Reflections on theory building and theory integration following a system
dynamics approach
Birgit Kopainsky’ and Luis F. Luna-Reyes”
' Corresponding author. Swiss Federal Institute of Technology, Institute of Agricultural Economics, 8092
Zurich, Switzerland, Phone: +41-44-632 53 28, Fax: +41-44-632 10 86, email:
birgit.kopainsky@ iaw.agrl.ethz.ch
2 Universidad de las Américas, Puebla, Business School, NE221J Santa Catarina Martir, Cholula, Puebla,
México, Phone: +52 (222) 229-2000 ext. 4536, Fax: +52 (222) 229-2726 email: luisf.luna@ udlap.mx
1 Abstract
Conceptualization is a critical task in the development of system dynamics models, which
starts early in the modeling process, and extends to later stages in the development of any
system dynamics project. The procedures and characteristics of model conceptualization
have striking parallels with the process of theory building as described in many different
strands of literature. Considering also that the modeling process as a whole is an iterative
process of comparing and contrasting data and current theories by means of a rigorous yet
intuitive process, it seems appropriate to reflect on the modeling process as a theory
building effort, which is the main purpose of this paper. In order to illustrate the differences
between theory building approaches, the paper presents two examples of system-dynamics-
based theory building efforts. Thinking of the model development work as a theory
building process has the potential of bringing new insights to the conceptualization of
system dynamics models, and to the criteria used to assess the suitability of our models.
The paper concludes with the introduction of a set of criteria to assess good theories and
with reflections on the further development of these criteria for model validation purposes.
Reflections on theory building and theory integration following a system
dynamics approach
“Conceptualization is at once the most important and least
understood of all modeling activities. Conceptualization is
really jargon for the mysterious process of creating a new
idea, a word designed to make the creative act sound
scientific, scholarly and repeatable” (J ohn Sterman 1986).!
1 Abstract
Conceptualization is a critical task in the development of system dynamics models, which
starts early in the modeling process, and extends to later stages in the development of any
system dynamics project. The procedures and characteristics of model conceptualization
have striking parallels with the process of theory building as described in many different
strands of literature. Considering also that the modeling process as a whole is an iterative
process of comparing and contrasting data and current theories by means of a rigorous yet
intuitive process, it seems appropriate to reflect on the modeling process as a theory
building effort, which is the main purpose of this paper. In order to illustrate the differences
between theory building approaches, the paper presents two examples of system-dynamics-
based theory building efforts. Thinking of the model development work as a theory
building process has the potential of bringing new insights to the conceptualization of
system dynamics models, and to the criteria used to assess the suitability of our models.
The paper concludes with the introduction of a set of criteria to assess good theories and
with reflections on the further development of these criteria for model validation purposes.
‘A quote from the presentation of the reprint of Mass’ (1986) introductory address to the
Conceptualization Table at the 1981 System Dynamics Conference.
2 Introduction
The purpose of a system dynamics modeling project is to gain understanding about some
problematic behavior in order to design policies or strategies for improving system
performance over time (Richardson and Pugh, 1981; Sterman, 2000). Problem
conceptualization constitutes a key element to project success, given that this is the “stage
that establishes the focus of the study - the general perspective and the time horizon. The
critical decisions are made on what part of reality to study and how to describe it” (Randers
1980:118). Moreover, instead of a discrete step within the modeling process, Randers
(1980) describes conceptualization as a recursive process closely related with the
formulation of the model. Since Randers’ effort to describe the conceptualization process in
1980, influential texts in system dynamics agree on the idea that problem conceptualization
consists of an iterative process of analyzing data, clarifying the problem boundaries and
pinning a dynamic hypothesis (Richardson and Pugh 1981, Roberts et al. 1983,
Wolstenholme 1990, Sterman, 2000). In spite of many efforts to get better tools and
methods for the conceptualization stage,? model conceptualization still is “the most
important and least understood of all modeling activities,” where the modeler uses her
intuition, data from the field, and existing literature to focus on a specific problem.
Given the similarities of the conceptualization process as described in the previous
paragraph with the process of theory building as described in many other strands of
literature (Benbastat 1987, Glaser and Strauss 1967, Eisenhardt 2002, Hanneman 1987,
Klein and Myers 1999, Lee 1989, Walsham 1995, Y in 1994, Strauss and Corbin 1998), the
proposition of this paper is to enrich our understanding of the modeling process by
analyzing it as the development of dynamic theories in order to explain phenomena or
design policy. Looking at the modeling process from this perspective will not only
See Richardson and Pugh (1981), Morecroft (1982), and Sterman (2000) for progress in mapping tools to
represent structure; Saeed (1992), and Sterman (2000) for techniques to analyze reference modes; and
Fey and Trimble (1993), Lane and Oliva (1998), and Keating (1998) for efforts to support the
conceptualization process with tools and techniques used in other scientific disciplines.
contribute to better modeling practices, but also enrich the criteria used to assess system
dynamic models.
The purpose of this paper is to explore theory building and integration using system
dynamics from different points of view. We analyze the benefits and pitfalls in the model
building process and differentiate between different approaches to modeling (Section 3). To
illustrate the differences between approaches, we present two examples of theory building
and integration in public policy. The first example (Section 4) follows a top-down theory
building approach which can often be found in the economic literature. The second
example illustrates a bottom-up approach which can be found in the information technology
management literature (Section 5). Section 6 shows a comparison of these two approaches
in terms of several criteria developed to assess “good theory,” and the paper ends with
some concluding remarks and further work (Section 7).
3 Model conceptualization as a theory-building process
In the problem identification and model conceptualization phase the modeler develops a
statement of the context and symptoms of the problem, sketches reference modes of
behavior, articulates the purposes of the modeling study, settles on a system boundary, and
develops a view of the system structure in terms of feedback loops. The last step represents
the model conceptualization in the narrow sense and consists in developing the physical
structure of the problem and adding information flows, perceptions and pressures arising
from perceptions that influence change in the system (Richardson and Pugh 1981).
The process just described fits the general description of a theory, considered to be made of
4 main components (Wacker 1998): definitions (of the variables and assumptions), a
specific domain (model boundaries), a set of relationships between variables (feedback
structure), and specific predictions (past or forecasted behavior over time explained by the
feedback structure).
Besides the data from a specific phenomenon, it is common to find a vast body of literature
speaking about aspects of the problem at hand. Existing theories are mostly static, either
listing elements of the physical structure of the system or describing specific information
flows. Given that the feedback nature of the problems is frequently described “between the
lines,” representing these static theories in a dynamic model represents an important
challenge to the modeler.
The challenge of building a dynamic simulation model from the data at hand and existing
theories is twofold. A first task is to elicit the feedback complexity inherent to static
theories. Second, different theories that cover different aspects of the problem have to be
combined and integrated into a consistent theory about the nature of the problem. Often, the
variables and linkages that comprise the relevant processes of a dynamic problem are well
established in the literature, but taken together they provide a new, more parsimonious view
of the processes. The development and analysis of a simulation model helps characterize
the range of organizational outcomes that these processes generate. The end result is an
internally consistent theory that is firmly grounded in data and previous work, but reaches a
new level of specificity concerning the determinants of the processes underlying the
problem at hand (Patrick 1995, Repenning 2002). The dynamic simulation model is, in the
end, a concrete realization of this theory (Hanneman and Patrick 1997).
A dynamic simulation model is based on both data and theoretical statements about the
operation of causal processes over time and makes concrete and explicit the concepts and
causal processes identified by actors in the problem and previous researchers. The study of
simulation models can be very useful in understanding and revising theory because they
provide an explicit and systematic way of deducing the implications of a theory as it
operates under particular circumstances to make predictions about outcomes over time
(Hanneman 1987).
There are two broad possibilities to build a dynamic theory. A top-down approach is theory
driven and mostly characterized by a high level of aggregation. The purpose of the model
building process is mostly to analyze some general case of a problem (similar to the
concept of homo economicus in economics). The simulation models resulting from such an
approach are simplified models that mimic the general behavior of a system but fail to
explain the observed behavior of individual cases in detail. The conclusions drawn from
model analyses provide general decision support and strategic guidelines. A bottom-up
approach, on the other hand, is more data and problem oriented and directed at
understanding and managing individual cases of an observed problematic behavior. The
resulting simulation models include all the elements essential to the case and are able to
replicate the relevant processes. Implications from model analyses lead to a set of concrete
policies that can be directly implemented.’ The difficulties of a bottom-up approach lie in
the generalization of the results, i.e. in identifying the generic features of the theory (Lee
and Baskerville 2003, Yin 1994, Wacker 1998). Table 1 summarizes the main
characteristics of the two modeling approaches.
The central problem is therefore to find the right balance between sufficient theoretical
orientation and sufficient data concem. On the one hand, data and problem oriented
research runs the risk of lacking an appropriate theoretical framework, maturity,
effectiveness, persistency, and consensus conceming concepts and methods. On the other
hand, theoretical orientation may again reduce problem orientation to questions of a newly
developed disciplinary matrix (Conrad 2002). The next two sections will therefore explore
the issues of theoretical orientation and data concem for the two model building
approaches.
Table 1: Key differences between a top-down and a bottom-up modeling approach (adapted from
Eckert 2004: 695)
Top-down approaches Bottom-up approaches
Research goal schadenindiee valid Understanding specific phenomena
Assumptions about reality Reality is objectively given Reality is subjectively constructed
Method of analysis Highly standardized Not or weakly standardized
Number of units of analysis High Low
In a report on best practices of system dynamics modeling, Martinez-Moyano and Richardson (2002)
found these two approaches to modeling as one of the main points of disagreement among the experts.
While some modelers in their sample prefer to work on a specific problem, others think that the modeling
process should focus in a more generic kind of problem to which the particular case belongs.
4 Top-down approaches: theory building in economics
Theoretical orientation is predominant in economics. Positive methods that contribute to
theory building formulate hypotheses based on an existing body of theory. The hypotheses
are tested against an empirical background. The size of the sample for the empirical
background needs to be statistically representative and the relevant criterion for assessing
the validity of a theory is the significance level (Eckert 2004). Positive methods explain a
given economic situation and its development. They aim at improving the basis for
forecasts about the future performance of the system (Sterman 1988).
Normative approaches in economics, on the other hand, identify the best ways to achieve a
given goal (Keusch 2000). Normative economics must have recourse to the analysis and
findings of positive economics. As it is not possible to prescribe what should be done
unless there are clear ideas about what would happen if certain economic measures were
taken or withdrawn. Such knowledge involves acceptable hypothesis about the structure of
the system (Mishan 1981). Positive methods also provide the basis for other methods of
mathematical economics in general and of economic dynamics in specific (Shone 2005).
4.1 Theories about employment and population dynamics in lagging rural regions
Econometrics is the measurement of economic relations that derive from preexisting
theories (Gujarati 1995, Maddala 2001). However, in applied socioeconomic contexts, a
specific research problem often encompasses elements from different strands of theory.
Theories that conceptualize the driving forces behind economic development in rural
regions of industrialized countries, for example can be found in various disciplines.
Regional economics and rural studies offer promising prospects as the former focuses on
regional economic development and the latter concerns rural development. The debate on
economic development in rural studies is especially concemed with the organizational
aspects of the rural economy. Regional economics, on the other hand, focuses more on the
interplay of the production factors of capital and labor, often affected by several other
factors (Terluin 2003). Theories in regional economics are therefore often called factor-
oriented theories while theories in the field of rural studies are labeled as being actor-
oriented (Egger 1998).
Contrary to factor-oriented theories that use only few variables and formal models to
explain development, in actor-oriented theories a considerable number of variables are
introduced but only very few attempts to formally model them have been undertaken so far.
Factor-oriented theories focus on explaining growth of a region’s output. As rural
development policy is not only concerned with output growth but with providing
employment opportunities as well, Figure 1 introduces a diagram of the interaction between
aregions’ product market and its labor market.
Figure 1: Basic scheme of a regional economy with linkages between the product market and the labor
market (adapted from Armstrong and Taylor 2000: 30)
R ai |_,| Final demand for Extemal demand
oolonal income region's output for region’s output
Output +) Demand for labor
Region's | .
aeaniopient Supply of labor
Price . neon Soe Wage rate Participation rate
Competitiveness of Net inward
region migration
I
The figure shows that employment growth depends on the growth of a region’s output,
which is itself determined by the competitiveness of its firms, i.e. the ability of these firms
to produce a certain share to meet the region’s own demand and the demand of other
regions.
Actor-oriented theories seek to understand the interaction between spatial structures and
sociospatial processes in rural areas. They address a wide range of issues in rural areas such
as people, settlements, landscape, environment, agriculture, economy, policy, minorities,
gender and cultural issues (Terluin 2001). A high capacity of local actors and strong
internal and external networks - often indicated as self-help capacity - are supposed to be
main factors behind employment growth (Terluin 2003).
Figure 2 illustrates the dynamic complexity inherent in actor-oriented theories on regional
tural development. The figure constitutes a summary of a theory integration effort that is
described in detail in Kopainsky (2005) and will be further commented in the final version
of this paper. The variable ‘external demand’ at the upper right hand side of the figure links
actor-oriented theories to the factor-oriented theories in Figure 1. Net migration, the
variable at the left hand side in Figure 2, is the second link between the two bodies of
theories.
Figure 2: Feedback complexity in actor-oriented theories on regional rural development
DY pamed
initiatives +
¢ « external demand
+ successfully 1a
implemented
ideas initiatives
v support of ideas ¥4
+ WS
B7- initiatives only success of threshold
tales rem initiatives commitment
net migration pressure to counteract
demographic changes
R5- reinforcement -d s
z : RG- reinforcement -
Hs Sapecty: poptletion commitment - success »
entrepreneurial + commitment ratio
capacity population| \ +4
commitment to
r initiatives
4, =
6 commitment necessary erortin
+ ‘through motivation
ne
available effort motivation
wader
+ 2
4.2 Dynamic implications of these theories
The dynamic simulation model derived from the factor- and actor-oriented theories
explores the evolution of the regional economy over time for a growth oriented strategy. It
analyses how robust these strategies are and how a region’s economy and population
interact over time.
Factor-oriented theories suggest a variety of strategies directed at regional economic
growth (Armstrong and Taylor 2000). Investment grants and venture capital initiatives are
examples of a development strategy based on the provision of infrastructure and capital
goods. Figure 3 summarizes the population effect of variations in the inflow into the capital
stock. The inflow into the capital stock is determined by the depreciation rate, the ratio
between current and desired capital and the fraction of available capital investment goods.
In Figure 3, the fraction of available capital goods is varied between 0.5 and 1.5, i.e. from
half its normal value to 150% of it.
The results are presented in the form of a three-dimensional response surface. In the plot,
the vertical axis represents the outcome variable of interest, in this case the 0 to 65 years
old population. The horizontal axis represents time and the third axis, which extends into
the page, captures the input variable being manipulated in the experiment, in this case the
fraction of available capital goods. Reading from left to right along the horizontal axis, any
given line shows the time path of the outcome variable given a specific input variable.
Reading from front to back along the input variable axis, any given line shows how the
value of the outcome variable, at one specific point in time, changes in response to changes
in the input variable. Viewing the resulting surface presents a dynamic view of how the
evolution of the outcome variable is influenced by changes in the input variable.
10
Figure 3: Population development as a reaction to changes in available capital investment goods
150
Population
0-65
fraction available
investment goods
Figure 3 contains several implications. It shows that making more capital goods available
results in a better-before-worse behavior pattern of the 0 to 65 years old population.
Restricting capital availability to half of the desired value generates constant population
decline. The normal equilibrium - transition - equilibrium development pattern is almost
completely suppressed. Increasing the availability of capital above its desired value, on the
other hand, leads to an initial population increase. After a short growth period, however,
population starts to decline and drops below the value of the situation with restricted access
to capital goods. In the long run, population recovers to a level that is approximately
identical for all the values in the fraction of available investment goods. If investment in the
capital stock is not followed by massive investment in an adequate increase in external
demand the costs of maintaining the capital stock become too high after the initial growth
period.
Better-before-worse behavior as shown in Figure 3 bears important policy implications. If
population stabilization or population growth are development goals in lagging rural areas,
a strategy based on increasing the availability of capital goods results in initial success. The
unintended consequence of it is, however, that the initial success is not sufficient to keep
population at such a high level. After an initial growth period, a distinct decline occurs.
11
Regional economic growth can also be brought about by locally and regionally initiated
economic activities. The number of initiatives that are created and flow through the system
until they end as determinants of endogenously created external demand depends on
initiative creation, initiative support and initiative implementation decisions as illustrated in
Figure 2. The successful implementation of initiatives is coupled to actors’ commitment to
the initiatives. Commitment is influenced by the success of (past) initiatives and the
motivation that actors experience during the implementation phase.’
If commitment and success are at minimum levels, the reinforcing feedback loop
reinforcement commitment - success keeps the system at minimum performance. A shift in
loop direction can only be caused by the balancing feedback loop commitment through
motivation. If commitment is lower than the threshold commitment for success, motivation
is necessary to raise commitment. Necessary motivation increases the lower the ratio
between current and necessary commitment is. Entrepreneurial capacity and external
support determine the level of available motivation which in tum determines whether
commitment can be sufficiently raised or not. A third set of experiments therefore analyzes
the reaction of commitment to changes in necessary and available motivation. For this
purpose, the system is initialized to equilibrium and hit with a step increase in necessary
success as above. Available motivation is varied between 0.2 (low motivation) and 0.8
(high motivation).
This experiment can be interpreted as varying entrepreneurial capacity either by increasing
management skills, political and administrative support or improving communication
infrastructure. The results are presented in Figure 4. In the plot, the vertical axis represents
the outcome variable of interest (in this case actors’ commitment to an initiative) and the
horizontal axis represents time. Each line captures the reaction of commitment to the input
variable being manipulated in the experiment (in this case the degree of motivation that
* The conceptualization of this model part is taken from Repenning (2002) and adjusted to the public domain.
12
local actors experience). Reading from left to right, any given line shows the time path of
the output variable given a specific input variable.
Figure 4: Reaction of commitment to changes in capacity to motivate actors involved in an initiative
1985 1995 2005 2015 2025 2035 2045
Figure 4 shows that the success of initiatives (resulting from actors’ commitment) depends
critically on the motivation that actors experience. The behavior is determined by the
interaction between the reinforcing loop reinforcement commitment - success and the
balancing loop commitment though motivation. As a reaction to the step increase in the
success threshold, the reinforcing loop works in a downward direction and thus drains
commitment and success. As some entrepreneurs start motivating the actors involved in the
initiative, the reinforcing loop is less of a drain on commitment. If motivation is high
enough, commitment continues to grow and success becomes sufficient to generate more
commitment. At this point, the reinforcing loop shifts direction and begins to work in an
upward direction. Once this shift occurs, reinforcement commitment-success generates
rapid growth in commitment.
Policy failure can therefore have two reasons: underinvestment and wrong choice of policy.
The exemplary results show that development strategies based on strengthening local
collaborative activities have the potential to influence employment and population
development and that policy failure mainly arises from underinvestment in management
skills and entrepreneurial capacity.
13
5 Bottom-up approaches: theory building in information systems
development
Similarly to the field of economics, factor analysis starting with theory is a common
approach to understand information systems development (ISD).° These approaches
contribute to our understanding of ISD by offering a linear-static view of a complex-
dynamic process such as information systems development and implementation (Newman
and Robey 1992).
A complementary approach consists in the use of case study research, which offers a
process-oriented, and dynamic view of the information systems development process
(Newman and Robey 1992). Similarly to the actor-oriented theories in economics described
in previous sections, case-based research in ISD introduces many variables when compared
with factor-oriented research. Moreover, case-based system dynamics models have proven
useful for studying information systems development processes given its ability to deal
with complex and dynamic systems (Abdel-Hamid 1988, A bdel-Hamid and Madnick 1990,
Bennett et al. 1999, Lehman and Ramil 2002).
Recognizing the differences of system dynamics with other mathematical modeling
methods, Black (2002) equiparates the modeling process with other qualitative theory-
building approaches where the researcher builds iteratively a theory by interpreting,
comparing, and contrasting observations, and patterns of behavior with previous theories
(Glaser and Strauss 1967, Walsham 1995, Eisenhardt 2002).
5.1 Theories about trust and collaboration in information systems development
The example outlined in this section deals with information resources for programs serving
the more than 29,000 homeless people who receive emergency shelter and a diversity of
support services each day in New Y ork State. Homeless services costs are estimated to be
$350 million each year, $130 of which are spent on service programs (CTG, 2000). The
5 See Larsen (2003) for an extensive review of a sample of 212 studies on information systems
development from 1954 to 1999.
14
information needed to assess the effectiveness and impact of the services provided to the
homeless is distributed in several agencies and nonprofits, such as the Bureau of Housing
Services (BHS), and the New York City Department of Homeless Services (DHS). The
lack of integration of the data sources makes very difficult to assess them. Starting in 1998,
the Office of Temporary and Disability Assistance (OTDA), Bureau of Housing Services
(BHS) started a series of efforts to create an integrated decision support system to help both
government and nonprofit organizations to manage and assess homeless services called the
Homeless Information Management System (HIMS). The system would integrate
information from a variety of sources. Demographic data would be obtained from the
individual shelters, payment information would come from the state Welfare Management
System (WMS), shelters’ information would be gathered from the BHS’s providers
certification database, medical information from the State Department of Health, and data
on substance abuse or other services from other State Agencies. Although BHS is an
oversight agency, which manages and regulates temporary housing programs in New Y ork
State, it shares its regulatory functions in New Y ork City with the NYC Department of
Homeless Services.
Problems like the one just described make collaborative approaches appealing for many
managers (Gray 1989, McCaffrey et al. 1995, Bardach 1998). However, there is an
important gap between managers’ appreciation and the actual proportion of initiatives using
a collaborative approach (McCaffrey et al. 1995). The gap between the managers’ beliefs
and current practices can be understood as a lack of theories to understand the processes
and phenomena involved in collaboration, and to guide our current practices.
Luna-Reyes (2004) developed a knowledge-and-trust-based collaboration theory based on
the HIMS case. Through the analysis of interviews and archival documentation’ of the case,
Longitudinal data from the case was gathered as a component of a Project in which the main objective
was to develop a better understanding of knowledge creation and sharing in interorganizational networks.
The project is part of the research program at the Center for Technology in Government (CTG) in
Albany, NY.
15
he identified three main themes that became the main backbone of the modeling effort:
trust development, stakeholder engagement, and requirement definition as a social process.
For this illustration, we present a model that focus in the first theme, analyzing
interpersonal trust dynamics (see Figure 5 for a high-level representation of the model). The
model constitutes a generic representation of the interaction between two actors, BHS and
homeless service providers. The model is grounded in the longitudinal case study of the
HIMS. The case study produced observational and interview data about these interactions
that indicated substantial growth in the levels of interpersonal trust among these
participants. There was considerable evidence of feedback and learning as important factors
in how trust developed over the roughly two-year course of the project.
Figure 5: Developing trust among providers
Noise
po Developing service
Collaboration BHS's knowledge <«————__ model
experiences of providers
@
(@y von
Confirmation Knowledge-based Providers’ Institutional
bias Trost knowledge of BHS Tnst
Broviiars Rereaved
roviders ‘ ;
perception of BHS's Providers mst n ww
ames an “___ Calallative Perceived
tust ~<—— _ benefit
A priori perception of
BHS's trustworthiness
As shown in the figure, the model identifies two important feedback processes associated
with the development of trust. The feedback process R1 represents the confirmation bias
identified in the trust literature as our tendency to assess positively our experiences with
people that we perceive as trustworthy, or to assess negatively our experiences with people
that we perceive untrustworthy (Klimoski and Donahue 2001). Providers’ collaboration
16
experience is represented in the model as their memories of good and bad experiences in
their interaction with the Bureau of Housing Services (BHS). These experiences can be
distorted by external noise that interferes in the perceptual process, and when there is no
previous experience, an a priori component of the perception appears to operate. The
feedback process R2 represents BHS’s ability to build its reputation as a trustworthy party.
It was possible to compare some portions in the model with several mechanisms of “trust
production” identified in the literature: Institutional trust, calculative trust, knowledge-
based trust, and identification-based trust.
Institutional trust refers to the existence of an institutional framework that regulates the
relationship between the trustor and the trustee. In any case, the existence of this
mechanism to facilitate trust reduces the trustor’s perception of risk in the interaction
(Williamson 1993). Calculative trust refers to the trustee’s estimation of the risks and pay-
offs intertwined in the interaction (Rousseau et al. 1998). Knowledge-based trust is related
to the ability of the trustor to assess the trustworthiness of the trustee (Mayer et al. 1995),
and it is associated with the history or the process of the relationship. Finally,
identification-based trust is associated sometimes with emotional bonds, or with the
existence of shared values or objectives between the actors (Shapiro et al. 1992).
The model assumptions about the way in which the “trust production” processes operate is
consistent with the views proposed by Rousseau et al. (1998), who consider that the
calculative trust plays a more important role in early stages of the relationship, changing
towards a knowledge-based trust as the relationships matures. In this way, providers’
knowledge of BHS can be interpreted as a weight between these two types of trust.
Although the model does not add any new term to the trust literature, it presents a new way
of interrelate all the existing concepts in a dynamic framework.
17
5.2 Dynamic implications of these theories
The model described briefly in the previous section was tested for internal consistency by a
series of experiments with diverse inputs, testing model sensitivity to changes in diverse
parameters. Some experiments were inspired in some studies of negotiation that suggest
that trust outcomes vary according to the rate of early versus late concessions. A pattern of
small early concessions leading to larger later ones tends to produce better outcomes than a
large-to-small concession pattern or constant-rate small concessions (Hamner and Y ukl,
1977). The behavior of the model reflects this same pattern (see Figure 6). In the first
experiment (Figure 6a), the pattern of concessions began small and then increased in size
later in the time period. The growing divergence in results shows one type of evidence of
path dependence. In addition, a higher proportion of the path’s lead to increased rather than
lower trust, as would be expected from the theory.
Figure 6: Behavior of trust in BHS for (a) low initial concessions gradually increasing at different later
times and (b) high initial concessions gradually decreasing at different times.
(a) (b)
downandhigh8a highanddownBa
‘Trustin B ‘Tnustin B
08 08
06
2
Time (Month) Time (Month)
Figure 6b above shows the results of the alternative concession pattem, ie., larger initial
concessions decreasing at a later time. The high initial concession rate leads to high trust
levels in the early stages, which drop steeply down as the concession rate drops. A high
proportion of the paths lead to overall lower trust. This is consistent with the generally
accepted view of trust as being susceptible to betrayal, which is how the high-to-low
concession pattem can be viewed. That is, the early pattern of high concessions can be
18
thought of as establishing high expectations for the outcome, which are “betrayed” by the
shrinking size of later concessions.
Other experiments with the trust structure suggest that, although the a priori component has
an important impact in trust development, the efforts to build trust in the day-to-day
interactions can overcome the initial weight of the a priori component. Moreover, early
efforts to develop trust are more effective than those that occur in later stages of the
interaction. Although the development of trust, because of the attention to the relationship,
is a gradual process, the lack of attention to the relation can revert the process much faster.
Finally, managing the institutional component of trust (i.e., reducing risk) could be a
strategy to break the initial trap of distrust.
The model suggests that is hard to create a history-based trust in as short a period of time.
Simulation experiments with the whole model (not only the trust portion of it) suggest that
the most important component of trust during the development of the HIMS prototype was
the calculative one. Given this situation, the team could have accomplished its goals in a
very similar period of time in a situation in which there was little interest in fostering a
trusting environment. The behavior of trust, however, suggests that the knowledge-based
component will be more important in subsequent project stages.
6 Comparison of the approaches and examples
Reflection about the selection of a top-down or bottom-up approach requires careful
consideration. From the two examples presented in this paper, it may seem that top-down
approaches could be well suited to problem areas where well-established competing
theories claim to have the best explanation to the observed behavior. Dynamic simulation in
those situations can contribute to identify the “true” differences among the approaches, the
elements where the existing theories are complementary, and the elements where there is
controversy. The bottom-up approach appears to be useful in areas like the research on
trust, where many competing points of view attempt to describe a particular phenomenon
recognizing that there is a lack of clarity in the academic debate. In these situations, the
19
rigor of the modeling process can help to clarify concepts and relationships, ordering the
main concepts and causalities.
Although both approaches need to rely on data and current theories, it seems that top-down
approaches rely mainly on current theory, while bottom-up approaches rely mainly on data
from concrete cases. A commonality in these two approaches is the iterative review of data
and theory, guided by the intuition and interpretation of the researcher (Black 2002).
System dynamicists share the belief that judgments about the validity of a model must be
linked to the purpose of the modeling effort (Richardson and Pugh 1981, Forrester and
Senge 1980, Barlas 1996, Sterman 2000). Accordingly, some distinctions can be made
between judgments about the adequacy of a model developed to help a client group, and a
model developed to increase understanding in a specific phenomenon from the academic
point of view (Coyle 2000). Considering the modeling process as a theory building process
to increase our scientific understanding of a particular phenomenon opens a whole new set
of criteria to use when judging the adequacy of a system dynamics model.’ A clear example
of one of such criterion could be the claim of the generalizability of the theory to other
specific instances.
Wacker (1998) presented a list of common criteria used to assess “good theories” (See
Table 2). From our point of view, it is needed to review the current tests to build confidence
in models to assess their suitability to support judgment in this set of criteria.
7 In our discussion of the validity of system dynamics models as “good science” we adhere to the relativist
philosophy of science rather than the logical empiricist one. See Barlas and Carpenter (1990) for a
complete discussion of these two philosophical approaches.
20
Table 2. Virtues of “good theory.” Extracted from Wacker (1998: 365)
Criterion Description
Uniqueness The proposed theory is different from other theories existing in the literature
Parsimony The better theory is that with less assumptions, variables and causal relationships,
yet with power of explanation of observed behavior
Conservatism Current theories cannot be replaced but by better theories (more parsimonious,
generalizable, etc.)
Generalizability The theory can be applied to several areas
Fecundity The ability of a theory to raise new questions and hypotheses
Internal Consistency All relationships inside the theory are well justified and explained
Empirical riskiness The theory must be refutable
Abstraction Independence from time and space
Deciding between a top-down or a bottom-up approach to theory development and
integration constitutes an important decision to be made by the modeler. The consequences
of this decision affect all the criteria listed in Table 2, specifically the issues of
generalizability and abstraction.
7 Concluding remarks
The purpose of this paper was to reflect on the theory building process in system dynamics.
Based on the premise that a model is a concrete realization of prior theories we analyzed
the implications of building a dynamic theory from a body of related static theories and
data. While the additional insights gained from such an approach have been widely
discussed in the system dynamics literature some aspects have received less attention. We
specifically emphasized the similarities between model conceptualization in the system
dynamics literature and theory building in other strands of literature such as sociology or
operations management. A fter reviewing some of the literature on model conceptualization
and theory building we introduced two case study examples of theory building in public
policy. The examples differed in the underlying discipline and the aim of the model
building process. The first example was rooted in economics and aimed at identifying
generic processes and determinants of regional rural development. The second example was
21
situated in the management of technology innovations and designed to analyze a real case
about trust and collaboration in developing technology innovations in the public sector.
As additional guidelines for assessing the validity and usefulness of a system dynamics
model we introduced a list of criteria to assess good theories. Adding these elements to the
validation of a system dynamics model seems to be promising. The criteria to assess good
theories ensure that the model not only serves its purpose in terms of the specific problem
at hand. Instead, they also shift the focus to more general issues such as the contribution of
the simulation model to existing theories. By asking questions about uniqueness of the
developed theory or others, learning from the model building process can be improved. The
contributions of a well validated system dynamics model to static theories and to the
various existing theories that speak to a specific issue can also be made more transparent.
However, the criteria have to be further operationalized to be fully applicable for evaluating
the usefulness and contribution of a system dynamics model. In future versions of this
paper we will use the two case study examples to develop a preliminary set of criteria that
can be used both for top-down and bottom-up approaches to theory building and
integration.
8 References
Abdel-Hamid, T. K. (1988). "The economics of software quality assurance: A simulation-
based case study." MIS Quarterly 12(3): 395-411.
Abdel-Hamid, T. K. and S. E. Madnick (1990). "The elusive silver lining: How we fail to
learn from software development failures." Sloan Management Review: 39-48.
Ammstrong, H. and J. Taylor (2000). Regional economics and policy. Oxford, Blackwell
Publishers. 3” edition.
Bardach, E. (1998). Getting agencies to work together: The practice and theory of
managerial craftmanship. Washington, DC, Brookings Institution Press.
Barlas, Y. (1996). "Formal aspects of model validity and validation in system dynamics."
System Dynamics Review 12(3): 183-210.
22
Barlas, Y. and S. Carpenter (1990). "Philosophical roots of model validation: Two
paradigms." System Dynamics Review 6(2): 148-166.
Benbastat, I., D. Goldstrein and M. Mead (1987). "The case research strategy in studies of
information systems." MIS Quarterly 11(3): 369-386.
Bennett, K., E. Burd, C. Kemerer, M. M. Lehman, M. Lee, R. J. Madachy, C. Mair, D.
Sjoberg and S. Slaughter (1999). "Empirical studies of evolving systems." Empirical
Software Engineering 4(4): 370-380.
Black, L. (2002). Collaborating across boundaries: Theoretical, empirical, and simulated
explorations. Sloan School of Management. Cambridge, MA, MIT.
Chiang, A.C. (1984). Fundamental methods of mathematical economics. Singapore,
McGraw-Hill International Editions. 3” edition.
Conrad, J. (2002). “Limitations to interdisciplinarity in problem oriented social science
research.” The Journal of Transdisciplinary Environmental Studies 1(1): 1-15.
Coyle, R. G. and D. Exelby (2000). "The validation of commercial system dynamics
models." System Dynamics Review 16(1): 27-41.
CTG (2000). Building trust before building a system: The making of the homeless
information management system.
http://www.ctg.albany.edu/guides/usinginfo/Cases/printable_bss_case.htm. Last
revised June, 2002
Eckert, S. (2004). “Grounded Theory als methodische Konzeption fir
wirtschaftwissenschaftliche Forschungsprojekte.“ WiSt 12/ 2004: 694-699.
Egger, U. (1998). Offentlich-private Partnerschaften zur regionalen Entwicklung.
Theoretische Grundlagen und Fallstudien in der Schweiz. St. Gallen, Hochschule
fur Wirtschafts-, Rechts- und Sozialwissenschaften.
Eisenhardt, K. (2002). Building theories from case study research. The qualitative research
companion. A. M. Huberman and M. B. Miles. London, Sage Publications.
Fey, W. R. and J. Trimble (1993). An expert system to aid in model conceptualization.
International System Dynamics Conference, Cancun, Mexico, System Dynamics
Society.
Forrester, J. W. and P. M. Senge (1980). Tests for building confidence in system dynamics
models. System dynamics. A. A. Legasto, Jr. and e. al. New Y ork, North-Holland.
14: 209-228.
Glaser, B. G. and A. L. Strauss (1967). The discovery of grounded theory: Strategies for
qualitative research. Chicago, Aldine Publishing Company.
23
Gray, B. (1989). Collaborating: Finding common ground for multiparty problems. San
Francisco, CA, Jossey-Bass Inc.
Gujarati, D.N. (1995). Basis econometrics. New York, McGraw-Hill, Inc.. 3" edition
(International Edition).
Hamner, W. C. and G. A. Yukl (1977). The effectiveness of different offer strategies in
bargaining. Negotiations: Social-psychological perspectives. D. Druckman. Beverly
Hills, CA, Sage Publications.
Hanneman, R. (1987). Computer-assisted theory-building. Newbury Park, CA, Sage
Publications, Inc.
Hanneman, R. (1995). “Simulation modeling and theoretical analysis in sociology.”
Sociological Perspectives 38(4): 457-462.
Hanneman, R. and S. Patrick (1997). “On the uses of computer-assisted simulation
modeling in the social sciences”. Sociological Research Online 2(2).
Keating, E. K. (1998). “everthing you ever wanted to know about developing a system
dynamics model, but were afraid to ask." 16th International Conference of the
System Dynamics Society, Quebec '98, Quebec City, Canada, System Dynamics
Society.
Keusch, A. (2000). Modellierung ressourcendkonomischer Fragestellungen am Beispiel der
Erosion im Gebiet des Baldeggersees. Doctoral Thesis ETH Zurich.
Klein, H. K. and M. D. Myers (1999). "A set of principles for conducting and evaluating
interpretive field studies in information systems." MIS Quarterly 23(1): 67-94.
Klimoski, R. J. and L. M. Donahue (2001). Person perception in organizations: An
overview of the field. How people evaluate others in organizations. M. London.
Mahwah, Lawrence Erlbaum Associates: 5-43.
Kopainsky, B. (2005). A system dynamics analysis of socio-economic development in
lagging Swiss regions. Aachen, Reihe V olkswirtschaft, Wissenschaftsverlag V auk.
Morecroft, J. D. W. (1982). "A critical review of diagraming tools for conceptualizing
feedback system models." Dynamica 8(1): 20-29.
Lane, D. C. and R. Oliva (1998). "The greater whole: Towards a synthesis of sd and ssm."
European Joumal of Operational Research 107(1): 214-235.
Larsen, K. R. T. (2003). "A taxonomy of antecedents of information systems success:
Variable analysis studies." Journal of Management Information Systems 20(2):
169-246.
Lee, A. S. (1989). "A scientific methodology for mis case studies." MIS Quarterly 13(1):
33-50.
24
Lee, A. S. and R. L. Baskerville (2003). "Generalizing generalizability in information
systems research." Information Systems Research 14(3): 221-243.
Lehman, M. M. and J. F. Ramil (2002). "Software evolution and software evolution
processes." Annals of Software Engineering 14: 275-309.
Luna-Reyes, L. F. (2004). Collaboration, trust and knowledge sharing in information-
technology-intensive projects in the public sector. School of Information Science
and Policy. Albany, NY, University at Albany.
Maddala, G.S. (2001). Introduction to econometrics. Chichester, John Wiley & Sons. 3m
edition.
Martinez-Moyano, I. J. and G. P. Richardson (2002). An expert view of the system
dynamics modeling process : Concurrences and divergences searching for best
practices in system dynamics modeling. Proceedings of the 20th Intemational
Conference of the System Dynamics Society, Palermo, Italy, The System Dynamics
Society.
Mayer, R. C., J. H. Davis and F. D. Schoorman (1995). "An integrative model of
organizational trust." The Academy of Management Review 20(3): 709-734.
McCaffrey, D., S. Faerman and D. Hart (1995). "The appeal and difficulties of participative
systems." Organization Science 6(6): 603-627.
Mishan, EJ. (1981). Introduction to normative economics. New Y ork/ Oxford, Oxford
University Press.
Newman, M. and D. Robey (1992). "A social process model of user-analyst relationships."
MIS Quarterly 16(2): 249-266.
Patrick, S. (1995). “The dynamic simulation of control and compliance processes in
material organizations” . Sociological Perspectives 38(4): 497-518.
Randers, J. (1980). Guidelines for model conceptualization. Elements of the system
dynamics method. J. Randers. Cambridge MA, Productivity Press: 117-138.
Repenning, N. P. (2002). "A simulaton-based approach to understanding the dynamics of
innovation implementation." Organisation Science 13(3): 109-127.
Richardson, G. P. and A. L. Pugh, III (1981). Introduction to system dynamics modeling
with dynamo. Cambridge MA, Productivity Press.
Roberts, N. H., D. F. Andersen, R. M. Deal, M. S. Grant and W. A. Shaffer (1981).
Introduction to computer simulation. Waltham, MA, Pegasus Communications.
Rousseau, D. M., S. B. Sitkin, R. S. Burt and C. Camerer (1998). "Not so different after all:
A cross-discipline view of trust." Academy of Mangement Review 23(3): 393-404.
25
Saeed, K. (1992). "Slicing a complex problem for systems dynamics modeling." System
Dynamics Review 8(3): 251-262.
Shapiro, D. L., B. H. Sheppard and L. Cheraskin (1992). "Business on a handshake."
Negotiation Journal 8(4): 365-377.
Shone, R. (2005). An introduction to economic dynamics. Cambridge University Press.
Sterman, J.D. (1988). A skeptic’s guide to computer models. Managing a nation: The
microcomputer software catalog. Bamey G.O., Kreutzer W.B., Garrett MJ.
Boulder CO, Westview Press: 209-229.
Sterman, J. D. (2000). Business dynamics : Systems thinking and modeling for a complex
world. Boston, Irwin/McGraw- Hill.
Strauss, A. and J. Corbin (1998). Basics of qualitative research: Techniques and procedures
for developing grounded theory. Thousand Oaks, Sage Publications.
Terluin, IJ. (2001). Rural regions in the EU: Exploring differences in economic
development. Utrecht/ Groningen, Netherlands Geographical Studies 289.
Terluin, I.J. (2003). “Differences in economic development in rural regions of advanced
countries: an overview and critical analysis of theories”. Journal of Rural Studies
19(3): 327-344.
Wacker, J. G. (1998). "A definition of theory: Research guidelines for different theory-
building research methods in operations management." Journal of Operations
Management 16: 361-385.
Walsham, G. (1995). "Interpretive case studies in is research: Nature and method."
European Journal of Information Systems 4(2): 74-81.
Williamson, O. E. (1993). "Calculativeness, trust, and economic organization." Journal of
Law and Economics 36: 453-486.
Wolstenholme, E. F. (1990). System enquiry - a system dynamics approach, John Wiley.
Yin, R. K. (1994). Case study research design and methods. Newbury Park, CA, Sage.
26
CC BY-NC-SA 4.0