Best Practices in System Dynamics M odeling
Ignacio J. Martinez** and George P. Richardson?
Ignacio J Martinez
is a doctoral
student at the
Rockefeller
College at the
University at
Albany. His
current research
focuses on
discovering the
underlying
structures that
condition the
creation and use
of best practices.
George P.
Richardson is a
professor of public
administration and
policy at the
Rockefeller
College at the
University at
Albany. His work
focuses on
understanding the
dynamic
implications of
elements related to
public policy and
management. He
is currently the
Chair of the
Public
Administration
and Policy
Department.
Abstract
This research is set out to (a) discover of a set of core practices in the system dynamics
modeling process and then (b) find the best of them according to the knowledgeable
opinion of a world wide recognized group of experts in the field. The paper will address
(1) what areas of the system dynamics modeling process are common to all model
building regardless of the modeler, the model, the type of practitioner, the tool used or the
purpose of the model? (2) Which of these areas can we describe as “best practice”?
We used a multi-method approach starting with interviews, then we conducted a virtual
meeting with the former presidents and award winners from the System Dynamics Society
to elicit the practices, and lastly, we developed a discussion on the results and the
implications for further research.
The paper identifies 41 Best Practices grouped into categories of problem identification
and definition (15), system conceptualization (8), and model formulation (11). Most
importantly, the study identified seven practices of which experts appear to disagree.
Keywords: System Dynamics, Theory, Conceptual Framework, Best Practices, Modeling
Process, and Knowledge Management.
Introduction: the importance of best practices
This research is set out to (a) discover of a set of core practices in the system
dynamics modeling process and then (b) find the best of them according to the
knowledgeable opinion of a world wide recognized group of experts in the field.
It is an initial effort that will latter be extended to a wider group of practitioners.
The paper will address (1) what areas of the system dynamics modeling process
are common to all model building regardless of the modeler, the model, the type
of practitioner, the tool used or the purpose of the model? (2) Which of these
areas can we describe as “best practice”?
We used a multi-method approach starting with interviews, and included a virtual
meeting with the former presidents and award winners from the system dynamics
society. Elicited in that virtual meeting the “best” practices, and lastly, we
developed a discussion on the results and the implications for further research.
The system dynamics literature brings together examples related to the concept of
“best practices” from a number of threads. Start with the earliest work done by
Jay Forrester in Industrial Dynamics (Forrester 1963) and World Dynamics
“ Corresponding author
* University at Albany, Milne Hall 121-A 135 Westem Av. Albany, NY 12222, USA E-
mail: im7797@ albany.edu
> University at Albany, Milne Hall 103 135 Western Av. Albany, NY 12222, USA E-mail:
gpr@ albany.edu
2The 19" Intemational Conference of the System Dynamics Society, Atlanta, Georgia, USA July 23-27, 2001
(Forrester 1973). Second, we find important collections of papers such as
Modeling for Management (Richardson 1997) or Modeling for Learning
Organizations (Morecroft and Sterman 1994). Thirds include specific pieces on
what practices are currently used as best like Benchmarking the System Dynamics
Community (Scholl 1995). Finally, we find textbooks in which we locate many, if
not all, the best practices that are there in the field such as Business Dynamics
(Sterman 2000) or Introduction to System Dynamics Modeling (Richardson and
Pugh 1981).
Because the field has been expanding with respect to the types of systems
modeled and the number of practitioners across the world, the different views of
the field are multiplying and the need to agree on a set of “core” practices
emerges. The set of practices identified in this paper should be independent of the
type of system modeled, the tool used to develop it, the purpose of the model, and
the type of practitioner or the individual modeler. Practices that meet these criteria
are, by definition, core practices. Despite the accomplishments of many talented
individual practitioners, the lack of concurrence over core practices makes it
difficult to broadly evaluate system dynamics as a modeling practice and can
prevent the field from continued development (Scholl 1992).
The system dynamics model building process involves six key activities as shown
in Figure 1.The activities are (1) problem identification and definition, (2) system
conceptualization, (3) model formulation, (4) model testing and evaluation, (5)
model use, implementation and dissemination, and (6) design of leaming strategy
/ infrastructure. We used these six activities as conceptual framework in this
study. The key products are the understandings of the model and of the problem
and the system.
I. J. Martinez and G.P. Richardson: Best Practices in System Dynamics Modeling 3
Figure 1. - Model Use,
renee of the Implementation, and
ystem Dynamics : Sagi
Modeling Dissemination
Approach
[Adapted from .
(Richardson and ee of the
Pugh 1981)] problem le system
Understandings of
the Model Problem Identification
and Definition
i Design of Learning
Model Testing and
Evaluation Strategy / Infrastructure
Model System ;
Formulation Conceptualization
Method of Study
This research uses a multi-method approach that includes interviews, a web-based
elicitation process, and an analysis of the results. The description of the method
appears in Figure 2. We used the web-based participation method in light of its
applicability as group decision support system and the fit to the needs! of this
specific study (Rohrbaugh 2000).
| Having many people to contact geographically disperse and with very different time
slots available for the study.
AThe 19" Intemational Conference of the System Dynamics Society, Atlanta, Georgia, USA July 23-27, 2001
Figure 2. -
Description of the
method of study.
Process
1) Preliminary Interviews
‘A group of experts from the System Dynamics Group at Albany was interviewed to generate the
framework to be used for the study.
The framework was generated and implemented in the research design and an asynchronous
facilitated web-based meeting was selected to elicit the ideas from the participants.
2) WebW ide Participation Meeting
Idea Elicitation
The experts wrote their ideas
The lists were used as a trigger for new ideas.
Idea Clustering
The complete list per area was available for the participants.
The participants put together the ideas that belonged together in clusters.
The minimum common multiple was obtained for the clusters in which all participants agreed.
Idea Prioritization
The complete list of clusters was available for the participants.
The clusters were equi-weighted initially.
The participants changed the weights on the clusters to reflect their priorities.
The weights expressed by the participants became the realtive importance of the clusters,
Products
T) Raw Data (Appendix 3)
A series of statements directly typed by the participants are reported presented by area and priority.
age The original clusters are presented as prioritized by the participants.
Four tables are presented, one for each area studied and one for the controversial statements.
2) Summary the Results (Section Ill of the paper)
The summary presents an slighly edited version of the raw data reducing each cluster to a single
summary statement.
Again, four tables are presented, one for each area studied and one for the controversial statements.
As described in Figure 2, the facilitated meeting had three parts. These parts are
consecutive and designed to generate the highest participation possible in the
study. The parts are (1) idea elicitation, (2) idea clustering, and (3) idea
prioritization.
Participants of the Study
The group of people who participated in the research includes the former and
future presidents of the society and the winners of awards from the society (Jay W
Forrester Award and The Lifetime Award). We selected this purposeful expert
sample to be able to have a group of individuals with the highest level of
recognition in the field. One important consideration made regarding the
composition of the sample was their busy schedules and the probable low-level
time available for this project. Out of 23 people invited, only two declined at the
very beginning due to time constraints. The participation level was 70%, 16 out of
the 23 experts invited participated in the study.
I. J. Martinez and G.P. Richardson: Best Practices in System Dynamics Modeling 5
The WebWide Participation Meeting
The total time span for the facilitated meeting was six weeks. All three stages
each lasted two weeks. In the first stage, the participants listed ideas related to the
eliciting question posted on the web site for the meeting. The participants
browsed in the web and looked at a screen presented as Figure 3.
Figure 3. - Web Is Help
Page for tie ie ae Eee
Facilitated [Ads [BT Ho stech ance Coton a 1 ete [jtie®
Meeting (Part 1) 5
Welcome to Best Practices in System Dynamics 4
Modeling.
Ifyou were offering advice on the best way to undertake system dynamics modeling, what specific core activities
‘would you say are essontial for exemplary PROBLEM IDENTIFICATION AND DEFINITION? In this area,
what are the most important practices of modeling work?
If you were offering advice on tha hest way to undertalze system dynamics modeling, what specific core activities
‘would you say are essential for exemplary SYSTEM CONCEPTUALIZATION? In this area, what are the most
important practices of modeling work’)
Iryou were offering advice on the hest way ta undertake system dynamics modeling, what specific core activities
wwould you say are essential for exemplary MODEL FORMULATION? In this area, what are the most important
practices of modeling work?
Instructions for Participants: To join in ou certing, please cick the button below and begin isting your ideas. You also
‘may want to browse through the ideas below that already have been proposed,
Liseeae
ere ~ lest Te
The three elicitation questions for this part were:
(1) If you were offering advice on the best way to undertake system dynamics
modeling, what specific core activities would you say are essential for
exemplary PROBLEM IDENTIFICATION AND DEFINITION? In this area,
what are the most important practices of modeling work?
If you were offering advice on the best way to undertake system dynamics
modeling, what specific core activities would you say are essential for
exemplary SYSTEM CONCEPTUALIZATION? In this area, what are the
most important practices of modeling work?
If you were offering advice on the best way to undertake system dynamics
modeling, what specific core activities would you say are essential for
exemplary MODEL FORMULATION? In this area, what are the most
important practices of modeling work?
(2
(3
After the two-week period of idea generation, the participants then clustered the
ideas elicited in the first stage into categories that included ideas that they
considered similar or that belonged together. Individually generated clusters were
6The 19" Intemational Conference of the System Dynamics Society, Atlanta, Georgia, USA July 23-27, 2001
Figure 4.- Web
Page for the
Facilitated
Meeting (Part 3)
compared and extracted the minimum common multiple, or the ideas that
everyone considered that belonged to the same cluster. The final “group” clusters
were the ones used in the next part of the study.
In the third part, the participants assigned priority scores to the clustered ideas
according to the relative importance of each one as essential for the particular area
covered. To complete this task, the participants received the next set of
instructions.
Instructions for Participants: A fter clicking on the button "Prioritize
Categories" you will see X categories with one to ten ideas listed in
distinct clusters. At first, all X categories are shown with 100 points as
equally important, but you may believe that some categories of best
practices may be more or less important in specific area. You may
raise or lower the 100 points for each category as you prefer: a
category with 1000 points would be interpreted as a more important
best practice by ten times a category with 100 points. Any time you
click on a "Sort" button, your screen will be refreshed with all the
categories reordered by your changes. The full set of X ideas is
displayed below.
For Problem Identification and Definition 81 ideas were generated and placed in
49 categories, for System Conceptualization 65 ideas were generated and placed
in 38 categories and, for Model Formulation 69 ideas were generated and placed
in 42 categories.
einet Explorer
[Ele Ech View Favorites Tools Hela
|e
bi |ia- & El
Beck 7 Fuel "Slop Aish Hane Seach Faveies_Hitow | Mal Part Dine
(ides [1 ton cae conrnin Conroe eh =] 26. | [tis *
Welcome to Best Practices in System Dynamics j
Modeling.
Ifyou were offering atvice on the best way to undertake system dynamics modeling, what specific core activities
would you say are essential for exemplary PROELEM IDENTIFICATION AND DEFINITION? In this area,
what are the most important practices of modeling work?
Instructions for Participants: Alter clicking on the button "Priontize Categories" you wil see 49 categories with one to ten
ides isted in distinct clusters. At frst, all 49 categories are shown with 100 points as eanually important, but you may bellewe
‘that some categories ofbest practices may be more of lees important in problem sdensfication and defniion. You may raise or
Jower the 100 points for each category as you prefer a category with 1000 points would be interpreted as a more important
best practice by ten times a category with 100 points. Any time you click on a ‘Sort" button, your screen will be refreshed with
all the categories reordered by your changes. The fill et of 81 ideas is displayed below
Piiotize Categatios
List of Proposed Ideas:
ask: why is current behavior of key variables generated, whatis causing it?
scentiy issues
[Biber 17 Pe Tetemat
I. J. Martinez and G.P. Richardson: Best Practices in System Dynamics Modeling 7
Figure 4 shows the screen that the participants saw during the prioritization part
of the meeting. Each participant used a different scale to evaluate the categories of
practices; the sum of the ratings accounts for a total that served as a normalizing
factor for the evaluations. Then it was accumulated with the others’ responses and
weighted to obtain the total relative assessment for the category or practice’.
There were four thresholds chosen for the final selection process. The meeting
was facilitated by 404 Tech Support (L.L.C.) that administers the WebWide
Participation pages that were used for this study.
This paper reports on the first phase that included (1) Problem Identification and
Definition, (2) System Conceptualization and, (3) Model Formulation. Phase 2
will include (4) Model Testing and Evaluation, (5) Model Use, Implementation,
and Dissemination, and (6) Design of Learning Strategy /Infrastructure’,
Summary of the Results
The results of the three-part meeting are presented ‘ in tables 1, 2, and 3 below.
These tables are part of what constitutes best practice in systems dynamics
modeling as seen by the group of experts from the system dynamics society that
participated in the study. Each table focuses on one of the three key areas of
activity in system dynamics modeling - problem identification and definition,
system conceptualization, and model formulation. - Each item in these tables
summarizes a cluster of ideas generated by study participants and ranked highly
by all participants. The exact words in Table 1, 2, 3 and 4 was selected by author
team and represents a slight editing on one of the participants own words shown
in the “raw data” tables in Appendix 2. Table 4 differs from tables 1, 2, and 3 in
that it represents in summary form clusters of practices on which study
participants disagreed. These are practices that tend to divide the opinion of
leaders in the field.
2 For details on the computation method, see appendix 1 of the paper.
3 This area was not considered in the beginning of the study and was added thanks to an
especially meaningful contribution to the project from a very committed participant, Barry
Richmond. The authors want to acknowledge and thank his contribution.
’ For the raw data, see appendix 2 of the paper.
8The 19" Intemational Conference of the System Dynamics Society, Atlanta, Georgia, USA July 23-27, 2001
Table 1. - Best
Practices in
Problem
Identification and
Definition
Highest Rated
1) Talk and listen reflectively to problem owners (clients).
2) Clarify the purpose (e.g. strategy/policy, theory building, education,
training).
3) Identify the reference mode: The central “process” or time development
to be studied and use reference mode diagrams to explore people's
expectations of future behavior.
4) Ask why is current behavior of key variables generated, and what is
causing it.
5) Formulate the dynamic hypothesis (i.e., “this behavior is caused by that
structure”).
Highly Rated
6) Clearly identify clients of the model or the person to whom you need to
answer.
7) Identify and engage key stakeholders.
8) Describe clearly the symptoms that initiated the modeling proposal.
9) Identify carefully the time horizon and the time unit of the model (years,
months, weeks).
10) Develop and sketch out desirable vs. undesirable futures of key variables
over time.
Moderately Highly Rated
11) Verify whether problem stated by client is suitable for system dynamics
study.
12) Form a study team consisting of technical people and system
participants.
13) Generate a concise and specific dynamic feedback time-dependent
problem statement.
14) Identify available time and budget for the study.
15) Identify all available data sources.
I. J. Martinez and G.P. Richardson: Best Practices in System Dynamics Modeling 9
Table 2. - Best
Practices in
System
Conceptualization
Highest Rated
1) Recognize that conceptualization is creative - there are no recipes-
approach it from different angles and avoid rigid separation of the
identification / conceptualization stages.
2) Generate a dialogue with the problem's owners that addresses their
mental models and the dynamic hypothesis.
3) Start with major stock variables to describe the system, draw their
reference modes and make sure their names are nouns, not verbs or action
phrases.
Hi Rated
4) Set main goal to generate an endogenous dynamic hypothesis.
5) Be sure dynamic hypothesis boundary is large enough for endogenous
orientation.
6) Identify key variables representing behavior.
Moderately Highly Rated
7) Be sure that each variable is measurable - at least in principal.
8) Look at all available data.
10The 19" Intemational Conference of the System Dynamics Society, Atlanta, Georgia, USA July 23-27, 2001
Table 3. - Best
Practices in Model
Formulation
Highest Rated
1) Start small / simple and build out / add complexity later quantifying the
structure a bit at a time.
2) Leverage the power of dimensional consistency; use it from the very
beginning.
3) Be sure equations make sense: all parameters must have real life
(explicable) meaning.
Highly Rated
4) Set main goal to generate the smallest model that captures dynamic
hypothesis.
5) Simulate as early as possible and often, testing even simple models
extensively.
6) Discuss model and simulation outcomes with a study team that includes
the client, and revise as necessary.
Moderately Highly Rated
7) Develop a small (<100 equations) prototype (full scope not detailed) and
use it to test dynamic hypothesis and identify shortcomings.
8) Avoid making equations unnecessarily complicated and avoid chained
table functions.
9) Bear bounded rationality in mind, especially in rate equation formulation
(but also in general).
10) Try always to describe truthfully what happens in real world (limited
rationality / information).
11) Take an apprenticeship (1 - 2 years) with an experienced system
dynamics coach and acquire experience with many types of models from the
literature.
I. J. Martinez and G.P. Richardson: Best Practices in System Dynamics Modeling 11
Table 4. -
Controversial Best
Practices.
Problem Identification and Definition
1) Identify the class of systems to which the particular case belongs.
2) Model the class to which the case belongs, not the case at hand.
System C onceptualization
1) Iteratively sketch causal loop diagrams, identify state variables / levels,
identify system boundary.
2) Draw the structure of your dynamic hypothesis as a causal-loop diagram
if stock-and-flow structure presents difficulties. Concentrate first on
identifying main connections and major loops (loop explanations for
reference modes).
3) Identify / draw stock-flow structures (resources, customers, products /
services) and identify influences on flows.
Model Formulation
1) Select a “core” piece of structure and grow from it (select / add / analyze)
never straying too far from a running model.
2) Think of extreme condition tests in writing equations; simulate different
extreme conditions and check if equations work in those conditions;
otherwise modify the model.
12The 19" Intemational Conference of the System Dynamics Society, Atlanta, Georgia, USA July 23-27, 2001
Figure 5. -
Percentages of
Agreement on
Clusters related to
Problem
Identification and
Definition.
Figure 6. -
Percentages of
Agreement on
Clusters related to
System
Conceptualization.
Figures 5, 6, and 7 present the percentages of agreement across the different areas
of study. We can see that the percentages are very similar. Roughly 66% of the
issues received mixed or low priority ratings. These issues are called “indistinct
issues.” The indistinct issues are those issues that were proposed by the experts as
exemplary work in system dynamics modeling and the group did not ranked them
even moderately highly. This means that for the individuals of the group two out
of three clusters of ideas did not rank even moderately highly, even though the
design invited all participants through the eliciting question to contribute
examples of exemplary work in system dynamics.
[ Best Practices in System Conceptualization ]
Highest Rated
3 (8%)
Controversial Issues| | Indistinct Issues | |Highly Rated 3
3 (8%) 27 (71%) (8%)
Moderately
Highly Rated 2
(5%)
Total Number of Clusters Generated = 38 ]
[[BestPractices in Problem Identification and Definition _|
Highest Rated
5 (10%)
Controversial Issues Indistinct Issues | |Highly Rated 5
2 (4%) 32 (66%) (10%)
Moderately
Highly Rated 5
(10%)
Total Number of Clusters Generated = 49 ]
I. J. Martinez and G.P. Richardson: Best Practices in System Dynamics Modeling 13
Figure 7. -
Percentages of
Agreement on
Clusters related to
Model
Formulation.
[ Best Practices in Model Formulation ]
Highest Rated
3 (7%)
Controversial Issues| | Indistinct Issues | |Highly Rated 3
2 (5%) 29 (69%) (7%)
Moderately
Highly Rated 5
(12%)
[ Total Number of Clusters Generated = 42 ]
In addition, we can see that the number of controversial issues is also steady,
(between 2 and 3) which represents an average of 6 % of the practices generated.
Discussion and Conclusions
The main implications of this study group into four categories to (1) tangible
results and their implications, (2) the general process followed in this study, and
finally (3) the controversial category. The discussion and conclusions follow.
Tangible Results
The results of the study (Tables 1, 2, and 3) represent what these group of experts
in the field think are the key elements in problem identification and definition,
system conceptualization, and model formulation. There are no big surprises in
these results. The distinctive element of these lists is the advantage of having
them presented in a concise form. Additionally, the group of experts that
participated in the study shares the perception of these practices being “best
practices” in the field, which brings credibility and guidance to a larger group of
practitioners.
General Process
The processes followed in the study allowed the participation of experts in the
field that are geographically disperse. This, in itself, is an interesting product of
the study that enlightens us about the capacity of collaboration that the field has
today. A great deal of what happened during the course or this research was a
group process of alignment of visions.
14The 19" Intemational Conference of the System Dynamics Society, Atlanta, Georgia, USA July 23-27, 2001
Controversial Category
The controversial category is a very interesting finding of the study because it
represents what these experts in the field do not agree on and, therefore, can
generate (or has already generated) distinct threads of thinking within the field.
These disagreements are not necessarily detrimental for the field; these
disagreements can be a natural and even beneficial event in the growth of the
field. When different worldviews collide, a different environment emerges; this
new status can generate a major break-through way of thinking that would expand
the borders of the field that experiences these differences. Alternatively, they
could lead to divisions, miscommunication, and a lot of annoyance.
In problem identification and definition, there is a clear debate over modeling the
class of the system or the case at hand. This difference of opinions can be
explained by the differences in the types of practitioners who participated in the
study. This difference tell us that there is a group of practitioners who consistently
prefer to model the case at hand, as opposed to other group who thinks that the
adequate way is by modeling the class to which the system belongs.
In system conceptualization, even though there is agreement on starting with
major stock variables (best practice C), there is disagreement on iteratively using
a casual-loop diagram approach or a stock-and-flow approach to conceptualize.
Most of these experts agreed on where to start the conceptualization (stocks) but
not on how to proceed from there.
In model formulation, there are two major areas of disagreement on how to
formulate models. The first relates to the issue of starting small and continuously
simulate and preferably, always having a running model. This disagreement
indicates us that there is a group of experts who formulate piece by piece and
always try to have a running model at hand, and another group who prefers to
formulate in big chunks and is not concemed about continuously having running
prototypes.
The second disagreement in this category relates to the use of extreme condition
tests on the model. This disagreement seems to indicate that some experts think
that the use of the extreme condition tests is crucial and some do not think of it as
important. The authors just do not have a way to understand why is this issue
controversial.
The results presented in this paper are consistent with the literature and indicate
us areas of opportunity for growth in the field. The disagreements encountered
can be a vehicle to expand the limits of the approach. The issue of identifying a
comprehensive list of practices and ranking them is not addressed in these
publications, perhaps because what makes a determined practice in a field (e.g.,
system dynamics) a "best practice" is the relation between the personal judgment
I. J. Martinez and G.P. Richardson: Best Practices in System Dynamics Modeling 15
of the practitioner and the social judgment of the community of that field. The
interaction of the individuals of the community and the community itself
generates the social construction of “best practices.” In fact, the very existence of
the community is the result of the creation of it by individuals. The world is as
complex as we create it through our worldview and through the systems that we
create to explain it, because we know that the systems we create are conceptual
formulations that aid us in the process of handling complexity in the world as we
experiment with it (Checkland and Scholes 1990).
The two most important findings of the study are that (1) we are not in agreement
with respect to how to do exemplary work in system dynamics modeling
(indistinct issues), and that (2) there are specific controversies regarding the way
to do it (controversial category). One plausible explanation for these results can
be related to the process followed in the study. How we conducted the elicitation,
the amount of time available to do it, the impossibility of clarification of the
meanings of the contributions through a discussion of the issues, among other
factors.
Next Steps
The next steps include exploring the last three areas of the system dynamics
modeling process used as theoretical framework for the study as presented in
Figure 1. These areas are (4) Model Testing and Evaluation, (5) Model Use,
Implementation, and Dissemination, and (6) Design of Leaming
Strategy/Infrastructure. Additionally, we plan to involve more practitioners in the
study so we can get a more comprehensive view of “best practices” and the actual
use of them. We want to explore more the emerging result that roughly 2/3 of
proposed best practices are not embraced as best by experts in the field.
The results of this study have generated additional questions related to “best
practices”. The questions are (1) does this results hold up under different ways of
eliciting or clustering and with larger groups? and (2) is this in the nature of
practice in a complex field?
16The 19" Intemational Conference of the System Dynamics Society, Atlanta, Georgia, USA July 23-27, 2001
References
Checkland, P. and J. Scholes (1990). Soft Systems Methodology in Action.
Chichester, UK, Wiley.
Forrester, J. W. (1963). Industrial Dynamics. The Encyclopedia of Management.
C. Heyel. New Y ork, Reinhold Publishing Company: 313-319.
Forrester, J. W. (1973). World Dynamics. Cambridge MA, Productivity Press.
Morecroft, J. D. W. andJ. D. Sterman, Eds. (1994). Modeling for Learning
Organizations. System Dynamics Series. Portland, OR, Productivity Press.
Richardson, G. P., Ed. (1997). Modeling for Management: Simulation in Support
of Systems Thinking. International Library of Management. Brookfield, Ashgate
Publishing Company.
Richardson, G. P. and A. L. Pugh, III (1981). Introduction to System Dynamics
Modeling with DY NAMO. Cambridge MA, Productivity Press.
Rohrbaugh, J. (2000). The Use of System Dynamics in Decision Conferencing:
Implementing Walfare Reform in New Y ork State. Handbook of Public
Information Systems. D. Garson. Raleigh, North Carolina, Marcel Dekker, Inc.:
618.
Scholl, G. J. (1992). “Benchmarking the system dynamics community.” System
Dynamics Review 8(3): 263-266.
Scholl, G. J. (1995). “Benchmarking the system dynamics community: research
results.” System Dynamics Review 11(2): 139 to 155.
Sterman, J. D. (2000). Business Dynamics: Systems Thinking and Modeling fora
Complex World. Boston MA, Irwin McGraw-Hill.
I. J. Martinez and G.P. Richardson: Best Practices in System Dynamics Modeling 17
Appendix 1 Method of Calculation
Computation Method of the Third Stage of the Facilitated Meeting.
Each participant used a different scale to evaluate the categories of practices; we
built a matrix with those unstandardized scores of the participants. The matrix a
(1) has elements ayy which represent the unstandardized score for element (i)
from participant (j).
a1 Ayn Ly
(1) a= 421 422 42)
a, a. ay
Then a standardized score was calculated using the total sum of the scores of the
participant to normalize the scores to 100 and to capture the relative weight given
to the specific element. The standardized scores (2) y,,; represent the relative
weigh put on element ( i ) by participant (j ).
(2) X)=3
2...)
Matrix (3) b was built using these standardized scores to calculate the top
elements for the group.
Xr M2 Mi
(3) D= Mr Xr Xj
Xi Xa. Xi
A total score (4) A, was calculated per element (category) using the elements
from matrix b.
(4) A Fx,
And a vector (5) with these total scores was built to be used as determinant
element for distinction among elements.
Ay
(5) TotalScores = A,
Ay
Four thresholds were selected for the selection process. (1) Threshold I “Highest
Rated”, (2) Threshold II “Highly Rated”, (3) Threshold III “Moderately Highly
Rated”, and (4) Threshold IV “Indistinct Rated”.
18The 19" Intemational Conference of the System Dynamics Society, Atlanta, Georgia, USA July 23-27, 2001
As a metric for general agreement, a dispersion measurement d was calculated
using the means and the variance of the scores (6).
a
XX) Xa)?
J
_—— 2
(6) d, = —"—_ *100 =" 100
dX /n M
ja
I. J. Martinez and G.P. Richardson: Best Practices in System Dynamics Modeling 19
Appendix 2 Raw Data of the Study
Appendix Table 1. - Best Practices in Problem Identification and Definition
Highest Rated
Listen carefully to Client Stories
Let most senior client say “What brought us together”
Talk and Listen reflectively to problem owners (clients)
Make sure you understand the client’s problem
Ask client sufficient questions - avoid giving premature answers
Check whether (dis) agreement on problem exists (When you are working with more
than 1 person)
Clarify purpose (e.g. strategy/policy, theory building, education, training)
Dynamic thinking - drawing graphs over time
Have client draw about 5 to 7 reference modes
Use reference mode diagrams to explore many people's expectations of future
behavior
Identify the reference mode: The central “process” or time development to be studied
Develop history of key measures
Sketch a graph of the time behavior of the supposed problem
Observe the behavior of key variables of interest over time
Select subgroup of time histories with simpler patterns to represent behavior of
interest
Draw reference modes of behavior
Plot time histories of what ever is available
Ask why is current behavior of key variables generated, and what is causing it.
Formulate the dynamic hypothesis (i.e., “this behavior is caused by that structure”)
Highly Rated
Clearly identify clients of the model
Pick or invent the person to whom you need to answer
Create a common ground of understanding between me (the modeler) and the issue
owner (the client)
Identify key stakeholders
Immerse yourself in the organization and engage stakeholders
Describe clearly the symptoms that initiated the modeling proposal
Identify carefully the time horizon
Select carefully the time unit
Develop desirable vs. undesirable futures of key measures
Sketch out the desired behavior of key variables over time
20The 19" International Conference of the System Dynamics Society, Atlanta, Georgia, USA July 23-27, 2001
Moderately Highly Rated
Check whether problem stated by client is suitable for System Dynamics Study
Form a study team consisting of technical people and system participants
Set main goal to generate an interesting dynamic feedback problem
Define the dynamic feedback problem
Generate a concise and specific problem statement
Find a puzzling time-dependent problem
Identify available budget for study
Identify available time for study
Identify available data sources
Look at all available data
Appendix Table 2. - Best Practices in System C onceptualization
Highest Rated
Avoid rigid separation of identification / conceptualization / formalization
stages
Approach conceptualization from different angles like a new creation
Recognize that conceptualization is creative - there are no recipes
Discuss the dynamic hypothesis with a study team
Engage in conversations around conceptual building blocks
Elicit client’s mental models
Identify levels / states first to describe system with and without symptoms of
interest
Identify the few (critical) main system variables (normally levels; 1-3)
Select stock variables in reference mode
Make sure stock variable names are nouns, not verbs or action phrases
Select one key stock variable in a single conservative system if more than one
variable is present
Write names of selected stock variables with space between them to draw
perceived causal links
Start with major stock variables, try to impose your feedback loops
Identify “essential” asset stock accumulations
I. J. Martinez and G.P. Richardson: Best Practices in System Dynamics Modeling 21
Highly Rated
Set main goal to generate an endogenous dynamic hypothesis
Be sure dynamic hypothesis boundary is large enough for endogenous
orientation
Identify key variables representing problematic behavior
Moderately Highly Rated
Be sure that each variable is measurable -at least in principal
Look at all available data
Appendix Table 3. - Best Practices in Model Formulation
Highest Rated
Start small / simple and build out / add complexity later
Work up through a series of simple to more comprehensive models
Quantify the structure a bit at a time
Add detail to prototype as needed to improve realism and show policy
impacts
Check dimensional consistency from the beginning
Check the units of the equations
Leverage the power of dimensional consistency
Support every concept with data or common experience
Be sure equations make sense: All parameters must have real life meaning
Be clear about what, in reality, the algebra represents
Highly Rated
22The 19" International Conference of the System Dynamics Society, Atlanta, Georgia, USA July 23-27, 2001
Keep the model simple / not too detailed
Set main goal to generate smallest model that captures dynamic hypothesis
Assess carefully whether additional structure is required
Require very good reasons to diverge from the simplest molecules
Push back hard on demands for more and more detail
Simulate early / as soon as possible
Simulate often
Test even simple models extensively
Involve client in discussions about simulation outcomes
Discuss model with a study team and revise as necessary
Moderately Highly Rated
Develop small (<100 equations) prototype (full scope not detailed)
Use Prototype to test dynamic hypothesis and identify shortcomings
Avoid making equations unnecessarily complicated
Avoid chained table functions (especially when concepts are overlapping)
Bear bounded rationality in mind (specially) in rate equation formulation (and
in general)
Try always to describe truthfully what happens in real world (limited
rationality / information)
Acquire experience with many types of models from the literature
Take an apprenticeship (1 - 2 years) with an experienced system dynamics
coach
I. J. Martinez and G.P. Richardson: Best Practices in System Dynamics Modeling 23
Appendix Table 4. - Controversial Best Practices
Problem Identification and Definition
Identify the class of Systems to which the particular case belongs
Model the class to which the case belongs, not the case at hand
System C onceptualization
Iteratively sketch causal loop diagrams, identify state variables / levels,
identify system boundary
Create comprehensive set of dynamic hypotheses (loop-explanations for
reference modes)
Identify loops and develop initial dynamic hypothesis
Identify major causal loops determining development over time of the main
variables
Draw the structure of your dynamic hypothesis as a causal diagram
Form dynamic hypothesis before modeling to depict major feedback loops
across sectors
Draw causal loop diagrams if stock-flow structure presents difficulties
Identify feedback loops
Look for a few potentially important feedback loops
Concentrate first on main connections and major loops
Identify / draw stock-flow structures (resources, customers, products /
services)
Identify influences on flows
Model Formulation
Select a “core” loopset, add loopset operationally, analyze model; iterate
select / add / analyze
Never stray too far from a simulatable model
Think of extreme condition tests in writing equations, check if equation works
in that condition
Simulate different extreme conditions and modify model
CC BY-NC-SA 4.0