Table of Contents
The “standard method”: Scripts for a group model building
intervention
Peter Otto Jeroen Struben
Dowling College, School of Business Massachusetts Institute of Technology
Information Systems Sloan School of Management
ottop@dowling.edu jirs@mit.edu
Abstract
The process of model building with a client group involves techniques and procedures for
a modeler to elicit knowledge and mental models from clients, and to guide the whole
team through the conceptualization of causal structure into formal models and potentially
simulation models. The literature provides a comprehensive overview of system
dynamics model-building processes, which are commonly used in client group
interventions. This article extends the discussion of group model-building procedures
with a description of conceptual activities and scripts using the “standard method” in a
client group intervention. While the “standard method” is not explicitly aimed as group
model building intervention, we have used this approach to work with a small client
group from the Gloucester Community Development Cooperation to support system
conceptualization, model formulation, and decision making for the build-up of a fish
factory. The “standard method” is discussed in light of this group modeling intervention
and some of the insights gained from client feedback and reflection by the stakeholders
are presented. The case discussion describes how we used a set of scripts in a group
modeling intervention to help understand and agree upon the basic structures, rather than
examining quantitative validation of the simulation model.
Keywords: Group Model Building, Scripts, Insights
Introduction
The purpose of this article is to describe the scripts used in a group model-building
initiative with the Gloucester Community Development Cooperation and to discuss the
lessons learned from applying the standard method as framework in a group environment.
Building system dynamics models with client groups has a long tradition in our field and
is well documented (e.g. Morecroft and Sterman 1994; Richardson and Andersen 1995;
Vennix 1996). In the literature several approaches to group model building are discussed
(e.g Richardson and Pugh 1981; Roberts et al. 1983; Vennix 1994) with varying stages on
how the process of constructing a computer simulation model involves a number of
conceptual activities. This article discusses the outcome of a group model initiative, using
“the standard method”.
The “standard method” refers to a framework of steps to identify key variables,
reference modes to formulate a dynamic hypothesis and finally to conceptualize a
simulation model. The idea behind the standard method is not to think directly towards a
solution at the beginning of the modeling intervention, but to gain knowledge and
insights from each individual phase in the project together with the client. Besides the
iterative nature of the model-building process, Hines’ “standard method” specifies steps
that a system dynamics modeler should consider in a consulting environment. For
example, the importance to identify key variables, which usually involves in-dept
discussion with the client, a reference mode to express a “hope” and “fear” scenario, and
in-depth analysis of the different loops in the system. The steps of the standard method
(Hines 2001) are:
1. Problem definition
a. List of variables
b. Reference modes
c. Problem statement
Momentum policies
Dynamic hypothesis (i.e. causal loops)
Model first loop
Analyze fist loop
Model second loop
Analyze second loop
Ete.
RA NU Be
The method is an evolutionary descendent from Randers guidelines for Model
Conceptualization (Randers, 1980). Randers’ emphasis on explicitly testing the dynamic
hypothesis (“dynamic behavior or reference modes” and their hypothesized “basic
mechanism or causal structures”) conform this approach as well as the emphasis on the
iterative process (“reference modes and hypotheses can change”) (step 1 and 3).
Momentum policies are a novelty; the term is borrowed from GM’s problem solving
process and they are defined as “a solution that your client would implement now if he or
she had to make a decision immediately” (Hines 2001). Another essential difference is
the loop by loop building approach, rather than starting from stocks and flows (step 4).
Together these two differences show the deviation in emphasis: where Randers (1980)
focused mainly on how to “address a meaning full whole”, the standard method
emphasizes generating insights throughout the process.
Besides applications in regular consulting, the method is mainly used as a
pedagogical framework for the course “Applications of System Dynamics” at the
Massachusetts Institute of Technology. Over the term of the semester, students address a
problem that has been predefined on a coarse scale by a real client and use the standard
method to gain insights towards and during conceptualization and analysis of a
simulation model and. Throughout the 15 week semester, students discuss with each
other their specific experiences insights and issues in breakout sessions to each other,
while in a general session potential next steps are discussed. Although students are
guided through the process more or less in phase; each will have their challenges and
insights at different steps in the process. We discuss our experience gained from a project
while taking this course. The project was continued after the semester project. In most of
the cases, students deal with one client. Because the project we have chosen for this
course involved the interaction with a group of policy makers and stakeholders, we
applied the standard method in a group modeling environment.
Group Model Building Processes
Richardson and Pugh (1981) define seven stages in building a system dynamics model:
problem identification and definition, system conceptualization, model formulation,
analysis of model behavior, model evaluation, policy analysis, and model use or
implementation. Roberts at al. (1983) suggests a similar approach to construct a
simulation model. Vennix el al. (1994) summarizes the steps and stages in model
building as shown in table 1.
Stage Steps
Problem formulation e Define time horizon
e Identify reference mode
e Define level of aggregation
e Define system boundaries
Conceptualization e — Establish relevant variables
e Determine important stocks and flows
e Map relationships between variables
e Identify feedback loops
e Generate dynamic hypothesis
Formulation : F
e Develop mathematical equations
e Quantify model parameters
Analysis/evaluation
e Check model for logical values
¢ Conduct sensitivity analyses
e Validate model
Policy analysis
e Conduct policy experiments
e Evaluate policy experiments
Table 1: Stages in model building initiatives
The stages and steps as shows in table 1 draw on long years of experience in group
model-building initiatives from leading scholars in this field. Besides the established
procedures in group model-building, Andersen et al. (1997) suggests that modelers who
engage in modeling with groups rely on fairly sophisticated pieces of small group
process, which he calls “scripts”. He defines as “a continuous stream of small-group
activity that generates produces such as a stakeholder analysis, a precise description of a
problem to be solved, a sketch of model structure, or the determination of a set of actions
to be taken”. Vennix (1996) defines a group model-building process as an initiative to
support a decision making group in structuring a messy problem and designing effective
policies to deal with it.
While Vennix el al. (1994) proposes a highly structured and well defined
approach for group modeling interventions (see table 1), Hines’ “standard method” seems
to emphasize a more emergence and intuitive framework as some of the steps are not
explicitly stated. For example, Hines’ “standard method” does not explicitly propose to
define the system boundary or to calibrate the model, and yet it is assumed that a good
modeler will do this intuitively. It is suggested that because the “standard method” is less
rigorous in terms of the level of detail, it provides a learning experience for a modeler
when going through the different stages of a model intervention. Following a less
rigorous framework in a group modeling intervention might lead to possible errors during
the conceptualization of the model. However, the “standard method” provides enough
guidelines to capture the important stages in a model building initiative, while it requires
a combination of skill and intuition to bridge the procedures which are not explicitly
stated. As Andersen et al. (1997) concludes “it becomes clear that group model building
is still more art then science”. We have used the “standard method” following the explicit
procedures and at the same time used our intuition about what will work in a group model
intervention.
More importantly, the explicit challenge according to the standard method is to
learn from each step and appreciate that, rather than working towards a solution.
Emphasis on “slowing down” the building process and therein the loop-by-loop building
approach, facilitate this process. Slowing down is difficult with a client that sets high
expectations, based upon monetary rewards and is used to translate that into “solutions”
to “known problems”. In that sense the course context makes life easier, because of the
absence of the reward system.
A Messy Problem
Gloucester in Massachusetts is one of the oldest fishing ports in the United States, with a
370-year history of harvesting a variety of fish species. Particularly with the harvest of
groundfish (the National Oceanic and Atmospheric Administration [NOAA] classifies
groundfish as a group of fish, which consists of a mixture of bottom-dwelling species
including Atlantic cod, haddock, redfish, hakes, and flounders), Gloucester became
economically and culturally an important fishing community in New England. With the
growing pressure on the stock of groundfish, primarily from distant-water foreign fishing,
and fleets of factory-based trawlers from Eastern Europe, Asia, and elsewhere during
1960-1975, the stocks declined rapidly (NMFS 1999) which forced the government to
impose new fishery controls and regulations.
The constraints from the traditional groundfishing, which changed the economic
situation for the local fishing industry, posed a challenging task for the Gloucester
Community Development Cooperation (GCDC). How could the declining revenues from
groundfishing be compensated to achieve a sustainable fishing community? Besides the
lack of revenues from traditional groundfishing, the community is also losing its identity
for being an important fishing port. Fishers fear that empty wharf space will attract real
estate developers to create condominiums, motels, and retail outlets, inalterably changing
the landscape of Gloucester. One of the possible answers was found in a new and
patented process to extract Surimi out of pelagics or dark fish (for example Herring and
Mackerel). Surimi in Japanese means "Minced Fish", it's pronounced "Sir-Ree-Mee”, and
is traditionally produced with skinless Alaska Pollack (a white fish). Surimi in brief is
fish minced meat that has been leached by washing with water then mixed with sugar and
other additives then frozen. It's widely used in Japan for the manufacturer of Fish jelly
products such as imitation crabstick (http://www.surimithailand.com/Surumi.html).
The new and unique technology for processing Surimi out of inexpensive dark
fish, which is available in almost unlimited resources, (according to NOAA, Herring and
Mackerel are underexploited species) proved to be a feasible alternative to taken into
consideration for subsidizing the lack of revenues from traditional groundfish. Although
the pelagics stocks are considered to be underutilized based on current research survey
results and historic landing pattern, there is the likelihood that an intensification to catch
these fish species could lead to unexpected interactions in the biomass. For example,
there is little information about the function of pelagics in the food chain for predators
(mainly groundfish), so if the pelagics stocks decrease, predators will find less prey,
which then influences the sustainability of the traditional groudfish stocks.
Under normal conditions, we could assume that there are enough pelagics to
harvest, considering the current stock assessments. However, a successful launch of the
Surimi factory in Gloucester could invite other fishing communities to also tap into this
lucrative business. According to GCDC the total market for Surimi is approximately
760,000 mt, growing at 10 — 20 percent per year, with Japan consuming 60 percent of the
total production. In other markets like Europe and the US, where consumers become
more health conscious, consumption of Surimi could easily reach staggering numbers.
With all the uncertainties to determine sustainability of fish stocks, lacking of decision
points to build a sustainable Surimi factory, and uncertainty of the socio-economic
implications for the community GCDC faced a highly unstructured decision environment.
Articulating the problem the client was facing was also not easy because of this highly
unstructured environment.
Stage 1: Problem Definition
Together with the client group, who consisted of Dr. Carmine Gorga, Dr. Steve Kelleher,
Dr. Damon Cunnings, and Joe Sinagra, we conceptualized a simulation model in an
iterative process with weekly meeting within the team, and bi-weekly meeting with the
client during a fifteen week (one semester) period. In the fist phase of the project we
challenged our client’s assumptions of what the boundary and the problem of the project
should be. The initial focus from our client was related to sensitivity issues around the
factory project, for example water and electricity usage, and other constraints influencing
the desired Surimi throughput. Following the “standard method”, we elicitated in our first
meeting with the client about 60 variables and parameters. By focusing on the
identification of variable, we also wanted to keep the client from thinking about solutions
at the beginning of the project.
Variables
et
Figure 1: Screen Shot of Variables
We then narrowed the list down to a number of key variables, which we clustered into
three sectors as shown in figure 2. The purpose to cluster the key variables into three high
level sectors was primarily to focus our attention to the interactions between the key
variables and for clarification of the system boundaries.
Operations Sector Community Sector
Potential Factory Output
Revenues from Fishing
Potential Demand Sustainability of communi
Attractiveness to Join
Co-operation
Potential Return on
Investment
Resource Sector
Fleet Composition
‘otal allowable Catch (TAC)
Fish Stocks
Figure 2: Key Variables and Sectors
The clustering of the variables was done within the model building team and then
discussed with the client. In reflecting the selection of key variables with the client, we
made sure the group as a whole shared the same view of the system. Visualizing the high
level view of the system - as illustrated in figure 2 - helped the client team to focus their
attention on the important factors and levers influencing the project. While we
conceptualized a number of reference modes at the beginning of the project, we only used
those as discussion boards to define the dynamic hypotheses, explained in the next
section.
With the knowledge we have gained after the first two meetings with the client
and in discussion with individual stakeholders, we formulated seven reference modes, of
which one formed the based for our problem statement (fig. 3) to represent the expected
behavior of the system. We presented the reference mode using the visual as shown in
figure 3 to the client group to first get their agreement on the expected system behavior.
Second, after the client group agreed on the reference mode, we have used this stripped-
down representation of the system in dialog with some stakeholders. The reference mode
communicated very easily even without lengthy explanations about the underlying
assumptions, to establish a shared framework of what this project is all about.
Total Revenues
Revenues from
Ground Fish
~~
Revenues from
pelagics
1996 2002 2005 2012 t
Figure 3: Reference Mode
The reference mode in figure 3 captures the decay of revenues from groundfish, due to
declining fish stocks and curtailing from the government. By 2004 the new Surimi
factory should be operational to compensate for the lack of revenue from groundfish. We
hypothesize that traditional fish stocks would bounce back due to a multispecies
approach, taking pressure away from groundfish stocks. The underlying assumption in
this scenario is that the factory would attract a number of fishermen willing to retrofit
their boat for harvesting pelagics. By retrofitting a number of groundfish trawlers into
pelagics trawlers, we assume to take pressure away from groundfish stocks. This
assumption does not hold however, if the retrofitted trawlers are replaced by other
groundfish trawlers. Using the reference mode as “concept” board, we formulize the
following problem statement in discussions with the client group:
The decline of traditional fish species and the curtailing of fishing efforts by the
Government require the fishing industry of Gloucester to identify alternative
resources to sustain their industry...
..A Surimi factory — harvesting fast renewable fish stock — might compensate for
the missing revenues from traditional white fish until their stock returns to a
sustainable level...
After the first two workshops with the client group, we were able to identify the key
variables, determine the boundaries and scope of the project, and agreed on the problem
statement, following the first stage in the standard method. The whole modeling group
moved rapidly in the first stage of the project, because the role of the main contact person
and the group structure was well defined right at the beginning, an important factor in
group modeling interventions, which is also emphasized by Richardson et al. (1992), and
Richardson and Andersen (1995).
Stage 2: Momentum Policies
Hines (2001) summarizes the benefits of recording momentum policies as following:
Momentum policies clarify for the client what solutions are currently
implemented, being implemented or simply “in the air”. The modeler will learn
more about the concern if he or she understands how people are thinking of
solving it. Momentum policies may also help to create dynamic hypotheses,
because each momentum policy is implicitly based on a dynamic hypothesis.
Recording momentum policies will provide the team with a mile- maker for
judging how far the team and the client travel during the project.
Because the project was in a very early stage, recording momentum policies was difficult.
The client group was not able to formulate policies for immediate implementation.
However, we discussed a number of possible momentum policies related to the build-up
of the Surimi factory. For example, resource constraints to use the factory to compensate
for declining revenues from white fish, and extending the capabilities of the factory in
converting fish leftovers in byproducts such as oil for the pharmaceutical industry. We
used the outcome of these discussions in formulating the dynamic hypothesis rather then
recording individual momentum policies.
Stage 3: Dynamic Hypothesis
The primary purpose of a dynamic hypothesis is an explanation for the reference mode
(not solutions), including a structure and an expected behavior pattern, while making the
assumptions explicit. In that sense, dynamic hypothesis are theories that a certain
structure or process could contribute to certain behavior patterns (Hines 2000). Based on
the information we gathered in the first two stages of the project, we formulated a number
of dynamic hypothesis, which we presented to the client group.
We were astonished how easy those concept modes communicated, even for
people who never before have been exposed to causal loops or similar types of
diagramming techniques. The client group was immediately involved in what we
assumed was going on in the system and was able to articulate agreement or
disagreement of the structure and expected behavior. The conceptualization of the
dynamic hypotheses was en emergent process and not only helped to define the clusters
of hope and fears but also to discuss and reevaluate the boundaries for the project. From a
list of eight dynamic hypotheses, which we discussed with the client, we identified the
following two (fig. 4 and fig. 5) as most relevant to capture the structure and behavior of
the system.
‘Cunailing from
Governmen
Total allowable
catch
TAC
pelagies
:
+ ereived
Available sh stock ane 5
Hope ‘Control by
R + Government Control
Overshoot Attractiveness of
+ and collapse rie
Fear
Fishing rate
+ Potential factory
output Time for soc
Total catch _ trenew
ai stock of we
white fish
Figure 4: Dynamic Hypothesis “Control and Utilization”
The dynamic hypothesis shown in figure 4 captures the influence from the control and
overutilization loops on the number of “total allowable catch” (TAC) provided by the
government. The reinforcing loop “overshoot and collapse” contributes to the decay, or
the fear scenario, whereas the two balancing “control” loops stabilize the system,
indicated with the hope scenario in our graph over time. Thus our hypothesis states that
without control from the government, the fish stock could deplete due to an increased
attractiveness for pelagics and less available white- or groundfish. The underlying
assumption from the hypothesis suggests that the sustainability of the system is
maintained by the government and not because of self-control from the fishing industry.
The dynamic hypothesis shown in figure 5 was used to capture the dynamics of
the variable “community quality of living”, a term which the client used to relate the
impact of the Surimi factory to the quality of the fishing community in Gloucester. This
variable does have multiple facets of quality for the community, like creating or keeping
jobs in the fishing industry, generating revenues, and enable Gloucester to remain what is
was; an important fishing port in the USA.
Changeovers to,
Finan’ AS) bark
Entrance
é Barrier equality
Community
Quality of Hope - Dark Fish An) +
Living ‘Attractiveness Dark Fish
—_ = i Catch
Increasing Seale
> eal
ZZ Fear, 4) om, 2 Surimi
VENUES eee Throughput
Revenues from ee
Dark and White
Fear) white fish , arial
= 0s ite sh it ah
Yield
AL White Fish 27 b+
Stocks
2:
Depletion
Figure 5: Dynamic Hypothesis “Community Quality of Living”
In this hypothesis it was the total revenues on the right that contributed to the community
quality of living’. The hope scenario assumes that there are enough renewable resources
' The variable as defined in the vertical axis on the reference mode should coincide (in name and meaning)
with one of the variables of the causal diagram. This is essential for hypothesis testing. Here we show our
exact “script”, which shows that we were not fully accurate.
in the ecosystem (both white and dark fish) to make reinvestment in plant and to create
rising stability, which reinforces the “quality of community living”. As noted before, the
factory would also act as an “incubator” to facilitate research in new fish process
technologies. Thus reinvestment in factory enables funding of these research activities,
which results in rising stability. The behavior of the slope “fearl” suggests that the
factory could have too much success, where increasing revenues result in increasing
competition from other fishing communities, which could lead to fish stock depletion and
or unequal/unfair profits within the fishing community of Gloucester. With the “fear2”
scenario, we hypothesize a delay in takeoff due to a lack of the FDA (Food and Drug
Administration) approval, sales below expectation, or increased competition from other
fishing ports along the North Atlantic cost.
In reflection, the process to formulate and discuss the dynamic hypotheses with
the client helped the whole team to clear and consistent communicate of what the
problematic and preferred behavior of the system could be. These scripts were also used
to communicate with other stakeholders, e.g. NOAA, and to draw their attention to the
important aspects and issues throughout the project. Finally we tested the hypotheses by
building the loops. Especially failure of the test provides a rich source for generating
insights. For instance balancing resources (the hope scenario) turns out to be extremely
hard, because of the natural self-defeating disconnection between fishermen and regulator
in combination with measurement delays: success of “adding an additional resource”
naturally leads to seeking the limits of sustainability. This will be discussed later.
While numerous approaches exist within the systems thinking, soft systems, and
system dynamics literatures for eliciting problem statements from groups (Lane 1993;
Morecroft and Sterman 1994; Richardson et al. 1994), Hines’ standard method focuses to
sketch graphs over time of problematic and preferred behavior, following the classical
tools suggested by Randers (1980) and Richardson and Pugh (1981).
Stage 4: Conceptualizing the Model
The standard method suggests conceptualizing the model structure of the system loop
after loop. This approach is also emphasized by Andersen and Richardson (1997);
beginning with a very simple picture of the system and add successive layers of
complexity. After we conceptualized the dynamic hypotheses, we merged then into three
sector causal loop diagrams, as shown in figure 6. We used those causal loop diagrams to
discuss the model boundaries and scope for the project. After presenting the sector causal
loop diagrams to our client group, we realized that (a) we exceeded the level of
comprehension for a group modeling environment and (b) pushed the system boundaries
too far.
Resource Sector
Figure 6: Sector Causal Feedback Loops - Spaghetti’s
At this point in the project, we went back to the reference mode and the dynamic
hypotheses to re-focus the boundaries of the project. Before we drew any new causal loop
diagrams, we discussed the scope of the project at an aggregated level with our client one
more time extensively. The diagram shown in figure 7 was used in discussion with the
client group to reflect what we considered as project scope and to get agreement for the
indicated model boundaries.
Groundfish
Trawlers
Groundfish
Stock
Figure 7: Scope and Model Structure
Figure 7 shows the suggested scope of the project and the model structure, which is
composed of nine modules: 1) Processing quality of Surimi, which could differ
depending on the composition of biomass, 2) Factory capacity, initially adjusted to
10,000 MT output per year, 3) Total Output, which it the actual output of the factory per
year, 4) Pelagics Trawlers, as number of boats used for catching the necessary quantity of
fish for the factory, 5) Relative attractiveness of pelagics, 6) Groundfish stock, which
includes primarily white fish, 7) Herring stock, 8) Mackerel stock, and 9) Groundfish
Trawlers, as a number of boats harvesting white fish (Inshore and offshore).
This script turned out to be quite useful in determine the system boundaries as
well as lay out the ground for the model structure. Even without explicit feedback loops,
the script capture the interrelation between variables and sectors, and provided a
structure, which we used as framework to conceptualize the simulation model.
Stock and Flow Scripts
Before we presented first stock-and-flow diagrams to the client group, we introduced the
methodologies of quantitative system dynamics simulation in very broad terms. We felt
this short introduction was necessary to help the client understand the diagrams, which
we presented in the meetings. Due to time constraints, we were not able to facilitate a
group session in which we conceptualized the stock-and-flow diagrams together with the
client group. However, we used simple model scripts to capture structural elements of the
system, which we then discussed with the client and if necessary changed during the
meeting. The starting point for the different stock-and-flow diagrams which we initially
presented to the client, were based on the dynamic hypotheses. Because we already had
agreement on the structure and expected behavior pattern of the sectors, conceptualizing
the stock-and-flow diagrams was relatively easy. Form our dynamic hypotheses, we
selected a first loop to conceptualize in a stock-and-flow model, which was central to the
problem and easy to represent, and captures one of the major concern from the client. The
picture below illustrates the level of detail with which we presented the stock-and-flow
script to the client group.
Reinvestment
Pactory Revenues
—_
|
Desired Surimi Funded |
Production Capacity Capacity |
w/J ow bh
ome uni
Growth —— +h ie
a % if
ae = c, | \
Sure |
per Ur
Demand
Time To
Expand
Maximum Suri ‘Actual Factory
Factory Output Output
Sari
> Production
Figure 8: Stock-and-flow diagram
One argument for of using a direct and straightforward approach, presenting relatively
detailed stock-and-flow diagram is time efficiency. The disadvantage is that this approach
does not involve the client group in the detail conceptualization of the model structure.
To choose either approach is not only a matter of time but also based on the level of
previous exposure to quantitative methods in the client group. Because our client group
consisted of experienced people in economics, control theory, and natural science, we felt
confident to use the direct approach to sketch the model structure.
Quantitative Model, Mental Models and Client Engagement
While building a model is an exciting phase for a system dynamics practitioner, it is also
the moment to lose ones’ client. As soon as this process starts, there are great risks of
divergence of mental models for several reasons: the client is less involved,
operationalization changes level and form of representation and finally the mathematics
is more abstract. In that case, a likely response after an intense and successful qualitative
stage is purely confirmative validation — a self-defeating danger of a successful initial
stage. Critical client discussions are as important during the simulation model building
process and in the analysis stages.
While true importance depends on the ultimate purpose of the project, we
(normally) want to refrain ourselves from selling an end product or just achieving buy-
inn. Moreover, there are lots of direct learning opportunities for the client as well. How to
keep the client engaged during this stage of the simulation building process, a process of
specialization and thus separation, in the case that the client is not engaged in the actual
building itself? How can we make sure to keep mental models in line, while remaining
productive?
The standard method supports this crucial process by building up the model loop
by loop (dynamic hypothesis by hypothesis). Each time a loop is added, behavior should
be fully understood. All “surprises” marked down. After about 2 loops, the outcomes,
“surprises” (and or insights) should be communicated to the client. In this gradual way
the client learns along the process and in addition (not less important) mental model
differences are resolved (and can generate new insights or work to be done).
On the other hand the 15 week course also implied serious time constraints, and
this is where we could not fully follow the suggested path. Our approach resulted in the
two intermediate communication topics. The first was centered on “assumption
discussion” and the second involved a model behavior check on one specific sector. Both
will be discussed in more detail below. While in reality those steps took place through a
more iterative and emergent process, we will discuss them as two separate entities.
Assumption discussion
During the building process all assumptions and its sources had been carefully
documented (i.e. beyond mathematical formulation). The parameters or relations that we
considered most crucial for careful communication, were those that either contained high
levels of uncertainty in their values (e.g. resource regeneration fraction, carrying
capacities), had emotional load (e.g. “fisherman desired days at sea”), or yielded large
implications for the dynamics (such as the table function relations as for yield per unit of
effort, price elasticity to quality and fractional birth/ death rates).
The first of those three will be addressed in a subsequent section; the second class
implied a careful process of selection and discussion and is something that comes up
regularly in group model processes. Communicating on the last group is crucial not only
for “validation purpose” (elicitation), but offers an opportunity for constructing sense of
what the model dynamics can do, under specific assumptions.
After discussion of the role of a table function, and the relevance of shape versus
quantitative values, several table functions (e.g. harvest yield per unit effort, fractional
birth and death of fish, effect of investment on quality of product) were sketched through
a “democratic group process”. Figure 9 shows a sample of charts that we used for
discussing the shape of the table function. We drew the shapes of 7 different table
functions (spread over the various sectors). As discussed in Ford and Sterman (1998),
who address the issues of “knowledge elicitation”, we focused on the value at the
extremes, some intermediate points, the transient shape and finished with fitting the
estimates (i.e. "drawing the derivatives”).
Sketch of “Effect of Relative Density on
Mackerel Yield’ Sketch of “Effect of Quality on Price”
Not
Applicable |
Range
Not
“| Applicable
Range
full boat retum after a 4 day trip”
Price Relative to that of average (B Grade)
Yield “1
0 1@) 2(A)
Relative Abundance (Density) of stock “1 = good times” or Quality (Conform Grades)
Figure 9: Script for Table Function (regenerated sketches)
While in the discussion of Ford and Sterman (1998) the emphasis was on
“knowledge elicitation”, for us the key objective really was on taking the mental model
of the group one step further and bridging between qualitative dynamic hypotheses to that
of quantitative model and model behavior.
We noticed that the active discussion on the role and shapes of table functions
adds a lot of value to understanding the interrelation between structure and behavior. We
also believe that this process was valuable as an investment for later stages, when model
behavior was to be analyzed at group level. Because of time-constraints, we only
discussed a few of those sketches, other table functions were presented to the client for
validation only; we doubt that in these cases we obtained the necessary feedback as well
as provided contributions to increased understanding for the group.
Analyzing model behavior — a sector
We choose to discuss one sector in detail with the group, what became the resource
sector, since it was tangible in terms of central loops. In addition, especially from the
clients’ perspective it contained a lot of “uncertainties” in terms of data, structure and the
modeling process had revealed some key insights in these.
Since most uncertainty and doubts had emerged around this area, we involved an
additional specialist group in the process, which also was a stakeholder in the process: the
NOAA (National Oceanic and Atmospheric Administration). This meeting had various
benefits: first it provided a valuable means for validation. Second it increased (positive)
contacts and understanding among Gloucester community and a key stakeholder / actor in
the system. Finally, it was an excellent trust-builder for the model (as concept).
Discussion of the model dynamics was centered initially around one simple stock-
and-flow diagram, its core assumptions, its behavioral implications and interpretation of
it (figure 10). Throughout the project, concerns had been raised on the uncertainties about
specific parameter values, to which our response was the relatively little importance
when focusing on behavior patterns. However, we feel that only after this phase this
insight was shared among the group.
GF T NDTC
aa Mk Normal ie
tk Fattonal
ral DeathRale
Mackoro ftir
Seat
2 \
s |
Wacker
Harvest Rate es }
~
4 Mueller: 3
. Dana” “tik Canyin z
Be | + i
\ ceric } GF T NBTC
\ <x. Eee Perceived MK 1 Le
ww FNBToO F F
Machorai eld
Meee
Fiesi Mackerel
s° ___Fratonat ets
Table Effect of Retative '
Mackerel Phase
60,000 = Maximum Net Regeneration
3 per Time Unit
2
45,000 3
ie
ae
33
20,000 33
a+ X% Fractional
"Harvest Rate
15.000 (pe Tine Unit:
° =
0.30 050 070 0.90 1.10 130 1.50 Nonmialized.
MK Relative Dersity 1 Population
Absolute
Harvest Level
Figure 10: Sample of level of detail for discussion: Basic Structure, Table Functions,
Dynamics and Explaining Dynamics
Yet, while we always emphasized that the process was about insights and generating a
common perspective on the problem to enhance discussion, it had probably not always
been clear to the participants what that meant and the attitude had mainly been dispersed
around one of “healthy benefit of the doubt skepticism”. From this time on, however,
when the numbers began to play, and the generated patterns were confirmed by the
audience and third parties, the atmosphere seemed to have switched to “wildly
enthusiastic” and “quantitative truth teller”. Here our role as moderators of the process
was extremely important, to emphasize the qualitative results, rather than the quantitative.
Results of the simulation effort
The project started with a diverse client group with an ill defined problem statement.
Given the complexity of the problem, the quantitative analysis contributed enormously to
the qualitative level of understanding. Central in this is the criticality of the relative
values of regeneration parameters, non-linearity therein and its relation to factory and
market response times as well as scale. Throughout the process, the separate steps each
had their independent contribution. However, we think, the understanding generated,
would not have been possible with detailed involvement of the client during the whole
process. In addition, the trust in the analysis would not have been sufficient. The next
step for the team is to bring these qualitative understandings under the light for a wider
audience.
Discussion and reflection
The following discussion is based on our observation and lessons learned from the
outcome of the group modeling intervention for the Gloucester Community Development
Cooperation. Our insights and conclusions are based on only one case where we applied
the “standard method” thus, we have no empirical evidence that the results of using this
method would differ from applying another approach for a group model building
initiative.
We believe that the method we applied in our group model building project
provides enough explicit procedural steps to guide a team through the different phases of
elicitation and model conceptualization. The scripts helped us to communicate with our
client and stakeholders throughout the different phases of conceptualization and
simulation. We were able to visualize a complex system in an easy understandable way,
using causal loop diagrams, and graphs over time.
Based on our observations from using the “standard method” we would argue that
the initial stage of the process and in particular the use of dynamic hypotheses together
with “hope and fear” scenarios, results in making the system structure and expected
behavior pattern more explicit than a reference mode. We have used a reference mode to
help define a problem statement and then used dynamic hypotheses to capture the
expected hope and fear behavior of the system. In combining the dynamic hypothesis
with the hope and fear mode of the system in one script, the client group was able to
articulate their interpretation of the system behavior and challenged the assumptions from
the expected hope and fear modes. These scripts easily communicated with stakeholders
and proofed to be a valuable tool throughout the initial stage of the project. The group
model intervention helped the client to gain insights into the dynamics of the relationship
between fishermen, the community and pelagics fish stocks. Using causal loop diagrams
and dynamic hypotheses improved our understanding into the dynamic behavior of the
system in the first stages of the group model building intervention, even without a
running simulation model.
The project also raises the issue on when and how to formulate the system
boundary, in the case of a “messy problem’. Strictly following the loop-by-loop
“standard method” implies keeping flexibility high to make use of insights gained and
knowledge acquired along the process. This results in a late formulation of the boundary.
Although we definitely benefited from this perspective in our project, we also felt the
need for some structure along the way. This resulted for example in the “indicated project
scope” as shown in figure 7.
The experience we have gained from applying the “standard method” in a group
modeling process, lead to the conclusion that (a) especially combining the dynamic
hypothesis with a hope and fear scenario in one script provides not only a check for
“meaningful and focused representation”, but directly provides an opportunity for client
group and stakeholders to obtain important insights into the structure and behavior of the
system, even before using the simulation model. This script, which can be subject to
change, can be used throughout the process for different purposes and provides thus a
very powerful addition to those normally used in group model building initiatives. And
(b) while the framework of steps and processes guides the team through the process, a
modeler still needs a certain amount of intuition to facilitate a group model initiative, as
indicated in the literature (Andersen et al. 1997). Specifically, this flexible process on
what to do when and how leaves room for intuition and thus, emphasizes the specific
learning experience in a model building initiative. For us the most insightful phases were
in the dynamic hypothesis formulation (which enabled us to go back identify the actual
problem statement) and the modeling of the hypotheses that related to the “pelagics
versus ground fish” dynamics. A critical condition for this is a client that is willing to
learn from the steps, rather than from the conclusion.
In reflecting the experience we have gained from this case, we would argue that
the strengths of the “standard method” in the context of group-model intervention with
“clear-cut guidelines” is to leave more flexibility for intuition, as well as that it provides a
unique script to visualize and test the expected behavior of the system by combining
causal feedback loops with “hope and fear” reference modes.
References
Andersen, D. F., Richardson, G. P., and Vennix, J. A. M. (1997). “Group Model
Building: Adding More Science to the Craft.” System Dynamics Review, 13(2),
187-201.
Andersen, D . F, and Richardson, G.P. (1997) "Scripts for group model building." System
Dynamics Review, 13(2), 107-129
Ford, D. N., and Sterman, J. D. (1998). “Expert Knowledge Elicitation to Improve
Formal and Mental Models.” System Dynamics Review, 14(4), 309-340.
Hines, J. (2000). “Momentum Policies and Rough Dynamic Hypothesis.” , Massachusetts
Institute of Technology.
. (2001). “The "Standard Method".” , Massachusetts Institute of Technology.
Lane, D. C. (1993). “The Road Not Taken: Observing a Process of Issue Selection and
Model Conceptualization.” System Dynamics Review, 9(3), 239-264.
Morecroft, J. D. W., and Sterman, J. D. (1994). “Modeling for Learning Organizations.”
System Dynamics Series, Productivity Press, Portland, OR, xxiiit+400.
NMFS. (1999). “Our Living Oceans. Report on the status of U.S. living marine
resources.” , U.S. Dep. Commer., NOAA Tech. Memo. NMFS-F/SPO-41.
Randers, J. (1980). “Guidelines for Model Conceptualization.” Elements of the System
Dynamics Method, J. Randers, ed., Productivity Press, Cambridge MA, 117-138.
Richardson, G. a. A. L. P. I. (1981). Introduction to system dynamics modeling with
DYNAMO, MIT Press, Cambridge, Mass.
Richardson, G. P., and Andersen, D. F. (1995). “Teamwork in Group Model Building.”
System Dynamics Review, 11(2), 113-137.
Richardson, G. P., Andersen, D. F., Maxwell, T. A., and Stewart, T. R. “Foundations of
Mental Model Research.” 1/994 International System Dynamics Conference,
Sterling, Scotland, 181.
Richardson, G. P., Andersen, D. F., Rohrbaugh, J., and Steinhurst, W. “Group Model
Building.” Proceedings of the 1992 International System Dynamics Conference of
the System Dynamics Society, Utrecht, the Netherlands, 595-604.
Roberts, N. H., Andersen, D. F., Deal, R. M., Grant, M. S., and Shaffer, W. A. (1983).
Introduction to Computer Simulation: The System Dynamics Modeling Approach,
Addison-Wesley, Reading, MA.
Vennix, J. A. M. “Building Consensus in Strategic Decision-Making: Insights from the
Process of Group Model-Building.” 1/994 International System Dynamics
Conference, Sterling, Scotland, 214.
- (1996). Group Model Building: Facilitating Team Learning Using System
Dynamics, Wiley, Chichester.
Back to the T
