Very Large System Dynamics Models - Lessons Learned
Jacob J. Jacobson!
Leonard Malczynski”
Vincent Tidwell?
Abstract
This paper provides lessons learned from developing several large system dynamics (SD) models.
System dynamics modeling practice emphasize the need to keep models small so that they are
manageable and understandable. This practice is generally reasonable and prudent; however, there are
times that lange SD models are necessary. This paper outlines two large SD projects that were done at
two Department of Energy National Laboratories, the Idaho National Laboratory and Sandia National
Laboratories. This paper summarizes the models and then discusses some of the valuable lessons
learned during these two modeling efforts.
Key Words: Large models, model validation
Introduction
The Idaho National Laboratory and Sandia National Laboratories have been developing capabilities in
system dynamics. As national laboratories, the researchers are tasked with large complex problems that
can have long and significant impacts to national policy. The nature of the problems that the labs
address with system dynamics typically places the models in the category of large System dynamics
models. Although large models are not the preferred modeling practice within the SD community there
are times when itis necessary.’ There are some valuable lessons to be extracted from the work we have
done at the national laboratories developing large system dynamic models. The purpose of this paper is
to discuss the lessons leamed from developing very large SD models. This will be done through two
recent case studies, one at the Idaho National Laboratory and the other at Sandia National Laboratories.
Case 1 - VISION
Model Description
! Jacob. J. Jacobson, Advisory Scientist, Idaho National Laboratory, 2525 N. Fremont Avenue, Idaho Falls, Idaho, 83415.
acob jacobson@inl.gov, 208-526-3071
? Leonard Malczynski, Principal Member of Technical Staff, Sandia National Laboratories, Albuquerque, New Mexico.
lamalcz@ sandia.gov, 505-844-7219
3 Vincent Tidwell, Principal Member of Technical Staff, Sandia National Laboratories, Albuquerque, New Mexico. vctidwe@ sandia.gov,
505-844-7219
The United States Department of Energy’s Advanced Fuel Cycle Initiative’s (AFCI) System Analysis
group is performing broad system analyses of future nuclear energy in the United States.? The Idaho
National Laboratory (INL) has been collaborating with Argonne National Laboratory (ANL) and Sandia
National Laboratories (SNL) in developing a system dynamics (SD) model of the U.S. commercial
nuclear fuel cycle. The Verifiable Fuel Cycle Simulation (VISION) model is being used to analyze and
compare various proposed technology deployment scenarios.’ The model is designed to give a better
understanding of the linkages between the various components of the nuclear fuel cycle that includes
uranium resources, reactor number and mix, nuclear fuel type and waste management. The model has
evolved into a very large dynamic simulation model. At the outset, the model was envisioned to be
complex and complete but the relative size was not expected to be nearly as large as it has evolved to.
Two of the reasons for the evolution from a mid-sized model to a large model are necessity and success.
As the model evolved, it because evident that more detail was needed to be able to properly capture the
dynamics of the system at the level of detail required by the customers. In addition, as the model
developed other groups saw the value of this type of model are added their requirements to the mix of
previously established requirements.
There are a number of distinct components in the nuclear fuel cycle. These components, as outlined in
Figure 1, include mining and milling of raw uranium, fuel fabrication, reactors, spent fuel storage, spent
fuel separations, and waste management. Each of these components although separate components are
tightly connected to the nuclear fuel cycle but usually analyzed in isolation of the other parts. This
model links these components into a single model for analysis and includes both mass flows and
economics. VISION is intended to assist in evaluating “what if’ scenarios and in comparing fuel,
reactor, and spent fuel separation alternatives at a systems level for U.S. commercial nuclear power.
The model is not intended as a tool for process flow and design modeling of specific facilities or for
tracking discrete units of fuel or other material through the system. VISION is intended to examine the
interactions among the components of the U.S. nuclear fuel system as a function of time varying system
parameters and provide a comparative analysis of different scenarios; this model represents a dynamic
rather than steady-state approximation of the nuclear fuel system.
QBtion tortaking |
Natural uranium U from outside |
available inopecan
Enrichment 5
Natural Uranium |UsOs UFe EUF. | Fuel Fabrication
shag an in caavern ad Sn Suc
[Module A] [Module G2} [Module D1]
EUFe vox
7 DU
Uranium Fuel Fabrication —
Re
[Module D2}
‘Option for taking | il
Pu from outside | —) =
program fo
Storage ee Gare we
[Module £3]
Recovered U (AU) Module | Np.Pu
Toute Ke, Ka) = Am,Cm Interim SNF
‘Separation — e — Wet
Aqueous Sosuie E11
RU Recycle Zr [Module F1]
nee, Interim SNF
Liw Zr, Cs, Sr acs
Te,|, La, Zr, UDS, residual (Meat
TRU, balance of FPs ‘SNF
Conditions Conditioning
ioning,
Storage, and Storage, and TRU Storage, and SNE Consitioning
rin erg ing
[Module G3} [Module G4] [Module Modus ea *iModule 2}
‘TRU Conditioning, r
and term
mea ey Rotrisvabie Storage
Ifadequate [Modi
‘decay, etc mee [Module 1]
Near-Surface Other Disposal
Disposal Re Concepts.
[Module J) [Module LY [Module My
IC Transportation Processes for High-Level Radioactive Materials (SNF, HLW, TRU, LLW-GTCC, recycled fuel) [Module O1] »)
Transportation Processes for Low-Level Radioactive Materials (LLW-near-surface, unirradiated fuel, RU, DU, EU) [Module 02] )
Figure 1: Schematic of VISION modules representing the nuclear fuel cycle processes and facilities and
showing mass flow.
VISION is a large model by SD measures. The following list describes some of the
model details:
e Composed of 42 modules (a module represents a separate component of the
system)
Approximately 3,300 variables
68 ranges (named arrays or matrix dimensions)
2,233,794 elements e.g. a variable with the dimension
4 sub-models.
In fact, the model is so large that the original version of the model which was developed
in Stella* had to be rebuilt in Powersim Studio® because it became too large to add any
more variables.
AFCI Systems Analysis group developed a set of four possible strategies that they were
most interested in understanding and comparing. These four strategies basically covered
the entire option space in terms of separations and waste management paths. In this
context, a strategy is a general approach to fuel management that encompasses a range of
options with similar characteristics. A strategy identifies nuclear reactor mix and recycle
strategies. The four strategies are:
e The current U.S. strategy is once-through - all the components of spent fuel are
kept together and eventually sent to a geologic repository. This strategy uses
existing types of nuclear power plants, which are all thermal reactors.
e The second strategy is limited recycle, recycling transuranic elements once.
Remaining transuranic elements and long-lived fission products would go to
geologic disposal. Uranium in spent fuel, depleted uranium, and short-lived
fission products would be disposed as low-level waste. This strategy uses
existing types of nuclear power plants, which are all thermal reactors.
e The third strategy is continuous recycle, recycling transuranic elements from
spent fuel repeatedly until destroyed. Continuous recycle is more technically
challenging than limited recycle and therefore more research, development, and
deployments would be required. Uranium in spent fuel can be recycled or
disposed. Essentially no transuranic elements would go to geologic disposal.
Long-lived fission products would either go to geologic disposal or some could be
transmuted in power plants. Short-lived fission products would be disposed as
low-level waste. This strategy would primarily use thermal reactors; however, a
small fraction of fast reactors may be required.
e The fourth strategy is sustained recycle, which differs from transitional recycle
primarily by enabling the recycle of depleted uranium to significantly extend fuel
resources, This strategy would primarily use Generation IV fast reactors.
Figure 2 presents the four strategies in a diagram that outlines the path for each strategy.
VISION was designed to simulate each of the above four strategies. Below are some
results that demonstrate the outcomes and comparison capabilities that are available
through VISION.
redibility of multiple
repositories enables
once-through
Inventories increase
gi Many costly packages
Complete once-through strategy ———>> Many repositories
Eventual U ore constraint
Toda Limited Intermediate steps gives
u recycle immediate benefits &
partial long-term benefits,
faster implementation,
Start Continuous | * less R&D required,
recycling recycle less economic uncertainty
. Inventories stabilize
Sustained Few HLW packages
recycle One repository
AFCI enables No U ore constraint
recycle options
* Fast reactors required for sustained recycle,
likely required for continuous recycle
Figure 2: This diagram depicts the four basic strategies for nuclear growth in the US.
For each strategy, there are a myriad of options that need to be set, growth rate, fuel type,
reactor parameters, process times. In order to make the comparisons useful, most of the
parameters were held constant through out the scenario comparisons. Only those
parameters necessary to match the basic strategy were modified.
We developed these five scenarios that covered the 4 key strategies listed above. The scenarios
are:
* Once-through (current US strategy)
* Multiple recycles mixed oxide fuel
* Multiple recycles inert matrix fuel
*Thermal fuel supplying fuel to burner fast reactor with multiple recycles
¢Thermal fuel supplying fuel to breeder fast reactor with multiple recycles
Each scenario starts in 2000 with spent fuel recycling starting in 2020 (where appropriate), and
growth is 1.8% per year starting in 2010. Figure 3 shows a comparison of the 5 scenarios for
total fresh uranium usage.
Total consumed uranium ore
6,000
5,000
w 4,000 —vUOX
Q —MOX PuNpAm
8 3,000 —IMF PuNp
2 —FR Bumer
~ 2,000 —FR Breeder
1,000
0 + + + +
2000 2020 2040 2060 2080 2100
Year
Figure 3: This chart shows a comparative graph of uranium consumption across the 5
different scenarios.
Figure 4 shows a chart for a single run that breaks the costs down across the five major
cost elements, front end costs, back end costs, recycling costs, reactor operation costs and
reactor capital costs. These types of charts are very useful in identifying the components
that are most influential on the overall costs.
Annual Total Cost Breakdown - Run 1 ($)
Total Front End Costs each year @ Total Back End Costs Each Year
m Total Recycling Costeach Year m Total Annual Capital Cost P er Year
@ Total Reactor Operation Cost Per Year
$M/yr
2000 2020 2040 2060 2080 2100
Figure 4: This charts shows the break down of the costs for a single run across the five
major cost components.
VISION is able to produce a multitude of comparison charts such as the one shown above
that allows for a quick comparison of different strategies on a variety of metrics.
Modeling Process
The INL has a set of requirements for every software development project which closely
follows ANSI/IEEE 1058.1-1987 Standard for Software Project Management Plans.
Prior to any modeling activities it was necessary to develop a software management plan
(SMP) and software requirements specifications (SRS) document.® The SMP outlines
the development responsibilities, the development process, procedures for revision
control and procedures for verification and validation. The SRS establishes the
customer's minimum requirements that the model must meet at completion. In the SRS,
it is important to outline not only what the model should do but also what the model is
not intended to do. The document should capture the end-users expectations and make it
clear what is expected. The SRS is a “living” document and can and should be updated
as the modeling progresses. For very large models it is unlikely that all the specifications
will be captured at the start of the process. Any changes to the SRS should be revision
controlled so that the new requests are documented.
The SRS will also be helpful in determining the correct platform that the modeling should
be done in. It is at this point that the programmer/modeler determines whether this is in
fact a system dynamics model or if there is a more appropriate too for the application. It
the end user desires discrete tracking or components throughout the lifecycle of the
component then SD is probably not appropriate and forcing the model into that platform
is going to lead to an inferior product.
VISION is about process flow of material, so the most natural way to divide up the model
is by “plumbing”. There were advantages to this process, such as, when working with
subject matter experts in each area they were able to visually understand how we were
modeling their sector by the views.
Modeling
VISION is the collective knowledge of subject matter experts from every area of the
nuclear fuel cycle.
Fuels
Reactor Physics
Waste Management
Separations
There are many experts in each of the areas identified above but very few experts that
understand every area of the nuclear fuel cycle and how those areas interact together in a
unified system. VISION was able to capture the knowledge of subject area experts in
each of the above areas and link the processes together into a functional system.
Benchmarking and Validation
Validation of large complex dynamic models is very difficult. It is important that good
modeling practice be exercised from the start in order to facilitate the validation process.
Naming convention, model structure, units should all follow a strict guideline that is
established before any modeling is started.
During the modeling process it is very important to manage the customer and their
requests. Any new request for enhancements needs to be evaluated for its value versus
the complexity to add it to the model. Many requests add very little in terms of supplying
understanding but a great deal in terms of complexity. The effects are in the “noise” but
require a lot of modeling effort and validation. The modeler needs to be continuously
asking the questions,
e “What is the value added with this enhancement”
“How difficult is it to add to the model”
“How difficult is it to validate the change”
From these questions, a return on investment can be estimated and the customer can
determine if they are willing to pay the price for the enhancement. There are times when
an enhancement will retum little but the customer feels strongly enough about it that they
want it added anyway. This can be very frustrating for a modeler when time and
resources are being stretched to meet deadlines.
Models like most analysis activities have but one life. Anyone who uses the model and
determines the model produces invalid results will lose confidence in the model results. It
is very difficult if not impossible to re-establish their trust in the model. Therefore, it is
essential that prior to any release of the model to verify and validate the model.
Due to the nature of the model and the intended end-use it was imperative that VISION
be tested and validated at a very rigorous level. There were several levels of verification
and validation performed on VISION. The first was done in-house. The model went
through a number of tests including unit consistency, boundary adequacy, extreme
conditions and integration error. In addition, the model was tested against other models.
The second step in the validation process was to have an outside source check the model.
Modelers from Sandia National Laboratories that were not involved in the development
of the model were requested to do an independent evaluation of the VISION model.
Ideally they might have been involved in the modeling process as the model progressed.’
The results were compiled in a report delivered to the INL upon completion. The basic
finding of their report, using objective tests, verified that VISION was producing
replicable and reasonable results.
The benchmarking activity was conducted to verify whether results obtained with the
VISION model reasonably approximate system performance, as well as to provide a basis
against which future model revisions can be subjected to regression. The five benchmark
calculation methods included the DYMOND®, CAFCA-II°, and NFCSim!° system
analysis codes, analysis results from a recent NEA/OECD report, and Microsoft Excel
spreadsheet calculations. Comparisons are made for alternative scenarios for four
altemative model strategies of nuclear power plant fuel cycles, including the current
once-through cycle, recycling of fuel through thermal reactors, a two-tiered combination
approach for recycling using both thermal and fast reactors, and recycling through fast
reactors alone.
Model Applications
It is important to note that VISION has been used for system level analyses since its first
revision. The model was supplying an understanding of the complexity of the nuclear
fuel cycle from the onset and so has been involved is supporting the AFCI and GNEP
activities early on. It is important to note that an operational version of VISION has been
necessary since the beginning. Because of this requirement the development of VISION
has followed an incremental development process. Each installment of VISION has
added new options and additional complexity to the analyses. Because VISION was
demonstrating its benefits early on, the funding for the project was never interrupted or
constrained even when other programs were experiencing cutbacks. Listed below are
several of the activities that VISION has supported:
e VISION has been used to support the analyses that compile the annual AFCI
Report to Congress. (2005, 2006, 2007)
e VISION is being used to support the analyses for the Global Nuclear Energy
Partnership (GNEP) report to Congress. (Spring 2008)
e Three U.S. Universities are using VISION in their graduate level nuclear fuel
cycle classes and six others have requested a copy. (Fall 2007, Spring 2008)
e Three masters of Science graduate students have developed sub-models as their
master’s projects.
e Five other students are working on sections of VISION for their master’s and PhD
projects.
VISION is proving to be a good analysis tool for AFCI and GNEP but is also proving to
be a good educational tool for universities.
Case 2 - Rio Grande Model
Model Description
The MRG Model is structured as a dynamic water budget with each supply and demand
component treated as a spatially aggregated, temporally dynamic variable. The spatial
extent of the region is defined by the boundaries of Bernalillo, Sandoval, and Valencia
counties in New Mexico. The various water supply, demand, and conservation terms are
generally aggregated over the three-county region; however, in some instances features
outside the planning region were simulated to accomplish required calculations (e.g., Rio
Grande Compact balance is calculated for the entire Middle Rio Grande Basin).
+
Rio Grande Compact SS ¥
7 Deficit Schedule Delivery ara
pe + Otowi Gage Inflows
“Storage
Reservoir tt
+ Evaporation +
t
a ie ral ai Grande Flows Tributary Inflows
Consum ey,
Climate oS Open Water” Waste Water +
ne + Return Flows
San Juan-Chama +
ce Diversion s
+ ee
Municipal =<)
Groundwater
~ Discharge Groundwater +
Z nnn _
River ae
Septic Return +
Flows:
J out Storage
Mountain Front Recharge _A Ca
Interbasin Flows
Figure 5: Causal loop diagram depicting the key elements influencing water supply and
demand in the Middle Rio Grande Planning Region.
+
The MRG Model is also a large model by SD measures. The following list describes
some of the model details:
e Composed of 13 modules (a module represents a separate component of the
system
e Approximately 899 variables
e 3 ranges (named arrays or matrix dimensions)
e 5,437 elements e.g. a variable with the dimension
Modeling Process
As aresult of the Middle Rio Grande cooperative water planning project growing in
complexity it was determined that a model could assist in the planning process. A
modeling project was initiated to:
1. provide a quantitative basis for comparing alternative water conservation
strategies in terms of water savings and cost,
2. help the public understand the complexity inherent to the regional water system,
and
3. engage the public in the decision process.
The team took a mediated modeling approach to the model building process. Although
the actually model was built at Sandia National Laboratories, many people participated in
bi-weekly modeling meetings. Table 1 lists the stakeholders and their roles in the
modeling project. The Sandia National Laboratories team was a part of the CMT.
Table 1: List of stakeholders and their roles |!
Stakeholder Role
Interstate Stream Commission (ISC) Manages treaty and interstate compact deliveries of
water. Oversight of the statewide water planning
process.
Mid Region Council of Govemments A board comprised of city and county officials.
Purpose is to coordinate regional (MRCOG)
planning.
Middle Rio Grande Conservancy District Responsible for managing and delivering irrigation
water to the farmers (MRGCD) of the Middle Rio
Grande region.
City Utilities and Water Cooperatives Responsible for managing and delivering water to
urban and rural water users for domestic,
commercial and industrial purposes. Also
responsible for capturing and treating resulting
wastewater.
Federal/State Agencies Management of waters and ecosystems of the state.
Provided data, models and system understanding in
the water planning process.
Middle Rio Grande Water Assembly Commissioned by the ISC with the responsibility of
preparing the 50-year water (MRGWA) plan in
cooperation with the MRCOG. Membership open to
the public.
Cooperative Modeling Team (CMT) Subset of MRGWA and MRCOG participants.
Purpose was to develop an interactive model to
assist in the water planning process.
General Public Participation through volunteering on the MRGWA
and/or participation at quarterly public forums.
Modeling
The model operates on an annual time step encompassing the period 1960-2050. This
period includes a 41-year calibration period (1960-2000) and the prescribed 50-year
planning horizon (2001-2050). An annual time unit was used because it matched the
annual basis of calculation for key metrics in regional water planning (i.e., Rio Grande
Compact obligations and groundwater depletions).
Sixty-six variables can be controlled in the model interface by slider bars or switches.
The MRG Model like VISION is an application; an application is a model plus a detailed
and complex interface or flight simulator. Users can easily simulate various combinations
of hydrological, economic or demographic conditions, and then run the model and view
output in seconds. This interactive modeling environment allows users in private or
public settings to experiment with competing management strategies and evaluate the
comparative strengths and weaknesses of each.
Benchmarking and Validation
The years 1960 to 2000 serve as the verification period for the MRG water-planning
model. The verification process compares historical data with modeled data for four
different variables, including groundwater depletions, Rio Grande Compact balance, Rio
Grande flows at the San Acacia gage (located just south of the planning region), and
storage in Elephant Butte Reservoir. The Rio Grande Compact legally delineates the
water delivery requirements of water in New Mexico to downstream users (e.g. Texas
and Mexico).
Model verification played an important role in the overall planning process. First, this
effort provided a sense that the model was being tested for credibility and that the model
was based on some level of reality. Second, verification of the model demonstrated that at
an aggregated, surface/groundwater level, the modeled terms in the water budget achieve
balance. Requiring the water budget to balance against historical data was important for
several reasons; in particular, balancing the budget helped set reasonable bounds on
parameters subject to uncertainty (e.g., mountain front recharge, agricultural consumption
and bosque consumption). Balancing also increased confidence that the model could
indeed produce output values consistent with other models, models that many of the
stakeholders already had confidence in. Historical balancing also caused careful
consideration of whether data gathered from disparate sources were all measured and/or
calculated in a self-consistent manner. Finally, there were critics who argued, during the
model development process, that a term in the water budget was incorrect. However,
within the context of a historically balanced model any change made to one portion of the
model required an equal and opposite change to another part of the model, and so
indiscriminate changes to the model were precluded. Most importantly, the verification or
balancing process made the team think more in the context of the whole system,
dynamically, rather than the individual terms, detail.
Figure 6 shows the results of sensitivity analysis applied to surface and groundwater
while varying rainfall and consumption via the Monte Carlo capabilities of the modeling
tool.
0
-500
-1,000 - as
s jes —| |—— 25th percentile
& -1,500 E anaes Mean
/ 75th percentile
2,000
siseae Low
2,500} ~ Z
3,000
= High
25th percentile
| Mean
———75th percentile
seeeee Low
-1,000
-1,5004 8B
-2,000
1960 1970 1980 1990 2000 2010 2020 2030 2040 2050
Year
Figure 6: (A) Groundwater depletion and (B) Rio Grande Compact balance
Model Applications
Since the development of the MRG model several activities have taken place. The first
was the realization that we now had model components, much larger than Molecules, that
pertained directly to river basin management and that the model building process itself
would lend itself to other resource allocation problems. The success of the MRG
(partially due to its flight simulator nature) allowed the team to promote a modeling
process that included stakeholder participation and rigorous hydrology and although
mentioned, did not specifically promote system dynamics as the solution. Several other
externally and internally funded projects have bee the result as shown below:
e River Basin modeling of the Willamette Basin to examine issues related to fish
habitat and water temperature.
e Modeling of the Estancia Basin, in New Mexico, a basin with inflow but no
surface water outflow.
e Modeling of the Gila and San Francisco River Basins in New Mexico to examine
changes to come after legal settlement of water rights issues between New
Mexico and Arizona.
e Water quality modeling for the Jordan and Zarga Rivers in Jordan.
e Modeling of the entire surface water system in Iraq.
e Development of a Toolkit that contains components useful in watershed
management including tools for surface and groundwater, population dynamics,
non-use valuation, and regional economics.
¢ Collaboration with several universities and state and federal agencies.
Summary -- Big versus Small
It is important to justify why a system dynamics model develops into a large model and
to separate it from the tendency for inexperienced modelers to develop large models. SD
models that become very large are because they include a lot of detail complexity. But,
as Peter Senge notes in his book, “The Fifth Discipline” !?, there are two types of
complexity, detail complexity and dynamic complexity.
With VISION and the MRG Model it was necessary to capture both the detail complexity
of the system as well as the dynamic complexity. The detail complexity was necessary
because as is often times the case the “devil is in the detail”. In the nuclear fuel cycle, it
is necessary to track all flows, not at a mass level but at an isotopic level. In other words,
it is necessary to track the flow of material at the level of isotopes; approximately 70
isotopes are currently tracked in VISION. Small changes in isotopic mixtures can cause
significant changes in behavior. In addition, because of radioactive decay the inventory
of isotopes is constantly changing as material sits in the various storage facilities.
With the MRG model, hydrologic rigor was required to alleviate concems that the tools,
which aggregated the hydrologic processes more that the standard hydrology modeling
tools, could ‘reproduce’ the output of the latter. Although in many cases the system
dynamics model performs as well as the standard tools, building confidence with the
stakeholders typically requires more data and calibration than necessary.
Dynamic complexity is when an action has one set of consequences locally and a very
different set of consequences in another part of the system at perhaps a much later time
period. The nuclear fuel cycle also involves dynamic complexity. As an example,
decisions on separations techniques could be very important for the type of separations
facility needed but the impacts to future disposition of the waste which may not take
place for years will be greatly impacted. The complex surface and groundwater
interactions in the Middle Rio Grande Basin have been altered by modern human water
use and management practices. The primary misunderstood interaction is the almost one-
sided transfer of groundwater to the surface water system and how residential and
commercial water conservation practices can harm the surface water system. As Senge
notes, “The real leverage in most management situations lies in understanding dynamic
complexity, not detail complexity”. In the case of VISION and the MRG Models, it was
necessary to capture both the dynamic complexity and the detailed complexity in order to
capture the realistic behavior of the system.
When developing a model there are several ways to accomplish it. One is to start with a
simple model that contains all the components of the finished product but with very little
detail or complexity.
Ways of dividing up a model’
e By Loops
-- Good for building a model
e By “plumbing” and decisions
-- Good for dividing model between views - Visual understanding
e By Sectors
-- Not so good, but common
Determining which way to divide up the model is very important. Remember, one of the
strengths of system dynamics modeling is that the model diagrams should be visually
stimulating and add to the understanding of the system. In fact some say that the actual
model layout should be as informative as a causal loop diagram. Choosing the wrong
method of dividing up the model may be detrimental to the model visualization.
The real danger of developing a large model, as Jim Hines noted in his Modeling for
Insights class!°, is inadequate time and resources. So, it is important to understand the
difficulties and time necessary when developing a large detail and dynamic complex
model and to plan accordingly. It is also important to continuously manage the
customer’ s expectations. Only promise the customer what you can deliver on the time
line agreed upon. In this sense, the modeling process resembles the software
development process. Unfortunate as this may be, moving the customer away from static
spreadsheet-like analysis is not a trivial task. Both the VISION and the MRG Model
teams did extensive planning to understand the question and its scale.
Conclusions
Both projects were well funded and had an appropriate time lines to assure success. Most
importantly, the models were able to show significant results early in the modeling
process. Without the early success, continued funding and support from management
would have been questionable.
The VISION model has been under development for over 2 years and is currently in its
second version release. The model is supporting several important efforts which include
a report to Congress on the Global Nuclear Energy Partnership (GNEP) and AFCI’s
annual report to Congress. Because of the nature of the use of the model it is important
that the model is reporting reasonable and prudent results. Take note that it was not
stated as precise and accurate since those terms in any predictive model are over
specifications of the models abilities.
The real value of the MRG Model has been advancing the premise that models can truly
be of use, especially in contentious situations with various stakeholder groups. In addition
the leaning acquired in stakeholder management and model building has led to further
work permitting much of the modeling team to stay together and continue work in this
area and others.
The common and advocated approach to system dynamics model building does not
encourage large model development. Both the VISION and MRG teams were well aware
that much system insight could be gained from much simpler models. Nevertheless both
teams succumbed or were forced to accept the detail complexity demanded by their
customers. Actually, given the capabilities of today’s system dynamics software, if
dynamic complexity is handled well, detail complexity becomes more of a time and data
management issue. Some lessons have been learned:
e When succumbing to detail complexity it is always important to ask the question:
Is this added detail going to add to the quality of the model or will it simply make
it more complicated?
e Validate the model early and then validate after every major update. Waiting till
the end will make validation a daunting task.
e As model size increases, modeling becomes more difficult to manage. Applying
the software engineering practices developed in the 1970s such as modularization,
are essential.
e Use a standard naming convention for all variables and make sure you adhere to it
all along the way.
e Add units to every variable that requires one. Unit consistency is the first and
foremost objective check for model validity.
One of the ways of judging the success of a modeling project is by how many insights
that a model generates. Insights can come from running a model or even when you are
building the model. If at the end of a modeling exercise there are no “new” insights then
the exercise probably has not been very fruitful. It simply verifies that you had a good
understanding of the system prior to the modeling exercise. If however, there are many
insights then the effort has at least generated some new understanding and knowledge.
VISION and the MRG Model have, from the start, generated an uncountable number of
new insights. As indicated above, this is directly a result of handling the dynamic
complexity of the problem.
The most important lesson to take from these efforts is that before you embark on
developing a large system dynamics model you be fully engaged with your customer and
establish the requirements early in the process. The model specifications will help
determine the time and funding requirements. At each step of the modeling process, keep
the customer engaged and, as appropriate, modify the requirements specifications to meet
any new requirements. The real success of the model is the acceptance of the model by
the customer. Does the model end up developing a better understanding of the system
and does the customer find it useful? That determines the success of the modeling effort.
U.S. DEPARTMENT OF ENERGY DISCLAIMER
This information was prepared as an account of work sponsored by an agency of the U.S.
Government. Neither the U.S. Government nor any agency thereof, nor any of their
employees, makes any warranty, express or implied, or assumes any legal liability or
responsibility for the accuracy, completeness, or usefulness of any information,
apparatus, product, or process disclosed, or represents that its use would not infringe
privately owned rights. References herein to any specific commercial product, process, or
service by trade name, trademark, manufacturer, or otherwise, does not necessarily
constitute or imply its endorsement, recommendation, or favoring by the U.S.
Government or any agency thereof. The views and opinions of authors expressed herein
do not necessarily state or reflect those of the U.S. Goverment or any agency thereof.
References
| Barlas, Y., “Leverage points to march ‘upward from the aimless plateau’”, Letter to the
Editor, System Dynamics Review, Volume 23, Number 4, Winter 2007, Wiley
Publications.
? Report to Congress on A dvanced Fuel Cycle Initiative: “Advanced Fuel Cycle
Initiative: Objectives, Approach, and Technology Summary,” U. S. Department of
Energy, Office of Nuclear Energy, Science, and Technology, May 2005.
3 Jacobson, J.J. etal., “VISION 2: Enhanced Simulation Model of the Next Generation
Nuclear Fuel Cycle,” ANS Summer Meeting Transactions, Boston, Massachusetts, 2007.
* Software, ISEE Systems, Inc. http://www.iseesystems.com, Web page accessed
February 2008.
> Powersim Software, http://www.powersim.com, Web page accessed February 2008.
® AFCI Economic Benefits and System Analysis Team, Software Requirements
Specification Verifiable Fuel Cycle Simulation (VISION) Model, INEEL/EX T-05-02643,
Rev. 1, November 2005.
7 Zagonel, Aldo and Thomas F. Corbet, “Levels of Confidence in System Dynamics
Modeling: A pragmatic Approach to the Assessment of Dynamic Models”, International
Conference of the System Dynamics Society, Nijmegen, The Netherlands, 2006.
51. Van Den Durpel, A. Y acout, D. Wade. Development of Integrated Systems
Dynamics Models for the Sustainability Assessment of Nuclear Energy. Proceedings of
GLOBAL 2005, Tsukuba, Japan, October 2005.
° T. Boscher, P. Hejzlar, M.S. Kazimi, N.E. Todreas, A. Romano. Altemative Fuel Cycle
Strategies For Nuclear Power Generation in the 21* Century. Revision 1. Center for
Advanced Nuclear Energy Systems. Massachusetts Institute of Technology, Cambridge,
Massachusetts, June 2005. MIT-NFC-TR-070-REV.1.
10C.G. Bathke, E.A. Schneider, S.F. DeMuth, and M.R. James. Report of LANL
Advanced Fuel Cycle Systems Analyses for FY 2003. Los Alamos National Laboratory
(2003). LA-UR-03-874
"Vincent C.Tidwell, Howard D. Passell, Stephen H. Conrad and Richard P. Thomas
(2004) System dynamics modeling for community-based water planning: A pplication to
the Middle Rio Grande. A quatic Science 66 (2004) 1-16
12 Senge, Peter (1990) The Fifth Discipline: The Art and Practice of The Leaming
Organization. New Y ork. Doubleday
8 Advanced Study Program in System Dynamics, Massachusetts Institute of Technology,
“System Dynamics for Insight”, Prof. Jim Hines, Spring 2000.
