From Teaching System Dynamics towards Total Immersion in
Advanced Model-Based Policy Analysis
Erik Pruyt?, Bert Enserink, Wil Thissen
Faculty of Technol Policy,
Delft University of Technology
Abstract: In this paper, we discuss (i) important lessons related to the latest innovations in our case-based blended System
Dynamics education, and (ii) the redesign of a masters programs in which System Dynamics plays an important role. instead
of teaching System Dynamics 101, supervising System Dynamics projects, teaching Advanced System Dynamics, and
supervising System Dynamics theses, we will from 2016 on rapidly ramp up the level from basic to intermediate System
Dynamics with intensive workshops, teach students the most advanced methods and techniques, integrate System Dynamics
with other methods and analytical approaches, and embed our Master thesis students as much as possible in real modelling
and simulation projects and research teams. The envisaged changes are expected to enable us to transition from introducing
engineers to model-based policy analysis to training the best model-based policy analysts.
“Most of the challenging issues faced by society today cannot be solved by alone. Engi must
be coupled with insight into societal needs and the mastery of project and process management tools” (EPA 2013)
1. Introduction
The aim of TU Delft’s master program in Engineering and Policy Analysis (EPA) is to train technical
students to become strategic advisors in complex societal, technical and political contexts. Model-
based policy analysis is an important aspect of this master program. The level at which these model-
based policy analytical skills are taught in this program needs to be raised tough. The issues dealt
with today and in the future are getting ever more complex, uncertain, urgent, and thus challenging.
Figure 1 shows a possible future of model-based policy/decision analysis in which different modelling
methods are integrated with each other, with data and data science, gaming, etc.
This is a future we believe we need to prepare our students for. In order to do so, we need to
redesign the EPA curriculum. The current paper focusses on some major changes in the Engineering
and Policy Analysis (EPA) master program in which System Dynamics (SD) (Forrester 1961; Sterman
2000) has a prominent (mandatory) place. We also discuss the latest experiments, innovations,
success stories, failures, lessons learned, and new directions in our System Dynamics teaching and
testing at Delft University of Technology’s faculty of Technology, Policy and Management (TPM) that
enable to make some of the envisaged changes. By sharing our teaching materials the results of our
teaching experiments, innovations, success stories, failures, lessons learned, and changes ahead, we
believe that we can help others improve their SD education program. That is the main aim of this
paper. Doing so, this paper adds to the growing body of literature on teaching and testing System
Dynamics in bachelors and masters programs (Pruyt 2013a, Bosch and Cavana 2014, Pavlov et al.
2014, Davidsen et al. 2014).
% Corresponding author: dr. Erik Pruyt, e.pruyt@tudelft.nl
1
(iil
Goal:
- Understanding
Experimentation)
FO exploration
Robust policie:
(I)
uncertainty
Inference of ens. of models’
(ML & Data ics 2)
(ul)
Figure 1: A possible future of model-based policy/decision analysis. Source: (Pruyt, 2015)
In this paper, we will first provide an overview of the current EPA program and SD’s place in the EPA
program. Then we will discuss some recent experiments, after which we will present the future EPA
program and place of SD in it. Section three contains concluding remarks.
2. EPA 1.0
2.1 Why?
The core program of EPA combines systems modeling techniques, policy analytical problem
structuring methods and political science courses. Systems modeling techniques are considered to be
a valuable tools for these future policy analysts and consultants as they contribute to informed
decision-making. We know, at the same time, that excellent modeling skills alone will not suffice as
these future engineers and consultants will inevitably work in an often highly politicized, mostly
international and interdisciplinary environment, which requires excellent intercultural and
communication skills (Kroesen et al 2012). The program therefore aims at training policy analysts,
who can communicate and cooperate and can apply a variety of analytical and modeling techniques
to structure and analyze the behaviors of the multi-actor systems they analyze and study. The focus
of their training is on wicked, ill-structured or complex problems in the sense that natural,
technological, social, and human elements interact (Rittel and Webber, 1973). As a result, a variety of
problem perceptions exists, values and interests may be conflicting, and power and resources to
change things are distributed over multiple actors, who may be playing strategic games (Dunn, 1994;
de Bruijn and Porter, 2004; Koppenjan and Klijn, 2004). Such multi-actor complexity is the everyday
2
reality of analysts and problem solvers and asks for participative approaches to problem structuring
by involving stakeholders in the actual conceptual modeling steps (Lei et al 2011; Enserink et al 2010;
Enserink et al 2013). That is why ERPA was founded in the first place.
2.2 What?
The EPA program combines its intensive modeling courses with policy analysis courses teaching
students general problem structuring techniques and stakeholder analysis techniques and uses these
elements as building blocks to introduce them to the conceptual modeling of systems. We use the
so-called system diagram (Sage, 1992; Sage and Olson, 2001) as a core concept to present a
structured view of a problem situation. As extensively described by Lei et al (2011) the complete
diagram is constructed through seven iterative steps, where each step is a sub-analysis. This
structuring of an unstructured problem and defining an appropriate scope for further analysis and
action are essential skills that all policy analysts should master. Hence, when developing the current
Engineering and Policy Analysis curriculum we included a number of participatory modeling
approaches (Shaw et al, 2006), which aim to support a diverse collection of actors in addressing
problematic situations of shared concern. Moreover the conceptual modeling and systems modeling
courses are supporting one another due to the overall (sequential and parallel) set-up of the program
(see Figure 2).
[ First semester | Second semester |
Fist period Second pared Thi pared Fourth period
EPA1222 EPA1233
Economics and Regulation Economy of Infrastructures
SEC SEC
LD oicyanaysis [] sotlo-sconomic conte [] management ystems modeling xl | oo
Figure 2: The current EPA MSc program
2.3 How?
The concept of active learning has had a central role in the training of TU Delft engineers (Graaff and
Andernach, 2006) but recently changed to online and blended learning approaches. EPA was one of
the programs participating in TUDelfts experiment with online and blended learning approaches. In
the past application followed the theoretical basis in projects, games, simulations etc. Over the
course of time, there was a transition from a more cognitive approach to learning towards a more
constructivist approach, and from teacher centered to student centered learning (Rullmann, 2006).
Many courses in the EPA curriculum are turning into blended courses. Some of the courses recently
also flipped the classroom (intense work sessions at the university instead of lectures which are
largely replaced by short movies and reading at home). EPA courses also start to be available as
online courses. As a result, these courses have significantly invested in e-learning, the products of
which are made available to regular EPA students too.
2.4 SD in EPA 1.0
EPA students are trained in basic policy analysis up to intermediate policy analysis, including model-
based policy analysis. System Dynamics modelling is one of the core modelling methods students are
trained in. During the 2-year EPA master, students start with a large mandatory SD course (EPA1322),
which consists of an introduction to SD modelling of 1 quarter (8-9 weeks) and a SD project of 1
quarter (5-7 weeks). This course is also available as an online course for anyone interested in learning
SD (see https://online-learning.tudelft.nl/courses/systems-modelling/ ).
Those interested in SD can take an advanced SD course, as well as modelling masterclasses, and write
a SD master thesis. Note, however, that they are trained as policy analysts, not System Dynamicists
pur sang.
The first quarter of the 2-quarter SD course is taught following a blended case-based approach based
on a free e-book (Pruyt, 2013b), with, in addition, special topics not covered in the e-book. The
objective of the first quarter is to teach students the necessary technical skills to make and use SD
models. During the second quarter, pairs of students do a supervised SD project on a topic of their
choice. The goal of the second quarter is to learn how to build SD models from scratch and use them
for policy analytical purposes. Most of the SD projects are good, that is, for a 5 week SD101 project
of, in total, about 70 hours per student. In fact, many of these SD projects have been presented at
the International SD Conferences. Examples include Kovari & Pruyt (2014) and Schwarz, Fakkert and
Pruyt (2015).
Since many EPA students do not take the advanced SD course, there are nevertheless quite some
EPA master students graduate with only basic to intermediate SD skills. The current curriculum
redesign offers the opportunity to teach basic, intermediate and advanced SD (and other modelling
and simulation) skills to all EPA students.
2.5 Recent Experiments and Lessons Learned
Over the past two years, several experiments regarding the SD teaching were conducted. Some
succeeded, others failed but offered very valuable lessons to learn.
A first experiment that worked out well was to reduce the number of weeks for acquiring the basic
modelling skills (i.e., the first part of the course based on the e-book) from (previously) 9-10 weeks
down to 5 weeks.
An experiment that related to the reduction of the amount of face-to-face teaching/coaching by their
professor failed to some extent: students were asked to develop their own modelling skills using the
aforementioned e-book. Although their modelling skills at the exam turned out to be of the same
level, about 66% of the students wanted more involvement and guidance by a professor. Without
this personal guidance, 2/3 of the students found learning the System Dynamics method (too) hard,
frustrating and time consuming. The remainder of the students enjoyed learning the System
Dynamics method with the e-book without further teaching/coaching by a professor. But 2/3 of the
students would enjoy SD much more with more personal guidance.
The inverse experiment, providing intense workshop-like teaching using the e-book was more
successful: this intense teaching approach allowed to further reduce the time required to learn the
basic modelling skills to three days without much practice time and 5 days with practice time
included.
Other experiments related to the testing cases used at the exam. Apart from having to answer 10
multiple choice questions related to the SD method, students also need to the make a SD model
following a completely new case description (like the ones in the appendices) and need to use the
model for a particular policy analytical purpose. Previous experiments to use multiple choice
questions to capture some of their model specification skills as in appendix A and C-E worked out
well: almost all students build their models without first looking at the answers of the multiple choice
questions and it tremendously speeds up the correction of these model-based exams (5 minutes
instead of 30 minutes). The experiment to also capture the answers to model use questions was not
successful: it did not improve student grades. To the contrary, it caused lock-in problems for students
who had made severe mistakes in the specification phase. In the end, it resulted in a lot more waste
of time than could possibly be gained. Using bigger testing cases and providing about 50% of the
model structures and equations as in the Ebola II&Ill cases in appendix C was not successful either:
students found the size of the overall model too hard to deal with given the limited amount of time
of 3 hours in total.
Another experiment that worked out well was to reduce the number of weeks of the SD project from
7-8 weeks down to 5 weeks.
We also tested whether it was possible to facilitate large groups of students (45-200 students) in
open projects (i.e., projects of their own choice) instead of pre-specified case-based SD projects.
From these experiments we concluded that models of open projects may not always be as good as
models of pre-specified projects, but also that students practice and develop more skills in open
projects (which is what matters in our SD project setup). Interestingly, the open projects also
required less instead of more hours of supervision per group than traditional pre-specified cases.
Finally we tested whether it was useful to require students to use a template with a very strict format
to report their SD projects. Although students found it difficult to abide to the strict page limit, it
helped students to provide exactly those pieces of information we believe modelling reports should
comprise.
3. EPA 2.0
3.1 Why?
In the summer of 2014, two external triggers started off discussions about repositioning the EPA
program: (i) a ruling by the Ministry of Education about termination of the so-called ‘doorstroom
masters’ (the automatic continuation of a bachelor study into a specific master program) and (ii) an
opportunity to relocate the program from Delft to The Hague.
Although successful in itself and one of TUDelft’s forerunners with respect to blended and online
courses, changes in the ruling by the Minister now enable us to adapt our entrance requirements and
to broaden our market. Where we had restricted our inflow to students with a mono-disciplinary
engineering background or natural sciences background, we are now allowed to open up to those
with a more multidisciplinary training, such as our own TB bachelors and bachelors from industrial
design or biology and even to liberal arts students with a sufficient background in math and physics.
Opening the gates to a wider variety of students also enables us to choose a more prominent
modelling, simulation and gaming profile. The new EPA program will therefore be positioned as an
internationally oriented master of science program in policy analysis with a focus on analytics,
modelling and simulation (EPA, 2015). We will, at the same time, pay attention to the political
context within which the products of the analytical work need to be used. That is, in real-world
decision-making. This tension between analysis and modelling and the dynamics of the strategic
decision-making context is an important intellectual challenge for the students. This challenge is
reflected in the design of the program.
The second external trigger for change is the envisioned relocation of the program from Delft, the
city of engineers, to The Hague, the international city of justice, seat of the Dutch national
government, and city hosting many international (UN) agencies, consultants, research institutes and
multi-national organizations. The presence of the latter institutions at the new location is an
Opportunity to make strategic alliances with these institutes and link up with policy analysts and
policy makers, and structurally bring their real-world experience into our teaching and (student)
research.
Content wise the program will be oriented towards dealing with grand challenges: global health,
water security, environmental justice, climate change adaptation, energy security food security,
water management, cyber security and the management of natural resources. Alumni from the new
program will be real change agents: engaged technical policy analysts who understand the value and
the limitations of analytical and modelling techniques and who want to make the world a better
place for current and future generations (EPA Curriculum Committee, 2015).
3.2 What?
Understanding Intercultural :
: : Political
International Grand Relations and Project pers o ee
Mameniere ecision-making icy Analy:
Ethics and Impacts
Policy Analysi of Global
zl of MAS Interventions
id
2 Computer Advanced
Technology Engineering for Diterete
Simulations \ . ji
\
Data Analytics & | \} Actor and Strategy Advanced System | /
Visualization Models Dynamics
Figure 3: 2-lines philosophy of the new EPA curriculum
The new EPA program will be a blended, case based program; blended as most of the materials will
be online and case based as the application and training will be done by using real cases as teaching
materials in the intensive groups work and individual assignments that are part of the flipped
classroom which is essential to the blended learning experience. Meanwhile there is a conscious
design choice underlying the new curriculum; the story of the new EPA curriculum is about the
relation between two learning lines; the modelling and simulation line and a policy and politics line
(see Figure 3). These two lines are intertwined and courses are phased in such as to confront these
different intellectual perspectives. To create learning windows we consciously created a number of
courses that can be seen as boundary objects; modelling techniques are challenged in the policy and
politics courses and stakeholder oriented courses are part of the modelling and simulation line.
Moreover, concepts and models of one course are used in the next or even in parallel courses. The
model-based decision-making course in the 4" quarter deliberately focusses on the limitations of
modelling approaches and the preconditions for effective use of model in policy processes.
As indicated the new EPA program is positioned to attract student with a wider variety of
backgrounds than the current program. Therefore in the new curriculum we have consciously chosen
to integrate 10 ECTS alignment courses. In the first semester non-TB bachelors will have to follow
dedicated crash courses on multi-actor systems and “TPM modelling and simulation” to allow for
advanced modelling courses to be taken in later semesters (see Figure 4).
Policy
Analysis of
Computer
engineering for
Simulations
MAS
Introduction
to TPM
Modelling
Political
Decision-making
Ethics and
Impacts of Global
Interventions
EPA curriculum 2016/17
7 Data Analytics &
2 Visualization
E
a
cy Intercultural
A, ' Relations and Project
ones Management
as Advanced System Advanced Discrete
q Dynamics Simulation
a
z
s Hbased Macro-Economics
making for Policy Analysis
g
a
3
£
&
2
a
2
3g
£
&
quawuaijy
NB1. First semester is
different for TB bachelors and
others. To align the students
TB bachelors do a technology
course in thesis research area
and a computer engineering
course. Non-TB students learn
about multi-actor systems
and TB modeling
for
or
thesis
project this allows for 50/60
ects specialization in total.
NB3. 10 ects Societal
Challenge project for
customer organization
offered to students studying
in Delft/The Hague . Can also
be individual specialization
courses.
Figure 4: Design of the new EPA curriculum
3.3 How?
There are rather big challenges, apart from the challenge to move a master program to another
town. First, the EPA curriculum will be pushed towards grand challenges. This means new cases need
to be developed. Second, the teaching needs to be further intensified to enable students to quickly
ramp up to intermediate level, no matter what their background is, from which level the core
program will actually start, to reach the advanced level during the first year. Third, this redesign
means that all teaching materials need to move towards advanced model-based policy analysis. This
will also require furthering the methodological integration between methods and techniques. Finally,
we will develop a system in which all MSc theses students are embedded in professional research
teams and at relevant organizations.
3.4 SD in EPA 2.0
SD will remain a crucial method in the new curriculum, possibly more crucial than before. It will be
one of the methods students will be familiarized with at the very beginning of their EPA studies.
Moreover, students will already reach the advanced level during the second semester. Even more
importantly, SD will be used in many other courses. Finally, many methods and techniques will be
linked to SD models or applied on SD simulation results. This will require excellent SD cases, like the
SD housing model in the appendix, which can be linked right away to the real-world data about the
Dutch housing sector. Finally, SD is an excellent method for investigating and dealing with grand
challenges. Hence, SD will most likely become even more central to the new EPA curriculum than it is
in the current one. SD modeling could even become the core of an integrated model-based policy
analysis cycle as visualized in Figure 5.
1. Ensemble
Generation
a PR WB / Cookbook / Libraries (e.g., matplotlib)
=
3. Exploration, |
Analysis, /
Searches/
/
Se Uncertainties,
\Policies, etc.
S
~ air
2. Ensemble / EES
data handling
Figure 5: Integrated (exploratory) model-based policy analysis cycle
4. Concluding Remarks
The EPA curriculum is currently being restructured. System Dynamics will remain a crucial method in
the curriculum. More, it may become even more important if the integration challenge with other
methods and techniques is handled well. This requires furthering the field and embracing other
methods and techniques.
Recent experiments have shown paths to be taken but also paths not to taken for teaching SD
modelling. It seems we need more intensive approaches and that these intensive approaches could
enable novices to quickly develop the necessary basic to intermediate skills required to slowly learn
the advanced skills needed in an even more complex and uncertain world. A world with many grand
challenges!
The Scientist Personality of System Dynamics
Camilo Olaya
Department of Industrial Engineering - TESO Research Group
Universidad de los Andes, Bogota
colaya@ uniandes.edu.co
Abstract
System dynamicists frequently seem to defend the “scientific” status of their
models by recurring to scientific principles. This is not surprising since the
philosophy of science has been the usual place to look for the philosophical
standing of System Dynamics. However, SD typically aims at designing artifacts of
different types, e.g. models, policies, plans, organizational schemes, etc. that
address a specific situation that is wanted to be improved. Such an attitude is the
trademark of engineering, a stance that is easy to see in the underpinnings of the
field that Jay Forrester shaped. This paper delineates some issues that show why
the philosophy of engineering provides a more suitable ground for SD. Once sucha
ground is acknowledged, the questions of the “scientific status” of SD, with all the
demands that come from such a concem, e.g. validation, confirmation of
knowledge, truth of statements, scientific method, predictability, generalizability,
replicability, empirical basis, etc. become truly irrelevant.
Keywords: science, engineering, validation, knowledge, epistemology
+ Paper presented at the 32nd International Conference of the System Dynamics Society, 2014.
1
References
Andernach en E. de Graaff. Active Learning at TU Delft. Spotlight on Three Best Practices, OC Focus
Centre of Expertise in Education, Delft University of Technology, pp. 30-33.
Bosch, O. and Cavana, RY, 2014. Systems -- Special Issue: Systems Education for a Sustainable Planet.
2(3)
Deegan, M, Stave, K, MacDonald, R, Andersen, D, Ku, M, and Rich, E, 2014. Simulation-Based
Learning Environments to Teach Complexity: The Missing Link in Teaching Sustainable Public
Management. Systems 2(3): 217-236
Davidsen et al. 2014. Systems Education at Bergen. Systems 2(3): 159-167
de Bruijn H and Porter AL (2004). The education of a technology policy analyst to process
management. Technol Anal Strateg Mngt 16(2): 261-274.
Dunn WN (1994). Public Policy Analysis: An Introduction. Prentice-Hall: Englewood Cliffs, NJ.
Enserink, B., Koppenjan, J.F.M. and Mayer, |.S. (2013) A Policy Sciences View on Policy Analysis. In: W.
A.H. Thissen and W. E. Walker (eds.), Public Policy Analysis International Series in Operations
Research & Management Science, 2013, Volume 179, Part 1, 11-40, DOI: 10.1007/978-1-4614-4602-
6_2 Springer Science+Business Media New York
Enserink, B., Hermans, L., Kwakkel, J, Thissen, W., Koppenjan, J., and Bots, P. (2010). Policy Analysis of
Multi-Actor Systems. Lemma. isbn 978.90.5931.538.9
EPA Curriculum Committee, (2015). Engineering and Policy Analysis, curriculum 2016, 13 March 2015
Graaf, E. de and T. Andernach (2006) “Methods of Active Learning”, In: Veen, E. van de, T.
Koppenjan J and Klijn EH (2004). Managing Uncertainties in Networks. Routledge: London, New York
Kovari A and Pruyt E (2014). A Model-Based Exploration and Policy Analysis Related to Prostitution
and Human Trafficking. International Journal of System Dynamics Applications 3(4)
Kroesen, O., Klaassen, R.G., Enserink, B. (2012) The flowers of global citizenship, Widening
Participation and Lifelong Learning, Volume 13, Number 3, Winter 2011/12: 74-88
Lei, T. van der, Enserink, B, Thissen, W.A.H. Bekebrede G, 2011. How to use a systems diagram to
analyse and structure complex problems for policy issue papers. JORS 2011: 1391-1402
http://www. palgrave-journals.com/jors/journal/v62/n7/abs/jors201028a.html
Pruyt, E., (2013a). Small SD Models for BIG ISSUES - The Book. In: Proceedings of the 2013 ISDC, St.-
Gallen, CH.
Pruyt, E., (2013b). Small System Dynamics Models for Big Issues: Triple Jump towards Real-World
Complexity. Delft: TU Delft Library. ISBN/EAN: 978-94-6186-195-5
10
Pruyt E (2015). From Building a Model to Adaptive Robust Decision Making Using Systems Modeling.
In: M. Janssen et al. (eds.), Policy Practice and Digital Science. Springer International Publishing.
Rittel WJ and Webber MM (1973). Dilemmas in general theory of planning. Policy Sci 4: 155-169.
Rullmann, P. (2006) “Learning With All Our Senses”, in Veen, E. van de, T. Andernach en E. de Graaff.
Active Learning at TU Delft. Spotlight on Three Best Practices, OC Focus Centre of Expertise in
Education, Delft University of Technology, p. 5.
Sage AP (1992). Systems Engineering. IEEE Wiley: New York.
Sage AP and Olson SR (2001). Modeling and simulation in systems engineering. Simulation 76(2): 90—
91.
Schwarz P, Fakkert S, and Pruyt E, (2015). Kidney transplantations in the United States: A System
Dynamics approach to reduce the waiting list and illegal trafficking. In: Proceedings of the 2015
International System Dynamics Conference, Cambridge, MA, USA.
Shaw D, Franco A and Westcombe M (2006). Problem structuring methods: New directions in a
problematic world. J Oper Res Soc 57: 757-758.
Sterman JD (2000). Business Dynamics. Systems Thinking and Modeling for a Complex World.
McGraw-Hill Higher Education: Boston.
11
1 The Scientist Personality
Science is frequently taken as the paradigm of academic activities. This tendency is puzzling when
itis applied to the arts, humanities and engineering. The scientific goals, its principles of reasoning,
methods, validation criteria, among other elements, sometimes become examples to follow. I see
such type of attitude in several academic engineering initiatives, including diverse modeling and
simulation approaches. System Dynamics and its critics are no exception. Consider for instance
questions of validation, identity (“what is System Dynamics?”), legitimation, the discussions on the
“theory” of System Dynamics, among many other topics, e.g. “the modeling process must follow
the scientific method”, “hypotheses must be testable”, “modelers must build theory”, “models must
be built on theory”, “models must be scientifically evaluated", “we must have scientific credibility",
etc. The problem that I see is that along the way such demands contradict, without noticing, the
engineering heritage of System Dynamics.
But it is not a matter of history or genealogy. Engineering is, in fact, very different from science.
With “different” I mean that the principles of reasoning of engineering are different from the ones
used for generating scientific knowledge. To recognize the engineering “personality” of System
Dynamics helps to dissolve various SD debates and to guide System Dynamics to “get its job
done”. I say this without implying that System Dynamics cannot contribute to science or to the
generation of scientific knowledge, e.g. Sterman (1994) highlights the enhancement of scientific
reasoning via SD virtual worlds. Moreover, SD can be also used as the basis for scientific activities,
e.g. it is very powerful for model-based research in social sciences, but this is only one of many
possibilities. I do not imply either that SD models should not be rigorous and done with the highest
standards, of course not. But what kind of rigor? What kind of standards? I do not mean either that
scientific rationality can not be beneficial for System Dynamics practice; scientific reasoning can be
useful for building certain models in many situations. But usefulness is one thing. To elevate the
principles of scientific reasoning as values or ideals to follow in SD modeling, for the sake of it, is
quite another.
For example, with the goal of having confident models for decision making, Eckerd, Landsbergen
and Desai, (2011) clarify that the results of more scientifically “rigorous” models will be seen by
users more confidently. However, apart from being a pragmatic requisite, they see the generation of
scientific knowledge as the ideal to pursue: “modelers should consider the scientific validity of the
model results. Ideally, models should contribute to knowledge and theoretical understanding of the
system in question. If a model is applicable to only a few specific scenarios, then the results are not
generalizable and do not provide us with a deeper understanding of the system in question.” (p. 8).
For these authors, the goals of System Dynamics are and should be the same goals of any
respectable scientific enterprise: to generate theoretical and generalizable knowledge that permits to
understand phenomena.
The previous example shows how sometimes the scientist personality is explicitly shown. I suspect
that this attitude is very common in the unconscious mind of the SD community. If we believe that
SD modeling has a scientific nature, then we answer some important questions in a particular way.
But if we believe that SD is primarily an engineering activity, intrinsically different from scientific
activity (even regarding “academic SD”), then our answers change.
2. The Engineering Personality
The trademark activity of engineering is design (Pitt, 2011; Van de Poel & Goldberg, 2010). A
design shows a “know-how”, as opposed to the scientifically valued “know-that”. Engineering
expresses a distinctive type of knowledge (Mitcham, 1994). However, the prevalent bias towards
knowledge-that undermines engineering’s knowledge-how, i.e. “engineering knowledge is practice-
generated... it is in the form of ‘knowledge-how’ to accomplish something, rather than ‘knowledge-
that’ the universe operates in a particular way” (Schmidt, 2012, p. 1162). Engineers know what to
do in non-ideal situations, engineering knowledge is defined by such a know-how (McCarthy,
2010). Knowledge-how is not concemed with the truth or falsehood of statements that concerns
knowledge-that, “you cannot affirm or deny Mrs. Beetons recipes” (p. 12). Engineers know how to
do things.
Justification philosophy—the search for epistemic authorities—has been the dominant style of
Westem scientific philosophy. This is the view of knowledge as justified true belief that looks for
"well-grounded" (positive) knowledge, that is, “knowledge-that”. This popular position supports
most of current Western thinking about what science should be: it is rational to accept only those
positions that have been justified according to the rational authority. Lately such authority is
“empirical evidence”. Another popular authority is the collective endorsement of a knowledge
claim by a scientific community. But the epistemology of engineering does not need epistemic
justifications. The intentional creation of artifacts is done by experimental methods that are more
fundamental than (and not derived from) any type of theory (Doridot, 2008). The origin of designs
is irrelevant, they do not necessarily have to be a priori supported by anything, including theories or
data. They can be freely generated with the help of any procedure, sourced from reason, or guided
by previous expectations (“theoretic” or not) (Stein & Lipton, 1989), guided with the help of a
model, or guided just based on imagination or instincts. “Empirical evidence”, or any other indirect
mechanism of representing the world, is just another option, but it is not a requisite. For instance,
“the inventor or engineer... can proceed to design machines in ignorance of the laws of motion...
These machines will either be successful or not” (Petroski, 2010, p. 54). An artifact is not false or
true (or closer to), simply it works, or it doesn’t. If it works, engineering succeeds. The popular
notion of knowledge as “justified true belief’ means nothing in a pragmatic approach in which
knowledge is unjustified. In the words of Pitt: “If it solves our problem, then does it matter if we fail
to have a philosophical justification for using it? To adopt this attitude is to reject the primary
approach to philosophical analysis of science of the major part of the twentieth century, logical
positivism, and to embrace pragmatism” (2011, p. 173). In engineering “what works is what
counts”, justification is optional and dispensable. Consequently, its method it is not the “scientific
method” (on any of its variants or interpretations). Engineering uses heuristics, that is, fallible and
unjustified means to address any problem (Koen, 2003).
The previous characteristics bring special criteria that actually oppose to scientific “principles”.
Several SD discussions take a different light if we see them through such engineering glasses. I will
list some of them.
The Relevance and Primacy of Design
The significance of design for engineering and for the Industrial Dynamics that Forrester envisaged
is straightforward. For instance, Sterman (2000) underscores the main purposes of SD modeling for
management: organizational design. In fact, design is ubiquitous in System Dynamics literature and
practice.
Validation and purpose
Forrester (1961) stated that the “validity” of a model is a question of usefulness for a purpose. Let
us consider the paper of Epstein (2008) entitled “Why model?” in which he expresses that “I can
quickly think of 16 reasons other than prediction...to build a model”, for instance:
1. Explain (very distinct from predict)
2. Guide data collection
3. Illuminate core dynamics.
4. Suggest dynamical analogies
5. Discover new questions
6. Promote a scientific habit of mind
7. Bound (bracket) outcomes to plausible ranges
8. Illuminate core uncertainties.
9. Offer crisis options in near-real time
10. Demonstrate tradeoffs / suggest efficiencies
11. Challenge the robustness of prevailing theory through perturbations
12. Expose prevailing wisdom as incompatible with available data
13. Train practitioners
14. Discipline the policy dialogue
15. Educate the general public
16. Reveal the apparently simple (complex) to be complex (simple)
Many purposes, many uses. If the goal is to develop scientific theory then perhaps some of the
principles of scientific reasoning might apply. But this is not “the” sole possibility. A design
process can be supported by a model that can be used in many ways. A model is not like a scientific
theory. Its purpose is not necessarily to generate a scientific theory either. A model is fallible,
uncertain, and many different (useful) models can be built for the same situation (there is no single
“best” theory, as science pursues). Given the pragmatic philosophy of engineering then if the model
works for the purpose in hand, then the engineer succeeds. It does not really matter if there is no
justification for a model that works.
Methodology: Heuristic Trial-and-Error
The questions on the “scientific method” address the possibility of generating scientific theoretical
knowledge, that is, abstract, generalizable, confirmed, justified knowledge. The issue of scientific
validity rests mainly on the method. However, any system dynamicist knows that there is no one
“best” or “standardized” method for building SD models. In fact, since there are many possible
purposes, then the methods should vary according to them. Moreover, given the heuristic nature of
design activities, then any procedure is welcome. Modeling is a creative activity. It is largely a
matter of trial-and-error. This heuristic quality is easily recognized in some well-known
recommendations for building SD models, e.g. “there is no cookbook recipe for successful
modeling... Modeling is inherently creative... Modeling is iterative... Models go through constant
iteration, continual questioning, testing, and refinement (Sterman, 2000, p. 87). Homer (1996)
recognizes that model development is “a process that is iterative, involving a certain amount of trial
and error, and often requiring significant time and effort to come to fruition” (p. 1). System
Dynamics, as any engineering enterprise, is experimental.
Methodology: Problem-Oriented
For engineers the first issue to consider is a problem to solve. Indeed, the very first step of the
“Industrial Dynamics” approach of Forrester (1961) is “to identify a problem” (p. 13). Sterman also
underscores the task-oriented primacy in SD: “The most important step in modeling is problem
articulation. What is the issue the clients are most concemed with? What is the real problem, not
just the symptom of difficulty? What is the purpose of the model?” (Sterman, 2000, p. 89). Indeed
Sterman makes the important warning: we should model problems, not whole systems. Engineers
are very aware of such a risk.
Critics: “SD has no Theory”
Over the years various critics have expressed the accusation that System Dynamics has no social
theory behind. However, it should be unmistakable that SD, as engineering activity, does not
pretend to build theories of human behavior or alike. Moreover, from an heuristic stance, SD does
not need theories to build models either. Theories may be useful, but that is another matter. SD is
not interested in individual action, furthermore, it does not assume that structures determine human
behavior either—the sort of determinism that Burrell and Morgan (1979) oppose to free-will. This
type of criticism has already been answered and clarified by Lane: SD is concerned with aggregate
social phenomena and not with individual meaningful actions (2000). Moreover, SD does not
propose invariant causal laws, as Lane (2001) already showed. SD’s engineering personality should
help to dissolve these issues.
Critics: “SD is Unscientific”
Some critics utter that SD has abandoned (or diverges from) the scientific method. A variation in
this sort of “attack” is that SD lacks scientific rigor. Connected with these criticism is the charge of
lacking empirical evidence (which is scientifically used for justifying knowledge). In the light of the
points above, it should be straightforward that an engineering enterprise is not concemed with
empirical evidence as such (again, it may be useful). The alleged scientific rigor of the “empirical
evidence”, for guaranteeing “legitimate” models or recommendations, is not even relevant within
engineering activities. And that is a good thing (I do not mean that data is not useful, for example
for testing behavior reproduction, but that is quite a different use for data). Akkermans and
Romme (2003) make the closest point to the fact the SD is about design, though for them it is a
“design science” enterprise. Their invitation is worthwhile, although the differences between a
“design science” and engineering can be a matter of debate. | find the “unscientific” criticism highly
misplaced. Instead, SD could be charged, if ever, of not following the engineering method. But then
the verdict would be: not guilty!
Identity Crisis: What is System Dynamics?
It is not uncommon the concern about “the identity of System Dynamics”, e.g. (Vanderminden,
2006). In fact SD has been labeled as a theory (Flood & Jackson, 1991; Jackson, 2003), a method
(Coyle, 1979; Lane, 2001; Meadows, 1980; Sterman, 2000; Wolstenholme, 1990), a methodology
(Roberts, 1978), a field of study (Coyle, 2000; Richardson, 1991), a tool (Luna-Reyes & Andersen,
2003), a paradigm, among other nouns. I want to highlight that the “engineering roots” of SD
address the question of identity in a way that unmistakably discards various scientific traits that I
see as a source of identity confusion. In fact Homer recently expressed his concern regarding the
“lack of progress and success” of the field of System Dynamics (p. 124), which for him it is “a
problem with how we think of ourselves and how we project ourselves to others. Perhaps the right
metaphor here is a psychological one. In particular, it seems to me that SD has for many years
suffered with an identity crisis”. I can’t agree more. But I disagree in the nature of such identity.
The pursue of a scientific credibility for SD undermines the engineering character of SD. The full
potential of SD will no be found in meeting the “scientific” demands made by some academic
communities but in its actual power for supporting the design and redesign of complex systems.
What is a Good Model?
That is always a central SD question. And I guess the answer is straightforward (and not simplistic
at all): A good model is a model that works, for its given purpose. To establish if a model “works”
5
is not an uncomplicated matter. But the ultimate razor for judging the “goodness” of (engineering)
models should be, unmistakably, effectiveness.
SD for Engineering Education
Goldberg (2008) states succinctly the problem in his article Bury the Cold War Curriculum:
Pushing science and math at the expense of design may have worked once but is now doing
students a disservice... The global economy places a premium on more creative engineering
activities at home. Furthermore, the death march of math and science disillusions some
otherwise able students, causing them to drop out. Disproportionate numbers of the
departing are minorities and women, whom engineering schools should instead be
attracting. Moreover, students who come to engineering to be entrepreneurial, socially
responsible or both wonder why business and ethics are merely bolt-on topics. When design
is finally taught, students are unable to solve other than rote problems and struggle to
communicate their results.
Engineering faculties are fertile soil for SD because of its modeling and simulation-based truly
creative, integrative and design possibilities. But paradoxically SD is rather scarce in engineering
curriculums. This means a significant diffusion potential for SD. In the past there have been
convincing claims in this direction, e.g. (Caulfield & Maj, 2002; Saeed, 1997). Perhaps the
strongest one is reflected in the title of an article of Radzicki and Karanian (2002): Why Every
Engineer Student Should Study System Dynamics.
3. Perspective
Milestones of engineering over extremely challenging problems, like the flight of the Wright
Brothers, the Chilean mine rescue, or the landing of the NASA rover Curiosity on Mars, are often
attributed, erroneously, to science (Petroski, 2009, 2011, 2012). I believe that this is a matter of
historic and cultural prejudice that associates “knowledge” (to solve problems) with “science”, and
“engineering” with mere “application”. Goldman, in his text “The Social Captivity of Engineering”
makes the case that:
Engineering is today captive of society... to a cultural prejudice that denies the very
existence of a theory of engineering—that is, of a distinctive conceptual framework, a
theoria, or perspective on the world, of engineering’s own—by reducing engineering to
devising applications of the products of scientific theorizing... [which] provides a rationale
for concluding, quite incorrectly, that all of the ‘serious’ intellectual problems... attach to
science, the principles of whose practice are supposed to comprehend the practice of
engineering as well” (Goldman, 1991, p. 121)
Since Plato, Westem culture value universals over particulars, theory over practice, thinking over
making and doing, and representations as copies over representations as models (Floridi, 2011). Our
culture favors the elegance of values such as certainty, truth, universality, abstraction. The
engineering way of doing things works under undervalued principles that favor uncertain, context-
dependent, contingent, practical solutions (Goldman, 2004). Indeed thinking or acting by “trial-and-
error” has been traditionally used in a pejorative sense. However, the engineering way of facing the
world represents a truly effective enterprise. Engineering, SD included, is not “applied science”.
Perhaps the situation is the other way around, as Goldberg (2010) expresses: ’science is merely the
application of engineering method to the evolution of models or concepts” (p. 8), as in the
Popperian trial-and-error sense. The same iterative framework that system dynamicists have been
developing for the last 60 years.
References
Akkermans, H., & Romme, G. (2003). System Dynamics at the Design-Science Interface: Past,
Present and Future. In Proceedings of the 21st International Conference of the System
Dynamics Society. New Y ork, NY.
Burrell, G., & Morgan, G. (1979). Sociological Paradigms and Organizational Analysis. London:
Heinemann.
Caulfield, C. W., & Maj, S. P. (2002). A Case for System Dynamics. Global Journal of
Engineering Education, 6 (1), 25-34.
Coyle, G. (2000). Qualitative and quantitative modelling in system dynamics: some research
questions. System Dynamics Review, 16 (3), 225-244.
Coyle, R. G. (1979). Management System Dynamics. Chichester: John Wiley & Sons.
Doridot, F. (2008). Towards an ‘Engineered Epistemology’? Interdisciplinary Science Reviews, 33
(3), 254-262.
Eckerd, A., Landsbergen, D., & Desai, A. (2011). The Validity Tests Used by Social Scientists and
Decision Makers. In Proceedings of the 29th International Conference of the System
Dynamics Society. Washington, DC.
Epstein, J. M. (2008). Why Model? Journal of Artificial Societies and Social Simulation, 11 (4).
Flood, R., & Jackson, M. (1991). Creative Problem Solving. Chichester: John Wiley & Sons.
Floridi, L. (2011). A Defence of Constructionism: Philosophy as Conceptual Engineering.
Metaphilosophy, 42 (3), 282-304.
Forrester, J. W. (1961). Industrial Dynamics. Cambridge, MA: Productivity Press.
Goldberg, D. E. (2008). Bury the Cold War Curriculum ASEE Prism - The American Society for
Engineering Education, 17 (8), 68.
Goldberg, D. E. (2010). The Missing Basics and Other Philosophical Reflections for the
Transformation of Engineering Education. In D. Grasso & M. B. Burkins (Eds.), Holistic
Engineering Education (pp. 145-158). New Y ork: Springer.
Goldman, S. L. (1991). The Social Captivity of Engineering. In P. T. Durbin (Ed.), Critical
Perspectives on Nonacademic Science and Engineering. Bethlehem, PA: Lehigh University
Press.
Goldman, S. L. (2004). Why we need a philosophy of engineering: a work in progress.
Interdisciplinary Science Reviews, 29 (2), 163-176.
Homer, J. B. (1996). Why we iterate: scientific modeling in theory and practice. System Dynamics
Review, 12 (1), 1-19.
Jackson, M. (2003). Systems Thinking: Creative Holism for Managers. Chichester: John Wiley &
Sons.
Koen, B. V. (2003). Discussion of The Method. In. Oxford: Oxford University Press.
Lane, D. (2000). Should System Dynamics be Described as a ‘Hard' or ‘Deterministic’ Systems
Approach? Systems Research and Behavioral Science, 17, 3-22.
Lane, D. (2001). Rerum cognoscere causas: Part I— How do the ideas of system dynamics relate to
traditional social theories and the voluntarism/determinism debate? System Dynamics
Review, 17 (2), 97-118.
Luna-Reyes, L. F., & Andersen, D. L. (2003). Collecting and analyzing qualitative data for system
dynamics: methods and models. System Dynamics Review, 19 (4), 271-296.
McCarthy, N. (2010). A World of Things Not Facts. In I. Van de Poel & D. E. Goldberg (Eds.),
Philosophy and engineering. An emerging agenda (pp. 265-273). Dordrecht: Springer.
Meadows, D. H. (1980). The Unavoidable A Priori. In J. Randers (Ed.), Elements of the System
Dynamics Method (pp. 23-57). Cambridge, MA: Productivity Press.
Mitcham, C. (1994). Thinking Through Technology. The Path Between Engineering and
Philosophy. Chicago: The University of Chicago Press.
Petroski, H. (2009). Want to Engineer Real Change? Don't Ask a Scientist. Washington Post, 25
Jan 2009.
Petroski, H. (2010). The Essential Engineer. Why Science Alone will not Solve our Global
Problems. New Y ork: Vintage Books (Ed. 2011).
Petroski, H. (2011). Engineering Trumps Science. ASEE Prism, 20 (6).
Petroski, H. (2012). Landing on Mars. ASEE Prism, 22 (3).
Pitt, J. C. (2011). Doing Philosophy of Technology. Dordrecht: Springer.
Radzicki, M. J., & Karanian, B. A. (2002). Why Every Engineering Student Should Study System
Dynamics. In Proceedings of the 32nd ASEE/IEEE Frontiers in Education Conference,
Boston, MA.
Richardson, G. P. (1991). Feedback Thought in Social Science and Systems Theory. Waltham, MA:
Pegasus Communications.
Roberts, E. B. (1978). System Dynamics - An Introduction. In E. B. Roberts (Ed.), Managerial
Applications of System Dynamics. Waltham, MA: Pegasus Communications Inc.
Saeed, K. (1997). System Dynamics as a Technology of Learning for New Liberal Education. In
Proceedings of the 29th Frontiers in Education Conference, Pittsburgh, PA.
Schmidt, J. A. (2012). What Makes Engineering, Engineering? In J. Carrato & J. Burs (Eds.),
Structures Congress Proceedings (pp. 1160-1168): American Society of Civil Engineers.
Stein, E., & Lipton, P. (1989). Where Guesses Come From: Evolutionary Epistemology and the
Anomaly of Guided Variation. Biology and Philosophy, 4, 33-56.
Sterman, J. (1994). Learning in and about complex systems. System Dynamics Review, 10 (2-3),
291-330.
Sterman, J. (2000). Business Dynamics. Systems Thinking and Modeling for a Complex World.
Boston, MA: McGraw-Hill.
Van de Poel, I., & Goldberg, D. E. (2010). Philosophy and engineering. An emerging agenda.
Dordrecht: Springer.
Vanderminden, P. (2006). System Dynamics - A Field of Study, a Methodology or Both. Paper
presented at the 24th International Conference of The System Dynamics Society, Radboud
University Nijmegen, The Netherlands.
Wolstenholme, E. F. (1990). System Enquiry. Chichester: John Wiley & Sons.
