Assessing the Efficacy of Microworlds for Promoting Systems Thinking
Steven A. Cavaleri, Central Connecticut State University
James P. Thompson, Gemini Consulting, Inc.
Abstract
Constructing a system dynamics model and analyzing its behavior is a well documented practice
for enriching the builder’s mental model. When there is a larger audience for the use of a model —
for example, to help transfer the builder’s discoveries — modelers can construct an interface that
allows a user to change parameters in a model. Depending on the interface design, these
microworlds (or management flight simulators) may be useful to introduce a larger audience to
some of the concepts of system dynamics and systems thinking. Our work examines the use of
such microworlds to boost system dynamics skills in the classroom and strategic thinking in
commercial settings. The users report that microworlds are useful tools for helping them to grasp
and master the intended concepts.
Introduction
System dynamics models have established their place in the formulation of strategy. (Thompson
and Weiner, 1996) But the value of microworlds as a tool for attaining deep levels of learning is
the subject of much debate. (Langley, 1993) There is considerable discord among researchers
concerning the practical efficacy of microworlds in knowledge transfer. (Bakken etal, 1992)
These concerns must be addressed for microworlds to achieve their promise. However, our work
focuses more narrowly: (1) Is a generic microworld valuable for introducing system dynamics
concepts in a university classroom? and (2) Are microworlds valuable for stimulating strategic
thinking with a system-wide perspective in a commercial setting?
Research, Methods, and Results
In both the classroom and the commercial (“boardroom”) setting, we have some common goals.
We used microworlds to
* Provide a logical and reasonable view of a system and the interaction of its parts
Reach and inform a group larger than the model development team about observations and
opinions of the model developers
Calibrate, confirm and socialize the critical thinking skills of a group larger than the model
development team
For the boardroom, we wished to create an environment for developing strategy by generating
simulation-based experiences that parallel real world experiences.
Our observations come from microworld use by four groups of Cavaleri’s graduate and
undergraduate business students (about 65 students over two years) and Thompson’s group of 24
managers at a Fortune 500 company. The students employed commercially available microworlds.
The managers employed a custom microworld from a system dynamics model that simulates
disease propagation, market development, drug development and production, U.S. federal
approvals process and marketing strategy for a pharmaceuticals company. As such our
observations are somewhat informal and rely on anecdotes.
Of the student groups, three had no significant prior exposure to the notion of systems thinking or
the discipline of system dynamics. One group, Group | in the data below, had a introduction to
some systems thinking (i.e., feedback, causal loops and system archetypes) in about ten hours of
classroom lecture and discourse. Aaron™ was used by Groups I-III, and Group IV employed the
Service Quality microworld.' The students responded anonymously in writing within a week of
44
completing their experience with the microworlds to eight questions. The questions and response
data are summarized below:
Mean Response
Group / Size —-— = =
Question
1. Microworld’s overall value as a learning tool in
comparison to all other approaches you have used HON, 6:43) 7.80) 31:60
(1 = Not Useful; 10 = Very Useful)
2. Effectiveness of microworld in helping to develop
insight about new ways to manage wets. iOHL ISS, 925
(1 = Not Effective; 10 = Very Effective)
3. Effectiveness of microworld in helping to better
understand the process of managing 733 661 736 7.41
(1 = Not Effective; 10 = Very Effective)
4. Does the learner consider the microworld to be user-
friendly 6.75 5.14 5.68 8.50
(1 = Unfriendly; 10 = Very Friendly)
5. Efficiency of the microworld in helping to develop
insights as compared to other methods you have used T88 833) '679' 730
(1 =Very inefficient; 10 = Very Efficient)
6. Has the learner ever used a microworld before 100% na. na. 42%
(Yes; No) vss a
7. Value of the microworld as a learning tool compared
to other computer simulations used Sal owe; fas “TL
(1 = Much less useful; 10 = Much more useful)
8. Have your beliefs about managing changed as a result
of using this micro world; if so approximately by how
much
Al na. na. .68
(expressed in a range from 0 to I)
Assessing the Efficacy of Microworlds for Promoting Systems Thinking 2
Steven A. Cavaleri, Central Connecticut State University
James P. Thompson, Gemini Consulting, Inc.
qs
The data from the surveys indicate that Cavaleri’s learners found the microworlds to be a useful
and effective tool for deepening their understanding of managing. By observation Cavaleri notes
that advanced undergraduates and graduate students were better able to take advantage of Aaron’s
sophisticated capabilities while novices found the ease-of-use of the Service Quality microworld to
be helpful.
Since the primary focus of the manager workshop was to develop strategy, it was important to
keep the pace lively. So, Thompson’s users were guided through some of the mechanical
processes of manipulating the interface.
Our Observations On The Use Of Microworlds
The above data apply to the use of microworlds in the classroom. But the overarching goals for
the classroom and the boardroom use of microworlds go beyond ease-of-use questions.
Specifically, we use microworlds to steepen the learning curve — to shorten the time from
introducing to grasping a topic with rich dynamic complexity. Our approach to these goals is
similar in the classroom and the boardroom. Here, the boardroom is a workshop centered on the
microworld.
The common steps we follow are —
* Familiarize participants with apriori of system dynamics
¢ Introduce participants to the concepts of feedback and reinforcing and controlling feedback
loops
* Review the structure of the simulation model that underlies the microworld
— Review the issues addressed in the simulation model
— Provide a high-level view of the major feedback loops
— Review model boundaries through discussion of inputs and outputs from simulation
¢ Analyze simulation outcomes and explore choices for policy changes to improve system
performance
In the classroom, these program steps are usually accomplished with Problem Sets. In the
boardroom, the same program steps are taken with a Scenario Generation and Evaluation
Workshop. Emphasis is placed on observing performance of important variables in various
simulation settings. Students and managers learn to trace the causes of systemic performance to
the structure of the system.
Both microworlds employ an interface that allows access to a graphical representation of the
system dynamics model. In the student’s microworlds, this portion of interface was a causal loop
diagram. For the manager group, the interface accessed a complete stock and flow diagram. For
both, the users could access all model equations and trace causes of simulation performance
through the application environment. We believe that all user groups benefited from using these
tools. In particular, the explicit representation of the interconnectedness of variables helped users
to relate model behavior to their experience in the real world. This validation-by-use may place an
extra burden on the model’s creators, but it is an important test that enables users to gain
confidence in what they are seeing.
The problem sets required the students to repeatedly use the same model with variations in _
parameter settings for simulations run over a period of weeks. The manager workshop required
the participants to run simulations with different parameter settings over the course of about eight
hours.
Assessing the Efficacy of Microworlds for Promoting Systems Thinking 3
Steven A. Cavaleni, Central Connecticut State University
James P. Thompson, Gemini Consulting, Inc.
Ge
The completeness of model sectors allowed the users access to data that they may not be likely to
consider in other environments. We take this as both a microworld weakness and a strength — a
weakness because the users sometimes experienced information overload and a strength because
the users found that previously held notions of cause-and-effect were challenged by the structure of
the model. The users often found important and subtle policy levers by thoroughly tracing causes
for simulation performance.
Here are some comments from our follow-up interviews with the managers —
* This confirms the importance of improving our analytical skills to shorten the time to high
quality decision-making. In today’s intensely competitive environment, we can no longer
afford the time to ponder endlessly and avoid making the tough decisions. We need to think
crisply and be wise stewards of our time. CEO
¢ liked the part where we ran the scenario and then talked about the real world. The computer
[microworld] deflected a lot of the negative thinking. Market analyst
* The group surprised me. 1 expected everyone to duck when they realized [our product
offering] might not be a world-beater. Research physician
¢ The frightening thing for middle managers is that people with a few years’ experience can learn
all their tricks in a day or two. You better watch out — by enabling the inexperienced, you
threaten the proprietary knowledge of middle management. Industry senior consultant
Summary
Inexperienced users gained, or increased their appreciation of, critical thinking skills. Our
microworlds’ dynamic complexity, lively behavior and believable outputs encouraged users to
relate simulation experience to their observations of the real world, and so to socialize the concepts
captured in the model among a group larger than the model developers. We are reluctant to draw
more conclusions from such a small amount of data. But we are encouraged by the results so far.
The microworlds have helped the users to quickly develop a system-wide view and enhance their
knowledge of interconnectedness between parts in that system.
References:
Bakken, B., J. Gould and D. Kim. (1993) “Experimentation in Learning Organizations: A Management Flight
Simulator Approach” in Modeling for Learning Organizations, John D.W. Morecroft and John D. Sterman,
editors. Portland, OR: Productivity Press
Langley, Paul A. (1993), “Learning with Model-Supported Case Studies” in Proceedings of the International System
Dynamics Conference 1993, pp. 245-254, Cambridge: The System Dynamics Society
Thompson, James P. and Joan Weiner. (1996) “Strategic Planning As A Tool For Building Learning Capacity” in
Managing in Organizations that Learn, Steven A. Cavaleri and David Fearon, editors. Cambridge: Blackwell
Publishing, Inc.
‘ Aaron™ is a highly configurable model of competitive dynamics and is published by LeapTec, 81 Hazard Avenue,
Providence, RI] 02906. Phone: (401) 467-9292. Fax: (401) 941-0412. The Service Quality Game is available from
GKA, Inc., 125 Cambridge Park Drive, Cambridge, MA 02140 Phone: (617) 441-7766, Fax: 617-491-6744
Assessing the Efficacy of Microworlds for Promoting Systems Thinking 4
Steven A. Cavaleri, Central Connecticut State University
James P. Thompson, Gemini Consulting, Inc.
cal
CC BY-NC-SA 4.0