Learning from Microworld Environments: A Summary of the
Research Issues
Paul A. Langley and John D.W. Morecroft
London Management Centre London Business School
University of Westminster Sussex Place
London. NW1 5LS UK London. NW1 4SA UK
Email: langleyp@wmin.ac.uk Email: jmorecroft@lbs.lon.ac.uk
Introduction
The system dynamics community is interested in the ways in which modelling and simulation
tools may be used to enhance learning about complex dynamic systems. We hope that this
enhanced learning will result in more robust policy making, i.e., improved decisions by the
policy maker, and henceforth subsequent improvements in organisational performance (as
measured by revenues, profits, market share, returns on sales, returns on capital invested,
improved social welfare, and so on). There is general consensus within the SD community that
the process of building and simulating formal models is a valuable team learning activity for
participants involved in the model building activity (Morecroft and Sterman, 1994).
Microworlds are software tools which enable a wider audience to experience the dynamics of
a particular complex system (e.g., upstream oil production) or a particular management issue
(e.g., growing a service business). They facilitate the dissemination of SD principles relating
systemic structure to behaviour. The question of whether any valuable learning takes place
from the use of microworld environments generates widespread disagreement within the SD
community. Some see microworlds as “no different from the business games in use 25 years
ago” whereas others see them as “significant technological developments for management
education and development”. Sometimes the reference point for comparison is lost in these
statements. Is the comparison being made with conventional paper case studies, electronic
learning environments, or the process of building and simulating models?
The Story so Far
Graham et al (1994) discussed the issues relating to the expected efficacy of model-supported
case studies (microworlds). The close link between a real-life case study (rich in detail) and a
simulation model (focuses on specific dynamic issues) seems to provide a very effective tool
for learning. Within the field, studies to investigate the efficacy of microworld-based learning
tools have been undertaken using a range of research methodologies from observations in
workshops (field anthropology) to controlled experiments. All these methodologies have their
relative merits. These studies have generated many observations, including:
e Much time is required to accrue benefits from experimentation with a number of different
policies. This is often far more time than is available in a typical workshop activity. The
issue then becomes “how much value-added is there from using a microworld for a couple
of hours?”. For example, in People Express, the policy space of growth rates (low,
medium, high) vs marketing mix (cheap/no-frills, expensive/full service) is at least a three
by two matrix and thus requires at least six runs to explore it properly.
e Typically, microworlds are similar to business games. The user makes periodic (quarterly,
yearly) decisions and receives outcome feedback (graphs, tables) on the state of the system.
300
There seems to be some difficulty for users in translating policies (e.g., “let’s grow fast at
30% per annum and price well below competitors”) to actual quarterly/yearly decisions.
Even with a planning sheet in front of them, users start off following the intended policy
carefully, but then react to recent outcome feedback in making decisions and forget about
the intended policies. This may well be the best way of managing the system, but does not
allow rigorous reflection on the success or failure of a particular policy.
¢ The learning objective of the workshop or activity plays a large part in determining the
efficacy of the microworld activity. For example, experiencing a typical disaster scenario in
a complex system is a valuable learning experience if the student is then given insight into
the systemic structure and learns how manage the system more effectively. Both the Beer
Game and People Express work well in this respect.
¢ User performance (as measured by cumulative profit or value of assets) is poor relative to
simple behavioral benchmarks - typically 30% below a benchmark and well below optimal
performance. Learning to improve performance with outcome feedback is poor (Paich and
Sterman, 1993 and Langley, 1995). Learning about systemic structure from outcome
feedback is also poor.
¢ Learning about some management policies is more easily transferred between case studies
than others. For example, in People Express, “demand-side marketing policies must be used
in conjunction with supply side capacity investment policies to grow smoothly” and “with a
selective hiring policy you can’t hire your way out of a declining service quality problem”
seem like they could apply to many service businesses. Similarly, in Beefeater Restaurants
(1996), “cutting staffing and maintenance costs to meet short-term profit targets results in
long-term failure of the business” is a dynamic we are all familiar with.
e When microworld users are asked about insights gained from the workshop/learning
activity they typically have much to say. They often can cite several examples from their
own organisation and their own department. This suggests that transfer of learning to their
own managerial activities does in fact take place.
« There are many process differences between individual and team learning. Transparent
microworld-based learning environments, which give insight into systemic structure online
may be more suitable for individual learners than for management teams, due to the
different learning styles of individuals and teams.
¢ There is some doubt within the field (see for example Paich and Sterman, 1993) as to
whether any substantive learning about systemic structure can take place without the
subject being involved in some or all of the processes of model building (ie.,
conceptualisation, mapping, flow-diagramming, equation writing, validation).
¢ There is some evidence (Langley, 1995) that using microworlds to support conventional
case studies improves the quality and richness of the student’s written assignments,
typically involving the critical evaluation of strategic options and policies.
Work in Progress
We are currently undertaking two studies (one a controlled experiment with MBA students
and the other based on field work with MBA students and executives) to explore some of
these research issues. The first issue relates to the question of whether subjects can learn
something about the systemic structure of the simulated system, without actually doing any
modelling themselves, and then use that learning to improve performance. Given that learning
from outcome feedback is so poor, to what extent can the subject performance be improved
through the provision of cognitive feedback about the task structure, i.e., the operating and
policy structure of the system? Does the subject reach a certain performance level more
Bol
quickly with the cognitive feedback, i.e., is the learning process accelerated? If indeed it is,
then that is an important finding. Users’ time is limited (both student and executive), so there
are many advantages for the design of microworld workshops if the learning can be
accelerated. The second issue relates to how adequately microworld users can identify
successful/unsuccessful “policies” after a period of experimentation. Also, do they learn more
if we allow them to formulate “policies” rather than make periodic “decisions” thus moving
the user interface closer to “simulation” rather than “gaming” mode.
Study 1 - Oil Producers Microworld - Cognitive Feedback (task structure)
In two related experiments, over 120 Masters subjects completed a series of six tasks
involving dynamic decision making in a complex system of the global oil industry (Langley,
1995). Each task involved making yearly upstream capacity approval decisions, over a
simulated 25 year period. Each task involved different industry demand/competitor strategy
scenarios. The tasks were completed as part of an assignment, with a generous time constraint
of two weeks to encourage learning. Performance in each task (relative to a benchmark
performance for each scenario) was measured by cumulative net income over 25 years.
Performance improvement was used as a surrogate for learning. Subjects were randomly
allocated to three cognitive feedback treatment groups and a control group which received
only outcome feedback. The cognitive feedback groups also received outcome feedback.
Cognitive feedback was only available during the first three trials, and was removed during the
last three. Both treatments and control groups still received outcome feedback during the last
three trials. The cognitive feedback included graphical representations of the systemic
structure (causal loops) with hypertext “explanations” and “tips” of structure/behaviour links.
Subjects in the cognitive feedback groups reached the limit of performance more quickly (trial
three), compared to the control group (by trial five). The ceiling on performance was the same
by trial six for all groups, and was still 25% below a behavioral benchmark. Overall the time
spent on the six trials was encouragingly high (mean 79 minutes on trial one), and 75%
subjects ran the software at home. But the time spent by subjects using the cognitive feedback
in the treatment groups declined significantly after trial one. Subjects seemed to use the
cognitive feedback as an “electronic book”, to be skimmed before playing the game. In the
second experiment, over 20 subjects (30%) did not use the cognitive feedback made available
to them. As a self-selected (non-random) group, they performed significantly worse than the
subjects who used the cognitive feedback.
Study 2 - Beefeater Restaurants Microworld - Policy formulation
The Beefeater Restaurants Microworld simulates the growth of a retail service business, and
explores the tension between serving the needs of customers and meeting the financial aims of
investors. In a typical microworld interface, users take the role of the Beefeater Restaurants
division management team, and try to cope with the diverse problems of start-up, growth and
business maturity. They steer the strategy by making quarterly decisions (over ten years) on
pricing, staffing, marketing, maintenance, product development and capital spending (on new
restaurants). After participating in a number of management challenges (“keep it going”, “pick
up the pieces”, “coping with maturity”) using the microworld, 40 Masters subjects were asked
to specify the policies or decision rules they had been using. They had been taught how to
make simple policy formulations using both algebra (e.g, price = price. *
(backlog/capacity)*) and pseudo-code (e.g., price was kept more or less the same, but was
adjusted by + 5% to match competitor price).
Sor
The cues that subjects used varied widely, both in terms of numbers of cues (range two to
eight, mean four) and the formulation. It was not clear if subjects could accurately specify the
policies they had been experimenting with. Analysis of the subjects’ actual decisions recorded
on disc showed inconsistency with their own perception of the decision rule they had used.
Consistent with earlier studies, performance was poor relative to simple behavioral
benchmarks - whatever policies they had been using they were not very effective.
Research Questions/Issues
There is no doubt that individuals and teams learn much from building and simulating models,
but this activity takes considerably longer than the time that student and executive learners can
spend on a workshop/assignment activity. Can these learners discover the same insights from
microworld use as they do from building and simulating models? Providing either online or
paper-based model transparency (giving insight into model structure and its link to behaviour)
is clearly a valuable addition to the microworld environment. However, we must accept that
the understanding of certain generic structures is more easily transferred (to other simulated
environments and to the real world) than others. For example, we often meet the dynamics of
growth management and capital investment in microworlds (e.g., People Express, B&B
Enterprises and Beefeater Restaurants) but less frequently the dynamics of capital investment
with long lead times in a cartel controlled industry (e.g., Oil Producers). The experiments with
task structure cognitive feedback (described earlier) indicated that learning is indeed
accelerated by the provision of structure/behaviour relationships, but there is plenty of scope
for improvements in the design of the cognitive feedback.
Switching the mode of user interaction with the microworld from gaming to simulation,
allowing users to specify policies rather than decisions and run the model for continuous
periods may improve their ability to identify high leverage policies. It is likely to allow more
efficient use of time available, and an opportunity to apply the scientific method to
systematically investigating the policy space.
We also hope that individual learning within the microworld environment can be extended to
learning within management teams as well as being transferred to the real world. Studies to
examine the transfer of learning (both individuals and teams) within simulated environments
are underway. The issue of how well insights into the dynamic complexity of a system, and the
ability to formulate high leverage policies, can be transferred from a microworld environment
to the real world is still an open research question.
References
Beefeater Restaurants Microworld 1996. Software and documentation. Further details on the
web page http://www.stargate.co.uk/farthing/smsim or email smsim@farthing.stargate.co.uk
Graham, Alan K., John D.W. Morecroft, Peter M. Senge and John D. Sterman 1994. Model-
Supported Case Studies for Management Education, in Modeling for Learning Organizations,
Productivity Press.
Langley, Paul A. 1995. Using Cognitive Feedback to Improve Performance and Accelerate
Individual Learning in a Simulated Oil Industry, Working Paper, University of Westminster,
April 1996.
Morecroft, J.D.W. and J.D. Sterman (Eds), 1994. Modeling for Learning Organizations,
Productivity Press.
Paich, M. and J.D. Sterman 1993 Boom, Bust, and Failures to Learn in Experimental Markets,
Management Science, 39, 1439-1458.
303
