( 254
8/17/81
DIAGNOSING SURPRISE MODEL BEHAVIOR:
A T0CL FOR BVOIVING BEHAVIORAL AND POLICY INSIGHTS
by
Nathaniel J. Mass
Associate Professor of Management
or, System Dynamics National Model Project
Alfred P. Sloan School of Management
Massachusetts Institute of Technology
Cambridge, Massachusetts 02139
Pree
August 1981
To be presented at the
1961 System Dynamics Research Conference
The Institute on Nan and Science
Rensselaervilie, New York
October 14-17, 1981
ame aities
ABSTRACT
Both in the incipient and later phases of developing a model,
unexpected behavior is ‘frequently encountered--that is, behavior which is
he
at odds with the initial expectations of the model builder or client.
appearance of such surprise behavior inmediately raises two possibilities:
either the behavior is implausible, and the model therefore must be
revised; or the behavior withstands ecrutiny and reveals previously
unappreciated aspects of the eystem. In either instance, the process of
diagnosing and interpreting surprise behavior gives a powerful basis for
model evolution and generating policy insights. But frequently, it-is
quite difficult in practice to discem whether the incidence of surprise
model behavior reveals errors or suggests insights.
This paper is designed. to contribute to the literature on model
formulation, testing, and policy analysis, by discussing the criteria for
diagnosing surprise model behavior. Several case examples are presented in
which appropriate resolution of surprise behavior led to significant model
improvements and/or behavior insights. Moreover, operational guidelines
are presented to increase the likelihood of uncovering and successfully '
treating surprise behavior.
D-3323 3
8/17/81
1. PROCESS OF GENERATING BEHAVIORAL INSIGHTS FROM SYSTEM DYNAMICS MODELS
Mathematical models are designed for different purposes and with
correspondingly different underlying approaches. At one end of the
spectrum, time series models do not purport to be built up from a causal
atructure, but rather are designed to extract in a sophisticated way
secular trends, cyclical fluctuations, or other patterns of behavior that
are contained in e series of historical data, and project those patterns
into the future on an assumption of continuity. In the middle range, many
statistical and econometric models are designed both to replicate
particular sets of time series data, but also to capture key accounting
identities and behavioral relationships that characterize the aysten
structure. System dynamics models are noteworthy, if not completely
unique, in insistence on a high degree of structural realism, and wet
important from the standpoint of this paper, a high level of explanatory
content for relating system structure to observed behavior patterns,
pathologies (that is, problematic behavior), and policy alternatives. In
other words, a system dynamics model is intended, beyond objectives of
forecasting or prediction, to yield operational insights about the feedback
relationships that can produce or contribute to problems, can counteract
the efficacy of policy interventions, or alternatively, can reinforce
benefits of policy actions aimed at high leverage ‘points.
What I have said thus far about the nature of models is not new, but
sinply reiterates the empha:
is of system dynamics models on explanatory
power in practical terms and at a managerially-relevant level. On the
D-3325 4
8/17/31
other hand, what is not well-documented in the literature on models, and
may not even be well appreciated by many model builders, is the process
through which behavioral insights are arrived et using a model. Several
articles in the system dynamics literature over the yeare have advised the
model builder to begin a new effort with a clear “reference node" thet
describes the time path of the problematic behavior being adiressed, end
also a “dynamic hypothesis" that lays out an initial theory of the
.Principal forces that could interact to produce the reference behavior.
From these initial constructs, and from additional data, descriptive
information, literature, and theory that the modeler can bring to bear, a
first model is developed. “The model is then improved through successive
rounds of analysis and consequent refinement. This progression of problex
statement, initial hypothesis, first model, and successive nodel versions
through iterative improvement, seems logical end is in fect frequently
helpful as a guide to the phases of model construction. But in my
experience, and I believe as well in the experience of many other nodel
builders, the usual description of the model building process is much too
orderly and free of tumult, and thereby misses one of the ajest inportent
dimensions of the model building process.
In a variety of major modeling and policy etudies in vhich I have
been involved either as a direct participant or close observer, the
understanding of real system behavior held by modelers and clients alike,
and sometimes even the very concept of what the modeling study is about,
has changed dramatically in mid-course as a consequence of surprisir
behavior revealed by an early model version. As a result, the course end”
stated objectives of the project were altered substantially fron the
Yass,
Feindustrielization,” Technology Review, August, 1981.
5
8/17/81
initial problea statements to reflect the new understanding of system
functioning. Let me give three examples:
4. Early on in the development of the System Dynamics National
Nodél, criginally around 1975, we assembled versions of a “standard
production sector" representing a detailed behavioral model of
industrial operations, to portray a consumer goods producing sector
and a capital goods producing sector, and the interactions between
them as a consequence of the demand and supply for capital goods.
The resulting model. revealed a 50-year fluctuation of large
periodicity originating in the capital producing sectors, and vith
recurring sharp peaks in economic activity separated by brosd values
of depression and subsequent recovery to a new peak. Until that
time, the main objective of the National Model Project had been seen
ae exploring in a national context issues of inflation and the
le of economic development involving resource and energy
Moreover, none of the project staff had been
aware of any process of recurring great depressions in
t it might be important for the National Model to
As the reasons for large amplitude fluctuating behavior
-year periodicity vere, understood from the perspective of
‘Ss producing the behavior within the model, the behavior
appear nore a6 a plausible managerial and economic
phenomenon, and less es an aberration. Related literature and
21 data on long-term economic behavior were marshalled as an
ticnal medium of refutation or support for the model behavior.
tine, we have gained increasing confidence that the originally
xpected model behavior in fact represented a significant set of
public and corporate policy issues that the National Model could
telp to exrose.* Exploration of causes and implications of
long-wave tehavior subsequently became a major thrust, although not
only objective, of the National Model project. This brief
ount provides an example of surprise model behavior that brings
to light a completely new phenomenon or pattern of behavior that the
noiel could significantly addres
2... In an industrial research project based at MIT, a model was
constructed to explain the sources of long-term decline in market
shere exrerienced by a major equipment manufacturer. Early versions
of the nodel in fact generated from internal causes the reference
pattern of declining market share. The replication of actual
behavior experienced in real life was, of course, significant. But
the nore important question was: Why does the behavior arise and
what policies could be exercised to reverse the declining trend? On
this question, the model suggested that declining market share
occurred during periods of low overall industry demand for the
corpany's product; and moreover, the prime cause of loss of market
share iuring these times was a high delivery delay (meaning lack of
thaniel J., and Peter M. Senge, “Behind the Clamor for
~ 256
D-3523 6
B/IT/Et
availability) of the company's product. The timing relationship
between overall industrial downturns and loss of market share for
this particular company was unanticipated; nonetheless, scrutiny
of company records confirmed the correspondence. However, the
incidence of high delivery delays during a period-of low industry
demand seemed intuitively implausible, implying that the company's
product was least available when no one wanted it. However,
review of the company's records again revealed that precisely this
had been occurring. The resulting insights into systen behavior
changed dranatically the directions for further model development
and ongoing policy analysis using the model. ‘This case thus
provides an example of a situation where a model did indeed
replicate an anticipated reference pattern of behavior, but where
the cause of that behavior was almost entirely unexpected.
3+ A preliminary version of a financial model developed for s
major bank suggested that the bank's policies for paying out cesh
and stock dividends, as they hed been described by executives in
the bank actually responsible for those decisions and thereby
incorporated in the model, could substantially reduce, or even
eliminate, growth in earnings per share. At the same time though,
data showed a clear growth in actual earnings per share
experienced by the bank. The process of reconciling the factual
circumstances and the initial model output highlighted several key
relationships that had-not been mentioned by the executives as
important considerations in their dividend decision process and
were therefore not included in the initial model, but which later
in fact appeared responsible for the continued grovth in earnings
per share. In this example, then, the process of understanding
surprise model behavior led to important changes in model
specification, ag well as to the realization of potentially
conflicting elements of the managerial decision process in the
actual firm.
Each of the three examples cited above shares several common
elements. First, behavior emerged from a preliminary model that vas
surprising to all participants in the modeling process, including both
model builders and clients. Second, in each case, the surprising behavior
could not inmediately be rejected as being either factually incorrect or
implausible as « prediction of future behavior. Third, the process of
interpreting the surprise behavior required the development of new
frameworks for viewing available data and knowledge about behavior and
management policies, Fourth, the process of resolving the surprise model
behavior led to appreciable shifts in the basic thrust of the modeling and
policy analysis effort.
D~352: 1
aes 8/17/81
In general terme, the appearance of surprise behavior froa a model
immediately raises tvo possibilities: either the behavior ie implausible,
and the model therefore must be revised; or the behavior withstands
scrutiny and reveale previously unappreciated aspects of the aysten. But
even more subtly, it frequently appears quite difficult in practice to
discern whether the incidence of surprise model behavior reveale errore or,
alternatively, suggests insights. When confronted with surprising model
behavior, and especially behavior that appears at odds vith initial
impressions or hypotheses about system operation, many model builders would
be tempted to assume that the model is behaving unreasonably, and to “cover
up" the surprise behavior through parameter changes or structural modifica~
tiotis. Om the other side, I have seen a variety of instances vhere a model
Duilder will accept surprise model behavior as providing a source of
significant policy insights, vhere in fact the behavior points up flaws in
basic model design. In many respects, it is the very behavioral richness
of system dynamics that is the source of this "identification" problem.
‘This paper attempts to contribute to the literature on model
formulation, testing, and policy analysis by discussing the criteria for
diagnosing surprise model behavior. In particular, Secton 2 presents
genera) guidelines as well as specific categories of tests for increasing
the likelihood of uncovering and successfully treating surprise behavior.
257
D-3323 8
8/17/81
2. QUIDELINES AND TESTS FOR RESOLVING SURPRISE BEHAVIOR
2.1 Follow Up All Unanticipated Behavior to Appropriate Resclution
‘The most basic guideline for effectively treating surrrise model
behavior is that whenever such behavior is ‘encountered, it should be
assessed, understood, and followed through to an appropriate resolution,
whether in tera of model improvements or adoption of new perspectives on
system behavior. The model builder must adopt perspective that views the
encountering of surprise model behavior as a significant opportunity to be
capitalized upon. In contrast, the inexperienced model tuilder who con-
fronts surprise or anomalous model behavior, is prone to pursue parameter
combinations that make the anonalous behavior less evident, or simply to
dismise the behavior as being outside of the intended use of the sodel.
One of the significant aspects of system dynamics models, whether in the
corporate policy or public policy realm, i8 that a well-structured model
will frequently come to have uses beyond those originally envisioned. In
other words, a effective aystem dynamics model is probably best viewed as a
multi-purpose or general purpose model, even if it was originally designed
only for narrower uses.
4’n important dimension of resolving surprise model behavior is to
balance model-besed results, empirical data, and client knowledge about
system behavior. As seen in the brief examples cited in Section 1, some of
the most important insights into resl system behavior can arise from model
results that at first appear to be at odds with knowledge of the real
258
Do332: 9
22 8/17/81
eysten, but which in fact suggest important new interpretations of
Ferceived facts. The system dynamics model builder/analyst/consultant can
have an important, though difficult to play, role as a change agent. On
the one hand, the model builder must recognize and accept the possibility
that much of the surprise behavior encountered, particularly in early
Fhases of model development, may point up defects in a model more than
Farticular insights. But on the other hand, especially as the model
builder is core experienced and more knowledgeable of the real system, and
as the model improves progressively over time, the likelihood increases
that surprise model behavior points to new ideas that bear on policy
formulation. In order to play the role of change agent effectively, the
analyst must te sure that he has a broad appreciation of available data,
literature, and managerial experience (including effects of previously
inplenented policy changes), and he must be sensitive to the actual
organizational pressures and relationships. But on the other hand, the
nodel tuilder nuat delve sufficiently deeply into the sources of model
behavior to be able to explain in novel, although practical terms,, the
forces that may produce unextected results in the actual system.
Especialiy in a consulting (as opposed to research) environment, a system
dynamics analyst can be rendered ineffective if he appears unaware of
existing deta and points of view about organizational behavior, and if he
is unable to relate to that existing knowledge in a conscious and creative
way. Fron this standpoint, a danger is that the consultant be “captured”
by the client, so that significant new policy perspectives that could
emerge from the model are not successfully cultivated to fruition.*
. For exarple, see Charles W. Gibson, “Using Models in Financial
Planning,” Journal of Business Strategy, Vol. , No. 4, 1981.
D-3323 10
a/i7/er
2.2 Importance of A Priori Expectation of Model Behavior
If we accept the importance of surprise model behavior as a
@iagnostic for model improvement and policy formulation, then a basic point
emerges. Appearance of “surprise” behavior implies a discrepancy between
results actually produced and previous expectations. of those results.
‘Thus, it is absolutely essential that the model builder have a strong a
priori expectation of nodel outcones, to establish a baseline against which
surprise model behavior can be recognized through the appearance of &
discrepancy that evokes "cognitive dissonance.”
In dtecuseing the form that an a priori expectation of model
behavior may take, I believe it ie useful to distinguich three classes of
models. I define a ‘type 1 motel as a model that is addressed to a vell-
established set of problems or circumstances observed in the past. A Type
1 model thus corresponds most naturally to the “classical” statement of
purpose for a systen dynamics model, where a historical reference mode
provides a basis for model development. The historical reference node nay
portray declining market share in a corporation, results following
implementation of a particular public policy that showed changes in the
opposite direction from that intended, or similar phenomena. Such an
historical reference mode provides the a priori expectation of results. If
the model does not replicate the historical reference mode, then the model
needs to be revised, while admitting the possiblity that model results may
cast the historical circumstances in a new light.
1
8/17/81
A Type 2 model can be defined as being addressed to a defined set of
policy issus rathern than to a particular historical circumstance. For
example, I and several others have been working on a system dynamics model
for a foreign government to help anticipate effects of alternative
strategies for oil development and oil export. This particular nation has
had no history of significant oil export in the past, 20 it is clearly not
relevant to draw a historical reference mode to guide model develoyment.
Of course, other nations have gone through stages of oil development, with
varying degrees of success or failure, and in some sense these alternative
experience curves comprise a historical base. Experiences of nations or
orgenizations other than the client organization being studied may have
relevance to the kinds of futures that should be encouraged or avoided for
a client. But I see such experiences as forming somewhat more equivocal
and less direct reference for model development than established wast:
history for a client orgenization. Thus, while there is probably no sharp
dividing line tetween the reference point for starting a Type 1 or Type 2
model, the extent to which the model must replicate reference behavior is
clearly different.
For developing a Type 2 model, then, experiences of related
organizations or systems may comprise part of the a priori expectation of
system behavior. But frequently still more important is a priori
expectation of possible effects of implementing the policies that the model
is being designed to analyze. In the case of the oil policy model, that a
priori expectation includes guesses as to the answer of questions such as
the following: What would be the effects on inflation, employment, and
other economic aggregates of substantially higher or lower levels of oil
259
D-3323 12
8/17 /8s
exports? What would be the short-run and long-run effects of setting ¢
domestic price of oil at world levels, or maintaining it at substantially
lower levels through explicit or implicit government subsidies? For a Tye
2 model, it is important to recognize that the a priori expectations of
behavior or policy impacts are not established facts thet the mete! Bust
replicate as a basis for validity. Rather, they comprise expectations that
enable a rigorous comparison of eventual results with the results thet were
originally expected. If a difference arises, then the appearance of that
difference calls for some resolution. ‘The model builder and client can
either adhere to the a priori expectation and elect to modify the defects
in the model that cause the model to fail in producing the expected
results; or alternatively, the a priori expectation nay te consciously
revised to conform to the new-understanding of system interactions. The
importance that I am attaching to the a priori expectation of behavior sey
seem exaggerated to some readers. But in my experience and that of others
in building modele of eyatens that lack a “hard” historical reference rode,
oth assessment of model validity and generation of policy insights can be
impeded by the absence of criteria, however transient and subject te revi-
sion, for evaluating the plausibility and significance of model tehsvior at
any point in time.
A Type 3 model in this classification is an extension of the Type 2
ues that the
model to systems for which there is again a list of policy
model should address but only a weak historical precedent for the
interactions being modeled. For example, I am now involved in developing a
growth strategy for a new company. ‘The company will be a sutsidiary of an
existing company in a line of business that represents a significant
260
13
8/17/81
extension of the parent company’s traditional areas. Moreover, the chief
executive and ruch of the management team for the new subsidiary has not
yet been appointed. ‘The parent company has clear ideas about the criteria
for success or failure of the new subsidiary, meaning that there are
identifiable patterns of behavior that would represent varying degrees of
success or failure. Moreover, corporate management has a list of potential
policies for the subsidiary whose effects it would like to understand
better. Because the ew subsidiary company has no past history, there is
certainly no historical reference mode to provide a point of departure for
2 rodel building effort. Purthermore, there do not seem to be any
histories of allied companies or companies in the same general area of
business to the subsidiary to comprise a sharply defined historical
reference ncde. In my view, a Type 3 model, while much more difficult to
cor
truct ard to evaluate than a Type 1 or even Type 2 model, is no less an
appropriate subject for system dynamics. Interactions can be identified,
albeit somewhat hyrothetically, that take the form of feedback loops and
tie the new conpany to its market, and significant policy issues can be
identified for enelysis using the model. In developing such a model,
aving a priori expectations of possible patterns of system behavior and
effects of possible policy changes becomes extremely important for tying
the odel to realit
Ny point cen be summarized as follows: system dynamics models can
start fren different points and different degrees of historical precedent.
4n historical reference mode is by no means a requirement for beginning a
model. Nonetheless, the model builder must have sharp @ priori
expectations about possible model results. These a priori expectations may
D-3323 16
7 8/17/81
take different forms, and can be articulated in more or less creative ways-
Without such expectations, there is no basis for judging when significant
surprises or anomalies appear in the model building process that should
motivate changes in the model or in the modeler‘s or client's viewpoint.
2.3 Confirm All Behavioral Hypotheses Through Appropriate Model Tests
When surprise model behavior is encountered, the nodel builder must
identify why the model produces the unexpected results. ‘The question of
why a model produces certain patterns of behavior can always be answered
with enough tine and effort relative to the model framework. Once the
model behavior is understood, the realism of both the behavior and the
underlying mechanisns must be challenged against corresponding behavior and
structure in real life. :
‘The mechanisms that produce surprise model behavior may take several
forms. For example, the model builder may identify a positive feedback
loop that was not previously recognized to exist that can cause major
excursions in particular variables. Alternatively, a model builder may
come to identify a negative feedback loop thet counteracts policy changes.
As still a third possibility, while not pretending to offer an inclusive
list, the model builder may identify a combined structure of positive and
negative feedback loops that can diminish the effectiveness of the policy
intervention, while calling for more and more of that intervention with
ever-declining efficiency over time. Whatever set of feedback structures
the model builder may hypothesize to yield the unexpected behavior that is
2-332! 15
ia 8/17/81
observed in the model, it is important for the model builder to devise
appropriate behavioral tests to confirm or reject his hypothesis.
The process of evaluating the hypothesis about sources of behavior
will always in some way involve segmenting and neutralising the forces in
question. For example, if the model builder feels that varying prices are
an indispensible part of a fluctuating mode of behavior in an, economic
model, he may artificially force price to be constant, through the
equivalent of a full price control. Alternatively, if behavior ie
hypothesized to result from fluctuating adequacy of household liquidity
and consequent interactions between consumer purchases, employment, and
wage payments, then adequacy of liquidity may be held at a neutral value,
or assumed to exert substantially greater or lesser impact on consumer
purchases. Such tests are necessary to verify that direct link from
liquidity to consumer purchases is in fact a sensitive point in producing
the behavior in question.
Sometines, a given hypothesis relating behavior to underlying causes
ean be tested from different points of view; alternatively, a given
hypothesis about behavior may be difficult to teat in a natural and
operationally significant way. For’ example, in working on the System
Dynamics National Model, we have variously tried to isolate particular
financial or real mechanisms underlying economic behavior. As a concrete
example, suppose we hypothesize that limits on the availability of
short-term debt from the comercial banking system are an indispensible
pert of producing a particular fluctuating mode. In testing this kind of
hypothesis, we have at times neutralized the availability of credit from
261
D-3323 16
B/i7/e1
the banking system, so that eligible loan requests are always met with no
limits from supply. But if supply of credit is unconstrained, then debt
may tend to fluctuate over larger ranges, with one possitle cutcome being
that the control of debt variations is shifted from pure availatility
considerations, to limitations from permissible debt to asset ratios. ‘The
analyst must then decide whether the relaxation of constraints ca
“availability” of bank funds means either simply the elimination of supply
constraints, or whether it implies elimination of both supply constraints
and limitations from credit worthiness on the eligibility of loan requests.
Different ways of casting a particular behavioral hypothesis test may yield
different results, so it is important that the analyst ccnsider carefully
the alternative ways of implementing a given hypothesis test, ari the fll
dimensions of evaluating a hypothesis through a given channei of
Despite the attendant difficulties in formulating hypothesis tests
appropriately, I believe that it is extremely valuable to insist on
behavioral confirmation of all hypotheses regarding sources of surrrise
model behavior. I have on any number of occasions spent substantial tine
diagnosing an unexpected behavior pattern, eventually arriving et en
apparently satisfactory explanation, but failed to develop appropriate
confirmatory tests only to discover much later that my hypothesis did not
stand up as a convincing explanation of the results. In this respect, the
model builder suffers the same difficulty as the policy maker: until his
intuition is sufficiently honed through experience in the real systen or
with models of the real system, then careful analysis of model output ray
fail to distinguish true causes of behavior from concomitant syrptors. I
find that the short-term effort devoted to model-based verification or
262
17
8/17/81
rejection of behavioral hypotheses is well worth the time and can diminish
the likelihcod of long detours down particular research and policy
irections that eventually are perceived as misdirected.
‘The renainder of this paper is devoted to developing a preliminary
ist of specific tests that may be helpful in increasing the likelihood of
excountering and resolving surprise model behavior.
2.4 Identifying Symmetry of Policy Response
Cne test thet is extremely valuable in model testing for revealing
unanticipated tehavior is to evaluate the symmetry of model response to
changes in upward and downward directions. For example, if an analyst is
testing a production model with upward step functions in consumer orders,
hould equally test the response to downward steps, representing
declines in denand. As one example of where such testing proved valuable,
severel years ago I was working on a financial model of how corporate
policies for maxing investments, paying out cash and stock dividends, and
gricing nev issues, affected the firm's average cost of capital, and
thereby the attractiveness of new investments. I started out by isolating
the part of the model that represented the stock market and subjecting it
to different external inputs from the standpoint of that module. These
ented, for example, assumptions about alternative patterns of
puts repres
growth in earnings per share, and investor confidence throughout the
economy. As one test, I started the initial value of stock price above
what I knew to te its equilibrium value, ran the model, and observed a
rapid decline tack to the equilibrium price, as was expected. It was only
D-3323 18
eft T/at
substantially later, that I performed the opposite test, of starting the
initial stock price below its equilibrium value. The model exhibited an
unexpectedly slow response over five to six years through which stock price
rose back toward ite equilibrium value. The eventual source of this slow
response turned out to be a formulation for “speculative risk,” through
vhich investors vere assumed to evaluate whether or not a stock price had
‘been driven to a value that could not be sustained on the basis of
financial realities ouch as grovth in earnings per share, but which
formulation was- not sufficiently robust to input conditions. In
particular, speculative risk in this early model version was formulated as
a simple function of the perceived growth in the stock price in relation to
the growth in earnings per share. Certainly, in an equilibrium situation
of stable earnings growth, stock price should grow at the sane rate as
earnings per share and consequently cash dividend per share. But in a
disequilibrium situation that is not characterized by stable growth, faster
growth in share price than in earnings does not necessarily indicate
speculation. For example, as encountered in the test when the price per
share was started below its equilibrium value, the total stock yield,
meaning both cash dividend yield and expected capital gains, would exceed
its equilibrium value. Higher total yield, in turn, should drive up the
stock price. But in the faulty model formulation, increasing stock price
was being taken as a signal of speculation, which was tending to increase
the perceived risk associated with holding the company’s shares, and
thereby generating a negative feedback pressure to restrain the rise in
share price. The net result was a slow rise in stock price in which the
pressures of undervaluation were tending to drive price upward, and the
faulty perception of speculation was exerting downward pressure to
D-3523 19
8/17/81
compensate in part. Once the source of this behavior was understood, a
broader end more robust formulation for speculative risk was developed that
encompassed not only the appropriate steady-state response, but also the
response to disequilibrium pressures.
I include this detailed example here because in my experience it is
very easy to be caught in the trap. of subjecting a model to an
insufficiently narrow set of tests, to progressively “tune” the model to
fit the limited input circumstances, and thereby to miss potential defects
or behavioral insights that would be quickly revealed by a different set of
test conditions. (Several other of the tests described in this chapter
treat analogous problems of testing that is too limited.) Sometimes,
asyrnetric response to different directions of model input or initial
conditions may be defensible. To take a simple example, if utilization of
capacity (meaning, length of work week, number of work shifts, and
efficiency of utilization) is more easily reduced than it is increased,
then model response to a growing demand should be slower than to a falling
denand, starting from the same level of full utilization. Thus, exenining
symmetry of model response to upward and downward conditions may sometimes
help to illuminate asymmetric time constants or other important behavioral
mechanisms, besides revealing potential defects in model formulation.
2.5 Mesting Large Amplitude vs. Small Amplitude Response
A form of testing that can be used to precipitate surprise behavior
is ‘to examine model response to both large and small amplitude input
variations. Very frequently, different adjustment mechanisms may be
263
D-3323 20
8/17/81
involved in regulating large and small departure from nornal operating
conditions. Por example, consider an oligopolistie industry such as
automobile manufacture. Small excess inventories of eutonotiles ray lead
producers to curtail production in order to liquidate inventories, with
little force for concessions on price compared with list price, much less
nventories suck 28
to change list price. On the other hand, large excess
are now being encountered in the automobile industry may force toth price
and output responses. We thus see an example of emphasis on different
corrective mechanisms depending on the degree of disequilibriua.
Sometimes, tests of model response to large amplitude input
variations can reveal important nonlinearities wnose omission ca: lead to
implausible model behavior. Alternatively, model tests in response to
large amplitude input variations may make evident important behavioral
mechanions that have policy significance and that may have been obscuted
when model behavior was examined only over a nerrow input rerge. For
example, I have recently been working on model of banking internediation
activity in the context of a national financial syster. Wher. the model is
run with high and slowly growing rates of inflation, bank profits are seen
to rise along with the higher nominal loan demands and interest rates
produced by inflation. On the other hand, if government deficits
accelerate rapidly, leading to rapid increase in inflation, then bank
profitibility can be reduced substantially, even to zero, if a substantial
fraction of loan revenues are not indexed to changes in the cost of funds.
This important distinction between the effects on profitability of high
rates of inflation versus large increases in the rate of infletion was
always latent in the model results," but was not appreciate! ustil a test of
2
8/17/81
large anplitude response made the differing patterns of behavior evident to
the eye.
2.6 Testing Policies Entering at Different Points of the System
Another important principle in model testing is to evaluate a wide
range cf policies whose direct effect occurs at different points in the
system being modeled. For example, in testing the production sector of the
System Dynaniecs National Nodel, we have tried to evaluate response of pro-
duction, price, liquidity, and other variables to external (meaning exter-
tal to the sector) assumptions about consumer demand, level of interest
rates, svailability of short-term and long-term credit, delay in filling
vacencies, delivery time for capital goods, national productivity trends,
increase in labor costs, and other inputs,
ry moce builder is often tempted to "fine tune” model response to
2 particular set of input conditions because the resulting outputs yield a
tangible result that aatches real system behavior. But I believe such
testing is misdirected. In our experience in working on the National
Yodel, ve find that a more balanced testing approach of evaluating model
tehevior in light of various stimuli at different points of intervention is
much more likely to reveal flaws or suggest insights from surprise
behavior. Such emphasis is called for even if the primary objective of
rodel analysis lies in understanding response to a particular set of input
corditions, such as labor shortage or faltering productivity. Testing
model respcnse to alternative input circumstances can point up model
defects cr even highlight important mechanisms that bear on the primary
264
0-323 22
eA T/er
purpose. For example, a recent corporate model I have been developing
suggests particular marketing and pricing policies to achieve a better
customer mix and improve profitability. But the potential desirability of
these marketing and pricing policies became most evident when realistic
Mmitations were imposed on the expansion of primary capacity. Thus, model
teating should never be limited inordinately to the immediate area of the
model surrounding the point of primary issue concern.
2.7 Testing Different Patterns of Behavior
Many system dynamics models have the potential for generating more
than one basic pattern of behavior. Yor example, the System Dynamics
National Model can generate fluctuating modes ranging from the 3-7 year
business cycle to the 50-year long wave, as well as a separate mode of
sustained inflation fron monetization of government deficits. Similarly,
the stock market model mentioned earlier can exhibit both an internelly
generated stock market cycle as vell as patterns of long-term grovth or
decline in share price. The importance of multiple modes of behavior can
ve two-fold. "First, symptoms are easily confused between the separate
nodes, thereby complicating policy formulation; and moreover, different
policies may be appropriate for treating the separate modes. If model
testing is insufficiently broad, the modeler may not even be aware that a
model is capable of exhibiting separate patterns of basic behavior.
Because many system dynamics models can exhibit more than one basic
mode of behavior, it is important to adopt a balanced approach for testing
than can expose behavioral implications of the structures that underlie
D~3323 23
8/17/81
different modes. In addition, the model can be subject to external inpute
representing different modes of behavior. An example can help to
illustrate some of the pitfalls of overnarrow model testing. In the early
stages of developing the production sector of the National Model, we
devoted a great deal of tine to refining the model structure relative to
the three cyclical nodes that the model was exhibiting--the business cycle,
the intermediate Kuanets or construction cycle, and the 50-year long wave.
However, upon subjecting the production sector model to external growth in
demand, representing a secular trend as opposed to a fluctuation around a
constant average value, whole new problems were disclosed. One formulation
that appeared defective and thus in need of improvement involved the
ordering function for capital equipment. The original model formulation
assumed that capital ordering depended first of all on the replacement of
depreciating capital; second, on a correction for the existing stock of
cepital and for the outstanding amount of capital on order; and third, on a
grovth term to increase the capital stock in line with expected future
growth in sales. ‘The formulation thus recognized that in order to maintain
a “neutral” growth path in the face of growing final demand and with no
ongoing shortages of capital equipment, capital must be expanded at the
rate of growth of demand even though at each point in time, no discrepancy
exists between desired and actual capital stocks. However, when the model
wes run with growing demand, a shortage of capital arose, manifested in
above normal delivery delays, lower than average inventory coverage, and
relatively high price end profit margin. Analysis of these biases avay
fron @ neutral steady-state growth path eventually revealed that the order
rate function needed to be extended to provide for expansion in the number
of capital units on order in line with the rising final output demand, as
265
D-3323 24
e/1 7/8
well as expanding capital stock along with growing demand. This
formulation defect might have been identified through careful enough
scrutiny of other model-generated modes, such as the large excursions in
capital ordering that took place over the course of a long wave.
Nonetheless, the point remains: behavioral implications of different
structures embodied in a model may be best revealed through @ broad testing
approach that attempts to isolate different behavior patterns and thereby
bring to the foreground structural problems that may be latent but
undiscovered in other modes of model behavior. Such a testing approach can
also contribute to policy analysis by revealing the relative efficecy of a
given policy under different circumstances and in different zodes of
behavior,
2.8 Evaluate Both Real and Nominal Changes
A number of system dynamics models have now been built that treat
money and financial markets, spanning from issues of corporate pricing end
financial policy to evaluation of policies for controlling national
inflation. These models extend the thrust of earlier system dyrenics
models, such as the models underlying Urban Dynamics and World Pynanice
that emphasized only real or physical changes, to treat moverents in rea?
activity, as well as nominal changes in prices’ and financial variables.
Where both real and nominal changes are combined in the model, it is
important to isolate the behavior resulting from each set of processes. An
example may help to illustrate the process and the kinds of results that
may emerge. In developing the National Model, we have attempted to follow
25
8/17/81
an increnental approach of successively activating various model sectors
and processes. At one stage, we were working with a combination of.
Froduction sectors, a household sector for savings and consumption
decisions, and 2 labor network connecting the production and household
sectors. The model was being tested with active price and wage equations.
However, in order to simplify the analysis conditions, we neutralized
effects of money flows between production and household sectors by
mainta:
ng neutral adequacy of liquidity in each sector; by neutral
adequacy of liquidity, we mean that through implicit borrowing and
repaynent c debt from the finahcial system, each sector is able to
ent liguidity to support its desired expenditure rates
dictated ty need for output, relative prices and wages, balance of factors
of production, and other similar factors. In ‘this configuration, we found
that real variables tended to exhibit the same basic cyclical modes that
ad been seen in analyzing the physical system alone. However, wages and
Frices showed a strong tendency to drift together, either in the upward or
downward direction and seeningly unpredictably as to what direction of
adrift would result in a given computer output. After careful study, we
concluded that auch behavior was in fact both necessary and appropriate
given the nodel configuration being assumed. With an implicitly unlimited
supply of credit, money supply in the model economy was likewise unlimited
and indeterminate. But if money supply does not have a determinate steady
state, then neither do absolute wage and price levels. In other words,
suppose thet from an initial equilibrium point, all prices, wages, and
Roney stocks were doubled. Then producers would be indifferent to the
gher absolute wage and prices levels, since their relative profitability
would be the same, and since there would have been no shift in the relative
D-3323 26
BATA
price/wage ratios that dictate factor intensities. Likewise, the household
would be able to support the same real flows of purchases, albeit at higher
prices, given twice the money level and twice the wage income and dividend
stream from the production sectors. In fact, such behavior relates
directly to the monetarist theories of inflation which argue that the
absolute level of money eupply is essential in determining the absolute
level of prices, as distinguished from the behavior of relative prices
(such as relative price-wage ratio).
In this instance, then, the appearance of surprise model behavior
did not suggest a flaw in underlying model formulation, but rather
suggested a policy insight: that control of money supply has an inportant
impact on the absolute price level, and therefore on inflation rate.
Moreover, the model results provided a vantage point for relating to an
economic literature covering both theoretical and policy issues that were
of obvious relevance to the project, but whose connection had never
previously been so clear.
In summary, then, it is important that models that incorporate both
real and nominal processes of change be analyzed so as to isclate the
relative behavioral contributions of each. For example, an important issue
is to understand the extent to which real and nominal changes are either
separable or intrinsically connected in a given mode of behavior. Such
evaluation can be performed by a variety of experiments that control the
environment surrounding nominal changes: for example, prices and wage
levels can be held constant; availability of credit from a financial
sector supplying money capital can be held neutral so that all eligible
D332) 27
28? a/17/8t
demands are met; adequacy of money or liquidity in various sectors can be
neutralized, and so’on. In order for systen dynamics models of economic
and financial activity to exert a significant impact on the literature and
on public policy, it is important that they add substantively to
understanding of how real and nominal changes in the economy are related
and how those processes contribute to problens such as persistent
inflation, high interest rates, and periodic credit shortages
2.9 Isolating Uniqueness of Equilibrium/Steady State
+ Although most system dynamice modele are designed to understand
disequilibrium or transient behavior of a system, equilibrium analysis of
nodel properties can still be revealing. For example, it is a common
tecknique to initialize model in equilibrium, and then perturb the system
through controlled exogenous inputs to understand the transient properties
such as periodicity, frequency response, and damping ratio. As an
additional example, in the Urban Dynamics book, Forrester uses a form of
comparative equilibrium analysis of computer simulations to see how
proposed policies for urban revival affect the long-run equilibrium of the
city, meaning population densities, population mix, unemployment rate, and
similar indicators.
Although a model such as the Urban Dynamics model is fairly complex,
containing more than 25 state variables, it has an important property that
bears on both model testing and policy evaluation: the final model
equilitriun for a given set of policies, constants, and exogenous inputs,
is independent of the initial values of level variables such as numbers of
267
D-3323 28
. e/17/81
business firms 4nd population levels. For example, Forrester develops @
particular set of urban revival policies by applying the revival policies
to a city that has already reached an equilibrium characterized by high
unemployment rate. He then shows that the same policies could also be
applied to a young and growing city, with identical long-run consequences:
Policies used for reviving a decayed area should, if
continuously applied, prevent decay. With rare and very
special exceptions, the ultimate equilibrium in a syster
does not depend on the systen's history. It depends
only on those policies and system parancters that ect
during the period when equilibrium is being establisted.
This means that the revival policies....could be applied
to a city throughout its growth period and shoul?
produce the same final equilibrium conditions as they do
when applied to a stagnant city... New and more
satisfactory urban-development policies can be initieted
at any point in the growth-maturity-stegnation cycle.
Transient conditions will be affected, tut the final
equilibrium depends on the policies thenselves and not
the initial conditions at the time the policies are
implenented.*
" As Forrester asserts in the above quotation, the majority of state
determined systems, of which system dynamics models are a subset, have the
dual properties that:.
a) model equilibrium is independent of initial conditions for given
policies and exogenous inputs (including constants); and
b) for a given set of initial conditions, model equilibrium is
independent of the time paths of the exogenous veriables before those
exogenous variables take on eventual constant values under which a system
equilibrium is reached.
¥ ‘J.W. Forrester, Urban Dynamics, (Cambridge: MIT Press,
1969), pg. 106.
D-3323 29
8/17/81
Although the above conditions characterize most system dynamics
models, there are models end systems that have more complex properties.
For exanple, consider a system consisting of a flat table surface and a
marble that is placed on the table. If the table is truly flet, then the
marble can cone to rest at an unlimited number of equilibrium points, each
identical to the initial conditions. Although systems with multiple
equilibrie exist, and models of these systems can be.usefully developed,
the models will heve unique properties. In my experience, it is a common
hagard for model builders to produce a model that exhibits a non-unique
equilibriun without appreciating the unusual nature of the results, and
erefore failing to question whether the results stem from a defect in the
nodei that fails to capture the pressures characterizing the real system
equilibrium, or whether in fact the real system has unique properties that
have important implications for policy design.
4g cne example of surprise model behavior that raised important
issues about equilibrium properties, several months ago I vas show the
betavior of a fairly simple teaching model that was designed to give
students exercise in model formulation and analysis. The subject of the
nodel was addict-related crime in an urban neighborhood. The model showed
the surprising result with apparently significant policy implications, that
an increase in rolice effort to control incoming drug supply, represented
in the mode! as a step function in an exogenous level of police effort,
yielded fever addict-related crines in the short run, but a sustained
higher crime rate in the long run. As I have tried to argue throughout
this paper, the appearance of such surprise behavior calls for careful
scrutiny of whether the surprise behavior reveals model flaws or
268
D-3323 30
B/NT/e1
alternatively policy insights. In exploring the reasons for the unexpected
policy result, we examined another model simulation in which a temporary
surge in police effort was represented, characterized in the model as 2
atep-up in police effort at one point in time, followed by a step down to
the originel level of police effort sonetine later in the same simulation.
We discovered that the level of crime did not return to its original
equilibrium value (an effort was made to confim that the initial condition
wes indeed a sustainable equilibrium), even though all exogenous inputs
including police effort returned to their initial values. The question of
whether such results correspond to real life is equivocal, and probably
cannot be enowered on purely logical grounds. But such behavior certainly
raises important issues from the point of view of model analysis and policy
evaluation. or example, if model equilibrium is dependent on the time
paths of exogenous variables such as police effort, then it is probably not
possible in principle to say that & higher eventual level of police effort
either raises or lowers the crime level: the outcome may be sensitive to
the exact time path of police effort before reaching the final higher
level, to the initial values of systems levels, and to the initial extent
of disequilibriun. An analytical study of this simple crime model using
basic algebra revealed that an unlimited number of equilibrium points could
be reached as long as the ratio of drug supply to addict population
attained a particular value. Thus, for example, there were no forces in
the model that limited the fraction of the local population that was
susceptible to drug addiction: the fraction could settle anywhere from
zero to 100% with equanimity. My point is not to argue definitively
whether these particular outcomes are realistic or not, but to emphasize
the importance of the underlying issues that they raise for model testing
D-3323 3
8/17/81
and policy analysis. In other words, it is possible that surprising: and
seemingly provocative results about the effects of a policy on the
direction of change in key system indicators may be attributable to defects
in the model that distort the equilibrium outcomes.
On the other side of the issue, a number of system dynamics models
have exhibited more than one equilibrium or steady-state set of conditions
under circumstances that seem potentially defensible and significant for
policy. ¥or example, an unpublished Ph.D. dissertation by William Shaffer*
that was done at MIT developed a model of crime rate in the atate of
Massachusetts and its relationship to deterrence measures in the form of
police effort and eventual length of prison sentence. Under normal ranges,
the model exhibited stable and well-bounded behavior. However, the model
was also capable of exhibiting @ very different mode in which prison
capacity was significantly overloaded, and thereby rising crime rates had
the potential for triggering a strong positive feedback loop that yielded
exponential growth in crime: more crime led to additional arrests and
additional court sentences; but in order to accommodate new prisoners in
jail, average length of prison stay for existing prisoners had to be
reduced; thus, turnover rate of prisoners increased and resulting lower
average prison sentence reduced the deterrent effect of the prison system
on crime rate, leading to further escalation in crime. While the
deterrence theory underlying this particular model may be questioned on a
variety of empirical grounds, the model nonetheless has important
= haffer, William A. “Court Management and the Massachusetta
Criminal Justice System,” Ph.D. dissertation, Alfred P. Sloan School of
Management, M.I.T., 1976
269
D-3523 32
B/iT/8t
properties. Under the overload conditions on prison capacity described
earlier, the model probably exhibits exponential growth in crime away from
an unstable equilibrium point. Thus, the model may have both a “normal”
stable equilibrium point, as well as an unstable equilibrium point. The
possibility of growth in an undesirable socio-economic veriecte, such a3
crime rate, away from an unstable equilibrium point certainly hes potentie?
policy relevance.
In summary, the issue of uniqueness of equilibriuz/steady state in a
system dynamics model is an important one for the analyst to evaluate. A
model that exhibits multiple equilibria may be the result of insufficient
structuring of the social and economic pressures that produce en
equilibrium in real life, or alternatively, may reflect cn the real nature
of the underlying system. The whole subject of multiple equilibria in a
system dynamics model (and indeed in other types of models) has barely been
touched in the literature and needs further study. For the rreeent, I
would argue that the appearance of model behavior characterized ty multiple
equilibria is an important departure from the vast majority of nedels with
a determinate equilibrium point, and should lead ‘the model builder to
serious and skeptical evaluation of model plausibility.
2.10 Understanding Forces Producing Equilibrium Positions
A related issue to that discussed in the previous section involves
the forces that produce the one or more equilibrium positions thet e system
dynamics model of an actual system may exhibit. In generel terms, an
equilibrium point can be neutral and pressure-free, or alternatively, it
12-3523 33
8/17/81
can be produced by offsetting pressures. To see the difference between
these two categories, consider a firm that utilizes labor and capital as
factors of production to produce an output stream. Moreover, suppose for
simplicity of exposition, that annual wage costs and capital charge rate
for capital equipment remain constant, so that the optimal intensity of
labor and capital in the production process does not change over time. A
“neutral” or pressure-free equilibrium would be one in which the production
sector replaces workers who quit or retire, and invests sufficiently to
offset depreciation of capital equipment, but does not confront pressures
to add or sv
tract lebor or capital equipment due to shortage or excess of
cutput or high or low marginal productivity of one factor in relationship
to the o'
In other vords, if the sector has just sufficient output
capacity to meet denend and maintain appropriate levels of output inventory
and order backlog, and if each production factor is in the appropriate mix,
en neither cutput pressures nor relative productivity of factors of
production sould produce incentives to change labor or capital stock over
tine. If, starting from a neutral equilibrium point, the sector
experienced 1C¥ more demand for its end product, it would eventually come
to add 10% more labor and 10% more capital, thereby yielding 10% more
output; once this point were reached, output rate would again match demand,
and both labor and capital would have risen by the same percentage, so
relative productivities would remain unchanged.
On the cther hand, suppose that bottlenecks in the supply system for
capital equipnent prevent expansion of capital goods. If demand for the
output of a firm that required capital goods for production were to expand,
that firm could augment production only through adding labor or through
270
D-3523 2 ies
increased utilization, such as longer work week. Suppose, then, that final
output demand went up by 10% and that the higher demand could be met with
fixed capital stock through a 15% increase in employment and utilization.
The new equilibrium that was thereby reached would no longer be
pressure-free. Inetend, acquisition of capital goods would be encouraged
by @ long output delivery delay and by a high marginal productivity of
capital equipment (meaning that it would be efficient to add capital in
relation to the outstanding number of employees); but capital acquisition
would be restrained by insufficient supply of capital goods. On the other
side, acquisition of labor would be discouraged by low marginal
productivity, but encouraged by above-normal delivery delays resulting from
unavailability of capital plant to supplement production capacity. ‘Tus,
the new equilibrium for each factor of production would be characterized by
@ balance between pressures to expand and forces to contract (or
Limitations on expansion) of that factor. Such occurence of offsetting -
equal but opposite pressures is what I mean by a non-neutral equilibrium
point.
Very frequently, evaluation of the forces producing a model-
generated position, be it neutral or non-neutral, can provide insight into
the adequacy of the model or into the forces determining equilibriur velues
in real life. A good example of such analysis of equilibrium position is
contained in a Ph.D. dissertation done at MIT several years ago by Barry
Richmond on forces underlying the long-term expansion of government
employment and expenditures in the United States.* As one thread of
7 Richmond, Barry M- "Government Growth in a Fixed Economy,”
unpublished Ph.D. dissertation, Alfred P. Sloan School of Management,
MAT.T., 1979.
D-3325, 35
8/17/81
argument in developing a theory of government growth, Richmond considers
the prevalent argument that government expansion is the result of
successive criges or incidents that create a temporary need for additional
governrent intervention, but with the long-term result that government size
‘271
‘D-3323 36
e/17/et
only grows over time, failing to fell significantly following the periods
of increased intervention. This hypothesis of government expansion is
sonetimes called a "racheting" theory, with the term “rachet" connoting a
progressive, stair-step form of increase in activity over tine with growth
interrupted only by periodic intervals of level activity. Pichzond
eventually develops simulation results to suggest that the rachet
hypothesis does not provide a tenable explanation for government growth.
The essential counterargument is that once the pressures producing greater
apparent need for government intervention wane following a crisis, unless
basic social values of the society have changed, then governrent activity
would decline back to its original relationship to private output, although
possibly with a very long downward adjustment tine.
In a similar vein, in ongoing work in the National Model Project on
causes of inflation and public policies to control inflation, we have
argued that various “cost-push" theories cf inflation do not provide @
plausible theory of ongoing inflation in the absence of conconitent
increase in money supply. Without increase in money and liquidity, rising
price and wage levels produced from cost-push strains would eventually
depress liquidity sufficiently to yield counterpressures that exactly
offset the upward thrust on prices and wages due to the cost-push force.*
In summary then, evaluation of the balance of forces producing a
model~generated equilibrium can be a powerful tool for evalusting nodel
ai See Mass, Nathaniel J. “Cost-Push Inflation and the Politics of
Monetary Expansion," Large-Scale Systems, Yol. 1, No. 2, pp. 107-115,
Vorth-Holland Publishing Company, Ansterdem, The Netherlands, 1¢S0.
(D-3098-1). Also Mass, Nathaniel J., "Monetary Sources of Inflation,”
System Dynamics Group Working Paper D-3254, February 19St.
BABE
37
8/17/81
adequacy and yielding policy suggestions. If, for example, a model
generates a non-neutral equilibrium, the analyst should carefully consider
whether the balance of opposing forces observed in the model would be
likely to occur in real life, or whether in fact, mechanisms have been
cnitted from the model that vould help to restore a neutral equilibrium
fron the standpoint of all impinging forces. On the other hand, if
opposing forces can reasonably sustain an equilibrium, then appreciation of
the nature of the balancing process may yield insights into the controlling
eechanisns that may either act in concert with, or in opposition to, policy
tiatives tried within the system.
272
0-323 38
e/T/8:
3. CONCLUSIONS
As argued in the introduction to thie paper, the very structural
richness of a system dynamics model yields a certain degree of a priori
unpredictability of model output. Certainly, as a model builder or
decision maker working with a model become more familiar with the inner
vorkings of the system being represented, the incidence of unexpected
behavior may diminish somewhat. But a variety of experience suggests that
some degree of unpredictability alwaye remains. From this point of view,
the experienced systen dynamics model builder may indeed be more capable
than the novice in anticipating behavior of a complex feedback model. But
to an even greater extent, he may becone nore effective end creative in
utilizing surprise behavior as a tool for diagnosing difficulties in tasic
model concept and in developing policy recommendations. There is 20
extensive practical literature that advises the model builder on guidelines
for improving models, and even more so, on guidelines for evolving policy
insights from a model. Thus, for the foreseeable future, both of these
skills are likely to revolve uround a high degree of art coupled with
experience and good judgment. In this vein, the basic thesis of this paper
is that appearance of surprise model behavior provides one of the events
that can precipitate fruitful improvement and application of system
dynamics models.
CC BY-NC-SA 4.0