Graham, Alan K., "Parameter Formulation and Estimation in System Dynamics Model", 1976

~ sh =

PARAMETER FORMULATION AND

ESTIMATION IN

SYSTEM DYNAMICS MODELS
Bs
by
Alan K. Graham
System Dynamics Group
Massachusetts Institute of Technology con
ABSTRACT
ur.

The purpose of this paper is to convey the techniques and considerations

normally involved in formulating and estimating parameters in system dynamics
models. Ideally, model equations should be formulated so that the associated
parameters each describe some unique observable characteristic of the real
system. Thereby, translating observations and measurements below the level
of aggregation of model structure (estimation from disaggregate data) into
specific parameter values becomes very straightforward. Fewer assumptions
about the structure of the system are needed than if the parameters were set
by equation estimation or model estimation from data at the level of aggrega-
tion of model structure. Making additional assumptions provides more oppor- Wve
tunities for systematic errors to creep into the parameter-setting process.
Rather than using data at or above the level of aggregation of model structure
to set parameters, such information might better be reserved for validity test-
ing. When such data are not already used to set parameter values, the validity
tests become simpler and depend upon fewer assumptions.

Parameters need only be set accurately enough to allow the model to ful- VI.
fill its purpose. One time-saving research strategy is to determine, by us-
ing only roughly-set parameters at first, how accurately the parameters must
be set before investing time and effort in setting them accurately. Then,
sensitivity testing can identify the relatively small number of parameters
whose values significantly alter the model behavior or response to policy
changes. The model can then be reformulated, the policies redesigned, or
the sensitive parameters reset by more elaborate and hopefully more accurate
techniques.

vir.

~ 52 -

TABLE OF CONTENTS

INTRODUCTION

‘A. Purpose and Organization
B. A Houping Model

CONSIDERATIONS IN PARAMETER FORMULATION

A. Realism
B, Effective Data Utilization

ESTIMATION FROM DISAGGREGATE DATA

A. Overview of the Three Cateogries of Parameter Estimation
Techniques

B. Estimating Between Limits

€, Estimating Table Functions with Extreme Conditions,
Normal Points, and Smooth Curvature

D. Calculating a Parameter Estimate from Disaggregate Data

EQUATION ESTIMATION

A, Estimating a Normal Fractional Rate of Flow
B. Estimating a Conversion Factor

MODEL ESTIMATION
PLANNING PARAMETER ESTIMATION EFFORTS
A, Strategies

B. Sensitivity Testing

C. Dealing with Sensitive Parameters

CONCLUSION

BIBLIOGRAPHY
~ 53 =

I. INTRODUCTION

System dynamics as a discipline diverges in several respects from more
traditional scientific disciplies, such as economics or physics. The most
apparent difference concerns the methods of selecting numerical values for
model parameters. In economics or experimental physics, a significant por-
tion of the total research effort is devoted to determining the precise
values of the parameters that characterize the system under study. A sig-
nificant part of professional communications in journals and conferences
concerns measurement, data, and statistical technique. In contrast, the
literature of system dynamics describes the complex structure of models,
and devotes considerable apace to analyzing the behavioral consequences of
that structure. Description of the parameter-setting process is usually
brief or nonexistent. It should not be surprising that practitioners of
traditional disciplines incorrectly perceive glaring deficiencies in system

dynamics models. Careful and laborious parameter setting, a part of research

long presumed necessary, appears totally lacking in system dynamics model

A. Purpose and Organization
The purpose of this paper is to convey the considerations and techniques
used to formulate and estimate parameters in system dynamics models. Section
I begins by discussing issues in formulation of equations and their associ~
ated parameters. The issues revolve around a parameter's dual purpose of
accurately describing some real process, and lending itself to straightforward
estimation. Estimation 1s divided into three broad categories, described

respectively in Sections ITI, IV, and V: estimation from disaggregated data,

~ Shh ~

equation estimation, and model estimation. Section VI describes the procedures
by which an initial model with roughly-estimated parameters is transformed into
a reliable guide to policy-making: sensitivity testing isolates the parameters
that require reformulation, reestimation, or policy redesign. Finally, Section

VII summarizes the main points of the paper.

B. A Housing, Model

This paper discusses various means of selecting parameter values in the
context of @ small model of an urban housing stock. Although very simple, the
nodel illustrates most of the issues and problems which accompany parameter
selection in moré complex system dynamics models. Each parameter in a properly-
formulated system dynamics model corresponds to some real process or processes.
To set a parameter in a aystem dynamics model is therefore to characterize or
describe some process with a numerical value, (Section III gives examples.)
‘The problems and issues entailed in such a characterization are virtually the
same, regardless of how many other processes also must be characterized——that
is, regardless of the size of the model.

The model describes the aggregate structure of an urban housing market,
and is designed to trace the broad history of housing growth and stabilization
in a central-city area. Figure 1 shows a DYNAMO flow diagram of the model.+

‘The level represents the total number of housing units H within a specified

Jphe model leaves a large number of factors implicit within the formulation.
For example, the model assumes that enough economic development occurs
close to the residential area being modeled so that jobs will be available
to support occupants of the housing units. For a more explicit treatment
of a housing market within the context of an urban economy, see Forrester
1969 (Appendix A), Goodman 1974b (Exerciae 12), or Alfeld and Graham 1976
(Chapters 6 and 7).
= 5h5 =

AREA
LAND
' PER HOUSE
HOUSING-LAND ‘ LPH
MULTIPLIER °
7
7
Housine
UNIT
UFETIME
AL

HO
Housing
DEHO-
ution

Figure 1, DYNAMO flow diagram

urban area. The rates are housing construction HC and housing demolition HD,
both of which are measured in housing units per year. The equation that spect~
fies the rate of housing demolition ND assumes some constant average housing
unit lifetime HL. Sections III and IV deseribe the equations for this model
in the context of estimating their associated parameters, with the exception

of the level equation:

He R=Hs JECDT) CHC JK HD SID tek
‘ tele N
IN=14000 1e2y 6

u ~ HOUSING UNITS (UNITS)

uc ~ HOUSING CONSTRUCTION CUNITS/YEAR)

Hn HOUSING BEMOLITION (UNTTS/YEAR)

HN ~ HOUSING INITIAL (UNITS)

= 5h6 -

It CONSIDERATIONS IN PARAMETER
FORMULATION

Much of the effort in system dynamics modeling is devoted to development
of the appropriate equation formulations and their associated parameters.
Only careful formulation can create parameters whose values can be set rela-
tively straightforwardly. This section therefore discusses the considerations
that enter into parameter formulation, as a prerequisite to the parameter-

setting techniques discussed in Sections II, IV, and V.

. AD Realism

A model 4s constructed to represent a set of real processes for a purpose.
The ultimate test of a model's validity, therefore, 1s whether the character-
istics of the representation agree closely enough with the characteristics of
the real processes to allow the model to fulfill its purpose. ‘The modeler can
set a very high standard for the realiam of a model structure by'requiring that
each equation in a model, and each parameter in each equation, correspond
simply and directly to some specific characteristic of the real. processes
being modeled.”

For an example of direct correspondence to real characteristics, consider

the equation for the rate of housing demolition HD in the simple housing model

in Figure

HI KLE 6 KHL: Qe
Qty &

~ HOUSING PEHOLITION CUNTTS/VEAR)

~ HOUSING ‘UNTTS (UNITS)

~ HOUSING UNTT LIFETINE (YEARS)

2See Forrester 1967, pp. 63-64, for a discussion of this subject in the context
of managerial models.
= out =

‘The equation defines the annual rate at which houses are demolished, which ts
quite directly observable. HD is defined in terms of the number of housing
units H in the area being modeled, and the average housing unit lifetime HL
(the average age of housing units when they are demolished). Both quantities
correspond to observable characteristics of the real system.

While an equation and its associated parameters may provide a good descrip-
tion of the real system, it may not be the best description. To avoid becoming
fixated upon one set of parameters, the modeler must realize that, for the most

part, parameters only describe the real system; they have no direct structural

counterpart in the real system. For example, the housing-land multiplier table HLM

exists only as a set of model parameters. In the real system, people buy
and sell land, and sometimes erect buildings. HLMT merely describes those
processes, and other descriptions are possible.?

Striving for a general description is sometimes more desirable and easier
than creating a formulation that describes only a specific case. For example,
suppose a model of a large corporation requires equations that describe pric~
ing decisions. Is the price determined by supply and demand in a somewhat
competitive market, or is the price determined by a traditional mark-up above
costs in a somewhat oligopolistic market? The modeler could choose a pricing
equation that reflects one or the other hypothesis, but a pricing equation
capable of representing both hypotheses and every alternative in between (by

means of different parameter values) would be clearly superior.

3see Masa 1974a and Miller 1975 for different, more detailed descriptions of
the markets for urban land and housing.

= 5h ~

One symptom of a non-general formulation is the presence of parameters
whose values cannot be realistically set independent of the values of other
parameters. If each parameter truly describes some unique, individual, and
observable characteristic of some part of the system, then every combination
of parameter values should have some plausible analogue in a real system.

But it is quite possible to formulate a model in which only some combinations
of parameters have a realistic interpretation, while other combinations give
nonsensical results. As a relatively obvious example, a model of budget
allocation within a firm could have parameters that could be set to continually

allocate 200 persent instead of 100 percent of the firm's income.“

B. Effective Data Uttlization

Every parameter in a system dynamics model must be assigned a specific
numerical value before the system behavior can be simulated. Therefore, the
equation formulations and their associated parameters should not only provide
a realistic description of the real system, but should also facilitate param-
eter estimation. .
What equation and paramter formulation best facilitates parameter esti~

mation? The answer depends on the kinds of data to be used to estimate the

parameter value. If the model parameters are to be set on the basis of de~
tailed, firsthand observations, the model parameters should correspond simply
and directly to observable characteristics of the real processes being repre-

sented. On the other hand, if an abundance of aggregate statistical information

4y superficial cure for such difficulties 1s to make algebraic constraints (such
as allocating 100 percent of income) part of the model structure, either in ini-
tial computations or in the auxiliary equations. However, interdependent param
eters (especially parameters describing allocations) often indicate missing lev-
els or overaggregation. For example, a budget constraint 1s an aggregation of
the feedback structure that surrounds the level of cash possessed by a firm or a
household. (The feedback causes spending to increase when cash builds up and
spending to decrease when cash is short.) The budget constraint is a behavioral
consequence of that feedback structure.
~ 59 -

is to be used to estimate the value, the model equations should be formulated
to facilitate the necessary computations (even though such formulations may
not match the features of the real system very closely). These two types
of data will be called (in this paper) data below the level of aggregation
of model structure and data at the level of aggregation of model structure,
respectively.

Data below the level of aggregation of model structure are observations
and measurements of the processes whose aggregate is represented by a model
equation. For example, consider the processes {volved in housing demolition.
One can observe the processes of aging and obsolescence which gradually ren-
der a housing unit less and less habitable. One can observe the other pro-
cesses by which houses are destroyed such as fire or replacement by new con
struction in urban redevelopment. One can observe the details of the demoli-
tion of individual housing units, whose aggregate is represented in the model
by the rate of housing demolition HD. If such observations are to be used
to set model parameters, the parameters should directly correspond to observ-
able characteristics, such as the average housing unit lifetime HL at the time
of demolition.

‘The other kind of information that can be used to set model parameters
ie data at the level of aggregation of model structure. Such data closely

correspond to model variables. For example, a model variable might be the

Sthe form of equations suitable for statistical estimation must often utilize
relatively aggregate data, so that the equations usually do not depict the
detailed processes through which the independent variables influence the de~
pendent variable. Also, analytic tractability often restricts the form of
such equations to be linear, with only one parameter per independent variable
(even though in reality the independent variable may act upon the dependent
variable linearly or nonlinearly through a variety of channels).

~ 550 =

annual rate of housing demolition HD within an area, and the corresponding
data then would be the number of housing units destroyed each year. Both the
model variable and the associated data represent the aggregation of a number
of objects: apartments, condominiums, single-family wooden houses, old houses,
and so on. The model variable and the data also represent the effects of a
nunber of processes: obsolescence of facilities within housing units,
gradually-accumulating damage to the interior and structure of housing units,
declining rent levels, declining maintenance expenditures, and condemnation
proceedings, to name a few. The data and the model variable are therefore

on the same level of aggregation. How can the modeler infer parameter values
from data that correspond to model variables? The data alone do not suffice,
since they describe the behavior of model variables, but not the model param~
eters. The. modeler must also use a model equation or several equations to
compute parameter values from data on model variables. One difficulty with
data at the level of aggregation of model variables is that the computations
require the use of assumptions about one or more equations. Such assumptions
always constitute "more rope to hang yourself with." The more assumptions, the
more opportunities for error. In contrast, setting parameters from data below
the level of aggregation of model variables allows each parameter to be set
and judged independently, without computations based on the rest of the equa~

tion which it helps to specify.©

Stconometricians are aware of an analogous situation in estimating simultaneous-
equation models. Even though simultaneous-equation estimation methods theoret~
feally deliver greater accuracy than multiple applications of single-equation
methods, the simultaneous-equation methods are more sensitive to minor viola-
tions of assumptions (less robust) than single-equation methods. Similarly,
paramter estimation from data at the level of aggregation of model variables
fs less robust than parameter estimation from data below the level of agerega-
tion of model variables.
at iad

One potential hazard exists in using data below the level of aggregation
of model structure. The hazard lies in formulating a model structure and param-
eters that are aggregated to the point where one cannot relfably observe the
processes being characterized by the parameter values. For example, in the sim-
ple housing system, a variety of processes determine how long it takes the sys~
tem to make a transition between growth and equilibrium--incentives to con-
struct housing, supply and demand effects in the land market, and housing depre-
elation, for instance. In the model, a number of different parameters charac~
terize these diverse processes. An alternative formulation of the model might
have contained a single parameter that specified the time constant for the
transition from growth to equilibrium. Urban experts may very well be willing
to give estimates of such a quantity, but the number would be a conclusion or
opinion drawn from their mental models of how the system behaves, rather than
a report on direct observations of events in the city.

Another example of confusing observations with conclusions occurs in the
field of international trade, where experts needed to predict the time it would
take for the voluwe of trade to adjust to the Smithsonian currency realignment.
Junz and Rhomberg 1973 show that statistical estimates of this delay time dif-
fer from the expert opinions by about a factor of two. That a difference ex-

ists is not surprising. That the difference is only a factor of two ii

surprising. Consider: the experts were attempting to predict, on a
purely intuitive basis, the behavior resulting from a very high-order, non-
linear, multiple-loop feedback system, involving a wide diversity of processes,

including marketing, inventorying, production, hiring, financing, and

- 552 -

pricing.’ If one formulates an equation with parameters that characterize the
result of a complex set of interactions by a single number, then one mist of
course experience great difficulty in obtaining reliable expert opinion or
other data below the level of aggregation of model structure: the model
structure is aggregated well above the point where a person can reliably wit-

ness the workings of its components.°

Tahere is an epistomological difficulty associated with the distinction between
paraneters that describe system behavior and parameters that describe processes
within the system. ‘The difficulty 1s that ultimately, all that one ever ob-
serves is behavior. For example, the observed average lifetine of a housing
unit discussed above could be considered as the behavioral result of a more
detailed system of interactions anong rent levels, maintenance and capital
costs, population and income levels, social traditions about housing, and
many other variables. So regardless of how detailed one makes a model, in a
strict philosophical sense, one alvays has parameters that are descriptions
of the outcome of processes not explicitly represented. These descriptions
are adequate for the purposes of the model provided that the outcomes of the
processes not explicitly represented do not change significantly as a result
of the dynamics being modeled. For example, one could ask whether the aver-
age lifetime of a housing unit changes as the system makes the transition
from abundant land to scarce land.

Sthere are tuo possible courses of action when one has formulated a model whose
parameters are too aggregated to be set reliably from available data below the
level of aggregation of model structure. One course is to restructure (usually
disaggregate) the model so that its parameters do correspond directly to observ-
able unchanging characteristics of processes within the system. ‘The disaggre-
gation will usually involve not only subdivision of levels into more levels,
but also explicit addition of feedback loops that control the levels. For
example, Mass 1974a and Miller 1975 disaggregate the relationship between land
availability and urban housing construction discussed in Section 111.B of this
paper. The other course of action when a parameter is too aggregated to be set
reliably from available data below the level of aggregation of model structure
4s to use another estimation technique and data at the level of aggregation of
model structure--uaually statistical techniques. It seems unwise, hovever, to
attempt to estimate a simple relationship if the actual system is complex enough
to render expert opinion unreliable. In the foreign-trade example above, for
instance, the aggregated delay between exchange rates and trade volume could
(for some model purposes) be completely inadequate: numerous other variables
impinge on that part of the system, including, anong others, forward exchange
rates; availability of arbitrage capital; relative interest rates; expected ex-
change rates; availability of capital, labor, and financing; and transport costs.
For many model purposes, the total aggregate delay tine must be regarded as an
endogenously-determined dynamic variable, and not as a constant parameter.
= 553 -

The final consideration about effective data use concerns model validation.
After a model has been formulated and the parameter values set, validation tests
demonstrate whether or not the model's structure and behavior agree (are consis~
tent with) available data about the system being modeled. The more a model's
equations, parameter values, and behavior resemble the known characteristics of
the real system, the more valid the model.

If parameters are computed from data at the level of aggregation of model
structure, the model is to some extent forced to replicate the characteristics
of those data, These characteristics can range from phase and magnitude rela~
tionships between individual variables to the entire system's behavior mode.

But such a replication cannot increase confidence in the validity of the model.
‘The method of parameter estimation forces the model behavior and real behavior
to agree.

One possible strategy for data use would be to use aggregate numerical
data (data at the level of aggregation of model structure) to set parameter
values. The information left over for testing the validity of the model is
then the data below the level of aggregation of the model structure (that is,
observations of the details of the processes being modeled), plus whatever
aggregate information remains to be extracted by more elaborate statistical
tests. This strategy seems to provide many opportunities for error. It is
too easy to declare paraméter values reasonable and characteristic of the real
system after the fact, even when they may not be (especially since a model
whose parameters are set by statistical or numerical procedures may not have
paraenters that correspond directly to observable characteristics of the real
system; see footnote 5). Statistical testing procedures do not seem appropriate

for validity testing, both because of logical limitations (discussed in Mass

- 554 -

and Senge 1976), and because the tests are predicated upon many assumptions,
which when false can cause the tests to indicate good results spuriously (see
Senge 1975a).

Another strategy for data use, more commonly employed in system dynamics
models, is to reserve the more aggregate numerical information for validity
testing, and to set model parameters from data below the level of aggregation
of the model structure. This strategy maximizes the opportunities for unbiased
utilization of data below the level of aggregation of model structure. By
excluding the use of aggregated data from parameter setting, this strategy
also maximizes the amount of aggregate data that can legitimately be used to
test model validity, without resorting to complex and non-robust statistical
tests. -

To summarize this section, realistic equation formulation should provide
a recognizable yet general description of some real process, in which each
parameter describes some independent characteristic of the process being
modeled. Formulation at the proper level of aggregation can facilitate param~
eter setting and validity testing, by allowing parameters to be set from data
below the level of aggregation of model structure. This reduces the number of
assumptions made in parameter settings, and allows the more aggregate data to

be used in validity testing.
= 339 =

TIE, ESTIMATION FROM
DISAGGREGATE DATA

A. Overview of the Three Categories
of Parameter Estimation Techniques

Each of the next three sections describes one category of parameter setting
technique. (Several examples of each type are given.). In order to delineate
the differences between the types, we will discuss all the types in this
subsection.

‘This section describes estimations from disaggregate data or, more precise~
ly, data below the level of aggregation of model structure. As an example, the
text has already discussed how the average housing unit HL can be estimated by
observing the age of individual housing units when they are demolished. Esti~
mations from disaggregate data may or may not involve computation, but the com-
putations never involve the actual model equations.

Section IV describes equation estimation, which uses data at the level of
aggregation of model structure in a computation based on the equation that con-
tains the parameter being estimated. For example, assuming that Equation 2 de~
fining housing demolition HD 4 correct, dividing the number of housing unite
by the number of housing units in a year within an area yields an estimate of
average housing unit lifetime HL.

Section V describes model estimation, which uses data at the level of

aggregation of model structure in a computation based on the entire set of

- 556 -

model equations.” For example, one could simulate the housing model with var-
ious vaiues of the average housing unit lifetime HL until the model behavior

approximately matches observed historical behavior.

Bo Estimating Between Limits

There are several variations in the technique of estimation from disag-
gregate data; the example of estimating the average housing unit lifetime
HL is the simplest variety, where a single parameter is directly set equal to
an easily-measured quantitative characteristic of the real system. A slightly
more complex situation arises when not enough observations are available to
set a single unique value. Even so, the modeler can obtain a parameter esti-
mate by considering upper and lower limits, which the parameter values should
not approach.

For example, Equation 3 defines the rate of housing construction HC as
the product of the number of housing units H, the housing construction normal

HCN, and the housing-land multiplier HLM.

HC. KLe¢H+R) CHEN) CHLM 6K) Bek
HCN#0.07 Sele ©
He ~ HOUSING CONSTRUCTION (UNITS/YEAR)
H ~ HOUSING UNITS (UNITS)
HCN ~ HOUSING CONSTRUCTION NORMAL (FRACTION/YEAR >
HUM > HOUSING-LAND MULTIPLIER CLIMENSTOHLESS»

The number of housing units H indicates the size and inherent ability to en-

gender growth of the developing community or city being modeled. ‘The housing

9single-equation methods in econometrics (such as the family of least-squares
estimators) are equation estimations. The econometric full-information
maximum-likelihood (FIML) techniques and the control engineering full~
information maximum-Likelihood via optimal filtering (FIMLOF) techniques
are both model estimations. For descriptions of FIMLOF techniques, see
Peterson 1976, Peterson 1975, Peterson and Schweppe 1974, or Schweppe 1974,
Chapter 14.
construction normal HCN is the proportion of additional new houses a community
or city can build under some defined set of normal conditions. Under such
normal conditions, the third factor, the housing-land multiplier HIM, assumes
a value of 1.0, HIM represents the effects of deviations from the set of
normal conditions on construction by exceeding or falling below 1.0,.!°

What values are appropriate for housing construction normal HCN? One
set of answers comes from observations of houses being built and neighborhood
expansions. If normal conditions are defined to apply when the community can
still experience substantial growth, an HCN value of 0.01 is too small. Such
a value would imply that a community of 100 houses, despite the availability
of acceptable construction sites, would have only one more house built in it in
a year, which ds not substantial growth, At the other extreme, a value of HCN
of 1.0 is clearly too large. With such a value, every year, new housing units
would be constructed in numbers equal to the size of the housing stock at the
beginning of that year. Neighborhoods seldom, 1f ever, grow so rapidly. How
fast do neighborhoods grow? A realistic value of HCN must Lie somewhere be-
tween 0.01 and 1.0, Simply choosing a value for HCN somevhere between these
two extremes may suffice for the purpose of the model. Alternatively, the
nodeler may have to seek more precise information to further narrow the range
of possible parameter values. (Section V discusses methods of determining

how accurately parameters must be set.)

10rhe normal conditions chosen for this model are the conditions that occur
when the growing housing stock first occupies 80 percent of the land area
being modeled. "Normal" is used here in the scientific sense of normalized
quantities, rather than in the sense of either "typical" or "healthy."
Alfeld and Graham 1976 (Section 5.3) and Graham 1974 further discuss the
use of normal conditions in model formulations.

~ 558 ~

GC Estimating Table Functions
with Extreme Conditions,
Normal Points, and Smooth Curvature

Table functions seem to constitute a formidable estimation problem, since
they are typically specified by 5 to 15 numbers. But the problem can be broken

estimating the value and the slope of the function at one

into subproblem
extreme, at the normal value, and at the other extreme. The remaining sub-
problem is connecting the known values and slopes with a swooth curve.

For example, consider Equation 4, which specifies the housing-land multi-

plier HIM as a table function of the land fraction occupied LFO.

HLM RETABLE CHLNT 9 LFO Ky Or 12064) ara
HUMT=44/47/2/1 6 25/1 AG/L 65/1 65/1 4/1/8570 Aydy 1
HLM ~ HOUSING'-LAND MULTLPLIER CUMENS LONLESS >

HLMT — ~ HOUSING~
LFO | = LAND FRat

AND MULTIPLIER TABLE
TON OCCUPIED (DIMENSIONLESS >

Under the extreme condition of very low land occupancy, incentives for
construction should be appreciably lower than under normal conditions. When
the land fraction occupied LFO approaches zero (near the left side of the curve
in Figure 2), the area being modeled is mostly vacant land. The area's via~
bility as an urbanizing entity has not yet been demonstrated. Developers can~
not count on continuing demand for the housing units they construct. Many
services taken for granted in more heavily-settled areas must be installed
new neighborhood—by—new neighborhood: roads, sewers, electricity, gas, and
schools. Theseservices will by no means be complete in an area too sparsely~
settled to make even city water or sewers an economical proposition, let alone
public transportation. So the housing-land multiplier HIM should be well below
1.0 when LFO 4e 0.0; Figure 2 gives HLM a value of 0.4 when LFO equals 0.0

(Point A).
Se
2.0 1
1
er Ota i
christ
iN
3 1o re
at)
: oy
Al 1 im
Ne
00 tyre
00 o2 o4 06 08 10

LFO

Figure 2. Housing-land multiplier table

Adding housing units to a sparsely-settled area gradually makes more and
more urban services economical, paying propositions, thereby making housing
construction more attractive and more obviously profitable. But adding a
few houses cannot pay for the infr:

tructure--schools, roads, Libraries,

utilities--necessary to deliver a complete ensemble of urban services. The
curve for the housing-land multiplier table should slope upwards, but not
very ateeply from where LFO equals 0.0. (See the line segment between Points
A and B on Figure 2.)

Now consider the normal condition, defined as the condition that occurs
during the normal period, when the land fraction occupied LFO equals 0.8.
By definition, under normal conditions, the rate of housing construction HC
equals the product of housing units H, and housing construction normal HCN,
Therefore, when LFO equals 0.8, HIM must exert no influence on HC, and must

equal 1.0 (Point € on Figure 2).

“560%

Consider the other extreme condition in which the land fraction occupied
LFO equals 1.0. The land area within the community, city, or district being
modeled is fully and totally occupied. Bven the least desirable sites have
been built upon. Regardless of whatever incentives exist to construct hous~
ing, no housing can be constructed within the area being modeled until there
is some physical space available upon which to build--until LFO ceases to
equal 1.0, So the housing-land multiplier HLM should equal 0.0 when LFO
equals 1.0, which establishes Point E on the graph of the housing-land mul~
tiplier table HLMT in Figure 2.

If the land fraction occupied LFO was not 1.0 but close to 1.0 (nearly
full land occupancy), urban services such as sidewalks, schools, libraries,
roads, and public transportation would already be installed and fully devel-
oped. To be sure, the crowding and lack of desirable construction sites
implied by an LFO close to 1.0 would not permit housing construction to take
place so rapidly under the normal conditions. Nonetheless, any small reduc~
tion of LFO from 1.0 opens up the possibility of appreciable housing construc-
tion, Therefore, the curve of the housing-land multiplier should probably
be fairly steeply-sloped as LFO approaches 1.0, (See the line segment be-
tween Pointe D and E on Figure 2.)

So far we have estimated the values at and near two extreme conditions
and at the normal condition. Now all that remains is to draw a curve through
the estimated points. Any sharply bent or kinked curve is probably not very
realistic. A bend or kink implies something special and unique about the
exact conditions at which the bend or kink occurs. Since the housing-land
multiplier table HLMI represents a very large number of processes, the prob-

ability is vanishingly small that all of the processes represented would show
- 561 ~

major changes under a single unique set of conditions. Accordingly, the curve
for HLMT (and in general, all highly-aggregated relationships) should change
smoothly, without kinks or bends.

Solving the subproblems of extreme conditions, normal conditions, and con-
necting known points with smooth curves, allows the modeler to estimate a non~
linear table function with a high degree of confidence. The estimated table
summarizes observations of a large number of processes below the level of ag-
gregation of model structure. HLMT is the aggregate representation of these

processes and their effect on housing construction.*!

D Calculating a Parameter Estimate
from Disaggregate Numerical Data

The modeler can combine numerical estimates or observations of processes
below the level of aggregation of the model structure into values for model
parameters. For example, consider an equation in an ecological model which
specifies the birth rate of rabbits BR (measured in rabbits per month) as
the product of the total rabbit population RP and some constant function, the
rabbit birth fraction RBF:

Ro BR-KL = RP.RARBP
‘The average person may not seem to have enough information to specify a value
for RBF, but most people in fact know enough about the biological characterist-

ics of rabbits to specify at least an approximate value. Suppose that a mature

Mute can be considered to be the composite of two nonlinear functions, one of
which represents the simulating effect of infrastructure development on hous
ing construction. The other represents the inhibiting effect of low land
availability on housing construction. In general, if a curve becomes any
more complex than the hump-shaped WLMT curve, it should be broken into com-
ponents and its componente each estinated separately. Customarily, most mul~
tipliers in system dynamics wodels have a simple monotonic form.

~ 562 -

female rabbit litters about every 5 months, and that about 4 bables per litter

survive, Since about half the mature rabbit population is female, that makes

1 litter, 4 bables _,_1_ mature female
5 months 1 mature female 2 mature rabbits

= 0.4 babies / mature rabbit / month.

But not every rabbit 1s mature. If rabbits live about 4 years or 48 months,
and require about 6 months to mature, and if the rabbit population is evenly
distributed, then (48 - 6)/48, or 0.875 of the rabbit population will be mature.
If the rabbit population is growing, then there will be proportionately more
young rabbits in the population, which reduces the fraction of mature rabbits.
Assume that the tabbit population being modeled is growing rapidly. There~
fore, the fraction of mature rabbits should be Jess than 0.875, say around 0.5.
For the whole rabbit population, there are:

0.4 babies x 0:5 mature rabbits
mature rabbit—month rabbit

= 0.2 babies/month/rabbit
So, setting the rabbit birth fraction RBF equal to 0.2 should be fairly close
to the value that would be derived by direct observation.” (Senge 1975b dis-

cusses a more complex computation in a managerial model.)

12rhe parameter estimation uses only the author's impressions of the biological
characteristics of rabbits. The reader may wish to check the parameter value
derived above against more detailed observations and measurements. An ency~
clopedia should have information about the average longevity, maturation time,
gestation period, and litter aize and frequency of rabbits. ‘(Don't forget to
account for infant mortality when carrying out the computation, and make sure
the lifetimes and infant mortality rates apply to a growing rabbit population.)
~ 563 ~

IV. EQUATION ESTIMATION

Estimation from disaggregate data employs data below the level of aggre-
gation of model structure, and never uses a model equation to compute a param-
eter value. In contrast, equation estimation employs data at the level of
aggregation of model structure, and must alvays use a model equation to com
pute a parameter value.!? subsection ITI.A described estimating the average
housing unit lifetime HL by using the equation for housing demolition HD and
data on WD and housing units H, The following subsection describes a slightly

more complex example.

A, Eotimating
a Normal Fractional Rate of Flow
The format for many rate equations in system dynamics models is
Rate = Level * Normal Fraction * Multipliers

By simple algebra, the value for the normal Fraction is given by:

. Rate
Normal Fraction = [37a Haltipliers

Under the normal conditions (at whatever time period it is defined), the
multipliers, by definition, assume values of 1.0. So the normal fractional
flow rate can be computed by dividing the observed rate by the observed
level, both measured during the period of normal conditions. For example,

suppose that one defines the year 1960 as the normal period for the urban

Vonty in rare instances can the modeler use simple manipulation of the model

equation to estimate more than one parameter. Single~equation econometric
techniques routinely estimate many parameters simultaneously, with corre-
spondingly more stringent requirements for specification and data accuracy.

= 56h =

area being modeled. Then, if the data are available, one can divide the num
ber of housing units constructed in the area during 1960 by the number of
housing units in the area in 1960 to obtain a value for housing construction
normal HCN.

Equation estimation requires several assumptions--in this case, that the
data apply to the normal period, that the data are accurate, and that the
equation is accurate. These assumptions provide opportunities for errors.
One example occurred in an attempted revision of the Urban Dynamics model
(Forrester 1969) in Babcock 1970. Babcock attempted to set normal con-
stants using dgta on levels and rates of flow, but neglected to use data
only for,the normal period. He used data for cities near equilibrium also.
The simple housing model presented here can show what happened as a result.
The housing model réaches equilibrium after the housing stock grows until
a shortage of land suppresses further housing construction. Because the
normal conditions in the model are growth conditions, the housing-land
multiplier HLM must suppress housing construction by going well below 1.0.
Suppose we divided the actual rate of housing construction HC, by the actual
number of houses Hl, to obtain a computed value for the housing construction
normal HCN,. If the model equations are accurate, using equilibrium data
to compute HCN:

uC, +H *HCN
en, = —* MgMHON HEM HCN, AHL
* A a a a

which means that

HN, < RCN, .

Using the computed value of HCN in a model reduces the model's impetus to

grow, and thus reduces the extent to which HLM must drop to bring the model
= 565 -

into equilibrium. Similarly in Urban Dynamics, growth ceases when land short-
age and unfavorable internal conditions (principally a job shortage and pre~
dominance of lower-income groups) depress construction. Using data from near~
equilibrium to compute normal fractions considerably reduces the extent to
which internal conditions in the model must decline to halt growth. In fact,
the model will no longer reproduce and account for depressed urban conditions.
Babcock's modified Urban Dynamics model therefore no longer even fulfills its
purpose, merely because the implicit assumptions used in parameter setting

were violated.

B. Estimating a Conversion Factor

A large number of parameters are conversion factors, which ‘convert quan-
tities from one dimension to another. For example, land per house LPH con-
verte housing units to an equivalent number of acres. Equation 5 uses LPH

in the definition of land fraction occupied LFO:

LFO.K= CHS IOKLPH) /AREA SA
LPH=044 Gedy ©
AREA=9000 5.29 €
LEO ~ LAND FRACTION OCCUPIED (DIMENSIONLESS)
im - HOUSING UNITS (UNITS)

LPH ~ LAND PER HOUSE (ACRES/UNIT)
AREA =~ AREA (ACRES)

Equation 5 could be manipulated to compute LPH as a function of LFO, housing
units H, and AREA, The only difficulty with such a computation lies in mak-
ing sure that the definitions of the data used are appropriate for the model.
For example, land per house LPH must include not only the land directly be~
neath each housing unit, but also the associated land used for yards, eide~
walks, roads, garages, driveways, and schools and stores serving the. neigh~

borhood. The modeler might suppose that the land per house LPU for a

= 566 ~

particular area could be calculted from the land area zoned for residential
use (minus the atea of vacant lots), divided by the number of dwelling units
within the area. However, many cities have land that is zoned for both resi-
dential and commercial use; some fraction of that land must be included in the
residential land area as well. Once the definitional considerations have been
laid to rest, conversion factors are relatively straightforward. Schroeder
and Strongman 1974 deseribe the use of such procedures to adapt the Urban

Dynamics model (Forrester 1969) to a real city.

Ve. MODEL ESTIMATION

As just described, equation estimation consists of manipulating one model.
equation to compute a parameter value. In contrast, model estimation consists
of manipulating all of the model equations to compute a parameter value. For
example, the housing construction normal HCN could be estimated by finding the
value of HCN that causes housing growth to fit the observed rate of growth.

The fitting could either be performed with repeated simulations or (if possible)
by an ad hoc computation. For example, say that the stock of housing grew at
4.0 percent per year under normal conditions. Also suppose that, from observa-
tion of housing demolition, the housing unit Lifetime HL is estimated to be 66
years--that is, 1/66 of the houses are demolished each year. If the model
equations are assumed to be correct, then the housing construction normal HCN
must exceed 1/66 by 0.04 to produce the observed rate of growth during the
normal period. Therefore, HCN can be inferred to be 1/66 + 0.04 = 0.07. As

‘another example, suppose a real system exhibits fluctuations of some specific
~ 567 -

period. The modeler can choose the magnitudes of time constants of the system
80 as to produce oscillations near the real period. (Forrester 1968, Chapter
10, derives a simple rule of thumb: for a system with two time constants T,
and T,, their geometric average approximately equals the period divided by 2m:
Vet, & P/2n.)

One danger of model estimation is misattributing the observed behavior to
the value of a particular parameter. In the oscillation example just cited,
if T, is inaccurate, model estimation will compute an inaccurate value for t,
as well in order that Y,T, 3 P/20. As a subtler example, the Urban Dynamics
model was once being modified to match the historical growth and decline of

Lowell, Massachusetts. A period of rapid growth early in the city's hietory

was being modeled by altering a table function similar to the housing-land
had

multiplier HIM. The table, arrived at through repreated simulations
about the same values at the extremes and normal points as the curve in Fig-
ure 2, but Point B was well above 1.0, Although the altered curve allowed

the model to reproduce the historical behavior quite accurately, it no longer
constituted a realistic representation of the true cause-and-effect relation—
ships within the city.!4

The modeler can choose one parameter value over another merely because

it yields model behavior closer to real system behavior.- But such a technique

‘errors in model estimation are often detected by checking the results with
other data. In the urban example above, the table function was deemed in-
compatible with day-to-day observations on the process of industrial de-
velopment (which is data below the level of aggregation of model structure).
This use of two independent sete of data is equivalent in principle to the
Jong-standing econometric practice of estimating parameters with data from
one time period and evaluating the parameter estimates with data from
another time period.

= 568 -

presumes that the entire model structure 1s correct, which is equivalent to
making the maximum possible number of assumptions. ‘Then, the falsity of any
‘one of the assumptions can in principle cause serious problems. The modeler
might better avoid making chains of assumptions, where possible, by setting
parameter values from easily-observed characteristics of the processes being
modeled (data below the level of aggregation of model structure). A model

is more credible if each formulation and each parameter value stands inde—

pendently as ‘a plausible and realistic representation of a real process.”

VI. PLANNING PARAMETER ESTIMATION
EFFORTS

The preceding sections have discussed considerations in parameter formu~
lation (Section II) and a variety of techniques for estimating parameter val-
ues (Sections ITI, IV, and V). Those discussions cover the parameter-related
issues involved in arriving at an initial model, the accuracy of whose param-
eter values may or may not suffice to allow the model to fulfill its purpose.

What are the appropriate next steps?

anis 1s not to say that model estimation techniques cannot increase one's
confidence in a model. If one has a means of detecting errors in model
estimations, such estimations can be quite useful in formulating and vali-
dating a model. For example, Peterson 1975 used statistical consistency
checks and strong prior paraneter values to uncover flava in developing a
model of energy denand. After the flawe were corrected, confidence in the
model was mich increased vhen model estimation (which 1s rather sensitive
to specification problems) failed to indicate further problems. If, how-
ever, one does not have an independent means of checking the model estima
tion, the resulting parameter values seem highly Likely to contain system-
atic errors.
~ 569 -

A. Strategies

The modeler could devote considerable time and effort to estimating real-
istic and accurate parameter values, and defer further work on model testing
and refinement of formulations. Or, one could continue the development of
the model formulation through model testing, and defer the parameter-est imation
effort. (Forrester 1961, Chapter 13, and Mass and Senge 1976 further discuss
general model testing.) What strategy best allows the model to fulfill its
purpose? Most system dynamics models do not have the purpose of precise nu-
merical prediction. Instead, they are usually aimed at replicating the causes
of an undesirable behavior mode, and investigating policies that diminish or
eliminate the undesirable behavior. 16

Such a purpose allows the system dynamics model to capitalize upon a
remarkable fact: system dynamics models usually represent nonlinear, high-
order, multi-Loop feedback systems, whose responses infrequently show sensi-

7 system dynamicists capitalize on this

tivity to a parameter variation.?
fact by constructing models using very rough, very quick parameter estimates.
‘The completed model 1tself can then serve to assess the wodel’s need for
accurate parameter values: by testing model behavior when paraneters are

changed, the modeler sees whether or not altering the parameter value after

Wrorrester 1961, pp. 123-128, describes this distinction in terms of predict-
ing a future system state versus predicting the system behavior.

phere appear to be four structural causes of parameter insensitivity. One,
minor negative feedback loops; they tend to compensate for parameter changes
within them. Britting and Trump 1975 further discuss this subject. Two,
structure outside dominant loops; usually, only a relatively amall number of
feedback loops (the dominant loops) produce the system behavior. Paraneters
that characterize processes not involved in any of the dominant loops cannot
have mich affect on behavior. ‘Three, redundancy; a feedback loop can have
several branches, so that parameter changes that inactivate one branch cannot
prevent the feedback from functioning. Four, numerical insignificance; for
example, doubling the time constant or a relatively short delay in a series
of delays does not significantly change the overall response time.

= So =

a laborious redetermination of the value could possibly have an effect on the
outcome.

Usually, only a few parameter values significantly influence the outcome.
Those parameters alone warrant further effort in formulation and estimation.
The initial rough estimates of most of the parameters are accurate enough for
the purpose of the model. The following subsection discusses the simulation

tests that distinguish the sensitive from the insensitive parameters.

B. Sensitivity Testing

Uncertainties and inaccuracies in parameter values may affect either the
model behavior ér the policy recommendations derived from the model. Testing
behavior sensitivity requires a comparison of two simulations: a reference
simulation, and a simulation with an altered parameter value.!® for an example
of behavior sensitivity, suppose that minor parameter variations cause the
housing model to exhibit several distinct modes of behavior. Perhaps the
behavior of real urban areas depends critically on the processes represented

by the sensitive parameters. If each of the several model-behavior modes

18, model can be subjected to two types of parameter variations. One type is

to evaluate model behavior or policy impact only over the range of values
that the parameter could realistically assume. For example, if housing
construction normal HCN could plausibly lie only between 0.04 and 0.4, then
0.04 and 0.4 would be the extreme values of HCN tested. The other type

of parameter variation is to raise or lower a parameter value progressively
to find the point at which the parameter variation substantially alters the
model behavior or policy impact. For example, lowering HCN far enough would
cause the rate of housing demolition HD to exceed the rate of housing con-
struction HC 60 that the number of housing units H would shrink instead of
grow. Both types of parameter variation are appropriate for either behavior
or policy sensitivity testing; the former variation gives more information
about the realism of a model or workability of a policy, and the latter
variation gives more information about the possible behavior modes of the
system.
a - 572 =

corresponds to a real situation or example, the parameter sensitivity builds
confidence that the model captures the essential features of the real system.
However, if the model exhibits a behavior sensitivity that does not correspond
to the behavior sensitivity of real urban areas, then the model requires care-
ful reexamination. The sensitive parameter indicates an area that requires
either reformulation or reestimation.

Parameter variation may also alter or reverse the impact of simulated
policy changes. The model user needs to know whether a policy yielding favor-
able results with one set of parametera can algo yield unfavorable results
with a different set of parameters. Susceptibility of policy results to
parameter changes i called policy sensitivity. Passing a policy-sensitivity
test builds confidence in the policy recommendation, while passing a behavior-
sensitivity test builds confidence in a model structure.

A policy-sensitivity test requires at least four simulations: a reference
simulation, a simulation of the policy change, a reference simulation with
an altered parameter value, and a policy simulation with the same altered
parameter value. Figure 3 shows the procedure for comparing simulations.
First, determine the impact of the policy change by comparing the reference
simulation and the policy simulation. Second, determine the impact of the
policy change under the conditions depicted by the altered parameter value:
compare the reference simulation with the altered parameter value to the policy
simulation with the altered parameter value. At this point, the modeler knows
the impact of the policy change upon the original model and upon the model with
an altered parameter value. Comparing the two policy impacts provides a measure
of whether or not the given parameter variation affects the desirability of the

policy--the policy sensitivity.

- 572 -
SIMULATE SIMULATE MODEL,
ORIGINAL MODEL ‘WITH ALTERED
PARAMETER
SIMULATE SIMULATE
POLICY CHANGE POLICY CHANGE
WITH ALTERED
PARAMETER:
FIND IMPACT FIND IMPACT
OF POLICY ON ‘OF POLICY ON

MODEL BEHAVIOR. MODEL BEHAVIOR
WITH ALTERED
: PARAMETER

FIND SENSITIVITY
E OF POLICY IMPACT

TO PARAMETER

VARIATIONS

Figure 3, Procedure for testing policy sensitivity

In performing various types of model tests, the modeler might be tempted
to avoid thoroughly analyzing the model behavior and examine only the end re-
sultt whether or not the overall behavior changes in response to a parameter
value change. Especially for a model suspected of being faulty, or during
numerous sensitivity tests, one might not take the time to analyze exactly
why the model behaves as it does in each simulation. (Senge and Mass 1976
give an example of model analysis.) Such a purely technical analysis, however,
provides several benefits. First, a technical analysis simplifies model
testing. If the structural causes of a system's insensitivity are known
(see footnote 17), one can immediately identify whole areas of the model
structure that are not important (in the behavior mode being tested). More-

over, one can identify the areas in the model structure that are important
to the behavior mode being tested, and that require further Investigation.
So, contrary to initial suspicions, performing a technical analysis may shorten,
instead of lengthening, the testing process.

Furthermore, purely technical analysis of model behavior also begins to
address the problem of sensitivity to multiple parameter changes. lt is
almost feasible to test the effect of ali single parameter variations for
most models, However, the number of possible conbinations of multiple paran-
eter changes in medium-sized or large models is far too large for the modeler
or a computer to test all possible combinations of parameter changes.1? But
if a tested theory is available to explain the underlying causes of the model's
behavior and its insensitivity to parameter variations, the modeler can dis~
tinguish between the multiple parameter changes that have a significant impact
and those that do not. Exhaustive testing 1s therefore not necessary in such

cases.

CG Dealing with Sensitive Parameters
The identification of behavior-sensitive and policy-sensitive parameters
can help to guide model reformulation, parameter estimation, and policy devel-
opment, as shown in Figure 4. The upper part of the figure illustrates

behavior-sensitivity testing (1) as a means of evaluating the realism of the

197 46 not clear at all, however, that changes in large numbers of parameters
are reasonable tests. Assuming a Bayesian viewpoint, assign each parameter
value a probability, and assume that the parameters ate independently distrib-
uted. Changing a eingle parameter avay from its most probable value reduces
its probability by some factor, For example, assume the factor is 0.5, The
probability of the entire set of parameters is reduced in proportion to the
product of the reductions in the individual probabilities. The probability
of the entire set of parameters therefore is reduced by 0.5. Then a multiple
parameter change reduces the probability density of each parameter by 0.5.
So making four such parameter changes reduces the aggregate probability by
0.0625, Investigating the consequences of such an unlikely event as four
such parameter changes does not seem very worthwhile.

~oThe=

w
EL > TEST BEHAVIOR

SENSITIVE
AREA IN MORE SENSITIVITY
DETAIL
(3) (2)
IS SENSITIVITY.
REALISTIC ?
ves
4)
DESIGN —-» FORMULATIVE POLICY
INSENSITIVE RECOMMENDATIONS
POLICY
(8)
" vesuaneere
emer Teeny ale
3 i
VALUES, Ho) REGOHWENDATIONS
(6)
we ARE SENSITIVE
2 PARAMETERS YES
CONTROLLABLE

Figure 4. Sensitivity testing in policy analysis

model. If the behavior sensitivity of the model does not correspond to the
behavior sensitivity of the real system (2), the model formulation must be
refined until they do correspond (3). The model then can be used to identify
policies that improve the behavior of the system (4).

To establish confidence in the policy recommendations, the policies should
be tested for their sensitivity to parameter variations (5). Suppose variations
in a parameter influence the desirability of a policy. The first question to
pose is whether or not the model user can control or influence the real processes
represented by the sensitive parameter (6). For instance, suppose a policy of

encouraging business expansion is sensitive to the average housing unit lifetime HL.
~ 515 ~

To some extent, policy-makers may be able to manipulate the average lifetime
of housing by altering assessment and property-tax practices or zoning. Ad-
justing the values of controllable parameters should be incorporated inta the
policy recommendations (7). Of course, the policy of encouraging business
expansion and altering housing lifetimes then requires further testing for
sensitivity to other parameters (5).

If policies are sensitive to uncontrollable parameters, model users have
three options.. First, perhaps the easiest option is to model in more detail
the processes represented by the sensitive parameters (3). Model parameters
describe the aggregate effects of processes below the level of aggregation of
model structure. These processes may occur over time periods much shorter or
longer than the time horizon the model is intended to portray. For example,
the housing-land multiplier table HLMT implicitly represents both the purchase
and sale of parcels of land and the elevation of land prices when unoccupied
land becomes scarce. HLMT gives the longer-term, aggregate results of these
short-term processes: building construction slows down as the land approaches
full occupancy. If the feedback loops that regulate land use are explicitly
represented, the revised model may show significantly less parameter sensitiv-
ity than the original representation, (Mass 1974a and Miller 1975 give more
detailed models of land use.)

The second option is to use the model to search for combinations of pol~
icies not sensitive to model parameters (8). Single policy changés which pro-
duce only moderate improvements and are fairly insensitive to parameter vari-
ations occasionally may be combined into a potent, insensitive policy. (Mass
1974b and Forrester 1969, pp. 227-237, give examples.) Third and finally, if

model reformulation and policy redesign both fail, the modeler must resort to

~ 576 -

some form of empirical research to determine more accurately the value of the

sensitive parameter (9).

VII, CONCLUSION

This paper discusses parameter-related issues that span the process of
modeling, from initial model formulation to final policy recommendations.
Of necessity, there are a large number of specific conclusions, principal
among which are:

(2) Each parameter should describe a separate and independent characteristic
of the real processes being modeled.

(2) The equation formulation should be general enough to allow parameter
values to describe many different cases.

(3) Preliminary parameter estimation should utilize data below the level of
aggregation (estimation from disaggregate data) where possible.

(4) "If parameters are to be estimated from data below the level of aggrega-
tion of model structure, the model structure should be disaggregated
enough to allow the parameters to be based on reliable observations of
relatively unchanging characteristics of the elements of the system,
rather than on (possibly 111-founded) conclusions or opinions about the
dynamic behavior of some subsystem.

(5) Equation estimation and model estimation should be used as secondary tech~
niques if at all, since they are much more vulnerable to error than esti~
mation from disaggregate data.

(6) Data at the level of aggregation of model structure should be reserved for
validity testing.

(7) Testing a model's behavior sensitivity and policy sensitivity can help to
identify the parameters and equations that require further estimation or
reformulation.

(8) Technical analysis of model behavior identifies the structural causes of
patameter sensitivity, and diminishes the need to test every parameter
and combination of parameters.
STs

(9) There are three ways to derive workable policy recommendations in the
Presence of uncontrollable policy-sensitive parameters: reformulate
the model to reduce the sensitivity, redesign the policy recommendation
to reduce its sensitivity, or reestimate the parameters in question with
more accuracy.

The number of techniques, even nonstatistica) techniques, for setting
parameter values is very large. The appropriateness. of each technique de~
pends on the needs of the model for accurate parameter values to fulfill its
purpose, the information available, and the strategy followed for model con=
struction and validity testing. These same considerations motivate both the
traditional uses of data (in experimental physics or economics, £or example)
and the typical system dynamics use of data described here. Researchers
should respond to different purposes, models, and availability of data by

choosing a method of setting parameters appropriate to the problem being

investigated.

Alfeld and Graham 1976

Babcock 1970

Britting and Trump 1975

Forrester 1961

Forrester 1968

Forrester 1969

Forrester 1975a

Forrester 1975b

Graham 1974

Goodman 1974a

~ 578 -

BIBLIOGRAPHY

Alfeld, Louis Edward, and Alan K. Graham. Introduc~
tion to Urban Dynamics. Cambridge, Mass.: Wright~
Allen Press, 1976.

Babcock, Daniel L. “Analysis and Improvement of a
Dynamic Urban Model." UCLA Ph.D. Thesis (unpub-
lished), 1970. Available from University Microfilns,
Ann Arbor, Michigan.

Britting, Kenneth R., and John G. Trump. "The
Parameter Sensitivity Issue in Urban Dynamics."
In Schroeder, Walter W. ITI, Robert E. Sweeney,
and Louls Edward Alfeld, eds. Readings in Urban
Dynamics Vol. 2. Cambridge, Mass.: Wright~Allen
Press, 1975, pp. 91-114.

Forrester, Jay W. Industrial Dynamics. Cambridge,
Mass.: MIT Press, 1961.

Forrester, Jay W. Principles of Systems. Cambridge,
Mass.: Wright-Allen Press, 1968.

Forrester, Jay W, Urban Dynamics, Cambridge, Mass.:
Weight-Allen Press, 1969.

Forrester, Jay W. “Business Structure, Economic Cy-
cles, and National Policy.” Prepared for the t7th
Annual Meeting of the National Association of Busi-
ness Econoinists, Boca Raton, Florida, 1975. Avail-
able also as System Dynamics Group Working Paper
D-2245-3, MIT, Room E40-253, Cambridge, Mass. 02139.

Forrester, Jay W. et al. "The Use of Data in Model-
ing: A Discussion," In Schroeder, Walter W. IIT,
Robert E. Sweeney, and Louis Edward Alfeld, eds.
Readings in Urban Dynamics Vol. 2. Cambridge, Mass.:
Wright-Allen Press, 1975, pp. 81-90.

Graham, Alan K. "Modeling City-Suburb Interactions."
In Mass, Nathaniel J., ed. Readings in Urban Dynamics
Vol. 1. Cambridge, Mass.: Wright-Allen Press, 1974,
pp. 155-168,

Goodman, Michael R. "Aggregation and Definition: The
Underemployed, A Case Study.” In Mass, Nathaniel J.,

ed. Readings in Urban Dynamics Vol. L. Cambridge,
Mass.: Wright-Allen Press, 1974, pp. 59-64.
Goodman 1974b

Hami1ton 1976

Junz and Rhomberg 1973

Mass 1974a

Mass 1974b

Mass and Senge 1976

Miller 1975

Peterson 1975

Peterson 1976

Peterson and Schweppe
1974

Schroeder and Strongman
1974

Schweppe 1973

7 519 -

Goodman, Michael R. Study Notes in System Dynamics.
Cambridge, Mass.: Wright-Allen Press, 1974,

Senge 1975a

Hamilton, Margaret $. “Estimating Lengths and Orders
of Delays in System Dynamics Models." This volume,
p+ XXXX-XXXK.

Senge 1975b

Junz, Helen B., and Rudolph R. Rhomberg. "Price Com
petitiveness in Export Trade and Industrial Countries."
American Economic Review Papers and Proceedings 63:
412, 1973.

Mass, Nathaniel J. "A Dynamic Model of Land Pricing
and Urban Land Allocation." In Mass, Nathaniel J.,
ed. Readings in Urban Dynamics Vol. Cambridge,
Mass.:. Wright~Allen Press, 1974, pp. 175-196.

Mass, Nathaniel J. "Self-Learning Revival Policies
in Urban Dynamics." In Mass, Nathaniel J.,-ed.
Readings in Urban Dynamice Vol. 1. Cambridge, Mass.:
Wright-Allen Press, 1975, pp. 133-164.

Mass, Nathaniel J., and Peter M. Senge. "Statistical
‘and Behavioral Tests for the Selection of Model Vari-
ables." This volume, pp, XXXX-XXXK.

Miller, John S. "Urban Dynamics and Land Rezoning."
In Schroeder, Walter W. III, Robert E. Sweeney, and
Louis Edward Alfeld, eds. Readings in Urban Dynamics
Vol. 2. Cambridge, Mass. Wright-Allen Press, 1975,
pp. 133-164.

Peterson, David W. “Hypothesis, Estimation, and Val-
idation of Dynamic Social Models." MIT Ph.D. Thesis
(unpublished), 1975,

Peterson, David W, “Statistical Tools for System
Dynamics." This volume, pp. XXXX-XXXX.

Peterson, David W., and Fred C. Schweppe. "Code for
a General Purpose System Identifier and Evaluator
(GPSIE)." IEEE Tr. Auto. Control AC-19: 852, 1974.

Schroeder, Walter W. III, and John E. Strongman.
“Adapting Urban Dynamics to Lowell." In Mass, Na-
thaniel J., ed, Readings in Urban Dynamics Vol. 1.
Cambridge, Mass.: Wright-Allen Press, 1975, pp.
197-224,

Schweppe, Fred C. Uncertain Dynamic Systems. Engle
wood Cliffs, N.J.: Prentice-Hall, 1973,

~ 580 -

Senge, Peter M. "Testing Estimation Techniques for
Social Models." Publication forthcoming. Available
as System Dynamics Group Working Paper D-2199-4, Room
40-253, MIT, Cambridge, Mass. 02139, 1975.

Senge, Peter M. “Future Electronics Company." Sys-
tem Dynamics Group Working Paper D-2310, Room E40-253,
MIT, Cambridge, Mass. 02139, 1975.