Towards the use of model structure analysis for designing flexible learning
itineraries
Martin Schaffernicht '
Abstract
Some large system dynamics models drive simulator interfaces used for teaching; this
is the case of the MacroLab model. Such a model may be useful for making students
with basic instruction in system dynamics explore the economy as a dynamic system,
allowing for diverse inquiry itineraries. The question is if different exploration
itineraries yield sufficiently similar learning outcomes. This has been tried with ten
student groups. The results are encouraging, but also indicate that the inquiry
scenario design should be based on systematic analysis of the model’s structure:
some variables may not be reachable from everywhere. An ad-hoc structure
exploration found such isolated areas. The use of a reachability matrix is suggested
and an initial example is shown. Also, students need systematic guidance in
constructing a loop set that will frame their exploration. Concluding, it is argued that
this kind of instructional design may bring other large system dynamics models
closer to instructional use.
Keywords: education, model structure, feedback loop set, reachability,
macroeconomics.
Introduction
Large system dynamics models have been used in teaching macroeconomics
as engine behind gaming interfaces for some years now; one example is thee
MacroLab environment (Wheat, 2007b). The interface shields students from
the system dynamics part, concentrating on “causal loop diagram” (CLD)
based argumentation and using the simulator to generate behaviors. This has
been shown to allow focusing cognitive resources to the dynamic processes
going on in the economy.
However, it may have other advantages as well. One such advantage may
stem from the fact that in system dynamics-models the many feedback loops
tie together the parts (Kampmann and Oliva, 2008). A. typical
macroeconomics model deals with different markets that are usually
discussed separately, and the models’ variables are distributed across these
markets (compare with Table 2 below). In contrast, with system dynamics
model, due to its feedback loops one would expect that for investigating the
economy’s working, the variable where one starts out does not have a large
influence on the set of variables inquired during the process. For example,
one student could start in the labor market and another in the money market —
still their inquiry would lead them across the entire model and all the markets.
This would be an interesting complement for textbook-based teaching, where
markets (and their immediate variables) are treated chapter by chapter in a
linear manner. Additionally, a whole set of learning itineraries might exist
where the textbook privileges one *.
' Facultad de Ciencias Empresariales, Universidad de Talca, Chile; martin@utalca.cl
? Even though textbooks may encourage the reader to define an individual reading
itinerary, the mere fact of the linear organization of chapters in a book privileges one
pathway over the others.
This possibility fueled the idea of a course where it was hoped that despite the
individually different itineraries, all students would learn the same relevant
items. The present paper reports of the journey that started by recognizing
this opportunity and subsequent problems and discoveries that lead the author
to believe that model structure analysis tools (Kampmann and Oliva, 2006;
2008; Moijtahedzadeh, 2008) may become used by instructional designers to
create flexible itineraries for learner centered teaching (Richmond, 1993;
Forrester, 1998), thus making large models useful for teaching processes that
go beyond “causal loop diagrams” (CLDs).
An initial attempt was carried out with one course during 2008. Ten groups
of third year undergraduate students with a first introduction into system
dynamics worked through ten different scenarios each starting out with a
shock to a different variable. These shocks consisted of exogenous
perturbations in the variable’s behavior. The students’ resulting mental
models of the situation — represented as variables and causal links * - were
compared amongst each other and also to a representation of the underlying
simulation model, using the “distance-ratio” method (Markovski and
Goldberg, 1995).
The groups’ mental models were found to be similar, which could at first
sight be interpreted as success. However, only small parts of the reference
model were used by the students and when the small size of their mental
models is taken into account, the differences appear to be too large to pass the
test.
In order to shed light onto the reasons for these differences, the links between
the different feedback loops taken into account by the students were used to
construct a reachability matrix amongst loops. Indeed, while there were
groups of maximally connected loops, other loops were isolated. Inspection
of the groups’ mental models and shock variables revealed that “missing”
variables were due to shock variables that belong to isolated loops. Ex-post
analysis revealed that the apparent isolation was due to mistakes in the
students’ mental models; however, during the work process, they only had
these mental models to work with.
After using this ad-hoc approach to analyze the situation, it appeared to the
author that the design of learning experiences would benefit from the use of
the tools developed for structural analysis; this would allow to:
1. design a range of shock scenarios that assure the reachability of a given
set of variables (in order to guarantee equality across scenarios and to
take students from simpler structures to more complex ones).
2. work with a set of feedback loops that is adequate for students. The
determination of the loop set used is not a trivial manner. In large
models, the set of feedback loops is huge and complex and currently there
is no precise indication concerning which loops students will work with.
The paper is organized in the following way. First the MacoLab model is
briefly introduced. Next, the goals, methodology and procedure for the initial
attempt are presented. Then the results are explained together with how they
called for further analysis. The ad-hoc analysis is then discussed, leading to
the insight that instructional designers cannot trust in the general idea that in
system dynamics models everything is connected with everything else.
3 For an introduction to the subject of mental models, refer to Schaffernicht and
Grésser, 2009
Eventually, a preliminary procedure for designing learning activities is
proposed, together with some reflection concerning the use of published
analysis methods and the two issues mentioned above — followed by an
outlook on the future of large models in education that appears to be possible
to the author.
MacroLab
The MacroLab model represents the worldwide economy as consisting of two
blocks of countries, which is sufficient for the case of teaching introductory
macroeconomics. Economically speaking, it is a fusion model that does not
try to represent any particular school of thought and its time horizon is about
five years, excluding growth processes from the analysis. The model consists
of two groups of 10 sectors each, one for the USA and an equivalent one for
the “rest of the world”.
[Submodel _|Sector
Production [Labor
(Capital
Productivity
Price
income distribution
[Consumption
Government
Banking Money
Monetary policy
Exchange rate
Table 1: MacroLab model sectors
A detailed description of the model is to be found in Wheat (2007a). The
whole model counts more than 400 variables and its corresponding stock-and-
flow diagram covers more than 20 computer screens. While this amount of
detail is required to carry out the simulation task with sufficient fidelity (to
textbook models and to historical data), it is an overwhelmingly complex
situation for the students exposed to our experience.
In order to work with a manageable amount of complexity and to assure
comparability amongst the reference model and students’ mental models, a
simplified causal loop diagram was derived using a procedure that also was
used by students (as described in the following section) and used as reference.
The model simplification yielded a set of 81 variables (details reported in
Quiroz y Aravena, 2008; Schaffernicht et al., 2008):
(Code [Refers to [Code [Refers to
[ADet [expected real demand M resverves (money offer)
ic [consumption (nominal) mbr mean bond rate
lceP cost effect on price mip multifactor productivity
[cpRat cost productivity ratio mpc mean propensity to consume
lcpRel cost productivity relation IN [employment
lcProp propensity to consume Ince inet capital entry (US)
ct compliance time Na [desired labor
ldeP demand effect on price Indr Inet deposit rate
di disposable income business lr inet lending rate
ldivN dividends (nominal) fopSrp operating surplus
lap dividends pet P price level
ldpst deposits pFac production factors
faTr Inet change of FF target rate pimp payments for Im (US)
leN effective labor PoB purchase of bonds
eti extra-time index PRW Price level (RW)
ExpNN __ [Exports (US) net nominal Exp nominal revenue from Ex (US)
ExpNNRW |net nominal Ex (RW) ros revenue % of salaries
lexR lexchange rate rr reserves rate
lexRstv _ [excess reserves rRW revenues (RW)
iG government spending s savings (W)
Gb government borrowing spe salaries % of capital
GBat government budget sr (nominal) sales revenue
GDbt [government debt sttp social transfer pmts % of budget
GDef government deficit Tax taxes
GPch [government purchases TaxB
i Interest rate TaxHh s households
idi income / disposable income ip transfer payments nominal
ieC interest elasticity of consumption tpB business tax pet
ieS interest elasticity of savings ipHh personal tax pet
imp [Imports real (US) ir FF target rate
ImpVW___ [WT of Imp volume (USA) tri [FF target rate indicator
Inv investment jactmp lunit cost of imports
rec interest rate effect on consumption fuer unemployment rate
iRise rise ini [UPC lunit production cost
IK capital lw nominal salary
Ke capital cost wap working age pupulation
KDes desired capital wd withdrawels
KDpr capital depreciation wre workforce
L [demand for reserves (liquidity demand) Ya laggregate demand
Ip loan payments Ys production
Iter long term expected results,
Table 2: variables used
In this table, variable names are abbreviated in order to facilitate
diagrammatic representation. The variables in bold typeface correspond to
the typical textbook variables.
The pilot experience
Objectives
The aim of the effort was to test if different groups of students, analyzing
different shocks, i.e. entering their inquiry at different variables (and sectors)
would still learn the same conceptual contents. The research question was:
Do student groups who use the MacroLab model from different entry
points learn the same variables and causal links amongst groups and in
relation to macroeconomics textbooks?
This yields two hypotheses:
H1: students’ metal models stemming from inquiring different shock
scenarios in the context of the MacroLab are similar to the underlying
model (which is compatible with the standard textbook model).
H2: students’ metal models stemming from inquiring different shock
scenarios in the context of the MacroLab are similar amongst each
other
Methodology
For the purposes of exercises carried out in the context of MacroLab, the
underlying simulation model is the “reality” against which students develop
their mental models, which consist of variables and causal links. Since CLDs
also contain (and are based upon) variables and causal links, we interpret
students’ CLDs as expression of their mental models: without the “loop”
component, they transform into “causal diagrams” — the representation used
by standard methods (Markévski and Goldberg, 1995). This is widely
practiced in the system dynamics community- (Capelo and Ferreira, 2008).
In order to make students’ mental models comparable against the “real”
MacroLab model, the following procedure is applied to the full stock-and-
flow model in order to obtain a comparable causal diagram:
1. select and mark a MacroLab variable that represents a standard
macroeconomic variable and put it on the causal diagram;
2. follow a link to a connected MacroLab variable which has not been
marked yet:
— if itis a standard macroeconomic variable or it receives a link
from another variable, return to step 1;
— else repeat step 2.
Application of this procedure leads to a causal diagram with only standard
macroeconomic variables and such variables that are needed to maintain any
feedback loops; other variables are “collapsed” and the number of variables
reduced to 81. Any individual who follows these instructions will obtain the
same causal diagram.
This representation of the system dynamics model has the same structure as
mental models as captured from students. Accordingly, the differences
between students’ causal diagrams and this “true” causal diagram indicates
how far or close their mental models come to the “truth”. Also, students’
causal models are compared amongst each other to discover if they are rather
similar or different.
In order to produce a systematic and generally understandable comparison,
the “Distance Ratio” (DR) method was selected. The method was initially
developed by Langfield-Smith et al. (1992) and improved by Markovski and
Golberg (1995) which allows measuring the distance between different
mental models. The details of this method are explained in Appendix 1.
Procedure and population
Students work through a sequence of phases as follows:
1. Each group is assigned a “domestic” sector of the MacroLab model
and a scenario with a shock that affects one of its variables.
2. The group applies the procedure described in the previous section to
their respective sector in order work out a CLD.
3. All groups assemble a general CLD from their sector diagrams that
will serve as navigation plan; this generates discussion amongst
groups and is the opportunity to discover mistakes and produce a
shared causal diagram. (Observance of the procedure assures that the
causal diagram is comparable to the “true” one without the personal
bias each individual would otherwise bring to the process of
representing the simulation model.)
4. For the variables included in the CLD, the MacroLab variables’
behavior resulting from the shock is inquired; variables which do
react noticeably are treated as main variables and their behavior is
drawn on the causal diagram; other variables (see supplementary
material in
http://dinamicasistemas.utalca.cl/5_Educacion/MacroLab/MacroLab.
html);
5. A report is written with a step-by-step explanation of how the shock
produces its effects on production and employment.
During the whole process, students have full access to the stock-and-flow
diagram and the equations of the MacroLab model.
Afterwards, each group’s final CLD is interpreted as a causal diagram
(without the loops) and distance ratios were calculated between groups’
models and between each group’s model and the reference model (Quiroz y
Aravena, 2008; Schaffernicht et al., 2008)
The exploration was carried out with a group of 25 students from the third
year of undergraduate studies in “management and information systems” at
the University of Talca. The context is a year-long course which starts out
with an introduction into system dynamics that deals with the fundamentals
of causal models, stock-and-flow thinking and the basic structures and
behaviors during 32 hours of instruction.
The following table indicates the shock scenarios:
[Submodel [Sector [Shock
Production Labor Population rises 30 million
Capital Earthquaque destroys capital
Productivity [Innovations rise productivity
Price Import costs rise
Income distribution [Household taxes lower 50%
[Consumption Propensity to consume augments
Government Federal budget rises during years 2 to7
Banking Money [Additional reserves
Monetary policy [Aggegate demand rise causes inflation
Exchange rate [Lower currency offer
Table 3: shock scenarios
Each of the student groups thus had a different scenario, but their tasks had
the same goal.
Results from the first experience
Good news and bad news
The students constructed the following causal loop diagram (included in
supplementary material):
In the diagram, causal links that belong to feedback loops are printed bold.
This helps to visually recognize the loops detected by the students. It also
makes visible the fact that many variables and links are not on any loop for
the students.
Overall, students identified 26 feedback loops, which appear with number and
polarity in the figure. A detailed presentation would be beyond the scope of
this paper; the interested reader will find a detailed description at the web-site
http://dinamicasistemas.utalca.cl/5_Educacion/MacroLab/MacroLab.html
The DR analysis produced the following results:
[Submodel _|Sector DR (%) Variables _|_%Variables
Production [Labor 374 21 26%
(Capital 3,62 15 19%
Productivity 3,75 9 11%
Price 3,82 17 21%
Income distribution 3,05 7 21%
Consumption 3,75 19 23%
Government 3,59 21 26%
Banking Money E
Monetary policy 4,12 17 21%
Exchange rate E
Table 4: distance ratios
The Money and Exchange rate groups did not finish their assignment on time
and were excluded from the analysis. For the remaining 8 sectors (groups),
the DRs range from 3.05% to 4.12%. At first sight this looks like what had
been hoped for: students’ models are hardly different from the reference
model.
However, a look at the columns “Variables” and “%Variables” reveals that
students’ models cover only a small portion (between 9 and 21 out of 83) of
the variables contained in the reference model. This means that the groups
excluded the majority of variables from their thinking about the shocks they
had to understand. This makes it necessary to ask two new questions:
1. did their models contain different parts of the reference model?
2. did their models include all the textbook variables?
As can be seen in the following figure, the DRs between student models were
higher than in relation to the underlying model:
=
2
3 3
2 & & a
= © Ee =
a z S @ E s
B 8 8 8 § 8 &
pag oO o a = o} =
Labor 9,30, 13,76, 10,38 14,53
(Capital 18,44) 10,79 18,26
Productivity 15,12 18,60
Price 16,18
Income distribution 72,11, 12,86) 15,70, 9,93 70,40 15,07,
[Consumption 8,95, 11,23) 13,04, _6,19| 10,42 _10,50| 13,36
[Government 70,19, 10,48) 13,41, 10,12 14,16
Table 5: inter-group distance ratios
If one takes into account the fact that student’ models covered about 20
variables each, these models were quite similar: this is a reassuring aspect,
since the sheer fact of having different numbers of variables already makes
out part of these differences. However, with respect to the second question, a
different picture emerges:
Shock sector Variables __Principales Covered
Consumption 19 12 67%
Monetary policy 17 12 67%
Price 17 9 50%
Labor 24 8 44%
Capital 15 8 44%
Government 21 8 44%
Income distribution irs 6 33%
Productivity 9 4 22%
Table 6: groups' coverage of main variables
None of the student groups included all of the textbook variables into their
CLDs. The Consumption, Monetary policy and Price scenarios made their
respective groups include between half and two thirds of these main
variables; all the other groups have been working with less than half of these
variables. This impression is complemented by looking at the coverage of
each of these variables (across groups):
Variable Code| Covered
Employment N 100,00%|
Production Y 100,00%|
Unemployment rate uer 100,00%|
Aggregate demand Y 87,50%|
Nominal wage w 87,50%
Price level Pe 62,50%|
Interest rate i 50,00%|
‘Consumption [ej 37,50%
Investments | 37,50%
Savings W. 37,50%
Propensity to consume c 25,00%
Capital K 25,00%
Reserves demand i. 25,00%
Reserves offer M 25,00%
Government deficit Gdef 12,50%
Government spending G 12,50%|
Exchange rate exR 12,50%
Exports Ex 0,00%|
Table 7: coverage of main economic variables
As one might expect from the shared goal of all the inquiries, employment,
production and the unemployment rate were always in the student models.
But none of the other main economic variables has been considered be all of
the groups.
The following figure presents the variables with text sizes corresponding to
the frequency of their use across the student groups:
Figure 2: frequencies of use for the MacroLab variables in the students’ CLD
(included as “students_loops_and_frequencies.mdl” in supplementary material)
10
This is not what was hoped for: in this project, students learned roughly about
the same variables, but not enough. Neither of the two hypotheses can be
accepted in this case.
It then becomes important to understand why this happened. While some
differences may safely be attributed to mistakes made by the students, it is
unlikely that this would be a good explanation: in this case, the percentage of
covered variables should be similar, which is clearly not the case.
One possible explanation may be that some variables cannot be reached from
just any other variable in side the reference model. In this case, the different
starting points of the respective scenarios may disclose different parts of the
reference model. In order to find out if this was a valid explanation, some
additional inquiry was necessary.
Connections between feedback loops
In a first step, the membership of each variable to the respective feedback
loops detected by the students was marked in a table with the following
structure:
Variables
2 iRise
18 dN
1 1|4
Table 8: variables and loops
This arrangement allows not only to see which variables belong to a given
loop, but —and this is what it is used for here — it shows how the loops are
connected amongst each other: if a variable is member of, say, two loops (like
C in the table, which is on loops 15 and 16), then these loops are connected
by means of this variable.
In such cases, if one of these loops is touched
upon by a shock, the other one will be impacted, too. In this sense, such
loops are like a group of loops, and such a group is always reached as whole
structure.
Based upon this table (complete version in appendix 2), it is possible to draw
a map of all the 26 loops and their interconnections, like shown in the
following figure:
ll
Figure 3: groups of feedback loops
In this diagram, each feedback loop is represented by a circle containing the
respective numbers (black = positive, gray = negative). Usually the
connection is drawn as an undirected line. For instance, between loops 1 and
2, there is a connection: both have at least one shared variable, and since it is
part of each of the loops, the connection is bidirectional.
It turns out that there are six such groups of loops (from a to f), inside which
each loop is connected with all the others. Please note that groups d and e
have only been separated for the sake of readability (by drawing them
together, the number of connections would grow strongly) — this is an
inconvenient aspect, due to the rather ad-hoc nature of the diagram. System
dynamics does not yet have a specific diagram language to represent loops by
themselves (other than the CLDs that force the reader to look at the single
variables). In the future, better representations like the loop inclusion graph
(Oliva, 2004:329) shall be used.
Between some of the groups, there is an arrow. This indicates that there is at
least one causal link from one of the first loop group’s variables to one of the
second loop group. (This has been done inside group c, too, in order to avoid
having two one-loop groups.)
If a shock scenario hits a variable in loop 20, say, then all the variables on the
other loops of groups d and e will be affected — and no other. If, in turn, the
shock impacts a variable of group a, all the variables of all the other groups
will be concerned, too. This allows to populate a loop reachability matrix,
shown in the table below:
12
Loops [1] 2]3)4]5)|6|7]| 8) 9 | 10) 11) 12) 13) 14) 15] 16) 17| 18] 19| 20] 21 | 22) 23| 24) 25] 26
1 T/t/i[t[t{i{7
204 trata fafasa
3 tia[t{a]4 7 Tiassa Titi tittii{i{t
4 1 TTi[i[a 1 Thifafifiaa Titfhttsastfr[a
5 Tit T[i}i 1 Thifififiai Tit ttif titi i
6 Tht]4 cd Tiifi[ififi Tatas
7 Tifa 4 Alla Tiifitifisi Tass aa
8 Tli[i[i]t 1 Tit[i[tfifi Wilts
9 4
70 1
1
12 i tii pitayayatt
13 1 Taya
14 1 TiiTi[t tif i{t
15 1 ae
16 7 Tit Titit iii i{t
17 Tiifa[ifi T{7
18 Tli[a[ift 1
19 Titiajata 1
20 Tiififit {1
21 Tlt[a[ift {4
22 Titiaia{a {4
23 tit]a{a{a {1
24 Titfifi{t {7
25 Tlif[a[ift {1
26 Tittiii{a {1
Table 9: reachability of loops
In this matrix, entries are read from row to column, like “loop 4 allows to
reach loop 3” for instance. Based on this information and the membership of
each variable to one sector and to certain loops, one can construct the
following table:
(Sector [Directly reachable loops [Reachable loops.
Labor 18 19,12, 13, 14, 15, 16, 17, 20, 21, 22, 23, 24, 25, 26
Capital 17, 23 12, 13, 14, 15, 16, 17, 20, 21, 22, 24, 25, 27
[Productivity 22, 24, 25 12, 13, 14, 15, 16, 17, 20, 21, 22, 24, 25, 28
[Price 20, 24, 25 12, 13, 14, 15, 16, 17, 20, 21, 22, 24, 25, 29
Income distribution 12,16 13, 14, 15, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26
[Consumption 15,16 12, 13, 14, 17, 20, 21, 22, 23, 24, 25, 27
[Government 12, 13, 14 15, 16, 17, 20, 21, 22, 23, 24, 25, 28
[Money 3,4,5,6 7,8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26
[Monetary policy 3, 12, 13, 14, 15, 16, 21, 25, 26 4,5, 6, 7, 8, 9, 10, 11, 17, 18, 19, 20, 22, 23, 24
|Exchange rate 12, 13, 14, 15, 16, 21, 25, 26 17, 18, 19, 20, 21, 22, 23, 24, 25, 26
Table 10: loops reaches from each sector
In this table, the loops that can be reached from variables of the different
sectors are listed. In terms of loop groups, one can now appreciate some
differences:
[Sector [Initial [Reached % of Loops
[Consumption D cd,e,f 92%
Money D cd.e,f 92%
Monetary policy b,d,e o,f 92%
Labor f ed 58%
Income distribution e d, f 58%
Government e d, f 58%
Exchange rate de f 58%
(Capital d @ 50%
Productivity d @ 50%
Price d e 50%
Table 11: loop coverage of sector variables
This table has been constructed using the membership of each loop to their
respective group. Some sectors’ variables are located on loops such as to
make most parts of the model reachable. At the same time, other sectors do
not allow to reach more than half of the feedback loops (and their variables).
The ordered list of variables used per sector (by the students groups: see
Table 6, p. 9) does not correspond exactly to the order of sectors in this last
table, but this can be attributed to mistakes made by the students. However,
at this point of the argumentation, we are independent from this particular
group of students and the quality of their respective work: if the perceived
model structure does not allow to reach certain loop groups starting in certain
sectors, then the different shock scenarios are deemed to fail the ambition to
allow equal or comparable learning about variables. For instance, as long as
all the scenarios start in the consumption, money or monetary policy sector,
there will be equality of opportunity; but as soon as the /abor sector is the
starting point of a forth scenario, some students will have less opportunity to
learn about the whole set of textbook variables.
Clearly, this is not a desirable state of affairs. Accordingly, some
consequences shall be explained in the following section.
Consequences from the first exploration — new challenges
This whole reflection and argumentation is made from the viewpoint of the
lecturer as the designer of learning activities: the students will realize the
MacroLab activity and learn from it. The goal of this design activity is that
students learn about the important variables and their causal connections
inside the dynamic system “macroeconomy”. That they could do so along
various itineraries would add flexibility in the sense of learner-directed
learning (Forrester, 1998). However, it has to be assured that they have the
same opportunities. Additionally, learning activities should provide a
progression from simpler to more complex tasks.
It has become evident that the simple heuristic “one sector — one scenario” is
not advisable: the hope that the complex of feedback loops will tie everything
together is not a good enough guide. In the remainder of this section, a first
sketch of a procedure for designing learning activities with large models is
introduced.
The goal of the outlined design method is to provide a series of shock
scenarios going from simple to complex by involving growing percentage of
the model’s parts.
The baseline of such instructional design is knowledge about which parts of
the model are reached by any particular shock scenario. This can be achieved
by two types of analysis, which shall be briefly discussed now.
In the exploration above, feedback loops have been used, because they are the
fundamental component of social systems in system dynamics (Forrester,
1968). However, in models of the size of MacroLab, the concrete feedback
loops are a tricky affair. The complete set of loops is breathtakingly huge:
14
Variable Code Loops
Employment N 251
Production ¥= 653
Unemployment rate uer 99
Aggregate demand a 725
Nominal wage w 510
Price level P 625
Interest rate i 9
Consumption Cc 122
Investments I 355
Savings Ww. 0
Propensity to consume Cc )
Capital K 354
Reserves demand L 4
Reserves offer M 9
Government deficit Gdef 286
Government spending G 4
Exchange rate exR )
Exports Ex 19
Table 12: feedback loops per textbook variable (loop count done with VenSim)
As the table indicates, most of the textbook variables lie on a big number of
feedback loops. It certainly calls our attention that some variables have
between few and no loops; this may indicate that these variables are not of
great importance for the purposes of MacroLab’s developer. But this is a
different question and may be analyzed by a new paper.
Clearly, the variables that have been central for the students’ efforts are part
of more feedback loops than the naked eye can see —- many more than the 26
distinguished by students. Any analyst would work based on a subset of all
these loops — so will the instructional designer. But there are many such
possible loop sets — so which to chose? And like stated by Kampmann and
Oliva (2008), any particular independent loop set is relative to some previous
decision like the variable one starts from. In this sense, the loop set may be
intuitive (like in our case, where it has been constructed by focusing on the
smallest loops) or based on one of the methods for “independent loop sets”
(Kampmann, 1996; Oliva, 2004), it will always be one of many possible sets,
and the instructional designer heavily intervenes in this by making a selection
for the students.
Is there a best loop set, or a criterion to select one? This is an open question,
and answers would be welcome (but will have to be provided by future
research). Besides this, it may be little desirable in educational terms to
construct the loop set for the students: wouldn’t it be indicated to give them a
procedure and make them construct their loop set?
How to design shock scenarios that would be robust in the face of different
loop sets used by different student (groups)? The reachability matrix (Oliva,
2004) comes to mind. It clearly shows the set of variables to be reached from
any starting point. Any well-constructed loop set will conserve the
reachability expressed in this matrix. So reachability questions may be
addressed based on this matrix rather than feedback loop sets (going counter
to what has been naively done during the initial exploration).
Then the suggested method is:
1. define a collection of sets of target variables that shall be involved in
shock scenarios;
2. construct the reachability matrix;
15
3. select the sets of entry variables from which the target variables can
be reached;
4. define one or several shock scenarios that impact the corresponding
entry variables.
This procedure will help designing sets of scenarios in such a way that the
complexity of the inquiry activities carried out by students remains under
control of the designer. Also, the matching with goals referring to specific
sets or subsets of variables will become easier. Of cause, it would be
desirable to have computer tools that help in performing the steps. While
some tools have been developed and freely made available (refer to Oliva,
2004 and Kampmann and Oliva, 2008), specific tools for the purpose of
instructional design have to be developed yet.
It would be very desirable to provide students with a method for constructing
a loop set that is an “independent loop set” (ILS) or “minimal independent
loop set” (MILS). The currently available methods (Kampmann, 1996;
Oliva, 2004) can be followed, but they may be too complex for students who
have recently been introduced to system dynamics. The question if there is a
way to guide “intuitive” loop set construction in a way such as to assure the
result is an ILS or even MILS, remains open for now and is a challenge for a
future paper.
Still this is a very important issue: during the first application of the
suggested procedure, it turned out that the students who participated in the
first experience had overlooked some links and thus worked inside a mistaken
picture of the “system” (as mentioned above). In a way, this may be
reassuring — after all, many variables were more reachable than perceived by
students. But beyond this, it shows the second weakness of the intuitive
approach behind the first experience: guidance in loop set construction is
needed.
In order to make clear the degree of difficulty of visually distinguishing these
loops, the following figure displays the “correct” CLD:
16
LI
Azeyuaurayddns ut _[purursseiqresne9,, SB popnpoUul) GTO 1901109 oy -p ONT
As compared to the students’ CLD, there are only few differences at the level
of the causal links: Inv->nlr, ros->w, rRw->exR, rExp->expNN and
expRWNN->expNN. The reader will agree that they are hard to detect.
However, several of them belong to the set of textbook variables, and not
perceiving the links had severe consequences for the reachability and
students’ results in the exercise.
Looking ahead — towards the use of model structure analysis for instructional
design
The steps explained here were made following Oliva (2004). The causal
diagram shown in the previous figure can be transformed in an adjacency
matrix; for MacroLab’s set of 81 variables, this is an 81X81 matrix A with
one row and one column for each variable. For reasons of readability, it is
only included in the additional material. When the variable in row r precedes
the variable in column c in the causal diagram, a “1” is marked in the cell a,...
After adding the identity matrix I to A, the reachability matrix R can be
constructed by successively elevating A to the next boolean power until no
more differences exist between two successive versions of A.
In our example, A> # A® #A*#A°=A°. The steps executed for this task are
described in appendix 3 (p. 25). The resulting matrix R can be used to
identify the variables from which they can be reached or not be reached. Two
examples may be:
1. from which variables can I reach all the textbook variables in the case
of a closed economy (no exports and imports)? This question will be
answered by setting the textbook variables’ filters (except for exports
and imports) to “1”. Only the variables from which one can, directly
or indirectly, reach the textbook variables are selected: it turns out
that all the variables in the CLD allow to reach the entire set of these
target variables. However, this includes the possibility of reaching
result variables (or dead-end variables) that do not have successors
(so the trip stops there). (Again, it seems noteworthy that savings
appears as a result variable.)
2. from which variables can I reach only variables i, L and M (and no
other of the textbook variables)? Answering this question takes
setting the filter of i, L and M to “1” and the filters of the remaining
textbook variables to “0”. However, it turns out that with the
exception of imports, all other textbook variables can be reached
from any other variable. So it appears that simple exploration
scenarios could only be designed by using MacroLab’s capability to
switch of certain sectors, like “Government”, for instance. Analyzing
this aspect would require to have a CLD for each “active sectors set”
of the MacroLab model.
Of cause this is a primitive way to proceed: clearly, it would be easy to
develop a piece of software that constructs and uses the reachability matrix in
a more comfortable manner. This shall be done in the following months and
will be reported in time.
Structural partitions - level and cycle partitions as described by Oliva
(2004:319) will be explored, too. The intuition behind this step is that a
clearer image of the dependencies inside the model (level partition) and of the
relationships amongst feedback loops (cycle partition) will allow students to
improve their systems thinking (Richmond, 1993) beyond the description
18
level of variables — a way to “see the forest and the trees” (Richmond,
1994:140).
Preliminary conclusions
This paper referred to the MacroLab model. It reported work started naively
trusting in the all-spanning feedback loops and not taking into account the
feedback loop complexity inherent to large system dynamics models. Even
though some success has been achieved by making students work from in
different scenarios — they had a large share of their mental models in common
— it became clear that some previous analysis has to be done in order to
design adequate studying scenarios. At the time being, the reachability
matrix is being used to tailor the scenarios for the class of 2009.
There are several tasks ahead. First, the issue concerning the construction of
a convenient feedback loop set has to be settled. Then the work procedure
shall receive more comfortable tools to work.
But there is more: this particular case touches upon something larger. There
seem to be two kinds of system dynamics books nowadays. The first kind are
“textbooks” (Forrester, 1961; Forrester, 1969; Morecroft and Sterman, 1994;
Sterman, 2000; Morecroft, 2007; Schaffernicht, 2008); these have been
written to help students learn system dynamics, and they contain models with
one, two, three or four feedback loops. This is because most of the important
issues are well captured by these rather little models, and it would not be
desirable to needlessly add complexity.
The other kind are books like “Urban dynamics” (Forrester, 1969), “World
dynamics” (Forrester, 1972), “Limits to growth” (Meadows et al., 2004).
These are texts written above a model that has been built and used to
understand a given problem and maybe find ways out of it. These models are
much more complex than their textbook cousins.
Why not use the “world3” model from “Limits to growth” to generate a set of
exploration scenarios, using the same procedure and tools? Wouldn’t this
bring together learning about a system/problem and learning about system
dynamics (on the task)? Wouldn’t it make some important features of the
system dynamics method accessible to users, and allow individuals with
relatively little expertise in systems dynamics to engage with complex models
in a meaningful way? When Forrester called our attention to the fact that
modeling is not the same as using a model like a simulation game (1985), he
was concerned by the fact that “consuming” a game does not trigger enough
reflection. Looked upon from this angle, students who work through their
way from a previously unknown perturbation to understanding its effect and
even to designing a solution or mitigation policy, must reflect a lot: sure, they
receive some guidance, and the model is already there — but they have to
construct their understanding (or mental model) themselves.
In this sense, the use of structural analysis methods and tools (Kampmann,
1996; Oliva, 2004; Kampmann and Oliva, 2006; 2008; Moijtahedzadeh,
2008) for instructional design promises to bring larger — and more applied or
problem-related- system dynamics models to a larger audience. As stated by
Moijtahedzadeh (2008:451), if these methods and currently still “targeted
towards a small interest group. However, the ultimate goal is to reach a wider
audience [...]”. It seems possible that specific sets of scenarios can be
developed by the system dynamics community and recommended for
teaching in wider fields, bringing the benefits of system dynamics to a wider
public without requiring full training in modeling.
The author hopes that MacroLab will be but the first step in this direction.
19
References
Capelo, C. and Ferreira, J. 2008. A system dynamics-based simulation
experiment for testing mental model and performance effects of using
the balanced scorecard, System Dynamics Review 25(1): 1-34
Forrester, J. 1961. Industrial dynamics, MIT Press
Forrester, J. 1968. Principles of Systems, MIT Press
Forrester, J. 1969. Urban dynamics, Productivity Press
Forrester,J. 1972. World Dynamics. Productivity Press; 2 edition (1971)
Forrester, J., 1985. The “model versus a modeling process”, System Dynamics
Review 1: 133-134
Forrester, J. 1998. Designing the future, Universidad de Sevilla
Kampmann CE. 1996. Feedback Loop Gains and System Behavior,
http://www.cbs.dk/content/download/10577/1579 14/file/FB %20loops.pd
f (3/13/2009).
Kampmann, C.E. and Oliva, R. 2006. Loop eigenvalue analysis: three case
studies, System Dynamics Review 22(2): 141-162
Kampmann, C.E. and Oliva, R. 2008. Structural dominance analysis and theory
building in system dynamics. Systems Research and Behavioral Science
25: 505-519
Langfield-Smith, K. and Wirth, A. 1992. Measuring differences between
cognitive maps, Journal of Operational Research 43(12): 1135-1150
Markovski, L and Goldberg, J. 1995. A method for eliciting and comparing
causal maps, Journal of management 21(2), p. 305-333
Meadows, D., Randers, J and Meadows, D. 2004. Limits to Growth — the 30
year update. Chelsea Green
Moijtahedzadeh, M. 2008. Do parralel lines meet? How can pathway
participation metrics and eigenvalue analysis produce similar results?
System Dynamics Review 22(4): 451-478
Morecroft, J. and Sterman, J. 1994. Modeling for learning organizations,
Productivity Press
Morecroft, J. 2007. Strategic modelling and business dynamics: A feedback
approach. John Wiley.
Oliva, R. 2004. Model structure analysis through graph theory: partition
heuristics and feedback structure decomposition. System Dynamics
Review 20(4): 313-336
Quiroz, JF. y Aravena, C., “Brisqueda de Posibles Diferencias en Modelos
Mentales Resultantes de la Exploracién de Sectores en un Modelo de
Simulacién Macroeconé6mico, el caso MacroLab ILE’, Memoria de
grado de Ingenieria comercial, Universidad de Talca, Agosto 2008
Richmond, B. 1993. “Systems thinking: critical thinking skills for the 1990s
and beyond”, System Dynamics Review 9(2):113-133
Richmond, B. 1994. “System Dynamics/Systems Thinking: Let's Just Get On
With It”, System Dynamics Review 10(2-3):135-157
20
Schaffernicht, M. 2008. Indagacién de situaciones complejas mediante la
dindmica de sistemas. Editorial Universidad de Talca
Schaffernicht, M. and Grésser, S. 2009. What’s inside a mental model of a
dynamic system? Conceptual structure and model comparison? —
Proceedings of the 27th Internatoinal Conference of the System
Dynamics Society, Albuquerque, July 2009
Schaffernicht, M., Quiroz, J. and Aravena, C. 2008. Hacia el uso de los bucles
de retroalimentacion en el disefo instruccional — el caso de MacroLab;
proceedings of the 6th Anual meeting of the Latinamerican Chapter of
the System Dynamics Society, Santiago de Chile, October 2008
Sterman, J. 2000. Business Dynamcis — systems thinking and modeling for a
complex world, Irwing McGraw Hill
Wheat, David. 2007a. The Feedback Method - A System Dynamics Approach
to Teaching Macroeconomics, Doctoral dissertation, University of
Bergen, March 2007.
Wheat, David. 2007b. The Feedback Method of Teaching Macroeconomics: Is
It Effective? System Dynamics Review, 23(4), Winter 2007, p. 391-413.
21
Appendix 1: the “distance ratio” method
The DR is a number that expresses how different one model is from another
one, based upon the comparison between the two sets of variables and causal
links. It varies from 0 (identical models) to 1 (no shared variable or link). In
order to compare two mental models, each of them is represented as
association matrix, A and B respectively, where each of the model’s variables
is arow and a column. Rows will be numbered from | to p using an index i
and columns from 1 to p using index j. Each variable is assigned a row and
column with a specific number and i=j. If variables x and y are located at row
r and column c respectively, possible links between them will appear in cells
a,- and b,,. Links from a variable x to a variable y are denoted as “1” for
positive polarity and “-1” for negative polarity; “0” means “no link”. So if a,
= | and b,,=-1, it means that in the model A, there is a positive link from x to
y, while in model B the link from x to y is negative.
We will us p to denote the total number of possible nodes; P, is the set of
common nodes in A and B, and p, is the number of common nodes. P,, is the
number of nodes unique to A and P,, the number of nodes unique in B. Ny
and Nz, are the sets of nodes in the two models.
The complete formula of the distance ratio is then
PP
YS dMaif GD
DR(A,B) = : os
(B+ yp. +7 2P.(Pus + Pus) + Pus + Pus) ~ CEB + 6) P+ (Pua + Pus)
diff(i,j) =
- 0 ifi=jando=1;
— Taj, by) if either i or j € P, andij € Naori,j € Np;
lay — byl +8 if ay * by <0;
lay — bj! otherwise.
Taj, bj) =
- 0 ify=0;
— 0 ify= land aj=bj=0
— 1 otherwise
- 0 ify=0;
— 1 otherwise
The parameter f represents the highest possible link strength, which is 1 in
our case; thus f is replaced by 1. It follows that 6 — which would give
different importance to polarity change according to the strength of links
involved — will be set to 0 (so that nothing will be added to the difference).
Since we are not interested in analyzing models where a variable influences
itself directly (a “self-loop”), @ = 1. The parameter € is the number of
possible polarities, which must be 2 in our case. The last parameter, 7, is a
22
little more complicated. In two models, a potential link may be absent
because the subject believes there is no causal link between the two variables,
or one (or both) of the variables is not part of the model. If this is taken to
mean something different, then y= 2, which is our case. When substituting
the chosen values into the parameters, the equation transforms into
Yai.)
DR(A, B) = ae
2p) +127, (Pus + Pun) + Pra + Pin) ~ (2D. + UP, + Pu)
Since the method takes into account variables and causal links with positive
or negative polarity, it can in principle be used to compare system dynamics
oriented mental models.
Appendix 2: the table of loop membership
Variables [Loops
Tl2] 374] se] 7] 8] s]w] [2] w]e
Tap Socal Transfer pms %e of Budget
Dinise vise ini 1
aC consumplion (namin)
Tore Propensity to consume
Bate net change of FF target rate tft
6K capital
TRDSS desired capital
@ PoB purchase of bonds 1
9 GPa ‘government purchases tpa|s
TO Ke capital cost
TOPS Unit production cost
12 GDet government defict
ERG ‘aggregate demand tfala
14 Der ‘expected real demand
15 dpst deposits 1 tpaft aa
16 KDpr capital depreciation
17 GDar government debt
18 dwn dividends (nominal) i
19 deP demand effect on price
20 reo Tntorest rate effect on consumplion
Bi oo” cost affect on price
22 eS interest elasticity of savings
2a ec interest elasticity of consumption
Banco et capital entry (US)
25 opSip operating surplus tpafa
26 exRsw excess resorves taht
27 ExpNNAW. ‘Ret nominal Ex (AW)
28 ExpNN Exports (we) net nominal
20 prac, production factors
30. wFre worktoree
EKG government spending
32 Inv investment
33. Imp, im real US)
3a Tax Taxes rpaft
35 Taxi Taxes business iia
36 TaxHh Taxes households i
EA FF target rate indicator 7
36 PAW, rice level (RW)
39-@t ‘exractime index
40RW revenues (AW)
A Esp minal revenue fom Ex US)
aL demand for reserves (iquidty demand) ta
ran resverves (monoy offer) nn
aN ‘employment
a5 FF target rate ta
46? pice level
ap Toan payments i
48 pimp payments for im (US)
48 spc salaries % of capital
50 108 revenue % of salaries
51 wap working age pupulation|
52 dp aividends pot
53 1pHh personal tax pet
Ea business Tax pet
55 Gb {government borrowing
55 GEG government budget tfala
37 mip) multifactor productiy
58 Ya production
59 mps, mean propersiy to consume
60-epRat cost productivity ratio
61 opel cost productivity relation
628 disposable income business ta
63 her Tong term expected results
Gi wd. withdrawals 7 i 7
65 a income / disposable come
66 usr Uunemployment rate
Cal Tnferest rate ae
68 nih net lending rate 1 1 1
or reserves rate
Tonge hot deposit rate t Wa
Th mr mean bond rale 1
Tet complance time
Too ‘exchange rate
TaN desired labor
75 aN, ‘fective labor
769 Transfer payments nominal
77-uclinp unit cost at impors
Test nominal) sales revenue tfata
795 savings (W)
Ww nominal salary
81 Impvw WT of Imp volume (USA)
Appendix 3: constructing MacroLab’s reachability matrix
The author is not in possession of the Matlab used by Oliva(2004), nor did he
have previous exposition to the use of graph theory and its software tools.
Accordingly, the steps have been carried out by simpler means, which can be
replicated by any person, using MS Excel and GNU Octave, a software tool
available at http://www.gnu.org/software/octave/download.html.
The first step is to create the adjacency matrix A in an Excel spreadsheet (see
additional material of the paper). Then a text-only file is generated for import
into Octave. The format is as follows:
# Created by Octave 3.0.3, <date> <unknown@unknown>
# mame: {name of the variable that stocks the matrix;
in the example, I use “M”}
# type: matrix
# rows: 81
# columns: 81
57 57 57 57 0 0 57 0 57 57 57 0 57 0 57 57 57 57 57
57.57 57 57 0 57 57 57.57 0 0 0 57 57 57 57 57 57 57
57.57 0 57 57 57 0 0 57 57 57 57 57 57 57 57 57 57 57
0 57:0 0 57 58 0 57 0 57 57 57 57 0 00 0 0 57 57 57
0 57 57
Note that the cells of the matrix appear without row or column header. The
values are separated by a BLANK character. An easy way to convert the
Excel cells into this format is:
1. copy the cells
2. special-paste “without format” into a MS Word document to make
cells appear with values separated by a TAB character.
3. search-replace all TABs by a BLANK
hand insert a BLANK before the first value
5. save as text only with a “.mat” extension into the Octave folder on
your computer. For instance “myMatrix.mat”
Then you create a new variable in octave:
load myMatrix.mat M
MM=M*M
save myMatrix.mat MM
Now you have to import the contents of the “.mat” file back into a new sheet
in Excel. Then generate a sheet with the same format and — for each cell —
convert the values to binary values (because this must be a_ binary
multiplication) by EXCEL’s built-in min(1;{value}) function. The last step is
to compare the current matrix with the previous one. Copy the structure to
the right side of the matrix and a formula like “IF({current value }={ previous
value};0;1)”: this leaves a 1 for each cell that has a different value from its
previous version. Now sum up all the cells: once the result is “0”, the
previous version of the matrix corresponds to the reachability matrix. If the
result is greater than 0, start over the whole cycle. The files
“matrices_all.xls” and ‘“‘matrices_A_R.xls” in the additional material of the
paper contain the data.
The last sheet of “‘matrices_A_R.xls” has auto-filters set for the variables. If
one defines the filter of variable v to “1”, this hides all rows representing
25
variables from which one cannot reach v. It follows that by adequate
definition of the filters, one can precisely find the set of variables that would
be convenient for a desired scenario.
26
