Group Learning Supported by Simulation model an Experiment
Design
Mirjana Kljajié BorStnar, Andrej Skraba, Miroljub Kljajié, Davorin Kofjaé
University of Maribor, Faculty of Organizational Sciences
Kidriceva 55a, SI — 4000 Kranj, Slovenia
Tel: + 386-4-237-4236, Fax: + 386-4-237-4299
e-mail: {mirjana.kljajic, andrej.skraba, miroljub.kljajic, davorin.kofjac} @fov.uni-mb.si
Abstract
This paper addresses the influence of individual and group information feedback on a
decision process supported by the application of system dynamics model. The experiment
considered the task of strategy determination with an explicitly defined criteria function
under three experimental conditions: a;) individual strategy determination supported by
just Causal Loop Diagram (CLD) explanation, a2) determination of strategy with
application of the system dynamics (SD) model without group interaction, and a3)
determination of strategy with application of the SD model with restricted subject
interaction supported by group information feedback. The hypothesis that model
application and group feedback information positively influence the convergence of the
decision process and contribute to higher criteria function values was confirmed.
However, the difference of frequency of simulation runs suggested that group membership
might have affected the group work. To eliminate this possibility, we have introduced a
pseudo-Solomon experimental design. A model of learning was developed as well.
Keywords: group decision, system dynamics, simulation, feedback, experiment design
(Solomon)
1. Introduction
Decision processes in contemporary enterprises are primarily based on the participating
subjects. Decisions generated in organizational systems are, therefore, not dependent on the
individual decision of a subject but rather on a group of experts working in a specific field.
The group better understands the considered system and provides synergistic effects (Hale,
1997). Their interaction in the process of problem solving (decision-making) supported by
advanced group support tools and interactive business simulators could enable more effective
individual and group analyses of the problem (Vennix, 1996; Richardson and Andersen, 1995;
Kwok and Khalifa, 1998; Langley and Morecroft, 2003, Skraba et al. 2003). Quality decisions
can be made only if the decision group has the appropriate information: both feedback and
anticipative. This assumes knowledge of a model of a system, criteria function and the state of
nature. These were intensively discussed in the literature (Chekland, 1994; Forrester, 1973;
Rosen, 1985; Simon, 1997; Sterman, 2000). The ideal of learning organizations can be
approached by application of SD models (Warren in Langley, 1999). Use of SD models for
testing the vision of evolution of business systems is widely used (Forester, 1973; Simon,
1997; Sterman, 2000). However, the interconnection of SD models with group support
systems (GSS) for the purpose of decision-making support is not commonly used and
researched. An interesting model intended to explain group learning phenomena was
described in (Lizeo, 2005), where the group learning process was modeled from structural,
interpersonal and cognitive factors in the form of a causal loop diagram (CLD) and SD
technique. Experiential learning as learning from the enterprise simulation is researched in the
experiment of Gopinath and Sawyer (1999), where effects of learning during determination of
broader business strategy on a business simulator was examined. Application of SD models
for strategy determination encourages strategic decision-making and systematic work. In the
experiment with the global oil microworld computer of Langley and Morecroft (2004), they
explore the effects of various types of feedback on the individual learning (outcome feedback
and structure feedback). Results suggest that structure feedback positively influences the
understanding of the problem and time for the task completion.
However, in complex systems, to make a formal experiment to prove that efficacy and
usefulness of group decision and using simulation model for decision assessment is a
demanding task. There are problems of validity in the design of the research (Chun and Park,
1998). It is difficult to create a laboratory environment in which subjects are motivated to
creatively participate in finding the solution as they would in a real world. The dilemma is
also in planning of the problem (organizational systems), which is inherently complex.
Further, it is not merely important that the problem is logically correct, but how the
information is framed (Kahneman and Tversky, 1979). There is also a problem of user
interface layout, as it affects the effectiveness of the subject in the process of problem solving
(Howie et al., 2000).
Three learning methods (case learning, simulation method, and action learning) were
researched in Jennings (2002). The participants rated the simulation method as superior to the
action learning and case learning methods. In the paper (Skraba et al., 2003) the process of
strategy determination was described as well as the impact of group interaction on subject
performance by applying the SD model of simplified business process. The hypothesis that
model application and group feedback information positively influence the convergence of the
decision process and contribute to higher criteria function values was confirmed. The
experiment was enhanced with the new group in order to analyze criteria function as well as
frequency distribution of members in using simulation model during searching for optimal
parameters. The goal of the repeated experiment was to acquire knowledge of the decision
process supported with the SD model and the influence of group feedback information.
Although the result of criteria function was similar as in previous experiments, it was
surprising that the frequency distribution among experimental groups was different at the
beginning of the experiment. The difference of frequency of simulation runs suggested that
group membership might have affected the group work.
This paper addresses the influence of feedback information on the group decision process
supported by the application of system dynamics models. The model of learning due to group
information is developed as well. Additionally, a pseudo-Solomon experimental design will
be presented in order to eliminate or confirm the effect of group membership on criteria
function and frequency distribution.
2. Method
2.1 Simulation Model
Figure | shows the model of the production process as a black box with input parameters 1},
T2, Y3 and r4 (where r is Product Price, r2 Salary, r; Marketing Costs and r4 Desired
Inventory) and criteria function J as the output under the experimental conditions a;), a2) and
a3). The task of the participants is to find the parameter values r; in order to maximize the
criteria function. The experiment was conducted under three experimental conditions: a1)
determination of strategy on the basis of a subjective judgment of the task, az) determination
of strategy with the application of a system dynamics model without group interaction, and
a3) determination of strategy with the application of a formal model with subject interaction
supported by group feedback information.
Ay, Ay, As
h
hb
+ __-|
b
ta
——
Decision
makers
Si'Sx nSh
Figure1. Business model with input parameters under different experimental conditions
The model m developed by the SD method, which was used in the experiment, described in
(Skraba et al., 2003) is shown in Figure 2. The model consists of: production; workforce and
marketing segments, which are well known in literature (Forrester 1973; Hines 1996; Sterman
2000). It was stated that product price (U1) positively influences income. However, as prices
increase, demand decreases below the level it would otherwise have been. Therefore, the
proper pricing that customers would accept can be determined. If marketing costs (U3)
increase, demand increases above what it would have been as a result of marketing
campaigns. The production system must provide the proper inventory level to cover the
demand, which is achieved with the proper determination of the desired inventory value (Us).
Surplus inventory creates unwanted costs due to warehousing; therefore, these costs have to
be considered. The number of workers employed is dependent on the production volume and
workforce productivity, which is stimulated through salaries (U2). Proper stimulation should
provide reasonable productivity.
Inventory
Production
+ +
t ae
‘Ya Inventory
Gap
ae a +
Ship ping Productivity )
+
Desired
Workforce Production
RR Desired *,
Workforce
+
Figure 2: Causal Loop Diagram of Production Model
Participants had the task of promoting a product, which had a one-year life cycle, on the
market. They had to find the proper values of parameters u; defined in the interval Umin < Ui <
Umax. The model was prepared in the form of a business simulator (Skraba et al., 2003). The
participants changed the parameter values via a user interface, which incorporated sliders and
input fields for adjusting the values. After setting the parameters in the control panel, the
simulation could be processed. The end time of simulation was set to twelve months. Output
was shown on graphs representing the dynamic response of the system and in the form of a
table where numerical values could be observed. Each participant had no limitations of
simulation runs, which he/she intended to execute within the time frame of the experiment.
The parameter values for each simulation run were set only once, at the start of the
simulation. It was assumed that the business plan was made for one year ahead. The criteria
function was stated as the sum of several ratios, which were easily understood and known to
the participants. It was determined that Capital Return Ratio (CRR) and Overall Effectiveness
Ratio (OER) should be maximized at minimal Workforce and Inventory costs determined by a
Workforce Effectiveness Ratio (WER) and Inventory / Income Ratio (IIR). The simulator
enabled simultaneous observation of the system response for all variables stated by the criteria
function during the experiment. The criteria function was dependent on the chosen values of
parameters and stated as:
d+Ydt) p+ Y pt) s+ Yoh) w+ dvb)
i=0 i=0 i=0 i=0
J = w+ r Ww, — i Ww; — W,
by k ty
. +d ot) pr+dipty) Py+>d pct)
i=0 i=0 i=0
: Ql)
where d, is the initial value of Income, d(t,) the Income function where d(t,) = p(t,)—o(t;),
p(t,) the Revenue function, o(t,) the Expenses function, t, the final time of observation, ¢
Capital, p, the initial value of Revenues, 0, the initial Expenses, s, the initial Workforce
Expenses, s(t,) the Workforce Expense function, v, the initial Inventory costs and v(t,) the
Inventory costs function. The weight values were prescribed as: w,= 0.5, w,= 0.35, w, = 0.1
and w, = 0.05. The goal of the participants was to maximize the criteria function in Equation
1;
2.2 Subjects and Procedure
In our experiment, 146 senior graduate students (86 female and 60 male) from the University
of Maribor participated in the experiment in order to meet the requirements of their regular
syllabus. The students were randomly assigned to nine groups, which were then assigned to
work at one of the three experimental conditions: aj, a2 and a3. The subjects who participated
in the experiment became accustomed to the business management role facing the stated goal
objective, which was in our case presented in the form of criteria function. The presentation of
the decision problem was prepared in the form of an electronic presentation where the model
and the task were explained. A printed version of a problem description was provided for each
subject. The structure of the considered system was presented and the main parameters of the
model were explained. The evaluation criteria for the business strategies were also
considered. The work with the simulator was explained for experimental conditions a2 and a3.
The participating subjects were familiar with SD simulators; therefore, working with the
simulator was not a technical problem. The participants formed a strategy according to the
stated problem and passed their decisions to the network server or filled in a paper form in the
case of the a; experimental condition. The search for the best parameter values was conducted
under three experimental conditions:
a;) Individual choice of parameter using CLD of the problem
Experimental condition a;) assumed the individual assessment of the decision-maker
supported only with CLD of the model as shown in Figure 2, paper and pen. It was assumed
that the subjects would find the best business parameter values {T1, 2, 3, 14} on the basis of
intuitive judgment. At the end of the experiment, the subjects recorded the best parameter
values on the form provided.
ay) Individual optimization using simulation model
Experimental condition a2) assumed the individual assessment of the decision-maker when
determining the model parameters values {I, f2, 13, T4} by maximization of the criteria
function in equation (1), using the SD model. At the end of the experiment, the subjects
submitted the best-achieved parameter values to the network server.
a3) Optimization using group feedback information
Experimental condition a3) assumed the application of the SD model by the participants with
group feedback information. The time of conducting the experiment under this condition was
divided into four time intervals, 8+8+8+6 minutes. Each participant submitted the best-
achieved set of parameter values {T), 2, 13, 4} to the network server at the end of each time
interval. Information about the best-achieved parameter values was fed back into the group
support system. The participants got feedback on the defined strategies of all the participants
in the group Ri = {f, To, Ts, Ta}; 1 = 1, 2,...n as well as the aggregated values in the form of
parameter mean values (1.0.5.0). For example, if the considered parameter was Product
Price and there were ten participants involved in the decision process, then all ten values for
Product Price, recognized as the best by each participant, were mediated via feedback as well
as the mean value of Product Price. Mean value provided the orientation for the parameter
search and prevented information overload. In addition to criteria function as the results of
decision making at different condition, simulation frequency in order to follow decision
maker activity was also analyzed.
Formally, the experiment under conditions a), az and a3 is summarized in Table 1, where J and
Oj represent values of observed criteira function at intervals gn minute, 16" minute, 24"
minute, 30" minute in both groups with individual feedback and group feedback. In this case,
it means that each participant had to send the selected parameter values to the network server
in the prescribed time intervals while their simulation activity continued and analyzed after
experiment. Variable F (frequency of simulation runs) represents every simulation run
(combination of four parameter values forming the business strategy) recorded for each
participant in a second time for the duration of experiment.
Treatment Observed Pretest Posttest
variable
ay J Osomin
a2 J,F Osmin Ortomin Oramin Osomin
a3 J,F Osmin Oromin Ooamin Osomin
Note: J - Criteria function values; F - number of simulation runs
Table 1: Pretest — post-test experimental design of comparing criteria J of Groups ap, a3
Table | presents the experimental design which assumes one group working individually
without the simulator and two groups working under two conditions supported by the
simulator (one individually and the other supported by simulator and group information
feedback). Members of two groups had to turn in their decisions after 8, 16, 24 and finally
after 30 minutes. Members of the first group had to present their decisions only once, after 30
minutes. During the first 8 minutes of working with the simulator, the same technical
conditions were in force for both groups (individual exploration supported by the SD model).
After they turned in their 8-minute results, group az continued working in the same manner
and group a3 received the group information feedback, therefore such design can be
interpreted as a pretest — post-test experiment.
We stated the following Hypothesis:
Hypothesis 1. Individual decision-making supported by the simulation model yields higher
values of criteria function than Individual decision-making without the simulation model.
Hypothesis 2. Individual decision-making supported by the simulation model and group
feedback information yields higher values of criteria function than Individual decision-
making supported only by the simulation model.
Hypothesis 3. There is no significant difference among the criteria function values obtained
by the groups a2 and a; after the first eight minutes of the experiment (pretest).
3. Results and discussion
A total of 146 students (86 female, 60 male) randomly assigned into 9 groups of 14 to 15
subjects participated in the experiment; 30 students (two groups) participated in the condition
aj, 58 (four groups) participated in the condition a2, and 58 (four groups) participated in the
experimental condition a3. For the purpose of results analysis, the criteria function was
optimized by Powersim Solver™ using two methods: incremental and genetic algorithms.
The optimal value of the criteria function was thus set to 1,5. The highest values of criteria
function were selected by the participants of group a3 (Sas =1,1676, o,; =0,34205,
J min,, =0,01, J max,, =1,49), followed by the results of the group a (J, =0,9832,
0) = 0,37135, J min,, =—0,29, J max,, =1,48) and the lowest results were gathered by
the group a; supported by just paper and pen Sa = 0,3735, o,, =0,59257, J min,, =—1,42,
J max,, =1,29). Criteria function values selected by the participants working at three
different conditions after 30 minutes of experiment time are presented in Figure 3. On the X-
axis the relative number of participants is shown and on Y-axis the values of criteria function
arranged from highest to lowest is shown.
Criteria function value under three experimental conditions
15
12
0,9
0,6
0,3
0
-0,3
-0,6
-0,9
-1,2
-15
Criteria function (J )
Relative No. of participants
—s—a2 ~—a3 —~al
Figure 3: Values of Criteria Function achieved by the participants under experimental
conditions: a; (individual decision-making supported by CLD), ap (individual decision-
making supported by the simulation model), and a3 (decision-making supported by the
individual use of the simulation model and group feedback information).
Hypothesis 1, stating that individual decision-making supported by the simulation model
yields higher values of criteria function than individual decision-making without the
simulation model, was confirmed by the U-test (Mann Whitney) with a p-level of 0,0000.
Hypothesis 2, stating that individual decision-making supported by the simulation model and
group feedback information yields higher values of criteria function than individual decision-
making supported by the simulation model only, was confirmed by the U-test (Mann
Whitney) with a p-level of 0,002.
Hypothesis 3, that there is no significant difference among the criteria function values
obtained by the groups a2 and a3 after the first 8 minutes of the experiment (pretest), was
rejected by the U-test (Mann Whitney) with a p-level of 0,0002. This means the difference in
criteria function values cannot be explained by random events.
We had expected the results of the two groups’ after the first 8 minutes to be similar, as they
had same technical conditions. We have examined the homogeneity of groups and eliminated
all possible sources of disturbances (randomization, motivation of participants to actively
cooperate, anonymity). Nevertheless, the group that had expected to share their views after
the first observation time (8 minutes) might have been motivated by that factor.
For the purpose of further analysis of dynamics of problem solving of Groups az and a3, we
have recorded every simulation run performed by each subject of the two groups during the
experiment at a sampling frequency of one second. The cumulative frequencies of the
simulation runs during the 30 minutes experiment time of Groups az and a3 are presented in
Figure 4. On the Y-axis, the frequency of testing is presented as the ratio of cumulative
frequency of simulation runs and time in seconds. On the X-axis, the time frame of the
experiment is presented in seconds, highlighting the observation times at 8°, 16" and 24"
minute.
Dynamics of testing Frequency
Frequency [1/s]
°
roe)
™.,
0 8min 16min 24min
Time [min]
——a2 ——a3
Figure 4: Cumulative frequencies of simulation runs over 30 minutes of experiment time of
Group a2 and Group a3
We can observe from Figure 4 that the frequencies of Group a3 are much higher than the
frequencies of simulation runs of Group az in the first 8 minutes of the experiment time. After
the first eight minutes, the frequency of experimentation on the simulator decreases. This is
consistent with the time of passing the selected parameter values to the network server. We
can observe such interruptions after the 16" and 24" minutes as well. The frequency of
simulation runs of Group a3 slows down after they receive the group feedback information
(after the first eight minutes). But the frequencies of simulation runs of Group az are
increasing until the end of experiment. At the end of observation time, the two groups have
performed almost the same number of simulation runs (Noa2=2925, Noa3=2930), but Group
a3 performed significantly better than Group a2 in achieving greater values of criteria function.
This is shown in Figure 5, where Y-axis represents the ratio of average value of criteria
function and cumulative frequency in time (seconds) and X-axis represents timeframe of the
experiment in seconds.
Dynamics of Average Criteria Function Value
Average Criteria Faunction Value
2
wu
0 8min 16min 24min
Time [min]
——AVG J_a2 —AVGJ_a3
Figure 5: dynamics of average criteria function values within 30 minutes of experiment time
of Groups az (individual decision-making supported by simulation model), and Group a3
(decision-making supported by simulation model and group feedback information)
Two curves in Figure 5 represent the average value of criteria function of Groups a2 and a3
during the experiment time. The two curves are increasing almost exponentially until they
reach a steady state approaching the optimum value. The average value of criteria function
achieved by Group a2 is lower than that achieved by Group a3 from the start of the
experiment. The disturbance which we can observe at the beginning of the experiment can be
accounted for familiarizing with the simulator. The interruptions at each observation time
(after 8", 16" and 24" minute) are notable as well.
Hypotheses | and 2 were stated to test the effect of information on problem solving. The
results of decision making at three experimental conditions are shown in Figure 3. It is seen
that Hypotheses 1 and 2, which test the effect of information on problem solving, are
confirmed at the level of significance p=0.00. However, Hypothesis 3, which refers to group
homogeneity, was rejected. It is shown in Figures 4 and 5 that in the first 8 minutes both the
criteria function and frequency of simulation activity of participants of Group a3 are higher.
After that time, the criteria function stays permanently higher in Group a3, as was expected,
while the activity (frequency) of the subjects in both groups converges to the equal stationary
value.
3.1 Solomon Four-group Experimental Design
Although Hypotheses 1 and 2 were proved at the end of experiment, the main task of
optimization was found in the fact that we cannot accept Hypothesis 3. We expected, due to
the homogeneity of population and its random selection into groups, that the results of criteria
function and frequency of testing in the first 8 minutes would be identical. However, from the
time course of variables on Figures 4 and 5 a difference was noted. This phenomenon cannot
be explained by the pretest - post-test experiment. Therefore, we plan a new experiment
according to Solomon Four-group Experimental Design. We expect to estimate the effect of
group belonging and pretest effect on the results using this test. Solomon’s design for the
suggested experiment is shown in Table 2.
Treatment Observed Pretest Posttest
variables
ag J,F Osmin Or6min Ooamin Osomin
a3 J,F Osmin Otomin O2amin O30min
ag J,F é Osomin
as J,F = Osomin
Note: J - Criteria function values; F - number of simulation runs
Table 2: Solomon four group experiment design
The first two groups in Table 2 represent the pretest - post-test design described earlier. The
last two groups of experiment will solve their task as the first two groups do but without
sending the selected parameter values every 8 minutes. They will work on their tasks without
interruptions for the whole observed time (30 minutes) and send their selected parameter
values at the end of experiment. All measurements will be automatic, which means theat
every simulation run will be recorded in the database in a second time thus the information
feedback about the group members’ decisions will be available at all times. In this manner it
would be possible to estimate effect of result sampling every 8 minutes, as well as the factors
group belonging or motivation versus group information. For that purpose, a new interface for
data acquisition and proceeding has been developed.
In order to explain the influence of individual feedback and group information feedback using
simulation model on efficacy on problem solving, we developed a CLD model of learning
during decision-making. The model shown in Figure 6 was modified according to (Lizeo,
2005) and consists of three B and one R loops. Loop B1 represents decision problem solving
only using CLD from Figure 2 and could associate with experimental condition a;. Loop B2
represents individual problem solving using simulations of the problem and corresponds to
experimental conditions of a2. Loop B3 represents direct contribution of group information
while loop R suggest reinforcing effects of group influence on problem solving at Group a3.
Loop R could probably explain phenomena regarding hypothesis 3. This effect could be
deduced from the proposed Solomon’s four group experiment design.
+
Group
Performance
+
@.
+
Motivation a
to Learn Individual
Ss # Learning with
Simulator
£2.)
Satisfaction
Proper Effort in
Choice of Problem -
Parameters wndersanaing
“Performance
oO
Goal
Figure 6: Learning model of decision group, with the feedback information obtained from
simulation model
4. Conclusions
In this paper, we have discussed the influence of individual information feedback obtained by
the simulation model and group information feedback on a decision-making process during an
experiment with decision groups. A system enabling interactive work with a business
simulator and restricted group interaction was developed; it was conducted with 146
participants under three experimental conditions: a;) individual assessment supported by CLD
explanation, a2) individual assessment supported by the simulator and a3) assessment
supported by the simulator and group information feedback. Two dependent variables were
observed in the experiment: criteria function (J) and frequency of simulation runs (F). The
hypothesis, that groups supported by the simulator achieved higher values of criteria function
than groups supported merely by the explanation of the CLD, was confirmed. Also confirmed
was the hypothesis that the group supported by the simulator and group feedback information
achieved better results than the group supported only by simulator. However, the hypothesis
that the results of groups a2 and a; didn’t differ in the first 8 minutes of work, when they had
identical conditions, had to be rejected. Further analysis of the dynamics of business strategy
findings had revealed that the group expecting to share their results after the first 8 minutes
performed almost twice as many simulation runs as did the group working with the simulation
model alone. For these differences that cannot be explained by the current experiment design,
we propose a Solomon four group experiment design. The CLD model of group learning in
decision problem solving using simulation models has been proposed with regard to new
experiment design in order to explain Hypothesis 3. The research is in the progress.
5. Acknowledgments
This research was funded by the Ministry of Education, Science and Sport of the Republic of
Slovenia; Program No. PP-0586-501 and Grant No. Z5-3313.
6. References
Hale R, Whitman P. 1997. Practical problem solving & decision making. Kogan Page.
Vennix JAM. 1996. Group model building: facilitating team learning using system dynamics.
Wiley: Chichester.
Richardson GP, Andersen DF. 1995. Teamwork in group model building. System Dynamics
Review 11: 113-137.
Kwok RCW, Khalifa M. 1998. Effect of GSS on knowledge acquisition. Information &
Management 34: 307-315.
Skraba A, Kljaji¢ M, Leskovar R. 2003. Group exploration of system dynamics models — Is
there a place for a feedback loop in the decision process?. System Dynamics Review. 19: 243-
263.
Checkland PB, Haynes MG. 1994. Varietes of systems thinking: the case of soft systems
methodology. System dynamics review. Vol.10, No. 2-3: 189-198.
Forrester JW. 1973. Industrial Dynamics. MIT Press. Cambridge. MA.
Rosen R. 1985. Anticipatory Systems. Pergamon Press.
Simon H. 1997. Models of Bounded Rationality: Empirically grounded Economic Reason.
Vol. 3. MIT.
Warren K, Langley P. 1999. The effective communication of system dynamics to improve
insight and learning in management education. Journal of Operational Research Society 50:
396-404.
Lizeo E. 2005. A dynamic model of group learning and effectiveness. Conference
proceedings, The 23" International Conference of the System Dynamics Society. Boston.
Gopinath C, Sawyer JE. 1999. Exploring the learning from an enterprise simulation. J ournal
of Management Development. Vol. 18. No. 5: 477-489.
Langley PA, Morecroft JDW. 2004. Performance and learning in a simulation of oil industry
dynamics. European Journal of Operational Research 155: 715-732.
Chun KJ, Park HK. 1998. Examining the conflicting results of GDSS research. Information &
Management 33: 313-325.
Kahneman D, Tversky A. 1979. Prospect theory: An analysis of decision under risk.
Econometrica 47: 263-291.
Howie E, Sy S, Ford L, Vicente KJ. 2000. Human-computer interface design can reduce
misperceptions of feedback. System Dynamics Review 16: 151-171.
Senge PM. 1990. The Fifth Discipline: The Art and Practice of The Learning Organization.
Doubleday/Currency. New York.
Jennings D. 2002. Strategic management: an evaluation of the use of three learning methods.
Journal of Management Development, Vol. 21 No. 9: 655-665.
Hines J. 1996. Molecules of Structure, Version 1.1, Building Blocks for System Dynamics
Models. LeapTec and Ventana Systems, Inc., USA.
Sterman JD. 1994. Learning in and about complex systems. System Dynamics Review 10:
291-330.
Sterman JD. 2000. Business Dynamics: Systems Thinking and Modeling for a Complex
World. Irwin/McGraw-Hill: Boston MA.
