OUR TEN YEARS OF WORK ON TRANSPARENT BOX BUSINESS
SIMULATION
José Antonio Dominguez Machuca* (jmachuca@ cica.es)
Jose Carlos Ruiz del Castillo*
Miguel Angel Domingo Carrillo*
Maria del Mar Gonzalez Zamora*
*G.LD.E.A.O. (Research Group on Computer Aided Business Management)
Dpto. Economtfa Financiera y Direccién de Operaciones
Universidad de Sevilla
Avda. Ramon y Cajal, 1, 41018 Sevilla, Spain
ABSTRACT
Traditional business games are of the so-called black-box type (BBBS=Black box business
simulator); that is to say, the intemal structure which generates the results of the simulation after
decision-making is not known. As a result, the player normally operates by trial and error and bases
his decisions on the symptoms of the problem (the observed behaviors of the system's variables)
and not on the real causes of the problem (the system's structure). Since 1988 José A.D. Machuca
has insisted that the business games based on System Dynamics models should be Transparent-box
business simulators (TBBSs). That means that, during the game, the user has access to the structure
of the underlying model and is able to relate it to the observed behaviors. The hypothesis is that
such transparency would facilitate causal reflection and favor systemic learning of business
problems.
In 1990, the G.I.D.E.A.O. Research Group took action on this idea and centered one of its
lines of research on this matter, with three main objectives: a) Creation of TBBSs, b) Introduction
of TBBSs in undergraduate and graduate Management courses as well as in executive training, c)
Experimentation in controlled environments in order to test the hypothesis mentioned in the above
paragraph. Now, ten years after the birth of the idea, we would like to share in this paper the results
obtained during that period.
1. About the need for transparency in Business Simulators (J.A.D.
Machuca)
As is well known, the internal structure of a system is the main cause of the behavior
observed in it. Consequently, an awareness of this structure helps understand the reasons
behind this behavior and, in principle, should favor decision-making and an understanding
of the phenomena observed, as well as greater knowledge of the system under study. My
conviction of the importance of the use of the causal structure together with the observed
behaviors as a vehicle for reflection aimed at a closer understanding of systems and for
easier decision-making is as firm as it is long-standing.
For this reason, as I stated in a previous paper (Machuca et al., 1993, p.298), since
1988 I have been criticizing the use of black-box business simulators (BBBSs) and
defending the creation of transparent-box business simulators (TBBSs), although my first
publication on this topic appeared in 1990 (Machuca and Roman, 1990). Transparency
refers to the possibility of relating the system’s structure to its behaviour when using a
Business Simulator. To facilitate transparency, the qualitative structure (for example, in the
form of a causal-loop diagram) and even the main equations of the model on which it is
based, could be at all times accessible in the computer to the user, who would therefore
find it easier to make decisions based on previous study of the possible causes of the
different behaviors of the variables, and not only on observation of the latter, which are
simply effects. This possibility, however, is not offered by black-box games, which only
allow access to the results that can be observed, that is, the effects or symptoms, but not
the causal structure giving rise to them. With black-box games a trial-and-error procedure
is usually followed, based on the mental models each player has of the case being
examined. An added problem is that these mental models differ from one person to
another, depending on his or her particular perception of the reality in question, and this
generally entails serious communication problems (Machuca et al. [1993, p.301]; Sterman
[1994, p.295]). This is one of the reasons why it is difficult to reach agreement in the
debate on real problems relating to the functioning and behaviour of complex systems.
The problem is further aggravated by the fact that these mental models are not made
explicit by those taking part in this discussion, who only know their own mental model and
assume that it is shared by the others. The consequences of this phenomenon can be
observed when the case method is applied or when attending a meeting of the board of
directors of any company. As a result, the mental model which the user of a business game
has formed of the sytstem represented by it may well not coincide fully with that of the
game’s creator. The player may therefore be making decisions based on a (simulated)
reality different from the one he has formed in his own mind, thus facilitating neither
decision-making nor correct leaning (Machuca et al., 1993, p.301). All this leads me to the
hypothesis that by using a transparent simulation, the learning process and acquisition of
a systems approach for decision-making would be improved (Machuca 1991 and 1992).
Although other authors have subsequently touched on this topic, directly or
indirectly1, the idea has scarcely been put into practice in game-design. In my view, one of
the most valuable tools that System Dynamics offers to the development of a systems
dynamics approach was being lost (Machuca, 1992, p. 176).
In 1990 I put my idea into action, starting a line of research I called “Learning
Laboratories in Computer-Aided Systemic Business Management,” which has been and is
1 See for example: Isaacs and Senge (1992, p. 195), Kemeny and Kreutzer (1992, p. 305), Morecroft (1992,
p. 465), Peterson (1992, p. 117), Sy-feng and Y oung (1992, p. 765), Davidsen (1994, p. 305), Langley and
Morecroft (1996, p. 301, 303), Grossler (1997, p. 370).
being sponsored within different complementary frameworks: regional (Junta de
Andalucia— the Government of Southern Spain— as well as private and public companies),
national (CICY T— Interministerial Commission of Science and Technology) and European
(COMETT and Leonardo Projects). The main objective of this line of research was to
create, develop and use TBBSs as well as to verify the hypothesis mentioned above. In my
opinion, TBBSs should facilitate causal reflection and favor systemic learning of social
and business problems, helping to prevent the “video game syndrome” which often arises
in black-box games. I am currently coordinator of a European project researching along
these lines called “New information technologies and transparent business simulators for
improving business competitiveness.” The French company K.B.S and the University of
Bergen are two of the partners of this project.
Other matters which have attracted my attention during the last six years are the
following ones:
* how to promote the expansion of Systems Approach/System Dynamics to a large
number of people in the absence of trainers who are experts in this approach.
* how to be effective in training managers who, being overloaded with work, have
little time at their disposal. This lack of time is particularly a problem if their training
involves moving outside the place of business.
The above considerations led us to move to the creation of TBBSs which greatly
facilitate self-learning and distance learning.
In the following sections, we will briefly comment on some of the results we have
obtained.2
2. A controlled experiment to verify the suitability of TBBSs (Machuca,
J. A. D. and Domingo, M. A.)3.
To verify the above-mentioned hypothesis we started a controlled empirical
experiment in 1994 which lasted three academic years. A secondary aim of the experiment
was to compare the use of business simulators with traditional teaching methods.
2.1. The sample
In order to guarantee the consistency of the results, we wanted to work with a wide
sample, which was comprised of 234 individuals, chosen from a total of 550 applications
received. In addition, every effort was made to make the sample as homogenous as
2 Here we do not comment on a simulator which facilitates distance Jeaming, (a friendly electronic version of
the Beer Game which works over a network and via the Intemet) because it has been already described in the
System Dynamics Review (1997).
3 Partial results of this experiment were already published (see, for example, Machuca et. al. (1995 and 1997).
We would like to thank Dr. J. L. Pérez de los Rios, Assistant Professor of Statistics at the University of
Sevilla, who gave us statistical support and advice.
possible so that the subsequent comparison would not be tainted by factors other than those
whose effects we were attempting to assess. To reach this goal we chose people with the
same academic background: students from the two last years in Business Management
studies at the University of Sevilla. Moreover, the individuals with the best academic
records were selected so as to have people with a similar level of knowledge.
The composition of the initial sample was the following: 54 people in 1994, 81 in
1995, and 99 in 1996, although, in reality, our work in 1994 was taken as a pilot study in
order to refine the questionnaires, as well as the simulator and the procedure to be used.
Thus, the final sample consisted of 180 individuals, who were organized into 60 groups of
three, since we believed that the sinergy produced by the group work would benefit
learning. These groups were subdivided into two sets of 24 and 36 groups; while the first
set worked with BBBSs, the second one worked with TBBSs.
2.2. The tool
We used a medium-complexity simulator representing a firm manufacturing a
single product with a total of six decisions to be made related to the areas of production,
personnel and finance (Machuca et. al., 1992). However, to allow the students to become
familiar with the software, before using this TBBS, all of the groups played a very simple
one in which only two decisions were made.
2.3. The questionnaire
We used a questionnaire covering different objective and subjective points:
a) Objective points (related to the understanding of the system's structure)
a.1) Questions about basic knowledge (questions B1 to B7).
a.2) Questions related to variables connected by relatively short chains of interactions
(questions D1 to D4).
a.3) Questions related to variables connected by longer chains of interactions (questions
11 to 13).
The aim of the above kinds of questions was to verify a new hypothesis: The more
conplex a question is the more useful TBBSs are. All of the questions were rated by us on
a scale from 0 (completely incorrect) to 10 (completely correct), the most commonly-used
kind of scale in our country.
a) Subjective aspects
b.1) Questions linked to the student’s perception of the increase in knowledge they
believe they have attained through the use of the simulator (BBBS or TBBS) in
relation to different topics previously studied through traditional methods of leaming
(questions S1 to S8).
b.2) Questions related to other points (to be compared with traditional learning), such as
the student’s possible increase in motivation or in general knowledge about the
topics under study (questions S9 to S11).
The participants were asked to rate questions S1 to S11 using a scale from 0 (no
increase) to 10 (maximum increase).
2.4. The procedure
As we mentioned, we tried to prevent the results of the experiment from being
tainted by factors other than those whose effects we were attempting to assess. In order to
do so we wanted to control the experiment as much as possible: We tried to make sure that
all of the groups had the most similar conditions possible throughout the experiment,
which had four stages. Thus, they all always played with one or more facilitators, whose
job it was to make sure that the game was played the same at all times. These facilitators
indicated when the decisions had to be made (so that all the players had the same amount
of time); they insisted that the players who were using TBBSs use the transparency offered
by the simulator (which is not a natural tendency, but rather an acquired one); they made
sure that there was no communication among groups; etc.
The stages were the following ones:
a) Grounding in Systems Approach. All the students were initiated into the Systems
Approach, with an emphasis on the differences between this approach and the Analytical
Approach.
b) Use of a very simple simulator. They all played with such a game with the sole aim of
becoming familiar with the software to be used
c) Use of a more conplex simulator. In a session for all the groups, the system which they
were going to simulate was explained in the traditional manner, without going into an
explanation of the causal structure. They had previously been provided with the relevant
written documentation. In a special session for the TBBS groups, the causal structure was
explained to them as well as how to use it to find the causes of the behaviors observed.
They were also provided with the relevant written documentation. In both this session and
the one mentioned above, how the company functioned was discussed and the students
claimed to have understood it. As the gaming sessions unfolded, modifications were
gradually introduced into the games, activating new variables and relationships which
made the structure more complex, and increasing the number of decisions the player could
make; the demand was also changed.
The groups playing with TBBSs had to make use of the causal-loop diagram and, if
they considered it necessary, the principal equations of the model as these two tools are
related to the observed behavior and provide a basis for improving subsequent decisions.
In order to increase the amount of information obtained from the experiment, we did the
following: Each year, once the simulations were finished, we formed a subset from the
total number of BBBS users (5 groups in 1995 and 9 groups in 1996). These students were
given a session similar to that given to the TBBS users regarding the causal structure of the
simulator, the Systems Approach, and so on. Afterwards, they were asked to play twice
more, but with the TBBS version.
2.5. Filling out the questionnaire
After each simulation, every user (those of both the TBBSs and BBBSs)
individually filled out, in the presence of the facilitator(s), the part of the questionnaire on
objective points (see 2.3) related to the simulation carried out. After they were all finished,
they answered the questions on the subjective points (see 2.3, b). The results from this
form can be treated as independent samples, representative of the TBBS and BBBS users
(see Section 2.6).
The students belonging to the subset which shifted from BBBSs to TBBSs filled
out the questionnaires again after playing with the TBBS version. As a result, apart from
the comparison between the TBBS and the BBBS users, based on the results obtained from
the independent samples, we were able to make a different comparison based on the
related samples that we had generated by having some of the BBBS users play with the
TBBS version.
This procedure required that the facilitators dedicate an enormous amount of times;
however, we believe that it was worth the effort if it made the experiment more reliable.
2.6. Statistical analysis of the results obtained
As we Said before, the sample to be taken into consideration was made up of the
students who played in 1995 and 1996.
First, we proceeded to verify the reliability’ of the questionnaire using the
Cronbach’s a coefficient. Various authors allow values between 0.55 and 0.7’. In our
study, the results obtained (see table 1) reflect adecuate interitem consistency.
Samples Independent Related
Questionnaire Objective | Subjective Objective Subjective
aspects aspects aspects aspects
Simulator TBBS and | TBBS and TBBS BBBS TBBS BBBS
BBBS BBBS
Cronbach's a 0,6235 0,7984 0,6163 0,5986 0,6384 0,6324
Table 1: Mesure of interitem consistency of the questionnaire
4 The total net time was aproximately 364 hours for the entire experiment: these hours were shared throughout
three academic years.
5 We would like to thank R. Garcia Sanchez and M. M. Gonzdlez Zamora, members of our research team who
colaborated on this arduous work, for their help.
6 Fora discussion about the relevance of the different types of reliability see, among others: Anasatai (1997),
Brown (1983), Gulliksen (1987).
7 See Nunnally (1978) and Van de Ven and Ferry (1979).
2.6.1. Analysis of the data obtained from the questionnaire regarding the objective
aspects--independent samples.
Given that the results do not follow a normal distribution, we decided to use the
equality of proportions test (Novales, 1997), with which we would obtain an approximate
result, and the Mann-Whitney test.®
a) Equality of proportions test
For each of the questions on the questionnaire about objective points, the number
of students in each of the experimental groups (Px for the TBBS users and Py for the
BBBS users) who answered correctly was added up in order to compare them. The null
hypothesis Ho: Px=Py was compared to the alternative hypothesis H1: Px*Py at a
significance level =0.05. Table 2 shows the results; here, the statistics with a minus sign
(-) indicate that with the TBBSs better results are achieved than with the BBBSs. The next
column indicates whether this value is significant (when P<=0.05).
BBBSs TBBSs
Correct % Correct % Statistics [Significativity (p)
Bl 67 93,1% 103 96,3% -0,9127 0,1807
B2 67 93,1% 96 89,7% 0,7953 0,2132
B3 53 73,6% 83 77,6% -0,6021 0,2736
B4 67 93,1% 107 100,0% -2,3180 0,0102
B5 52 72,2% 97 90,7% -3,0814 0,0010
B6 57 79,2% 98 91,6% -2,2639 0,0118
B7 64 88,9% 96 89,7% -0,1758 0,4302
D1 32 44,4% 84 78,5% -4,8137 0,0000
D2 45 62,5% 75 70,1% -1,0516 0,1455
D3 47 65,3% 93 86,9% -3,3345 0,0004
D4 35 48,6% 83 77,6% -4,0568 0,0000
I 52 72,2% 93 86,9% -2,3684 0,0089
12 29 40,3% 81 75,7% -4,9798 0,0000
13 39 54,2% 87 81,3% -3,8899 0,0001
N 72 107
Table 2: Results of equality of proportions test.
Except for question B2, the percentage of correct responses for the experimental
group of TBBS users is always higher. It is worth noting that the greater the complexity of
the questions asked, the greater the degree of significance, which affirms the hypothesis
presented in Section 1.
8 This is a good approximation of the Wilcoxon test (see Conover, 1981), which is the most commonly
accepted one for cases like ours. We used the Mann-Whitney method because it is the one which uses SPSS
software, which is what we used for the statistical analysis of our results.
b) Application of the Mann-Whitney/Wilcoxon test
Due to limits on space we present the aggregated results of all three types of
questions asked together in Table 3.
BBBSs TBBSs
Questions Mean Stan.dev. | Mean |Stan.dev.|Signif. (P)
B1 to B7 8,8254 1,0150 |9,3711| 0,8437 | 0,0001
DitoD4 | 6,0764 2,8055 |[8,1472| 2,1057 | 0,0000
11 to 13 5,7176 3,2268 |8,1776| 2,4620 | 0,0000
Table3: Results of Mann-W hitney/W ilcoxon test (objective aspects for independent samples)
The results clearly confirm the conclusions of the previous section. The TBBS
users again got better results and the significance of the results was great; likewise, in this
experimental group, there was less dispersion in the results surrounding the mean value.
Again, as the questions became more difficult, the superiority of the TBBS became more
evident.
2.6.2 Analysis of the data obtained from the questionnaire about subjective points—
independent samples (questions S1 to S11; see 2.3)
For all the questions on the questionnaire the results for the TBBS users were
superior. When all of these results were submitted to the Mann-Whitney/Wilcoxon test, a
significance smaller than 0.05 was yielded in nine of the eleven questions and also in the
average of the eight questions on the increase in understanding (S1 to S8, see Table 4),
among which are the two questions, mentioned above, with a significance of more than
0.05.
BBBSs TBBSs
Questions Mean Stand.dev. Mean Stand.dev. | Significance (P)
Si 5,62 1,88 6,32 2,41 0,003
$2 5,94 2,28 7,39 1,43 0,000
$3 5,82 2,06 7,07 1,95 0,000
S4 5,87 2,35 7,76 1,53 0,000
$5 5,27 2,17 6,07 2,11 0,009
S6 5,49 2,27 6,13 1,96 0,065
S7 6,31 2,07 6,86 1,82 0,168
S8 6,07 2,29 7,05 1,86 0,044
Mean (S1-S8) 5,82 1,73 6,82 1,09 0,000
sg 6,92 1,73 ii 1,62 0,018
$10 6,74 1,86 7,86 1,59 0,002
S11 5,28 1,91 6,72 1,56 0,000
Table 4: Results of Mann-Whitney/Wilcoxon test (subjective aspects— independent samples)
2.6.3, Analysis of the data obtained from the questionnaire about objective points—
related samples
Remember (see 2.4) that in the present case the groups that make up the sample
first used the BBBSs and filled out the corresponding questionnaire; later, after receiving
adequate training, they went on to use the TBBS version and then filled out the new
questionnaire. In this section, we compare the results from both questionnaires by means
of the Mann-Whitney/Wilcoxon method. The results of both tests appear in Table 5. As
this table shows, results similar to those of the independent samples are obtained. Due to
lack of space, we will not make any further comments about this, except to say that it
confirms our hypothesis.
BBBS TBBS
Questions Mean Stan.des}] Mean |Stan.des.|Signif. (P)
BltoB7 | 9,0170 | 0,8642 | 9,5272 | 0,9869 | 0,0111
D1toD4 | 6,3155 | 2,9213 | 9,0060 | 1,4351 | 0,0000
11 to 13 5,6587 | 3,0995 | 8,5238 | 2,4706 | 0,0001
Table5: Results of Mann-Whitney/W ilcoxon test (objective aspects— related samples)
2.6.4, Analysis of the data obtained from the questionnaire about subjective aspects—
related samples
In this case the students gave their perception of the improvements of the TBBSs
over the BBBSs on a scale of 0 (no improvement) to 10 (maximum improvement). In
Table 8, a clear preference for the TBBSs can be seen, both in regard to the students’
increase in understanding (questions S1 to S8) as well as other aspects, notably the
increase in motivation (question S9).
Questions | S1 S2 $3 S4 $5 s6 | S7 | $8 | $9 | S10] S11
Mean 6,737| 6,737] 6,316] 6,842] 4,211] 4,263] 6,723] 6,638] 7,063] 6,250] 6,438
Stand.dev. | 1,408] 1,881] 2,237] 2,243] 3,225]2,600} 2,530] 2,279} 1,873] 2,709] 1,809}
Table 6: Results of descriptive analysis (subjective aspects— related samples)
3. SITME 1.0: A “self-learning facilitator” transparent-box business
simulator (J .A. D. Machuca, J. C. Ruiz)?
In this section, we comment on one of our most representative TBBSs, SITME 1.0.,
which we began in 1991 and finished in 1995. While designing and developing it, we
mainly took into account the transparency necessary, which we talked about in Section 1,
along with other aspects related to the improvement of the learning process. We refer
primarily to the growing need for self-leaming (E.U. Commision, 1996); we have made
such learning easier thanks to the use of new computer information technologies.
This TBBS simulates the behavior of a hypothetical firm making a single product. It
considers aspects related to Operations Management, Personnel Management, Financial
Management and Marketing. The player is given the task of making decisions in these
areas, one of the primary objectives of the game being the understanding of the workings of
the business system in all its complexity, based on a study of the underlying causes of the
effects produced after decision-making during the game. Another basic objective is to show
the close inter-relationships between the different functional areas of the firm making the
user aware that a decision in one particular area will have consequences in the other areas
as well as in the general behavior of the company.
At the beginning of each simulation the facilitator of the game must make up to 15
of decisions. These decisions make up the environment in which the business will work
(price of raw materials or machinery, subcontracting, short- and long-term interest rates, the
firm's credit limits, the sort of demand there is likely to be for the product, how long the
simulation is to last or the frequency with which information will be received from the
simulator, etc). Throughout the simulation only the facilitator of the TBBS will have access
to these decisions in order to change the enviroment framework if he/she judges it
necessary. This can be done from the computer used by a player or from another computer
linked to the player’s computer by a local network.
Throughout the simulation, the user can make up to 16 decisions which we have
grouped into the four business areas mentioned above. The player uses a menu to select the
area; the decisions appear in the Control Window (Figure 1). Using the mouse, the
decisions to be made are chosen, changing their previous value and also indicating how
many periods to simulate. These decisions can be made at any moment within the time
limits set for the complete simulation.
9 We would like to thank M.A. Dominguez Machuca and A. Ruiz, who collaborated during the first modelling
stage.
Production
Finance
Personnel
Marketing
Enviroment
Decisions Area
[Periods to be simulated | [Remaining periods |
La
To access to fez
business areas
menu
Desired production
Number of machines to buy
Raw materials inventory coverage
Raw materials inventory adjustment time 96
—=
To see other decisions Periods passed
[Pause ]
FIGURE 1. Control window
mo
The information offered by the interface is one of the fundamental bases for
decision-making on the part of the player. We will now go on to analyse the different types
of information incorporated into our transparent-box business game. The incorporation of
the causal-loop diagram (linked to variables behavior) into the interface is one of the basic
elements for achieving transparency in our simulators, and one which we have paid
particular attention to. Although, as is well known, causal-loop diagrams lack the
indentification of level and flow variables (Forrester, 1994, p. 202), in order to prevent this
danger we have used symbols to allow a clear identification of the above mentioned kind of
variables. Other elements are the access to the main equations of the model representing the
business system under study, a hypermedia training help module, and various reports which
will be commented on below.
3.1. Information on the causal loop diagram
In this TBBS, owing to the complexity of the model under simulation, the
corresponding causal-loop diagram is sufficiently extensive to surpass the physical
boundaries of the computer screen; we have therefore chosen to represent it in two different
versions: a reduced one (RCLD) occupying just one screen, dispensing with the names of
the variables represented, and another, more detailed one (DCLD), which offers the user
more complete information. When the mouse is positioned on one of the variables of the
less detailed causal-loop diagram (RCLD), the complete name appears in an independent
window. It is also possible to search for a specific variable by choosing from an
alphabetical list; when clicked on, the variable is identified in the RCLD by sound and
visual effects. Then, clicking on the highlighted variable in the diagram accesses the more
detailed causal-loop diagram, to facilitate the analysis of the information about the structure
of the model .
When the mouse is positioned on one of the variables of the DCLD, all the others
influencing it change color (see Figure 2). Should the player wish to do so, it is also
possible to observe in highlighted form those variables influencing those directly linked to
the variable being analysed (see Figure 3). In this way we make it easier to analyse the
causes of the behavior of the variables belonging to the model.
+,
Manpower
Standard
Capacity
Equipment
Capacity
Desired
Production,
Raw
Materials
Inventory
Desired
Production
Manpower
‘Standard
Raw
Materials
Inventory
FIGURE 2. Looking for the causes (first step).
FIGURE 3. Looking for the causes (second step).
To help to understand the relationship between the structure of the system and its
behavior, one can also analyse the behavior of the different variables graphically, within the
causal loop diagram, by visualizing a small diagram which will display the evolution of the
variable selected!°, allowing the causes of
Figure 4).
the behavior to be detected at a glance (see
FIGURE 4. Investigating the relation structure/behaviour
Although the behavior of the different variables can be seen more precisely on the
screen in the form of larger-scale graphics (see section 3.3), this initial examination has the
advantage of being carried out on the causal structure, thus facilitating a systemic
understanding of the relationship between the systen’s structure and its behavior, the main
objective of our TBBSs.
10 The first time we saw this feature was in the 1993 System Dynamics Conference (Kemeny andKreutzer
(1992)).
3.2. Training help module
We have created a hypermedia mouse-tracking system that allows the user to
access all kind of information: when clicking on the name of any variable (the name can
appear in any graphic, table, report, causal diagram, etc.), a hypertext textual description of
that variable appears, showing, in highlighted form, the variables related to the one clicked
on, as well as the graphics, tables and reports. This feature can enlarge the leamner’s
knowledge about the variable under study (see Figure 5). This system is also linked to the
model equations and to the causal diagram (see figure 6).
The standard productionis based upon the standard capacity of thefimn and |}
the inventory of rawmaterials. The graphic, the table andthe report about
production will show you more information.
LZ =
The[standard capacity of a firm is defined as the humber of products which canbe [4] [1
produced ina period of time, without using overtime. It is calculated as the =
minimum between the manpower standard capacity and the equipment capacity. To
find out more about these variables, look at the corresponding graphic, table and
report. | [a
TEE T aaa
|
cd
B
cd
a
cd
bd
cf
4
ced
%
lbeaeReaRaEBeREREaEaEaaE
|
Ey > vec]
FIGURE 5. Training help module: Access to reports, graphics and tables
Manpower standard capacity || Production o)
Standard capacity
Equipment capacity By
SI)
The standard capacity of a firm is defined as the humber of products which can be [42 a
produced ina period of time, without using overtime. It is calculated as the =
minimum between the manpower standard capacity and the equipment capacity. To
find out more about these variables, look at the corresponding graphic, table and
report. im
© Standard capacity, = MIN (manpower standard capacity, , equipment capacity, )
Eli) 3}
sy
FIGURE 6: Training help module: access to equations and “textual diagram”
This hypertext system is available at any moment of the simulation, providing the
player with a conceptual help system which improves the learning process. This formative
help module is essential for various reasons. Among other things, it allows:
access to the explanation of concepts and relationships which the user might not be
familiar with or might have forgotten over time (imagine, for example, financial
concepts for someone who works in production).
* the user to learn without a teacher. This facilitates self-learning at work or even at
home. Moreover, this training is completely flexible, so that, thanks to the
hypermedia system, the user can navigate according to his or her needs.
* the user to gain a deeper understanding of the model used by the simulator (e.g.
access to the main equations of the model)
3.3. Other characteristics
Another module of our interface is specifically devoted to the graphic representation
of the evolution of different variables. The player can see the corresponding numerical
values from any point on a curve with the use of the magnifying glass. It is also possilbe to
manipulate the size or colors, print the graphic, and so on. There are 10 predefined
graphics and, in addition, the player can define as many graphics as he wants to represent
by selecting the variables he/she wishes from the complete list included in the simulator;
we thus give each player complete freedom to study the variables he considers important to
his decision-making process. The information which is represented graphically under the
previous heading can be visualized and printed numerically in table format, showing the
values adopted by the variables in the course of the simulation.
Our interface contains information screens that are specific to the model. The
player can access these at any point since the reports are updated period by period. The user
can also select the information for the last period simulated or for any previous period. We
have incorporated into some reports a "what if..." system. For example, Figure 7 shows the
report on Capacity, containing icons with the + and - signs, which allow the player to
consider, in a preliminary examination, the behavior of the variable standard capacity if the
values of some of the following variables are changed: productivity, manpower or the
number of machines. What we offer here is, therefore, a simulation tool to be used before
decision-making.
Capacity a
Manpower Productivity
100 =a] IS] 10.00 =:
Manpower
Standard Capacity
LE
Equipment Standard Capacity
Capacity
1000 1000
aess Q CS
FIGURE 7: Capacity Report
Given the ongoing interaction between player and interface, we have incorporated
the possibility for the player to decide to move backwards, that is, go back a number of
periods in the simulation until he finds himself where he wants to be, in search of more
suitable decisions with which to rectify those considered erroneous. The simulation can
also be stored on disk, to be recovered any time later; this allows two simulations to be
simultaneously activated in order to compare the behavior resulting from different policies.
We have also incorporated a help feature for interface use; by activating it, the
player can get help with how to handle the business game at any stage. The various
possibilities offered by the interface (graphics, tables, reports, causal-loop diagrams, etc.)
are activated by using the mouse to select the corresponding option from the system of
menus incorporated. We have incorporated moving graphics, sound and video in some of
the modules of the interface, in an attempt to make the information on offer to the player as
user-friendly and motivating as possible.
4, SITECOM 1.0: A multi-functional simulator of competing companies"!
(.A.D. Machuca and M.M. Gonzalez)
As a next step in the development of transparent box simulators, taking the model of
the TBBS mentioned previously (SITME 1.0) as a basis we incorporated a key element
from the business environment— competition— in order to complete the relationships and
elements that affect the way a business works. This makes the simulator more real and
brings the user closer to the complex workings of businesses these days. Thus, the new
simulator we are working on, created for use on a PC, and whose initial prototype could be
finished in July, represents three industrial companies!3 dedicated to making the same
widely-used product which compete for orders in a common market. The philosophy of
use of the simulator, is similar to the one described in the previous section (from the
decision-making process to the observation of results based on grafics and tables to the use
of a causal-loop diagram and a training help module). Therefore, we will not re-explain
that process, but rather we will dedicate the following paragraphs to pointing out the
fundamental differences between the competitive simulator and the one that served as a
starting point for its development.
4.1. New decisions and events
Just as with the simulator described previously, each of the companies comprises four
areas— production, finance, personnel and marketing—, but the number of decisions to be
made by the player has been increased to a total of 24. To these decisions, add another 13
decisions concerning the business environment, which serve to configure the environment
conditions at the outset of the game.
Along with the changes mentioned regarding the decisions, certain events which
were not thought of in the SITME 1.0 simulator but which make the simulation more
realistic are modelled in this one. Among them, the following are worth mentioning:
x The possibility of problems with the machinery. The preventive maintenance decision
has been created in order to try to diminish such problems.
The possible existence of defective products, which form part of the final inventory.
Leave-taking by workers, (when workers take time off, there are fewer people to do a
job that was planned for more).
4.2. Competition factors
As was previously mentioned, the fundamental characteristic of the game stems
from the fact that there is competition among the simulated companies. The number of
customer orders depends on how attractive the company’s product is compared to those of
its competitors. Based on certain elasticity coefficients, this depends on the function of the
sales price, the quality, the payment period, the quality of service offered by the company,
and the image created through advertising; the definitive market response to each of these
11 A simulator of this kind was developed by Davidsen and Myrtveit (1994).
12 We would like to mention J.C. Ruiz who is also collaborating in this work.
13 This number can be increased; there can be up to ten companies without much difficulty.
factores takes time to see. Moreover, because of the interdependence of these factors
among the different competitors, the consequences of a certain company’s decision can not
be known until the other companies make their decisions and the market reacts to them,
generating a dynamic behavior that imitates what could happen in reality.
4.3. Demand
In regard to the market demand, one can choose (at the beginning of the game)
between one that represents all of the fases of the classic product life cycle (introduction,
growth, maturity and decline) or another from a specific phase (growth or maturity); one
can also choose a demand with stationary factors and add (or not) different degrees of
randomness to any of the possibilities mentioned. These make up the initial demand, which
will then be affected throughout the game by the actions of the players, that is, by the
service level, price policies, quality investment policies, and advertising of the different
companies.
4.4. Decision levels
The simulator is presented as a useful instrument for improving decision-making.
In developing it, two levels of decisions have been taken into account: strategic and
operative! decisions. Both types can be employed simultaneously during the simulations.
At the strategic level the players look to the future and try to make an analysis of their
possible decisions based on a series of hypotheses about the decisions of their competitors
(e.g. regarding the price or quality of their product); in this case a real interaction with the
rest of the virtual companies would not be produced, but rather the results would depend
only on the hypotheses put forth and the decisions made by the company. This process
allows the player to consider and analyse different hypothetical scenarios, by asking
themselves, “What would happen if...” As soon as the players consider it appropriate, they
can abandon the strategic level and move on the the operative level. Here the decisions
made by the players have real effects both on their own companies as well as their
competitors’. At this decisional level there is competition in real time, so in each decision
period all the players have to make their decisions before the game simulates the results
derived from them.
4.5.Operation using a network
This competetive game is played using the possibilities offered by new information
and comunication technologies. Thus, although each company has to be simulated in a
different computer, the connections necessary for creating a competitive game in real time
are achieved through a local area network or even through the use of the Intemet.
Consequently, the different players can be far away from each other, or even in different
countries if the Intemet is used. By using these technologies, the simulations also become
more realistic, and thus, the users may become more motivated. This should make these
new tools more effective as teaching tools.
14 Terms used by Davidsen and Myrtveit (1994, p. 17).
4.6 Experimentation
As a result of the innovative nature of these transparent-box competetive business
simulators, it will be necessary to complement the development of these products with
empirical tests of their value. To do this, it will be necessary to design experiments that
allow us to test our hypothesis that the use of such simulators improves training and
understanding of business systems in an ever-changing environment
5. Final remarks
The aim of this paper has been to present only some of the results of our research
during the last ten years; for reasons of space we have had to be very brief. In the near
future, we will further develop this paper, amplifying its content and analysing other works
related to our research.
In our judgement, the results from our experiment demonstrate the superiority of
TBBSs to BBBSs in regard to the learning process, especially in relation to complex
questions linked to an understanding of the structure and operation of the system under
study. However there are still several points to study and test; we would encourage
everyone who is interested to work along these lines, carrying out truly controlled
experiments which will continue to shed light on this subject.
References
ANASTASI, A. (1997), Physological Testing, MacMillan.
BROWN, F.G. (1983), Principles of Educational and Psychological Testing, Holt, Rinehart
and Winston.
CONOVER, (1981), Practical nonparametric statistics, Wiley.
DAVIDSEN, P. (1994), "The System Dynamics Approach to Computer-Based
Management Learning Environments", in J.A.W. Morecroft and J.D. Sterman (Editors),
Modeling for Learning Organizations, Productivity Press.
DAVIDSEN, P.I. AND MYRTVEIT, M. (1994), “Der Rutli Management Simulator - a
New Concept in System Dynamics Based Management Flight Simulators”. Proceedings of
the 1994 International System Dynamics Conference.
EDEN, C. (1994), "Cognitive mapping and problem structuring for System Dynamics
model building", System Dynamics Review, 10, 2-3.
E.U. Commission (1996), Report of the Task Force educational software and multimedia.
FORRESTER, J.W. (1994), "System Dynamics, Systems Thinking and Soft OR", System
Dynamics Review, 10, 2-3.
GROSSLER, A. (1997), “Giving the Black-Box a Lid-Providing Transparency in
Management Simulations”, in Barlas Y., Diker V. G. And Polat S. (Editors). Systems
Approach to Learning and Education into the 21% Century, Bogazicy University.
GULLIKSEN, H. (1987), Theory of Mental Test. Lawrence Erlbaum Associates
Publishers.
ISAACS W.N. AND SENGE P. (1992), "Overcoming learning limits in CBLE'S", EJOR,
59, 1.
KEMENY J.M. AND KREUTZER B. (1992), "An archetype based Management Flight
Simulators", in Vennix J.A.M., Faber J., Scheper W.J., Takkenberg C.A.T. (Editors),
Proceedings 1992 International System Dynamics Conference, The System Dynamics
Society.
DANCE, P.A. AND MORECROFT, J.D.W. (1996), “Learning from Microworlds Environments: A
summary of the Research Issues”, in Richardson G. P. And Sterman J. D. (Editors): System Dynamics’ 96.
The System Dynamics Society.
MACHUCA, J.A.D. (1991), “A new generation of business games for management
education”, in Lasker G.E. and Hough R.R. (Editors): Advances in Support Systems
Research, IIASRC.
——(1992), "Are we losing one of the best features of System Dynamics?", System
ics Review, 8, 2.
MACHUCA J.A.D and ROMAN C. (1990), "Economic Policy and Monetary Policy: A
System Dynamics Conceptualization, in Andersen D., Richardson G.P., Sterman J. (Eds),
ics'90, 2. The System Dynamics Society.
MACHUCA, J.A.D; RUIZ, A.; MACHUCA, M.A.D AND RUIZ, J.C. (1992), “El juego de
la mano de obra y gestion de materiales, D.E.F.D.O.
MACHUCA J.A.D, MACHUCA M.A.D, RUIZ DEL CASTILLO J.C. and RUIZ A.
(1993), "Systems Thinking Learning for Management Education. What are our ideas and
how are we going about it in Sevilla", in Zepeda E. and Machuca J.A.D (Editors), The role
of strategic modelling in international competitiveness. The System Dynamics Society.
MACHUCA J.A.D, DOMINGO M.A, GARCIA R. and GONZALEZ M.M. (1995), "On
the possible influence of Transparent-box business games in the learning process", in: Klein
(Editor): Teaching and interactive methods , WACRA.
___(1997),"On the evaluation of Transparent-box business games as compared with other
teaching methods", in Watts F. and Carbonell A. (Editors), Simulation now , Diputacion
Provincial de Valencia (Spain).
MACHUCA, J.A.D. AND POZO, DEL R. (1997), “A computerized network version of the
Beer Game via Internet”, System Dynamics Review, 13, 4.
MORECROFT J.D.W. (1992), "Design of a leaming environment", in Vennix J.A.M.,
Faber J., Scheper WJ., Takkenberg C.A.T. (Editors), Proceedings 1992 International
System Dynamics Conference. The System Dynamics Society.
NOVALES C.A. (1997), “Estadistica y Econometria”, McGraw-Hill.
NUNNALLY, J.C. (1978), Psychometric Theory, Mc Graw Hill.
PETERSON. S. (1992), "Software for Model-Building and simulation: An illustration of
design philosophy", EJOR, 59.
STERMAN, J.D. (1994), "Learning it about complex systems", System Dynamics Review,
10,2-3.
VAN DE VEN, A. AND FERRY, D. (1978), Messuring and assessing organizations,
Wiley.
SY-FENG WANG AND SHOWING YOUNG (1992), "A preliminary experiment on
Examining Thinking in a Meta-Dynamic Decision-Making Environment", in Vennix
J.A.M., Faber J., Scheper WJ. Takkenberg C.A.T. (Editors), Proceedings
1992International System Dynamics Conference. The System Dynamics Society
CC BY-NC-SA 4.0