Table of Contents
Go Back
Development of multiplayer games through group modelling
Morten Ruud
Operations Division
Norwegian Defence Leadership Institute (NODLI)
Norwegian Defence Education Centre (NODEC)
Oslo mil/Akershus
NO-0015 Oslo
Norway
Tel: +47 95 24 03 85
Email: morten.ruud@sikt.net
Bjorn Tallak Bakken
Operations Division
Norwegian Defence Leadership Institute (NODLI)
Norwegian Defence Education Centre (NODEC)
Oslo mil/Akershus
NO-0015 Oslo
Norway
Tel: +47 23 09 73 95
Fax: +47 23 09 37 71
Email: btbakken@fsaf.mil.no
Abstract
Group model building can be used as a tool to increase the efficiency of game
development. In a development project done for an Air War College, group model
building was used as a core method to create a multiplayer, decision training game.
The method made it possible, within two working days, to progress from the first
meeting between domain experts and modeller to a tested, multiplayer game. The
development work was perceived as involving college instructors in a positive and
useful process of knowledge elicitation and sharing. The final product was evaluated by
both instructors and students as a suitable tool for decision-making training at strategic
level. At the same time the development required very limited use of resources.
Background
Involving clients has always been an important issue in System Dynamics. At the
beginning of System Dynamics, client involvement was seen as an important technique
for eliciting knowledge during the modelling process (Forrester 1961). Later studies
have supported the concept of client participation as a highly efficient way of improving
knowledge elicitation for model development (Rouwette et al. 2002). It has also
become evident that that deep client involvement contributes to the learning process
among the participants in the modelling group (Morecroft 1992, Lane 1992). It also
seems likely that client involvement has an important influence on subsequent
implementation of a product (Weil 1980). The degree of involvement has differed, but
has found its most distinct expression in Group Model Building, a term created by
system dynamicists at SUNY Albany.
The importance of client involvement, both for the modeling process itself, and for the
implementation of model results, has led to a number of studies in which Group Model
Building may be seen as a common feature (Vennix 1992). Studies have produced
detailed descriptions of steps in the group modeling process and the different roles of
individual group members (Richardson and Andersen 1995; Andersen and Richardson
1997). Detailed procedures for controlling and directing the modelling process have
been described (Randers 1978). The knowledge elicitation process and techniques to
support and make the process more efficient have been developed (Ford and Sterman
1998; Vennix 1990). Some studies have focused on the learning process among the
participants (Lane 1992), while others have been more interested in how the team
discussion could function as a decision support process in management groups
(Morecroft 1992). In most cases, the modelling setting has been quite uniform. In the
course of several meetings, the client group, together with a facilitator and a modeller
create a computer model that utilises several techniques such as a “behaviour over time
diagram”, “causal loop diagram”, “graphical function diagram” and “stock and flow
diagram”. Different ways of organising and accomplishing group model building have
been employed, but with limited variations (Vennix 1996).
In this paper we describe how system dynamics and group model building can be used
to speed up and improve the process of developing multiplayer games. We also discuss
how game playing itself can be a part of the group modelling process, and thus
influence how knowledge is elicited. We also describe how intervention in the group
modelling process influences the outcome and the participants’ view of the process.
A traditional way of handling game development projects would include the
development of a system description from the client, followed by programming and
ending up with a final software product to be delivered to the client. Such a process is
liable to be rather time-consuming and expensive, but most importantly, in most cases it
will create an information gap between the domain experts who hold the knowledge that
is to be incorporated into the game, and the game developers. There is also some
experience that suggests that this approach in many cases would divert attention from
the underlying model structure responsible for creating the game behaviour, over to a
interface focus.
We wanted to experiment with an alternative development strategy. Emphasizing the
problem related to information collection, we wished to reduce the delay in the feedback
loop between expert descriptions, system implementation and expert comments and
corrections. At the same time, we were interested in how to increase the ability to gather
information about a system with ill-defined relationships, partly based on conflict
between opponents. By applying group model building to the development process we
hoped not only to increase the efficiency of knowledge elicitation, but also to establish a
higher level of expert confidence in the solution that emerged. We also wished to
maintain a sharper focus on the underlying game structure than on the game interface.
In addition we wanted to include game testing as a part of the group model-building
development setting.
Case
Our starting point was a war college which wished to develop a game for decision-
making training at a strategic level. The domain was air defence, the topic air power.
The client had a clear idea about what to train and how. The challenge was how best to
develop a solution that would satisfy the client.
The game was to be at a highly aggregated level, putting the players into typical
decision dilemmas. The dilemmas should be a result both of the system’s internal
structure and the uncertainty created by other players’ decisions. The dilemmas in the
game should correspond to the accepted domain theory and in this way underline and
strengthen players’ existing knowledge and stimulate them to engage in theoretical
reflection.
A full day was set aside for a modelling workshop with the domain expert group.
Armed with a general understanding of the topic based on readings and some hours’
discussion in a mixed defence group, the modellers met the expert group. The group
was made up of four to six experienced instructors from the college, all of whom had
deep insight into air power and related topics.
For their part, the modelling group consisted of an experienced modeller/facilitator and
several observers. The domain experts were first given a brief introduction to decision-
training games in general and to modelling tools. They were then introduced to the
modelling project, and were requested to decide on the model boundary, time units,
decisions steps, level of aggregation, etc. Once the work had been framed, the rest of the
day was used for modelling. The modelling was based on the experts’ understanding of
the actual problems, with the principal focus on system structures being the cause of
central dilemmas. Traditional tools for the setting were used, such as “causal loop
diagrams” and “level and flow diagrams”. At the end of the first day of the modelling
workshops the expert group was given the choice of which model variables should be
player decisions, and the opportunity to offer suggestions about interface design.
On the first evening of the modelling workshop, the model was refined by a small group
of modellers and a draft game interface was added. The game, which was designed for a
game leader and two teams playing against each other, was then installed on a 3-node
LAN.
The following morning, the expert group briefly inspected the model, and were
introduced to the draft game interface. They were then divided in two teams, and asked
to run a number of games in order to test the behaviour of the model and the game
interface and design. On the basis of this experience the modelling group turned back to
the model, discussing game behaviour and model design. A number of changes were
proposed in both the model itself and the interface.
During the following week the list of changes was implemented and a modified game
was submitted to the client for testing. Only a limited number of changes were actually
made, leaving unchanged 80-90% of the original structure that had been developed at
the first workshop.
As a part of a training and exercise week, where Army and Navy war college students
were learning about air defence, the modified game was used as a means of reinforcing
students’ understanding of air power dilemmas. After an introduction, the students were
divided into six teams and allocated to separate rooms. Each pair of teams, unknown to
each other, was given a scenario description and would then run the game followed by a
debriefing. A number of games were run in this fashion, each of them with a new
opposing team.
The game software and the game itself were left on the college computer network, and it
turned out a few weeks later that the instructors who had taken part in the model-
building process had started using the game as a part of their courses. Coming back
afterwards, they not provided feedback on how the game itself had functioned in the
training situation, but also ideas for changes in the underlying model. Through the
games played by the students, the instructors gained experience about weakness and
errors, as well as potential enhancements to the model.
Modelling games
The objective of developing games for decision training in a professional environment
is to encourage learning. This is not necessarily a matter of learning directly from the
game itself, but may be as much learning as a consequence of reflecting on the totality
created by the game, the game scenario, one’s own and opposing teams and instructors
(Bakken et al. 1992; Isaacs and Senge 1992). Games for professional training need not
necessarily have advanced interfaces, as professionals are usually capable of
comprehending a situation based on information presented in a simple way. In fact this
also reflects the reality for many decision-makers at strategic level, where information is
only available in an aggregated, text-oriented manner. The primary functions of the
interface are to plot decisions into the system and read information out of it. The most
important issue is to make the game deliverer credible behaviour related to basic theory
and the assumptions given.
The expectations of a client when starting the development of a game are often
concerned with the interface. This was also the situation in our case. Even though the
goal had been set well in advance, for a game at a highly aggregated strategic level,
quite clear expectations emerged for a detailed game interface with all sorts of ‘bells
and whistles’. However, modelling the underlying structure altered the mindset within
the expert group. Creating a visual representation of the mental models possessed by the
participants attracted the full attention of the group. The discussion focused on the
domain, and the participants themselves took care not to become too detailed in the
description of the system, keeping away from discussions about events and staying
firmly with a highly aggregated description. The main issue for the development
process became that of which decision dilemmas the group wished to create and how
these dilemmas could best be illustrated.
During the modelling session we experienced a change in both the role and the focus of
the experts in the modelling group. As the discussion progressed, the experts changed
from a customer-like style of asking for solutions to a more participant role in which
they produced solutions on their own and in this way moved their own fields of
competence closer to the centre of the development process. These changes in
discussion focus were expressed by the way in which the group went from detailed
event-oriented thinking to high-level descriptions of a few important structures, just
enough to create the decision dilemmas they were looking for.
The pedagogical element in a game developed for training must obviously play a central
role. In our case we experienced the conflicts that pedagogical aspects created, with
their demand for realism. In terms of realism, information about the situation of the
opposing forces (intelligence) should be different from the true situation. This
difference might depend on the level of one’s own intelligence service in addition to a
random function. From a pedagogical point of view this sort of misleading information
feedback was not regarded as appropriate. Making the effect of one’s own actions less
traceable, it would reduce the students’ possibilities to reflect over the relationship
between action and effect. The behaviour created by a structure dominated by feedback,
delay and non-linearity was, from a pedagogical point of view, more then enough to
challenge the students’ ability to comprehend complexity. When the effects of the
opposing players’ decision-making was added, the picture for the student was more than
complex enough to create challenging decision training. Even though one ended up in
on
Defensive’
Command and
control
Surface forces
Logistics
Loss Loss | Loss Loss
Northland Southland
Figure 1: Conceptual model for Air Power Decision Training developed during first day of group model
building
this case with “noise” that the instructor could switch on or off, the focus on
pedagogical usefulness remained central throughout the project. Rather than a correct
and detailed description of a system, it was decided to develop a model that would
supply the information needed by students to conceptualise situations that arose,
dilemmas they needed to face, theory they should exploit and decisions they would have
to take.
Group model building is a setting tailored for knowledge elicitation. But it is also a
setting for knowledge dissemination. Over and over again, the group model building
setting enabled us to see how people who have worked side by side for a long time
could “update” their perception of each other’s understanding during the modelling
process. The combination of qualitative and quantitative system description during the
group model building process enforces rich discussions that enabled us to formalise
mental models. But it also has the capability to involve and alter the mental models of
the participants.
Modelling games — with games
Group model building usually includes a description of reference modes. Somewhere in
the modelling process, model runs and the reference mode will be compared. Even for a
game group model-building setting it is natural to describe reference modes for certain
situations. Running the model in order to compare the reference mode with actual
model runs is a little more complicated. To create a run that could be compared with the
reference mode it is usually necessary either to model players’ policy or actually to play
the game. As the modelling process was aimed from the very beginning at creating a
game in which players are responsible for implementing their chosen policy, adding
policy elements to the model for testing purposes seems rather pointless. A much more
obvious method is to complete the model with a game interface and let domain experts
from the modelling group experience the model behaviour by playing.
Most multiplayer games have relatively few variables for player information output and
even fewer decision variables for input. This makes it possible to develop a quick and
straightforward draft game interface within a reasonable time and with minor
consumption of resources. Establishing this sort of sketch game interface opens up a
number of possibilities. Testing an unlimited number of runs vis-a-vis reference modes
is one; trying out different decision policies is another possibility that can be
implemented with little effort.
Having brought the model the short distance to a playable game also opens up a
different modelling discussion. Moving back and forth between model and game brings
up new and exciting elements in the model development process. One might well say
that the game interface gives the group model building setting a “new” tool for
discussion and development. This is certainly true of situations in which the model
describes systems with multiple stakeholders, and stakeholders in a_ situation
characterize it in terms of competition or conflict. The possibility of playing the game
during the modelling session provides an opportunity to learn more about the
consequences related to this properties of the system and to move back to the model for
corrections and enhancements.
The expert group that took part in our group model-building process had the opportunity
to offer their opinions of the modelling process, both in a questionnaire and in open
interviews. They pointed out the process had been quite different from what they had
expected. Instead of describing a set of system requirements they ended up in the
middle of the development process itself, and experienced how the model and game
emerged during the group model-building process. They also expressed satisfaction
about how much had been achieved within such a short period of time.
SS aioe
Timer etterstart- 0
E-bilde
awacs @ 0
AA eo
Foie @ 3
Ledche = @ 10
Kenmunicajoc® 15
Lge @ 0
a)
Operative 00
Northland
Besletningcr om sre per time for 6 timer framove
Typeomiras Offa
Deferve- Offensive Komi Ledehe Lat
ea ee
1:23 fe=a]
[Aveta pit] Arsakste J Swuttur |
Figure 2: Draft game interface developed during first day of group
model building workshop for testing purposes
‘Timer etter start- 0 Southland
E-bilde Northland ©
Transit
=iot
Status
Hiybase e
Kommando & kontrol @
‘Kormunikasjon e
Lovistkls e
Overfatestyrer e
Tiystyke e
Southland
AWACS
AAR
Fiybase
‘Kommanda & kontroll
‘Koaunikasjon
Logistik.
Overflatestyrker
ose 10 —
rn ara. | —— al
Totale tap av fy (antal) ° ‘Tid tgjengelig fi hestoring 03 Send
Ceeeeoee:
Figure 3: Final game interface for “Air Power Decision Trainer”
The development processes evidently created a sense of ownership of the product
among the expert group. This sense of ownership was reflected in a number of ways.
Weaknesses and errors in the model were not regarded as a problem for the modeller,
but rather as challenges for the expert group and their system knowledge, both during
the modelling workshop itself and later. The instructors participating in the expert group
immediately began to use the game in their classes and they did internal marketing of
the project via-a-vis the rest of the college. Given their detailed knowledge of the
underlying model and the user interface, they were able to come up with ideas for
enhancements and more comprehensive plans for new versions.
Game use
The game has been used in a number of decision-making training settings in which
professionals under training took part. The training sessions were all quite extensive,
lasting from one to three days and with a well-defined structure. The teams that played
comprised from four to eight persons and the sessions mainly followed a structure that
consisted of a scenario briefing, playing the game and a debriefing session. The
instructors’ evaluations, external observations and feedback from the participants
obtained by questionnaire, all give an impression of a successful solution. The
instructors in particular pointed to the extensive group processes involved in the training
session and emphasized how the game is a tool for triggering group discussions on
important domain issues.
We do not underestimate the importance of the development style employed in this
project for the success of the application. It is quite natural that a sense of product
ownership among the instructors involved would influence how they integrated the
game into their training sessions. Insight gained during the development process would
be likely to influence confidence in the product and the ability to explain its
assumptions and behaviour.
The development process established confidence among the instructors that the game
supports important theoretical issues that are emphasised in the college’s training
programme. Relying on this, from the instructors’ point of view there is no drawback in
a possible lack of detail in the model. On the contrary, this simplification makes it
possible to focus more directly on the training elements that one wishes to emphasise,
enabling an “in-depth” focus to be placed on central dilemmas and dynamics in
conflicts involving air power
Conclusions
Group model building obviously has certain qualities that make it suitable as a
framework for game development. The most striking experience gained from the
development process clearly was the very short time involved, from initiation to
completion. This was related both to the actual resources employed and to the amount
of time put by the participant in to the project from start to the first version of the
product. The main work was compressed into two days, from when the domain experts
first met the modelling team until they had already run an early version of the game.
Group model building is an effective way of eliciting knowledge. This experience also
applies to situations ion which game development is the goal. In traditional game
development the development team would take the responsibility to push forward both
the interface design and the game behaviour control. With a game software product in
mind, the client can easily perceive the interface as the most important part of game
development and thereby end up with a focus on design issues. The group model
building setting provided a framework for drawing and keeping attention on what
should be considered to be the main theme, the model structure.
On the one hand group model building seems to be an efficient tool for game
development, ..not only because it offers quick results, but also because the underlying
model that creates the game behaviour has a sounder professional foundation when
domain experts from the client have taken part in the development process. Group
model building also has the strength of creating involvement for the client, which
ensures trust and ownership, making implementation of the modelling result more
likely.
At the same time, gaming can be regarded as a useful tool in the group model-building
process itself. When looking for techniques to develop client involvement and
engagement, to improve the facilitators’ capability to understand the expert’s informal
mental models and, in some cases, to bring out the important element of conflict that are
built into many systems, group model building could be a useful tool.
References
Bakken, B. E., J. M. Gould, et al. (1992). "Experimentation in Learning Organizations:
A Management Flight Simulator Approach." European Journal of Operations Research
59(1): 167-182.
Ford, D. N. and J. D. Sterman (1998). "Expert Knowledge Elicitation to Improve
Formal and Mental Models." System Dynamics Review 14(4): 309-340.
Forrester, J. W. (1961). Industrial Dynamics. Cambridge MA, Productivity Press.
Isaacs, W. and P. M. Senge (1992). "Overcoming Limits to Learning in Computer-
Based Learning Environments." European Journal of Operations Research 59(1): 183-
196.
Lane, D. C. (1992). "Modelling as Learning: A Consultancy Methodology for
Enhancing Learning in Management Teams." European Journal of Operational
Research 59(1): 64-64.
Morecroft, J. D. W. (1992). "Executive knowledge, models and learning." European
Journal of Operations Research 59(1): 9-27.
Randers, J. (1978). How to be a Useful Builder of Simulation Models. Current Topics in
Cybernetics and Systems. J. Rose. New York, Springer.
Richardson, G. P. and D. F. Andersen (1995). "Teamwork in Group Model Building."
System Dynamics Review 11(2): 113-137.
Rouwette E A J A et al. (2002): Group model building effectiveness: a review of
assessment studies. System Dynamics Review 18(1), pp 5-46.
Sterman, J. D. (2000). Business Dynamics : Systems Thinking and Modeling for a
Complex World. Boston, Irwin/McGraw-Hill.
Vennix, J. A. M. (1990). Mental models and computer models: design and evaluation of
a computer-based Teaming environment for policy-making., Univ. Nijmegen.
Vennix, J. A. M., D. F. Andersen, et al. (1992). "Model-building for group decision
support and alternatives in knowledge elicitation." European Journal of Operations
Research 59(1): 28-41.
Vennix, J. A. M. (1996). Group Model Building: Facilitating Team Learning Using
System Dynamics. Chichester, Wiley.
Weil, H. B. (1980). The Evolution of an Approach for Achieving Implemented Results
from System Dynamics Projects. Elements of the System Dynamics Method. J.
Randers. Cambridge MA, Productivity Press: 269-289.
Back to the Top
