Supporting Material is available for this work. For more information, follow the link from
the Table of Contents to "Accessing Supporting Material".
Table of Contents
Scripts for Interrupted Group Model Building: Lessons from
Modeling the Emergence of Governance Structures for Information
Integration Across Government Agencies |
Luis Felipe Luna-Reyes David F. Andersen Theresa A. Pardo
lluna@mail.udlap.mx fadum@albany.edu tpardo@ctg.albany.edu
George P. Richardson Brian Burke
Universidad de las gpr@albany.edu bburke@ctg.albany.edu
Ameéricas-Puebla, Business Yi-jung Wu
School, NE-221, Sta. Rockefeller College of wyi-jung@ctg.albany.edu
Catarina Martir, Cholula, Public Affairs and Policy Anthony M. Cresswell
Puebla, MEXICO 72820 University at Albany teresswell@ctg.albany.edu
135 Western Avenue, Tamas Bodor
Mohammad Albany, NY 12222 tbodor@ctg.albany.edu
Mojtahedzadeh Donna Canestraro
dcanestr@ctg.albany.edu
mohammad@attunegroup.c Sharon Dawes
om sdawes@ctg.albany.edu
Fikret Demircivi
Attune Group, 16 Regina fdemircivi@ctg.albany.edu
Court, Suite 16, Albany, NY Carrie Schneider
12054 cschneid@ctg.albany.edu
Fiona Thompson
fthompson@ctg.albany.edu
Center for Technology in
Government, University at
Albany, 187 Wolf Road,
Suite 301
Albany, NY 12205
Abstract
The system dynamics group at Albany has been developing approaches to decision conferencing
using a combination of group facilitation techniques linked to projected computer models in the
room for more than 20 years. Over the years, the group has developed a series of pieces of small
group processes to build system dynamics models with groups, i.e. scripts. The Group Model
Building (GMB) process reported here has several characteristics that make it different from
most other experiences in the group. While the common setting involves managers interested in
tackling a specific problem, this work involves a research team interested in building theory
about the complexity of intergovernmental information integration. Additionally, the reported
GMB process took place in small sessions of two to three hours, while the common practice at
The research reported here is supported by National Science Foundation grant # ITR-0205152. The views and
conclusions expressed in this paper are those of the authors alone and do not reflect the views or policies of the
National Science Foundation.
Albany involves intensive one or two-day meetings. In this way, the paper will include general
thoughts about the implications of these differences for the GMB process.
Key Words
Group Model Building, System Dynamics Modeling, Theory Building, Information Integration.
Introduction
Methods for Group Model Building (GMB) have been developed at the University at
Albany to enable the integration of decision conferencing and system dynamics practices
(Reagan-Cirincione et al., 1991; Richardson and Andersen, 1995; Andersen and Richardson,
1997; Rohrbaugh, 2000; Zagonel, 2002). These methods employ decision conferences as a
particular kind of group decision support activity in which groups create and refine system
dynamics models. Instead of the computer-mediated collaboration typical of group decision
support systems, however, these GMB methods employ face-to-face meetings in which “verbal
and nonverbal communication takes a completely connected, ‘each to all’ pattern enhanced by
the presence of a group facilitator” (Schuman and Rohrbaugh, 1991, p.148). These meetings use
a combination of group facilitation techniques linked to projected computer models in the room
to support the model development. (Mumpower et al., 1988; Schuman and Rohrbaugh, 1991;
Rohrbaugh, 1992). These methods and the related GMB research at Albany builds on the larger
body of literature in GMB (Vennix, 1996; Rouwette et al., 2000).
The particular GMB methods reported here were used to support theory building rather
than managerial decision making. The purpose was not to guide a strategy or policy development
but to create a dynamic theory of a complex interorganizational process: the social processes
inside an interagency team with the task of creating a “framework” for Justice Integration in
New York State. This interagency team, made up of government professionals, had the task of
creating a governance framework to guide integration technology development. They worked
with a research team that facilitated and studied the creative process. In a typical problem
solving setting the group involved in the GMB sessions would be the interagency team, tackling
a specific problem related to policy or strategy decisions. In this work, by contrast, the group
consisted of the research team seeking to build a dynamic social science theory about the
complexity of intergovernmental information integration. As described below, the research team
was working as part of a larger theory-building project supported by the National Science
Foundation (NSF). A significant amount of qualitative field research preceded the group
modeling process and empirical research involving both focus groups and a survey followed the
group modeling exercise.
Over the years, the Albany group has developed a series of scripts, i.e. “fairly
sophisticated pieces of small group processes” (Andersen and Richardson, 1997, p. 107). This
paper documents the scripts and products for the GMB effort using the approaches developed at
Albany from November 2003 to May 2004. The theory building GMB process took place in
small sessions of two to three hours, separated by one or more months. The previous common
practice at Albany employed intensive one or two-day meetings. The paper describes the new
GMB process in more detail and discusses the implications of these method differences for the
GMB process in general.
The modeling group consists of the System Dynamics Group at Albany, and the action
research team at the Center for Technology in Government” (CTG) who worked with
representatives of the Criminal Justice Information Technology (CJIT) group of New York State
(NYS). Their goal was to develop a governance framework as part of an initiative to create a
“one-stop shopping” capability for users of criminal justice information. The actual model
developed from these GMB sessions is described elsewhere in these proceedings (Luna-Reyes et
al., 2004).
Project Background: A NSF Funded Research Project
The GMB process was conducted as one of the several modeling approaches used in a
basic research project funded by NSF and focusing on increasing the understanding of
interorganizational information integration. See Figure | for project timeline and flow of
research activities. The project aimed to develop and test models of information integration in
multi-organizational government settings. Integrating and sharing information in these settings
involves complex interactions among social and technological processes. Organizations must
establish and maintain collaborative relationships in which knowledge sharing is critical to
resolving issues of data definitions and meaning. The interagency team involved in the system
design and development faced problems of multiple platforms, diverse database designs and data
structures, highly variable data quality, and incompatible network infrastructure. These
integration processes often involve new work processes and significant organizational change.
Moreover, designing and implementing cross-agency information integration is a lengthy
process, involving learning and evolving interorganizational relationships. Thus the processes
appeared to involve important feedback effects, making it an appropriate focus for dynamic
modeling.
Field Visits
ia NYS Reflection Survey to test Final Models)
enmetn a Workshop model(s) and Theories
nitiatives
Modeling
Workshops
Data Collection and Analysis
Product Development
Figure 1. NSF funded project timeline
The goal of this group model building was an empirically grounded theory of the social
and technical processes observed in the work of the interagency team. The plan was to develop
The Center for Technology in Government at the University at Albany develops applied research and
partnership projects to foster innovative ways to improve government services through the understanding of the
management, policy, and technology dimensions of information use in the public sector
(http://www.ctg.albany.edu).
that theory using the tools of system dynamics to represent the processes of interest, forming the
basis for substantive theory. Therefore the group engaged in the model building had to bring
together knowledge of what would constitute relevant theory with understanding of modeling
and sufficient data about the process to be modeled. The data about the processes to be modeled
was available in the notes, recording, and memories of the research team. Both the research team
and the modeling team brought knowledge about relevant theory to the modeling. It was decided
not to include members of the interagency team in the initial modeling sessions, since they were
not equipped with relevant theory or modeling knowledge to participate. They would instead be
asked to review and comment on the modeling work at later stages.
The NYS criminal justice information integration case provided the CTG researchers
with an excellent opportunity to study how IT and social factors interact to influence the
effectiveness of interorganizational information integration. Prior to their work with the CJIT
group, the CTG researchers hypothesized that there was a structure to the social and technical
processes of interaction for information integration. Drawing on preliminary research data from
the team’s work the CJIT group, the CTG researchers’ approached the GMB effort with an
interest in exploring this hypothesis further. Moreover, from the team’s action research with the
CJIT group, the CTG researchers had observed group interaction that was comprised of a set of
social processes that formed and reformed technical artifacts. The research team further
hypothesized that the effectiveness of interorganizational information integration hinged on the
interaction of this set of social processes with the technical artifacts produced. Consequently, the
CTG researchers and the modeling team agreed that a GMB effort involving the CTG research
team focused on modeling these interactions was the appropriate approach.
Case Background: Integrated Criminal Justice Information for New York State
In April 2003, the Criminal Justice Information Technology (CJIT) group of New York
State (NYS) was tasked with developing a framework to fulfill the goal of giving users of
criminal justice data and information systems “one-stop shopping” access to the information
needed to accomplish their mission.
Collaborating with the Center for Technology in Government for an 8-month period, the
group specified the business problem, its context, and identified feasible solutions and
alternatives. The vision for this project was for the “cop on the street” to be provided with quick,
consolidated responses to support life-and-death decisions. Delays in retrieving data from
multiple organizations cannot be tolerated. Systems must focus on ease of use, simplicity of
administration, and functional relevancy to law enforcement personnel, correction officers,
judges, and other justice system participants. Integrated systems must be central to the work and
processes of the front line workers and not viewed as add-on, superfluous activities that require
additional time and effort. In addition, these systems must also provide decision makers with
access to summary, trend, geographic, and other strategic and tactical information that can be
derived from operational information in near real-time.
The formation of an Integrated Justice Advisory Board was seen as a critical first step in
the establishment of the governance process necessary to achieve the goals of NYS Integrated
Justice. In particular the team focused on the development of recommendations for a statewide
integrated justice governance body. The recommendations were submitted to the NYS Director
of Criminal Justice in December 2003 with the first meeting of the recommended Criminal
Justice CIO Council occurring in January 2004. The development of the governance framework
is a “first of its kind” cross-agency decision process in support of a new initiative of the governor
and the director of criminal justice. This work is critical to research as it provides foundational
“street-level” knowledge of integration processes to inform the model development. Data
collection for this action research effort includes transcripts and observations notes from over
120 meetings, workshops, debriefs, and interviews.
Scripts for Interrupted Group Model Building
The group modeling effort spanned a six-month period and consisted of five separate
meetings held at the University at Albany’s Center for Technology in Government (CTG). This
paper includes a description of each of the five GMB sessions.*
Group Model Building Sessions
1_| November 26, 2003 | Problem boundary and reference mode elicitation
2 | December 10, 2003 | Story-telling, reference mode clustering
3__| March 2, 2004 Towards a dynamic hypothesis
4 | May 4, 2004 Fishbowl with preliminary model
5 | May 21, 2004 Revisiting model and exploring scenarios
The first three GMB sessions focused on developing reference modes from which the
team began the model building and creation of a dynamic hypothesis. During the third meeting
the modelers presented a series of dynamic models reflecting what they had heard over the
previous four months as well as a first look at a dynamic hypothesis. The team spent the final
meetings of the process experimenting with two versions of a “collaboration model” in order to
elicit feedback for refinement. The fifth, and final meeting, concluded with a final draft that
modeled the interactions between social and technical artifacts in this information integration
project. Results of the theory construction process were shared with a panel of information
professionals who were involved in system development at all six of the research sites. In
addition, based on feedback at that meeting and on qualitative insights drawn from the larger
project, CTG is presently engaged in a wide-scale survey effort to test and validate some of the
theories proposed in the modeling projects.
Problem boundary and reference mode elicitation
One of the goals of the first meeting was to more fully familiarize the CTG research team
with system dynamics modeling. After this the research team members were able to be more
fully engaged in the model building process by using system thinking to provide modelers with
information for model development. The first meeting also helped to probe the boundary of the
* See supplemental file for detailed agendas, meeting minutes, presentations, and group exercise products from all
five GMB sessions.
model and to realize the expectations and desired outcomes of the CTG research team by using
the system dynamics model building approach.
In the beginning of the first meeting, the modelers reviewed the core concepts of system
dynamics, stocks and flows, with the CTG research team using a Concept Model as described in
Andersen and Richardson (1997). The first group exercise was designed to have the research
team work in pairs to identify Reference Modes of influential and significant variables. The
exercise also included a short story telling section, which allowed each individual story to be
verified or modified by the entire CTG research team. The story-telling section also helped the
modelers to obtain a more comprehensive picture of the project and to gain important
information about the overall system boundary. Figure 2 illustrates one Reference Mode
example created during the first meeting.
Level of Understanding the Reality vs. Level of Involved Effort-
© The goup started with a rroderate level of effort. However, when
they realized thatthe realty wes not whet they originally thought, the
level of effort was withdrawn, And the low level of invalved effort
lasted until Oct
© Citcal poin-Oct. meetings: The Oct. meetings were the avenues:
for the group to understand the reality and the big picture ofthe
project. They were givena lot of "assigrments’ in these meetings
and started to fed! the ownership of the product. Besides, they also
began to realize they could actually make some influence in the
process by meking recommendations. Once this concept is
erbedded in ther minds, the level of involvernent and engagerrent
Stated torise.
Figure 2. Example of results from the reference modes exercise
The dialogue between modelers and the CTG research team in the first meeting were
extremely valuable in terms of identifying and eliciting some key variables in the research. This
also leaded to an effective first attempt of building a dynamic hypothesis.
Story-telling, reference mode clustering
The second meeting was designed to pursue a dynamic hypothesis to work toward the
entire model development. This was a process of lining up all the significant variables to address
the overall picture of the project.
The second meeting grouped the CTG researchers into three teams and gave each team
all the stories and graphs produced in the first meeting. In this assignment, modelers asked the
teams to cluster the variables into different groups by naming them with core concepts (such as
social factors, technical factors, intra-organizational, inter-organizational). Each team presented
the story by illustrating the interactions among grouped variables. Figure 3 illustrates one team
working during the clustering exercise.
6
Figure 3. Clustering exercise
The assignment helped the researchers and modelers to review the process of the
information integration project, and to identify the challenges and obstacles to it. The assignment
introduced the physics of the system (such as workload accumulation) and the perceptions of the
system (such as work pressure). In addition, the second meeting moved the overall group toward
a more comprehensive dynamic hypothesis and allowed modelers to identify specific critical
stocks and flows in the model development.
Towards a dynamic hypothesis
In the third meeting, the modelers moved forward the conclusions of the three teams from
the second meeting by specifying some of the casual structures that might generate the reference
mode behavior within a dynamic model. Prior to the third meeting the modeling team had met
and proposed a stock-and-flow image consisting of an aging chain of three technical artifacts
surrounding by four social accumulations, two of which involved individual understanding and
commitment to the project and two of which represented group understanding and engagement.
The group spent considerable time understanding and agreeing upon this backbone structure.
The reflections and interactions between modelers and CTG research team in the third
meeting allowed the modelers to modify the first draft hypothesis and models in the preparation
of the actual running simulating model for the next meeting. Figure 4 illustrates one of the
dynamic models proposed by the modeling team.
Green group conversation
Stock and Flow eancesteiproducr onthe ork
black and white”
Individual
Individual
understanding
commitment
laeas [rev es Se
Brainstorming Cafifying Formalizing
Green group
conversation:
le “Engagement as an
important element in
the process”
Group Group
understanding engagement
Figure 4. Dynamic model example
Fishbowl with preliminary model
The purpose of the fourth meeting was to reflect and comment on the meetings that had
been conducted to date through a fishbowl exercise’, and then to present the running model to
the group for questions and suggestions.
The meeting raised a number of issues related to concept clarification and terminology
usage between modelers and CTG research team. The first running model was also employed to
show the behaviors of the model and to confirm whether the behaviors seemed logical according
to the researchers’ experience on working with the information integration project. If the
simulating behaviors were shown differently from research team members’ past experiences, the
modelers would explain the feedback loops in the model, and then confirmed and comprehended
the logic with the research team.
The fourth meeting granted the modelers the opportunity to check the first running model
and brought the thread of exploring possible contingency implications in the information
integration project.
Revisiting model and exploring scenarios
The fifth meeting was held to revisit the system dynamics model which had been
modified after the first running model check in the fourth meeting and to test more contingency
scenarios to better understand project and model behaviors.
During a fishbowl exercise, the modeling team sits in front of the group and talk about alternative ways to
conduct the meeting, choosing one of those alternatives.
Modelers picked several core variables in the project and laid out a scenario matrix in
order to test different applications and discover the leverage/tipping point of each variable.
Through this exercise, the project and model behaviors were realized in a system thinking
approach that helped the CTG research team to obtain another comprehensive picture for
research and to acknowledge the feedback effects in the project. Figure 5 illustrates one of the
experiments shared with the GMB team.
Experiments with Model
Legitimate proposals
40
o 7 z 3 4 3
Time (Month)
Legitimate proposals : Base
Legitimate proposals : lots of ctg in
Legitimate proposals : no cig inf
Legitimate proposals less grp inf, dea
Figure 5. Model Exercise Example
In other words, as long as the variable leverage points were recognized, it is easier for the
research team to locate the critical parameters and to understand the problem of information
integration.
Discussion
Using System dynamics as one of the theory-building methodologies delivered a number
of benefits in the project. First, the group had an opportunity to observe and express the project-
related issues through a dynamic analytical lens, capturing the story at a different level of
analysis. Prior to the GMB sessions, the CTG team had developed a set of propositions from
their preliminary research data. These propositions formed a foundation from which the CTG
team and the modelers were able to explore the interaction of social processes and technical
artifacts through a systems thinking approach. The graphical representations of the model proved
useful to facilitate conversation and promote new insights into the already rich thinking of the
CTG team.
Having a period of time between sessions also proved to be useful for theory-making
purposes. It provided a unique opportunity for the CTG research team to reflect on the model
building effort after each phase of the process. It allowed the researchers the time to review the
large amount of data collected from the case after each GMB session in order to help them and
the modelers continue building and refining dynamic models that most accurately captured, in
terms of systems, thinking the team’s experiences and observations.
Formulating the diagrammed theory in mathematical terms also brought some
advantages. The mathematical formulation of every relationship and feedback process eliminates
any ambiguity associated with them, facilitating conversations about their nature, and the
appropriateness of each of them. During one of the sessions, a member of the group commented
“apparently all depends on the math in this model”, and effectively that is true. The group needs
then to decide if the math represents appropriately the observed phenomenon or if it needs to be
reformulated in a way more consistent to their observations.
Conclusion
Although the GMB approach proved to be effective to facilitate conversation and
generation of insight in the theory-building process by providing a graphical and mathematical
language to promote critical reflection, system dynamicists need to reflect about the
appropriateness of our current validation techniques to the theory —building process. That is to
say, our current validation techniques are oriented to build confidence in a model to be used by a
particular group to make decisions about either policy or strategy. However, the generalizability
of the theory embedded in the model is not a concern of the group of managers involved in its
development and use. Using the same tool to build general theories may need additional tests and
process “safeguards.” In this way we need to revisit the tests to build confidence in system
dynamics models to assess its suitability to “build confidence” in the generalizability of a model
(Forrester and Senge, 1980; Barlas, 1996). Work of social scientists developing theory through
the use of qualitative techniques and case studies has the potential to contribute to this reflection
process as well (Lee, 1989; Walsham, 1995; Strauss and Corbin, 1998; Eisenhardt, 2002; Lee
and Baskerville, 2003).
References
Andersen, D. F. and G. P. Richardson (1997). "Scripts for group model building." System
Dynamics Review 13(2): 107-129.
Barlas, Y. (1996). "Formal aspects of model validity and validation in system dynamics." System
Dynamics Review 12(3): 183-210.
Eisenhardt, K. (2002). Building theories from case study research. The qualitative research
companion. A. M. Huberman and M. B. Miles. London, Sage Publications.
Forrester, J. and P. Senge (1980). Tests for building confidence in system dynamics models.
Modelling for management: Simulation in support of systems thinking. G. P. Richardson:
414-432.
Lee, A. S. (1989). "A scientific methodology for mis case studies." MIS Quarterly 13(1): 33-50.
Lee, A. S. and R. L. Baskerville (2003). "Generalizing generalizability in information systems
research." Information Systems Research 14(3): 221-243.
Luna-Reyes, L. F., M. Mojtahedzadeh, D. F. Andersen, G. P. Richardson, T. A. Pardo, B. Burke,
Y.-j. Wu, A. M. Cresswell, T. Bodor, D. Canestraro, S. Dawes, F. Demircivi, and F.
Thompson (2004). Emergence of the governance structure for information integration
across governmental agencies: A system dynamics approach. 22nd International
Conference of The System Dynamics Society, Oxford, England, System Dynamics
Society.
Mumpower, J. L., S. P. Schuman and A. Zumbolo (1988). Analytical mediation: An application
in collective bargaining. Organizational decision support systems. R. Lee, A. McCosh
and P. Migliarese. Amsterdan: 61-73.
Reagan-Cirincione, P., S. Schuman, G. P. Richardson and S. A. Dorf (1991). "Decision
modeling: Tools for strategic thinking." Interfaces 21(6): 52-65.
Richardson, G. P. and D. F. Andersen (1995). "Teamwork in group model building." System
Dynamics Review 11(2): 113-137.
Rohrbaugh, J. (1992). Cognitive challenges and collective accomplishments. Computer
augmented teamwork: A guided tour. R. W. R. Bostrom, and S.T. Kinney. New York,
Van Nostrand Reinhold: 299-324.
Rohrbaugh, J. (2000). The use of system dynamics in decision conferencing. Handbook of public
information systems. D. Garson. New York, Marcel Dekker: 521-533.
Rouwette, E. A. J. A., J. A. M. Vennix and C. M. Thijssen (2000). "Group model building: A
decision room approach." Simulation and Gaming 31(3): 359-379.
Schuman, S. P. and J. Rohrbaugh (1991). "Decision conferencing for systems planning."
Information & Management 21(3): 147-159.
Strauss, A. and J. Corbin (1998). Basics of qualitative research: Techniques and procedures for
developing grounded theory. Thousand Oaks, Sage Publications.
Vennix, J. A. M. (1996). Group model building: Facilitating team learning using system
dynamics. Chichester, Wiley.
Walsham, G. (1995). "Interpretive case studies in is research: Nature and method." European
Journal of Information Systems 4(2): 74-81.
Zagonel, A. A. (2002). Model conceptualization in group model building: A review of the
literature exploring the tension between representing reality and negotiating a social
order. 20th International System Dynamics Conference, Palermo, Italy, System
Dynamics Society.
11
Back to the
CC BY-NC-SA 4.0