Group Model Building Wins: The results of a comparative analysis
Michael Dwyer and Krystyna Stave
University of Nevada Las Vegas
Department of Environmental Studies
4505 Maryland Parkway Box 454030
Las Vegas, Nevada 89154-4030
702-895-4833 (office) and 702-895-4436 (Fax)
michael.dwyer@ unlv.edu, krystyna.stave@ unlv.edu
Abstract
This paper presents clear evidence of the value of group model building for supporting
group decision processes. It responds to Rouwette et al.’s (2002) challenge to take GMB
assessments beyond unstructured single case descriptions that cannot be easily
compared. This paper compares two parallel, real-world problem solving teams
examining urban growth issues in Las Vegas, Nevada over the same two-year time
period. One followed a system dynamics group model building process. The other used
a more traditional group facilitation process. Data about the dynamics of discussions
and the outcomes were collected from meeting transcripts, participant interviews, written
documents and direct observations. The results reveal a marked difference in the content
and timing of discussions over the life of each group project, strongly supporting the
hypothesis that system dynamics provides a better foundation for structuring discussions,
eliciting mental models, and generating sound decisions.
Keywords: Group Model Building (GMB), public participation, public policy,
group facilitation, team performance
Introduction
In 2000, Vennix argued that Group Model Building offered significant advantages over
traditional group problem solving approaches. Since that time, researchers have been
working to find ways to measure the difference. In 2002, Rouwette, Vennix, and van
Mullekom (2002) published a meta-analysis of group model building assessments. Of
the 107 cases they examined, most were single case studies that examined the change that
occurred over time. While 88% of cases produced positive results, the authors noted that
the diversity of methods and definitions made it difficult to compare the cases, and
challenged the system dynamics community to move away from single case study
descriptions. This study responded to their challenge by comparing the performance of
1
Proceedings of the 26" International Conference of the System Dynamics Society. Athens, Greece, July
20-24, 2008. Available at: http://www.systemdynamics.org/conferences/2008/proceed/index.htm
two similar groups examining urban growth issues in Las Vegas over the same two-year
time period. Stakeholder groups of this type are increasingly being used by municipal
governments to support public policy development. The two stakeholder groups studied
were formed independently in March 2004 by different government entities with the
purpose of making recommendations related to managing growth in the Las V egas
metropolitan area. The first group, called the Clark County Community Growth Task
Force (Task Force), reported to the Clark County Board of Commissioners and consisted
of 17 citizen members representing one of seven different interest areas. The Task Force
met 21 times over a 13-month period and was facilitated by two professional facilitators
working together. The second group, the Land Use, Transportation, and Air Quality
(LUTAQ) Working Group, reported to the Southern Nevada Regional Planning
Coalition, and consisted of 12 to 15 staff representatives of 11 different local government
entities. The LUTAQ Working Group met 42 times over an 18 month period. The
LUTAQ Working Group was facilitated by the authors of this paper using a system
dynamics group model building approach.
The relative performance of and the nature of the discussion that occurred in each group
were assessed and compared. The results reveal a marked difference in both performance
and the nature of discussion. The assessment of group performance revealed that despite
a slight disadvantage in ‘input’ variables, the group model building team achieved a
higher level of performance in both process and output variables. Analysis of the
discussion - examining why outcomes might differ - reveals a distinct difference in the
process.
Key Differences
The most significant difference between the two groups is shown in Figure 1, which
displays the dynamics of the nature of the discussions in both groups. Each column of
graphs shows the distribution of participant comments relating to problem definition,
causes of the problem, or potential alternatives or solutions to the problem. The first
2
Proceedings of the 26" International Conference of the System Dynamics Society. Athens, Greece, July
20-24, 2008. Available at: http://www.systemdynamics.org/conferences/2008/proceed/index.htm
column describes discussions in the Task Force group, which was facilitated using
traditional group facilitation methods. The second column describes the LUTAQ group,
facilitated using a group model building approach.
Proceedings of the 26" International Conference of the System Dynamics Society. Athens, Greece, July
20-24, 2008. Available at: http://www.systemdynamics.org/conferences/2008/proceed/index.htm
Traditional Facilitation Group Model Building
Task Force LUTAQ
1,657 Comments 1,112 Comments
100% 100%
0% am La
Problem i c
Definition 40% iii aoe HUT RI |_|
20% a 20%
‘ Messin “MM TT
& 3 3 8 J 3 8 8 8 g
eo § € € € 2 a 2 & PeERLEER Ee
BOE BoB Bo 8 & 85825 8 8 8 a8
100% ial
ws 0%
a ans
Causes
0% 0% m
% 2 Hei
ox (OCEE — % a
3 3 8 Fy Bg 8 g 8 8 g $2 8 8
g g & B88 25
Bb go eg fe § FE See 8 §
100% a0
20% jj 80%
Aitemativess | 4 || 0% a
matives,
Solutions | ** ii ii Bo a al 4
2% [11 | | aw LY LIU
«ll : 0 » pollen I
3 3 3 8 3 3 8 8 $3838 8 8 8 8 BB 8
e a o¢ 8 € 2 a € 8 z 8 5 5
i Poa oR a pO: EEEEEEEERS ES i
Figure 1. Percent of Comments by Category over the Project Life
(Trend lines were added using a polynomial best-fit operation)
Proceedings of the 26" International Conference of the System Dynamics Society. Athens, Greece, July
20-24, 2008. Available at: http://www.systemdynamics.org/conferences/2008/proceed/index.htm
Assessing the nature of the discussion that occurred over the life of each project was
accomplished by categorizing all the comments made during meetings that were available
in minutes, notes, or on video. A total of 1,657 comments made during 20 of the 21
meetings of the Community Growth Task Force (traditional facilitation), and 1,112
comments made during 40 of the 41 of the LUTAQ Working Group (group model
building) were categorized into one of five categories:
e Mission or process
e Problem definition
e Causes
e Alternative/Solution development (including discussion of consequences).
e Other
The coding protocol was developed using grounded theory (Strauss and Corbin’s 1990),
to draw keywords from the data itself. The results are arrayed as percentages by major
category (except ‘other’ ) over the life of each project (figure 1).
The trends in the problem definition, causes, and alternatives graphs for the traditionally
facilitated group (figure 1, column 1) show that discussion jumped quickly from problem
definition to alternative generation, with virtually no discussion of causes. The trend
lines of all three graphs show a lack of central tendency. The relative amplitude of the
trend lines shows that alternative development dominated the discussion for almost the
entire life of the project with one exception: a mid-project return to discussion of the
problem definition.
Despite professional facilitation, the traditionally facilitated group skipped over problem
definition and causes and went right to alternative generation which dominated the
discussion throughout the process. Roughly 30% of the discussion in all the traditionally
facilitated meetings for the entire two-year period focused on the problem, over 40% of
the discussion focused on solutions, and less than 5% of the discussion focused on
problem causes. (The remaining comments concemed the mission of the group.)
By contrast, the same graphs for the group facilitated using a group model building
approach (figure 1, column 2)show a distinctive central tendency of similar amplitude in
each of the graphs. The trend lines move sequentially across the graphs in a wave-like
motion, showing that the group discussion moved through the stages (problem definition,
causes, alternative generation) sequentially.
While these result provide insight into the difference in the discussion, they do not tell us
whether one process was better than the other -- only that they are different. The
performance analysis provides the other half of the story.
How did the Groups Perform?
The results of the performance analysis are presented in a ‘scorecard’ format (tables 1
and 2). The relative degree of performance of each group for each performance variable
category is indicated by the position on an inverted triangle in the far right column. An
explanation of how the analysis was conducted follows the presentation of the results.
Table 1: Relative Degree of Achievement: Output
Variable Degree of achievement
Category (MSIRRES Notatall Partially Marginally Largely Fully
Mission The mission was accomplished.
Consensus Consensus was achieved
4 Members feel pride in their accomplishment.
Commitment Members exhibit support for the final product.
'Y =Group Model Building (LUTAQ Working Group)
V=Traditional Facilitation (Community Growth Task Force)
Table 2: Relative Degree of Achievement: Process Variables
Variable ; Degree of achievement
Category EMELINE Notatall Partially Marginally Largely Fully
Members communicate actively, openly, and
effectively with each other. All members are Vv
ae engaged in discussions, and practice two-way
Communication communication (talking and listening). Vv
Discussions are rich in breadth and substance. | I~
Diverse views are valued and sought.
Morale is high. Members play-off each other in
a manner that creates synergy and creativity.
Collaboration Members recognize their interdependence. v
Members care about each other, trust each V
other, and treat each other with respect. J
Members exhibit back-up behavior.
Conflict Conflict is managed and resolved in a v
Management productive (win-win) manner. Vv
7 ‘inns Vv
Decision- Issues are fully explored before a decision is
making called for. Vv
Le
Members are engaged. They show-up and Vv
Commitment participate actively. Members exhibit zeal,
enthusiasm, and esprit-de-corps.
‘Y =Group Model Building (LUTAQ Working Group)
V=Traditional Facilitation (Community Growth Task Force)
Table 3: Relative Degree of Achievement: Input Variables
Variable 5 Degree achieved
Category EMELINES Notatall Partially Marginally Largely Fully
The mission, goals, and performance objectives Vv
Mission are clear, shared, supported, and are used to Vv
guide and adjust plans, deliberations, and
activities be ~|
The team is structured and populated to include Vv
Team Structure | the fewest number of members necessary to Vv
and
Composition
represent all relevant interests, to provide the
knowledge and skills needed, and to provide
balance in terms of interests, views, and skills.
Roles
Roles are clearly articulated, are assigned to
members with the knowledge, skill and ability to
carry-out their assigned duties, and are
accepted by members. Members carry-out their
duties in an effective manner.
Boundary
Management
The rules for interaction between the team and
sponsoring entity and other interested parties
are clearly articulated, are understood and
supported, and are followed.
Project
Management
The project is managed according to a plan
designed to accomplish the mission in an
efficient and effective manner. The plan and
associated operating procedures (including
ground rules) are clearly articulated, and are
understood and supported by the members.
Implementation is monitored, evaluated, and
plans and procedures are adjusted as needed.
Resources
All resources required for the team to achieve
its mission are delivered in a timely and
effective manner.
Conflict
Management
A strategy for resolving conflict is identified,
understood, supported and followed.
Decision-
making Process
A decision making process is established,
understood, supported by team members, and
followed.
Commitment
Members are committed to the mission.
'Y =Group Model Building (LUTAQ Working Group)
V=Traditional Facilitation (Community Growth Task Force)
The performance scorecards reveal that despite a slight disadvantage in several input
variables (table 3), the group facilitated using a group model building approach
outperformed the traditionally facilitated group in all process (table 2) and output
categories (table 1). This finding is based on the findings that the groups exhibit similar
characteristics in input variables (even a slight advantage in the traditionally facilitated
group), as well as in task and context characteristics, but differed significantly in the
degrees of collaboration (and other process variables depicted in table 2) and consensus
(and other output variables depicted in table 1) achieved. While the assessments are
qualitative, they are derived from multiple sources of evidence and were made according
to a detailed research protocol that was designed to produce the same results independent
of the observer/analyst.
Performance Assessment Methodology
Several methodologies for deriving team performance measures were assessed for
application in this project. Dickenson and MclIntyre’s (1997) framework for developing
teamwork measures was selected for its scientific rigor. The steps in the methodology
(1997) are:
1. Identify a model of team performance for the particular type of team
2. Use the model to identify variables to be measured in each category (input,
process, and output).
3. Identify attributes for each variable
4. Identify observable behavior for each attribute
5. Develop decision rules and a measurement scale for coding each behavior
Dickenson and McIntyre’s methodology was supplemented by Strauss and Corbin’ s
(1990) grounded theory for qualitative research to develop decision rules for assessing
behavior, and for coding comments.
Step 1 - Team Performance Model
After reviewing several team performance models, Gladstein’s ‘General model of group
behavior’ (figure 2) was selected (Gladstein 1984: 502). The Gladstein model was
selected for its consistency with the conceptual framework for assessing group model
building interventions (Rouwette and Vennix 2003). The Gladstein model also
reinforces the importance of similarity in task and environmental characteristics in
conducting a comparative analysis as these are ‘moderating’ factors in group
performance.
INPUTS PROCESS OUTPUTS
GROUP LEVEL
GROUP TASK
GROUP COMPOSITION }—>»
= Task complexity
Adequate skills Environmental uncertainty
Heterogeneity Interdependence
Organizational tenure
Job tenure
GROUP STRUCTURE meal
Role & goal clarity GROUP PROCESS { X)} GROUP EFFECTIVENESS
Specific work norms
Task control Open communication Performance
Size Supportiveness Satisfaction
Formal leadership Conflict
Discussion of strategy
Weighing of individual inputs
ORGANIZATIONAL LEVEL Boundary management
RESOURCES AVAILABLE [>
Timing & technical
consultation
Markets served
ORGANIZATIONAL
STRUCTURE
Rewards for group
performance (©) Indicates a moderated relationship
Supervisory control
Figure 2. General model of group behavior: C onstructs and measured variables
(Gladstein 1984: 502)
10
Step 2 - Variables
Several researchers (Wheelan 2005; Leholm and Vlasin 2006; Dickenson and McIntyre
1997; Hensey 2001; Keen 2003; and Innes and Booher 1999) have identified the
characteristics exhibited by successful teams. Table 3 shows these characteristics in 12
categories.
Table 3. Characteristics exhibited by successful teams.
Variable iy
Category Characteristics
Mission The mission, goals, and performance objectives are clear, shared, supported, and are used to
guide and adjust plans, deliberations, and activities.
Team Structure
and Composition
The team is structured and populated to include the fewest number of members necessary to
represent all relevant interests, to provide the knowledge and skills needed, and to provide
balance in terms of interests, views, and skills.
Members roles (including leadership and facilitation) are clearly articulated, are assigned to
Roles members with the knowledge, skill and ability to carry-out their assigned duties, and are accepted
by members. Members carry-out their duties in an effective manner.
Boundary The rules for interaction between the team and sponsoring entity and other interested parties are
Management Clearly articulated, are understood and supported, and are followed. Members do not feel undue
pressure or coercion from forces outside the team.
The project is managed according to a plan designed to accomplish the mission in an efficient and
Project effective manner. The plan and associated operating procedures (including ground rules) are
Management clearly articulated, and are understood and supported by the members. Implementation is
monitored, evaluated, and plans and procedures are adjusted as the team sees fit.
R All resources required for the team to achieve its mission are delivered in a timely and effective
esources ftianiner.
Members communicate actively, openly, and effectively with each other. All members are
Communication engaged in discussions, and practice two-way communication (talking and listening). Discussions
are rich in breadth and substance. Diverse views are valued and sought.
Collaboration, and
Morale is high. Members play-off each other in a manner that creates synergy and creativity.
Members recognize their interdependence. Members care about each other, trust each other, and
Cohesiveness treat each other with respect. Members exhibit back-up behavior.
Conflict i ; ; il
Management Conflict is managed and resolved in a productive (win-win) manner.
Decision-making
Process
A decision making process is established and followed. Decisions are made in a manner that is
appropriate for the context, and that is supported by team members. Issues are fully explored
before a decision is made.
Members are engaged. They show up and participate actively. Members exhibit zeal,
Commitment enthusiasm, and esprit-de-corps. Members exhibit a sense of responsibility and accountability for
the outputs and outcomes.
Consensus Consensus is achieved. Members support the final product.
11
Step 3 - Attributes
Between two and 12 attributes were identified for each performance category from more
detailed descriptions of each attribute and Gladstein’s model of group behavior (figure 2)
was used to organize them into input, process, and output categories.
While the process and output variable categories speak most directly to group
performance, ‘input’ characteristics, such as: the degree to which the mission is
understood and supported by the members, the unique set of skills and personality traits
each members brings, that availability of resources, decision-making procedures, the
effectiveness of participants with special roles, and the nature of the relationship the
group has with the sponsoring entity, affect group performance. Ina laboratory
environment, these are variables that would be controlled. Because this research makes
use of real-world groups for which controlling these variables would be impossible, they
are included with the variables being measured and compared (table 3).
Step 4 - Observable Behavior
Data sources were identified (shown in table 4, column 1) and a set of observable
behaviors was developed for each attribute by source (shown in table 5).
Table 4, Data Sources
Data Source Community Growth Task Force LUTAQ Working Group
Agendas & minutes j q
Hand-outs & slide presentations ieee agenias & edad
‘ Charter and Process Plan eg ine st een Hons
Content Analysis Website contents ace ocumentation
Benefit cost analysis Final Report
Final Report hi
11 meetings (52%) -45 hours ‘ "
Observation 2 Live i peeiings {08%} alive
10 on video
yi 5 members (29%) 5 (45% of those attending at least 10 meetings)
Interviews 3 managers 2 managers
12
The data collection matrix (Table 5) was used to develop data collection instruments for
each data source. A cross reference scheme maintained the relationship between
observable behaviors and specific questions and observation guidelines in data collection
instruments.
Table 5. Data Collection Matrix (single entry)
Variable é Content r 7
Category Attribute Analysis Observation Interview
cad F Presented consistently “ae
Mission Clearly articulated | In writing? in meeting(s)? Mission clear?
Step 5 - Decision Rules and Measurement Scale
Behavioral observation scales (Figure 3) and an associated set of decision rules (Table 6)
were developed for each attribute using Dickenson and McIntyre’ s framework.
Notatall
0
Partially Marginally
1
Largely
3
Figure 3. Dimensional scale: Degree to which a behavior of interest was achieved
Table 6. Decision Rules
Degree to which a behavior of interest was achieved
Source Not at all Partially Marginally Largely Fully
Active opposition Mostly passive Evenly split Mostly passive | Active support
Content and/or completely | opposition and/or | support/behavior. | supportand/or | and/or completely
Analysis inconsistent inconsistent No indication consistent consistent
behavior behavior either way. behavior behavior
Active opposition Mostly passive Evenly split Mostly passive | Active support
observation and/or completely opposition and/or Support/behavior. Support and/or and/or completely
inconsistent inconsistent No observation consistent consistent
behavior behavior either way. behavior behavior
ea 4 Opinion split
Opinion split but = «
Interviews None support ied do not ae Ne ae eee ee All support
way.
13
Scorecards: Presenting the Results of the Performance Analysis
The position of the marker (an inverted triangle) for each team’s performance level
(tables 1, 2 and 3) represents the mean of the all attributes associated with a particular
variable. For example, in the input variable table (table 5), under the mission category
(line 1) the position of the markers along the performance scale (column 3) represents the
combined mean of the six attributes in this category:
e The mission is articulated
e The mission is clear
e The mission is understood by members
e Thee mission is supported by members
e Progress toward achieving the mission is monitored throughout the process
e Feedback on progress is used to adjust activities
While there is some risk in combining the results in this way, the purpose is to provide a
summary level assessment of the degree to which each team achieved the characteristics
associated with a particular variable category.
Why are these differences important?
The clear differences in the pattern of discussion and the performance measures both
support the value of group model building for group decision processes and help explain
why the differences occurred. The discussion analysis revealed a marked difference in the
content and timing of discussions over the life of each group project, strongly supporting
the hypothesis that system dynamics provides a better foundation for structuring
discussions, eliciting mental models, and generating sound decisions.
From the process analysis, it is clear that the traditional problem-solving facilitation
process did not include a step leading the group to a common view of what is causing a
problem. One participant in the traditionally facilitated group observed: We kept mixing
the problem and potential solutions in the same pot and that confused us. By failing to
14
foster a common view of what causes a problem, traditional facilitation fails to provide
the means to integrate diverse views at a critical stage of the problem solving process.
This in turn inhibits the achievement of consensus in assessing the benefits and costs
associated with alternatives, and selecting an alternative.
Analysis of the group model building process revealed that it supplied a more complete
and coherent problem solving process, including significant discussion of the causes.
Member comments confirm that the group model building approach provided the means
to incorporate diverse views. For example, members of the group model building group
observed:
e We had a very eclectic group - and the diversity was a good thing. The model was
a good vehicle for discussion - it was how we learned.
e It would have been hard for a single person to steer it in some direction because
the model simply wouldn’t allow it.
e [The modeling process] removed the isolation. As we got into the process, the
jurisdictional boundaries disappeared and we really started looking at the
problem from a regional basis.
e We achieved strong consensus because we were involved in the development of
the model - the hood was up.
Note about Traditional Facilitation
While the findings reported here suggest that a system dynamics group model building
approach can produce higher degrees of group performance under certain circumstances,
they should not be interpreted as a rejection of the tools and processes associated with
‘traditional’ facilitation. These tools and processes have their place. Zagonel (2004)
found that effective group problem-solving requires a balance of attention to the problem
and the group dynamics. While the system dynamics group model building approach
assessed here appears to have provided a more effective balance in this case, more work
remains to blend the two approaches.
15
Finally, it is worth noting that there is a risk in labeling the professionally facilitated but
non model-building approach ‘traditional facilitation.’ Doing so opens debate on
whether there is such a thing as a ‘traditional’ facilitation approach. As McFedzean
(2002) points out, there is considerable latitude in how facilitation is carried out in
practice. But the same argument is made for the group model building approach.
Rouwette et al. (2002) observed considerable variation in group model building
interventions. Rather than debate the question of whether each specific process is
representative of its label, the specific mechanisms of each intervention were documented
using the proposed framework for assessing group model building interventions
(Rouwette et al. 2002) so as to allow this research to be added to the greater universe of
group model building assessments that can be used to better understand the relationships
between context, mechanism, and outcome.
References
Dickenson, T. and R. McIntyre, 1997, A Conceptual Framework for Teamwork
Measurement, in Team Performance Assessment and Measurement: Theory,
Methods, and Applications, Brannick, M., E. Salas, and C. Prince (eds.), Lawrence
Erlbaum Associates, Mawah, New Jersey, London
Gladstein, D., 1984, Groups in context: A model of task group effectiveness,
Administrative Science Quarterly, 29 (1984): 499-517
Hensey, M., 2001, Collective Excellence: Building Effective Teams (2nd ed.), ASCE
Press, U.S.A.
Innes, J., 1992, Group Processes and the Social Construction of Growth Management,
Journal of the American Planning Association, Vol. 58, No. 4, pp. 440-453
Innes, J. and D. Booher, 1999, Consensus Building and Complex Adaptive Systems: A
Framework for Evaluating Collaborative Planning, Journal of the American Planning
Association, Vol. 65, No. 4, pp. 412-423
Keen, T., 2003, Creating Effective and Successful Teams, ICHOR Business Books,
Purdue University Press, West Lafayette, Indiana
Leholm, A. and R. Vlasin, 2006, Increasing the Odds for High-Performance Teams:
Lessons Leamed, Michigan State University Press, East Lansing
16
McFadzean, E. 2002, Developing and Supporting Creative Problem Solving Teams: Part
1- A conceptual Model, Management Decision, Vol. 40, No. 5, pp. 463-475
Rouwette, E. and J. Vennix, 2003, Process and outcomes of modeling: An attempt at
formulating a conceptual framework, Proceedings of the 21° International Conference
of the System Dynamics Society, New Y ork 20-24, 2003
Rouwette, E., and J. Vennix, T. van Mullekom, 2002, Group model building
effectiveness: a review of assessment studies, System Dynamics Review, Spring
2002, Vol. 18:1 p. 5-45
Ryan, C. and P Walsh, 2004, Collaboration of public sector agencies: reporting and
accountability challenges, International Journal of Public Sector Management, Vol.
Vol. 17, No. 7, pp. 621-631
Strauss, A. and J. Corbin, 1990, Basics of Qualitative Research: Grounded Theory
Procedures and Techniques, Sage Publications, Newbury Park, London, New Delhi
Vennix, J. A. M., 1996, Group Model Building: Facilitating Team Learning Using
System Dynamics, John Wiley and Sons, Chichester, New Y ork, Brisbane, Toronto,
Singapore
Wheelan, S., 2005, Creating Effective Teams: A quide for members and leaders, 2ed.,
Page Publications, Thousand Oaks, London, New Delhi
Yin, R., 1994, Case Study Research: Design and Methods, 2nd Edition, SAGE
Publications, Thousand Oaks, London and New Delhi
Zagonel, A., 2004: Reflecting on Group Model Building Used to Support Welfare Reform
in New York State, Dissertation, University at Albany, State University of New Y ork
17
