Supporting Material is available for this work. For more information, follow the link from
the Table of Contents to "Accessing Supporting Material".
Table of Contents
Go Back
Research Problem
Can SD Models Have Greater Relevance to Practice When Used Within
Participatory Action Research Designs?
By
Hans J (Jochen) Scholl
University of Washington
The Information School
Mary Gates Hall, 370C
Seattle, WA 98195-2840
Phone (206) 616-2543
Fax (206) 616-3152
Email jscholl@u.washington.edu
Bio Sketch
Hans J (Jochen) Scholl,
Assistant Professor
University of Washington
The Information School
Mary Gates Hall, 370C
Box 352840
Seattle, WA 98195-2840
Phone: (206) 616-2543
Fax: (206) 616-3152
E-mail: jscholl@u.washington.edu
Personal Web Page: http://faculty.washington.edu/jscholl/
Hans J. (Jochen) Scholl is Assistant Professor at the
University of Washington. He earned a Ph.D. in
Information Science from the University at Albany and a
MBA from the GSBA Zurich, Switzerland. His research
interests focus on modeling complex systems using
system dynamics. He also engages in qualitative research.
Study areas include information systems success, e-
Government, and firm survival. Previously, Jochen was a
researcher at the Center for Technology in Government at
Albany,NY.
Can SD Models Have Greater Relevance to
Practice When Used Within Participatory
Action Research Designs?
Abstract
Over the years, the field has produced numerous rigorously researched SD models, which
have helped suggest detailed policy changes to organizations. However, the application
of model-based insights and the implementation of practical changes to policies,
structures, and processes has not been observed as frequently, even though, various
approaches have been used to increase ownership in models and results among
practitioners and decision-makers, for example, via group model building. In this paper, a
more radical approach is considered, which would amalgamate SD and its analytical
wealth with Participatory Action Research and its practical problem-solving and change
orientation, such that the relative strengths of both disciplines complement each other and
reliably produce an SD-influenced organizational outcome. The feasibility of the
proposed approach needs to be empirically tested yet.
Problem Definition
In both academic and commercial system dynamic modeling a major concern has been
model credibility among practitioners and stakeholders who “own” the problem modeled.
First, those problem owners may neither agree in degree nor in kind that the model
represents the problem they face. Second, even if the problem owners accept a model as
an adequate representation of “their” dynamic problem along with the diagnoses and
prescriptions for policy change derived from analysis, still no action may be taken in
practice. While learning occurs and insights seem to follow from GMB exercises, policy
changes are made rather rarely (Rouwette, Vennix, & Mullekom, 2002). Hence, although
a scientifically rigorous model has been created, it may have little or no relevance to
practice. So far, various approaches have been proposed to overcome this implicit rigor-
versus-relevance dilemma in SD. In this research problem notice, as a candidate for a
a ae
potential remedy, the amalgamation of Participatory Action Research and System
Dynamics is proposed for empirical testing. Both methods, Action Research and System
Dynamics deal with what Ackoff calls messy (Ackoff, 1974) and Checkland labels ill-
defined problems (Checkland, 1981), which are described as incompletely defined and
rudimentarily understood problems due to their systemic complexity in detail or in
dynamics, or both. Both research methodologies also employ iterative and spiral designs,
which resemble each other in various phases. Like in Action Research, practitioners have
collaborated with system dynamicists, for example, in group-model-building exercises
aimed at improving both model validity and utility when intervening in a practical
organizational situation. However, while Group Model Building aims at both rapidly
developing a rigorous model with input from practitioners and creating ownership
regarding the model and its uses among practitioners, Action Research is specifically
geared to create a change and solve a practical problem within an organizational setting.
Participatory Action Research designs with an adopted SD component might pave a more
direct path to new policy implementation than other approaches. An empirical test of
such a design would also entail the attempt of measuring the contribution of the SD
modeling exercise, and, hence, the credibility and utility of the model.
Tapping the Mental “Database”
The study of social phenomena, for example, has encountered a myriad of problems of
the type Ackoff and Checkland describe. Forrester has made the point that modeling such
complex social systems cannot rely on hard data such as numerical and written accounts
alone (Forrester, 1980). Getting access to this important source of data, namely, the
mental “database,” has been the primary focus of the SD traditions of, for example,
ze
Group Model Building Albany-style (Andersen & Richardson, 1997; Richardson &
Andersen, 1995), European style (Vennix, 1996; Vennix, Akkermans, & Rouwette, 1996;
Vennix, Andersen, Richardson, & Rohrbaugh, 1992; Vennix & Gubbels, 1992; Vennix,
Gubbels, Post, & Poppen, 1988), or Mediated Modeling (Van den Belt, Videira, Antunes,
Santos, & Gamito, 2000). Action researchers, on the other hand, tap into the exact same
“database” of mental models when attempting to find a practical solution for a “real-
world” problem together with practitioners. However, in Action Research designs the
application of formal and quantitative modeling techniques has not been found in the
literature studied. Action Research designs garner practitioners’ confidence in the
produced practical solution by iteratively probing its viability. In this particular context of
iterative probing, the use of system dynamic modeling may prove a formidable tool for
policy analysis and design within and between action research stages and cycles. The
integration of quantitative SD techniques with qualitative approaches (such as soft
systems methodology (SSM)) has been proposed before, for example, (Lane & Oliva,
1998). Although, participatory action research and SSM may belong to the same family
of research traditions, they are distinct (for space reasons this discussion must be omitted
here). Using Lane’s adaptation of Burrell and Morgan’s framework of research
paradigms (Burrell & Morgan, 1979), the approach proposed here would be located
between or in the close neighborhood of both what he calls Holon Dynamics and Agency
Dynamics while SD is predominantly located in the functionalist quadrant posing the
problem of commensurability when crossing paradigmatic boundaries. The same author,
however, argues more recently that SD may act as a powerful driver of integrative
paradigmatic efforts (Lane, 2001).
The Crux with Verification, Validation, and Confirmation of SD Models
In traditional research designs, the academic community is used to scrutinize both
internal and construct validity in any research design. Those models, however, are rather
simple relative to SD models. The structural complexity in SD models, hence, has always
been a core concern for modelers and their non-modeler audiences alike: If dynamic
behavior endogenously results from the model structure, then obviously this structure
should capture the main traits of the problem under study. However, how can the expert
modeler convince herself and the audience, that her particular conceptualization
accurately describes the problem at hand? Also, will two expert modelers working
independently on a problem come down with the precise same model? If hundreds of
variables and equations are involved, then the likelihood of finding an exact same model
formulation in two independently developed SD models is obviously infinitesimally little.
But if this is so, what does this mean for the model’s internal and construct validities? SD
modelers have worked on two major avenues for coping with these challenges and for
increasing confidence in a given model by (1) reducing uncertainty through
demonstrating the model’s robustness and structural soundness (Forrester & Senge, 1996)
including minute outcome fit with times series (for an extreme example, see (Graham,
Choi, & Mullen, 2002) and by (2) involving subject experts and target audiences in the
model building process typically in so-called Group Model Building (GMB) exercises
(Vennix, 1996). In my concluding remarks, I will point at additional issues regarding
verification, validation, and confirmation of numerical models.
Group Model Building
Today, quite a few strands of GMB and mediated modeling coexist. This short overview
uses the Albany school and the Vennix school of GMB as an illustration of the approach.
For space constraints, Ford and Sterman’s expert knowledge elicitation technique cannot
be contrasted to the GMB approaches here (Ford & Sterman, 1997). GMB was first
proposed as a vehicle for simultaneous, structured knowledge elicitation from a multitude
of experts (Vennix et al., 1988). As Homer states detail knowledge used in SD modeling
may be widely dispersed (Homer, 1996). In order to capture the level of desired detail,
gathering the subject field experts and then eliciting their knowledge through facilitated
group interaction appears advantageous. This approach had been practiced even before
the term of GMB was coined (Vennix et al., 1992). Besides knowledge elicitation for
purposes of model building, the GMB process has also been portrayed as a vehicle for
influencing and actively changing GMB participants’ mental models, that is, their way of
thinking about the problem at hand, in order to gestate commitment and intention to bring
about organizational change (Vennix, 1996; Vennix et al., 1996). Vennix at al describe
the group-based knowledge elicitation process through the various stages of model
building. (Vennix et al., 1992; Vennix et al., 1988). These stages comprise (a) identifying
and defining the problem under study, (b) formulation of dynamic
hypothesis/conceptualization, (c) formulation of simulation model, (d) model testing, (e)
model evaluation/policy design, and (f) model implementation and dissemination
(Forrester, 1975; Richardson & Pugh, 1981; Sterman, 2000). A facilitator/modeler guides
groups of experts through the stages of model building leading to what the authors call
convergent thinking among participants (Vennix et al., 1992, 30). Richardson and
iB
Andersen expand the number of essential and distinct roles in GMB to five (Richardson
& Andersen, 1995). GMB, the authors summarize, is speedier than other formats of
modeling. Richardson and Andersen also observe an increased sense of ownership of the
model and its simulation outcomes within the target audience. In other words, through
GMB it became clear rather quickly that this particular approach to model building also
helped raising the confidence in SD model building and model outcomes for both the
potential target audience and the modelers. Andersen and Richardson discuss a yet
refined version of the GMB process based on what they call scripts which help
orchestrate and control the GMB exercise (Andersen & Richardson, 1997). Since the
exercise is confined to a total of two consecutive business days for completing the first
three stages of model building, the authors see the necessity for organizing the GMB
process within strictly “controlled experimental settings” (p. 115). The authors define the
GMB exercise explicitly as an “intervention” (p. 126). In his 1999 prize lecture, Vennix
discusses how human perception and reality construction as sources of messy problems
impact GMB (Vennix, 1996). In Vennix’s view humans “process information and
construct models of reality” (p. 381). The information process is depicted as an
acquisition of “information from the environment” (p. 386). Vennix identifies three
“deficiencies” (p. 385) of human group interaction leading to distorted views of reality
and the creation of messy problems: (a) “mixing up of cognitive tasks, in particular the
production and evaluation of information” (ibid.), (b) lack, or even suppression, “of
critical investigation” (ibid.) leading to sort of a distorted perception of reality, and (c)
defensive, “low-quality communication” (p. 386). Those three deficiencies taken together
can lead, as Vennix argues, to group-think, arriving at conclusions prematurely, self-
Bex
fulfilling prophecies, and outright denial (“humans are inclined to explain away mistakes
and failures” (p. 388)). The author emphasizes that the facilitator’s instrumental role in
the GMB process based on specific attitudes and skills is to neutralize those deficiencies
and limitations resolving or, at least, addressing the messy problem through the modeling
exercise. Hence, the facilitator role is so fundamental Vennix asserts that it seems to be
incompatible with that of a modeler at the same time. Though not said explicitly, the
author seems to contend that GMB, when done correctly, can lead to an accurate
representation of reality in an ontological sense.
Akkermans and Vennix analyze a total of six GMB cases finding that (a) larger group
sizes negatively affect the GMB process, (b) good communication among participants in
the GMB process fosters a sense of model ownership, and (c) SD modeling expertise is
no prerequisite for good communication (Akkermans & Vennix, 1997). Rouwette et al
report in their meta-analysis of 107 GMB cases in the SD literature that (a) learning about
the problem at hand among participants was observable in most cases followed by (b) an
increase in insight (Rouwette et al., 2002). The exposure to hands-on modeling in GMB
seemingly leads to better learning among practitioners compared with practitioner use of
pre-fabricated micro-world models according to the authors. The implementation of
suggested policy changes does not seem to follow as a norm after GMB exercises, even
though commitment to model findings and consensus among participants seem to
increase. Quantitative models appear to produce more commitment and consensus among
participants according to the Rouwette et al study. Andersen and Richardson believe that
more research in GMB needs to be focused on the research team interaction and the
efficiency of the researcher-led modeling process (Andersen & Richardson, 1997). They
as ae
also propose integrating GMB techniques into the SD curricula (ibid.). Remarkably, the
“subjects” of the intervention and their ex-ante and post-hoc perceptions of the GMB
experience including outcomes and processes are nowhere mentioned as a worthwhile
area of academic discovery.
In summary, GMB practice serves three purposes (1) rapid model development through
parallel and simultaneous expert knowledge elicitation in tightly controlled and managed
environments, (2) increased model utility (as a proxy for validity), that is, structural and
simulation outcome acceptance within target audiences, and (3) increased application and
implementation of policy/strategy insights from modeling (cf., (Richardson, 1999)) and
ownership based on changed mental models (Vennix, 1996; Vennix et al., 1996). The
image of the SD researcher in this setting appears as an omnipotent and external expert
interventionist with maximum control over the process geared for modeling efficiency
and audience buy-in. Lane classifies this approach as interactive SD, which falls into the
paradigmatic quadrant of functionalist sociology (with regulation views on society while
claiming to produce objective knowledge in the social sciences) (Lane, 1999).
Action Research
The research tradition has drawn attention from a wider academic community after Fred
Blum’s now famous article appeared in Philosophy of Science in 1955 (Blum, 1955).
Action Research (AR) has roots in a number of disciplines including psychology, health
care/medicine, and education (cf., (McKernan, 1996)). While the term Action Research
was coined by Kurt Lewin in the mid 1940s (Sussman & Evered, 1978), the research
tradition has earlier roots, for example, in John Dewey’s experimentalist educational
research (McKernan, 1996)). Using a different research approach in social sciences than
a oe
in natural sciences turns out a necessity, as Checkland argues: In the study of a physical
phenomenon, for example, the researcher is almost naturally confined to an observer’s
role, however, remaining such an external observer in studying social phenomena is
“almost impossible” (Checkland, 1981, 153). When addressed via intervention, social
phenomena typically defy the application of an engineering perspective according to the
author. AR, hence, incorporates action and reflection upon the action. There is no
standardized approach in the AR tradition, it rather comes in many flavors and formats
such as educational AR, Action Learning, Action Science, Soft Systems Methodology,
and others (Baskerville & Pries-Heje, 1999; Flood, 2001). However, these different
formats have in common that (1) they actively involve practitioners and researchers in a
project; (2) both groups jointly pose a problem, target one or more practical actions to
address this problem, and study their outcomes; and (3) the project is equally dedicated to
the research side and to the action/outcome side (cf., (McKernan, 1996; McTaggart,
1997; Sussman & Evered, 1978)). In Rapoport’s definition AR “aims to contribute both
to the practical concerns of people in an immediate problematic situation and to the goals
of social science by joint collaboration within a mutually acceptable ethical framework”
(Rapoport, 1970, 499). Grundy emphasizes that AR projects take a “social practice,
regarding it as susceptible to improvement,” proceed “through a spiral of cycles” of
action and reflection, and involve “those responsible for the practice in each of the
moments of activity...maintaining collaborative control of the process” (Grundy, 1982,
23), As AR projects have the distinct characteristic to be designed in an iterative and
circular fashion, Sussman and Evered distinguish five phases of AR: “diagnosing, action
planning, action taking, evaluating, and specifying learning” (Sussman & Evered, 1978,
Gu
588), in which the learning phase leads to the next diagnosing phase in the next cycle. As
opposed to traditional sequential research methods seeking for evidence in testing
hypotheses, the follow-up cycles in AR implicitly or explicitly attempt to disconfirm
findings of previous cycle analog to the hermeneutic cycle (ibid.). Hence, confidence in
the soundness of AR findings builds inasmuch as rigorous attempts to establish counter-
evidence fail and as the practical problem appears adequately addressed in the view of
both researchers and practitioners. The action planned and taken in an AR project is
aimed at informing and improving the understanding about the very action by iteratively
reflecting upon the results of the action and by contemplating the meaning of the action
within a context of external constraints (Carr & Kemmis, 1986). As Grundy observes, AR
is conducted from three different philosophy-of-science perspectives (“modes”), which
lead to different AR designs: (1) the technical, (2) the practical, and (3) the emancipatory
perspective (Grundy, 1982, 1987). The technical perspective would resemble traditional
research in that it is expert-driven and geared to control the research process as well as
the outcome (ibid). In Burrell and Morgan’s framework, this approach would qualify as
functionalist. Though practitioners participate, the process and its facilitation unfold
within the constraints of the researcher’s design. Technical action research intends to
improve a situation according to externally defined criteria (Carr & Kemmis, 1986).
Participants may be personally committed to and actually “play “the ‘action research
game’. Their actions and deliberations are authentic within the context of the project and
designed to achieve the action research goal, but once the ‘game’ is over they are no
longer obliged to act according to its rules’ (Grundy, 1982, 26). The technical AR
researcher typically finds that practitioners “revert” to their old ways of doing and
my ¢ ee
thinking shortly after the intervention (ibid.). As opposed to technical AR, practical or
participatory AR (also referred to as mutual-collaborative) (Grundy, 1987; McKernan,
1996), gives practitioners and researchers an equal footing with respect to the definition
of the problem at hand, what actions might be taken, and how results are interpreted. In
Grundy’s words, while “{t}echnical action research seeks to improve practice through
the practical skills of the participants,” participatory AR “seeks to improve practice
through the application of the wisdom of the participants” (Grundy, 1982, 27). While
technical AR is rooted in a positivist understanding of science, in which a single,
discoverable, objective reality is assumed, and consequently knower and known are seen
as separable, in participatory AR, reality is assumed to be socially constructed, and
knower and known appear as intertwined (ibid.). According to Burrell and Morgan, this
variant of AR would belong to interpretive sociology (Burrell & Morgan, 1979).
Emancipatory AR is rooted in Critical Sciences (Habermas, 1974) and emphasizes an
egalitarian and value-oriented approach to science geared at social change and
emancipation. In the Burrell-Morgan framework, emancipatory AR would be situated in
the quadrant of radical humanism (Burrell & Morgan, 1979). Grundy argues, though
practitioners and researchers entertain and maintain a relationship of quasi-peers within
the AR project, that participatory AR reaches its limitations when it comes to power-
based “institutional restrictions” to desired change (Grundy, 1982, 28). Hence,
emancipatory AR “focuses not only upon a particular practice, but also the theoretical
and organizational structures and social relations which support it” (ibid.). It assumes
multiple realities based on vested interests and on inequity in a social milieu and seeks
emancipation and conscious change based on enlightenment towards increased equity
ey fe
(ibid.). Grundy summarizes “that it is not in the methodology that these three modes (of
AR-insertion mine) differ, but the underlying assumptions and worldviews of the
participants that cause subtle variations in the application of methodology” (p. 33).
In summary, AR is an iterative, action-reflection research methodology relying on active
practitioner involvement within a social setting geared at improving a problematic
(social) situation. In this, AR provides usually no causal explanations and rests on
primarily qualitative data. The definition of the problem under study is typically driven
by practitioner needs and is not as precise as in other research formats. Even for technical
AR, researcher impartiality and experiment-like control over the research process are
limited, in participatory or emancipatory AR, they are not even claimed. AR is
situational, that is, the process if repeated would not be identical, nor would it produce
the identical results. While AR projects help build and partially test theory,
generalizabilty as typically pursued in traditional research is not a thrust of AR.
How to Use SD Modeling within a Participatory Action Research Design?
As Homer states, SD modeling is an iterative process, in which due to the “wide range of
known detail” (Homer, 1996, 3) revisions are incorporated throughout the whole
modeling effort until the time of completion. Also, in modeling surprises regarding model
behavior are rather the norm than the exception. The modeler assumes the roles of “data
detective, compiler, and analyst” at the same time (p. 17). Revising a model can be as
insightful a process as conceptualizing it according to the Homer.
< Insert Figure 1 about here>
ey i
Sterman illustrates this and shows how modeling is governed by feedback processes, in
which “{m}odels go through constant iteration, continual questioning, testing and
refinement” (Sterman, 2000, 87), see Figure 1.
Like SD modeling, AR also has a circular research design comprising five phases as
depicted in Figure 2.
< Insert Figure 2 about here >
While at first sight those phases and stages may appear similar between the two
approaches, a closer look reveals that they are geared towards different ends. While in the
SD cycle a sound and useful model is the desired result, in the case of AR, organizational
and social change in form of an iteratively implemented and tested solution to a practical
problem is expected. This difference in orientation becomes evident when looking at the
AR cycle stages:
In the diagnosing/posing phase, the problem to be studied and acted upon is diagnosed
(or posed) as McTaggart asserts (McTaggart, 1997). Problems in this regard the author
maintains are not seen as a “pathologies” (p. 39) but as an intermediate result from
previous action within an organizational setting which has a capacity for improvement
via change. The diagnosing/posing phase provides the stage for reflection upon the
results of previous action taken as they became manifest within the organizational
context. As Baskerville points out, in this phase participants, that is, practitioners and
ey be
researchers, jointly develop a theory “about the nature of the organization and its problem
domain” (Baskerville, 1999, 15). The relationship between researcher and practitioners is
not one of experts imparting their knowledge on students, or doctors treating patients, but
one of peers (cf., (Schein, 1988)). In the action planning the participants consider and
define actions that may have the capacity to improve the diagnosed situation. The theory
developed in the anteceding phase provides the frame of reference for this planning
phase. Actions are planned and their expected outcomes are specified. In the third phase
of action taking the participants undertake the action within the organizational setting.
Such action taking can span over longer periods of time. Directive and non-directive
change tactics may be applied (cf., (Baskerville, 1999)). In the next phase of evaluating,
the effects and outcomes of the change actions are analyzed and compared with the
expected outcomes as specified in the planning phase. AR project participants critically
assess whether or not the pre-specified results and expected outcomes were achieved.
They attempt to understand whether the theory had sufficiently guided the action, and
also whether the actions were undertaken as planned. In the AR cycle’s concluding phase
of specifying learning the new insights from the previous phases are formally accounted
for, and new theory emerges. All participants also jointly perform this last phase. As
Baskerville remarks, “the success of failure of the theoretical framework provides
important knowledge to the scientific community for dealing with future research
settings” (p. 16). AR cycles are iterated at least once geared at disconfirming the findings
of the previous cycle. Depending on the nature of the problem diagnosed and re-
diagnosed there may be numerous iterations. AR projects, hence, facilitate and fuel
organizational change processes.
=[4=
Practical Use of SD in a Participatory Action Research Project
From an AR perspective, using the SD cycle could facilitate every phase in the AR cycle.
For example, the diagnosing phase could incorporate a complete SD cycle, that is,
formulating and evaluating a formal model of the stated problem, which would help
guide the next phase of action planning by considering policy alternatives as they become
apparent through the iterative modeling process. While the action is taken, the simulation
model could be instrumental in identifying and assessing alternatives. Data from action
would help calibrate the model. In the AR evaluation phase, the SD model would provide
arich frame of theoretical references. Action outcomes would be compared to the
simulated outcomes. The specification of learning would also take the form of extending
and reformulating the model. Arriving at the diagnosing phase would provide for a well-
founded theoretical base rooted in organizational practice and augmented by quasi-
experimental insights from the calibrated model.
Why the Integration of Participatory AR into the SD cycle might be problematic. As
the discussion on GMB indicated, SD researchers seek and practice the collaboration
with practitioners in model building. However, the collaboration is anything but one of
peers. The SD researchers maintain the role of expert facilitators who control the GMB
process along the lines of rapid model building. This approach, if at all, most closely
resembles the technical AR variant. Widespread modeling skills among practitioners
would be desireable (cf., (Sterman, 1994)) and would be even necessary from the outset
in order to facilitate participation and create a peer group situation. However, the “action”
planned and taken would focus on the modeling exercise, not on changing an
organizational setting. The modeling routine might limit the exploration of other avenues
Py be
of practical experimentation and action. Open or hidden hierarchies along the lines of
modeling proficiency or personal reputation might emerge. There also seems to be the
tisk of modeling to dominate the process and not practical problem solving.
Different Distances to Consulting. SD is so widely used in commercial consulting
practice that the term “client” has become the proxy for practitioners in the field’s
academic papers and even textbooks alike (cf., the System Dynamics Review, the papers
cited here, as well as (Sterman, 2000)). Not only in this paper, but also in almost the
entire SD literature the practitioner audience is referred to as “clients” (p. 107), and GMB
is also labeled as “client-centered system dynamics modeling” (p. 108). Vennix labels the
researcher in GMB as “interventionist” (Vennix, 1996, 382). When attempting to
understand the deeper meaning of the term “client” in this context, it is noteworthy that
“modeling for management” (Sterman, 1996) is a common notion in the SD community
indicating not only a close connection to managerial practice and decision making but
also to the widespread use of SD as a commercial consulting tool (cf., (Thompson,
1999)). In Schein’s taxonomy of consulting, the SD community seems to assume a
doctor-patient relationship rather than engaging in process consultation (Schein, 1988). It
is also noteworthy to remember that academic research and commercial consulting, even
if the latter is conducted on solid scientific foundations, are serving different ends. As
Gill and Johnson argue, those differences are numerous and have important consequences
(Gill & Johnson, 2002).
<Insert Table | about here >
wl Rw
In their synopsis of traditional research consultancy, and action research (see Table 1),
the authors point at differences between the approaches to problems from the initial
problem formulation through the final results in theory and practice. While in traditional
research, the researcher defines both the problem and the research design, in AR
practitioners and researchers jointly do this including specifying the research goals. In
this regard, consultancy marks the middle of the road between the two, in that the client
reports the problem, while the consultant defines the treatment. The traditional expert
researcher (and also the consultant) performs the detailed diagnosis of the problem on the
basis of client data, while in participatory AR researchers and practitioners jointly carry
out this diagnosis on the basis of practitioner-provided data and researcher concepts. In
consulting, if action is taken, the consultant prescribes it, while in participatory AR the
action is jointly planned and executed. The detailed results remain unpublished in
consulting, while in AR and traditional science they are made public to the academic
community. Evaluations of results for practice are rarely conducted in the cases of both
traditional science and consulting, while they are the norm in AR. It is in this phase
where new problems are articulated and new theory begins to form. The most striking
difference is observable after the intervention is completed. While in traditional research
as well as in consultancy, the client remains dependent, participatory AR aims at the
practitioner’s self-support (for an overview, cf. also (Darwin, 1999)). The motivation in
AR is oriented towards advancing scientific understanding, while consulting first and
foremost serves a defined commercial interest. While the consultant’s commitment is
dedicated to her client, the AR scholar is committed to both the practitioner and the
academic community. Consultants approach a problem typically along the sequence of
oy Wt fe
the classical prescription of “engage-analyze-act-disengage” (Schein, 1988) whereas AR
is cyclical, iterative, and collaborative by definition. Hence, change emerges within a
jointly undertaken self-experiment in AR, while it is based on the consultant’s external
and independent analysis. Before this background, it seems obvious that SD research has
its roots in traditional (positivist) science where SD experts design the models and tightly
control “real-world” interventions (cf., also (Lane, 1999)). In this regard, the consulting
paradigm is much closer to traditional (positivist) science than to AR. However, if
leading SD researchers have a strong practical engagement in consulting, the borders
between the two domains may blur. More importantly, the two domains reinforce the
notion of expert-controlled, tightly scripted interventions in those with one foot in either
domain, where the practitioner merely appears as a source of data. Moreover, based on
insights from experiments, influential SD scholars have developed a deep skepticism
regarding the capacities of rational reasoning and learning in human beings when dealing
with complex systems (Sterman, 1989a, 1989b). This may, consciously or not, also
reinforce the tendency in SD research to rely on tight expert control rather than on
participatory designs geared at jointly gained insight and resulting consensus.
A Sketched Proposal for Empirical Testing
When discussing the various observed and potential uses of SD modeling across
paradigms, Lane seemingly sympathizes with integrative approaches that help
practitioners and researchers alike to arrive at a shared interpretation and an inter-
subjective “reality”, that is, an understanding of their practice mediated by means of
dynamic modeling (Lane, 1999). Uses of SD that support learning and “further
communicative competence within groups” (p. 518), he sees as most far-reaching and
ey bee
promising (cf., also, (Lane & Oliva, 1998)). This proposal follows this avenue and briefly
sketches out an empirical test of intertwining the AR and SD. In a participatory action
research project researchers become co-subjects and peers to practitioners. For
experienced SD modelers, this may be an unusual and even uncomfortable position in a
group and a group process.
Entry Interview. The empirical test needs to establish a point of departure first. In semi-
structured entry interviews, the practitioners and researchers who will work together in a
group will be questioned regarding their practical experience with participatory action
research, their exposure to literature on AR including SSM as well as their exposure and
proficiency regarding formal modeling techniques including system dynamics modeling.
The information will be shared among co-subjects/co-researchers.
Establishing a Peer Relationship. During the diagnosing phase of the first AR cycle the
SD modeler assumes the role of a participant who helps identify the focus of the inquiry,
the particular question, and the specific practical problem, the group wants to focus on.
Shee helps select and plan the action to be taken. As Heron and Reason propose, in this
phase, also procedures for gathering and recording data from action taking are selected
(Heron & Reason, 2001). In the diagnosing and problem definition phase, no reference to
feedback structures should be made, since it is assumed that not everybody understands
that concept from the outset.
Introducing the Concept of Feedback. During the planning phase, the SD modeler
introduces the concept of feedback to the group, thus, providing an additional lens for
observation and reflecting on experience to be made when taking the action. Also,
expected outcomes from taking the action are qualified and quantified. The SD modeler
~{0=
may begin to identify important variables in his personal recordings. During the next
stage, the SD modeler immerses herself along with the other group members into taking
the planned action. She observes and records outcomes. Unexpected outcomes and
experiences are carefully recorded.
Early Conceptualizing. Since she has little chance to abstain from it, the SD modeler
might begin to identify feedback structures, but without sharing them with the other
members of the group yet. In taking action, new opportunities for attacking the problem
may emerge and be tested without prior planning. In the next phase, the group members,
that is, the co-subjects/co-researchers including the SD modeler share their data and
observations comparing them to their original assumptions, expected outcomes, and
ideas. They begin to discuss ideas how to adjust the approach to solving the problem. It is
here where the SD modeler begins to present his findings and evaluation in feedback
view.
Becoming a Peer Educator. In small experiments, it has been found that group members
become attracted to the feedback perspective and begin to provide examples and
additional structure from their own experience. Since the SD modeler at this stage has
become a co-subject/co-researcher and peer member in the group with the capacity to
draw from own experience regarding the problem at hand, the views she presents have
become internal views of a group member rather than an imposed frame of external
thinking. The SD modeler may now begin to capture for herself the feedback structures
observed in one or more small models. She may begin to experiment with those mini-
models. She may also let other group members have a look at those mini-models, and the
behavior they expose. At this stage, some group members may become interested in
ts @ ere
acquainting themselves with the SD modeling principles and techniques. This is when the
SD modeler takes on the role of a peer-educator. It is essential at this point not to
sacrifice the role as a peer and co-subject. In other words, heavy-duty SD teaching would
not be conducive to that end, but change the SD modeler’s role and the group
composition. The SD modeler could, however, introduce self-learning material on SD at
this point.
Using Mini-Models. With more group members becoming familiar with the feedback
approach and having introductory-level modeling experience, the next AR cycle of
problem diagnosis and action planning begins. Early insights from the emerging feedback
structure inherent in the problem lead to refined formulations of planned action and
ensuing reflection. Since the AR project typically spans multiple months, if not years,
sufficient time is available to more systematically educate those group members
interested in SD modeling. This task shall not be assumed by the project-internal SD
modeler, but by an external qualified teacher. While the AR cycles unfold, the use of SD
modeling increases when analyzing and interpreting the outcomes and experiences from
dealing with the practical problem.
Gradually Expanding Model Use and Scope. The mini-models may grow into sectors
of a larger model. Since the model remains closely tied to the action surrounding the
practical problem, its relative usefulness remains evident to the group members. The
modeling process in this approach becomes an integral part of all stages of the AR cycle.
The group members remain focused on the problem they have identified. Modeling and
feedback thought is not imposed by an external authority but in a bootstrapped fashion
from inside the project. The usage of SD is limited to those aspects of the problem, the
apie
SD modeler and the SD-educated group members are able to identify and model. In this
design, ownership in the model is expected to emerge naturally. Of whatever quality the
final model will be, it will have served (along with the SD modeler) as a sounding board
and vehicle in the AR inquiry and problem solving cycles.
Using Journals and Exit Interviews. In AP projects, individuals and the group as a
whole keep journals, in which they record project progress, important observation,
decisions, and actions throughout the project. The SD modeler needs to maintain a
separate journal, in which the progress of the modeling component and the group’s
observed understanding and use of SD modeling is recorded. The SD modeler also
assesses individual group members’ SD modeling proficiency. In semi-structured exit
interviews, each group member will be asked to portray her individual learning in the
project. Particular emphasis will be laid on having group members describe the influence
of the feedback perspective and the modeling component on project outcome. Finally, the
SD modeler assesses the degree, to which policy changes relatable to insights garnered
from the SD models were incorporated into organizational practice.
Concluding Remarks
Participatory AR and SD have a number of intersections and similarities that make it
worthwhile to consider research designs, which attempt to intertwine the two
methodologies. Paradigmatic consensus exists at least between SD GMB and technical
AR. In both approaches, though practitioners are involved throughout the process, the
expert/facilitator maintains maximum control over the process, and the distribution of
power regarding the process is asymmetrical. As a consequence, the practitioner
commitment to action and change beyond the intervention seems to be weak (cf.,
a) ws
(Grundy, 1982; Rouwette et al., 2002)). In other words, though the SD researcher may
perceive the emerged model as a correct and flawless representation of the “real world”
according to accepted external criteria, and even though the practitioners confirm a high
degree of learning and insights gained, no change action is taken, and the intervention
leads to no further consequences. From a model relevancy and model utility standpoint
thi
is a truly unsatisfactory, and even frustrating, outcome. As outlined before, SD
modelers have responded to this situation with increased model validation and
verification efforts. However, as Oreskes et al demonstrate, those approaches to
quantitative model verification, validation, or confirmation fail to produce the desired
results (Oreskes, Shrader-Frechette, & Belitz, 1994). For example, the strategy of
demonstrating a model’s fit to a time series is riddled by at least the two problems of
affirmation of the consequent as well as of underdetermination. In the first case, the three
authors demonstrate that no model verification follows from time-series fit for an obvious
fallacy in syllogistic reasoning. Even if the deductive reasoning was sound, it could not
claim truth as Austin shows (Austin, 2002). The case of underdetermination presents
another principal pitfall for any model’s verification and validation, since its uniqueness
in producing the observed outcome cannot be demonstrated. Therefore, Oreskes et al
propose to shift model assessment from numerically verifying, validating, or confirming
to testing for the model’s usefulness when challenging existing theory (Oreskes et al.,
1994). Forrester also argues in favor of testing a model’s utility against a stated purpose
rather than pursuing its unattainable validation (Forrester, 1961). If a model’s utility is
what counts, then its role in practical policy/organizational change defines the test. The
advance from learning to action in this regard hinges upon practitioners’ commitment to
Pe
act. The commitment, in turn, depends upon the ownership of and involvement in the
process. If practitioners are excluded from finding and defining the criteria of reference in
the process, they have no control over their learning process. Hence, generative (Senge,
1990) or double-loop (Argyris & Schon, 1974) learning on behalf of the practitioners is
effectively barred. In other words, the tight control comes at a high price. The consulting
practice also plays a role in inhibiting learning at the technical level. If not contractually
forced, consultants typically do not share the “tricks of the trade,” that is, the technical
skills of SD modeling building with practitioners they are serving. Keeping the
practitioner, “the client” rather, dependent makes good business sense for the consultant
but produces little learning for the practitioner, particularly, in terms of double-loop
learning. Developing the practitioners’ skills and self-support including SD modeling,
however, would be the key to sustainable change. It would, thus, be worthwhile to design
a longitudinal, participatory AR project with a SD modeling component along the lines
discussed before. The rules of engagement as well as the roles of practitioners and
researchers need to be re-thought, though. In Carr and Kemmis’s words, in participatory
AR, “participants monitor their own ... practices with the immediate aim of developing
their practical judgment as individuals. Thus, the facilitator’s role is Socratic: to provide a
sounding board against which practitioners try out ideas and learn more about the reasons
for their own action, as well as learning more about the process of self-reflection” (Carr
& Kemmis, 1986, 203). In the same vein, Grundy observes that while in technical AR
“the facilitator controls the project’, in participatory AR “power is shared” directly
leading to action (Grundy, 1982, 33). Participatory AR projects could benefit in various
ways from SD researchers as peers in the project team. Practitioners would learn from SD
4 =
researchers how to express aspects of the problem in feedback structures and modeling
terms. The SD researchers would be “sounding boards” rather than expert facilitators. As
modeling proficiency among practitioners increases, the SD researcher’s educator role
decreases, and the project progresses towards action planning and action taking. The SD
model would not act as an embodiment of an overarching and imposed theoretical
framework but naturally evolve with the skills and the insights from action planning and
action taking. Rather than being perceived and intended as correct representations of the
“real-world” problem, the evolving models would likewise play the role of sounding
boards with great utility for learning and insight. Ongoing modeling while proceeding
through the AR cycle, hence, would add the critical stage of continuous comparing and
checking the model(s) against observations and experiences via action (Checkland,
1981). The main difference between current GMB practice and the here proposed SD-
enriched participatory AR lies in the recognition of and the reliance on the social process
and the shared social codes of participants in actively creating knowledge and
understanding (cf., (Glasersfeld, 1995)) leading to jointly created perceptions of reality
upon which is jointly acted. This approach explicitly rejects the notion of “management
flight simulator,” which treats human organization like a deterministic physical system.
As Sussman and Evered point out, as opposed to such systems, “the nonrandomness or
the structuredness of a social system results from shared codes of conduct or rules of its
members...” which accommodate to “personal investments. ..conflicts over power,
prestige, and attention” (Sussman & Evered, 1978, 594). The “solution” to any problem
in human organization is inescapably a socially moderated, not a mere technical one.
Model utility resides in this context, too.
ap ew
Appendix
1. Problem Articulation
(Boundary Selection)
5. Policy |
Formulation & 2. Dynamic
Evaluation Hypothesis
4. Testing 3. Formulation
w_ESESee
The System Dynamics Modeling Cycle
Sterman (2000)
Figure 1 The SD Modeling Cycle
i ee
1. Diagnosing /
Posing the
Problem
5. Specifying 2. Action
Learning Planning
3. Action
4. Evaluating Taking
TheAction Research Cycle.
Sussman & Evered (1978)
Figure 2 The Action Research Cycle
Pt af fee
Stages Action Consulting Basic Research
Research
Entry Client or researcher | Client presents Researcher
presents problem, | problems and presents problems
mutually agreed defines goals and defines goals
goals
Contracting Business and Business contract, | Researcher
psychological consultant controls | controls as expert.
contracting, mutual | client Keeps client
control happy. Minimal
contracting
Diagnosis Joint diagnosis. Consultant Researcher carries
Client data / diagnosis. Often out expert
researcher’s minimal. Sells diagnosis. Client
concepts package provides data
Action Feedback, Consultant Report often
dissonance, joint prescribes action. | designed to
action plan. Client | Not published impress client with
action with how much
support. Published researcher has
learned and how
competent he or
she is. Published
Evaluation New problems Rarely undertaken | Rarely undertaken
emerge. Recycles. | by neutrals
Generalizations
emerge
Withdrawal Client self- Client dependent Client dependent
supporting
Table 1 Action Research versus Consulting (Gill & Johnson, 2002, p. 76)
References
Ackoff, R. L. (1974). Redesigning the future: a systems approach to societal problems.
New York,: Wiley.
Akkermans, H. A., & Vennix, J. A. M. (1997). Clients' opinions on group model
building: An exploratory study. System Dynamics Review, 13(1), 3-31.
Andersen, D. F., & Richardson, G. P. (1997). Scripts for group model building. System
Dynamics Review, 13(2), 107-129.
Argyris, C., & Schon, D. A. (1974). Theory in practice : increasing professional
effectiveness (1st ed.). San Francisco: Jossey-Bass Publishers.
Austin, M. (2002). Deductive and inductive arguments. Retrieved 2/23/02, 2002, from
http://webpages.shepherd.edu/maustin/rhetoric/deductiv.htm
Baskerville, R. L. (1999). Investigating information systems with action research.
Communications of the Association for Information Systems, 2(19), 1-32.
Baskerville, R. L., & Pries-Heje, J. (1999). Grounded action research: a method for
understanding IT in practice. Accounting, Management, and Information
Technologies, 9(19), 1-23.
Blum, F. (1955). Action research: A scientific approach? Philosophy of Science, 22, 1-7
Burrell, G., & Morgan, G. (1979). Sociological paradigms and organisational analysis :
elements of the sociology of corporate life. London: Heinemann.
Carr, W., & Kemmis, S. (1986). Becoming critical : education, knowledge, and action
research. London ; Philadelphia: Falmer Press.
Checkland, P. (1981). Systems thinking, systems practice. Chichester Sussex ; New York:
J. Wiley.
Darwin, J. (1999). Action research: Theory, practice and trade union involvement.
Retrieved 1/20, 2001, from http://www.shu.ac.uk/schools/sbs/cslc/99-06 - J
Darwin.doc
Flood, R. L. (2001). The relationship of 'systems thinking' to action research. In P.
Reason & H. Bradbury (Eds.), Handbook of action research : participative
inquiry and practice (pp. 133-144). London ; Thousand Oaks, Calif.: SAGE.
Ford, D. N., & Sterman, J. D. (1997). Expert knowledge elicitation to improve formal and
mental models. System Dynamics Review, 14(4), 309-340.
Forrester, J. W. (1961). Industrial dynamics. Cambridge, Mass.: M.I.T. Press.
Forrester, J. W. (1975). Collected papers of Jay W. Forrester. Cambridge, Mass.: Wright-
Allen Press.
Forrester, J. W. (1980). Information sources for modeling the national economy. Journal
of the American Statistical Association, 75(371), 555-566.
Forrester, J. W., & Senge, P. M. (1996). Tests for building confidence in system
dynamics models. In G. P. Richardson (Ed.), Modelling for management:
simulation in support of systems thinking. (Vol. 2, pp. 414-434). Aldershot, Hants,
England ; Brookfield, Vt.: Dartmouth.
Gill, J., & Johnson, P. (2002). Research methods for managers (3rd ed.). London ;
Thousand Oaks, Calif.: Sage Publications.
200 x
Glasersfeld, E. v. (1995). Radical constructivism : a way of knowing and learning.
London ; Washington, D.C.: Falmer Press.
Graham, A. K., Choi, C. Y., & Mullen, T. W. (2002, 1/7-1/10). Using Fit-Constrained
Monte Carlo Trials to Quantify Confidence in Simulation
Model Outcomes. Paper presented at the Hawaiian International Conference on System
Sciences, Waikoloa, Hawaii.
Grundy, S. (1982). Three modes of action research. Curriculum perspectives, 2(3), 23-24.
Grundy, S. (1987). Curriculum : product or praxis? London ; New York: Falmer Press.
Habermas, J. (1974). Theory and practice. London: Heinemann.
Heron, J., & Reason, P. (2001). The practice of co-operative inquiry: Research 'with'
rather than 'on' people. In P. Reason & H. Bradbury (Eds.), Handbook of action
research : participative inquiry and practice (pp. 179-188). London ; Thousand
Oaks, Calif.: SAGE.
Homer, J. B. (1996). Why we iterate: scientific modeling in theory and practice. System
Dynamics Review, 12(1), 1-19.
Lane, D. C. (1999). Social theory and system dynamics practice. European Journal of
Operational Research, 113, 501-527.
Lane, D. C. (2001). Rerum cognoscere causas: Part II-opportunities generated by the
agency/structure debate and suggestions for clarifying the social theoretic position
of system dynamics. System Dynamics Review, 17(4), 293-309.
Lane, D. C., & Oliva, R. (1998). The greater whole: Towards a synthesis of system
dynamics and soft systems methodology. European Journal of Operational
Research, 107, 214-235.
McKernan, J. (1996). Curriculum Action Research: A handbook of methods for the
reflective practitioner (2nd ed.). London: Kogan Page.
McTaggart, R. (1997). Guiding principles for participatory action research. In R.
McTaggart (Ed.), Participatory action research : international contexts and
consequences (pp. 25-43). Albany: State University of New York Press.
Oreskes, N. K., Shrader-Frechette, K., & Belitz, K. (1994). Verification, validation, and
confirmation of numerical models in the earth sciences. Science, 263, 641-646.
Rapoport, R. N. (1970). Three dilemmas of action research. Human Relations, 23(6),
499-513.
Richardson, G. P. (1999). The 1999 Jay Wright Forrester award. System Dynamics
Review, 15(4), 375-377.
Richardson, G. P., & Andersen, D. f. (1995). Teamwork in Group Model Building.
System Dynamics Review, 11(2), 113-137.
Richardson, G. P., & Pugh, A. L. (1981). Introduction to system dynamics modeling with
DYNAMO. Cambridge, Mass.: MIT Press.
Rouwette, E. A. J. A., Vennix, J. A. M., & Mullekom, T. v. (2002). Group model
effectiveness: A review of assessment studies. System Dynamics Review, 18(1), 5-
45.
Schein, E. H. (1988). Process consultation: Its role in organizational development (Vol.
1). Reading, Mass.,: Addison-Wesley Pub. Co.
Senge, P. M. (1990). The fifth discipline : the art and practice of the learning
organization (1st ed.). New York: Doubleday/Currency.
ay ws
Sterman, J. (2000). Business dynamics : systems thinking and modeling for a complex
world. Boston: Irwin/McGraw-Hill.
Sterman, J. D. (1989a). Misperceptions of feedback in dynamic decision making.
Organizational Behavior and Human Decision Processing, 43, 301-335.
Sterman, J. D. (1989b). Modeling managerial behavior: Misperceptions of feedbackin a
dynamic decision making experiment. Management Science, 35(3), 321-339.
Sterman, J. D. (1994). Learning in and about complex systems. System Dynamics Review,
10(2-3), 291-330.
Sterman, J. D. (1996). Modeling managerial behavior: misperceptions of feedback in a
dynamic decision making experiment. In G. P. Richardson (Ed.), Modelling for
management : simulation in support of systems thinking (Vol. 1, pp. 209-227).
Aldershot, Hants, England ; Brookfield, Vt.: Dartmouth.
Sussman, G., & Evered, R. (1978). An assessment of the scientific merits of action
research. Administrative Science Quarterly, 23, 582-603.
Thompson, J. P. (1999). Consulting approaches with system dynamics: Three case
studies. System Dynamics Review, 15(1), 95.
Van den Belt, M., Videira, N., Antunes, P., Santos, R., & Gamito, S. (2000). Mediated
modeling in Ria Formosa, Portugal. Retrieved 1/18, 2004, from
http://gasa.dcea.fct.unl.pt/ecoman/projects/RiaFormosa/modelo_e.htm
Vennix, J. A. M. (1996). Group model building : facilitating team learning using system
dynamics. Chichester ; New York: J. Wiley.
Vennix, J. A. M., Akkermans, H. A., & Rouwette, E. A. J. A. (1996). Group model-
building to facilitate organizational change: An exploratory study. System
Dynamics Review, 12(1), 39-58.
Vennix, J. A. M., Andersen, D. F., Richardson, G. P., & Rohrbaugh, J. (1992). Model-
building for group decision support: Issues and alternatives in knowledge
elicitation. European Journal of Operational Research, 59(1), 28-41.
Vennix, J. A. M., & Gubbels, J. W. (1992). Knowledge elicitation in conceptual model
building: A case study in modeling a regional Dutch health care system.
European Journal of Operational Research, 59(1), 85-100.
Vennix, J. A. M., Gubbels, J. W., Post, D., & Poppen, H. J. (1988, July). A structure
approach to knowledge acquisition in model development. Paper presented at the
International Conference of the Systems Dynamics Society, La Jolla, CA.
ee } es
Back to the
