Dynamics of Innovation Implementation and
Organizational Performance in Mental Health Services’
Peter S. Hovmand, Ph.D.
George Warren Brown School of Social Work
Washington University in St. Louis
Campus Box 1196
One Brookings Drive
St. Louis, MO 63130, USA
(314) 935 7968
phovmand@ wustl.edu
David F. Gillespie, Ph.D.
George Warren Brown School of Social Work
Washington University in St. Louis
Campus Box 1196
One Brookings Drive
St. Louis, MO 63130, USA
(314) 935 6674
davidfg@ fidnet.com
Abstract
While organizational variables play an important role in the adoption and
implementation of evidence based practices in mental health, most researchers have assumed
that successful implementation leads to improving organizational performance. Yet existing
organizational theory suggests that implementation differs by organizational characteristics, and
certain configurations can lower organizational performance. This study shows how
implementation of evidence based practice impacts organizational performance. Specifically, we
present a system dynamics simulation model of implementation and organizational performance
based on existing theory, system dynamics research, and key informant interviews. By varying
organizational characteristics we learn how implementation affects organizational performance,
and then explain these effects through subsequent behavioral analysis. These analyses led to a
simplification of the theory and model for understanding performance following the
implementation of evidence based practice. The theory implies that benefits from evidence-based
practice depend on how fast managers can implement the innovation relative to the quality
improvement process.
' The research and preparation of this paper was supported in part by the Center for Mental Health Services
Research, George Warren Brown School of Social Work, Washington University; through an award from the
National Institute of Mental Health (P30 MH068579). Paper presented at the Intemational System Dynamics
Conference July 29-August 2, 2007 in Boston, MA.
Innovation Implementation 2
Keywords: innovation implementation, mental health services, evidence-based practice
1. Introduction
How does evidence-based practice help mental heath agencies improve performance?
Mental health agencies have long fought a battle to defend the legitimacy of mental health
treatment and recovery through an appeal to empirical research and evaluation of outcomes. The
current version of this struggle for legitimacy is the evidence-based practice (EBP) movement,
based on evidence-based medicine, defined as “the conscientious, explicit, and judicious use of
current best evidence in making decisions about the care of individual patients” (Sackett et al.
1996). The term has now spread to nursing, teaching, management, and mental health, where it is
claimed that EBP has the potential to improve the quality of care, contain costs, and shape better
policies (Gonzales, Ringeisen, and Chambers 2002).
Despite the EBP potential, gaps persist and researchers continue pushing for more studies
to understand the implementation processes of EBP (Sliverman, Kurtines, and Hoagwood 2004;
Proctor 2004; Gonzales, Ringeisen, and Chambers 2002). This research has recognized the
importance of organizational barriers to implementing EBP (Bartels et al. 2002; Gonzales,
Ringeisen, and Chambers 2002; Rosen 1994; Hoagwood et al. 2001; Newman, Papdopoulous,
and Sigsworth 1998; Rosenheck 2001; Schoenwald and Hoagwood 2001). However, most
researchers have assumed that once implemented, these innovations will benefit the agency and
thereby improve organizational performance.
Developing a theoretical understanding of how implementing EBP improves performance
is critical to helping mental health organizations plan and manage innovation and organizational
change. It also helps us understand when it is better to focus on building the capacity of
organizations as opposed to implementing evidence-based practices. In this paper, we address
this gap by presenting results from a simulation model of implementation and organizational
performance. The model is based on previous system dynamics models of organizational change,
organizational theory, and key informant interviews with administrators of mental health
services. We then use the model of implementation and organizational performance to answer
the following two questions:
1. Under what initial conditions does organizational performance improve as a
consequence of implementing evidence based practice innovations?
2. Which mechanisms account for the organizational performance trajectories within
each region of performance change?
In addressing these questions we develop insights into the dynamics confronting
managers and policy makers on an important problem in mental health services research. We
also demonstrate the feasibility and benefits of integrating and extending existing system
dynamics models to develop and test social theories.
2. Background
Innovations with the potential to improve both service outcomes and legitimacy are
especially appealing to mental health and other social service organizations. Mental health
service organizations, relative to other industries such as health care or manufacturing, generally
Innovation Implementation 3
have difficulties demonstrating outcomes and benefits of their services. These difficulties make
mental health service organizations seek to protect their funding by ensuring legitimacy with
stakeholders. Traditionally, this legitimacy was values-based and defended ideologically.
Innovations that help demonstrate the scientific merits of mental health interventions will
enhance the organization’s legitimacy with stakeholders and reduce the likelihood of
organizational failure. Innovations such as EBP, therefore, hold the promise of maintaining
organizational survival in increasingly competitive sectors and improving organizational
performance. Accordingly, mental health organizations have strong incentives to adopt and
implement EBP.
At the same time, adopting and implementing EBP can set in motion a new set of internal
and external organizational demands that can threaten performance. For example, the epitome of
successful implementation of EBP might be when the organizational culture is oriented toward
ensuring the highest quality of evidence-based services. Y et, this very commitment to a way of
doing things, as well as its demonstrated success, will make it more difficult for the organization
to adapt to new demands— an example of organizational inertia. Externally, increased quality of
services from successful implementation of EBP is likely to increase the demand for services.
This can push the agency past its capacity and force staff to restrict access to services or sacrifice
quality, either of which will undermine subsequent organizational performance. The special
appeal of EBP combined with delayed effects creates a situation in which a service organization
could enter a vicious cycle of increasing implementation and declining performance, eventually
terminating with organizational failure.
While barriers to implementing EBP are well-recognized (Bartels et al. 2002; Gonzales,
Ringeisen, and Chambers 2002; Rosen 1994; Hoagwood et al. 2001; Newman, Papdopoulous,
and Sigsworth 1998; Rosenheck 2001; Schoenwald and Hoagwood 2001), no studies have
considered the impact of implementing EBP on organizational performance. That is, scholars
have largely assumed that implementing EBP will lead to improved organizational performance.
Yet, for some organizations, EBP may be a “poison fruit” for organizational performance.
Developing a theoretical understanding of how this can happen is critical to helping
organizations plan and manage the innovation associated with implementing EBP. In this paper,
we present a dynamic theory of adopting and implementing EBP, and evaluate conditions under
which implementation of EBP leads to higher and lower levels of organizational performance as
well as conditions resulting in no change.
2.1. Innovation Implementation
Organizational scholars have long known about the difficulty organizations face with
implementing new ideas (Zaltman, Duncan, and Holbek 1973). The difficulty of getting
empirically supported treatments into practice has drawn attention to four different social
processes: diffusion, dissemination, adoption, and implementation. Diffusion refers to the
sharing of information through ad-hoc mechanisms (e.g., word of mouth) and is contrasted with
dissemination, which is a deliberate strategy to transmit information from one group to another
(Sliverman, Kurtines, and Hoagwood 2004). Adoption refers to the decision to use an
innovation, while implementation refers to the process of its actual use (Klein and Knight 2005;
Rogers 1995). Less attention has been paid to re-examination of adoption decisions and the
process of discontinuing a practice, which has led to a pro-innovation bias in diffusion of
innovation research (Rogers 1995). In this study, our focus is on implementation and its impact
on organizational performance.
Innovation Implementation 4
2.2 Organizational Efficiency, Effectiveness, and Performance
We are concerned with the situation in which an organization adopts and implements
evidence-based practices because it wants to improve some aspect of performance; for example,
to increase legitimacy with stakeholders, improve client outcomes, or provide services at a lower
cost. To capture this, we draw on Pfeffer and Salancik’s (1978) distinction between
organizational efficiency and organizational effectiveness.
Organizational efficiency refers to how well the organization pursues its activities.
Efficiency is intemal to the organization and determined by how much the organization
produces. For mental health agencies, efficiency includes finances, services, and utilization
(Ozcan, Shukla, and Tyler 1997). It can also include client outcomes, which may vary from
client satisfaction to changes in severity of symptoms or behavior. Many questions about quality
of services are, in fact, questions about efficiency in delivering services. For example, the
emphasis in Total Quality Management (TQM) is on reducing the number of defects per unit of
output, not on changing what is produced (Deming 1986).
Organizational effectiveness refers to whether or not the activities are seen as appropriate
by stakeholders. The basis of the criteria for evaluating effectiveness is external to the
organization and depends on the environment. Institutional environments can be characterized
along two dimensions: technical and institutional (Scott and Meyer 1991). In technical
environments, organizations are rewarded for their outputs. In institutional environments,
organizations are rewarded for their conformity to rules, regulations, or organizational form.
Organizations can face demands from technical environments, institutional environments, or
both. Public utilities, banks, and hospitals face strong demands from both technical and
institutional environments, whereas manufacturing companies experience strong demands from
technical environments, but weaker demands from institutional environments (Scott and Meyer
1991).
Mental health organizations are often characterized as facing strong demands from
institutional environments, but weak demands from technical environments (Scott and Meyer
1991; Powell 1991; Ozcan, Shukla, and Tyler 1997). That is, mental health organizations have
historically been judged more by the organization’s credibility and therapists’ conformity to
expectations about how mental health services should be organized than by measurable clinical
outcomes. Interventions are tolerated and even promoted in spite of weak, lacking, or even
harmful scientific evidence.
Within this framework, organizations can have any combination of efficiency and
effectiveness (Ostroff and Schmitt 1993; Ozcan, Shukla, and Tyler 1997). There are
organizations efficient at producing unwanted goods or services, just as there are organizations
inefficient at producing highly valued goods or services. The best organizations do both; that is,
they are known for both producing services that are effective and for doing so with great
efficiency. Likewise, there are organizations that do neither, and yet they continue to survive as
permanently failing organizations (Meyer and Zucker 1989). Following Sastry (1997), we
consider organizational performance as the product of organizational efficiency and
effectiveness.
Figure 1 illustrates this framework by carving the phase space of organizational
efficiency and effectiveness into four quadrants: organizational excellence, organizational
inefficiency, organizational ineffectiveness, and organizational failure. Organizational excellence
involves the efficient production of highly valued goods or services. Organizational inefficiency
involves organizations inefficiently producing highly valued goods or services. Organizational
Innovation Implementation 5
ineffectiveness is the efficient production of unnecessary or inappropriate goods or services; for
example, a court mandated counseling program that delivers low cost services and makes a
profit, but only provides its clients with an increased awareness of their problem, as opposed to
treatment, which is what the courts, probation officers, and community would expect.
Organizations are failing to the extent that they are inefficient at producing unnecessary or
inappropriate goods and services.
Figure 1 Organizational type by effectiveness and efficiency
High
Organizational Organizational
inefficiency excellence
Effectiveness
Organizational Organizational
failure ineffectiveness
Low
Low Efficiency High
2.3 Adopting and Implementing Evidence-based Practice
Evidence-based practice changes the basis for evaluating performance by changing the
environmental demands placed on organizations, from primarily institutional to both institutional
and technical. Specifically, instead of judging interventions based on practitioners’ beliefs or
intuition or scientifically unsupportable theories of human behavior, interventions are only
considered acceptable if they meet the “gold standard” of demonstrating clinical benefits to
clients that are equal to or exceed the benefits of other interventions. What is radical in this shift
is not the use of science to inform practice decisions; rather, it is what happens to the
organization that must now face technical demands in addition to institutional demands about
how to conduct business. For the mental health organization adopting EBP for treatment
decisions, it means that external and changing scientific standards now determine what kinds of
services should be provided and how. Organizational performance can decline if agencies are
unable to adapt to these changing demands.
2.4 Organizational Inertia
Organizational inertia represents existing monetary and psychological investments by the
organization—a sunk cost in the status quo (Hannan and Freeman 1984). These investments
include existing policies and procedures, technology, personal relationships and loyalties,
political structures within the organization, organizational culture, and ties to other organizations
and networks. The ability of an organization to produce outputs reliably depends on
institutionalization and enactment of standardized routines; that is, inertia (Hannan and Freeman
Innovation Implementation 6
1984). This has important implications for mental health organizations and their organizational
ecology.
A central question from evolutionary-ecological organizational theories is whether or not
organizations can learn and adapt to their environments as fast as the environment is changing
(Hannan and Freeman 1984). The answer depends on understanding the relationship between the
nature of the change in the environment and its impact on organizations. Organizational inertia is
a relative concept that emphasizes how quickly an organization can change to address emerging
needs and secure new resources (Larsen and Lomi 1999). Organizations with high inertia are
slower to adapt to changes in their environment than organizations with low inertia.
Organizations build inertia in stable environments and lose inertia when routines are not
continually practiced (Hannan and Freeman 1984). Changes that affect the structural core of the
organization— mission, authority structure, technology, and marketing—are more likely to
decrease structural inertia, which can lead to declining organizational performance and thus
increase the likelihood of organizational failure, whereas changes that are peripheral to the
organizational core are less of a threat and might even enhance the organization (Carroll and
Hannan 2000; Hannan and Freeman 1984). EBP can potentially affect all four levels of the
structural core and hence impact inertia and performance.
Thus, it is plausible that large, established mental health organizations will find it more
difficult to adapt to the shifting environmental demands associated with adopting evidence-based
practice than newer, smaller, and less stable mental health organizations. While organizations
with less inertia will be less reliable in their service outputs initially, their ability to adapt to the
changing environment gives them an advantage over more established agencies. This means that
the adoption and implementation of evidence-based practice can lower the performance of stable
and more established organizations, relative to the less stable and smaller organizations.
If funding agencies and state policies reward organizations implementing evidence-based
practice, then these newer organizations could displace the established and stable organization.
This, in turn, could destabilize the organizational ecology of mental health agencies in a
community, potentially to the point of undermining the overall quality of services. However, the
transition could also dampen the quality of services only temporarily and then raise the quality of
services to a higher level. It is, therefore, vital for us to have a better understanding of how the
adoption and implementation of evidence-based practice impacts organizational performance
within mental health services.
3. Method
In this section, we describe the methods used to develop and test the model, along with
the procedures used to answer the two main questions in this study. We review literature,
summarize earlier work on replicating system dynamics models, and report on key informant
interviews. Our goal is to advance a kind of understanding that progresses and cuts across many
different situations, for example, from transforming a state mental health system in the United
States to building a service system in a developing country or responding to acute mental health
needs after an environmental disaster. To do this, we need to demonstrate how we take existing
conceptual and empirical work and build models to answer specific questions.
3.1 Systematic Review of Literature
The initial conceptualization of the Implementation and Organizational Performance
(IOP) Model was based on a systematic review of the literature on diffusion and implementation
Innovation Implementation 7
of EBP in mental health agencies. This included reviewing existing literature on diffusion and
implementation of EBP in mental health organizations, system dynamics models, and
organizational theory. We used content analysis to identify key constructs and causal
relationships in relevant articles, and then coded each fragment into a set of cause and effect
concept pairs (Wrightson 1976). These were combined to form an initial conceptual model of the
problem of adopting and implementing EBP in community mental health organizations.
In addition to the existing mental health services literature, we also reviewed system
dynamics models related to the diffusion and implementation of innovation, and planned
organizational change. Because we were interested in understanding the impact of
implementation on organizational performance, we excluded models that focused primarily on
diffusion of innovation. The models reviewed had to be published in joumals or books, and list
equation or provide the models on the web. When we rebuilt models from equation listings, we
replicated the simulations in published studies to ensure the accuracy of our model
reconstruction.
3.2. Formulation
We started model formulation by working with Sastry’s (1997) model of Tushman and
Romanelli’s (1986; 1985; 1985) theory of punctuated organizational change, and then added
structure to reflect the processes identified through our systematic review of the mental health
literature and structures from other models of organizational change (Levin and Roberts 1976;
Repenning 2002; Samuel and Jacobsen 1997; Sastry 1997). This approach revealed equivalent
mechanisms and differences in the meaning of similar terms. For example, the models we
considered had some reinforcing mechanism that increased commitment through experience.
Functionally, these mechanisms drove the implementation of innovations, although they tended
to represent the same phenomena using different mechanisms.
Simulation testing revealed that apparently similar concepts such as resistance to change,
commitment, and organizational inertia were functionally distinct. For example, commitment is
sometimes used to mean worker commitment to change, but at other times refers to managers’
commitment to implement change. Resistance is sometimes included as an element or indicator
of organizational inertia, and other times thought of as the result of change. Where we
discovered this type of ambiguity, we drew on organizational theories such as resource
dependence theory (Pfeffer and Salancik 1978), organizational ecology (Hannan and Freeman
1977, 1984), theories of punctuated change (Romanelli and Tushman 1994; Sastry 1997), and
new intuitionalism (Scott and Meyer 1991) to clarify and extend our model of implementation
and organizational performance.
Initial conditions for each of the stocks were calculated to start the model in equilibrium
for high inertia organizations. In some cases, this was a straightforward exercise of finding the
roots for the net rate of change. Other situations proved more complicated and required
derivation of expressions based in organizational theory. For example, an important assumption
in our model is that only organizations with high inertia are in a dynamic equilibrium since
inertia accumulates in stable environments. From these assumptions, we worked out a series of
lemmas describing the initial conditions so that the model initialized in equilibrium independent
of initial effectiveness, efficiency, and organizational performance.
3.3. Model Testing
We used a variety of tests throughout the modeling process to identify errors in
formulation and theory. In addition to dimensional consistency tests, we ran the model through a
Innovation Implementation 8
series of behavior reproduction tests comparing the behavior of the relevant organizational
theory against model behavior. For example, implementing new ideas in an organization
frequently leads to an initial decline in performance before any improvement can be seen. Since
our model incorporated theories and structures that could produce this effect, we expected our
model to replicate these behaviors as well.
3.4 Key Informant Interviews
We compared structures in the IOP Model against seven key informant interviews with
administrators of mental health services. Key informants were asked questions about their
experiences implementing evidence-based practice. Interviews were recorded, professionally
transcribed, and independently coded by two members of the research team. Administrators
identified barriers such as costs of training and supervision, high caseloads, resistance to change
among experienced workers, shortage of master’s level graduates ready to use EBP in clinical
practice, and an urgent need for “evidence-based management” to inform the implementation
process. These interviews shifted our focus from modeling tactical questions about
implementation to addressing strategic questions. When we could not find excerpts in the
interviews to corroborate the mechanism as specified in the model, we either modified the
mechanism to reflect what key informants were saying or dropped the mechanism entirely.
3.5 Representing Adoption and Implementation of EBP
Our main focus in this study was on understanding what happens to a mental health
agency that decides to adopt and then implement EBP with the expectation of improving
organizational performance. This had two components: the strategic decision to adopt and
implement EBP, and the goal of improving organizational performance. The decision to adopt
and implement EBP means that the organization changes the basis of its legitimacy from one
based on ideology to one based on evidence. The implication is that organizational effectiveness
drops, and creates what will appear as an initial shortfall in the strategic direction. We
represented this change as a 30% increase in the required strategic direction using step input at
12 months.
Equally important is the fact that the organization initiates this change in a strategic
direction with the intention of improving organizational performance. That is, the issue here was
not that the environmental demands changed and the organization sought to stay at the current
level of organizational performance. Rather, the organization entered a change process with the
goal of improvement, which we represented as a 30% increase in the desired level of
organizational performance using a step input at 12 months.
3.6 Sensitivity Analysis
We answered the first question by conducting a simulation study of the implementation
process for organizations with different initial conditions. Initial efficiency and _ initial
effectiveness both varied from 0.1 to 0.9 in increments of 0.05 to cover the phase space of Figure
1. To capture the effect of organizational inertia on performance, we needed to vary initial
organizational inertia for each combination of initial effectiveness and initial efficiency. The
value of initial inertia for an organization to be in equilibrium in a stable environment is unique
and varying inertia places organizations into disequilibrium.” Our approach was to vary the ratio
? Note that in a changing environment as opposed to a stable environment, an organization can be in a dynamic
equilibrium with multiple values of initial inertia.
Innovation Implementation 9
of initial inertia to equilibrium inertia from 0.2 to 0.9 in increments of 0.1. This resulted in a
total of 2,312 simulations.
There are a number of ways to quantify the impact of implementation on organizational
performance. Perhaps the easiest measure is to compare post-implementation performance
against pre-implementation performance. This is generally what managers do most of the time.
We represent this by comparing performance at 120 months against the initial organizational
performance at 0 months, calculated as the difference Dj:
_ P.(120)—P (0)
BO)
where P(t) is the performance at time t for the organization implementing evidence-based
practice. In order to compare the size of change across different types of organizations, the
difference is normalized by dividing by the initial performance.
A major disadvantage with this approach is that it attributes all changes in organizational
performance to the implementation process. In fact, the organizational performance might
already have been improving or declining without the intervention. What we want to know is
not whether or not the organization changed in absolute terms, but whether or not the change in
organizational performance was due to the intervention. That is, we want to compare the
dynamic behavior of the factual (implementation case) with the counterfactual (no
implementation case) to assess what the impact of implementation is on the organizational
dynamics. The factual case is implemented as described in Section 3.5 above. The
counterfactual case does not introduce the step inputs, and the organizational change that is
observed is a result of organizational inertia building in a stable environment. We calculate D (t)
as:
, (1.1)
p,() =A (1.2)
where P,(t) is the performance of the organization at time t with implementation (factual), and
P(t) is the performance of the same organization but without implementation (counterfactual).
The difference is then normalized by dividing by the initial performance at the start of the
simulation for the case without implementation. This metric, D.(t), is continuous over time.
While this is useful for the more detailed behavioral analysis discussed in the next section, it is
difficult to use as a summary of what happened for each organization. Thus, we might consider
three summary metrics of D2(t) based on taking the average, maximum, and minimum of the
difference over the simulation period:
Dean = D2 (t), Dine =max(D,(t)), and D,,,, =min(D,(t)) . (1.3)
mean max
We also want to know whether or not the organization eventually improved relative to
what might otherwise have happened. This tells us whether or not the long-term expectations are
met for improvements in organizational performance, which is a central concern to managers.
For this, we can calculate D3 as the difference between the two scenarios (factual and
counterfactual) at 120 months, normalized by the initial performance of the non-implementation
case:
Innovation Implementation 10
D; =
P,(120) - P,, (120) fia)
aaa ne ;
(0)
To answer the first question, we first determine whether or not organizational
performance increased, remained the same, or declined using the normalized change metric D;
for each simulation. We use D3 to determine if that change can be attributed to implementation
of EBP. To identify where changes in the strategic direction impacts performance, we identify
regions in the phase plots of Figure 1 for both for D; and D3.
3.7. Behavioral Analysis
To answer the second question, we seek to explain changes in organizational
performance in terms of the model's feedback mechanisms. In principle, one can do this for
each of the 2,312 simulated cases. However, our main interest is in understanding what
differentiates the cases where performance increases from implementation from those that
decline in performance. So instead we purposefully select cases to develop a comparative
understanding of the successful trajectories. For example, we compare cases 1, 2, and 3 to see the
effect of increasing initial efficiency on improving organizational performance; compare case 3
with case 4 to see why two organizations with the same initial organizational performance have
different outcomes; and compare case 3, 6, and 10 to see if our explanation for an improvement
in performance varies with inertia.
For each case, we simulate and compare the trajectory of the factual case (adopted and
implemented EBP) against its counterfactual case (did not adopt EBP). This generates 2
simulations for each hypothetical organization resulting in 28 simulations for the behavioral
analysis. For each pair of trajectories, the factual and counterfactual are compared to identify
time periods with similar and different behavior. A command file for replicating the simulations
in Vensim and data files for each simulation is available from the first author.
4. Model
This section describes the Innovation and Organizational Performance (IOP) Model. The
IOP Model represents a dynamic theory of how implementation of evidence-based practices
impacts organizational performance. The focus of the IOP Model is on understanding the
consequences of implementation at the organizational level. We therefore exclude diffusion
mechanisms both within the organization and at the sector level. For example, we do not attempt
to model how successful or unsuccessful experiences with an innovation by clinicians affect the
likelihood that they will adopt the innovation. Instead, we simply represent implementation of an
innovation among workers as a function of commitment, which is largely driven by managers.
Likewise, we also do not attempt to model how positive or negative experiences with an
innovation diffusion within a service sector between practitioners or organizations. Table 1
shows the boundary chart for the current IOP Model indicating the variables and mechanisms
that are treated as endogenous, exogenous, and excluded from the model altogether.
A causal loop diagram of the main mechanisms in the Implementation and Organizational
Performance Model is shown in Figure 2. Table 2 describes each of the main mechanisms and
provides excerpts from key informant interviews showing the relevance for some or all of the
mechanism. Mechanisms that did not have support from the key informant interview transcript
Innovation Implementation 11
were either modified to reflect what key informants were saying, or dropped from the model. In
the next sections, we briefly describe each of the major feedback mechanisms in the IOP Model.
Table 1 Boundary chart of IOP Model
Endogenous Exogenous Excluded
Effectiveness Required strategic orientation to Diffusion of innovation among
i services workers
Efficiency
Perf Managers’ commitment to strategic Diffusion of innovation among
Scrormance: direction organizations within a sector
Resources needed Desired performance Cost of incentives
Demand for services Treatment efficacy” Pacing of implementation
Reliability or intervention fidelity
Staff commitment to strategic
direction
Interaction with service networks
Staff and managers’ commitment to
structural change
Organizational inertia Need for services or size of market
* Calculated as a function of initial efficiency and reliability for initial conditions to set the initial conditions of the
organization, but not modified during the simulation.
4.1 Reorientation
The process of reorientation entails a balancing feedback mechanism (B1 in Figure 2)
where an agency changes it strategic orientation to services to meet environmental demands.
Examples of this happen when foundations begin to expect program evaluation outcomes from
an organization or the legitimacy of services shifts to technical outcomes related to evidence
based practice. What constitutes a large, moderate, or small change is relative.
The model presently represents Required Strategic Direction and Strategic Direction in a
manner similar to Sastry (1997) with the environment impacting the organization through the
absolute difference between the Required Strategic Direction and Strategic Direction. This
shortfall lowers the Effectiveness of the organization, which decreases Performance and thus
Perceived Performance following a delay. This increases the Pressure to Change and causes a
Change in Strategic Direction that will reduce the shortfall. It is worth noting that in this
representation of strategic orientation, agencies do not make mistakes in adjusting their strategic
orientation. That is, they only experience delayed information and always move closer to the
environmental demands.
4.2 Funding
Levin and Roberts’ (1976) theory of human service delivery systems includes a balancing
mechanism where shortfalls in performance lead to increases in community resources. This is
represented by the balancing mechanism where the agency changes direction to meet the
demands of its environment to secure additional funding (B2 in Figure 2). Specifically, an
increase in the Strategic Direction Shortfall lowers Effectiveness, which causes a decline in
Community Support and reduction in the Funds Allocated to Agency. This leads to less
Resources for Services and lowers the Ratio of Available to Needed Resources, which contributes
to less time for providing services. Thus Reliability and Efficiency decline, which lowers
organizational Performance and Perceived Performance with the result of increasing the
Performance Shortfall and increasing the Pressure to Change, which causes Change in Strategic
Direction and leads to an adjustment in Strategic Direction provided that the agency has the
Innovation Implementation 12
ability to change. This describes how organizations change their strategic direction in response to
changes in funding.
4.3. Caseload Pressures
Caseload refers to the number of clients the agency is currently serving. It is generally
assumed to be stable and managed through a number of feedback mechanisms as suggested by
Levin and Roberts (1976). When caseloads increase, caseload pressures limit the quality and
thereby the growth in demand for services. Conversely, agencies that experience declines in
caseloads will initially have more time to provide higher quality services and this can lead to an
increase in demand. This is balancing mechanism represented by B3 in Figure 2. Specifically, an
increase in Caseload increases the Resources Needed. This lowers the Ratio of Available to
Needed Resources, and decreases Reliability, which lowers Efficiency and leads to fewer
Referrals and lowers Caseload relative to what it would have been if Referrals remained
constant.
44 Commitment
Both Samuel and Jacobsen (1997) and Repenning (2002) describe the process of
managers setting goals for implementation. For example, Samuel and Jacobsen discuss how the
pacing of change affects managers’ use of incentives, and Repenning discusses managers’
commitment to an innovation as the primary determinant of successful innovation
implementation. In contrast, Sastry (1997) treats managers’ commitment as endogenous to the
organization and determined by an organization changing its strategic direction. In the IOP
Model, commitment refers to staff members willing to put an innovation into practice. This is
represented within a feedback mechanism whereby managers apply normative pressures to
increase staff commitment to use evidence-based practice as shown by B4 in Figure 2.
Specifically, an increase in the Commitment Gap leads to an Increase in Commitment that
increases Commitment, and thereby reduces the Commitment Gap. It is important to note that in
this process, managers’ ability to motivate staff is essentially perfect, and that commitment only
decreases when the strategic direction changes.
4.5 Implementation
Implementation as a process refers to situation where increasing commitment leads to
greater implementation, which improves organizational performance, and feeds back to reinforce
commitment. This is represented as R1 in Figure 2. Specifically, as Implementation increases, so
does Efficiency, which leads to improved Performance and Perceived Performance. This
reduces the Performance Shortfall, which decreases the Pressure to Change, and hence slows the
Change in Strategic Direction. The result is that the Decrease in Commitment stemming from
Change in Strategic Direction slows, which allows Commitment to build even more. Thus
organizations that are improving performance through greater commitment and implementation
will become more stable around the current strategic direction and “lock in” on a specific
innovation. This has benefits until the environment changes.
13
ation Implementation
Innovi
“doot
yoeqpesy Surpuepeg e sajousp ,q, doo] speqpeey Hurarojurare sxjousp AY, ‘sjpeye pebie] s}ousp syuT] Tenseo ssawe sour] eqnod
‘souSN GUT Jo uo_sarIp Bureorput subs yy ‘sdrysuoyeyal [esneo ayeoIpul SMALTY ‘SeTeUPA gyE]s JO 3po}s 10 feu yuesadar soxog
“RPO (dOT) soueuntopeg jeuoTezuehio pue uoTeEwe Puy jo weep dooy jesnea z ambi
Innovation Implementation
14
Table 2 Main feedback mechanisms in the IOP Model
Mechanism Label! Description Support ”
Reorientation. Bl Agency changing its strategic “As you know, the evidence changes.
orientation to meet the demands _That’s my challenge is to stay on top of that
of its environment. change and continue to involve the staff in
looking at the change.”
Funding B2 Providing external support to “We worked with the county in getting a tax
agency to meet the demands of _ passed, which will create a children’s
its environment. services fund.”
Caseload pressures B3 Caseload pressures limiting the “There needs to be some limits on
quality and growth in demand caseload.” “We don’t have enough other
for services. staff to really do a lot of sort of talk therapy
with patients.”
Commitment B4 Supervisors applying normative “They get immediate feedback about what
pressures to increase staff they’ re doing, whether it’s effective or not
commitment to use evidence- effective.”
based practice.
Implementation R1 Increasing implementation by “So the medical center director, his bonus,
gaining commitment through in large part, is determined by how the
improved organizational medical center does on its performance
performance. measures.”
Quality R2 Improving the reliability of “And so, if we have what we perceive to be
improvement services through organizational an inordinate amount of runaways in our
leaming and quality residential program, we will do a QI study
improvement. for 18 months.”
Ability to change R3 Changing strategic direction “Some people appreciated the power of
decreases organizational inertia, _ being trained a certain way, and once you
which increases the agency’s leave school that it’s very difficult to take
ability to change. Likewise, on new ideas and embrace them and move
increasing stability increases forward.”
inertia, which makes it harder to
change.
Demand R4 Increasing the effectiveness of “And I think there is an appeal from the
services leads to more demand public for that kind of thing, and so we'll
for services, reinforcing pressure increase the numbers of people, we'll
to change and further increase increase our profile in the community, we'll
effectiveness. increase our revenues through doing that
because... they want to make sure they’ re
not just throwing their money away.”
Institutionalization R5 Establishing a way of doing “People have to have some experience with
things in the agency, that is,
organizational culture.
it, they have to see it working, and they
have to hear their peers talk to them about
how it’s working.”
Notes: '’B’ prefixes denote balancing or negative feedback mechanisms, while ‘R’ prefixes denote reinforcing or
positive feedback mechanisms. ? The quotes are excerpts from key informant interview transcripts with
administrators of mental services.
4.6
Quality Improvement
Quality improvement refers to the process where reliability of services increases through
organizational learning as shown in R2 in Figure 2. Sastry (1997) represents this process as a
function of growth of inertia where organizations develop routines based on previous experience
and performance that lead to additional improvements. It is important to note that in a quality
improvement process, it is usually not just that the quality of the outputs have improved, but that
the effects of these improvements help the organization and reinforce the initial investments in
Innovation Implementation 15
the change. This is represented as the reinforcing mechanism R2 in Figure 2 where an increase in
Reliability improves Efficiency which increases Performance and Perceived Performance. This
reduces the Performance Shortfall and Pressure to Change, which slows the Change in Strategic
Direction and decreases inertia. This allows Inertia to grow faster than it would have otherwise,
and reinforces the initial increase in Reliability.
4.7 Ability to Change
An organization's ability to change is limited by its organizational inertia (Carroll and
Hannan 2000; Hannan and Freeman 1984), which decreases when organizations undergo a
change process that affects organizational structure (Sastry 1997; Tushman, Newman, and
Romanelli 1986; Tushman and Romanelli 1985; Tushman, Virany, and Romanelli 1985). This
forms the reinforcing mechanism R3 in Figure 2. Specifically, high Inertia lowers the
organization’ s Ability to Change, which limits the Change in Strategic Direction. This slows the
decrease of inertia and allows Inertia to grow.
48 Demand
In Levin and Roberts’ (1976) theory, demand for services increases as the community
becomes aware of new services or improvements in quality. All other things being equal,
increasing demand leads to more clients receiving service and to further increases in demand.
This process is represented in the current IOP Model by the reinforcing mechanism R4 in Figure
2. Specifically, an increase in Referrals leads to a higher Caseload, which increases the
Resources Needed and reduces Ratio of Available to Needed Resources. This lowers Reliability
and Efficiency, leading to a decrease in Performance and Perceived Performance, and thus
increases the Performance Shortfall. This creates a Pressure to Change and change in Strategic
Direction to reduce the shortfall in performance, which contributes to an increase in
Effectiveness, more Community Support, and an additional increase in Referrals.
It is important to note that this representation in the IOP Model is problematic. While R4
does capture the effect that demands for services depends on effectiveness and community
support, it is primarily a resource allocation mechanism. Specifically, it reflects a mechanism
where organizational growth in the form of increased caseloads and funding is fueled through
improving the fit between the organization and environmental expectations. This is a different
mechanism than the word of mouth or marketing effects more commonly discussed.
4.9 Institutionalization
Institutionalization refers to the process where organizational inertia accumulates through
the natural development and transmission of rules, procedures, and routines within an
organization, often discussed as organizational culture. Institutionalization is sometimes
described as organizational learning (Sastry 1997). This is a simple reinforcing mechanism (R5
in Figure 2) where more Inertia leads to a further increase in Inertia.
5. Results
In this section, we present the results from the simulation analysis of the IOP Model. We
begin with the sensitivity analysis used to answer the first question about how the initial
conditions of the organizations and implementation impact organizational performance. From
this, we identify specific regions of change that we consider in behavioral analysis.
Innovation Implementation 16
5.1 Sensitivity Analysis
Under what initial conditions does organizational performance improve as a consequence
of implementing evidence based practice innovations? Figure 3 shows plots for D; and D3 by
initial efficiency and effectiveness with one pair of plots for each condition of initial inertia.
Inertia increases going from left to right in the panels of Figure 3. The top row of panels has D,,
as the dependent variable while the bottom row of panels shows D3. White regions are neutral,
dark gray represent declining performance, while light gray regions indicate improving
performance. The numbers identify cases used for the behavioral analysis to understand the
structure-behavior relationship.
Most organizations will experience improving performance when initial inertia is low
(top-left panels in Figure 3). As initial inertia increases (top-middle panels in Figure 3),
improvements in organizational performance begin to vary, and depend on initial conditions of
efficiency and effectiveness of the organization. For example, failing organizations will see a
decline in performance, whereas organizations with high efficiency or high effectiveness will see
improvements. As initial inertia increases further, only organizations with high efficiency will
see improvements in performance (top-right panels in Figure 3).
However, the results look different when we compare change in performance between the
factual (implementation) and counterfactual (no implementation) using D3. The bottom row of
panels in Figure 3 shows that only a small portion of organizations will improve in
organizational performance relative to what would have happened if they had not adopted and
implemented EBP. First, for organizations with low to moderate inertia at the start of the
simulation, only those with high efficiency but low effectiveness see improvements in
organizational performance relative to the counterfactual case (e.g., regions 3 and 6 in Figure 3).
Moreover, the region where D3; is positive shrinks and then expands with increasing initial
inertia.
In no cases do organizations with high initial effectiveness see increasing organizational
performance from implementation when compared with the counterfactual case of not
implementing EBP. In particular, agencies demonstrating organizational excellence are likely to
decline in performance, whereas organizations seen as efficient but ineffective are likely to
improve in organizational performance from implementing EBP when they have 1) low to
moderate initial inertia, or 2) high initial inertia. The next question is, why?
Figure 3 D, and D3 by initial efficiency, effectiveness, and inertia.
020406 08 020406 08 02040608 020406 08
po
inertia Linertia Thertia inbria Ineria inertial inertia T inertia 4
14 5 8 9 13 44/08
a pos 2
4 O04
go4 128 76 a2 1119.
s 3 03 03 03 03 D3 03 D3. 8
8 inertia Linertia hentia inbrtia Ineria inertial inertia T inertia
i874 6 8 9 13° 44
06-4 r
O44 r
02-2 3 76 a2 1119 4
eS aS SS a a oo
02040608 02040608 02040608 02040608
Efficiency
Innovation Implementation 17
5.2 Behavioral Analysis
In our behavioral analysis, we first seek to understand why cases improve in
organizational performance from implementation relative to the no implementation
counterfactual. Next, we seek to understand why this effect seems to vary by initial
organizational inertia. Lastly, we seek to understand the interaction between high initial
efficiency and high initial effectiveness that makes organizational performance decline for cases
starting out in the region of organizational excellence.
In the first set of comparisons, we consider cases 3, 6, and 10 (see Figure 3). All six
simulations show improvement in organizational performance relative to their initial
performance (see Figure 4). Case 3 and 10 show marginally higher final performance for the
implementation case relative to the no implementation case, while case 6 is more or less neutral
(which has to do with the fact that case 6 is close to neutral on D3 in Figure 4.
In all three cases, performance drops sharply at the time of the initial adoption decision
and change in the organizational environment. This drop in Performance is caused by a decline
in Effectiveness, and indirectly by a decline in Community Support. The decline in Community
Support affects Referrals, Funds Allocated to the Agency, and ultimately the availability of
resources relative those needed to maintain quality services. The result is an immediate decline
in the Reliability of services at 12 months® that lowers Efficiency and compounds the effects of
the initial decline in Effectiveness.
The magnitude of this initial decline is proportional to the Efficacy of the service
technology, which is assumed to be constant throughout the simulation.” Efficiency is
formulated as the product of Implementation, Reliability, and Efficacy:
Efficiency(t) = Implementation(t) x Reliability(t) x Efficacy (t) (1.5)
A drop in Reliability is therefore multiplied by the value of Efficacy. For organizations with low
inertia and high efficiency, this means that high Efficiency is the result of using highly
efficacious interventions as opposed to achieving high reliability through resources or high
inertia. This explains why the drop is larger for organizations with high initial efficiency, and
why this effect declines with inertia, but it does yet explain why this translates into the
organizational performance being higher than the non-implementation case. Intuitively, one
might expect a shallower drop to be more in line with higher performance as opposed to the
other way around.
3 That this happens instantly is unrealistic. For example, agencies will have operating budgets that can sustain
temporary loses in revenue and staff are likely to continue providing quality services with temporary shortfalls in
time or increases in caseloads. Although not sustainable in the long-run, most agencies do have ways of making it
through temporary short-term transitions, and we would expect such mechanisms to come into play and buffer the
immediate shock in loss of community support. There would also be ways that organizations could slow the decline
in community support by, for example, participating in sector-wide change efforts or educating supporters about the
expected transition. This would introduce a delay between the initial change and the loss of community support.
“This is also unrealistic since what we are saying is that the organization switches from one practice to another
practice, and we would expect this to mean that the organization adopted and sought to implement interventions
with higher treatment efficacy.
Innovation Implementation
18
Figure 4 Implementation versus no-implementation performance trajectories for three cases
where performance increased by initial levels of organizational inertia.
(a) Initial inertia = 0.2
Dm
0
0 12 24 36 48 60 72 84 96 108 120
Time (month)
Performance : Case 3 counterfactual —t—3——3—33—_4 434
Performance : Case 3 factual
(b) Initial inertia = 0.6
0.6
Dm
0 12 24 36 48 60 72 84 96 108 120
Time (month)
Performance : Case 6 counterfactual —--—-+—4—_-—_ 4 4-4-3
Performance : Case 6 factual
(b) Initial inertia = 0.9
0.6
Dm
0 12 24 36 48 60 72 84 96 108 120
Time (month)
Performance : Case 10 counterfactual +——t——t——_ 4-4. 4-4-4
Performance : Case 10 factual
Innovation Implementation 19
The key to why the larger initial drop contributes to a long-term performance gain is the
role that Pressure to Change plays in driving improvements in organizational performance.
Pressure to Change can influence organizational performance through changes in commitment
via the implementation loop (R1); organizational inertia through the quality improvement loop
(R2); and through strategic direction through the reorientation loop (B1). For most cases, long-
term improvements in organizational performance will come from the quality improvement
process as Inertia accumulates in a stable environment. However, the accumulation of Inertia is
not goal directed around a specific outcome in performance or particular strategic direction.
Moreover, Inertia has an upper-bound associated with employee turnover, the forgetting of
routines, etc. So there is an inherent limit in how far an organization can improve performance
via a quality improvement process, and it will not offset the problems associated with an
organization moving in the wrong strategic direction. Thus, a critical question for an
organization is whether or not it can complete its reorientation before the quality improvement
loop (R2) takes hold.
When improvements from earlier changes in Strategic Direction combined with increases
in Inertia and Reliability are sufficient to close the Performance Shortfall, then the incentives for
the organization to continue the reorientation process disappear. This locks-in a Strategic
Direction Shortfall and limits the potential long-term Effectiveness of the organization.
Organizational performance continues to improve through the accumulation of inertia, but it will
ultimately be less than the no-implementation case. However, the larger drop in Performance
pushes the Pressure to Change past a critical point where growth in Inertia slows, causing a
decline in Inertia (see Figure 5a). This leads to an increase in the organization’s Ability to
Change and reinforces change in strategic direction (see Figure 5b), allowing the organization to
close the Strategic Direction Shortfall before month 36 and the quality improvement process
takes hold (see Figure 5c).
To understand why this effect varies by initial inertia, we need to explain why it 1)
declines with initial inertia, and then 2) reappears for organizations with high inertia. The first is
easy in that we have already shown how for an organization to have high efficiency with low
inertia, it must deploy an intervention that is highly efficacious. This translates into a larger
initial drop, increasing the pressure to change, which allows for the gap in effectiveness to be
closed. Since reliability increases with inertia, the treatment efficacy required for an organization
to have high initial efficiency decreases. This lessens the initial drop and lowers the pressure to
change until it no longer pushes the organization past the critical threshold of being able to
complete the reorientation process before quality improvement sets in.
To understand the second case, it is important to note that the for higher inertia
organizations, the Ability to Change is lower, making them less responsive, and allowing
Pressure to Change to build. As initial inertia approaches the right side of Figure 4, Pressure to
Change increases until Inertia and Ability to Change are low enough for significant Changes in
Strategic Direction. This creates sufficient momentum in the system to allow the reorientation
process to complete before the quality improvement process gets established.
Innovation Implementation
Figure 5 Dynamics of Inertia, Ability to Change, and Strategic Direction Shortfall from
simulation of implementation for cases 6 and 7
(a) Inertia
i
04
0 12 24 36 48 60 72 84 96 108 120
Time (month)
Tnertia : Case 6 factual ——+—+— Inertia : Case 7 factual —2—2—
(b) Ability to Change
1
a
0
0 2 4 36 48 6 72 84 96 108 120
‘Time (month)
Ability to Change : Case 6 factual
Ability to Change : Case 7 factual
(c) Strategic Direction Shortfall
SDU
0.6
0 12 24 36 48 60 72 84 96 108 120
‘Time (month)
Strategic Direction Shortfall : Case 6 factual ——2——2—3—3_ 334
Strategic Direction Shortfall : Case 7 factual —--—-2—-2—2—_2—2—_ 2
Innovation Implementation 21
Finally, to understand why organizational excellence does not translate into improving
organizational performance, consider that an organization with performance in the region of
organizational excellence in Figure 1 has both high efficiency and high effectiveness, which
means higher levels of community support. This translates into more client referrals and
resources. One implication of this is that for two organizations with the same level of initial
Inertia, Reliability will be higher for the organization with higher Efficiency. If the initial
efficiency for both organizations is the same, higher Reliability means that the organization with
higher effectiveness can achieve the same level of efficiency with less efficacious treatments
according to equation (1.5). Consequently, Efficacy is lower for organizations with higher initial
effectiveness; and, lower treatment efficacy means a lower drop in organizational performance,
which lessens the Pressure to Change and decreases the likelihood that the reorientation process
will finish before the quality improvement process takes over. This effect is independent of
initial efficiency.
6. Discussion
This research has implications for strategic planning and policies to promote the use of
EBP. Based on our conceptual model, ineffective but highly efficient organizations have the best
chance to see improvements in organizational performance from implementation in otherwise
stable environments (shaded region in Figure 6). This means that organizations with strong
reputations that are considered excellent in a community should consider the possibility that
implementation may not be an innovation for improving organizational performance, and may
even lead to a decline in performance. While improving organizational performance is not the
main point of EBP, we do believe it is important to consider as declining performance can lead to
a loss in managerial support for implementation that could in tum lead to low intervention
fidelity or abandonment of the implementation process.
Figure 6 Zone of expected organizational performance improvement from implementation
High
Organizational Organizational
inefficiency excellence
Effectiveness
Organizational Organizational
failure ineffectiveness
Low
Low Efficiency High
The results also point to different strategies for organizational development and
implementation of EBP. For inefficient and failing organizations seeking to implement EBP and
improve organizational performance, the best initial investments could be on increasing
Innovation Implementation 22
organizational efficiency through a quality improvement process. That is, it may be better to
delay the strategic reorientation until the organization understands and has sufficient control over
its service outcomes. For agencies in the region of organizational excellence, EBP represents a
disruptive innovation (Christensen 2003). These organizations may be better off developing
standalone programs in the shaded region of Figure 6.
For the behavioral analyses, we found that Pressure to Change played a key role in
understanding whether or not the organization was able to complete the reorientation process
(feedback mechanism B1) before the quality improvement process (feedback mechanism R2)
took effect. The higher the Pressure to Change, the faster the reorientation process, and the
greater the likelihood the organization would be able to close the gap between the organization’s
actual and required strategic direction. This race between completing the reorientation process
before the quality improvement process sets in was the main explanation for why some
organizations were able to see improvements in organizational performance while other
organizations experienced declines.
For the most part, Pressure to Change was influenced in this model by treatment
efficacy, which was calculated as a function of the initial conditions of the organization. This
brought attention to the different ways that organizations can bring about the same level of
efficiency and clinical outcomes. Some organizations will achieve high levels of efficiency by
reducing the variability of their services, other organizations will achieve this by providing a
higher level of resources to their clinical staff relative to need, and a third group of organizations
achieve this by deploying clinical interventions that are highly efficacious. On the surface, they
would all appear to achieve similar results, but the underlying differences have implications for
how the organizations are impacted by or benefit from implementing EBP in terms of
organizational performance.
These results suggest a simpler theory and model for understanding the dynamics of
implementation on organizational performance (shown in Figure 7). Community support (B2)
causes an initial decline and recovery in organizational performance, followed by a period where
the reorientation process (B1) dominates, and then the quality improvement process (R1) takes
over as inertia begins to build. Whether or not the organization experiences an overall benefit
from the transformation process depends on how fast the reorientation process is relative to the
quality improvements process. If the organization can move fast enough to align with the
requirements in its environment, then the organization will benefit. Otherwise, the quality
improvement process will lock into limits to improvements in organizational performance.
This model (Figure 7) moves us more in the direction of strategy as opposed to tactics of
implementing evidence based practices. At the level of individual organization, issues are raised
about the relative timing and speed of change relative to organizational inertia, community
support, and environmental requirements. It suggests that a key decision for managers is
deciding when to focus on strategic direction versus quality improvement. At the level of
organizational ecology, the model raises questions about how a population of organizations with
differences in treatment technology, community support, and organizational inertia will fare
through broad changes in public mental health policies.
There are, however, a number of limitations that need to be addressed in future research.
First, the work so far has only considered individual organizations and then assumed only stable
environments. That is, we have not sought to include the effects of organizations on service
networks. More work is clearly needed to understand how broader changes such as state-wide
transformation efforts play out over time to impact public mental health and financing of
Innovation Implementation 23
services. To make such studies more meaningful, it will be essential to have a better empirical
basis for the organizational demographics within a regional mental health service sector.
Figure 7 Simplified causal loop diagram of implementation and organizational performance
Required Strategic
Direction
Strategic |
Direction ey
FS
"
yee we Effectiveness
Performance Pressure to
Goal Change BL €
Reorientation
Community
Support
Performance ai
B2
Ri # :
Quality Community
\ improvement Efficiency support Resources
Inertia
ine
me
ms:
It is also important to recognize that for many organizations, the environment is changing
and this is likely to increase when effectiveness is judged on technical as opposed to institutional
criteria. This is happening as funding sources begin to push for empirically supported treatments
and establish systems for monitoring the fidelity of mental health interventions. Thus, we need
to understand the organizational demography along with the interplay between the organizational
demography and changing environment in the form of new evidence based practices, changing
funding requirements, and restructuring of private-public and state-local partnerships in
delivering mental health services. We see these as important opportunities for applying system
dynamics for advancing services research, improving mental models of health delivery systems,
and increasing stakeholder participation and consensus for more consumer-driven and accessible
services.
7. Conclusion
We have made an argument for the importance of understanding the dynamics of
implementation and organizational performance in mental health services. Using theory,
preliminary empirical research, and models, we built and simulated a model of the
implementation process to understand how changes in organizational performance varied by
organizational characteristics. We then conducted a more detailed behavioral analysis to
understand these variations in terms of the causal structure. This led us to a simplified
Innovation Implementation 24
understanding relating strategic reorientation, community support, and quality improvement
processes.
The model we have presented is still largely conceptual and in the early stages of
development. Nonetheless, it does demonstrate the potential effects under idealized conditions
of implementation on organizational performance. It also represents an important advance in
services research, which has often been characterized as lacking an adequate social theory for
guiding research and public policy. Specifically, we have demonstrated the utility of applying
system dynamics for combining what we know from existing models and theory with pilot data
to develop and test a model that leads us to substantive insights and into the next iteration of
research.
We believe the IOP Model is an improvement over existing and often static mental
models of mental health policy regarding implementation of evidence based practice. Moreover,
we see attention to organizational dynamics and strategic as opposed to tactical issues as
innovations in services research. Our research demonstrates an application of system dynamics
to one cycle of research that can develop into a longer program of evaluation and policy design.
Such programs are essential to making significant gains in public mental health in years to come.
Innovation Implementation 25
References
Bartels, Stephen J., Aricca R. Dums, Thomas E. Oxman, Lon S. Schneider, Patricia A. Arean,
George S. Alexopoulos, and Dilip V. Jeste. 2002. Evidence-Based Practices in Geriatric
Mental Health Care. Psychiatric Services 53 (11):1419-1431.
Carroll, Glenn R., and Michael T. Hannan. 2000. The demography of corporations and
industries. Princeton, NJ: Princeton University Press.
Christensen, Clayton M. 2003. The innovator's dilemma: The revolutionary national bestseller
that changed the way we do business. New Y ork, NY : HarperCollins Publishers, Inc.
Deming, W. Edwards. 1986. Out of the crisis. Cambridge, MA: Massachusetts Institute of
Technology, Center for Advanced Engineering Study.
Gonzales, Junius J., Heather L. Ringeisen, and David A. Chambers. 2002. The tangled and
thorny path of science to practice: Tensions in interpreting and applying "evidence".
Clinical Psychology: Science and Practice 9 (2):204-209.
Hannan, Michael T., and John Freeman. 1977. The population ecology of organizations. The
American J ournal of Sociology 82 (5):929-964.
———. 1984. Structural inertia and organizational change. American Sociological Review 49
(2):149-164.
Hoagwood, Kimberly, Barbara J. Burns, Laurel Kiser, Heather Ringeisen, and Sonja K.
Schoenwald. 2001. Evidence-Based Practice in Child and Adolescent Mental Health
Services. Psychiatric Services 52 (9):1179-1189.
Klein, Katherine J., and Andrew P. Knight. 2005. Innovation implementation: Overcoming the
challenge. Current Directions in Psychological Science 14 (5):243-246.
Larsen, E. R., and A. Lomi. 1999. Resetting the clock: A feedback approach to the dnamics of
organisational inertia, survival, and change. The J ournal of the Operational Research
Society 50 (4):406-421.
Levin, Gilbert, and Edward B. Roberts. 1976. The dynamics of human service delivery.
Cambridge, MA: Ballinger Publishing Company.
Meyer, Marshall W., and Lynne G. Zucker. 1989. Permanently failing organizations. Newbury
Park, CA: SAGE Publications.
Newman, M., I. Papdopoulous, and J. Sigsworth. 1998. Barriers to evidence-based practice.
Intensive and Critical Care Nursing 14 (5):231-238.
Ostroff, Cheri, and Neal Schmitt. 1993. Configurations of Organizational Effectiveness and
Efficiency. The Academy of Management J ournal 36 (6):1345-1361.
Ozcan, Y asar A., Ramesh K. Shukla, and Laura H. Tyler. 1997. Organizational Performance in
the Community Mental Health Care System: The Need Fulfillment Perspective.
Organization Science 8 (2):176-191.
Pfeffer, Jeffrey, and Gerald R. Salancik. 1978. The external control of organizations: A resource
dependence perspective. Stanford, CA: Stanford University Press.
Powell, Walter W. 1991. Expanding the scope of institutional analysis. In The new
institutionalism in organizational analysis, edited by W. W. Powell and P. J. DiMaggio.
Chicago, IL: The University of Chicago Press.
Proctor, Enola. 2004. Leverage points for the implementation of evidence-based practice. Brief
Treatment and Crisis Intervention 4 (3):227-242.
Repenning, Nelson P. 2002. A simulation-based approach to understanding the dynamics of
innovation implementation. Organization Science 13 (2):109-127.
Innovation Implementation 26
Rogers, Everett M. 1995. Diffusion of innovation. 4th ed. New Y ork: The Free Press.
Romanelli, Elaine, and Michael L. Tushman. 1994. Organizational transformation as punctuated
equilibrium: An empirical test. Academy of Management Journal 37:1141-1166.
Rosen, Aaron. 1994. Knowledge use in direct practice. Social Service Review 68 (4):561-577.
Rosenheck, Robert A. 2001. Organizational Process: A Missing Link Between Research and
Practice. Psychiatric Services 52 (12):1607-1612.
Sackett, David L., William M. C. Rosenberg, J. A. Muir Gray, R. Brian Haynes, and W. Scott
Richardson. 1996. Evidence based medicine: what it is and what it isn't. British Medical
Journal 312 (7023):71-72.
Samuel, Yitzhak, and Chanoch Jacobsen. 1997. A system dynamics model of planned
organizational change. Computational and mathematical organization theory 3 (3):151-
171.
Sastry, M. Anjali. 1997. Problems and paradoxes in a model of punctuated organizational
change. Administrative Science Quarterly 42 (2):237-275.
Schoenwald, Sonja K., and Kimberly Hoagwood. 2001. Effectiveness, transportability, and
dissemination of interventions: What matters when? Psychiatric Services 52 (9):1190-
1197.
Scott, W. Richard, and John W. Meyer. 1991. The organization of societal sectors: Propositions
and early evidence. In The new institutionalism in organizational analysis, edited by W.
W. Powell and P. J. DiMaggio. Chicago, IL: University of Chicago Press.
Sliverman, Wendy K., William M. Kurtines, and Kimberly Hoagwood. 2004. Research progress
on effectiveness, transportability, and dissemination of empirically supported treatments:
Integrating theory and research. Clinical Psychology: Science and Practice 13 (3):295-
299.
Tushman, Michael L., William H. Newman, and Elaine Romanelli. 1986. Convergence and
upheaval: Managing the unsteady pace of organizational evaluation. California
Management Review 29 (1):29-44.
Tushman, Michael L., and Elaine Romanelli. 1985. Organizational evolution: A metamorphosis
and model of convergence and reorientation. In Research in Organizational Behavior,
edited by L. L. Cummings and B. M. Staw. Greenwich, CT: JAI Press.
Tushman, Michael L., Beverly Virany, and Elaine Romanelli. 1985. Executive succession,
strategic reorientations, and organization evolution. Technology and Society 7:297-313.
Wrightson, Margaret Tucker. 1976. The documentary coding method. In Structure of decision:
The cognitive maps of political elites, edited by R. Axelrod. Princeton, NJ: Princeton
University Press.
Zaltman, Duncan, and Holbek. 1973. Innovations and organizations. New Y ork, NY: Wiley.
