Table of Contents
Go Back
Copyright 2001, by The Regents of the University of California.
Reprinted from the California Management Review, Vol. 43, No. 4.
by permission of The Regents.
Past the Tipping Point:
THE PERSISTENCE OF FIREFIGHTING
IN PRODUCT DEVELO PMENT
Nelson P. Repenning
Paulo Goncalves
LauraJ. Black
oth managers and scholars increasingly realize the central role that
product development plays in creating competitive advantage. Given
this interest, it is not surprising that the design of effective product
development processes has received considerable attention from acade-
mics.* Yet despite this wealth of information and the best intentions of many
product development managers, it is quite clear that practice does not always
follow theory. A growing number of studies suggest that the development
processes prescribed in the literature and the actual sequences of steps used to
create products are two very different things.* Consider for example, the case
of Project X, a major development project that we observed at a large U.S. rec-
reational products manufacturer:
Project X started a little late. In fact, it wasn’t initially scheduled in the product
plan for the year in which it was launched. However, an unmet market need
coupled with the postponement of another project suffering technical difficulties
meant that Project X suddenly became critically important to maintaining cus-
tomer loyalty and business growth. Because it started late, the project team
designing X didn’t meet until well into the development cycle and initially the
team members, the company’s best developers, didn’t have time to give X as
much attention as it deserved. Furthermore, in order to match the development
cycle of other projects launching at the same time as X, the team rushed through
much of the project's up-front work and instead spent most of their time on
actual design activities.
Rapid progress on design encouraged everyone and led the marketing organi-
zation to expand the scope of Project X. When prototypes revealed that the design
Work reported here was supported by the MIT Center for Innovation in Product Development under
NSF Cooperative Agreement N umber EEC-9529140
44 CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
had some serious flaws, however, it was clear X was in trouble. Significant rework
would be needed to address the design concerns and the changes to requirements.
With just a few months remaining before the planned launch date, X seemed at
risk. However, postponing Project X would jeopardize other projects whose
designs had accommodated the geometry of X. Postponing X would also wreak
havoc on future product plans that were premised on X being part of the product
line.
With so much at stake, the organization marshaled its considerable resources
to make sure X succeeded. Developers worked almost non-stop for months, hand-
carrying parts to testing stations and personally visiting suppliers to resolve issues.
More senior managers reviewed X's progress on a weekly basis and provided
additional resources where needed. Even the vice-president of development
pitched in, attending project reviews and making suggestions.
Asit turned out, X ended a success. The product launched on time, and while
a few minor issues required patching up once the product was in the field, on the
whole the organization felt good about its effort. After all, just months earlier the
entire model year was in jeopardy.
The last-minute heroics required to save Project X exemplify one of the
most common and costly departures from the development processes prescribed
in the literature, a phenomenon we label firefighting. The metaphor of fighting
fires is widely used in the management literature, typically referring to the allo-
cation of important resources to solve unanticipated problems or “fires.” In prod-
uct development, firefighting describes the unplanned allocation of developers
and other resources to fix problems discovered late in the development cycle.
The costs associated with firefighting are well documented and this mode of
product development has been widely criticized in the product development
literature.
Yet, despite its costs, it should come as little surprise that firefighting
occasionally occurs in the development of new products. Creating new products
is a fundamentally uncertain task, often involving unproven technologies and
processes. Further, the alternatives to firefighting, such as letting a defective
product reach the market or canceling it altogether, are often even less appeal-
ing. Indeed, many organizations show a remarkable ability to fight fires and
transform troubled projects into marketable products. Further, managers often
view the ability to engage in firefighting as an important complement to more
formalized development processes. As one manager at the company that created
Project X said, “We are a good fourth-quarter team.”
There is, however, a significant problem with this view. Our research
suggests that the major problems that create the need for extra resources and
departures from the standard development process—although usually attributed
to outside forces (changing customer requirements, problematic suppliers) —are
themselves the result of past firefighting efforts. Further, while the ability to
fight fires is often viewed as a necessary skill, required by the messy reality of
developing complex products in a competitive environment, we find that even
the isolated use of firefighting can quickly spread to other projects, driving out
CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001 45
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
the disciplined execution of the desired development process. In other words,
organizations that resort to firefighting in a few projects are soon likely to find
that its use has completely replaced the more structured development process.
In many organizations, firefighting is the development process. Thus, firefighting
is not a complement to a more structured approach to new product develop-
ment, but is, instead, an organizational pathology that, left unchecked, can sig-
nificantly degrade an organization’s ability to create high-quality products.
Here, we develop this idea by reporting the results of a system dynamics
analysis of firefighting. Our results arise from over five years of in-depth study
of product development in major U.S. manufacturing firms. Our research began
with four in-depth case studies of efforts to improve product development at
companies in four different industries (automotive, telecommunications, semi-
conductor, and recreational products) from which we identified the phenome-
non of firefighting as a key impediment to performance in product
development.? In the second phase, one of the original four companies allowed
us to study its development process in even more depth. During this phase, we
conducted in-depth post mortem analyses for each of three consecutive model
years.* Producing these “launch reports” entailed observing the actual launch
of new products in the factory, interviewing every development team that com-
pleted a product in the given model year, reviewing problem logs kept during
the development process, and collecting related quantitative data such as open
concern reports. In total, the three researchers producing these reports spent
over fifteen months at the research site. These data and the original four cases
were then used as the basis for a series of system dynamics models targeted at
explaining both the existence and persistence of firefighting in product
development.>
The most important insight arising from our study is that product devel-
opment systems have a tipping point. In models of infectious diseases, the tipping
point represents the threshold of infectivity and susceptibility beyond which a
disease becomes an epidemic. Similarly, in product development systems there
exists a threshold for problem-solving activity that, when crossed, causes fire-
fighting to spread rapidly from a few isolated projects to the entire development
system. In other words, firefighting is a self-reinfordng syndrome: Once it occurs
in one project, it can quickly spread to others, permanently degrading the capa-
bility of the entire development system.
Practically, this finding has three implications. First, it suggests that fire-
fighting, while initially used only when a project gets “in trouble,” can quickly
become the de facto development process. A reliance on firefighting has a ten-
dency to drive out proper process execution. Second, this also implies that even
a temporary increase in workload can initiate the firefighting dynamic and cause
a permanent decline in system performance. While the costs of permanently
overloading a development process are now well recognized, the existence of a
tipping point suggests that such systems are even more fragile. Even a temporary
overload can initiate a vicious cycle of costly firefighting. Third, our analysis
46 CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
FIGURE 1. 0 verview of the Development Process
launch date for product n-1 launch date for productn —_ launch date for product n+1
' ' '
Product n
Concept Development ff Product Design and Testing
|
Product n+1
Concept Development J Product Design and Testing
Le
Product n+2
Concept Development ff Product Design and Testing
Model Year s Model Year s+1 Model Year s+2 Model Year s+3
shows that the location of the tipping point, and therefore the susceptibility of
the system to the firefighting phenomenon, is determined by resource utilization
in steady state. As more work is introduced into the system, progressively
smaller shocks are needed to initiate the downward spiral of increasing problems
and fewer resources available for their prevention. Thus, development managers
face an important trade-off between steady-state performance and the system’s
ability to accommodate unanticipated changes in resource requirements without
descending into the firefighting cycle. Taken together, these insights suggest that
many of the current methods for aggregate resource planning, while necessary,
are insufficient to prevent firefighting and that managers wishing to avoid it
must rethink their approach to managing multi-project development
environments.
Why Does Firefighting Spread?
To understand both the existence and persistence of firefighting in prod-
uct development, we have developed a number of system dynamics models.®
While some of the models are quite complex, the most important insights can
be developed in a fairly simple framework (see the stylized model of a product
development system in Figure 1).
Suppose an organization has a two-step development process. First,
there is a concept development phase designed to identify the customer's needs,
develop the product concept, and select the supporting technologies. Second,
there is a product design and testing phase, in which the concept developed in
the previous phase is turned into an actual product. For simplicity, we assume
CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001 47
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
FIGURE 2. The Feedback Structure of Multi-Project Product D evelopment
Concept Development Goal for
Activities on Next Desi Problems: Number of
Year's Product. inThis
a oO Design Problems
of o df
Problem Gap
Tipping Loop Rework Loop
+
Resources
Dedicated to This
Year's Product
dt
Resources
Dedicated to Next
Year's Product
that the organization works on only two products at once, each in a different
phase. As shown in Figure 1, in this simple model the organization launches
one new product every year and at any moment in time must divide its atten-
tion between completing the design on this year’s product and developing the
concept for next year’s product. Our modeling studies show however that our
results are robust in more realistic settings.’
To understand the dynamics possible within this simple framework, we
need to capture the interconnections between the two phases. We first assume
that executing each phase requires allocating development resources to com-
plete a set of tasks. In the concept development phase, tasks include activities
such as documenting customer requirements and making the final selection
between candidate concepts. In the design phase, the tasks are related to creat-
ing and testing the product. The primary role of concept development activities
is to make the downstream design work more effective by, for example, docu-
menting customer requirements, establishing the project scope, and demonstrat-
ing the feasibility of the chosen technologies. However, when resources are
scarce, the tasks in the concept development phase can be skipped—products
can be passed to detailed design with few documented customer requirements,
an ambiguous scope, or unproven technologies.® The consequence of skipping
these tasks, however, is that the probability of correctly completing a design task
goes down. For example, as we have often observed, an organization can skimp
on documenting customer requirements but may have to make substantial
changes later in the development cycle when early prototypes reveal mis-
matches between the initial concept and the customers’ desires.°
48 CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
The inner feedback loop in Figure 2 represents the decision process of
managers allocating resources among competing projects in the development
process. Our field studies suggest that when resources are scarce, the projects
nearing launch receive priority. Thus, for example, if there is a rise in problems
in the design phase of the project nearest launch, then the Problem Gap—the
accumulation of unresolved issues in design, components that don’t work, bugs
in the software, and so forth—rises, leading the organization to allocate its
resources to fixing the design problems. As the Design Problems in This Year’s Prod-
uct decrease, the Problem Gap falls, and fewer resources are devoted to this year’s
product. This is the desired balancing dynamic of the Rework Loop—the system
acts to balance, or counteract, a change in any variable in the loop and reduces
the Problem Gap to zero.
The often unintended consequence, however, is that allocating more
developers to the design phase of the product nearest launch decreases Resources
Dedicated to Next Year's Product. Fewer resources dedicated to the concept develop-
ment phase of next year’s product reduce the completion rate of the concept
development activities for next year’s product. This in turn increases Design
Problems in This Year’s Produc when next year’s product transitions to the down-
stream phase. The Tipping Loop is a reinforcing feedback that amplifies a change
in any variable in the loop. It can operate as a virtuous cycle—more resources
devoted to concept development lead to fewer errors in the design phase, freeing
still more resources for concept development on the next year’s product—or it
can become a vicious cycle in which more problems in the downstream phase
draw resources away from the concept development activities that might pre-
vent those problems in future products.
While simple, the mode! captures a set of interconnections that have
repeatedly emerged from both our field studies and subsequent modeling efforts
as central to understanding the firefighting phenomenon. Further, while we
have conducted extensive simulation analysis on these models, the core insights
can be developed graphically using the phase plot (which was derived from the
model’s structure) shown in Figure 3.°
The horizontal axis of the phase plot represents the fraction of the con-
cept development work completed in the current model year, while the vertical
axis captures the fraction of the concept development work that will be com-
pleted next year. The solid, upwardly sloping line shows how, given the struc-
ture of the system, the amount of concept development work will evolve from
year to year. Thus, to read the plot, start at any point on the horizontal axis, go
straight up until you reach the solid line and then go straight left to the vertical
axis. By relating what happens this year to what will happen next year, the
phase plot provides a simple summary of the system's behavior and provides
a useful tool to discuss how product development processes behave over time.
To understand the diagram and its implications for managing product
development, suppose that our simulated organization accomplishes about 60
CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001 49
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
FIGURE 3. A Phase Plot of the D ynamics of Multi-Project Product D evelopment
100
80
60
40
%of Upfront Work
Completed Next Year
20
0 20 40 60
%of Upfront Work
CompletedThisYear
80 100
percent of its planned concept development work. In this case, to determine
what will happen next year, we can, following the dotted line, read up and over
to find that, if it accomplishes 60 percent of the up-front work this year, the
dynamics of the system are such that it will accomplish about 75 percent of the
up-front work next year. If it completes 75 percent of the early-phase work next
year, what will happen the year after that? Return to the horizontal axis, start
at 75 percent, and again read up to the solid black line and over to the vertical
axis to find that in year number three it will accomplish almost 100 percent of
the concept development work. In the example so far, then, the system is driven
by a virtuous cycle. The positive tipping loop shown in Figure 2 works in the up-
ward direction: each year a little more concept work is done, decreasing errors
in the actual design phase and freeing downstream resources to complete even
more concept development work in the next year. Thus, if this cycle continues,
each successive model year will require less firefighting and the system con-
verges to the point where the organization accomplishes all its desired up-front
work every year.
In contrast to this desirable state of affairs, consider another example.
Imagine this time that, for whatever reason in a particular year, our organization
accomplishes only 40 percent of its planned concept development activities.
Now, reading up and over, we find that instead of completing more early-phase
work next year, due to the greater defect rate in down stream activities, the
50 CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
FIGURE 4. Two Stable Execution Modes in the Multi-Project Product
Development System
100
80
Desired
Execution Mode:
100% of up-front
tasks completed
60
%of Upfront Work
Completed Next Year
&
Tipping Point:
Determines shift
20 from virtuous to
vicious
Y
, 0!
Fire Fighting \ = 60
Execution Mode: 7 ,
0% of up-front %of Upfront Work
tasks completed CompletedThisYear
organization actually completes less—in this case only about 25 percent. In
subsequent years, the situation deteriorates further in a vidous cycle of declining
attention to up-front activities and increasing error rates in design work. In this
case, the reinforcing tipping loop works in the downward direction. Here the
growing need to fight fires in the downstream phase progressively reduces the
completion of the early-phase work until the system converges to a mode in
which concept development work is not done. Here, the organization's ongoing
struggle to fix problems in the project closest to launch effectively drives out the
desired development process.
The analysis offers some interesting insights into the behavior of product
development systems. First, as shown in Figure 4, it suggests that such systems
tend to converge to one of two execution modes (highlighted by the two solid
black dots at the upper right and lower left corners). Either they will benefit
from a virtuous cycle of increasing attention to up-front activities and decreasing
error rates in downstream work, or they will be trapped in a vicious cycle of
increased firefighting and decreased attention to the early-phase work that
might have prevented those crises in the first place. In other words, two product
development systems, despite being identical in every way, could potentially
produce very different levels of performance, depending only on the initial
attention given to up-front work.
CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001 51
FIGURE 5.
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
Three Simulated Scenarios
- 0%, 20%, and 25% O ne-
Time Temporary Increase
in Workload
Fraction Competed
Fraction Defective
al
o
=
un
o
05
FRACTION OF CONCEPT
DEVELO PMENT WORK COMPLETED
~ =
4 Bi Zz, 4
Gn
Fy aa
NZ 20% ncrease
r New
5 25% Increase
25% Increase
Second, Figure 4 also highlights
that these two modes are separated by
a tipping point (highlighted by the solid
black dot at the center of the diagram
where the phase plot crosses the 45-
degree line). The tipping point is critical
to understanding the dynamics of the
system because it represents the fraction
of concept development completion at
which the reinforcing tipping loop shown
in Figure 2 switches directions, changing
from a virtuous cycle that eliminates
firefighting to a vicious circle that rein-
forces it. This means that if the system
starts in the desired execution mode and,
for whatever reason, the fraction of con-
cept development work falls below the
tipping point, the system will never
recover. Instead, it will descend into a
downward spiral of increasing error rates
and decreasing resource adequacy, even-
tually becoming trapped in the firefight-
ing mode.
The following experiment high-
lights the role of the tipping point in
determining the system’s behavior. We
simulate our mode! under three different
scenarios.” First, we run the model in
steady state to show its behavior in the
absence of any outside shocks. In the
second case, we simulate the model
again, but this time, in model year one, we temporarily increase the workload
by 20 percent. Such an increase in workload could also represent a particularly
challenging mode! year or the cost of introducing a new technology. In the case
of Project X, the increase represents the actual workload due to a late start and
changes in the product scope. Finally, in the third case, we introduce another
one-time change in workload, but this time it’s a 25 percent increase. The results
are shown in Figure 5.
As the figure highlights, before the change in workload, the simulated
organization operates in the desired mode: It completes all the concept devel-
opment work and consequently finds relatively few defects in the downstream
phase. Not surprisingly, when the 20 percent increase in workload is introduced,
performance declines. The extra workload means that less concept develop-
ment work is done and, consequently, the quality of the final product suffers.
52 CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
However, the decline in quality is temporary; eventually the system recovers to
its initial capability. In contrast, however, the system's response to a 25 percent
increase in workload differs. This is the mode of operation characterizing Project
X, where much of the team’s time was spent on actual design activities instead
of up-front conceptual work, and, consequently, prototypes revealed serious
design flaws. Unfortunately for the real organization launching X, and for the
organization simulated here, the system never recovers. Instead, it descends into
a firefighting cycle in which up-front activities receive little attention and quality
permanently declines.
Why isthe system able to handle a 20 percent temporary increase in
work but not 25 percent? A 20 percent increase is not quite sufficient to push
the system over its tipping point. Thus, with time, the system recovers to its
original capability. In contrast, the 25 percent increase pushes the system over
the threshold, sending the simulated organization on a new path towards the
firefighting equilibrium.
What does this mean for the management of product development
systems? First, it suggests that a temporary increase in workload (and the con-
sequent need to engage in firefighting) can cause a permanent decline in per-
formance. Most managers are now well aware of the costs of permanently
overloading a development system. Our analysis suggests that the situation is
worse—even a temporary overload can ignite the vicious cycle of self-reinforcing
firefighting. Thus, managers considering the possibility of tackling just a little
more work should be aware that such efforts, while possibly successful in the
short run, can jeopardize the long-term health of the development system.
The second important implication for managing product development
comes in realizing that the level of resource utilization determines the location
of the tipping point. The tipping point can be viewed as the balance between
workload and resources beyond which an organization cannot accomplish all
of the tasks necessary to properly execute the projects currently underway. As
managers put more projects in the product plan, thereby increasing resource
utilization (the percentage of people working at full capacity during all available
work hours), the tipping point moves up and to the right. As steady-state
resource utilization increases, smaller and smaller increases in workload (no
matter how temporary) can propel the organization into the firefighting syn-
drome. Thus, a fully utilized product development system is also a system con-
stantly on the verge of descent into firefighting. The phase plots in Figure 6
depict how the location of the tipping point shifts with changes in resource
utilization.
Not surprisingly, managers frequently want to know where their develop-
ment system is relative to its tipping point. While determining the exact location
of the tipping point is difficult, a number of symptoms indicate that resource
utilization is precariously high. For instance, interviews at one organization
yielded numerous indicators of a system caught in a vicious dynamic of firefight-
ing. From design engineering, we heard: “There are insufficient resources for the
CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001 53
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
FIGURE 6. Resource
Utilization in Steady
State Affects the
Location of the
Tipping Point
HIGH UTILIZATION
°%of Upfront Work
Completed his Year
MEDIUM UTILIZATION
P
40
20
%of Upfront Work
Completed Next Year
0 0 ~ 40 60 80 100
of Upfront Work
CompletedThisYear
LOW UTILIZATION
80
%of Upfront Work
Completed Next Year
0 2 40 60 80 100
of Upfront Work
CompletedThisYear
work.” “Morale suffered [with ongoing problems.
in a high-visibility project].” “There’s a lot of ill-
ness among the group.” Similarly, manufacturing
engineers observed: “Manufacturing engineers are
working round the clock, 24 hours a day.” “Turn-
over in manufacturing engineering is high—in five
years maybe 15 percent of the original staff is left.”
Why Does Firefighting Persist?
So far we have tackled the problem of
understanding why firefighting exists. As the
analysis shows, this syndrome ultimately has
its roots in the fact that early-phase tasks, while
highly effective in the long run, can be skipped
when resources are scarce. Managers who try to
“stretch” and accomplish just a little more, while
producing success in the short run, often push
their development systems over the tipping point
and into the downward spiral of decreasing atten-
tion to up-front tasks and increasing problems in
downstream projects. As mentioned earlier, this
scenario repeats itself in numerous organizations.
So the question naturally arises: Wouldn’t man-
agers eventually figure this out? The unfortunate
answer to this question is that, without a detailed
understanding of the dynamics such a system gen-
erates, managers are unlikely to recognize—much
less overcome—the firefighting syndrome.
Consider what happens when a manager
in a system like the one described above decides
to take some resources away from a project in its
early concept development phase and allocate
them to fighting fires in a struggling downstream
project. As the discussion above suggests, this deci-
sion has two consequences. First, with the addi-
tional resources, the troubled project improves.
This happens quickly, the impact of the decision is
easy to assess, and it has a fairly certain outcome.
Later, however, these same actions, if they push
the development organization over the tipping
point and ignite the firefighting cycle, degrade the
performance of the product development system.
In contrast to the improved performance of the
54 CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
specific project, this outcome doesn’t happen right away—it can take a number
of years for these dynamics to unfold—and the impact of these actions is difficult
to assess. Thus, managers making resource allocation decisions in such systems
face a “better-before-worse” trade-off in which the positive but transient conse-
quence of the decision to engage in firefighting happens quickly and is easy to
assess, while the negative but permanent consequence occurs only with a delay
and is difficult to characterize.
In such situations, psychologists, economists, and others who study deci-
sion making have reached a clear conclusion: People do not learn to manage
such systems well.?? In experiments ranging from managing a simulated produc-
tion and distribution system to fighting a simulated forest fire to managing a
simulated fishery, people overweight the short-run positive benefits of their
decisions while ignoring the long-run negative consequences.? People who play
these games produce wildly oscillating production rates and inventory levels,
they allow the firefighting headquarters to burn down and they destroy the
fishery through over-fishing. Applying these results to the product development
context suggests both that managers will typically favor downstream projects
and that they are unlikely to learn to overcome the undesirable dynamics that
such a bias creates.
The problem is still worse. The fact that the health of the development
system rests on managing across projects makes identifying and analyzing the
source of difficulties challenging. When a development organization is caught in
the firefighting cycle, people know something is wrong, but they don’t necessar-
ily know exactly what the problem is. When performance is persistently low,
managers are more likely to blame the people who work for them than the
structure of the development system.** Thus, as performance begins to decline
in a downward spiral of firefighting, not only are managers unlikely to learn to
manage the system better, but they are also likely to blame their designers,
developers, and project managers. To make matters worse, the system provides
little evidence to discredit this hypothesis. Once firefighting starts, system per-
formance continues to decline, even if the workload returns to its initial level.
Furthermore, managers will observe developers spending less and less time on
up-front activities like concept development, providing powerful evidence to
confirm managers’ mistaken belief that developers are to blame for the declining
performance. The result is one example of the more general dynamics discussed
in another article in this issue (see Nelson P. Repenning and J ohn D. Sterman,
“Nobody Ever Gets Credit for Fixing Problems that Never Happened: Creating
and Sustaining Process Improvement,” California Management Review, 43/4 [Sum-
mer 2001]). Management teams become increasingly frustrated with a devel-
opment staff they perceive as lazy, undisciplined, and unwilling to follow the
prescribed development process, and the development staff becomes increasingly
frustrated with managers who, they feel, do not understand the realities of
launching new products and, consequently, set unachievable objectives.
CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001 55
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
To summarize, we tip our systems because actions that appear rational
at the project level—the use of firefighting to compensate for unplanned prob-
lems—undermine effective management of the development system. Once the
system has entered the firefighting zone, escape is difficult. The firefighting syn-
drome persists because few actions can push the product development system
back over the tipping point into a virtuous cycle of improved process execution.
Policies: W hat To Do?
If your organization is prone to these dynamics, what actions can move
you onto the virtuous side of the tipping point and eliminate the need for costly
firefighting? Most important, realize that managing such systems well doesn’t
come naturally. As highlighted above, the “better-before-worse” behavior cre-
ated by firefighting coupled with basic human tendencies and intuitions leads
decision makers down the wrong path when managing complex systems like
multi-project development environments. Thus, standard managerial responses
such as admonitions to “do less firefighting” or to “focus on the process, not the
product” are likely to be inadequate in combating the deeply seated conditioning
created by years of operating in the firefighting mode. Firefighting is so preva-
lent precisely because it comes so naturally.
Creating a fire-resistant product development system thus requires more
significant, permanent changes. However, these need to be executed with con-
siderable care. Both our field data and subsequent analysis suggest that many
interventions that appear to eliminate firefighting actually have the potential
to make the situation worse rather than better. Three such strategies are par-
ticularly damaging to the long-run health of the development process.
= Don’t invest in new tools and processes if you're already resource-constrained. A
common reaction to the low performance created by continuous firefight-
ingisinvestment in an improved set of development tools and processes.
Although successful deployment of such tools would often unequivocally
help the organization, they require the development of knowledge and
experience. As a consequence, introducing tools actually lowers produc-
tivity in the short run while people learn and incorporate them into
existing processes. Problems arise when managers ignore this worse-
before-better tradeoff. Recall that in the simulation experiments shown
earlier, a temporary increase in workload was sufficient to push the sys-
tem over its tipping point. Introducing a new set of tools into an already
over-utilized development system creates a similar set of dynamics. The
increase in workload, arising from the additional training, learning, and
practice time required to use the tools proficiently, further raises resource
utilization, potentially pushing the system over its tipping point. Thus,
if new tools are not accompanied by a reduced workload, their introduc-
tion is likely to lead to more firefighting and a further decline in process
capability.
56 CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
= Fixing only a fraction of the problems identified late in the development cyde does
not prevent the spread of firefighting. A second, and often observed, approach
to reducing the use of firefighting is to fix only the “important” problems.
However, although fixing a fraction of problems can lead to a short-term
decrease in workload, it rarely has the desired effect in the long run.?©
Leaving some problems unresolved usually means that the product leaves
customers unsatisfied, thereby increasing the pressure on the next gener-
ation of the product while simultaneously leaving even less time to do the
up-front work that it requires. Further, and even more damaging, lower-
quality products can push the organization into a death spiral of declining
competitiveness: low-quality products generate less revenue to fund
research and development, thereby limiting the ability to invest in the
development of future products. Leaving defects unattended is not an
effective strategy for eliminating firefighting.
= Postponing problematic projects does not prevent the spread of firefighting. A final
popular response to firefighting is the postponement of troubled projects
under the rationale that the reduced workload can be used to improve
both the design of the current project (allowing it to be launched with
higher quality) and next year’s product concept. However, because post-
ponement does not address the root cause of over-utilized resources
(postponed projects continue to consume resources until they are
launched), it does little but defer the problems to the next model year.
In the context of our model, this strategy amounts to doubling the quan-
tity of outstanding downstream work in the model year following the
postponement.?” Having more projects in the downstream phase increases
the need for firefighting, reduces the attention to up-front work in sub-
sequent generations, and further speeds the system's descent into low
capability.
Thus, as is common in complex systems, many of the policies that would
seem to help alleviate a problem actually exacerbate it. The problem with each
of the aforementioned approaches is that, while they address the symptoms of
firefighting, they do nothing to eliminate the root cause. Any policy targeted
at eliminating firefighting must improve the balance between the outstanding
workload and the available resources. Building on our field data and subsequent
analysis, we find that creating a fire-resistant development system requires four
complementary policies.
= Aggregate resource planning is critical to fire prevention. Although managers
are increasingly aware that resources must be managed across as well as
within projects, most development organizations work on too many pro-
jects.® As our analysis shows, when a system is consistently overloaded,
firefighting quickly becomes the development process. Thus, there is no
substitute for a portfolio-level plan that matches the number of ongoing
projects to available development resources. Further, the existence of a
tipping point implies that slack resources provide a valuable buffer against
CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001 57
58
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
the uncertainties inherent in new product development. Thus, to prevent
firefighting, the aggregate project plan should allow for uncertainty by
maintaining a reserve of unassigned resources. Managers who attempt to
utilize fully their development resources are virtually guaranteed to spend
much of their time engaged in firefighting.
Cancel projects with inadequate concepts before they reach the design phase.
While necessary, aggregate resource planning is insufficient to prevent the
spread of firefighting. As discussed above, even a single troubled project,
if it draws resources dedicated to other projects, can push the system over
its tipping point. Thus, beyond improving the resource planning process,
preventing firefighting also requires minimizing the chance that troubled
projects actually reach the downstream phases of the development cycle.
The key is to cancel projects early, before they have tipped the system.
Canceling a troubled project shortly before its scheduled launch means
that an organization pays all of the costs of firefighting and receives none
of the benefit of canceling. Our simulation experiments suggest that an
aggressive policy of canceling those projects with inadequately developed
concepts before they enter the design phase is very effective in preventing
firefighting.®
When a project does experience trouble in the later phases of the devdopment qyde,
don’t try to “catch up”— revisit the product plan instead. Although it can clearly
be minimized, the late discovery of some errors is inevitable when devel-
oping products with new technologies for new markets. These problems,
if they are to be solved, constitute temporary increases in resource
requirements with the potential to push the system over the tipping
point. Unfortunately, in the organizations we studied, managers rarely
accounted for these contingencies in the resource planning process.
Despite the fact that such problems often required significant allocations
of resources not captured in the initial aggregate resource plan, managers
rarely updated their plans when troubles arose, implicitly assuming (as
was the case in Project X) that they could “catch up” by just “working
alittle harder” or “squeezing a little more.” Yet, this was rarely (if ever)
the case. Instead, resources were stolen from other projects in the earlier
stages of the development process, trapping the development system in
the firefighting syndrome. For example, despite the fact that the late dis-
covery of defects in Project X required substantial engineering resources
be pulled off other projects, the schedules for those other projects were
not changed. As a consequence, those later projects were also moved to
the design phase with inadequately specified concepts, creating the need
for further firefighting and, thereby, perpetuating the cycle. Thus, if a
significant problem does arise late in the development cycle, ensuring
that firefighting does not spread from the troubled project to the rest
of the portfolio requires that managers revisit the plans for all of the proj-
ects currently in progress. If for example, a given project requires three
CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
months of additional work, then the product plan must be updated to
reflect the fact that those people working on the project will not be avail-
able for another three months. In our model, such a strategy amounts to
either extending the model year whenever a product gets in trouble or
simply skipping a model year altogether. In either case, the effective
reduction in workload stabilizes the system and prevents the firefighting
required by the troubled project from spreading to the rest of the develop-
ment system.
= Don’t reward developers for being good firefighters. As discussed in the intro-
duction, many managers we interviewed came to view their ability to
fight fires as an important source of competitive advantage. Not surpris-
ingly, many organizations reward and promote engineers based on their
ability to save troubled projects. Consider, for example, one senior man-
ager’s reflection on how developers in his organizations were rewarded:
“Occasionally there is a superstar of an engineer or a manager that can take
one of these late changes and run through the gauntlet of all the possible ways
that it could screw up and make it a success. And then we make a hero out
of that person. And everybody else who wants to be a hero says, ‘Oh, that is
what is valued around here.’ It is not valued to do the routine work months in
advance and do the testing and eliminate all the problems before they become
problems. What is valued is being able to make a change in the last minute and
ramrod it through.”
Thus, to be successful, improvements in the resource planning process
require complementary changes to the incentive and reward systems.
While a number of changes are possible, perhaps the most important
one isto simply not let project leaders jump into projects with the hope
of saving them at the last minute. As the quote highlights, allowing man-
agers to “save” troubled projects, and therefore receive accolades and
benefits, creates a situation in which, for those interested in advance-
ment, there is little incentive to execute a project properly from start to
finish. While allowing such heroics may help in the short run, the long
run health of the development system is better served by not rewarding
them.
There are, of course, major challenges associated with implementing
these changes. First, it is clear both from the field studies on which this article
is based and from other studies reported in the product development literature
that admonitions to “do fewer projects” and “cancel more,” though often heard
in R&D organizations, rarely if ever have appreciable impact. Although people
may understand and subscribe to these ideas at an abstract level, when decisions
concerning specific projects must be made on a day-to-day basis, these general
guidelines are simply inadequate to combat the deeply seated cognitive biases
created by experience uninformed by a structural understanding of the multi-
project development system. Recall the first set of simulations. In the first model
year, the system ably accommodates an increased workload (even one that
CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001 59
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
pushes the system beyond the tipping point), delivering the additional work
with high quality. A manager making such a decision receives powerful and
salient feedback that working a little harder was a good thing to do. Unfortu-
nately, the short-run gain comes at the expense of long-run performance. In
subsequent model years, performance can progressively decline—even though
workload returns to normal. Because the decline in performance occurs only
with a significant delay, it is less likely to be associated with the earlier increase
in workload in any manager’s mind. This creates a strong belief that they can
always “do just a little more.”
Second, even if managers seek to rebalance resources with the workload,
another problem arises: Nobody likes having her project canceled or postponed.
Developers and project managers are rewarded for delivering projects, not for
canceling them. During project reviews there is a strong tendency to overstate
the level of project completion and understate the remaining resource require-
ments. Managers don’t always receive accurate information concerning the state
of projects in the pipeline, thus creating additional uncertainty and further
reducing the likelihood that they will take the difficult step of canceling projects.
Given the strength of managers’ biases and the incentives facing project
managers, we believe that a particularly strict and inflexible version of the can-
cellation policy offers the highest potential to produce significant improvement
in product development. Specifically, we propose that managers establish a strict
policy of allowing only a fixed number of projects into the detailed design phase
and, more important, uphold a strict policy of canceling projects with unfinished
concept development work.
Why might such draconian measures be needed to produce the desired
outcome? First, recognize that no development system can produce beyond its
steady state capacity for very long. Thus, organizations cancel and postpone pro-
jects all the time, but usually only after such projects have consumed substantial
resources, thus making these interventions ineffective in eliminating the spread
of firefighting. Implementing such a policy puts a decision that is normally made
implicitly and with little forethought—through the day to day actions of devel-
opers and project leaders—under the explicit control of managers. Second, as
previously highlighted, the psychological biases, institutional structures, and
incentive schemes that create the pathologies we studied here are deeply
ingrained and unlikely to change easily. A more flexible version of the cancella-
tion policy is subject to modifications by managers that would erode its efficacy
over time. If managers are given discretion to increase the number of projects,
they are likely to use it, thus putting the system back in the undesirable situa-
tion in which it started.
Furthermore, such a policy gives both developers and managers a strong
incentive to develop competence in evaluating projects mid-cycle and perform-
ing thorough up-front work. There is an additional reason why organizations
may often fail to do up-front work: They may not know how. Organizations
that do not invest in up-front activities are unlikely ever to develop significant
60 CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
competence in executing them. Thus, the decision to change the balance of
resources not only necessitates reducing the number of projects; it may also
require a significant investment in learning. Individual managers and project
leaders are unlikely to make such investments for fear of incurring the inevitable
but temporary decline in performance. More senior managers may be unable to
create incentives for people to undertake such learning activities.
The policy we propose has the potential for reversing many of the feed-
backs that otherwise work against higher performance. Managers required to
cancel projects on an annual basis will need defensible criteria for doing so. Pro-
ject managers will soon learn that, to survive the cut, they must demonstrate
the likelihood that their project will succeed. In contrast to the situations we
have observed (in which there are few incentives to do up-front work), ina
world where some projects are always canceled before entering the design phase,
project managers face strong incentives to invest in early-phase activities and
develop clever ways to demonstrate the efficacy of a proposed product far in
advance of its detailed design. Similarly, while project managers often portray
their projects in the kindest light, more senior managers, if they are required
to cancel a certain number of projects, will also have a powerful incentive to
develop more objective methods of determining which projects are allowed to
proceed into the detailed design phase.
What do these suggested policies mean for Project X? First, given its
late start, managers probably should not have agreed to undertake Project X
in the first place. Without adequate time to proceed through the concept devel-
opment phase, Project X was almost guaranteed to experience serious difficul
ties during the design and testing phase. Second, when those problems were
revealed by early prototypes, Project X probably should have been canceled. By
continuing the project despite outstanding issues that required significant addi-
tional resources, managers put the entire product plan at risk. Finally, given that
it did proceed, despite its major problems, managers should have revisited the
product plan and adjusted the schedules for the other projects in the pipeline
that were not receiving attention due to the firefighting required by Project X.
Admittedly, the implications for Project X are grim. When faced with an
urgent market need, the company’s most talented developers are usually willing
to give it their best shot. However, that is precisely why avoiding situations like
Project X requires policies that both prevent projects from getting in trouble in
the first place and, when they do experience difficulties, ensure the cure is not
worse than the disease. Only by vigilantly managing the balance between work
and resources will organizations be able to prevent widespread firefighting and
maintain the long-term health of their product development systems.
Notes
1. See, for example, R.G. Cooper, Winning at New Products (Reading, MA: Addison
Wesley, 1993); S. Wheelwright and K. Clark, Leading Product Development (New
CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001 61
10.
11.
62
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
York, NY: The Free Press, 1995); K. Ulrich and S. Eppinger, Product Design and
Development (New York, NY: McGraw-Hill Inc., 1995).
. See, for example, A. Griffin, “PDMA Research on New Product Development
Practices: Updating Trends and Benchmarking Best Practices,” J ournal of Product
Innovation Management, 14 (1997): 429-458; P. O'Connor, “From Experience:
Implementing a Stage-Gate Process: A Multi-Company Perspective,” J ournal of
Product Innovation Management, 11 (1994): 183-200.
. For summaries of these case studies, see E. Keating and R. Oliva, “A Dynamic
Theory of Sustaining Process Improvement Teams in Product Development,” in
M. Beyerlein and D. J ohnson, eds., Advances in Interdisciplinary Studies of Teams
(Greenwich, CT: J Al Press, 2000); R. Oliva, S. Rockart, and J. Sterman, “Managing
Multiple Improvement Efforts: Lessons from a Semiconductor Manufacturing
Site,” in D. Fedor and S. Ghosh, eds., Advances in the Management of Organizational
Quality (Greenwich, CT: J Al Press, 1998), pp. 1-55; N.P. Repenning and J .D. Ster-
man, “Getting Quality the Old-Fashioned Way: Self-Confirming Attributions in
the Dynamics of Process Improvement,” in R. Scott and R. Cole, eds., Improving
Research in Total Quality Management (Newbury Park, CA: Sage, 2000), pp. 201-235.
Documentation on the “launch reports” can be found in C. Morrison, “Product
Development Process Assessment,” unpublished M.Sc. thesis, Massachusetts Insti-
tute of Technology, 2000; M. Van der Wel, “Product Development Process Assess-
ment,” unpublished M.Sc. thesis, Massachusetts Institute of Technology, 2001.
. Formal models can be found in N.P. Repenning, “A Dynamic Mode! of Resource
Allocation in Multi-Project Research and Development Systems,” System Dynamics
Review, 16/3 (2000): 173-212; L.J. Black and N.P. Repenning, “Why Firefighting
Is Never Enough: Preserving High Quality Product Development,” System Dynamics
Review, 17/1 (2001): 33-62; N.P. Repenning, “Understanding Fire Fighting in New
Product Development,” J ournal of Product Innovation Management, 18/5 (2001);
K. Pilon and G. Herweg, “System Dynamics Modeling for the Exploration of
Manpower Project Staffing Decisions in the Context of a Multi-Project Enter-
prise,” unpublished M.Sc. thesis, Massachusetts Institute of Technology, 2001.
. _Repenning (2000), op. cit.; Black and Repenning, op. cit.; Repenning (2001),
op. cit.; Pilon and Herweg, op. cit.
. See, for example, Pilon and Herweg, op. cit.
. Unfortunately, according to Cooper and Kleinschmidt, these up-front tasks are
often skipped. See R.G. Cooper and E.J. Kleinschmidt, “New Products: What Sep-
arates Winners from Losers,” J ournal of Product Innovation Management, 4/3 (1987):
169-184.
. The importance of concept development for the success of the product develop-
ment process is stressed by Cooper, op. cit. Also, this observation is supported by
empirical studies. R.G. Cooper and E.J . Kleinschmidt, “An Investigation into the
New Product Process: Steps, Deficiencies and Impact,” J ournal of Product Innovation
Management, 3 (1986): 71-85.
In particular, the system in Figure 2 serves as the basis for the development of a
dynamic non-linear model. A reduced form of this model provides an equation
relating the fraction of concept development tasks completed in a given year, the
amount of work in the system, the annual capacity, and the fraction of concept
development tasks completed in the previous year. For details, see Repenning
(2001), op. cit.
For additional documentation of the model discussed here, see Repenning (2001),
op. cit. Also, a running version of the model can be downloaded at
<http://web.mit.edu/nelsonr/www/>.
CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001
12.
13.
Past the Tipping Point: The Persistence of Firefighting in Product D evelopment
See, for example, E. Diehl and J.D. Sterman, “Effects of Feedback Complexity on
Dynamic Decision Making,” Organizational Behavior and Human Dedsion Processes,
62/2 (1995): 198-215.
See B. Brehmer, “Dynamic Decision Making: Human Control of Complex Sys-
tems,” Acta Psychologica, 81 (1992): 211-241; E. Moxnes, “Misperceptions of Bioe-
conomics,” Management Sdence, 44/9 (1998): 1234-1248; J .D. Sterman, “Modeling
Managerial Behavior: Misperceptions of Feedback in a Dynamic Decision Making
Experiment,” Management Sdence, 35/3 (1989): 321-339.
, See Repenning and Sterman, op. cit.
. The role of new tools in initiating firefighting is formally analyzed in Repenning
(2000), op. cit.
. This policy in analyzed in Black and Repenning, op. cit.
. This is also analyzed in Black and Repenning, op. cit.
. S. Wheelwright and K. Clark, op. cit.
. Black and Repenning, op. cit.
Back to the Top
CALIFORNIA MANAGEMENT REVIEW VOL.43,NO.4 SUMMER 2001 63
