Explaining Capacity Overshoot and Price War: Misperceptions of
Feedback in Competitive Growth Markets
Paul A. Langley*, Mark Paich*, J ohn D. Sterman*
*McKinsey & Company
**S loan School of Management, Massachusetts Institute of Technology
Abstract
Companies consistently get into trouble in rapid growth markets. Frequently they
grow too fast, overshoot when the market saturates, then get into price wars and
suffer huge losses due to low prices and excess capacity.
The companies that grow most aggressively sometimes lose the most, contrary to
the new conventional wisdom that you have to be the largest player to benefit from
increasing returns and positive feedbacks that confer success to the successful. How
can the prevalence and persistence of this dynamic be explained? Is it just bad luck
or is there a systematic explanation. And how can firms do better?
To explore these issues, we designed an experiment involving over 270 subjects
(MBA and short course Executives). Subjects played the role of a management team
for one firm in a simulated duopoly market, with a rapidly growing demand for the
new product. As in the real world, market potential and the course of the product
lifecycle were highly uncertain. Subjects made quarterly capacity, pricing and
marketing decisions over a simulated ten year period. Performance was measured
by cumulative net income.
The results showed that subjects systematically made pricing decisions that were not
only far from the "optimal" price, but were often in the opposite direction from the
optimal change. Subject performance was very poor, compared to a benchmark
performance computed using simple behavioral decision rules. Subjects did not
substantially modify their policies under different market structures or different
competitor strategies. Neither did they modify their policies over trials - little learning
took place.
The poor performance is explained in terms of flaws in the subjects’ mental models -
their "misperceptions of feedback". We close with discussion of implications for
improved senior management strategies in new product markets.
Introduction
The traditional “learning curve” perspective applied to the growth of new product
markets suggests that early entrants can achieve sustainable competitive advantage
through rapid investment in capacity and by pricing aggressively to pre-empt
competition (Moore, 1996). Recently, aggressive strategies designed to gain dominant
market positions have been reemphasized as theories of so-called ‘increasing returns’
based on positive feedback (due to network externalities, complementary assets,
economies of scale and scope, and others) have gained considerable attention (e.g.
Arthur 1994). However, previous work based on simulated markets has shown that
aggressive strategies are suboptimal when the market is dynamically complex (Paich
and Sterman 1993, Sterman et al, 1995). In recent years, the dynamic complexity of
markets has increased dramatically, through shrinking product life-cycles, the
acceleration of globalization and intensified competition.
Frequently, competing firms chase market share with the perception that “early
dominance will lead to near monopolies as customers become locked in and reluctant
to switch to competitors” (Wall Street J ournal, 12 December 1996, p. Al). Many cases
of “boom and bust” suggest that overcapacity, price wars and bankruptcies are chronic
dysfunctional behaviors in industries which experience rapid growth and sudden
saturation. These markets include consumer durables (such as bicycles and chain
saws), consumer electronics (such as video games, personal computers, CB radios
and VCRs), toys and games, fashions, and fads (such as wine coolers and fashion
watches).
One characteristic of industries prone to profit-destroying boom and bust is the dynamic
complexity of the strategy formulation problem. The existence of network effects, scale
and scope economies, high fixed costs, and the strong role of complementary assets
(e.g. software for PCs) does create positive feedbacks that favor an aggressive
strategy aimed at market share dominance through rapid growth and low prices. But at
the same time, short and unpredictable product lifecycles, rapid growth, intense
competition, and long delays in adjusting capacity favor more conservative strategies.
In addition to the tension between these opposites, the forces at work interact strongly.
Hence managers have new dynamically complex issues to face, specifically relating to
strategies on product pricing and capacity acquisition, for which their mental models
and prior experience are inadequate, if not simply wrong.
The misperceptions of feedback (MOF) hypothesis (Sterman 1989a and 1989b)
suggests that decision makers systematically misperceive dynamic environments that
include multiple interacting feedback loops, time delays and nonlinearities. These
misperceptions result in decisions that are far from optimal, and, in contrast to the
economic view of decision making, leave a great deal of "money on the table". The
misperception hypothesis has been supported by several experimental studies in the
fields of system dynamics and behavioral decision theory (e.g. Diehl 1992, Kleinmuntz
and Thomas 1987, Brehmer 1990 and 1992, Smith et al 1988, Funke 1991).
This paper applies the MOF perspective to the critical issue of pricing strategy. This
paper presents the results of several experiments that test the MOF theory in the
context of pricing decisions made in a rapidly growing market. The experiment required
2
the subjects to make price and capacity decisions in a simulated market for a new
product. The results show that many subjects made pricing decisions that were
consistently in the opposite direction from the optimal change. Subjects systematically
lowered price when it should have been raised and raised price when it should have
been lowered. Other subjects changed price in the correct direction, but the magnitude
of the change was much smaller than optimal.
In addition, the paper demonstrates that the incorrect price decisions can be interpreted
as the result of the subjects’ misperception of the feedback structure of the
environment. The greater the dynamic complexity of the market environment, the
worse performance is relative to the benchmarks (i.e., over and above changes in
intrinsic task difficulty). The subjects’ decisions would have been reasonable if the
environment were simpler and did not include important feedbacks among price,
orders, order backlog and lead-time. In fact, the subjects’ decisions were close to
optimal for the simple environment in which the subjects presume the market is in
equilibrium. Unfortunately, the environment was not as simple as the subjects
presumed. Price strategies that would have been effective in a simplified environment
close to equilibrium were dysfunctional in the actual, disequilibrium environment of
rapid growth markets.
The Misperceptions of Feedback (MOF) Hypothesis
The MOF hypothesis holds that decision makers systematically misperceive
environments characterized by interacting feedback loops, time delays and
nonlinearities. This misperception of feedback results in decisions that are far from
optimal and are often much worse than decisions generated by simple, naive
decision rules. The source of the misperception is the combination of the complexity
of environment and the bounded rationality of the decision maker. Systems
comprised of multiple feedback loops and time delays are too complex to be
analyzed completely. Consequently, boundedly rational decision makers resort to
simplification strategies that may ignore important feedback relationships but make
the decision problem more tractable. The analysis of several experiments has shown
that decision rules that would have been successful in simplified settings yield very
poor outcomes in the actual environment.
Earlier work (Paich and Sterman, 1993) designed an experiment using business school
subjects involved in a decision making task portraying new product dynamics in a
simulated duopoly market structure. The experimental treatments were the strength of
the key feedback processes (the strength of word of mouth and product durability) in a
simulated market. Over a number of repeated trials, performance relative to potential
was poor and was severely degraded when the feedback complexity of the
environment was high. However, the behavior of the simulated competitor in the
original experiment was quite simple. The competitor set price using a “cost-plus”
strategy with a constant markup. In this study, the simulated market structure has been
modified to include four different pricing strategies for the competitor (“cost plus”,
“margin oriented”, “share oriented”, “tit for tat”). These pricing strategies provide a
more realistic range of competitor behavior and allow us to test the robustness of prior
results.
Model Structure
We used an interactive computer game for the experiment, the “B&B Enterprises
Management Flight Simulator” (Sterman 1991; Graham et al 1992 discuss design
principles and give examples). The flight simulator embodies a model representing a
firm, its market, and its competition. Subjects manage a new product from launch
through maturity, making price, capacity, and marketing decisions each quarter year
through a ten-year simulation.
Market Sector
The market model is based on well known diffusion models in the tradition of Bass
(1969), Kalish and Lilien (1986), Mahajan and Wind (1986), Homer (1987), and
Mahajan, Muller, and Bass (1990). The essence of these models is the feedback
structure through which potential purchasers become aware of and choose to buy
the product. Adoption increases the customer base, generating word of mouth which
leads to additional sales (a positive feedback), but also depleting the pool of potential
customers (a negative feedback). The customer base follows an s-shaped pattern,
while sales rise exponentially, then peak and decline to the rate of replacement
purchases as the market saturates. Key features of the market sector include:
«Product price affects the number of potential adopters. The elasticity of industry
demand is less than unity, quite typical for many goods (Hauthakker and Taylor
1970).
« The greater the aggregated marketing expenditures of the firm and the competition,
the larger the fraction of potential customers who purchase each quarter.
Diminishing returns set in for high marketing expenditure levels.
«Demand is also generated by word of mouth. Word of mouth is driven by recent
purchasers (people who are still excited by the product and have not yet come to
take it for granted). The strength of the word of mouth effect (the number of
purchases generated per quarter by each recent purchaser) was a treatment
variable in the experiment.
«A fraction of the customer base re-enters the market each quarter to replace worn
or obsolete units. The repurchase fraction was a treatment variable in the
experiment.
* Total orders for the product are divided between the firm and the competition in
proportion to the attractiveness of each product. Attractiveness depends on price,
availability (measured by delivery delay), and marketing expenditure. Firm demand
is highly but not infinitely elastic - price is important to consumers but availability
and marketing can differentiate the two products.
Firm sector
While many diffusion models implicitly equate shipments with orders, the model here
explicitly represents the supply side of the market. The key assumptions of the firm
sector are:
«Product is built to order. Customer orders flow into a backlog until they are
produced and shipped. The firm will ship the current backlog within one period
unless capacity is inadequate, in which case the backlog and delivery delay rise,
reducing the attractiveness of the firm’s product and the share of orders it receives.
4
* Subjects set a capacity target each quarter. Actual capacity adjusts to the target
with a delay representing the time required to plan for, acquire, and ramp up new
production facilities. Capacity adjustments follow a distributed lag with a mean of
four quarters. Some investments can be realized sooner than four quarters
(purchasing equipment), while some take longer (building new plant). For simplicity
the delay is symmetrical in the case of capacity reduction.
«The firm benefits from a learning curve which reduces unit costs as cumulative
production experience grows. A standard “80%” learning curve is assumed - each
doubling of cumulative production reduces unit variable costs by 20%. The
competitor's learning curve has identical strength. Learning is assumed to be fully
appropriable.
* Profit is revenue less total costs. Total costs consist of fixed and variable costs,
marketing expenditures, and investment costs. Revenues are determined by the
quantity shipped in the current quarter and the average price received for those
units. Customers pay the price in effect when they booked their order, even if the
price has changed in the interim.
« Fixed Costs are proportional to current capacity. Unit fixed costs are constant.
Variable costs are proportional to output. Unit variable costs fall as cumulative
production increases. The fraction of revenue spent on marketing is a decision
made by the subject each quarter.
«Investment costs represent administrative, installation, training, and other costs of
increasing capacity. Symmetric decommissioning costs are incurred whenever
capacity is decreased. Investment costs are proportional to the magnitude of the
rate of change of capacity.
* Subjects may lose as much money as they like without facing bankruptcy. The task
is therefore more forgiving than reality since losses leading to bankruptcy in real life
can in the game be offset by subsequent profits.
Competitor Structure and Strategy
The subject's firm faces competition from another firm which has launched a similar
product at the same time. The playing field is level - the structure and parameters for
the firm and its competitor are identical. But while the subjects make price and target
capacity decisions for their firm, the competitor's price and target capacity decisions
are simulated.
The competitor sets target capacity to meet expected orders and maintain normal
capacity utilization. Expected orders are determined by the current order rate and the
expected growth rate of orders. Extrapolative expectations are assumed: the recent
growth rate of orders is projected four quarters ahead - the length of the capacity
acquisition lag - to account for the growth in demand likely to occur while awaiting
delivery of capacity ordered today. The forecast of future demand is adjusted in
proportion to the balance between desired production and capacity. If desired
production exceeds current capacity, additional capacity is ordered to reduce the
backlog, and vice-versa. The decision rule for competitor capacity acquisition is
extensively used in simulation models and is well supported empirically and experi-
mentally (Senge 1980, Sterman 1987a, 1987b). The competitor price decision varies
according one of four competitor scenarios (C1-C4) as follows:
Competitor Scenario 1 - “cost plus”
Competitor 1 is a simple base case. Costs are the only determinant of competitor
price. The competitor here is totally unresponsive to subject decisions, the
competitive situation, or market forces. The competitor sets price at a constant mark-
up over cost. Competitor price falls as costs move down the learning curve. A poor
outcome for subjects in scenario 1 cannot be blamed on a sophisticated or wily
competitor.
Competitor Scenario 2 - “share oriented”
Competitor 2 represents an aggressive market-share oriented player. The
competitor's goal for market share is 75%, and the competitor will aggressively cut
mark-up whenever actual share is less than this goal. If share exceeds the goal,
mark-up is raised only slightly. Likewise, the competitor will cut price dollar for dollar
when the subject's price is less than its own (but not below unit variable cost). If the
subject's price is higher than its own, however, mark-up is raised less than
proportionately to boost share. Finally, mark-up is cut aggressively when there is
excess capacity, but raised only weakly if capacity is inadequate. In scenario 2 the
competitor prices low from the beginning to gain share and is extremely likely to
retaliate to any move by the subject to lower price. Scenario 2 is the most difficult, for
several reasons. First, by pricing low, the competitor increases the size of the market
and growth of demand during the boom phase, often leading to a bigger bust as the
market saturates. Secondly, the aggressive competitor retaliates strongly to any
move by the subject to lower her price, often engaging subjects in a price war they
did not intend to fight.
Competitor Scenario 3 - “margin oriented”
Competitor 3 represents a margin oriented competitor. The competitor's goal for
market share here is an equitable 50%. If actual share is less than this goal the
competitor cuts mark-up only slightly, preferring to give up share if necessary rather
then sacrificing margin. If share exceeds the 50% target, the competitor in this
scenario will raise mark-up to boost profitability even if such action pushes share
down again. Similarly, when the subject's price is lower, the competitor lowers mark-
up only slightly, but will raise mark-up aggressively when it finds its product selling
for less. The competitor also raises mark-up when capacity is insufficient to meet
demand, but only cuts mark-up slightly when there is excess capacity. Essentially,
the competitor seeks a collusive equilibrium in which both firms split the market at
the collusive rather than competitive equilibrium price. The competitor's response to
disequilibrium is to signal its desire to achieve the collusive equilibrium by keeping
prices high even at the cost of market share or capacity utilization. This competitor
can be exploited by the subjects. If the subject wants to build up her market share,
she may easily undercut the competitor without provoking strong retaliation. If she
wants to increase her own margins, she may do so easily without losing market
share. Most important, however, by encouraging higher prices for both the
competitor and the subject, scenario 3 slows the growth of demand and smoothes
out the transition from boom to bust. A strategy like scenario 3 is in fact the optimal
strategy for this environment.
Competitor Scenario 4 - “tit for tat”
Competitor 4 responds aggressively to imbalances in both directions. The target
market share here is 50%. The competitor adjusts mark-up strongly in the face of
imbalances in either direction in market share, relative price and demand/supply
balance. Scenario 4 puts the player in an environment where the competitor makes
strong moves. The direction of these moves depends to a great extent on the
subject's own decisions. The game may evolve to an implicitly collusive equilibrium
in which both subject and competitor price high, smooth the industry life cycle, and
reap large profits, or it may degenerate into a price war, severe boom and bust, and
large losses for both.
It is important to note that the model of competitor behavior used here does not
presume the competitor is omniscient. The competitor price is set without recourse to
any complex game-theoretic reasoning, nor does the competitor rely on information
the player does not have. On the contrary, the competitor is modeled as an entity
with bounded rationality, who uses simple but realistic rules of thumb in setting price
(see for example Morecroft 1985 for models and empirical evidence supporting the
decision rules for price used here). The competitor uses only its own costs, market
share, capacity, and backlog, along with subject's price, in making its price decision
(in scenario 1 the competitor utilizes cost information only). In fact, the subject knows
and can utilize far more information about the competitor and the market.
Experimental Design
The experiment was repeated six times over a three year period at a major US
management school. Most of the subjects were second-year full-time MBA students,
taking a System Dynamics for Business Policy elective. One group of subjects were
mid-career executives. Each subject played the game five times, providing the
equivalent of 50 years of simulated experience. Overall, there were 271 subjects
making a total of 1352 trials. After trials were eliminated due to incorrect sequences
of trials, repetition of trials etc., the dataset was reduced to 1119 trials for 253
subjects (226 MBAs and 27 Executives). The five game tasks were assigned as
homework to be done individually over two weeks. Subjects were given a full written
briefing guide (Sterman, 1992) together with an in-class demonstration of the
software. Subjects were allowed to take as long as they wished to make each of the
40 quarterly decisions for each game, and to suspend play between trials as
required. There was no time pressure (other than the overall due date).
The experimental design used a Graeco-Latin square, with five market scenarios
(M1-M5) and four competitor scenarios (C1-C4). The five market scenarios had
different replacement fraction (r) and word-of-mouth (w) factors according to Figure
1. The stronger the word of mouth, the faster the growth and sooner the saturation
of the market. The smaller the repurchase fraction, the lower the equilibrium
replacement demand and the steeper the bust when the market saturates.
Figure 1 Experimental Treatments - Five Market Scenarios M1-M5
(labeled A, B, C, D, E) and the Resulting Market Dynamics
Strength of Word of Mouth/
Base Case Value
05 | 1/2
05 | €E D
Repurchase
Fraction/ 1 A
Base Case Value
2 Cc B
(Million Units/Quarter)
0
Quarter
Note: Assumes no capacity constraints and constant margin pricing. Actual demand patterns also
depend on subject decisions.
Task Sequences
Five task sequences are used to complete the Latin Square with five trials. Each
sequence of tasks involves each one of the five market scenarios (M), and each one
of the four competitor scenarios (C) just once. The fifth competitor scenario is
chosen at random for each sequence. A total of 20 scenarios (five market scenarios
by four competitor strategies) are thus available, of which each subject plays just
five, according to the sequences shown in Table 1. For example, a subject playing
sequence 3 will face market scenario 3 and competitor scenario 1 in trial 1, followed
by market scenario 1 and competitor scenario 4 in trial 2. Hence, each subject plays
each of the five market scenarios just once, and plays each of the four competitor
scenarios at least once. The market scenario and competitor strategies used in
each trial were not revealed to the subjects.
Table 1 Five Sequences of Market Scenario, M (1-5) and Competitor
Scenario, C (1-4) used in 5 Trials. Notation M, C.
Cris a competitor scenario randomly selected from C1-C4.
Trial 1 Trial 2 Trial 3 Trial 4 Trial 5
Sequence | M2,C4 M4,C3 M3,Cr M1,C2 M5,C1
Sequence M4,Cr M3,C2 M1,C1 M4,C5 M2,C3
Sequence M3,C1 M1,C4 M5,C3 M2,Cr M4,C2
sequence M1,C3 M5,Cr M2,C2 M4,C1 M3,C4
Sequence M5,C2 M2,C1 M4,C4 M3,C3 M1,Cr
The design is reasonably well balanced, with an average of 45 subjects in each of
the 25 cells.
Benchmark Decision Rules
The potential cumulative net income over 40 quarters varies considerably for each of
the 20 scenarios, due to the variance in total industry sales generated by the
different market competitor scenarios. Hence a benchmark rule is needed to
compare subject performance in the 20 different scenarios. The subject's cumulative
net income for each game of 40 quarters is divided by the benchmark cumulative net
income for the particular scenario, giving a “performance relative to benchmark”.
Performance Relative to Benchmark = Subject cumulative net income
Benchmark cumulative net income
The functional form of the benchmark decision rules for Target Capacity and product
Price are formulated following examination of subject logs, and reference to the
literature:
Target Capacity = Target Market Share * Expected Industry Demand
#(1+£ xpected Demand Growth) ® * (Current Backlog/C apacity) ”
a, =0, a, 20
(1)
Price =Unit Costs * (1+Gross Margin) * (Current Backlog/Capacityy
B20
(2)
where Target Market Share =0.50 and Gross Margin =0.25
The target capacity rule seeks to capture 50% of expected demand, where demand
9
is forecast by extrapolating current industry sales at the current growth rate. Target
capacity is increased (decreased) relative to the demand forecast when capacity is
insufficient (excessive) relative to desired production.
The benchmark pricing decision rule assumes cost-plus pricing with a constant gross
margin and an adjustment for the demand/supply balance. Price simply follows costs
down the learning curve, with a markup sufficient to cover marketing expense,
investment costs, and provide a reasonable profit (at normal capacity utilization).
The behavioral benchmarks are simple, even naive, rules. They utilize only four
cues (costs, industry sales, backlog, and current capacity) rather than full
information. The rules naively extrapolate demand growth even though the subjects
know the product will go through a life-cycle of growth, saturation, and decline (the
benchmark rule’s forecast of demand is guaranteed to miss the peak in the market).
They do not use pricing to clear the market, control profitability, or signal intentions.
There is no game-theoretic reasoning. There is no explicit consideration of
investment costs, no anticipation of market saturation and no response to competitor
price or capacity, much less the competitor's strategy. The rules cannot learn. They
are much less sophisticated than the decision making typically presumed in
economic models and strategy texts. Subjects should be able to outperform the
benchmark performance quite easily.
The decision parameters a,, a, and B were chose to maximize average cumulative
net income per scenario over all 20 market and competitor scenarios, subject to the
constraint that cumulative net income is not negative (i.e., no losses) in any single
scenario. The global maximum of cumulative profits was generated by:a,=0.50,
@,=2.375, B=2.50, implying a modest response to growth in industry demand, and
strong adjustments of capacity and price to the demand/supply balance.
As expected, these parameters yield the lowest cumulative net income of $18.9m for
M=2, C=2 (the most rapid product lifecycle coupled with the most aggressive
competitor strategy). The highest is $1722m for M=4, C=4. The total cumulative net
income over 20 scenarios is $19,170m, the average per scenario being $959m. This
scenario generates the highest total cumulative net income over all 20 scenarios,
and hence is used for calculating the benchmark profits.
Subject Performance
Figure 2 compares subject performance to the benchmark in all five trials, and
against all four competitors. On average, performance is extremely poor.
Performance does improve over the five trials against competitors 1 and 2. Against
competitors 3 and 4, initial improvement for the first three trials then worsens for
trials 4 and 5. In all five trials, and against all four competitors, the mean subject
performance is below the benchmark performance.
Table 2 and Figure 3 show how the mean subject performance relative to the
benchmark (PRB) varies over the five trials, and against three competitor scenarios.
Competitor scenario 2 is omitted in Figure 3 because large outliers distort the scale.
10
Figure 2 Mean Subject Cumulative Net Income (P) in 5 Trials (T) against
4 Competitor Strategies (C)
Mean Subject Performance (P) over 5 Trials against 4 Competitors (C)
1500
—*— Competitor 1
—s=— Competitor 2
1000, —k— Competitor 3
—a— Competitor 4
500 —*— Benchmark
0
3 4 Ni
-500
1000
~1500
Trial T
A PRB=1.0 would mean the subjects achieve the same cumulative net income as the
benchmark decision rule. Similarly, PRB=0.5 means 50% of benchmark profit,
whereas -0.5 means a negative cumulative net income of magnitude equal to 50% of
the benchmark profit. Large negative values indicate the subjects sustained huge
losses. Though performance relative to the benchmark does vary from trial to trial,
the variation is not significant. Subjects do not learn to improve performance over the
five trials. Performance relative to the benchmark does depend on the particular
competitor scenario, with competitor 2 generating significantly worse scores than
competitors 1, 3 and 4. The mean subject performance relative to benchmark in all
five trials and against all four competitors, is below the benchmark score of 1.0.
Table 2, Summary Results for Mean Subject Performance Relative to
Benchmark (PRB): mean (standard deviation in parentheses).
Trial T
T=1 T=2 T=3 T=4 T=5 Overall
Competitor
Cc
C=1 -0.822 -0.809 0.443 0.745 0.669 -0.059
(8.00) (3.93) (0.533) (1.39) (0.554) (4.521)
Cc=2 -0.754 -4.5114 -16.199 -11.415 0.712 -6.241
(4.45) (11.5) (28.7) (23.1) (0.841) (17.9)
c=3 -0.801 -0.125 0.527 0.423 -1.767 -0.341
(7.56) (3.46) (0.712) (1.68) (11.9) (6.58)
c=4 -0.753 0.406 0.542 0.551 0.039 0.149
(3.67) (0.602) (0.495) (1.09) (2.48) (2.12)
Overall -0.784 -1.302 -3.142 -2.368 -0.156 -1.544
(6.25) (6.68) (15.0) (12.5) (6.41) (9.98)
11
The percentage of subjects who score above the benchmark performance (over all
trials) is 17.9%. This does vary across trials, from - 4.8%, 11.5%, 19.1%, 26.5% and
3.1% for trials 1-5 respectively. Clearly some subjects do appear to improve relative
to the benchmark performance, but the mean subject score for each trial is still well
below the benchmark. This percentage does not vary much against competitors -
16.2%, 19.2%, 17.4% and 18.2% for competitors 1-4 respectively.
Figure 3 Mean Subject Performance Relative to Benchmark as a Function of
Trial T (1-5) and Competitor Scenario C (1,3,4).
—*— Competitor 1
—4+— Competitor 3
2 —a- Competitor 4
—*— Benchmark
Trial T
Vertical Axis is PRB, Performance Relative to Benchmark
General Linear Model
To gain insight into the determinants of performance, we estimate a general linear
model for the dependent variable Performance Relative to Benchmark (PRB), with
independent variables Subject (SUB), Trial (T), Competitor (C), the log or the Word-
of-Mouth Effect (logw), and the log of the Replacement Fraction (logr). Principal
interactions of the main effects are also included. The additive error term e is
assumed to be a normally distributed random variable with zero mean. The linear
model we are estimating is:
PRB =constant +SUB +T +C +logw +logr + logw*logr +C*T +T*logw
+T*logr +C*logw + C*logr + C*logw*logr +€
The results of the analysis of variance are shown in Table 3.
The competitor scenario, and the determinants of the speed and severity of the
product lifecycle (the replacement fraction, strength of word-of-mouth, and their
interaction) are highly significant. Further, the interactions of competitor strategy
with the determinants of the product lifecycle are also highly significant. The
magnitude and sign of the WOM and replacement fraction are as expected. The
stronger the word of mouth, the faster the growth of demand, higher the peak, and
sharper the transition to decline. Subjects have more difficulty when the positive
feedback driving growth is strong. Similarly, the smaller the replacement fraction,
12
the worse the subject performance, since a small replacement purchase rate means
a greater demand decline from peak to equilibrium. The worst performance arises
when rapid growth is combined with a highly durable product. The greater the
dynamic complexity of the market, the worse is subject performance relative to
potential.
Table 3. Analysis of Variance for Dependent Variable PRB
R?=51%
Source DF Seq SS AdjSS AdjMS_ F P
Sub 253 24743 18359 72 1.11 0.137
T 4 1048 280 70 1.08 0.367
Cc 3 7380 1646 548 8.43 0.000
logw 1 3807 687 687 10.56 0.001
logr 1 4452 1035 1035 15.89 0.000
TC 12 6598 413 34 0.53 0.897
logw*logr 1 1130 776 776 11.92 0.001
THogw 4 574 427 106 1.64 0.161
THogr 4 2174 263 65 1.01 0.401
C*logw 3 1693 430 143 2.20 0.086
C#logr 3 1410 1793 597 9.18 0.000
CHogwlogr | 3 2648 2648 882 13.56 0.000
Error 826 53791 53791 65
Total 111 111454
The coefficients of the significant main effects are:
Constant -1.2239
G1 1.4523 Competitor strategy “cost-plus”
C2 -3.8061 Competitor strategy “share-oriented”
C3 0.6996 Competitor strategy “margin-oriented”
C4 1.65 Competitor strategy “tit-for-tat”
logw -1.5765 Word-of-mouth (WOM) effect (speed of growth)
logr 1.7407 Replacement fraction (severity of bust)
logr*logw 1.3306 Interaction of WOM and Replacement Fraction
Competitor strategy 2 generates by far the worse performance relative to benchmark
(coefficient of C2 is -3.8) and is what we expect given the strategy of competitor 2. In
fact, Bonferroni post-hoc tests show that only C2 is significantly different from the
other competitor scenarios C1, C3, and C4.
We also tested for a “sequence effect” to see if the particular order in which the
scenarios are being completed contributes significantly to performance. The trial
sequence is, in fact, significant but post-hoc tests reveal that only sequence four is
significantly worse than the others. This sequence included market scenario two and
competitor scenario two which in combination are particularly difficult for the
subjects. Sequence is therefore not included in subsequent statistical models.
13
Analysis of Subjects’ Decision Rules
The general linear model and ANOVA show that the greater the dynamic complexity
of the market and of the competitor strategy, the worse the subjects do relative to
potential. Next we explore the nature of the decision rules subjects used to see if we
can identify the sources of their poor performance and gain insight into the mental
models they bring to the task. Behavioral decision rules are estimated for target
capacity and price. These rules are generated from analysis of subjects’ logs and
written reports of their strategies. The rules we tested represent just two of many
possible rules. But they do reveal how the weights on the cues that subjects use to
make their decisions vary over trials and treatments (i.e., market scenario M and
competitor scenario C).
A more complete description of the form of these decision rules is given in Paich and
Sterman 1993, pp. 1450-1451. The postulated rules generalize the benchmark
decision rules. We postulate that subjects select the share of the market they seek
to capture, estimate future market demand from prior information, current demand,
and recent demand growth, and invest to balance capacity with demand.
Specifically:
C,=S'[D,"D,,)1(1 +g)" (B/C),
Ger =(D,. - D.2) / D.,
. (3)
where S is target market share (assumed constant), Dg is the prior expectation of
average industry demand, D is actual demand, gt is the expected fractional growth
rate of demand, B is the backlog (desired production), and C is current capacity.
The proposed decision rule for price P assumes subjects use markup pricing:
P,=(UPC,) (M’),
M’,=M, (UPC,/UPC,)® (B/C)® (CP. / Pas)”
(4)
where UPC = unit product cost (fixed plus variable) and M’ = gross margin. Gross
margin depends on the subject's response to the demand/supply balance and the
policy for passing cost reductions on to the consumer. As the firm moves down the
learning curve, the subject must decide how much of the cost reduction to pass on to
consumers. All cost reductions are passed into price when B =0, while -1 <= RB) <=
0 indicates price falls less than costs. Positive values of Bi indicate price falls faster
than costs, perhaps indicating an attempt to build market share and move more
rapidly down the learning curve than the competitor. We further expect that the gross
margin will increase when backlog is high relative to capacity (B2 > 0).
Re-arranging equations 3 and 4 and taking logs gives the following equations for
estimation by regression:
log(C*) = ao tailog(D,,) + arlog(1+9+-1) + aslog(BYC:) + &
log(P) =bo +bilog(UPC,) + bslog(ByC,) + bslog(C Pu1)+ &
14
(6)
Each rule was estimated separately for all the 22 Executive subjects, and for a
similarly sized random sample of 24 MBA subjects. The mean and variance of the
performance relative to benchmark for the random sample of MBA subjects (mean=
2.49, s.d=12.2) are within 95% confidence limits of the population mean and
variance (mean=1.55, s.d.=10.23), and hence the sample is considered to be
representative. The error terms are serially correlated, so the Cochrane-Orcutt
procedure for first-order autocorrelation was used. Table 6 shows the regression
coefficients mean and standard deviations.
Table 6 Mean and Standard Deviations of Estimated Parameters for Subjects’
Capacity and Pricing Rules (Equations 5 and 6)
MBA (N=91) Exec (N=54)
Subjects subject
s
Capacity Mean StdDev %Sig Mean Std.Dev %Si
Rule: g
ao 8.399 5.107 73% 8.975 7.480 72%
a 0.374 0.348 62% 0.329 0.490 57%
a 0.316 0.949 39% 0.195 0.683 22%
a 0.214 0.436 53% 0.064 0.401 22%
fest 0.570 0.330 71% 0.634 0.255 91%
R2 0.821 0.192 0.828 0.156
Pricing
Rule:
bo 1.343 9.375 43% -0.335 10.419 48%
bi 1.038 4.260 52% 1.191 2.940 63%
bz -0.106 0.391 51% -0.098 0.112 67%
bs 0.0684 0.470 51% 0.137 0.361 39%
ford 0.0631 0.392 78% 0.309 0.270 87%
R2 0.876 0.131 0.902 0.131
(i and p» are the first-order autoregressive coefficients.
“%Sig” column indicates the percentage of trials in which the parameter was significantly different
from zero.
The estimated parameters for the MBAs and executives are very similar. The
greatest difference is in the mean estimate for a,, (0.214 for MBAs vs. 0.064 for the
executives), indicating the executives paid less attention to the supply/demand
balance than MBAs (only 22% of the estimated values for the executive group were
significantly different from zero, compared to more than half for the MBAs).
However, the difference in the mean estimate of a, between the two groups is not
statistically significant.
The coefficient b3 in the pricing rule is very small (0.0684 for MBAs and 0.137 for
executives) with less than half significantly different from zero (47% MBAs, 39%
Executives). Surprisingly, subjects appear to pay little attention to the competitor
15
price when formulating their own pricing decision. This seems unlikely, apart from
the case of Competitor 1 (cost-plus pricing). Eliminating Competitor 1 from the
analysis, the parameter b3 mean (s.d) is 0.247 (0.293) for MBAs, and 0.257 (0.198)
for Executives. These values are much higher than those estimated with Competitor
1 included, and show that more attention is paid to the Competitor price when it
varies.
Next, we investigate how subject decision weights change with experience and
across market environments by estimating linear models for each estimated
parameter in turn (ao, a1, a2, a3, Do, bi, b2, bs) with subject (Sub), trial (T), word-of-
mouth effect (logw), replacement fraction (logr) and competitor scenario (C)
treatments as explanatory variables (equation 7). EXEC is a dummy variable to
distinguish between MBA and Executive subjects.
Estimated parameter = constant +Sub +EXEC +C +T +logw t+logr+e
(7)
Most of the estimated parameters do not change significantly with experience (trial),
or as the market environment (word of mouth, replacement fraction and competitor
strategies) change. The estimation of parameter b3; (governing the response to
competitor price in the pricing rule) does depend significantly on the competitor
strategy C, with an overall R2 of 47%. The competitor scenario effect has coefficients
(significantly different from zero) for C1=0.48, C2=0.10, C3=0.18 and C4=0.26. The
weight that subjects place on competitor price does vary significantly with the
particular competitor scenario, but the adjustments are small. There is no clear
pattern to the coefficients, and in estimating six terms in eight models we would
expect at least one effect to be significant at the 5% level just as a matter of chance.
Overall, the results show that subjects’ decision rules are not sensitive to the
dynamic complexity of the market or the competitor strategy. More important, there
is no evidence of learning from experience: subjects do not modify their strategies
over time.
Discussion
The results show, first, that performance relative to potential is poor on average.
Second, performance relative to potential is significantly worse in dynamically complex
environments. Third, the decision rules of the subjects are not responsive to the
market environment or competitor strategy. Fourth, in general, subjects are not
sufficiently responsive to the demand supply balance, either in adjusting their capacity
or in adjusting prices. Finally, there is little evidence that subjects learned from
experience, and no evidence that they learned from their experience how to do better in
dynamically complex environments. These are sobering results, but consistent with
prior work (Paich and Sterman 1993, Sterman 1994).
The pricing behavior of the subjects warrants special consideration. Recall from the
analysis of the benchmark decision rule that the optimal value of b,, the coefficient that
relates the demand/supply balance to price, is 2.5, which implies that a 1% increase in
the demand/supply balance causes a 2.5% increase in price. The average estimated
value of b, for both the MBAs and the executives was -0.1; over 75% of the estimated
coefficients were negative. A negative value for b, implies that the subjects reduced
16
price in response to lower product availability, exactly the opposite of the optimal
response. Those estimates of b, that were positive were generally much smaller than
the optimal value.
Apparently, the subjects in the experiment, focused on maintaining market share,
reason that they must offset the reduction in product attractiveness caused by long
delivery times by lowering prices. However, lower prices, by increasing product
attractiveness, actually worsens the delivery situation. The result is a positive feedback,
a vicious cycle of deteriorating customer service, lower prices, and still worse service.
Furthermore, since in the real world customers have heterogeneous tastes, the effect
of lower prices and long delivery times is to systematically drive away those customers
who are delivery sensitive and price insensitive while attracting those customers who,
for reasons of limited funds, are willing to wait longer to save a few dollars.
Consequently, a firm that cuts price to maintain attractiveness as service deteriorates
will find its customer mix changing to favor price sensitive bargain hunters. That is, the
firm systematically increases the elasticity of its demand curve, increasing its
vulnerability to price competition at the same time they invite a price war.
The decision to lower price when product availability is poor has been observed in
many real situations. For example, Apple Computer suffered severe shortages of
innovative new products (Wall Street J ournal August 11, 1995). The backlog of unfilled
orders grew substantially and market share declined. However, Apple reduced price by
as much as 40% on some of the items that were in short supply. In addition, IBM has
made similar decisions in the PC market.
As another example, People Express Airlines fell victim to just this dynamic. Initially,
People Express offered low fares and good service. But its very low fares led to rapid
growth, causing staff shortages, rapid hiring, a decline in employee experience, and
other operational problems resulting in a sharp decline in customer service.
Consequently, the customer mix shifted from both business and discretionary (student,
vacationer) passengers in the early days to a customer base almost exclusively made
up of price-sensitive discretionary travelers. When competitors then cut prices to match
People’s fares, People lost the only remaining dimension of product attractiveness in
which it had an advantage and was soon forced to the brink of bankruptcy and a forced
sale to Frank Lorenzo's Texas International (Sterman 1988, Graham et al 1992).
The MOF hypothesis can account for the significant difference between subjects’
responses to supply/demand imbalances and the optimal response. The MOF holds
that in determining their choices decision makers ignore important feedback loops. In
the model, an important feedback loop links product availability, market share, and
orders. As shown in Figure 4, an increase in new orders increases the order backlog
and reduces product availability. Lower availability reduces market share and orders.
This causal chain creates a goal seeking feedback loop that balances supply and
demand.
17
Figure 4 Feedback Structure of Simulated Model
Market Share Production Capacity
+
+ -
a Order Backlog
New Orders
(~ XN B
Price Product Availability
The causal link between orders and product availability makes it optimal to raise price
when availability falls. Assume that orders unexpectedly increase. Because capacity
acquisition takes time, additional orders reduce product availability and market share.
By itself, raising price would further reduce market share and orders. However, the
reduction in orders caused by the higher price improves availability and pushes share
back up. The higher price reduces orders to some degree, but not enough to offset the
revenue gain from a higher price per unit. Consequently, raising price in response to
lower availability significantly increases profit.
The MOF holds that decision makers could find it difficult to determine the policy
implications of the feedback structure described in Figure 4. In this situation, the
decision maker might use a simpler mental model of the situation that ignores the goal
seeking loop. An example of a simplified mental model is shown in Figure 5. In the
simplified mental model, product availability is not connected to the order backlog and
does not depend on past pricing decisions.
Figure 5 Simplified Feedback Loop Structure of Simulated Model
Market Share
"
New Orders
+
Price Product Availability
The simplified mental model shown in Figure 5 has very different policy implications
than the mental model shown in Figure 4. Assume that market share is determined by
the same equation as in the full simulation and that the decision maker maximizes profit
18
for a single period. For the model in Figure 5, it can be shown that the optimal response
to lower product availability is always to lower price.
In other words, a decision maker who used the simplified mental model would decide to
lower price in response to lower availability. In addition, for the specific parameters in
the simulation model, the optimal value of the parameter b, is about -0.12 which is very
close to the average estimated value for the subjects. The stated rationale for reducing
price in response to poorer availability could be something like the following. Lower
availability has reduced market share and, in order to compensate, price should be
reduced. The lower price will induce customers to wait until the product is available,
increase orders, and improve profitability. A price increase would "punish the customer"
and ultimately reduce profitability.
One explanation for the subjects’ pricing behavior is that they found an excellent
solution to the wrong problem. The subjects’ pricing strategy works well in the simplified
environment described in Figure 5. The same strategy performs poorly in the actual
environment described in Figure 4. A mental model that ignores the feedback
connection between price and future availability generates decisions that are the
opposite of the correct decisions.
In essence, if the system were in equilibrium (if capacity adjusted rapidly to demand),
delivery delay would be independent of price, and a drop in market share could be
offset by lower prices. But in the simulation as in the real world, capacity adjustment
takes time, the firm is often in disequilibrium, and price is strongly couples to delivery
delay. The subjects appear to assume implicitly that the market and firm are in
equilibrium, despite the fact that evidence to the contrary was available to them at all
times during the experiment. We conjecture that the emphasis on equilibrium and
comparative statics in economic theory and course work may contribute to this error.
Our study did not collect concurrent verbal protocols or other real-time metrics of
people's reasoning, so we cannot prove that they used a simplified mental model like
the one described in Figure 5. We can say that a hypothetical decision maker who
used the simplified mental model and attempted to maximize profit would have
generated price decisions that were consistent with the subjects' actual price decisions.
However, the subjects' actual decisions could have resulted from some other mental
model. Follow up study would be required to answer the question definitively.
It is possible that factors not included in the model could make it optimal for real-world
firms (computer manufacturers, for example) to reduce price in the face of product
shortages. Price cuts could be necessary to maintain the loyalty of the dealer network
or to promote an image of price competitiveness or fairness. Many real markets involve
more competitors than our experiment, so the scope for collusive behavior is less. It is
not obvious to what extent these factors would change the optimal response to
supply/demand imbalance. Given the importance of pricing for both short-term and
long-term profitability, this is an important topic for future research.
19
References
Arthur, W. (1994) Increasing Returns and Path Dependence in the Economy. Ann Arbor MI: University of
Michigan Press.
Bass, F. M. (1969) A New Product Growth Model for Consumer Durables. Management Science. 15, 215-227.
Brehmer, B. (1990) Strategies in Real Time, Dynamic Decision Making, in Hogarth, R. (ed) Insights in Decision
Making. Chicago: University of Chicago Press, 262-279.
Brehmer, B. (1992) Dynamic Decision Making: Human Control of Complex Systems. Acta Psychologica 81, 211-
241.
Diehl, E. (1992) Effects of Feedback Structure on Dynamic Decision Making. PhD. Dissertation, MIT Sloan
School of Management.
Funke, J. (1991) Solving Complex Problems: Exploration and Control of Complex Systems, in R. Sternberg and P.
Frensch (eds.), Complex Problem Solving: Principles and Mechanisms. Hillsdale, NJ: Lawrence Ebaum
Associates.
Graham, A. K., Morecroft, J. D., Senge, P. M., & Sterman, J. D. (1992) Model Supported Case Studies for
Management Education. European J ournal of Operational Research, 59(1), 151-166.
Hauthakker, H. S., & Taylor, L. C. (1970) Consumer demand in the United States. Cambridge, MA: Harvard
University Press.
Homer, J. (1987) A Diffusion Model With A pplication to Evolving Medical Technologies. Technological
Forecasting and Social Change, 31, 197-218.
Kalish, S. and Lilien, G. (1986) Market Entry Timing Entry for New Technologies, Management Science 32(2) 194-
204.
Kleinmuntz, D., and J. Thomas (1987) The value of action and inference in dynamic decision making,
Organizational Behavior and Human Decision Processes, 39(3), 341-364.
Mahajan, V., E. Muller, and F. Bass (1990) New Product Diffusion Models in Marketing: A Review and Directions
for Research. Journal of Marketing 54(1), 1-26.
Mahajan, V. and Y. Wind, eds. (1986) Innovation Diffusion Models of New Product Acceptance. Cambridge, MA:
Ballinger.
Moore, G., Inside the Tornado, 1996.
Morecroft, J. (1985), Rationality in the analysis of behavioral simulation models, Management Science 31(7), 900-
916.
Paich, M. and J.D. Sterman, “Boom, Bust, and Failures to Leam in Experimental Markets”, Management Science,
39, 12 (1993), 1439-1458.
Senge, P. (1980) A System Dynamics A pproach to Investment Function Formulation and Testing, Socioeconomic
Planning Sciences, 14, 269-80.
Smith, V., Suchanek, G., and A. Williams (1988) Bubbles, Crashes, and Endogenous Expectations in Experimental
Spot Asset Markets, Econometrica, 56(5), 1119-1152.
Sterman, J. D. (1987a) Testing Behavioral Simulation Models by Direct Experiment. Management Science, 33(12),
1572-1592.
Sterman, J. D. (1987b) Expectation Formation in Behavioral Simulation Models. Behavioral Science, 32, 190-211.
Sterman, J. D. (1988), People Express Management Flight Simulator. Simulation Game (software), Briefing Book,
and Simulator Guide, Available from author, MIT Sloan School of Management, Cambridge, MA 02139.
Sterman, J. D. (1989a) Misperceptions of Feedback in Dynamic Decision Making. Organizational Behavior and
Human Decision Processes, 43(3), 301-335.
Sterman, J. D. (1989b) Modeling Managerial Behavior: Misperceptions of Feedback in a Dynamic Decision Making
Experiment. Management Science, 35(3), 321-339.
Sterman, J. D. 1991. B & B Enterprises Management Flight Simulator. Software, Briefing Book, and Instructor's
Guide. Available from author, Sloan School of Management, MIT E53-351, Cambridge MA 02142.
Sterman, John D., R. Henderson, E. D. Beinhocker and L.I. Newman, “A Behavioral Analysis of Learning Curve
Strategy”, Working paper D-4354, MIT Sloan School, System Dynamics Group, 1995.
20
CC BY-NC-SA 4.0