Informing Mental Models for Strategic Decision Making
with ERPs and the Balanced Scorecard:
A Simulation-Based Experiment
James L. Ritchie-Dunham
Doctoral Candidate
University of Texas at Austin, MSIS
3615 Aspen Creek Parkway
Austin, Texas 78749
USA
(512) 925-9824 Cell
(512) 291-0658 Fax
jimrd@ strategic-clarity.com
http://www.strategic-clarity.com/
Abstract
To make efficient, effective decisions, organizational leaders use strategic frameworks to see
how well the organization is accomplishing its goals and implementing its strategies. These
strategic frameworks filter great quantities of information about different parts of the
organization into a few, critical elements. To process this vast amount of information, these
frameworks depend increasingly on sophisticated information systems. Prior behavioral decision
making research supports this practice, suggesting that guiding cue selection and increasing
information credibility enhance decision performance. Common practice, however, focuses
primarily on increasing information credibility and not enhancing cues. This research develops a
theoretical model to explain how the decision maker’s understanding of the decision context is
influenced by the strategic framework and by the information processing system. This research
tests the theoretical model with a simulator-based experiment, where 118 MBA and MEng
students run a wireless telecommunications firm.
Key words: experiment, mental model, ERPs, balanced scorecard, simulation
Introduction
We inform decision makers to help them make efficient, effective decisions - decisions that lead
to positive results. In this direction, decision makers use strategic frameworks, which inform
them about how well the organization is accomplishing its goals and implementing its strategies.
These frameworks guide the decision maker by making explicit the key strategic elements and
how they relate. For example, the strategic framework economic value added (EVA) focuses the
decision maker on shareholder value creation (the goal) through its components of revenues,
operating expenses, cash taxes, and capital charges (the implementation). To inform the decision
maker about what is happening in the real world of the organization, these strategic frameworks
filter great quantities of information about different parts of the organization into a few, critical
elements These critical elements tend to focus on the organization’ s stated goals and actions (i.e.,
shareholder value creation). To process this vast amount of information, for effective and
efficient decision making, these frameworks depend increasingly on sophisticated information
systems, such as SAP’s Strategic Enterprise Management module (SAP, 2000).! These modules
use best practices in strategic management to select the information that best represents the status
of the entire organization (for examples using economic value added and the balanced scorecard,
see PeopleSoft, 2000; SAP, 1999).
Belief in the potential benefit is evidenced by the heavy investment in these systems. In 1997,
over 20,000 companies around the world spent over $10 billion on these sophisticated enterprise
systems (Martin, 1998). By 1998, over 40% of companies with revenues exceeding $1 billion
had begun to use them (Caldwell and Stein, 1998). Even with such intense investment in
information, research indicates that firms currently use them primarily to replace previous
information collection activities, and very few use them to manage the business more efficiently
(Davenport, 2000; Vandenbosch and Higgins, 1996). Whether these systems lead to increased
decision effectiveness or efficiency is still under debate (Sharda et al., 1988; Todd and Benbasat,
1999). So, though the solution seems obvious - buy a big information system with a great
framework - the value that selecting specific strategic information adds to decision making is not
clear to or understood by many decision makers. These decision makers often lack the
experience (empirical base) from which to evaluate the value added and the framework
(theoretical base) with which to communicate it.
Experimental data on the value these information systems provide would increase the ability of
leaders to appraise them. A theoretical model explaining the experimental data would provide
the framework with which to communicate the value added. To increase credibility, the
experiment should include realistic strategic decisions. To increase communicability, the
theoretical model should be simple and complete.
This research develops a theoretical model to explain how the decision maker’ s understanding of
the decision context is influenced by the strategic framework and by the information processing
system. This research tests the theoretical model with a simulator-based experiment, where
graduate students run a wireless telecommunications firm. This paper presents the results.
Influencers of Strategic Mental M odel Similarity
For this research, the unit of analysis is the decision maker. The context is strategic resource
allocation at the top of a large organization. In this setting, the human-computer interaction
literature describes five elements related to informing the decision maker about the decision
context (Norman, 1983):
1 An Andersen Consulting 1999 study (Andersen, 1999) surveyed 200 CEOs of companies that had implemented an
integrated information system, also referred to as ERPs (Davenport, 2000). This study reports that the top three
reasons why CEOs invested in an integrated information system were (% that stated this): (1) to improve
information accuracy and availability (67%); (2) to improve management decision making (61%); (3) to reduce
cost/improve efficiency (51%).
the target system
the conceptual model of the target system
the system image
the user's mental model of the target system
the scientist’ s conceptualization of the mental model.
eda a
The target system is the organizational decision context. The conceptual model of the target
system is a teaching tool created by the decision maker's advisors or teachers as an accurate,
consistent, and complete representation of the target system. The system image is the image of
the target system that is seen by the decision maker - it is the strategic framework filled with
data that the decision maker sees. The system image may or may not be consistent with the
conceptual model of the target system. The user’s mental model of the target system is created
as the decision maker interacts with the target system or system image (Craik, 1943). The user's
mental model might not be equivalent to the conceptual model. Finally, the researcher extracts a
conceptual model of the subject’s mental model (Staggers and Norcio, 1993). A model, as used
here, is defined as a representation of structure (cf Oxford English Dictionary), focusing on the
component parts (elements) and how they interrelate (relations) to make the structure function.
The strategic framework and supporting information systems exist to inform the decision maker
about how well the organization implements and achieves its goals. Therefore, the intent is for
the decision maker’s mental model to accurately match the organization’s decision context. In
that sense, the theoretical model in this research (see Figure 1) proposes that System Image
Structural Similarity (SISS) measures how well the system image structure correlates with the
target system, approximating the integrity of the system image. Structural similarity is measured
in this research as the proportion of structural elements and relations shared between the
information selected and that of an expert map of the decision context (Mathieu et al., 2000;
Schvaneveldt, 1990). In the strategic setting, SISS measures how well the strategic framework
(system image) describes the organizational decision context (target system).
System Image Mental Model
Structural Similarity Similarity. “
System Image
Behavioral Similarity
Firm Value
Figure 1; The Research Model
Mental Model Similarity measures how well the user's mental model of the target system
correlates with the target system (Mathieu et al., 2000). In the strategic setting, this captures how
well the decision maker s understanding (user’s mental model) correlates with the organizational
decision context (target system). Mental models are commonly used in the behavioral literature,
and most often described in terms of their elements and their interrelationships (Borgman, 1999;
Craik, 1943; Doyle and Ford, 1998; Johnson-Laird, 1983; Mathieu et al., 2000; Staggers and
Norcio, 1993; Stout et al., 1999). Similarity is measured, in this research, as the proportion of
elements and relations shared between the decision maker’s mental model and that of an expert
map of the decision context (Carley and Palmquist, 1992; Rowe and Cooke, 1995).
System Image Behavior Similarity (SIBS) measures how the information in the System Image
behaves compared to the target system. SIBS measures the accuracy, timeliness, and consistency
of the information, provided by the information system from the target system, about the
effectiveness and efficiency of the decision maker's previous decisions. Behavioral research in
dynamic decision making suggests these three attributes greatly influence decision performance
(Einhom and Hogarth, 1986; Kleinmuntz, 1993; Sterman, 1994). Previous research across many
other disciplines has also identified these attributes as key attributes in information processing
(Barua, Kriebel, and Mukhopadhyay, 1989; Davenport, 2000; Feltham, 1968; Fuerst and
Cheney, 1982; Huang, Lee, and Wang, 1999; Zmud, 1978). Similarity is measured, in this
research, as the degree to which a specific information system matches the best-in-class status
for each attribute. Firm Value measures the economic value created by the firm.
The relationships among the variables in the theoretical model have been well studied in the
behavioral decision making literature. Research in social judgment theory has studied how
selecting more predictive cues (elements and relations) from the background (target system)
influences judgment accuracy, which is related to MMS (Hammond, 1955; Hursch, Hammond,
and Hursch, 1964). This research hypothesizes that SISS positively influences Mental Model
Similarity. This also finds support in the decision support system literature on decision aids
(Cats-Baril and Huber, 1987; Kleinmuntz and Schkade, 1993; Silver, 1990; Todd and Benbasat,
1999).
Behavioral research in dynamic decision making has found that people are very susceptible to
noisy feedback, leading to much worse than nd ve behavior (Einhom and Hogarth, 1986;
Kleinmuntz, 1993; Moxnes, 1998; Paich and Sterman, 1993; Sterman, 1989). These studies
suggest SIBS positively influences Mental Model Similarity. Empirical decision support research
shows that as the information accuracy, timeliness, and consistency degrade, decision maker
confidence in the information falls, thus they are less likely to use it (Fuerst and Cheney, 1982).
Behavioral and strategy research link Mental Model Similarity with improved performance on
multiple organizational objectives (Bantel, 1993; Bourgeois, 1980; Mathieu et al., 2000; Porter,
1980; Snow and Hrebiniak, 1980). These studies suggest Mental Model Similarity positively
influences Firm Value.
This research contributes to previous behavioral research by providing an integrated theoretical
model and new, more rigorous measurement tools. Previous behavioral research provided
rigorous analysis of probable decision rules used by subjects, inferring the mental model that
generated it. This paper provides a tool to measure that mental model, which now permits
research into whether the scientist’s conceptualization of the user's mental model was able to
generate the observed behavior.
This research also explains some apparent inconsistencies in the decision support system
literature on the effect of SIBS on decision making performance (Sharda et al., 1988; Todd and
Benbasat, 1999). Most of the research manipulated SIBS and not SISS. For tactical decisions,
which are well structured, increasing SIBS improves performance, because the target system is
well understood and easy to describe (well-structured). For strategic decisions, which are ill
structured, increasing SIBS may not improve performance, because the target system is not well
understood and is difficult to describe (ill-structured).
Finally, this research contributes to the mental model literature by applying the methods and
tools to more complex, social systems. Early mental model research focused on physical
systems, like calculators. Recent research has delved into team mental models of jet fighters,
still focusing on managing physical systems.
While the research cited above examines pieces of the research model, often with simple tasks,
this paper contributes a theoretical model that integrates the pieces and tests the theory with a
task that more closely reflects strategic resource allocation.
Research Questions
In summary, this research addresses the questions:
¢ Is strategic mental model similarity influenced by system image structural similarity and
system image behavioral similarity?
* Does strategic mental model similarity influence decision performance?
In practical terms, these questions ask whether current, best-practice strategic frameworks and
more accurate, timely, consistent information improve the decision maker’ s understanding of the
decision context.
The Task: Strategic Resource Allocation for a Wireless
Telecommunications Firm
The experimental task is to make strategic resource allocation decisions for a simulated wireless
telecommunications firm every six months over a seven-year period. To test the theory
presented above, the experiment will divide the subjects into four groups, altering system image
structural similarity (low and high) and system image behavioral similarity (low and high). The
subjects will run the firm under these different treatments in an interactive game. This
interactive game or “management flight simulator” (for examples, see Moxnes, 1998; Paich and
Sterman, 1993; Sterman, 1989) represents a model of a firm, its customers, and its competition.
Subjects manage the firm through a period of strong growth in the market, replicating the current
difficulties faced by wireless telecommunication firms (Shepard, 2000).
The model was developed with experts in wireless telecommunications, enterprise systems, and
the balanced scorecard, including senior managers in three wireless telecommunications firms,
academics, and strategy consultants. The model was validated using expert knowledge and data
from two firms in the industry. The model parameters were then scaled to protect the
confidentiality of the two firms.”
2 The elements in the model include 69 operational variables and 54 financial variables. Of the relationships, 179
were informational flows and 31 were material flows.
The model encompasses external and internal components of the supply chain, as seen in Figure
2. External components include the suppliers of parts for the switching equipment and the
customers. Internal components include human resources, technology, service-supporting
information technology, and financial capital.
Financials
Customer Base
+ Economic Value
Added
* Cash Flow
+ Economic Capital
+ Expenses
+ Financing
Revenues
+ Market Share %
+ Population
Retention Rate
+ Fim Customers
Operating Costs [
‘Customer Satisfaction
+ Customer Disposable Income
* Competitor Customer
Satisfaction
+ Fim Price
* Utility of Perceived Call
Quality and Customer Service
Technology Network Coverage
Network Quality
>
Investment in
Base Stations
+ Base Stations
Technology Obsolescence
—__v
Human Resources
HR Service
Index
Training >). Obsolete Skilled People
+ Relevant Skilled People IT Facilitation
Index
SsIT $ Service Supporting IT
ef er Retiring Rate
* Service Supporting
Information Technology
Figure 2: Overview of the Simulator Model
External Components
In the Technology component, the firm orders new base stations to be installed by external
contractors. The contractors respond based on their available capacity, which is a function of the
size of the overall market. On the other end of the supply chain, in the Customer Satisfaction
component, the firm provides services to customers for a fee. The customer chooses to continue
with the firm, if satisfied with the service provided for the price paid. Satisfaction is measured
relative to both the competitive offering in the marketplace and the customer's disposable
income dedicated to wireless telephone services. Satisfaction is measured as a utility function of
the price paid for a perceived level of call quality and customer service. Each of these satisfiers
is determined by the model endogenously, as explained in the following section. Demand for
wireless services is assumed to increase linearly over the seven-year simulation period, with the
customer's average disposable income for telecommunication services remaining constant. In
the Customer Base component, relative Customer Satisfaction affects movement of customers
between the firm and the competition.
Internal Components
Following the logic of the internal value chain, in the Technology component, the technicians
maintain the base stations and switching infrastructure. The Human Resources component
describes the dynamic of human resource skills as employees are hired or fired, and as they are
trained or their skills are obsoleted by changing technologies. The Service Supporting
Information Technology component depicts the dynamics of the information systems that
support the call centers. The Financials component calculates the financial statements based on
the status and flow of resources in the firm. The Investment Decisions component translates the
subject’s periodic capital allocation decisions on a continuous basis. The model permits hiring,
firing, and training people, building new hase stations, and buying new information technology.
See Appendix A for the logic explaining these decisions.
Method
The subjects were 118 students in the University of Texas at Austin MBA and M.Eng. programs,
taking a first-year course in Operations Management. Attendance was given as a homework
assignment. The students had an average five years of work experience, ranging from one to
thirteen years.
The experiment is a 2x2 design. The two independent variables are system image structural
similarity (low or high) and system image behavioral similarity (low or high). This creates four
cells, to which the subjects were randomly assigned.
After answering four questions about previous education and experience, the subjects read a
written introduction to the case and software. This introduction corresponds to the “conceptual
model of the target system,” described earlier in this paper.
The subjects ran the simulator twice. Each time, they ran it over a seven-year period, reviewing
performance and allocating strategic resources every six months, for a total of fourteen decision
points. The subjects were instructed to run through the first simulation quickly to leam how it
works, taking approximately 15 minutes. The subjects then ran the simulator a second time, for
which the results were captured, allowing approximately 40 minutes. After the second run, the
subjects were asked to describe their understanding of the structural elements and relationships,
as described below. Allowing for 5 minutes of set-up time and 5 minutes to save the results, the
total time elapsed allowed was 2 hours and 15 minutes.
The subjects were given the same initial resource conditions, the same overall firm structure, and
the same competitive environment. The only differences in the four treatments were the
operationalization of the two independent variables in the interface, as described below. The
interface formalizes the subject’s decisionmaking process, as this is the only information they
receive.
The Independent Variables
Low SIBS was operationalized in the interface with random noise (accuracy) applied to delayed
(timeliness) information. Information from different parts of the firm’s supply chain have
different noise and delays, creating inconsistent information (consistency) in the interface.
Low SISS was operationalized in the interface as a linear combination of the financial
components driving profitability. High SISS was operationalized in the interface by grouping the
cause-effect linkages of three key leading indicators of financial performance (customers,
internal processes, and learning and growth). See Appendix B for the user interface screen for
each treatment. For the analysis, SIBS and SISS were operationalized as depicted in Table 1.
Treatment | Treatment Description SIBS SISS
1 Low SIBS with Low SISS 1 1
2 Low SIBS with High SISS 1 -l
3 High SIBS with Low SISS -1 1
4 High SIBS with High SISS -1 -1
Table 1: Operationalization of Independent Variables
Results
Of the 118 subjects, the results for four subjects were not properly saved after the simulation run,
so they were not included in the analysis. The 114 remaining were evenly distributed across the
four treatments (29 in #1, 27 in #2, 29 in #8, 29 in #4).
The Dependent Variables
The subjects in the experiment develop a mental model of the simulated telecommunications
firm. By providing a simulated model of the firm, this experiment provides a known entity, thus
the target system is comprehensively described. To capture the subject’s mental model, the
dependent variable Mental Model Similarity was operationalized with measures of the subject's
mental model. The measures were calculated from the mental model elicited from the subjects.
To elicit this, the subjects followed a laddering process, where they were asked to describe their
understanding of the following structural elements and relationships (see Carley and Palmquist,
1992 for an example of this process):
¢ Structural Elements
0 Objective function
"The firm’s goal/purpose
o Constraints
= Relevant stakeholders
= Stakeholder goals
* Value-driving resources
= Actions
¢ Relationships
o Between constraints and objective function
o Between individual constraints
o Between objective function and constraints
In Norman’s framework, presented earlier, the subject's elicited map is the scientist's
conceptualization of the user’s mental model. The simulator model is the target system. Mental
Model Similarity was operationalized with the C (closeness) statistic, a measure of the similarity
of the two maps (Schvaneveldt, 1990). The C statistic (Goldsmith and Davenport, 1990) is a
measure of the shared links across two different networks, ranging from 0 (low similarity) to 1
(high similarity). Recent research has provided some evidence to the reliability of the C statistic
(Gualtieri, Fowlkes, and Ricci, 1996; Stout, Cannon-Bowers, and Salas, 1996).
Since the task was to operate the modeled firm, not a real firm, the subjects had no prior specific
knowledge. As for general knowledge, the results showed positive, but not significant, effects
for experience and education on Firm Value.
The dependent variable Firm Value was operationalized as the market value achieved by the
subject. Firm Value was calculated as the market value added (MVA) for the seven simulated
years plus the continuing value of the firm (Copeland, Koller, and Murrin, 1996, 285), which
incorporates the known cash flows during the simulation and predicted value afterward.? Table 2
presents the results.*
Paired-samples t-tests were run to test for significant differences in the means for each treatment.
Controlling for SISS, the difference in mean Firm Value between low and high SIBS is
significant between Treatments #1 and #3 (ty9=3.286, p<0.002) and between Treatments #2 and
#4 (tz7=2.696, p<0.010). Controlling for SIBS, the difference in mean Firm Value between low
and high SISS is not significant.
Treatment # | Description (Count) Firm Value Standard
Mean MVA Deviation
1 Low SIBS with Low SISS (29) | -6.3e9 7Ae9
2 Low SIBS with High SISS (27) | -5.7e9 4.5e9
3 High SIBS with Low SISS (29) | -2.2e9 4.0e9
4 High SIBS with High SISS (29) | -2.5e9 3.69
Table 2: Firm Value for Treatments
a Continuing value was calculated with the long explicit forecast (Copeland, Koller, and Murrin, 1996), which takes
the last three years of EVA and calculates the linear trend forward twelve years, then calculates the NPV of this
stream.
“The negative mean MVAs reflect capital charges that exceeded profits. While many of the firms were profitable
over the seven simulated years, on average they destroyed economic value, not unlike real firms. Nevertheless,
there are significant differences in the degree to which the subjects destroyed value, on average.
The Hypotheses
The hypotheses developed in the beginning of the paper were tested through regression analysis.
The correlation matrix for the regression analysis, in Table 3, shows a suggestive correlation
between the independent variable SIBS and the dependent variable Mental Model Similarity.
SIBS shows a significant relationship with Firm Value. SISS shows a significant relationship
with Mental Model Similarity and not with Firm Value. There is no significant interaction effect
of the independent variables and Firm Value.
SISS__| SIBS* SISS_| Mental Model Similarity | Firm Value
SIBS 0.018 | -0.018 0.154~ 0.358***
SISS 0.018 0.229* 0.021
SIBS * SISS -0.108 -0.039
Mental Model Similarity 0.221*
** D<0.001 ** p<0.01 * p<0.05 ~p<0.1
Table 3: Correlation Matrix for Theoretical C onstructs
To test whether Mental Model Similarity mediates the relationship between the independent
variables and Firm Value, as proposed in Figure 1, Mental Model Similarity must show a
significant relationship with the independent variables and Firm Value must be significant with
Mental Model Similarity (Baron and Kenny, 1986). Mediation attempts to “explain how external
physical events take on internal psychological significance (Baron and Kenny, 1986, 1176).”
As shown in Figure 3, regressing Mental Model Similarity on the independent variables
(R’agjustea=0.069, p<0.026) showed a suggestive effect for SIBS (4=0.166, t=1.636, p<0.105), a
significant effect for SISS, and no significant interaction effect for SIBS*SISS.° Regression of
Firm Value on the independent variables (R’adjustea=0.140, p<0.002) showed no effect for SISS, a
significant effect for SIBS, and a very suggestive effect for Mental Model Similarity (4=0.181,
t=1.735, p<0.086).
Three conditions support the hypothesis that Mental Model Similarity mediates the effect of SISS
on Firm Value (Baron and Kenny, 1986):
1. SISS significantly influences Mental Model Similarity
2. SISS does not significantly influence Firm Value
3. Mental Model Similarity significantly influences Firm Value
There is also support for the hypothesis that Mental Model Similarity mediates the effect of SIBS
on Firm Value (Baron and Kenny, 1986):
5 To control for potentially low measurement validity due to the subject's frustration with their poor performance,
these analyses block on “mental model coherence” a measure of the consistency of the subject’s mental model
(Schvaneveldt, 1990). Only coherence levels above 0.0 are used, resulting in treatment #1 having 20 subjects, #2-
22, #3-26, and #4-26.
SIBS suggestively influences Mental Model Similarity
SIBS significantly influences Firm Value
When Mental Model Similarity is added to the regression of Firm Value on SIBS, the
influence of SIBS decreases and it is significant.
or
Combining these results for Mental Model Similarity with the decreasing standard deviation for
higher SIBS and SISS, as seen in Table 2, seems to indicate that the subjects tended to converge
in performance with increasing mental model similarity to the target system - they had more
similar mental models and tended to perform more similarly.®
System Image 0.250* Mental Model
Structural Similarity Se Similarity
0.166 0.181~
System Image Kagph (corr 0.281 pt.012)
Behavioral Similarity
ee
0.01
~p<0.1 *p<0.05 **p<0.01 ***p<0,001
For regression of MMS (PFCInfo) on independent variables, with MM Coherence > 0.0, R2-adj=0.069 (p<0.026).
For regression of Firm Value (MVA7 + Continuing Value), R2-adj=0.140 (p<0.002).
Figure 3: Test for Mediation of Mental Model Similarity
SISS * SIBS
These findings (difference in means, correlation, mediation, regression) provide support for the
hypotheses that SISS and SIBS positively influence Mental Model Similarity and that Mental
Model Similarity positively influences Firm Value.
A closer look at the dependent variable Firm Value shows that while SIBS and Mental Model
Similarity significantly influenced overall value created (MVA plus “continuing value”), the
results were not significant for MVA and were significant for the “continuing value.” SISS does
not influence either. SIBS did not significantly influence MVA, but did significantly influence
the “continuing value” (as=0.345, t=3.47, p<0.001). A detailed analysis of the SIBS-Firm Value
relationship shows that lower SIBS leads to a significant increase in investment in infrastructure
(i.e., base stations and information systems), which might indicate a hampering of the subjects’
ability to calibrate their investment decisions with lower SIBS. This finding supports the future
refinement of the MMS construct into two constructs, Mental Model Structural Similarity and
Mental Model Behavioral Similarity. This disaggregation might absorb some of the strong
influence of SIBS on Firm Value, and explain the lesser influence of the SIBS construct on MMS.
5 This is a potential area of research for influencing the holy grail of “shared mental models” with technology, in
addition to the well-known group decision support techniques of the learning organization (Senge, 1990).
Finally, MMS did not significantly influence MVA, but did suggestively influence the
“continuing value” (as=0.179, t=1.734, p<0.087). These results potentially indicate that MMS
does not explain short to medium-term financial indicators, but does partially explain long-term
performance.”
Refining the regression model shows in Figure 4 that the amount of Time the subject spent,
during the simulation, had no influence on MMS and a significant influence on Firm Value.
There is also a suggestive negative interaction effect of Time and Mental Model Similarity on
Firm Value. The negative interaction effect might be interpreted as subjects with lower MMS
that spent more time performed worse than those that spent less time, while subjects with higher
MMS that spent more time performed better than those that spent less time. This finding is
consistent with organizational research on strategic issue diagnosis (Dutton, 1993; Dutton,
Fahey, and Narayanan, 1983). In this revised model, the influence of MMS on Firm Value is
significant.
System Image 0.250* Mental Model
Structural Similarity Similarity
System Image ue
Behavioral Similarity
Time Firm Value
Time*MMS
~p<0.1 *p<0.05 **p<0.01 **p<0.001
For regression of MMS (PFCInfo) on independent variables, with MM Coherence > 0.0, R2-adj=0.069 (p<0.026).
For regression of Firm Value (MVA7 + Continuing Value), R2-adj=0.201 (p<0.000).
Figure 4: Revised Regression Model
Discussion
Summary
The findings in this research provide support for the theory developed in the beginning of the
paper. The results suggest the following potential contributions of this research. First, Mental
Model Similarity influences Firm Value. Though seemingly obvious, this suggests that for
practitioners, the goal is to provide the decision maker with a rich image of reality rather than the
current trend of overloading them with evermore financial and operational information.
7 This finding would be particularly interesting in industries where the “continuing value” represents between 50%
and 125% of the total value (Copeland, Koller, and Murrin, 1996, 286).
In demonstrating the influence of System Image Structural Similarity on Mental Model Similarity
and Firm Value, this research informs the use of strategic frameworks for decision making. This
research provides System Image Structural Similarity as a measure for determining which
frameworks provide a more complete yet parsimonious image of reality, thus permitting
comparison of frameworks such as EBIT and the Balanced Scorecard.
This research also confirms the influence of System Image Behavioral Similarity on Mental
Model Similarity and Firm Value. For practitioners, these finding suggest that accurate, timely,
and consistent information reinforces the benefits of more complete strategic frameworks. As
for generalizability, this experiment tested graduate students in a simulated experiment. Though
they had an average of five years work experience, and there was no significant experience
effect, their general knowledge of the elements and relations in a business model might not
reflect that of senior executives. Further, resource allocation decisions in real firms might
potentially include many more variables, though great care was taken to describe the main
elements of a service provider offering wireless telecommunications using standard strategic
elements.
Future Directions
Future research in this area might explore the generalizability of the findings, further detail of the
mental model, shared mental models, and cognitive tradeoffs in the System Image. Further
testing with executives might indicate the degree to which a strong business understanding
provides a stronger base mental model. The Mental Model Similarity construct should be
divided into the structural elements and relations (Mental Model Structural Similarity) and the
calibration of the mental model (Mental Model Behavioral Similarity).
The preliminary findings in this research that the technology (SIBS and SIBS) led to less variance
in performance should be further developed to determine the potential contribution this would
make to attainment of shared executive mental models. Cognitive limitations also constrain the
number of elements and relations that the decision maker can process. Further research on the
ratio of convergence to number of elements and relations might indicate an optimal level for
strategic decision making.
Appendix A
The Decisions
The subjects make decisions by allocating financial resources, every six months, to human
resources, information technology, and infrastructure. Assuming that the task is to maximize
shareholder value maximization each time period, the objective function of the model can be
described with Equation 1.° The left-hand side describes the price per customer of monthly
service. On the right-hand side, the first set of brackets describes the firm’s cost exposure per
5 This equation is equivalent to economic value added (EVA, see Ehrbar, 1998), which is revenues less
operating expenses less cash taxes less capital charges. Market Value Added (MV A) is the present value of future
EVAs. See http://www.sdsg.com/new/sd01paper/model_formulas.htm for the complete model formulation and
description. The variables in Greek letters are time invariant. The others may vary over time.
customer, reflected in its operating costs. The second set of brackets describes the firm’s net
capital exposure per customer, reflected in its capital investments and depreciation.
EVA = Revenues - Operating Expenses - Cash Taxes - Capital Charges
For EVA >0,
Revenues > Operating Expenses + Cash Taxes + Capital Charges
Op*on c*po*Pa , tinfra*Ptoc d*cy O
oy oferoh o* Pc *Pa, Tinta" Ploo d*oy f
i] c c c c i
fh *wace “infra tKssit +H) (0, *Cwace —MKinfra * Paintra *Kssit * Passit *Kot * Pall
q ° ° g
Equation 1
Starting with price per customer fi,, the model assumes a very competitive market and that
reducing price would only result in a price war, so price cannot be changed. As for cost
exposure, the resource of employees p can be changed by hiring or firing. Assuming the firm
has been reengineered, administrative costs c*fi;*n, and technical operating costs for the
infrastructure tinfra*ftoc are assumed to be efficient and not changeable. The cost of debt d*cq is
not changeable. In the second set of brackets, the resources of infrastructure Kinfra, service
supporting information technology k;si, and training k,, can be changed by investing in them or
letting them depreciate. The after-tax effect 4: equals the inverse of one minus the tax rate and is
assumed to be constant. The amount of asset depreciated each time period depends on the
existing asset amount and the depreciation rate fy, which depend on prior investments, so path
dependency assumes that depreciation on previous assets cannot be changed going forward. In
summary, the model permits hiring, firing, and training people, building new base stations, and
buying new information technology.
Appendix B
The user interfaces used in the experiment are presented in the following screen captures.
Figure 6: Treatment #2 - Low SIBS with High SISS
size
Figure 8: Treatment #4 - High SIBS with High SISS
References
Bantel, K. 1993. Strategic Clarity in Banking: The Role of Top Management-Team
Demography. Psychological Reports 73: 1187-1201.
Baron, R.M., and D.A. Kenny. 1986. The Moderator-Mediator Variable Distinction in Social
Psychological Research: Conceptual, Strategic, and Statistical Considerations. Journal of
Personality and Social Psychology 51 (6): 1173-1182.
Barua, A., C.H. Kriebel, and T. Mukhopadhyay. 1989. MIS and Information Economics:
Augmenting Rich Descriptions with Analytical Rigor in Information Systems Design. Paper
read at The Tenth International Conference on Information Systems, December, at Boston.
Borgman, C.L. 1999. The User's Mental Model of an Information Retrieval System: An
Experiment on a Prototype Online Catalog. International Journal of Human-Computer
Studies 51: 435-452.
Bourgeois, L. 1980. Performance and Consensus. Strategic Management J ournal 1: 227-248.
Caldwell, B., and T. Stein. 1998. Beyond ERP: New IT Agenda. Informationweek Online,
November 30.
Carley, K., and M. Palmquist. 1992. Extracting, Representing, and Analyzing Mental Models.
Social Forces 70 (3): 601-636.
Cats-Baril, W.L., and G.P. Huber. 1987. Decision Support Systems for I]]-Structured Problems:
An Empirical Study. Decision Sciences 18 (Summer): 350-372.
Copeland, T., T. Koller, and J. Murrin. 1996. Valuation: Measuring and Managing the Value of
Companies. New Y ork: John Wiley & Sons, Inc.
Craik, K. 1943. The Nature of Explanation. Cambridge: Cambridge University Press.
Davenport, T.H. 2000. Mission Critical: Realizing the Promise of Enterprise Systems.
Cambridge, MA: Harvard Business School Press.
Doyle, J.K., and D.N. Ford. 1998. Mental Models Concepts for System Dynamics Research.
System Dynamics Review 14 (1): 3-29.
Dutton, J.E. 1993. Interpretations on Automatic: A Different View of Strategic Issue Diagnosis.
Journal of Management Studies 30 (3): 339-357.
Dutton, J.E., L. Fahey, and V.K. Narayanan. 1983. Toward Understanding Strategic Issue
Diagnosis. Strategic Management J ournal 4: 307-323.
Ehrbar, A. 1998. EVA: The Real Key to Creating Wealth. New Y ork: John Wiley & Sons, Inc.
Einhorn, HJ., and R.M. Hogarth. 1986. Judging Probable Cause. Psychological Bulletin 99 (1):
3-19.
Feltham, G.A. 1968. The Value of Information. Accounting Review 43 (4): 684-696.
Fuerst, W.L., and P.H. Cheney. 1982. Factors Affecting the Perceived Utilization of Computer-
Based Decision Support Systems in the Oil Industry. Decision Sciences 13: 554-569.
Goldsmith, T.E., and D.M. Davenport. 1990. Assessing Structural Similarity of Graphs. In
Pathfinder Associative Networks: Studies in Knowledge Organization, edited by R.
Schvaneveldt. Norwood, NJ: Ablex.
Gualtieri, J., J. Fowlkes, and K. Ricci. 1996. Measuring Individual and Team Knowledge
Structures for Use in Training. Training Research J ournal 4: 117-141.
Hammond, K.R. 1955. Probabilistic Functioning and the Clinical Method. Psychological Review
62: 255-262.
Huang, K.-T., Y.W. Lee, and R.Y. Wang. 1999. Quality Information and Knowledge. Upper
Saddle River, NJ: Prentice Hall PTR.
Hursch, CJ., K.R. Hammond, and J.L. Hursch. 1964. Some Methodological Considerations in
Multiple-Cue Probability Studies. Psychological Review 71: 42-60.
Johnson-Laird, P.N. 1983. Mental Models: Towards a Cognitive Science of Language, Inference,
and Consciousness. Cambridge, MA: Harvard University Press.
Kleinmuntz, D.N. 1993. Information Processing and Misperceptions of the Implications of
Feedback in Dynamic Decision Making. System Dynamics Review 9 (3): 223-237.
Kleinmuntz, D.N., and D.A. Schkade. 1993. Information Displays and Decision Processes.
Psychological Science 4 (4): 221-227.
Martin, M.H. 1998. Smart Managing. Fortune, February 2, 149-151.
Mathieu, J.E., G.F. Goodwin, T.S. Heffner, E. Salas, and J.A. Cannon-Bowers. 2000. The
Influence of Shared Mental Models on Team Process and Performance. Journal of Applied
Psychology 85 (2): 273-283.
Moxnes, E. 1998. Misperceptions of Bioeconomics. Management Science 44 (9): 1234-1248.
Norman, D.A. 1983. Some Observations on Mental Models. In Mental Models, edited by D.
Gentner and A. L. Stevens. Hillsdale, NJ: Lawrence Erlbaum A ssociates.
Paich, M., and J.D. Sterman. 1993. Boom, Bust, and Failures to Leam in Experimental Markets.
Management Science 39 (12): 1439-1458.
PeopleSoft, I. 2000. PeopleSoft Balanced Scorecard PeopleSoft Inc., 2000 [cited March 14
2000]. Available from
http://checkers. peoplesoft.com/prodindsol .nsf/572fb441b9716d788256672006d656d/3bac44
dfa79bd70888256732007d1859?0 penD ocument.
Porter, M. 1980. Competitive Strategy. New Y ork: Free Press.
Rowe, A.L., and NJ. Cooke. 1995. Measuring Mental Models: Choosing the Right Tools for the
Job. Human Resource Development Quarterly 6 (3): 243-255.
SAP. 1999. SAP AG Strengthens Its SAP Strategic Enterprise Management Solution By Adopting
Proven Approach for Strategic Management Processes From The Balanced Scorecard
Collaborative SAP America, 1999 [cited April 1999]. Available from
http://www.sap.com/press/04_99/04_99 5.htm.
SAP. 2000. SAP Strategic Enterprise Management: Enabling Value Based Management --
Translating Strategy into Action SAP E-Business Solutions (Business Intelligence), 2000
[cited February 17 2000]. Available from
http://www.sap.com/solutions/bi/sem/sem_over.htm.
Schvaneveldt, R., ed. 1990. Pathfinder Associative Networks: Studies in Knowledge
Organization. Norwood, NJ: Ablex.
Senge, P. 1990. The Fifth Discipline. New Y ork: Doubleday Currency.
Sharda, R., S. Barr, H. McDonnell, and C. James. 1988. Decision Support System Effectiveness:
A Review and an Empirical Test. Management Science 34 (2): 139-159.
Shepard, S. 2000. Telecommunications Convergence. New Y ork: McGraw-Hill.
Silver, M.S. 1990. Decision Support Systems: Directed and Non-directed Change. Information
Systems Research 1 (1): 47-70.
Snow, C., and L. Hrebiniak. 1980. Strategy, Distinctive Competence, and Organizational
Performance. Administrative Science Quarterly 25: 317-336.
Staggers, N., and A.F. Norcio. 1993. Mental Models: Concepts for Human-Computer Interaction
Research. International Journal of Man-Machine Studies 38: 587-605.
Sterman, J.D. 1989. Modeling Managerial Behavior: Misperceptions of Feedback in a Dynamic
Decision Making Experiment. Management Science 35 (3): 321-339.
Sterman, J.D. 1994. Learning in and about Complex Systems. System Dynamics Review 10 (2-3):
291-330.
Stout, RJ., J.A. Cannon-Bowers, and E. Salas. 1996. The Role of Shared Mental Models in
Developing Team Situational Awareness: Implications for Training. Training Research
Journal 2: 86-116.
Stout, R.J., J.A. Cannon-Bowers, E. Salas, and D.M. Milanovich. 1999. Planning, Shared Mental
Models, and Coordinated Performance: An Empirical Link Is Established. Human Factors 41
(1): 61-71.
Todd, P., and I. Benbasat. 1999. Evaluating the Impact of DSS, Cognitive Effort, and Incentives
on Strategy Selection. Information Systems Research 10 (4): 356-374.
Vandenbosch, B., and C. Higgins. 1996. Information Acquisition and Mental Models: An
Investigation into the Relationship Between Behavior and Learning. Information Systems
Research 7 (2): 198-214.
Zmud, R.W. 1978. An Empirical Investigation of the Dimensionality of the Concept of
Information. Decision Sciences 9 (2): 187-195.
CC BY-NC-SA 4.0