The Application of the Technology Acceptance Model: A New Way to
Evaluate Information System Success
Wei-Tsong Wang
School of Information Science and Policy, University at Albany
840 Washington Avenue, Albany, NY 12203
212-404-7657
ww8256@albany.edu
Chao-Yueh Liu
Rockefeller College of Public Affairs and Policy, University at Albany
14 Niblock Court, 2™ FL, Albany, NY 12206
212-404-7651
chaoyueh.liu @ gmail.com
Abstract:
In the modern era, the advances in information technology have been dramatically shaping
the ways people live as well as the ways organizations manage their businesses in their
professional business domains. Implementing various kinds of information systems, such as
Decision Support Systems, has been recognized as one of the most crucial tasks for organizations
in order to continue to be competitive or even to survive. Although considerable effort has been
devoted to improving the performance of information system implementations, organizations are
still constantly suffering from the failures of information system implementations. In this study an
extensive framework that depicts the context of information system implementation is developed. A
system dynamics approach is used to investigate the dynamic nature of information system
implementations. By using the proposed system dynamics model, we contend, executives and
information system professionals of organizations can gain comprehensive insights into
organizational behaviors and substantial policy-making implications regarding information
system implementations.
Keywords: Information System Success, System Dynamics, TAM
1. Introduction:
In the modern era, the advances in information technology have been dramatically shaping
the ways people live as well as the ways organizations deal with their businesses in their
professional business domains. Implementing various kinds of information systems, such as
Enterprises Resource Planning (ERP) systems, Decision Support Systems (DSS), and Knowledge
Management Systems (KMS), has been recognized as one of the necessary tasks organizations
have to perform in order to continue to survive. Given the tremendous amount of efforts
organizations have devoted to the implementation of information systems, organizations are still
continuously suffering from the failures of information system (IS) implementation. The purpose
of this study is to provide a comprehensive framework that can help information system
professionals understand the context of information system implementation. By having accurate
assessments, the framework can in turn help IS professionals develop effective strategies or
policies in order to maximize the probability of success in implementing information systems.
Among all potential causes that might be responsible for the success or the failure of
information system implementation, users’ attitude and acceptance of an information system have
been recognized as factors that have critical impacts on the performance of information system
implementations (Davis 1989; Pikkarainen, Pikkarainen, Karjaluoto, and Pahnila 2004; Succi and
Walter 1999; Venkatesh and Davis 1996). The Technology Acceptance Model (TAM) has been
well known as the most cited and influential model for understanding the acceptance of
information technology since it was developed by Davis (1986) and Davis, Bagozzi, and Warshaw
(1989) in the late 80’. The DeLone and McLean (D&M) IS Success model, ten years after it was
first proposed by DeLone and McLean (2002), has become a standard for the developing and
justifying the measurement of the dependent variable in information systems research. A total of
285 papers, including published papers as well as proceedings, have referenced this model to
discuss the evaluation of IS success during the period 1993 to mid-2002 (DeLone and McLean,
2003). Given both models are expanded and empirically validated by various scholars and
practitioners, one question still interests many information system professionals: Is it possible to
have a better and more thorough model for evaluating IS success?
In this research project we seek for the possibility of creating a new model for evaluating IS
success by applying the concepts of both TAM and D&M IS Success Model. The discussion starts
with a brief introduction of Technology Acceptance Model and an overview on the development of
DeLone and McLean’s (1992; 2002; 2003) IS success model. An integrated model for evaluating
IS success that is generated by encompassing the fundamental theories of both the TAM and the
D&M update IS Success Model is proposed. The approach of system dynamics is adopted for this
study so as to demonstrate how the proposed model can be beneficial for decision makers in
organizations on evaluating the implementation of information systems.
2. Development of the theoretical foundation:
2.1 Technology Acceptance Model (TAM):
Davis (Davis 1986; Davis 1989; Davis 1993; Davis, Bagozzi, and Warshaw 1989) introduced
TAM, which is presented in Figure 1, for modeling user acceptance of information systems in 1986.
TAM starts by proposing external variables as the basis for tracing the impact of external factors
on two main internal beliefs, which are perceived usefulness and perceived ease of use, while
perceived ease of use also affects perceived usefulness over and above external variables (Taylor
and Todd 1995). These two beliefs both influence users’ attitude toward using IS. Attitude toward
using sequentially has influence on behavior intention to use, which is the key factor for
determining actual conditions of system use, while belief of perceived usefulness also affects
behavioral intention to use over attitude toward using (Taylor & Todd, 1995).
Perceived
| Usefulness
Aw)
Attitude Behavioral heal
External Toward Intention to Sane
Variables Using (A) Use (Bi) *
Perceived ]
Ease of Use |~
(E)
Figure 1: Technology Acceptance Model
2.2 The D&M IS Success Model:
The original D&M IS success model was proposed by DeLone and McLean (1992) back in
1992. The main purpose of this model was to “identify those factors that contribute to information
systems success” (DeLone and McLean 1992, 60). They finally identified six most important
categories of factors for evaluating IS success, which are information quality, system quality, use,
user satisfaction, individual impact, and organizational impact, and a model was created using
these factors as presented in Figure 2. “System quality” refers to the performance of an IS system
itself, while “Information quality” refers to how good is the output from a particular IS system.
“Use” is used to measure how well the output of the IS system, such as information or physical
reports, are used. “User satisfaction” represents users’ overall comments on the IS system.
“Individual impact” that DeLone and McLean address here refers to the influence of the outputs of
IS systems on individual users’ behaviors, while “Organizational impact” refers to the effects of
the usage of IS systems on the organizational performance. It is proposed that Information quality
and System quality have influence on both Use and User satisfaction. Use and User satisfaction
affects Individual impact, and Individual impact in terms influences Organizational impact. One
thing needs to be addressed is that the model was built based on a process nature, and the focus
while utilizing this model should be to examine how these six categories of factors are interrelated
and interdependent with one another instead of concerning the causal relationships among them.
Information Use
Quality |. Vi XS
\ : \
; Individual Organizational
Impact Impact
System User
Quality Satisfaction
Figure 2: The Original D&M IS Success Model in 1992
DeLone and McLean (2002; 2003) reevaluate their original IS success model, which has been
released more than ten years, by taking into consideration the opinions as well as criticisms from
other scholars and practitioners in the last decade. DeLone and McLean slightly modify their
original model by including some factors that are newly considered important for evaluating IS
success, especially factors that are necessary for measuring e-commerce systems success. This
new D&M IS Success Model is presented in Figure 3. In DeLone and McLean’s update model,
they do not separately consider about individual impact, organizational impact, and some other
kinds of emerging impact measures, such as work group impact and consumer impact. On the
contrary, DeLone and McLean use the term “net benefits” to represent all the impact measures for
the sake of simplifying the model. In addition, DeLone and McLean think the variable “use” in
their original model is defined without considering the actual complexity of the usage behaviors.
They realize that perspective system users, especially the e-commerce system users, are not always
required to use the system. The usage of the systems performed by these users may not be able to
totally represent the complex conception of “use” for the purpose of evaluating net benefits under
certain circumstances. They state, “declining usage may be an important indication that the
anticipated benefits are not being realized” (DeLone and McLean 2003, 16). As a result, they
suggest “intention to use” as an alternative of “use” for some particular circumstances in their
update model. Moreover, DeLone and McLean agree with the concepts, which are proposed by a
number of scholars (Kettinger and Lee 1994; Pitt and Watson 1995), on the importance of service
quality for evaluating IS success. Corresponding to these propositions, DeLone and McLean adopt
service quality, along with system quality and information quality, as an important category of
factors that has influence on intention to use/use as well as user satisfaction.
Information
Quality €
Intention to
Use Use
System
Quality Net Benefits
User
Satisfaction
Service
Quality
Figure 3: The Update D&M IS Success Model
DeLone and McLean address the point that their update model is still constructed in a process
sense. However, it is flexible to utilize the model in a causal sense under certain circumstances.
“The nature of these casual associations should be hypothesized within the context of a particular
study” (DeLone and McLean 2003, 23).
2.3 The Integration of TAM and the D&M update IS Success Model:
From the comparison of TAM and the D&M update IS success model we may see more
comprehensive concerns on the system use in TAM model than in D&M IS success model. It is
reasonable since TAM was mainly developed to focus on evaluating system usage from users’
perspective. However, the D&M update IS success model concerns about the relationships among
actual system usage, user satisfaction, and their influence on the overall benefits, while TAM does
not. Both TAM and the D&M update IS success model have their own strengths and weaknesses in
terms of evaluating the success of an information system.
However, by integrating the concepts of these two models, we can to a certain extent create a
more comprehensive and solid model for evaluating IS success model, since these two models are
complementary to each other in a certain way. The proposed model is presented in Figure 4.
System
Quality f
4 Perceived
Usefulness |
Information |/ : Belavion
l . javiorial
Quality | Attitude eves - [Actual System User Net
toward Using 4 Use Usage Satisfaction Benefits
Pal ae ry
Perceived
Service Ease of Use
Quality ¥
Figure 4: The Integrated IS Success Model
The proposed model is constructed by taking three variables, system quality, information
quality, and service quality, in D&M IS success model as the replacements for the external
variables in TAM, with a perception that they are the most important three external variables for
evaluating system usage. As for the broad concept of system usage, we adopt the idea of TAM
since TAM has more comprehensive concepts related to it. In the new model, we include the
variables “perceived usefulness” and “perceived ease of use” which the D&M update IS success
model does not have. By including these two variables, we argue that users’ perceptions on
usefulness and ease of use, instead of system quality, information quality, and service quality, have
direct influence on system usage, while system quality, information quality, and service quality
serve as the most important variables that affect perceived ease of use and perceived usefulness. In
addition, we replace “intention to use/use” in the D&M update IS success model with the three
factors that are proposed in TAM for evaluating system usage, which are attitude toward using,
behavioral intention to use, and actual system usage in a sequential order (see Figure 4).
However, for the purpose of evaluating IS success, we keep the variable “user satisfaction” in
our model. We contend with DeLone and McLean’s (2003) idea that actual system usage has direct
impact on both user satisfaction and the overall benefits that are generated by the implementation
of the information system, and user satisfaction also directly affects the overall benefits. In
addition, instead of adopting DeLone and McLean’s argument that user satisfaction has direct
influence on intention to use/use, we suggest that user satisfaction has direct influence on
perceived ease of use and perceived usefulness, and in turn affects actual system usage indirectly.
We argue that this suggestion is the key to link TAM and the D&M update IS success model
together. We also argue that this new model can be utilized in either a process sense or a causal
sense, as DeLone and McLean’s claim on their update D&M IS success model.
3. Context of the Development of IS Success System Dynamics Project:
3.1 Purpose of the Project
Although the fundamental structure of the proposed model is developed by referring to a
number of existing theories in the area of management information systems, this study is
conducted for the purpose of making policy recommendations instead of proving a theory. By
conducting this study we can have a better understanding on the dynamics of information system
implementations in organizations. The model of this study is developed based on the well known
Technology Acceptance Model and D&S IS Success Model from the area of management
information systems. The purpose of constructing this model is to identify key variables that are
associated with the performance of information system implementations in order to have a basic
foundation for study rather than starting from scratch. Since both reference models have been
empirically validated and widely extended by researchers, it is plausible that the variables selected
from the two reference models are the key factors that are associated with the dynamics of
information system implementations. In addition, we also use the proposed model as the skeleton
for identifying important variables and causal relationships that are related to the implementations
of information systems in a more detailed manner. By doing so, we can develop a more
comprehensive model in terms of properly exploring the dynamics of information system
implementations.
3.2 Target Audience
There are two groups of target audience for this study, IS researchers and IS practitioners.
This study is expected to benefit IS practitioners by providing them better insight into what major
factors and causal relationships among these factors are dominating the implementation of
information systems. IS practitioners can utilize the insights provided by this study to make better
policies or to employ more proper strategies for information system implementations, in terms of
cost effectiveness and efficiency, desired benefits from information systems, forces of support and
barriers within organizations, etc., by taking advantage of a more appropriate and systematic logic
of thinking.
In addition, since the context of information system implementation is very complex and
somehow vague, it is difficult to identify all the dynamic structures of IS implementation across
various perspectives. However, we are expecting that this study would give researchers more
comprehensive ideas on what situations people are really facing when implementing information
systems. By understanding it, researchers are able to have some leads regarding discovering
dynamic structures that have not been identified or drawn the attention they deserve. These efforts
will in turn contribute to the knowledge base of policy making in IS implementation.
4, Boundaries and Development of the System Dynamics Model of IS Success:
4.1 Endogenous, Exogenous, and Excluded Variables
From the perspective of system dynamics, the proposed model contains only endogenous
variables that describe the story of information system implementation. However, we believe that
there are two main excluded exogenous variables which also have notable impact on IS success,
which are Competition and Advances of information technology. The variables “Increase in user
requirements on information system“ and ” Efforts on enhancing IS quality”, which are considered
as endogenous variables, are embedded in the model to serve as the bridge between the internal IS
implementation environment and the quality of information systems.
There are many exogenous variables proposed in the existing literature that are associated
with information system implementation. At current stage we manage to include three most crucial
exogenous variables whose values can be adjusted for the purpose of studying the behaviors of the
system. These three exogenous variables are “Training efforts’, “User involvement in system
development”, and “Perceived sufficiency of organizational resources”. The model can be further
expanded in terms of the complexity of policy-making implications by identifying more key
exogenous variables in the future.
4.2 Model Structure
A system dynamics model of IS success is developed based on our previous discussions and
its aggregated casual loop diagram is presented in Figure 5.
Behavioral
+ intention to use
Ciena Attitude toward
‘ser involvement in ; ‘
ust R2: "User's +
system development “4 * psseetibsfal Actual system
* IS quality and 4_tse rate
system usage
adjustment
loop
+
Perceived
usefulness
Perceived ease
of use
+h +
User satisfaction
Effort on user R1:"Benefits ~
training Overall IS from the use
it of IS
quality :
adjustment as ®
- loop"
p Actual task
completion rate
Effort on enhancing 4 Increase in user
system quality ~*——_________— expectation on IS
+
- +
Performance
AS investment
-
Perceived necessity
to invest onIS + Executives’
support on IS
Perceived sufficiency of
organizational resources
Figure 5: The System Dynamics Model of IS Success
From Figure 5 we can identify two reinforcing loops and one balancing loop that are
dominating the behaviors of the system. The first reinforcing loop R1, which is named “Benefits
from the use of information systems adjustment loop‘, is presented in Figure 6. The story behind
the R1 loop is that the more the users in an organization are willing to use their information system,
the more they are satisfied with the system. As a result, the users can use the information system in
more effectively and efficiently ways and in turn increase the actual task completion rate. When
the benefits the users get from the information system increase, users’ expectations on the
information system will increase. The increasing expectations of users will encourage or force the
IS professionals of the organization to put more efforts on enhancing the quality of the information
system, and in turn result in improvement in the overall IS quality. When the quality of the
information system increases, the system are assumed to be capable of providing more useful
Expected task
completion rate
information, services, and user-friendly interfaces, which can make users feel that the system is
becoming more useful and easier to use. As a result, the users will become even more willing to
use the information system than they used to be in order to get even more benefits from it.
Behavioral
ae intention to use ON
Attitude toward “Actual system
using use rate
+ +
+
User satisfaction
Perceived ease Perceived R1:"Benefits
of use usefulness
from the use
of IS
y
adjustment Actual task
\ ) loop" completion rate
Overall IS
quality
\ .
Effort on enhancing Increase in user
system quality <i expeectation on IS
Figure 6: The R1 reinforcing loop in the IS Success Model
The second reinforcing loop R2, which is named “User’s perception on information system
quality and system usage adjustment loop”, is presented in Figure 7. The story behind the R2 loop
is that the more the users in an organization are willing to use their information system, the more
they are satisfied with the system. The more the users are satisfied with the system, the more likely
they will start to feel that the information system is becoming more useful and easier to use. As a
result, the users will become even more willing to use the information system than they used to be.
i Ss
Attitude toward + Behavioral
using intention to use
+ +
R2: "User's
7 perception on IS
Perceived ease Perceived quality and
of use usefulness system usage
+ Be adjustment loop
+
User satisfaction + Actual system
—4+___—— _ use rate
Figure 7: The R2 reinforcing loop in the IS Success Model
The balancing loop B1, which is named “JS benefits and IS investment adjustment loop*, is
presented in Figure 8. The story behind the B1 loop is that the more the users in an organization are
willing to use their information system, the more benefits they can acquire, which reflect on the
increase in actual completion rate as we discussed previously. When the benefits the users get from
the information system increase, the gap between the expected and actual task completion rate,
which is represented by the variable “performance ratio” in the model, will decrease. As a result,
the users will have the perception that the information system is good enough for them and this
kind of performance of the information system will last even though the organization did not do
anything afterwards. This kind of perception will lead to the perception that there is no need to
continue to invest as well as put more efforts on the information system since it is functioning
pretty well. However, an information system must be properly maintained by devoting constant
efforts in order to continue to provide the same quality of services to the users. These inadequate
reactions will deteriorate the quality of the information system and in turn make it less useful and
harder to use for the users. As a result, the users will become less willing to use the information
system than they used to be.
Attitude toward
(3
Perceived ease Perceived
of use usefulness
ne I
Overall IS
quality
f
Effort on enhancing
system quality
+
Investment on
information systems
Behavioral
ms intention to use oo,
Ketual s ystem
use rate
(
User satisfaction
B1: "IS \. 4
benefits and Actual task
IS investment
adjustment
loop"
completion rate
Performance
ratio
Perceived necessity +
to invest on IS
Figure 8: The BI balancing loop in the IS Success Model
The stock and flow diagram of the proposed IS model is presented in Figure 9 below. There
are four stocks in the model, which are "Actual system use rate", "Actual task completion rate", “IS
investment”, and "Overall IS quality". These four stocks represent the main measures for
organizations to evaluate the performance of the implementations of information systems.
Generally, the ideal scenario for organizations is that the larger the IS investment is, the better the
Overall IS quality is, the higher the Actual system use rate is, and the higher the Actual task
completion rate is. Each stock has an expected or indicated value, which is influenced by a number
of related variables. The net increase of a stock is mainly determined by the gap between its current
value and its expected value. Each stock is also accompanied by a SMOOTH structure in order to
reflect the effect of time delay on the stock.
Behavioral
+ intention to use Time to adjust
system use rate
User involvement in
system development Ridin’ _
ost “User's system use
i perception on Net increase in te
* 1S quality and systemuse 7
system usage -
Effort on user “nee
training oop >
Expected system
N use rate
¢ Perceived
Perceived ease usefulness +
a ee satisfaction
+
Tine to adjust IS .
quality
‘Time to adjust task Indicated task
completion rate completion rate
Overall IS R1:"Benefits
- quality from the use ine ttidl
Net increase in : of IS ae completion rate
. ss ctual tas]
= ame .
overall IS quality je sdleciest dual tail
Net increase in task rate.
E
Expected IS
quality wt Effort on enhancing
system quality $f Inerease in user
+ expectation on IS
completion rte 4g ——/
+
Time to adjust
investment
nie Performance
IS investment benefits and male
Net increase in IS IS investment
adjustment
ag
investment
’
Perceived sufficiency of
organizational resoure
Perceived necessity .
to invest on IS _+ Executives!
: +
cs Shieeal IS gee SS6__Sjaenans
investment
Figure 9: The stock and flow diagram of the IS Success Model
4.3 Application of System Archetype
The “Limits to Success” archetype structure in the proposed IS success model is composed of
one balancing loop and one s reinforcing loop (see Figure 10). The balancing loop B1 represents
the structure that contains the constraint and limiting action of the system. The s reinforcing loop
RI represents the structure that contains the efforts made for the purpose of enhancing
performance.
Actual system
Performance
if use rate \ Perceived sufficiency of
Co \ organizational resources
Effort on enhancing i) Actual task
IS quality completion rate Investment on
+ 4 information systems _+
Figure 10: “Limits to Success” structure of the IS success model
The “efforts” loop R1 starts from Actual task completion rate (see Figure 10). When the
Actual task completion rate increases, Efforts on enhancing IS quality increases, and eventually
leads to the increase in Actual task completion rate. Although the structure of “efforts” pushes the
system to move toward the direction for success, the effect of the “constraint” loop B1 includes a
constraint that can limit the potential of the “efforts” structur in creating benefits. The indicated
constraint is the limited organizational resources. The balancing loop B1 (see Figure 10) that
includes the constraint starts from the Actual task completion rate. When the Actual task
completion rate decreaes, the Investment on information systems increases, and eventually leads
to the increase in the Actual task completion rate. However, when the Perceived sufficiency of
organizational resources is lower than a certain level, it constaints the increase in the Investment
on information systems, and in turn limits the potential for increasing Actual task completion rate.
4.4 Time Horizon
As far as we are concerned, there has not been a commonly accepted estimation on the
average lifespan of an information system. However, since the length of the diffusion of an
information system can be a few years to a few decades (Yang and Huang 2004), ten to fifteen
years would be a plausible estimation of the time horizon for this system dynamics study. As a
result, the time horizon for the proposed IS success model is set as ten years.
4.5 Sector Overview of the IS Success Model
The sector overview diagram of the IS Success model is presented in Figure 11 below. The
diagram is presented in order to provide us a comprehensive insight into the key behaviors of the
proposed model. There are six sectors and multiple flows in the system. These six sectors are “Use
of information system’, “Benefit assessment of information system”, “Contribution of information
system”, “Organizational investment on information system”, “Quality of information system’, and
“User's assessment and attitude on information system’. Each sector represents an important
behavior in regard to the use of an information system in an organization. Instead of isolating from
other objects in the system, these sectors are associated with one another through the transfer of
materials and information among them. The material or information flows provide necessary
information to their designated sectors for the purpose of performing the main tasks of the sectors.
Among all the flows in the diagram, only two of them are material flows, which are “/nvestment on
13
IS” and “Training efforts”, while the others are information flows. The “Investment on IS”
contains money, equipment, human resources, etc., while the “Training efforts” represents hours of
efforts spent on training information system users.
soca User involvement in
-
oT system development
3 pert ng 7
Z v7 Users overall Use of Information System
ta attitude toward _ .
@ Actua e1 IS
User's Assessment and_ using IS Actual system use
Attitude on Information Tate
System
© Perceived usefulness
© Perceived ease of use *
© Attitude toward using \ ‘
© Behavior intention to Actual 9 \
e -. '
use information ‘ Expected information
support from IS support from IS
s
4 User satisfaction,
1 N
'
1 fa _
H a
i Information System Benefit Assessment of
Training efforts Informat stem
} © Actual task ”
/ completion rate @ Expected task
/ © User satisfaction completion rate
eqeusbl Expected @ Performance ratio
Overall quality of IS¢
Pa
a
uality of Information
System S)
Inde ase in user's requirements :
on IS I
Organizational a
@ System quality
attitude on IS J ¥
investment
--
© Information quality
© Service quality Organizational Investment
‘ived necessity to
invest on IS
@ IS investment
@ Efforts on enhancing
IS quality
Investment on IS
\
1
1
Sutficicency of
organizational
resources
Figure 11: Sector overview diagram of the IS success model
4.6 Policy Structure Diagram of the IS Success Model
The policy structure diagram of the IS Success model is presented in Figure 12 below. The
diagram provides a conceptual representation of the policy structure that is embedded in the
proposed IS success model. The diagram is beneficial to the general public since it reveals the
policy structure of the proposed model in a manner that people can easily comprehend. In this
diagram, the information from two sectors, which are “Use of information system” and
“Contribution of information system’, help decision makers in an organization determine their
performance gap in benefits from the use of an information system. This gap in turn serves as the
main criterion for decision makers in the organization to determine how they are going to allocate
their organizational resources. The amount of organizational resources assigned to information
system implementation and the previous contribution of the information system determine the
quality level of the information system. The quality level of the information system is used as the
determinant for information system users to assess the usefulness of the information system.
User’s perception on the information system is the key for assessing the use rate of the information
system. Finally, the level of use rate of the information system is used to evaluate the benefits that
are generated from the use of the information system.
Use of
Information
System
Expected
User's Overall Information Support
Attitude toward Using
IS Actual Information
Support
Expected Benefits
and Attitude 4 from IS Performance
on Information 5 i oi Contribution of -<@ gap in
System User Satisfaction | tformation System benefits from
IS usage
Actual Benefits
from IS
Increase in Desired Oo icational
IS Quali rganizationa
Overall Quality Q . Perception on IS
of IS Investment
Quality of Organizational
Information Organizational Resurce
System tea Efforts on IS allocation
Figure 12: Policy structure diagram of the IS success model
4.7 Reference Mode
The reference modes of the dynamics system in this study is discussed based on the behaviors
of the most four important variables, “Actual system use rate”, “Actual task completion rate”, “IS
investment”, and “Overall IS quality”. These four major variables are responsible for the success
or failure of information system implementation. One fundamental assumption for this model is
that when certain normal level of investment on a particular information system is devoted
continuously, the quality of information system and the actual system use rate of users are
expected to become stable at certain normal levels. However, having the normal actual system use
rate only allows an organization to perform its regular operations properly, but cannot significantly
generate extra benefits for the organization. As a result, the task completion rate is also expected to
stay at a certain normal level.
Another key assumption is that an organization can significantly increase the benefits
generated from implementing an information system by boosting up the system use rate by taking
advantage of employing accurate as well as effective organizational strategies or policies.
Corresponding to these two assumptions, all of the four stocks are expected to start at certain
equilibrium levels, and then increase gradually and eventually reach certain higher equilibrium
levels if proper organizational policies were implemented. The reference modes of the IS success
model are developed and presented in Figure 13, 14, 15, and 16.
Actual system use rate Actual task completion rate
80 40
0 30
40 2
20 10
0 0
0 2 4 % 8 © 2 S % 108 10 0 2 4 % 4 6 72 S& % 108 120
‘Tire (nn) ‘Tine (north)
‘Actual systemuse rate IS ideal mnde. —+—+—+—+-hour(person*month) Actual task competion rate : IS ideal mode. +——+—+—+—+ task/month
Actual systemuse rate :IS original mde —2—2—2—2 hour(person*month) Actual task completion rate : IS original ede. —-2-—2—2—2— task/month
Figure 13: Reference mode- Actual system use rate — Figure 14: Reference mode- Actual task completion
rate
IS investment Overall IS quality
800,000 08
600,000 06
400,000 04
Hs
=
200,000 02
0 0
0 2 &@ % & © 2 8 % 18 1D O 2 4 % 4 OO 72 4 % 108 120
‘Tine (nonth) ‘Tie (nnth)
IS investment IS ideal mode. —+—+—+—+—+—+—+— dollu/onth Overall IS quality =1IS ideal mode. —4—+—+—+—+—+—+—+ Dn
IS imestrent :IS original ede —2—2—2—2—2—2—2— dolkarfmonth Overall IS quality : IS original nde. —2—2—2—2—2—2—2—2 Drm
Figure 15: Reference mode- IS investment Figure 16: Reference mode- Overall IS quality
5. Application of the IS success System Dynamics model:
5.1 The Base Run
Three exogenous variables are included in the model for adjusting the behaviors of the system
of IS implementation, which are “User involvement in system development”, “Effort on user
training”, and “Perceived sufficiency of organizational resources”, as presented previously in
Figure 9. The changes in the values of these three variables are expected to significantly change the
behaviors of the system. The graphs of the base run of the model are presented in Figure 17, 18, 19,
and 20 below. In this base run, all of the four stocks stay at certain equilibrium levels throughout
the whole simulation time period.
Actual system use rate Actual task completion rate
40 10
35 95
30 9
25 85
2 8
9 RB 4% 8 OM PR MH 6 WE RH 9 2 4 % & O 2 SM % 108 120
‘Tine (month) ‘Tine (nnth)
Actual systemuse rate :1S base +—+—+—+—+—+ howwfpersontmonth) acre tak completion rte: 1S base wagons
Figure 17: Base run - Actual system use rate Figure 18: Base run - Actual task completion rate
IS investment Overall IS quality
400,000 04
350,000 035
300,000 03
250,000 025
200,000 02
0 2 4 % & © 2 8 % 108 120 0 2 M % B® O 2 Sf % 108 120
Tine (nnth) ‘Tine (nnth)
IS investrent : IS base. —t—+—+ ++ —+—+—+— dolhw/prth Overall IS quality :IS base. ++ —+—+—++ ++ +++ Dl
Figure 19: Base run - IS investment Figure 20: Base run - Overall IS quality
5.2 Adjustment on “User involvement in system development”
In this section we will demonstrate the behaviors of the system by adjusting the value of
“User involvement in system development” while keep the values of all other variables constant.
The summary of the simulation results is presented in Figure 21, 22, 23, and 24. From the results
we can conclude that as the rate of User involvement in system development increases, “Actual
system use rate”, “Actual task completion rate”, “IS investment”, and “Overall IS quality” rise up
to relatively higher equilibrium levels, while there are some oscillatory patterns on all four stocks
in the beginning of the simulation period. In addition, it seems that the change in the value of
“User involvement in system development” does not have significant effects on the final
equilibrium levels of “JS investment”, and “Overall IS quality” but have effects on how fast these
two stocks reach their final equilibrium levels.
Actual system use rate Actual task completion rate
69.46 33.15
57.07 2687
44.68 20
32.29 13.12
19.90 6243
0 2 4 3% 4 © 2 84 % 108 120 0 12 4 36 4 6 2 84 % 108 120
Tine (wath) ‘Tine (rath)
Actual systemuse rate :1S Ul pts. —+—+—+—+—+- hou/(person*nonth) Actual task completion rate : IS UL pS). ——+—+—+—+—+- task/month
Actual systemuse rate IS Ul po? ¢—e—2—2—e—2 hour(person*npnth) Actual task completion rate: IS Ul pt? 2—2—s—2—s—e— task/nonth,
‘Actual systemuse rate IS Ul 9. —3—3—8—s—8— hour(peson*month) Actual task completion rate :IS Ul pt). —3—8—s—s—s—s— task/nnonth
Figure 21: Actual system use rate Figure 22: Actual task completion rate
IS investment Overall IS quality
371,707 06
332,926 os
294,146 04
255,365 03
216,585 02
0 2 24 % 4 60 72 4 % 108 120 0 2 4 3% 8 O&O 2 SM % 108 120
‘Tine (month) ‘Tine (nonth)
IS investrent :IS UlptS. —+—+7—+—+—+ ++ dollarnpnth Overall IS quality :IS Ul pt). —+—+—+>—+-++—+++—+ Dl
IS imestrent :IS Ul p?. —2—e—e—2e—2e—s—e—2— dolhinonth Overall IS quality :1S Ul pt?) —-2—2—2—2—2 2» >» Dm
IS imestnent :IS Ulpt9 ¢—s—s—s—s—s—s—s—+ dolarinorth Overall qualty :IS Ul pt? -s—3—s—s—s—s—s—_3—s— Dl
Figure 23: IS investment Figure 24: Overall IS quality
Note: “User involvement in system development” = 0.5 in Run 1; = 0.7 in Run 2; = 0.9 in Run 3
5.3 Adjustment on “Effort on user training”
In this section we demonstrate the behaviors of the system by adjusting the value of “Effort
on user training” while hold the values of the other two exogenous variables constant. The
summary of the simulation results is presented in Figure 25, 26, 27, and 28. From the results we
can conclude that as the value of “Effort on user training” increases, “Actual system use rate”,
“Actual task completion rate”, “IS investment’, and “Overall IS quality” rise up to relatively
higher equilibrium levels, while there are some oscillatory patterns on “Actual system use rate”,
“TS investment’, and “Overall IS quality” in the beginning of the simulation period. In addition, it
seems that the change in the value of “Effort on user training” does not have significant effects on
the final equilibrium levels of “JS investment” and “Overall IS quality” but have effects on how
fast these two stocks reach their final equilibrium levels.
Actual system use rate Actual task completion rate
69.07
56.78
4448
32.19
19.90
0 2 4 3% & 60 2 8S % 108 120 0 2 4 3% 4 6 2 S % 108 120
Tine (nronth) ‘Tine (nth)
Actual systemuse rate :1S UP pS. —t—+—+—+—+ hour(persont*nonth) Actual task conpetion rate : IS UP pS —+—+—+—+—+—+ tas/nnonth,
Actual systemiuse rate :IS UT pt?) —-2—2—2—2—2 hour/(person*month) Actual task completion rate : IS UT pt? 2—2—2—2—2—2— task/inonth
Actual systemuse rate :IS Up -3—s—s—s—s— hourperson*npnih) Actual task conpletonrate :1S UT p'9. —s—s—s—s—s—s— task/tnonth
Figure 25: Actual system use rate Figure 26: Actual task completion rate
IS investment Overall IS quality
366,829 0.5531
327,560 0.4785
288,292 0.4039
249,024 0.3292
200,756 0.2546
0 12 2% 3% 4 © 2 8 9% 108 120 0 2 4 3% 8 © 2 8 % 108 120
‘Tine (tron) ‘Tie (nonth)
IS investrent :IS UT ptS. -—+—+—4+—+—++—+— dolk/inonth
IS investrent :IS UT pt?) —2-—2—2—2—s—2—s—2— dolkw/inwnth
IS imestrent :IS UT pt9) —s—s—s—3—s—s—3—-dolkw/onth
Figure 27: IS investment
Overall IS quality : 1S UT pS. +4 —+—+ +—+ Dil
Overall IS quality :IS UT p7?. —2—2—2—2—2—2—2—2—2 Dm
Overall IS qualty :1S UT x9 =—3—s—s—s—2—ss—2— Dl
Figure 28: Overall IS quality
Note: “Effort on user training” =0.5 in Run 1; =0.7 in Run 2; = 0.9 in Run 3
5.4 Adjustment on “Perceived sufficiency of organizational resources”
In this section we demonstrate the behaviors of the system by adjusting the value of
“Perceived sufficiency of organizational resources” while keep the values of the other exogenous
variables unchanged. The summary of the simulation results is presented in Figure 29, 30, 31, and
32. From the results we can conclude that as the rate of “Perceived sufficiency of organizational
resources” increases, “Actual system use rate”, “Actual task completion rate”, “IS investment”,
and “Overall IS quality” rise up to relatively higher equilibrium levels, while there are some
oscillatory patterns on “JS investment” in the beginning of the simulation period.
Actual system use rate
Actual task completion rate
48 «0
‘Tine (month)
R 8 9% 108 120 0 8 60 2 8 %
‘Tine (month)
108 120
Actual systemuse rate IS PSOR pS. +—t—+—+—+-_hour(personi*nnpnth)
‘Actual systemuse rate :IS PSOR pt? —2-—2—2—2—2 hour(persori*nrpnth)
‘Actual systemuse rate IS PSOR p99. —s—s—s—s— hour(personi*nrpnth)
Figure 29: Actual system use rate
Actual task completion rate 1S PSOR pS +#—+——+—+—+ tasl/nonth
Actual task completion rate IS PSOR pt? —-—2—2—2—2— taslnonth
Actual task completion rate 1S PSOR p'9. —s—s—s—s—s— taslvnpnth
Figure 30: Actual task completion rate
20
JS investment Overall IS quality
IM 1
750,000 0.75
500,000 0s
250,000 0.25
0 0
0 2 4 3% 4 © 2 & % 18 120 O 2 M 36 48 6 72 S& % 108
‘Tine (nth) Tine (month)
Figure 31: IS investment Figure 32: Overall IS quality
Note: “Perceived sufficiency of organizational resources” =0.5 in Run 1; =0.7 in Run 2;
= 0.9 in Run 3
6. Conclusion and Future work:
From the previous discussion, we have demonstrated the usefulness of the proposed IS
success model by showing how it can help decision makers to develop policies of information
system implementation in order to make the best use of these information systems. The model has
shown us that by making proper decisions on policies of information system implementation, such
as user training and user involvement in system development, the major reinforcement loop R1,
which is the “Benefits from the use of information systems adjustment loop”, will dominate the
behaviors of the model. As a result, organizations can facilitate the usage rate of their information
systems and in turn increase their net benefits generated from the information system usage.
Inevitably, there exist few weaknesses of the proposed model. Although the proposed model
has a solid theoretical ground, it is limited to a certain extent since it is generated mainly based on
two specific models. More efforts on literature review are expected in order to seek for relevant
concepts that can be used to reevaluate or to refine the proposed model. In addition, due to the lack
of empirical data and the time constraint at the current stage, it is difficult to either identify more
variables and stocks or to develop more detailed structures that are associated with the dynamics of
the information system implementation in organizations. Furthermore, the lack of data also makes
it difficult to formulate the variables in a way that allows the model to show behaviors that better
fit with the reality. As a result, the next step is to do a more extensive literature review and to gather
empirical data in order to develop a more persuasive and comprehensive model, and in turn
acquire more insights into the behaviors as well as the policy-making implications of information
system implementation.
21
120
PSOR pS —+—+—+ +++ + dolhi/tnonth Overall IS quality :1S PSOR pS. —4—+—+ +++ 4+ Dl
PSORp7? —-2—2—2—2—2—s—s— dolhw/ironth Overall S quulty "IS PSOR pp7. <2 222222» Dm
SORp9 -3—s—s—s—s—s—s—3- dolhwimmnth Overall S quality :18 PSOR p9 —s—s—s—s—s—3—s—s— Drm
References:
Davis, F. D. 1986. A Technology Acceptance Model for Empirically Testing New End-User
Information Systems: Theory and Results. Doctoral dissertation, Sloan School of Management,
Massachusetts Institute of Technology
Davis, F. D., Bagozzi, R. P., and Warshaw, P. R 1989. User Acceptance of Computer Technology:
A Comparison of Two Theoretical Models. Management Science 35(8): 982 — 1003
Davis, F. D. 1989. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of
Information Technology. MIS Quarterly 13(3): 319 — 339
Davis, F. D. 1993. User Acceptance of Information Technology: System Characteristics, User
Perceptions and Behavioral Impacts. International Journal of Man Machine Studies 38(3): 475 —
487
DeLone, W. H., and McLean E. R. 1992. Information Systems Success: The Quest for the
Dependent Variable. Information Systems Research 3(1): 60 — 95
DeLone, W. H., and McLean E. R. 2002. Information Systems Success Revisited. Proceedings of
the 35" Hawaii International Conference on System Sciences, January 7 - 10, Big Island, Hawaii,
US, 1-10
DeLone, W. H., and McLean E. R. 2003. The DeLone and McLean Model of Information Systems
Success: A Ten-Year Update. Journal of Management Information Systems 19(4): 9 — 30
Kettinger, W. J., and Lee, C. C. 1994. Perceived service quality and user satisfaction with the
information services function. Decision Sciences 25(5/6): 737 — 765
Pikkarainen, T, Pikkarainen, K., Karjaluoto, H., and Pahnila, S. 2004. Consumer acceptance of
online banking: an extension of the technology acceptance model. Internet Research 14(3): 224 -
235
Pitt, L. F, and Watson, R. T. 1995. Service quality: A measure of information systems
effectiveness. MIS Quarterly 19(2): 173 — 188
Succi, M. J., and Walter, Z. D. 1999. Theory of user acceptance of information technologies: an
examination of health care professionals. Proceedings of the 32™ Hawaii International
Conference on System Sciences, January 5 — 8, Maui, Hawaii, US, 1-7
Taylor, S., and Todd, P. 1995. Understanding Information Technology Usage: A Test of Competing
Models. Information Systems Research 6(2): 144 — 176
Venkatesh, V. and Davis, F D. 1996. A model of the antecedents of perceived ease of use:
development and test. Decision Sciences 27(3): 451 — 481
Yang, H., and Huang, H. J. 2004. Modeling user adoption of advanced traveler information
22
systems: a control theoretic approach for optimal endogenous growth. Transportation Research
Part C 12(3/4): 193 - 207
23
