Some Issues in Building System Dynamics Models designed
to improve the Information Systems Investment Appraisal
Process
Michael Kennedy
Information Management and Modelling Group
School of Computing, Information Systems and Mathematics
South Bank University
Borough Road, LONDON SEI OAA, UK
Tel: (+44) 171 815 7416; Fax: (+44) 815 7499; e-mail: kennedms @sbu.ac.uk
Key words: Investment Appraisal, System Dynamics, Information Systems, Finance
Abstract
This paper examines issues in the Information Systems (IS) Investment Appraisal
process. It discusses factors that should be incorporated in a system dynamics (SD)
model designed to conduct an IS Investment Appraisal exercise.
There is evidence that organisations perceive that they are not getting a satisfactory
financial return from their IS investments.
There may be a variety of underlying reasons for this problem, ranging from
difficulties in recognising and measuring the benefits realised by the information
systems to managerial and technical failures in the IS development process, but for
this perception to have arisen, there must be problems in the techniques utilised for
evaluating IS investments.
The paper assesses the potential usefulness of SD in exploring IS Investment
Appraisal issues. A conceptual model of the 'IS Investment Appraisal Process ‘is
produced.
1. INTRODUCTION
The "traditional" investment appraisal techniques such as Payback, Accounting Rate
of Return [ARR], Net Present Value [NPV] and Internal Rate of Return [IRR], as
commonly used, are not able to measure many of the benefits offered by IS
investments that are intended to gain tactical or strategic business advantages. This is
a particular problem with those projects designed to achieve a ‘transformation’ of the
business processes.
The lack of a consensus as to the existence and importance of the problem of IS
Investment evaluation may be judged by comparing the view of Tony Cleaver as chief
executive of IBM UK [reported by Farbey et al (1995a)] that finding reliable ways of
assessing investments in information systems (IS) was the issue the government
needed to address most urgently in relation to IT with the statement of Rt. Hon.
George Young MP [reported by Ballantine et al (1995)] that "There can be no excuse
for treating investment on computers any differently from any other capital
expenditure".
Despite the evidence that IS investments can, in some circumstances, yield
competitive benefit, reported performance is very mixed [ Qureshi (1993), Porter &
Millar (1995), Ward et al (1995), Strassmann (1985), Hayes & Garvin (1982), Lincoln
(1990), Meiklejohn (1989) and Strassmann (1990)]. In the author's opinion the need
for concern in the state of IT evaluation within industry may be gauged from the
numerous instances of unsatisfactory findings from research conducted to date.
From their own and other's investigations, Remenyi et al (1991) reported the
following findings:
¢ IT is not linked to overall productivity increases.
* 70% of firms report that their IT system were not returning the company investment.
¢ IT overheads are consistently larger than anticipated.
* 31% of firms surveyed report a successful introduction of IT.
* 20% of IT spend is wasted.
* 30%-40% of IS project realise no net benefit whatsoever.
* 90% of firms did not have a systematic evaluation process.
+ 24% of firms surveyed report an above average return on capital from their IT.
These figures illustrate why managing IS/IT is one of the major business challenges.
Organisations are not able to effectively evaluate the costs and benefits of IT and so
make poor investment decisions. This is confirmed by a survey, which investigated
how 48 organisations assess the value of IT and what techniques they use to assess the
value of IT [Qureshi (1993)], which concludes that many organisations are not getting
‘value for money’ from their investments.
Organisations have a variety of reasons for investing:
* To enable them to produce a new product or service or more of an existing product
or service (expansion)
* To replace an asset that has worn out or become obsolete (Replacement)
* To cut expenditure on our current or future expenditure (Cost displacement)
* To change the method of operation (Transformation)
* To meet changes in the law or regulations (Regulatory)
It has been suggested that different investment rationales may require different
investment appraisal techniques.
Most organisations are still using "traditional" financial management investment
appraisal techniques [Hutchinson (1995)], [Weston & Copeland (1988)], such as
Payback, Accounting Rate of Return [ARR]., Net Present Value [NPV] and Internal
Rate of Return [IRR] for evaluating all IT investments [Hares & Royle (1994)],
[Remenyi et al (1991)]. It is argued that although these "traditional" investment
appraisal techniques are suitable for evaluating IT investments that automate
operations where the prime motive of the project is cost displacement they are not
suitable for evaluating IT investment that are intended to gain tactical or strategic
business advantages.
Two example IT investments will attempt to clarify this. The first example is a Payroll
System, while the second example is a Management Information System.
Example | - An organisation is installing IT to automate a process, say a payroll. This
is a cost displacement project, the reduction in wages will be compared with the initial
cost of the new system and operating cost after which cost/benefit analysis can be
applied to calculate the investment's NPV (Net Present Value). If care is taken to
include all costs and benefits then the organisation can be fairly confident of the
accuracy of the NPV.
Example 2 - An organisation wishes to install a Management Information System.
The evaluation of this investment is not as straight forward as the evaluation of the
payroll system. The system may change the structure of the entire organisation. The
main benefits will not be a saving in wages, or head count, but the value of the
information supplied by the new MIS. How much is information that is more accurate,
timely and flexible worth to the organisation? If through using the information,
management are able to reduce development cycle time what is the impact to the
organisation's financial performance? The system will facilitate organisation change
creating new ways of working so that outputs may not be comparable to those
previously produced. It is not suggested that we should ignore financial analysis but it
is argued that conventional accounting systems have great difficulty in capturing the
benefits in financial terms.
It is argued that because one of the major benefits offered by a MIS is gained by an
increase in the quality of the decision making within the organisation it can not be
evaluated effectively by techniques that only consider quantitative and financial data.
For IT investments that are designed to gain strategic advantage the benefits are likely
to be even more difficult to measure than those of an MIS because of the increased
number of external and internal factors which are involved.
The main problem lies in the effective measurement and quantification of benefits in
practice, as described in section 2 below.
This paper will suggest that organisations that are using narrowly focused evaluation
techniques are missing opportunities, while taking on large risks and costs that can be
avoided through the use of appropriate and effective evaluation techniques.
2. ELEMENTS OF AN INVESTMENT EVALUATION:
* Cost
+ Benefits
¢ Risks
* Flexibility
2.1 Costs
The conceptual problems with the costs of IT projects are mainly concerned with the
identification, categorisation and measurement of "indirect costs'. IT costs are
consistently larger than anticipated and overheads, especially in user departments, are
frequently understated [Remenyi et al (1991), Willcocks (1992), Earl (1989) and
Hochstrasser (1992)].
Wilcocks (1992a) identifies some problems with specifying and measuring costs for
evaluating IT Projects:
* Understating human and organisation costs
* Overstating costs
* Problem of apportioning costs to user of the system
2.2 Benefits
Perhaps the most serious problem with traditional methods is their inadequate
treatment of the benefits of IS developments. There are conceptual problems with the
identification and measurement of ‘intangible benefits' [Remenyi et al (1991), Ward et
al (1995) and Hochstrasser (1992)].
Remenyi et al (1991) identify the following types of Benefits:
* Regulatory Compliance
+ Financial Benefits
* Quality of service Benefits
* Customer perception Benefits
¢ Internal Management Benefits
+ Dis - Benefits
Ward et al (1995) suggest that anticipated benefits from a project are often not
realised. This is because , in many organisations, no steps are taken to ensure that
expected benefits materialised. A benefits management programme is therefore
required. They suggest a model :
¢ Identifying and Structuring benefits
+ Planning benefits Realisation
¢ Execution of the realisation plan
¢ Evaluating the results of post implementation review - potential for future benefits
2.3 Risk Evaluation
The author would suggest a risk management programme comprising:
¢ Risk identification - recognise risks of project
+ Impact Assessment - Quantification of the damage / loss if the adverse risk occurs
* Probability - What is the likelihood of the event happening
* Avoidance - What steps can we take to minimise the changes of the event
2.4 Flexibility
Flexibility is the degree to which a project is able to adapt to uncertain issues at the
time it was being planned. Hares & Royle (1994) identify three base and two support
Flexibilities:
Base Flexibilities
¢ Product or Service
¢ Volume of business
¢ Robustness
Support Flexibilities
¢ Organisation
¢ Technology
3. IT PROJECT TYPES
It has been suggested that IT investments should be analysed into different categories
of IT Project Types that have differences which are as a result of both its
characteristics and how it is perceived by management. Appropriate investment
methodologies may then be selected for each IT Project Type.
Remenyi et al (1991) categorise IT investment as:
* Mandatory / obligatory
¢ Investment to improve performance
¢ Investments to gain competitive edge
¢ Infrastructure investment
¢ Research investment
Wilcocks (1992a) however categorised IT investments into six namely:
+ Effectiveness
¢ Architecture
¢ Research and development
¢ Competitive edge
« Must do
Efficiency
Other categorisations include Hochstrasser's (1990) who listed eight categories but
Farbey et al (1995a) have gone one step further with their categorisation by including
the ranking of each project type in what is described as a project ladder:
«Rung 8: Business transformation
«Rung 7: Strategic systems
«Rung 6: Inter - organisational systems
«Rung 5: Infrastructure
*Rung4 MIS and DSS systems
¢Rung3 Direct value added
¢Rung2 Automation
¢Rung1 Mandatory change
4. IMPORTANT SELECTION CRITERIA.
For any technique to be accepted in this domain, it needs the following attributes:
¢ A technique which takes account of all important aspects of company performance - not just
short run financial returns.
¢ A technique which offers indicators of future as well as past performance - financial figures
are held to be poor on the former.
¢ A technique which takes account (albeit with difficulty) of intangible costs & benefits - the
traditional view is to accurately measure that which can be done easily & ignore the rest.
These subsidiary criteria may also be important:
¢ A track record - a tried and tested approach is likely to be received much more readily than
an untried idea.
¢ Inherent simplicity - busy executives are more likely to apply a straightforward approach
Wilcocks’ (1992a) Evaluation Guidelines:
¢ Link evaluation across stages & time
5.
¢ Flexibility - each organisation is unique in terms of its strategy, its competitive position, and
other key criteria and will need an approach which is able to accommodate such diversity.
¢ Measurability - an approach where the information required is obtainable is eminently more
than one which needs considerable time and effort to master.
useful than one which requires figures which are impractical to obtain.
¢ Involve key stakeholders in evaluation at all stages
¢ Assess the actual against the planned impact of IT
¢ Evaluate & re-evaluate at all stages of the project
¢ The concept of learning should be central to the evaluation proces:
adequate techniques may reveal a ‘quick-fix' orientation; in the long run getting it right
may prove more difficult but add greater value
INVESTIGATING EVALUATION TECHNIQUES
Introduction
Table | lists a selection of evaluation techniques that have been suggested by their
proponents as possible solutions to the problems described in the introduction. The
. The clamour for
suggestions embrace both academic and practitioner techniques. They were described
and evaluated in Kennedy (1996).
Table 1: A Summary of the Sixteen Evaluation Techniques Examined.
Tech.
[Technique Name
Proponent(s)/Ref,
Quant
[Qual
tative
tative
[Numeric Techniques
1 [Adapting DCF (Discounted Cash Flow) Kaplan (1986) .
2 [Calculating the new value of IT. Kent (1991) :
3[IVAN Primrose (1989) .
4[Return-On-Management Strassmann (1990) .
[Non - numeric Techniques
IIT Investment Mapping
Peters cited in Willcocks (1992)
[Guidelines for surviving without numbers
[Clemons cited Freedman (1990)
[Modelling/ Management Science Techniques
[Process Flight Simulation
Rubin (1994)
[Multi Objective , Multi Criteria
Kenny & Raiffa (1976)
[Value Analysis
[Melone & Wharton (1982)
[Eclectic Approaches
[Information Economics
Parker, Benson & Trainor (1988)
Kobler Unit
[Beat Hochstrasser (1992)
[The Balanced Scorecard
[Kaplan & Norton (1992)
[Business Wide Value Model
Esther & Brooks (1995)
[Framework Approaches
[Enterprisewide Information Economics
Parker & Benson (1992)
[Guidelines for assessing the strategic and
[Banker, Kauffman
leconomic value of IT
[& Mohmood (1993)
BSI
Business Science Internat.
[cited in Freedman (1990)
Particular attention is paid to the perspective(s) from which the investment is
evaluated by the technique, the technique's application area, and whether the
techniques seek to evaluate the proposed project in a quantitative and/ or qualitative
manner.
6. Why System Dynamics
The potential combination of SD to IS/IT management appears to lie in five areas:
Firstly, at the most basic level, in some cases it may be possible to replicate existing
models developed originally using other modelling styles or techniques. In this
domain this is normally spreadsheets. The author has described various replications of
this sort (Kennedy 1997(a), Kennedy 1997(b)). This proc may be of value in
convincing managers that they are not losing desirable aspects of their current
systems, in better handling any dynamic behaviour incorporated in the previous
model, in building confidence in SD models and in giving some secondary benefits
such as better documentation, but it will not generally realise the full potential of SD.
In some cases it may allow for the incorporation of other factors (e.g. intangible
benefits) that were not incorporated before, or any dynamic behaviour known of but
not previously incorporated, in an enhanced model. It could also form the basis for a
more radical reconstruction incorporating tried and tested elements of the previous
{non SD] model.
Secondly, it should be possible to produce SD models of some of the methods
mentioned in this paper. This would aid in their evaluation and implementation.
Thirdly, it is possible to construct new models to suit the business environment being
examined. A prototype example, (Figure 1) follows, as a basis for discussion. The
requirements for such a model need to be identified and analysed effectively,
particularly as requirements change over time, as management changes its decision
styles, and staff need new operational data and information.
Potential benefits
Return on management
Degree of Business + Project Goals
and IS integration’
Project performance
+ No project perceived
Managerial expertise in eeding funding
planning IT projects .
Le
capacity for managers to learn
Criticality in achieving
Business goals
+
Management perception
of importance of project
Importance of project i
to business manager's competence
+ use of architypes Total investment
- Projectlife funding desired
Management committment
+
ia Hurdle return needed for
Willingness to allocate project to be accepted
funds to the project
Figure 1: High Level Dynamic Hypothesis of IS/ IT investments
Fourthly, we may develop models of a business showing business processes before
and after a proposed process change. The anticipated value of the benefits derived,(in
terms of greater revenues, resources saved or perceived improvements in quality or
reputation), can be compared to the estimated costs. This would be of considerable
value in “Transformation” type projects.
Fifthly, we may develop “"Process Flight Simulators". The concept is that a dynamic
model is built of an organisation which allows managers to simulate and study
situations before encountering them in reality and so deepen their understanding of the
organisation and the likely impact of policies and decisions. Rubin (1994) describes
the use of "Process Flight Simulation", using SD techniques in this domain. He
describes the construction of a dynamic model of an organisation. This model may
consist of processes, events, patterns of behaviour, structures and information
feedback flows. Once managers are confident that they have developed a satisfactory
model of their organisation, they can simulate a wide variety of business
circumstances and scenarios.
7. Conclusion
SD has a direct, but yet to be realised, application for evaluating IT investments. It
may be possible to implement several of the evaluation approaches examined in this
paper within a system dynamics modelling tool so that when an IT investment is
proposed, management could simulate its likely effect within the organisation.
Additionally, an organisation could consider cost, time and human resources which
would enable the model to estimate intangibles costs that traditional cost/benefit
analysis can not measure.
The introduction outlined some difficulties with the current methodologies but there
may be problems with the alternative methods described above:
* Over-complexity - while strategic planning departments may well be at ease with
some of the more involved techniques, others are less likely to find them appealing or
practicable.
+ Untried - some potentially promising ideas have yet to be applied successfully in
organisations; without a track record many concerns are not prepared to entertain such
approaches.
Qureshi's (1993) survey, that investigated 48 organisations and their attitude towards
IT, found that organisations’ difficulty in assessing intangible benefits was
exacerbated by the culture barrier existing between business and IT managers. The
survey also uncovered some interesting information on the on how organisations’
business and IT plans are linked While 60% of business had separate business and IT
plans, 67% of the IT plans were de-coupled from the business plans. SD, though an
increased understanding of structure and process, may facilitate a closer “alignment.
The conclusion I would draw from the investigation of these evaluation techniques is
that, as yet, no single technique is suitable for evaluating all IT investments. The
evaluators should choose the most appropriate technique on the basis of the
investment's application area and the perspective from which they hope to gain
advantage. The matching of categories of IT investments to evaluation methods is
problematic but of fundamental importance. Farbey et al (1995b), Hochstrasser (1990)
and Willcocks (1992) have offered valuable suggestions but the search for a definitive
linkage methodology continues. From this paper I would contend that the various
potential ways that SD may add to the evaluator’s armoury should be further
investigated.
Acknowledgements
The assistance of Ddembe Williams, Steve Fisher, Neil Bardsley and Ade Samuel in
preparing this paper is gratefully acknowledged.
8. References
Ballantine J., Galliers R.D., Powell P.L. (1995) Daring to be Different : Capital
appraisal and Technology Investments. Proceedings of the Third ECIS
Banker R.D. , Kauffman R.J. , Mahmood M.O.(1993) Measuring the Business Value
of IT; in - "Strategic Information Technology Management: Perspectives on
Organisational Growth and Competitive Advantage". Idea Group Publishing
Earl M. (1989) Management Strategies for Information Technology. Prentice Hall.
Esther J.A.K. , Brooke C. (1995) A business- wide value model for evaluating IT
investments. Proceedings European Conference on IT Investment Evaluation.
Farbey B., Land F., Targett D. (1995a) IT investment methods and practice.
Management Today July.
Farbey B., Land F., Targett D. (1995b) System- Method- Context. Proceedings
European Conference on IT Investment Evaluation,
Freedman D. (1990) The ROI Polloi. CIO - April
Hares J., Royle D. (1994) Measuring the value of IT. John Wiley & Sons.
Hayes R.H., Garvin D.A. (1982) Managing as if Tomorrow Mattered. Harvard
Business Review, May-June
Hochstrasser B. (1992) Justifying IT Investment. AIS
Hochstrasser B. (1990) Evaluating IT Investment - Matching Techniques. Journal of
Information Technology .
Hutchinson R. (1995) Corporate Finance. Stanley Thornes.
Kaplan R.S. (1986) Must CIM Be Justified By Faith Alone?. Harvard Business
Review Apr.
Kaplan R.S., Norton D.P. (1992) The Balance Scorecard - Measures That Drive
Performance. Harvard Business Review - Jan. -Feb.
Kennedy, M., (1996) Investment Appraisal of information Systems Projects:
Alternative Methodologies Explored., Financial Information Systems Conference,
Sheffield Hallam University UK.
Kennedy M., (1997a), Transforming Spreadsheets into System Dynamics Models;
Proceedings of 15th System Dynamics Conference, Istanbul, Turkey.
Kennedy M., (1997b), The transformation of a Power Plant Investment Appraisal
Application from a Spreadsheets into a System Dynamics Model; Proceedings of 15th
System Dynamics Conference, Istanbul, Turkey.
Kenny R.L. and Raiffa .H. (1976) Decision with Multiple Objectives: Preferences
value trade-offs. Wiley.
Kent A. (1991) How Much Such You Spend on Information Technology ? Australian
Accountant - Aug.
Lincoln. T. [Ed.] (1990) Managing Information Systems for Profit. John Wiley.
Meiklejohn I. (1989) Computing Matters : Making IT Work to Best Effect.
Management Today - Jan
Melone N.P., Wharton T.J (1982)Strategies for MIS Project Selection. Journal of
System Management 32 No2 pp. 26-37
Parker M.J. , Benson R.J. (1992) Enterprisewide Information Economics Latest
Concepts. Journal of Information Systems Management - Fall
Parker M.M. , Benson. R.S. & Trainor H.E. (1988) Information Economics. Prentice
Hall.
Porter M.E., Millar V,E.(1985) How Information Gives You Competitive Advantage.
Harvard Business Review - Jul.-Aug.
Primrose P.L. (1989) AMT Investment And Cost Systems. Management Accounting
Oct.
Qureshi M. (1993) Calculating Returns On IT Investment. Management Consulting.
Remenyi D.S. J. , Money A ., A Twite A. (1991) A Guide To Measuring And
Managing It Benefits. NCC Blackwell.
Rubin H.A., Johnson M., Yourdon E. (1994) Using software Process "Flight
Simulation" to predict the impact of improvements in process maturity.
Strassmann P. (1990) The Business Value Of Computers. Information Economics
Press.
Strassmann P. (1985) Information Payoff. The Free Press
Ward J. Griffiths P., Whitmore P. (1990) Strategic Planning for Information Systems.
John Wiley.
Ward J., Taylor P., Bond B. (1995) Identification , Realisation and Measurement of
ITS Benefits: An empirical study of current practice. Proceedings European
Conference on IT Investment Evaluation.
Weston J.F., Copeland T.E. (1988) Managerial Finance. Cassell.
Willcocks L.(1992) IT Evaluation : Managing The Catch 22. European Management
Journal vol. 10 No 2 June.
