The contribution of system dynamics to cost benefit analysis
- a case study in planning new mental health services in the
UK
Professor Eric Wolstenholme (Director, Symmetric SD and Professor of Business
Learning, London South Bank University),
David Monk (Director, Symmetric SD)
Douglas McKelvie (Associate, Symmetric SD)
David Todd, (Director, Symmetric SD)
Registered address: Symmetric SD, 85 Church Road, Hove, BN21BV.
+44 (0)1273 811092
e.wolstenholme @symmetricsd.co.uk; d.todd@symmetricsd.co.uk
Abstract
This paper presents an example of the value that system dynamics can add to
conventional cost benefit analysis. A static cost benefit analysis is described for
planning the supply of new mental health services across the UK and the development
of this analysis into a system dynamics model is explained. By developing a bigger
picture of the issue, both upstream to where patients go after treatment and
downstream from where patients originate in the labour market, and by simulation of
the enhanced vision, the dynamic cost benefit analysis is shown to advance
understanding of the issue and plans. It questions the magnitude of the potential
benefit, introduces phasing issues, surfaces structural insights, takes account of the
dynamics of the labour market and forces linkages between the plan and other
initiatives. An overall conclusion is that dynamic factors are often left out of cost
benefit analysis simply because they cause too much complexity for decision makers,
whereas system dynamics allows these factors to be included without masking the
clarity of the case. The paper suggests that cost benefit analysis and system dynamics
are very complimentary and should be used together in strategic planning.
Please note that any views or opinions expressed are solely those of the authors and
do not necessarily represent those of any health organisation.
Introduction
There have been numerous applications of system dynamics in health in recent years.
These have covered health reform (Hirsch et al, 2005), capacity planning (Lacey,
2005; Lane et al, 2000; Royston, 1999; Taylor and Dangerfield, 2005), older people’s
services (Wolstenholme, 1993, 1996 and 1999) and disease management
(Dangerfield, and Roberts, 1999).
In particular the authors of this paper have specialised in patient flow modelling. First,
at a national level to influence government policy on delayed hospital discharges
(Wolstenholme et al, 2004a) and more recently to assist local heath and social care
communities in the UK to interpret and apply national policy frameworks for older
Copyright Symmetric SD March 2007 1
people (Wolstenholme et al, 2004 b and c) and for mental health reform
(Wolstenholme, et al 2006).
This paper addresses the application of system dynamics to enhance traditional cost
benefit analysis (CBA) applied to planning a significant increase in the number of
mental health therapists in the UK for the purpose of providing a range of new mental
health treatments (Observer, 2006). The paper describes the cost benefit analysis and
a system dynamics model developed to improve understanding of the people flows
and complexities of transition associated with the initiative. The issues and insights
raised by the dynamic analysis are presented and comments made about the role of
system dynamics modelling in this type of strategic planning.
The cost benefit hypothesis
In early 2006 plans were developed in the UK to assess the benefit of making
cognitive behavioural therapy (CBT), as recommended by the National Institute for
Clinical Excellence (NICE, 2006), widely available to treat depression and chronic
anxiety. The plan suggested that investing £600 million in 10,000 therapists would
enable the treatment/recovery of 800,000 people per year and create benefits to the
exchequer and society of over £2 billion per year when fully established. The benefits
would arise from extra output, reduced absenteeism, extra quality of life years,
savings in medical costs associated with these conditions, increased tax payments and
savings in invalidity benefits. It was known that there was an accrued backlog of
around | million people on incapacity benefit (IB) due to mental illness and the CBA
assumed that this level of investment and numbers treated might be expected to
eliminate the backlog in 7 years.
Benefit coefficients per person were derived based on well researched sources,
averaged over 6 classes of mental health condition and taking account of spontaneous
recovery rates. Cost coefficients were derived based on the costs of employing the
number of therapists required to treat this number of people per year based on the
number of people each therapist might treat. The overall net cost and benefit
coefficients were then multiplied by estimates of the number of people whom it was
thought might be possible to treat with CBT to give annual returns achievable in
steady state, once the new treatment system was established. Estimates were also
given as to where the extra therapists might be sourced.
The system dynamics model
The system dynamics model initially incorporated the derivation of the cost and
benefit coefficients per person as in the cost benefit analysis. The idea of this was to
demonstrate that the derivations were reproducible in a dynamic model and to allow
changes in assumptions within these calculations. However, as the project developed
the focus of the investigation centred more and more on modelling and questioning
assumptions about the numbers of people who might be treated based on the flow of
these people into and out of the proposed new service and the availability of therapy
capacity. This trend resulted eventually in the cost and benefit coefficients being fixed
within the model to simplify the overall structure.
Copyright Symmetric SD March 2007 2
Figure 1 shows the system dynamics perspective broken down into 3 sectors
surrounding the single state of treatment used in the cost benefit study. These are the
treatment and recovery sector — the dynamics of where people flow to, the labour
market sector — the dynamics of where people flow from and the therapist sector — the
dynamics of therapist capacity available.
The treatment and recovery sector allows for assumptions about waiting for treatment,
length of treatment, drop out rates, recovery rates, employment rates (dependent on
jobs available) and relapse rates.
The labour market sector allows for assumptions about the labour market and the
flows of people into the new treatment system from those with mental health
conditions in employment, from those out of work and claiming IB, both short term
and long term.
The therapist sector allows for assumptions about therapist recruitment, therapist
leaving and case load.
Each of these sectors raised interconnected issues.
Length of
treatment
Drop outs/
failures
Short term
1B backlog
Long term
IB backlog
Figure 1 Overview of the structure of the system dynamics model
Issues raised in the treatment and recovery sector
The main issues raised here were:
1. The distinction between treatment, recovery and employment. Although it was
stated in the CBA that there was a 30% failure rate associated with cognitive
Copyright Symmetric SD March 2007 3
behavioural therapy, it was not clear whether it was being suggested that 2.5
million people would be treated to achieve 800,000 recovered OR 800,000 would
be treated to achieve 270,000 recovered. The benefits expressed in the CBA were
based on 800,000 people recovered and back in employment, but the feasibility of
using 10000 therapists to treat 2.75 million people was very questionable since the
length of treatment achievable would be less than the minimum considered
clinically feasible.
The cost benefit analysis assumed that every one treated had a job to return to
whereas the perspective here suggested that some people (varying with location)
might not have a job to return to and that job availability and the sigma attached to
mental health treatment would conspire against people in seeking a return to work.
. The CBA did not allow for any relapses after treatment.
The focus on length of treatment and source of people led to thinking about
having a mixed rather than single programme with two treatment channels using
different types of therapists as indicated in Figure 1. The first channel would
contain a high intensity programme for those with greater need and the second a
low intensity programme for those with lesser needs. An additional consideration
here was whether people failing the low intensity treatment would ‘step up’
afterwards to receive the high intensity treatment.
Issues raised in the labour market sector
The main issues raised here were:
1.
What would be the candidate population for the new treatment and how many
people would be available for treatment? Although it was known that there was an
accrued backlog of around | million people on incapacity benefit (IB) due to
mental illness with inputs and outputs of around 260,000 per year, there was
limited data available about the overall flow of people with mental health
problems through the labour market. For example, the numbers off work due to
mental illness. Figure 2 shows a more general structure of the labour market from
which the states of Figure 1 were derived.
Copyright Symmetric SD March 2007 4
Available
For Work
Out of Work Out of Work
Well Mental Health
Problem
‘At Work Out of Work Out of Work ||| Out of Work
Well Unwell OnIB || Recovered
Out of Work
At Work Physical
Unwell Problem
‘At Work
Off Sick
Figure 2 A broader overview of the labour market
2. How did people recover from mental in illness at present? The modelling led to
thinking about and determining estimates for how long people were out of work
due to mental illness and this led to an explicit statement that the main benefit of
the new system would be to not only to increase the return to work of many more
people than at present, but also to speed up their return.The modelling also led to
the thinking that there were really two states of people claiming invalidity benefit.
A short term cohort who recovered either without treatment of from the current
limited current NHS provision and a long term cohort whose chances of returning
to work diminished the longer they were in this state.
The link to other initiatives for assisting people to return to work from both
physical and mental illness. Once the perspective of Figure 1 was developed it
became clear that there were overlaps between the proposed use of CBT and
numerous other central and local initiatives to help people back to work. Clearly
ant case for funding CBT needed to explain its how it complimented these other
ideas.
Issues raised in the therapists sector
1.
The number of therapists needed in steady state? Following on from the idea of
two service channels using two different types of therapists, the modelling
hypothesised that fewer therapists would be needed overall for a mixed
programme than for a single programme.
The build up of therapist capacity. The CBA focussed mainly on the steady state
situation after full implementation of the new system, but suggested that there
would need to be a progressive increase in therapist recruitment and training from
two sources over a seven year period. The modelling added to this thinking by
considering therapist turnover and by being able to test alternative training
policies.
Copyright Symmetric SD March 2007 5
3
. The number of therapists needed beyond seven years once the accrued backlog of
people in the labour market had been drawn down and a new equilibrium had been
established.
Model experiments
The issues raised by the modelling led to data assumptions and experimental design
for running the model of Figure | for policy testing. The model was run in each case
for 7 years in days and the range of experiments fell into 3 groups:
1.
Running the model to show the benefits when the programme was fully
operational as a comparison with the CBA for both single and mixed treatments,
with and without step up from the low intensity to high intensity channels for
some of those who failed to respond to the low intensity treatment. These runs
assumed all therapists to be in place and a constant number of people per year
presenting for treatment.
Running the model to show a phased build up to therapist capacity over 7 years
for mixed treatments, again with and without step up and for a progressive
drawdown of people from two states of the labour market (‘off work with mental
health conditions’ and those people receiving ‘short term invalidity benefit’).
Assumptions were made here about the flow of people through the labour market.
This data covered the length of time spent out of work due to mental illness and
the current rates of return to work.
Testing the sensitivity of the new programme to lapses after treatment and job
availability.
Results from running experiments to show the benefits of the ‘Fully operational’
CBT programme
A Single Programme
Figure 2 shows the number of therapists, number of people treated per annum,
number of people recovered per annum, length of treatment in days and the overall
net benefits per annum in £ billion to be expected from a full operational single
programme
Base
comparison Alternative 1 Alternative 2
with the CBA
Therapists 10,000 10,000 6,800
Numbers treated per 2,750,000 800,000 900,000
annum
Numbers recovered 800,000 270,000 450,000
per annum
Length of treatment in 44 7 49
days
Net benefits per 227 0.56 0.8
annum (E£billion)
Copyright Symmetric SD March 2007 6
Figure 2 Model results for a single programme when established
The first column on Figure 2 is a direct validity comparison with the basic CBA and
shows 10,000 therapist being used to treat 2.75 million people per year with a 30%
success rate, giving net benefits of £2.27 billion. As indicated earlier, this situation
was considered to be somewhat infeasible since a 44 day treatment time is
incompatible with the number of sessions of treatment and the associated assimilation
time needed for CBT.
The second column of Figure 2 (alternative 1) shows the much more feasible
interpretation of 10,000 therapists being used to treat 800,000 people per year with a
30% success rate. This result uses a minimum feasible figure for the average
treatment rate of 77 days, but since only 270,000 people per year recover the net
benefits per year are only 25% of those claimed by the CBA.
A slightly better return is indicated by column 3 in Figure 2 which gives results for a
further option introduced at a later date than the original CBA (alternative 2). This
option assumes an alternative service configuration capable of treating 900,000
people per year. This requires only 6800 therapist, 49 days treatment time, 900,000
people per year presenting and a 50% success rate.
A Mixed Programme
Figure 3 shows the results for a mixed programme when fully established for
alternatives | and 2 from Figure 1, for both ‘no step up’ and ‘step up’ between the
programmes. These results assume a 50/50 spilt of people between the high intensity
and low intensity programmes and a 50% failure rate from both the programmes.
Alternative 1 Alternative 1 Alternative 2 Alternative 2
no step up step up no step up step up
Therapists 4700/1500 6400/1500 3000/1500 4500/1500
high intensity/low (6200) (7900) (4500) (6000)
intensity (total)
Numbers treated per 800,000 1,000,000 900,000 1,125,000
annum
Numbers recovered 400,000 500,000 450,000 562,500
per annum
Length of treatment
High intensity/ 77/25 77/25 49/25 49/25
low intensity in days
Benefits per annum 1.0 1.2 14 1.3
(billions)
Figure 3 Model results for a mixed programme when established
Column 1 of Figure 3 shows alternative 1 using a high intensity channel with 77 days
treatment time and a low intensity channel with a 25 days treatment time. This
configuration can treat 800000 people per year and requires 4700 high intensity
therapists and 1500 low intensity therapists. The net benefits per year of this
configuration are double those from the single programme version of alternative | due
to a higher assumed success rate and fewer therapists needed in total (6,200 compared
with 10,000).
Copyright Symmetric SD March 2007 7
Column 2 of Figure 3 again shows results for the mixed programme version of
alternative 1, but this time assuming that people failing the low intensity treatment
will step up to the high intensity channel. This effectively means treating 200,000
more people per year in the high intensity programme — | million in total per year
rather than 800000. So this configuration means that 6400 high intensity therapists are
needed in total not 4700. However, the net benefits per year are increased beyond the
costs because more people are treated.
Column 3 of Figure 3 shows the results from alternative 2 using a high intensity
channel with 49 days treatment time and a low intensity channel with a 25 days
treatment time. This configuration can treat 900000 people per year and requires
fewer therapists in total compared with alternative 1 (3000 high intensity and 1500
low intensity therapists), giving improved net benefits per year over both the single
programme version of alternative 2 and the mixed programme version of alternative
1.
Column 4 of Figure 3 shows the results assuming a step up of failures from the low
intensity channel to the high intensity channel. This means treating 225,000 more
people in the programme per year — 1.125m in total rather than 900000. So this
configuration means that 4500 high intensity therapists are needed rather than 3000.
However, the net benefits are again increased because more people are treated.
Overall, the mixed programme results would appear to give higher benefits than the
single programmes and the use of ‘step up’ gives higher benefits than ‘no step up’.
Summary of first insights from the system dynamics model
Although the system dynamics model was not really used as a dynamic model at this
stage the stock- flow thinking of the model surfaced some interesting insights:
1. The differentiation between people treated and recovered. The clarification
here indicated that the original net benefit claims of the CBA were
unachievable since to attain 800,000 recovered people per year meant treating
an infeasible high number of people per year.
2. The mixed programme provides higher returns than the single programme and
requires fewer therapists.
3. Step up. The concept of step up had not really been thought through prior to
the modelling and it was not appreciated that stepping up 50% of people who
did not respond to the low intensity treatment to the high intensity channel
would require 50% more high intensity therapists.
Results from running experiments to show the benefits of a phased build up to
full operation of the CBT programme over 7 years
This section of the work actually used the system dynamics model as a dynamic
model. Figure 4 shows a similar output summary of results to Figure 3. However,
although the number of therapists is that achieved at the end of 7 years, the figures for
the number of people treated per annum, the number of people recovered per annum
Copyright Symmetric SD March 2007 8
and the overall net benefits per annum in £ billion are the average figures over 7
years.
These results are again presented with and without step up and, additionally, this time
for a progressive drawdown of people from two states of the labour market (‘off work
with mental health conditions’ and those people receiving ‘short term invalidity
benefit’).The final 3 rows show the percentage reduction in these stocks for each
situation modelled.
Assumptions were made here about the flow of people through the labour market.
This data covered the length of time spent out of work due to mental illness and the
current rates of return to work. It was assumed that the ‘mental health off work’ stock
was 1.25 million people and had an inflow per year of | million people. Also that the
‘short term IB’ stock was 0.8 million people and had an inflow of 260,000 people per
year. The calculations for these draw down figures is a truly dynamic calculation
since the draw down is in addition to the underlying normal throughputs of the stocks.
Alternative 1 Alternative 1 Alternative 2 Alternative 2
no step up step up no step up step up
Therapists 4700/1500 6400/1500 3000/1500 4500/1500
high intensity/low (6200) (7900) (4500) (6000)
intensity (total)
Average numbers 530,000 640,000 628,000 760,000
treated per annum
Average numbers 260,000 320,000 314,000 380,000
recovered per annum
Length of treatment
High intensity/ 77/25 77/25 49/25 49/25
low intensity in days
Benefits per annum 0.7 0.9 0.8 1.0
(billions)
Change in stock of
‘mental health off work -50% -50% -58% -58%
sick’ over 7 years
Change in stock of
‘short term IB backlog’ -15% -33% -15% -33%
over 7 years
Change in stock of
‘long term IB backlog’ -36% -36% -42% -42%
over 7 years
Figure 4 Model results for a mixed programme phased in over 7 years and draw down
of people from the labour market.
An important general point about the results of Figure 4 is that the overall net benefits
per year of the plan are reduced relative to the fully operational situation, due to the
progressive build up of capacity.
Copyright Symmetric SD March 2007 9
However, before discussing the Figure 4 results in detail it is useful to relate some of
the dynamic factors underpinning them. Figures 5 and 6 show example model results
supporting the results summarised in Figure 4. Figure 5 shows an example output of
the progressive build up of high and low therapy capacity and people in treatment
over 7 years. Figure 6 shows the corresponding draw down of the accrued labour
market stocks over this period.
These graphs highlight two interesting counterintuitive factors.
The first is that the ‘real’ short term IB backlog is not just the one shown in Figure 1,
but this plus those people failing to respond to CBT treatment.(Note that these people
cannot be placed back in the short term IB stock because then we would be assuming
that they could be treated again). Consequently, and particularly in the case of no step
up, as was the situation in Figure 5, the real total short term IB backlog will not be
drawn down as fast as linear, static thinking might predict. It will also start to rise
whenever therapy capacity is fully utilised.
The second factor is that although there is no direct draw down from the long term IB
backlog, this stock will decline as an indirect consequence of the drawdown of the
short term IB stock. Since this is upstream of the long term stock, drawdown of it will
reduce the flow into the long term stock.
@ te biintrestment ‘fin treatment 3: hi capacity 4: li capacity 5: hi wait
ie Raia
: Bs ey
. = rr igh intensity,
ea ra people
ee oon
1 0000 "
fe
| cs
fg low intensity peoale
. -_ = : |
:
a i oe peer
aoe me he aa
Naas 92 veo
Figure 5 Example model output showing the progressive build up of high and low therapist
capacity and people in treatment over 7 years.
@ times iaaaienll Teoh bat laa pti
aa
x Rea
3; a
ont
Pert! pre
fey a |
=
1 ‘1900000 =
Ese laa
i get Se =r
= fool of people
So || inven ere so
k ‘ aE
f : ' . cgi
foo fe
ton wes Tah ra 5D
pace Dare v1 27 Fab zy
Nae 2 unted
Figure 6 Example model output showing the draw down of labour market accrued backlogs over
7 years.
Copyright Symmetric SD March 2007 10
Returning to Figure 4, column | shows the alternative 1 mixed model results with ‘no
step up’. The ‘mental health off work sick’ backlog is reduced by 50%, the short term
IB backlog by 15% and by default the long term IB backlog is reduced by 30%.
Column 2 of Figure 4 shows the same results as column 1, but with ‘step up’. Here
the impact on the ‘short term IB’ backlog ‘ is doubled, as more people are treated per
year rather than placed back into the backlog (but obviously using more therapists).
Column 3 of Figure 4 shows the alternative 2 mixed model results with ‘no step up’.
Here short term IB backlog reductions are increased because this configuration allows
more people to be treated than in the alternative 1 mixed model and introducing step
up (column 4) permits even greater backlog reductions and more people to be treated
per year (but again obviously using more therapists).
Although these results are not directly comparable with the CBA because two
backlogs are modelled and a greater treatment rate used, it would seem that the impact
of the plan on the labour market would take some time to show significant results.
One reason for this is the treatment rate must not only be big enough to have an
impact on the backlogs but also on the underlying inflow rates into the backlogs. In
health terms it must be large enough to cope with both prevalence and incidence. In
system dynamics terms it must be large enough to cope with both stock correction and
inertia.
Summary of further insights from the system dynamics model
The simulation results from the system dynamics model created further insights about
the overall plan in addition to those given earlier:
1. The annual net benefits of the plan would be on average 30% less during the 7
year transition period to full build up than when fully established.
2. The results from the dynamic model indicate that at best with a mixed
programme and step up the impact of the CBT programme on the estimated
accrued backlogs of people out of work in the labour market would take
longer than 7 years to have a significant impact.
3. Due to the diminished effects of CBT highlighted in point (2) it would appear
very important to run all other return to work initiatives along side the new
CBT programme.
4. Structurally the modelling suggested that the best strategy would be to reduce
the expensive backlogs of claimants first and then to raw down people onto
TAPT from as far ‘upstream’ as possible in the mental health/workforce chain.
5. Step up of from the low to high intensity programmes for those people failing
the low intensity programme gives a bigger impact in reducing the accrued
labour market backlogs of people off work sick and on short term IB benefits,
but requires more therapists.
Copyright Symmetric SD March 2007 ll
6. The long term IB backlog will decline as an indirect consequence of the
drawdown of the short term IB stock.
Sensitivity results
All of the foregoing results still assume that all people being treated by the new
programme will recover, have no relapses and return to employment. The system
dynamics model allows the impact of these factors to be tested. Figure 7 Shows how
devastating the effect on the net benefits would be if 15% relapse occurred after 2
years and 50% of the recovered had no jobs to return to after 3 years. The result
assumed that new job creation would be developed to counter these trends after 4
years.
Diode! net banetits: 1-2 +
base un
= Lt bereits
‘
‘
het benefits whan
186 lapses coeur
ee se : | after 2 years, 56
jrimee Pm gl a people haven
jabs torturnto
sfter 8 years and
rex job cretion
1 1+ ieundertaken ster
Boo woe78 1a 60 101825 ct ae
Pave 1 Days tio 27 Feb 2082)
Naar ? ves
Figure 7 Example model output showing the effect of relapse and lack of job availability on the
net benefits per year of the CBT treatment programme over 7 years.
Conclusions on the content of the CBA
The use of system dynamics suggested that although generally very beneficial the
proposed plan developed in the CBA could be less than anticipated from a static
analysis. Whist the benefit coefficients derived are sound, the main loss of impact is
associated with assumptions about the flows of people. The real results would depend
on the intensity of the programmes created, the phasing of introduction, the step up
assumptions, the knowledge of the sources of people in the labour market, actions to
counter treatment lapses and particularly actions to help recovered people back into
employment.
Conclusions on the use of system dynamics to supplement CBA
It was shown possible to quickly develop a dynamic model to supplement the CBA
which allows analysis and understanding to develop well beyond the CBA.
The characteristic of traditional CBA is that enormous technical and clinical research
effort is put into determining the cost and benefit coefficients, but then these are
multiplied by very simple estimates of the underlying resource flows.
A characteristic of system dynamics is that it enormous effort is put into studying the
resource flow structure of a problem, but often little effort in converting these to flow
to financial outputs.
Copyright Symmetric SD March 2007 2
It is suggested here that dynamic cost benefit analysis combines the best of both
worlds by combining these characteristics. System dynamics can include the
derivations of the coefficients, but its major contribution is to take up where the CBA
leaves off and to apply the coefficients to an investigation of the underlying flow
structure of the issue.
It is interesting that there was some resistance initially in the project to the
development of a dynamic model of the cost benefit analysis on the grounds that this
would complicate an already complex problem and confuse decision making. In the
event the use of system dynamics enabled a more comprehensive analysis without
losing decision clarity.
It is suggested that dynamic cost benefit analysis as described here should be a
prerequisite on all planning projects and starting with a CBA is an excellent way of
showing the value of the system dynamics method.
References
Dangerfield B. and Roberts C.1999. Optimisation as a statistical estimation tool: an
example in estimating the AIDS treatment free incubation period distribution. System
Dynamics Review, Vol. 15, No. 3.
Hirsch G., Homer J., McDonnell G. and Milstein B. 2005. Achieving Health Care
Reform in the United States: Towards a Whole System Understanding, Paper
presented at the International System Dynamics Conference, Boston, USA.
Lacey P. 2005. Futures through the Eyes of a Health System Simulator, Paper
presented to the System Dynamics Conference, Boston, USA.
Lane D. C., Monefeldt C. and Rosenhead J. V. 2000. Looking in the Wrong Place for
Healthcare Improvements: A system dynamics study of an accident and emergency
department, Journal of the Operational Research Society 51(5): 518-531
National Institute for Clinical Excellence. Depression: management of depression in
primary and secondary care - NICE guidance, National Clinical Practice Guideline
Number 23, December 2004
Observer Leader, 2006, The Depression Report: a new deal for depression and
anxiety disorders. Centre for Economic Performance’s Mental Health Study Group,
the London School of Economics and Political Science, June 2006,
Roysten G., Dost A., Townsend J. and Turner H. 1999. Using System Dynamics to
help develop and implement policies and programmes in Health Care in England,
System Dynamics Review, Vol. 15, No 3.
Taylor K. and Dangerfield B. 2005 Modelling the feedback effects of reconfiguring
health services, JORS Vol.56 pp658-675.
Wolstenholme E.F. 1993 A Case Study in Community Care using Systems Thinking,
Journal of the Operational Research Society, Vol. 44, No. 9, September, pp 925-934.
Copyright Symmetric SD March 2007 13
Wolstenholme E.F. 1996. A Management Flight Simulator for Community Care, In
Enhancing Decision Making in the NHS, Ed. S. Cropper, , Open University Press,
Milton Keynes
Wolstenholme E.F. 1999 A Patient Flow Perspective of UK Health Service, System
Dynamics Review. Vol. 15, no. 3, 253-273.
Wolstenholme E.F. 2003. Towards the definition and use of a core set of archetypal
structures in system dynamics, System Dynamics Review, Vol. 19, No. 1 Spring,
2003.
Wolstenholme E. F., Monk D., Smith G. and McKelvie D. 2004a. Using System
Dynamics to Influence and Interpret Health and Social Care Policy in the UK.
Proceedings of the 2004 System Dynamics Conference, Oxford, England.
Wolstenholme E. F., Monk D., Smith G. and McKelvie D. 2004b. Using System
Dynamics in Modelling Health and Social Care Commissioning in the UK.
Proceedings of the 2004 System Dynamics Conference, Oxford, England.
Wolstenholme E. F., Monk D., Smith G. and McKelvie D. 2004c. Using System
Dynamics in Modelling Mental Health Issues in the UK. Proceedings of the 2004
System Dynamics Conference, Oxford, England.
Wolstenholme E.F, Repper D, Monk D, Todd D and McKelvie D. 2006. Reforming
Mental Health Services in the UK - Using System Dynamics to support the Design
and Implementation of a Stepped Care approach to Depression in North West
England. Proceedings of the 2006 System Dynamics Conference, Nijmegen,
Netherlands
Copyright Symmetric SD March 2007
14
