Initial Experiences of Introducing SD through a MH Project
Initial Experiences of Introducing System Dynamics
through a Mental Health project in North West England
Gill Smith — Symmetric (gill.smith@symmetricsd.co.uk)
Eric Wolstenholme — Symmetric (eric. wolstenholme@symmetricsd.co.uk)
Dean Repper — Director NIMHE North West (dean.repper@nimhenorthwest.org.uk)
Abstract
The paper describes a partnership project between three parties, centred on the use of
System Dynamics (SD) in a Mental Health Trust (MHT). The main learning
experiences relate to the politics of introducing challenging concepts in a situation
where participants prefer to avoid confrontation. The rigorous nature of SD modeling
and simulation raises questions which operational managers may fear reflect badly on
the organization and their own capabilities. They may be uncomfortable with questions
about the evidence-base for current treatments, or the research to back their ideas for
future developments. A further issue is the ambiguity inherent in definitions of SD and
the likelihood that many managers prefer single-issue projects, based on their comfort
zone of practice/expertise. A hypothesis is developed to describe the observed reactions
in the project and suggest alternative approaches in carrying out SD projects in the UK
public sector.
1. Context
A development project was set up in early 2004, involving the north west region of the
National Institute for Mental Health England (NIMHE), a Mental Health Trust (MHT)
and OLM Consulting. The basis of the partnership was that NIMHE had an interest in
developing an approach to modernization in mental health, the MHT were keen to
demonstrate their progress in this respect (eg in meeting the requirements of the
National Service Framework’) and OLM Consulting was willing to work on a research
basis, sharing risks and costs with the funding organization, (NIMHE), to facilitate the
use of System Dynamics (SD) in Mental Health.
A detailed description of the legislative context, policy directions and challenges facing
Mental Health then (and now) is given in a previous paper’. The modernization agenda
in Mental Health spawns a bewildering array of initiatives and one of the key ways in
which SD can make a contribution is to provide a whole systems model, against which
both national and local initiatives can be evaluated. For instance, separate projects for
commissioning, or introducing new services, or re-skilling staff, or tackling budget
issues can be drawn together in one discourse about current capabilities and plans for
"NSF for Mental Health (published by Department of Health in 1999) lays down various service
directions, standards and targets to be met by commissioners and providers over the following 10 years
> Using System Dynamics in Modelling Mental Health Issues in the UK: paper presented to International
SD Conference, Oxford, July 2004
Symmetric May 2005 1
Initial Experiences of Introducing SD through a MH Project
the future. This might fulfill the role of an over-arching business plan, common in the
private sector but sometimes missing in the public sector.
In the context of the north west project, the proposed discourse was indeed wide,
covering:
e Mapping the flows of clients/patients across the range of social care/health
services
e Understanding the current drivers of these flows (eg access routes, referral
patterns, eligibility criteria, clinical judgements, availability of services)
e Comparing outcomes of different care pathways (eg length of stay in different
routes, effect on level of independence in terms of subsequent placement)
e Investigating the impact of current policies (service directions, capacity,
funding) and possible future policies (eg shifts in the balance of services, use of
pooled budgets)
e Reviewing leverage points demonstrated through the SD model, in terms of
identifying sensitive areas where PIs are needed for monitoring performance and
exploring possible future intervention points
The proposed role for SD in complementing existing work in Mental Health is
described in figure | (red pathway).
Fig 1: The Role of SD in Mental Health Toolsets
Government
MH Policy NIMH(E)
| |
Modernisation Agency Strategic testing (simulation
NHS Teams of likely national scenarios)
| Priorities —> Workstreams
Service By /
Improvement Leadership, Extended toolset
new ways of working for local projects }#—_ Projects
+ shared models
mulation
Innovation & Knowledge Group gs een Service Improvements|
interventions
| + choosing best Focus for | advocacy
Research, innovation, | ,!'Y¢2e" Po!) improvement _| - improved access
Beacene ‘ ae - reduced wait times
nplementing - quality of life
powiliating - better data
Capacity and demand: - better outcomes
5000 staff trained in PDSA
Tmprovement techniques
Symmetric May 2005 2
Initial Experiences of Introducing SD through a MH Project
A model was developed to illustrate patient flows through the various agencies involved
in mental health services, with particular attention to proportions flowing down each
route and lengths of stay. The MHT and the local social services department provided
the key input. Social services were interested in tracking the effect of the new common-
access point they had introduced to channel mental health referrals. The MHT was
interested in the value of the community mental health teams in managing cases in the
community and gatekeeping access to acute facilities. At an operational level, there
were few ideas about radical shifts in service.
The modeler attempted to elicit insights into:
e Relative proportions of the vulnerable population in each service
e Common pathways and exceptions
e Assumptions driving models of working (for instance, the attitude to closing
cases/re-referral)
e Leverage points in particular services and their impact on the whole system
network
This section has described the partners to the project, the proposed scope and the
interests of each party. Section 2 describes the conduct of the project and the results, as
well as discussing the initial perceptions of risk and the emerging dynamics resulting
from the differing agendas of the partners.
2. Undertaking the Project
OLM Consulting’s first approach was to offer a template of MH services, produced a
few months earlier with informal input from various MH practitioners and managers.
The partnership, however, wanted to develop its own model and saw the process of
doing this as an essential part of the learning experience of SD.
An expert group of practitioners and managers was convened, to map the care pathways
and populate the model with data. The intention was to develop an operational model to
take to the steering group for validation, and to use this group to pose the scenarios
which would harness the model to provide the strategic benefits. Three meetings of the
expert group were held between April and June, as well as some meetings on data to
feed into the model. There was a review meeting with the steering group in August.
The modeling process is shown in figure 2. By August, progress on searching out data
had slowed (which meant an unwelcome number of iterations of the model, to re-
balance it with occasionally emerging new pieces of data), and the proposed scenario
stage was put into abeyance.
Symmetric May 2005 3
Initial Experiences of Introducing SD through a MH Project
Fig 2: Method
1. Expert group 2. Data Sources
MAPPING the Whole System connections Getting the DATA >
-what proportion are admitted to this service? “whetactual numbers enter?
-where do they come from? -how many are in there?’ «———_
-where do they go next? shaw long do they stay?
g
|
we.
a ea
lar 2 = et
Running the SIMULATION
-we have no data on... —
-this data contradicts that
3. Validation and Scenarios -that doesn't look right
-resolving issues over the model or the data
-helping interpret what the model is saying
-deciding the “what if?” experiments to run
At the current time, the model represents the flow of clients/patients through tier 1-4
services but:
e It does not have any data on capacities (which are needed to allow evaluation of
supply/demand and potential bottlenecks). Test data has been added, but this
needs validating
e It does not show any feedback loops. Some test loops have been added (eg a
low rate of crisis admissions resulting from those newly discharged from social
services or health)
Feedback loops are particularly important in establishing the links across the model
where the actions of one party may result in unintended consequences on the system
as a whole. For instance:
e Assumptions and attitudes may cause a behavioural response. An example of
this is the emphasis on the pathology of mental health (diagnosis and
treatment), which may deter patients from seeking help
e ‘Fixes’ or adaptive behaviour, instigated when the system is under pressure,
may set off a chain reaction. An example of this is that delays in assessment
may result in patients deteriorating and needing more intensive service.
Risk factors identified at the outset were:
Symmetric May 2005 4
Initial Experiences of Introducing SD through a MH Project
Objections to the abstract nature of the work (particularly the need to ‘take a
view’ of the degree of detail to be modeled)
Political hurdles to the sharing of mental models or exposing hidden
assumptions; lack of trust that all parties gain from a clear view of mutual
issues; fear that there will be ‘winners and losers’
‘Drift’ in obtaining input to the modeling process (eg delays in getting data, in
obtaining views of what has been done and still needs to be done....)
Inability to obtain a shared view of how the elements of the whole system ‘link
up’
Inability to obtain data or reach consensus on estimated values
Change of participants in the project (de-stabilising, takes time to induct new
members)
Outside events detracting from the focus (or even causing the project to be re-
defined)
Many of these risks developed into problems, and:
The modeling and data stages took longer than expected
Arranging expert group meetings was difficult and commitment to follow-
through issues was also problematic
The data was incomplete
Apart from the August review, there was no opportunity to present to the
steering group. This also meant that the ‘focus of interest’ required to guide the
modeling stage was largely absent
With hindsight, there were other risks relating to the difficulties of managing a research
partnership with several participants. The relationship of NIMHE (as a national agency,
promoting strategic approaches) and the MHT (with its own concerns about maintaining
its position in a local market) was not straight-forward. A Trust will want to be seen as
forward-thinking, and association with national initiatives is important (particularly if
they are centrally-funded), but managing budgets and ensuring long-term funding will
take precedence. The MHT will be particularly sensitive to anything which appears to
put its local reputation at risk, and this could include:
Questions which seem to challenge its grasp of operational issues (eg questions
about the rationale of who gets services and who does not, length of stay and its
relation to outcomes)
Questions which probe its insights into the future (eg relating to changing
profiles of need over a 5-10 year horizon)
Questions which relate to shifts in power (eg ‘softening the edges’ between
acute services and primary care)
Questions which might expose weaknesses (eg relating to missing data, financial
imbalances, initiatives which have not yet delivered)
Symmetric May 2005 5
Initial Experiences of Introducing SD through a MH Project
It may also be significant that the MHT was not bearing any part of the cost, other than
the opportunity cost of staff time.
NIMHE north west also had a position to ‘defend’. It was progressing a number of
strategies within the central structure of NIMHE (which has 8 regional centres), as well
as a number of initiatives with local economies. SD is potentially intrusive, in that it
questions the reason for a particular initiative in the context of the whole system view.
Any apparent conflict in priorities emerging in the modeling process might be seen as
disruptive to people’s planned work and their position as project sponsors.
For both partners, SD ran the risk of being seen as ‘just another project’ (and potentially
less important than those in the NIMHE national mainstream or relating to the MHT’s
market). At worst, it could be seen as too challenging — both in terms of the time
consumed to ‘think differently’ or gather data, and also in terms of the questions it
raised.
A further dynamic in the relationships was that NIMHE offered the MHT a free hand in
how they scoped and progressed the project. The MHT led the project, while the
NIMHE project lead maintained a ‘friendly overview’. The steering committee
consisted of MHT managers and the NIMHE project lead, but there was no prior
discussion about how to resolve issues if divergent priorities emerged and any of the
partners were unhappy about progress or direction.
This section has indicated the difficulties encountered in finding common interests
amongst the partners to an SD project. Section 3 describes some of the conflicting
interests.
3. Evaluating the Response to System Dynamics
The champion of the partnership was undoubtedly the senior manager in NIMHE north
west, who saw an opportunity to introduce new ways of thinking. He has been
supportive throughout, but is not able to spend much time on the project and has not
viewed the model or simulations. He appointed a project lead from NIMHE, who is also
supportive, but whose time is also thinly spread. His main role has been to oversee
progress and act as a conduit to the project lead in the MHT, to solicit their continued
support. It seems likely that both these key managers in NIMHE appreciated that SD
was a useful adjunct to strategic planning tools, which was justification enough for them
to support the project.
The response in the MHT is more difficult to understand. The CE of the MHT was also
able to appreciate the potential of SD and was happy to recommend its use. As a close
associate of the sponsor in NIMHE, it is likely that both had a shared view of what
could be achieved with a strategic tool. However, the operational directors of the MHT
appear to have had mixed views:
Symmetric May 2005 6
Initial Experiences of Introducing SD through a MH Project
A new project may have seemed to undermine their own projects, generally
aimed at modernizing the services. In particular, there was a cross-borough
project to demonstrate the use of data in managing performance (Intelligence in
Progress - [iP) which was portrayed as a good fit with the SD project, but
subsequently seemed to become a source of friction (since data required by the
project did not appear to be available)
They were guarded about ‘exposing too many people to the project until a
sufficiently robust model had been developed’. What this meant in practice was
that the modeling was directed by the MHT project lead, with input from
someone on the IiP and occasional participation by 1-2 operational managers. It
was difficult to stimulate the level of interest and debate needed to bring SD to
life
The question of data was seen as ‘challenging’ — the IT manager of the MHT
suggested it would be better to shelve the project for 18 months while they
carried out a proposed system implementation. Despite discussions from the
outset about running simulations based on ‘good enough’ data, there were no
managers willing or able to provide estimates (eg proportions flowing down a
route, length of stay)
Figure 3 represents the balance of what was achieved and not achieved.
Achieved Still to achieve
Fig 3: Results of the SD Project
Also interest expressed in wider
community :
people would like to see a whole
systems MH model
* involvement of MHT project |* fuller involvement — everyone
lead and a few others very busy so continuity of expert
group difficult to sustain
* oversight by NIMHE ~— useful
for QA and their ab ility to + data still needs more work
demo model within Trust
+ decisions on scope/number of
* initial model models (current model may be
used as an overview — context
* initial data for more detailed models on
specific issues/client groups?)
Need to extend relatively simplistic
model with feedback
Section 3 has described a lack of engagement in the project. Section 4 investigates some
possible causes.
Symmetric May 2005 7
Initial Experiences of Introducing SD through a MH Project
4. Lessons Learned: a Tentative Hypothesis
This section offers a number of observations on the project — some relating to mental
health, some to the project itself and some to more general issues associated with SD.
These observations are used to derive a tentative hypothesis, describing the issues in
this type of work.
Observation 1: it is easy to describe the task-based issues relating to a
project, but a terms of reference needs to look at people’s motivation,
allegiances and likely behaviour. This goes beyond vague assertions of the
importance of change management, and is not easy to discuss.
The original terms of reference for the partnership identified risks, of which the key
ones turned out to be:
e Inability to obtain data or reach consensus on estimated values
e Outside events detracting from the focus
However, there was a growing realization that the relationships in the partnership were
even more significant as factors in the outcome.
The aim of the partnership was to create a whole systems model for mental health. What
did that mean to different people?
e Something impressive to show national policy-makers?
e Something that would provide insights into useful directions for the future — to
offer guidance in terms of commissioned services and perhaps even pointers to
less tangible subjects such as attitudes and value systems?
e Something to demonstrate competence in the MHT?
e Something to act as a lever for inward investment to the MHT?
e Something to raise the profile of particular services (especially those felt to be
“Cinderella services’ and in need of championing)?
e An opportunity to develop a new skill?
If this is compared to other projects in NIMHE and the MHT, a difference emerges. The
SD project is capable of multiple interpretations: in fact, OLM Consulting described it
as providing a learning environment, which predisposes it to a certain ambiguity.
By contrast, NIMHE north west is currently engaged on specific areas of work (figure
4) and the MHT also has a number of specific projects (figure 5).
Symmetric May 2005 8
Initial Experiences of Introducing SD through a MH Project
Fig 4: NIMHE north west areas of work
Acute in-patient care .
Ageing and mental health co)
Assertive outreach .
BME issues °
Children and young people .
Commissioning °
Crisis resolution
Dual diagnosis °
Early intervention cd
Knowledge management °
Leadership cd
LIT support and development e
Mental health promotion °
Personality disorder °
.
Positive practice in psychosis
Primary care
Prison mental health care
Psychological perspectives
Recovery and values
Service improvement and
redesign
Service users and carers
Social inclusion
Social perspectives
Substance misuse
Suicide prevention
Voluntary sector issues
Women's issues
Workforce development
Example NIMHE north west project: Stepped care
e to demonstrate savings (and better outcomes) where patients enter at the
lowest tier of services and are only moved up the chain on the basis of
agreed principles
Fig 5: excerpt from MHT Clinical Governance plan: Clinical Effectiveness
Recommendation
Comments/suggested actions
acted upon when the trust reviews implementation of its
care programme approach
23 The trust needs to continue with plans to set up a clinical | 23.1 Develop/access a knowledge
effectiveness and audit sub-committee to improve management/librarian function
coordination, reporting and dissemination arrangements
across all groups that have a role in implanting
evidence-based practice
24 Plans to develop an over-arching clinical effectiveness 24.1 Implement the trust’s
strategy should be implemented effectiveness strategy
25 Service user and carer involvement should be further 25.1 Develop a user/carer CPA
developed handbook
25.2 Review carer assessments
26 Service user and staff views need to be considered and 26.1 Include service users and staff in
reviews of the implementation of the
CPA policy
Symmetric
May 2005
Initial Experiences of Introducing SD through a MH Project
27 Further training for staff to support research and 27.1 Provide further training in
evidence-based practice should be provided research and evidence-based practice
27.2 Service development groups to
develop training plans
27.3 Develop training in research
design and critical appraisal skills
27.4 Review the trust’s training
strategy and incorporate clinical
effectiveness
Example MHT project: crisis admission
e review the use of planned admissions to the ward as a part of the care plan, anda
means to avoid crisis admissions
Observation 2: NHS organisations are generally more at-ease with simple (single
issue) projects, particularly where these are related to specific areas of practice or
client groups.
It is less common to find projects relating to overall strategy. Where broader
projects exist, they tend to relate to quality, staff issues (recruitment, retention,
training), user participation or IT. Health seems relatively comfortable with these
concepts, but less able to manage the ambiguity of questions such as “Are we doing
the right thing? Are we doing it right?”
It appeared that SD introduced enquiries that were not part of the normal managerial
agenda.
For instance, questions during the mapping stage include:
e Is this care pathway complete/correct? Can people go anywhere else?
e What proportions go down this route? What assumptions are built into routing?
What clinical or social developments could change the use of this route?
¢ How long will people spend in each stage? What can affect that? What clinical
or social developments could shorten or extend the length of stay?
e Where are the pressure points? What do you do if demand exceeds capacity?
Questions during the model validation stage include:
e Does the overall pattern of movements represented in the model match what you
would expect? (eg numbers in each sector, overall change in numbers receiving
care over time)
e Does the resource usage match what you would expect? (eg the apparent over-
or under-usage of various services)
e How can we represent the ‘gains and penalties’ in the system? For instance, if
half the projected people access this service (or receive it for half the time), what
Symmetric May 2005 10
Initial Experiences of Introducing SD through a MH Project
would be the penalty? Conversely, what would be the gain if twice the people
had access, or their length of stay was doubled?
Questions posed at the outset (but reinforced in the lead-up to the scenario stage)
include:
e What scenarios do you want to test?
e Can you express those scenarios in terms of a hypothesis? (eg we are thinking of
doing X, by which we expect to achieve Y, but there are risks that the outcome
may be Z)
The response to these questions was often (initially) a silence: it did not appear that
managers normally talked in these terms and people were reluctant to express a view.
This might be followed by a discussion of exceptions to various pathways (eg eligibility
tules which de-barred dementia sufferers from some services), or enumeration of factors
which affected decisions (like severity of a patient’s condition, risk attached to their
home situation). The discussion often then moved to whether it was possible to
generalize sufficiently to justify a model. If it was pointed out that whole system
modeling requires a degree of pragmatism about the level of detail (and analogies were
given of the benefits obtained even with high levels of approximation — as in models of
climate change), then silence once more descended.
The difficulty of obtaining data was also raised frequently. If it was suggested that the
data available was already being used for strategies and plans in the organisation (so
that using them for models was no more inaccurate), then silenced again ensued. The
invitation to estimate figures which were unavailable from IT systems (eg proportions
flowing down a route) was generally not taken up.
Observation 3: managers do not tend to think in terms of ‘flows’ (how many, how
often, how long). They may assert they need more resources, but often do not justify
the request with analysis of current issues or projections of what the additional
resource will achieve (other than in general terms relating to bottlenecks or bed
usage). They are reluctant to leave their safety zones in discussing performance —
particularly when it comes to broad subjects outside their immediate remit
One of the aspects of the project was a lack of urgency. It seems unlikely that anybody
thought that either NIMHE or the MHT depended on the outcome of the project.
Observation 4: radical change is more likely when there is no alternative. At other
times, the principle of autopoiesis predominates (the organization responding to
change in a way which tends to minimize the impact and restore the status quo)
Symmetric May 2005 11
Initial Experiences of Introducing SD through a MH Project
Taken together, these observations add up to a tentative hypothesis, describing an
organisation’s response to the sort of challenge represented by SD:
When faced with challenging concepts, which may expose weaknesses in the
organisation, it is tempting for managers to marginalize attempts to find radical
solutions. The behavioural traits most likely to dominate are:
e Control — keep those involved to a minimum
e Reductionism — argue for a narrower scope
e Skepticism — express doubts in the validity or likelihood of success of the
project
e Diversion — create difficulties in finding time for project tasks
e Detachment — avoid entering into discussion on ‘difficult’ topics
Displacement — assert, for instance, that the project cannot be done unless
the IT system is improved
A search for radical solutions may only be possible where the sponsor is actively
involved and ‘won’t take no for an answer’ and where there is an acceptance that
the situation is grave and ‘something has to be done.’
This section has introduced a possible hypothesis for the difficulties encountered in the
project. It is based on the observed behaviour of middle managers in a risk-averse
organisation, when faced by difficult questions aimed at “uncovering” insights.
“Uncovering” may have negative connotations in this situation. Section 5 explores the
ways in which managers may perceive an SD project in these circumstances.
5. Developing the Hypothesis
The hypothesis depends on an assumption that SD presents a challenge to organizations.
The nature of that challenge includes the effort to embrace new concepts, the risk that it
will expose organizational weaknesses and the questions it raises about the role of
management. This section discusses each aspect of the challenge.
Embracing the concepts
SD practitioners are familiar with the problems of describing the concepts. Recent email
exchanges between members of the International SD Society throw an interesting light
on the choices of how to communicate what SD does, how much of the complexity to
reveal, and how to answer the “so what?” group of listeners.
Topic 1: why is it difficult to explain the distinct nature of SD?
“Last spring I heard a talk by James Bailey, most famously of Thinking Machines
Corp., who talked about the history of understanding. He broke it into three levels:
Symmetric May 2005 12
Initial Experiences of Introducing SD through a MH Project
religious, logical, and complex. Religious understanding attributes causality to
supernatural powers, and people described this in books. Logical, scientific, or
enlightenment understanding attributes causality to articulable laws, logics, and
mathematical relationships that once again can be described linearly in books.
While such descriptions have great power and have enjoyed great success, our
understanding of complex systems and our ability to express such systems
computationally, according to Bailey, have outstripped our ability to express or
describe them in books.
This is another way of saying that it is hard to make a complex systems-based argument
in a logical, linear, or convincing fashion. I thought this was a pretty smart
observation, and it says something about social, economic, and environmental policy
formation. Namely, as Galileo experienced, it's one thing to make a true argument
supported by observations and evidence, it's quite another to make a convincing
argument that will carry political weight”,
“It's important to remember that there are actually two components to each
debate: a set of objective questions of causality (does CO2 warm the atmosphere, and
will that wipe out polar bears?) and a set of subjective valuations (what's more
important to you, polar bears or a snowmobile?)
Models can't help much with the subjective component, but they make it easier to solve
problems by making the objective component as clear as possible, so interested parties
with different values can at least negotiate an efficient solution. The problem is that the
subjective component is complex, and parties on both sides of the debate sometimes
resort to disinformation to confuse the issue rather than addressing the fundamental
questions of values and distribution of costs and benefits. The challenge for us as
modelers is how to get at the truth and be heard in a polluted information
environment”.*
Topic 2: SD models the future, but does it predict outcomes?
“Forecasting is a risky business. Herbert Simon provided a special caution for
predictive models of social systems, the systems most likely to be modelled in system
dynamics. He observes that there are likely to be several entities making guesses —
rational expectations — about the future of such systems for the purpose of changing
outcomes in those very systems.... Simon argues that the better question (rather than
attempting forecasting) is ‘Which combination of policies is more robust and less prone
to cause problems?’. In a nutshell, he argues in favour of policy testing models when
boundedly rational expectations are part of the system to be modelled”.’
3 Corey Lofdahl: email January 6" 05
4 Tom Fiddaman: email January 6" 05
> James TI hhompson: email January 5” 05
Symmetric May 2005 13
Initial Experiences of Introducing SD through a MH Project
Topic 3: surely SD is only useful for business plans? — not for health strategies
“In the 1970s, Richard O. Foster's MIT EE MS thesis formulated a dynamic model of
glucose regulation and the onset of diabetes. The analyses proposed a causal theory of
the onset (or more precisely, showed that some prevailed theories at the time were
inconsistent with the observed behaviour in the model and real life).
(However) I would suggest that the centre of gravity of SD applications is probably
feedback systems where human decision-making is an integral part of the dynamics. a
Topic 4: can we really trust models and simulation?
“We are trying to help ordinary executives to appreciate how their strategies deliver
performance by building, developing and sustaining resources that are interdependent.
They pretty much get the idea that ‘stuff accumulates' and understand too that the
arithmetic of this is tricky... so, we explain, it makes sense to have a computer do the
tricky arithmetic for you.””
Computer models are not the object of study, they are the means of study.
While there are meaningful insights about the basic physics and chemistry of the
atmosphere to be had with pencil and paper, I don't know of any serious climate science
that can be done without computers. Whichever you use, it's hard to see how one could
proceed without a model, as the essence of science is formulating models and testing
them against data to see which work. You simply can't skip the first step (modeling), so
it's hard to see how the "modeling is not science" claim can be supported, except to the
extent that modeling is a necessary step in science, but not sufficient without also doing
the data work (verifying the map against the territory). ”
Several conclusions can be drawn from these exchanges. Firstly, these are the gurus of
SD and they often disagree about what it is, and how it should be used and
communicated. Secondly, they note that it is impossible to predict what will happen, as
decisions depend as much on subjective elements as on reasoned arguments....but we
can use SD to provide people with a rounded view of the facts. Thirdly, they are
(mostly) talking about a ‘hot topic’ — climate change, where the arguments for not doing
anything (until we are sure what the problem is) are seductive.... However, ‘wait and
® Alan Graham: email January 6" 05
7 Kim Warren: email January 6" 05
® Tom Fiddaman: email Dec 16'" 04
Symmetric May 2005 14
Initial Experiences of Introducing SD through a MH Project
see’ is only legitimate if the stakes are not too high, and if we are likely to know enough
in time to avert the problem. Arguably, there is no time to wait on climate science (nor
the future of mental health), and the consequences of inaction are dire (for both). Hence
it is particularly important to make the assertion that, whatever you think of the current
state of understanding (or the data available), it is better to try and model your best
guesses on outcomes than to wait and see. Fourthly, they are at pains to emphasise that
modeling is something we all do naturally: to make sense of a complex world, we
reduce it to symbolic elements that we can manipulate in our heads. Science does it
more rigorously; SD does complex maths more easily over time. It may be worth noting
that the usual tool for managers is the spreadsheet, and there is copious discourse
available on the relative merits of spreadsheets (a linear tool) and SD (a dynamic tool,
encompassing feedback).
In summary, the particular email discussions quoted over this 3-week period illustrate
some of the ambiguity inherent in SD and the difficulties of explaining it....unless the
listener’s ear is attuned to a vital issue and sympathetic to scientific method.
Risk of exposing organisational weaknesses; questions about the role of management
Risk operates at two levels: risk to the organization (eg of attracting unfavourable
attention from inspectors, press comment, ‘losing a star’ in the performance league
tables) and risk to individual managers. Rationally-speaking, it is unlikely that an SD
project will attract the former set of risks. Indeed there is some evidence that, at least
with inspectors, it is ‘good’ to demonstrate that the organization is undertaking some in-
depth analysis. To be even more cynical, it is often ‘good enough’ that they have
summoned in consultants to help with any aspect of performance.
So what are the risks to individual managers? They include:
e Being seen as ‘not competent’: since SD is unlikely to investigate individual
competences, the probable risk is in managers being unable to describe
operational matters (with sufficient cogency or data)
e Being seen as ‘not in control’: key risks here are (massive) over-usage or any
under-usage of services. Modest over-usage is probably seen as evidence of
‘doing a good job’; wild variations are not
e Being seen as not ‘adding value’: key risks here are that the service is not
appearing as ‘central’ in key care pathways. This can lead to managers behaving
defensively, asserting that referrals come from many sources, or that the service
is not to be confused with similar-sounding services
e Being ‘out-of-step’ with the party line on any subject (for instance, venturing a
theory or supporting a project which is not favoured by line managers)
From a cynical point of view, the manager’s best strategy is to go along with the
project, arguing passionately for the unique contribution of their own service,
Symmetric May 2005 15
Initial Experiences of Introducing SD through a MH Project
complaining slightly about the demands on their (busy) time, and largely eschewing any
kind of wider debate on futures or performance.
A particular sensitivity arises over questions about capacity. In SD terms, it is vital to
know ‘how many people we can deal with’ — otherwise we cannot tell the impact of
scenarios that increase or decrease demand. The unit of capacity may be ‘beds’ or
‘places’. For case-holding teams, however, it is a unit of productivity — how many cases
can they assess or manage? It is rare to get an answer to this question, and common to
trigger the argument of whether it is possible to generalize sufficiently to justify a
model (given, in this case, the range of complexities in cases, the sudden demands when
a case goes into crisis, the other demands on staff time etc). If the suggestion is made
that all these activities and events can in fact be modeled, to provide a ‘rich picture’ of
the demands on the team, then silence again descends. I have seen (in another
organization) a real row develop over the ‘carrying capacity’ of a particular team, where
the service head argued that they could ‘cope with anything’ and the commissioner
replied that that wasn’t good enough.
This raises another issue: what do teams do when the service is under pressure? They
are generally reluctant to talk about the ‘fixes’ they apply in order to manage the
situation, and senior managers talk somewhat glibly about ‘elastic capacity’, as if they
expected (and encouraged) a degree of over-stretching. In practice, this can mean that
performance data represents the organization ‘coping’ and is meaningless in terms of
best practice, or predictions of how the organization would manage additional demand.
This culture of ‘heroic management’ (and its impact on the ability to analyse
performance) is dealt with in depth in a further paper from OLM Consulting’.
In summary, managers often behave as if rational debate on performance (informed by
data, supplemented by estimates and enhanced by educated guesses on cause and effect)
is not part of their job. They demonstrate sensitivities to certain types of questions,
promote the importance of their own service and tend not to engage in broader debate.
This section has illustrated the difficulties of “explaining SD” and the defensiveness
exhibited by managers who feel the project puts them outside their comfort zone.
Section 6 reviews the lessons learned and suggests approaches to use in future.
6. From Hypothesis to Potential Strategies in Delivering SD
Projects
The critical success areas in a typical SD project are:
Coping but not Coping in Health and Social Care: masking the reality of running
organisations well beyond safe design capacity (in preparation, for ISDS Boston
conference, July 2005)
Symmetric May 2005 16
Initial Experiences of Introducing SD through a MH Project
Defining the purpose
Scoping the project
Setting up the infrastructure and participants
Introducing the concepts
Providing mechanisms for managing the politics
Maintaining momentum
Reinforcing the benefits
Cr
At each of these points, it is important to bear in mind the hypothesis:
When faced with challenging concepts, which may expose weaknesses in the
organisation, it is tempting for managers to marginalize attempts to find radical
solutions. The behavioural traits most likely to dominate are:
e Control — keep those involved to a minimum
e Reductionism — argue for a narrower scope
e Skepticism — express doubts in the validity or likelihood of success of the
project
¢ Diversion — create difficulties in finding time for project tasks
Detachment — avoid entering into discussion on ‘difficult’ topics
Displacement — assert, for instance, that the project cannot be done unless
the IT system is improved
A search for radical solutions may only be possible where the sponsor is actively
involved and ‘won’t take no for an answer’ and where there is an acceptance that
the situation is grave and ‘something has to be done’
The final section discusses how these critical success areas were handled in the project
and suggests alternative approaches which may be useful in future projects.
Defining the purpose
Managers may be initially attracted to SD because it offers some numbers to put behind
the business plan, or enables strategies to be dry-run to lessen risk. There is a danger
that the reassuring message belies the real work (intellectual and emotional) required to
put SD into effect.
The approach in this project was defined as:
¢ Mapping the flows of clients/patients across the range of social care/health
services
e Understanding the current drivers of these flows (eg access routes, referral
patterns, eligibility criteria, clinical judgements, availability of services)
e Comparing outcomes of different care pathways (eg length of stay in different
routes, effect on level of independence in terms of subsequent placement)
Symmetric May 2005 17
Initial Experiences of Introducing SD through a MH Project
e Investigating the impact of current policies (service directions, capacity,
funding) and possible future policies (eg shifts in the balance of services, use of
pooled budgets)
e Reviewing leverage points demonstrated through the SD model, in terms of
identifying sensitive areas where PIs are needed for monitoring performance and
exploring possible future intervention points
What did managers ‘hear’ when this was discussed? ‘Mapping flows’ may signify little
more than providing verbal descriptions, or perhaps a list of possible steps in a care
pathway. Perhaps they were unprepared for the rigours of stock-flow diagrams
supported by equations. Similarly, discussion of outcomes may mean no more to them
than making broad assertions that one model of working is ‘better’ or reduces the risk of
an adverse event. Even the term ‘whole systems’ (which is commonly used in the UK
public sector) is generally only associated with the need to ‘work together’ (rather than
carrying any implication of a scientific approach to understanding dependencies).
It is obviously important to ensure managers understand what they are undertaking with
an SD project. In the MHT project, managers were shown a completed model from
another work area and taken through the logic and data, the reason for particular
experimental runs and the interpretations of the results. What did they absorb from this?
Possibly, they saw a neat solution without appreciating that science rarely yields simple
answers without a messy journey. E=MC? did not arrive in a shrink-wrapped package.
Perhaps they saw something like an arcade game, (press the right buttons and you can
achieve the top score), without appreciating that there are no ‘right answers’ in SD —
only perspectives and options derived by patient exploration of the alternatives.
Scoping the Project
The approach described above emphasized care pathways. Initial modeling concentrated
on representing flows that operational managers could relate to (such as the numbers of
people referred to a central intake point from different sources, the different inputs to
the community mental health teams). This was also the level at which data should
(notionally) have been available.
It became clear early in the project that it would be helpful to raise the model to a more
abstract level and focus discussion on broad movements of clients/patients rather than
the detail of individual services. Starting ‘bottom-up’ reinforces the tendency to argue
the importance of individual services rather than looking at the potential to make more
significant shifts across sectors. However, it became clear that the MHT would not ‘let
go’ of the detail and the argument that ‘we can’t expose staff to the model until we are
sure it is correct’ was difficult to counter.
Symmetric May 2005 18
Initial Experiences of Introducing SD through a MH Project
Setting up the infrastructure and participants
The infrastructure of the project consisted of a steering group to oversee the project, an
expert group to participate in modeling, and project leads from NIMHE and the MHT.
The steering group was a pre-existing senior management group within the MHT, to
which the NIMHE project lead was invited. Participants for the expert group were
chosen by the MHT project lead. OLM Consulting discussed the requisites for the
expert group:
e People who know the care pathways and are able to describe performance (in
terms of actual data or “best guesses’)
e People who have ideas about the future of services (including an ability to
critique current process and practice)
e People who are willing and able to work with abstract concepts and ask ‘what
if?’ questions
The question arises again: what did managers hear when this was discussed? Perhaps
they considered that managers automatically qualified for assisting in any study relating
to their practice area, or that all managers possessed skills in critical analysis and
abstract reasoning.
Introducing the concepts
An introductory session was held for the expert group, explaining the components of an
SD model and the thinking behind it. This took approximately one hour. The session
then continued with the MHT lead describing care pathways, using a flip chart to draw
diagrams and note key features. At the next session, the OLM consultant demonstrated
the translation of these notes in the form of an initial stock-flow model, with a
commentary covering:
e What the mapping meant (people in this stage of the care pathway go to X, Y or
Z. They cannot go anywhere else. Is that right?).
e Drivers of flows (can you tell me the proportion which go down each route? The
reasons why they do? Any constraints eg capacity? The length of stay at each
point?)
e The overall ‘sense’ of the model (this says that people move in these main
directions so we need to ask about the significance....what assumptions underlie
these care pathways? what might change? what sort of questions would you like
to be able to answer?)
The intention was to continue to develop the model, populate it with data and move into
the scenario stage. However, the attendees varied at each meeting of the expert group,
so time was taken in repeating enough of the introductory material for people to make
sense of why they were there. An additional problem was that the MHT lead missed a
key meeting and it was not possible to make decisions (or progress) in their absence. It
transpired (at the review meeting) that at least one key pathway had been omitted in the
model as the MHT lead had not included it in their initial notes on the flipchart, but the
Symmetric May 2005 19
Initial Experiences of Introducing SD through a MH Project
membership of the expert group was so small (and variable) that nobody had noticed the
omission. The degree of control exercised by the MHT lead had therefore been less than
helpful.
If mapping was difficult, then obtaining data was very difficult. The culture was to
request data from IT staff, who asked for a detailed specification of the required items.
The OLM consultant explained that this was not necessary: ‘good enough’ data was the
first priority, to show patterns of movement and validate the model structure. IT staff
were not comfortable with this, so the consultant demonstrated the model and attempted
to engage the IT staff in discussion about broad areas of data which must already exist
in management reports (for instance, the total number of mental health patients, the
current usage of the day hospital). Some management reports were offered, but attempts
to elucidate the meaning of conflicting figures, or to elicit estimates where no data was
available (typically on flows, length of stay) were largely unsuccessful.
The impression was that the attendees of the expert group (as well as IT staff) were
outside their comfort zone. Estimating data or speculating on reasons for particular
flows were apparently not part of their mental model. Discussion of feedback loops was
even further removed from their sense of role.
Providing mechanisms for managing the politics
It became clear to the NIMHE project lead that expectations were not being met and he
attempted to engage his opposite number in the MHT in discussion. Questions about the
consistency and commitment of the expert group were generally met with apologies
about workload and the need to reschedule. It was not possible to ask bluntly ‘Do you
want the project to succeed? What are you prepared to do to help it succeed?’ The
NIMHE project lead called the review meeting held in August. Unexpectedly, the MHT
managers on the steering group said that the project had been very helpful. What did
they mean? Certainly, a model was produced and it had enough data in to offer some
insights into current performance. However, there had been no meaningful analysis of
the experimental runs. Was the MHT simply restating its willingness to ‘go along with’
NIMHE without offering any evidence of real commitment?
Maintaining momentum
OLM Consulting initially suggested that meetings of the expert group were held every 3
weeks, to allow time for follow-up between. However, the meetings began to slip due to
the crowded schedule of the MHT lead, and follow-up was not possible without the
cooperation of managers and IT staff. In particular, requests to expedite the collection
of data were met with apologies about workload and the difficulty of accessing data.
Suggestions that the OLM consultant be given access to the system were not answered.
Symmetric May 2005 20
Initial Experiences of Introducing SD through a MH Project
Reinforcing the benefits
The scope was reasonably detailed (covering mapping, data, scenarios, search for
leverage points etc), but this was never crystallised in terms of the expected benefits.
Although the topic was raised on several occasions, nobody in the MHT expressed a
view of what the model might help them with, other than a general statement about
informing service development.
In summary, the approach to the project assumed that the MHT would create an expert
group of people who could “rise to the challenge” of SD and that the Steering Group
would maintain an active interest and intervene to address any issues. Instead, a series
of issues arose that were “un-discussable”, rendering many areas of activity “undoable”.
The experience of this project suggests some alternative approaches in each of the
critical success areas:
e Defining the purpose: the project should have explored the motives of the
sponsor (NIMHE) and participants (the MHT and OLM Consulting) in a more
searching way. It needed to cover awkward areas (like progress, potential
conflicts of interest, who could raise and resolve issues). It needed to anticipate
the politics (see below). It is likely that ‘producing an SD model’ was a
surrogate to the real aspirations of the parties, which were probably
irreconcilable from the outset. The MHT was not offering the necessary
intellectual and emotional effort and NIMHE was not in a position to force the
degree of openness required.
e Defining the scope: greater attention should have been paid at the outset to the
appropriate level of modeling detail. It would have been better to use process-
mapping initially and introduce a more abstract SD model (which could
nevertheless be shown as deriving from the process maps) later in the project.
e Setting up the infrastructure and participants: as well as covering areas like
progress and resolving issues, the project should have retained more control on
the MHT’s choice of participants.
e Introducing the concepts: early concerns about the consistency and commitment
of the expert group should have been acted on. If this had been addressed, then
additional time could have been given to ensuring participants did ‘hear’ the
messages. Any who did not feel comfortable with extending their normal mode
of operation should then have been replaced
e Providing mechanisms for managing the politics: this was probably only
possible through direct involvement of the most senior managers in NIMHE
and the MHT. In our experience, it is relatively rare for senior public sector
managers to participate directly in projects, perhaps due to the long tradition of
delegating work. In this case, the experimental nature of the project and the
need for ‘leaps of faith’ (despite the risk of exposing organisational
weaknesses) required direct input at the highest level.
Symmetric May 2005 21
Initial Experiences of Introducing SD through a MH Project
e Maintaining momentum: someone needed to be able to say ‘progress is not
good enough’. This was difficult given the spirit of partnership in which the
project had been set up With the direct participation of the most senior
managers, lack of progress would have been seen and addressed (rather than
relying on someone to be the bearer of bad tidings)
e Reinforcing the benefits: the project needed to be associated from the outset
with a tangible use. Aspirations such as service development (or improved care
pathways or better performance management) are too nebulous.
In short, there needed to be direct involvement of the most senior managers in NIMHE
and the MHT, with a willingness to lead by example in exploring genuine matters of
significance to the future of each organization. Difficulties relating to participants,
progress, data and the risk of exposing weaknesses needed to be confronted in a firm
but helpful way. The need for “management buy-in” is often quoted in the project
management literature, but usually applies in the relatively restricted context of setting
goals, providing resources and monitoring progress. In this case, the concept of
“champions” was more appropriate. There was a need for active involvement of the
project sponsors, modeling the behaviour they expected of participants through
surfacing conflicts of interest, addressing defensive routines and demonstrating a spirit
of enquiry.
A suggestion that emerged from discussion with NIMHE was that SD should be used
initially to support ‘single issues’, rather than aiming for a whole systems model. For
instance, smaller models could be developed to illustrate variable workload in the
community mental health team or the impact of out-of-area placement on the MHT’s
budget. These smaller models could theoretically be linked at a later date. Indeed,
experience in other public sector projects shows that managers who start with a modest
aim typically come to question the outer limits of their models, and ask for them to be
extended, to track cause and effect back to more distant origins.
7. In Conclusion
The partnership project between NIMHE, the MHT and OLM Consulting has provided
a learning experience for all three. SD projects are not for the faint-hearted and more
time needs to be taken at the inception in testing the resolve of participants and
anticipating the politics of the situation. The experience may shed light on the role of
managers in the public sector. It is not enough to ‘manage’ in the sense of occupying a
role or even carrying out assigned tasks. A good manager is able to work with abstract
concepts and apply critical thinking, however uncomfortable the process, in order to
seek continual improvement. That improvement must be judged for the ‘whole system’
and may even be at the expense of their own area of control.
The hypothesis developed in the course of this project relates to the politics of
behaviour in organisations faced with challenges. SD is challenging in terms of the
Symmetric May 2005 22
Initial Experiences of Introducing SD through a MH Project
intellectual and emotional effort required to use it constructively, as well as the risk of
exposing weaknesses in individuals, services or the organization as whole. It is very
human to preserve comfort zones and avoid potentially damaging revelations, but this is
the realm of the emperor’s new clothes. Whether SD is used or not, it will be necessary
to embrace challenges. Managers will need to go beyond verbal descriptions of a care
pathway or broad assertions on outcomes. Modernisation in the public services is
closely associated with evidence-based decisions and a more rigorous approach to
business planning and operational management. However, in Lofdahl’s words
(reference 3):
“it's one thing to make a true argument supported by observations and evidence, it's
quite another to make a convincing argument that will carry political weight”
References
1. NSF for Mental Health (published by Department of Health in 1999)
2. Using System Dynamics in Modelling Mental Health Issues in the UK: paper presented to International
SD Conference, Oxford, July 2004
3. Corey Lofdahl: email SD listserv January 6" 05
4, Tom Fiddaman: email SD listserv January 6" 05
5. James Thompson: email SD listserv January 5" 05
6. Alan Graham: email SD listserv January 6" 05
7. Kim Warren: email SD listserv January 6" 05
8. Tom Fiddaman: email SD listserv Dec 16" 04
9. Coping but not Coping in Health and Social Care: masking the reality of running organisations well
beyond safe design capacity (in preparation, for ISDS Boston conference, July 2005)
Background
Zagonel, A (2002), Model Conceptualization in Group Model Building: A Review of the Literature
Exploring the Tension Between Representing Reality and Negotiating a Social Order (ISDS Conference)
Zagonel, A (2004), Developing an Interpretive Dialogue for Group Model Building (ISDS Conference)
Vennix, J.A.M. (1998), Group Model Building: Facilitating Team Learning Using System Dynamics,
New York, Brisbane, and others 1998
Richardson, G.P. and Andersen, D.F. (1995), Teamwork in Group Model Building, in: System Dynamics
Review, 11, 1995, 2, pp. 113 — 137
Otto, P. and Struben, J. (2004), Gloucester Fishery: Insights from a Group Modeling Intervention, in:
System Dynamics Review, 20, 2004, 4, pp. 287 — 312
Symmetric May 2005 23
