Application of System Dynamics to Project Prioritisation
Author: Nombuso Sibeko Co-author: Corne du Plooy,
Lower Germiston Road, Rosherville, Eskom Lower Germiston Road, Rosherville, Eskom
SOC, Johannesburg, South Africa SOC, Johannesburg, South Africa
Telephone: +27 11 629 5488 Telephone: +27 11 629 5072
Email: sibekonl@ eskom.co.za Email: dplooyjo@ eskom.co.za
Abstract— This paper details the process that was followed in the development of a project
prioritisation model using system dynamics methodology. Key project prioritisation criteria for a
specific research business unit were identified. The objectives in developing the model was to be able
to prioritise mega-projects and to identify interdependencies, however, it was found that although the
tool could prioritise projects using information obtained from the project proposals,
interdependencies still required individual engagements with project team members and remains a
time intensive process. This paper will focus on the development process followed for the simulation
model.
Keywords: Causal loop Diagram, system dynamics; project prioritisation; project interdependencies
I. INTRODUCTION
Within a resource and financially constrained business environment, project prioritisation becomes a
crucial and critical step in ensuring that projects aligned with company objectives, which provide
minimal risk and satisfactory return on investment, are prioritised.
In the absence of a project prioritisation tool, decision makers will naturally defer to their mental
models in order to prioritise projects. Each decision maker’s mental model is unique, thus each person
will use their tacit knowledge and own criteria to prioritise projects. The criteria may change depending
on the project being evaluated; differing interpretations may exist as well as ambiguities and
contradictions that cannot be examined by the team of decision makers [1]. Mental models are also
subject to influence by emotions relating to the project itself or the project executor and are limited
when applied to complex problems with multiple feedback loops and time delays.
A project prioritisation tool would support a systematic approach to project prioritisation and allow the
playing field to be levelled for each project to be prioritised by catering for the following:
+ Repeatability - the same criteria are used to evaluate each project
* Reduction in bias - the projects are evaluated on their merits with respect to alignment with
company objectives
+ Removes ambiguity - clear definitions and measurement scales are assigned for each criterion.
The objective of the simulator development was to assess whether a system dynamics methodology
could be applied in developing a project prioritisation model to prioritise mega projects while
incorporating the project interdependencies and the project prioritisation mental models of the decision
makers within a research organisation. The model would serve as a decision support tool for senior
to better und 1 the sensitivities, risks and outcomes of various project prioritisation
scenarios and options, in order to make the best use of the limited organisational resources, within a
research organisation.
This paper details the process that was followed in the development of the simulator which was
modelled in iSee STELLA.
Il. METHODOLOGY
A Causal Loop Diagram
Causal loop diagrams assist in identifying the potential key driving forces of a system and their cause
and effect relationships. Figure. 1 shows the initial causal loop diagram that was developed at project
commencement.
The initial prioritisation of a set of projects is typically based on the impact the projects are expected
to have on achieving the organisational key performance indicators (KPIs), which should align with
the company objectives. The ones that have the highest expected benefit will be ranked higher; this
then determines the initial relative importance of the project. Figure. 1 analyses the forces that come
to play to reinforce or reduce the relative importance of the project.
eo
ee eet aa
Pe. inepotis neiocton
ee op Or.
Pa Relative importance eo ~~
y oy poate .
j rojoct' nick
ae ad of completion
Project's & :
Bee 5 R
of sub-projects 3
promeesceares
(oa . a
Avdlible eccurer ,
‘ OS
Resource allocation vo eb Sige 5 te S Number of
to non-aligned to ject Sub nrolects
eae and supa eC ;
3 oo
- eereneshl
Ky “Probabity of
Se project delays
a s
Management's relative pressure
Yo deliver in the shortcterm
2 ae welt, Brojoct's oxpoctod
time to completion
di time to i
benefits attainment
Figure 1: Project Prioritisation Causal Loop Diagram
Loop R1 captures the effect of resource allocation. Loop R2 looks at senior management's need to
deliver and their preference to fund and resource projects that will deliver in a short time frame. Loop
R3 traces the effect of return on investment as the gap between the expected benefit of the project and
its actual benefit decreases.
In loop R1, if the importance of a project is relatively high, the importance of its sub-projects (projects
it depends on) will also be high. The resource allocation to this project and its sub-projects should
increase. The resource allocation is bound by the company resources that are available. The large
number of resources allocated to the project and its sub-projects should ensure that the number of sub-
projects that reach completion increase, this is limited by project management abilities. As the number
of completed sub-projects increase, the project’s degree of dependency on its sub-projects will reduce
along with its risk of completion. As the risk of completion is reduced, the likelihood of the realisation
of the expected benefits increases and the gap between the project's expected benefits and its actual
benefits gets less; this in turn drives up the relative importance of the strategic project.
Loop R2 looks at senior management's need to deliver and their preference to fund and resource
projects that will deliver in a short time frame, thus as the project's expected time to benefit attainment
gets closer more resources will be assigned to the project that is highly prioritised as it aligns with
their objectives over any other quick hit projects.
In loop R3 we trace the effect of retum on investment as the gap between the expected benefit of the
project and its actual benefit decreases, the return on investment increases which results in
management feeling less pressured to deliver in the short-term (as the return on investment of the
highest prioritised project is coming to fruition) this results in lower resource allocation to less
strategically aligned quick hit projects and more allocation to the highly prioritised strategically
aligned project.
B. Prioritisation Technique
The Simple Multi-A ttribute Rating Technique (SMART) was selected as the prioritisation system.
This technique was incorporated in the Stella model. The SMART process is as follows [2]:
* Define problem
* Identify stakeholders
* Identify evaluation criteria
* Assign performance measurement methods for each criterion
* Determine a weight for each criterion
* Identify alternative projects to compare
* Evaluate the performance of each project on each criterion
* Take a weighted average of the values assigned to each project
+ Make a provisional decision
¢ Perform sensitivity analysis for final decision
This technique was chosen for ease of use; it is simple to understand, and not time intensive which is
important for those involved in the decision making process [3]. SMART is also utility-based and has
the ability to handle both quantitative and qualitative data. One of the limitations of this technique is
that it ignores the interrelationships between parameters, and the ratings of the altematives are not
relative, thus changing the number of projects evaluated will not in itself change the decision scores
of the original projects.
Steps 1 to 5 of SMART facilitated the process of capturing and unpacking each decision maker’s
project prioritisation mental model. The research managers in this business unit, are responsible for
setting the technology and research strategy. They approve and oversee existing and new projects.
Workshops were held with the research managers; where each research manager discussed the criteria
they use to prioritise projects as well as their evaluation methods for the criteria. Through these
workshops a shortlist of several key criteria was identified, namely: strategic alignment to company
objectives, project criticality, potential retum on investment, execution risk, project time frame,
project enablers, and public perception.
III. SIMULATION MODEL
The simulator prioritises a maximum of 5 projects and has a time frame of 5 years with a monthly
resolution. Projects are prioritised by assigning a score out of 100 to each project through the
application of the simple multi-attribute technique.
The scope of the model is summarised in the model boundary chart captured in Table 1. A model
boundary chart lists the key variables that are included endogenously, exogenously and excluded from
the model [4].
Table 1: Model Boundary Chart
Exogenous Variables Endogenous Variables Excluded Variables
Evaluation criteria Criteria weights Delays
Criteria ranks Prioritisation schedules Knowledge transfer
Project costs Funds allocation Human resource constraints
Project output Minimum time to benefit
interdependencies attainment
Once the criteria have been identified, the oversight committee assigns unique ranks to the criteria
and sub-criteria. The criteria are ranked from 1 to 7 where 1 indicates the most important criteria and
7 the least important.
The simulator converts the ranks to weights using the Rank Order Centroid (ROC) method [2];
equation 1. The ranks assigned should remain the same for all projects to be evaluated.
sy (1)
Where:
W; — weight of project ranked i
n—number of criteria
A. Model Overview
Figure 2 shows the system architecture map (SAM) of the simulator that details the high-level
interaction of the simulator structures.
Five projects are evaluated against each criterion and a criterion score assigned for each. The final
project scores are calculated using a weighted average for each project. Projects are prioritised in
descending order starting with the highest scoring project to the lowest scoring project.
The prioritisation schedule is adjusted for projects with a legislative or regulatory requirement through
ascaling method of the initial project scores. The algorithm uses the initial prioritisation scores before
legislative requirements are considered as well as the time to deadline to the legislative or regulatory
requirement and the project time frames. A project with a shorter legislative/regulatory deadline will
be assigned a higher score than one with a legislative/regulatory deadline that is further away. For
projects with the same time to legislative/regulatory deadline, the project time frames are used as
scaling factors to allow the project with the least slack to be prioritised higher and thus assigned a
higher score.
A second prioritisation schedule is also calculated for projects that are interdependent. This
prioritisastion is based on the output dependencies of the projects. The model will then allocate funds
to each project based on the prioritisation schedules and available funds.
Projects criteria Portfolio Prioritisation Scenario Analysis
Figure 2: System Architecture Map of Project Prioritisation Simulator
IV. TESTING AND DISCUSSION OF RESULTS
The simulator has been completed and will be calibrated using empirical data and stakeholder knowledge
on more suites of projects, besides the 5 projects within the Water Research Portfolio, which was chosen
as the initial pilot test case. The simulator was initialised by a contingent of research strategy managers.
The criteria weights calculated by the simulator are shown in Figure 3.
Four of the 5 projects had a legislative requirement with the same deadline of 28 February 2022. The
project scores were between 61.87 and 49.57. The project data for project evaluation was very similar as
the projects are from the same environment and work together in fulfilling their intended purposes.
However, there was still enough variability between projects for the simulator to determine a prioritisation
schedule. The project prioritisation schedule after legislative requirements were considered resulted in a
normalisation in ratings of the 4 projects to within the same range (62.81-62.66), the results are shown in
Figure 4 and Figure 5.
Research Manager's Critera Weights
40 37.04
30 22.76
20 15.61
10.85 7.28
10 . 442
2.04
a | =
Strategic Execution _Crrticality ROI Enablers Time Frame Public
Alignment Risk Perception
Figure 3: Criteria Weights
100, Project Scores Betore Legislative Requirements
80
61.87 m= Nesalination & water recovery
55.59 2.49 55.63
60 52.49 = Waste
40 lm System Management
30 m Water Quality
@ Water applications & treatment
oO
Projects
Figure 4: Project Prioritisation Schedule before Legislative R
Project Scores after Legislative Requirements
100
80 @ Desalination & water recovery
62.66 62.72 G?.81 67.68
53.01
m Waste
m= System Management
m Water Quality
= Water applications & treatment
Projects
Figure 5: Project Prioritisation Schedule after Legislative R
V. CONCLUSION
The development of the simulator served as a discussion vehicle that enhanced the decision-makers’
understanding of the mental models they utilise to prioritise projects. The engagements and workshops
held with the research managers allowed for the comparisons of the various decision makers’ project
prioritisation mental models. The vast differences identified through this process further emphasised the
need fora non-biased project prioritisation tool. Using system dynamics methodology the decsion makers’
in the organisation were able to reach a consenus on the relevant critical criteria for project prioritisation,
within their business unit, and to establish how they affect strategic decisions.
Initial engagements with subject matter experts validated the results from experience. The system
dynamics methodology was effective in applying prioritisation to a suite of projects.
Identifying project interdependencies still remains an onerous task and requires extensive time from
project team members.
VI. ACKNOWLEGEMENTS
The authors would like to acknowledge Dr. Simon Higgins for the initial concept of the project and
Danie Booyens and Come du Plooy for the assistance with STELLA modelling.
VIL REFERENCES
[1] J.D Sterman, “System Dynamics modelling for project management”, Sloan School of Management,
Massachusetts Institute of Technology, 1992
[2] M.B Barfod, S. Leleur, “Multi-criteria decision analysis for use in transport decision making,”
Technical University of Denmark, Second edition, 2014
[3] N.H Zardari, Z.B Y usop, K Ahmed, “Weighting Methods and their Effects on Multi-Criteria Decision
Making Model Outcomes in Water Resources Management,” Springer, 2015
[4] J.D Sterman,”In: Business Dynamics: Systems Thinking and Modeling for a Complex World.” The
McGraw-Hill Companies, Inc, pp. 97, 2000
