Performance Evaluation of Management Information Systems in
Clinical Trials: A System Dynamics Approach
Céline Bérard
Doctoral Candidate
School of Management
University of Quebec at Montreal
berard.celine@ courrier.ugam.ca
L. Martin Cloutier
Department of Management and Technology
School of Management
University of Quebec at Montreal
315 Ste Catherine East
Montreal, Quebec, H2X 3X2 Canada
Tel. (514) 987 3000, ext. 3732# Fax (514) 987 3343
cloutier.martin@ ugam.ca
Luc Cassivi
Department of Management and Technology
School of Management
University of Quebec at Montreal
315 Ste Catherine East
Montreal, Quebec, H2X 3X2 Canada
Tel. (514) 987 3000, ext. 4084#
cassivi.luc@ ugam.ca
Abstract
The complexity and characteristics of the pharmaceutical firm present an intriguing context for
underlying information management issues during clinical trials for new drug development. This
paper reports on the evaluation and performance of MIS for information management in clinical
trials in new drug development. The main objective of the study is to examine the economic and
business impacts of automating that process, to enhance our understanding of informational
stakes involved, using a system dynamics (SD) model. The SD method is enriched in this paper
with other conceptual frameworks such as Alter’s (2001) Work Centered Analysis (WCA) and the
activity diagram offered by the Unified Modeling Language (UML). Results of the simulations for
alternative sensitivity analyses on errors rates in data transmissions, that is, on alternative error-
rate specifications, do not necessarily influence project delay, but rather work intensity. A
discussion details the usefulness of enriching the SD modeling process with alternative
conceptual frameworks in the problem definition in such complex settings.
Keywords: information systems, information management, clinical trials, project management,
performance measurement.
Introduction
The pharmaceutical industry, one of the largest manufacturing sectors worldwide with annual
sales of US$ 300 billions, is well known for its intense R&D spending (Santos, 2003). Even
though its activities are focused on the research, the development and the commercialization of
new drugs, time-to-market is slow (somewhere between 8 to 12 years) and expensive (about US$
800 millions per new drug), in part, due to the complexity of the process and heavy regulation.
Several steps are required for a new drug to reach the market. The process involves a series of
trials, known as pre-clinical and clinical trials required for approving and commercializing a new
drug. These clinical trials are divided into four main phases (Alshawi, 2003). The phase I
involves tests on an important number of voluntary healthy patients. The effectiveness and side
effects are then assessed on these voluntary patients in phase II clinical trials. The purpose of
phase III clinical trials is to verify and confirm results of this assessment on a larger number of
voluntary patients. Finally, in phase IV trials, a regulatory agency (such as the Food and Drug
Association (FDA) in the US, or Health Canada, in Canada) reviews and approves the
development process and the final product (drug) for commercialization.
Knowledge and information management are key to success for firms involved in clinical trial
studies. Pharmaceutical firms, looking to lower R&D costs and time-to-market, tum to
information systems (IS) to improve the management and guarantee the flow of information
during the many clinical trial phases. Thus, automating the clinical trial process through IS is a
means for these firms to meet project performance goals, that is to keep cost, time delay, and data
quality in check, notably during the more extensive clinical trial phases II, III, and IV, where
many events take place for which a large quantity of data are gathered, processed, and analyzed.
This paper looks into measuring how IS can improve the execution of a complex information
management process within the R&D process of clinical trials for new drug development. The
objective of this research is to highlight the impact of automating information management
during clinical trials through the use of a system dynamics (SD) model. The results generated by
the model can help firms involved in that process determine the economic value associated with
software investment decisions, in conjunction with their business objective. The remainder of the
paper is organized as follows. The next section introduces some issues associated with IT and
health care. Then, the research methods followed during the course of this research are presented.
An emphasis is placed on the research steps followed. The model is applied and some sensitivity
analyses are conducted and illustrative results are presented. Finally, a conclusion follows where
some learning points and the contribution of the research are emphasized.
IT and health care
Information technology (IT) is becoming ubiquitous to health care and related activities. For
instance, Anderson (2002) analyzes the impact of information systems (IS) on the organizational
context in which it is implemented, notably on health (medical) services. These highly dedicated
IS have enabled medical decision-making in hospital settings, and also have helped uncover and
lower the number of medical errors.
Augen (2002) emphasizes the important role of IT in the drug discovery process. Over the past
25 years, the drug discovery process, coupled with IT, has evolved to become inseparable
components of the product life cycle, or of the “drug discovery pipeline” (Augen, 2002). Hence,
the contribution of IT to the drug discovery process is a critical strategic element for firms
involved in these activities. Information management during clinical trials relies on specific
software, technology that often are critical to the economic and business livelihood of that
process due to the complexity and hurdles associated with the project management of clinical
trials and strict deadlines associated with the process (Rowe et al., 2002). The negative impact of
such time delays on the drug discovery process is easily foreseen, as firms want to rush the
product to market. Furthermore, stricter regulation has pushed, over the last ten years, to double
the number of patients required to complete a clinical trial study (Rowe et al. 2002), thus
increasing data management requirements. These constraints have pushed firms towards
automating data extraction and processing with the end goal of reducing time delays, costs and
information risks.
Over the past 20 years, the pharmaceutical industry has witnessed a large portion of its clinical
trial activities outsourced to specialized clinical research organizations (CROs), more specifically
to conduct phases II to IV of clinical trials. Large pharmaceutical firms, that outsource some of
these activities, are more able to focus on their core capabilities, to gain in flexibility and
effectiveness in their R&D activities, to improve internal capabilities, and to gain superior access
to technology, knowledge and competencies (Piachaud, 2002). The number of CROs worldwide
is estimated to be about 1,300 firms and the market value of these activities at US$ 8.8 billion
(Piachaud, 2002). This is a support activity to an industry that has experienced a 20% growth rate
over the last few years (Azoulay, 2003).
In health care related activities, most studies that use SD address specific problems, such as: the
diffusion of medical technology, the flow of patients in an establishment, etc. (McDonnell, 2004).
Anderson (2002), who contributed significantly to this field, has shown how with many
simulation methods, it is possible to represent and evaluate decision-making processes using IS in
a healthcare environment. Indeed, the health care system is complex, and issues associated with it
should be treated accordingly (Gloubermann and Zimmerman, 2002). Researchers in this field
are hence increasingly tackling research questions related to health care management with
flexible approached such as SD. As stressed by McDaniel and Driebe (2001), several
characteristics of health care organizations are amenable to this type of analysis using approaches
such as the ones found in complexity science. This interdisciplinary approach, based on the
notion of nonlinear dynamics, and on the management of interactions/feedback loops, enables a
better understanding and the potential for learning about complex systems (Sterman, 2000). Other
authors also insist on the benefits of simulation, one of the quantitative methods contiguous to the
approach (Wiendahl and Worbs, 2003; Morecroft et al., 1991). Anderson (2002) notes that with
SD, changes envisioned to a system, or process improvement, may be tested risk-free before
engaging or continuing on a given project effort (Davidson, 1994). The future behaviour of a
system structure is thus anticipated under various conditions without disturbing the actual system.
According to Wiendahl and Worbs (2003: 30), computer simulation “offers the possibility to
describe and analyze the behavior of existing systems under changing conditions and different
parameters”. These methods facilitate the generation of quality data for scenario building
(Wiendahl and Worhbs, 2003). This set of principles and methods have long been used to evaluate
the performance of IS (Wolstenholme et al., 1993).
Several studies focus on IS use in health care, and very many few have used SD methods.
However, given that drug development and clinical trials are an extreme case of information flow
management, the paucity of IS research in this area using SD is surprising
Research method
The management of information intensive processes carried out during clinical trials can be
generalized through a dynamic set of interrelations in a specific timeframe. This process is
basically akin for all firms in a clinical trial context, and similar constraints (rules and norms)
pressure information management within these firms. In order to support the management of
information, alternative software technology is available on the market. This research focuses on
a case study in which a generic information management simulation model was designed. The
objective of the model is to represent in a generic structure the results of this research, so that it
can be applied to other organizations involved in clinical trials.
The case study was realized within a contract research organization (CRO) in the pharmaceutical
sector. A Canadian-based CRO has contributed to this research effort by making available all
expertise and data required to develop a SD model, and more importantly, by allowing one of the
team’s researchers to participate in the day-to-day activities of the firm to gain an intimate
understanding of the work methods, information management needs, business objectives; and to
provide data on information management activities and processes. The research context provided
by the CRO was appropriate due to the fact the firm, at the time, faced an IT investment dilemma
in its clinical trial infrastructure. The automation level of its processes being limited, the firm
could not address its market growth potential, given available resources. Semi-structured
interviews were conducted at every two or three weeks with the three main respondents who
participated in the data collection effort over a seven-month period: the business manager, the IT
manager and the financial officer. The data collected were synthesized from the firm’s experience
with four previously completed projects. These projects were conducted for Phase IV clinical
trials, the most frequent type of projects conducted by the CRO. Moreover, a secondary
information source was required from software providers, on the availability of different product
offerings; as well as their characteristics and functionalities. The analysis of several documents
and brochures was first conducted to understand product availability on the market. To obtain
more detailed information, this documentation was further enriched by attending product
demonstration and holding meetings with software providers such as ClinTrial, Datafax, Oracle
Clinical and Teleform. The results from the case study along with the analysis of the IS were
used to elaborate the SD model. The complexity of the information management process, and of
the context in which it is carried out, prompted the use an approach which is more structured than
the ones typically suggested in the SD literature. The framework employed in the study is
generically based on Sterman’s (2000) “Steps of the Modeling Process” to structure the research
sequence of the study. This provided a generic research framework that allowed to stock and
manage the information required for the development/evaluation of the level-rate model.
In a first step, the definition of the problem and model’s objectives were clarified. The
information management process was represented using the Unified Modeling Language (UML).
The actual process represents the case of low IT support, and the improved work process, that is
with the use of a performing IS, was represented using activity diagrams. Then, to develop or
improve the IS supporting the process within the firm, Alter’s (2001) Work Centered Analysis
(WCA) was applied to analyze the process. This approach has allowed for a better understanding
of the business process to be automated, while providing a link between the system and the
information treatment service offered to clients. Five perspectives were treated, with respect to
the WCA framework: clients, products and services, business processes, clinical trial participants
and IT infrastructure.
The second step involved the development of influence diagrams that represent the dynamic
hypothesis of the interacting feedback loops within the information management process. The
third step included the design and calibration of the SD level-rate model using the Powersim
software. The level-rate model was then assessed, in a fourth step, by evaluating whether its
behavior over time was similar to the real system's behavior. The fifth and final step focused on
the scenario building to analyze future business strategies with the analysis for software decision.
Information management in clinical trials
The information management process for clinical trials amounts to a set of information exchanges
between the organization in charge of clinical trials, physicians who meet with patients, and the
regulatory agency. Clinical trials involve the testing of new drugs on a certain number of patients
in good health or matching a specific criterion. The firm conducting clinical trials receives forms
that contain data relative to clinical tests that it treats and analyzes. It is important to note the data
can be collected intemally, using a call center, or can originate from physicians external to the
firm. The information to treat is mainly extemal to the firm and the data treatment process differs
little between clinical trial phases.
The first important step of that process is the definition of a clinical trial protocol. This is a
document that determines the basis of the clinical study to execute and indicates the number of
physicians to recruit, the number of patients to enroll, the number of visits for each patient, the
contents of data fields to fill up on the form, etc. When physicians are recruited and the forms
(questionnaires) are ready for distribution to physicians, the study can begin. The forms, referred
to as Case Report Forms (CRF), are then completed by the physicians recruited following a
patient's “visit”. These forms contain entry fields such as the name, blood or other test results,
the comfort level of the patient, etc. The external physician can transmit one or many forms with
the required data entries completed. Forms are typically transmitted by fax. The faxed data are
treated upon receipt, in other words, verified, corrected and then, stored. If the CRF contains
errors (non-evident error in data or missing entry on the form), a form listing these error entry
data is generated and will be faxed back to the physician for correction or completion. This form
is called a Data Clarification Form (DCF). When all the required visits are completed, that is
when the study has met its target, the stored data is analyzed, and a statistical report analysis is
generated.
This data management process is represented using a Unified Modeling Language (UML) as
shown in figure 1. The UML approach is convenient to illustrate this business process, and more
generally, the activity diagram (OMG Unified Modeling Language Specification, Version 1.5,
2003) models data and control flows, and explicitly displays the role of actors involved in the set
of activities. This is a useful means to show the dynamic aspects of the information management
process at a general level.
Clinical Sites Data Management Team Statistical team
Begr
Fax theCRF aS
Verity CRF oR
u
F
Few
Noten corey sve coer
(Create DCF
é
Ea Mi —y
ines | (iraseD0> ¢ Bee)
v L
(Faxteccr)|_O — oN
Upeated "i
- oF
oe Create anaiss Analyzes
reser
Vataats
Se
Legend
oe
2 =| =
Conditional = ae
Figure 1 - UML activity diagram
In this context, the main role of an IS, in addition to support project management, is the reception
of forms by fax, optical character recognition or OCR (conversion of data from a paper to an
electronic format), data control, fax and error tracking, and finally the archiving of data for
further consultation and analysis. These functionalities may vary from one software to another.
The different software available on the market do not always offer the same set of functionalities,
and functionalities may vary as to their performance levels. Furthermore, the information
management need differs amongst firms. For example, according to organizational objectives
(some prefer speed in information treatment, others favor 100% error free), or to financial
resources (a small or medium size CRO can invest a more limited amount into its hardware and
software). The problem selection relates to the choice of an IS appropriate for the specific criteria
of the clinical trial process and firm policies.
As seen in figure 2, this information management system includes a number of balancing
feedback loops. Feedback loops B1, B2, and B3 relate to the generation visits. First, these require
the enlistment of patients, which is function of a target stated in the protocol, that is, for the
“enrollment objective”. The greater is the enrollment objective, the greater will be the enrollment.
This has for consequences to increase in the number of “enrolled patients” and to reduce the
objective of patients to enroll, as seen in the balancing loop B1. However, a number of patients
already enrolled are suppressed from the study, because a portion of patients give up on the study
or because it fails to meet a criterion. The rejection rate, directly linked to the number of enlisted
patients, negatively impacts the number of patients to visit, as seen in the balancing loop B2. The
balancing loop B3 concems the generation of the number of visits, and depends on the number of
patients to visit, and these visits require a certain time delay for completion.
Rejection
a
+ B2
me a
Patients to vist -
a ® Enlisted Sa
t Y Objective of
B1 patients fo
% enrol
Completed visits of
*\enroiment eet CRF data corrected
a rate in database
CRF workiow
oi . 3 cRF \
5 87
\ * Aes i
is _ CRF Se Erroneous data
CRF saved
Rate CRF a3 in database
linked to DCF
ocr j--*
workfow
OCF with ‘ error rate
bgt fate i
7m “.
we y ,
Statistical analysis
ocr .
+ managing \ f/
oN} eee Corrected CRF
Figure 2 - Influence diagram of information management process
The generation of visits leads to the transmission of CRFs. As seen in the balancing loop B4,
these CRFs are transferred upon receipt to the IS, and this, through the OCR functionality of the
information system. These CRFs can then be directly stored in the database, which may require
some manual intervention for the fixing of any error, or lead to the generation of a DCF. In the
case of a DCF, the rate of a CRF linked to a DCF determines the number of DCF to be generated.
As seen in the reinforcing loop R1, following the issuance of a DCF, the initial CRF can be
manually fixed or at times, a new CRF is faxed back to a physician, following a certain time
delay. It is important to note that some CRFs will remain unanswered by physicians and, thus, the
corresponding CRF will never be treated or analyzed. The number of outstanding CRFs depends
in part on whether the IS can manage DCFs. For the specific case of the CRO under
consideration, the return fax of the DCF back to physicians was not automated, which may lead
to a time delay in the execution of the task, leading to a non-response on the part of physicians
for returning the DCF. The CRFs requiring a manual correction are a function of the CRF rate to
correct, which depends, among other things, on the performance of the IS relative to the OCR,
and this will eventually impact the number of corrected CRFs. This will allow the reduction in
the rate of CRFs to correct. This sequence is shown in the balancing loop B5. It is important to
note that some software can support only to a limited extent the error correction process, and this
will have an impact on the time it takes to treat CRFs. Finally, the corrected CRFs can be saved
in the database and a statistical analysis will be performed.
Two other feedback loops are shown in this diagram. The balancing feedback loops B6 and B7
are related to the tracking of CRFs and to data quality. Indeed, errors can crop up, when for
example, when the CRF status is conferred (B6), or fields have missing or false entries in the
database itself (B7). Information systems that manage CRFs with strong data control
functionalities can help avoid this type of error. However, this was not the situation for the case
under study and an additional effort was required on the project management front.
The main feedback loops that have been emphasized from the modeling effort lead to the
quantitative representation of the model structure. The data collected to build the model (CRO or
software suppliers), have been used to calibrate this model, with the quantification of the level-
rate model, and associated parameters. The general structure of the model is shown in figure 3.
Theme 1 Theme 2
Generated New project
visits
configuration
Costs, detys
Theme 5
Global strategy
Number of visits realized
Costs, dekye
Rate analysis
Costs, dekys
Nb of CRF transtemd
if
CRF saved in DB
y
Theme 3
CRF treatment
Theme 4
Nomber of OF CRF Analysis
Figure 3 - General structure of the model
This process can be analyzed using five themes to which correspond five level-rate sub-models
linked up with interacting variables. These themes are:
1) Visits generated by physicians, linked to the characteristic of the study, that are linked to
the size of the study;
2) New project configuration to which are related the required configuration for each new
project, and that includes all preliminary steps prior to begin the information management
process;
3) The treatment of CRFs, that is the data, at the center of information management during
the clinical trial process;
4) The statistical analysis of CRFs; and
5) The general strategy of the organization that underlies the study and orients the many
actions.
The model is a detailed representation of the management structure of clinical trials supported by
an IS. It takes into account parameters that represent both organizational and technological
dimensions of clinical trials management. This model includes 44 level variables and 49
parameters, and a large number of balancing feedback loops.
The complex structure of clinical trials makes it difficult to fully understand this system. The
application of Alter’s (2001) WCA or the process modeling using the UML language, have been
key elements to structure the simulation model. The use of these approaches have helped
delineate the clinical trial process, to represent the key feedback loop structure at a level general
enough to justify a generic SD model. The use of these descriptive frameworks is treated only to
a limited extent in the literature, and this further justifies the interest in the proposed research.
Results
Sterman’s (2000) research process has contributed to the structuring and design of the many
analytical components, such as: the formulation of a new strategy and the evaluation of
simulation results. The simulation model was examined using the analysis of altemative
specification for sensitivity analyses on a parameter linked to the use of an IS, like variations in
the business throughput (that depends, among other things, on the treatment capacity of the IS in
terms of volume), error rates relative to the performance of the optical recognition (OCR) of the
IS, or even, the number of human resources on the IT team. The results presented examine the
main impacts of the automation of the information management process during clinical trials, by
parameterization of altemative software functionalities. The software functionalities and
specifications were obtained from dedicated products commercially available on the market.
This section presents some illustrative results of sensitivity analyses conducted on error variation
linked to optical recognition. This choice is motivated by the fact this parameter is directly linked
to the performance level of one of the functionalities of the IS. Indeed, the basic required
automation in information management during clinical trials requires the optical character
recognition (OCR) to electronically read the information received by fax from the physicians.
However, some errors can be linked to the OCR, which is translated into an “OCR error rate”,
thus in the percentage of forms for which one or more fields on the form are not appropriately
recognized. The error rate is mainly linked to the performance of the IS implemented but also to
the readability of the fax itself. Readability originates both from the layout of the form (space
between fields) and from the physician’s handwriting. Initially, the OCR error rate is set at 15%
(base case specification, or specification 1), the error rate is then set to 0% (specification 2), 50%
(specification 3), and 70% (specification 4). Thus, the impact on the process performance
associated with the OCR and efforts on improving the form layout are measured with the results
obtained from the specification 1.
The impacts of such a variation in error have first been examined for a single project. It is
important to note that a clinical trial project can last over a year. From the calculated means of
data from several Phase IV projects from the CRO under study, the different specifications have
been tested on the basis of 1,000 patients enrolled by project, with three required visits per
patient. The initial number of CRFs to be treated is 6,000, and the total revenue generated
amounts to $180,000.
The results show that the variation in the OCR error rate has a direct impact on the number of
CRFs to correct, either higher or lower, according to the variation in that rate, CRFs treatment
activities, and project costs. In table 1, the results show a variation according the rate of error.
Table 1 - Impact of alternative OCR error rate specifications
Specification 1 2 3 4
Error rate due to OCR 15 0 50 70
CRF to correct due to the number of OCR errors 900 0} 3000{ 4200
CRF processing (number of days) 185 176 207 220
Costs associated with CRF processing ($) 14 815 | 14.056 | 16 586 | 17 597
Gross benefit margin per project (% of revenue) 40,9 41,3 40,1 39,6
It appears that an error-free OCR rate would lead to a 20% cost reduction relative to a 70% error
rate. Relative to the base case, with a 15% error rate in the OCR, the CRF treatment costs would
increase by $1,770 for specification 3 and close to $2,800 for specification 4. This due to a
variation in the duration of activities for the treatment of CRFs: the number of days allocated for
the treatment of CRFs increases with the number of errors. For the base case, the number of days
is 185 and decreases to 176 in the error-free case, and the number of days would be 207 and 220
with a 50% and 70% error rate, respectively. The gross benefit margin is impacted by this error
rate but the variations in margins are the highest at 41.3%, with an error-free specification, and
are lowered to 39.6% with an error rate of 70%. However, it remains true that an error-free
situation does not depict a real life situation, and is virtually impossible.
It can be assumed that the smaller the number of CRFs to correct, the more rapidly the project
will be completed. However, as seen on figure 4, this is not necessary the case. This result is
shown on the flow of archived CRFs in the database, and available for analysis at the end of the
process. Even though CRFs can be saved more rapidly, the analysis of the data only takes place
in month 13, and this, no matter the scenario under consideration. One can understand this
phenomenon by looking at figure 5, where the dynamic paths show CRFs are corrected over the
same period, and this, no matter the number of CRFs to be treated. The organization actually
corrects more CRFs in the case of specifications 3 and 4, but uses the same time interval to get
the work completed as in the specifications 1 and 2. It is important to note that even in the case of
specification 2 (error-free) the number of CRFs to be corrected is not entirely zero. This is
because errors other than from the OCR can occur.
10
0 12 3 4 5 6 7 8 9 10 11 22 13:14:15 16
Months
ses@+ + 15% —a&— 0% 50% senses 70%
Figure 4 - Impact of the OCR error rate specification on project duration and intensity
9 10 11 12 13:14:15 16
Months
s++@++ 15% —a— 0% 50% +++ e ee 70%
Figure 5 - Impact of OCR error rate specification on the number of CRF with error
To sum up, it is important to note that the error variation in the OCR error rate has almost no
impact on the duration of the project. Indeed, no matter the strategy adopted, the time delay
associated with the treatment of CRFs, that is, of the project, is much more dependent on the
velocity of patient enrollment and on the frequency of visits that follow, than factors exogenous
to the information management process. Thus, increases and decreases in the number of CRFs to
correct have only a minimal impact on the project execution.
The results emphasize that errors rates impact costs, but also the human resources efforts required
for fixing these errors. The rate of errors from OCR should have a real impact in multi-project
situations, while resources can be allocated to other tasks if error rates are low. The capacity to
conduct several projects simultaneously is a source of competitive advantage. For the case under
study, and given available capacity in resources, only the equivalent of two large scale projects
can be conducted over a year. The selection on an IS, that could support the execution of several
projects by limiting the data correction effort, could play a greater role in a business growth
strategy.
11
Conclusion
This paper was concemed with the IS performance evaluation of information management for
drug development, and more specifically in clinical trials. The approach taken to research this
question was SD. The purpose of the model designed was to examine the repercussions of
automating the process and to capture the informational issues that must face organizations
involved in clinical trials. The analysis of the information management process during clinical
trials, and its modeling, has stressed how complex is such a system. Indeed, it is characterized by
a number of interrelated mostly negative feedback loops that generate non-linear behavior. The
resulting dynamics make it difficult to structure an understanding of the actual and future
behavior of the system. To accomplish this, the WCA and the UML were used as complements to
help scope the research problem. The resulting model has helped realizing a comparative analysis
of software technologies, and in addition, alternative scenarios were proposed. A few sensitivity
analyses were conducted on the error rate, and these results were presented in the paper. The
results of the research emphasize that since the IS has no real impact on the duration of a project,
the choice of an IS should favor multi-project execution.
The conceptual framework brings a genuine contribution in terms of the modeling process, while
following the general SD modeling process (Sterman, 2000). The proposed model in this
complex context is even more interesting, as the advantages that SD can bring to decision-
making are emphasized.
Given the generic nature of clinical trial projects, the management needs and constraints in
clinical trials management may differ from one organization to another. Applying this model to
other CROs would be interesting to evaluate the generic model and confirm some conclusions. It
would also be interesting to include a parameterization of biopharmaceutical firms, which have
not been included in this research. Large pharmaceutical groups, or smaller biotechnology firms,
may possess characteristics that differ from CROs.
References
Alshawi, S., Saez-Pujol, I., Irani, Z. 2003. Data warehousing in decision support for
pharmaceutical R&D supply chain. International Journal of Information Management 23: 259-
268.
Alter, S. 2001. Information Systems: The Foundation of E-Business. Upper Saddle River, New
Jersey: Prentice Hall.
Anderson, J.G. 2002. Evaluation in health informatics: computer simulation. Computers in
Biology and Medicine 32: 151-164.
Augen, J. 2002b. The evolving role of information technology in the drug discovery process.
Drug Discovery Today 7: 315-323.
12
Azoulay, P. 2003. Acquiring knowledge within and across firm boundaries: evidence from
clinical development. Available at: http://www.nber.org/papers/w10083.
Glouberman, G, Zimmerman, B. 2002. Complicated and complex systems: What would
successful reform of medicare look like? Commission on the Future of Helath Care in Canada.
Discussion Paper No 8.
McDonnell, G. 2004. Using System Dynamics to analyse Health System Performance within the
WHO Framework. International Conference of the System Dynamics Society.
McDaniel, R.R., Driebe, DJ. 2001. Complexity science and health care management. Advances
in Health Care Management 2: 11-36.
Morecroft, J.D.W., Lane, D.C., Viita, P.S. 1991. Modelling growth strategy in a biotechnology
startup firm. System Dynamics Review 7: 93-116.
Piachaud, B.S. 2002. Outsourcing in the pharmaceutical manufacturing process: an examination
of the CRO experience. Technovation 22: 81-90.
Rowe, J. C., Elling, M. E., Hazlewood, J. G., Zakhary, R. 2002. A cure for clinical trials. The
McKinsey Quartely 2.
Santos, F.M. 2003. The coevolution of firms and their knowledge environment: insights from the
pharmaceutical industry. Technological Forecasting & Social Change 70: 687-715.
Specification UML : Version 1.5. Available at: http://www.omg.org/docs/formal/03-03-01.pdf
Sterman, J.D. 2000. Business Dynamics: Systems Thinking and Modeling for a Complex World.
New Y ork, NY: Irwin-McGraw- Hill.
Wiendahl, H.P., Worbs, J. 2003. Simulation based analysis of complex production systems with
methods of non-linear dynamics. J ournal of Materials Processing Technology 139: 28-34.
Wolstenholme, E.F., Henderson, S., Gavine, A. 1993. The Evaluation of Management
Information System: A Dynamic and Holistic Approach. Wiley: Chichester.
13
CC BY-NC-SA 4.0