Implementation of adolescent SBIRT 1
Application of system dynamics to inform impl ion of
care settings
SBIRT in primary
D. W. Lounsbury,’ S. G. Mitchell,? Z. Li,? R. P. Schwartz,” J. Gryczynski,” A. Kirk,’ M. Oros,° C. Hosler,° K.
Dusek,” Laura B. Monico,” B. Brown*
Albert Einstein College of Medicine, Montefiore Medical Center, Bronx, NY, United States.
Friends Research Institute, Baltimore, MD, United States.
College of Global Public Health, New York University, New York, NY, United States.
Total Health Care, Baltimore, MD, United States.
The Mosaic Group, Baltimore, MD, United States.
Gig OS be
Abstract
We apply system dynamics (SD) modeling to better understand the influence of different
implementation strategies on the effective implementation of screening and brief intervention. Using
qualitative and quantitative data from an on-going cluster randomized trial in 8 federally qualified health
center sites, two implementation conditions were compared: Generalist vs. Specialist. In the Generalist
Approach, the primary care provider (PCP) delivers brief intervention (Bl) for substance misuse (n=4
clinics). In the Specialist Approach, Bls are delivered by behavioral health counselors (BHCs) (n=4 clinics).
We used our SD model to compare ‘basecase’ dynamics to strategic approaches to deploying
Continuous Technical Assistance (TA) and Performance Feedback Reporting (PFR). We calibrated our
basecase to effectively represent the SBIRT intervention, which reflected actual monthly volume of
adolescent primary care visits (N=9,639), screenings (N=5,937), positive screenings (N=246), and brief
interventions (Bls; N=50) over the 20-month implementation period. Insights gained suggest that
implementation outcomes are sensitive to frequency of PFR, with bimonthly events generating the most
rapid and sustained screening results. Simulated trends indicate that availability of the BHC directly
impacts success of the Specialist model. Similarly, understanding PCPs’ perception of severity of need
for intervention is key to outcomes in either condition.
Financial Support: National Institute on Drug Abuse grant 1RO1DA034258-04
Implementation of adolescent SBIRT 2
Introduction
In this paper we present an application of system dynamics (SD) to examine strategies for implementing
the Screening, Brief Intervention and Referral to Treatment (SBIRT) model in a multi-clinic, urban
primary care facility for low income families. The SBIRT model described here targets substance use and
other risky behaviors in high-school aged youth who visit the facility for either well- or sick-care. Clinical
trials and meta-analyses evidence supports the effectiveness of SBIRT for adolescents with substance
misuse, but primary care providers have been slow to adopt this evidence-based approach in routine,
clinical practice. Thus, research is needed to determine effective ways to implement SBIRT for
adolescent substance misuse so that this approach can be brought to scale. SD modeling was used to
help inform new organizational procedures and policies that adequately address implementation
challenges and support the long- term sustainability of the intervention.
Procedures for building and validating the SD model were embedded within a parent research project
(Mitchell, Pl; National Institute of Drug Abuse grant 1RO1DA034258-04), which aims to contribute to the
growing field of implementation science field. Implementation science is a nascent field and a number of
conceptual models have recently been proposed to guide research efforts (Damschroder, Aron, Keith, et
al., 2009; Fixsen, Naoom, Blase, Friedman, & Wallace, 2005; Proctor, Landsverk, Aarons, et al., 2009;
Simpson & Flynn, 2007). Proctor and colleagues’ model of implementation research (Proctor et al., 2009;
Proctor, Silmere, Raghavan, et al., 2011) links key implementation strategies with implementation
outcomes.
Preval and impact of use and other risky behaviors in adolescents. Alcohol, tobacco,
and other drug use remains highly prevalent among US adolescents and is a threat to their well-being
and to the public health. In the United States, approximately one in ten youth under the age of 18
reports using illicit drugs, tobacco, or binge alcohol drinking in the past month (Substance Abuse and
Mental Health Services Administration, 2010) and while recent studies have shown an overall decline in
reported use of illicit drugs, findings from the most recent Monitoring the Future survey indicated that
37% of 12th graders reported past-month use of alcohol, 35% reported use of marijuana in the past
year, and that use of non-prescribed medications such as Adderall, Vicodin, and tranquilizers in the past
year ranged from 4.7 to 6.8% (Johnston, O’Malley, Bachman, Schulenberg & Miech, 2014).
Youth with more severe substance use issues often experience significant difficulties with school
performance (Miller, Naimi, Brewer, & Jones, 2007), while many more were no longer engaged in school
by the 12th grade, indicating that these figures likely under-represent the extent of substance use and
related problems. More particularly, studies have found that deleterious short-and long-term
consequences of substance use include mental health problems (Mathers, Toumbourou, Catalano,
Williams, & Patton, 2006; Moore, Fiellin, Barry, et al., 2007), deteriorating school performance (Miller et
al., 2007), risky sexual activity and victimization (Fergusson & Lynskey, 1996; Miller et al., 2007), placing
oneself in danger by riding with an impaired drive (Miller et al., 2007), suicide attempts (Windle, 2004),
and elevated risk of mortality (Clark, Martin, & Cornelius, 2008). Moreover, it has been reported that
substance misuse during adolescence may negatively impact critical stages of brain development
(Volkow & Li, 2005; Lubman, Yucel & Hall, 2007). Perhaps the most obvious long term consequence of
substance use in adolescence is that it increases the risk for substance use disorders later in life
(Englund, Egeland, Oliva, & Collins, 2008; Hingson, Heeren, & Winter, 2006; Mathers et al., 2006;
McCambridge, McAlaney, & Rowe, 2011; Swift, Coffey, Carlin, Degenhardt, & Patton, 2008). These
findings have led to a concern with developing better screening and interventions for the range of
substance use issues, which has led in turn to the development of the SBIRT model.
Screening, Brief Intervention and Referral to Treatment. SBIRT typically uses universal screening (S)
with validated brief self-report questionnaires to identify those at-risk for substance use problems
Implementation of adolescent SBIRT 3
(Knight, Sherritt, Harris, Gates, & Chang, 2003; Knight, Sherritt, Shrier, Harris, & Chang, 2002; Reinert &
Allen, 2007). Those who screen positive are given a Brief Intervention (Bl), or a referral to treatment (RT)
if specialized treatment for substance use disorders appears warranted. In this way, SBIRT can be
employed to address varying degrees of substance use severity. In randomized clinical trials with
adolescent populations in school and health care settings, brief interventions were found to significantly
impact consumption of alcohol, tobacco, and marijuana (McCambridge & Strang, 2004); smoking
frequency and to increase long-term cessation (Colby, Monti, O'Leary, et al., 2005; Heckman, Egleston, &
Hofmann, 2010; Hollis, Polen, Whitlock, et al., 2005; Peterson, Kealey, Mann, et al., 2009); use of alcohol
(Monti, Colby, Barnett, et al., 1999; Spirito, Monti, Barnett, et al., 2004); both use of alcohol and anti-
social aggressive behaviors (Walton, Chermack, Shope, et al., 2010); attempts to quit drinking
(Bernstein, Heeren, Edward, et al., 2010); frequency of drug use and related consequences (Winters &
Leitten, 2007); marijuana use; number of friends smoking marijuana (Bernstein, Edwards, Dorfman, et
al., 2009; D'Amico, Miles, Stern, & Meredith, 2008; Martin & Copeland, 2008); and referral to substance
abuse treatment (Tait, Hulse, & Robertson, 2004). Meta-analyses of RCTs in a variety of settings of Bls
for adolescent substance use have obtained positive findings (Tait & Hulse, 2003; Tripodi, Bender,
Litschge, & Vaughn, 2010).
Most adolescents in the US see a healthcare provider at least annually, making primary care an ideal
venue in which to deliver substance misuse interventions for this population (Newacheck, Brindis, Cart,
Marchi, & Irwin, 1999). Unlike traditional substance abuse treatment or extended prevention programs,
SBIRT is a service model that is well-suited for integration into primary care (Erickson, Gerstle, &
Feldstein, 2005). Although the USPSTF states that support for providing Bls in primary care is limited,
both the American Academy of Pediatrics and the NIAAA recommend that pediatricians provide
substance use screening and counseling to all adolescents (American Academy of Pediatrics, 2010).
However, the majority of physicians do not follow this recommendation (Millstein & Marcell, 2003).
i 1 ‘if
of i strategies via sii Although human service delivery
systems, such as the multi-site primary care facility used in the current project, are complex and
arguably “... not predictable... it is possible to achieve a level of understanding of a complex system
by studying how it operates” (p. 177). Working in collaboration with selected clinic-based project
stakeholders, we developed a system dynamics (SD) model that simulates the effects of two key
implementation strategies deployed in the current research project, namely: Continuous Technical
Assistance (TA) with site-specific Performance Feedback Reporting (PFR). TA includes integrated team
development of SBIRT service delivery model, including useful modifications to the electronic medical
record, initial training of staff (i.e., Medical Assistants, MAs; primary care physicians, PCPs; and clinic
administrators), staff debriefing/informal following guidance, and periodic educational ‘booster’
sessions. PFR includes periodic aggregated (i.e., clinic-level) and provider-specific performance reporting
to with to all staff accompanied with supervision.
The SD model is designed to simulate the effects of both TA and PFR on dynamics associated with the
adolescent SBIRT, namely aggregate adolescent screenings, positive (risk endorsed) adolescent
screenings, and Bls — relative to hypothesized an expected positive screening rate adjusted for
adolescent ‘under reporting’ and PCPs’ perception of the severity of adolescents’ substance use and/or
potential harm due to self-reported risky behavior(s). The specific aim of this work was to develop and
validate a SD model as a visual interface for structured play, or gaming, by health care providers and
other important stakeholders who to effectively implement and sustain the adolescent SBIRT model in
their setting.
Implementation of adolescent SBIRT 4
Methods
Parent project design. The parent research project’s protocol is a multi-site, cluster randomized trial (N=
8) guided by Proctor’s conceptual model of implementation research (Proctor et al., 2009; Proctor,
Silmere, Raghavan, et al., 2011) and comparing two principal approaches to SBIRT delivery within
adolescent medicine: Generalist vs. Specialist. In the Generalist Approach, the primary care provider
delivers brief intervention (Bl) for substance misuse (n=4 clinics). In the Specialist Approach, Bls are
delivered by behavioral health counselors (n=4 clinics).
Statistical analyses of intervention data showed that there were no significant differences by Generalist
or Specialist condition during the 20-month implementation period on penetration of screening (p=.55),
brief advice (p=.70), or brief intervention (p=.58). There were significant time period differences in
screening (above and beyond differences by Site and Condition), but not for Brief Advice or Brief
Intervention (Note: This is likely due to very low numbers receiving BA or Bl). There was significant
variation by Site in penetration of screening, BA, and BI. While both service delivery models showed
promise for delivering Bls, the high rates of variability within sites demonstrate a need for further
examination.
Overview of SD modeling approach. SD model-building deploys an iterative research process that is
complete when the model achieves sufficient structural and behavioral validity to its intended purpose
(Barlas, 1989, 1996; Martinez-Moyana & Richardson, 2013; Martinez-Moyano, 2012). Procedures for
establishing structural and behavioral validity are organized around the purpose of the model, the type
and quality of the sources of evidence, and model calibration. In the current study, the purpose of the
SD model is to help inform new organizational procedures and policies that adequately address
implementation challenges and support the long term sustainability of the intervention. The featured SD
model is a working set of algebraic and ordinary differential equations, generally shown as a stock-and-
flow diagram. With input and deliberation among the research project team and other project
stakeholders, we followed established procedures for model building (see Roberts, Anderson, Deal,
Garet, & Shaffer, 1983), including problem identification, system conceptualization, model formulation,
model simulation, and, finally model evaluation.
Assessment of model performance. We conducted a variety of tests to ensure that the current SD
model adhered to established validation tests, following recommendations by (Forrester & Senge,
1980). Specifically, we conducted (1) verification tests to confirm that parameter estimates were logical,
supported by one more sources of information, and properly entered; (2) validation tests to address the
extent to which the simulated behavior of the model was realistic or like the actual ‘real world’
dynamics it is intended to represent; and (3) legitimation tests, which affirm that the differential
equations use to construct the model followed commonly accepted mathematical principles, namely
that they were dimensionally valid (i.e., the units of measurement or quantification of the constructs or
variables on each side of the equation should be the same), and, for material (i.e., physical) variables,
the model should maintain ‘conservation of flow.’ (i.e., what enters the system should be accounted for
at any point within the model’s time horizon). Face-to-face and on-line meetings with key project
stakeholders were conducted at project milestones (start-up, implementation phase initiation,
implementation phase completion, sustainability phase completion). Vensim software was used to
create the SD model and simulate outcomes (Ventana Systems Inc., 2008).
Sources of SD modeling data. Our system dynamics model is informed by quantitative data obtained
through the parent project’s multi-site, cluster-randomized design and by qualitative input obtained
from stakeholders, namely MAs, nurses, PCPs, BHCs, and administrators. Five data sources are available
to support our proposed SD modeling: (1) Training data — detailed records of initial and booster training
sessions (longitudinal; see Appendix); (2) Patient visit and screening data — number of adolescents and
Implementation of adolescent SBIRT 5
non-adolescents (medical records, individual patients, longitudinal); (3) Staffing data — clinical staffing
and staffing turnover (longitudinal); (4) Structured provider interviews and semi-structured qualitative
provider interviews about knowledge of barrier and facilitators (baseline and follow up during
sustainability period); and (5) Organizational impact data about either facilitators or inhibitors of the
intervention’s implementation, such as catastrophic breakdown of a clinic’s electronic medical records
systems or an abrupt change in clinic leadership priorities relating to the intervention.
SD model variables and parameters. Figure 1 presents a stock-and-flow diagram of our SD model of
adolescent SBIRT implementation strategy dynamics. The SD model structure is designed to simulate
patient flow and SBIRT delivery across health system. The model is organized into three substructures
(‘implementation strategy,’ ‘Generalist,’ and ‘Specialist’). The ‘Implementation Strategy’ substructure
includes the structures that represent patient visits, screenings, positive screenings, and interventions
(Bls) for adolescents aged 12-17 who visit any of the seven clinics that comprise the participating care
facility. We utilized the first-order smooth to simulate effect of key implementation constructs:
Performance Reporting (PFR) frequency; Quality of Technical Assistance (TA). Three first-order smooth
structures represent staff performance and engagement, including the performance of MAs (who are
charged with administering the screening of adolescents), the clinic administrators (who are charged
with overseeing implementation and sustainability of the SBIRT intervention, and the PCPs (who are
charged with actually delivering the Bl). Note that organizational buy-in by PCPs and administrators
partially mediate the effect of TA and PFR on the MA’s screening performance.
We applied the ‘smooth’ structure to represent information delays for each of these major constructs,
as it allowed us to test hypotheses about the potential effect of deployed implementation strategies
(i.e., TA and PFR) on how long it takes, on average, to change the behavior of clinic MAs, PCPS, and
administers. An additional smooth structure was used to represent the effect of ‘training and trouble-
shooting,’ which was derived from parameters that defined the quality of TA and the frequency of PFR.
The ‘Generalist’ and the ‘Specialist’ substructures, respectively, allow for comparison of simulated
outcomes between the two intervention conditions of the parent study. These substructures simulate
rates and accumulation BA and BI delivery, over time.
Finally, there are 16 exogenous parameters (shown in red font in Figure 1) that are used to define our
basecase (calibrated) scenario and comparison scenarios. These exogenous parameters are essentially
‘drivers’ or effect sizes, of the implementation strategy (parameters circled in green), or the intervention
itself (BA or BI), which is a function of either the performance of the PCP (parameters circled in purple)
or the performance of the Behavioral Health Counselor (BHC; parameters circled in aqua blue). The
definitions and the range of values for these drivers were informed by qualitative and quantitative data
from the study.
Minimum and maximum values for exogenous parameters were purposefully defined with available
data as well as discussion among the research team and about how best establish meaningful, easy-to-
interpret, range of values that supported comparison scenarios of interest. For example, quality of TA
was defined as on-going assistance, punctuated by monthly site visits by research staff, to address
questions that clinic staff may have had and to check to see if new staff were aware of the aSBIRT
implementation effort. The range of values for quality of TA was set to be as low as 1 and as high as 10.
We chose 5 for our basecase because, although the research staff manager and trainer made regular
visits, reach to staff at any given visit was sporadic. Similarly, time to change PCP Buy In to the
intervention (i.e., PCP’s awareness, understanding and motivation to support the SBIRT intervention)
was set to 6 months. This ‘average’ parameter value was chosen considering all PCPs in the clinics and
PCP turnover rates. Comparatively, the ‘average’ time to change Administrator Buy In was set to 12
months and the average time to change MA Screening Performance was, on average, just 1 month. The
Implementation of adolescent SBIRT 6
higher (longer) parameter value for Administrators reflects qualitative observations about how all
administrators in the health system, including evidence that Administrators had more turnover than
PCPs. The very low (short) time to change MA Screening Performance was assigned based on the
assumption that their job was dedicated to the intervention, and that they would be response to their
PCPs’ and Administrators’ supervision.
Other important drivers included PCP Perceived Severity Factor and PCP Willingness to Handoff Factor.
PCP Perceived Severity was set to 1.3 for the basecase. This parameter was defined as the PCPs'
likelihood to respond to a ‘high’ risk vs. a ‘low’ risk patient, which was an artifact of the PCP’s personal,
subjective judgment. This parameter range was from 0 to 2, which allowed for use of interesting
mathematical properties (i.e., if set to 0, no Bls would be delivered by the PCP). In our basecase, PCP
Willingness to Handoff Factor was set to 0.26, to match the proportion that were documented in charts
as handed off to the BHC. And this value could be as low as 0 or as high as 1.0, which, again, allows for
examination of special circumstances, where BHC may be complete obstructers or facilitators of BI
delivery.
| IMPLEMENTATION STRATEGY |
Legend:
red font = exogenous parameter
oO Implementation driver
Cc Primary Care Provider BA|Bl driver | SPECIALIST *
oO Behavior Health CounselorBI driver
@F Structure of interest
Implementation of adolescent SBIRT 7
Results
Our basecase implementation scenario (i.e., the scenario representing actual deployment of the SBIRT at
participating clinics) was compared to 8 scenarios of interest (see Tables 1-3). Supported by qualitative
and quantitative data from the study, we calibrated our basecase to effectively represent the SBIRT
intervention, which reflected actual monthly volume of adolescent primary care visits (N=9,639),
screenings (N=5,937), positive screenings (N=246), and brief interventions (Bls; N=50) over the 20-
month implementation period. We then varied the value of five selected drivers, to illustrate tradeoffs
in implementation outcomes. Selected drivers included to implementation drivers (i.e., frequency of PFR
and quality of TA), two PCP performance drivers (i.e., PCP perceived severity and PCP willingness to
handoff adolescent patients to the BHC), and one BHC driver (i.e., BHC availability).
Table 1 shows parameterization for our basecase and three scenarios that explore the medical assistant’
screening performance. Comparison of simulated output shows that screening performance varied in
relationship to the frequency of performance feedback reporting (PFR). Our basecase scenario
(bimonthly PFR) showed screening rates to rise rapidly over the initial 6 months of the implementation
period to 68% of all adolescent visits, dipping slightly due to transition to a new electronic medical
records (EMR) system, then recovering and leveling off at approximately 70%. Decreasing PFR to
quarterly, semiannual, or annual intervals generated diminished screening patterns (Scenarios 1-3; see
Table 1 and Figure 1).
Table 1 - MA screening performance rate
Parameter
quency Quality oF
Biicccum performance technical PCP perceived | PCP willingness | BHC availability
feruril feedback reporting| _assistance__| severity (dmnl)_| to handoff (dmnl) (dmnt)
Parameter range
25to 12 Tto 10 0.0 to 2.0 0% to 100% 0% to 100%
Basecase Bimonthly here Somewhat more | Somewhat Half of the time
severe unwiling
Z 5 13 26% 50%
Scenario 1 Quarterly Average Somewhatmore | Somewhat Half of the time
severe unwiling
3 5 13 26% 50%
ISzanaes'2 Zenilennull erage Somewhatmore | Somewhat Half of the time
severe unwiling
6 3 13 26% 50%
Scenario 3 wil erage Somewhatmore | Somewhat Tatofthetime
severe unwiling
2 5 13 26% 50%
Figure 1 - Medical Assistarts' Screening Performance
0 2 4 6 8 10 12148 16 618 ~=—(20
Time (Month)
Implementation of adolescent SBIRT 8
Table 2 shows the parameter values for BI delivery rate in the Specialist condition only. Examination of
BI delivery rates for the Specialist condition, where availability of the Behavioral Health Counselor (BHC)
varied from 25% to 100%, showed that, as expected, higher BHC availability generated higher BI delivery
rates, although never exceeded 10% of positively screened adolescents (Scenarios 4-6; see Table 2 and
Figure 2).
Table 2 - Bl delivery rate, Specialist Only
Frequency of ‘Quality of
performance technical | PCP perceived | PCP willingness | BHC availability
assistance _| severity (dmni)_| to handoff (dmni) {amnl)
Parameter range
250 12 to 70 0.0to 2¢ T% to 100% O% to 100%
Basecase Somewnatmore | Somewhat
Bimonthy Average Seats wing Half of the tme
2 5 13 25% 50%
‘Scenario 4 Bimonthty jaiacaige Somewhat more ‘Somewhat ‘A quarter of the
severe unwilling time
2 5 13 25% 25%
‘Scenario 5 Somewhat more ‘Somewhat Three quarters of
Bimonthly Average severe unwillirg the tme
2 5 13 25% 75%
Scenario€ Somewnatmore | Somewhat
Bimonthy Average patie cunwilicg Allof the time
Z 5 13 25% 700%
Figure 2 - BI Delivery Rate - SPECIALIST Only
0.12
0.06
oO
0 2 4 6 8 0 2 4 «16 18 2
Time (Month)
Table 3 shows the parameter values for BI delivery rates, comparing Generalist to Specialist conditions.
Comparison of simulated differences in the PCP’s likelihood to respond to a positive vs. a low risk
adolescent patient (i.e., perceived severity) revealed high sensitivity, with BI delivery rates increasing
from 39% to 61% (GENERALIST) and from 5% to 8% (SPECIALIST) by the end of the implementation
period. Notably, results for the GENERALIST condition were substantively higher than in the SPECIALIST
condition for all simulated values of PCP’s perceived severity (Scenarios 7-8; see Table 3 and Figure 3).
Table 3 - BI delivery rate GENERALIST vs SPECIALIST
Parameter description
Frequency of Quality of
Situ performance technical | PCP perceived | PCP willingness | BHC availability
Scenario i assistance | severity(dmnl) |tohandoff (dmni)| __(dmnl)
Parameter range
to 12 Tt 10 0.0 to 2.0 O% to 100% O% to 100%
Basecese Bimonthly Average Somewhatmore | Somewhat | tairoF the time
severe unwiling
z 5 13 26% 50%
\Scenerior7: Bimonthly Average Same severity Somewhat Half of the time
unwiling
z 5 10 26% 50%
Scenenee: Bimonthly Average Extremely more Somesiniat Half of the time
severe unwiling
z 5 2.0 26% 50%
Implementation of adolescent SBIRT 9
Figure 3 - BI Delivery Rate - GENERALIST vs. SPECIALIST
1 2 3 4 5 6 7 6 9 0 1 12 15 14 18 16 17 18 19 2
Time (Month )
Initial feedback about current SD model. During a brief webinar offered in late 2015, key stakeholders
from the participating health system affirmed the utility of simulated analysis for future orientation and
training sessions with providers and other ‘front line’ providers. Preliminary experience in sharing the SD
model out on key stakeholders from the parent project was successful. For example, one stakeholder
noted that it would be useful and interesting to use the model to conduct a simulated break-down
analysis by age group and gender. There was a hypothesis that high school age girls and boys would
present with different levels of risky behavior, and that girls may be biased to endorse risk behaviors
more than boys.
There was also interest in expanding the SD model to include structures that would simulate the
dynamics of referral to SA treatment for those adolescents with greater risk. Another suggestion was to
use the SD model to examine clinic size and clinic flow relative to size (i.e., the adolescent visit rate).
Last, there was interest expressed in further consideration of how the provider interview data
(qualitative) may be useful data to inform staff-related effects. Further examination of this via the SD
modeling is also is warranted. More generally, there was concern from one participant about having
sufficient evidence to support modeling assumptions, although there was an appreciation for simply
using the model to explore various possible contingencies, over time.
Discussion
The current paper features an application of system dynamics (SD) to examine strategies for
implementing the Screening, Brief Intervention and Referral to Treatment (SBIRT) model in a multi-clinic
urban primary care facility for low income families. The simulation results presented here are offered as
a demonstration of the potential utility of using a SD approach to guide implementation planning. The
scenarios used to illustrate the SD model are limited in that, although there are thoughtful and
plausible, they generate an incomplete assessment of the myriad options could be explored.
Insights gained from this set of scenarios suggest that implementation outcomes are sensitive to
frequency of PFR, with bimonthly events generating the most rapid and sustained screening results.
Simulated trends indicate that availability of the BHC directly impacts success of the Specialist model.
Similarly, understanding PCPs’ perception of severity of need for intervention is key to outcomes in
either condition. Additional application of the SD model will explore post-implementation outcomes.
Conclusion
SD modeling is a robust method for comparative analyses of implementation strategies. This approach
facilitates synthesis of multiple sources of information/data and can foster important insights about how
to deploy limited resources for training and support in diverse clinical sites.
Integration of Innovation and Technology as a Strategic Advantage for
an Organ Transplant Program
Todd Astor, M.D., M.B.A. Medical Director, Lung and Heart-Lung Transplant Program, MGH Organ
Transplant Center and Department of Medicine, Massachusetts General Hospital; Assistant Professor of
Medicine, Harvard Medical School
Lung transplantation has become the definitive therapy for patients with end-stage lung
disease. The lung transplant rate at the Massachusetts General Hospital had previously
been considered suboptimal by national standards. To identify opportunities for improved
patient care and overall program growth, a comprehensive competitive strategic analysis
utilizing system dynamics models was performed to better conceptualize the complex
interactions between the medical, financial, marketing, and operational variables and
capabilities that contribute to the functioning of an organ transplant program. This
analysis included an exploration into the enhancement of program capabilities to create
an optimal substrate for innovation, an understanding of the dynamics between
innovation and external transplant program “appeal”, a recognition of the impact of time
delays and need for synergism of capabilities, and the identification of potential
vulnerabilities in a strategic plan that is centered on clinical innovation. The conclusions
from this analysis directly led to the identification of a distinct strategic competitive
advantage possessed by MGH over other transplant programs, highlighted by the
capability to rapidly integrate several technological and operational innovative
approaches into the clinical care of patients. The implementation of a strategic plan based
on this paradigm shift led to an unprecedented increase in the annual MGH lung
transplant volume, and has rapidly placed MGH among the elite transplant programs in
the U.S. This paper highlights the utility of system dynamics models in performing a
competitive analysis of a clinical program that deals with the care of complex patient
populations.
Implementation of adolescent SBIRT 10
References
Aalto, M., Pekuri, P., & Seppa, K. (2003). Obstacles to carrying out brief intervention for heavy drinkers
in primary health care: a focus group study. Drug Alcohol Rev, 22(2), 169-173.
American Academy of Pediatrics (2011). Policy Statement: Adolescents and HIV infection: The
pediatrician's role in promoting routine testing. Pediatrics, 128(5), 1023-1029.
Babor, T. F., & Kadden, R. M. (2005). Screening and interventions for alcohol and drug problems in
medical settings: what works? J Trauma, 59(3 Suppl), S80-87; discussion $94-100.
Barlas, Y. (1989). Multiple tests for validation of system dynamics type of simulation models. European
Journal of Operational Research, 42(1), 59-87.
Barlas, Y. (1996). Formal aspects of model validity and validation in system dynamics. System Dynamics
Review, 12(3), 183-210.
Bernstein, E., Edwards, E., Dorfman, D., Heeren, T., Bliss, C., & Bernstein, J. (2009). Screening and brief
intervention to reduce marijuana use among youth and young adults in a pediatric emergency
department. Acad Emerg Med, 16(11), 1174-1185.
Bernstein, J., Bernstein, E., Tassiopoulos, K., Heeren, T., Levenson, S., & Hingson, R. (2005). Brief
motivational intervention at a clinic visit reduces cocaine and heroin use. Drug Alcohol Depend, 77(1),
49-59.
Bernstein, J., Heeren, T., Edward, E., Dorfman, D., Bliss, C., Winter, M., & Bernstein, E. (2010). A brief
motivational interview in a pediatric emergency department, plus 10-day telephone follow-up, increases
attempts to quit drinking among youth and young adults who screen positive for problematic drinking.
Acad Emerg Med, 17(8), 890-902.
Boekeloo, B. O., & Griffin, M. A. (2007). Review of clinical trials testing the effectiveness of physician
approaches to improving alcohol education and counseling in adolescent outpatients. Current Pediatric
Reviews, 3, 93-101.
Bower, P., & Gilbody, S. (2005). Managing common mental health disorders in primary care: conceptual
models and evidence base. BMJ, 330(7495), 839-842.
Clark, D. B., Martin, C. S., & Cornelius, J. R. (2008). Adolescent-onset substance use disorders predict
young adult mortality. J Adolesc Health, 42(6), 637-639.
Colby, S. M., Monti, P. M., O'Leary Tevyaw, T., Barnett, N. P., Spirito, A., Rohsenow, D.J.,...Lewander,
W. (2005). Brief motivational intervention for adolescent smokers in medical settings. Addict Behav,
30(5), 865-874.
Implementation of adolescent SBIRT 11
Cowell, A. J., Bray, J. W., Mills, M. J., & Hinde, J. M. (2010). Conducting economic evaluations of
screening and brief intervention for hazardous drinking: Methods and evidence to date for informing
policy. Drug Alcohol Rev, 29(6), 623-630.
D'Amico, E. J., Miles, J. N., Stern, S. A., & Meredith, L. S. (2008). Brief motivational interviewing for teens
at risk of substance use consequences: a randomized pilot study in a primary care clinic. J Subst Abuse
Treat, 35(1), 53-61.
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering
implementation of health services research findings into practice: a consolidated framework for
advancing implementation science. Implement Sci, 4, 50.
Drummond, M., Sculpher, M., Torrance, G., O'Brien, B., & Stoddart, G. (2005). Methods for the economic
evaluation of health care programmes (Third ed.). Oxford: Oxford University Press.
Englund, M. M., Egeland, B., Oliva, E. M., & Collins, W. A. (2008). Childhood and adolescent predictors of
heavy drinking and alcohol use disorders in early adulthood: a longitudinal developmental analysis.
Addiction, 103 Suppl 1, 23-35.
Erickson, S. J., Gerstle, M., & Feldstein, S. W. (2005). Brief interventions and motivational interviewing
with children, adolescents and their parents in pediatric health care settings. Arch Pediatr Adolesc Med,
159, 1173-1180.
Fergusson, D. M., & Lynskey, M. T. (1996). Alcohol misuse and adolescent sexual behaviors and risk
taking. Pediatrics, 98(1), 91-96.
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research:
A synthesis of the literature. Tapma, FL: University of South Florida, Louis de la Parte Florida Mental
Health Institute, The National Implementation Research Network (FMHI Publication #231).
Fleming, M. F., Mundt, M. P., French, M. T., Manwell, L. B., Stauffacher, E. A., & Barry, K. L. (2000).
Benefit-cost analysis of brief physician advice with problem drinkers in primary care settings. Med Care,
38(1), 7-18.
Fleming, M. F., Mundt, M. P., French, M. T., Manwell, L. B., Stauffacher, E. A., & Barry, K. L. (2002). Brief
physician advice for problem drinkers: long-term efficacy and benefit-cost analysis. Alcohol Clin Exp Res,
26(1), 36-43.
Gentilello, L. M., Ebel, B. E., Wickizer, T. M., Salkever, D. S., & Rivara, F. P. (2005). Alcohol interventions
for trauma patients treated in emergency departments and hospitals: a cost benefit analysis. Ann Surg,
241(4), 541-550.
Gonzales, A., Westerberg, V.S., Peterson, T. R., Moseley, A., Gryczynski, J., Mitchell., S. G., Buff, G., &
Schwartz, R. P. (2012). Implementing a state-wide SBIRT service in rural health settings: New Mexico
SBIRT. Substance Abuse, 33(2), 114-123.
Implementation of adolescent SBIRT 12
Goplerud, E., & Anderson, D. R. (2008). 2008: The breakout year for SBI. Alcoholism and Drug Abuse
Weekly, 20(1), 5.
Gryczynski, J., Mitchell, S. G., Peterson, T. R., Gonzales, A., Moseley, A., & Schwartz, R. P. (2011). The
relationship between services delivered and substance use outcomes in New Mexico's Screening, Brief
Intervention, Referral and Treatment (SBIRT) Initiative. Drug Alcohol Depend, 118(2-3), 152-157. PMCID:
PMC21482039.
Heckman, C. J., Egleston, B. L., & Hofmann, M. T. (2010). Efficacy of motivational interviewing for
smoking cessation: a systematic review and meta-analysis. Tob Control, 19(5), 410-
416.
Hingson, R. W., Heeren, T., & Winter, M. R. (2006). Age at drinking onset and alcohol dependence: age
at onset, duration, and severity. Arch Pediatr Adolesc Med, 160(7), 739-746.
Hollis, J. F., Polen, M. R., Whitlock, E. P., Lichtenstein, E., Mullooly, J. P., Velicer, W. F., & Redding, C. A.
(2005). Teen reach: outcomes from a randomized, controlled trial of a tobacco reduction program for
teens seen in primary medical care. Pediatrics, 115(4), 981-989.
Holmes, B. J., Finegood, D.T., Riley, B. L., & Best, A. (2012). Systems thinking in dissemination and
implementation research. In Brownson, Colditz and Proctor (Eds). Dissemination and Implementation
Research in Health: Translating Science to Practice. Oxford Press. New York, NY.
Hovmand, P. S., & Gillespie, D. F. (2010). Implementation of evidence-based practice and organzational
performance. Journal of Behavioral Health Services Research, 27(1), 79-
94.
Hovmand, P. S., & Gillespie, D. F. (2010). Implementation of Evidence-Based Practice and Organizational
Performance. Journal of Behavioral Health Services & Research, 37(1), 79-94. doi: 10.1007/s11414-008-
9154-y
Johnston, L. D., O“Malley, P. M., Bachman, J. G., Schulenberg, J. E., & Miech, R. A. Monitoring the Future
national survey results on drug use, 1975-2013: Volume |, Secondary school students. The University of
Michigan. Ann Arbor: Institute for Social Research.
Knight, J. R., Sherritt, L., Harris, S. K., Gates, E. C., & Chang, G. (2003). Validity of brief alcohol screening
tests among adolescents: a comparison of the AUDIT, POSIT, CAGE, and CRAFFT. Alcohol Clin Exp Res,
27(1), 67-73.
Knight, J. R., Sherritt, L., Shrier, L. A., Harris, S. K., & Chang, G. (2002). Validity of the CRAFFT substance
abuse screening test among adolescent clinic patients. Arch Pediatr Adolesc Med, 156(6), 607-614.
Implementation of adolescent SBIRT 13
Kunz, F. M., Jr., French, M. T., & Bazargan-Hejazi, S. (2004). Cost-effectiveness analysis of a brief
intervention delivered to problem drinkers presenting at an inner-city hospital emergency department. J
Stud Alcohol, 65(3), 363-370.
Lubman, D. |., Yucel, M., & Hall, W. D. (2007). Substance use and the adolescent brain: a toxic
combination? Journal of Psychopharmacology, 21(8), 792-794.
Madras, B. K., Compton, W. M., Avula, D., Stegbauer, T., Stein, J. B., & Clark, H. W. (2009). Screening,
brief interventions, referral to treatment (SBIRT) for illicit drug and alcohol use at multiple healthcare
sites: comparison at intake and 6 months later. Drug Alcohol Depend, 99(1-3), 280-295.
Martin, G., & Copeland, J. (2008). The adolescent cannabis check-up: randomized trial of a brief
intervention for young cannabis users. J Subst Abuse Treat, 34(4), 407-414.
Martinez-Moyana, Ignacio J., & Richardson, George P. (2013). Best practices in system dynamics
modeling. System Dynamics Review, 29(2), 102-123.
Martinez-Moyano, I. J. (2012). Documentation for model transparency. System Dynamics Review, 28(2),
199-208.
Mathers, M., Toumbourou, J. W., Catalano, R. F., Williams, J., & Patton, G. C. (2006). Consequences of
youth tobacco use: a review of prospective behavioural studies. Addiction, 101(7), 948-958.
McCambridge, J., McAlaney, J., & Rowe, R. (2011). Adult consequences of late adolescent alcohol
consumption: a systematic review of cohort studies. PLoS Med, 8(2), e1000413.
McCambridge, J., & Strang, J. (2004). The efficacy of single-session motivational interviewing in reducing
drug consumption and perceptions of drug-related risk and harm among young people: results from a
multi-site cluster randomized trial. Addiction, 99(1), 39-52.
Miller, J. W., Naimi, T. S., Brewer, R. D., & Jones, S. E. (2007). Binge drinking and associated health risk
behaviors among high school students. Pediatrics, 119(1), 76-85.
Millstein, S. G., & Marcell, A. V. (2003). Screening and counseling for adolescent alcohol use among
primary care physicians in the United States. Pediatrics, 111(1), 114-122.
Mitchell, S. G., Gryczynski, J., Peterson, T., Gonzales, A., Moseley, A., & Schwartz, R. P. (in press).
Screening, brief intervention, and referral to treatment (SBIRT) for substance use in a school-based
program: Services and outcomes. American Journal on Addictions.
Monti, P. M., Colby, S. M., Barnett, N. P., Spirito, A., Rohsenow, D. J., Myers, M., ... Lewander,
W. (1999). Brief intervention for harm reduction with alcohol-positive older adolescents in a hospital
emergency department. J Consult Clin Psychol, 67(6), 989-994.
Implementation of adolescent SBIRT 14
Moore, B. A., Fiellin, D. A., Barry, D. T., Sullivan, L. E., Chawarski, M. C., O'Connor, P. G., & Schottenfeld,
R. S. (2007). Primary care office-based buprenorphine treatment: comparison of heroin and prescription
opioid dependent patients. J Gen Intern Med, 22(4), 527-530.
Moyer, A., & Finney, J. W. (2004). Brief interventions for alcohol problems: factors that facilitate
implementation. Alcohol Res Health, 28(1), 44-50. Access at:
http://pubs.niaaa.nih.gov/publications/arh28-41/44-50.pdf.
Newacheck, P. W., Brindis, C. D., Cart, C. U., Marchi, K., & Irwin, C. E. (1999). Adolescent health
insurance coverage: recent changes and access to care. Pediatrics, 104(2 Pt 1), 195-202.
Ondersma, S. J., Svikis, D. S., & Schuster, C. R. (2007). Computer-based brief intervention a randomized
trial with postpartum women. Am J Prev Med, 32(3), 231-238. Peterson, A. V., Jr., Kealey, K. A., Mann, S.
L., Marek, P. M., Ludman, E. J., Liu, J., & Bricker, J.
B. (2009). Group-randomized trial of a proactive, personalized telephone counseling intervention for
adolescent smoking cessation. J Natl Cancer Inst, 101(20), 1378-1392.
Peterson, J. A., Schwartz, R. P., Mitchell, S. G., Reisinger, H. S., Kelly, S. M., O“Grady, K. E., Brown, B.S., &
Agar, M. H. (2010). Why don“t out-of-treatment individuals enter methadone treatment programs?
International Journal of Drug Policy, 21, 36-42.
Pincus, H. A. (1980). Linking general health and mental health systems of care: conceptual models of
implementation. Am J Psychiatry, 137(3), 315-320.
Pincus, H. A. (1987). Patient-oriented models for linking primary care and mental health care. Gen Hosp
Psychiatry, 9(2), 95-101.
Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation
research in mental health services: an emerging science with conceptual, methodological, and training
challenges. Adm Policy Ment Health, 36(1), 24-34.
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A.,... Hensley, M. (2011).
Outcomes for implementation research: conceptual distinctions, measurement challenges, and research
agenda. Adm Policy Ment Health, 38(2), 65-76.
Reinert, D. F., & Allen, J. P. (2007). The alcohol use disorders identification test: an update of research
findings. Alcohol Clin Exp Res, 31(2), 185-199.
Reisinger, H. S., Schwartz, R. P., Mitchell, S. G., Peterson, J. A., Kelly, S. M., O“Grady, K. E., Marrari, E. A.,
Brown, B. S., & Agar, M. H. (2009). Premature discharge from methadone treatment: Patient
perspectives. Journal of Psychoactive Drugs, 41(3), 285-296. ~
Repenning, N.A. (2002). A simulation-based approach to understanding the dynamics of innovation
implementation. Organization Science, 13(109-127).
Implementation of adolescent SBIRT 15
Roberts, N., Anderson, D., Deal, R., Garet, M., & Shaffer, W. (1983). Introducation to computer
simulation: A system dynamics modeling approach. Reading, Massachusetts: Addison-Wiley Publishing
Company.
Simpson, D. D., & Flynn, P. M. (2007). Moving innovations into treatment: A stage-based approach to
program change. J Subst Abuse Treat, 33(2), 111-120. Spirito, A., Monti, P. M., Barnett, N. P., Colby, S.
M., Sindelar, H., Rohsenow, D. J.,... Myers,
M. (2004). A randomized clinical trial of a brief motivational intervention for alcohol-positive
adolescents treated in an emergency department. J Pediatr, 145(3), 396-402.
Sunstance Abuse and mental Health Service Administration (2010). Results from the 2009 National
Survey on Drug Use and Health: Mental Health Findings. Center for Behavioral Health Statistics and
Quality (NSDUH Series H-39, HHS Publication No. SMA 10-4609). Rockville, MD
Swift, W., Coffey, C., Carlin, J. B., Degenhardt, L., & Patton, G. C. (2008). Adolescent cannabis users at 24
years: trajectories to regular weekly use and dependence in young adulthood. Addiction, 103(8), 1361-
1370.
Tait, R. J., & Hulse, G. K. (2003). A systematic review of the effectiveness of brief interventions with
substance using adolescents by type of drug. Drug Alcohol Rev, 22(3), 337-346.
Tait, R. J., Hulse, G. K., & Robertson, S. |. (2004). Effectiveness of a brief-intervention and continuity of
care in enhancing attendance for treatment by adolescent substance users. Drug Alcohol Depend, 74(3),
289-296.
Tripodi, S. J., Bender, K., Litschge, C., & Vaughn, M. G. (2010). Interventions for reducing adolescent
alcohol abuse: a meta-analytic review. Arch Pediatr Adolesc Med, 164(1), 85-91.
Ventana Systems, I. (2008). Vensim: Simulator for Business Open Education Resources. Ventana
Systems, Inc. (2008). Vensim®: Simulator for Business Open Education Resources (OER). Volkow, N. D. &
Li, T. K. (2005). Drugs and alcohol: treating and preventing abuse, addiction and their medical
consequences. Pharmacol Ther, 108(1), 3-17.
Wachtel, T., & Staniford, M. (2010). The effectiveness of brief interventions in the clinical setting in
reducing alcohol misuse and binge drinking in adolescents: a critical review of the literature. J Clin Nurs,
19(5-6), 605-620.
Walton, M. A., Chermack, S. T., Shope, J. T., Bingham, C. R., Zimmerman, M. A., Blow, F. C., &
Cunningham, R. M. (2010). Effects of a brief intervention for reducing violence and alcohol misuse
among adolescents: a randomized controlled trial. JAMA, 304(5), 527-
535. Windle, M. (2004). Suicidal behaviors and alcohol use among adolescents: a developmental
psychopathology perspective. Alcohol Clin Exp Res, 28(5 Suppl), 29S-37S. Winters, K. C., & Leitten, W.
(2007). Brief intervention for drug-abusing adolescents in a school setting. Psychol Addict Behav, 21(2),
Implementation of adolescent SBIRT 16
249-254. Wutzke, S. E., Shiell, A., Gomel, M. K., & Conigrave, K. M. (2001). Cost effectiveness of brief
interventions for reducing alcohol consumption. Soc Sci Med, 52(6), 863-870. Zarkin, G. A., Bray, J. W.,
Davis, K. L., Babor, T. F., & Higgins-Biddle, J. C. (2003). The costs of screening and brief intervention for
risky alcohol use. J Stud Alcohol, 64(6), 849-857.
Zarkin, G.A., Dunlap, L.J., & Homsi, G. (2004). The substance abuse services cost analysis program
(SASCAP): a new method for estimating drug treatment services costs. Evaluation and Program Planning,
27 (1):35-43.
Implementation of adolescent SBIRT 17
APPENDIX
Implementation of adolescent SBIRT 18
Table 1 - Screening rate
(implementation phase reference mode)
100%
90%
80% =
70%
60%
50%
40% FA
30% —<<Y
20%
10%
0%
nam nmnmomnmnom tort tr ttt tr t er
PS2PPESERRERE ERE RSSS
— Gen Screening ——Spec Screening ——All Screening ——Poly. (All Screening )
Table 2 - Endorsed risk rate
(implementation phase reference mode)
100%
80%
60%
40%
20%
0%
nam nmnmnmnmnom tort tt ttt ter
PS2ZPFSsRRSPESSFSRTFZSEB
—Gen Endorsed Risk ——Spec Endorsed Risk
——All Endorsed Risk —Poly. (All Endorsed Risk )
Implementation of adolescent SBIRT 19
Table 3 - Intervention rate for screened
adolescents (Implementation phase ref. mode)
100%
80%
60%
40%
20%
ee
~. ———
rmnynnnmnnmnnmnmnrst ts fF ttt rtrtr rs fF t+ Ft
SSaSeSeasABesBSSSSSBSRSZB ERS
><cs Pov e BGtCesse BE FZFEBYA BB B
$2728 0268S PS ESRBrZ8 026
—Gen Interv (Screened) —Spec Interv (Screened)
——All Interv (Screened) —Poly. (All Interv (Screened))
Table 4 - Intervention rate for adolescent visits
(Implementation phase reference mode)
100%
80%
60%
40%
20%
0%
ae eaaaaeaea ae SSAA STS SAS
PELPRSELERERFESER ERE
—Gen Interv (Pop) —Spec Interv (Pop)
——All Interv (Pop) —Poly. (All Interv (Pop))
