Evaluating Group Model Building in
Mental Health and Vocational Rehabilitation Service Delivery
Steven Huz, MPA, New York State Office of Mental Health
David F. Andersen, PhD., State University of New York at Albany
George P. Richardson, PhD., State University of New York at Albany
Roger Boothroyd, PhD., New York State Office of Mental Health
In recent years, facilitated system dynamics group model building has been used
to facilitate and support decision making in management teams working to solve
complex problems in both private and public sector settings. This approach is
considered to be an important tool for improving the system thinking and decision
making capabilities of participating managers. The goal of these interventions is to
strengthen a teams' ability to work toward developing an agreed upon course of action
that will ultimately lead to resolving the problem at hand. Although there are strong
indications that these approaches provide great benefits, little research has empirically
explored the degree to which group system dynamic model building successfully
facilitates this outcome. In New York State, a system dynamics group model building
strategy is being systematically examined as an approach to promote public sector
system change in mental health and vocational rehabilitation service delivery systems.
To test the impact of group model building researchers at the State University of
New York at Albany and the New York State Office of Mental Health (NYSOMH) have
collaboratively designed an experiment which builds on an innovative ongoing policy
initiative aimed at improving access to vocational rehabilitation services for individuals
with serious mental illness. The project, supported by the National Institute of Mental
Health funded Center for the Study of Issues in Public Mental Health Services,
measures and compares system change outcomes for local areas in which key actors in
the mental health and vocational rehabilitation are actively engaged in examining the
issue of services integration between these systems and receive differential
interventions to assist their efforts.
Public Policy Problem
Delivery of publicly funded vocational rehabilitation services to persons with
severe mental illness has been problematic. The problem is compounded by the
structural fragmentation of service delivery promoted at the state level. Currently, the
435
Evaluating Group Model Building in Mental Heaith and Vocational Rehabilitation Service Delivery 2
Steven Huz, David F. Andersen . George P. Richardson, and Roger Boothroyd
New York State Office of Mental Health (NYSOMH) has primary responsibility for service
provision to people with serious mental illness, while the Office of Vocational and
Educational Services for Individuals with Disabilities (VESID), an administrative entity
located in the state's Department of Education, is charged with overseeing delivery and
coordination of vocational rehabilitation services. This arrangement has resulted in
difficulties in access and coordination of vocational rehabilitation services for individuals
with severe mental illness.
Through a recently signed, formal memorandum of understanding, the chief
administrative officers of the NYSOMH and VESID committed their agencies to
addressing this difficult situation. To further encourage this general statement of policy,
a demonstration, Services Integration Project (SIP) was implemented in seven counties
in NYS. In these areas, teams comprised of representatives of both service systems
were empowered to actively explore the integration of services between the mental
health and vocational rehabilitation systems in their local areas. The study to test the
efficacy of Group Model Building was developed in the context of this policy setting. THE
NYSOMH and VESID both agreed to use the opportunity presented by this new policy
initiative as a means to test group model building as an innovative approach to
promoting system change.
Research Design for Examining Outcomes of Group Model Building
The study examining the impact of group model building focuses on individuals
from eight counties in New York State who are key actors in the local mental health and
vocational rehabilitation systems. In four counties these individuals will participate in a
Group Model Building decision support conference specifically designed to facilitate
mental health/vocational rehabilitation services integration. Instrumentation designed to
measure system change and group functioning outcomes are administered to these
individuals both prior to and following the model building exercise. Equivalent data
collection instrumentation are also administered to individuals in four counties where the
group model building is not implemented. These data will be analyzed to assess the
degree to which progress toward achieving an integrated service delivery system has
been made in each local area. System change outcomes measured are the degree to
334
Evaluating Group Model Building in Mental Health and Vocational Rehabilitation Service Delivery 3
Steven Huz, David F. Andersen , George P. Richardson, and Roger Boothroyd
which important characteristics of an integrated system and processes which move a
system toward integration are present in a local area. A Organizational Network
Analysis is also administered to assess shifts in relationships among key provider
organizations in the area. Instrumentation designed to measure group member
impressions of group functioning is also administered. These data will be analyzed to
assess the impact of a group model building on the group of individuals who participate.
Change in group cohesion, goal clarity, group fragmentation, and the openness of group
process will be assessed in each comparative study condition. In addition, analytic
techniques to explore shifts in variability of perceptions of study participants on the goals
of services integration and the means to achieving integration are also being examined
to assess the impact of group model building using system dynamics on moving
participants toward a greater shared vision and shared understanding of the services
integration issue and on what steps need to be taken to achieve the desired outcome.
The Group Model Building Intervention for Mental Health/Vocational Rehabilitation
Services Integration
The Group Model Building Intervention that is tested by this research was
developed in a pilot phase of the project. The intervention consists of four meetings
which occur over approximately six months. The initial meeting serves to orient the
participants to the process in which they will participate. The system dynamic method is
explained along with precise expectations from participants. The second meeting is a
two-day group model building conference where the group is interactively engaged in the
process of developing a client flow system dynamics model which reflects the groups
consensus view of how individuals flow through the local service system. Modelers
attending the conference use data elicited by the facilitator to construct a system
dynamics simulation model. The model is used to explore the impact of various policy
scenarios on the behavior of the system. The system dynamic modelers share insights
from the group constructed model with the group. The session ends with an action plan
and responsibility list for moving toward a more integrated system.
The third and fourth meetings of the intervention are one day sessions in which
the facilitators continue to explore insights into system behavior based on simulations of
ABS
Evaluating Group Model Building in Mental Health and Vocational Rehabilitation Service Delivery 4
Steven Huz, David F. Andersen , George P. Richardson, and Roger Boothroyd
the system dynamics model constructed by the group. Group members report on
progress in action areas identified in previous meetings and leave each session with a
revised action plan for moving toward a more integrated service system.
Preliminary Outcomes
A pilot test of the group model building intervention and the instrumentation to
measure system change and group functioning showed that some change could be
detected. In the area of system change, the network analytic data suggest that
movement toward establishing important pre-conditions for system change had occurred
in the short time between the pre- and post- group model building data collection.
Overall, respondents reported a significant increase in the presence of situational factors
necessary for change. Inspection of specific situational factors revealed that participants
perceived the greatest increase in consensus about the vocational needs of persons
with mental illness. With respect to structural conditions within the network, no
significant changes were found from pre- to post- assessment. Although there was no
significant increase reported in the effectiveness and utility of the interorganizational
relationships, a significant increase in the level of satisfaction with these relationships
was present. Attitudinal data from participants at the pilot site also show changes that
coincide with observed changes in behavior of the group and new activities directed at
changing processes involved with delivering vocational rehabilitation services to
individuals with serious mental illness. Analysis of pilot data on movement toward a
shared understanding of the problem showed change in the direction of greater
agreement on dimensions of services integration for mental health and vocational
rehabilitation.
13
