Table of Contents
Evaluating the Effectiveness of Simulation-based Instruction about Water Resources in the
Middle School Science Classroom
Krystyna Stave, Lisa Upham, and Stephanie Fincher
UNLV Department of Environmental Studies
Kim Barchard
UNLV Department of Psychology
Abstract
This poster describes a research study comparing the effect of system dynamics-based
instruction versus traditional teacher-directed instruction about water resource management in 6"
grade science classes. The study tests the hypothesis that learner use of a simulation model
promotes greater understanding and better retention of the material presented than a teacher-led
presentation of the same material. Approximately 180 sixth grade students from a Middle
School in Las Vegas, NV participated in the study. Half received a “traditional” teacher-directed
lesson using visual aids (in the form of a powerpoint presentation) followed by a discussion led
by the teacher. The other half received the same introduction followed by a session in which
students used a simulation model to explore the issues on their own. Both groups were given a
pre-test, a test immediately following the instruction, and another test approximately one month
after the instruction. This poster presents and discusses the results of the study.
Introduction
Barry Richmond (1990) and others (e.g., Forrester 1992) have made a strong case for
using system dynamics models as a learner-directed teaching technique in K-12 education. They
claim that system dynamics-based instruction promotes a more constructive, versus an
assimilative, learning process than traditional teacher-directed instruction, and that when students
have to construct their own knowledge, they learn better and retain more. There is a good deal of
qualitative evidence supporting this view, but few experiments have been conducted that directly
compare the two instructional approaches and attempt to quantify the difference in learning.
The study described in this poster is an experiment that measures differences in learning
between students exposed to the same material in teacher-directed versus system dynamics-based
instruction. Study subjects are sixth grade students. The study is one part of a larger study
evaluating the effectiveness of system dynamics models for communicating resource
management issues in general. The overall study uses a simple model addressing water resource
management in Las Vegas, Nevada. Stave (in press) describes the model and its development.
In the first part of the overall study, the model was used in seven public workshops to test its
effectiveness as a tool for increasing public understanding of water management issues. Eighty-
three adult community members participated in the study, which is described in Cloud and Stave
(2001). Following the study described in this poster, a similar experiment will be conducted with
college students in Fall 2003. Finally, the same material will be put on the web and used to test
the potential for web-based public education using system dynamics simulation. Preliminary
ideas for the web-based experiment are described in Little and Stave (2001).
Hypothesis
The purpose of this specific study is to determine if among classes of sixth grade
students, those who are exposed to a learner-directed instructional technique, based on using
system dynamics modeling, will more completely understand the material presented, score higher
on an immediate assessment, and retain the information longer than those exposed to a teacher-
directed instructional technique involving lecture and a related activity not involving the use of a
computer-simulated model.
Procedures
Participants for this study were six classes of sixth grade students from Roy Martin
Middle School in Las Vegas, NV. Classes were randomly assigned to receive either the
“traditional” teacher-directed instruction or system dynamics model-based instruction. All
students began with a pre-test to determine their current level of knowledge related to the issues
of the Las Vegas Valley water supply and demand. All students received a powerpoint
pres entation introducing them to water management issues in Las Vegas, followed by a
discussion of management options. Following the discussion of options, the model users were
introduced to the model’s input/output screen, shown how to run the model, given a worksheet to
record the results of their experimentation with the model, and given approximately a half-hour
to experiment with the model. The non-model users were given graphs showing the results of
choosing different management options and led in a discussion about the differences. At the end
of the session, both groups were led in a discussion about the most effective management
strategy. Immediately after the instruction took place, students were tested to see how well they
learned some of the basic concepts. Finally, approximately one month after the instruction,
students were given a retention test to see how well the information they leamed was retained.
Student scores on the evaluation tests immediately following and one month after the
instruction were compared with scores on the pre-tests to measure the amount of learning and
retention. The learning and retention measures for the non-modeling group were compared with
the measures for the model users to evaluate the hypothesis.
Results and Discussion
This study is currently underway. The presentations and testing were conducted in the
week of April 28, 2003. Results are being analyzed in May. All analysis will be complete for
the conference presentation in July and will be included in the poster. Papers describing the
results, instructional material, evaluation instruments will be available at the conference.
References
Cloud, S.W. and K.A. Stave. 2001. Part II: Evaluating the potential of system dynamics
modeling for improving public participation in water resource management. (Abstract).
Proceedings of the 19" International Conference of the System Dynamics Society.
Atlanta, GA, August 6-10, 2001.
Forrester, J.W. 1992. System Dynamics and Learner-Centered-Learning in Kindergarten
through 12" Grade Education. Publication D-4337, Sloan School of Management, MIT.
Available online as a document in Roadmaps at: URL:
http://sysdyn.mit.edu/road-maps/rm-toc.html.
Little, R.E. and K.A. Stave. 2001. Part IIE Preliminary development toward a web-based
learning environment to broaden the audience for systems learning about water
management in Las Vegas, Nevada. (Abstract). Proceedings of the 19" International
Conference of the System Dynamics Society. Atlanta, GA, August 6-10, 2001.
Richmond, B. 1990. Systems Thinking: A critical set of Critical Thinking Skills for the 90's and
beyond. Pages 934-947 in: Andersen, D.F., Richardson, G.P. and J.D. Sterman (eds.)
System Dynamics ‘90 volume 2. Proceedings of the 1990 International System Dynamics
Conference. System Dynamics Society. Cambridge, MA.
Stave, K.A. (in press). A System Dynamics Model to Facilitate Public Understanding of Water
Management Options in Las Vegas, Nevada. Journal of Environmental Management.
