Do school students learn more about the environment
from a system dynamics model by themselves or with a
partner?
Kate Thompson and Peter Reimann
The University of Sydney
Building A35,
The Faculty of Education and Social Work
The University of Sydney
Australia, 2006
461290365001 and +61293516365, Fax: +61290365205
k.thompson@ edfac.usyd.edu.au and p.reimann@ edfac.usyd.edu.au
Abstract
As part of the first author’s PhD project, year 9 and 10 students were given a system
dynamics model of the impacts of visitors on a National Park. The students were given a
pre- and post-test to determine whether their knowledge of the environment changed.
Students were randomly assigned to either the individual learning condition (students
interrogated the model as individuals) or the collaborative learning condition (students
interrogated the model with one other student). There was a significant increase in the
environmental knowledge score for those students in the collaborative learning
condition, but not in the individual learning condition. The implications of this finding for
the use of system dynamics models in educational settings are discussed.
Introduction
Despite the literature that says that learning by modelling is an important method of
learning, there are few empirical studies of the success of modelling in science
classrooms (see for example Stylianidou, Boohan, & Ogbor, 2004). In particular, there
is little empirical data to support the claim that system dynamics modelling is a valuable
educational tool. Instead, case studies and anecdotal accounts from teachers who have
used such models are available on the web (Ragan, 1999; Verona, Ragan, Shaffer, &
Trout, 2001). Other studies have raised areas for further investigation. Studies
investigating the effect of systems thinking interventions on participants’ mental models
(Doyle, 1997), the effect of systems modelling on the development of higher cognitive
skills (Jonassen, 2003) and general learning effectiveness (Spector, 2000) are needed.
This study aims to investigate the use of a system dynamics model (SDM) as an
instructional tool to understand a complex socio-environmental system (a socio-
environmental system is one that incorporates society’s use of, or human impact on, the
environment). The system that was used in this study was visitor impact in a National
Park. In this paper, we will examine cooperative learning as one strategy to encourage the
acquisition of knowledge in the areas of the nature and function of ecosystems, how they
are related, and the impact of people on environments. Davies (2002) suggested that one
of the features of a simulation that was important for student engagement was allowing
for cooperative learning, prompting the investigation of collaborative learning in this
study. Some studies have shown that both model building exercises and learning with
models can promote systems thinking, improve learning outcomes and student attitude
toward the class (Friedman & McMillian Culp, 2001; Kiboss, Ndirangu, & Wekesa,
2004; Kurtz dos Santos, Thielo, & Kleer, 1997). Due to time limitations, it was decided
that students would participate in a learning-with-models activity.
System dynamics modelling was chosen as an appropriate modelling tool to represent this
socio-environmental system for a number of reasons. System dynamics modelling
requires higher level cognitive skills than those often addressed by computer-based
learning (Jonassen, 2000). Simulation models have been used successfully in physics
where improvements were found in students’ achievement, problem solving skills, and
overall understanding of the concepts (Jimoyiannis & Komis, 2001). However, there has
not been agreement in the literature on this success (de Jong & van Joolingen, 1998;
Hopson, Simms, & Knezek, 2001/2002). Reasons for poor results include: under
utilisation of the features available; different types of learners not accounted for; and an
inability to assess individual learners due to poor experimental design (de Jong & van
Joolingen, 1998). Generally, the skills required to fully participate in discovery leaning
are the ability to generate a hypothesis, applying a systematic and planning process, and
the use of high-quality heuristics for experimentation (de Jong & van Joolingen, 1998).
Of those studies that did test individual measures, certain characteristics typified
successful learning outcomes. They are direct access to domain information, students
receiving assignments, and the availability of a learning environment that allows model
progression (de Jong & van Joolingen, 1998).
Cooperative learning can be contrasted to collaborative learning because “cooperation
only requires that learners work together, each learner completing a part of the task,
rather than negotiating with others about all aspects of the task, as is necessary in
collaboration” (Beatty & Nunan, 2004, p. 166). Often, a group of students working
together at a single computer encourages collaboration because not everyone can have
control of their tasks (Beatty & Nunan, 2004). Cooperative learning can be an effective
learning environment, although some differences have been found in the way that male
and female students work in groups (Edwards, Coddington, & Caterina, 1997). Studies
have found that students in cooperative learning groups out performed individual learners
in a biology subject (Singhanayok & Hooper, 1998). Cooperative learning encourages
interaction with the tool (Singhanayok & Hooper, 1998); supports a range of learning
styles (Wang, Hinn, & Kanfer, 2001); and allows group members to explain concepts to
each other (Kramarski, 2004), which is a metacognitive strategy.
This paper will outline the results of one part of a larger study conducted by the first
author as part of her PhD project. A description of the model will be given, followed by a
description of the sample and instruments used. Finally the results and discussion will
address the key finding of the comparison of the individual and collaborative learning
environments, and implications for the use of SDMs in educational settings will be
discussed.
Methods
The SDM was developed between December 2005 and January 2006. The SDM was built
using STELLA™. STELLA™ has been used in a variety of both instructional and
experimental or explorational studies (for example, (Davies, 2002)). The model shows
visitor usage of a National Park (based on real data of the number of visitors per day
(Davison, 2000)) and the possible effects that this can have on the ecosystem. The model
used in this study showed a National Park picnic area used by visitors for seven
consecutive days (see Figure 1). Each day has a unique pattern of usage. As visitors leave
the park, they put one piece of rubbish each into the rubbish bin. Half of the rubbish left
is organic rubbish and the other half is inorganic. The garbage person comes at the same
time every day to collect the rubbish. If organic rubbish is left in the rubbish bins at the
end of the day, then rats eat half of it and the other half enters a process of
decomposition. The decomposition process takes three days, after which the organic
waste becomes additional nutrients in the environment. Graphs on the screen used by
students to experiment with the model monitor the amount of waste accumulating in the
park and the additional nutrients in the environment (Figure 2). Students are able to
interact with both models: the time the person comes to collect the garbage (5pm,
5:30pm, 6pm, 6:30pm, 7pm, or 7:30pm); the proportion of garbage collected (90%, 95%
or 100%); and the number of pieces of rubbish per person (1, 2, 3, or 4) are able to be
altered by students.
Visitors at picnic ground 1
o-—O— O—18
arrival cycle 1 — “departure cycle 1
4g waste accumulating
ot “9 te a eaten by animals
seoumulation Waele Eat the site oro yite Moone ten by anima ’
—OO—— KF 2: = = DE
(3 oe \
number of pieces of =" —\ sesstinfog waste )
rubbish per person ae )
ee ——_—_——~ / /
ss - : =
wT BS ¢ iQ
garbage collection time" a arbage Services decomposition
Z | Additional Nutrients in the soil
‘ =
@ 4 r—o—_
amount of garbage collected decomposing waste wtrients
to the soil
Figure 1: Stella™ model designed for this study
Figure 2: Experiment screen showing items that can be altered and graphs recording output
The dependent variable reported in this paper is knowledge of a socio-environmental
system. The independent variable is the learning environment (individual or
collaborative). Other independent variables including SES, attitudes towards both the
environment and science, experience with computers, motivation/enjoyment of the
learning experience, understanding of the system modelled and engagement with the
materials were measured but not reported in this paper. Four instruments were developed
to measure these variables. A background questionnaire identifies the gender, SES,
computer skills, attitudes towards science and attitudes towards the environment of the
students. In order to measure student’s knowledge about the socio- environmental system
an assessment task was developed. The assessment addresses general environmental
knowledge and knowledge specific to the topic of National Parks management. Students
were given this as a pre-test and a post-test. In this way, prior knowledge was controlled
for, and we were able to determine what students learnt as a result of being exposed to the
different materials. There was space on the pre-test to record post-test answers. When
students completed the post-test, they either retained their pre-test response (by recording
“as above” in the post-test space) or they recorded a different answer from that in the pre-
test. Students were also encouraged to record the response “I don’t know” if they did not
know the answer to a question. A final assessment task was also developed in order to
assess students’ understanding of the model itself. Finally, an evaluation was used to
determine students’ motivation/enjoyment of the learning experience. In addition, screen
captures of the students were recorded. These were processed into Quicktime movies, to
determine the length of time students spent on task, running and manipulating the
models, and the variables manipulated (Levy & Wilensky, 2005).
Altogether there were 15 students from two schools who participated in this part of the
study. The first school, school 1, was an academically selective girls high school;
students who participated from this school were in year 10. The second school, school 2,
was a girls 7-10 middle school; students who participated from this school were in year 9.
There were six students in the collaborative learning experiment, all from school 1, nine
students were involved in the individual learning experiment, four from school 1 and five
from school 2. Students in both schools were given an introduction to the purpose of the
experiment and a brief introduction outlining how the model would look, what could be
changed for experiments, and expectations regarding their work.
Due to the scheduling requirements at each school, the progression through the
experiment was different for each school. At school 1, students were introduced to the
experiment and completed the background questionnaire, and one week later the first
author returned to conduct the remainder of the experiment. At this point, students were
given 20 minutes to complete the pre-test, 20 minutes to look at the materials, and 20
minutes to complete the post-test, ten minutes to complete the final assessment and five
minutes to complete the evaluation. At school 2, students were introduced to the
experiment and completed both the background questionnaire and the pre-test during the
first meeting. The first author returned the followed day and students were given 20
minutes to examine the materials, and the remainder was as for school 1.
Results and Discussion
As was outlined above, two schools were used in the individual learning condition. The
first step then was to determine whether the school had an effect on the results. A Mann
Whitney test was used to compare the pre-test (school 1 Mdn=21; school 2 Mdn=11) and
post-test (school 1 Mdn=17.5; school 2 Mdn=13.5) environmental knowledge scores for
the individual learning (SI) group between the two schools. The results showed that there
was no significant difference for either the pre-test environmental knowledge scores
(U=3, p=111, r=.572) or the post-test (U=2, p=.063, r=.653), although large effect
sizes were present in both comparisons. For this reason it was decided that the results
from school 1 and school 2 could be combined for use in the SI group.
The pre-test and post-test scores were then compared for the SI group using a Wilcoxon
Signed Ranks test. In the SI group there was no significant difference between the pre-
test (Mdn=11.5) and post-test (Mdn=15.5) environmental knowledge scores (T=28,
p=514, r=.217). The same test was performed on the environmental knowledge scores
from the collaborative learning (SC) group. There was a significant difference found
between the pre-test (Mdn=18.75) and post-test (Mdn=23.5) environmental knowledge
scores (T=15, p=.043, r=.826).
The pre- and post-test environmental knowledge scores were then compared between the
two groups using Mann-Whitney tests. There was found to be no significant difference
between the pre-test environmental knowledge scores for the SI and SC groups (U=13.5,
p=114, r=.414). There was, however, a significant difference between the post-test
environmental knowledge scores for the SI and SC groups (U=4, p=.005, r=.701). Figure
3 shows that the post-test environmental knowledge scores were higher in the SC group
than in the SI group.
Post-test environmental
1B knowledge score (possible
25 total = 32)
‘|
Group
Figure 3: Box plot of the environmental knowledge pre- and post-test scores for the SC and
SI groups
The similarity in the pre-test environmental knowledge scores in the SI and SC groups
means that students in the two groups had similar levels of environmental knowledge
before being exposed to the SDM. After the two groups were given the SDM, only those
students in the collaborative learning condition increased their environmental knowledge,
and in fact this environmental knowledge was not only significantly higher than their
previous knowledge, but also significantly higher than that of those students in the
individual learning condition. This indicates that if students are in a dyad they may be
able to better understand the representation of a SDM than those who are approaching the
model as an individual.
Further investigation of the remaining data collected as part of this study is required to
explain these results. This will include analysis of students’ understanding of the system
itself (as opposed to the knowledge of environmental processes as explained above) as
well as students’ previous experience with computers and their enjoyment of the learning
experience. It is expected that enjoyment of the learning experience will play a role
because of the findings of the pilot study (Thompson & Reimann, 2006), which examined
only those students in the individual learning condition. In the pilot study, it was found
that students using the SDM did not find the content interesting, found the representation
difficult to understand, did not want more time with the materials, and found it difficult to
visualise the real situation. They interacted less with the model than students with other
representations. This may go some way to explaining the above results.
Although still a preliminary finding, the results reported in this paper, that students who
interrogated a SDM in a collaborative learning environment significantly increased their
environmental knowledge, while those in an individual learning environment did not, has
interesting implications for the use of SDM in educational settings. It may be that school
students need this collaborative learning environment when learning from SDMs in order
to understand this abstract, yet powerful representation.
References
Beatty, K., & Nunan, D. (2004). Computer-mediated collaborative learning. System, 32,
165-183.
Davies, C. H. J. (2002). Student engagement with simulations: a case study. Computers
& Education, 39, 271-282.
Davison, K. (2000). A Discrete Event Simulation Model of Visitor Usage at Botany Bay
National Park, Kurnell., The University of New South Wales, Sydney.
de Jong, T., & van Joolingen, W. R. (1998). Scientific discovery learning with computer
simulations of conceptual domains. Review of Educational Research, 68(2), 179-
201.
Doyle, J. K. (1997). The cognitive psychology of systems thinking. System Dynamics
Review, 13, 253-265.
Edwards, L. D., Coddington, A., & Caterina, D. (1997). Girls teach themselves, and boys
too: Peer learning in a computer-based design and construction activity.
Computers & Education, 29(1), 33-48.
Friedman, W., & McMillian Culp, K. (2001). Evaluation of the CoreModels Project.
Maryland: Education Development Center/Center for Children & Technology.
Hopson, M. H., Simms, R. L., & Knezek, G. A. (2001/2002). Using a technology-
enriched environment to improve higher-order thinking skills. Journal of
Research on Computing in Education, 34(2), 109-119.
Jimoyiannis, A., & Komis, V. (2001). Computer simulations in physics teaching and
learning: a case study on students’ understanding of trajectory motion. Computers
& Education, 36, 183-204.
Jonassen, D. (2000). Computers as mindtools for schools. engaging critical thinking.
(2nd ed.): Merrill.
Jonassen, D. (2003). Using cognitive tools to represent problems. J ournal of Research on
Technology in Education, 35(3), 362-381.
Kiboss, J. K., Ndirangu, M., & Wekesa, E. W. (2004). Effectiveness of a Computer-
Mediated simulations program in school biology on pupils' learning outcomes in
cell theory. Journal of Science Education and Technology, 13(2), 207-213.
Kramarski, B. (2004). Making sense of graphs: does metacognitive instruction make a
difference on students' mathematical conceptions and alternative conceptions?
Learning and Instruction, 14, 593-619.
Kurtz dos Santos, A. C., Thielo, M. R., & Kleer, A. A. (1997). Students modelling
environmental issues. Journal of Computer Assisted Learning, 13, 35-47.
Levy, S. T., & Wilensky, U. (2005). An analysis of student’ patterns of exploration with
NetLogo models embedded in the Connected Chemistry environment: Center for
Connected Learning and Computer-Based Modeling, Northwestern University,
Evanston, IL.
Ragan, S. (1999). Student-built computer models enhance learning in science and
mathematics. Retrieved 18/4/05, 2005, from
http://mvhs1.mbhs.edu/mvhsproj/coremodels/mast.html
Singhanayok, C., & Hooper, S. (1998). The effects of cooperative learning and learner
control on students' achievement, option selections, and attitudes. Educational
Technology, Research and Development, 46(2), 17-32.
Spector, J. M. (2000). System dynamics and interactive learning environments: Lessons
learned and implications for the future. Simulation & Gaming, 31(4), 528-535.
Stylianidou, F., Boohan, R., & Ogborn, J. (2004). Science Teachers' transformations of
the use of computer modeling in the classroom: using research to inform training.
Science Education, 89(1), 56-70.
Thompson, K., & Reimann, P. (2006). Knowledge and Understanding of an
Environmental System Using Two Different Types of Computer-based Models - a
Pilot Study. Paper presented at the Sharing wisdom for our future, Environmental
education in action: Proceedings of the National Conference of the Australian
Association for Environmental Education., Bubury, Western Australia.
Verona, M. E., Ragan, S., Shaffer, D., & Trout, C. (2001). Working paper: A case study
of materials development fostered by the MVHS CoreModels Project. Retrieved
18/4/05, from mvhs1.mbhs.edu/mvhsproj/workmatdev.pdf
Wang, X. C., Hinn, D. M., & Kanfer, A. G. (2001). Potential of computer-supported
collaborative learning for learners with different learning styles. Journal of
Research on Technology in Education, 34(1), 75-85.
