Learning from erroneous models using SCYDynamics
Yvonne G. Mulder, Lars Bollen, & Ton de Jong
Abstract
Dynamic phenomena are common in science education. Students can lear about such system
dynamic processes through model based learning activities. This paper describes a study on
the effects of a learning from erroneous models approach using the leaming environment
SCY Dynamics. The study compared three conditions. Two experimental conditions where
students had to correct errors in a model were contrasted to working with a correct model. The
experimental conditions differed on whether or not the students had to detect the errors before
correcting them. Results indicate that this approach enhanced students’ model testing and
revising activities. Furthermore this approach was found to have a beneficial effect on
learning common errors. Contrary to expectations this approach showed no learning effect on
domain knowledge acquisition. The discussion further elaborates on improvements that might
enhance this learning from erroneous model approach.
Keywords: System Dynamics, modeling, learning, erroneous examples
Introduction
Science education often requires students to learn about dynamic systems, which are
notoriously difficult to understand. An example of such a complex dynamic system
commonly found in high school biology curricula, is the topic of the human glucose
regulatory system. Students are taught that the human body constantly needs glucose as it is
the basic source of energy. However, the glucose level should stay within a narrow range as
either too high or too low blood glucose levels can cause damage to nerves, blood vessels, and
organs. The dynamic process of glucose-insulin regulation keeps the blood glucose level
within this narrow range. Students often have difficulties understanding this system because it
consists of multiple variables that are interrelated in intricate ways. Also the dynamic
behavior of this complex system in which glucose accumulates and recedes over time is
difficult to understand (Grésser and Schaffernicht, 2012) and requires reasoning on multiple
levels (i.e., on the structure of the system and the behavior over time).
To facilitate students’ learning of such complex dynamic systems, the potential of learning by
modeling approaches is increasingly recognized in the Netherlands and elsewhere around the
world (CCSSO, 2013; NGSS Lead States, 2013; van Dijk et al., 2013). System Dynamics
(Forrester, 1968) models (hereafter: models) have the opportunity to aid students’
understanding of dynamic systems as they provide an overview of the relevant variables and
their intricate relations (Grosser and Schaffernicht, 2012; Mulder, Lazonder, and de Jong,
2014). Moreover, when these models have the form of an executable computer model, they
can show the behavior of the entire system over time. As such, these models give students the
opportunity to explore the effects of the components and their relations on the behavior of the
system as a whole.
Model based learning activities typically require students either to leam from an existing
model, or to construct a model themselves (e.g., Alessi, 2000; de Jong and van Joolingen,
2008). Existing models give students a direct overview of the model structure. Through
simulation, students can explore the model by changing the values of input variables and
observe the resulting behavior of the system. In contrast, the learning by creating models
approach requires students to first construct the model from scratch before it can be simulated.
The learning by creating models approach is in line with the basic ideas behind
constructionism, and as such presumably enhances knowledge acquisition. Through iterative
phases of model building, testing, and revising (cf. Hogan and Thomas, 2001) students
acquire a deeper understanding of the domain. Unfortunately, the advantages of this learning
by creating models approach are often hindered by students’ lack of model building skills.
Researchers repeatedly conclude that students need support for their model building activities
in science education in order to reap the benefits (e.g., Louca and Zacharia, 2012; Mulder,
Lazonder, and de Jong, 2010; VanLehn, 2013).
As creating models from scratch is too difficult for novice students, we propose an alternative
model based learning activity (i.e., erroneous model approach) which might bridge the gap
between learning from existing models and learning by creating models. Like learning from
existing models, this alternative approach presents students with a pre-constructed model.
However, to actively engage students in the modeling process, the provided model contains
errors which students have to correct. Learning from erroneous models requires students’ to
detect and correct the errors. In this manner, students have to engage in testing and revising
behavior like during learning by creating models, but students are less likely to become
overwhelmed by the actual model construction process.
Leaming from erroneous examples is gaining interest in a variety of domains, such as math
(e.g., Booth et al., 2013; Durkin and Rittle-Johnson, 2012; Grofe and Renkl, 2007; Isotani et
al., 2011; Tsovaltzi et al., 2012) and concept mapping (e.g., Chang, Sung, and Chen, 2002;
Hilbert, Nickles, and Matzel, 2008). Gro&e and Renkl (2007) summarize multiple arguments
in favor of learning from erroneous examples, indicating that encountering errors during the
learning process might lead to deeper understanding (e.g., because errors can trigger
reflections). Several studies have shown that compared to problem solving or learning from
correct examples, learning from erroneous examples leads to higher learning gains (e.g.,
Booth et al., 2013; Chang et al., 2002; Durkin and Rittle-Johnson, 2012; GroBe and Renkl,
2007; Hilbert et al., 2008; Tsovaltzi et al., 2012). However some studies could not find this
effect (e.g., Hilbert et al., 2008; Isotani et al., 2011).
These mixed findings suggest students do not always reap the benefits of this approach. One
of the risks of learning from erroneous examples is that students fail to detect and correct the
errors and instead gain incorrect knowledge. To compensate for this pitfall, Hilbert and
colleagues conclude that a prerequisite for effective learning from erroneous examples is the
availability of feedback. In learning by modeling, students have the opportunity to engage in
testing and revising activities by simulating their models. This gives them feedback on the
quality of the model and will indicate the errors in the model. Additionally, to compensate for
lack of error detection, the errors in the model can be indicated (for instance by highlighting)
leaving students with only the correction task. As such it could be expected that applying a
learning from erroneous examples approach in modeling will have a positive effect on
students leaming.
Research design and Hypotheses
The study described in this paper assessed the effects of a learning from erroneous model
approach and the differential effects of detecting and correcting the model errors. To do so
this study contrasted three conditions. Students in the first experimental condition (i.e.,
detection and correction; D&C condition) received an incorrect model where they had to
detect and correct the mistakes. Students in the second experimental condition (i.e., correction
only; C condition) also received an incorrect model where the errors were highlighted and
thus they only had to correct the mistakes. To assess the effects of the learning from erroneous
model approach, both experimental conditions were contrasted to a control condition (i.e.,
simulation; S condition) where students received a correct model which they could simulate.
Contrasting both experimental conditions will shed light on the differentiated effect of
detecting and correcting errors in models. It is expected that both detecting and correcting
errors increases students’ testing and revising behavior and enhances students’ knowledge of
the domain.
Learning environment SCY Dynamics
All students worked with the learning environment SCY Dynamics. SCY Dynamics is a stand-
alone modeling tool that originates from the SCY project (de Jong et al., 2010) where it was
created to allow students to build and work with System Dynamics models in an interactive
fashion. The main part of this learning environment is the model editor (Figure 1) where
students can create, inspect, and adjust their models. A dditionally the tool provides two tabs
where students can get feedback on the structure (bar chart) and dynamic behavior (graph
tool) of their models (Figure 2). SCY Dynamics is intended for secondary school students to
learn about system dynamic phenomena from the biology, chemistry, or physics curricula.
vom east iae
Figure 1. Model editor tab
Model editor. In the model editor part of SCY Dynamics the students can represent and
structure their knowledge of a particular domain in an executable computer model. As shown
in Figure 1 the editor uses principles from the System Dynamics formalism (Forrester, 1968).
In addition to the typical language elements in this context, e.g. stocks, constants, flows,
relations and auxiliary variables, SCY Dynamics provides a selection of qualitative relation
types that can be used to describe the nature of a relation between a stock and an auxiliary
variable or between two auxiliaries (e.g. linear, parabolic, or sigmoid). Internally,
SCY Dynamics replaces the qualitatively specified relation with a quantitative relation to
create an executable model. The quantitative representation is taken from a set of pre-defined
functions that can be specified by a teacher or a modeling expert. This feature alleviates the
mathematical complexity and skills needed by a learner to create sound models of complex
phenomena.
Figure 2. Bar chart (left panel) and graph tool (right panel)
Feedback. Whilst constructing their models, students can get instant feedback from the
SCY Dynamics tool on their model by means of a bar chart and graph tool. The bar chart
offers feedback on the current structure of the model, by indicating the number of correct,
incorrect, and non-specified variables, relations, and directions of relations. Using the graph
tool, students can inspect and learn about the dynamic behavior of their model by running the
model and evaluating the output. The information about correct and incorrect variables and
relations is derived from the above mentioned expert model, which also provides the
quantitative representations of qualitative relations.
Modeling task .In this study the SCY Dynamics tool was used to teach all students about the
glucose insulin regulatory system. Students could find information about the glucose-insulin
regulatory system in an instructional text. This text described the ‘supply and demand’
mechanisms that ensure that cells in the human body receive blood that contains the right
amount of sugar. Students received an assignment that was based on three scenarios: (1) the
case of homeostasis, where the blood glucose level reaches an equilibrium over time, (2)
eating high-calorie food, which creates a spike of glucose in the bloodstream, and (3) the case
of people with diabetes Type 1, where the body cannot control the blood glucose level. The
first scenario, homeostasis, served as a starting point as all students were instructed to inspect
and get feedback on the model, and correct any errors in the model. The subsequent scenarios
required students to apply the model to real life cases that affect the glucose-insulin regulation
process.
Although the assignment was identical for all students, the pre-defined model with which
students started the assignment differed across conditions. Students in the experimental
conditions started with a pre-defined model containing errors in the variables and relations as
well as in the type of relations. Consistent with studies on erroneous concept maps by (Chang
et al., 2002) and (Hilbert et al., 2008), 30% of the model was incorrect which resulted in a
total of six errors (2 elements, 2 relations and 2 relation types). In Figure 3 these errors are
indicated by highlighting. Students in the D&C condition started with the non-highlighted
version of this model and had to detect and correct the errors in this erroneous model.
Students in the C condition received a highlighted erroneous model as shown in Figure 3, so
they only had to correct the errors. Students in the S condition worked with a correct version
of the model.
Figure 3. Erroneous model
Procedure
Participants were 62 Dutch high school students aged 15-17 years. Participants were matched
to conditions based on class-ranked prior knowledge test scores (S (Simulation) condition: n =
21; C (Correction only) condition: n = 22: D&C (Correction & Detection) condition: n = 19).
Data were collected during two sessions: a 50-minute introduction, and a 100-minute
experimental session that were carried out in regular classrooms where students worked
individually. During the introductory session participants first completed a domain knowledge
pretest, then received a brief plenary introduction on learning by modeling, which was
followed by a brief tutorial that familiarized students with the learning environment. During
the experimental session participants first read an instructional text about the glucose-insulin
regulatory system before students worked on the modeling task. The leaming environment
SCY Dynamics stored all participants’ actions in a logfile, so students’ testing and revising
activities could be retrieved as the number of times students ran their models to inspect the
bar chart and graph tools. At the end of the experimental session, students filled out a domain
knowledge posttest and an error recognition test. The domain knowledge posttest was
identical to the domain knowledge pretest and consisted of 9 items addressing key domain
concepts and students’ understanding of the glucose-insulin regulatory system. Students’
answers to the items were scored using a rubric that allocated one point to each correct
response. The Cohen’s « inter-rater reliability estimate of this rubric was 0.89. The error
recognition test required students to indicate and correct the six errors on a paper version of
the models. The coding rubric of this test allocated one point for each correctly identified
error, and one point for each correctly corrected error, leading to a 12 point maximum score.
The Cohen’s « inter-rater reliability estimate of this rubric was 0.98.
Analysis of Results
Table 1
Summary of Participants’ Performance
S (n =21) C (n =22) D&C (n =19)
M sD M sD M sD
Domain knowledge
2.76 1.34 3.14 1.46 2.63 1.38
pretest scores
Domain knowledge 419 g.g4 436409 4AT.Ss«d:'SL
posttest scores
Enorrecognition 399 197 741 329 695 2.70
test scores
Testing and
revising activities
Bar chart runs 8.95 6.35 23.27 1468 3432 26.80
Graph runs 6.14 7.18 3.64 3.98 3.95 5.15
Table 1 reports students’ scores on the knowledge tests and the number of times students ran
their models to inspect the bar chart and graph tool. Students’ overall scores on the domain
knowledge pretest was 2.85, indicating that students in our sample had little prior knowledge
of the glucose-insulin regulation. Univariate analysis of variance (ANOVA) were performed
which confirmed that the slight between-group differences in pretest scores did not
significantly differ between conditions, F(2,59) =0.74, p =.482.
The number of times students ran their models to inspect the bar chart and graph tools as
shown in Table | are an indication of students’ testing and revising activities. Multivariate
analysis of variance (MANOVA) produced a significant effect for experimental condition,
F(4,118) =4.82, p <.001, indicating that the learning from erroneous model approach
influenced students’ testing and revising activities. Subsequent ANOVA’s showed that the
leamming from erroneous model approach significantly affected the number of times students
ran their models to inspect the bar chart, F(2,59) = 10.48, p <.001, but not the number of
times students ran their models to inspect the graph tool, F(2,59) = 1.26, p =.292. Planned
contrasts were performed to pinpoint the effects of detecting and correcting errors on the
number of times students ran their models to inspect the bar chart. Significant differences
were found contrasting the S condition to both experimental conditions, t(61) =-4.20,
p <.001, r =.47, and when contrasting the C condition to the D&C condition, t(61) =-2.00,
p =.050, r =.25. This indicates that both detecting and correcting of errors independently
increase how often students use the bar chart for testing and revising activities.
Having established the influence of detecting and correcting errors on students’ testing and
revising behavior, additional analyses were performed to reveal the influence of the learning
from erroneous model approach on students’ learning. The domain knowledge posttest scores
reflect students’ understanding of the glucose-insulin regulatory system following the
modeling task. A mixed-design ANOVA was performed which combined the between-group
variable condition and the repeated-measures variable domain knowledge scores (on both
pretest and posttest), to analyze the effect of erroneous examples on students’ domain
knowledge increase. There was a significant main effect of the repeated-measures variable
domain knowledge scores, F (1,59) = 49.35, p <.001, which indicates that the students leaned
during the experiment. There was no significant main effect of the between-groups variable
condition, F (2,59) =0.55, p =.579, nor was there an interaction effect of the between-group
variable and the repeated-measures variable, F(2,59) = 0.78, p =.455. This means that we
found no indication that the learning from erroneous model approach influences students’
performance on the domain knowledge posttest, nor that this approach influences students’
increase on domain knowledge. Together this shows that both erroneous model learning and
learning from existing models approaches enhanced students’ domain knowledge, but that
detecting and correcting errors did not influence how much students’ learned from the domain
during the modeling activity.
The error recognition test score reflects how well students’ recognize errors and how capable
they are in correcting these errors following the modeling task. Students’ overall score on the
error recognition task was 6.08, indicating that, on average, students recognized (and were
able to correct) about half of the errors in the model. As can be seen in Table 1, there were
large differences between conditions on the number of recognized errors. ANOVA confirmed
that these between-group differences were significant, F(2,59) = 10.32, p <.001. Next,
planned contrasts were performed to pinpoint the effects of detecting and correcting errors on
the number of errors students recognized. Significant differences were found contrasting the S
condition to both experimental conditions, t(61) =-4.48, p <.001, r =.49, but not when
contrasting the C condition to the D&C condition, t(61) = 0.54, p =.590. This indicates that
correcting a model increases the number of errors that students recognize, but that detecting
errors in a model has no added effect on the recognition of errors.
Discussion
The aim of the study presented in this paper was to assess the effects of a learning from
erroneous model approach and to further differentiate on the effects of detecting and
correcting model errors. Compared to learning from (correct) existing models, detecting and
correcting errors was expected to increase both students’ testing and revising behavior and to
enhance students’ knowledge of the domain.
First, in line with expectations, detecting and correcting errors in models was found to
influence students’ testing and revising behavior. As expected, students used the bar chart
most often when they had to detect and correct errors in the model and least often when they
only had to simulate a correct model. However, contrary to expectations, this effect of
detecting and correcting errors was not found regarding the number of times students ran their
model with the graph tool. This could easily be explained in terms of the feedback function of
these tools. When confronted with an erroneous model, testing and revising activities are
more likely to occur compared to when confronted with a correct model. Furthermore, two-
thirds of the errors in the erroneous model regarded the structure of the model. Since the bar
chart tool offers direct feedback on the model structure, it makes sense that the effect of the
erroneous models in this study on testing and revising activities was more pronounced in the
number of times the students used the bar chart tool compared to the graph tool.
Second, the hypothesized effect of detecting and correcting errors in models on students’
domain knowledge acquisition was partially confirmed. The model based learning activities in
the three conditions were all found to increase students’ learning of the domain. However, the
effect of detecting and correcting errors on students’ learning only showed with regard to how
well students’ can recognize errors and are capable of correcting these errors, but not with
regard to acquiring knowledge of the domain. These results could help explain why the
existing research on the effectiveness of learning from erroneous examples paint a mixed
picture as only some, and not all, studies report that erroneous examples leads to higher
learning gains (Booth et al., 2013; Chang et al., 2002; Durkin and Rittle-Johnson, 2012;
Grofe and Renkl, 2007; Hilbert et al., 2008; Isotani et al., 2011; Tsovaltzi et al., 2012). This
study indicates that erroneous examples only influences acquisition of knowledge on the
targeted errors.
However, this conclusion is not in line with the Hilbert et al. (2008) study, where learners
were found to acquire incorrect knowledge during a concept map correction task. Based on
their findings, Hilbert and colleagues conclude that feedback is essential for a learning from
erroneous examples approach, as to prevent the students from acquiring incorrect knowledge.
Students in the current study did have this suggested feedback option and did not show
acquisition of incorrect knowledge. This supports Hilbert’s conclusion that feedback is a
prerequisite in order for a learning from erroneous examples approach to be effective.
The results of this study have a clear practical implication for science education. The leaming
from erroneous models might be a fruitful approach in teaching students about dynamic
phenomena on which students typically have persistent misconceptions. By creating models
which harbor these misconceptions and having students correct these errors, students gain a
more correct understanding of the domain. As this study showed no difference between the
experimental groups on learning, the most practical approach is to highlight the errors in the
model so students can fully focus their attention on correcting them.
Future research in this area should focus on advancing this learning from erroneous model
approach, in such a way that it also enhances students’ acquisition of domain knowledge. The
question remains whether, and how, erroneous models can facilitate acquisitions of correct
domain knowledge. The present findings suggest that students focus only on the errors in the
model and neglect the correct aspects and the system as a whole. Traditional leaming from
worked example approaches are typically enhanced by applying self-explanation prompts.
These prompts trigger self-explanations during the learning activity which are commonly
known to substantially foster learning outcomes (e.g., Chi et al., 1989; Renkl, 1997).
A first attempt to apply these prompts to erroneous examples by Grofe and Renkl (2007)
showed no effect of these prompts, presumably because the errors in the model diminished the
quality of students’ self-explanations. Future research should find a means to compensate for
this negative side effect. Instead of a general self-explanation prompt, students should receive
prompts that specifically direct them to explain the whole model and not only the errors. This
might pave the way for a broad practical application of the learning from erroneous models
approach.
Acknowledgements
This study was conducted in the context of the project “Learning through modeling and self
explanations” which is part of the National Initiative Brain and Cognition (NIHC) funded by
the Dutch Organization for Scientific Research (NWO), grant no. 056-31-011.
The authors gratefully acknowledge A nnelot Adolfsen, Odette Bunnik, Evelien Hannink, and
Anita Hoefakker for their help in collecting the data.
Bios
Yvonne G. Mulder is post doc researcher at the department Instructional Technology of the
University of Twente. She specializes in learning by modeling.
Lars Bollen studied Physics and Computer Science for Higher Education at the University
Duisburg-Essen in Germany, where he finished his PhD. in 2009 in the field of applied
computer science and mobile learning. Currently, he is working as a post-doctoral research
fellow at the Department of Instructional Technology at the University Twente in the
Netherlands. His research interests include model-based learning, modelling & sketching,
visual modelling languages and environments, mobile devices and pen-based devices in
leaming scenarios, and (inter)action analysis.
Ton de J ong is full professor of Educational Psychology at the University of Twente, Faculty
of Behavioral Sciences where he acts as department head of the department Instructional
Technology.
References
Alessi SM. 2000. Building versus using simulations. In Spector ].M., T.M. Anderson (eds.),
Integrated & holistic perspectives on learning, instruction & technology: Improving
understanding in complex domains. Kluwer, Dordrecht, The Netherlands, pp. 175-196.
Booth JL, KE Lange, KR Koedinger, KJ Newton. 2013. Using example problems to improve
student learning in algebra: Differentiating between correct and incorrect examples. Learning
and Instruction 25(0): 24-34.
CCSSO. 2013. The common core state standards for mathematics, Retrieved december 16,
2013 from www.corestandards.org.
Chang KE, YT Sung, ID Chen. 2002. The effect of concept mapping to enhance text
comprehension and summarization. J ournal of Experimental Education 71(1): 5-23.
Chi MTH, M Bassok, MW Lewis, P Reimann, R Glaser. 1989. Self-explanations: How
students study and use examples in learning to solve problems. Cognitive Science 13(2): 145-
182.
de Jong T, WR van Joolingen. 2008. Model-facilitated learning. In Spector M., M.D. Memill,
et al. (eds.), Handbook of research on educational communications and technology. Lawrence
Erlbaum Associates, New Y ork, pp. 457-468.
de Jong T, WR van Joolingen, A Giemza, I Girault, U Hoppe, J Kindermann, A Kluge, AW
Lazonder et al. 2010. Leaning by creating and exchanging objects: The SCY experience.
British Journal of Educational Technology 41(6): 909-921.
Durkin K, B Rittle-Johnson. 2012. The effectiveness of using incorrect examples to support
leaming about decimal magnitude. Learning and Instruction 22(3): 206-214.
Forrester JW. 1968. Principles of systems. Pegasus Communications, Waltham, MA.
GroRe CS, A Renkl. 2007. Finding and fixing errors in worked examples: Can this foster
leaming outcomes? Learning and Instruction 17(6): 612-634.
Grosser SN, M Schaffernicht. 2012. Mental models of dynamic systems: taking stock and
looking ahead. System Dynamics Review 28(1): 46-68.
Hilbert T, M Nickles, S Matzel 2008. Concept mapping for learning from text: evidence fora
worked-out-map-effect. In Proceedings of the Proceedings of the 8th international conference
on International conference for the learning sciences. Utrecht, The Netherlands. International
Society of the Learning Sciences, pp. 358-365.
Hogan K, D Thomas. 2001. Cognitive comparisons of students' systems modeling in ecology.
Journal of Science Education and Technology 10(4): 319-345.
Isotani S, D Adams, RE Mayer, K Durkin, B Rittle-Johnson, BM McLaren. 2011. Can
erroneous examples help middle-school students leam decimals?, Towards Ubiquitous
Learning. Springer, pp. 181-195.
Louca LT, ZC Zacharia. 2012. Modeling-based learning in science education: cognitive,
metacognitive, social, material and epistemological contributions. Educational Review 64(4):
471-492.
Mulder YG, AW Lazonder, T de Jong. 2010. Finding out how they find it out: An empirical
analysis of inquiry learners' need for support. International Journal of Science Education
32(15): 2033-2053.
Mulder YG, AW Lazonder, T de Jong. 2014. Key characteristics of successful science
learning: the promise of learning by modelling. Manuscript under review.
NGSS Lead States. 2013. Next generation science standards: For states, by states.
Washington, DC, The National Academies Press.
Renkl A. 1997. Learning from worked-out examples: A study on individual differences.
Cognitive Science: A Multidisciplinary Journal 21(1): 1-29.
Tsovaltzi D, BM McLaren, E Melis, A-K Meyer. 2012. Erroneous examples: effects on
learning fractions in a web-based setting. International J ournal of Technology Enhanced
Learning 4(3/4): 191-230.
van Dijk G, M Hajer, R Scharten, B de Vos. 2013. Werken aan vaktaal bij de exacte vakken.
Nationaal expertisecentrum leerplanontwikkeling, Enschede. Available from www.slo.nl.
VanLehn K. 2013. Model construction as a leaning activity: a design space and review.
Interactive Learning Environments 21(4): 371-413.
Anja Kreidler, Meike Tilebein
12
members. For insight into correlations on team level, for example how diversity affects
communication and innovativeness, a system dynamics model may more appropriate.
Table 3: Summary and A pplication Examples
System Dynamics Agent-based Modeling
Level Problem structures, dynamic effects, | Individual agents and interactions
feedback, and nonlinearity can be between agents.
modeled. Conclusions about the team can only
Statements about the behavior of be made as a function of single
single team members are not possible, | agents.
only the overall structure can be Aggregated data and emerging effects
modeled. cannot be modeled directly. It is
difficult to depict measurable effects
on a valid basis using agent-based
modeling.
Empirical Numerous empirical studies examine | Effects on the behavior of single
Data the behavior on team level and show _| persons are barely examined in
the basic correlations between
diversity and creativity,
innovativeness and performance.
Thus, empirical data exists as a basis
for creating a model.
(cognitive) empirical studies.
Additional studies that examine the
behavior of individual team members
are necessary.
Further factors that might have an
influence on the diversity/innovation
relationship must be considered: for
example it must be examined if
individual team members’ moods or
opinions influence team work.
Modeling the Dynamic Aspects of Team Diversity
13
Application
examples
Examination of the basis structures
and behaviors of diversity,
communication, and conflicts on
creativity, innovativeness, or
performance.
Example:
The effect of communication
structures on team level on
cooperation.
Examination of the behavior of
individual team members and
communication/conflicts between
team members.
Example:
Examination of fault lines and
subgroup formation.
Depiction of (dynamic) network
positions of individual team members.
Table 3 summarizes the focus of both methods, System Dynamics and agent-based modeling,
for diversity effects in teams. The modeling level of each method for the diversity/innovation
relationship is given, as well as the available data from empirical studies. Examples are listed
for which aspect of team diversity the modeling approach can be more applicable.
This paper offers a guideline for using simulation as a complementary research method for
investigating the effects of diversity in teams. We give researchers some insight to help
decide, which of the two simulation methods — System Dynamics or agent-based modeling —
might be better suited for analyzing a specific research question within the field of diversity
effects in teams.
Anja Kreidler, Meike Tilebein 14
References
Akgin, Ali E.; Dayan, Mumin; Di Benedetto, Anthony (2008): New Product Development
Team Intelligence: Antecedents and Consequences. In Information & Management 45 (4), pp. 221-
226.
Amabile, Teresa M. (1998): How to Kill Creativity. In Harvard Business Review 76 (5), pp. 76-
87.
Ancona, Deborah Gladstein; Caldwell, David F. (1992): Demography and Design: Predictors of
New Product Team Performance. In Organization Science 3 (3), pp. 321-341.
Blindenbach-Driessen, Floortje (2015): The (In)Effectiveness of Cross-Functional Innovation
Teams. The Moderating Role of Organizational Context. In IEEE Trans. Eng. Manage. 62 (1), pp. 29-
38.
Bunduchi, Raluca (2009): Implementing Best Practices to Support Creativity in NPD Cross-
Functional Teams. In International Journal of Innovation Management 13 (4), pp. 537-554.
Cabrales, Alvaro Lopez; Medina, Carmen Cabello; Lavado, Antonio Carmona; Cabrera, Ramén
Valle (2008): Managing Functional Diversity, Risk Taking and Incentives for Teams to Achieve
Radical Innovations. In R&D Management 38 (1), pp. 35-50.
Chen, Chung-Jen; Hsiao, Yung-Chang; Chu, Mo-An; Hu, Kae-Kuen (2015): The Relationship
Between Team Diversity and New Product Performance. The Moderating Role of Organizational
Slack. In IEEE Trans. Eng. Manage. 62 (4), pp. 568-577.
Chi, Nai-Wen; Huang, Yin-Mei; Lin, Shu-Chi (2009): A Double-Edged Sword? Exploring the
Curvilinear Relationship Between Organizational Tenure Diversity and Team Innovation: The
Moderating Role of Team-Oriented HR Practices. In Group & Organization Management 34 (6),
pp. 698-726.
Davis, Jason P.; Eisenhardt, Kathleen M.; Bingham, Christopher B. (2007): Developing Theory
Through Simulation Methods. In Academy of Management Review 32 (2), pp. 480-499.
Gebert, Diether; Boerner, Sabine; Kearney, Eric (2006): Cross-functionality and Innovation in
New Product Development Teams: A Dilemmatic Structure and its Consequences for the Management
of Diversity. In European Journal of Work and Organizational Psychology 15 (4), pp. 431-458.
Gibson, Cristina B.; Gibbs, Jennifer L. (2006): Unpacking the Concept of Virtuality: The
Effects of Geographic Dispersion, Electronic Dependence, Dynamic Structure, and National Diversity
on Team Innovation. In Administrative Science Quarterly 51 (3), pp. 451-495.
Guillaume, Yves R. F.; Dawson, Jeremy F.; Woods, Steve A.; Sacramento, Claudia A.; West,
Michael A. (2013): Getting diversity at work to work. What we know and what we still don't know. In
J Occup Organ Psychol 86 (2), pp. 123-141.
Hambrick, Donald C. (2007): Upper Echelons Theory: An Update. In Academy of Management
Review 32 (2), pp. 334-343.
Hambrick, Donald C.; Cho, Theresa Seung; Ming-Jer Chen (1996): The Influence of Top
Management Team Heterogeneity on Firms' Competitive Moves. In Administrative Science Quarterly
41 (4), pp. 659-684.
Modeling the Dynamic Aspects of Team Diversity 15
Happach, R. M.; Tilebein, M. (2015): Simulation as Research Method: Modeling Social
Interactions in Management Science. In Catrin Misselhorn (Ed.): Collective Agency and Cooperation
in Natural and Artificial Systems. Cham: Springer International Publishing, pp. 239-259.
Harrison, David A.; Klein, Katherine J. (2007): What's the Difference? Diversity Constructs as
Separation, Variety, or Disparity in Organizations. In The Academy of Management Review 32 (4),
pp. 1199-1228.
Homberg, F.; Bui, H. T. M. (2013): Top Management Team Diversity. A Systematic Review. In
Group & Organization Management 38 (4), pp. 455-479.
Jehn, Karen A.; Greer, Lindred L.; Rupert, Joyce (2008): Diversity, Conflict, and their
Consequences. In A.P. Brief (Ed.): Diversity at Work. Cambridge: Cambridge University Press.
Jehn, Karen A.; Bezrukova, Katerina (2004): A field study of group diversity, workgroup
context, and performance. In J. Organiz. Behav 25 (6), pp. 703-729.
Jehn, Karen A.; Mannix, Elizabeth A. (2001): The Dynamic Nature of Conflict: A Longitudinal
Study of Intragroup Conflict and Group Performance. In The Academy of Management J ournal 44 (2),
pp. 238-251.
Kratzer, Jan; Leenders, Roger Th. A. J.; van Engelen, Jo M. L. (2004): Stimulating the
Potential: Creative Performance and Communication in Innovation Teams. In Creativity and
Innovation Management 13 (1), pp. 63-71.
Kreidler, Anja; Tilebein, Meike (2013): Diversity and Innovationess in New Product
Development Teams. Adressing Dynamic Aspects with System Dynamics. In R. Eberlein, I. J.
Martinez-Moyano (Eds.): Proceedings of the 31st International Conference of the System Dynamics
Society. Cambridge, Massachusetts, USA.
Lau, Dora C.; Mumighan, J. Keith (1998): Demographic Diversity and Faultlines: The
Compositional Dynamics of Organizational Groups. In The Academy of Management Review 23 (2),
pp. 325-340.
Leenders, Roger Th A. J.; van Engelen, Jo M. L.; Kratzer, Jan (2003): Virtuality,
Communication, and New Product Team Creativity: A Social Network Perspective. In Journal of
Engineering and Technology Management 20 (1-2), pp. 69-92.
Leenders, Roger Th. A. J.; van Engelen, Jo M. L.; Kratzer, Jan (2007): Systematic Design
Methods and the Creative Performance of New Product Teams: Do They Contradict or Complement
Each Other? In Journal of Product Innovation Management 24 (2), pp. 166-179.
Milliken, Frances J.; Martins, Luis L. (1996): Searching for Common Threads: Understanding
the Multiple Effects of Diversity in Organizational Groups. In Academy of Management Review 21
(2), pp. 402-433.
Perry-Smith, Jill E.; Shalley, Christina E. (2003): The Social Side of Creativity: A Static and
Dynamic Social Network Perspective. In The Academy of Management Review 28 (1), pp. 89-106.
Schieritz, Nadine; Milling, Peter (2003): Modeling the Forest or Modeling the Trees - A
Comparison of System Dynamics and Agent-Based Simulation. In: The 21st International Conference
of the System Dynamics Society.
Sethi, Rajesh; Smith, Daniel C.; Park, C. Whan (2001): Cross-Functional Product Development
Teams, Creativity, and the Innovativeness of New Consumer Products. In Journal of Marketing
Research (J MR) 38 (1), pp. 73-85.
Anja Kreidler, Meike Tilebein 16
Simon, H.; Meyer, S.; Tilebein, M. (2008): Modelling the Coevolution of Organizations and
their Environments — The Case of Bounded Rationality in Computational Models of Organizations. In:
8th Annual Conference of the European Academy of Management (EURAM).
Sterman, John D. (2000): Business Dynamics. Systems Thinking and Modeling for a Complex
World. Boston [u.a.]: Irwin/McGraw-Hill.
Stolarski, Vera; Tilebein, Meike (2009): Diversity as a Knowledge Resource in Top
Management Teams - A Framework for Agent-Based Modeling. In: HICSS '09. 42nd Hawaii
International Conference on: System Sciences, 2009, pp. 1-10.
Tilebein, Meike; Stolarski, Vera (2009): The Contribution of Diversity to Successful R&D
Processes. The R&D Management Conference, 21.-24 Juni 2009 Wien Osterreich.
Vissers, Geert; Dankbaar, Ben (2002): Creativity in Multidisciplinary New Product
Development Teams. In Creativity and Innovation Management 11 (1), pp. 31-42.
West, Michael A. (2002): Sparkling Fountains or Stagnant Ponds: An Integrative Model of
Creativity and Innovation Implementation in Work Groups. In Applied Psychology: An International
Review 51 (3), pp. 355-387.
