Rubrics for Understanding
Lees Stuntz William Costello Debra Lyneis
Creative Learning Champlain Valley Union High Creative Learning Exchange
Exchange School RR#3, Box 160 1 Keefe Rd.
1 Keefe Rd. Hinesburg, VT 05461 Acton, MA 01720, USA
Acton, MA 01720, USA Phone (802) 482-7100 Phone-978-287-0070
Phone- 978-287-0070 will@ cvumail.cvu.cssd.k12.vtus Fax- 978-287-0080
Fax-978-287-0080 Lyneisd@ clexchange.org
Stuntzln@ clexchange.org
Abstract
A group of K-12 educators have developed a set of rubrics for assessing the
understanding generated by the use of system dynamic tools. These rubrics were used at the
DynamiQUEST exposition for grades 5-12 students in May of 2000. This is a presentation of the
rubrics which includes standards for behavior over time graphs, causal loop diagrams, stock/flow
maps and system dynamic models as well as an over-arching rubric for assessing the
understanding generated by the use of these tools. Feedback from those who read the rubrics is
sought to enhance efforts to bring standards to the use of system dynamics in K-12 education.
Introduction
DynamiQUEST 2000 was a celebration of student work in the field of system dynamics
designed to bring together students and teachers who are learning to think systematically and
employing the tools and method of system dynamics to increase understanding. For six years
there has been a fair of system dynamics projects by high school students in the Portland, Oregon
area called SyM Bowl. This fair has highlighted the work of students who are taking system
dynamics modeling classes.
A group of K-12 educators in the New England area wished to build on the extensive
work and energy poured into the SyM Bowl by creating an exposition on the East Coast of the
United States. There were a few significant characteristics, which the educators wished to
capture in their event:
1. Anemphasis on grades 5-12 (ages 11-18)
2. Anatmosphere of encouragement and learning
3. Demonstrations of the system dynamics tools put in the context of the understanding they
generate
4, A set of understandable standards for teachers and students to measure their work
One of the primary concentrations of the event was to give students appropriate and helpful
feedback in order for their participation to be a learning experience. Instead of Judges, there
were Coaches. It was a lot harder for the adults running the exposition to be consistent about the
lack of the competitive process than it seemed to be for the students to accept and enjoy the
freedom. There were no “prizes”. In order to enhance the learning, the group felt that the
standards had to be clearly delineated. Although there has been some work done in this realm
both by the SyM Bowl committee and individually by several school systems, no clear set of
standards existed for such projects.
Students could enter original work or their application of other work (from the Creative
Learning Exchange or CC-STADUS/SUSTAIN in Portland, etc.) This might include a unit, a
lesson, or an application of one or more of the systems tools. The objective was to meet the
standards that apply to the work submitted for DynamiQUEST, whether it is a BOTG or an
extensive project with a working model. The philosophy of DynamiQUEST is to improve student
learning by honoring the work of students as they learn the tools and method of SD/ST.
In order to set the standards, the group designed a set of rubrics for four system dynamics
tools: behavior over time graphs, causal loop diagrams, stock/flow diagrams and system
dynamics models. Over arching all of these is the Rubric for Understanding which delves into
the understanding generated by the use of the SD tools. The rubrics were used by coaches to
provide feedback related to students’ use of systems tools and development of systems
understanding. One of the major goals of presenting these Rubrics at this conference is to
encourage you all to participate with us to make the Rubrics standards that can be used for any
age learner.
RUBRICS FOR UNDERSTANDING:
USING SYSTEM DYNAMICS TOOLS
Available at www.clexchange.org under the name of SE1999-11RubricsF orSDT ools
These rubrics were created to be used as a complete package. Use of any one
of the Rubrics without the Rubric for Understanding does not further the
powerful learning which can occur when system dynamics tools are utilized.
Creating a system dynamics model, for instance, could necessitate the use of
all of the other four Rubrics. If only a Behavior-Over-Time-Graph (BOTG)
were being created, the BOTG Rubric and the Rubric for Understanding
would be pertinent.
Created by
The DynamiQUEST 2000 C ommittee*
November, 1999
* The DynamiQUEST 2000 committee:
Dan Barcan and Sue Jamback, Chelmsford Public Charter School
Alan Ticotsky and Rob Quaden, Carlisle Public Schools
Larry Weathers and Dick Maki, Harvard Public Schools
Will Costello, Waters Grant Project and Chittenden South School District
Steven Roderick, Lincoln-Sudbury Regional School District
Lees Stuntz and Debra Lyneis, Creative Leaming Exchange
Permission granted to photocopy and/or electronically distribute for educational, nonprofit uses only.
Copyright Creative Leaming Exchange
RUBRIC FOR UNDERSTANDING
For use with all other Rubrics
CATEGORY MEETS STANDARD COMMENTS
The question is clearly defined
DEFINITION OF THE The question has a manageable
QUESTION focus (is not so broad as to defy
description or analysis)
Time horizon The time over which the system
Boundaries of the system
operates is defined and
appropriate.
The limits of the system of
interest are indicated and
exogenous factors are
differentiated from endogenous
Key factors in the system factors.
The choice of key factors in the
system is supported.
A written description of the
DESCRIPTION OF behavior of the system is
MENTAL MODEL OF
SYSTEM
Behavior
Assumptions
included, with details about how
the system behaves including
major feedback loops and
delays.
Reference mode or hypothetical
BOTG is evident for major
stocks (levels) in the system.
Assumptions or simplifications
used to develop the mental
model of system behavior are
clearly described.
PRESENTATION
Graphs, Pictures and
Tables
Easy to Understand
Display
The presentation clearly
describes the system and system
behavior under study for the
appropriate audience.
Visual tools are used to support
the presentation and analysis.
The display is clear, neat, and
aesthetically pleasing.
All elements of the diagram are
clearly and accurately labeled.
REFLECTION
“What have you learned?”
In this project the student has
addressed the following questions:
a How has your original mental
model been changed?
Did anything surprise you?
What part was the most difficult
to do?
a Did your project answer your
original question? If not, what
have you leamed from the
process?
a Is anything still missing from
your diagram or model? How
could it be improved?
a How can you use what you have
learned?
a Are there leverage points to
change the system?
a What better questions can you
ask now?
RUBRIC FOR BEHAVIOR-OVER-TIME GRAPHS
For use with the Rubric for Understandin
CATEGORY MEETS STANDARD COMMENTS
a Horizontal axis represents time.
VARIABLES: a Increments are clearly labeled
“What is changing?” and related to the behavior. Can
be numeric or descriptive. (Ex.
1. Horizontal Axis numbers of minutes; or “Begin-
(Time) ning, Middle, End” of story)
a Scale encompasses enough time
to show the complete pattem of
behavior.
a Vertical axis represents key
2, Vertical Axis variable being studied.
(Behavior) a All variables are nouns.
Increments are clearly labeled,
reasonable, and related to the
behavior. Can be numeric or
descriptive. (Ex. Degrees
Celsius, 1-100, or “Low,
Medium, High”)
a The graph is a continuous line
THE LINE: graph.
“How is the variable a The line accurately depicts the
changing?” behavior based on available
data.
a The graph shows the pattern of
behavior, focusing on the slope
of the line and change over time
rather than on only one event.
a_ If possible, the graph shows
lines for more than one related
variable.
USEFULNESS See Rubric for Understanding.
RUBRIC FOR CAUSAL LOOP DIAGRAMS
For use with the Rubric for Understandin
CATEGORY MEETS STANDARD COMMENTS
a All variables are nouns.
VARIABLES (the words)
a All nouns used represent
quantities worth measuring in
terms of the causality which the
loop is intended to explain.
a All +- (s/o) relationships
CONNECTIONS identified.
BETWEEN VARIABLES
a Arrows connect related
(+/- and S/O ) variables.
a Each arrow represents a causal
relationship.
a Arrows show some kind of
feedback.
a Connections are based on
evidence or data or reasonable
hypothesis.
a Delays are indicated.
USEFULNESS See Rubric for Understanding.
RUBRIC FOR STOCK/FLOW DIAGRAMS
For use with the Rubric for Understandin
CATEGORY
MEETS STANDARD
COMMENTS
STOCKS
(Accumulations)
a Stocks are nouns in the system
which have quantitative values
(absolute or relative) which can
change over time.
a They are the central elements of
the system.
a They are limited in number to
allow insight into system
behavior.
FLOWS
(Actions or processes that
change the stock. They can
flow into or out of a stock,
or both)
a Flows are described with terms
related to the same units as the
stock they change.
CONVERTER
(Information, events or
relationships which affect
a Converters are reasonably
perceived to influence the flow.
They are connected to flows by
arrows which show influence.
the rate of change of the
flows if they are not
constant)
FEEDBACK a Elements show circular causality
(Circular connections (feedback) within the system.
showing influences within [9 The diagram is easy to read,
the system) with a minimum of crossing
lines.
USEFULNESS a The structure of the problem is
evident from the diagram.
a Interdependencies and feedback
give insight into what makes the
system behave as it does.
See Rubric for Understanding.
RUBRIC FOR SYSTEM DYNAMICS MODELS
For use with all other Rubrics
CATEGORY
MEETS STANDARD
COMMENTS
CONCEPTUALIZATION
a The model has a clear purpose stated
in behavioral terms.
a_ A behavior over time graph shows
the behavior the model aims to
explain (a sketch based on
observation, reading, experience,
etc.).
a Key elements of the system and their
basic interrelationships are
hypothesized.
Boundaries are clear.
The topic is appropriate and useful to
model: it is dynamic (includes
feedback) and a model would provide
useful insight into the change. The
cause of the behavior is endogenous.
MODEL CONSTRUCTION
The Diagram
All Standards for Stock/Flow
Diagrams apply.
The Equations
(The assumptions)
a Equations describe causal
relationships in math terms.
a All stocks are accumulations and
their flows are in the same units per
time.
a Units are listed for each equation and
balance throughout the model.
a All equations are briefly documented
in simple terms.
a Long or complex equations which
cannot be explained simply are
broken into simpler converters for
clarity.
Equations (cont'd)
a All variables represent real world
entities; they can be “soft” or “hard.”
a All parameters (constants) have
reasonable, real world values.
a “Switches” are used sparingly.
a Graphic functions accurately capture
complex non-linear relationships,
Curves are smooth.
Simulation Mechanics
a The solution interval (DT) is ¢ 1/3 of
the smallest time constant to produce
asmooth curve.
a The length of simulation is chosen to
show the complete pattern of
behavior.
a Scales are appropriate for ease of
interpretation.
The Structure
a The model starts with a simple
structure and builds in complexity.
The simpler model is included if the
model is complex. Sectors may also
simplify a complex model.
THE OUTPUT
a The model accurately captures the
behavior, or an explanation tells what
is missing and why.
a The graphs show the behavior under
varying conditions.
a Supporting graphs show all other
important variables in the model,
their interrelationships.
a Tables may be used to explain
specific points.
USEFULNESS
See Rubric for Understanding.
