Structural Validation of Causal Loop Diagrams
James R. Burns
College of Business Administration
Texas Tech University
Lubbock, Texas 79409-2101
Philip Musa
College of Business Administration
University of Alabama - Birmingham
Birmingham, AL 35255
Abstract--In this paper, we present techniques for manually testing the validity of each
link in a causal loop diagram (CLD). A link is defined as consisting of an origination
quantity, a destination quantity and a connection edge between them. Each link is
considered as a separate and distinct causal hypothesis whose validity should stand on its
own merits. Following Goldratt, criteria for determining the validity of the CLD are
adapted and demonstrated. Eight possible validation criteria are considered: clarity,
quantity existence, connection edge existence, cause sufficiency, additional cause
possibility, cause/effect reversal, predicted effect existence, and tautology.
Bums & Musa: Structural Validation of Causal Loop Diagrams—July 2001, Atlanta SD Conference—p. 1
I. Introduction
In his book Goldratt’s Theory of Constraints, William Dettmer (1997) discusses
in detail the basics of the Goldratt thinking process. Of particular interest in this paper is
the discussion of categories of legitimate reservation. Goldratt originally asserted seven
such categories (1992). An eighth rule was added later. The purpose of these “logical
tools” is to espouse criteria that govern the acceptability of the connections. The criteria
constitute a framework of tests or proofs used to validate cause-and-effect logic.
Sterman (2000, p. 846) re-iterates what has been believed within the system
dynamics community for many years. Namely, models cannot be perfectly validated. It
may be impossible to create a perfect model that is perfectly valid. That is not the point
here. Some models are better than others; some models, while not completely valid,
possess a greater degree of authenticity than others. Sterman argues that all models are,
in a sense, wrong because there could always exist a counter example to which the model
did not conform to completely. While we would concede that, we would also
acknowledge that this is not our point. Some models are wrong more frequently than
others. And, we would argue that a model that fits known data and observations is better
than one that fits only some of the data and observations. Some models are better than
others and we are looking to create and validate such models.
Sterman (2000, p. 138, 139) presents material regarding the traditional
understanding that causal linkages have. He does not like the use of “s” and “o” to
Bums & Musa: Structural Validation of Causal Loop Diagrams—July 2001, Atlanta SD Conference—p. 2
designate link polarity and instead reverts to the more accepted use of “+” and “-“ to
designate positive and negative links. According to Sterman, “a positive line means that
if the cause increases, the effect increases above what it would otherwise have been, and
if the cause decreases, the effect decreases below what it would otherwise have been... .A
negative link means that if the causes increases, the effect decreases below what it would
otherwise have been, and if the cause decreases the effect increases above what it would
otherwise have been.” The links do not tell you what will happen, but rather what will
happen if the cause changes value. In this paper we shall adhere to these definitions of
links as exhibited in the CLD (Causal Loop Diagram).
Before these criteria can be applied, the purpose of the CLD should be asserted.
For example, the purpose of the CLD might be merely to capture the dynamic cycles of
influence that would serve to pinpoint where leverage points in the system exist. On the
other hand, the purpose of the CLD might be to facilitate the construction of a stock-and-
flow diagram leading to a simulation model. In the former case not very much detail is
required to capture and communicate the dynamics to others. In the latter case, extensive
detail is needed that will allow model builders to convert the CLD into a SFD (stock-and-
flow diagram).
Bums & Musa: Structural Validation of Causal Loop Diagrams—July 2001, Atlanta SD Conference—p. 3
II. Assumptions
System dynamicists want to construct CLD’s that communicate, that have some
semblance of reasonableness and that are accepted by stakeholders and others.
Specifically, the following is assumed by modelers who build CLD’s and present them to
others. For one, modelers want to build logically sound CLD’s. Second, modelers, at
some point, will also present their CLD’s to others to communicate and elicit action.
Third, modelers naturally develop an emotional attachment to their CLD’s (“pride of the
inventor”). Modelers, often express cause/effect connections that are intuitive to them
but not to others. Occasionally, intermediate quantities appear to be missing. At other
times there may be cause insufficiency; then there is the possibility that the effect could
be created by additional causes not considered by the modeler; still other instances of
cause-effect reversal may surface. Modelers don’t want to be embarrassed by presenting
CLD’s that appear to have flaws or are not intuitive. Rather, they look for affirmation as
well as advice regarding their constructions, although they may be somewhat sensitive to
offered in a non-threatening way. The stakeholders are assumed to be actively interested
in helping modelers improve their CLD’s and in contributing to the analysis of the
subject. The stakeholders are not interested in humiliating the modeler. The audience
has considerable substance and understanding of the subject matter, that the modeler may
want to take advantage of.
Before CLD’s are published and distributed for wide consumption, we would
argue that they should be scrutinized and examined by a few outsiders for face validity.
Bums & Musa: Structural Validation of Causal Loop Diagrams—July 2001, Atlanta SD Conference—p. 4
Goldratt is a strong proponent of the use of a person called a “scrutinizer” to assess
validity of the CLD-like structure before that structure was employed in any real-world
diagnosis/prognosis context. We would agree.
III. The Categories of Causal Reservation
Clarity. The first criteria for assessing the appropriateness of a CLD might be clarity.
Clarity refers to the extent to which the model clearly communicates the implied
causality. In the beer game, Senge (1990, p. 49) suggests that MY ORDERS PLACED
somehow impacts MY SUPPLIER’S INVENTORY BACKLOG. This is to say, a link is
directed from MY ORDERS PLACED to MY SUPPLIER'S INVENTORY BACKLOG.
There are many other causal connections hypothesized in the CLD exhibited on page 49,
but we will address just this one.
Occasionally, the causal model is presented without definition of what the causal
connections mean, what the quantities are and so forth. There is opportunity for
confusion here. One has to assume that the causal connections have their conventional
meaning as defined by Sterman. But the quantities require some explanation in order to
avoid confusion.
Questions that should get addressed during the clarity investigation of the CLD
include:
1) is any additional verbal explanation required for the cause and its effect to be
understood;
2) is the connection between cause and effect convincing at “face value;”
Bums & Musa: Structural Validation of Causal Loop Diagrams—July 2001, Atlanta SD Conference—p. 5
3) is this a “long link” (i.e., missing intermediate quantities and edges).
The quantities of interest in any CLD are parameters (constants) and variables.
This is in stark contrast to Goldratt’s trees where the entities are statements like “gas
mileage is deteriorating” and “my car is an older car.”
Any CLD construction exercise that takes place within a group context ought to
begin with a statement of purpose, a declaration of mode (descriptive or prescriptive) and
a determination of perspective (what manager, what team, or what stakeholder group,
etc.). This puts all participants on the same wavelength. Then the participants ought to
begin by listing all of the parameters and variables (quantities) that impact the problem of
interest. This list should then be transferred to Post-it notes with one quantity written on
each note. The Post-it notes permit the quantities to be re-arranged. The group can then
begin the process of pair-wise consideration of causal linkages between the quantities and
can draw-in those connections as they are “discovered.”
Returning to the Senge illustration of a link fom MY ORDERS PLACED to MY
SUPPLIER'S INVENTORY BACKLOG., it becomes clear that a lot of assuming must
“go under the table” to buy into this hypothesis. For example, evidently, the supplier has
no inventory on hand; if he did, then the orders placed would simply reduce the inventory
on hand. The causal hypothesis assumes the supplier carries a backlog of inventory;
many suppliers do not exercise this policy. Finally the causal assertion assumes the
customer is willing to be “wait-listed” until the supplier has inventory. This is just one
causal link among many that comprise the beer game. Assumptions like this ought to be
made explicit; if people knew the underlying assumptions of the beer game up front,
perhaps they would not make such unintelligent ordering decisions. For clarity sake, all
Bums & Musa: Structural Validation of Causal Loop Diagrams—July 2001, Atlanta SD Conference—p. 6
of the assumptions ought to be made explicit and stored in a database that could he easily
accessed by all who have an interest in the model.
Quantity Existence and Units Associated Therewith.
In this test, the analyst is concerned about the outright existence of the variable or
parameter in question. The variable may not be real, currently, but only postulated.
Nevertheless, if the variable or parameter provides a quantitative view of an issue or
subject area of interest, it is legitimate even if it has actually just been conceived. The
quantity should be catalogued and its “units” defined. By this we mean, the quantity
should be entered into a database along with its units.
Causality Existence.
In causality existence, the reality of the causal link between a pair of quantities is called
into question. Someone has some doubts as to the reality of the causal link, that is,
whether the source quantity actually causes the destination quantity on the other end of
the link. Causality existence challenges the existence of the link between the pair of
quantities. In order to verify the causal link several conditions are necessary. First both
quantities must be measurable quantities. An example of an unverifiable pair of
connected quantities might be observation> intuition. What are the units associated with
observation? What are the units associated with intuition? Moreover, what experiment
can we set up to validate the assertion that “the more we observe a phenomenon, the
more intuition we have about it?” How would we measure “observation? How would
we measure “intuition?” Y et, the systems thinking literature is replete with just these
Bums & Musa: Structural Validation of Causal Loop Diagrams—July 2001, Atlanta SD Conference—p. 7
kinds of causal connections— causal relations that are unverifiable. However, this is only
a problem when someone dishelieves the causal conjecture and challenges it. Then, it is
impossible to empirically verify it. Going back fifty years in time and rejoining some
philosophy scholars in tea talk, we would hear them tell us that empirically non-verifiable
assertions were “nonsensical,” based on their positivist approach to logic.
Cause Insufficiency.
Here we examine the target of a causal linkage and ask “can the causal link, by itself,
create the effect we are expecting in the target quantity?” If the answer is NO, there is
cause insufficiency. This is where Behavior Over Time charts are useful because it is
possible to ascertain whether behavior in the cause variable is producing the behavior
expected in the effect variable. We ask, “are there any significant cause factors
missing?” Taking the perspective of the effect variable and looking back to all of its
immediate cause antecedents, we ask, “are the exhibited cause variables sufficient to
produce the stated effect?” If our answer is no, we haven't been thorough in our
inclusion of all of the possible causal factors. In the original and subsequent world
models developed by Forrester (1961, 1968, 1970, 1971) and Meadows, et al., (1971,
1992), there is no disease (or the absence thereof) component in these models that
explicitly considers the effects of such epidemics as AIDS. This could be construed as
cause insufficiency in terms of the effect variables aggregate birth rate, aggregate death
rate, and population for these models; authenticity and accuracy are diminshed. Similar
statements could be made for the lack of inclusion of technology effects in the early
models that were developed before 1972.
Bums & Musa: Structural Validation of Causal Loop Diagrams—July 2001, Atlanta SD Conference—p. 8
Additional Cause.
Here, the argument is presented that the cause/effect assertion is not unique and that other
cause variables could independently produce the same effect. If may become necessary,
therefore, to postulate all of the cause/effect relations to fully capture all of the inherent
causality that could possibly produce the effect. For example, it may be possible to
verify that a particular effect is reality. It may not follow that the cause must therefore
exist as well, however, because several possible cause variables could have
independently produced the effect. A marketing vice president would examine the source
of low sales figures in the northeastem part of his territory. He could attribute this effect
to any number of possible sources: a new regional marketing rep for the region, recent
high interest rates, an extremely cold winter with high heating costs, a loss of consumer
confidence, etc.
Cause- effect Reversal.
Those who would scrutinize and challenge a CLD might do so on the basis of their
beliefs about the direction of one or more causal links. If they were to assert that the
direction of the link should be reversed, then they would be making a case for cause-
effect reversal. Thus, they would be stating that the stated effect is really the cause and
conversely. Frequently, in system dynamics modeling two variables enter into a tight
cause-effect cycle or loop in which each variable is both cause and effect. This is quite
acceptable and does not constitute a problem in terms of understanding which variable is
cause, which is effect. Both variables are cause variables; both are effect variables.
As an example of cause effect reversal, consider the postulated link high retail
Bums & Musa: Structural Validation of Causal Loop Diagrams—July 2001, Atlanta SD Conference—p. 9
sales > high level of consumer confidence. Clearly, the latter is the cause, and not the
effect. This type of cause effect reversal occurs frequently when indicator variables are
brought into the model. The model builder must be careful to get the edge going in the
right direction when using indicator variables. Indicator variables are always effect
variables first. It is much less likely that they actually cause something else to change in
the model, although this is entirely possible.
Similar statements could be made for symptoms taken in relation to causes. For
example, if a patient’s body temperature is high and he experiences pain in his abdomen,
then he may have appendicitis. However, it would be wrong to conclude that high body
temperature combined with abdominal pain caused the appendicitis. Clearly, these are
symptoms and the direction of the edges should be from appendicitis to body temperature
and to abdominal pain as effect variables.
Predicted Effect Existence.
If the suggested cause variable is really the culprit, what other effects could we also
observe as a result of this hypothesized cause? Let’s suppose there are several.
However, we observe none of those other effects in reality. Then we begin to have our
doubts as to the existence of the cause variable, since none of its predicted effects have
actually been observed. While the residual effects of a certain cause variable may be
uninteresting to the modeler in terms of the system that is being modeled, these
nevertheless would be helpful in assessing the existence of the cause variable and should
comprise a part of the variable’s documentation, in a database say.
Bums & Musa: Structural Validation of Causal Loop Diagrams—July 2001, Atlanta SD Conference—p.
10
Tautology.
A tautology is a re-statement of the term itself, a statement that is true by definition. In
many such cases, the effect is offered as a rationale for the existence of the cause; but, in
reality, these are one and the same. Thus tautologous statements of causality are circular
in terms of their reasoning.
Consider the argument that Sales > Revenues. Depending upon the units used in
conjunction with Sales and whether there are sources of Revenue other than Sales, it may
be true that Sales = Revenues. This would be true when Sales are measured in dollars (or
some other monetary unit, consistent with Revenues) and Revenues, also measured in
dollars come only from Sales. The assertion Sales > Revenues is a tautology. However,
if Sales are measured in units of the product and/or there are other sources of Revenue,
then the assertion Sales > Revenues is a legitimate causal assertion. This is why it is so
important to document the units associated with each quantity in the CLD and to
document the assumptions under which the link is true. Tautologies should be eliminated
from CLD’s because they provide absolutely no additional clarifying content regarding
the exact nature of the underlying causality; in effect they confuse and complicate the
causal map and prevent readers from comprehending the causal system with clarity.
Leaps of Faith
Occasionally, causal connections contain leaps of faith in which intervening quantities
are left out. This makes the causal connection less than transparent and subject to
question by scrutinizers. Consider the link Revenues > Sales. Following our logic
above, this could just be a tautology, given the right circumstances in terms of units used
for Sales, for Revenues and whether Sales are the only source of Revenues. However,
Bums & Musa: Structural Validation of Causal Loop Diagrams—July 2001, Atlanta SD Conference—p.
11
another interpretation is possible. More Revenues could lead to more Salespeople which
could lead to more Sales; that is, Revenues Salespeople Sales. Now, the causal
connection makes much more abundant sense. Before, the causal connection
Revenues Sales was either a tautology or contained a lead of faith. Now, it is clear
there is no tautology and that more Revenues does lead to more Sales. Taken in total, the
following cycle of influence is apparent.
Sales
Revenues
Salespeople
IV. Summary and Conclusion
The appropriateness of Goldratt’s Categories of Legitimate Reservation is apparent.
CLD facilitators and developers should encourage their participants to use these
categories to scrutinize CLD’s so that the postulated causal hypotheses could be more
authentic and accurate. A dialogue should develop in which each of the above-mentioned
categories is further explored in terms of its usefulness in validating CLD’s. One useful
supporting tool for CLD’s would be a database that provides a record for each and every
causal link in the CLD and provides along with that same record a verbal statement of
what the link means exactly and a reference supporting its validity. Such a database
might serve the same purpose as a data dictionary that is often used in database
development within information technology. Such a database might provide a definition
Bums & Musa: Structural Validation of Causal Loop Diagrams—July 2001, Atlanta SD Conference—p.
12
and existence justification for each of the quantities that make up the CLD and for each
of the links that comprise the CLD as well. If there was ever any doubt as to what
exactly the creator of the CLD intended, the database could be consulted to help resolve
any possible ambiguities.
References
1. Dettmer, H. William, Goldratt’s Theory of Constraints, Milwaukee, Wisconsin, ASQ
Quality Press, 1997.
2. Goldratt, Elihau, The Jonah Program, Nashua, New Hampshire: The Goldratt Institute,
1992.
3. J.W. Forrester, Principles of Systems. Cambridge, MA: Wright-Allen, 1968.
4, ___, Industrial Dynamics. Cambridge, MA: MIT Press, 1961.
5. ___, Urban Dynamics. Cambridge, MA: MIT Press, 1969.
6. ___, World Dynamics. Cambridge, MA: Wright-Allen, 1971.
7. D.H. Meadows et al., The Limits to Growth. New York: Universe Books, 1972.
8. , Beyond the Limits: Confronting Global Collapse/Envisioning a
Sustainable Future, Post Mills, Vermont: Chelsea Green Publishing Company, 1992.
9. Sterman, John, Business Dynamics: Systems Thinking and Modeling for a Complex
World, Boston: Irwin McGraw-Hill, 2000.
Bums & Musa: Structural Validation of Causal Loop Diagrams—July 2001, Atlanta SD Conference—p.
13
