Developing Causal Map Codebooks to Analyze Policy
Recommendations: A preliminary content analysis of floodplain
management recommendations following the 1993 Midwest Floods
Michael A. Deegan, Ph.D.
National Academies of Science Post-Doctoral Research Fellow
The Institute for Water Resources, U.S. Army Corps of Engineers
1200 First Street, Unit 222, Alexandria VA 22314
(518)257-2554
deeganphd @gmail.com
Abstract
In July 2008, a congressional hearing was held to better understand the damages
incurred during the Midwest Floods of 2008. In March 2009, the Army Corps of Engineers
responded to a request made by Senator Boxer, who asked the Corps to report back on the status
of the 97 recommendations made in Sharing the Challenge (1994). The codebook for analyzing
recommendations helps answer a fundamental question asked by scholars and practitioners in
policy analysis: Which recommendations are most likely to be implemented?
Policy recommendations in public policy venues provide a course of action by framing
issues in ways that lead decision-makers to preferred solutions. This paper presents a codebook
for developing causal maps of policy recommendations. The codebook's strengths and
weaknesses are discussed as it is applied to a set of recommendations that were made to improve
floodplain management in the U.S. By developing codebooks that are reliable, consistent, and
transparent, the internal validity of causal maps constructed from qualitative data will improve
substantially.
Keywords: flood mitigation, policy recommendation, public policy, codebook, content analysis,
floodplain management, causal maps, causal loop diagram, issue framing
1 Introduction
Floodplain management policy recommendations made in public policy venues take on
several forms, such as congressional hearing testimonies after a major event or solicited advice
in reports provided by a task force (e.g., Sharing the Challenge in 1994). Strong policy
recommendations frame issues in ways that lead decision-makers towards a preferred set of
solutions.
The Midwest Floods of 2008 was a potential focusing event that shows despite efforts to
mitigate damages, the United States is still vulnerable to flooding. Many of the areas affected in
the 2008 flood had experienced flood damages in the Great Flood of 1993. After the 1993 flood,
a task force was commissioned to develop policy recommendations to improve the nation’s
floodplain management in the United States. The resulting document, Sharing the Challenge,
contained eight themes and 97 recommendations to change many aspects of federal policies on
floodplain management. The Midwest Floods of 2008 sparked Senator Boxer to ask the Army
Corps of Engineers to report back on the status of those 97 recommendations. The Army Corps
of Engineers think tank, the Institute of Water Resources, took the lead on this fact finding
mission and in April 2009, a status report was sent to Congress.
This paper presents a codebook for analyzing arguments that identify floodplain
management issues and support recommendations for floodplain management alternatives. In
this paper, I use qualitative data from the text in Sharing the Challenge to develop causal maps
of the recommendations made in that document. I believe the characteristics of the
recommendations may be used to predict their likelihood of adoption or implementation. The
causal map codebook developed for this research is used to identify the connection between
issues and solutions in this policy domain. Hopefully, the way in which data are coded using this
codebook will improve consistency, reliability, and transparency in research that relies on
qualitative causal mapping. Strengths and weakne! of the codebook will be reviewed, as well
as discussions about the future directions of this research.
2 Literature Review and Research Questions
In this section of the paper, I will illustrate a “feedback perspective” of a section of the
natural hazards literature which deals with the policymaking process. Natural hazards scholars
who study the policy process often present arguments in ways that resemble the concepts found
in system dynamics. While no one statement or thought provides the basis for a complete causal
map, the best scholars in the field often present arguments with the features of a closed feedback
loop. As I reviewed the literature, I noticed some strengths and weaknesses in my approach,
which prompted me to think about more precise ways for developing causal maps from a
document or text. This feedback perspective of the academic literature illustrates three
objectives: (1) establish a theoretical foundation for the analysis of policy recommendations in
the study; (2) show how policy scholars (who may have no background in system dynamics)
make elegant arguments containing feedback loops and system dynamics concepts; and (3)
reveal a weakness in my prior approach to causal mapping. Objective three is the main focus of
this paper, as my shortcomings have provided me with an incentive to develop a coding tool for
systematically constructing causal maps from qualitative data. This coding tool is presented in
sections three and four of this paper.
There are many important factors in the policy process literature that help scholars and
practitioners better understand why certain policies are selected and others are not. Three of
those factors include: policy windows, focusing events, and policy entrepreneurs. Since natural
hazards are often viewed as potential focusing events (see Birkland 1997), it is important to
explore how and why policies change over time and how these policy changes affect the
damages incurred during future events.
2.1 Policy Windows
Kingdon (1995) and Polsby (1984) examined patterns of policy adoption and innovation in
government. Kingdon’s (1995) streams metaphor of the policy process describes conditions for
policy change when the politics, problem and policy streams converge at a window of
opportunity.
Figure 2.1 Window of Opportunity
public
attention span
Cope] Window of
Opportunit
opening BE z closing
policy policy
/- windows {—) > windows
magnitude of
focusing event willingness to pursue
hazard mitigation
+
damage Vulnerable
- Property
development
The author discusses how changing indicators and focusing events can be used to open the
policy window. This theory has been applied by several researchers in a variety of policy
contexts (see Rabe 1986; Birkland 1997). Mileti, Nathe et al. (2004) conclude that windows of
opportunity can be used as an advocacy tool to increase hazard communication and public
education. Very often the window of opportunity opens very briefly, which suggests that those
who attempt to use the opportunity must be well prepared. Prater and Lindell (2000) suggest
there should be well-developed policy alternatives and a clear strategy of action established even
before the window opens.
2.2. Focusing Events
A major focusing event can open the window of opportunity for policy change (Kingdon
1995). In the case of hazard mitigation policy, this window usually occurs in the immediate
aftermath of a disaster when the community is most receptive to policy changes. The window
closes quickly, as the public’s attention span is short and soon shifts to other issues. For example,
Pennebaker and Harber (1993) reported that discussions about hazard mitigation following San
Francisco’s Loma Prieta earthquake virtually disappeared within three months of the event.
Figure 2.2 Disaster Knowledge
public
attention
span
Policy Window
opening closing policy
policy window
window +
willingness to
mitigate
Property
Developed in
Hazard developing
in hazard
+
ra protection from
environment
damage in nearby
building 4 community
Iknowledge-
Disaster
Knowledge
However, the recent experience of a disaster is a potential focusing event that can keep
the issue on the agenda (Birkland 1998). An experienced disaster is a powerful way to start the
policy process moving. Turner (1967) identifies circumstances where recent events influence
decisions to take precautions and how organized support is most likely to take place. In a hazards
context, the occurrence of an event (e.g., flood hazard) might be considered a focusing event if it
can be used to increase advocacy for the adoption of policies to take precautions against future
events (e.g., hazard mitigation policies). Prater and Lindell (2000) suggest that even a disaster
that has occurred within a neighboring community, especially one that is perceived to be similar
in its hazard vulnerability, can provide a very powerful agenda setting effect.
2.3 Policy Entrepreneurs
Not all potential focusing events become actual focusing events, nor do they guarantee
policy change in all cases (Birkland 1997). The window of opportunity is facilitated by very
important actors in system known as policy entrepreneurs. Policy entrepreneurs have expert
knowledge in specific policy alternatives, as well as political expertise in the policy process
(Kingdon 1995). They are able to use focusing events and changing indicators to maintain
support on the agenda for their preferred policy alternatives.
Figure 2.3 Policy Entrepreneurs
opening policy closing policy
window window
Policy
Window ==“)
+
adopting hazard
mitigation measures
Vulnerable
Property
developing
in hazard
+
Policy Entrepreneurs
for Mitigation perceived risk by
community
+
activating policy
entrepreneurs
The adoption of hazard mitigation measures is associated with the presence of strong
advocates who have access to policy makers and a high degree of legitimacy due to technical
expertise, political power, or the prospects of longevity in office (Wyner 1984; Alesch and Petak
1986; May and Williams 1986; Berke and Wilhite 1989).
Policy entrepreneurs have been studied in a variety of settings. In the case of state coastal
erosion policy, scholars have found that policy entrepreneurs who understood the technical
research as well as the policy implications were able to change the direction of coastal
management policy (Deyle 1994). For environmental programs, Borins (1998) found that
environmental ac s were valuable resources for policy entrepreneurs, as they explored
market mechanisms for innovative policy changes to environmental programs. Meo Ziebro et al
(2004) applied the Borins findings to their case study on Tulsa hazard mitigation policies. They
found that successful outcomes were influenced by policy entrepreneurs who facilitated the
interaction of political and nonpolitical actors.
The policy entrepreneur takes many forms and plays several roles in hazard mitigation
policy design. They can be champions for safer communities, advocates who define the problem
and maintain it on the institutional agenda, and people who mobilize support for preferred
policies (Berke and Beatley 1992; Olson and Olson 1993). Roberts and King (1996) break policy
entrepreneurs into five categories: policy intellectuals, political entrepreneurs, bureaucratic
entrepreneurs, policy entrepreneurs, and executive entrepreneurs. Based on this description, the
role of the public administrator comes into focus, as experts at government agencies have a long-
term interest in an issue and understands the efficacy of mitigation and prevention over strictly
response and relief policies (Prater and Lindell 2000).
Policy entrepreneurs work on both sides of hazard mitigation policy design. Community
planners and emergency managers with the necessary hazard knowledge could try to link
mitigation policies to urban development and environmental issues that are of interest to
neighborhood associations, particularly those who have become active in the aftermath of a
disaster (Prater and Lindell 2000). In this way, technical specialists can exert upward influence
upon policymakers my mobilizing a political constituency for more farsighted hazard policies
(Lindell 1994). On the other side of the issue, there can be powerful interests who have policy
entrepreneurs who promote solutions that do not include hazard mitigation. These entrepreneurs
work to keep hazard mitigation off the agenda, defining the issue as a condition rather than a
problem (Bachrach and Baratz 1962).
2.4 Research Questions
¢ Which issues and solutions are discussed by policy entrepreneurs when potential focusing
events open a policy window?
e How are the connections between issues and solutions framed in floodplain management
policy recommendations?
¢ What is the causal model (stated or implied) in the recommendation?
3 Developing the Codebook
The codebook developed for this research builds on existing system dynamics research,
where qualitative data has been used to construct causal maps and system dynamics models (for
examples see Luna-Reyes and Andersen 2003; Kim 2007). For my research, the task is
somewhat straight-forward, as recommendations carry a logical structure which often identify
causal relationships. A policy analysis report (e.g., Sharing the Challenge) presents the relevant
issues and shows how solutions address each of the issues. In doing so, the analyst (explicitly or
implicitly, by discussing indicators of the problem) identifies the causal map used in the analysis.
In a public policy setting, the task is slightly more complex, but not impossible to achieve. Public
forums allow advocacy groups to weigh-in on the discussion in venues such as: congressional
testimony, court hearings, and administrative agency public comment periods. In the policy
process literature, advocacy groups (and policy analysts to some extent) engage in issue framing,
a strategy whereby a causal model (or issue frame) is identified by a proponent for a particular
piece of the problem or solution. Issue frames can be viewed as pieces of a causal map.
According to the person or group making the recommendation, they are, in fact, the most
important pieces of the causal map. This research develops a codebook for constructing causal
maps of policy recommendations. The method I suggest for this study is very similar to a content
analysis, since there is a source who sends a message using a channel to a recipient.
3.1. Components of a content analysis
A content analysis contains four elements: a source, the person or group who produces a
message; a message that contains distinct characteristics; a channel or venue in which the
message appears; and a receiver or target of the message. The codebook developed for this
research was used to analyze each recommendation as a message unit. Since there is only one
document analyzed for this paper, the messages for the case study have only one source, the
Interagency Floodplain Management Review Committee. The messages were made in a single
channel, the recommendation document, Sharing the Challenge. Technically, the receiver of
every message was the Administration Floodplain Management Task Force, whose key members
were the Associate Director of the Office of Management and Budget, the Director of the White
House Office of Environmental Policy, and the A: nt Secretary of Agriculture for Natural
Resources. However, upon further examination one sees that some recommendations were
charged to the Executive Branch, some recommendations were charged to Congress, while other
recommendations were charged to individual agencies.
o Source = Interagency Floodplain Management Review Committee
o Message / Channel = each of the 97 recommendations / the report, Sharing the
Challenge
o Receiver = Administration Floodplain Management Task Force
Those who study flood mitigation would agree that Sharing the Challenge was an
influential document, whose recommendations were taken very seriously by the Administration
Floodplain Management Task Force following the 1993 Midwest Floods. For this reason, I
identified the Interagency Floodplain Management Review Committee’s chairman, Gerry
Galloway, as a policy entrepreneur in this policy domain. As the Corps put together its response
to Senator Boxer, I developed a codebook to analyze the recommendations (i.e., messages) in
Sharing the Challenge. It is my hope that the characteristics of the messages in this document can
help us identify which recommendations are more likely to be adopted.
3.2 Data Source, Unit of Analysis and Unit of Observation
The data source for this study is Sharing the Challenge, a policy analysis report with 97
recommendations covering eight important floodplain management themes. The unit of analysis
for this study is the recommendation. A recommendation is the argument in favor of a course of
action, which identifies a connection between relevant issues and a proposed solution. The unit
of observation for each recommendation is the causal link or causal relationship. There may be
several causal links in a given recommendation. A causal link is an statement made by the source
about the nature of a cause and effect relationship. At a minimum, a causal link will contain the
cause variable, the effect variable, and the direction of the relationship between cause and effect.
In addition, there may be information about the natural of the relationship, such as the linearity
(or nonlinearity) between the cause and effect, as well as any delays between the cause and
effect, both of which may result in important accumulations in the system.
3.3. Coding Rules for Constructing Causal Maps of Recommendations
The coding rules defined in this section of the paper include a series of steps for the coder
to follow in sequence, as well as advice on how to address different types of language in the
document. Ideally, it would be best to address each step in the codebook as a question, whose
answer helps define the causal relationship. The coder should respond “not clear” in cases where
a relationship is not clearly defined. Otherwise, observations should be coded and answers
should be provided for each of the 10 rules in this codebook. There are two coding forms for this
codebook. First, a hand drawn causal map should be kept for each observation (in this case,
observation=recommendation). Second, a table, spreadsheet, or database should be maintained
that transforms the causal map into its respective components.
The codebook designed for this research provides instructions for developing causal maps
from documents and hearings with policy recommendations. The coding rules that follow
provide guidance in two ways: (1) instruction for coding causal relationships; and (2) instruction
for identifying issues and solutions discussed in the recommendation. Here are the coding rules
developed for this study.
Coding Rules
Section A: Causal Relationships
1. Read the recommendation carefully. Identify the recommended solution. Identify the
arguments which support the recommendation. This step defines the boundary of the
recommendation and provides evidence for the steps that follow.
- Directly quote the text whenever possible.
- Use brackets [ ] around any language that is not a direct quote from the
source. Use these brackets sparingly — in a similar fashion as you would read a
news article, use brackets to clarify a statement that makes reference to an
earlier statement in the argument, when quoting out of context.
2. Divide the argument into segments for each cause and effect relationship.
- Place a letter at the end of text which supports the causal relationship.
- Each argument segment provides evidence for a cause and effect
relationship.
3. Clearly identify the cause variable and effect variable. Variable names should be
nouns, not verbs. Variable names must be quantities could graphs over time.
- Acausal relationship will be an explicit or implicit “if/then’” statement.
- The coder should be able to identify the units for the variable names. Soft
variables, such as knowledge or willingness, are perfectly acceptable.
4. Identify the direction of the relationship (+ or - )
- The direction of the relationship is positive if the statement explicitly or
implicitly suggests an increase to X (cause) will increases Y (effect).
- The direction of the relationship is negative if the statement explicitly or
implicitly suggests an increase to X (cause) will decrease Y (effect).
5. Use the information in steps 1 through 4 to draw a causal map. Label the positive or
negative relationship (ie., the causal link) with the letter used to define the
relationship in step 2.
- Provide evidence for the relationship by using direct quotes from the text.
- Highlight the text and mark the supporting quote with the same letter used to
identify the causal relationship in step 2.
Steps 1 through 5 produce the causal map of the argument, as stated by the source. Save this
causal map.
Section B: Defining the Argument - Issues and Solutions in the Causal Map
Review the extant literature on the policy domain, before the study. This will help
understand how issues have been framed in the past and will support steps seven
through ten.
6. Identify the causal links that make up the major issues discussed in the argument used
to support the argument.
7. In the causal map, place a box around the major issues identified in the
recommendation.
8. Identify a causal link between the solution proposed in the recommendation.
9. In the causal map, place a circle around each solution identified in the
recommendation.
10. Under a new file name reproduce the causal map and close the feedback loops. The
coder should use their existing knowledge of the literature to justify feedback loops.
- Use dotted lines to link the cause and effect variables. Do NOT use capital
letters to identify “new” causal links.
- Use the literature to provide references for all links identified in step 10.
Table 1: Coding template for cause, effect, direction, and evidence of relationship
Causal link ID Cause Effect Direction Evidence
Observation 1A
Observation 1B
Observation 1C
4 Discussion with a Coding Example
To illustrate how the text is coded using the codebook developed for this research, I have
selected an example from Sharing the Challenge. Action 7.6 is a good example for a few
reasons: (1) the argument to support the recommendation connects flooding with other policy
domains; (2) there is excess information in the argument the coder must filter in order to identify
the causal relationships; (3) with respect to the number of causal relationships used, the argument
is not overly complex; and (4) the coder makes a couple of errors, which may help illustrate
some challenges for any causal map codebook.
The following example will show how the codebook steps can be used to generate a causal
map. The example selected appears in Chapter 7 of Sharing the Challenge, a chapter that focuses
heavily on environmental impacts of flooding and flood policies. This example explains how the
codebook steps are used and some challenges the coder may face during the coding process.
Step 1: Identify the recommended solution. In this document, the recommended solutions
are clearly marked as sub-headers in each chapter. Thus, the task was rather straight-forward.
However, in other documents the coder may have trouble defining the solution. The coder should
look for language to describe a course of action that is intended to resolve an issue. As in Action
7.6, a solution provides direction on a course of action that should be taken to resolve an issue.
The coder would record step one as follows:
Action 7.6: Require agencies to Co-fund ecosystem management using Operations and
Management funds
Step 2: Divide the argument into segments for each cause and effect relationship. There are three
cause and effect relationships clearly identified in Action 7.6. The challenge for the coder in step
two is determining how much information to include as supporting evidence for the causal
relationship or merely to just identify those sentences. For this research, I decided to include
supporting evidence, as this would help in another layer of analysis, coding causal links for
themes. To be clear, the first or last sentence of the argument segment is not necessarily the
causal relationship. Thus, there will probably be some variation among coders on how segments
of the argument are defined. Ultimately, the coder should define segments of the argument in a
way that provides evidence to support a causal relationship.
The coder would record step two as follows':
Construction of various federal navigation and flood control projects have impacted federal trust
resources in many rivers of the upper Mississippi River Basin.3 Operation and maintenance of
some of these projects continue to impact fish and wildlife resources and, in some cases, may
accelerate those losses. [A]
In the 1970s and 1980s, concerns related to these impacts on the upper Mississippi River
resulted in formation of cooperative interagency management efforts, such as the Great River
Study,4 Upper Mississippi River Master Plan,5 and Upper Mississippi River Environmental
Management Program.6 These programs, which address both development and natural resource
needs, have resolved many interagency conflicts and problems. Across the upper Mississippi
River Basin, though, federal agencies need to develop and implement ecosystem management
plans. Especially on the Missouri River, such plans would help ensure protection of fragile
ecosystems and address the needs of plant and animal species that are of inter-jurisdictional
federal interest. Presently a funding mechanism to develop and implement ecosystem
management plans does not exist. [B]
Action 7.6: Require agencies to co-fund ecosystem management using Operation and
Maintenance funds.
Ecosystem management planning would document natural resource needs and identify actions
that federal agencies can take to offset development impacts and enhance ecosystem
sustainability. Funding for development and implementation of ecosystem management plans
should be an annual standard component of each federal agency’s
operation/maintenance/construction budgets along with annual funding for development
projects, which often impact the ecosystem. Funds should provide for participation of outside
agencies and the states. Once costs of minimizing environmental impacts become a standard part
of project costs, they can be reflected more closely in federal benefit-cost ratios. [C]
Step 3: Clearly identify the cause variable and the effect variable.
Step 4: Identify the direction of the relationship
Steps three and four will most likely be completed in close sequential order, as it is
difficult for one to conceptual a causal relationship without thinking about the direction of that
relationship. With that said, there will be circumstances when the direction of that relationship is
unclear, weak, or nonlinear. Therefore, steps three and four are separate activities. In this
example, the coder identified three argument segments in step two. In steps three and four, the
coder’s task is to clearly identify the causal relationship in each segment of the argument. Table
2 summarizes the three causal relationships in Action 7.6.
! Appendix A1 provides the verbatim text of 7.6
As you can see in Table 2, the coder makes a few errors in step 3. First, a few of the
variables are not expressed clearly as quantities. For example, the cause identified in causal link
7.6A, the cause is operations and maintenance of federal navigation and flood control projects.
Phrased as such, the variable is at best a dummy variable and one could argue there is no
ible graph over time for 7.6A. The cause identified in 7.6C, using operations and
maintenance of federal navigation and flood control projects, is also poorly phrased. Can
“using” be graphed over time? Perhaps yes, but another phrases, such as “level of’, “number of”
or “quality of’ would be better suited in these circumstances.
Why did the coder make these mistakes? The mistakes were made because the coder was
trying to maintain the “golden” rule in step 1, to use the source’s words and directly quote the
text whenever possible. In order to accurately represent the variable as a quantity, the coder runs
the risk of making the variable less precise. The tension between precision and accuracy is an
important issue and possible shortcoming in most codebooks. Should the coder change the
wording of the text to transform a categorical variable into a quantity? I don’t have the answer to
this question, but I hope this paper generates some debate on the topic.
A summary of how the coder would record steps three and four (with “evidence” from step two)
is provided in table 2:
Table 2: Causal Relationships for Action 7.6
ree | | Cause | +e | Effect | | Evidence
A operations and maintenance of 4. accelerated losses to fish and {Construction of various federal navigation and flood
federal navigation and flood wildlife resources control projects have impacted federal trust resources in
control projects many rivers of the upper Mississippi River Basin.3
(Operation and maintenance of some of these projects
continue to impact fish and wildlife resources and, in some
cases, may accelerate those losses.
B upper miss river basin agencies 4, protection of fragile ecosystems fin the 1970s and
developing and implementing and address the needs of plant —_|1980s, concerns related to these impacts on the upper
ecosystem management plans and animal species that are of | Mississippi River resulted in formation of cooperative
intesjurisdictional federal interagency management efforts, such as the Great River
interest study.4 Upper Mississippi River Master Plan, and Upper
Mississippi River Environmental Management Program.6
‘These progranis, which address both development and
natural resource needs, have resolved many interagency
‘conflicts and problems
‘Across the upper Mississippi River Basin, though, federal
agencies need to develop and implement ecosystem
|management plans. Especially on the Missouri River, such
plans would hielp ensure protection of fragile ecosystems
‘and address the needs of plant and animal species that are
of interjurisdictional federal interest. Presently a funding
‘mechanism to develop and implement ecosystem
‘management plans does not exist.
free | [| Cause x | Effect 1 | Evidence
c ‘using operation and. “f eavironmental impacts TAction 7.6: Require agencies to co-fund
management finds to co-fund becoming a standard part of [ecosystem management using Operation and
ecosystem management project costs so they can be faintenance fands
reflected more closely in
federal benefit-cost ratios. [Ecosystem management planning would document natural
esource needs and identify actions that federal agencies
lcan take to offiet development impacts and enhance
lecosystem sustainability. Funding for development and
limplementation of ecosystem management plans should be
fan annual standard component of each federal agency's
loperation maintenance/construction budgets along with
lannual funding for development projects, which often
limpact the ecosystem. Funds should provide for participation
lof outside agencies and the states. Once costs of
jminimizing environmental impacts become a standard part
lof project costs, they can be reflected more closely in
lfederal benefit-cost ratios
Step 5: Use the information in steps 1 through 4 to draw a causal map.
Step 6: Identify the causal links that make up the major issues discussed in the argument used to
support the argument.
Step 7: In the causal map, place a box around the major issues identified in the recommendation.
Step 8: Identify a causal link between the solution proposed in the recommendation.
Step 9: In the causal map, place a circle around each solution identified in the recommendation.
After the arguments have been summarized in some format, such as the one shown in
table 2, the coder maps the arguments either by hand or preferably with system dynamics
software, such as Vensim. For Action 7.6, there are three causal links identified in the text and
one recommended solution. The process is relatively straight-forward from here, as most of the
work has been done in steps one through four. By constructing a causal map, we see the logic
used by the source to “frame” the issue. I strongly believe the visual representation of this data
provides a more elegant way of defining issue frames in public policy recommendations.
Figure 4.1 shows how the coder would use the information in steps one through four to
record steps five through nine’.
Figure 4.1: The Causal Map for Action 7.6
2 Step 10 is discussed briefly in the final section of this paper. Rules for closing the feedback loops will be the
subject of the next paper.
p> acceretated losses (0 Bsn
operations and enauice of = + — and wildlife resources
Require agencies to federal navigation and flood A
{ — co-find ecosystem \ control projects
\
| Managementusmg |
Operations and }
Mamtenance finds upper miss river basin agencies ———————™ protection of fragile ecosystems and
developing and implementing B address the needs of plant and anmal
ecosystem management plans species that area of interjurssdictional
federal mterest
sing operation and = ; = = — environmental mpacts becoming a
manggement finds to cofind Cc standard part of project costs so they can
eqpsystem management be reflected more closely m federal benefit
cost ratios
5 Conclusions
In my early research, I developed a conceptual model based on major themes in the public
policy literature, such as Policy Windows, Focusing Events, and Policy Entrepreneurs.
Arguments use elements of these and other important themes. However, very little empirical
work has been done to show how arguments for recommendations can be analyzed to connect
themes and arguments using elements of systems thinking, causal mapping and content analysis.
This paper takes a small step in that direction by presenting a codebook, whose final product can
be empirically verified and whose process is transparent.
As mentioned earlier, Sharing the Challenge contains eight major themes and 97
recommendations. Examples of coded data for each of the eight major themes in the document
have been provided in the appendix. The reader can examine those examples to get a sense of the
issues and challenges the author experienced as he developed the codebook for this study. A
codebook needs to be precise, flexible, and accurate. I believe that a balance has been achieved
for the purpose of the study identified in this paper.
6 Next Steps and Future Research
This paper argues in favor of using system dynamics concepts to construct causal maps from
reports and hearings that contain policy recommendations. The codebook presented in this paper
contributes to the system dynamics literature which uses qualitative data to build causal maps.
The coding rules presented here provide a way to analyze and deconstruct the arguments made
by experts and advocates. In addition, the coding rules provide a way to compare the reliability
between coders to minimize the amount one must “read between the line” in order to produce
causal maps. Hopefully, this paper will continue the discussion that leads modelers towards
measures of inter-coder reliability.
With that said, there is much work to be done in the future. The next versions of the
codebook will need to address other types of policy problems and types of data. The codebook
presented in this paper was designed to address a specific set of research questions. For this
research, it was not essential to close all feedback loops, as the task was focused on how issues
and solutions were framed. In the current version of the coding scheme, causal links with weak
or no supporting evidence were allowed on the diagram as a dotted-line. This approach needs to
be revisited. In future research the author needs to be more explicit about ways for closing loops
when evidence is not clearly stated in the text.
Additional Coding: From causal map to system dynamics model & Coding Major Themes
1. Identify variables that appear frequently in the arguments. Then, determine which of
these variables should be considered stocks in the model
2. Identify the strength of the relationship.
- Record “not clear” if strength is not explicitly stated.
Tf one exists, identify nonlinearities in the causal relationship.
Tf one exists, identify delays in the causal relationship.
Coding for major themes: After all of the recommendations have been coded in one
document, review the themes identified in the literature review. Code the issues and
solutions for the relevant themes in the literature.
Sige
Table: Coding for strength of relationship, nonlinear relationships and delays
Causal link ID Strength Nonlinearity Delays
Observation 1, A
Observation 1, B
Observation 1, C
Table: Example with Recommendation 4.1
Causal link ID Strength Nonlinearity Delays
4.1A Moderate NA NA
4.1B stronger when | NA over time federal
federal monies are monies have been
limited limited
4.1C strength is relative | NA NA
to size of structural
project
Table: Themes in the Literature - Recommendation 4.1
Recommendation | Causal Links Issue or Solution | Literature Theme
4.1 A,B,C Issue urban centers at risk
4.1 D Solution defining vulnerability: standard
project flood instead of the 1% flood
Next Steps: Example of Complexity
In Step 10, the coder creates a duplicate version of the causal map and uses their
knowledge of the extant literature to close the feedback loops. The diagram below shows two
ways to represent the same argument made to support recommendation 4.1, which says “reduce
vulnerability of population centers to damages from the standard project flood discharge and
move from a | percent flood design standard to an SPF design standard.” The diagram on the left
is an exogenous view of the problem, where cause and effect variables are neatly separated from
each other. The diagram on the right is an endogenous view (step 10 in the codebook) of the
same argument.
Complexity in the Argument: Causal Map 4.1
ttt cdgiaciniin 4
muni lpopeyne ptboideap cbs (ot a + ielpeaetagiat SS
coment oped + hati a pny tedear
rede henbityot enone kein . <a ;
[ popltincetes io |
{damages fmt std | soa eS
| pogctboddecge | ‘med Aint > sngbenecot am a te
\ stench pet } anal fr od dame [3] snetnd to seecthgh elof nba te es
\ food desnsanird sanjay: eta “Silat & ae .
\ dei } gfe \ Ne
t<— = ebony 7
: keeloftcomar niko - > food pmiein rc fom a on
comms ear iso emer ett asain sete
cpp ls atk cosas ee |
alma pret,
\ ees 00 a vienityot \ ssid
yecbde “Bj comm
reatal mk
> foodphincomgrs develope srateges br —_
esting staegs fr enlug bei \ \ #
redo couerg ofthe osc ent \ aide
smunmlaprovces ease verb Pit
pope
png at seksi ae
sebeamr eatin bebe rygtied
~ > fing fr smear appones ‘
stapes ve
sah ofininde_— ty
vied ore Suramar ios xii
fooipin ‘prec fed tin + etude
> inka fesrerediggcuces
References
Alesch, D. and W. Petak (1986). The Politics and Economics of Earthquake Hazard Mitigation.
Boulder, Institute of Behavioral Science, University of Colorado.
Bachrach, P. and M. Baratz (1962). "Decisions and non-decisions: An analytical framework."
American Political Science Review 57: 632-642.
Berke, P. R. and T. Beatley (1992). Planning for Earthquakes: Risk, Politics and Policy.
Baltimore, Johns Hopkins University Press.
Berke, P. R. and S. Wilhite (1989). "Influences on Local Adoption of Planning Measures and
Earthqyake Hazard Mitigation." International Journal of Mass Emergencies and Disasters 7(1):
15-33.
Birkland, T. A. (1997). After Disaster: Agenda Setting, Public Policy and Focusing Events.
Washington, D.C., Georgetown University Press.
Birkland, T. A. (1998). "Focusing Events, Mobilization, and Agenda Setting." Journal of Public
Policy 18(1): 53-74.
Borins, S. (1998). Innovating with integrity: how local heroes are transforming American
governmnet. Washington, D. C, Georgetown University Press.
Deyle, R. (1994). "Conflict, uncertainty, and the role of planning and analysis in public policy
innovation." Policy Studies Journal 22: 457-473.
Kim, H. (2007). Gaining Confidence in Maps Elicited from Qualitative Data. Albany-MIT PhD
Colloquium, University at Albany, SUNY.
Kingdon, J. W. (1995). Agendas, Alternatives and Public Policies. New York, Harper Collins.
Lindell, M. K. (1994). "Are local emergency planning committees effective in developing
community disaster preparednees?" International Journal of Mass Emergencies and Disasters 12:
159-182.
Luna-Reyes, L. and D. Andersen (2003). "Collecting and analyzing qualitative data for system
dynamics: methods and models." System Dynamics Review 19(4): 271-296.
May, P. and W. Williams (1986). Disaster Policy Implementation: Managing Programs Under
Shared Governance. New York, Plentum Press.
Meo, M., B. Ziebro, et al. (2004). "Tulsa Turnaround: From Disaster to Sustainability." Natural
Hazards Review 5(1): 1-9.
Mileti, D., S. Nathe, et al. (2004). Public Hazards Communication and Education: The State of
the Art. Boulder, CO, Natural Hazards Research and Applications Information Center.
Olson, R. S. and R. A. Olson (1993). "The rubble's still standing up in Oroville, California: The
politics of building safety." International Journal of Mass Emergencies and Disasters 11: 163-
188.
Pennebaker, J. W. and K. D. Harber (1993). "A social stage model of collective coping: The
Loma Prieta Earthquake and the Persian Gulf War." Journal of Social Issues 49: 125-145.
Polsby, N. A. (1984). Political innovation in American: The politics of policy innovation.
Berkeley, CA, University of California Press.
Prater, C. and M. K. Lindell (2000). "Politics of Hazard Mitigation." Natural Hazards Review
1(2): 73-82.
Rabe, B. G. (1986). Fragmentation and integration in state environmental management.
Washington, D.C, Conservation Foundation.
Roberts, N. C. and P. J. King (1996). Transforming public policy: Dynamics of policy
entrepreneurship and innovation. San Francisco, Jossey-Bass.
Turner, R. H. (1967). "Types of solidarity in the reconstruction of groups." Pac. Sociol. Rev 10:
60-68.
Wyner, A., J. (1984). "Earthquakes and Public Policy Implementation in California."
International Journal of Mass Emergencies and Disasters 2(August): 267-284.
Appendix A1: Direct Quotes from the Text to Support the Causal Map in Action 7.6
Require agencies to co-fund ecosystem management using
Operation and Maintenance funds_
Before Rec:
USING O&M FUNDS TO MANAGE ECOSYSTEMS
\Construction of various federal navigation and flood
control projects have impacted federal trust resources in
many rivers of the upper Mississippi River Basin.3
Operation and maintenance of some of these projects
continue to impact fish and wildlife resources and. in some
eases. may accelerate those losses. [In the 1970s and
1980s. concerns related to these impacts on the upper
Mississippi River resulted in formation of cooperative
interagency management efforts, such as the Great River
Study.4 Upper Mississippi River Master Plan.5 and Upper
Mississippi River Environmental Management Program.6
These programs, which address both development and
natural resource needs, have resolved many interagency
conflicts and problems.
Across the upper Mississippi River Basin. though. federal
agencies need to develop and implement ecosystem
management plans. Especially on the Missouri River, such
plans would help ensure protection of fragile ecosystems
and address the needs of plant and animal species that are
of interjurisdictional federal interest. Presently a funding
mechanism to develop and implement ecosystem
management plans does net exist.
As a matter of practice. agencies responsible for operating
and maintaining major development projects should
procure funding for representation and participation of
other federal agencies in their major study and implementation,
efforts. The USACE-FWS Memorandum of
Agreement for fund transfers related to Fish and Wildlife
‘Coordination Act compliance makes such participation
possible during the planning process, but no authority
exists to transfer funds for support of post-construction
ecosystem planning. Similarly no funding mechanisms
exist for state or local participation in either the planning
or post-construction phases of federal water resources
development.
\Action 7.6: Require agencies to co-fund
ecosystem management using Operation and
Maintenance funds.
Ecosystem management planning would document natural
resource needs and identify actions that federal agencies
can take to offset development impacts and enhance
ecosystem sustainability. Fundins for development and
implementation of ecosystem management plans should be
an annual standard component of each federal agency”s
operation/maintenance/construction budgets along with
annual funding for development projects. which often
impact the ecosystem. Funds should provide for participation
of outside agencies and the states. Once costs of
minimizing environmental impacts become a standard part
of project costs. they can be reflected more closely in
federal benefit-cost ratios |
(Comment (T1318
- {Comment [Fa3k ©
