Table of Contents
Emotional Decision Making in System Dynamics
J. Parvizian! and H. Tarkesh?
Abstract
This paper is about decision making agents in system dynamic models. Decision makers control the rate
variables. The decision is made based on the information of the up- and down-stream levels of each rate
received at the decision points. Inspired by the similar concepts created for servo-mechanisms, in socio-
economic dynamic systems it is common practice to assume that decisions are made according to a definite
law or a guidance table or graph. This deterministic approach is hardly able to model systems in which
decisions are taken by humans. Since humans may decide differently in the same conditions not because
they are rational but because they, sometimes, decide emotionally. Rationality is assumed to be
independent of persons; therefore understandable for all, i.e., the decision maker is always trying to
maximize her/his explicit profits by taking decisions that are known to the modeler. On contrary,
emotionality is very personal and often leads to un-justifiable decisions. To capture the nature of decisions
made by people we have to consider the characteristics and personality of the person who is in charge. This
way, the rational decision maker may be replaced by a rational-emotional one. Following efforts to build
emotional robots, in this paper an emotional decision maker, which is called sometimes an agent, is
integrated into a socio-economic system dynamic model. This agent receives information from the
environment and decides in-line with its personality. The environment is being changed by the decisions
made. So the agent faces a new condition to decide in. The environment also encourages or punishes the
agent by the result of the decisions taken. Therefore, the personality of the agent is a set of dynamic levels
under the influences of the environment. However, these levels may accept rapid changes that cannot be
given by the integral equations common in socio-economic models. To consider this, emotions are modeled
as fuzzy mapping functions. Over longer periods the fuzzy values of the agent’s emotions change according
to the experiences gained by decisions.
1. Introduction
It was probably in 1981 that, for the first time, Sloman and Croucher interpreted the
achievements of naive psychology to conclude that “the need to cope with a changing and
partly unpredictable world makes it very likely that any intelligent system with multiple
motives and limited powers will have emotions”. Therefore, the belief that emotions and
intellect are somehow quite separate is mistaken. They identified some of the constraints
on intelligent systems as “recognition often requires the use of structural descriptions”,
“the collection of motives is not static”, “the environment is not static”, “the complexity
of the environment often leads to mistaken beliefs, plans, and actions”, and “different
motives in the same individual may be inconsistent”. To develop a general grammar of
emotional states they suggest that “an emotional state normally involves having at least
one fairly strong motive”, “the combination of motive and belief (or uncertainty) must be
capable of producing a disturbance, i.e. continually interrupting thinking and deciding,
and influencing one’s decision-making criteria and perceptions”, “the disturbance may or
may not involve specific new motives”, “new motives need not be selected for action”,
* Assistant Professor, Isfahan University of Technology, Isfahan, 84156 83111, Iran.
Phone: +98 (311) 391 5514, Fax: +98 (311) 391 5526, Email: japa@cc.iut.ac.ir, (Corresponding Author).
* Master Student, hamedtarkesh@ yahoo.com
“some emotional states arise out of actions performed by the individual”, “sensory
detectors may record local changes produced by the interruptions, and the system’s
perception of its own state will be changed. ... and... the ability to discriminate and
recognize complex internal states may have to be learnt, and may involve complex
perceptual processes”. To introduce a concept of emotions that can be used in a dynamic
system, Sloman [2004] proposes that “what are normally called emotions are a somewhat
fuzzy subset of a larger class of states and processes that can arise out of interactions
between different mechanisms in an architecture”.
Decision makers change their behavior according to the information they receive.
Those active objects that take in and manipulate information and vary their behavior as a
result of processing information are called agents. Autonomous agents generate their own
motivations. Each agent type has an associated collection of rule-sets. Each rule-set is a
collection of condition-action rules that interact via ‘databases’ or working memories
internal to the agent. Thus one nule-set might be concemed with interpretation of low
level sensory information, another with generating motivators in response to the
formation of new beliefs, another concerned with assessing the importance of motivators,
another concerned with planning, and so on. Within an agent, learning or developmental
processes may change individual rules, or introduce new rule-sets, or introduce new
interactions between rule-sets, e.g. by adding new communication channels [Davis et al,
1995].
Most autonomous agents are situated in a social context and need to interact with other
agents (both human and artificial) to complete their problem solving objectives. There are
many potential decision making functions which could be employed to make the choice.
Each such function will have a different effect on the success of the individual agent and
of the overall system in which it is situated. Therefore, Kalenka and Jennings [1997]
examine agents’ decision making functions to ascertain their likely properties and
attributes.
Marcia Macas et al. [2001] also stress on the point that efficient decision-making
depends heavily on the emotions underlying mechanism and that alternative courses of
action in a decision-making problem are emotionally (somatic) marked as good or bad.
These emotional marks not only guide the decision process, but also prune the options
leaving only the positive ones to be considered for further scrutiny.
Noting that, to date, the field of Artificial Intelligence has largely ignored the use of
emotions and intuition to guide reasoning and decision making, Velasquez’s [1998]
contribution is to show “how drives, emotions, and behaviors can be integrated into a
robust agent architecture, that uses some of the mechanisms of emotions to acquire
memories from past emotional experiences that serve as biasing mechanisms while
making decisions during the action-selection process”. The flexible agent architecture
presented integrates drives, emotions, and behaviors and focuses on emotions as the main
motivational system that influences how behaviors are selected and controlled. It is
shown how the mechanisms of primary emotions can be used as building blocks for the
acquisition of emotional memories that serve as biasing mechanisms during the process
of making decisions and selecting actions. This work is derived from a previous research
of Velasquez [1997] on computational models of emotions.
Seif El-Nasr and Skubic [1998] rely on Demasio’s suggestion that emotions lead an
active role in guiding the decision-making process by providing a selection mechanism
for eliminating bad altematives. Decision-making is then simplified, because there are
fewer choices left to be evaluated. They investigate the use of emotional agents in the
decision-making process of a mobile robot. They have expanded the traditional
Intelligent Agent (IA) framework to incorporate the emotional or the internal state
features. In the traditional IA model, the world belief and goals are the determining
factors of actions that the agent takes. In contrast, in this model the goals shape the
expectation levels of the events. The expectation levels, along with environmental inputs,
determine the mixture of emotions and their intensities. They propose “a fuzzy logic
model that captures the inherent uncertainty of emotions. The model is used to generate
decisions based on both internal and extemal states and incorporates the use of sensory
information to extract environmental conditions”. In this way, the agent will react to a
changing environment and can take an action according to a mixture of emotions
generated by multiple states. Later, Seif El Nasr et al. [2000], develop a model that is
using fuzzy-logic representation to map events and observations to emotional states. The
model also includes several inductive leaming algorithms for learning patterns of events,
associations among objects, and expectations.
Gmytrasiewicz and Lisetti [2002] use the principled paradigm of rational agent design
to formally define the emotional states and personality of an artificial intelligent agent.
The emotional states are viewed as “the agent’s decision making modes, predisposing the
agent to make its choices in a specific, yet rational way”. Personality is defined as
consisting of the agent’s emotional states together with the specifications of transitions
taking place among the states.
In the present work a system dynamic model is examined in which rate variables can
be controlled by emotional decision makers. The personality of each decision maker is
modeled by a set of fuzzy mapping functions. Each decision, in long-term may result ina
profit or loss for the agent. Therefore, the punishment (encourage) due to loss (gain) will
weaken (strengthen) the emotion that caused that decision.
2. Decision A gents
The purpose of a decision is to maximize, in long term, the rewards gained for the
decision maker. Classic economics defines different rational rules to model the society
trend and even to get the best decision in each situation fora DM. Thus, the decision
made in a system is independent of the agent, and of the environment which is not overtly
integrated into the system. Therefore, it is not wrong to describe “rational decision
making” as “deterministic”. This approach may fail, however, to explain the diversity of
decisions made by different agents or in different environmental conditions, unexpected
decisions, the speed a decision is made by, and the importance of suggestions made by a
specific agent, i.e., a new manager or salesman.
To build a more general framework for decision making, the personality of a decision
maker can be integrated into the model as the agent’s emotions. A rational decision is
considered as an optimum solution to maximize the reward subject to a set of constraints.
Thus, in most cases, it can be left to a computer to find the solution, if the objective
function and constraints can be defined properly using mathematical terms. It tumed out,
however, that to find a solution for a multivariable function under a large number of
constraints by known methods may take much longer, even by the fastest computers, than
it takes by a human. In one hand, it can be referred to a very high processing speed or
parallel computing abilities of the brain. On the other hand one can think of an alternative
approach that humans use to find the solution. This alternative is not contradicting any
assumption for trial and error learning that may also explain the diversity of decisions due
to different history each agent has had. The ability to avoid wrong decisions and to find
the best solution can be personal and improving. The set of characters that form this
ability is thought of, in recent literature, as emotions. Emotional decision making is
believed to be in harmony with what has been called rational thinking so far.
The action followed by a decision changes the environment. The environment is
modeled partly in the system by variables that their levels affect the decision making
conditions. However the human agents may he influenced by an environment usually
larger than the one considered in the model. For example, in the very abstract economic
models, the price is determined at the equilibrium point by the levels of the inventory and
the demand, while the demand is determined by the price. In reality, the price may
change due to the variables that are not considered in the model such as fear of a war, or
the rapid changes of the market consumers expect to happen following breaking news.
What an agent gains by a decision made may cause changes in the characteristic set of
the agent, i-e., its emotions. If the agent is gaining (losing) by taking a risk, this gain
(lose) strengthens (weakens) the agent’s level of accepting risks. This can be interpreted
as the reinforcement learning of the agent. It is important to note that the adjustment time
of a personal characteristic may be different than the time the system state or
environmental conditions change. Therefore, the decision maker receives information of
the state of the system and a broader environment according to which the decisions are
made and the emotional levels are adjusted in different time scales. Figure 1 shows a
schematic of a system dynamic model in which the decision maker personality is
influenced by the environment and the reward.
Reward is a personal measure of the consequences of a decision made by the agent. A
multivariable averaging function of all levels at each time step by different weights
assigned by the agent determines reward. The interested reader is invited to compare the
differences of the reward function as defined in this paper with similar definitions [Sutton
and Barto, 1998].
— = Dp
I >|
[Agent's Emotions
| Agent's Action
Agents Policy TS
>
4
Revers,
System’s Condition
Figure1 Schematic of a system dynamic model in which the decision maker personality is influenced
by the environment and the reward.
3. Fuzzy sets
If the value of a level is given by rather a qualitative expression, any deterministic
one-to-one deduction may not be justified for a decision. To capture the uncertainty
inherent in the information given to the decision maker, also the deduction rules the
decision maker employs, fuzzy sets are used to define the emotional personality of the
decision maker. The core idea is to quantify this uncertainty which is due not to chance
but to the absence of sharply defined criteria of class membership. Fuzzy sets are classes
with uncertain borders that pervade human language and thinking [Sangalli, 1998]. They
can be used to quantify the qualitative expressions usually given for emotions, e.g., the
fear of the agent or the level he/she accepts risks can be ‘high’ ‘moderate’, or ‘low’.
Fuzzy sets are also useful to construct the system's input and output space vectors in each
step of time.
An important feature of using fuzzy sets to quantify emotions is that they are able to
be adapted to new values according to the learning mechanisms inserted in the model.
When the decision making rules in different environmental conditions/system states are
defined by an expert team or using tools such as Neural Networks, Genetic Algorithms,
or Genetic Programming [Castillo 2001, Ishibuchi 2002], a fuzzy inference is designed so
that it can model the past of the system or satisfies the expectations of the expert team for
different input conditions. ANFIS (Adaptive Neuro-Fuzzy Inference System) is one
example of proper tools to do this job [Jang,1991].
When the fuzzy system is designed, the initial state of the agent is determined by levels
defined in the system and levels that determine fuzzily the agent’s emotional state. Using
the fuzzy inference the agent can make a decision. The decision results in changing the
state of the system and the environment. The reward is calculated and reported to the
agent. It can be an indication of closeness of the reality to the expectations of the agent.
Therefore, the membership functions are adapted accordingly. The new emotional state
of the agent depends on levels of the system and the agent in earlier time steps, and on
the environment. The system state may be affected by decisions made by other agents.
The environment also may change unexpectedly. The agent is facing the new conditions
and decides. In reality, different agents in a system with different rules play a game to
maximize their own profit. Their interaction defines the state of the system in time.
4, Example
A simple demand-supply system, figure 2, is considered [Whelan and Msefer, 1996].
A step increase in demand causes a decrease followed by an overshoot then by a
diminishing oscillation in the inventory to reach a new equilibrium state. Although the
levels included in the model remain stable at the equilibrium conditions, all the human
decision makers in the system are influenced by the changes independently happen in the
environment. This deterministic model, in which decision to buy or sell is determined
independently according to Table1, cannot explain the mechanism the system interact
dynamically with the decisions made by the agents to react to the environmental changes.
Emotional decision model however is designed specifically to reflect to the environment
and represent the personal differences of decision makers.
Figure 3 shows an emotional agent-based demand and supply system. Each of the two
agents, that regulate supply and demand, aims to maximize its own profit. Their decisions
are rooted in their experiences and the information they receive of the environment. The
fuzzy system that is pre-defined is used to make a decision. Although, the membership
functions are history-dependent and are changed by experience, i.e., the agents learn how
to play to gain higher values in long run. Figure 4 shows how the system react when a
rumor, say collapse of a competitor or fear of war or a sudden heat wave, is spread
through the market during time distance of (20,60). This shows a sharp increase in
demand followed by a permanent positive step despite the price is increased. The reason
is obviously the customer fear. There is no punishment or encourage for the agents in the
long term since the rumor did not come true. Authors, in another paper [2004] show how
the system behave if the rumors come true, ie., the supply for any reason is really
decreased, and how the emotional membership functions change in response to the gaine
or lose.
Table 2 defines different rules that are used in this example to define the fuzzy
emotional conditions of the agents. Figure5 shows different choices of the supplier with
moderate risk at system states: demand and price. This cannot be explained by classic
economics in which the demand and supply are functions of only the price but not the
agent.
5. Conclusion
In this paper a new model for dynamic systems is presented in which rates can be
controlled by emotional agents who are sensitive to the environment. The deterministic
decision making which is not a good model of reality is replaced by fuzzy, learning, and
interactive decision making. Rules can be defined initially according to the history of
each agent and are left to the computer to adapt/correct them using the reward gained by
agents.
This model can be developed to multi-agent systems to predict market trend and so on.
Price Quantity Quantity
Demanded (per Supplied
week) (per week)
$50 10 100
$45 14 97
$40 18 94
$35 22 89
$30 28 84
$25 35 77
$20 45 68
$15 57 57
$10 73 40
$5 100 0
Table 1 Demand and Supply Schedules
Supply
Inventory
ef Ka
Supt
Shipment
Supply Piide Schedule nyentory Rialo
Ds tired ventory
Price
Emecton Price Desired
Price Change Delay
Figure2 Traditional System Dynamic Model of Supply and Demand
Figure3 Agent-based Model of Supply and Demand
a
7 stot.
! { !
ory i 1
1
Price Curve
[shocks
quam Point |
0 Se a a a
Time
Figure4 Demand increases sharply following a rumor (information) at time 20. When the rumor is denied
at time 60 the demand goes back to the normal. The same happens for the price. The customer fear, that
causes the changes, is also back to normal.
Supplier
Customer
T. If (Price is Low) then (Supply is Low) (0.8)
2. If (Price is High) and (Inventory is not High) then (Supply is
High) (0.9)
3. If (Price is Normal) then (Supply is Medium) (0.8)
4, If (Supplier Risk is Low) and (Inventory is Low) then (Supply
is High) (1)
5. If (Supplier Risk is High) and (Inventory is High) then (Supply
is Low) (1)
6. If (Inventory is High) then (New Customer Fear Low)(New
Supplier Risk is High) (0.85)
7. If (Inventory is Low) then (New Customer Fear High)(New
Supplier Risk is Low) (0.8)
8. If (Supplier Risk is Low) then (New Supplier Risk is Low) (1)
9, If (Supplier Risk is Medium) then (New Supplier Riskis
Medium) (1)
10. If (Supplier Risk is High) then (New Supplier Risk is High)
(a)
11. If (Supplier Risk is Low) then (Supply is High)(New Supplier
Riskis Low) (0.7)
12. If (Supplier Risk is Medium) then (Supply is Medium) (New
Supplier Risk is Medium) (0.7)
13. If (Supplier Risk is High) then (Supply is Low)(New Supplier
Riskis High) (0.7)
1. If (Price is Low) then (Demand is High) (0.8)
2. If (Price is Normal) then (Demand is Medium) (0.8)
3. If (Price is High) then (Demand is Low) (0.8)
4. If (Customer Fear is Low)
then (Demand is Low)(New Customer Fear is Low) (0.65)
5. If (Customer Fear is High)
then (Demand is High)(New Customer Fear is High) (0.65)
6. If (Customer Fear is Medium)
then (Demand is Medium)(New Customer Fear is Medium)
(0.65)
7. If (Inventory is Medium) then (Supply is Medium) (0.45)
8, If (Customer Fear is Low) then (New Customer Fear Low) (1)
9. If (Customer Fear is Medium) then (New Customer Fear
Medium) (1)
10. If (Customer Fear is High) then (New Customer Fear High)
())
Table 2 Rules are defined by an expert team.
sue
baseeege
ss 2
an
Figure5 shows different choices of the supplier with moderate risk at system states: demand and price.
References
[1] Castillo, L., Gonzalez, A. and Perez, P. “Including a simplicity criterion in the
selection of the best rule in a genetic fuzzy learning algorithm,” Fuzzy Sets and Systems
120, 309-321 , 2001.
[2] Davis, D., Sloman, A., and Poli, R., “Simulating agents and their environments”, In
AISB Quarterly, Autumn 1995.
[3] Gmytrasiewicz, P. and Lisetti, C. L., “Emotions and Personality in Agent Design and
Modeling”, In Game Theoretic and Decision Theoretic Agents, Parsons, S., Wooldridge,
M. and Gmytrasiewicz, P. (Eds.), pp. 81-95, Norwell, MA: Kluwer, 2002.
[4] Ishibuchi, H. and Y amamoto, T., " Fuzzy Rule Selection by Data Mining Criteria and
Genetic Algorithms", 2002.
[5] Jang, J.-S. R., “Fuzzy Modeling Using Generalized Neural Networks and Kalman
Filter Algorithm, ” Proc. of the Ninth National Conf. on Artificial Intelligence (AAAI-91),
pp. 762-767, July 1991.
[6] Kalenka, S. and Jennings, N. R. “Socially Responsible Decision Making by
Autonomous A gents”, In Proc ICCS'97, 1997.
[7] McCauley, Thomas Lee, “Implementing Emotions in Autonomous Agents”, M.Sc.
Thesis, the University of Memphis, 1999.
[8] Macas, M., Ventura, R., Cust’odio L., and Pinto-Ferreira, C., “Experiments with an
Emotion-based Agent using the DARE Architecture”, In Proceedings of the Symposium
on Emotion, Cognition and Affective Computing (AISB 01 Convention), UK, March 2001.
[9] Parvizian, J. and Tarkesh, H. “A gent Based Dynamic of Emotional Decision
Making” submitted to CSIMTA, Site Universitaire de Cherbourg, France, 2004.
[10] Sangalli, A. “The Importance of Being Fuzzy”, Princeton University Press, 1998.
[11] Seif El-Nasr, M., Yen, J., and loerger, T., “FLAME - A Fuzzy Logic Adaptive
Model of Emotions”, A utomous A gents and Multi-agent Systems, 3, 219-257, 2000.
[12] Seif El-Nasr, M. and Skubic, M., “A Fuzzy Emotional A gent for Decision-Making
in a Mobil Robot”, Proc. of Fuzz-IEEE 98, 1998.
[13] Sloman, A., “What Are Emotion Theories A bout?”, Invited talk at cross-disciplinary
workshop on Architectures for Modeling Emotion at the AAAI Spring Symposium at
Stanford University in March 2004.
[14] Sloman, A., and Croucher, M. “Why Robots Will Have Emotions”, In Proceedings
TJCAI 1981, Vancouver.
[15] Sutton, R.S. and Barto, A. G., “Reinforcement Learning: An Introduction”, MIT
Press, Cambridge, MA, 1998.
[16] Velasquez, J. 1997. Modeling Emotions and Other Motivations in Synthetic A gents.
In Proceedings of the Fourteenth National Conference on Artificial Intelligence. Menlo
Park, CA: AAAI Press.
[17] Velasquez, J. 1998. When Robots Weep: Emotional Memories and Decision-Making.
In Proceedings of the Fifteenth National Conference on Artificial Intelligence, 70-75.
Menlo Park, CA: AAAI Press.
[18] Ventura, R. and Pinto-Ferreira, C., "Emotion-based agents: Three approaches to
implementation", In Proceedings of the Third International Conference on Autonomous
Agents editor by J. D. Vealsquez Seattle, U.S. A., 1999.
[19] Whelan, J. and Msefer, K., “Economic Supply & Demand”, prepared for MIT
System Dynamics in Education Project, 1996, Copyright ©1994 by MIT.
Back to the Top
