Danny the Grim

Why do I drink?

Because I want to.

Why do I want to?

Because it’s there.

Sometimes some people try to understand things by creating models of things and seeing how well the models reproduce the behavior of the things. The models can be theories, mental models, concept maps, or computer simulations. The things can be people, or people in groups, or just about anything, really. But I’m thinking of people.

One way to model people drinking is using a social contagion model. Social contagion models have been used to model all sorts of things. Like juvenile delinquency, adoption of a new product, spread of fads, even obesity. Social contagion models are a kind of social network models. They can be represented graphically using nodes and links, which are circles and lines. The people are the circles. The structure of the network impacts the changes in people. Who you’re connected to affects who you become more like. In an agent-based model, the people are agents. Usually, agents do stuff. In an agent-based social contagion model, the agents might only do simple stuff, like copy their neighbor. The agent represents a person, but they might be just a little black box inside a circle, not much like an actual person. A real person would have a past, a mind, and a lot of mental processes. There might be quite a lot of stuff inside the black box, all kinds of rules and variables, but maybe not a person. Social network models made out of simple agents can go a long ways. They demonstrate the role of structure and interaction in the emergence of group level properties and behaviors, and are one way to study those things.

If someone wants their models to be a lot more complicated, but more realistic, they can rip the lid off the black box and try to stick something more like a person in the circle. This might be a learning agent, a cognitive agent, an emotional agent, a cognitive-emotional agent. All sorts of things are possible. This gets into computational modeling of mental processes and personal behavior. The scope expands. It’s like moving from the level of the proton to the level of a complex protein. There’s no comparison. Like the difference between my little finger and infinity, between today and eternity. Well, almost. But depending on what someone wants to understand, it might be desirable to go there. Cautiously, cognizant of scope and scale, boldly, go there. Because it’s there.

Psychological science, and other branches of social science, have studied people. That’s people in general, people together, and the individual person. This work can help us stick a person in a circle in a model. There are seven facets of a person studied in psychological science. All are involved in observable behavior. All are necessary to construct a good model of a real person.  These seven are: learning, motivation, cognition, emotion, personality, sociality, and culture. Learning includes behavior in the sense that it was studied by behaviorists. Personality includes identity and the self. Lots of theories and experiments have produced insight into the seven facets in the last hundred years. Literature and philosophy have played a part. Sociology and social psychology have made primary contributions to the understanding of sociality and social behavior. Anthropology and sociology have dug into the cultural inheritance, which is the symbolic content, that shapes every lives.

Recently, neuroscience has produced other useful findings that are being integrated into the psychological literature. Part of the integration involves computational modeling. Neuroscience is revolutionizing our understanding of the seven facets of psychology. The neuroscientific investigations and their integration are not complete. This is happening today. We are on the threshold of something whole and useful. The outlines have emerged from the mist.

One of the results of this integration has been the discovery that the seven facets of psychology are not independent. They’re not really even separate. More like seven lines of sight into one thing. This is complex. Unbelievably complex. Luckily, there’s a thing called complexity science. Complexity science was sorta, not really, invented at the Santa Fe Institute. It can help.

A popular, older, but still productive, way to simulate a person, is by using the belief-desire-intention (BDI) model. This has a variety of implementations, as it has evolved with twists and turns and various psychological and computational influences (Bratman, 1987; Rao and Georgeff, 1998). BDI architectures are an advance over belief-desire theories of animal behavior (de Wit & Dickinson, 2009) because, by including intentions, they incorporate the function of anticipation, or prediction, which has been established as a foundational component of evolved cognition (Butz & Pezzulo, 2008; Castelfranchi, 2005; Clark, 2013; Pezzulo & Castelfranchi, 2009). By bringing in beliefs, BDI has made a place for symbolic thought, which ushers in language and culture as a recognized psychological facet. Anticipation, prediction, or forward modeling gives rise to expectations, which are a key component needed to explain human behavior. Anticipation is an interesting functional capacity whose depth and breadth are still being explored. It means that behavior can be selected not only by remembering consequences, as in trial and error and operant conditioning, but also by mentally simulating the results of hypothetical actions (Pezzulo, Candidi, Dindo & Barca, 2013).

One of the weaknesses of some BDI architectures is their treatment of emotion. It gets in there sideways, because of its relation to desire, but as an explicit component it is most often left out. There is, however, a belief-desire theory of emotion, and recent efforts have incorporated it in computational modeling of intentional agents. The belief-desire (aka belief-goal) theory of emotions rests on the contention that human emotion cannot be understand without its cognitive components. In this respect it is akin to the ideas of such researchers as Richard Lazarus (1991), Klaus Scherer (2001), and Nico Frijda (1986, 2004).

Early proponents of a belief-desire theory of emotions were Gerald Clore and Andrew Ortony (Ortony, Clore, & Collins, 1987; Clore & Ortony, 2013). Their version is also referred to as the OCC (Ortony-Clore-Collins) model. A nice NetLogo implementation of this theory was done by two Jordanian modelers (Abu Maria & Abu Zitar, 2007). Abu Maria & Abu Zitar include a nice discussion of several previous models. Another model was built by a graduate student at USC (Jiang, 2007), using an architecture referred to as EBDI, or Emotional BDI. The EMA model of Stacy Marsella and Jonathan Gratch (2009, 2014), also originated at USC, is another model integrating BDI principles with appraisal theories of emotion. Ema is a large model with a complex architecture incorporating a lot of psychological theory. Ema is our friend.

A more recent theoretical computational model of the belief-desire theory of emotion was developed by Rainier Reisenzein (2012). Reisenzein has a lot of good stuff to say about emotion, cognition, neuroscience, and computational models. He has coauthored an interesting article about modeling with a group of prominent computer scientists (Reisenzein, Hudlicka, Dastani, Gratch, Hindriks, Lorini & Meyer, 2013). The article discusses emotion theory, BDI models, and recent OCC-based models such as FAtiMA. FAtiMA was developed by modelers in Portugal and has been redeployed in a modular version (Dias, Mascarenhas, & Paiva, 2014). Another recent model, “EMO” was developed by the animal ecologist Ellen Evers at Utrecht (Evers, de Vries, Spruijt & Sterck, 2014). This model doesn’t seem to be OCC or EBDI based, but builds on a simplified neuroanatomy and emotion theory. Another emotional agent model has been developed by Salichs & Malfaz in Madrid (2012). This model has a decision process based on wellbeing, happiness, sadness, and fear. Their article includes a brief discussion of similar earlier models by other modelers. Some Malaysian modelers have published a nice review of norms in multiagent systems (Mahmoud, Ahmad, Yusoff & Mustapha, 2014).

Reisenzein, as well as Ortony and Clore, have influenced Cristiano Castelfranchi and Maria Marceli, the iconic grandparents of the psychology of emotion in Italy. Located in Rome, they have very thoroughly investigated every conceivable aspect of emotion over a very long period and recently published a book-length treatment of the role of anticipation and prediction in human emotion (Miceli & Castelfranchi, 2015).  Miceli and Castelfranchi acknowledge Ortony and Clore’s “belief-goal” perspective as the foundation of their work, and Ortony wrote a foreward to the book. Castelfranchi has an apparent ability to stay on the cutting edge of things and has written on computational modeling as a generative method for social science (2014), has shown an interest in neuroscience, and has co-authored with another insightful Roman researcher called Giovanni Pezzulo (Pezzulo & Castelfranchi, 2009).

Giovanni Pezzulo, another busy Italian living in Rome, has written some nice things. Pezzulo’s basic area of study is action selection and decision theory. Among his specific topics of interest have been behavioral control models (Verschure, Pennartz, & Pezzulo, 2014; Pezzulo & Cisek, 2016), and the more cutting edge active inference models (Pezzulo, Rigoli & Friston, 2015). These look like theoretical computational models of brain processes, not agent-based models or full-scale computer simulations, although some of the work does involve simulation.

The article Pezzulo publish with Paul Cisek in 2016 is interesting for a number of a reasons. First, because it was written with Cisek. Cisek was born in Poland but is professor at University of Montreal. Cisek has his name on a bunch of cool papers about neuroscience, decision making and action control (Cisek, 2007; Cisek & Kalaska, 2010, Cisek & Pastor-Berrier, 2014). And, this article takes a control system perspective emphasizing the importance of feedback, which places it in the tradition of both systems science and systems biology. And, they talk about intentional action. Not to mention, active inference. What’s more, the article focuses on affordances. Accordingy to their hypothesis of “hierarchical affordance competition”:

“. . . intentional action can be conceptualized as a “purposive” navigation in an ‘affordance landscape’: a temporally extended space of possible affrodance, which changes over time due to events in the environment but also – importantly – due to the agent’s own actions. The key for extending the simple competition among affordances toward intentional action is to recognize that brains are continuously engaged in generating predictions (e.g., about future opportunities) rather than just reacting to already available affordances.”

Pezzulo & Cisek, 2016

So yes, my beverage is there, and I drink because it’s there. But I am there too, and being me, I might not necessarily drink this time, or ever.

Clearly, not all brilliant minds are Italian or Dutch. The chief champion for active inference models of brain function is Karl Friston of University College London (Friston, Schwartenbeck, FitzGerald, Moutoussis, Behrens & Dolan, 2013). Active inference, I think, grows out of neuroscience findings, philsophy of biology, and connects with such things as embodiment and enactivism. There is an acknowledgment of both goal-directed (voluntary, intentional) and habitual action. Active inference theory also connects with decision theory and expected utility theory (Friston, et al., 2013). Generally speaking, active inference supercedes RL (reinforcement learning), but there has been an attempt at integration with work on reward and associative learning (Friston, FitzGerald, Rigoli, Schwartenbeck, O’Doherty & Pezzulo, 2016). It seems to be the hot new thing and is beginning to percolate down to psychology. It should also be showing up in computational agents within agent-based models.

Active inference theory has been applied to the emotional brain (Seth & Friston, 2016). There is also an interesting article describing how it applies to social interaction (Gallagher & Allen, 2016). This article takes exception to the prevailing Theory of Mind paradigm in psychology that grounds social behavior in mental representations of other people’s thoughts, wishes, and feelings. Gallagher and Allen have followed the lead of Karl Friston (Friston & Frith, 2015), in applying the new paradigm of active inference to social cognition and the social context.

Not too long ago, I had a plan to get a Masters in Systems Science and apply to the PhD program in Mechanisms of Behavior at Indiana University. This is an interdisciplinary program beween psychology, neuroscience, and the animal behavior people over in biology. I started thinking along those lines and doing background reading. An idea that makes sense to me is that associative learning, emotion, and voluntary cognitive control are three among several mechanisms of behavior control, represented by linked but somewhat independent control systems that activate and sometimes compete in the brain, depending on events and situations. It is one way to conceptualize behavior that might overcome some of the backbiting among psychologists, a neural peaceful coexistence model. Simple and inviting, but possibly not very realistic. A key peice in this line of thinking, as I understand it, is the research within neuroscience that has explained the central role of the centrally located brain region called the striatum as a gathering place for alternative potential actions. There are interconnections between striatum, prefrontal cortex control, anterior cingulate conflict resolution, and amygdala emotional valencing. Potential actions arrive at the striatum, constrained by threshold requirements, and hang out there in an inhibited state until they fade or are released for implementation. Presumably they can arrive from a variety of sources. Mere valencing is a different role for emotion than the action readiness function championed by some (of my favorites) in the psychology of emotion, which gives it a broader role in proposing and weighting potential actions. We are thus faced with the possibility that all actions are inhibited unless released, but that they generally get released, because they are presented for implementation in the absense of conflict or competition. Conflict, however, does not have to be with other ready actions sitting in striatum. It can be conflict with goals and values, or self image. In this model the brain has the responsibility to monitor the contents of striatum and do lookups to determine these things. This goes above and beyond any emotional valencing already attached to potential actions and their contexts. Voluntary or consicous participation of prefrontal cortex is not necessarily involved.

Luiz Pessoa, a native of Brazil with a PhD from Boston University, worked at Indiana for a while, although he has since moved to the University of Maryland. Pessoa is an advocate of the strong version of cognitive-emotional interaction and an integrative holistic perspective on brain function, as opposed to more modular, dualistic, or parallel processing models. He wrote a nice chapter for The Wiley Hanbook of Cogntive Control (2017), and also has a recent theory piece in Trends in Cognitive Sciences (2017), that describes his views about brain mechanisms of emotion within the whole brain.

An approach to computational modeling of intelligent agents with a long track record is the Reinforcement Learning (RL) tradition. Although this sounds like an umbrella term for operant conditioning or behaviorism in general, RL lives in both computer science and psychology. RL might have something to offer computational modeling of agents who drink, to that extent that other models ignore associative learning or the reward systems in the brain. Some of this territory has been explored from the standpoint of psychology by Sanne de Wit and his colleagues in the Netherlands and across the channel at Cambridge (Watson, Hommel, de Wit, & Wiers, 2012; Robbins, Gillan, Smith, de Wit, & Ersche, 2012). They have looked, in particular, at the role of habit and impulsivity in problem drinking and in addiction. De Wit also has a nice chapter in the new Wiley Handbook on cognitive control (De Wit, 2017).

Two neuroscientists working in the RL domain have independently described an expansion of the search for neural correlates of classical and operant conditioning toward full scale integrative theories of learning, motivation, and behavior. They are Bernard Balleine and Mathew Botvinick. They talk about cognitive control, but it feels like cognition is somewhat of a black box in this approach. Since these efforts have roots in the behaviorist tradition, it would not surprise me if the target of investigation is often animal behavior rather than human behavior. Nothing wrong with that, psychologists nowadays seem to forget that human are animals, or to believe that symbolic thought, language, self awareness and imagining the future make us so different that we can leave animal learning out and explain all human behavior in terms of thoughts, plans, and rational choice. This is as absurd as the the old school view that we can explain all human behavior in terms of conditioning. Balleine, along with Anthony Dickinson, was a central figure in translation of classical learning theory into neuroscience (Balleine & Dickinson, 1998). Balleine has continued to write about the distinction between goal-directed and habitual action (Balleine & O’Doherty, 2010; Dezfouli & Balleine, 2012), and interactions between the prefrontal cortex and basal ganglia, which includes the striatum (Balleine, Dezfouli, Ito, & Doya, 2015; Hart, Leung & Balleine, 2014). Botvinick has a similar focus, has written about planning as inference with a computer scientist (Botvinick & Toussaint, 2012), and has published a recent 30-page review of integrative theories of learning, motivation and cogntion, under the rubric of motivation and cognitive control (Botvinick & Braver, 2015).

BDI, BDIE, and RL are abstractions, that may be only loosely correlated with neuroanatomy. They are computational models that attempt to produce intelligent behavior in artificial agents. They meagerly simulate the cognitive architectures and  behavioral repertoires of actual human persons. With the incredible advances coming out of neuroscience, however, psychological and computational scholars are making efforts to get closer to comprehensive, integrated models that encompass most of the seven facets of psychological science, and could soon produce simulations of mental processes and human behavior that will be remarkably consistent with the real thing. A lot of this work is in the nature of theoretical synthesis, and is not necessarily reflected in computational models that are implemented in software. Possibly noone has the resources or wherewithal at this moment to tackle a modeling job that would involve the level of complexity that presents itself in current theory. But it can inspire those who seek to review and improve on the abstract models commonly in use today.

There is some recent work building on neuroscience findings to amplify the role of emotion in action readiness and selection, and it is presented in the publications of two Dutch researchers who built on Nico Frijda’s theory of emotion (1986, 2004). Ridderinkhof and Rietveld coauthored an article with Frijda on human action and emotional impulses (Frijda, Ridderinkhof, & Rietveld, 2014). They carefully describe, category by category, human action and only arrive at top-down, conscious, voluntary control in the last paragraph, where it warrants only a few sentences. This is clearly a different view than the usual rational choice and intentional action theories in which voluntary control takes the top role. This paper is short and sweet, but shouldn’t be overlooked. Ridderinkhof has written another beautiful paper outlining a comprehensive theory of action (2014), which he labels IMPPACT, for Impetus, Motivation, and Prediction in Perception-Action Coordination theory. He also coauthored two recent papers on the role of intention in behavior inhibition (Ridderinkhof, van den Wildenberg & Brass, 2014; Schel, Ridderinkhof, & Crone, 2014).

Ridderinkhof’s 2014 paper includes an extensive history review of the ideomotor principle, which captures the contention that mental representations become key at some point in evolution, as at some point in human development (early childhood), to action selection and control. Ideomotor processes succeed, but coexist with, sensorimotor processes. A key figure in the elaboration of this line of thinking has been Bernard Hommel (2009, 2017), with his theory of event coding (TEC). The 2017 chapter, “Conscisouness and Action Control,” has some interesting things to say about voluntary control and intentional action.

Eric Rietveld’s work is situated in the world of ecological psychology, which attempts to explain behavior by reference to environmental context, and the evolution of our capabilities in response to the environment, with a heavy emphasis on the concept of affordances. This is not a bad place to be, since the entire brain – perception, memory, attention, cognition, emotion, the whole gamut – is extremely sensitive to context, both to place and to time, whether time of day, time of year, or time of life. The world is richly filled with affordances, which present possibilities for action (Rietveld, 2012). This same perspective can be applied to the social world, as it is filled with social affordances (van Dijk & Rietveld, 2017). Rietveld has also written on norms, and their role in nonreflective action (2008).

We don’t respond to affordances randomly, though, and our responses are not preprogrammed at birth. Our “concerns” are involved, and our personal histories and characteristics, as well as our current state. But the power of the opportunties presented by the situation are not to be underestimated.

” . . . both humans and animals are selectively responsive to one affordance rather than another, in a way that is related to the individual’s dynamically changing needs. This phenomenon of adequate responsiveness to relevant affordances in context is crucial and can even be seen as a paradigmatic form of unreflective action. Relevant affordances are alluring and bodily activating possibilities for action. This responsiveness has a basic normative aspect that cannot be reduced to mechanistic causal explanation.

“Unreflective actions are performed without mediation of explicit deliberation or reflection. Of course not all of our life is spent in a state of unreflective action. Sometimes we lack the relevant skills, things go very wrong, or situations are too complex, thus forcing us to reflect or deliberate explicity. However, here I will restrict myself as much as possible to investigating those episodes where the activities of a skillful individual unfold without reflection on his or her part. Discussion of the many interesting issues related to the interactions between reflective action and unreflective action will have to be postponed to another occasion.”

Rietveld, 2012

I drink, in short, because it’s there.

References:

Abu Maria, K. & Abu Zitar, R. (2007). Emotional agents: A modeling and an application. Information and Software Technology 49, 695-716.

Balleine, B. W., Dezfouli, A., Ito, M., & Doya, K. (2015). Hierarchical control of goal-directed action in the cortical-basal ganglia network. Current Opinion in Behavioral Science, 5, 1-7.

Balleine, B. W., & Dickinson, A. (1998). Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology, 37, 407-419.

Balleine, B. W., & O’Doherty, J. P. (2010). Human and rodent homologies in action control: Corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology Reviews, 35, 48-69.

Botvinick, M., & Braver, T. (2015). Motivation and cognitive control: From behavior to neural mechanism. Annual Review of Psychology, 66, 83-113.

Botvinick, M., & Toussaint, M. (2012). Planning as inference. Trends in Cognitive Sciences, 16, 485-488.

Bratman, M. (1987). Intentions, plans, and practical reason. Cambridge, MA: Harvard University Press.

Butz, M. V., & Pezzulo, G. (2008). Benefits of anticipations in cognitive agents. In: G. Pezzulo, M. V. Butz, C. Castelfranchi, & R. Falcone (eds.), The challenge of anticipation, pp. 45-62. Berlin: Springer-Verlag.

Castelfranchi, C. (2005). Mind as an anticipatory device: For a theory of expectations. In: M. De Gregorio, V. Di Maio, M. Frucci, & C. Musio (eds.), Brain, vision, and artificial intelligence, First International Symposium, Proceedings. Cham, Switzerland: Springer International Publishing. 

Castelfranchi, C. (2014). Making visible “the invisible hand”: The mission of social simulation. In: D. F. Adamatti, G. Pereira Dimuro, H. Coelho (eds.), Interdisciplinary applications of agent-based social simulation and modeling. IGI Global.

Cisek, P. (2007). Cortical mechanisms of action selection: the affordance competition hypothesis. Philosophical Transactions of the Royal Society B, 362, 1585-1599.

Cisek, P., & Kalaska, J. F. (2010). Neural mechanisms for interacting with a world full of action choices. Annual Review of Neuroscience, 33, 269-298.

Cisek, P., & Pastor-Berrier, A. (2014). On the challenges and mechanisms of embodied decisions. Philosophical Transactions of the Royal Society B, 369: 1655.

Clark, A. (2013) Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences 36(3), 181-204.

Clore, G. L. & Ortony, A. (2013). Psychological construction in the OCC model of emotion. Emotion Review 5, 335-343.

De Wit, S. (2017). Control of behaviour by competing learning systems. In: T. Egner (ed.), The Wiley handbook of cognitive control, 190-206. New York: John Wiley & Sons.

De Wit, S., & Dickinson, A. (2009). Associative theories of goal-directed behavior: a case for animal-human translation models. Psychological Research 73, 463-476.

Dezfouli, A., & Balleine, B. W. (2012). Habits, action sequences, and reinforcement learning. European Journal of Neuroscience, 35(7), 1036-1051.

Dias, J., Mascarenhas, S., & Paiva, A. (2014). FAtiMA modular: Towards an agent architecture with a generic appraisal framework. In: Bosse, Broekens, Dias, van der Zwaan (eds.), Emotion modeling: Towards pragmatic computational models of affective processes, pp. 44-56. Cham, Switzerland: Springer International Publishing.

Evers, E., de Vries, H., Spruijt, B. M., & Sterck, E. H. M. (2014). The EMO-Model: An agent-based model of primate social behavior regulated by two emotional dimensions, anxiety-FEAR and satisfaction-LIKE. PLOS One, 9: e87955.

Frijda, N.H. (1986). The emotions. New York: Cambridge University Press.

Friston, K., & Frith, C. D. (2015). Active inference, communication and hermeneutics. Cortex, 68, 129-143.

Friston, K., FitzGerald, P., Rigoli, F., Schwartenbeck, P., O’Doherty, J., & Pezzulo, G. (2016). Active inference and learning. Neuroscience and Biobehavioral Reviews, 68, 862-879.

Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., & Dolan, R. J, (2013). The anatomy of choice: active inference and agency. Frontiers in Human Neuroscience, 7, 598.

Gallagher, S., & Allen, M. (2016). Active inference, enactivism and the hermeneutics of social cognition. Synthese, 1-22.

Hart, G., Leung, B. K., & Balleine, B. W. (2014) Dorsal and ventral streams: The distinct role of striatal subregions in the acquisition and performance of goal-directed actions. Neurobiology of Learning and Memory, 108, 104-118.

Hommel, B. (2009). Action control according to TEC (theory of event coding). Psychological Review, 73, 512-526.

Hommel, B. (2017). Consciousness and action control. In: T. Egner (ed.), The Wiley handbook of cognitive control, 111-123. New York: John Wiley & Sons.

Jiang, H. (2007). From rational to emotional agents, doctoral dissertation. University of Southern California.

Lazarus, R. S. (1991). Emotion and adaptation. New York: Oxford University Press.

Mahmoud, M. A., Ahmad, M. S., Yusoff, M. Z. M, & Mustapha, A. (2014). A review of norms and normative multiagent systems. The Scientific World Journal, 2014: 684587, 1-23.

Marcella, S. C. & Gratch, J. (2009). EMA: A process model of appraisal dynamics. Cognitive Systems Research 10, pp. 70-90.

Marcella, S., & Gratch, J. (2014). Computationally modeling human emotion. Communications of the ACM, 57, 56-67.

Miceli, M., & Castelfranchi, C. (2015). Expectancy & emotion. Oxford: Oxford University Press.

Ortony, A., Clore, G. L., & Collins, A. (1988). The cognitive structure of emotions. New York: Cambridge University Press.

Pessoa, L. (2017). A network model of the emotional brain. Trends in Cognitive Science 21(5), 357-371.

Pessoa, L. (2017). Cognitive control and emotional processing. In: Egner, T. (ed.), The Wiley handbook of cognitive control, pp. 392-407. New York: John Wiley & Sons.

Pezzulo, G., Candidi, M., Dindo, H., & Barca, L. (2013). Action simulation in the human brain: Twelve qeustions. New Ideas in Psychology, 31, 270-290.

Pezzulo, G., & Castelfranchi, C. (2009). Intentional action: from anticipation to goal-directed behavior. Psychological Research 73, 437-440.

Pezzulo, G., & Cisek, P. (2016). Navigating the affordance landscape: Feedback control as a process model of behavior and cognition. Trends in cognitive sciences 20(6), 414-424.

Pezzulo, G., Rigoli, F., & Friston, K. (2015). Active inference, homeostatic regulation and adaptive behaviouiral control. Progress in Neurobiology 134, 17-35.

Rao, A. S., & Georgeff, M. P. (1998). Decision procedures for BDI logics. Journal of Logic and Computation 8: 293-342.

Reisenzein, R. (2012). What is an emotion in the belief-desire theory of emotion? In: F. Paglieri, L. Tummolini, R. Falcone, and M. Miceli (eds), The goals of cognition: essays in honour of Cristiano Castelfranchi, pp. 193-223. London: College Publications.

Reisenzein, R., Hudlicka, E, Dastani, M., Gratch, J., Hendriks, K., Lorini, E., & Meyer, J. (2013). Computational modeling of emotion: Toward improving the inter- and intradisciplinary exchange. IEEE Transactions on Affective Computing, 4, 246-266

Reitveld, E. (2008). Situated normativity: The normative aspect of embodied cognition in unreflective action. Mind, 117, 973-1001.

Reitveld, E. (2012). Bodily intentionality and social affordances in context. In: F. Paglieri (ed.), Consciousness in interaction. The role of the natural and social context in shaping consciousness.

Robbins, T. W., Gillan, C. M., Smith, D. G., de Wit, S., & Ersche, K. D. (2012). Neurocognitive endophenotypes of impulsivity and compulsivity: towards dimensional psychiatry. Trends in Cognitive Science 16, 81-91.

Salichs, M. A., & Malfaz,, M. (2012). A new approach to modeling emotions and their use on a decision making system for artificial agents. IEEE Transactions on Affective Computing, 3, 56-68.

Scherer, K. R. (2001). Appraisal considered as a process of multilevel sequential checking. In: K. R. Scherer, A. Schorr, and T. Johnstone (eds.), Appraisal process in emotion: theory, methods, research, pp. 92-120. Oxford: Oxford University Press.

Seth, A. K., & Friston, K. J. (2016). Active interoceptive inference and the emotional brain. Philosophical Transactions of the Royal Society B, 371(1708),

van Dijk, L., & Rietveld, E. (2017). Foregrounding sociomaterial practice in our understanding of affordances: The skilled intentionality framework. Frontiers in Psychology, 7.

Verschure, P. F. M. J., Pennartz, C. M. A., & Pezzulo, G. (2014). The why, what, where, when and how of goal-directed choice: neuronal and computational principles. Philosophical Transactions of the Royal Society B, 369: 20130483, 1-14.

Watson, P., de Wit, S., Hommel, B., & Wiers, R. W. (2012). Motivational mechanisms and outcome expectancies underlying the approach bias toward addictive substances. Frontiers in Psychology 3, 440.