Decision‐making and Neuroeconomics

Tobias Kalenscher, University of Amsterdam, Amsterdam, The Netherlands

Published online: March 2009

DOI: 10.1002/9780470015902.a0021397

Decision-making is the process of choosing one out of several alternatives. The study of decision-making is inherently multidisciplinary and can be approached from many different angles. Traditional accounts in economics and biology have a normative flavour and prescribe, rather than describe decision-making. Recently, however, efforts in psychology and behavioural ecology have resulted in the development of theories with higher descriptive validity. Furthermore, neuroeconomics is a new interdisciplinary field that combines contributions from economics, biology and psychology and aims to identify a biologically valid, mechanistic, mathematical and behavioural theory of choice and exchange. Recent neurophysiological advances have further deepened our understanding of the mechanistic implementation of a decision in the brain.

Key Concepts

Utility function: A mathematical function that models the relationship between an objective measure of a commodity (such as money) and the subjective value or desirability of that commodity. The utility function is often assumed to be concave, which is supposed to underly risk aversion.

Risk attitude: Animals and humans are almost never risk neutral, i.e. when choosing between a very risky and a less risky (or even certain) option, most organisms prefer the less risky option (risk aversion) even if the payoff of both options is identical, or even better for the risky option. Occasionally, some individuals systematically prefer the more risky option (risk proneness).

Expected utility theory: Normatively flavoured model in economics that tries to prescribe how humans should make decisions under risk. It is assumed that decision-makers compute the probability-weighted sum of the utilities of all possible outcomes, integrate them with their current asset, and then choose the option the highest expected utility in final assets.

Discounted utility theory: Similar model to expected utility, but for decisions in time. The weighting factor in discounted utility theory is not probability, but a function of the delay between choice and consumption.

Game theory: Framework to formalize choice behaviour involving multiple interacting decision-makers. A game allows to isolate and investigate in a controlled environment a limited set of premises of economic decision theory.

Prospect theory: Alternative theory to expected utility theory with higher descriptive validity. It is assumed that outcomes of a decision are evaluated with respect to changes in wealth (gains versus losses), not in final states. Furthermore, the utility function is concave in the domain of gains, and convex in the domain of losses, and steeper for losses than for gains.

Choice heuristic: A simple decision rule, or set of rules, that humans and animals apply to make decisions in complex environments.

Optimal foraging theory: Based on the assumption that evolution should have favoured the development of decision mechanisms that maximize Darwinian fitness, optimal foraging theory aims to explore animal behaviour from an optimal choice perspective.

Neuroeconomics: Interdisciplinary field that attempts to identify a biologically valid, mechanistic, mathematical and behavioural theory of choice and exchange.

Keywords: neuroeconomics; reward; risk; intertemporal choice; optimal foraging; brain

Figure 1. Utility functions can explain risk attitude. (a) Utility as a function of the value of a commodity, expressed in monetary gains. The utility curve is concave and is a decelerating function of the current level of wealth because the marginal utility, i.e. the utility increment with each additional unit, decreases with increasing level of wealth. A concave utility function predicts risk aversion when choosing between a medium-sized, certain (MM) and a large and small risky monetary option (MS and ML). The average utility of the certain option exceeds the average utility of the risky option because the utility of the large outcome is only sublinearly larger compared to the utilities of the other outcomes. (b) A convex function predicts risk proneness.
Figure 2. Exponential versus hyperbolic discounting of future events. The figure describes a situation in which an agent chooses between a small, short-term outcome or a large, long-term outcome (proximal) and another situation in which both outcomes are deferred into the future by the same time interval (distant). (a) Exponential utility function of a large, delayed (grey line) and small, short-term commodity (black line). With exponential discounting, stationarity holds because the utility of the large reward (UL) is always higher than the utility of the small reward (US). This is true when both rewards are temporally proximal, and also when they are deferred by the same time interval into the future. (b) Hyperbolic discounting can explain preference reversals and the violation of stationarity. Owing to the steeper decay for short delays, the utility of the small, short-term commodity is higher than the large, delayed reward for temporally proximal outcomes, but the utility order reverses when both outcomes are deferred into the future.
Figure 3. Value and weighting function according to Prospect Theory. (a) The value function for the domains of gains and losses. The x-axis represents the objective measure of a commodity, such as money, the y-axis depicts the subjective value, or utility, of the commodity as a function of its objective value. (b) The weighting function. The dotted line is the objective measure of probability, the solid line is the weighting function. See text for description.
Figure 4. Neural representation of discounted value, and evidence for multiple choice systems in the brain. (a) A pigeon chooses between a short, always immediate and a large, gradually delayed reward. This figure shows the discounted value representation on single-cell level in the avian forebrain. In the left panel, the animal chooses the large reward, but the neuron's activity (grey area) decreases with increasing delay duration (left, from top to bottom). When the animal chooses the small, always immediate reward (right panel), there is no noticeable difference in neural activity. Further analysis has shown that this neuron was not only responsive to changes in delay, but also reward amount. Reprinted from Kalenscher et al. (2005). Copyright 2005, with friendly permission from Elsevier. (b) Humans choose between small, short-term or large, delayed rewards. Left panel: Activated brain regions for choices in which money is available immediately. PCC, posterior cingulate cortex; MPFC, medial prefrontal cortex; VStr, ventral striatum and MOFC, medial orbitofrontal cortex. Right panel: Brain regions that were activated while making choices independent of delay. RPar, right intraparietal cortex; DLPFC, dorsolateral prefrontal cortex; VLPFC, ventrolateral prefrontal cortex; LOFC, right lateral orbitofrontal cortex; VCtx, visual cortex; PMA, premotor area and SMA, supplementary motor area. Reprinted from McClure et al. (2004). Copyright 2004, with friendly permission from AAAS.
close
 References
    Ainslie G (1975) Specious reward, a behavioral theory of impulsiveness and impulse control. Psychological Bulletin 82: 463–496.
    Bateson M, Healy SD and Hurly TA (2003) Context-dependent foraging decisions in rufous hummingbirds. Proceedings. Biological Sciences 270: 1271–1276.
    book Bateson M and Kacelnik A (1998) "Risk-sensitive foraging, decision-making in variable environments". In: Dukas R (ed.) Cognitive Ecology, pp. 297–341. Chicago, IL: University of Chicago Press.
    Benzion U, Rapoport A and Yagil J (1989) Discount rates inferred from decisions, an experimental study. Management Science 35: 270–284.
    Bernoulli D (1954) Exposition of a new theory on the measurement of risk. Econometrica 22: 23–36.
    Brosnan SF, Jones OD, Lambeth SP et al. (2007) Endowment effects in chimpanzees. Current Biology 17: 1704–1707.
    Camerer C, Loewenstein G and Prelex D (2005) Neuroeconomics: how neuroscience can inform economics. Journal of Economic Literature 43: 9–64.
    Celebrini S and Newsome WT (1994) Neuronal and psychophysical sensitivity to motion signals in extrastriate area MST of the macaque monkey. Journal of Neuroscience 14: 4109–4124.
    Crockett MJ, Clark L, Tabibnia G, Lieberman MD and Robbins TW (2008) Serotonin modulates behavioral reactions to unfairness. Science 320: 1739.
    De Martino B, Kumaran D, Seymour B and Dolan RJ (2006) Frames, biases, and rational decision-making in the human brain. Science 313: 684–687.
    Durstewitz D (2003) Self-organizing neural integrator predicts interval times through climbing activity. Journal of Neuroscience 23: 5342–5353.
    Fehr E and Schmidt KM (1999) A theory of fairness, competition and cooperation. Quarterly Journal of Economics 114: 817–868.
    Fiorillo CD, Tobler PN and Schultz W (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299: 1898–1902.
    Frederick S, Loewenstein G and O'Donoghue T (2002) Time discounting and time preference, a critical review. Journal of Economic Literature 40: 351–401.
    Gigerenzer G and Goldstein DG (1996) Reasoning the fast and frugal way, models of bounded rationality. Psychological Review 103: 650–669.
    Glimcher PW, Kable J and Louie K (2007) Neuroeconomic studies of impulsivity, now or just as soon as possible? American Economic Review 97: 142–147.
    Glimcher PW and Rustichini A (2004) Neuroeconomics: the consilience of brain and decision. Science 306: 447–452.
    Goldstein DG and Gigerenzer G (2002) Models of ecological rationality, the recognition heuristic. Psychological Review 109: 75–90.
    Green L, Fristoe N and Myerson J (1994) Temporal discounting and preference reversals in choice between delayed outcomes. Psychonomic Bulletin & Review 1: 383–389.
    book Gul F and Pesendorfer W (2005) The Case for Mindless Economics. Princeton: Princeton Press.
    Güth W, Schmittberger R and Schwarze B (1982) An experimental analysis of the ultimatum game. Journal of Economic Behaviour and Organization 3: 367–388.
    Hare TA, O'Doherty J, Camerer CF, Schultz W and Rangel A (2008) Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. Journal of Neuroscience 28: 5623–5630.
    Huettel SA, Stowe CJ, Gordon EM, Warner BT and Platt ML (2006) Neural signatures of economic preferences for risk and ambiguity. Neuron 49: 765–775.
    Izawa E, Aoki N and Matsushima T (2005) Neural correlates of the proximity and quantity of anticipated food rewards in the ventral striatum of domestic chicks. European Journal Neuroscience 22: 1502–1512.
    Kable JW and Glimcher PW (2007) The neural correlates of subjective value during intertemporal choice. Nature Neuroscience 10: 1625–1633.
    Kacelnik A (1997) Normative and descriptive models of decision making: time discounting and risk sensitivity. Ciba Foundation Symposium 208: 51–67; discussion 67–70.
    Kacelnik A and Bateson M (1996) Risky theories – the effects of variance on foraging decisions. American Zoology 36: 402–434.
    Kahneman D and Tversky A (1979) Prospect theory, an analysis of decision under risk. Econometrica 47: 263–291.
    Kahneman D and Tversky A (1984) Choices, values, and frames. American Psychology 39: 341–350.
    Kalenscher T, Ohmann T and Gunturkun O (2006a) The neuroscience of impulsive and self-controlled decisions. International Journal of Psychophysiology 62: 203–211.
    Kalenscher T, Ohmann T, Windmann S, Freund N and Gunturkun O (2006b) Single forebrain neurons represent interval timing and reward amount during response scheduling. European Journal Neuroscience 24: 2923–2931.
    Kalenscher T and Pennartz CM (2008) Is a bird in the hand worth two in the future? The neuroeconomics of intertemporal decision-making. Progress in Neurobiology 84: 284–315.
    book Kalenscher T and Tobler PN (2008) "Comparing risky and inter-temporal decisions; views from psychology, ecology and microeconomics". In: Hoffmann KP (ed.) Psychology of Decision Making in Economics, Business and Finance, pp. 111–135. New York: Nova Science Publishers.
    Kalenscher T, Windmann S, Diekamp B et al. (2005) Single units in the pigeon brain integrate reward amount and time-to-reward in an impulsive choice task. Current Biology 15: 594–602.
    Kim JN and Shadlen MN (1999) Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nature Neuroscience 2: 176–185.
    Kim S, Hwang J and Lee D (2008) Prefrontal coding of temporally discounted values during intertemporal choice. Neuron 59: 161–172.
    King-Casas B, Tomlin D, Anen C et al. (2005) Getting to know you, reputation and trust in a two-person economic exchange. Science 308: 78–83.
    Knoch D, Gianotti LR, Pascual-Leone A et al. (2006a) Disruption of right prefrontal cortex by lowfrequency repetitive transcranial magnetic stimulation induces risk taking behavior. Journal of Neuroscience 26: 6469–6472.
    Knoch D, Pascual-Leone A, Meyer K, Treyer V and Fehr E (2006b) Diminishing reciprocal fairness by disrupting the right prefrontal cortex. Science 314: 829–832.
    Knutson B, Taylor J, Kaufman M, Peterson R and Glover G (2005) Distributed neural representation of expected value. Journal of Neuroscience 25: 4806–4812.
    Kobayashi S and Schultz W (2008) Influence of reward delays on responses of dopamine neurons. Journal of Neuroscience 28: 7837–7846.
    Koopmans TC (1960) Stationary ordinal utility and impatience. Econometrica 28: 287–309.
    Kosfeld M, Heinrichs M, Zak PJ, Fischbacher U and Fehr E (2005) Oxytocin increases trust in humans. Nature 435: 673–676.
    Laibson DI (1997) Golden eggs and hyperbolic discounting. Quarterly Journal of Economics 112: 443–477.
    other Louie K and Glimcher PW (2006) Temporal discounting activity in monkey parietal neurons during intertemporal choice. Annual Meeting of the Society for Neuroscience, Program no. 605.5.
    book Luce RD (1986) Response Times, Their Role in Inferring Elementary Mental Organization. New York: Oxford University press.
    MacDonald DH, Kagel JH and Battalio RC (1991) Animals’ choices over uncertain outcomes, further experimental results. Economic Journal 101: 1065–1084.
    Marsh B and Kacelnik A (2002) Framing effects and risky decisions in starlings. Proceedings of the National Academy of Sciences of the USA 99: 3352–3355.
    Maskin E (2008) Can neural data improve economics? Science 321: 1788–1789.
    Mazur JE (1984) Tests of an equivalence rule for fixed and variable reinforcer delays. Journal of Experimental Psychology, Animal Behavior Processes 10: 426–436.
    McClure SM, Laibson DI, Loewenstein G and Cohen JD (2004) Separate neural systems value immediate and delayed monetary rewards. Science 306: 503–507.
    McCoy AN and Platt ML (2005) Risk-sensitive neurons in macaque posterior cingulate cortex. Nature Neuroscience 8: 1220–1227.
    Metcalfe J and Mischel W (1999) A hot/cool-system analysis of delay of gratification, dynamics of willpower. Psychological Review 106: 3–19.
    Nash JF (1950) Equilibrium points in n-person games. Proceedings of the National Academy of Sciences of the USA 36: 48–49.
    Navarick DJ and Fantino E (1972) Transitivity as a property of choice. Journal of the Experimental Analysis of Behavior 18: 389–401.
    Navarro AD and Fantino E (2005) The sunk cost effect in pigeons and humans. Journal of the Experimental Analysis of Behavior 83: 1–13.
    book von Neumann J and Morgenstern O (1944) Theory of Games and Economic Behavior. Princeton, NJ: University Press.
    O'Doherty J, Dayan P, Schultz J et al. (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304: 452–454.
    Padoa-Schioppa C and Assad JA (2006) Neurons in the orbitofrontal cortex encode economic value. Nature 441: 223–226.
    book Pascal B (1948) Great shorter works (translated from French). Philadelphia, PA: Westminster.
    Platt ML and Glimcher PW (1999) Neural correlates of decision variables in parietal cortex. Nature 400: 233–238.
    Rangel A, Camerer C and Montague PR (2008) A framework for studying the neurobiology of value-based decision making. Nature Reviews. Neuroscience 9: 545–556.
    Ratcliff R (1988) Continuous versus discrete information processing, modeling accumulation of partial information. Psychological Review 95: 238–255.
    Roesch MR, Calu DJ and Schoenbaum G (2007) Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nature Neuroscience 10: 1615–1624.
    Roesch MR and Olson CR (2005) Neuronal activity in primate orbitofrontal cortex reflects the value of time. Journal of Neurophysiology 94: 2457–2471.
    Roesch MR, Taylor AR and Schoenbaum G (2006) Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation. Neuron 51: 509–520.
    Samuelson PA (1937) A note on measurement of utility. Review of Economic Studies 4: 155–161.
    Sanfey AG, Rilling JK, Aronson JA, Nystrom LE and Cohen JD (2003) The neural basis of economic decision-making in the ultimatum game. Science 300: 1755–1758.
    Schall JD (2001) Neural basis of deciding, choosing and acting. Nature Reviews. Neuroscience 2: 33–42.
    Schultz W (1998) Predictive reward signal of dopamine neurons. Journal of Neurophysiology 80: 1–27.
    Schultz W, Dayan P and Montague PR (1997) A neural substrate of prediction and reward. Science 275: 1593–1599.
    Shadlen M and Newsome WT (2001) Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. Journal of Neurophysiology 86: 1916–1936.
    Shafir S (1994) Intransitivity of preferences in honey bees: support for ‘comparative’ evaluation of foraging options. Animal Behavior 48: 55–67.
    Shafir S, Reich T, Tsur E, Erev I and Lotem A (2008) Perceptual accuracy and conflicting effects of certainty on risk-taking behaviour. Nature 453: 917–920.
    Simon H (1956) Rational choice and the structure of the environment. Psychological Review 63: 129–138.
    Singer T, Seymour B, O'Doherty JP et al. (2006) Empathic neural responses are modulated by the perceived fairness of others. Nature 439: 466–469.
    Stephens DW (1981) The logic of risk-sensitive foraging preferences. Animal Behavior 29: 628–629.
    Stephens DW (2002) Discrimination, discounting and impulsivity, a role for an informational constraint. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 357: 1527–1537.
    Stephens DW (2008) Decision ecology: foraging and the ecology of animal decision-making. Cognitive, Affective & Behavioral Neuroscience 8: 475–484.
    Stephens DW and Anderson D (2001) The adaptive value of preference for immediacy, when shortsighted rules have farsighted consequences. Behavioral Ecology 12: 330–339.
    book Stephens DW and Krebs JR (1986) Foraging Theory. Monographs in Behavior and Ecology. Princeton, NJ: University Press.
    Thaler RH and Shefrin HM (1981) An economic theory of self-control. Journal of Political Economy 89: 392–406.
    Thielscher A and Pessoa L (2007) Neural correlates of perceptual choice and decision making during fear-disgust discrimination. Journal of Neuroscience 27: 2908–2917.
    Tobler PN and Kalenscher T (2007) Awfully afraid? Dissociating decision- from motor- and sensory-related brain activation during perceptual choices. Journal of Neuroscience 27: 6081–6082.
    Tobler PN, O'Doherty JP, Dolan RJ and Schultz W (2007) Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems. Journal of Neurophysiology 97: 1621–1632.
    Tom SM, Fox CR, Trepel C and Poldrack RA (2007) The neural basis of loss aversion in decision-making under risk. Science 315: 515–518.
    Waite TA (2001) Intransitive preferences in hoarding gray jays (Perisoreus canadensis). Behav Ecol Sociobiol 50: 116–121.
    Yang T and Shadlen MN (2007) Probabilistic reasoning by neurons. Nature 447: 1075–1080.
 Further Reading
 Microeconomics:
    book Pindyck RS and Rubinfeld DL (2008) Microeconomics (7th edn). Englewood Cliffs, NJ: Prentice Hall Publishers.
 Psychology of decision-making:
    book Gigerenzer G (2008) Gut Feelings: The Intelligence of the Unconscious. New York: Penguin Group.
    book Kahneman D and Tversky A (2000) Choices, Values and Frames. Cambridge, UK: Cambridge University Press.
 Neuroeconomics:
    book Glimcher PW (2004) Decisions, Uncertainty and the Brain – The Science of Neuroeconomics. Cambridge, MS: MIT Press.

Related Sub-Topics:

Neural Networks and Behaviour | Behavioural Ecology
Contact Editor close
Submit a note to the editor about this article by filling in the form below.

* Required Field

Article Title: Decision‐making and Neuroeconomics
 
How to Cite close
Kalenscher, Tobias(Mar 2009) Decision‐making and Neuroeconomics. In: eLS. John Wiley & Sons Ltd, Chichester. http://www.els.net [doi: 10.1002/9780470015902.a0021397]