Grasping the Impalpable: The Role of Endogenous Reward in Choices, Including Process Addictions

ABSTRACT

The list of proposed addictions has recently grown to include television, videogames, shopping, day trading, kleptomania, and use of the Internet. These activities share with a more established entry, gambling, the property that they require no delivery of a biological stimulus that might be thought to unlock a hardwired brain process. I propose a framework for analyzing that class of incentives that do not depend on the prediction of physically privileged environmental events: people have a great capacity to coin endogenous reward; we learn to cultivate it, and, where it is entrapping, to minimize it, by managing internally generated appetites for it. The basic method of cultivating endogenous reward is to learn cues that predict when best to harvest the reward that has been made possible by the growth of these appetites. This hedonic management occurs in the same motivational marketplace as the instrumental planning that seeks environmental goods in the conventional manner, and presumably obeys the same laws of temporal difference learning; but these laws are no longer limiting. Furthermore, instrumental contingencies often provide the most productive structure for hedonic management as well, for reasons that I discuss; but the needs of hedonic management create incentives both to pursue instrumental goals in a suboptimal manner and to avoid noticing how the hedonic incentive affects this pursuit. The result is the apparent irrationality that is often observed in process addictions.

Acknowledgements

This paper was presented at the conference ‘Agency and Addiction’ at the Center for the Study of Mind in Nature, Oslo, Norway, on November 10–11, 2011. This material is the result of work supported with resources and the use of facilities at the Department of Veterans Affairs Medical Center, Coatesville, PA, USA. The opinions expressed are not those of the Department of Veterans Affairs or of the US Government.

⁷³

Notes

¹Robins and Regier, Psychiatric Disorders in America.

²Gearhardt et al., ‘Neural Correlates’.

³Blaszczynski et al., ‘Withdrawal and Tolerance’.

⁴Marks, ‘Behavioural (Non-Chemical) Addictions’.

⁵Becker and Murphy, ‘Theory of Rational Addiction’.

⁶Heyman, Addiction.

⁷For example, Gollwitzer and Bargh, Psychology of Action.

⁸For example, Dayan et al., ‘Misbehavior of Value’.

⁹For example, Skinner, Science and Human Behavior, 76–81.

¹⁰The word ‘reward’ may denote either the internal selective factor that selects the mental process it follows for repetition, or an external event that activates this factor. Unfortunately, there are no common words that distinguish these cases. I use ‘reward’ to denote the internal factor, and ‘a reward’ or ‘rewards’ to denote the external events.

¹¹Ainslie, Picoeconomics, 28–32; Montague and Berns, ‘Neural Economics’; Shizgal and Conover, ‘Neural Computation’.

¹²Ainslie, ‘Specious Reward’, ‘Précis of Breakdown of Will’; Green et al., ‘Discounting’; Kirby, ‘Bidding on the Future’.

¹³Ainslie, ‘Core Process’.

¹⁴Ainslie, Breakdown of Will, 48–70, ‘Précis of Breakdown of Will’, ‘Core Process’. Briefly, pain, negative emotion and other vivid but aversive experiences comprise two motivational components, reward for attention and a consequent lowering of general reward level. The simplest model of this combination is a rapid cycle of brief, high reward and longer unreward, hyperbolic discounting of which can enable the brief rewards to capture choice despite the aversiveness of the pattern as a whole. This explanation obviates classical conditioning as a separate selective principle. Other models are possible, for instance the competition of Pavlovian and instrumental ‘values’ in temporal difference theory, but all must entail what amounts to a positive reward for attention. Dayan et al., ‘Misbehavior of Value’.

¹⁵Buckner, Andrew-Hama, and Schacter, ‘Brain's Default Network’; Spreng, Mar, and Kim, ‘Common Neural Basis’.

¹⁶See Rick and Loewenstein, ‘Intangibility in Intertemporal Choice’.

¹⁷Dodge, ‘Laws of Relative Fatigue’, 103; Renninger and Hidi, ‘Revisiting the Conceptualization’; Silvia, Exploring the Psychology of Interest.

¹⁸Dodge, ‘Laws of Relative Fatigue’, 103.

¹⁹Ibid., 104.

²⁰Ibid., 103.

²¹Sandson and Albert, ‘Varieties of Perseveration’.

²²Dodge, ‘Laws of Relative Fatigue’, 104.

²³Ibid., 104.

²⁴Ainslie, Breakdown of Will, 164–74, ‘Uncertainty as Wealth’.

²⁵Wilson et al., ‘Pleasures of Uncertainty’.

²⁶The discrimination of appetites from each other is also an ambiguous process. Dodge provided a start: ‘Whenever in mental processes fatigue of one is regularly accompanied by fatigue of another there must be some dynamic factor common to both. Conversely, whenever the fatigue of one mental process does not show a fatigue of another, the two must depend on different dynamic conditions’. This is to say that psychologist Richard Herrnstein's test of separate satiability to delimit concrete appetites could be applied to modalities of endogenous reward. The 8 or 10 major categories of arousable emotion, which, because they fatigue, must also have (or be) appetites, are obvious examples; but otherwise practical benchmarks that might define separately fatiguing categories seem to be almost entirely lacking. Dodge, ‘Laws of Relative Fatigue’, 92; Herrnstein, ‘Method and Theory’; Izard, Patterns of Emotion.

²⁷For example, Ryan and Deci, ‘When Rewards Compete’, 16; Silvia, ‘Interest and Interests’, 278.

²⁸Renninger, ‘Individual Interest and Development’.

²⁹Hidi and Renninger, ‘Four Phase Model’; Renninger and Hidi, ‘Revisiting the Conceptualization’.

³⁰Ackerman, ‘100 Years without Resting’.

³¹Sansone and Harackiewicz, ‘I Don't Feel Like It’.

³²Mackworth, ‘Breakdown of Vigilance’.

³³Freud, ‘Introductory Lectures’, 335–57.

³⁴Ainslie, ‘Core Process’.

³⁵Pezzulo and Rigoli, ‘Value of Foresight’.

³⁶For example, Silvia, ‘Interest and Interests’.

³⁷Prelec and Loewenstein, ‘Red and the Black’.

³⁸Silvia, ‘Interest and Interests’, 284.

³⁹Coover, Universal Baseball Association.

⁴⁰Sternbergh, ‘Thrill of Defeat’, 18–20.

⁴¹Mellers, Schwartz, and Ritov, ‘Emotion-Based Choice’.

⁴²Berlyne, Conflict, Arousal, and Curiosity, 18–44.

⁴³Turner and Silvia, ‘Must Interesting Things Be Pleasant?’

⁴⁴Suedfeld et al., ‘Reactions and Attributes of Prisoners’.

⁴⁵Bexton, Heron, and Scott, ‘Effects of Decreased Variation’.

⁴⁶Johnson, Shimizu, and Ok, ‘Actors and Actions’.

⁴⁷Silvia, ‘Interest and Interests’, 280–1.

⁴⁸Jackson, ‘Zynga Abyss’; King, Delfabbro, Griffiths, ‘Role of Structural Characteristics’. I thank Ole Rogeberg for finding the Jackson reportage.

⁴⁹Griffiths, ‘Gambling and Video Game Playing’.

⁵⁰Turkle, Alone Together.

⁵¹Lea and Webley, ‘Money as Tool’.

⁵²Ainslie and Haendel, ‘Motives of the Will’; Kirby, ‘Bidding on the Future’; Rosati et al., ‘Evolutionary Origins’.

⁵³Wittmann et al., ‘Now or Later?’; Gregorios-Pippas, Tobler, and Schultz, ‘Short-Term Temporal Discounting’.

⁵⁴Wittmann et al., ‘Now or Later?’.

⁵⁵Karp, ‘Global Warming’.

⁵⁶Ibid.

⁵⁷Navarick, ‘Negative Reinforcement’; Solnick et al., ‘Experimental Analysis of Impulsivity’; Mazur, ‘Procrastination by Pigeons’.

⁵⁸Rosati et al., ‘Evolutionary Origins’.

⁵⁹Loewenstein, ‘Anticipations’; Berns et al., ‘Neurobiological Substrates of Dread’.

⁶⁰See Sansone, ‘What's Interest Got to Do with It?’. A social analog was the Great Plains Indians' practice of ‘counting coup’, arising from their valuation of striking the first blow against enemies more highly than actually overcoming them. McGinnis, Counting Coup and Cutting Horses.

⁶¹Rick and Loewenstein, ‘Intangibility in Intertemporal Choice’.

⁶²See Ainslie, ‘Motivation Must Be Momentary’.

⁶³Ainslie, ‘Recursive Self-Prediction’, 151–5.

⁶⁴Ibid., 154.

⁶⁵See Ainslie, Breakdown of Will, 161–74, ‘Précis of Breakdown of Will’.

⁶⁶Loewenstein, ‘That Which Makes Life Worthwhile’, esp. 92–100 .

⁶⁷See Broyles, ‘Why Men Love War’.

⁶⁸After years of experience any of these activities becomes less vulnerable to a recognition that it is not instrumentally effective, because the longstanding taking of this particular risk or collecting that particular object have made the activities singular in their own right.

⁶⁹Slovic and Tversky, ‘Who Accepts Savage's Axioms?’.

⁷⁰Hsee, Yang, and Wang, ‘Idleness Aversion’.

⁷¹Ainslie, ‘Money as MacGuffin’.

⁷²March, ‘Bounded Rationality’.

⁷³Many of the references of which I was author or co-author are downloadable from http://www.picoeconomics.org.