Katie M. J. Wood, Tina Seabrooke & Chris J. Mitchell

Abstract

We report a new, simple instrumental action-slip task, which sets goal-directed action against putative S–R associations. On each training trial, participants were presented with one of two stimuli (blue or green coloured screen). One stimulus (S1) signalled that one joystick response (R1—left or right push) would earn one of two rewards (O1—jellybeans or Pringles points). A second stimulus (S2) signalled a different instrumental relationship (S2:R2-O2). On each test trial, participants were told which outcome could be earnt (O1/O2) on that trial. They were required to withhold responding until the screen changed colour to S1 or S2. On congruent test trials, the stimulus presented (e.g., S1) was associated with the same response (R1) as the outcome available on that trial (O1). On incongruent test trials, in contrast, the outcome (e.g., O1) preceded a stimulus that was associated with a different response (e.g., S2). Hence, in order to obtain the outcome (O1) on incongruent trials, participants were required to suppress any tendency they might have to make the response associated with the stimulus (R2 in response to S2). In two experiments, participants made more errors on incongruent than congruent trials. This result suggests that, on incongruent trials, the stimulus drove responding (e.g., S2 increased R2 responding) in a manner that was inconsistent with goal-directed action (e.g., R1 responding to obtain O1)—an action slip. The results are discussed in terms of popular dual-process theories of instrumental action and a single-process alternative.

In changing environments, it is essential to flexibly adapt behaviour to the present circumstances. Instrumental learning is one important way in which animals learn to adapt behaviour to obtain valued outcomes, such as food or money, and avoid aversive outcomes such as pain (O’Doherty et al., 2017).

Dual-process theories of instrumental learning propose that instrumental behaviours are controlled by two systems: a goal-directed system and stimulus-triggered habitual system (e.g., Dickinson & Balleine, 1994; Verplanken, 2018). The goal-directed system is suggested to facilitate deliberate, reward-driven actions, while the habitual system is suggested to produce comparatively automatic responses based on stimulus–response (S–R) associations. In this way, dual-process theories of instrumental learning capture a wide range of both adaptive and counterproductive reward-based behaviours. The goal-directed system is slow to respond and effortful, but highly sensitive to environmental changes. In contrast, the habitual system operates quickly and effortlessly, but is error-prone when the environment changes and old responses are no longer appropriate.

The outcome revaluation procedure is the canonical test of goal-directed versus habitual behaviour (de Wit & Dickinson, 2009; Dickinson, 1985). Animals first learn to perform instrumental responses to receive a reinforcing outcome. That outcome is then devalued, before instrumental responding for the outcome is reassessed in an extinction test. If an animal’s behaviour is sensitive to the changed value of the outcome, then they should reduce their rate of responding for the outcome. If the animal has learnt an S–R association that does not incorporate any representation of the outcome (a habit), however, then responding for the devalued outcome should not change.

Adams (1982) provided a good example of the outcome revaluation procedure. In his first experiment, rats received either moderate or extensive training to press a lever to obtain sucrose pellets. For half of the rats, the sucrose pellets were then devalued by pairing them with lithium chloride-induced sickness. For the remaining animals, the sucrose pellets remained valuable. The number of lever-press responses performed by each group was then assessed in an extinction test, where no sucrose pellets were given. The rats that received moderate training performed fewer lever press responses when the sucrose was devalued than when it was still-valued. For the extensively trained group, by contrast, sucrose devaluation had no significant effect on lever-press responses. Thus, the rats demonstrated goal-directed control after moderate instrumental training, but habitual control after extensive instrumental training.

The outcome revaluation procedure has also been used with human participants. Valentin et al.’s (2007) participants, for example, learnt to perform two instrumental actions to obtain different drinks. One of the rewards was then devalued by allowing unlimited consumption. In a subsequent extinction test, the participants responded more for the still-valued outcome than the devalued outcome, a clear demonstration of goal-directed control. This finding has since been conceptually replicated many times (e.g., de Wit et al., 2012; Liljeholm et al., 2015; Piray et al., 2016).

In contrast to the evidence for goal-directed control in humans, clear evidence for habits in humans has been hard to come by. Tricomi et al. (2009) obtained initially promising evidence; their participants learnt to press a button to obtain pictures of two foods (in the presence of different stimuli), to be consumed later. After either moderate or extensive training, one of the outcomes was devalued through selective satiation. In a subsequent extinction test, participants who received moderate training performed significantly fewer instrumental responses for the devalued outcome than the valued outcome. Extensively trained participants, by contrast, responded similarly for the still valued and devalued outcomes. Like Adams’s (1982) findings in rodents, extended training promoted habitual behaviour.

Although Tricomi et al. (2009) provided exciting evidence of habitual responding in humans, subsequent research has not been so convincing. Notably, de Wit et al. (2018) reported five failures to induce habitual behaviour through overtraining in humans, including two failed replications of Tricomi et al.’s study. As de Wit et al. acknowledged, one can always argue that more training would have produced habitual control. The main conclusion from these studies, however, is that it is by no means easy to experimentally induce habits in humans through overtraining.

Another prominent line of work has sought evidence for habits in humans by combining the outcome revaluation procedure with an instrumental task in which the same stimuli, pictures of fruit, could serve as both stimuli and the outcomes (de Wit et al., 2007; de Wit et al., 2013). Here, participants learn to perform two instrumental responses (R1 and R2) to earn different fruit outcomes, with their reaction time determining the number of points received. Each response is preceded by a discriminative stimulus, which is also a picture of a fruit. On congruent trials, the discriminative stimulus matched the outcome (e.g., S_apple: R1–O_apple; S_strawberry: R2–O_strawberry). On control trials, the discriminative stimuli and outcomes were unique (e.g., S_orange: R1–O_plum; S_mango: R2–O_pineapple). The authors’ main focus, however, was on incongruent trials, in which the discriminative stimulus was paired with an outcome that served, in a separate contingency, as the discriminative stimulus for a different response. For example, one contingency might involve S_banana: R1–O_kiwi and another contingency would be S_kiwi: R2–O_banana. Hence, kiwi and banana were paired with both R1 and R2, but played different roles (S or O) in each case. It was with the stimuli trained on these incongruent trials that de Wit et al. (2007) found evidence of habits.

On each test trial, a specific fruit was devalued; participants were told that responding for this fruit would reduce the total number of points earned. The results showed that participants were unable to appropriately avoid responding for the devalued outcomes in the incongruent condition, and so they lost points on these trials. This suggested that their (very accurate) responses during training were not based on knowledge of the fruit outcomes. In contrast, test performance was very good on congruent and control trials, suggesting that outcome knowledge was good in these conditions (see de Wit et al., 2011; de Wit et al., 2009; de Wit et al., 2007; de Wit et al., 2013; de Wit et al., 2012; Sjoerds et al., 2013). However, this conclusion has since been called into question. De Houwer et al. (2018) provided evidence that participants were in fact behaving in a goal-directed way in the incongruent condition. They showed that participants’ responses were not directed towards the fruit outcomes but towards the outcome of the points earned. A manipulation of the points outcome (rather than the fruit outcome) revealed goal-directed behaviour that was equally strong in all groups.

In summary, the outcome devaluation procedure—the canonical assay of goal-directed versus habitual control—has, in two paradigms, failed to provide compelling evidence in humans for the habitual controller that is proposed by the dual-process theory of instrumental learning. Tricomi et al.’s (2009) study failed to replicate (de Wit et al., 2018), and participants in de Wit et al.’s (2007; de Wit et al., 2013) fruit task appear, on closer inspection (De Houwer et al., 2018), to be behaving in a goal-directed fashion. Another issue is that both paradigms rely on null effects to provide evidence of habitual control. To demonstrate habits in Tricomi et al.’s (2009) overtraining paradigm, it is necessary to observe no difference in rates of responding for the valued and devalued outcomes. Similarly, in de Wit et al.’s (2007; de Wit et al., 2013) fruit task, responding must be no different from chance on incongruent trials. As De Houwer et al. (2018) noted, such null effects could reflect habitual control, but they could equally reflect other factors such as poor learning. Thus, even when evidence of habits has been reported, that evidence is subject to alternative interpretation.

The high-profile failures to experimentally induce habits in humans (e.g., de Wit et al., 2018), and the concerns around interpretations of null effects (e.g., De Houwer et al., 2018), prompted Luque et al. (2020) to develop a new assay of habitual control. They argued that looking for changes in overt responding after overtraining was impractical in experiments with human participants, because too much training would be required. Instead, they argued that looking for changes in reaction times (RTs) would be a more viable approach. In Luque et al.’s (2020) experiments, participants learned that, in the presence of each stimulus, one response would earn a high-value diamond, and another response would earn a low-value diamond. Some of the diamonds were then made worthless (devaluation). When a previously high-value diamond was devalued, the optimal response would be to switch to the alternative response which produced the previously lower-value, but now still valuable, diamond. The authors observed an RT switch cost, where choosing the alternative, optimal response came with slowed responding, particularly with extended training and when participants were placed under time pressure.

Luque et al.’s (2020) work suggests that habits can be revealed in humans, following outcome devaluation, through slowed RTs rather than response selection (Luque et al., 2020). In our current study, we were keen to pursue the idea that a habit, learned in the laboratory, could produce more than just a slowing of an optimal response, but also result in non-optimal/incorrect responding. Our approach was slightly different to that usually taken in the learning field. We did not manipulate outcome value. Rather, we used a simple interference procedure in which a putative habit, possibly S–R in nature, was set against a goal-directed action. Evidence for habitual behaviour would then be seen when responses are determined by an S–R relationship, or habit, that undermines goal-directed action. In this way, our procedure is similar to that used by Brass et al. (2001) to show how observing one finger moving can lead to participants moving that finger, rather than the “correct” finger on that trial—an effect that could be described as “automatic imitation.” Our main aim was to create a simple procedure to show habitual behaviour that was easy to implement and interpret.

Read more on the Springer Link website >