Pavlovian Conditioning and Human Causal Learning

A strong undercurrent in thinking since Hume is that humans do not directly apprehend causality.  Instead, we make causal inferences based on a restricted set of experiences.  When (1) two events occur together in time and space, (2) one of the events precedes the other, and (3) the two events appear consistently together (that is, they do not occur alone), we normally infer the existence of a causal relationship between them (Hume, 1739/1964).

Human causal learning is affected by these primary Humean rules, which are the same factors that affect classical conditioning in animals:  contiguity, priority, and contingency (e.g., Fales & Wasserman, 1992; Shanks & Dickinson, 1987).  Moreover, both humans and animals exhibit behavioral phenomena such as “discounting” and “augmentation” (Kelley, 1973), which appear to implicate a sophisticated causal reasoning process; organisms not only take into account how a potential cause covaries with the effect, but also how this cause competes with rival explanations of the effect.  Interestingly, Hume himself amplified his three primary rules with three others that better pinpoint causality:  (4) the same cause always produces the same effect, and the same effect never arises but from the same cause, (5) where several different objects produce the same effect, it must be by means of some common feature, and (6) any difference in the effects of two resembling objects must proceed from that particular in which they differ.

One of the best-known cases of discounting is the cue validity effect, first reported by Wagner, Logan, Haberlandt, and Price (1968).  In their experiments, target Cue X was equally often paired with the outcome in all experimental conditions; Cue X was paired half of the time with Cue A and half of the time with Cue B.  In one condition, each AX trial was paired with the occurrence of the outcome and each BX trial was paired with the absence of the outcome; in the other condition, both AX and BX were assigned the same probability of occurrence of the outcome (0.50).  Rats’, rabbits’, and pigeons’ conditioned responding, and humans’ causal judgments to Cue X systematically decrease as the differential predictiveness of AX and BX increases—discounting (Wasserman, 1990).

Associative learning theories such as that of Rescorla and Wagner (1972) can readily explain these results.  Briefly, the Rescorla-Wagner model states that a reinforcer can sustain only a limited amount of associative strength; so, simultaneously presented cues must compete with one another as the best predictor—or cause—of the outcome.  When AX is consistently followed by the outcome and BX is not, Cue A becomes a strong predictor to the detriment of Cue X.  When both AX and BX are followed by the outcome half of the time and are not followed by the outcome half of the time, none of the cues can become a strong predictor, so Cue X can acquire moderate positive strength.

As promising as this associative account approach may be, problems have arisen when this model was applied to other types of discounting and augmentation effects, such as those involving absent cues:  so-called “retrospective revaluation” phenomena (also addressed by Martin Giurfa in his contribution to this conference), which have been observed in both Pavlovian conditioning (e.g., Kaufman & Bolles, 1981; Miller & Matute, 1996) and human causal learning studies (e.g., Dickinson & Burke, 1996; Wasserman & Berglan, 1998).  These phenomena involve the presentation of a compound of two cues, AB, that is followed by the outcome, so that each of these individual cues will become a moderate predictor or cause of the outcome.  After this training, Cue A is presented alone, either followed by the outcome or not followed by the outcome, with no further training of Cue B.   Even though Cue B is not given, its associative value changes.  When Cue A alone is paired with the outcome, subjects decrease their judgments of the causal strength of Cue B—backward blocking.  On the other hand, when Cue A alone is not paired with the outcome, subjects now increase their judgments of the causal strength of Cue B—recovery from overshadowing.

One way to deal with these challenges for associative learning theory is to dismiss the theory outright as either incorrect or inappropriate to causal understanding.  Another tactic is to reconsider some of the premises of associative learning theory and to modify it in light of the evidence.  For instance, it seems reasonable to believe that, if the presented cue were to gain strength in light of evidence, then nonpresented cues might immediately and correspondingly lose strength; conversely, if the presented cue were to lose strength in light of evidence, then nonpresented cues might immediately and correspondingly gain strength.  Van Hamme and Wasserman (1994) suggested that the Rescorla-Wagner (1972) model could be modified in such a way that different learning rate parameters are assigned to presented and nonpresented cues:  positive and negative learning parameters, respectively.  This theoretical maneuver allows the Rescorla-Wagner model to embrace results that at first had appeared to be so discomforting.

Associative learning theory has accordingly been modified and enriched by the similarities between human causal learning and animal conditioning.  We propose that the existence of such parallels speaks to a common underlying process.  If one assumes that, during a conditioning procedure, animals acquire information about the causal texture of their environment, then the correspondence between animal conditioning and human causal learning can be readily accepted.  However, some deem these parallels to be inadequate to prove causal understanding in animals, because these studies concern merely “making predictions about the temporal and spatial relations between observable events” (see, for example, discussion on Jennifer Vonk’s paper in this conference).  We disagree on this point, precisely because—as the studies mentioned above show—causal understanding even in humans seems to be based on the observation of temporal and spatial regularities in the environment.

As well, we are concerned with drawing a strong theoretical distinction between making predictions and making causal inferences.  We would suggest that the best predictor of an event is also the cause of that event.  It is unlikely that environmental contingencies are organized in such a way that a non-cause would be a reliable predictor of an event.  It would be peculiar if evolution were to have endowed organisms with the ability, not to detect causal relationships, but to detect predictive relationships, when the former ought to be more directly relevant to survival than the later.

What evolution may have done is to prepare organisms to preferentially forge some associative connections, thereby increasing the speed with which certain experiential contingencies promote learning.  Garcia and Koelling’s (1966) classical experiments showed that learning depends on the relevance of the potential cause to the potential effect.  Rats readily associated a taste with later illness, but with much greater difficulty they associated audiovisual cues with the same illness.  In the opposite fashion, rats readily associated audiovisual cues with shock, but they found it difficult to associate taste with shock.  Hence, learning best proceeded when the potential cause was combined with a relevant effect.  Similar results have been found in other species, such as pigeons (Shapiro, Jacobs, & LoLordo, 1980).  It might be that, when a plausible causal link can be inferred—even when the underlying connection is not truly causal—organisms are better prepared to associate stimuli that are presented together.

Instrumental Learning and Causation

Causal knowledge allows us not only to predict, but also to control our environment.  We are able to predict an effect on the basis of observed cues, but we are also able to predict the effects that our own actions will have on the environment.  If animals understand that there is a causal relationship between events, then one might argue that, when the effect is highly valuable, the animal should work to make the cause occur.  Instrumental conditioning relies on the ability of organisms to learn that their own actions can produce certain outcomes.  Humans’ and animals’ manipulation and control of their environment may be based on the inference of a causal relationship between their own behavior and the consequences of this behavior.

Man’s first experience with causes probably came from his own behavior:  things moved because he moved them (Skinner, 1971, p.7).

It is not difficult to imagine that all mobile organisms go through this very basic experience.  Hence, it is reasonable to ask:  Are nonhuman animals also able to distinguish between events that are caused by their own behavior from those that are not?  Killeen (1981) “asked” his pigeons whether or not they were responsible for key light offset.  The pigeons were able to discriminate whether it was their own behavior or “something else” that caused the change in the light.  The rudiments of causal understanding can easily be note here.

Arguably more compelling evidence of causal understanding in instrumental conditioning comes from studies of outcome devalution.  Adams and Dickinson (1981) trained rats to press a lever to get food pellets.  Later, an aversion to the food was induced by injecting the animals with a mild toxin that produced gastric illness.  During this aversive conditioning, the lever was not present.  The relevant issue here was to what extent this devaluation would affect lever pressing when the lever was again available.  If the animals had learned that there was a positive causal relation between lever pressing and the receipt of food pellets, then lever pressing should be influenced by this causal knowledge and the current desirability of the outcome.  Because the food pellets were no longer appetizing, the animals decreased their pressing of the lever.

Current challenges for associative theories

Several more recent findings pose new challenges for the adequacy of associative learning theory to explain causal understanding.  Let’s examine a recent study about the difference between observation and intervention in nonhuman animals.

Even when the same between-event contingencies are arranged, people make different causal inferences depending on whether they merely observe the occurrence of an effect or they know that someone or something else has intervened to produce that effect (e.g., Waldmann & Hagmayer, 2005).  Blaisdell, Kosuke, and Waldmann (2005) were interested in whether or not rats could also exhibit a similar tendency.  These researchers presented rats with a light followed by a tone and the same light followed by sucrose.  The light should be a potential cause and the tone and the sucrose should be potential effects.  Would the animals consider the light as the cause of both the tone and the sucrose?

To answer this question, after the above training, a lever was inserted into the chamber.  In the Intervention group, the tone was presented each time the rats pressed the lever, whereas in the Observation group, the tone’s presentation was not contingent on lever pressing, although the tone was presented the same number of times in each group.  Therefore, one of the effects of the light, the tone, was only “observed” in the Observation group, whereas it was “intervened” in the Intervention group.  If rats were to consider the light to be a common cause of the tone and the sucrose, then the presentation of the tone in the Observation group should lead the animals to infer that the light must have occurred and to expect sucrose as well.  On the other hand, if rats in the Intervention group attributed the tone’s presentation to their own lever pressing behavior, then they should not attribute the tone to the presence of the light, because it had been caused by their own behavior.  Thus, rats in the Intervention group should not infer that the light had occurred as well, so that they should not expect any other effects of the light—specifically, the sucrose—to be present.  This is exactly what Blaisdell et al. (2005) observed:  when the tone appeared, rats in the Intervention group did not look for sucrose, whereas rats in the Observation group did.

Thus, it seems that nonhumans’ behavior can evidence complex causal reasoning not based on the mere extraction of contingency information about given events.  Here, rats may show that they are not only capable of forward learning, from cause to effect, but that they are also able to observe effects and to diagnose whether the cause should have occurred or not.  Indeed, in the Blaisdell et al. (2005) study, we might be seeing hints of diagnostic abilities that are embedded in the notion of causal explanation (in the terms considered by Anne Reboul in this conference).  The rats’ behavior suggests that these diagnostic capacities may not be uniquely human; indeed, these abilities may emerge from bidirectional associations that have been the focus of learning theorists for over 100 years.  Numerous studies show that, during the course of learning, humans and animals do not acquire just forward associations, but also backward—or bidirectional—associations between paired events (Arcediano, Escobar, & Miller, 2005; Asch & Ebenholtz, 1962; Frank & Wasserman, 2005).  These bidirectional associations might help to explain the diagnostic abilities that both human and animals exhibit.

Conclusions

Parallels between Pavlovian conditioning and human causal judgment, research on instrumental conditioning, and recent work on the distinction between observed and intervened effects, all suggest that causal knowledge lies at the root of both human and animal behavior.  We do not deny that humans’ causal understanding is far more advanced than animals’; but, that advancement is likely to be premised on the basic rules of causal association that were proposed centuries ago by David Hume.  Whether that advancement is simply a further elaboration of these rudimentary rules or something qualitatively different is a live empirical question.

References

Adams, C. D., & Dickinson, A.  (1981).  Instrumental responding following reinforcer devaluation.  Quarterly Journal of Experimental Psychology, 33B, 109–122.

Arcediano, F., Escobar, M., & Miller, R. R. (2005).  Bidirectional associations in humans and rats.  Journal of Experimental Psychology: Animal Behavior Processes, 31,301-318.

Asch, S. E., & Ebenholtz, S. M. (1962). The principle of associative symmetry. Proceedings of the American Philosophical Society, 106, 135-163.

Blaisdell, A. P., Kosuke, S., & Waldmann, M.  (2005).  Seeing versus doing:  Two modes of assessing causal models by rats.  Proceedings of the 12th Annual International Conference On Comparative Cognition

Dickinson, A., & Burke, J.  (1996).  Within-compound associations mediate the retrospective revaluation of causality judgements.  Quarterly Journal of Experimental Psychology, 49B, 60-80.

Fales, E., & Wasserman, E. A.  (1992).  Causal knowledge:  What can psychology teach philosophers?  Journal of Mind and Behavior, 13, 1-27.

Frank, A. J., & Wasserman, E. A.  (2005).  Associative symmetry in the pigeon after successive matching-to-sample training. Journal of the experimental Analysis of Behavior, 84, 147-165.

Garcia, J., & Koelling, R. A.  (1966).  Relation of cue to consequence in avoidance learning.  Psychonomic Science, 4, 123-124.

Hume, D. (1964). Treatise of human nature (edited by L. A. Selby-Bigge). London: Oxford University Press. (Original work published 1739)

Kaufman, M. A., & Bolles, R. C.  (1981).  A nonassociative aspect of overshadowing.  Bulletin of the Psychonomic Society, 18, 318-320.

Kelley, H. H.  (1973).  The processes of causal attribution.  American Psychologist, 28, 107-128.

Killeen, P. R.  (1981).  Learning as causal inference.  In M. L. Commons & J. A. Nevins (Eds.), Quantitative analyses of behavior (Vol.1):  Discriminative properties of reinforcement schedules (pp. 89-112).  Cambridge, MA:  Ballinger.

Miller, R. R., & Matute, H.  (1996).  Biological significance in forward and backward blocking:  Resolution of a discrepancy between animal conditioning and human causal judgment.  Journal of Experimental Psychology:  General, 125, 370-386.

Rescorla, R. A., & Wagner, A. R.  (1972).  A theory of Pavlovian conditioning:  Variations in the effectiveness of reinforcement and nonreinforcement.  In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II:  Current research and theory (pp. 64-99).  New York:  Appleton-Century-Crofts.

Shanks, D. R., & Dickinson, A.  (1987).  Associative accounts of causality judgment.  In G. H. Bower (Ed.), The psychology of learning and motivation, (Vol. 21, pp. 229-261).  San Diego, CA:  Academic Press.

Shapiro, K. L., Jacobs, W. J., & LoLordo, V. M.  (1980).  Stimulus-reinforcer

interactions in Pavlovian conditioning of pigeons:  Implications for selective associations. Animal Learning and Behavior, 8, 586-594.

Skinner, B. F.  (1971).  Beyond Freedom and Dignity.  New York:  Knopf.

Van Hamme, L. J., & Wasserman, E. A.  (1994).  Cue competition in causality judgments:  The role of nonpresentation of compound stimulus elements.  Learning & Motivation, 25, 127-151.

Wagner, A. R., Logan, F. A., Haberlandt, K., & Price, T.  (1968).  Stimulus selection in animal discrimination learning.  Journal of Experimental Psychology, 76, 171-180.

Waldmann, M. R., & Hagmayer, Y.  (2005).  Seeing versus doing:  Two models of accessing causal knowledge.  Journal of Experimental Psychology:  Learning, Memory, and Cognition, 31, 216-227.

Wasserman, E. A. (1990).  Attribution of causality to common and distinctive elements of compound stimuli.  Psychological Science, 1, 298-302.

Wasserman, E. A., & Berglan, L. R.  (1998).  Backward blocking and recovery from overshadowing in human causal judgment:  The role of within-compound associations.  Quarterly Journal of Experimental Psychology, 51B, 121-138.