Introduction

In the course of the past ten years, I have been engaged in collaboration with the cognitive neuroscientist Marc Jeannerod both in setting up an interdisciplinary institute and in writing a book on vision. I have been asked by the organizers of this web seminar on interdisciplinarity to recount and reflect on this experience, as an example of interdisciplinary interaction. I do so sketchily, and will be happy, in answer to questions, to go more into details and issues that might be relevant to the general topic of the seminar.

Cognitive science — or should one say “the cognitive sciences”? — lies at the interface between the natural non-human sciences and the human social sciences. Notice that noun phrases referring to scientific disciplines or academic areas can often be used either in the singular or in the plural. So it is e.g., with “the cognitive sciences” and “the neurosciences”. In what follows, I will use the singular when some common research goal is involved and the plural to emphasize the relevant heterogeneity of the concepts and empirical methods used by the various disciplines involved.

Whereas the social sciences make liberal use of such human mental representations as intentions, perceptions, beliefs and desires in their various attempts at explaining human actions, the natural sciences are reluctant to explain the behavior of anything — from elementary particles, stars, galaxies, to molecules and cells — by attributing to them representations. To a naturalistically inclined philosopher of mind, like me, who pays proper respect to the natural sciences, cognitive science offers the prospect of providing scientific respectability to mental representations as “theoretical entities” (as philosophers of science call such things). I would not like to denigrate the social sciences. Nor am I willing to embrace a radical form of methodological dualism between the natural and the social sciences.

Ten years ago, on March 3, 1993, I took an early TGV from Paris to Lyon. I had been invited by Marc Jeannerod to attend a small interdisciplinary meeting designed to launch a discussion of how best to integrate linguistic and philosophical research into a projected Institute of Cognitive Science. In 1993, Marc Jeannerod, who was trained as a neurophysiologist and who has done seminal work in human neuropsychology and psychophysics, was the head of a research unit called “Vision and motricity”, located in Bron (on the eastern outskirts of Lyon). In what follows, I will tell the twofold story of our collaboration: first, I will describe the science policy process whose turning point was the creation of the Lyon Institute of Cognitive Science. Then, I will move to issues of scientific substance and evoke my collaboration with Marc Jeannerod in writing a book on vision.

The creation of the Lyon Institute of Cognitive Science

Whereas cognitive science programs developed in the 1970’s in the US, in the UK and elsewhere, it was not until the late 1980’s that the French scientific community came to recognize the scientific potential of cognitive science. Several eminent neurobiologists and neuroscientists such as Jean-Pierre Changeux, Michel Imbert and Alain Berthoz played a critical role in this process. In the late 1980’s the Lyon-based neuroscientist André Holley (working on olfaction) became the head of the CNRS program called “Cogniscience”. Then by the mid-1990’s, Marc Jeannerod had convinced the General Director of CNRS that CNRS should create its own Institute of Cognitive Science and furthermore that it should be located in the Lyon area.

In the Fall of 1995, I became the head of a small interdisciplinary CNRS unit (EP 100) called “A modular approach to cognitive processes: memory, language, action”, based in Lyon, involving a dozen members, and whose goal was to pave the way for the creation of the new Institute of Cognitive Science.

The group involved three neurobiologists from André Holley’s lab working on olfactory memory in rats, two cognitive neuroscientists (including Marc Jeannerod) a cognitive psychologist (working on human memory and especially on face recognition), a cognitive psychiatrist (who examines shizophrenic patients at the Vinatier Hospital), three syntacticians (working on the fundamental properties of universal grammar within a Chomskyan framework), and a naturalistically inclined philosopher of mind (myself). Marc’s sense of irony was, I think, tickled by the opportunity to do a little social experiment, i.e., to test Plato’s idea that philosophers should run labs.

From the Fall 1995 until early 1998, we held a weekly interdisciplinary seminar in one of the small rooms provided for us by the Medical School of the Lyon University Claude Bernard in the Rockefeller Center, near the subway stop Grange Blanche. During these sessions, one of us would attempt to give the others a precise sense of what he or she was doing.

One of my most memorable experiences at the time was a visit to the cognitive neuroscientist Driss Boussaoud’s lab. Driss was recording single neurons in the premotor cortex of a macaque monkey engaged in a task in which he was required to press a lever with his left hand if and only if he would see a red square on the upper right corner of the screen in front of him. The occurrence of the red square would always be preceded by the occurrence of a green circle in the lower left corner of the screen a few milliseconds earlier (that would prepare the monkey for the motor instruction). Driss was interested in disentangling the respective contributions of visual attention and motor intention to the preparation of action. For an hour, I watched the monkey who was quietly sitting with his back turned onto me. A thin electrode was coming out a metal plate covering his skull. He occasionally pressed a lever and sipped apple juice when he succeeded in the task. I paused over whether I was ready to move into moral philosophy. I decided that I was not.

In addition to our weekly interdisciplinary attempt at talking across disciplines, on Thursday afternoon once a month, we would invite two speakers on a common topic of relevance for the cognitive sciences. We selected the speakers either because they would approach a single topic from the perspective of two different disciplines (e.g., philosophy and neuropsychology or linguistics and cognitive psychology) or because they had some important theoretical or methodological disagreement. The topics included phenomenal consciousness, blindsight, implicatures in human verbal communication, numerical cognition, mirror neurons, visual imagery, the empty subject parameter in Romance languages, the cognitive study of religion, the memory of proper names, spatial cognition and autism.

The design of an Institute of Cognitive Science faces one major challenge: how to promote the collaboration between neuroscientists (i.e., biologists), computer scientists and social scientists? How to integrate the social sciences into a cognitive science context? Within the French institutional framework of CNRS, the challenge was to create a single CNRS research unit across the boundaries between at least three separate Departments: the Departments of the life sciences (SDV), of the computer sciences (STIC) and of the human and the social sciences (SHS).

Marc and I agreed on three related assumptions: (a) an institute of cognitive science should not be an institute of cognitive neuroscience. (b) Cognitive psychology is really the core of the cognitive sciences. (c) Linguists working within the tradition of generative grammar should provide a major input to the social sciences within an institute of cognitive science.

The first assumption is almost definitional, but the other two require a few justifications.
• First, in the past thirty years or so, in the study of motor cognition, perception and memory, cognitive psychology and cognitive neuroscience have increasingly come to share more experimental paradigms (e.g., brain imaging techniques).
• Secondly, as a result of the cognitive revolution of the 1960’s, cognitive psychology has been an integral part of the computational sciences as psychologists have accepted as a constraint to offer computational models of the various human cognitive processes that they study.
• Thirdly, one of the major inputs to the cognitive revolution has unquestionably been the Chomskyan revolution in linguistics. In a nutshell, in the Chomskyan framework, the study of the language faculty is the study of that fundamental piece of human knowledge, which allows a human child to learn the grammar of the language spoken by members of her linguistic community on the basis of her linguistic experience. While psychologists study memory, perceptual and motor processes, generative linguists study systems of knowledge that are distinctive of human cognition — something important to the social sciences within a cognitive scientific environment. To put it mildly, our assumption was not widely shared among French linguists and other social scientists who were in a position to affect the orientation of the projected Institute in the years 1995-1998 — either in the CNRS section devoted to the study of language or at the Head of the Department for the social and the human sciences.

Ultimately, in February 1998, roughly fifty cognitive scientists moved into a brand new building on 67, boulevard Pinel, in the neighborhood of the Psychiatric and the Neurological Hospitals in Bron. From February 1998 until the end of 2002, Marc Jeannerod has been the Director of the Institute. In January 2001, I left the Institute to become the director of the Institut Jean Nicod in Paris, a newly created interdisciplinary research unit at the interface between philosophy, the cognitive and the social sciences.

In retrospect, the Lyon Institute of Cognitive Science is, in my view, one of the best things that have happened to the French cognitive science community: it offers social scientists the unique opportunity to collaborate with experimentalists from the various areas of the cognitive sciences and to test some of their favorite social scientific theories. Two problems, I think, have arisen and will persist. On the one hand, within the French context of CNRS, it has proved difficult to hire the best scientists for the required special slots, on the basis of an international search committee, and not to depend excessively on existing CNRS research positions. On the other hand, it has turned out that the agenda of the best theoretical linguists is to keep doing theoretical linguistics, not to rush into experimental collaboration with psychologists and/or neuroscientists.

What is cognitive neuroscience?

Science policy was not our primary, let alone exclusive, topic of common concern. Soon the new cognitive neuroscience of vision became our favorite topic of discussion. In fact, in order to understand how a cognitive neuroscientist and a philosopher of mind could come to write a book on vision together, it is important to say what cognitive neuroscience is.

Cognitive neuroscience is the biological roots of the current cognitive sciences. Cognitive neuroscience, however, is only one component of the neurosciences. The goal of the neurosciences is to understand what is arguably the most complex physical object presently known in the universe: the human brain. A human brain contains in the order of one hundred billions neurons, which are involved in roughly one million billions synaptic connections. At the most elementary biological levels of organization of brain functions, molecular neurobiology tries to understand the chemical composition of neurons and the molecular structure of neurotransmitters that are involved in the communication between neurons. At a more complex level of organization, different functional areas of the human brain have been recognized and the connections between them have been mapped by means of the combined methods of neuroanatomy, histology and neurophysiology. A human brain does not arise out of an act of special creation. It results from two historical processes: the phylogenetic history of the species homo sapiens sapiens and the ontogenetic development of each human individual. Thus, the neurosciences involve the comparative study of the differences and the similarities between features of the human brain and features of the brains of members of other species — in particular non-human primates. And they involve the study of the genetic bases, the embryological and epigenetic developments of the plasticity of the human nervous system.

Current cognitive neuroscience is really motivated by the attempt to map specific cognitive activities onto specific brain areas. It in turn involves a heterogeneous ensemble of experimental techniques:
• electrophysiological recordings of either single neurons or assemblies of neurons in the course of a cognitive task in various animals either awake or anaesthetized;
• the neuropsychological study of impairments selectively produced by brain lesions in human patients;
• the psychopathological study of mental diseases in human patients without recognized brain lesions;
• the use of various techniques of brain imaging to study the activation of particular brain areas in normal human subjects during a cognitive task;
• the psychophysical study of the perceptual and/or motor responses of normal human subjects upon detection of an experimental stimulus.

Because its goal is to map selective brain areas onto cognitive activities, more than any other discipline within the neurosciences, cognitive neuroscience is likely to be of direct relevance to the social sciences. So in particular, cognitive neuroscience has brought new insights to our understanding of concepts of fundamental importance for the philosophy of mind such as perception and action. In the last twenty years, cognitive neuroscience has discovered that single neurons in particular areas of the brain of macaque monkeys fire preferentially in response to the perception of object-directed actions involving particular movements of the fingers or to the sight of a full face rather than to the same face viewed in profile. In the following, I shall concentrate on the concept seeing.

How a cognitive neuroscientist and a philosopher came to write a book together

Strong interdisciplinarity is exemplified when two or more well-established scientific disciplines merge and give rise to a new scientific discipline. Outside the cognitive sciences, molecular biology grew out of the interplay between genetics and biochemistry. Within the cognitive sciences, linguistics and cognitive psychology gave rise to experimental psycholinguistics. Marc and I were never involved in strong interdisciplinarity, since we never designed, let alone performed, a novel experiment together. Instead, we spent five years discussing the significance of many experimental papers in electrophysiology, neuropsychology and psychophysics.

The main result of our collaboration has been to clarify the “two visual systems” model first proposed by Mortimer Mishkin and Leslie Ungerleider (in the early 1980’s) and later revised by David Milner and Mel Goodale in their book, The Visual Brain in Action (Oxford University Press, 1995). On the basis of lesions in the brain of macaque monkeys, Ungerleider and Mishkin discovered that, in the primate visual system, there is a bifurcation between two anatomical pathways the ventral pathway projects the primary visual cortex onto inferotemporal areas. The dorsal pathway projects the primary visual cortex onto parietal areas. Ungerleider and Mishkin further hypothesized that the former is involved in the visual recognition of objects (the What system) and the latter is involved in localizing objects in extrapersonal space (the Where system). On the basis of neuropsychological evidence, Goodale and Milner conjectured that the ventral pathway underlies “vision-for-perception” and the dorsal pathway underlies “vision-for-action”. The basic idea of the two visual systems model of human vision is that humans can see one and the same object in two fundamentally different ways: they can build a perceptual appreciation of it and they can use visual information in order to act upon it.

Arguably, it is part of the human commonsense conception underlying what psychologists call “mindreading” that seeing leads to knowing. Human children seem able to understand early on that whether or not some object or event falls within the direction of one’s gaze makes a difference to one’s knowledge about the object or event. There is currently much discussion about whether apes share this commonsense human conception.

As a philosopher of mind with an interest in the philosophy of perception, I had assumed (a) that the goal of the human visual system is perception; (b) that human visual perception has a distinctive kind of phenomenology and (c) that the goal of perception is knowledge of the world. Although I had been exposed to the phenomenon of blindsight, I had failed to appreciate its full implications for a scientific understanding of human vision. Blindsight patients have a lesion in their primary visual cortex. As a result, visual inputs are disconnected from the rest of their visual cortex and they feel no visual phenomenal experience in their blind hemifield. However, in the 1970’s, it was discovered by a number of neuropsychologists (including Lawrence Weiskrantz, Marc Jeannerod and several collaborators from Lyon) that blindsight patients have surprising residual visuomotor capacities without visual phenomenal awareness of the stimulus.

By virtue of interacting with Marc, I came across a wide array of empirical work from the cognitive neurosciences of vision that converges on the conclusion that it is not true that the goal of the human visual system is to give rise to visually based knowledge of the world. Instead, much visual processing in humans is devoted to the guidance of object-directed actions. It dawned upon me that this evidence provided new arguments for the representational account of the visual mind. The cognitive neurosciences of vision were ready for some conceptual analysis. Philosophers had much discussed the idea that, whereas thoughts have conceptual content, perceptual representations have nonconceptual content. I wanted to argue that human hand actions directed towards objects are guided by special visual representations — visuomotor representations. The task was to analyze the differences between the nonconceptual content of visual percepts and the nonconceptual content of visuomotor representations. For example, the size, shape and location of an object are relevant for grasping it. But its color and texture are not.

One crucial area of the cognitive neurosciences turned out to be of great relevance for the assessment of the two visual systems model of human vision: it is the study of the normal unfolding of human visually guided actions of grasping objects. The dexterity of the human hand (and to a lesser extent of non-human primates) allows them uniquely to reach, grasp and manipulate objects. The action of grasping involves two components: a reaching component guided by a visual representation of the target located relative to the agent’s body and a grasping component guided by a visual representation of the size, shape and orientation of the target. Marc Jeannerod and others discovered that during reaching (i.e., transportation of the hand to the target), there is an automatic process of grip formation whereby the preshaping of the finger grip is programmed much before the hand contacts the object. At about 60% of the reaching phase, the opening of the fingers reaches its peak, which is called “maximum grip aperture” and which is linearly correlated with the size of the target. This visuomotor process can be selectively disturbed by brain lesions as in optic ataxia.

In his previous work (e.g., in his BBS 1994 paper “The Representing brain. Neural correlates of motor intentions and imagery” and in his 1997 book, The Cognitive Neuroscience of Action, Blackwell), Marc drew a functional distinction between two kinds of visual processing, which, borrowing from the study of language, he labelled respectively the “semantic” and the “pragmatic” processing of visual information. Whereas the former leads to visual identification and recognition of objects, the latter underlies visually guided actions onto objects. In the forthcoming book that Marc and I wrote together and that is entitled Ways of seeing, we try to decompose further the distinction between the semantic and the pragmatic processing of visual inputs. In the process, we argue for two particular qualifications of the two visual systems model of human vision.

First of all, we argue that the two visual systems hypothesis is restricted to the visual processing of objects that humans can reach, grasp and manipulate with their hands. But, or so we argue, human vision is not restricted to seeing objects that can be grasped between the thumb and the index finger.

Nor should human actions be restricted to grasping objects. Indeed, we argue that the “visuomotor transformation” that allows a human being to grasp an object is but a low level of human pragmatic processing of visual information. A higher-level of pragmatic processing of visual information is involved in the use and the recognition of tools, and in pantomimed actions with complex tools. After a lesion, apraxic patients may be impaired in their use and recognition of tools, and their understanding of actions with tools. But their visuomotor transformation is intact.

In addition to graspable objects, humans can also see holes, shadows, substances, gases, events and actions. They can see other humans act. Seeing a human act involves seeing an action, which is an event. It also involves seeing a human body composed of various articulated parts, including the face, which can carry much social information. Furthermore, much recent work has showed that humans have a special visual sensitivity to the perception of biological movements.

Human actions that humans can see can in turn be directed either towards inanimate or towards animate objects, including conspecifics. So in the last chapter of the book, we argue that the human brain contains two complementary systems: one is for the visual perception of object-oriented actions and the retrieval of motor intentions. The other is for the visual perception of human actions directed towards conspecifics and the retrieval of social intentions, i.e., intentions to affect a conspecific’s behavior, such as causing submission, fear or sexual desire. We submit that these two complementary perceptual systems send inputs to the human mindreading system, which in turn is a crucial component of human social cognition.

Secondly, we offer an explanation of the basic dissociation between visual perceptual processing and visuomotor processing of one and the same visual input. This dissociation is at work in the contrast between the visual behavior of respectively visual agnosic patients and optic ataxic patients. It is also exemplified by normal subjects’ responses to illusory stimuli.

There are three constitutive features of visual perception. First, to perceive objects in a visual array is to process their spatial relationships. Second, to perceive objects is to represent their relative sizes, shapes and orientations. Finally, the job of visual perception is to enable an individual to recognize objects, apply relevant concepts, and mentally classify them. So what is important to visual perception is the representation of those enduring properties of objects that allow recognition from many different points of view on many different occasions.

By contrast, the visual information relevant to reaching and grasping an object is its position relative to the agent’s physical position at a given time. For the purpose of grasping an object, what must be represented is its absolute (non-relative) size and shape. So two kinds of visual features of objects are relevant to the visuomotor processing: geometrical properties such as size, shape and orientation, and the distance of the object relative to the agent, which in turn changes during the action.

Philosophers have argued that beliefs and visual percepts have a mind-to-world direction of fit: if what a belief or a percept represents fits a fact in the world, then the belief or the percept is veridical; otherwise not. Intentions and desires, on the other hand, have a world-to-mind direction of fit: if what obtains in the world fits what the intention or the desire represents, then the intention or the desire is fulfilled; otherwise not. We argue that the function of visual percepts is to provide visual information relevant to the formation of beliefs. Following Ruth Millikan’s idea that there are what she calls “pushmi-pullyu” representations (after the Pushmi-Pullyu, an imaginary two-headed animal in Dr Doolitle’s stories), that fall in between beliefs and intentions, we argue that visuomotor representations have a hybrid direction of fit in virtue of which they provide motor intentions with visual information about affordances for action. We argue that the contrast between visual percept and visuomotor representation is corroborated by the double dissociation between the perceptual impairment of apperceptive visual agnosic patients and the visuomotor impairment of optic ataxic patients. The former cannot visually recognize the size, shape and orientation of an object that they can grasp between their thumb and index finger. The latter cannot reach and grasp objects whose size, shape and orientation they can visually recognize.

Marc and I wrote our first paper on the two visual systems hypothesis in 1998. We started writing a full book in the Summer of 2000: we decided to start by writing each selected chapters. Then each of us revised the other’s writing. Much of our initial collaboration was devoted to the analysis of dissociations between perceptual responses and visuomotor responses to illusory stimuli in normal human subjects. At the time, a new experimental paper on this topic would be published every other week or so. The question was whether the new psychophysical data were consistent with the two visual systems hypothesis, which was essentially based on neuropsychological evidence. I was a recent convert to the theory. In many conversations and email exchanges, I tried to argue that the evidence was compatible with the theory. Marc was more sceptical: he was more willing than I was to give up an overall framework in the face of prima facie recalcitrant evidence.

In 1995, I was a philosopher of mind whose questions were broadly metaphysical in character: could one naturalize intentionality? Until I spent time in Marc’s lab, it had certainly not occurred to me that by studying the calibration of the human finger grip in either a task of manual estimation of the size of an object or in a grasping task, one could get deep insight into the visual human mind. The human finger grip offers an elegant design for the experimental study of selected features of the human mind. When they are not immersed into historical questions, the best philosophers are, I think, irresistibly drawn towards “big pictures”. I do feel the impulse. But in the future I will also keep trying to ask questions about features of the human mind that can be put to an experimental test. As I see it, within a cognitive science context, the minds of philosophers of mind are bound to remain divided, if not hybrid.