4.1  Object vs. Agent File Fixation Cues

Once we acknowledge that object vs. agent tracking can be articulated at three distinct levels, we can raise the question of whether, in the case of tracking,² different classes of properties allow fixing of reference to an item in the case of agents and in the case of objects. Let us consider an example drawn from a classic arcade game of the ’80.

The Asteroids Game. A player must drive a space vessel in order to avoid and destroy both asteroids and enemy vessels. Asteroids are characterized by passive physical movements, while enemy vessels are characterized by motor patterns revealing purposeful behavior (like avoiding asteroids, actively changing speed and direction, and shooting other vessels). The player must be able to track both asteroids and enemy vessels and react appropriately to their movements in order to destroy them, survive and win the game. 

Figure 2:  The Asteroids Game: objects vs. agents

This example illustrates a case in which a subject needs to pick out and maintain reference to two different types of individual (objectual entities, like asteroids, vs. intentional entities, like enemy vessels). We assume that in order to do this, the subject must detect two classes of cues prior to any further identification or categorization. It seems that, to establish reference, the subject does not need to access properties that might be used to identify objects and agents (e.g., asteroids and enemy targets might have the same shape). In short, we are claiming that a subject becomes able to individuate and maintain reference to entities belonging to two distinct classes (proto-objects vs. proto-agents) in virtue of his ability to detect objectual vs. agency cues, and of the specific task requirements that force him to maintain perceptual reference to individuals over time. If it can be empirically demonstrated that in similar conditions subjects display robust capabilities to differentially detect reference-grabbing properties belonging to two mutually exclusive classes as a condition for tracking items, then we might plausibly conclude that two distinct and independent individuation processes are at work. This distinction would support the claim that object tracking and agent tracking are independent at the level of reference fixation properties.

The existence of two distinct classes of reference fixing properties — although sufficient to support the independence view at this level (agency cues can be segregated from objectual cues) — is not sufficient to conclude that the two mechanisms of agent and object tracking are necessarily independent. We must also consider the relation between agents and objects at two other levels, viz. reference preservation and feature encoding.

4.2  Object vs. Agent File Preserving Properties

We have insisted on the fact that fixing perceptual reference is still not enough for tracking individual entities over time. Tracking implies preserving a referential link to a perceptual item already picked out. This raises the question of determining in virtue of which properties reference to a single individual can be maintained over time.

In the example above, in order to be able to avoid an asteroid, a subject must be able to track it as persisting over time. Following Pylyshyn, we assume that the properties used for fixing reference when the individual is picked out need not be the same as those that preserve reference. Once an item is individuated, reference fixing properties can be discarded without breaking up the referential link itself. Provided there are some file preserving properties, an item can undergo major changes without losing its singularity: we argue that if preservation conditions are met, an individual will not cease to be treated as a single perceptual item even if the properties initially used for its individuation have disappeared. Assuming that an asteroid was picked out as an individual object through its shape, it is not necessary that its shape be maintained over time in order for the object to persist in a perceptual tracking task.

Recent literature on Multiple Object Tracking has demonstrated that tracked items can survive several kinds of disruption of their features. It seems, though, that certain properties are required for an item to preserve its individuality. It is reasonable to assume, on the basis of this literature, that in our asteroids game example, although asteroids might ‘survive’ temporary occlusions which do not alter their trajectory, they would fail to maintain a perceptual link in cases of spatio-temporal incoherence, like sudden shrinking or disappearing and reappearing at a different location (Scholl and Pylyshyn, 1999). Coherence of trajectory as well as cohesiveness (Van Marle and Scholl, 2003) are hence examples of properties that seem to be required in order to maintain reference to perceptual items in tracking tasks.

We might then ask whether the properties used to keep reference alive are different in the case of agent or object tracking.

4.2.1  An Objectual Bias in Agent Files Preservation?   Many observable properties might in principle be recruited to preserve agency. For instance, the reiteration of animacy cues (e.g. an enemy vessel periodically shooting at the player) or the lack of cues of agency disruption (e.g. the lack of observable indicators of an enemy vessel being destroyed) are good candidates for the properties that contribute to the preservation of agent files. This might suggest that at the level of file preservation as well, agents and objects do not share any property (and hence that object files are independent from agent files). Nevertheless, given the fact that intentional entities are usually coinstantiated with objectual entities (‘bodies’) in our environment, it is plausible to assume that agents can be perceptually tracked via objectual preservation properties. We call this phenomenon an objectual bias in agent files preservation. This amounts to saying that:

(a) there is a class of properties that are sufficient to maintain reference to an object, once an object file is opened.

(b)  there is another class of properties that are sufficient to maintain reference to an agent, once an agent file is opened;

(c) among each of the above classes there are single properties that are more or less strongly correlated with preservation of the file: in the case of objects, there might be properties strongly or weakly correlated with object persistence;

(d)  one and the same property can be correlated with different degrees of reliability to agency or objecthood preservation;

(e) object preservation properties that are weakly correlated with preservation of agency might nonetheless be recruited for preserving agency when other agency preservation cues are absent or not available.

The fact that one and the same property P might be relevant to different degrees for maintaining reference either to an object or to an agent does not threaten per se the very possibility that there are two distinct kinds of file. We are just suggesting that it is empirically possible that, in virtue of the robustness of the agent-body correlation in our environment, nonconceptual tracking of agents might exploit objectual properties.

Which classes of properties are actually being used for tracking a proto-agent (whether they are properly described as agency-related rather than object-related) is, thus, an empirical issue that calls for experimental work. We maintain that — until a number of explicit conditions for distinguishing between these two classes are formulated — empirical results supporting the hypothesis that infants track individual entities as persistent objects, as persistent agents or as agents persisting in virtue of their objectual features might be seriously undermined. As a possible suggestion, one might test whether the fact of using objectual properties for keeping track of agents’ persistence has any consequences on the number of items an infant is able to individuate and track at the same time.

4.3  Object vs. Agent File Content

Once the conditions for fixing and preserving an item are met, we have at our disposal a device (a file) to store information attached to this individual. The way in which a file attaches properties to an entity is radically different from the way in which the nonconceptual mechanism at work while tracking an individual establishes and preserves reference to it.

On the one hand, nonconceptual tracking provides a direct link to an individual, i.e. allows a particular entity to be grabbed and segregated from other individuals or other properties of the scene: tracking is thus a necessary condition for parsing an individual as persisting.

On the other hand, having a file attached to an individual is required in order to ascribe some properties to it.

In the object file literature, a file content typically consists of properties that can be predicated of an object and used in perceptual judgment, categorization and identification. Following Scholl et al. (1999, p.2):

[This kind] of property determines the object’s appearance — what a particular object looks like — including its color, shape, lightness, and texture. We call these featural properties.

If our hypothesis on the extensibility of the object file paradigm to the case of agent tracking is valid, then we can think of the content of an agent file as a temporary structure attached to an individual entity which can store information about this intentional entity.

It is an empirical issue to understand whether and what are the particular constraints on properties that can be stored in an agent file as opposed to an object file. Yet, we submit that the extension of the file notion to the case of agency can explain how perceptual agency properties can be ascribed to a particular agent and used in perceptual judgment.

When an infant is asked to identify among a number of perceptually available agents which is the agent displaying, say, aggressive behavior, we claim that she is making use of information stored in a file to perform this task. It should be noted that since properties stored in a file are those properties that are conceptually accessible, they need not have any relevance for understanding how the nonconceptual parsing and tracking of individual entities is done.

5  Are There Really Agent Tracking Mechanisms?

The previous paragraphs were meant to outline a number of empirical issues related to the possible distinction of object vs. agent tracking at three different levels: (a) the level of reference-fixing, (b) the level of keeping reference alive and, (c) the level of ascribing properties. It might be objected that our proposal of an extension from the domain of objects to that of perceptual agents is in fact a mere redescription of the functional role of object files and thus the idea that we are able to track agents in virtue of dedicated mechanisms would be brought into question. The proposal underlying such objection can be called the deflationary view on agent tracking.

Our reply to this objection can be articulated at different levels.

•    The rationale for the existence of agent tracking mechanisms is that if we want to account not only for detection of and sensitivity to agency or animacy cues as opposed to objectual cues, but also for the ability to maintain reference to an intentional entity persisting over time, then we need to explain how this representation of persistence is achieved.

•    From the fact that there might be significant similarities in the dynamics and nature of object vs. agent tracking (what we called the dependence view), it does not follow that the latter should be reduced to the former. In the previous paragraph we made some suggestions about possible empirical ways to assess the similarity/difference between the two mechanisms.

•    Our proposal is consistent with a large literature in developmental psychology that has demonstrated the existence in children of two distinct domains of perceptual properties: the domain of animacy and the domain of objecthood. Our contribution can be considered as a framework for extending these investigations to the question of how individuals endowed with animacy and agency can be grabbed as entities persisting over time.

•    The hypothesis of the existence of agent files and agent-related tracking abilities opens up some interesting research directions leading to the study of possible conflicts and dissociations between object and agent tracking. Empirical research might shed light on the fact that:

§       intentional and objectual entities can compete for the same attentional or computational resources: it is possible that the limit on the number of items a subject can track at one time is dependent on (or independent of) the class of tracked items (objects only, agents only or objects plus agents);

§       there may be interesting cases of dissociations, i.e. selective impairments of either of these abilities without functional consequences for the other: we might imagine cases of subjects being able to track objectual entities but not agents or viceversa.

§       inattentional blindness studies might investigate whether the existence of two distinct classes of entities has any effect on their neglect: this might suggest that the traditional list of ‘styles of attention’ (object-based vs. space-based attention) should be extended to include a third kind of style (agent-based attention);

§       developmental investigations might benefit from the notion of an agent tracking mechanism to establish the conditions under which infants represent the persistence of a perceptual item over time.

The above considerations suggest that dedicated mechanisms for agent tracking are likely to have psychological reality and account for a number of capacities involved in perceptual reference to persistent entities endowed with agency. The relation of such mechanisms to those involved in perceptual reference to objects remains, however, an open empirical issue deserving further investigation. We have given arguments in favor of a moderate dependence view suggesting that in some cases objectual properties might be recruited to establish and maintain reference to agents.

6  Conclusions

A crucial step for understanding our nonconceptual abilities to refer to individual entities consists in explaining how such individuals are picked out and tracked over time, prior to any form of categorization or conceptualization. Such mechanisms of direct reference to individuals lay probably at the basis of both infants’ abilities to parse objects and adults’ capacity to keep track of multiple perceptual items. We have proposed an extension of the studies on object individuation and tracking to the domain of perceptual individuation and tracking of entities endowed with agency. This extension is intended to fill a gap between the study of perceptual sensitivity to agency cues and a full-fledged understanding of how perceptual items tracked as agents (what we called proto-agents) can persist over time. We have argued that unless some explanation of the mechanisms underlying agent persistence is provided, many empirical results concerning infants abilities to track animate entities could be dramatically undermined. Our proposal of three distinct levels that might be involved in agency fixing, preservation and ascription is meant to provide the basic requirements for any explanation of perceptual capabilities to track agents. We argue that by analyzing the dynamics of agents at these three different levels, more principled answers might be given to the question of possible interferences and biases between mechanisms dedicated to agent vs. object individuation.

Institut Jean Nicod

Paris

References

Blaser, E., Pylyshyn, Z. W., and Holcombe, A. O. (2000). Tracking an object through feature space. Nature, 408:196–199.

Bloom, P. (1996). Possible individuals in language and cognition. Current Directions in Psychological Science, 50:90–94.

Bloom, P. (2004). Descartes’ Baby. How The Science Of Child Development Explains What Makes Us Human. Basic Books.

Carey, S. and Xu, F. (2001). Infant’s knowledge of objects: beyond object files and object tracking?  Cognition, 80:179–213.

Casati, R. (2004). Common sense, philosophical and theoretical notions of an object: some methodological problems. In Barcelona ESPP-SPP Joint Conference.

Heider, F. and Simmel, M. (1944). An experimental study of apparent behavior. American Journal of Psychology, 57:243–249.

Kahneman, D. and Treisman, A. (1992). The reviewing of object files: Object-specific integration of information. Cognitive Psychology, 24:175–219.

Kuhlmeier, V., Bloom, P., and Wynn, K. (2004). Do 5-month-old infants see humans as material objects?  Cognition, 94(1):95–103.

Premack, D. (1990). The infant’s theory of self-propelled objects. Cognition, 36(1):1–16.

Pylyshyn, Z. W. (2000). Situating vision in the world. Trends in Cognitive Sciences, 4:197–207.

Pylyshyn, Z. W. (2001). Visual indexes, preconceptual objects, and situated vision. Cognition, 80:127–158.

Pylyshyn, Z. W. (2004). Notes for the oleron summer workshop on objects.

Scholl, B. J. and Pylyshyn, Z. W. (1999). Tracking multiple items through occlusion: Clues to visual objecthood. Cognitive Psychology, pages 259–290.

Scholl, B. J., Pylyshyn, Z. W., and Feldman, J. (2001). What is a visual object: Evidence from target-merging in multiple-object tracking. Cognition, 80(1-2):159–177.

Scholl, B. J., Pylyshyn, Z. W., and Franconeri, S. L. (1999). The relationship between property-encoding and object based attention: Evidence from multiple object tracking.

Scholl, B. J. and Tremoulet, P. (2000). Perceptual causality and animacy. Trends Cogn Sci, 4(8):299–309.

Spelke, E. (1994). Initial knowledge: Six suggestions. Cognition, 50:431–445.

Van Marle, K. and Scholl, B. J. (2003). Attentive tracking of objects versus substances. Psychological Science, 14(5):498–504.

Note

2 We do not intend to discuss here cases of mere sensitivity to objectual cues vs. agency cues. See the paragraph in section 3.1.1 on the distinction between detecting and tracking agency.