| |
1 The evolution of linguistic representations as a cultural Darwinian process
Even since the elaboration of the theory of natural selection by Charles Darwin, researchers have proposed that the mechanisms of language evolution may have strong similarities with the mechanisms of biological evolution (Schleicher, 1863). More recently, several kinds of analogies have been proposed. A first kind, tries to map units and structures in the genetic space directly to units and structures in the linguistic space (Berlinski, 1972; Searls, 2002; Stegmann, 2004). A second kind of parallel was developed in which the focus was on the Darwinian process of evolution rather that on the units themselves (Mufwene, 2005; Croft, 2000, 2002; Steels, 2004). The common process between genome and language evolution is here the following: 1) there exists a population of units capable of replication, 2) replication is not perfect: modifications can appear, 3) the units have different levels of efficiency in replication, which produces differential replication. This high level formulation, sometimes conceptualized as a generalization of Darwin’s theory of natural selection (Hull, 1988), has the advantage of not specifying the structure of units as well as the mechanisms of replication and variation. And indeed, researchers found ways to instantiate it into biological or language evolution by filling in those missing slots with the corresponding specific structures and mechanisms (Croft, 2000). As far as biology is concerned, the units are genes, the mechanisms of replication are those associated with meiosis/mitosis, and the mechanisms of variation are mutations and cross-over. As far as language is concerned, a wide variety of instantiations have been proposed. The units of replication were conceived as ideas, mnemotypes, idene, culturetype, socio-genes, tuition (van Driem, 2003), ranging from simple abstract concepts like words or expressions to complex neural structures implementing associations between phonological forms and meaning. Perhaps the most well-known notion of cultural unit of replication is the meme introduced by (Dawkins, 1976). Linguistic memes, sometimes called linguemes (Croft, 2000), are themselves a population of very diverse kinds of units: phonological features, phonemes, syllables, rules of phoneme sequencing, lexicons, rules of syntax, semantic categories, systems of world categorization, constructions mapping combinations of words and complex meanings, prosodic structures, social conventions involving gestures and gaze to coordinate linguistic interactions, etc. Dawkins gives imitation as an example of mechanism of replication for units for language evolution. As a matter of fact, all kinds of linguistic activities, which can be much more complex than just imitation, like conversation or reading, provoke the replication of linguistic units. The consequence is that “leaping from brain to brain” is a very complex process that can happen through a variety of mechanisms. What provokes variation is therefore also very diverse: bad perception, erroneous interpretation, exaggeration, etc.
This shows that the conceptualization of language evolution as a Darwinian process may take quite different forms for different authors and is often presented only at a rather general level, especially in the memetics litterature. Yet, in order to be useful, we argue in this paper that this conceptualization must be precise, detailed and operational. Indeed, if language evolution is a Darwinian process, then it means that many of its features are systemic: they are the outcome of the complex interactions between replicators, replicating mechanisms and various kinds of constraints (e.g. learning biases, function or environment). Depending of each particular mechanism and of the particular ecological constraints, very different cultural dynamics might happen, and the adaptation of linguistic representation may (or may not) happen in many different manners. To make this point, we will review a number of computational models of the origins and evolution of language, showing the variety of replicating units and replication mechanisms that one can encounter at various levels, and showing what consequence they have on actual language evolution.
The common point of all the computational experiments we present is that they consist of populations of agents initially devoid of linguistic convention, and that will progressively and culturally build each time a new (simple) linguistic system. A first series of examples will focus on the functional constraints imposed on linguistic replicators due to their use as communication systems. In particular we will see that only very specific mechanisms of replication may allow for the efficient formation of shared linguistic conventions. Then, we will present an experiment showing how linguistic replicators can adapt and evolve under the specific constraints due to the external environment. We will then review an experiment studying the role of learning biases in the replication process and see how it can influence the evolution of linguistic representations.
2 Functional constraints on Darwinian dynamics
Linguistic replicators have specific properties compared to biological replicators: they form a system that permits communication. For instance, in a vocabulary in which words are associated to concepts/meanings and are used to draw the attention of several speakers towards a particular referent in a given context, synonymy and homonymy tend to be reduced ensuring efficient communication. We will see that not every differential replication process permits the emergence of such communication systems. Typically, each linguistic interaction involves the semiotic triangle: there is a form (e.g. a word), an associated meaning, and an associated referent in a particular context. This entails that three kinds of entities can be replicated through communication: forms, meanings, and associations within certain forms and certain meanings. As a matter of act, each of these kinds of entities consists itself of a variety of entities which are also replicators. For example, words are composed of sounds like vowels and consonants, which can be grouped into syllables through sets of phonotactic rules, which can themselves be sequences according to certain rules to build up words, and all these hierarchically entities can replicate differentially. Although all these replicators are constantly interacting, the experiments we will now present make a number of simplifications that allow us to develop a better understanding of the fundamental dynamics associated to various functional, internal and environmental constraints. For example, in the first experiment we will describe, we will suppose that there is only one meaning in the world that agents inhabit, and that two possible words can be associated to this meaning. This experiment will show some basic properties of the replication mechanism so that a simple convention can be adopted by a population, i.e. so that linguistic coherence can be reached (speakers associate the same word to the same meaning). This experiment will then be made more complex in following sections, allowing to study progressively more complex phenomena like linguistic distinctiveness (speakers associate different words to different meanings, next section).
2.1 Linguistic coherence and replication mechanisms: simple epidemiologic models are not enough
Let us consider a simple problem: N agents have to choose between two conventional names c1 and c2. We will consider three simple models representative for many more complex ones studied in the field. The first model is an imitation-based model (model A) and the two others are frequency-based (model B and C). In model A, the speaker simply produces the conventional name he heard last as a listener. In model B, the speaker produces the name that he has heard most frequently as a listener. In model C, the speaker produces any name that he has heard as a listener with a probability proportional to the frequency. These three types of replication processes could be seen as possible models of how cultural replication occurs. Yet, results show that the entailed dynamics are quite different (for details, see Kaplan (2005)). With imitation-based model A, the population eventually converges to a state of complete co-ordination. However, convergence happens typically only after a very long series of oscillations. With model B, convergence towards a single conventional name also occurs. However, the oscillations observed are much smaller. As soon as a convention spreads more in the population than the other, its domination seems to amplify even more over time and convergence happens quickly. For model C, on the contrary, dynamics tend to maintain the distribution of c1 and c2 over time, after an initial drift, and so there is no convergence. An in-depth study of these three models reveals that despite their apparent similarity the types of dynamics they create are extremely different. Among the three models studied, only model B creates self-reinforcing dynamics that permit a fast coordination of the entire population towards the use of a single conventional name. Model A is approximatively similar to a random walk, converging in quadratic time, and thus is impractical in real life. Finally, the dynamics of model C tends to maintain the distribution of the convention at a fixed non-convergent level.
What we must remember from these results is that not all cultural transmission systems create a differential replication process that ensures the domination of some linguemes over others in a reasonable time span. Simple models of cultural transmission solely based on imitation are not sufficient to permit linguistic coordination. In that sense the dynamics of linguistic replication are likely to be different from the ones characterizing epidemiological processes, which have sometimes been presented as a possible metaphor for cultural transmission (Sperber, 1984).
2.2 Linguistic distinctiveness and the adaptation of form-meaning pairs
For efficient communication, it is better that different words are associated with different meanings and vice versa. This obvious remark actually constrains systems of linguistic replicators in many important ways. Indeed, the replication process must not only address the issue on linguistic coherence but also permit linguistic distinctiveness. Let us consider a model in which individuals have to establish conventionalized associations between several words and several meanings. Each agent is now equipped with an associative memory, which is a list of word-meaning pairs with a numeric score. It is used to find the best word associated to a given meaning and reversely to find the best meaning associated to a given word. As in the model B of the previous section, the agents choose the association with the highest score when several solutions are possible. The associative memories of the agents are initially empty. Associations are progressively created as the agent interacts with other agents.
Studies of such systems were initiated by Steels in the mid 1990s (Steels, 1996). Several other experiments rapidly showed how collective dynamics could permit that each name eventually becomes associated with a single context and each context with a single convention (Hutchins and Hazlehurst, 1995; Oliphant, 1997; Arita and Koyama, 1998; De Jong and Steels, 2003; Vogt, 2005; Baronchelli et al., 2006; Kaplan, 2001). Some of the most interesting dynamics of such self-organizing lexicons are obtained in the presence of noise. Let us consider that each word/replicator ci is modelled with an integer value between 0 and 1000. Each time a word/replicator is transmitted, a random number between −B/2 and +B/2 is added to its value. Thus, B is a measure of the global noise level. Each agent is equipped with a filter permitting to select all the words/replicators in its associative memory of which the values are at a distance D less than D = B. The structure of an interaction is the following: Agent 1 randomly chooses a meaning s1 among the different meanings available and uses a word c1 to express this meaning. If it does not have words associated with this meaning, the agent creates a new one (a random integer between 0 and 1000). Then, c1 is transmitted to agent 2 with an alternation between −B/2 and +B/2. Then, agent 2 selects all the possible associations with a word close to the integer received (at a distance less than B). If several associations are possible, agent 2 chooses the one with the highest score: (c2, s2). If s1 = s2 the interaction is a success, in the other cases the interaction is a failure. If no association is close enough in agent 2’s memory, the agent creates a new association between the received integer and the meaning s1. In case of success, agent 2 increases the score of the association (c2, s2) with +_ and diminishes the score of competing associations ((c2, _) and (_, s2)) where * is any meaning or word in the memory of the agent) with −_. In case of failure, agent 2 decreases the score of (c2, s2) with −_ (see (Kaplan, 2001; De Jong and Steels, 2003; Vogt, 2005) for discussions of the importance of such forms of lateral inhibition). Associations are initially created with a 0 score.
Figure 1: Evolution in the word space (idealized acoustic space) of the words associated with 5 meanings in an experiment involving 10 agents with a noise level B = 100. After a first period of ambiguity, five well separated bands appear associated with each meaning. Evolution of the ’average’ values (in the population) of the words associated with each meaning is highlighted in the middle of each band.
Can collective dynamics lead to choose the “best” word/replicators? A good word/replicator is a replicator that an agent will not confuse with another one that has a different usage. A “good” lexical system should have sets of words clearly distinct from one another depending on the meanings they associated to. Results show that indeed, distinctive sets of word-meaning associations are progressively formed and selected through cultural interaction. Figure 1 shows the evolution in the word space (i.e. an idealized acoustic space) of the words associated with 5 meanings in an experiment involving 10 agents with a noise level B = 100. After an initial ambiguity period, five well separated bands in the word space are clearly identifiable. Agents do not converge towards a unique value for each context. Each agent uses a different one. But these values tend to be very close. The “band” for one context is clearly distinct from bands associated with other contexts. No confusion is possible. Figure 1 plots also the ’average’ value of each band. Thus, it is easier to see the collective optimization of distinctivity leading to a solution compatible with the level of noise present in the environment. We also see on this graph that with this level of noise we approach the limit of expressiveness possible in this medium. If the agents had to communicate about a larger number of distinct meanings, ambiguity will inevitably arise.
2.3 Neutral drift
External factors like language contact between populations are often cited as a major cause of language evolution. But it is also known that language can change spontaneously based on internal dynamics (Labov, 1994). We have seen with the previous model that in a noisy environment, agents can converge on a stable system in which distinct bands in the word/acoustic space are associated with distinct contexts. As we see in figure 1, this repartition in separated bands does not evolve anymore once a stable solution has been found. Figure 2 shows the evolution of the average word in the presence of an agent flux defined by a probability of replacing an old agent by a new one Pr = 0.01, for a population of 20 agents and 2 contexts. The centre of the bands is spontaneously evolving as new agents are entering the system. This is an example of a neutral drift.
Figure 2: Example of a neutral drift: Spontaneous evolution of the ’average’ forms in presence of an agent flux.
This effect is easily understandable. A new agent tends to converge on words belonging to the existing bands for each meaning to express. But within this band, it does not converge towards the exact centre of the band. Thus the centre is moving as the flux of new agents enters the system. The higher the agent tolerance on noise, the higher the amplitude of this drift (see (Steels and Kaplan, 1998) for a first description of this phenomenon).
In this experiment, words/replicators are drifting spontaneously without any functional drive. However external pressures can direct these dynamics in one direction or another. This neutral drift provides novelty and thus can lead to a more efficient reorganization if needed. In some way, this is effect is similar to role of neutral mutation in evolution (Kimura, 1983).
Experiments on computational models of phonological systems have shown how similar collective dynamics in the presence of noise lead a population of agents to converge towards a set of vowels optimally distributed in the phonological space in order to favour distinctiveness between them (de Boer, 1997; De Boer, 1999; Oudeyer, 2005b). Such emerging phonetic systems have high similarity with real ones as observed in natural languages.
To summarize, noise during word transmission favours sets of words that are clearly distinct from one another when they are associated with different meanings. Finally, in the presence of noise and agent flux, we experimentally observe a spontaneous non functional evolution. This continuous exploration can lead to a more efficient reorganization of the replicator system if needed.
3 Environmental constraints and Darwinian dynamics
Until now, we have only considered simple models of linguistic replication. Linguistic phenomena are obviously more complex. The previous experiments have focused on the replication of words, but their associated meanings and semantic categories are also entities that can replicate from brain to brain through linguistic interactions. In many computational models, these categories are modelled as points in category space. But more complex systems of meanings, also referred as categories or concepts, were also investigated (Steels and Belpaeme, 2005). The Talking Heads experiment conducted between 1999 and 2000 by Steels and co-workers has provided a large set of data on how systems of categories can adapt to particular environments (Steels and Kaplan, 2002; Kaplan, 2001). In this experiment robotic agents were capable of segmenting the image perceived through the camera into objects and of collecting various sensory data about each object, such as the colour, position, size, shape, etc. A couple of robots were placed in front of a white board on which various types of objects were placed. At each interaction, the speaker chose one object from this context, reused or constructed a category that would identify this object from the other object present in the background and uttered a word associated with that category. Based on this word, the other robot had to guess which object was named (Figure 3).
Figure 3: The Talking Heads set up. Two robotic cameras are placed in front of white board. On the board, objects of various shapes and colours are placed. The robots have to construct categories and words to name the objects on the board and have the other agent guess the right object based on that word. Categories referring to colour, position, size or shape can be used.
In the first run of the experiment, a total of 8000 words and 500 concepts were created, with a core vocabulary consisting of 10 basic words expressing concepts like up, down, left, right, green, red, large, small, etc. The dynamics that pushed the population towards coherence and distinctivity ensured the collective choice of a set of word-category associations adapted to the environment that the robots were perceiving. Interestingly, and in spite of a set of possible perceptual features with no internal cognitive bias for some over others, some features like shape were used very rarely whereas position, colour and size categories were preferred. This can be explained by the properties of the environment in which the language games took place. Indeed, it happened that the kind of objects places on the white boards were very similar in shape, but much more distinctive in terms of colors and positions.
Such types of indirect competitions between perceptual categories were observed during the whole experiment. Some categories were general and other specific (e.g. one was used to describe a particular shade of green, and another one to describe green contexts in general). Usually, general categories were preferred because they were both easier to learn by the agent and adapted to a larger number of contexts (see also Smith (2005) for another series of experiments in this line). However, in several cases, a precise category adapted to reoccurring specific context survived as other categories were present to “back it up”. Therefore, when analyzing these types of complex dynamics, considering competition between isolated categories is not always sufficient. The quality of a category needs to be evaluated regarding the category set to which it belongs and the adaptivity of the whole to particular environments.
Another interesting phenomenon was observed. Most of the words of the core vocabulary were coherently interpreted as having distinct meanings. However, in some cases, two competing meanings co-occurred for a long time. For instance the word ’bozopite’ was associated concurrently with two types of categories: large area (large) and large width (wide). This co-occurrence was due to the fact that in the types of environments that the robotic agents encountered most objects that were large in area were also large in width. This is an example of residual polysemy.
This brings us to a remark. As collective dynamics select sets of replicators that are well adapted to the environment in which the agents are communicating, we might be tempted to say that the “quality” of the replicators increases. But, like for species in natural evolution, optimization stops once adaptation is reached. We have seen in the previous section that in the presence of noise, well separated bands of replicators were emerging. However, once a stable solution was found, this optimization of distinctivity stops. The same effect occurs in more complex architectures where residual polysemy is observed. In all these situations, there is no absolute optimization, only the search for stable adapted solutions.
4 Cognitive constraints on Darwinian dynamics
When linguistic replicators leap from brain to brain, they in fact do so through perpetual cycles of encoding, production, perception, decoding and learning. Whatever these replicators are, they need to be incorporated into internal representations in the brain of speakers and hearers at some point. The process of updating one’s brain to incorporate some new information defines learning. Learning theory, and in particular machine learning theory (Mitchell and Weinmann, 1997), has shown that all learning systems are characterized by a number of biases which mean that every single system will be good at learning certain things and bad at learning other things. For example, learning algorithms such as recurrent neural networks are good to learn to predict complex time series but they are quite inefficient to learn fine categorical distinctions in high-dimensional static spaces, whereas support vector machines are good in high-dimensional static spaces but pretty bad when they have to learn time-dependent phenomena (Duda et al., 2001). Learning biases also apply to human brains. For example, when the human brain learns a new concept or a new sound, it will do so typically by using the representation of an already known concept or sound and modify it a little bit. The consequence is that learning a new concept or a new sound will only be effective if the corresponding brain already knows not too dissimilar concepts or sounds. This imposes strong constraints on the replication of linguistic memes, which are not only defined by the generic cognitive constraints of all human brains, but also by the particular cognitive structures that were built during the ontogeny of each of them. This means that for a given brain, some linguistic memes will be easily learnt and replicated, but some other linguistic memes will be strongly deformed often to the point that no replication at all takes place. And the linguistic memes which are easy to learn and replicate for a given brain may prove to be difficult for another brain which had a different history.
What is then the consequence of all this on the dynamics of language evolution? We will now present the outline of a computational model of the origins of syllable systems which sketches the outline of an answer (this model is described in detail in another article (Oudeyer, 2005a)). This model involves a population of agents which can produce, hear, and learn syllables, based on an auditory and a motor apparatus that are linked by abstract neural structures. These abstract neural structures are implemented as a set of prototypes or templates, each of them being an association between a motor program that has been tried through babbling and the corresponding acoustic trajectory. Thus, agents store in their memory only acoustic trajectories that they have already managed to produce themselves. The set of these prototypes is initially empty for all agents, and grows progressively through babbling. The babblings of each agent can be heard by nearby agents, and this influences their own babbling. Indeed, when an agent hears an acoustic trajectory, this activates the closest prototype in its memory and triggers some specific motor exploration of small variation of the associated motor program. This means that if an agent hears a syllable S that it does not already know, two cases are possible: 1) he already knows a quite similar syllable and has a great chance to stumble upon the motor program for S when exploring small variations of the known syllable, 2) he does not already know a similar syllable and so there is little chance that he incorporates in its memory a prototype corresponding to S. This process means that if several babbling agents are put together, some islands of prototypes, i.e. networks of very similar syllables, will form in their memory and they will develop a shared skill corresponding to the perception and production of the syllables in these networks. Nevertheless, the space of possible syllables was large in these experiments, and so the first thing that was studied was whether agents in the same simulation could develop a large and shared repertoire of syllables. This was shown to be the case (Oudeyer, 2005a). Interestingly, if one runs two simulations, the population of agents will always end up with their own particular repertoire of syllables.
Then, a second experiment was run: some fresh agents were tested for learning syllable systems that were formed by another population of interacting agents, and some other fresh agents were tested for learning a syllable system which was generated artificially as a list of random syllables. The results, illustrated in figure 4, were that the fresh agents were always good at learning the syllable systems developed by other similar agents, but on the contrary rather bad at learning the random syllable systems. In other terms, the syllable systems developed culturally by agents were adapted to their cognitive biases, and the random systems were not. Thus, the replicators constituted by syllables evolved and were selected in a cultural Darwinian process so as to fit to the pre-existing ecological niche defined by the cognitive structures of agents, fitness being here learnability. What is particularly interesting is to note that the pre-existing learning/cognitive biases are here not language specific: the same architecture could be used to learn hand-eye coordination for example. The linguistic structures that adapt to these biases seem to be idiosyncratic from an external point of view, and might be thought to rely on language specific cognitive modules, but this experiment shows that this is not necessarily the case.
Figure 4: Evolution of the rate of successful imitations for a child agent which learns a syllable system established by a population of agents (top curve), and for a child agent which learns a syllable system established randomly by the experimenter (bottom curve). The child agent can only perfectly learn the vocalization systems which evolved in a population of agents. Such vocalization systems were selected for learnability (Reprinted from (Oudeyer, 2005a))
Several other computational systems have been developed to study the mechanisms that allow the cultural selection for learnability of linguistic replicators. Zuidema presented abstract simulations of the formation of syntactic structures and detailed the influence of cognitive constraints upon the generated syntax (Zuidema, 2003). Brighton et al. presented a thorough study of several simulations of the origins of syntax (Kirby, 2001) which were re-described in the light of this paradigm of cultural selection for learnability (Brighton et al., 2005).
5 Conclusion
We have shown in this paper that conceptualizing language evolution in analogy to biological evolution, which has been proposed in the field of memetics in particular, requires to go down to the details of the replication mechanisms as well as the precise definition of replicating units and the ecological constraints in which their evolution takes place. Otherwise, there is a risk that this analogy may lead to misconceptions (such as for example the idea that simple epidemiologic models might be a good model of the formation of a consensus on the use of a (new) linguistic representation). However, we also think that the different examples we presented illustrate the fact that when the Darwinian concepts are applied operationally to the evolution of linguistic representations, they can provide important new insights. This conceptualization may participate in the refoundation of linguistics (Croft, 2000). Indeed, in the last fifty years language has been mainly considered as a fixed and idealized system which could be studied independently of its use and of its users. In this traditional body of theories, individual variation, and more broadly language evolution were either ignored or left unsolved, and biological evolution was used to explain the particularly good adaptation of our brains to the learning of nowadays idiosyncratic languages. On the contrary, viewing language as a system of replicators constantly replicating from particular brains to particular brains, with variation as a central concept, allows one to understand how languages change over time, why there is so much linguistic diversity, and provides a different account of the ease with which children learn languages. Indeed, as we showed in our last example, this framework allows us to understand that languages themselves probably evolved in a cultural Darwinian manner so as to become easily learnable by their users. And the peculiarities of the pre-existing learning systems of these users can explain the apparent idiosyncratic properties of languages. The paradigm shift induced by viewing language evolution as a Darwinian process also sets up new problems to be solved. In particular, it highlights the fact that the sharing of complex and intricate linguistic conventions must be explained: how can a system of competing replicators interacting at the level of individuals converge to a coherent and distinctive system adopted by all the population? We have shown with several computational experiments how this problem could be solved for lexical systems, thanks to the use of specific replication mechanisms based on positive feedback loops and self-organization. Yet, future work will have to show how several interacting levels of conventions, ranging from phonology to grammar and pragmatics, can be formed through a cultural Darwinian process.
Pierre-Yves Oudeyer
and
Frédéric Kaplan
6 Acknowledgements
This research has been partially supported by the ECAGENTS project founded by the Future and Emerging Technologies programme (IST-FET) of the European Community under EU R&D contract IST-2003-1940.
References
Arita, T., Koyama, Y., June 1998. Evolution of linguistic diversity in a simple communication system. In: Adami, C., Belew, R., Kitano, H., Taylor, C. (Eds.), Proceedings of Artificial Life VI. The MIT Press, Cambridge, MA, pp. 9–17.
Baronchelli, A., Felici, M., Loreto, V., Caglioti, E., Steels, L., 2006. Sharp transition towards shared vocabularies in multi-agent systems. Journal of Statistical Mechanics P06014.
Berlinski, D., 1972. Philosophical aspects of molecular biology. Journal of Philosophy 69 (12), 319–335.
Brighton, H., Kirby, S., Smith, K., 2005. Cultural selection for learnability: Three hypotheses underlying the view that language adapts to be learnable. In: Tallerman, M. (Ed.), Language Origins: Perspective on Evolution. Oxford University Press, Oxford.
Croft, W., 2000. Explaining language change. Linguistics. Longman.
Croft, W., 2002. The darwinization of linguistics. Selection (3), 75–91.
Dawkins, R., 1976. The selfish gene. Oxford University Press, Oxford.
de Boer, B., 1997. Generating vowel systems in a population of agents. In: Husbands, P., Harvey, I. (Eds.), Proceedings of the Fourth European Conference on Artificial Life. The MIT Press, Cambridge, MA.
De Boer, B., 1999. Self-organizing phonological systems. Ph.D. thesis, VUB University, Brussels.
De Jong, E., Steels, L., 2003. A distributed learning algorithm for communication development. Complex Systems 14 (4-5), 315–334.
Duda, R., Hart, P., Stork, D., 2001. Pattern classification. John Wiley and Son.
Hull, D., 1988. Science as a process: an evolutionary account of the social and conceptual development of science. Chicago: University of Chicago Press.
Hutchins, E., Hazlehurst, B., 1995. How to invent a lexicon: the development of shared symbols in interaction. In: Gilbert, N., Conte, R. (Eds.), Artificial Societies: The Computer Simulation of Social Life. UCL Press, London, pp. 157–189.
Kaplan, F., 2001. La naissance d’une langue chez les robots. Hermes Science, Paris.
Kaplan, F., 2005. Simple models of distributed co-ordination. Connection Science 17 (3–4), 249–270.
Kimura, M., 1983. The neutral theory of molecular evolution. Cambridge University Press, Cambridge.
Kirby, S., 2001. Spontaneous evolution of linguistic structure: an iterated learning model of the emergence of regularity and irregularity. IEEE Transactions on Evolutionary Computation 5 (2), 102–110.
Labov, W., 1994. Principles of Linguistic Change. Volume 1: Internal Factors. Blackwell, Oxford.
Mitchell, B., Weinmann, L., 1997. Creative design for the www. Lecture Notes, SIGGRAPH 1997.
Mufwene, S. S., 2005. Language evolution: The population genetics way. Vol. 29. Gene, Sprachen, und ihre Evolution, pp. 30–52.
Oliphant, M., 1997. Formal approaches to innate and learned communicaton: laying the foundation for language. Ph.D. thesis, University of California, San Diego.
Oudeyer, P.-Y., 2005a. How phonological structures can be culturally selected for learnability. Adaptive Behavior 13 (4), 269–280.
Oudeyer, P.-Y., 2005b. The self-organization of speech sounds. Journal of Theoretical Biology 233 (3), 435–449.
Schleicher, A., 1863. Die darwinsche Theorie und die Sprachwissenschaft : Offenes Sendschreiben an Herrn Dr. Ernst Häkel. Weimar : Böhlau.
Searls, D. B., 2002. The language of genes. Nature (420), 211–217.
Smith, A. D. M., 2005. The inferential transmission of language. Adaptive Behavior 13 (4), 311–324.
Sperber, D., 1984. Anthropology and psychology : towards and epidemiology of representations (the malinowsjumemorial lecture 1984). Man 20, 73–89.
Steels, L., 1996. A self-organizing spatial vocabulary. Artificial Life Journal 2 (3), 319–332.
Steels, L., 2004. Analogies between genome and language evolution. In: Pollck, J. e. a. (Ed.), Artificial LifeIX: Proceedings of the Ninth International Conference on the Simulation and Synthesis of Living Systems. The MIT Press, Cambridge, MA, pp. 200–207.
Steels, L., Belpaeme, T., 2005. Coordinating perceptually grounded categories through language: A case study for colour. Behavioral and Brain Sciences 28, 469–529.
Steels, L., Kaplan, F., August 1998. Spontaneous lexicon change. In: Proceedings of COLING-ACL 1998. Morgan Kaufmann, San Francisco, CA, pp. 1243–1250.
Steels, L., Kaplan, F., 2002. Bootstrapping grounded word semantics. In: Briscoe, T. (Ed.), Linguistic evolution through language acquisition: formal and computational models. Cambridge University Press, Cambridge, UK, pp. 53–74.
Stegmann, U. E., 2004. The arbitrariness of the genetic code. Biology and Philosophy 19 (2), 205–222.
van Driem, G., 2003. The language organism: The leiden theory of language evolution. In: Mırovsky, J., Kotesovcova, A., Hajicova, E. (Eds.), Proceedings of the XVIIth International Congress of Linguists. Prague: Matfyzpress vydavatelstv´ı Matematicko-fyzik´aln´ı fakulty Univerzity Karlovy, Prague.
Vogt, P., 2005. The emergence of compositional structure in perceptually grounded language games. Artificial Intelligence 167 (1–2), 206–242.
Zuidema, W., 2003. How the poverty of the stimulus solves the poverty of the stimulus. In: Becker, S.,
Obermayer, K. (Eds.), Advances in Neural Information Processing 15. Cambridge MA: MIT Press, pp. 51–68. |
 |
|
|
Alternative interpretations etc 
Monica Tamariz
Oct 26, 2007 17:31 UT
This is a most welcome paper reflecting an increasingly noticeable trend towards studying language evolution from usage-based perpectives within rigorous evolutionary epistemological frameworks.
Your first computer simulation models the effect of three innate imitation strategies on convergence in the cultural behaviour of agents. Imitation of the most recently observed variant (A) leads to convergence only after long fluctuation; imitation of the most frequently observed variant (B) obtains fast, self-reinforcing convergence; frequency-dependent probabilistic imitation (C) does not lead to convergence, the two variants eventually entrenching in a stable distribution. This is interpreted as evidence that pure imitation, akin to Sperber’s epidemiological model of cultural transmission, cannot explain replication of replication of linguistic units.
This simulation could be interpreted at a general level as a model of niche construction (Odling-Smee, Laland & Feldman 2003) by agents with different innate imitation strategies (although the effect of the changing niche on genes is not included); its linguistic interpretation is just one of many. Could it not also e.g. a model of the behaviour of rats with three different genotypes (A, B and C) for patterns of imitation of food selection in an environment where there are two types of food available (c1 and c2)? If so, perhaps its relevance to language evolution should be made more explicit. Also, the question of development – the plausibility of the correspondence between genotypes and behavioural phenotypes – is something that should be discussed and not simply assumed as it is in many models. An important related issue is the impact of the environment on development (e.g. the same genotype giving rise to different behaviours depending on the structure of the environment).
Random drift, in the absence of new variation, leads to eventual fixation of one variant and extinction of the others. This seems like a reasonable explanation of convergence towards a single variant in the population in models A and, especially, B. Still on 2.1, you say that “Model A is approximately similar to a random walk, converging in quadratic time, and thus is impractical in real life”. It would be interesting to investigate the ‘threshold of practicality’ empirically or with a simulation. Later you add “simple models of cultural transmission solely based on imitation are not sufficient to permit linguistic coordination”. Model A, though it takes longer than B, does converge eventually!
The next simulation seems more interpretable as “a system of associations between meanings and signals”. It is a model of adaptation of signals to being distinguishable from each other in a noisy transmission dynamics. A high noise:signal ratio results in stable signals and a low ratio allows for a degree of random drift when new agents (and therefore variants) are introduced in the variant pool. Your results can be interpreted as an argument in support of the hypothesis that the human anatomical and neural substrates for perception and production coevolved with language (or, more generally, with culture). Perception and production may have become finer-grained under the pressure to reduce the noise:signal ratio in linguistic (cultural) signals when more signals would have allowed encoding of new meanings (or, in the case of phonemes, when more phonemes would have increased the combinatorial power of the system).
An embodied model of convergence in the features of meaning during guessing games between two robots showed how perceptual biases influenced segmentation of the meaning space. Replicators here are categories which are under selective pressure to adapt to learnability (to the perceptual biases). The final simulation models convergence onto a shared syllable repertoire in a population of initially randomly babbling agents. Replication of babble syllable variants is favoured by previous knowledge of the syllable, and knowledge comes from random production during babbling. The agents converged on a syllable repertoire that turned out to be easier to learn by new agents (better adapted to learnability) than random repertoires. This simulation shows that, if one assumes that we can only learn things that are similar to things we already know, then cultural evolution is constrained by innate cognitive biases. One alternative, or perhaps a complement, to this “Evolutionary Psychology” assumption is the “starting small” principle (Jeffrey Elman, 1993), which allows you to learn things that you did not know beforehand. Additionally, the higher learnability of the evolved syllable repertoire could be explained as a consequence not of its adaptation to specific cognitive biases, but of its adaptation to a general learning bias favouring highly structured systems over random ones.
Finally, the term Darwinian has been mostly associated with selection; you do mention drift, and some interesting recent work has shown that the dynamics of some cultural variants is best explained by a neutral drift model (see Alex Bentley’s work). Other evolutionary mechanisms such as the equivalent of gene flow (language contact) and mutation are also worth exploring in an evolutionary framework (Baxter, Blythe, Croft & McKane (2005) discuss a variety of evolutionary mechanisms applied to language evolution.)
|
| |
|
1 reply to Alternative interpretations etc:
|
| |
|
|
Development and other complements
Pierre-Yves Oudeyer
Oct 30, 2007 18:12 UT
Dear Monica Tamariz,
thanks for you careful reading of our paper and your interesting comments. I kind of agree with most of them. I would just like to give further information on a few points. You propose that the question of development should be brought into these simulations. I cannot agree more. As a matter of fact, we (Frédéric and myself) think that development/epigenesis should be viewed as central to understanding language, and this is the reason why we are now involved in research in epigenetic/developmental robotics. It is true that we did not introduce this issue in this paper, but it might have been a good idea. For a glance on how we see developmental mechanisms introduced in language origins and evolution debate, the following preprint may be consulted:
Kaplan, F., Oudeyer, P-Y., Bergen B. (in press) Computational Models in the Debate over Language Learnability, Infant and Child Development. http://www.csl.sony.fr/~py/kaplan-oudeyer-bergen.pdf
In your fourth paragraph, you comment on our proposal that "simple models of cultural transmission solely based on imitation are not sufficient to permit linguistic coordination", remarking that our simulations show that in such models linguistic coordination actually happens, even if it takes a long time. You are right that at first sight, this seems to be only a quantitative speed difference between model A and model B. As a matter of fact, if you look at the precise figures, you will see that for a population of just a dozen agents, the apparently simple task of reaching a consensus on a few simple linguistic items, will take millions of interactions. In the real world, language requires a consensus on hundreds or thousands of complex linguistic items, which would then require billions of interaction if the dynamics of model A was followed. In practice, the life of humans is not long enough to allow for so many interactions. This is why we say that in practice, such a dynamics is not a very good candidate to explain how linguistic consensus can be reached.
In your 6th paragraph, you state that "This simulation shows that, if one assumes that we can only learn things that are similar to things we already know, then cultural evolution is constrained by innate cognitive biases." I would not agree with that. Cultural evolution is ALWAYS constrained by any cognitive bias. What our simulation shows is how linguistic representations can adapt, through cultural evolution, to a number of general innate cognitive constraints in such a way that these linguistic representations become easily learnable. Furthermore, I would not relate our approach to evolutionary psychology which is a quite different domain with quite different questions and theories (evolutionary psychology is concerned with the mechanisms of biological evolution that may explain the formation of innate behaviour). You also mention the "starting small" principle of Elman, which we think is very complementary to the simulation presented in this paper, and relates to the developmental aspects that we are working on in our most recent research.
|
|
The language-learning device as a
(5 replies)
Jose Luis Guijarro, Oct 18, 2007 16:49 UT
|
|
|
Ease of learning language
(1 reply)
Mike Tintner, Oct 16, 2007 15:35 UT
|
|
|
|
Note: yellow triangles ( ) indicate new messages that have been posted since your last visit to the site.
|
|
