1 The evolution of linguistic representations as a cultural Darwinian process

Even since the elaboration of the theory of natural selection by Charles Darwin, researchers have proposed that the mechanisms of language evolution may have strong similarities with the mechanisms of biological evolution (Schleicher, 1863). More recently, several kinds of analogies have been proposed. A first kind, tries to map units and structures in the genetic space directly to units and structures in the linguistic space (Berlinski, 1972; Searls, 2002; Stegmann, 2004). A second kind of parallel was developed in which the focus was on the Darwinian process of evolution rather that on the units themselves (Mufwene, 2005; Croft, 2000, 2002; Steels, 2004). The common process between genome and language evolution is here the following: 1) there exists a population of units capable of replication, 2) replication is not perfect: modifications can appear, 3) the units have different levels of efficiency in replication, which produces differential replication. This  high level formulation, sometimes conceptualized as a generalization of Darwin’s theory of natural selection (Hull, 1988), has the advantage of not specifying the structure of units as well as the mechanisms of replication and variation. And indeed, researchers found ways to instantiate it into biological or language evolution by filling in those missing slots with the corresponding specific structures and mechanisms (Croft, 2000). As far as biology is concerned, the units are genes, the mechanisms of replication are those associated with meiosis/mitosis, and the mechanisms of variation are mutations and cross-over. As far as language is concerned, a wide variety of instantiations have been proposed. The units of replication were conceived as ideas, mnemotypes, idene, culturetype, socio-genes, tuition (van Driem, 2003), ranging from simple abstract concepts like words or expressions to complex neural structures implementing associations between phonological forms and meaning. Perhaps the most well-known notion of cultural unit of replication is the meme introduced by (Dawkins, 1976). Linguistic memes, sometimes called linguemes (Croft, 2000), are themselves a population of very diverse kinds of units: phonological features, phonemes, syllables, rules of phoneme sequencing, lexicons, rules of syntax, semantic categories, systems of world categorization, constructions mapping combinations of words and complex meanings, prosodic structures, social conventions involving gestures and gaze to coordinate linguistic interactions, etc. Dawkins gives imitation as an example of mechanism of replication for units for language evolution. As a matter of fact, all kinds of linguistic activities, which can be much more complex than just imitation, like conversation or reading, provoke the replication of linguistic units. The consequence is that “leaping from brain to brain” is a very complex process that can happen through a variety of mechanisms. What provokes variation is therefore also very diverse: bad perception, erroneous interpretation, exaggeration, etc.

This shows that the conceptualization of language evolution as a Darwinian process may take quite different forms for different authors and is often presented only at a rather general level, especially in the memetics litterature. Yet, in order to be useful, we argue in this paper that this conceptualization must be precise, detailed and operational. Indeed, if language evolution is a Darwinian process, then it means that many of its features are systemic: they are the outcome of the complex interactions between replicators, replicating mechanisms and various kinds of constraints (e.g. learning biases, function or environment). Depending of each particular mechanism and of the particular ecological constraints, very different cultural dynamics might happen, and the adaptation of linguistic representation may (or may not) happen in many different manners. To make this point, we will review a number of computational models of the origins and evolution of language, showing the variety of replicating units and replication mechanisms that one can encounter at various levels, and showing what consequence they have on actual language evolution.

The common point of all the computational experiments we present is that they consist of populations of agents initially devoid of linguistic convention, and that will progressively and culturally build each time a new (simple) linguistic system. A first series of examples will focus on the functional constraints imposed on linguistic replicators due to their use as communication systems. In particular we will see that only very specific mechanisms of replication may allow for the efficient formation of shared linguistic conventions. Then, we will present an experiment showing how linguistic replicators can adapt and evolve under the specific constraints due to the external environment. We will then review an experiment studying the role of learning biases in the replication process and see how it can influence the evolution of linguistic representations.

2 Functional constraints on Darwinian dynamics

Linguistic replicators have specific properties compared to biological replicators: they form a system that permits communication. For instance, in a vocabulary in which words are associated to concepts/meanings and are used to draw the attention of several speakers towards a particular referent in a given context, synonymy and homonymy tend to be reduced ensuring efficient communication. We will see that not every differential replication process permits the emergence of such communication systems. Typically, each linguistic interaction involves the semiotic triangle: there is a form (e.g. a word), an associated meaning, and an associated referent in a particular context. This entails that three kinds of entities can be replicated through communication: forms, meanings, and associations within certain forms and certain meanings. As a matter of act, each of these kinds of entities consists itself of a variety of entities which are also replicators. For example, words are composed of sounds like vowels and consonants, which can be grouped into syllables through sets of phonotactic rules, which can themselves be sequences according to certain rules to build up words, and all these hierarchically entities can replicate differentially. Although all these replicators are constantly interacting, the experiments we will now present make a number of simplifications that allow us to develop a better understanding of the fundamental dynamics associated to various functional, internal and environmental constraints. For example, in the first experiment we will describe, we will suppose that there is only one meaning in the world that agents inhabit, and that two possible words can be associated to this meaning. This experiment will show some basic properties of the replication mechanism so that a simple convention can be adopted by a population, i.e. so that linguistic coherence can be reached (speakers associate the same word to the same meaning). This experiment will then be made more complex in following sections, allowing to study progressively more complex phenomena like linguistic distinctiveness (speakers associate different words to different meanings, next section).

2.1 Linguistic coherence and replication mechanisms: simple epidemiologic models are not enough

Let us consider a simple problem: N agents have to choose between two conventional names c1 and c2. We will consider three simple models representative for many more complex ones studied in the field. The first model is an imitation-based model (model A) and the two others are frequency-based (model B and C). In model A, the speaker simply produces the conventional name he heard last as a listener. In model B, the speaker produces the name that he has heard most frequently as a listener. In model C, the speaker produces any name that he has heard as a listener with a probability proportional to the frequency. These three types of replication processes could be seen as possible models of how cultural replication occurs. Yet, results show that the entailed dynamics are quite different (for details, see Kaplan (2005)). With imitation-based model A, the population eventually converges to a state of complete co-ordination. However, convergence happens typically only after a very long series of oscillations. With model B, convergence towards a single conventional name also occurs. However, the oscillations observed are much smaller. As soon as a convention spreads more in the population than the other, its domination seems to amplify even more over time and convergence happens quickly. For model C, on the contrary, dynamics tend to maintain the distribution of c1 and c2 over time, after an initial drift, and so there is no convergence. An in-depth study of these three models reveals that despite their apparent similarity the types of dynamics they create are extremely different. Among the three models studied, only model B creates self-reinforcing dynamics that permit a fast coordination of the entire population towards the use of a single conventional name. Model A is approximatively similar to a random walk, converging in quadratic time, and thus is impractical in real life. Finally, the dynamics of model C tends to maintain the distribution of the convention at a fixed non-convergent level.

What we must remember from these results is that not all cultural transmission systems create a differential replication process that ensures the domination of some linguemes over others in a reasonable time span. Simple models of cultural transmission solely based on imitation are not sufficient to permit linguistic coordination. In that sense the dynamics of linguistic replication are likely to be different from the ones characterizing epidemiological processes, which have sometimes been presented as a possible metaphor for cultural transmission (Sperber, 1984).

2.2 Linguistic distinctiveness and the adaptation of form-meaning pairs

For efficient communication, it is better that different words are associated with different meanings and vice versa. This obvious remark actually constrains systems of linguistic replicators in many important ways. Indeed, the replication process must not only address the issue on linguistic coherence but also permit linguistic distinctiveness. Let us consider a model in which individuals have to establish conventionalized associations between several words and several meanings. Each agent is now equipped with an associative memory, which is a list of word-meaning pairs with a numeric score. It is used to find the best word associated to a given meaning and reversely to find the best meaning associated to a given word. As in the model B of the previous section, the agents choose the association with the highest score when several solutions are possible. The associative memories of the agents are initially empty. Associations are progressively created as the agent interacts with other agents.

Studies of such systems were initiated by Steels in the mid 1990s (Steels, 1996). Several other experiments rapidly showed how collective dynamics could permit that each name eventually becomes associated with a single context and each context with a single convention (Hutchins and Hazlehurst, 1995; Oliphant, 1997; Arita and Koyama, 1998; De Jong and Steels, 2003; Vogt, 2005; Baronchelli et al., 2006; Kaplan, 2001). Some of the most interesting dynamics of such self-organizing lexicons are obtained in the presence of noise. Let us consider that each word/replicator ci is modelled with an integer value between 0 and 1000. Each time a word/replicator is transmitted, a random number between −B/2 and +B/2 is added to its value. Thus, B is a measure of the global noise level. Each agent is equipped with a filter permitting to select all the words/replicators in its associative memory of which the values are at a distance D less than D = B. The structure of an interaction is the following: Agent 1 randomly chooses a meaning s1 among the different meanings available and uses a word c1 to express this meaning. If it does not have words associated with this meaning, the agent creates a new one (a random integer between 0 and 1000). Then, c1 is transmitted to agent 2 with an alternation between −B/2 and +B/2. Then, agent 2 selects all the possible associations with a word close to the integer received (at a distance less than B). If several associations are  possible, agent 2 chooses the one with the highest score: (c2, s2). If s1 = s2 the interaction is a success, in the other cases the interaction is a failure. If no association is close enough in agent 2’s memory, the agent creates a new association between the received integer and the meaning s1. In case of success, agent 2 increases the score of the association (c2, s2) with +_ and diminishes the score of competing associations ((c2, _) and (_, s2)) where * is any meaning or word in the memory of the agent) with −_. In case of failure, agent 2 decreases the score of (c2, s2) with −_ (see (Kaplan, 2001; De Jong and Steels, 2003; Vogt, 2005) for discussions of the importance of such forms of lateral inhibition). Associations are initially created with a 0 score.

Figure 1: Evolution in the word space (idealized acoustic space) of the words associated with 5 meanings in an experiment involving 10 agents with a noise level B = 100. After a first period of ambiguity, five well separated bands appear associated with each meaning. Evolution of the ’average’ values (in the population) of the words associated with each meaning is highlighted in the middle of each band.

Can collective dynamics lead to choose the “best” word/replicators? A good word/replicator is a replicator that an agent will not confuse with another one that has a different usage. A “good” lexical system should have sets of words clearly distinct from one another depending on the meanings they associated to. Results show that indeed, distinctive sets of word-meaning associations are progressively formed and selected through cultural interaction. Figure 1 shows the evolution in the word space (i.e. an idealized acoustic space) of the words associated with 5 meanings in an experiment involving 10 agents with a noise level B = 100. After an initial ambiguity period, five well separated bands in the word space are clearly identifiable. Agents do not converge towards a unique value for each context. Each agent uses a different one. But these values tend to be very close. The “band” for one context is clearly distinct from bands associated with other contexts. No confusion is possible. Figure 1 plots also the ’average’ value of each band. Thus, it is easier to see the collective optimization of distinctivity leading to a solution compatible with the level of noise present in the environment. We also see on this graph that with this level of noise we approach the limit of expressiveness possible in this medium. If the agents had to communicate about a larger number of distinct meanings, ambiguity will inevitably arise.

2.3 Neutral drift

External factors like language contact between populations are often cited as a major cause of language evolution. But it is also known that language can change spontaneously based on internal dynamics (Labov, 1994). We have seen with the previous model that in a noisy environment, agents can converge on a stable system in which distinct bands in the word/acoustic space are associated with distinct contexts. As we see in figure 1, this repartition in separated bands does not evolve anymore once a stable solution has been found. Figure 2 shows the evolution of the average word in the presence of an agent flux defined by a probability of replacing an old agent by a new one Pr = 0.01, for a population of 20 agents and 2 contexts. The centre of the bands is spontaneously evolving as new agents are entering the system. This is an example of a neutral drift.