Communicative bottlenecks lead to maximal information transfer

ABSTRACT

This paper presents new analytic and numerical analysis of signalling games that give rise to informational bottlenecks – that is to say, signalling games with more state/act pairs than available signals to communicate information about the world. I show via simulation that agents learning to coordinate tend to favour partitions of nature which provide maximal information transfer. This is true despite the fact that nothing from an initial analysis of the stability properties of the underlying signalling game suggests that this should be the case. As a first pass to explain this, I note that the underlying structure of our model favours maximal information transfer in regard to the simple combinatorial properties of how the agents might partition nature into kinds. However, I suggest that this does not perfectly capture the empirical results; thus, several open questions remain.

KEYWORDS:

• Signalling games
• information transfer
• informational bottlenecks
• reinforcement learning
• emergent communication

Acknowledgments

Many thanks to Jeffrey A. Barrett, Aydin Mohseni, Cailin O’Connor and Oliver Lean for insightful comments and discussions on early drafts. Thanks also to the anonymous referees for helpful suggestions. This work was presented at the Philosophy of Biology Graduate Conference at the University of Calgary (2018), and I am grateful to the organisers and audience for helpful discussion. I would like to extend my appreciation to Mila – Québec Artificial Intelligence Institute and, in particular, Yoshua Bengio, for providing generous space and resources. Thanks also to Sarah and Atlas.

Disclosure statement

No potential conflict of interest was reported by the author.

Notes

1. See LaCroix (2019b) for further discussion.

2. Though this simple syntactic game is not itself compositional, see the discussion in Franke (2016); LaCroix (2019a); Steinert-Threlkeld (2016).

3. In a related paper, Barrett and LaCroix (2020) use these results to explain how the structural properties of a language come to reflect the world in which the language evolved. This shows how something like a principle of indifference (in a Bayesian sense) might arise naturally in an evolutionary context. This is discussed in further detail, in relation to the current analysis, in Section 5.

4. For now, I will assume that all states are equiprobable, so nature is unbiased, but this assumption can be relaxed.

5. See Huttegger, Skyrms, Smead, and Zollman (2010) for an overview of a number of different dynamics.

6. This simple reinforcement learning is based upon the ‘matching law’ proposed by Herrnstein (1970), which is itself a formalisation of the ‘law of effect’, due to Thorndike (1905, 1911, 1927). This learning rule is empirically tested in Erev and Roth (1998); Roth and Erev (1995). On the real-world effectiveness of simple learning, Schultz, Dayan, and Montague (1997) show that dopamine neurons in certain areas of primate brains seem to enact a reasonably similar learning procedure. See also Schultz (2004) and Glimcher (2011).

7. Punishment for miscoordination can be included in this model. This simply consists of discarding the ball when the agents fail to coordinate.

8. In the $2\times 2$ signalling game, when nature is not too biased, the sender and receiver converge towards one or the other signalling system with probability $1$ under this sort of dynamic (Argiento, Pemantle, Skyrms, & Volkov, 2009). Therefore, in the limit, the players learn to coordinate perfectly. Furthermore, after only $300$ time-steps, the communicative success rate of the sender and receiver is approximately $0.9$, on average (Skyrms, 2010b). Similar results hold for a variety of other dynamics, such as the replicator dynamic. See Huttegger (2007a, 2007c) for an overview.

9. The signal may also be understood as being about actions; see Huttegger (2007b); Zollman (2011) on separating indicatives and imperatives.

10. See Birch (2014); Godfrey-Smith (2011); Shea, Godfrey-Smith, and Cao (2018) for a critical discussion of Skyrms’ notion of informational content in the signalling game.

11. That is to say, there is a reduction of uncertainty from two possibilities to one. At a signalling system in a $4\times 4$ game, signals carry two bits of information (and so on). One bit of information, in this example, corresponds to the logarithm’s base being $2$. The units of this quantity can be specified similarly as nats or harts if we change the base to $e$ or $10$, respectively.

12. See, Shannon (1948) and Shannon and Weaver (1949).

13. In this context, the relative entropy of a particular signal can be understood as a measure of the information with respect to additional bits gained by moving from a prior to a posterior distribution, in a Bayesian sense.

14. See Huttegger (2007a) and Steinert-Threlkeld (2016).

15. This definition of a signalling game is more general than, e.g. that of Lewis (2002/1969).

16. See Donaldson et al. (2007). This follows from a proof due to Selten (1980).

17. See Donaldson et al. (2007). This follows from a proof due to Selten (1980).

18. Proof is given by Donaldson et al. (2007).

19. Proof is given by Donaldson et al. (2007).

20. Donaldson et al. (2007) refer to this as a ‘pool’, in order to draw a connection to pooling equilibria. I will refer to $\tau$ as a partition. Thus, ‘pool’ and ‘partition’ refer to the same thing.

21. Proof is given by Donaldson et al. (2007).

22. Proof is given by Donaldson et al. (2007).

23. Proof is given by Donaldson et al. (2007).

24. Again, I do not include in our frequency expectation the possible outcome for $\left(\begin{array}{c}4\\ 0\end{array}\right)=\left(\begin{array}{c}4\\ 4\end{array}\right)=1$. This would have us calculate our expected frequencies out of $16$ possibilities rather than $14$; however, these outcomes correspond to unstable strategies, and so they will be selected against in the long term.

25. In our urn-learning metaphor, the sender has 10 urns, each starting with two balls, and the receiver has two urns each starting with $10$ balls.

26. The cumulative success rate is a measure of success that takes account of the history of the game. It is calculated by dividing the number of plays that led to a success by the total number of plays in that run. When the players are successful, early failures are washed out as the number of plays increases.

27. The KS test on these data gives the statistic $D=0.0843$, which is the supremum of the set of distances between the empirical distribution function we observe and the expected distribution function from the combinatorial measure. This gives us a calculated $p$-value of $0.0017$, implying that we can reject the null hypothesis with high confidence.

28. Of course, since there is no punishment, every conditional action for the sender and receiver has some weight; however, in these cases, the weight for these acts is less than $1$ in $2500$.

29. More specifically, this includes the $⟨5,3,1⟩$ partition (occurring $0.073$); the $⟨4,4,1⟩$ partition (occurring $0.019$); the $⟨5,2,2⟩$ partition (occurring $0.066$); the $⟨4,3,2⟩$ partition (occurring $0.030$); and, rarely, the $⟨3,3,3⟩$ partition (occurring $0.005$). Again, in each case, we see all permutations occurring with approximately equal frequency.

30. See the discussion of Win-Stay/Lose-Shift in Barrett and Zollman (2009); Huttegger and Zollman (2011), and the related dynamic Win-Stay/Lose-Randomise in Barrett, Skyrms, and Mohseni (2019); Barrett and Zollman (2009).

31. For a discussion of sensory manipulation in the context of signalling games, see Barrett and Skyrms (2017).

32. This follows straightforwardly from the formula for entropy on the one hand and the formula for the binomial coefficient on the other.