Information flow in context-dependent hierarchical Bayesian inference

ABSTRACT

Recent theories developing broad notions of context and its effects on inference are becoming increasingly important in fields as diverse as cognitive psychology, information science and quantum information theory and computing. Here we introduce a novel and general approach to the characterisation of contextuality using the techniques of Chu spaces and Channel Theory viewed as general theories of information flow. This involves introducing three essential components into the formulism: events, conditions and measurement systems. Incorporating these factors in relationship to conditional probabilities leads to information flows both in the setting of Chu spaces and Channel Theory. The latter provides a representation of semantic content using local logics from which conditionals can be derived. We employ these features to construct cone-cocone diagrams, commutativity of which enforces inferential coherence. With these we build a scale-free architecture incorporating a Bayesian-like hierarchical structure, in which there is an interpretation of active inference and Markov blankets. We compare this architecture with other theories of contextuality which we briefly review. We also show that this development of ideas conveniently accommodates negative probabilities, leading to the notion of signed information flow, and address how quantum contextuality can be interpreted within this model. Finally, we relate contextuality to the Frame Problem, another way of characterising a fundamental limitation on the observational and inferential capabilities of finite agents.

KEYWORDS:

• Chu space
• channel theory
• bayesian inference
• contextuality
• information flow
• local logic
• cone-Cocone diagram
• active inference
• markov blanket
• frame problem

Acknowledgments

The authors wish to thank several anonymous referees for their constructive criticism, and helpful comments which assisted in improving the overall presentation of this paper. CF thanks the Federico and Elvia Faggin Foundation for financial support.

Disclosure statement

The authors report no conflicts of interest involved in this work.

Notes

1. Basically this means that the induced mapping $F:\mathfrak{C}\to \mathbf{C}\mathbf{h}\mathbf{u}(\mathbf{S}\mathbf{e}\mathbf{t},K)$ realises $F(\mathfrak{C})$ as an embedded subcategory of $\mathbf{C}\mathbf{h}\mathbf{u}(\mathbf{S}\mathbf{e}\mathbf{t},K)$ which consists of some objects of the latter and all of the arrows between them (see e.g., Adámek et al., 2004; Awodey, 2010). An example is the category of conditional probabilities (Culbertson & Sturtz, 2013) whose objects consist of countably generated measurable spaces, and whose arrows belong to a semigroup of Markov kernels.

2. In Pratt (1997), $K=\mathbf{2}$ is mainly taken, but other possibilities are discussed when working with the biextensional (full subcategory) $\mathbf{c}\mathbf{h}\mathbf{u}(\mathbf{S}\mathbf{e}\mathbf{t},K)$ of $\mathbf{C}\mathbf{h}\mathbf{u}(\mathbf{S}\mathbf{e}\mathbf{t},K)$.

3. In physics parlance, this amounts to ‘going to the Church of the Larger Hilbert Space’ where there are enough degrees of freedom to define pure states (cf. Chiribella et al., 2016).

4. We interpret ‘scale’ here broadly to include not only physical (e.g., length or time) scales but also organisational scale within a complex system.

5. The quantum-theoretic approach to contextuality is also developed by other means in e.g., Döring and Frembs (2019); Frembs and Döring (2019); Gudder (2019). In particular, Gudder (2019) introduces the notion of a ‘quantum channel’ formulated in purely functional-analytic terms, which differs (but is possibly relatable to) the category-theoretic based Channel Theory of information as adopted in this paper.

6. We thank an anonymous reader for pointing this out.

7. Allwein (2004) makes similar remarks when pointing to Barwise and Seligman (1997): ‘How do remote objects, situations and events carry information about one another without any substance moving between them?’. The general qualitative framework of Barwise and Seligman (1997), though not pinpointing specific measures of data flow within passive messaging, as in the Shannon information case, does nevertheless provide a flow of logical/semantic reasoning, as noted in e.g., Allo (2009), when the logical constraints of Definition 6.2 lead to structured inference. Given the premise that information is a physical mode of distinctions and the relationships between them, and causation is understood as transfer of information, Collier (2011) applies the logical formulism of Channel Theory to argue that causation itself may be viewed as a form of computation in view of the regular relations in a distributed system. In consonance, Old and Priss (2001) regard such information flow as a ‘conceptual’ channel between source and target. Accordingly, each classifier accommodates a ‘context’ in terms of its constituent tokens and types.

8. The Frame Problem has been popularised as a form of Hamlet’s problem: when to stop thinking? It describes the predicament that Fodor’s hapless robot falls victim to (Dennett, 1984; Shanahan & Baars, 2005)(cf. Fodor (1983)). The difficulty, of course, is that for any informationally unencapsulated inferential process, no a priori limit exists to information pertinent to that process.

9. At least as far as ‘heuristic solutions’ for humans go, such solutions emerge from the methodology of the massively parallel, competitively based, distributed system implemented by the GNW architecture (Dehaene & Naccache, 2001; Shanahan & Baars, 2005). Indeed, the GNW can be seen as a mechanism for determining relevance on the fly.

Additional information

Funding

This work was supported by the Federico and Elvia Faggin Foundation (CF).