Abstract
We use Category Theory notions to explore the conceptual axiom that, in a general sense, information flows through a neural network following a path of “least resistance”. To this end, we consider particular endofunctors, called preradicals, whereby we describe persistence in small shape diagrams defined in R-Mod. Specifically, we show that the α preradicals naturally describe persistence in commutative G-modules, for G a directed acyclic graph. Then, we use this results to generalize the notion of persistence to any diagram labeled by a quiver Q. These results, in turn, set the theoretical foundation for our formal framework that explores the notion of “paths of least resistance”. Lastly, we provide a notion of entropy for preradicals in R-Mod, and prove that it respects the order and operations between preradicals.
1. Introduction
Networks, as mathematical objects (formally graphs with additional properties on the edges and/or vertices), naturally capture and describe how the physical components that constitute the structure of a network support dynamics on top of that structure. This, in turn, leads to the encoding and processing of information by the network, determining its capabilities and the functions it can support. In this paper, we explore the critical intuition that, while descriptive models or algorithmic rules establish the conditions for the dynamics of a network, it is the resulting flow of signals through the network, constrained by physical factors, and the information captured by those signals, that dictate the prioritization and execution of computational events, ultimately shaping the dynamic function of the network. In fact, we assert that this foundational phenomenon, which lies at the core of all network dynamics, naturally follows a path of least resistance. This conceptual notion represents a universal principle that, at the most fundamental level, is responsible for the dynamics of any network. Yet, capturing and describing it mathematically in a universal way is not at all obvious without resorting to the modeling or description of the specific physical details that constitute a network.
We derived motivation for the universal theoretical framework proposed in this paper from our group's prior work, focusing on modeling information and dynamic theoretical concepts within biological neural networks composed of interconnected neurons. Specifically, our exploration involved studying neural network models to comprehend the foundational principles governing its autonomous operations. We delved into understanding what the drivers are that influence the prioritization, order, and decisions of internal computations and operations following an input received by a single neuron or a network of neurons (see Buibas and Silva Citation2011; Muotri, Silva, and White Citation2020; Silva Citation2019). In this endeavor, we rely on two mathematical theories for constructing a formal framework that describes the conceptual axiom that information flow through a network will always follow a path of “least resistance”. To the best of our knowledge, a theoretical framework that captures the universality of this effect does not exist.
One of the theories we use in this work is called Persistence Homology. Broadly speaking, Persistence Homology is employed in algebraic topology to analyze and characterize the topological features of a space. It focuses on the identification and tracking of homological features, such as connected components, loops, and voids, as they persist through different scales or levels of simplification. Here, we interpret this persistence of information as “the flow” of information. With this in mind, we use notions and concepts of Category Theory to generally describe the flow of information. Put simply, Category Theory examines mathematical structures and their relationships. In this context, it provides a high-level, formal language to analyze and connect diverse mathematical theories. Recently, Category Theory has shown to be a powerful tool to describe systems and their complex structures. Its broad applications vary from biological networks (Haruna Citation2013), Bayesian networks (Fong Citation2012), Machine Learning (Fong, Spivak, and Tuyéras Citation2019), Complex Systems (Baas Citation2019a, Citation2019b; Baas, Ehresmann, and Vanbremeersch Citation2004), and neural network signal processing (Armenta and Jodoin Citation2021; Parada-Mayorga et al. Citation2020), among others.
Considering the above two theories, we prove the main results in Chambers and Letscher (Citation2018) for single-source single-sink directed acyclic graphs through the lenses of Category Theory, and then, extend these results to diagrams whose underlying structure defines a directed graph (or more generally, a quiver). Specifically, we use particular endofunctors, called preradicals, to show how information flows through structures known as quiver representations. Typically, a quiver representation is visualized as an assignment of vector spaces (or R-modules) to nodes of a directed graph, and an assignment of linear morphisms (R-linear morphisms) to the arrows of the graph. We see that the flow of information, or persistence, can be computed using an α-type of preradical defined on a category generated by the quiver representations. The theoretical framework and results are then used to explore the conceptual axiom that, in a general sense, information flows naturally through “paths of least resistance”.
After elaborating on the framework for the flow of information through preradicals and the paths of least resistance, we introduce a concept of algebraic entropy that applies universally to preradicals. We recall that, as a mathematical concept, entropy was first defined in topology by Adler, Konheim, and McAndrew in Adler, Konheim, and McAndrew (Citation1965) for a continuous self-map of a compact space. From an algebraic perspective, Weiss was the first to suggest the idea of algebraic entropy for endomorphisms of abelian groups (Weiss Citation1974). Simultaneously, Peter provided a slightly different definition of entropy for automorphisms of abelian groups (Peters Citation1970), which was later extended in Dikranjan and Giordano (Citation2016) to consider endomorphisms. One substantial difference between Weiss's and Peter's definitions is that in the former, one takes the supremum over all finite subgroups, while in the latter, one considers a supremum over all finite subsets of a group G. Subsequently, algebraic entropy was generalized in Salce and Zanardo (Citation2009) beyond abelian groups (or -modules) to modules over a unitary ring R. There, the authors considered real-valued functions - called invariants, to define an algebraic i-entropy. In particular, one considers invariants with two minimal requirements with which one can associate an algebraic entropy to it. Here, we prove that each subadditive invariant i on R-Mod gives rise to an i-entropy for preradicals on R-Mod. Furthermore, each of these entropies respects the order and the operations between preradicals. We also provide some examples and utilize functoriality to demonstrate how the algebraic entropy relates to preradicals on S-Mod and R-Mod, whenever a ring homomorphism is provided.
The paper is organized as follows: Section 2 provides a brief description of persistent homology and preradicals defined on categories of the form R-Mod, where R is an associative ring. We discuss the role of preradicals as a tool to outline the flow of information in small shape diagrams within R-Mod. Section 3 contains original results describing our technical contributions to understanding the flow of information in the context of persistence. Initially, we establish existing but crucial results regarding persistence in commutative G-modules in terms of preradicals, a task not previously undertaken. Subsequently, by using universal constructions in R-Mod, we leverage these results and formalize the notion of persistence for any diagram labeled by a quiver. In Section 4, we discuss how our construction and results motivate the conceptual axiom that information flows through “paths of least resistance”. Finally, Section 5 introduces a definition of entropy for preradicals on R-Mod. We provide some examples and show some functorial properties. We have added an Appendix section containing some proofs and additional details to facilitate the reader's understanding of the paper.
2. Preliminaries
In this preliminary section, we introduce the two essential mathematical tools integral to our framework. On the one hand, we have persistence homology, which is a powerful tool to scrutinize the topological features of a space, specifically focusing on their evolution across different scales. On the other hand, we have the concept of preradicals for categories of the form R-Mod. Preradicals play a pivotal role in our framework as these will describe the flow of information. In order to provide the reader with a foundational understanding, we give a brief description of these two concepts. However, for a more in-depth exploration, we encourage readers to refer to dedicated resources (Carlsson and de Silva Citation2010; Carlsson and Zomorodian Citation2009; Chambers and Letscher Citation2018), and Bican, Kepka, and Němec (Citation1982), Horbachuk and Yu (Citation2011), and Raggi et al. (Citation2002a, Citation2002b), for a comprehensive understanding, respectively.
Persistent homology examines a family of topological spaces and their inclusions to identify common topological features among subsets within these spaces. Standard persistent homology, for instance, considers linearly ordered spaces with one space included in the next. Another variant, known as zigzag persistence (Carlsson and de Silva Citation2010), retains linear ordering but allows inclusions to occur in either direction. Additionally, multidimensional persistence (Carlsson and Zomorodian Citation2009) operates in multiple dimensions on a grid, with inclusion maps parallel to the coordinate axes. In Chambers and Letscher (Citation2018), Chambers and Letsher consider filtrations whose underlying structure defines a Directed Acyclic Graph (DAG):
Definition 2.1
Chambers and Letscher Citation2018, Definition 2.1
For a simple directed acyclic graph , a graph filtration of a topological space X is a pair such that
for all ;
If then is a continuous embedding (or inclusion) of into .
The resulting diagrams are commutative: in other words, for any path in G, one can naturally extend this to a function on the topological spaces . Then, commutativity means that for any two different directed paths γ and in G, connecting vertices u and v, one has that .
Building on these DAG filtrations, the authors in Chambers and Letscher (Citation2018) introduced algebraic representations for such structures, terming them commutative G-modules:
Definition 2.2
Chambers and Letscher Citation2018, Definition 2.2
Given a directed acyclic graph , a commutative G-module, , is a pair of families , such that for each vertex v, is an R-module, and for each edge in E, is an R-morphism, with the condition that the resulting diagrams are commutative.
Observe that the commutative relations in a commutative G-module arise from following distinct paths within the DAG. In other words, if and are two different paths in G, connecting vertices u and v, then . The above definition of commutative G-module is a particular case of what we know as a quiver representation. In fact, a quiver is a 4-tuple , where V is a set of vertices, E a set of edges, and are two functions that assign the starting vertex and the target vertex for each edge, respectively. This 4-tuple structure generalizes directed graphs, as it allows self-loops and multiple directed edges between vertices. With this in mind, we have the next
Definition 2.3
Given a quiver , a representation of Q over the ring R is a pair of families such that, for each , is an R-module, and for each edge with and , is an R-morphism.
Taking advantage of the completeness and cocompleteness of the category R-Mod, Chambers and Letsher obtained the persistence in a commutative G-module by considering the constructions of limits and colimits of the diagram defined the G-module. More precisely, they show that the persistence of is given by the image of the induced morphism It is worth mentioning that the authors primarily study single-source single-sink DAGs, for which there is only one induced morphism φ. In general, the number of morphisms from to depends on the number of source and sink vertices in the underlying DAG. Inspired by this categorical construction, we generalize the concept of persistence using preradicals. Initially, we present the main results for commutative G-modules whose underlying structure is a single-source single-sink DAG. Then, we extend these results to DAGs with n sources and m sinks. Subsequently, we use this construction to define persistence in any finite quiver representation in R-Mod.
Now, a preradical is an endofunctor that maps objects to subobjects and morphisms to restrictions. As noted in Pardo-Guerra, Rincón, and Zorrilla-Noriega (Citation2020), preradicals can be visualized as compatible choice assignments, where the assignments of objects to subobjects are all compatible with the morphisms of the category. This last property is, in fact, what makes preradicals useful for describing the flow of information in diagrams, as they preserve the underlying structure and provide invariant subobjects. In this work, we specifically consider preradicals in categories of the form R-Mod, where R is an associative ring.
Definition 2.4
Raggi et al. Citation2002a, Definition 1
Let R be an associative ring with a unit element. A preradical σ on R-Mod is an endofunctor that assigns to each module -, a submodule such that, for each R-morphism , one has the commutative diagram
Here, ι represents the inclusion map, and .
Example 2.5
Consider the category -Mod of all -modules. Now, given -Mod, we define Let us first notice that for each -Mod, is a submodule of M. Also, for any morphism in -Mod, the linearity of f implies that , for all . Thus, if one has that , which in turn implies that . Therefore, and hence,
is a commutative diagram.
As we will demonstrate in the next section, preradicals delineate the flow of information through the persistence of a quiver representation. To achieve this, we conceptualize each quiver representation as a diagram in R-Mod. Recall that a diagram of shape in the category R-Mod is a functor -. Small categories are commonly employed to define diagrams, as they have only a set's worth of arrows. For illustrative purposes, let us consider a linearly ordered shape diagram in R-Mod
and a preradical σ on R-Mod. By applying σ to the components of the diagram, we obtain the commutative diagrams
Note that the bottom row in the above diagram illustrates how information, related to σ, traverses through the diagram of shape . Also, observe that σ, as a compatible choice assignment, preserves the structure and relations of the linear shape diagram, thereby facilitating the description of the flow of information or persistence. With this understanding, we demonstrate that an α preradical, defined on a subcategory generated by the commutative G-module, naturally gives the persistence on the G-module structure. This result, in turn, leads us to a generalization of the notion of persistence in any finite quiver representation on R-Mod.
3. Preradicals and persistence
This section introduces our technical results and contributions, describing the flow of information through the lens of preradicals. We show that the persistence on a commutative G-module is given by an α-type of preradical. This allows to generalize the main results in Chambers and Letscher (Citation2018) to any DAG with n source and m sink nodes. Then, we formalize the notion of persistence to any diagram labeled by a quiver.
We start by noticing that, when a directed acyclic graph is depicted as a quiver, then, we can identify a commutative G-module with a quiver representation over a ring R. This representation satisfies particular commutative conditions that arise from following distinct paths within the DAG. Now, from a Category Theory perspective, these representations can be interpreted as small shape diagrams in R-Mod. Their shape is defined by a free category associated with the underlying quiver Q. The free category , also referred to as the path category, has as objects the vertices of the quiver, and as morphisms all finite directed paths between vertices. The composition operation is defined by concatenation. Consequently, the category encompasses all conceivable paths between the vertices of the quiver by which information flows.
Hereafter, we will refer to diagrams in R-Mod as those having the shape of a path category , where Q is a quiver. Such diagrams will be denoted by when there is no room for confusion. Now, in order to describe the flow of information via preradicals, we start by considering the category , whose set of objects coincides with the hereditary class generated by . In simpler terms, the objects of encompass all submodules of the modules lying in the diagram . Regarding morphisms, encompasses all the morphisms present in the diagram , in addition to inclusion maps between submodules, identity morphisms, zero maps, and, lastly, all possible compositions among these types of morphisms. For example, if is a morphism in the diagram , and N is a submodule of , then the restricted morphism is a morphism of . This is because can be expressed as the composition , where is the inclusion map. We will call the category the completed free category generated by the quiver representation . As we will see shortly, it is within these completed free categories that we describe the flow of information, or persistence, using an α-type of preradical.
Proposition 3.1
Let Q be a quiver, and let be a quiver representation. For any object , and any subobject N of L, the map , defined in each by (1) (1) is a preradical on .
Proof.
Let Q be a quiver, and let be a quiver representation over R. Given the definition of the completed free category , it is sufficient to demonstrate the result for a preradical of the form . The proof for any other choice of and W follows a similar approach. Thus, we will show that, for any morphism in , one has the commutative diagram
By definition, we have that Thus, thereby showing the commutativity of the diagram.
Let be a commutative G-module whose underlying structure G defines a single-source single-sink DAG:
As observed in Chambers and Letscher (Citation2018, Lemma 3.3), when taking the limit and colimit of the diagram defined by , the apex cones and coincide with and , respectively. Consequently, the legs of the limit cone correspond to morphisms of the form , where γ represents a path in G connecting vertex s with vertex j. Similarly, the legs of the colimit cone correspond to morphisms , where γ represents a path in G connecting vertex j with vertex t. In this context, due to the commutativity conditions on , only one induced morphism φ is obtained from to . Thus, the persistence of is given by (2) (2) As we next show, the above persistence is described by the following α-type of preradical:
Proposition 3.2
Let be a commutative G-module whose underlying structure G is a single-source single-sink DAG. If and denote the module representation in of the source and sink vertices of G, respectively, then the persistence mentioned in Chambers and Letscher (Citation2018, Lemma 3.3) is obtain through the preradical defined on the completed free category generated by .
Proof.
Let be a commutative G-module as illustrated in Figure , and let be the completed free category generated by . For the modules and in , the preradical evaluated in is However, as noted above, there is only one induced morphism , and consequently, only one in . Therefore,
The authors showed in Chambers and Letscher (Citation2018, Proposition 3.4) how DAG persistence generalizes the Standard persistence, the Zigzag persistence, and the Multidimensional persistence. To establish this, they relied on Chambers and Letscher (Citation2018, Lemma 3.3), which provides persistence in single-source single-sink DAGs. Following the same principles, Proposition 3.2 leads to the next
Proposition 3.3
Let G be a directed acyclic graph and let denote the k-dimensional persistence module for a filtration . Then:
(1) | (Standard Persistence) If is the graph corresponding to a filtration , and I is the subgraph of G consisting of vertices , then the persistence of the commutative G-module induced by I is where is a preradical defined on . | ||||
(2) | (Multidimensional Persistence) If is a multifiltration with as its underlying structure DAG, and J is the subgraph with vertices , then the persistence of the commutative G-module induced by J is given by |
where is a preradical defined on .
Proof.
Let be a DAG, and let be the commutative G-module corresponding to the k-dimensional persistence module for a graph filtration .
(1) Since the subgraph of G defines a single-source single-sink graph, we can use Proposition 3.2 to obtain the persistence of the commutative G-module associated to the subgraph I by means of an preradical. In this case, the commutative G-module associated to I is given by where the modules are indexed by the set , whereas the morphisms are indexed by the set . Thus, if we denote the persistence of the commutative G-module by , then, Proposition 3.2 implies that Here, is a preradical defined on the completed free category . However, as is a full subcategory of , then, the preradical coincides with the corresponding preradical , now defined on the category . Hence, by Proposition 3.2, it follows that (2) Let us first notice that the subgraph is a single-source single-sink graph (namely, the vertices u and v, respectively) so we can apply Proposition 3.2 to obtain the persistence of the commutative G-module associated to the subgraph J. In this case, the commutative G-module associated to J has the form where the modules are indexed by the set , whereas the morphisms are indexed by the set . Thus, if denotes the persistence of the commutative G-module , then, where is a preradical defined on the subcategory of . As noted above, we can also realize as a preradical on , since is a full subcategory of .
We now generalize Proposition 3.2 for G-modules whose underlying graph has n source vertices and m sink vertices. For that, we start by computing the persistence of a diagram in terms of arbitrary cones and cocones. This persistence, in turn, will ultimately be given by the construction of universal cones, leading us to our definition of persistence in small shape diagrams labeled by a quiver. Consider then a commutative G-module whose underlying DAG has n sources, denoted by , and m sinks, denoted by . Let be a cone and a cocone of the diagram defined by (Figure ):
Notice that each leg morphism , with , is factorized by some with . Furthermore, due to the commutative property in the cone's definition, these factorizations all coincide. Indeed, if and are the morphisms in , corresponding to paths in G from the source vertices and to vertex l, respectively, then, one has that . Similarly, for the cocone , each leg morphism , with , is factorized as . This time, is a morphism in that corresponds to a path in G connecting vertex l with sink vertex . These observations show that the information flowing from the apex C of , through , and reaches apex D of , is ultimately represented by all morphisms where , and is a morphisms in corresponding to the path in G from vertex to vertex . Furthermore, when considering the completed free category , which is generated by along with and , we see that its underlying structure defines a single-source single-sink graph. Hence, by Proposition 3.2, we can compute the persistence by means of the preradical on , obtaining
(3) (3) The above resolution also holds for the universal cone and universal cocone of . As we next see, it is their universal properties that enable us to express persistence with arbitrary cones and cocones in terms of persistence using the limit and colimit cones. To illustrate this, consider the diagram in Figure , where defines a cone, defines a cocone, and denotes the limit cone whereas the colimit cone:
Now, by the limit's universal property, there exist a unique morphism such that , for . Likewise, by the cocone's universal property, we have a unique morphism such that , for . Thus, the flow of information is given by
This last expression, in turn, can be written as where is the image of morphism δ, and is the preradical defined on the completed free category , which is generated by along with and .
Theorem 3.4
Let G be a DAG with n source vertices and m sink vertices, and let be a commutative G-module. If denotes the completed free category generated by along with and , then the persistence of is given by (4) (4) where is a preradical defined on the completed free category .
We note that Theorem 3.4 naturally extends the notion of persistence stated in Proposition 3.2. Indeed, if is a commutative G-module whose underlying structure defines a single-source single-sink DAG, then and . Thus, the free category coincides with the path category , and hence, The above conclusions lead us to extend the conditions in Theorem 3.4 to consider any quiver representation over the ring R, regardless of its commutative properties. Thus, if is the completed free category generated by along with its limit cone and colimit cocone , we get
Definition 3.5
Let be a quiver representation over the ring R. Then, the persistence of is given by where is the preradical defined on the completed free category .
4. Preradicals and the flow of information
This section aims to explore the conceptual axiom that, in a general sense, information flows naturally through “paths of least resistance”. We investigate this phenomenon using our algebraic framework from Section 3, representing a neural network and its underlying directed graph with a quiver representation over a ring R. We claim that, by describing each neural network as a quiver representation, we can deduce that information flows through a network in such a way that it is factorized by distinguishable morphisms that are induced by the network itself and its local configurations. Thus, any network over which information is transmitted as discrete dynamic signals can be interpreted in two ways. On the one hand, from a static point of view, the directed graph of neurons represents the passage on which electrical signals flow. On the other hand, in the dynamic mode, information flows in the network through these distinguishable objects and morphisms that encode the essential information of the local configurations, thus defining the “paths of least resistance”.
For illustrative purposes, we start examining a neural network responding to a single stimulus, resulting in activations of specific neuron sets. These activations, along with their ensuing interactions, give rise to emergent dynamics. Now, we will refer to a coordination neuron as a common post-synaptic neuron activated due to stimulation of a group of neurons. Algebraically, we represent this scenario with a cocone with apex the coordination neuron and legs morphisms (Figure ):
When considering a learning process induced by continuous stimuli, Hebb's rule (Ehresmann and Vanbremeersh Citation2007, 290) synchronizes the assembly of neurons, causing them to evolve as a unit. This unit encodes information for the specific type of stimuli received by the assembly. Accordingly, we represent each unit as a “cat-neuron”–a higher-order neuron containing the essential information associated with a particular neural activity configuration. These cat-neurons will arise from constructing limits or colimits for the assembly of neurons. This practice of taking limits and colimits for an assembly of objects and morphisms–typically small shape diagrams–has been widely employed in designing neural network architectures (see, for instance, Ehresmann and Vanbremeersh Citation2007; Healy Citation1999). From our perspective, this process represents the smallest possible configuration of information interacting with any other object interacting with the given assembly.
Taking the above into consideration, we now delve into describing the flow of information from a local perspective and elucidate the notion of the path of least resistance. In this context, recall that the category R-Mod is complete and cocomplete for any commutative ring R, allowing thus the construction of limits and colimits for any small shape diagram. Let us consider initially a discrete assembly comprising n pre-synaptic neurons , and a coordination neuron . By introducing the cat-neuron associated with this assembly, namely, the direct sum alongside the inclusion maps (Figure ),
we get a unique morphism , such that , for all . It's noteworthy that the full information received at the coordination neuron is (5) (5) which can also be described by an preradical. Indeed, consider the limit for the discrete set of source components , namely the direct product together with the projections for each , and then take the enlarged diagram (Figure ):
Then, the preradical defined on the completed free category provides the persistence of information that flows from the discrete source of components to the coordination neuron . This persistence is given by the sum of the images of all morphisms in . As each of these morphism is of the form for , it follows that (6) (6) Here, each morphism in , with domain and codomain , is factorized by η and , for some .
Let us now consider an assembly of neurons that are related by synaptic interconnections among each other. For any coordination neuron , by taking the cat-neurons and of the assembly, along with their morphisms, we obtain a diagram as displayed in Figure :
With this setup, the persistence of information is obtained by first considering the enlarged diagram , and then, evaluate the preradical , defined on the completed free category , at . Here, the persistence of information is given by the sum of the images of all morphisms in with domain and codomain . Note that each of these morphisms is factorized by ξ and , for some .
Lastly, let us consider the scenario where more than one assembly of neurons share a common coordination neuron N. For the sake of simplicity in notation, we will illustrate the case with two assemblies, as the general case with a finite number of assemblies follows the same pattern. Thus, by taking the cat-neurons for each of these assemblies, we get a diagram as in Figure . There, we observe that all leg morphisms of the cocone with apex N are factorized by the morphism provided in the colimit's construction. Consequently, since the total information received at neuron N is essentially the sum of the images of all leg morphisms (i.e. both and ), we may interpret this factorization as a manifestation of the conceptual principle that information follows 'paths of least resistance,' which emerges from the colimit's constructions.
5. Entropy for preradicals
In this section, we derive a notion of algebraic entropy that applies to preradicals from an overall perspective. We demonstrate that each algebraic entropy respects the operations between preradicals on R-Mod. Furthermore, we provide examples and use functorial properties that allow for comparing different entropies.
We start by recalling some definitions and basic facts that lead to the notion of entropy for objects and endomorphisms in R-Mod. Additionally, we offer background results that support the development of algebraic entropy, which are detailed in the Appendix section. For a comprehensive description, we direct readers to Dikranjan and Giordano (Citation2016, Citation2019), Dikranjan et al. (Citation2009), Giordano and Salce (Citation2012), and Salce and Zanardo (Citation2009).
Definition 5.1
Dikranjan and Giordano Citation2013, Definition 1.1
An invariant in R-Mod is a function such that and , whenever M and are isomorphic objects in R-Mod.
From now on, we will consider subadditive invariants, this is, invariants that satisfy the following two conditions:
for all submodules of M.
for any submodule N of M.
Here, we will follow Weiss' approach when computing entropy, this is, we will consider the collection of submodules of M such that , which we denote by
Definition 5.2
Dikranjan and Giordano Citation2019, Definition 5.25
Let i be a subadditive invariant on R-Mod, M an R-module, and an endomorphism. The algebraic i-entropy of η with respect to is The algebraic i-entropy of η is defined as
Definition 5.3
Salce and Zanardo Citation2009, Definition 3
Let i be a subadditive invariant of , and let denote the set of all endomorphism of the module . The algebraic i-entropy of M is
With the two above definitions in mind, we proceed to define the i-entropy for preradicals on R-Mod.
Definition 5.4
Let i be a subadditive invariant on , and let . If , then the algebraic i-entropy of σ with respect to M is given by where .
Remark 5.1
If i is a subadditive invariant on R-Mod, then, for every preradical and every module -Mod, one has that
Proposition 5.5
Let i be a subadditive invariant on R-Mod. If σ and τ are two preradicals such that , then (7) (7) for any .
Proof.
Let and let σ and τ be preradicals such that . As , then, for any endomorphism , we have that . Hence, and thus, Therefore, .
Corollary 5.6
Let σ and τ be two preradicals, and let . Then,
Proof.
Since holds in the lattice R-pr of module preradicals, it follows from Proposition 5.5 that for any -Mod.
We now provide two examples of different i-entropies applied to the same preradical. We will observe that depending on the chosen invariant i, the outcomes vary.
Example 5.7
Let -Mod be the category of all -modules, and let Tor be the preradical on which assigns to each -module M its torsion submodule . Consider now the invariant defined by whenever M is finite, otherwise . Recall that for any endomorphism , and any , one defines This is, is the logarithm of the η-trajectory of L. This way, the algebraic i-entropy of η, with respect to the finite subgroup L, is Hence, the algebraic i-entropy of η is Considering the above, one has the algebraic i-entropy of M is Now, let us define the -module , where denotes the cyclic module of order p, for each . We observe that, as M is a torsion module itself, then . Also, for the Bernoulli shift endomorphism, whose correspondence rule is we get that . Having this in mind, when taking the subgroup of M, we have that for each . Hence, for each . Therefore, , which in turn implies that . Moreover, since the β-trajectory of the finite subgroup reaches M, then, it also covers any other β-trajectory for any finite submodule F of M. Thus, , whereby This shows that .
In contrast, we have the next
Example 5.8
Let Tor be the torsion preradical on -Mod, and let us now consider the rank invariant , defined as follows: for each -module M, whenever this dimension is finite, otherwise . In this case, as for any torsion module -Mod, we have that for all Mod.
We now demonstrate how the entropy associated with the invariant log relates to categories of the form S-Mod and R-Mod, whenever there exists a ring homomorphism between the base rings. Suppose is a ring homomorphism. Then, each S-module M inherits an R-module structure through the homomorphism t. Indeed, for each -Mod, t induces a scalar multiplication but as an R-module given by , where and . With this in mind, t gives rise to a functor that assigns, to each the induced R-module M, and assigns to each S-morphism the same map f but now considered as an R-morphism.
Proposition 5.9
Let and be the log invariant defined on and , respectively. If is a ring homomorphism, then, for any and its induced R-module , one has (8) (8)
Proof.
Let be a ring homomorphism. For clarity sake, we denote an S-module by and its image under the functor by . First, if , then . In fact, the functor maps the inclusion map of S-modules into the inclusion map of R-modules. Thus, defines a submodule of . Furthermore, if , then , implying that . Consequently, for each endomorphism , and each , where denotes the endomorphism arising from evaluating functor in η. Taking the above into consideration, we have that and hence, Therefore,
In the case where is a surjective ring homomorphism, by Fernández-Alonso, Gavito, and Pérez-Terrazas (Citation2018, Theorem 2.5) the induced functor is full. Furthermore, since respects inclusion maps and is injective on objects, Fernández-Alonso, Gavito, and Pérez-Terrazas (Citation2018, Theorem 2.4) provides an injective assignment that is order preserving. This assignment is given by where is the alpha preradical on R-pr defined by the images of and M under functor .
Proposition 5.10
Let and be the invariants defined on and respectively. If is a ring epimorphism, then (9) (9) for each preradical .
Proof.
Let σ be a preradical on S-Mod. For clarity sake, let us denote again an S-module by and its image under functor by . Following this notation, given , we denote by the R-module corresponding to the image of under functor . Thus, as preserves inclusion maps, the inclusion is mapped to the inclusion map . With this in mind, we have that Further, . Lastly, for each , it follows that Therefore,
Discussions
In conclusion, our exploration, rooted in Category Theory, has provided a formal framework to investigate the fundamental notion that information traverses neural networks via paths of least resistance. By representing neural networks as quiver representations within an algebraic framework, we elucidate the persistence of information in these quiver representations through the lens of preradicals. We also extend the utility of preradicals by demonstrating their aptitude in delineating information flow within small shape diagrams labeled by quivers overall. Furthermore, our algebraic framework opens avenues to explore the concept of “paths of least resistance”. Future research trajectories may entail employing preradicals to characterize information persistence in categories inspired by dynamical systems, such as the emerging field of Markov categories.
Acknowledgments
The authors wish to express their profound gratitude to the referee for their suggestions, comments and corrections, which greatly improved this work.
Disclosure statement
No potential conflict of interest was reported by the author(s).
Data availability statement
Data sharing is not applicable to this article. No dataset was generated or analyzed during this research.
Additional information
Funding
Notes
1 An endofunctor is a functor whose domain is equal to its codomain
2 A subobject of an object is a monomorphism with codomain C.
References
- Adler, R., A. Konheim, and M. McAndrew. 1965. “Topological Entropy.” Transactions of the American Mathematical Society 114 (2): 309–319. https://doi.org/10.2307/1994177.
- Armenta, M., and P.-M. Jodoin. 2021. “The Representation Theory of Neural Networks.” Mathematics9 (24): 3216. https://doi.org/10.3390/math9243216.
- Baas, N. 2019a. “On the Philosophy of Higher Structures.” International Journal of General Systems 48 (5): 463–475. https://doi.org/10.1080/03081079.2019.1584894.
- Baas, N. 2019b. “On the Mathematics of Higher Structures.” International Journal of General Systems48 (6): 603–624. https://doi.org/10.1080/03081079.2019.1615906.
- Baas, N., A. Ehresmann, and J.-P. Vanbremeersch. 2004. “Hyperstructures and Memory Evolutive Systems.” International Journal of General Systems 33 (5): 553–568. https://doi.org/10.1080/0308107042000193534.
- Bican, L., T. Kepka, and P. Němec. 1982. Rings, Modules and Preradicals. New York: Marcel Dekker.
- Buibas, M., and G. Silva. 2011. “A Framework for Simulating and Estimating the State and Functional Topology of Complex Dynamic Geometric Networks.” Neural Computation 23 (1): 183–214. https://doi.org/10.1162/NECO_a_00065.
- Carlsson, G., and V. de Silva. 2010. “Zigzag Persistence.” Foundations of Computational Mathematics10 (4): 367–405. https://doi.org/10.1007/s10208-010-9066-0.
- Carlsson, G., and A. Zomorodian. 2009. “The Theory of Multidimensional Persistence.” Discrete & Computational Geometry 42 (1): 71–93. https://doi.org/10.1007/s00454-009-9176-0.
- Chambers, E., and D. Letscher. 2018. “Persistent Homology Over Directed Acyclic Graphs.” Research in Computational Topology abs/1407.2523: 11–32. https://doi.org/10.1007/978-3-319-89593-2.
- Dikranjan, D., and B. Giordano. 2013. “Entropy in a Category.” Applied Categorical Structures 21 (1): 67–101. https://doi.org/10.1007/s10485-011-9256-1.
- Dikranjan, D., and B. Giordano. 2016. “Entropy on Abelian Groups.” Advances in Mathematics298:612–653. https://doi.org/10.1016/j.aim.2016.04.020.
- Dikranjan, D., and B. Giordano. 2019. “Entropy on Normed Semigroups (towards a Unifying Approach to Entropy).” Dissertationes Mathematicae 542:1–90. https://doi.org/10.4064/dm791-2-2019.
- Dikranjan, D., B. Goldsmith, L. Salce, and P. Zanardo. 2009. “Algebraic Entropy for Abelian Groups.” Transactions of the American Mathematical Society 361 (7): 3401–3434. https://doi.org/10.1090/S0002-9947-09-04843-0.
- Ehresmann, A., and J. Vanbremeersh. 2007. Memory Evolutive Systems; Hierarchy, Emergence, Cognition. New York, NY: Elsevier Science Inc.
- Fernández-Alonso, R., S. Gavito, and J. Pérez-Terrazas. 2018. “On the Connection Between the Representation Type of An Algebra and Its Lattice of Preradicals.” Communications in Algebra 46 (1): 176–190. https://doi.org/10.1080/00927872.2017.1319478.
- Fong, B. 2012. “Causal Theories: A Categorical Perspective on Bayesian Networks.” Master's thesis, University of Oxford.
- Fong, B., D. Spivak, and R. Tuyéras. 2019. “Backprop as Functor: A Compositional Perspective on Supervised Learning.” In 34th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS), 1–13. Vancouver, BC, Canada.
- Giordano, B., and L. Salce. 2012. “A Soft Introduction to Algebraic Entropy.” Arabian Journal of Mathematics 1 (1): 69–87. https://doi.org/10.1007/s40065-012-0024-3.
- Haruna, T. 2013. “Theory of Interface: Category Theory, Directed Networks and Evolution of Biological Networks.” Bio Systems 114:125–148. https://doi.org/10.1016/j.biosystems.2013.08.002.
- Healy, M. 1999. “Colimits in Memory: Category Theory and Neural Systems.” In IJCNN'99. International Joint Conference on Neural Networks, Washington, DC, USA. Proceedings, Vol. 1, 492–496.
- Horbachuk, L., and B. Yu. 2011. “Preradical and Radical Functors in Different Categories.” Matematychni Studii 35 (2): 121–127.
- Muotri, A., G. Silva, and C. White. 2020. “Understanding the Human Brain Using Brain Organoids and A Structure-Function Theory.” bioRxiv.
- Parada-Mayorga, A., H. Riess, A. Ribeiro, and R. Ghrist. 2020. “Quiver Signal Processing.” arXiv:2010.11525v1.
- Pardo-Guerra, S., H. Rincón, and M. Zorrilla-Noriega. 2020. “Some Isomorphic Big Lattices and Some Properties of Lattice Preradicals.” Journal of Algebra and Its Applications 19 (7): 2050140. https://doi.org/10.1142/S0219498820501406.
- Peters, J. 1970. “Entropy on Discrete Abelian Groups.” Advances in Mathematics 33:1–13. https://doi.org/10.1016/S0001-8708(79)80007-9.
- Raggi, F., J. Ríos, H. Rincón, R. Fernández-Alonso, and C. Signoret. 2002a. “The Lattice Structure of Preradicals.” Communications in Algebra 30 (3): 1533–1544. https://doi.org/10.1080/00927870209342395.
- Raggi, F., J. Ríos, H. Rincón, R. Fernández-Alonso, and C. Signoret. 2002b. “The Lattice Structure of Preradicals II: Partitions.” Journal of Algebra and Its Applications 1 (2): 201–214. https://doi.org/10.1142/S0219498802000136.
- Salce, L., and P. Zanardo. 2009. “A General Notion of Algebraic Entropy and the Rank-Entropy.” Journal Forum Mathematicum 21 (4): 579–599.
- Silva, G. 2019. “The Effect of Signaling Latencies and Node Refractory States on the Dyamics of Networks.” Neural Computation 31 (12): 2492–2522. https://doi.org/10.1162/neco_a_01241.
- Ward, T. 1994. Entropy of Compact Group Automorphisms. Ohio, USA: Ohio State University .
- Weiss, M. 1974. “Algebraic and Other Entropies of Group Endomorphisms.” Mathematical Systems Theory 8 (3): 243–248. https://doi.org/10.1007/BF01762672.
Appendices
Appendix 1.
Preradicals
A preradical on a category can be defined in terms of the 2-category of all endofunctorsFootnote1 of . This 2-category coincides with the functor category whose objects are all endofunctors and whose morphisms are natural transformations, usually denoted by . Within this context, a preradical on is the same as a co-pointed endofunctor of ; this is, an endofunctor for which there is natural transformation to the identity functor on . Observe that the natural transformation gives a family of morphisms -indexed by the objects of - such that, for each and each morphism in , the following diagram commutes
We can also think of a preradical σ on a category as a subfunctor of the identity functor (see Horbachuk and Yu Citation2011). This means that σ is an endofunctor that assigns to each object a subobjectFootnote2 of C such that, for any morphism in , the diagram
commutes. Here, ι denotes the inclusion map, and denotes the restriction and corestriction of f to and , respectively. In particular, when considered the category R-Mod, the subobjects of an R-module correspond to submodules. Thus, we get
Definition A.1
Let R be an associative ring with a unit element. A preradical σ on R-Mod is an endofunctor that assigns to each module , a submodule such that, for each R-morphism , one has the commutative diagram
Here, ι represents the inclusion map, and .
We now describe some properties and definitions of preradicals on R-Mod. For a complete introduction, we refer the reader to Bican, Kepka, and Němec (Citation1982) and Raggi et al. (Citation2002a, Citation2002b). We denote by R-pr the collection of all preradicals on R-Mod. There is a natural partial ordering in R-pr given by if, and only if, for all -Mod. Further, there are four basic operations between preradicals. These are defined as follows: for -Mod and -pr, we have
;
;
;
is the submodule of M satisfying
The above operations are called the meet, the join, the product, and the coproduct, respectively. Notice that the product corresponds to the composition of functor τ followed by functor σ; while the coproduct involves taking a quotient module (induced by σ), applying functor τ, and then use the Correspondence Theorem for Modules to obtain the submodule of M. These four operations satisfy the following condition: given any -pr one has In R-Mod, we define the α and ω preradicals as follows: given -Mod, with , the preradical is such that for every -Mod. Likewise, the preradical is such that where denotes the inverse image of N under the morphism f.
Appendix 2.
Invariants and algebraic entropy
Let M be an R-module, and let be an endomorphism. If L is a subset of M and n is a positive integer, we define the n-th η-trajectory of L as Observe that, when L happens to be a submodule of M, then is a submodule of M. Also, we define the η-trajectory of L as This is also a submodule of M, which we denote by .
Let us now consider a subadditive invariant i on R-Mod. Then, for any submodule L of M with , one has that for every . Indeed, proceeding by induction over n, we first consider the base case n = 2, as n = 1 is trivial. Since , and i is a subadditive invariant, we have that Hence, Assume now that the inductive hypothesis is valid for n>2. Then, as , we get that Furthermore, due to and the inductive hypothesis, we get that We now notice that for every subadditive invariant i, and for any submodule L of M, one has Thus, as i is a subadditive invariant, it follows that where the last relation follows from (b) and the fact that .
Remark A.1
For L a submodule of M with , one has that every endomorphism η of M induces a sequence of positive reals such that .
In the set of real numbers, every sequence of positive numbers with is convergent. Indeed, by mathematical induction, as , we get that for all . This, in turn, implies that the sequence is bounded above by , and below by 0. Therefore, exist, and futher, coincides with , as stated in the next
Proposition A.2
Ward Citation1994, Exercise 6.5
Let be a sequence of positive real numbers such that for all . Then the sequence converges to .
Hence, based on the aforementioned results, one can formally define the algebraic i-entropy on R-Mod for each invariant i.