Category theoretic properties of the A. R\'enyi and C. Tsallis entropies

The problem of embedding the Tsallis and R\'{e}nyi entropies in the framework of category theory and their axiomatic foundation is studied. To this end, we construct a special category MES related to measured spaces. We prove that both of the R\'{e}nyi and Tsallis entropies can be imbedded in the formalism of category theory by proving that the same basic functional that appears in their definitions, as well as in the associated Lebesgue space norms, has good algebraic compatibility properties. We prove that this functional is both additive and multiplicative with respect to the direct product and the disjoint sum (the coproduct) in the category MES, so it is a natural candidate for the measure of information or uncertainty. We prove that the category MES can be extended to monoidal category, both with respect to the direct product as well as to the coproduct. The basic axioms of the original R\'{e}nyi entropy theory are generalized and reformulated in the framework of category MES and we prove that these axioms foresee the existence of an universal exponent having the same values for all the objects of the category MES. In addition, this universal exponent is the parameter, which appears in the definition of the Tsallis and R\'{e}nyi entropies.


Introduction
The discovery of two related generalizations of the classical Shannon entropy [1] is a remarkable coincidence in the history of abstract probability theory and statistical physics.A. Rényi introduced a possible generalization [2] of the classical Shannon entropy by pure axiomatic extension of the Fadeev axioms [3], [4] that define uniquely the Shannon entropy.On the other hand, the generalized entropy [5], [6] introduced by C. Tsallis was useful to extend the classical maximum entropy principle such that the heavy tailed distributions observed in a large scale of physical processes [7], [8], [9], [10], could be derived from (generalized) maximum entropy principles.The interest in the study of the generalizations of the Shannon entropy in the recent years is due to the multiple applications of the Tsallis and Rényi entropy or the associated Rényi divergence [7], [8], [11], [12].We mention also that similar to the classical H theorem of L. Boltzmann, the generalizations of the Rényi entropy, as well as the original Rényi entropy, is a Liapunov functional for a large class of stochastic processes described by generalized Fokker-Planck equations, more exactly by Fokker-Planck equation where the drift term and the diffusion tensor are itself dependent on some external random variable [13].We mention that in the case of suitable singular limiting procedure, both the Tsallis and Rényi entropies give the same limit: the Shannon entropy.The Rényi entropy is additive while the Tsallis entropy is not.Despite the Rényi and Tsallis entropies give the same results in the case of problems associated to the determination of the probability density function from the Maximum Entropy principles, because they are algebraically related by simple formulae, the non-additivity of the Tsallis entropy generated many discussions in the physical literature.On the other hand, by formulating the basic axioms [2], A. Rényi introduced new concepts (incomplete random variables and incomplete distributions) that are not included in the standard terminology of the probability theory.Also the formulation of the Postulate 5' [2], is not the simplest, mathematically natural.In this work we develop a formalism in the framework of the category theory [14] and [15] for the study of generalized entropies.The category theory is the branch of mathematics that plays a central role in the logical foundation and synthesis of the whole contemporary mathematics.In particular, the category theory allows avoiding the paradoxes of the classical set theory.In order to highlight the natural structures related to generalized entropies, we use the central concepts of the modern mathematics.The paper is organized as follows.In the Section 2, Subsection 2.1, we define a special category related to measurable spaces (referred to as M ES), enabling the introduction an associated basic functional Z p (see the forthcoming Section for his exact definition).Both the Tsallis and Rényi entropies, as well as the distance in L p spaces, may be expressed in terms of this functional.In the Subsection 2.2, we define the direct product of the objects in M ES and we prove that the functional Z p satisfies a compatibility relation with respect to this product i.e., it is multiplicative.This multiplicative property is equivalent to the additivity of the Rényi entropy.In the Subsection 2.3 we define the disjoint sum (or the coproduct) of the objects in M ES, and we prove that the functional Z p satisfies a compatibility relation with respect to coproduct i.e., it is additive.Note that this property is equivalent to one of the postulates characterizing the Rényi entropy.The prove that both product and coproduct possess a universal property and that the direct product and coproduct can also be defined for morphisms of the category M ES, can be found in the Subsection 2.4.In the Subsection 2.5 we show that, by extending the category M ES with the introduction of the unit object and the null object, the category M ES became to a monoidal category.Section 3 deals with the axiomatic characterization of the functional Z p .We demonstrate that there exists a universal exponent p (the same for all the objects of the category) that characterizes completely the functional Z p (hence, also the Tsallis or Rényi entropies) up to an arbitrary multiplicative factor.Appendix 5.1 shows that the Rényi divergence can be expressed in terms of the Rényi entropy.The proof of the universality (with respect to all the objects of the category M ES) of the exponent defining the Rényi or Tsallis entropies can be found in Appendix 5.2.

The category-theoretic properties related to
Rényi and Tsallis entropies.

Definitions
Our definitions include as a particular case the original definition of the generalized entropies [5], [6] and [2].Our basic construction that will play the role of the object of the category MES is derived from the well known concept of measurable space [16], [17].Guided by statistical ideas, in order to take into account the negligible sets we specify also an sub-ideal of the σ-algebra of measurable sets.The objects of the category M ES consist of triplets M X := (X, A X , N X ) with X denoting the phase space (for instance, it is a symplectic manifold in the case of statistical physics or, in the case of elementary probability models, finite or denumerable set) and A X is the σ−algebra generated by a family of subsets of X, respectively.We also denote with N X ⊂ A X an ideal of the σ-algebra A X having the meaning of negligible sets.Let us now postulate the completeness property.From N ∈ N X and N ′ ⊂ N results N ′ ∈ N X .The morphisms of the category M ES with the source M X and range M Y are the measurable maps Φ from X to Y , which are nonsingular i.e., such that Φ −1 (N Y ) ⊂ N X .From the completeness property results the ideal property i.e., if N ∈ N X and A ∈ A X then A ∩ N ∈ N X .Note that it is possible that N X contains only the empty set (as, for example, in the case of atomic spaces).
Remark 1 At first sight it would be more natural to consider the objects as measure space triplet (X, A X , µ X ) containing the measure µ X , and the morphisms as the measure preserving transformations.However, in this case we cannot define direct product or coproduct having universal property.
We denote with C(M X ), or with C(X, A X , N X ), the cone with all σ− finite positive measures over (X, A X , N X ) that are compatible with N X (i.e., µ ∈ C(X, A X , N X ) iff for all N ∈ N X we have µ(N ) = 0).For a given µ X ∈ C(X, A X , N X ) and p > 0, we denote with L p (M X , µ X ) the Banach space (p ≥ 1) or the Fréchet space (0 < p < 1) of functions f X : X → R that are measurable modulo N X and have finite norm (pseudo norm, respectively): more precisely, In the sequel, we shall denote for some non-negative density ρ X ∈ L p (M X , µ X ).The generalized entropies are defined for probability density functions (PDF) satisfying the conditions where p > 0 and p = 1.The probability P (A) can be represented by PDF as follows In this framework, for a given measurable space M X := (X, A X , N X ) and measure µ X ∈ C(M X ), the classical Boltzmann-Gibbs-Shannon entropy functional is given by For a given measurable space M X , the generalizations of the A. Rényi [2] and C. Tsallis [5], [6] entropies, involves the functional Z p (M X , µ X , ρ X ) given by Eq.( 1).The functional Z p is related to the norm of the density ρ in the Banach space for p ≥ 1 [17], and to the pseudo-norm N p [ρ] for 0 < p < 1 [16] [18], through the obvious relations These relations give the geometrical interpretation of the generalized entropies (for further information Refs to [13]).

Remark 2
The study of the generalized entropies helps us to better understand the classical entropy.For p ≥ 1, the functional ρ X p is the classical L p norm, and for 0 < p < 1 the functional N p [ρ X ] is the exotic L p -norm [18].For p > 1 the L p spaces are reflexive, the Maxent problem is equivalent to the minimal L p distance problem with restrictions [13], or to the minimal Z p (M X , µ X , ρ X ).For 0 < p < 1, the L p spaces has, in general, trivial duals, the Maxent problem is equivalent to the maximal L p distance or the maximal Z p (M X , µ X , ρ X ) (see [13]).The case p = 1, which corresponds to the classical Shannon entropy, is just the border point between two radically different functional-analytic properties.
The corresponding generalized entropy S R,p , proposed by A. Rényi [2], and the entropy, S T,p , proposed by C. Tsallis [5], [6] are given by Consider now a measure space N = (Ω, A, n) with σ-finite measure n, and let us denote with P (x), Q(x) two probability densities: is related to the Rényi entropies, Eq.( 9), by Eq.(48) (see Appendix 5.1).Note that when X is a finite or denumerable set, if we denote with p k the probabilities of element x k ∈ X, the measure µ X is the counting measure on the space X (equal to the number of elements in a subset), and the family of null sets N X = {∅} then, from the previous Eqs.( 9, 10, 1) we get the original definitions from Ref. [2], [5], [6] Remark that, in this particular case, S T,q [M X , µ X , ρ X ], as well as Z q (M X , µ X , ρ X ), are Lesche stable [19].Note that, from Eqs (6, 9 and 10), results 2.2 Direct product of measurable spaces and the multi- In the framework of the our formalism, the multiplicative property is the counterpart of the Postulate 4 in the Rényi theory [2].In the following we overload the tensor product notation " ⊗ "; its meaning results from the nature of the operand.Denote the direct product of two measurable spaces Here X × Y is the Cartesian product of the phase spaces X and Y , while the σ-algebra A X ⊗ A Y is the smallest σ-algebra such that it contains all of the elements of the Cartesian product (we denote it also by the same symbol).The measure µ X ⊗ µ Y acting on (A X ⊗ A Y )/N X⊗Y are defined by extension by denumerable additivity, starting from the product subsets: Consider now the measures µ X ∈ C(M X ), µ Y ∈ C(M Y ), and the densities The following function is also denoted with the same symbol with We have the following basic proposition Then we have The validity of this statement follows directly from the definitions of the direct product, the Rényi entropy and the functional Z p .

Coproduct of measurable spaces and the additivity of the functional
Let us study now the property encoded in the Postulate 5' related to the Rényi entropy theory (Ref.[2]), transcribed in the measure theoretic and category language and re -expressed in the term of the functional Z p [M X , µ X , ρ X ].Also in this case, we overload the notation ⊔, for the disjoint sum from the set theory.Its precise meaning will be clear from the nature of the operands.In the following we investigate the functorial properties, related to Postulate 5', of the functional Z p [M X , µ X , ρ], in analogy to Proposition 3. To this end we introduce the following Definition 4 The coproduct of measurable spaces M X = (X, A X , N X ) and M Y = (Y, A Y , N Y ) will be denoted by M X ⊔M Y and have the following structure Here, X ⊔Y is the disjoint sum of the sets X and Y , and A X ⊔A Y is the smallest σ-algebra that contains all of the sets of the form A 1 ⊔ A 2 , with A 1 ∈ A X and A 2 ∈ A Y , respectively.Moreover, the new null set ideal N X ⊔N Y is the smallest σ-algebra generated by the family ) and the weights w 1 ≥ 0, w 2 ≥ 0 and w 1 + w 2 = 1.The measure µ := w 1 µ X ⊔ w 2 µ Y acts on the σ -algebra A X ⊔ A Y and it is defined uniquely as the continuation by denumerable additivity from the property We restrict our definition of coproduct to finite terms.An example of (denumerable infinite) coproduct is the grand canonical ensemble.
Remark 5 If ρ X dµ X and ρ Y dµ Y are probability measures, then the measure From the previous definition of the direct sum and the functional Z p [M X , µ X , ρ X ] the following obvious proposition results

Proposition 6
The reformulation of the Postulate 5' (Ref.[2]) reads: the functional Z p [M X , µ X , ρ X ] is additive with respect to the direct sum of measurable spaces

Universal properties of the direct product and direct sum in the category of measurable spaces
In the following we prove that the basic binary operations on measurable spaces, the direct product and the direct sum, defined in the previous section, have universality properties in the category of measurable spaces M ES.
We have the following

Proposition 7
In the category M ES the applications π X : Proof.The measurability of π X is direct consequence of the fact that the canonical projection maps are measurable, in fact the measurability of the canonical projections is an alternative definition of the product of σ algebras.The nonsingularity property p From the previous Proposition 7 results immediately the following Theorem Theorem 8 In the category M ES, the direct product has the universal property.Let M X = (X, A X , N X ).M Y = (Y, A Y , N Y ) and M = (Z, A Z , N Z ) measurable spaces that are objects of the category M ES, such that there exists morphisms φ X ∈ Hom(M, M X ) and φ Y ∈ Hom(M, M Y ).Then there exists an unique morphism θ ∈ Hom(M, M X ⊗ M Y ) such that where π X , π Y are the morphism defined in Proposition 7 .
Proof.The morphism θ is induced by the application and it is unique.In order to prove that θ is a morphism we have to prove that T is measurable and it is nonsingular.
To prove that T : Z → X × Y is measurable, we recall that it is sufficient to prove that, for all A ∈ A X , B ∈ A Y , we have the property T (−1) (A × B) ∈ A Z , a property resulting from the measurability of φ X and φ Y .Note that to prove the inclusion T −1 (N X×Y ) ⊂ N Z , it is sufficient to demonstrate for the generating subsets T −1 (N X × A Y ) ⊂ N Z (which follows from the nonsingularity of φ X and In conclusion the direct product operation has the natural functorial property, so the multiplicative property Eq. ( 23) of the functional Z p (M X , µ X , ρ X ) appears as an algebraic compatibility property.By simple reversal of the arrows, we are lead to the corresponding universality property of the coproduct in the category M ES.We have the following obvious proposition Proposition 9 In the category M ES, consider the objects M X , M Y .The applications ι X : so i X and i Y are nonsingular, which completes the proof that ι X , ι Y are morphisms in the category M ES.By reversing the arrows, in analogy to the Theorem 8, we obtain the following result Theorem 10 In the category M ES the direct sum of the objects has the following universality property.Let denote with spaces that are objects of the category M ES, such that there exists morphisms φ X ∈ Hom(M X , M ) and φ Y ∈ Hom(M Y , M ).Then, there exists an unique morphism γ where ι X ,ι X are the morphisms defined in Proposition 9. Proof.The morphism γ is induced by the map g : X ⊔ Y → Z defined as follows.If x ∈ X then g(x) := φ X (x) ∈ Z, and in the case x ∈ Y , then g(x) := φ Y (x) ∈ Z.The measurability of the map g results from the measurability of φ X and φ Y .The inclusion g (−1) (N Z ) ⊂ N X ⊔ N Y results from the nonsingularity of φ X and φ Y .
In conclusion, the direct sum operation has natural category theoretic properties.He,Hence, the additivity property Eq.( 28) of the functional Z p (M X , µ X , ρ X ) is not an artificial construction.

The monoidal categories associated to product and coproduct
We recall the following Proposition 11 [15] Let C be a category such that for all objects A, B ∈ Ob(C) exists their direct product A ⊗ B, having the universal property.Then, there exists a covariant functor F from the product category to C, C× C → C ,defined as follows.For the object (A, B) of C×C, where A, B are objects of C, we have For the pair of morphisms (u, v) ∈ Hom((A, B), (A ′ , B ′ )) with u ∈ Hom(A, A ′ ), v ∈ Hom(B, B ′ ), from the category C × C there exists an unique morphism w in the category C, w ∈ Hom(A ⊗ B, A ′ ⊗ B ′ ) uniquely fixed by the conditions We denoted with p A , p B the projections from Hom(A ⊗ B, A), Hom(A ⊗ B, B), and p A ′ are p B ′ the projections from Hom(A ′ ⊗ B ′ , A ′ ), Hom(A ′ ⊗ B ′ , B ′ ).The map (u, v) → F ((u, v)) has the functorial property.Let (u, v) ∈ Hom((A, B), (A ′ , B ′ )) and (u ′ , v ′ ) ∈ Hom((A ′ , B ′ ), (A", B")).Then, If in the category C we have an unit object, then C is a monoidal category.
Similarly, by duality arguments, we have the following result for the direct sum (coproduct) Proposition 12 [15] Let C be a category such that for all objects A, B from Ob( C) exists their direct sum A ⊔ B, having the universal property.Then, there exists a covariant functor G from the product category C×C →C defined as follows.
We denoted with i A , i B the canonical injections from Hom(A, A⊔B), Hom(B, A⊔ B), and with i A ′ , i B ′ the injections from The association (u, v) → G((u, v)) has the functorial property.Let (u, v) ∈ Hom((A, B), (A ′ , B ′ )) and (u ′ , v ′ ) ∈ Hom((A ′ , B ′ ), (A", B")) then, If in the category C we have a null object then, C is a monoidal category with respect to direct sum.
We emphasize that, despite the fact that the construction of the direct sum is dual to the direct product, from the previous proposition ( 12) the functor G is a covariant functor.In the category M ES we have an unit object as well as the null object.The unit object is denoted with , where 1 is the one point set [15], A 1 is the trivial σ-algebra consisting in the one point set 1, ∅, and N 1 = {∅}, The (more or less formal) null object M 0 , with respect to the direct sum, is the object generated by the empty set M 0 := (∅, A ∅ , N ∅ ).So we have the following

Conclusion 13
The category M ES is a monoidal category both with respect to the product ⊗ and the coproduct ⊔.

Axioms
We expose another approach, based on category theory, to the problem of the naturalness of the choice of the family of functions g α used in the definition of the entropy [2].We prove that this problem may be treated if we take into account the additivity and the multiplicative properties of the functional Z p .We mention that a possible candidate for the generalization of the symmetry Postulate 1 [2] is the requirement of invariance of the generalized entropy under measure preserving transformations.Recall that the group generated by finite permutations is the maximal measure preserving group with respect to the counting measure.The problem is that there are plenty of measures such that the measure preserving group is trivial (for instance, the atomic measure for 2 element set with µ(1) = µ(2)).To avoid this problem, we observe that Postulate 1 and Postulate 5' in the original Rényi theory [2] can be generalized as follows.For a given measurable function f (x) on the measured space M X and µ ∈ C(M X ), let us define Note that m f (M X , µ, t) is invariant under measure preserving transformations.In addition Then, the Postulate 1 (the symmetry property) and Postulate 5' (the additivity property expressed in Proposition 6) can be generalized as follows.
for some Borel measurable function h X (t) with The last requirement result by considering the case when the support of ρ X is concentrated on a proper subset of X and by using Eq.( 28) .The generalization of the Postulate 2 (the continuity property) is straightforward.Be h(x) continuos and ρ In our settings, the analog of the Postulate 4 (the additivity property) [2] is the multiplicative property given by Eq.( 23) and Proposition 3. By using Eqs.(23, 33, 35 and 36), and by continuity of the functions h XY , h X , h Y for all x, y ≥ 0, we obtain the following functional equation (valid almost everywhere) h XY (x y) = h X (x)h Y (y); x, y ∈ R (37) By arguments similar to the proof of the uniqueness, from Theorem 2 [2]), we get Eq.(32) (for details see the Appendix 5.2): there exists an universal family of functions, independent of X , parametrized by the positive parameter p such that

Summary and conclusions
We proved that the most natural setting for treating the axiomatic approach to the study of definitions of measures of information or uncertainty, is the formalism of the category theory, that was invented for the most difficult, apparently contradictory aspects of the foundation of mathematics.In this respect we introduced a category of measurable spaces M ES.We proved that in the category M ES exist the direct product and the direct sum, having universal properties.We proved that the functional Z p (M X , µ X , ρ X ) defined in Eq.( 1), which appears in the definition of both Rényi and Tsallis entropies, has algebraic compatibility properties with respect to direct product and direct sum, as shown in Eqs ( 23) and ( 28).
The main conclusions may be summarized as follows.
(1) The natural measure of the quantity of information is the family of functionals Z p (M X , µ X , ρ X ) given by Eq. ( 1), (defined in the Fréchet space for 0 < p < 1, and in the Banach space for p > 1), and the classical Shannon entropy by Eq. ( 6); (2)The category M ES is the natural framework for treating the problems related to the measure of the information, in particular in reformulating the Rényi axioms; (3) The category M ES is a monoidal category with respect to direct product and coproduct and the functional Z p (M X , µ X , ρ X ) has natural compatibility properties with respect to the product (it is multiplicative) and the coproduct (it is additive); (4) Up to a multiplicative constant, it is possible to recover the exact form of the functional Z p (M X , µ X , ρ X ) defining the generalized entropies from a system of axioms that generalize the ones adopted by Rényi [2].
For the object (A, B) of C× C, where A, B are objects of C we have G((A, B)) := A ⊔ B For the pair of morphisms (u, v) ∈ Hom((A, B), (A ′ , B ′ )), with u ∈ Hom(A, A ′ ) and v ∈ Hom(B, B ′ ), from the category C × C there exists an unique morphism w in the category C, w ∈ Hom(A ⊔ B, A ′ ⊔ B ′ ) uniquely fixed by the conditions w = G((u, v))