An Exceptional Generalization of the Poisson Distribution

A new two-parameter count distribution is derived starting with probabilistic arguments around the gamma function and the digamma function. This model is a generalization of the Poisson model with a noteworthy assortment of qualities. For example, the mean is the main model parameter; any possible non-trivial variance or zero probability can be attained by changing the other model parameter; and all distributions are visually natural-shaped. Thus, exact modeling to any degree of over/under-dispersion or zero-inflation/deflation is possible.


Introduction and the Main Result
In count data modeling the Poisson distribution is usually the first option, but real data can indicate a variety of discrepancies.These can be genuine features or secondary consequences of e.g.censoring, clustering, approximations or correlations.Specifically, the Poisson model has no dispersion flexibility because the mean determines the variance and the zero probability, σ 2 = μ, p 0 = e -μ , while the real data can display over or under-dispersion, σ 2 ≠ μ, or zero-inflation or deflation, p 0 ≠ e -μ [1].Such situations are usually handled e.g. by randomizing the Poisson mean, by mixtures, by adding a new parameter, by reweighing the Poisson point probabilities, or via generalizing the exponential increments in the homogeneous Poisson process [2][3][4][5].Our approach will be different.
We recall an elementary fact.The mean-deviation pair (μ, σ) of a non-binary count variable (non-negative integer-valued random variable) always satisfies the inequality where [μ] is the largest integer not exceeding μ.Thus, we will say that a count model (parameterized count variable) has full dispersion flexibility if every positive solution (μ, σ) of the inequality (1) is the mean-deviation pair for some parameter values.

2) Generalization of the Poisson model:
3) Full dispersion flexibility: If the numbers μ > 0 and σ > 0 satisfy inequality (1), then there is a β such that

 
The solution to be presented in this paper obeys the following cumulative probabilities: where g(t, x) and G(t, x) are the one-parameter gamma probability and cumulative distribution functions, respectively, with parameter x and variable t (Section 2).We begin with the derivation of fundamental inequalities in Section 2. These inequalities lead to a cumulative distribution H(x, μ), where the parameter μ > 0 is the mean.Then the insertion of a new independent parameter β > 0 provides an extended cumulative distribution H(x/β, μ/β) and the related non-negative two-parameter random variable X(μ, β), where μ is still the mean.Now the proclaimed count model N(μ, β) is defined as a mean-preserving discretization of X(μ, β), and the above properties 1), 2), 3) are proved.Thereafter the most immediate applications are given; namely, exact modeling of over/ under-dispersion or zero-inflation/deflation to any possible degree.In the last section, we propose motives for further research, and we compare N(μ, β) with well-established Poisson generalizations.
There is a nice probabilistic perspective on the gamma function: If the random variable T has a gamma density g(t, x), then E(ln(T)) = Ψ(x) and Var(ln(T)) = dΨ(x)/dx [7].In terms of our notation above, these simple observations can be written in the form . ( Additional work leads to a stronger result, Namely, integration by parts, the functional equations , and l'Hospital's rule allow us to write Next we derive two fundamental inequalities.For every fixed x > 0, the function a(t, x) has exactly one root , and it is increasing there.This and (3, left side) imply Now, taking into account ( 5) and (4, left side), we obtain the first inequality Further, for every fixed x > 0, the function b(t, x) has exactly two roots, 0 , and it is decreasing at t 0 and increasing at t 1 .From this one can conclude that B(t, x) has, for every x > 0, a positive local maximum at t 0 and, because of (3, right side), a negative local minimum at t 1 .Considering (4, right side) too, we finally arrive at the second inequality

A Mean-Preserving Discretization
We will also need a certain discretization procedure: If X is a non-negative random variable with cumulative distribution F(x), the discretization of X is a count variable N with cumulative probabilities equal to the mean F(x) on the interval (n, n + 1), i.e. (8) We shortly quote the basic properties from [6]: The mean and the variance of N exist (are finite) if and only if the mean and the variance of X exist, and in that case

A Generalization of the Poisson Model
In our construction of a new generalization of the Poisson model, the following one-parameter function will be the central ingredient: Recalling ( 5) and the notation A(t, x) = ∂G(t, x)/∂x from Section 2, we derive .
In (12) we first changed the integration order (as the integrand is positive) and then employed the limits The limits (13) follow from Chebyshev's inequality and the simple fact that the parameter x of the one-parameter gamma density g(t, x) equals the mean and the variance.
Proof of (20).Definition (11) and partial integration yield the identity The first term on the right side vanishes when M→∞, since MG(t, M) ≤ t M /Г(M).Now by changing the integration order in the latter term, one obtains where Then, by using (21) and part of (18), and changing integration variable, z = βt, one arrives at Further, the inequality s , s > 0, x > 0, yields a lower bound for L(s): This means that L(s) tends to ∞ as s→0, and so the average of L in the interval (0, z/β) approaches ∞ as β→∞ (22).Thereby, E(X(μ, β) 2 ) grows to ∞, so (17) and (10, left side) complete the proof of (20).

Computing and Applications
When working with N(μ, β), the following numbers are useful: The latter faster version follows from partial integration and the identities G(t, x) -G(t, x + 1) = g(t, x + 1), G(t, 0) = 1 (13a).Note also that most mathematical software offers fast computation of G(t, x).Employing (23) in ( 16), basic formulas can be written in the following form: We consider exact modeling of count variables.(For numerical examples, see Table 1).
Application 1. Generally, a non-binary count variable with desired mean μ and variance σ 2 exists if and only if In that case N(μ, β) always provides a solution.Indeed, because of full dispersion flexibility, Property 3), there Again N(μ, β) provides a solution.Arguments like those in Section 5 would show that there is a β > 0 such that = p 0 (24, n = 0).Application 3. Suppose there is a real non-censored random sample available of the unknown non-binary count variable to be modeled.Let  be the sample mean, 2   the standard variance and 0 the zero fraction.It is easy to prove that these UMVU estimates also meet (27, 28).Thus, there is a β 1 that satisfies p .Importance weighing provides a compromise β and an approximate solution    .

Further Research and Discussion
Additional work is needed to enlarge the applicability of N(μ, β).The computational behavior of the central formulas 23-26 should be further explored, and tools for stochastic simulation and statistical inference should be developed.We put forward two concrete problems.Problem 1. Numerical experimentation indicates that the numbers K n (23, n ≥ 1) increase with β (K 0 = μ).If this is true, all moments (25, k ≥ 2) increase with β, so the iteration of β in the applications in Section 6 can be made faster.
Problem 2. Find an algorithm for generation of random variates from N(μ, β).The alias method [8] can of course be used for truncated versions, but a tailor-made method would be welcome.Actually, a generation method for X(μ, β) would be enough since, according to [6], this can immediately be transformed to the discretization N(μ, β).
. Conversely, if (μ, σ) is a positive solution of (1), then it is the mean-deviation pair of a non-binary count variable; and as we have shown, there is always an N(μ, β) with this mean-deviation pair.Since the mean is an original model parameter of N(μ, β), only β needs to be solved from the equation Var(N(μ, β)) = σ 2 .We have called this feature "full dispersion flexibility", because it enables exact modeling for the first two moments, or for mean and zero probability.Full dispersion flexibility seems to be very rare even among well-established Poisson generalizations.The generalization of Consul and Jain [2], the negative binomial [3], the COM-Poisson distribution [4] and many others have severe shortcomings in dispersion flexibility, and also partly bad-shaped distribution functions.A positive exception is the General Poisson Law [5].However, here the mean is not a model parameter, so, if a certain pair (μ, σ) is wanted, the original parameters must be solved simultaneously from two equations, which both include laborious infinite series'.
Also note that the invariants (4) and ( 5), the inequalities ( 6) and (7), and the distribution (11) comprise, as such, a contribution to probabilistic treatment of the gamma function.

2  and a β 2
that satisfies 0 (both exactly), but of course, usu-