^{1}

^{*}

^{1}

^{*}

In the present communication, we have obtained the optimum probability distribution with which the messages should be delivered so that the average redundancy of the source is minimized. Here, we have taken the case of various generalized mean codeword lengths. Moreover, the upper bound to these codeword lengths has been found for the case of Huffman encoding.

Any message that brings a specification in a problem which involves a certain degree of uncertainty is called information and it was Shannon [

then there exists a uniquely decodable code with these lengths, which means that any sequence

where

Later, Campbell [

and

respectively, where

is Campbell’s [

is Kapur’s [

is Renyi’s [

Recently, Parkash and Kakkar [

and

Further, the authors provided two source coding theorems which show that for all uniquely decipherable codes, the mean codeword lengths

and

respectively where

is a Kapur’s [

is measure of entropy developed by Parkash and Kakkar [

This is to emphasize that in the entire literature of source coding theorems, one can observe that the mean codeword length is lower bounded by the entropy of the source and it can never be less than the entropy of the source but can be made closer to it. This phenomenon provides the idea of absolute redundancy which is the number of bits used to transmit a message minus the number of bits of actual information in the message, that is, the mean codeword length minus the entropy of the source. The objective of the present communication is to minimize this redundancy in order to increase the efficiency of the source encoding. For this purpose we have made use of the concept of escort distribution as follows:

If

where

The aim of the present paper is to obtain the optimum probability distribution with which the source should deliver messages in order to minimize the absolute redundancy. To obtain our goal, we have taken into consideration the above mentioned generalized mean codeword lengths. Moreover, the upper bound to these codeword lengths has been found for Huffman [

Let us assume that for discrete source

where

In order to minimize the average redundancy, we resort to the following theorem:

Theorem 1: The optimum probability distribution that minimizes the absolute redundancy

Proof: To minimize the redundancy, we need to minimize

subject to the constraint

To prove this, we first of all, find the extremum of

So, in order to extremize

where

Now

Letting

Substituting (2.6) in (2.4), we get

Substituting (2.7) in (2.6), we get the result (2.2).

Now,

We see that

Also,

So,

Thus,

Thus, the minimum value is given by

Again, the necessary condition for the construction of uniquely decipherable codes is given by

Therefore, from (2.9), we have

NOTE: It is to be noted that

Therefore, for this case, (2.2) becomes

Similarly, if we consider the codeword length

where

Theorem 2. The optimum probability distribution that minimizes the absolute redundancy

Proof: We will find the extremum of

Let us consider the Lagrangian given by

where

For an extremum, let

Using (2.14), we get

Substituting (2.17) in (2.16), we get (2.13).

Also,

and

So,

that is,

Note: Again in this case also, if the source is Huffman [

Next, we will find the upper bound on the codeword lengths

Theorem 3. The exponentiated codeword length

if the source is encoded using Huffman procedure.

Proof: The exponentiated codeword length

where

Considering (2.12), (2.19) becomes

where

We need to find the extremum of

For this purpose, we first of all, find the extremum of

So, we consider the Lagrangian given by

where

Letting

Now,

Using (2.23) in (2.22), we get

that is,

Now,

We see that

Also,

So,

Therefore,

Thus, the maximum value is given by

Theorem 4. The mean codeword length

if the source is encoded using Huffman procedure.

Proof: The exponentiated codeword length

We need to find the extremum of

So, we consider the Lagrangian given by

where

Letting

Since

Substitute (2.28) in (2.27), we get

Now,

Also,

So, the mean codeword length

Note-I: For the case of Campbell’s codeword length

where

The absolute redundancy in the case of Campbell’s [

Note-II: Absolute redundancy when we use Kapur’s[

where

Theorem 5: The optimum probability distribution that minimizes the absolute redundancy of the source with entropy

Proof: To minimize the redundancy, we need to minimize

subject to the constraint

To prove this, we first of all find the extremum of

So, in order to extremize

where

Letting

Substituting (2.33) in (2.31), we get

Substituting (2.34) in (2.33), we get the result (2.29).

Now,

We see that

Also,

So,

Therefore,

The minimum value is given by

Theorem 6. The Kapur’s [

if the source is encoded using Huffman procedure.

Proof: Proceeding as in Theorem 2.3, we can prove the Theorem 6.

The authors are thankful to Council of Scientific and Industrial Research, New Delhi, for providing the financial assistance for the preparation of the manuscript.