Optimum Probability Distribution for Minimum Redundancy of Source Coding


In the present communication, we have obtained the optimum probability distribution with which the messages should be delivered so that the average redundancy of the source is minimized. Here, we have taken the case of various generalized mean codeword lengths. Moreover, the upper bound to these codeword lengths has been found for the case of Huffman encoding.

Share and Cite:

O. Parkash and P. Kakkar, "Optimum Probability Distribution for Minimum Redundancy of Source Coding," Applied Mathematics, Vol. 5 No. 1, 2014, pp. 96-105. doi: 10.4236/am.2014.51011.

Conflicts of Interest

The authors declare no conflicts of interest.


[1] C. E. Shannon, “A Mathematical Theory of Communication,” Bell System Technical Journal, Vol. 27, 1948, pp. 379-423 (Part I) 623-656 (Part II).
[2] L. G. Kraft, “A Device for Quantizing Grouping and Coding Amplitude Modulated Pulses,” M.S. Thesis, MIT, Cambridge, 1949.
[3] L. L. Campbell, “A Coding Theorem and Renyi’s Entropy,” Information and Control, Vol. 8, No. 4, 1965, pp. 423-429.
[4] J. N. Kapur, “Entropy and Coding,” Mathematical Sciences Trust Society (MSTS), New Delhi, 1998.
[5] A. Renyi, “On Measures of Entropy and Information,” Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1, 1961, pp. 547-561.
[6] O. Parkash and P. Kakkar, “Development of Two New Mean Codeword Lengths,” Information Sciences, Vol. 207, 2012, pp. 90-97.
[7] D. Harte, “Multifractals: Theory and Applications,” Chapman and Hall, London, 2001.
[8] J. F. Bercher, “Source Coding with Escort Distributions and Renyi Entropy Bounds,” Physics Letters A, Vol. 373, No. 36, 2009, 3235-3238.
[9] J. F. Bercher, “Tsallis Distribution as a Standard Maximum Entropy Solution with ‘Tail’ Constraint,” Physics Letters A, Vol. 372, No. 35, 2008, pp. 5657-5659.
[10] C. Beck and F. Schloegl, “Thermodynamics of Chaotic Systems,” Cambridge University Press, Cambridge, 1993.
[11] D. A. Huffman, “A Method for the Construction of Minimum Redundancy Codes,” Proceedings of the Institute of Radio Engineers, Vol. 40, No. 10, 1952, pp. 1098-1101.

Copyright © 2023 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.