Near-Lossless Compression Based on a Full Range Gaussian Markov Random Field Model for 2 D Monochrome Images

This paper proposes a Full Range Gaussian Markov Random Field (FRGMRF) model for monochrome image compression, where images are assumed to be Gaussian Markov Random Field. The parameters of the model are estimated based on Bayesian approach. The advantage of the proposed model is that it adapts itself according to the nature of the data (image) because it has infinite structure with a finite number of parameters, and so completely avoids the problem of order determination. The proposed model is fitted to reconstruct the image with the use of estimated parameters and seed values. The residual image is computed from the original and the reconstructed images. The proposed FRGMRF model is redefined as an error model to compress the residual image to obtain better quality of the reconstructed image. The parameters of the error model are estimated by employing the Metropolis-Hastings (M-H) algorithm. Then, the error model is fitted to reconstruct the compressed residual image. The Arithmetic coding is employed on seed values, average of the residuals and the model coefficients of both the input and residual images to achieve higher compression ratio. Different types of textured and structured images are considered for experiment to illustrate the efficiency of the proposed model. The results obtained by the FRGMRF model are compared to the JPEG2000. The proposed approach yields higher compression ratio than the JPEG whereas it produces Peak Signal to Noise Ratio (PSNR) with little higher than the JPEG, which is negligible.


Introduction
Image content analysis is an important research issue in computer vision because applications such as multimedia, image retrieval through Internet, TV broadcast, and storage management etc. require high speed transmission of data and higher compression ratio with high quality.To fulfil these requirements, the size of the data needs to be compressed.Usually, image compression techniques are categorized into lossy and lossless.The lossy compression techniques concentrate on high compression ratio whereas the lossless compression techniques concentrate on high quality.Both the techniques have disadvantage: The lossy does not maintain high quality while the lossless does not maintain high compression ratio.Most of the image compression techniques come under either dictionary-based schemes or statistical-based approaches.The former is computational and time consuming when compared to the latter.The lossy compression technique is developed based on Discrete Cosine Transform (DCT) and on entropy coding while the lossless compression technique uses predictive coding technique, which also follows entropy coding.The lossless JPEG2000 or JPEG-LS is the current standard for lossless image compression [1][2][3].The JPEG2000 is developed based on wavelet transform, scalar or trellis coded quantizer, and arithmetic coder.In JPEG2000, the input image is transformed by wavelet transform and the transform coefficients are put into two groups.Each group consists of blocks in the wavelet domain.The transform coefficients are quantized with the help of scalar or trellis coded quantizer, and then the arithmetic coder is applied to code the coefficients.
Hewlett-Packard developed a coding scheme LOCO-I [4] during the standardization process, which is the base for the current standard of the lossless compression.LOCO-I utilises the features of simple fixed context model and complex context model for capturing higherorder dependencies, which attains compression ratios similar or superior to that of state-of-the-art schemes based on arithmetic coding.Memon and Wu [5] proposed a scheme, CALIC, which is presented as a competitor for the standard; it gives similar or better performance at the increased computational complexity when compared to that of LOCO-I.These schemes are based on the same idea of predicting a pixel value on the basis of the values of adjacent pixels.The simplicitydriven schemes like JPEG standard have some limitations in their compression performance by the first order entropy of the prediction residuals, which do not achieve the total decorrelation of the data [6].In complexitydriven schemes like LOCO-I [4], CALIC [5], and TMW [7], the compression gains are obtained by adjusting the model carefully to the image compression application.The literature reveals the significance of the gap between the complexity-driven schemes and simplicity-driven schemes.The gap between the two schemes is diminished by FELICS algorithm [8].FELICS maintains the performance with moderate compression ratio at the expense of low computational complexity.
Most of the compression community people, including those in the medical image processing, feel that instead of approaching in terms of the dichotomy between lossless and lossy, it is better to establish an algorithm, which consists of features of both techniques and also it can be a trade-off between them.It establishes a method to develop near-lossless compression algorithm.Near-lossless means no pixel is changed in magnitude by more than error tolerance gray levels compared to the original pixel [9].Error tolerance means that the quantity of the difference between the original image and decompressed image is negligible.Though many researchers [9][10][11] have developed near-lossless compression algorithm based on differential pulse code modulation (DPCM) for continuous-tone images, there is a lack of decreasing the computational time as well as increasing the compression ratio.Chen and Ramabadran [10] proposed a scheme based on DPCM coding and two types of uniform scalar quantizers with less distortion rate.Ke and Marcellin [11] have extended the concept by generalising and incorporating entropy-minimization of the quantized predictor error.It is observed from the literature that most of the algorithms demand more computational effort to attain moderate compression ratio.This motivated us to development the proposed algorithm.
The proposed technique uses the predictive coding based statistical approach.In this approach, after estimating the parameters, the model generates the pixel values within negligible time period.Also there is no dictionary-based data storage overhead.In the proposed technique, very sharp edges are slightly smoothed.But this problem is overcome by estimating the parameters precisely with the help of M-H algorithm as discussed in Sections 4 and 5.
In order to analyse image content, stochastic models like Random Field (RF) [12], Markov Random Field (MRF) [13][14][15][16], and Gibbs field (GF) [17,18] are investigated.Moreover, the degree of accuracy of parameter estimation plays a significant role in obtaining satisfactory results in low-level image processing.In earlier works, classical approaches such as Least Square (LS) [13,19] and Maximum Likelihood Estimation (MLE) [15,20] methods were used to estimate the parameters, which were found to be unsatisfactory for reconstructing the images after compression [13,16] or they require some post processing [16,21,22] or iterative procedures [23][24][25].Later the Bayesian approach was studied by [26][27][28][29] for parameter estimation that could give higher precision.It is also observed from the literature that the emphasis is on parameter estimation, not on identification of the order of the model [30].The reason for not concentrating on the order determination or model identification is that computational complexity arises as the order increases.Selecting the most suitable model for describing individual time series can improve the performance of the prediction process.
Collopy and Armstrong [31] suggest that there is no single model that performs better than the others in all types of data (images).In [32], the authors emphasize the need for incorporating knowledge into the model selection process by associating image characteristics with the model performance.Later, Arinze [32] proposed the use of learning algorithm for model selection and is adopted in other works [33][34][35].The approaches treat the model selection as a classification problem in which a learning algorithm is used as the classifier.This adds more to computational complexity.
In the proposed method, there is no question of model selection and order determination, for it has infinite structure with a finite number of parameters and so completely avoids the problem of order determination [28,29].The advantage of the present approach is that the number of parameters fixed is only four and the order does not increase the computational complexity, since the estimation of these parameters is the same irrespective of the order of the model, and hence it increases the efficiency of the model.So, the proposed model is more a generalized one for most images.
The rest of the paper is organized as follows: The overview of the proposed work is presented in Section 2. Section 3 discusses the proposed model and its features.In Section 4, the parameter estimation is discussed.The error model and its parameter estimation technique are discussed in Section 5.The measure of performance is presented in Section 6 and the results of the experiments are demonstrated in Section 7. The conclusion is drawn

Overview of the Proposed Work
In this paper, a Full Range Guassian Markov Random Field (FRGMRF) model is proposed for image compression.The proposed compression method performs the compression process at two stages: first, the actual input image is performed using FRGMRF model; second, the residual image is performed based on the error model discussed in Section 5.The proposed family of FRGMRF model is completely free from order determination as discussed in the previous Section.An input image   i f of size L  L is assumed to be a Gaussian Markov Random Field (GMRF), which is segregated into various non-overlapping subimages of equal fixed sizes of M  M where M < L. Using the Bayesian methodology, the parameters of the model are estimated for each subimage as discussed in Section 4. Using the estimated parameters K, α, θ and  , the coefficients r a of the model are computed.Based on the model coefficients and seed values, the subimage is generated.This process is repeated for the entire image.The reconstructed subimages are synthesised to form the entire image, which is denoted as i r f .The residual image e f is computed from the actual input image i f and the reconstructed image

Proposed Model
Generally, the pixels in a small image region are linearly dependent.The linear dependency among the pixels is extracted to predict the missing pixels in that region.As illustrated in Figure 2(a), the center pixel X(s) linearly depends on its neighbouring pixels.The properties of the FRGMRF model allow us to define the likelihood function   l f .This facilitates the statistical estimation of the unknown pixel values.As mentioned in Section 1, the LSE and MLE methods are not satisfactory.Hence, in this article, the Bayesian approach is adopted to predict the unknown pixels in the small image region by considering the likelihood function aforesaid, which gives the conditional distribution of the neighbourhood pixels given the original,   p g f and the prior information obtained from the domain block,   π f , where g represents the neighbourhood pixels.The joint posterior dis- f is obtained by using these two components and the Bayes's theorem.By keeping all these points, a model is proposed as in Equation (1).Let X be a random variable that represents the intensity value of a pixel at location (k, l) in an image of size M  M. The random variable X is assumed to be independent and identically distributed (i.i.d.) Gaussian random variables in discrete state space with mean zero and variance σ 2 , because the noise is independent and identically distributed Gaussian random variable which is incorporated with X.The noise is denoted with the symbol   , k l  and it follows the normal distribution, i.e.
 X s can be considered as a Markov process because it has the conditional probability: and s k belonging to the state space S and 0 1 w here   .Some of the models used in the previous workswhite noise, Markov model with finite order and infinite order can be regarded as special cases of the proposed model.Thus, 1) if we set  = 0, then the FRGMRF model reduces to the white noise process; 2) when α is large, the coefficients a r s become negligible as "r" increases.So the FRGMRF model reduces to a Gaussian Markov Random model with order r approximately, for a suitable value of r, where r is the order of the model; 3) when α is chosen to be less than one, then the FRGMRF model becomes an explosive infinite order Gaussian Markov Random model.The fact that

 
X s has regression on its neighbourhood pixels gives rise to the terminology of dependency process.However, in this case the dependence of   X s on neighbourhood values may be true to some extent.In fact, the process is Gaussian under the assumption that the are Gaussian and in this case its probabilistic structure is completely determined by its second order properties.The second order properties meant for the proposed FRGMRF model is asymptotically stationary up to order two, provided 1 -α < K < α -1.Finally the range of the parameters of the model is set as with the The non-causal model proposed in Equation ( 1) represents the pixel   X s as a linear combination of nearest neighbourhood values on each side as shown in Figure 2(a), and the influence of the horizontal and vertical pixels on the centre is higher than that of the diagonal pixels.The order of influence of neighbourhood pixels on its centre is illustrated in Figure 2(b).
While reconstructing the compressed image, only the previous values viz.upper triangular elements are known and are represented by one, and all the other elements are zero as depicted in Figure 3(b).Hence, the pixel   X s is predicted by using the upper triangular elements only, which are categorized into two sets according to the order of neighbourhood pixels as shown in Figure 3(a).
Hence, the non-causal model given in Equation ( 1) is redefined as a causal model, with the combination of two sets of elements as described in Figure 3(a): where, By using the above constraints in Equations (2a) and (2b), the causal model given in Equation ( 3) is obtained from Equation ( 1) where, , and r p q   .(3a) The model given in Equation ( 3) is used to reconstruct the compressed image data at stage one.

Parameter Estimation
In order to implement the proposed FRGMRF model, the parameters are to be estimated.The parameters K, α, θ and  are estimated, by taking the suitable prior information for the hyper parameters , , and , based on Bayesian methodology [28,29].The hyper parameters are meant for the parameters of the prior distribution of the actual model parameters K, α, θ and .The hyper parameters, approximately estimated by using the mean and standard deviation of the pixel values of the subimage are to be estimated.Just for the computational purpose, the pixel values of each subimage are arranged as one-dimensional vectors   X t , (M  M = M 2 = N).Since the error term in Equation ( 3) is independent and identically distributed Gaussian random variable, the joint probability density function of the stochastic process where, When the real data are analyzed with finite number of N observations, the range for the index "r" viz., 1 to , reduces to 1 to N and so the joint probability density function of the observations given in Equation (4); the summation 1 r    can be replaced by which gives By expanding the square in the exponent, we get where .
1 , , 0,1, 2, , The above joint probability density function can be written as where, and σ 2 > 0. The prior distribution of the parameters is assigned as follows: 1) α is distributed as the displaced exponential distribution with parameter β, i.e.
3) K,  and  are Uniformly distributed over their domain, i.e. , a constant; where, P is used as a general notation for the probability density function of the random variables given within the parentheses following P. Using ( 7), (10) and Bayes theorem, the joint posterior density of K, α, θ,  and σ 2 is obtained as x p 1 e x p 12 2 , 1, 0 π, 0 π 2 and 0.
Integrating (11) with respect to σ 2 , the posterior density of K, α, θ and  is obtained as where That is, where Thus, the above joint posterior density of K, ,  and  can be rewritten as where, 2 This shows that, given α, θ and , the conditional distribution of K is "t" distribution located at b 1 with (2d -1) degrees of freedom (d.f.).
The proper Bayesian inference on K, α, θ and  can be obtained from their respective posterior densities.The joint posterior density of α, θ and , namely,   , , P X    , can be obtained by integrating (15) with respect to K. Thus, , , where, n = (2d −1) d.f.Thus, the joint posterior density of α, θ and  is obtained as The derivation of Equation ( 18) from Equation ( 17) is discussed in APPENDIX.
The marginal posterior density of α, θ and  in ( 18) is a complicated function and is analytically not solvable.Therefore, we can find the original posterior density of α, θ and  numerically from the joint density (18).
The point estimates of the parameters α, θ and  may be taken as the means of the respective marginal posterior distribution i.e. posterior means.With a view to minimize the computations, we first obtain the posterior mean of α numerically.Then fix α at its posterior mean and evaluate the conditional means of θ and  fixing α at its mean.We fix α, θ and  at their posterior means respectively and then evaluate the conditional mean of K. Thus, the estimates are The estimated parameters K, α, θ and  are used in Equation (3a) to compute the coefficients r a s of the model presented in Equation ( 3) and then the model coefficients are applied in Equation ( 3) to reconstruct the compressed image.

Error Model
The residual image e f is computed from the actual input image i f and the reconstructed image ir f .Since the residual image e f has no spatial direction among the pixels, the angle θ is set to zero in the proposed model in Equation (1).Now, the proposed FRGMRF model becomes white noise model as in Equation ( 21), because the first term in Equation ( 1) becomes zero and the error term, that is, second term only remains.where r p q   ,   , e k l represents error term, which follows the Markov process [39] and r s  are the Markov coefficients.
By applying the constraints used in Equations ( 2a) and (2b), the model in Equation ( 21) is reconstructed as stationary second-order MR(2) model as follows: where The computed coefficient values 1  and 2  are substituted in Equation ( 22) to obtain the decompressed image.
The following algorithm is employed to estimate the coefficients 1  and 2  .

M-H Algorithm
Step 1: Check the stationarity conditions for the set of coefficient values Step 2: A sample of draws from the posterior distribution of the parameters   , , Step 3: Step 4: Generate candidates from the density function where nor f is the normal density function, Step 5: At jth iteration (the current value  2(j) ) draws a candidate   1 j a  from a normal density with mean  and covariance 1) If it satisfies stationarity conditions given in Step 1, then move to this point with probability Step 6: Repeat Step: 2 to Step: 5 until it satisfies the condition 1) of Step: 5.
After estimating the parameters 1  and 2  , the model expressed in Equation ( 22) is employed to reconstruct the compressed subimage.

Measures of Performance
The performance of the proposed FRGMRF model is evaluated by means of Root Mean-Squared Error (RMSE) and Peak Signal to Noise Ratio (PSNR) The compression ratios along with RMSE and PSNR values by the proposed method and the JPEG2000 are presented in Table 1.where i f and r f are actual input image and final reconstructed image of size 256  256 respectively.
Based on the actual input image i f and the reconstructed image ir f , the residual image e f is computed.The residual images of D9-Grass, D29-Beach-Sand and Barb images are presented in Figures 5(a)-(c) respectively.As discussed in Section 1, the sharp edges are smoothed while reconstructing the compressed images by the FRGMRF model.Smoothed means the loss of information.The information lost (residual image) is extracted by differentiating the images i f and ir f .The error model discussed in Section 5 is employed on the residual image to obtain furthermore quality.For more precise estimation of the parameters of the error model, the M-H algorithm discussed in Section 5 is used for accurate prediction of the pixels in the residual image.The procedure used for the actual input image (FRGMRF model) is adopted for the residual image (error model).The arithmetic coding is used on image data compressed by the FRGMRF model and the error model to achieve higher compression ratio.The output of the experiment is presented in Figures 6(d)-(f) and the results obtained are given in Table 2.The outputs are obtained by adding the images generated by the FRGMRF model and error model.The final outputs are compared with the outputs obtained by JPEG2000 with the quality factor 95 and presented in Figures 6(g)-(i).Conducting the experiment at various levels of quality factors and bpp, the final outputs are obtained.From the experiment, it is observed that the quality factor 95 yields good reconstruction quality at different bit rates.The optimum bpp is fixed at No.of bits required to store compressed image CR 1 No.of bits required to store original image 100

Experiments and Results
The     Quality Factor on the same images.But, due to the space constraint, it is not possible to present all the outputs of the experiment here.The obtained results show that the proposed method requires more or less the same bpp for textured type (Grass and Beach-Sand) images while it produces less PSNR with high bpp for structured (Barb) images.This is illustrated in Figure 7. From the experimental results, it is observed that the proposed method gives higher compression ratio with good reconstruction quality for both textured and structured images.
The proposed model can be applied in data mining for clustering, classification, prediction, and searching.In the proposed method, there is no question of model selection and order determination, as it has infinite structure with a finite number of parameters and so completely avoids the problem of order determination.The number of parameters fixed is only four and the order does not increase the computational complexity, since the estimation of these parameters is the same irrespective of the order of the model, and hence it increases the efficiency of the model.So, the proposed model is a

Conclusions and Future Work
In this paper, a family of FRGMRF model and an error model are proposed for monochrome continuous-tone still image compression.Since the residual image e f has no spatial direction among the pixels, the angle θ is set to zero in the proposed model in Equation ( 1).Thus, it facilitates to derive the error model from the proposed FRGMRF model.The parameters of the FRGMRF model are estimated using the Bayesian approach.A few generalized one for different categories of images.Hence, the proposed model can also be extended to colour image compression by restructuring it as a three dimensional model.
The expression given in Equation (1A) can be written as in Equation (2A), since the term b 1 can be ignored.Because b 1 represents the location of the variate K, which follows t-distribution.
, , , , The probability density function   f t of t-distribution with d.f.satisfies the condition Now put, in Equation (4A) 2 1 a nK y  The limits of "K" and "y" remain the same and differentiate the above equation.Using these values in Equation (4A) and making use of (6A), we get Equations (7A) and (8A) The probability density function of Beta distribution of second kind for variate 2 1 1 , 2 2 where The factor 2 disappearing since the integral form  to  must be unity.The expression in Equation (11A) is the t-distribution with (n-1) d.f.. Acording to Equations (10A) and (11A), the variate 1 , 2 2 n        in Equation (9A) follows t-distribution with n d.f. and its probability density function is equal to 1. Thus, Equation (9A) can be written as follows: As said earlier, the term b 1 represents the location of the distribution.So it can be ignored.The expression given in Equation (13A) gives the required result.

Figure 1 .
Figure 1.Block diagram of the proposed work. s

Figure 2 .
Figure 2. (a) Relationship among neighbour sets; (b) Neighbour sets for different FRGMRF model with orders.    sin cos , r r K r r r p q a       (1a) and K, α, θ, and  are real parameters.The r , model coefficients, which are computed by substituting the model parameters K, α, θ, and  in Equation (1a).The

Figure 3 .
Figure 3. (a) Order (priority) of the neighbour sets; (b) 1 represents availability of the pixels; 0 represents nonavailability of the pixels.
structed subimages are grouped together to form the entire image.The reconstructed images by the proposed method, corresponding to the original images shown in Figures4(a)-(c), are presented in Figures4(d)-(f) respectively.The proposed image compression scheme is also compared to JPEG2000.The compressed version of the original images for JPEG2000 is obtained with a quality factor 60, by conducting the experiment with Advanced JPEG Compressor V4.8[41] software, and the outputs corresponding to the images shown in Figures4(a)-(c) are presented in Figures 4(g)-(i).
The compression ratio (CR) is calculated in percentage as follows: proposed FRGMRF model and the error model discussed in Section 4 and Section 5 are experimented on different types of monochrome images for compression and decompression.For sample, two standard textured images D9 (Grass) and D29 (Beach-Sand) and a structured image viz. the Barb image, each of size 256  256 with pixel values in the range 0 to 255 are given in Fig- ures 4(a)-(c) respectively.The textured images are taken from standard Brodatz album [40].The input image is divided into various non-overlapping subimages of equal size of 8  8.The parameters of the proposed FRGMRF model are estimated for each subimage as discussed in Section 4. Using the estimated parameters, the coefficients of the model are computed.The coefficients and a few pixel values are stored.Based on the model coefficients and pixel values, the subimage is generated.This process is repeated for the entire image.All the recon-

Figure 6 .
Figure 6.(a), (b) and (c) are original images; (d), (e) and (f) are reconstructed images by the proposed model; (g), (h) and (i) are reconstructed images by JPEG with quality factor 95.
Based on the stored values, the error image is reconstructed.Now, the reconstructed image of actual input and the residual image are added together, which results in the final reconstructed image.The final reconstructed image is almost error free.Various steps involved in the proposed work are demonstrated with a block diagram in Figure 1.
constructed subimages are grouped together to form the entire image, which is denoted as er f .Again, error image ee f is computed from er f and e f .The average is computed on ee f and it is stored together with parameters values and seed values.