Magnetization Performance of Ldpc Reduced-complexity Decoding Algorithms

Low-density parity-check (LDPC) codes are very efficient for communicating reliably through a noisy channel. N.Sourlas [1] showed that LDPC codes, which revolutionize the codes domain and used in many communications standards, can be mapped onto an Ising spin systems. Besides, it has been shown that the Belief-Propagation (BP) algorithm, the LDPC codes decoding algorithm, is equivalent to the Thouless-Anderson-Palmer (TAP) approach [2]. Unfortunately, no study has been made for the other decoding algorithms. In this paper, we develop the Log-Likelihood Ratios-Belief Propagation (LLR-BP) algorithm and its simplifications the BP-Based algorithm and the λ-min algorithm with the TAP approach. We present the performance of these decoding algorithms using statistical physics argument i.e., we present the performance as function of the magnetization.


Introduction
Low-density parity-check (LDPC) codes were first discovered by Gallager [3], in his thesis, in 1962 and have recently been rediscovered by Mackay and Neal [3,4].These codes can get very close to the Shannon limit by mean of an iterative decoding algorithm called Belief-Propagation (BP) algorithm [5].
This performance combined with their relatively simple decoding algorithm makes these codes very attractive for the next generation of digital transmission systems.Indeed, these codes have been chosen as a standard for the second Satellite Digital Video Broadcasting normalization (DVB-S2).So, the search for efficient implementations of decoding algorithms is being a challenge.
The implementation of the BP algorithm is difficult.That's why different simplifications of this algorithm were discovered.The BP algorithm can be simplified using the BP-Based algorithm [6] and the λ-min algorithm [7].
N. Sourlas [1] has shown in 1989 that there is a mathematical equivalence of error-correcting codes to some theoretical spin-glass models.This analogy has contributed to the present proximity of the statistical physics and the information theory.Therefore, the methods of statistical physics developed in the study of disordered systems proved to be efficient for studying the properties of these codes.One of these methods is the Thouless-Anderson-Palmer (TAP) [2] approach which is shown equivalent to the BP algorithm by kabashima, et al. [8].Unfortunately, no study has been made for the other decoding algorithms.
In this paper, we develop the Log-Likelihood Ratios-Belief Propagation (LLR-BP), the BP-Based and the λ-min decoding algorithms with the TAP approach.Their performance is evaluated as a function of a statistical physics parameter which is the magnetization.
This paper is organized as follows.Section 2 corresponds to a general description of the LDPC codes and their decoding algorithm.Section 3 describes the similarity between the BP algorithm and the TAP approach.In Section 4, we develop the LLR-BP algorithm and its simplifications the BP-Based algorithm and the λ-min algorithm with the TAP approach.Finally and before concluding, simulation results as function of the magnetization are presented in Section 5. Using a notation similar to that in [4,6], let  

LDPC Codes and Decoding Algorithm
denote the set of bit j that participate in check  .Similarly, we define the set of checks in which bit j participates as     denotes the transmitted codeword.

Belief Propagation Algorithm
This section summarizes according to [4,5], the iterative decoding of LDPC codes based on the BP algorithm.The likelihood of s is given by  j a j f , with a j f  P(s j = a).
Let a j q  denotes the probability that bit j of s is a , given the information obtained via checks other than check  .The quantity a j r  is meant to be the probabil- ity of check  is satisfied if bit j of s is considered fixed at a and the other bits have a separable distribu- tion given by the probabilities The standard iterative decoding algorithm based on the BP approach consists on the following main steps. Initialization: The variables 0 j q  and 1 j q  are ini- tialized to the values 0 j f and 1 j f respectively. Iterative processing: , and for For each j and Create the word 1 ˆˆ( ... )

N s s s 
the detection of the transmitted codeword such that: the decoding process ends.Otherwise, we pass to another iteration of BP.A failure is declared and the decoding process ends if the number of iterations exceeds the maximum number of iterations and s ˆ is not a valid codeword.

Decoding Problem from Statistical Physics
Point of View

Statistical Physics Analogy
In the previous section, we have described the LDPC codes using the additive Boolean group      , 1 , 0 . However, in order to apply methods of statistical physics, it is convenient to introduce an equivalent group, the multi- . From a statistical physics point of view, the code can be regarded as a spin system.Each bit and the word can be seen as a collection of N spins, called a configuration.The parity-check matrix gives rise to the interactions between the spins [1].
The decoding problem depends on posteriori like where J is the evidence (received message or syndrome vector).By applying Bayes' theorem this posteriori can be written in the form [1].
In the statistical physics description, the probability (1) can be expressed as a Boltzmann distribution at the inverse temperature  [9]. where is the Hamiltonian of the system.In this Hamiltonian, we identify two components that are necessary for the analysis of LDPC codes. A term that guarantees that all parity checks are satisfied.It can be written with the Kronecker's delta According [9], the  's can be replaced by a soft constraint.

 
.  A prior term that provides some statistical information on the dynamical variables S .It can be represented by the prior distribution where F is the external field.It is determined by the flip rate f of channel noise as So, the Hamiltonian is written as follows: where  J is the muti-spin coupling.

Decoding in the Statistical Physics
The decoding process corresponds to finding local magnetization at the inverse temperature  , and calculating estimates as [1].
The decoding performance in the statistical physics approach can be measured by the magnetization [1] defined by the overlap between the actual message and estimate: Here the overage ... is performed over the matrices A .This value provides information about the typical performance of the code.
The Magnetization is an order parameter which has a standard of judgment whether the whole system exhibits an ordered state or not.
the system is in a disordered phase called paramagnetic phase.Two main methods can be employed for calculating the value of the magnetization: the replica method for diluted systems [9] and the TAP approach [8].In this paper, we are interested only on the TAP approach.

TAP Approach
Kabashima, et al. [8] have shown the similarity between equations derived from the TAP [2] approach and those obtained from BP.The fields a j q  correspond to the mean influence of sites other than the site j and the fields a j r  represent the influence of j back over the system (reaction fields) [10].
The similarity can be exposed by observing that the likelihood   The conditional probability j S j r  can be seen as a normalized effective Boltzmann weight (effective Boltzmann probability) obtained by fixing the bit j S [10]   Since spin variable j S takes only two values 1  , it is convenient to express the BP/TAP algorithm using spin averages rather than the distributions j S j q  and j S j r  themselves.We use . Additionally, it was shown in [10] that the following statements hold The pseudo-posteriori can then be calculated This provides a way for computing the Bayes optimal decoding as follows: The result that is demonstrated as follows:

LLR-BP Algorithm with TAP Approach
In the LLR-BP algorithm instead of handling probabilities as in [3,5] we are going to deal with the LLRs.In practice, using LLRs as messages offers implementation advantages over using probabilities or likelihood ratios because multiplications are replaced by additions and the normalization step is eliminated [11].
In this section, we try to develop the LLR-BP with the TAP approach.Let j x  and j y  be the LLR of bit j which are sent from bit node j to check node  and from the check node  to bit node j , respectively.From the statistical physics definition, the LLR is defined as follows [12]: We need also the following results   j j j j j j j j j j m q q q q q q q q x (12) Like in (12), the variable j m  ˆcan be also written as: So, from Equations ( 5), ( 12) and ( 13) we have Finally, the posteriori LLR of bit j S is It's not easy to implement (15) which has the form of a product.Our idea is to decompose the check node update into sign and magnitude processing like in [13].We have from (15) Then, taking the logarithm of the inverse of both sides of (19) yields The Equation (20) verifies So the Equation ( 21) can be written Using (10) and (11), the Equation (23) can be written as From ( 18) and (24), the check node processing in the statistical physics is denoted by

BP-Based with TAP Approach
In this section, we try to develop the approximation of the BP algorithm: the BP-Based [6] algorithm, with the TAP approach.The key idea of this algorithm is that on the Figure 1.The Equation ( 24) is then approximated by So, the extrinsic information in the BP-Based algorithm written with the TAP equation is given by: ) Therefore, there is an important simplification in the BP-Based algorithm with TAP approach since the check node update is replaced by a selection of the minimum input value.

λ-min Algorithm with TAP Approach
In this section, we try to develop the λ-min algorithm [7] with the TAP approach.In the check node update of (25), the magnitude processing is run using the function defined by (20).While depicting this function on the So, like Guilloud, et al. [7] we propose to compute (24) with only the λ bits that participate in check  and which have the minimum magnitude.  .Hence, for the second case, the computations have to be performed only once [13].

Simulation Results
Simulations have been performed using regular (3,6) LDPC code of length 1008  N and with 20 decoding iterations.We averaged the results over 10 codes.This ensemble of LDPC code is characterized by the same block length, the ones in each column and the ones in each row.For each run, a fixed code is used to generate 1008 bit codeword from 504 bit message.Corrupted versions of the codeword are then decoded using LLR-BP, BP-Based and λ-min decoding algorithms.
To observe the effect of the simplification in ( 27) and (29) on the performance from a statistical physics point of view, we have calculated the overlap between the actual message and the estimate (magnetization) for each fixed code and for each algorithm.After, we average the obtained results for the 10 different codes.Figure 2 depicts the magnetization performance of the LLR-BP, BP-Based and the λ-min algorithms.f(t) t According the results of Figure 2, we can conclude the effectiveness of the λ-min algorithm.At a magnetization of 0.9, the BP-Based algorithm introduces a degradation of 0.5 dB with 20 iterations.The 2-min algorithm introduces a degradation of 0.3 dB.The performance of the 3-min algorithm is slightly worse than that of the LLR-BP algorithm, the degradation is only 0.08 dB.
Another remark from Figure 2 is that at high signal to noise ratio the magnetization is concentrate at the value 1  m .This value corresponds to the ferromagnetic state in the statistical physics and to the perfect decoding in the information theory.
The analogy between the Bit Error Rate (BER) performance and the magnetization performance can be examined in [14].

Conclusions
In this paper, we have been interested in the decoding of LDPC codes from a statistical physics approach.First, we have examined the correspondence between LDPC codes and Ising spin systems.The relation between the BP algorithm and TAP approach is investigated.Then, we showed that LLR-BP algorithm and its simplification i.e. the BP-Based algorithm and the λ-min algorithm can be obtained using the TAP approach of Ising spin systems in statistical physics.Besides, we have presented the performance of the decoding algorithms using a statistical physics parameter which is the magnetization.
Finally, we concluded that the BP-Based algorithm reduces the complexity for updating extrinsic information but there is degradation in performance compared to the LLR-BP algorithm.The λ-min algorithm reduces the complexity for updating extrinsic information without degradation in performance compared to the LLR-BP algorithm especially when λ increases.These results con-  firm with the results obtained in the information theory.
As a perspective of our work, we propose to study the replica method.This is a method from the statistical physics applied to find the magnetization.Besides, we propose to simplify the equations of this method by the same approximation in the BP-Based algorithm and the λ-min algorithm in order to compare the analytic results with the experimental ones.

Figure 1 .L
Figure 1.Shape of the function f.
Two cases will occur: if the bit j belongs to the sub-