Sparse Representation by Frames with Signal Analysis

The use of frames is analyzed in Compressed Sensing (CS) through proofs and experiments. First, a new generalized Dictionary-Restricted Isometry Property (D-RIP) sparsity bound constant for CS is established. Second, experiments with a tight frame to analyze sparsity and reconstruction quality using several signal and image types are shown. The constant k δ is used in fulfilling the definition of D-RIP. It is proved that k-sparse signals can be reconstructed if 2 2 3 k δ < by using a concise and transparent argument1. The approach could be extended to obtain other D-RIP bounds (i.e. tk δ ). Experiments contrast results of a Gabor tight frame with Total Variation minimization. In cases of practical interest, the use of a Gabor dictionary performs well when achieving a highly sparse representation and poorly when this sparsity is not achieved.


Introduction
Let n p × Φ ∈  and p β ∈  be a signal such that with 2 z ε ≤ . In compressed sensing, one can find a good stable approximation (in terms of ε and the tail of β consisting of p k − smallest entries) of β from the measurement matrix Φ and the measurement y through solving an 1  -minimization, provided that Φ belongs to a family of well behaved matrices. A subclass of this family of matrices can be characterized by the well known Restrictive Isometry Property (RIP) of Candès, Romberg, and Tao, [3] [4]. This property requires the following relation for Φ δ . Some bounds on δ have been determined in previous publications [3]- [6]. Notably, Cai and Zhang have established several sharp RIP bounds that cover the most interesting cases of δ k and 2k δ [7] [8] showing 2 1 2 , 3 2 A key requirement in this setting is a signal being sparse or approximately sparse. Indeed, Many families of integrating signals have sparse representations under suitable bases. Recently, an interesting sparsifying scheme was proposed by Candès et al. [9]. In their scheme, instead of bases, tight frames are used to sparsify signals. Let p d D × ∈  ( d p ≥ ) be a tight frame and k d ≤ . Candès et al. [9] suggests that one use the following optimization to approximate the signal β : The traditional RIP is no longer effective in the generalized setting. Candès et al. defined the D-restricted isometry property which extends RIP [9]. Here the formulation of D-RIP is used as in Lin et al. [10].
holds for all k-sparse vectors d v ∈  . The RIP is now a special case of D-RIP (when the dictionary D is the identity matrix). For D being a tight frame, Candès et al. [9], proved that if 2 0.08 is a good approximation of β . Lin et al. [10], improved this result to 2 0.472 k δ < by using some techniques developed by Candès et al. [3].

Contribution
The proof in Section 2 establishes an improved D-RIP bound which states that 2 2 3 k δ < . This result was previously available [1] and it has been improved on by Wu and Li [2]. The main ingredient of the proof in Section 2 is a tool developed by Xu et al. [11]. This approach takes its inspiration from the clever ideas of Cai and Zhang [7] [8].
The practical application of this proof consists of experiments targeting the theory of using tight frames in this CS setting satisfying D-RIP. Similar experimental methods to those used by Candès et al. are followed [9] [12]. However, an expanded variety of relevant sparse and non-sparse signals are used to test the robustness of the Gabor transform. Additionally, these experiments analyze sparsity in the coefficient domain and show that a highly sparse representation is a good indicator of reconstruction quality. These results are contrasted with a commonly used CS approach of Total Variation (TV) 1  minimization. This paper is organized into four main sections. Background information is presented in Section 1. Section 2 describes a proof of an improved D-RIP sparsity bound by using a concise and transparent approach. Section 3 details the methods of experimentation used to apply the tight frame in practical applications of signals and image recovery. Section 4 shows and discusses the reconstruction simulation results with an analysis of the robustness and shortcomings of using tight frames in CS.

New D-RIP Bounds
where the constants 0 C and 1 C depend on 2k In order to prove Theorem 2, the following 1  -norm invariant convex k-sparse decomposition by Xu and Xu [11], and Cai and Zhang [8] are needed.
The following description is taken from Xu et al. [11]. Lemma 1. For positive integers k n ≤ , and positive constant C, let n v ∈  be a vector with for some nonnegative real numbers 1

Now Theorem 2 is proven.
Proof. This proof follows some ideas in the proofs of Theorems 1.1 and 2.1 by Cai et al. [8], incorporating some simplified steps. Some strategies from Cai et al. [5] [13] are also used. This proof deals only with the 2k δ case so that the key ideas can be conveyed clearly. [9], one can easily verify the following: By rearranging the columns of D if necessary, assume 1 , , ,0, ,0 and 0, ,0, , , .
Assume that the tight frame D is normalized, i.e., * DD I = , and Thus there is the following useful relation: , .
From the facts * * with each d t w ∈  being k-sparse, . From this and the Cauchy-Schwartz inequality, there is By the triangle inequality, In order to prove the theorem, it suffices to show that there are constants 0 1 , In fact, assuming (17) there is Now moving to the proof of (17). Denote On the other hand, as each *   2 1  2  2  2  3  3  3  3   2 1  2  2  ,  2  2  2  3  3  3  3 3  2  2  2  2  2  3  3  3  3  3  3  3  3  3 3 This demonstrates the use of Lemma 1 to get a good result. This could be pursued further to general cases for an even better bound. Indeed, this has been done recently by Wu and Li [2] to improve the result of this proof which has been available previously [1].

Experiments
The focus in these experiments is to show practical applications of a sparsifying frame that satisfies the new lower bound proven in the previous section. A time-frequency Gabor dictionary is used as a sparsifying transform in re-weighted 1  minimization [12]. The Gabor dictionary fulfills the requirements of D-RIP because it is a tight frame.
One of the advantages of a Gabor dictionary is its characteristic of translational invariance both in time and frequency. In a similar context, a translational invariant wavelet transform was used with good results in MRI reconstruction [14]. Invariance in the 1  minimization penalty term is advantageous for its ability to represent different signals sparsely.
Here comparison measurements of sparsity in Gabor and TV Coefficients of various signals are used to verify if good and robust results can be achieved. A selection of 5 different real valued signals and/or images with variants are used to simulate the complexity of practical signals. The use of the Gabor dictionary to reconstruct these signals utilizes a core optimization algorithm solver for the primal-dual interior point method provided by 1  -Magic [15]. The redundancy of the Gabor transform coefficients compared to the original signal is about 43 times. These results are compared to reconstructions of the same input signal using another CS optimization algorithm solver, TV minimization with quadratic constraint, which is also provided by 1  -Magic. These algorithms operate on the full sized image (or signal) without breaking up the data into segments. In both settings, the same under-sampling is performed by using a pseudo-random Gaussian measurement matrix with a factor of 2.
In Table 1, there is a comparison of two measures for this analysis, the Mean Square Error (MSE) and

Results and Discussion
The goal is to show how well a sparse tight frame representation of various signals performs in CS reconstruction. Analysis is done of the Gabor dictionary as a sparsifying transform on non-sparse signals and images. A large range of reconstruction errors and sparsity levels are observed for different image types and signals. The use of the Gabor frame with a reference of TV weighted 1  minimization in compressed sensing reconstruction is compared. Table 1 quantifies MSE and sparsity for each reconstruction test. Figures 1-8 are the corresponding image and signal reconstructions.
According to these measurements, sparsity in the coefficient domain will correlate to image reconstruction C. Baker success. For example, test 1 measures a Gabor coefficient sparsity of over 99% and a reconstruction success which reduces the MSE by 36 times compared to the linear reconstruction. Whereas, with the same signal, which is not sparse at all in the TV domain, TV minimization actually increases the MSE when compared with the linear reconstruction, see Figure 1. It is important to note that the sparsity calculated on the Gabor coefficient set is on a much larger set of redundant coefficients than the non-redundant TV coefficients. In test 2, the complexity and sparsity are adjusted by adding additional sinusoidal pulses which may overlap. The complexity of the pulses significantly increases to 20 pulses and the Gabor dictionary is able to sparsely represent the signal very well compared to TV minimization, see Figure 2. Similar to the results in test 1, the reconstruction MSE is reduced by 36 times.
In tests 3 and 4, images are chosen which are sparse in the TV domain but not in the Gabor domain. The TV reconstruction reduces the MSE to zero compared to the Gabor reconstruction reducing by only a factor of 2.6 and 2.4 respectively. The penguin image is an example with a different background magnitude from the Shepp-Logan phantom, see Figure 3 and Figure 4.
The signal in test 5 is a combination of a pulse from test 1 with the Shepp-Logan image from test 3. The same signal is plotted in the time domain for Figure 5 and in the image domain for Figure 6. Although both Gabor and TV CS reconstructions improve the result over the linear calculation, the error still remains quite large. It is noteworthy that the sparsity percentages are much lower for this case in both the TV and Gabor domains. This underscores the important connection between having a sparse representation and making a good reconstruction.
In the last experiments, tests 6 and 7, MRI images of the brain as either T1 weighted or TOF MIP are used. They appear not to be sparsely represented in either the Gabor domain or in TV. The MSE result is poor in both reconstruction algorithms, see Figure 7 and Figure 8. It is important to note that under-sampling is in the image domain and not in the native MRI signal domain of k-space. However, this is still an equivalent comparison for cases when the under-sampled k-space produces artifacts that are incoherent as in this experiment. A requirement of CS reconstruction is that artifacts due to sampling are similar to uniform noise with an even distribution across the image.
It is also important to point out that the linear reconstructions, calculated with a pseudo-inverse, have a consistent MSE for all experiments, see L MSE in Table 1. This is not the case for this tight frame. These findings show remarkably good results for some periodic signals. However, the Gabor tight frame does not appear to be advantageous for the images investigated here. The ability of the sparsifying transform to produce a high percentage of sparsity contributes greatly to the reduction of reconstruction error. When using CS in these cases, it is vital to pick a dictionary which will effectively and sparsely represent the signal of interest.

Conclusion
The use of a new D-RIP sparsity bound constant for compressed sensing is proven using a transparent and concise approach. Practical numerical experiments for this setting are performed. The use of a Gabor tight frame in CS is contrasted with TV weighted 1  minimization by simulation. Measurements of reconstruction error and coefficient sparsity in each domain are presented and analyzed. Sparse representation by frames does provide good results when the dictionary is chosen appropriately. Care must be taken to assure high levels of sparsity are achieved in the coefficient domain. Otherwise, poor fidelity in reconstruction may occur.