Compressive Sensing Based Robust Signal Sampling

Signal processing methods have been changed substantially over the last several decades. Traditional sampling theorem of Shannon-Nyquist states that the sampling rate must be at least twice the maximum frequency presented in the signal; however, sampling at the Nyquist rate is inefficient because the signals of interest contain only a small number of significant frequencies relative to the band limit, although the locations of the frequencies may not be known a priori. Recently, compressive sensing (CS) has made a paradigmatic step in the way information is presented, stored, transmitted and recovered, by which we can acquire and reconstruct sparse signals from sub-Nyquist incoherent measurements. Three key ingredients of successfully implementing compressed sampling technique are sparsible/compressible probed signal, reliable hardware design, and low-cost computational algorithm. In this paper, we focused on two aspects about the robust sampling of sparsible/compressible signal, in particular, the design of compressed sampling hardware and the robust reconstruction via sparse Bayesian analysis. Primary results showed the high performance of proposed strategies.


Introduction
Signal processing methods have been changed substantially over the last several decades.Traditional Shannon-Nyquist based sampling strategy states that the sampling rate must be at least twice the maximum frequency presented in the signal.However, sampling at the Nyquist rate is inefficient in many cases, for example, the signals of interest contain only a small number of significant frequencies relative to the band limit.Recently, researchers in the growing vigorously field of compressive sensing (CS) have made a step forward by relating a signal's sparsity structure with its acquisition and have showed that one can recover a sparse signal from a few linear non-adaptive measurements (Candes et al., 2006;Candes and Romberg, 2007;Candes and Tao, 2006;Donoho, 2006;Romberg, 2008;Li et al., 2009).Accordingly, CS has made a paradigmatic step in the way information is presented, stored, transmitted and recovered, by which we can acquire and reconstruct sparse signals from their sub-Nyquist incoherent measurements.Roughly speaking, CS can be an effective approach in situations such as: (a) The conventional Nyquist rate of sampling is too high to implement; for example, the current analogy-to-digital conversion (ADC) technology based on uniform sampling in the time or spatial domain is limited to within the order of 1GHz.(b) The number of sensors is limited due to implementation constraints or cost.(c) The sensing process may be slow so that only a small number of measurements can be collected.
Besides sparsity, other two key ingredients of implementation CS technique for practical applications are reliable hardware realization and robust signal reconstruction.It has been shown that the random matrix with entries independently drawn at random from a Gaussian distribution of zero mean and unit variance (or other distributions such as Bernoulli, and so on) can ensure the exact recovery of the signal which is sparse in arbitrary orthobase with overwhelming probability (Candes et al., 2006;Candes and Romberg, 2007;Candes and Tao, 2006;Donoho, 2006).Following this theory, the well-known single-pixel camera was constructed by Baraniuk et al.Later on, many efforts to design the universal CS measurement instruments have been done by many researchers, for example, the chip-level Analogy-to-Information converter (Laska et al., 2007), the single-shot compressive spectral imager (Gehm et al., 2007;Marcia et al., 2009), the random lens (Fergus et al., 2006), and so on.Unfortunately, these CS measurements cannot be usually used in practice (at least cannot used for the real-time purpose) because of its time-consuming data collection and the difficulty of physical realization.To address this drawback, the random convolution based sampling strategy has been investigated from the aspects of theoretical analysis (Romberg, 2008) and empirical attempt (Tropp et al., 2006).Application of a random filter for compressive sensing was first mentioned by Tropp et al. who proposed two equivalent realization structures of a random filter: 1) convolution with a random waveform in the time domain, and 2) multiplication with random weights in the frequency domain, both followed by equal interval down-sampling.Following the Romberg's theory, Jacques et al constructed the CMOS compressed imaging by using a lot of shift registers in a pseudo-random configuration.
In this paper we will firstly focus on the random-convolution based sampling strategy and its hardware realization.Theoretical and experimental results from the time-reversal in random media have shown that exploiting the multi-scattering of acoustic and electromagnetic wave in random media can dramatically enhance the imaging resolution.Motivated by this observation, we will mainly study the application of wave scattering/propagation in random medium in low-cost hardware design for signal sampling and diffraction tomography.The proposed strategy has the following two advantages: 1) the generation of the measurement ensemble is computationally efficient and requires less memory; and 2) the proposed sampling hardware is universal from the viewpoint of compressive sensing, in particular, it can be applied to several transform domains and leads to simple implementations.It is noted that there are several similar works by exploiting the multi-path effect for complex signal environment, for example, the super-resolution migration imaging has been numerically investigated when the point-like clutters with known characteristics appears (Cheney, 2008), the random lens has been constructed to obtain super-resolution optical imaging from sparse measurements (Marcia et al., 2009), and so on.Secondly, we will discuss the robust reconstruction of sparse signals used, in particular, to address sparse signal reconstruction under some unreliable measurements.Recovering information from the maximum number of dependable observations with some unreliable ones is critical for robust signal sampling.We will investigate this problem from the viewpoint of sparse Bayesian estimation.

Random Convolution Sampling Theory
In a nutshell, compressed sampling (CS) acquires a signal of interest indirectly by correcting a small number of its projections rather than evenly sampling it at the Nyquist rate which can be prohibitively high for broadband signals encountered in many applications.This new signal acquisition paradigm has revolutionized that way digital data are traditionally acquired.The sparsity is one of key issues involved in current CS theory, which express the idea that the "information rate" of a continuous-time signal may be much smaller than that suggested by its bandwidth.In other words, a discrete-time signal depends on a number of degree of freedom which is relatively much smaller than its length; actually, many natural signals are sparse or compressible in the sense that they have sparse or approximately sparse representations when expressed in an appropriate basis or dictionary.Empirically, a vast amount of facts showed that many sub-Nyquist sampled signal sparse or compressible in some transformed domain can be exactly reconstructed through the lp-norm ( 0 1 p   ) based recovery techniques, which is traced back to the work of Logan in 1965, Santosa and Symes in 1986, and Donoho and Stark in 1989.However, the sparse related concepts were generally agreed since the foundation of so-called compressed sampling (CS) developed by Candes, Tao, Romberg andDonoho in 2006 and2007 where stands for the inner product of two vectors.Using matrix notation the sensing process is described by   y Φx n , where n means the "small perturbation in the data" referring either to signals that are not exactly sparse but nearly sparse, or to presence of noise in the sampling process.
In the literatures the sensing matrix Φ required by CS theory is often chosen as a random matrix for it keeps small coherence with Ψ .However, it is also known that the aforementioned random structures are inefficient in terms of realization and computation, especially for large-scale problems.So far, many sensing matrices with certain structures have been investigated by many authors and it is noted that the random convolution based approach is the most popular due to its outstanding advantages in the aspect of hardware design and algorithm consideration (Bajwa et al., 2007;Romberg, 2008;Rivenson and Stern, 2009;Tropp et al.,2006).Mathematically, the random convolution can be formulated by       , where the input signal   x t and output signal   y t is related by the random filter response   h t .Apparently, one key issue is that what kind of   h t can be used to sample the sparse signal using as less number of measurement as possible.In terms of classical linear algebra, the received signal   y t can be expressed as Romberg's theorem universally works because the generated CS measurement matrix  will be incoherent with any fixed orthonormal matrix  with 'overwhelming' probability.
Although the important success of Romberg's theorem in design of digital circuit and numerical simulation, it is very hard to construct the analogy hardware that satisfies the unite amplitude-response within the working bandwidth.Here, we will investigate the CS theorem for convolution with white noise waveforms followed by deterministic subsampling.Performance of this framework is quite simple, straightforward and not costly.Only small adjustments are required to transform a traditional sensing system to a compressive sensing one.Practical examples include the famous coded aperture, radar system, CMOS compressed imaging, and so on.
The basic idea behind Romberg's work is that the randomness is designed in the frequency domain, where the spectrum of the random waveform has unit amplitude and independent random phase such that the random waveform is orthogonal with its shift which makes the convolution matrix orthogonal.In contrast, in our model the convolution matrix is not orthogonal and some frequency components of the interested signal will be filtered out.Accordingly, our model is not suitable for sensing signals which are sparse in the frequency domain; however, this sacrifice leads to an advantage of system realization.We showed that the suitability of the white convolution system for sensing a sparse signal depends on the coherence between the signal representation and the Fourier basis.Another important contribution is that the randomly selected sampling strategy is not necessary.We proved that subsampling at fixed locations also works well for the random convolution framework (Xiang et al., 2009).
For practical consideration, we assume to sense a continuous signal with limited control parameters.The signal is band-limited or approximated by a limited-resolution observation.For simplicity, we suppose the bandwidth of the signal to be 1.Since       , the full sampled data can be given by  . It comes back to the discrete framework of convolution.
However, the entries of the random sequence   , 1,..., are not independent of one another in general.
, where ,  is the Dirac function, we can still demonstrate the continuous problem by , which is independent when generated from a Gaussian distribution.Accordingly, we can extend our convolution model into the continuous case.
In summary, we analyzed the random convolution framework, i.e., convolving the tested signal with a white random waveform, followed by subsampling at fixed locations in the measurement domain, i.e., equal interval sampling.As an effect of the reduction in the freedom of randomness, the linear CS convolution system needs more measurements than the circular one with the same size.It also becomes inefficient when the waveform length is too short relative to the signal length.Indeed, in some applications the bandwidth of the random waveform is shorter than the tested signal.However, even in this case one may have super-resolution results when the original signal is sparse enough or when more prior information is used.

Application of Random Convolution Sampling Theory
The basic requirement of random convolution based sampling theory is to construct a system whose system response likes random number.In many applied fields such as communication, radar, navigation, etc., the point-spread response of system under complex wave environment usually is treated as random function in view of complex behaviour of propagation path and energy lossy.Especially, the amplitude-frequency response of it will be constant for ideal lossless system, which is required by Romberg's theory.Motivated by this observation, we will investigate wave phenomena in the random medium environment, and will explore its possibility of decreasing the number of signal sampled.
Before proceeding to details, let us qualitatively consider the simplified situation as shown in Figure 1.A source transmitting pulse of   s t is located at origin of coordinate and surrounded by rectangle-ring random media while a receiver is a detector is located at r ρ outside of this ring.It should be pointed out that in this work the structure is constructed by the dielectric (or metal) square cylinders with known electrical/geometrical parameters 8 r   randomly located around probed obstacles.Assuming that the signal from transmitter arrives at the receiver through N independent propagation paths {d i , i=1,2,…,N}, then the echoes recorded by the receiver can be approximately represented as where c is the light velocity, i a is decay coefficient for the ith propagation path.Readily we get the system response of random media as Theoretically the expected response required by random convolution theorem can be achieved by adjusting the value of   , , 1, 2,...
(equivalently, controlling random media); however, this is a challenge optimization problem due to nonlinear interaction between wave (whether electromagnetic or acoustic) and media, and should be deeply investigated in the near future.
From the viewpoint of electromagnetic/acoustic inverse problems, the above problem can be looked as the so-called inverse source problem; correspondingly the inverse scattering problem will be considered below, in particular, the diffraction tomography.Referring to Figure 1(b), three point targets are surrounded by the random media, while the transmitter and receiver are set to be outside the random medium.The location and reflectivity of ith target are i ρ and i  (i=1, 2,3), respectively, furthermore, we will assume there is no interaction between these three targets.Now our basic goal is to determine the distribution of reflectivity (i.e., find i from the time-domain data collected by the receiver located at r ρ .
The signal arriving at the ith objects is N is the number of wave paths connecting transmitter and ist object, , it j a and , it j d are the amplitude decay and electrical path of jth path, respectively.Now, the recorded wave scattered from 1st object can be expressed as , among ir N wave paths connecting the ist object and receiver, , ir j a and , ir j d are the amplitude decay and electrical path of jth path, respectively.Finally, the echo collected by receiver becomes which can be rewritten as in the frequency domain, , 1,2,...
where C is a constant, if the distance between any two pixels is larger than 0 0.25 , which means the use of random media can realize the super-resolution tomography ( 0 0.25 ) by the measurement configuration of one pair of transmitter/receiver, for convenience, call it compressive diffraction tomography.

Compressed sampling filter
We call a compressed-sampling filter (CSF) one for which the function relating the input signal to the output signal is pseudo-random (Li et al., 2009).In spirit of wave propagation in random media, one microwave circuit based CSFs working within the 2.0GHz to 4GHz to enable signal acquisition with sub-Nyquist sampling has been constructed, analyzed and tested.The results have empirically shown that by using the proposed architecture the S-sparse n-dimensional signal can be exactly reconstructed with   To investigate the performance of the proposed CSF, in particular, how many measurements M are required to reconstruct exactly the K-sparse N-dim signal by the proposed structure, the methodology commonly used in the field of compressive sampling is exploited to check empirically the relation between K, N and the number of measurements M. To end this, assuming the length of unknown sparse signal x is 400, and the sampling ratio is 1/3-Nyquist ratio.For each of 200 trials we randomly generate such sufficiently sparse vectors x (choosing the nonzero locations uniformly over the support in random and their values from N(0,1/400)).The graph presented in Fig. 3 shows that the success ratio for complex-valued data and real-valued data in recovering the true sparse signal, which shows that by the 1/3-Nyquist sampling, one realize the exact reconstruction of 120-sparse 400-dim signal for complex-valued measurements and 30-sparse 400-dim signal for real-valued measurements.Fig. 3 show that the for complex-valued measurements, the measurements with the order of O(Klog(N/K)) is enough to exactly reconstruct K-sparse N-dim signal; for real-valued measurements, the required measurements is order of O(Klog(N)).Moreover, this conclusion for signal which is sparse in the DCT and Harr-wavelet domain still exists.It should be pointed out that though the constructed CSF not satisfied the requirements from sampling theory studied previously, in particular, nonuniform amplitude-frequency response due to energy lossy dependent on frequency, and many other reasons, non-conjugate symmetry phase-frequency response, and so on, from the presented empirical results one can realize the exact reconstruction from the sub-Nyquist measurement by the proposed structure.
Reference to Figure 4, a simple example is demonstrated to show the application of proposed CSF in signal reconstruction, where the original signal which is the combination of two differential Gaussian pulses , where 0.6ns   and 9 0 2*10   rad are the input signal of the CSF, and the measured output signal is shown in Figure 4(b).The 1/3-Nyquist complex-valued measurements are used to reconstruct the original time-domain sparse signal and the reconstructed result Figure 4(a).

Compressive diffraction tomography
Another important application of random convolution theory is the well-known diffraction tomography (DT), where the electrical and geometrical properties of an object can be reconstructed from the diffracted field (Devaney, 1984;Pan and Kak, 1983).Under the assumption of weak scattering, one can use Born or Rytov approximations to derive the so-called Fourier diffraction projection theorem (FDPT), which relates the 2D Fourier transform of the object function to the measured data (Devaney, 1984;Pan and Kak, 1983).The input data of the FDPT algorithm and its variants should satisfy the so-called Nyquist-Shannon theorem to obtain the successful reconstruction; traditionally, the full-view and full-angle single-frequency measurements tightly packed over the sphere with radius k (k denotes the working wavenumber of homogeneous background media) are imperative.On the other hand, due to the use of Fourier transform the theoretical resolution limit of FDPT algorithm or its invariants are 0.5 (  is the working wavelength) (Devaney, 1984;Pan and Kak, 1983).Two appealing problems for the DT are how to design work system to relieve time-consuming data collection and how to improve the imaging resolution.In (Cui et al., 2004) by taking the rich information of decaying wave in the region of near field into account, an super-resolution imaging from the near-field measurement has been proposed within the framework of Born-based inversion.The further insight into the mechanism which leads to sub-wavelength resolution imaging from far-field measurement by means of multiple scattering has been carried out experimentally and theoretically in (Chen and Chew, 1998) and (Cui et al., 2004).On the other hand, the a prior information is that the gradient-magnitude image is highly sparse has been exploited to obtain perfect 0.5 resolution reconstruction from few-view and limited-angle Fourier data (LaRoque et al., 2008).In this paper we will focus on the use of random media to realize the "compressive" measurement in order to obtain super-resolution reconstruction from highly sparse data.
Here we restrict ourselves into two-dimensional scalar scattering problem as shown in Figure 1(b).The complex wave environment where the probed obstacles are located has been constructed; in particular, the random background medium to enforce the significant multi-path effect has been added.Under the assumption of Born approximation for weak scattering, the measurement signal   s j E k (j=1,2,…,M), representing the scattered fields whose working wavenumber is j k and is observed at R r due to incident field from the line source located at S r , can be expressed as where    r is assumed to be frequency-independent contrast function of probed objects

 
, ; is the incident wave due to the line source at S r ,i.e..   , ; is the Green's function for the random background media, which can be obtained and stored via in-situ measurement or numerical computation.Now (4) can be reformulated in the form of the matrix-vector, namely, where N is the total number for unknowns to be reconstructed, the entries of y come from   , ; r , n denotes the error due to measurement noise and discretization transformation from (4) to (5), the entries of A are x can be sparsely represented in some orthogonal basis Ψ such as the wavelet transform, the discrete cosine transform (DCT), and so on, i.e.,  x Ψθ , the equation (2.2) can be rewritten as For the purpose of the super-resolution reconstruction from the measurements of single receiver/transmitter pair, the formed complex-valued matrix A is highly underdetermined, i.e., M N .Obviously, the highly underdetermined system of linear equations (6) will generate infinite solutions.If one desires to narrow the choice to one well-defined solution, additional constraint should be imposed to guarantee a unique solution.To attack this, define the general optimization problem as where   J x is introduced to evaluate the desirability of a would-be solution x .Usually, the squared l2-norm   x , a measure of energy, is employed to obtain a unique solution.It is well known that the measure of energy is not optimal, even be a disaster in some cases.Fortunately, all most of obstacles to be probed for many applications such as medical imaging, geophysics, optical imaging, and so on, are sparse, or compressible in some framework such as DCT, wavelet, and so on.Consequently, the alternative to l2-norm regularization is so-called sparsity-promoted one, i.e.,   In the following a set of numerical simulations are provided to demonstrate above approach, where the investigation domain with size 2.4m by 2.4m is divided into the subgrids with size From Figs. 5, one can find that (a) the use of random media can realize the diffraction tomography even if just one pair of transmitter and receiver are employed which is impossible task if the objects are located at free space; (b) Although the reconstruction for the l2-constraint and random media background media can also yield visually reasonable reconstructions, its results are much poorer than the reconstruction with l1-norm constraints optimization, which can provide us super-resolution reconstruction.In summary, by compressive diffraction tomography (i.e., the random-media background plus sparse-enhance optimization), one can obtain exactly super-resolution reconstruction from the highly sparse data.

Robust Sparse Signal Reconstruction
In this section we will briefly discuss the algorithm of reconstructing sparse signal involved in previous sections.Generally, the considered problem formally can be stated via linear inverse problem with l1-norm based sparse constraint, in particular, noise n follows the Gaussian distribution with zero mean and covariance R , i.e.,

 
, n 0 R .In practice, due to non-idealities A in the analogy and digital implementation of CS, measurement mismatch or/and random malfunctioning of the instrument for measuring b, and other possible reasons, we have to explore the so-called robust optimization problem, in particular, where U we introduce uncertainty set, or the set of admissible disturbance of   , A b for constraint on perturbation   ,   A b .In this paper we assume that the perturbation matrix A and observations b are sparse, which means just a limited number of entries of  A or/and b are non-zero and non-ignorable.Correspondingly, the resulting problem of (9) can be reformulated as  , 2  and 3  are three regularization factors to be determined carefully.
We will provide new insight into problem of (9) within the framework of Bayesian analysis, whose distinguished advantages include 1) to do uncertainty analysis and 2) to estimate all of parameters involved adaptively.For the sake of simplicity, we assume  A 0. The first task of Bayesian analysis is to assign suitable probability distribution for uncertainty b and unknown signal x .Traditionally, the Gaussian distribution is the favourite assumption for modelling uncertainty in many research areas such as pattern recognition, computer vision, data clustering, and interpretation problems.However, even when such justification is absent, the Gaussian remains a popular choice due to its attractive analytical properties.For example, log likelihoods are often quadratic in parameters of interest, which facilitates optimization, and a Gaussian prior can be combined with a Gaussian likelihood analytically in Bayesian inference.However, as mentioned previously, both of  b and x are not controlled by a Gaussian, then the model based on the Gaussian assumption favours a Gaussian estimate that may interpret the data set in a misleading way.The results in an incorrect model for the data set, especially when possible outliers exist in the data set.To model this case, the Laplacian distribution about the noise is commonly explored; however, due to computational intractability, the so-called hierarchical Gaussian model is used instead (Tipping and Faul, 2003;Ji et al., 2008), in particular, , and the Gamma distribution is used to model the induced hyperparameters of B and Λ , i.e.,

 
. Now, the estimation of unknown x and induced hyperparameters of B and Λ can be readily made through the so-called type-I likelihood estimation, namely (Tipping and Faul, 2003;Li and Li., 2011;Ji et al., 2008), where the the posterior probability of x follows . Furthermore, we can estimate hyper-parameters of B and Λ under the type-II likelihood estimation, namely, After some basic manipulations, we can get Taking the gradient of L with respect to Λ, B , and setting them to be zeros, one can arrive at the closed-form solutions to Λ and B as (Li and Li., 2011) As discussed above, we can see that the proposed approach within framework of Bayesian analysis belongs to iterative one, in particularly, repeatedly computing ( 12), ( 15) and ( 16), which is computation demanding.

Conclusion
Compressive sensing (CS) has made a paradigmatic step in the way information is presented, stored, transmitted and recovered, by which we can acquire and reconstruct sparse signals from sub-Nyquist incoherent measurements.Three key ingredients of successfully implementing compressed sampling technique are sparsible/compressible probed signal, reliable hardware design, and low-cost computational algorithm.In this paper, we have investigated two aspects about the robust sampling of sparse signal, in particular, the design of compressed sampling hardware and the robust reconstruction via sparse Bayesian analysis.
A key to the success of CS is the design of the measurement ensemble satisfying the so-called restricted isometric property (RIP) or other rules.Theoretical and experimental results have shown that very limited random measurements of probed signals can suffice to provide valuable information of the signal subspace.On the other hand, the well-known time reversal theory shows that exploiting the multi-scattering of acoustic wave in random media can dramatically enhance the imaging resolution.In this paper, we first discussed the relationship between the wave scattering/propagation in complex medium and the classical compressive sensing theory; afterwards, study the design of circuit-level CS hardware prototype.The proposed variable sampling approach has the following advantages: 1) the generation of the measurement ensemble is computationally efficient and requires less memory; and 2) the proposed sampling hardware is universal from the viewpoint of compressive sensing, in particular, it can be applied to several transform domains and leads to simple implementations.In addition, another contribution of this paper is the development of robust sparse Bayesian analysis to address the robust optimization problem involved in sparse signal processing.
be the orthogonal basis in the sense of probability due to random sequences of   exactly what the random convolution theory needs.We can empirically show measurements with overwhelming probability.The proposed CSF (with size of 96mm by 60mm) is shown in Fig.2 (a), and the measured amplitude-frequency and phase-frequency responses of this CSF are provided in Fig.2(b) and Fig.2(c), respectively.So far there is no rigorous theoretical formulation for designing such microwave structure; however, one can design it under the guide of enriching the signal multi-scattering or multi-path effect to the maximum extent.
. The probed objects (see Fig. 5(a)) are assumed to be sparse in Harr-wavelet domain and their reconstructions are shown in Figs.5(b) and 5(c), where Fig. 5(b) is the result for diffraction tomography by l2-norm constraint while 5(c) is for l1-norm regularization.
Figure 3. Probability of success of developed CSF shown in Fig. 2 in the recovery of the sparest signal when the length of unknown signal is 400 and 1/3-Nyquist measurements, where x-axis denotes the cardinality of the solution, y-axis denotes the probability of success of the developed CSF shown in Fig.2 . Since then, a burst of intensive research activities in the new field of CS have been inspired in the past several years.