In signal processing, the Wiener filter is a filter proposed by Norbert Wiener during the 1940s and published in 1949.Its purpose is to reduce the amount of noise present in a signal by comparison with an estimation of the desired noiseless signal. The discrete-time equivalent of Wiener's work was derived independently by Kolmogorov and published in 1941. Hence the theory is often called the Wiener-Kolmogorov filtering theory. The Wiener-Kolmogorov was the first statistically designed filter to be proposed and subsequently gave rise to many others including the famous Kalman filter. A Wiener filter is not an adaptive filter because the theory behind this filter assumes that the inputs are stationary.[2]

Contents[hide]

1 Description 2 Wiener filter problem setup 3 Wiener filter solutions o 3.1 Noncausal solution o 3.2 Causal solution 4 Finite Impulse Response Wiener filter for discrete series o 4.1 Relationship to the least mean squares filter 5 See also 6 References 7 External links

DescriptionThe goal of the Wiener filter is to filter out noise that has corrupted a signal. It is based on a statistical approach. Typical filters are designed for a desired frequency responce. However, the design of the Wiener filter takes a different approach. One is assumed to have knowledge of the spectral properties of the original signal and the noise, and one seeks the linear time-invariant filter whose output would come as close to the original signal as possible. Wiener filters are characterized by the following: 1. Assumption: signal and (additive) noise are stationary linear stochastic processes with known spectral characteristics or known autocorrelation and cross-correlation 2. Requirement: the filter must be physically realizable/causal (this requirement can be dropped, resulting in a non-causal solution) 3. Performance criterion: minimum mean-square error (MMSE) This filter is frequently used in the process of deconvolution; for this application, see Wiener deconvolution.

[edit] Wiener filter problem setup

The input to the Wiener filter is assumed to be a signal, , corrupted by additive noise, The output, , is calculated by means of a filter, , using the following convolution:[3]

.

where

is the original signal (not exactly known; to be estimated) is the noise is the estimated signal (the intention is to equal ) is the Wiener filter's impulse response

The error is defined as

where

is the delay of the Wiener filter (since it is causal)

In other words, the error is the difference between the estimated signal and the true signal shifted by . The squared error is

where

is the desired output of the filter is the error , the problem can be described as follows:

Depending on the value of

If then the problem is that of prediction (error is reduced when is similar to a later value of s) If then the problem is that of filtering (error is reduced when is similar to ) If then the problem is that of smoothing (error is reduced when is similar to an earlier value of s) as a convolution integral:

Writing

Taking the expected value of the squared error results in

where

is the observed signal is the autocorrelation function of is the autocorrelation function of is the cross-correlation function of

and is zero), then

If the signal and the noise this means that

are uncorrelated (i.e., the cross-correlation

For many applications, the assumption of uncorrelated signal and noise is reasonable. The goal is to minimize , the expected value of the squared error, by finding the optimal , the Wiener filter impulse response function. The minimum may be found by calculating the first order incremental change in the least square error resulting from an incremental change in g(.) for positive time. This is

For a minimum, this must vanish identically for all

which leads to the Wiener-Hopf equation

This is the fundamental equation of the Wiener theory. The right-hand side resembles a convolution but is only over the semi-infinite range. The equation can be solved by a special technique due to Wiener and Hopf.

Wiener filter solutions :The Wiener filter problem has solutions for three possible cases : one where a non causal filter is acceptable ( requiring an infinite amount of both past and future data ) , the case where a causal filter is desired ( using an infinite amount of past data ) , and the finite impulse response ( FIR )

case where a finite amount of past data is used. The first case is simple to solve but is not suited for real-time applications. Wiener's main accomplishment was solving the case where the causality requirement is in effect, and in an appendix of Wiener's book Levinson gave the FIR solution.

Noncausal solution

Provided that

is optimal, then the minimum mean-square error equation reduces to

and the solution

is the inverse two-sided Laplace transform of

.

[edit] Causal solution

where

consists of the causal part of (that is, that part of this fraction having a positive time solution under the inverse Laplace transform) is the causal component of (i.e., the inverse Laplace transform of is non-zero only for ) is the anti-causal component of (i.e., the inverse Laplace transform of non-zero only for )

is

This general formula is complicated and deserves a more detailed explanation. To write down the solution in a specific case, one should follow these steps:[4] 1. Start with the spectrum components: in rational form and factor it into causal and anti-causal

where contains all the zeros and poles in the left hand plane (LHP) and contains the zeroes and poles in the right hand plane (RHP). This is called the WienerHopf factorization. 1. Divide by and write out the result as a partial fraction expansion. 2. Select only those terms in this expansion having poles in the LHP. Call these terms

.

3. Divide

by

. The result is the desired filter transfer function

.

[edit] Finite Impulse Response Wiener filter for discrete series

Block diagram view of the FIR Wiener filter for discrete series. An input signal w[n] is convolved with the Wiener filter g[n] and the result is compared to a reference signal s[n] to obtain the filtering error e[n]. The causal finite impulse response (FIR) Wiener filter, instead of using some given data matrix X and output vector Y, finds optimal tap weights by using the statistics of the input and output signals. It populates the input matrix X with estimates of the auto-correlation of the input signal (T) and populates the output vector Y with estimates of the cross-correlation between the output and input signals (V). In order to derive the coefficients of the Wiener filter, consider the signal w[n] being fed to a Wiener filter of order N and with coefficients , . The output of the filter is denoted x[n] which is given by the expression

The residual error is denoted e[n] and is defined as e[n] = x[n] s[n] (see the corresponding block diagram). The Wiener filter is designed so as to minimize the mean square error (MMSE criteria) which can be stated concisely as follows:

where denotes the expectation operator. In the general case, the coefficients may be complex and may be derived for the case where w[n] and s[n] are complex as well. With a complex signal, the matrix to be solved is a Hermitian Toeplitz matrix, rather than Symmetric Toeplitz matrix. For simplicity, the following considers only the case where all these quantities are real. The mean square error (MSE) may be rewritten as:

To find the vector respect to

which minimizes the expression above, calculate its derivative with

Assuming that w[n] and s[n] are each stationary and jointly stationary, the sequences and known respectively as the autocorrelation of w[n] and the cross-correlation between w[n] and s[n] can be defined as follows:

The derivative of the MSE may therefore be rewritten as (notice that

)

Letting the derivative be equal to zero results in

which can be rewritten in matrix form

These equations are known as the Wiener-Hopf equations. The matrix T appearing in the equation is a symmetric Toeplitz matrix. These matrices are known to be positive definite and therefore non-singular yielding a unique solution to the determination of the Wiener filter coefficient vector, . Furthermore, there exists an efficient algorithm to solve such Wiener-Hopf equations known as the Levinson-Durbin algorithm so an explicit inversion of is not required.

[edit] Relationship to the least mean squares filter

The realization of the causal Wiener filter looks a lot like the solution to the least squares estimate, except in the signal processing domain. The least squares solution, for input matrix and output vector is

The FIR Wiener filter is related to the least mean squares filter, but minimizing its error criterion does not rely on cross-correlations or auto-correlations. Its solution converges to the Wiener filter solution. .. In mathematics, Wiener deconvolution is an application of the Wiener filter to the noise problems inherent in deconvolution. It works in the frequency domain, attempting to minimize the impact of deconvoluted noise at frequencies which have a poor signal-to-noise ratio. The Wiener deconvolution method has widespread use in image deconvolution applications, as the frequency spectrum of most visual images is fairly well behaved and may be estimated easily. Wiener deconvolution is named after Norbert Wiener.

WIENER FILTERING: To find the filter to shape a wavelet to another wavelet is not an exact process, but the filter which produces the closest result can be obtained by a mathematical technique known as least squares. The Wiener filter is that which best (in a least squares sense) shapes a given wavelet to a desired wavelet. Applications include shaping a source wavelet to it's minimum phase equivalent, shaping a wavelet within the data to a spike (to improve resolution) or to shape a time-series with multiples to one without multiples (predictive deconvolution). Without going into the mathematics it turns out that the filter is found by dividing the cross-correlation of the input with the desired output by the auto-correlation of the input. This solution sets up a series of simultaneous equations which are solved rapidly in the computer by matrix inversion using the Levinson algorithm. A certain percentage of noise (called white noise or white light) is added to stabilise the inversion program.

Description The two-step noise reduction (TSNR) technique removes the annoying reverberation effect while maintaining the benefits of the decision-directed approach. However, classic short-time noise

reduction techniques, including TSNR, introduce harmonic distortion in the enhanced speech. To overcome this problem, a method called harmonic regeneration noise reduction (HRNR) is implemented in order to refine the a priori SNR used to compute a spectral gain able to preserve the speech harmonics as proposed by Plapous et al. ("Improved Signal-to-Noise Ratio Estimation for Speech Enhancement", IEEE Transactions on ASLP, Vol. 14, Issue 6, pp. 2098 - 2108, Nov. 2006). MATLAB release MATLAB 7.6 (R2008a)

Coding of the FFT Wiener Filter

The source code for the FFT Wiener Filter was created in Matlab as this is a strong mathematical programming language with fast in built Fourier functions which will increase the efficiency of the program . My experience in Matlab has been fairly limited in the past and hence studying Wiener Filtering in Matlab gives me an opportunity to learn a new language . The basic premise of the code below is taking the Fourier transform of the image and multiplying it by the transform of the filter function(h) . The process elimates low frequency signals in the Fourier domain which is the noise and emphasizes the high frequency signals which is the line . The output image is then created by taking the inverse Fourier transform of the product . The filter multiplies each pixel in the Fourier image by this filter to give the final filtered image . The original model for the FFT Wiener Filter code was taken from a website http://www.owlnet.rice.edu/~elec539/Projects99/BACH/proj2/wiener.html although many alterations have been made to the original Matlab functions and the program restructured. The original code was about twice as long and did not operate as efficiently or as effectively as it does now. Originally the code could only just make out a line at an intensity of 1.5 above the noise mean. Now it clearly shows a line at an intensity of only 0.5 above the noise mean. I have also commented all steps in the code so the Wiener Filtering Process may be followed and understood in relation to the code.

Matlab Code for Fourier Transform Wiener FilterWienerDriverclear; clf; %generate random noise array SIZE = 128; N = randn(SIZE,SIZE); components %generates a random Gaussian matrix with SIZExSIZE

N(50,:)=N(50,:) + 2; figure(1) colormap(jet) image(N*50) h = ones(10,10)/16;

%line is x=50 and raised by 2 above the noise mean.

%multiplies each pixel in the image by 50 %Filter function

sigma=1; % should always be 1 for a gaussian noise distribution as stated in the Matlab RANDN function Xf = fft2(N); %Fourier Transform of supplied image Hf = fft2(h,SIZE,SIZE); %Fourier Transform of blurring filter y = real(ifft2(Hf.*Xf)); %y is the real inverse fourier transform of the Fourier transform of %the initial image multiplied by the Fourier Transform of the filter %function (h). This creates the output image y % restoration using generalized Wiener filtering gamma = 1; alpha = 1; ewx = wienerFilter(y,h,sigma,gamma,alpha); figure(2) colormap(jet) image(ewx*50) return %magnifies each pixel in the image by 50

WienerFilterfunction ex = wienerFilter(y,h,sigma,gamma,alpha); % % ex = wienerFilter(y,h,sigma,gamma,alpha); % % Generalized Wiener filter using parameter alpha. When % alpha = 1,(The noise Power) it is the Wiener filter. It is also called % Regularized inverse filter. % SIZE = size(y,1); Yf = fft2(y); filter function Hf = fft2(h,SIZE,SIZE); Pyf = abs(Yf).^2/SIZE^2; %Fourier transform of original image N by % Fourier transform of filter function %Fourier transform of filter

sHf = Hf.*(abs(Hf)>0)+1/gamma*(abs(Hf)==0); function + its inverse iHf = 1./sHf; %inverse

iHf = iHf.*(abs(Hf)*gamma>1)+gamma*abs(sHf).*iHf.*(abs(sHf)*gammasigma^2)+sigma^2*(Pyf10 (mean=0). Table 3 presents the Matlab code to add white noise to an image. The noise is simulated by random values in the standard deviation area. % 3. Add a gaussian (white) noise: variance v>10 (mean=0) variance = 11; % Must be bigger than 10 std_dev = sqrt(variance); noise = std_dev.*randn(size(iBlur)); iBlurNoise = iBlur + noise;Table 3. Step 3

The resulting image after the addition of noise is presented in figure 4:Figure 4. Noisy and blurred image.

4. Perform a restoration of this built image by using the simplified Wiener restoration filtering:

HK WH + =2 *where the restored image

^

F

is obtained from the observed (degraded) image G in the Fourier space by:

F =WG^

Give the K value leading to the best visual restoration.In order to compute the K value leading to the best visual restoration an array from 0.001 to 0.1 with 100 intervals has been computed. Each value has been assigned to a specific K and the restoration has been performed. The selected K value corresponds with the minimum error obtained between the original image and the restored one. The Matlab code is presented in Table 4: Wiener filtering GARCIA Frederic VIBOT MSc

-6% 4. Perform a restoration of this built image by using the simplifier H = fft2(h); % Transform the filter into Fourier domain % Give the K value leading to the best visual restoration, calculated % by minimazing the error btw the initial image and the restored image K = linspace(0.001,0.1,100); errorVect = zeros(1,100); for i=1:length(K) % Generate restored Wiener filter W = conj(H)./(abs(H).^2 + K(i)); % Apply filter G = fft2(iBlurNoise); F = W.*G; iRestored = uint8(ifft2(F)); % Calculate error error = uint8(iSrc) - iRestored; errorVect(i) = mean(error(:))^2; end % Retrieve minimum error [minErrorValue minErrorPos] = min(errorVect); idealK = K(minErrorPos); W = conj(H)./(abs(H).^2 + idealK); G = fft2(iBlurNoise); F = W.*G; iRestored = ifft2(F);Table 4. Step 4

And the restored image is presented in figure 5:Figure 5. Restored image.

Figure 6 presents the results of all the computations until this step: Wiener filtering GARCIA Frederic VIBOT MSc

-7Figure 6. Results of the steps 1-4.

Although the information about the spectrum of the undistorted image and also of the noise is known, it is not possible to achieve a perfect restoration. The main reason is the random nature of noise 5. Try to introduce two different blurs: one (s1) for the square and another (s2