Semi-supervised Bayesian adaptive multiresolution shrinkage for wavelet-based denoising

We can use wavelet shrinkage to estimate a possibly multivariate regression function g under the general regression setup, y = g + ε. We propose an enhanced wavelet-based denoising methodology based on Bayesian adaptive multiresolution shrinkage, an effective Bayesian shrinkage rule in addition to the semi-supervised learning mechanism. The Bayesian shrinkage rule is advanced by utilizing the semi-supervised learning method in which the neighboring structure of a wavelet coefficient is adopted and an appropriate decision function is derived. According to decision function, wavelet coefficients follow one of two prespecified Bayesian rules obtained using varying related parameters. The decision of a wavelet coefficient depends not only on its magnitude, but also on the neighboring structure on which the coefficient is located. We discuss the theoretical properties of the suggested method and provide recommended parameter settings. We show that the proposed method is often superior to several existing wavelet denoising methods through extensive experimentation.     

Wavelet thresholding via a Bayesian approach

We discuss a Bayesian formalism which gives rise to a type of wavelet threshold estimation in nonparametric regression. A prior distribution is imposed on the wavelet coefficients of the unknown response function, designed to capture the sparseness of wavelet expansion that is common to most applications. For the prior specified, the posterior median yields a thresholding procedure. Our prior model for the underlying function can be adjusted to give functions falling in any specific Besov space. We establish a relationship between the hyperparameters of the prior model and the parameters of those Besov spaces within which realizations from the prior will fall. Such a relationship gives insight into the meaning of the Besov space parameters. Moreover, the relationship established makes it possible in principle to incorporate prior knowledge about the function's regularity properties into the prior model for its wavelet coefficients. However, prior knowledge about a function's regularity properties might be difficult to elicit; with this in mind, we propose a standard choice of prior hyperparameters that works well in our examples. Several simulated examples are used to illustrate our method, and comparisons are made with other thresholding methods. We also present an application to a data set that was collected in an anaesthesiological study.     

Empirical Bayes approach to wavelet regression using ϵ-contaminated priors

We consider an empirical Bayes approach to standard nonparametric regression estimation using a nonlinear wavelet methodology. Instead of specifying a single prior distribution on the parameter space of wavelet coefficients, which is usually the case in the existing literature, we elicit the ?-contamination class of prior distributions that is particularly attractive to work with when one seeks robust priors in Bayesian analysis. The type II maximum likelihood approach to prior selection is used by maximizing the predictive distribution for the data in the wavelet domain over a suitable subclass of the ?-contamination class of prior distributions. For the prior selected, the posterior mean yields a thresholding procedure which depends on one free prior parameter and it is level- and amplitude-dependent, thus allowing better adaptation in function estimation. We consider an automatic choice of the free prior parameter, guided by considerations on an exact risk analysis and on the shape of the thresholding rule, enabling the resulting estimator to be fully automated in practice. We also compute pointwise Bayesian credible intervals for the resulting function estimate using a simulation-based approach. We use several simulated examples to illustrate the performance of the proposed empirical Bayes term-by-term wavelet scheme, and we make comparisons with other classical and empirical Bayes term-by-term wavelet schemes. As a practical illustration, we present an application to a real-life data set that was collected in an atomic force microscopy study.     

Real nonparametric regression using complex wavelets

Summary.  Wavelet shrinkage is an effective nonparametric regression technique, especially when the underlying curve has irregular features such as spikes or discontinuities. The basic idea is simple: take the discrete wavelet transform of data consisting of a signal corrupted by noise; shrink or remove the wavelet coefficients to remove the noise; then invert the discrete wavelet transform to form an estimate of the true underlying curve. Various researchers have proposed increasingly sophisticated methods of doing this by using real-valued wavelets. Complex-valued wavelets exist but are rarely used. We propose two new complex-valued wavelet shrinkage techniques: one based on multiwavelet style shrinkage and the other using Bayesian methods. Extensive simulations show that our methods almost always give significantly more accurate estimates than methods based on real-valued wavelets. Further, our multiwavelet style shrinkage method is both simpler and dramatically faster than its competitors. To understand the excellent performance of this method we present a new risk bound on its hard thresholded coefficients.     

Wavelet analysis of uniformly time-modulated processes

Uniformly time-modulated processes (UTMP) offer a simple, though convenient model for many non stationary time series. However, in many applications, dependency is considered as a disturbing factor. For the UTMP using Weierstrass representation of modulating function, we demonstrated that the correlation between coefficients in wavelet domain is a decreasing function of number of wavelet vanishing moments and tends to zero in progressive rate as we move from finer scales to coarser scales.     

A Fast Wavelet Algorithm for Analyzing One-Dimensional Signal Processing and Asymptotic Distribution of Wavelet Coefficients With Numerical Example and Simulation

In this article, we consider a sample point (t _j , s _j ) including a value s _j  = f(t _j ) at height s _j and abscissa (time or location) t _j . We apply wavelet decomposition by using shifts and dilations of the basic Häar transform and obtain an algorithm to analyze a signal or function f. We use this algorithm in practical to approximating function by numerical example. Some relationships between wavelets coefficients and asymptotic distribution of wavelet coefficients are investigated. At the end, we illustrate the results on simulated data by using MATLAB and R software.     

Wavelet shrinkage for unequally spaced data

Wavelet shrinkage (WaveShrink) is a relatively new technique for nonparametric function estimation that has been shown to have asymptotic near-optimality properties over a wide class of functions. As originally formulated by Donoho and Johnstone, WaveShrink assumes equally spaced data. Because so many statistical applications (e.g., scatterplot smoothing) naturally involve unequally spaced data, we investigate in this paper how WaveShrink can be adapted to handle such data. Focusing on the Haar wavelet, we propose four approaches that extend the Haar wavelet transform to the unequally spaced case. Each approach is formulated in terms of continuous wavelet basis functions applied to a piecewise constant interpolation of the observed data, and each approach leads to wavelet coefficients that can be computed via a matrix transform of the original data. For each approach, we propose a practical way of adapting WaveShrink. We compare the four approaches in a Monte Carlo study and find them to be quite comparable in performance. The computationally simplest approach (isometric wavelets) has an appealing justification in terms of a weighted mean square error criterion and readily generalizes to wavelets of higher order than the Haar.     

Genetic Algorithm in the Wavelet Domain for Large p Small n Regression

Many areas of statistical modeling are plagued by the “curse of dimensionality,” in which there are more variables than observations. This is especially true when developing functional regression models where the independent dataset is some type of spectral decomposition, such as data from near-infrared spectroscopy. While we could develop a very complex model by simply taking enough samples (such that n > p), this could prove impossible or prohibitively expensive. In addition, a regression model developed like this could turn out to be highly inefficient, as spectral data usually exhibit high multicollinearity. In this article, we propose a two-part algorithm for selecting an effective and efficient functional regression model. Our algorithm begins by evaluating a subset of discrete wavelet transformations, allowing for variation in both wavelet and filter number. Next, we perform an intermediate processing step to remove variables with low correlation to the response data. Finally, we use the genetic algorithm to perform a stochastic search through the subset regression model space, driven by an information-theoretic objective function. We allow our algorithm to develop the regression model for each response variable independently, so as to optimally model each variable. We demonstrate our method on the familiar biscuit dough dataset, which has been used in a similar context by several researchers. Our results demonstrate both the flexibility and the power of our algorithm. For each response variable, a different subset model is selected, and different wavelet transformations are used. The models developed by our algorithm show an improvement, as measured by lower mean error, over results in the published literature.     

Detection and Estimation of Jump Points in Non parametric Regression Function with AR(1) Noise

In this paper, we propose a method based on wavelet analysis to detect and estimate jump points in non parametric regression function. This method is applied to AR(1) noise process under random design. First, the test statistics are constructed on the empirical wavelet coefficients. Then, under the null hypothesis, the critical values of test statistics are obtained. Under the alternative, the consistency of the test is proved. Afterward, the rate of convergence, the estimators of the number, and locations of change points are given theoretically. Finally, the excellent performance of our method is demonstrated through simulations using artificial and real datasets.     

Robust estimation and wavelet thresholding in partially linear models

This paper is concerned with a semiparametric partially linear regression model with unknown regression coefficients, an unknown nonparametric function for the non-linear component, and unobservable Gaussian distributed random errors. We present a wavelet thresholding based estimation procedure to estimate the components of the partial linear model by establishing a connection between an l ₁-penalty based wavelet estimator of the nonparametric component and Huber’s M-estimation of a standard linear model with outliers. Some general results on the large sample properties of the estimates of both the parametric and the nonparametric part of the model are established. Simulations are used to illustrate the general results and to compare the proposed methodology with other methods available in the recent literature.     

Wavelet Shrinkage with Double Weibull Prior

In this article, we propose a denoising methodology in the wavelet domain based on a Bayesian hierarchical model using Double Weibull prior. We propose two estimators, one based on posterior mean (Double Weibull Wavelet Shrinker, DWWS) and the other based on larger posterior mode (DWWS-LPM), and show how to calculate them efficiently. Traditionally, mixture priors have been used for modeling sparse wavelet coefficients. The interesting feature of this article is the use of non-mixture prior. We show that the methodology provides good denoising performance, comparable even to state-of-the-art methods that use mixture priors and empirical Bayes setting of hyperparameters, which is demonstrated by extensive simulations on standardly used test functions. An application to real-word dataset is also considered.     

On Combining Wavelets Expansion and Sparse Linear Models for Regression on Metabolomic Data and Biomarker Selection

Wavelet thresholding of spectra has to be handled with care when the spectra are the predictors of a regression problem. Indeed, a blind thresholding of the signal followed by a regression method often leads to deteriorated predictions. The scope of this article is to show that sparse regression methods, applied in the wavelet domain, perform an automatic thresholding: the most relevant wavelet coefficients are selected to optimize the prediction of a given target of interest. This approach can be seen as a joint thresholding designed for a predictive purpose. The method is illustrated on a real world problem where metabolomic data are linked to poison ingestion. This example proves the usefulness of wavelet expansion and the good behavior of sparse and regularized methods. A comparison study is performed between the two-steps approach (wavelet thresholding and regression) and the one-step approach (selection of wavelet coefficients with a sparse regression). The comparison includes two types of wavelet bases, various thresholding methods, and various regression methods and is evaluated by calculating prediction performances. Information about the location of the most important features on the spectra was also obtained and used to identify the most relevant metabolites involved in the mice poisoning.     

Detection of Change Point in Nonparametric Function with Unit-Root Noise by Wavelet

A wavelet method is proposed to detect jumps in a function which is observed with unit-root noise. We obtain critical values at any scale and prove the consistency of wavelet detection when the nonparametric function is smooth. It shows that the estimation of the number and locations of change points are consistent when there are change points in the nonparametric function. Simulation study supports our method.     

Non‐stationary Cross‐Covariance Models for Multivariate Processes on a Globe

Abstract. In geophysical and environmental problems, it is common to have multiple variables of interest measured at the same location and time. These multiple variables typically have dependence over space (and/or time). As a consequence, there is a growing interest in developing models for multivariate spatial processes, in particular, the cross‐covariance models. On the other hand, many data sets these days cover a large portion of the Earth such as satellite data, which require valid covariance models on a globe. We present a class of parametric covariance models for multivariate processes on a globe. The covariance models are flexible in capturing non‐stationarity in the data yet computationally feasible and require moderate numbers of parameters. We apply our covariance model to surface temperature and precipitation data from an NCAR climate model output. We compare our model to the multivariate version of the Matérn cross‐covariance function and models based on coregionalization and demonstrate the superior performance of our model in terms of AIC (and/or maximum loglikelihood values) and predictive skill. We also present some challenges in modelling the cross‐covariance structure of the temperature and precipitation data. Based on the fitted results using full data, we give the estimated cross‐correlation structure between the two variables.     

Wavelet processes and adaptive estimation of the evolutionary wavelet spectrum

This paper defines and studies a new class of non-stationary random processes constructed from discrete non-decimated wavelets which generalizes the Cramér (Fourier) representation of stationary time series. We define an evolutionary wavelet spectrum (EWS) which quantifies how process power varies locally over time and scale. We show how the EWS may be rigorously estimated by a smoothed wavelet periodogram and how both these quantities may be inverted to provide an estimable time-localized autocovariance. We illustrate our theory with a pedagogical example based on discrete non-decimated Haar wavelets and also a real medical time series example.     

Full Bayesian wavelet inference with a nonparametric prior

In this paper, we introduce a new Bayesian nonparametric model for estimating an unknown function in the presence of Gaussian noise. The proposed model involves a mixture of a point mass and an arbitrary (nonparametric) symmetric and unimodal distribution for modeling wavelet coefficients. Posterior simulation uses slice sampling ideas and the consistency under the proposed model is discussed. In particular, the method is shown to be computationally competitive with some of best Empirical wavelet estimation methods.     

Cross-correlating wavelet coefficients with applications to high-frequency financial time series

This paper uses a new concept in wavelet analysis to explore a financial transaction data set including returns, durations, and volume. The concept is based on a decomposition of the Allan covariance of two series into cross-covariances of wavelet coefficients, which allows a natural interpretation of cross-correlations in terms of frequencies. It is applied to financial transaction data including returns, durations between transactions, and trading volume. At high frequencies, we find significant spillover from durations to volume and a strong contemporaneous relation between durations and returns, whereas a strong causality between volume and volatility exists at various frequencies.     

Multivariate Bayes Wavelet shrinkage and applications

In recent years, wavelet shrinkage has become a very appealing method for data de-noising and density function estimation. In particular, Bayesian modelling via hierarchical priors has introduced novel approaches for Wavelet analysis that had become very popular, and are very competitive with standard hard or soft thresholding rules. In this sense, this paper proposes a hierarchical prior that is elicited on the model parameters describing the wavelet coefficients after applying a Discrete Wavelet Transformation (DWT). In difference to other approaches, the prior proposes a multivariate Normal distribution with a covariance matrix that allows for correlations among Wavelet coefficients corresponding to the same level of detail. In addition, an extra scale parameter is incorporated that permits an additional shrinkage level over the coefficients. The posterior distribution for this shrinkage procedure is not available in closed form but it is easily sampled through Markov chain Monte Carlo (MCMC) methods. Applications on a set of test signals and two noisy signals are presented.     

Solving Noisy ICA Using Multivariate Wavelet Denoising with an Application to Noisy Latent Variables Regression

A novel approach to solve the independent component analysis (ICA) model in the presence of noise is proposed. We use wavelets as natural denoising tools to solve the noisy ICA model. To do this, we use a multivariate wavelet denoising algorithm allowing spatial and temporal dependency. We propose also using a statistical approach, named nested design of experiments, to select the parameters such as wavelet family and thresholding type. This technique helps us to select more convenient combination of the parameters. This approach could be extended to many other problems in which one needs to choose parameters between many choices. The performance of the proposed method is illustrated on the simulated data and promising results are obtained. Also, the suggested method applied in latent variables regression in the presence of noise on real data. The good results confirm the ability of multivariate wavelet denoising to solving noisy ICA.     

Multiscale methods for data on graphs and irregular multidimensional situations

Summary.  For regularly spaced one-dimensional data, wavelet shrinkage has proven to be a compelling method for non-parametric function estimation. We create three new multiscale methods that provide wavelet-like transforms both for data arising on graphs and for irregularly spaced spatial data in more than one dimension. The concept of scale still exists within these transforms, but as a continuous quantity rather than dyadic levels. Further, we adapt recent empirical Bayesian shrinkage techniques to enable us to perform multiscale shrinkage for function estimation both on graphs and for irregular spatial data. We demonstrate that our methods perform very well when compared with several other methods for spatial regression for both real and simulated data. Although we concentrate on multiscale shrinkage (regression) we present our new 'wavelet transforms' as generic tools intended to be the basis of methods that might benefit from a multiscale representation of data either on graphs or for irregular spatial data.