Estimation of Variance Function in Heteroscedastic Regression Models by Generalized Coiflets

A wavelet approach is presented to estimate the variance function in heteroscedastic nonparametric regression model. The initial variance estimates are obtained as squared weighted sums of neighboring observations. The initial estimator of a smooth variance function is improved by means of wavelet smoothers under the situation that the samples at the dyadic points are not available. Since the traditional wavelet system for the variance function estimation is not appropriate in this situation, we demonstrate that the choice of the wavelet system is significant to have better performance. This is accomplished by choosing a suitable wavelet system known as the generalized coiflets. We conduct extensive simulations to evaluate finite sample performance of our method. We also illustrate our method using a real dataset.     

Wavelet shrinkage for unequally spaced data

Wavelet shrinkage (WaveShrink) is a relatively new technique for nonparametric function estimation that has been shown to have asymptotic near-optimality properties over a wide class of functions. As originally formulated by Donoho and Johnstone, WaveShrink assumes equally spaced data. Because so many statistical applications (e.g., scatterplot smoothing) naturally involve unequally spaced data, we investigate in this paper how WaveShrink can be adapted to handle such data. Focusing on the Haar wavelet, we propose four approaches that extend the Haar wavelet transform to the unequally spaced case. Each approach is formulated in terms of continuous wavelet basis functions applied to a piecewise constant interpolation of the observed data, and each approach leads to wavelet coefficients that can be computed via a matrix transform of the original data. For each approach, we propose a practical way of adapting WaveShrink. We compare the four approaches in a Monte Carlo study and find them to be quite comparable in performance. The computationally simplest approach (isometric wavelets) has an appealing justification in terms of a weighted mean square error criterion and readily generalizes to wavelets of higher order than the Haar.     

Functional logistic regression: a comparison of three methods

Functional logistic regression is becoming more popular as there are many situations where we are interested in the relation between functional covariates (as input) and a binary response (as output). Several approaches have been advocated, and this paper goes into detail about three of them: dimension reduction via functional principal component analysis, penalized functional regression, and wavelet expansions in combination with Least Absolute Shrinking and Selection Operator penalization. We discuss the performance of the three methods on simulated data and also apply the methods to data regarding lameness detection for horses. Emphasis is on classification performance, but we also discuss estimation of the unknown parameter function.     

Wavelet‐based estimators for mixture regression

We consider a process that is observed as a mixture of two random distributions, where the mixing probability is an unknown function of time. The setup is built upon a wavelet‐based mixture regression. Two linear wavelet estimators are proposed. Furthermore, we consider three regularizing procedures for each of the two wavelet methods. We also discuss regularity conditions under which the consistency of the wavelet methods is attained and derive rates of convergence for the proposed estimators. A Monte Carlo simulation study is conducted to illustrate the performance of the estimators. Various scenarios for the mixing probability function are used in the simulations, in addition to a range of sample sizes and resolution levels. We apply the proposed methods to a data set consisting of array Comparative Genomic Hybridization from glioblastoma cancer studies.     

Solving Noisy ICA Using Multivariate Wavelet Denoising with an Application to Noisy Latent Variables Regression

A novel approach to solve the independent component analysis (ICA) model in the presence of noise is proposed. We use wavelets as natural denoising tools to solve the noisy ICA model. To do this, we use a multivariate wavelet denoising algorithm allowing spatial and temporal dependency. We propose also using a statistical approach, named nested design of experiments, to select the parameters such as wavelet family and thresholding type. This technique helps us to select more convenient combination of the parameters. This approach could be extended to many other problems in which one needs to choose parameters between many choices. The performance of the proposed method is illustrated on the simulated data and promising results are obtained. Also, the suggested method applied in latent variables regression in the presence of noise on real data. The good results confirm the ability of multivariate wavelet denoising to solving noisy ICA.     

A wavelet-based approach for modelling exchange rates

This paper proposes a new approach, based on the recent developments of the wavelet theory, to model the dynamic of the exchange rate. First, we consider the maximum overlap discrete wavelet transform (MODWT) to decompose the level exchange rates into several scales. Second, we focus on modelling the conditional mean of the detrended series as well as their volatilities. In particular, we consider the generalized fractional, one-factor, Gegenbauer process (GARMA) to model the conditional mean and the fractionally integrated generalized autoregressive conditional heteroskedasticity process (FIGARCH) to model the conditional variance. Moreover, we estimate the GARMA-FIGARCH model using the wavelet-based maximum likelihood estimator (Whitcher in Technometrics 46:225–238, 2004). To illustrate the usefulness of our methodology, we carry out an empirical application using the daily Tunisian exchange rates relative to the American Dollar, the Euro and the Japanese Yen. The empirical results show the relevance of the selected modelling approach which contributes to a better forecasting performance of the exchange rate series.     

Testing for spurious and cointegrated regressions: A wavelet approach

This paper proposes a wavelet-based approach to analyze spurious and cointegrated regressions in time series. The approach is based on the properties of the wavelet covariance and correlation in Monte Carlo studies of spurious and cointegrated regression. In the case of the spurious regression, the null hypotheses of zero wavelet covariance and correlation for these series across the scales fail to be rejected. Conversely, these null hypotheses across the scales are rejected for the cointegrated bivariate time series. These nonresidual-based tests are then applied to analyze if any relationship exists between the extraterrestrial phenomenon of sunspots and the earthly economic time series of oil prices. Conventional residual-based tests appear sensitive to the specification in both the cointegrating regression and the lag order in the augmented Dickey–Fuller tests on the residuals. In contrast, the wavelet tests, with their bootstrap t-statistics and confidence intervals, detect the spuriousness of this relationship.     

Empirical Bayes approach to wavelet regression using ϵ-contaminated priors

We consider an empirical Bayes approach to standard nonparametric regression estimation using a nonlinear wavelet methodology. Instead of specifying a single prior distribution on the parameter space of wavelet coefficients, which is usually the case in the existing literature, we elicit the ?-contamination class of prior distributions that is particularly attractive to work with when one seeks robust priors in Bayesian analysis. The type II maximum likelihood approach to prior selection is used by maximizing the predictive distribution for the data in the wavelet domain over a suitable subclass of the ?-contamination class of prior distributions. For the prior selected, the posterior mean yields a thresholding procedure which depends on one free prior parameter and it is level- and amplitude-dependent, thus allowing better adaptation in function estimation. We consider an automatic choice of the free prior parameter, guided by considerations on an exact risk analysis and on the shape of the thresholding rule, enabling the resulting estimator to be fully automated in practice. We also compute pointwise Bayesian credible intervals for the resulting function estimate using a simulation-based approach. We use several simulated examples to illustrate the performance of the proposed empirical Bayes term-by-term wavelet scheme, and we make comparisons with other classical and empirical Bayes term-by-term wavelet schemes. As a practical illustration, we present an application to a real-life data set that was collected in an atomic force microscopy study.     

HAZARD RATE ESTIMATION FOR CENSORED DATA BY WAVELET METHODS

ABSTRACT

We study the estimation of a hazard rate function based on censored data by non-linear wavelet method. We provide an asymptotic formula for the mean integrated squared error (MISE) of nonlinear wavelet-based hazard rate estimators under randomly censored data. We show this MISE formula, when the underlying hazard rate function and censoring distribution function are only piecewise smooth, has the same expansion as analogous kernel estimators, a feature not available for the kernel estimators. In addition, we establish an asymptotic normality of the nonlinear wavelet estimator.     

Finite Sample Comparison of Parametric,Semiparametric, and Wavelet Estimators of Fractional Integration

ABSTRACT

In this paper we compare through Monte Carlo simulations the finite sample properties of estimators of the fractional differencing parameter, d. This involves frequency domain, time domain, and wavelet based approaches, and we consider both parametric and semiparametric estimation methods. The estimators are briefly introduced and compared, and the criteria adopted for measuring finite sample performance are bias and root mean squared error. Most importantly, the simulations reveal that (1) the frequency domain maximum likelihood procedure is superior to the time domain parametric methods, (2) all the estimators are fairly robust to conditionally heteroscedastic errors, (3) the local polynomial Whittle and bias-reduced log-periodogram regression estimators are shown to be more robust to short-run dynamics than other semiparametric (frequency domain and wavelet) estimators and in some cases even outperform the time domain parametric methods, and (4) without sufficient trimming of scales the wavelet-based estimators are heavily biased.     

Finite Sample Comparison of Parametric, Semiparametric, and Wavelet Estimators of Fractional Integration

In this paper we compare through Monte Carlo simulations the finite sample properties of estimators of the fractional differencing parameter, d. This involves frequency domain, time domain, and wavelet based approaches, and we consider both parametric and semiparametric estimation methods. The estimators are briefly introduced and compared, and the criteria adopted for measuring finite sample performance are bias and root mean squared error. Most importantly, the simulations reveal that (1) the frequency domain maximum likelihood procedure is superior to the time domain parametric methods, (2) all the estimators are fairly robust to conditionally heteroscedastic errors, (3) the local polynomial Whittle and bias-reduced log-periodogram regression estimators are shown to be more robust to short-run dynamics than other semiparametric (frequency domain and wavelet) estimators and in some cases even outperform the time domain parametric methods, and (4) without sufficient trimming of scales the wavelet-based estimators are heavily biased.     

Nonparametric estimation of a two dimensional continuous–discrete density function by wavelets

We consider the estimation of a two dimensional continuous–discrete density function. A new methodology based on wavelets is proposed. We construct a linear wavelet estimator and a non-linear wavelet estimator based on a term-by-term thresholding. Their rates of convergence are established under the mean integrated squared error over Besov balls. In particular, we prove that our adaptive wavelet estimator attains a fast rate of convergence. A simulation study illustrates the usefulness of the proposed estimators.     

A Note on the Effect of Wavelet Choice on the Estimation of the Evolutionary Wavelet Spectrum

The locally stationary wavelet process model assumes some underlying wavelet family in order to generate the process. Analyses of such processes also assume that the same wavelet family is used to obtain unbiased estimates of the wavelet spectrum. In practice this would not typically be possible since, a priori, the underlying wavelet family is not known. This article considers the effect of wavelet choice within this setting. A particular focus is given to the estimation of the evolutionary wavelet spectrum due to its importance in many reported applications.     

Full Bayesian wavelet inference with a nonparametric prior

In this paper, we introduce a new Bayesian nonparametric model for estimating an unknown function in the presence of Gaussian noise. The proposed model involves a mixture of a point mass and an arbitrary (nonparametric) symmetric and unimodal distribution for modeling wavelet coefficients. Posterior simulation uses slice sampling ideas and the consistency under the proposed model is discussed. In particular, the method is shown to be computationally competitive with some of best Empirical wavelet estimation methods.     

Wavelet Estimator in Nonparametric Regression Model with Dependent Error’s Structure

In this article, we consider a nonparametric regression model with replicated observations based on the dependent error’s structure, for exhibiting dependence among the units. The wavelet procedures are developed to estimate the regression function. The moment consistency, the strong consistency, strong convergence rate and asymptotic normality of wavelet estimator are established under suitable conditions. A simulation study is undertaken to assess the finite sample performance of the proposed method.     

Time–Threshold Maps: Using information from wavelet reconstructions with all threshold values simultaneously

Wavelets are a commonly used tool in science and technology. Often, their use involves applying a wavelet transform to the data, thresholding the coefficients and applying the inverse transform to obtain an estimate of the desired quantities. In this paper, we argue that it is often possible to gain more insight into the data by producing not just one, but many wavelet reconstructions using a range of threshold values and analysing the resulting object, which we term the Time–Threshold Map (TTM) of the input data. We discuss elementary properties of the TTM, in its “basic” and “derivative” versions, using both Haar and Unbalanced Haar wavelet families. We then show how the TTM can help in solving two statistical problems in the signal + noise model: breakpoint detection, and estimating the longest interval of approximate stationarity. We illustrate both applications with examples involving volatility of financial returns. We also briefly discuss other possible uses of the TTM.     

A trimmed translation-invariant denoising estimator

A popular wavelet method for estimating jumps in functions is through the use of the translation-invariant (TI) estimator. The TI estimator addresses a particular problem, the susceptibility of the wavelet estimates to the location of the features in a function with respect to the support of the wavelet basis functions. The TI estimator reduces this reliance by cycling the data through a set of shifts, thus changing the relation between the wavelet support and the jump location. However, a drawback of the TI estimator is that it includes every shifted analysis in the reconstruction, even those that may reduce, rather than improve, the effectiveness of the method. In this paper, we propose a method that modifies the TI estimator to improve the jump reconstruction in terms of the mean squared errors of the reconstructions and visual performance. Information from the set of shifted data sets is used to mimic the performance of an oracle which knows exactly which are the best TI shifts to retain in the reconstruction. The TI estimate is a special case of the proposed method. A simulation study comparing this proposed method to the existing wavelet estimators and the oracle is provided.     

Asymptotically minimax non–parametric function estimation with positivity constraints I

The spatially inhomogeneous smoothness of the non-parametric density or regression-function to be estimated by non-parametric methods is often modelled by Besov- and Triebel-type smoothness constraints. For such problems, Donoho and Johnstone [D.L. Donoho and I.M. Johnstone, Minimax estimation via wavelet shrinkage. Ann. Stat. 26 (1998), pp. 879–921.], Delyon and Juditsky [B. Delyon and A. Juditsky, On minimax wavelet estimators, Appl. Comput. Harmon. Anal. 3 (1996), pp. 215–228.] studied minimax rates of convergence for wavelet estimators with thresholding, while Lepski et al. [O.V. Lepski, E. Mammen, and V.G. Spokoiny, Optimal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimators with variable bandwidth selectors, Ann. Stat. 25 (1997), pp. 929–947.] proposed a variable bandwidth selection for kernel estimators that achieved optimal rates over Besov classes. However, a second challenge in many real applications of non-parametric curve estimation is that the function must be positive. Here, we show how to construct estimators under positivity constraints that satisfy these constraints and also achieve minimax rates over the appropriate smoothness class.     

Wavelet thresholding via a Bayesian approach

We discuss a Bayesian formalism which gives rise to a type of wavelet threshold estimation in nonparametric regression. A prior distribution is imposed on the wavelet coefficients of the unknown response function, designed to capture the sparseness of wavelet expansion that is common to most applications. For the prior specified, the posterior median yields a thresholding procedure. Our prior model for the underlying function can be adjusted to give functions falling in any specific Besov space. We establish a relationship between the hyperparameters of the prior model and the parameters of those Besov spaces within which realizations from the prior will fall. Such a relationship gives insight into the meaning of the Besov space parameters. Moreover, the relationship established makes it possible in principle to incorporate prior knowledge about the function's regularity properties into the prior model for its wavelet coefficients. However, prior knowledge about a function's regularity properties might be difficult to elicit; with this in mind, we propose a standard choice of prior hyperparameters that works well in our examples. Several simulated examples are used to illustrate our method, and comparisons are made with other thresholding methods. We also present an application to a data set that was collected in an anaesthesiological study.     

Genetic Algorithm in the Wavelet Domain for Large p Small n Regression

Many areas of statistical modeling are plagued by the “curse of dimensionality,” in which there are more variables than observations. This is especially true when developing functional regression models where the independent dataset is some type of spectral decomposition, such as data from near-infrared spectroscopy. While we could develop a very complex model by simply taking enough samples (such that n > p), this could prove impossible or prohibitively expensive. In addition, a regression model developed like this could turn out to be highly inefficient, as spectral data usually exhibit high multicollinearity. In this article, we propose a two-part algorithm for selecting an effective and efficient functional regression model. Our algorithm begins by evaluating a subset of discrete wavelet transformations, allowing for variation in both wavelet and filter number. Next, we perform an intermediate processing step to remove variables with low correlation to the response data. Finally, we use the genetic algorithm to perform a stochastic search through the subset regression model space, driven by an information-theoretic objective function. We allow our algorithm to develop the regression model for each response variable independently, so as to optimally model each variable. We demonstrate our method on the familiar biscuit dough dataset, which has been used in a similar context by several researchers. Our results demonstrate both the flexibility and the power of our algorithm. For each response variable, a different subset model is selected, and different wavelet transformations are used. The models developed by our algorithm show an improvement, as measured by lower mean error, over results in the published literature.