Robust estimation and wavelet thresholding in partially linear models

This paper is concerned with a semiparametric partially linear regression model with unknown regression coefficients, an unknown nonparametric function for the non-linear component, and unobservable Gaussian distributed random errors. We present a wavelet thresholding based estimation procedure to estimate the components of the partial linear model by establishing a connection between an l ₁-penalty based wavelet estimator of the nonparametric component and Huber’s M-estimation of a standard linear model with outliers. Some general results on the large sample properties of the estimates of both the parametric and the nonparametric part of the model are established. Simulations are used to illustrate the general results and to compare the proposed methodology with other methods available in the recent literature.     

A Fast Wavelet Algorithm for Analyzing One-Dimensional Signal Processing and Asymptotic Distribution of Wavelet Coefficients With Numerical Example and Simulation

In this article, we consider a sample point (t _j , s _j ) including a value s _j  = f(t _j ) at height s _j and abscissa (time or location) t _j . We apply wavelet decomposition by using shifts and dilations of the basic Häar transform and obtain an algorithm to analyze a signal or function f. We use this algorithm in practical to approximating function by numerical example. Some relationships between wavelets coefficients and asymptotic distribution of wavelet coefficients are investigated. At the end, we illustrate the results on simulated data by using MATLAB and R software.     

Testing for additivity and joint effects in multivariate nonparametric regression using Fourier and wavelet methods

We consider the problem of testing for additivity and joint effects in multivariate nonparametric regression when the data are modelled as observations of an unknown response function observed on a d-dimensional (d 2) lattice and contaminated with additive Gaussian noise. We propose tests for additivity and joint effects, appropriate for both homogeneous and inhomogeneous response functions, using the particular structure of the data expanded in tensor product Fourier or wavelet bases studied recently by Amato and Antoniadis (2001) and Amato, Antoniadis and De Feis (2002). The corresponding tests are constructed by applying the adaptive Neyman truncation and wavelet thresholding procedures of Fan (1996), for testing a high-dimensional Gaussian mean, to the resulting empirical Fourier and wavelet coefficients. As a consequence, asymptotic normality of the proposed test statistics under the null hypothesis and lower bounds of the corresponding powers under a specific alternative are derived. We use several simulated examples to illustrate the performance of the proposed tests, and we make comparisons with other tests available in the literature.     

Multiscale detection and location of multiple variance changes in the presence of long memory

Procedures for detecting change points in sequences of correlated observations (e.g., time series) can help elucidate their complicated structure. Current literature on the detection of multiple change points emphasizes the analysis of sequences of independent random variables. We address the problem of an unknown number of variance changes in the presence of long-range dependence (e.g., long memory processes). Our results are also applicable to time series whose spectrum slowly varies across octave bands. An iterated cumulative sum of squares procedure is introduced in order to look at the multiscale stationarity of a time series; that is, the variance structure of the wavelet coefficients on a scale by scale basis. The discrete wavelet transform enables us to analyze a given time series on a series of physical scales. The result is a partitioning of the wavelet coefficients into locally stationary regions. Simulations are performed to validate the ability of this procedure to detect and locate multiple variance changes. A ‘time’ series of vertical ocean shear measurements is also analyzed, where a variety of nonstationary features are identified.     

Bayesian analysis of threshold autoregressions

A nonasymptotic Bayesian approach is developed for analysis of data from threshold autoregressive processes with two regimes. Using the conditional likelihood function, the marginal posterior distribution for each of the parameters is derived along with posterior means and variances. A test for linear functions of the autoregressive coefficients is presented. The approach presented uses a posterior p-value averaged over the values of the threshold. The one-step ahead predictive distribution is derived along with the predictive mean and variance. In addition, equivalent results are derived conditional upon a value of the threshold. A numerical example is presented to illustrate the approach.     

Wavelet Shrinkage with Double Weibull Prior

In this article, we propose a denoising methodology in the wavelet domain based on a Bayesian hierarchical model using Double Weibull prior. We propose two estimators, one based on posterior mean (Double Weibull Wavelet Shrinker, DWWS) and the other based on larger posterior mode (DWWS-LPM), and show how to calculate them efficiently. Traditionally, mixture priors have been used for modeling sparse wavelet coefficients. The interesting feature of this article is the use of non-mixture prior. We show that the methodology provides good denoising performance, comparable even to state-of-the-art methods that use mixture priors and empirical Bayes setting of hyperparameters, which is demonstrated by extensive simulations on standardly used test functions. An application to real-word dataset is also considered.     

Semi-supervised Bayesian adaptive multiresolution shrinkage for wavelet-based denoising

We can use wavelet shrinkage to estimate a possibly multivariate regression function g under the general regression setup, y = g + ε. We propose an enhanced wavelet-based denoising methodology based on Bayesian adaptive multiresolution shrinkage, an effective Bayesian shrinkage rule in addition to the semi-supervised learning mechanism. The Bayesian shrinkage rule is advanced by utilizing the semi-supervised learning method in which the neighboring structure of a wavelet coefficient is adopted and an appropriate decision function is derived. According to decision function, wavelet coefficients follow one of two prespecified Bayesian rules obtained using varying related parameters. The decision of a wavelet coefficient depends not only on its magnitude, but also on the neighboring structure on which the coefficient is located. We discuss the theoretical properties of the suggested method and provide recommended parameter settings. We show that the proposed method is often superior to several existing wavelet denoising methods through extensive experimentation.     

Wavelet Threshold Estimators for Data with Correlated Noise

Wavelet threshold estimators for data with stationary correlated noise are constructed by applying a level-dependent soft threshold to the coefficients in the wavelet transform. A variety of threshold choices is proposed, including one based on an unbiased estimate of mean-squared error. The practical performance of the method is demonstrated on examples, including data from a neurophysiological context. The theoretical properties of the estimators are investigated by comparing them with an ideal but unattainable `bench-mark', that can be considered in the wavelet context as the risk obtained by ideal spatial adaptivity, and more generally is obtained by the use of an `oracle' that provides information that is not actually available in the data. It is shown that the level-dependent threshold estimator performs well relative to the bench-mark risk, and that its minimax behaviour cannot be improved on in order of magnitude by any other estimator. The wavelet domain structure of both short- and long-range dependent noise is considered, and in both cases it is shown that the estimators have near optimal behaviour simultaneously in a wide range of function classes, adapting automatically to the regularity properties of the underlying model. The proofs of the main results are obtained by considering a more general multivariate normal decision theoretic problem.     

Detection and Estimation of Jump Points in Non parametric Regression Function with AR(1) Noise

In this paper, we propose a method based on wavelet analysis to detect and estimate jump points in non parametric regression function. This method is applied to AR(1) noise process under random design. First, the test statistics are constructed on the empirical wavelet coefficients. Then, under the null hypothesis, the critical values of test statistics are obtained. Under the alternative, the consistency of the test is proved. Afterward, the rate of convergence, the estimators of the number, and locations of change points are given theoretically. Finally, the excellent performance of our method is demonstrated through simulations using artificial and real datasets.     

Wavelet-based LASSO in functional linear quantile regression

ABSTRACT

In this paper, we develop an efficient wavelet-based regularized linear quantile regression framework for coefficient estimations, where the responses are scalars and the predictors include both scalars and function. The framework consists of two important parts: wavelet transformation and regularized linear quantile regression. Wavelet transform can be used to approximate functional data through representing it by finite wavelet coefficients and effectively capturing its local features. Quantile regression is robust for response outliers and heavy-tailed errors. In addition, comparing with other methods it provides a more complete picture of how responses change conditional on covariates. Meanwhile, regularization can remove small wavelet coefficients to achieve sparsity and efficiency. A novel algorithm, Alternating Direction Method of Multipliers (ADMM) is derived to solve the optimization problems. We conduct numerical studies to investigate the finite sample performance of our method and applied it on real data from ADHD studies.     

Wavelet thresholding via a Bayesian approach

We discuss a Bayesian formalism which gives rise to a type of wavelet threshold estimation in nonparametric regression. A prior distribution is imposed on the wavelet coefficients of the unknown response function, designed to capture the sparseness of wavelet expansion that is common to most applications. For the prior specified, the posterior median yields a thresholding procedure. Our prior model for the underlying function can be adjusted to give functions falling in any specific Besov space. We establish a relationship between the hyperparameters of the prior model and the parameters of those Besov spaces within which realizations from the prior will fall. Such a relationship gives insight into the meaning of the Besov space parameters. Moreover, the relationship established makes it possible in principle to incorporate prior knowledge about the function's regularity properties into the prior model for its wavelet coefficients. However, prior knowledge about a function's regularity properties might be difficult to elicit; with this in mind, we propose a standard choice of prior hyperparameters that works well in our examples. Several simulated examples are used to illustrate our method, and comparisons are made with other thresholding methods. We also present an application to a data set that was collected in an anaesthesiological study.     

Genetic Algorithm in the Wavelet Domain for Large p Small n Regression

Many areas of statistical modeling are plagued by the “curse of dimensionality,” in which there are more variables than observations. This is especially true when developing functional regression models where the independent dataset is some type of spectral decomposition, such as data from near-infrared spectroscopy. While we could develop a very complex model by simply taking enough samples (such that n > p), this could prove impossible or prohibitively expensive. In addition, a regression model developed like this could turn out to be highly inefficient, as spectral data usually exhibit high multicollinearity. In this article, we propose a two-part algorithm for selecting an effective and efficient functional regression model. Our algorithm begins by evaluating a subset of discrete wavelet transformations, allowing for variation in both wavelet and filter number. Next, we perform an intermediate processing step to remove variables with low correlation to the response data. Finally, we use the genetic algorithm to perform a stochastic search through the subset regression model space, driven by an information-theoretic objective function. We allow our algorithm to develop the regression model for each response variable independently, so as to optimally model each variable. We demonstrate our method on the familiar biscuit dough dataset, which has been used in a similar context by several researchers. Our results demonstrate both the flexibility and the power of our algorithm. For each response variable, a different subset model is selected, and different wavelet transformations are used. The models developed by our algorithm show an improvement, as measured by lower mean error, over results in the published literature.     

Two-queue polling systems with switching policy based on the queue that is not being served

We study a system of two non-identical and separate M/M/1/? queues with capacities (buffers) C₁ < ∞ and C₂ = ∞, respectively, served by a single server that alternates between the queues. The server’s switching policy is threshold-based, and, in contrast to other threshold models, is determined by the state of the queue that is not being served. That is, when neither queue is empty while the server attends Q_i (i = 1, 2), the server switches to the other queue as soon as the latter reaches its threshold. When a served queue becomes empty we consider two switching scenarios: (i) Work-Conserving, and (ii) Non-Work-Conserving. We analyze the two scenarios using Matrix Geometric methods and obtain explicitly the rate matrix R, where its entries are given in terms of the roots of the determinants of two underlying matrices. Numerical examples are presented and extreme cases are investigated.     

Block‐threshold‐adapted Estimators via a Maxiset Approach

We study the maxiset performance of a large collection of block thresholding wavelet estimators, namely the horizontal block thresholding family. We provide sufficient conditions on the choices of rates and threshold values to ensure that the involved adaptive estimators obtain large maxisets. Moreover, we prove that any estimator of such a family reconstructs the Besov balls with a near‐minimax optimal rate that can be faster than the one of any separable thresholding estimator. Then, we identify, in particular cases, the best estimator of such a family, that is, the one associated with the largest maxiset. As a particularity of this paper, we propose a refined approach that models method‐dependent threshold values. By a series of simulation studies, we confirm the good performance of the best estimator by comparing it with the other members of its family.     

Bayesian False Discovery Rate Wavelet Shrinkage: Theory and Applications

Statistical inference in the wavelet domain remains a vibrant area of contemporary statistical research because of desirable properties of wavelet representations and the need of scientific community to process, explore, and summarize massive data sets. Prime examples are biomedical, geophysical, and internet related data. We propose two new approaches to wavelet shrinkage/thresholding.

In the spirit of Efron and Tibshirani's recent work on local false discovery rate, we propose Bayesian Local False Discovery Rate (BLFDR), where the underlying model on wavelet coefficients does not assume known variances. This approach to wavelet shrinkage is shown to be connected with shrinkage based on Bayes factors. The second proposal, Bayesian False Discovery Rate (BaFDR), is based on ordering of posterior probabilities of hypotheses on true wavelets coefficients being null, in Bayesian testing of multiple hypotheses.

We demonstrate that both approaches result in competitive shrinkage methods by contrasting them to some popular shrinkage techniques.     

Nonparametric estimation of the derivatives of a density by the method of wavelet for mixing sequences

The problem of estimation of the derivative of a probability density f is considered, using wavelet orthogonal bases. We consider an important kind of dependent random variables, the so-called mixing random variables and investigate the precise asymptotic expression for the mean integrated error of the wavelet estimators. We show that the mean integrated error of the proposed estimator attains the same rate as when the observations are independent, under certain week dependence conditions imposed to the {X _i }, defined in {Ω, N, P}.     

A Bayesian wavelet approach to estimation of a change-point in a nonlinear multivariate time series

ABSTRACT

We propose a semiparametric approach to estimate the existence and location of a statistical change-point to a nonlinear multivariate time series contaminated with an additive noise component. In particular, we consider a p-dimensional stochastic process of independent multivariate normal observations where the mean function varies smoothly except at a single change-point. Our approach involves conducting a Bayesian analysis on the empirical detail coefficients of the original time series after a wavelet transform. If the mean function of our time series can be expressed as a multivariate step function, we find our Bayesian-wavelet method performs comparably with classical parametric methods such as maximum likelihood estimation. The advantage of our multivariate change-point method is seen in how it applies to a much larger class of mean functions that require only general smoothness conditions.     

Wavelet-based estimation of regression function with strong mixing errors under fixed design

We consider wavelet-based non linear estimators, which are constructed by using the thresholding of the empirical wavelet coefficients, for the mean regression functions with strong mixing errors and investigate their asymptotic rates of convergence. We show that these estimators achieve nearly optimal convergence rates within a logarithmic term over a large range of Besov function classes B^s_{p, q}. The theory is illustrated with some numerical examples.

A new ingredient in our development is a Bernstein-type exponential inequality, for a sequence of random variables with certain mixing structure and are not necessarily bounded or sub-Gaussian. This moderate deviation inequality may be of independent interest.     

Mathematische aspekte zu einer methode zur numerischen bestimmung individueller thermodynamischer aktivitätskoeffizienten einzelner ionensorten in konzentrierten elektrolytlösungen

The mathematical problems of the – in an communication [3] described – principle for the calculation of individual thermodynamic activity coefficients of single ionic species in concentrated electrolyte solutions are specified. It is the Newtonian approximation method that makes possible the evaluation of the constants b ₁,…b ₄ in the concentration function (0.1) for the product of the activity coefficients.

The efficiency of the method is represented by the example of the activity coefficients of pure and of – with other electrolytes – mixed solutions of NaCIO₄. The individual activity coefficients of the single ionic species are evaluated for several electrolytes of the concentration range from m = 0 to m = 10 mole/kg and published at another place [3, 17, 18].     

A simulation study of the Bennett test and Miller test for coefficients of variation

In the article, properties of the Bennett test and Miller test are analyzed. Assuming that the sample size is the same for each sample and considering the null hypothesis that the coefficients of variation for k populations are equal against the hypothesis that k ? 1 coefficients of variation are the same but differ from the coefficient of variation for the kth population, the empirical significance level and the power of the test are studied. Moreover, the dependence of the test statistic and the power of the test on the ratio of coefficients of variation are considered. The analyses are performed on simulated data.