Asymptotic properties of LS and QML estimators for a class of nonlinear GARCH processes

We consider estimation of a class of power-transformed threshold GARCH models. When the power of the transformation is known, the asymptotic properties of the quasi-maximum likelihood estimator (QMLE) are established under mild conditions. Two sequences of least-squares estimators are also considered in the pure ARCH case, and it is shown that they can be asymptotically more accurate than the QMLE for certain power transformations. In the case where the power of the transformation has to be estimated, the asymptotic properties of the QMLE are proven under the assumption that the noise has a density. The finite-sample properties of the proposed estimators are studied by simulation.     

Nonparametric Regression with Sample Design Following a Random Process

Nonparametric regression is considered where the sample point placement is not fixed and equispaced, but generated by a random process with rate n. Conditions are found for the random processes that result in optimal rates of convergence for nonparametric regression when using a block thresholded wavelet estimator. Previous results on nonparametric regression via wavelets on both fixed and random sample point placement are shown to be special cases of the general result given here. The estimator is adaptive over a large range of Hölder function spaces and the convergence rate exhibited is an improvement over term-by-term wavelet estimators. Threshold selection is implemented in a data-adaptive fashion, rather than using a fixed threshold as is usually done in block thresholding. This estimator, BlockSure, is compared against fixed-threshold block estimators and the more traditional term-by-term threshold wavelet estimators on several random design schemes via simulations.     

Asymptotics of cross-validated risk estimation in estimator selection and performance assessment

Risk estimation is an important statistical question for the purposes of selecting a good estimator (i.e., model selection) and assessing its performance (i.e., estimating generalization error). This article introduces a general framework for cross-validation and derives distributional properties of cross-validated risk estimators in the context of estimator selection and performance assessment. Arbitrary classes of estimators are considered, including density estimators and predictors for both continuous and polychotomous outcomes. Results are provided for general full data loss functions (e.g., absolute and squared error, indicator, negative log density). A broad definition of cross-validation is used in order to cover leave-one-out cross-validation, V-fold cross-validation, Monte Carlo cross-validation, and bootstrap procedures. For estimator selection, finite sample risk bounds are derived and applied to establish the asymptotic optimality of cross-validation, in the sense that a selector based on a cross-validated risk estimator performs asymptotically as well as an optimal oracle selector based on the risk under the true, unknown data generating distribution. The asymptotic results are derived under the assumption that the size of the validation sets converges to infinity and hence do not cover leave-one-out cross-validation. For performance assessment, cross-validated risk estimators are shown to be consistent and asymptotically linear for the risk under the true data generating distribution and confidence intervals are derived for this unknown risk. Unlike previously published results, the theorems derived in this and our related articles apply to general data generating distributions, loss functions (i.e., parameters), estimators, and cross-validation procedures.     

Minimax estimation of a probability of success under LINEX loss

Minimax estimation of a binomial probability under LINEX loss function is considered. It is shown that no equalizer estimator is available in the statistical decision problem under consideration. It is pointed out that the problem can be solved by determining the Bayes estimator with respect to a least favorable distribution having finite support. In this situation, the optimal estimator and the least favorable distribution can be determined only by using numerical methods. Some properties of the minimax estimators and the corresponding least favorable prior distributions are provided depending on the parameters of the loss function. The properties presented are exploited in computing the minimax estimators and the least favorable distributions. The results obtained can be applied to determine minimax estimators of a cumulative distribution function and minimax estimators of a survival function.     

On the estimation of the parameters of the von mises distribution

The usual maximum likelihood estimators of the parameters of the von Mises distribution are shown to perform badly in small samples. In view of this and the fact that these estimators require a large amount of computation, alternative, simpler estimators are proposed. It is shown that these estimators are at least comparable to the traditional estimators and are, in many cases, superior to them. We also apply the procedure of jackknifing to the maximum likelihood estimator of the concentration parameter of the von Mises distribution and compare the properties of the jackknifed estimator with the other estimators considered in this paper.     

Adaptive nonparametric empirical Bayes estimation via wavelet series: The minimax study

In the present paper, we derive lower bounds for the risk of the nonparametric empirical Bayes estimators. In order to attain the optimal convergence rate, we propose generalization of the linear empirical Bayes estimation method which takes advantage of the flexibility of the wavelet techniques. We present an empirical Bayes estimator as a wavelet series expansion and estimate coefficients by minimizing the prior risk of the estimator. As a result, estimation of wavelet coefficients requires solution of a well-posed low-dimensional sparse system of linear equations. The dimension of the system depends on the size of wavelet support and smoothness of the Bayes estimator. An adaptive choice of the resolution level is carried out using Lepski et al. (1997) method. The method is computationally efficient and provides asymptotically optimal adaptive EB estimators. The theory is supplemented by numerous examples.     

Block‐threshold‐adapted Estimators via a Maxiset Approach

We study the maxiset performance of a large collection of block thresholding wavelet estimators, namely the horizontal block thresholding family. We provide sufficient conditions on the choices of rates and threshold values to ensure that the involved adaptive estimators obtain large maxisets. Moreover, we prove that any estimator of such a family reconstructs the Besov balls with a near‐minimax optimal rate that can be faster than the one of any separable thresholding estimator. Then, we identify, in particular cases, the best estimator of such a family, that is, the one associated with the largest maxiset. As a particularity of this paper, we propose a refined approach that models method‐dependent threshold values. By a series of simulation studies, we confirm the good performance of the best estimator by comparing it with the other members of its family.     

Maximum likelihood revisited under a semi-parametric context - estimation of the tail index

In this paper, and in a context of regularly varying tails, we study computationally the classical Maximum Likelihood (ML) estimator based on the Paretian behaviour of the excesses over a high threshold, denoted PML-estimator, a type II Censoring estimator based specifically on a Fréchet parent, denoted CENS-estimator, and two ML estimators based on the scaled log-spacings, and denoted SLS-estimators. These estimators are considered under a semi-parametric set-up, and compared with the classical Hill estimator and a Generalized Jackknife (GJ) estimator, which has essentially in mind a reduction of the bias of Hill's estimator.     

Boundary and Bias Correction in Kernel Hazard Estimation

A new class of local linear hazard estimators based on weighted least square kernel estimation is considered. The class includes the kernel hazard estimator of Ramlau-Hansen (1983), which has the same boundary correction property as the local linear regression estimator (see Fan & Gijbels, 1996). It is shown that all the local linear estimators in the class have the same pointwise asymptotic properties. We derive the multiplicative bias correction of the local linear estimator. In addition we propose a new bias correction technique based on bootstrap estimation of additive bias. This latter method has excellent theoretical properties. Based on an extensive simulation study where we compare the performance of competing estimators, we also recommend the use of the additive bias correction in applied work.     

A non linear wavelet based estimator for long memory processes

Two wavelet based estimators are considered in this paper for the two parameters that characterize long range dependence processes. The first one is linear and is based on the statistical properties of the coefficients of a discrete wavelet transform of long range dependence processes. The estimator consists in measuring the slope (related to the long memory parameter) and the intercept (related to the variance of the process) of a linear regression after a discrete wavelet transform is performed (Veitch and Abry, 1999). In this paper its properties are reviewed, and analytic evidence is produced that the linear estimator is applicable only when the second parameter is unknown. To overcome this limitation a non linear wavelet based estimator - that takes into account that the intercept depends on the long memory parameter - is proposed here for the cases in which the second parameter is known or the only parameter of interest is the long memory parameter. Under the same hypothesis assumed for the linear estimator, the non linear estimator is shown to be asymptotically more efficient for the long memory parameter. Numerical simulations show that, even for small data sets, the bias is very small and the variance close to optimal. An application to ATM based Internet traffic is presented.Financial support from the Italian Ministry of University and Scientific Research (MIUR), also in the context of the COFIN 2002 ALINWEB (Algorithms for the Internet and the Web) Project, is gratefully acknowledged.     

Tail density estimation for exploratory data analysis using kernel methods

It is often critical to accurately model the upper tail behaviour of a random process. Nonparametric density estimation methods are commonly implemented as exploratory data analysis techniques for this purpose and can avoid model specification biases implied by using parametric estimators. In particular, kernel-based estimators place minimal assumptions on the data, and provide improved visualisation over scatterplots and histograms. However kernel density estimators can perform poorly when estimating tail behaviour above a threshold, and can over-emphasise bumps in the density for heavy tailed data. We develop a transformation kernel density estimator which is able to handle heavy tailed and bounded data, and is robust to threshold choice. We derive closed form expressions for its asymptotic bias and variance, which demonstrate its good performance in the tail region. Finite sample performance is illustrated in numerical studies, and in an expanded analysis of the performance of global climate models.     

Bootstrap adaptive estimation: The trimmed-mean example

We consider the problem of choosing among a class of possible estimators by selecting the estimator with the smallest bootstrap estimate of finite sample variance. This is an alternative to using cross-validation to choose an estimator adaptively. The problem of a confidence interval based on such an adaptive estimator is considered. We illustrate the ideas by applying the method to the problem of choosing the trimming proportion of an adaptive trimmed mean. It is shown that a bootstrap adaptive trimmed mean is asymptotically normal with an asymptotic variance equal to the smallest among trimmed means. The asymptotic coverage probability of a bootstrap confidence interval based on such adaptive estimators is shown to have the nominal level. The intervals based on the asymptotic normality of the estimator share the same asymptotic result, but have poor small-sample properties compared to the bootstrap intervals. A small-sample simulation demonstrates that bootstrap adaptive trimmed means adapt themselves rather well even for samples of size 10.     

Asymptotic Properties and Variance Estimators of the M-quantile Regression Coefficients Estimators

M-quantile regression is defined as a “quantile-like” generalization of robust regression based on influence functions. This article outlines asymptotic properties for the M-quantile regression coefficients estimators in the case of i.i.d. data with stochastic regressors, paying attention to adjustments due to the first-step scale estimation. A variance estimator of the M-quantile regression coefficients based on the sandwich approach is proposed. Empirical results show that this estimator appears to perform well under different simulated scenarios. The sandwich estimator is applied in the small area estimation context for the estimation of the mean squared error of an estimator for the small area means. The results obtained improve previous findings, especially in the case of heteroskedastic data.     

Estimating risk reduction in stein estimation

It is shown that the unbiased estimator of the risk reduction in Stein estimation is unsatisfactory from a mean-squared-error point of view. A truncated form of the unbiased estimator and various empirical Bayes estimators of the risk reduction are shown to perform much better than the unbiased estimator. A simple practical estimator is proposed whose performance is a compromise between that of the truncated and empirical Bayes estimators.     

Wavelet threshold based on Stein's unbiased risk estimators of restricted location parameter in multivariate normal

In this paper, the problem of estimating the mean vector under non-negative constraints on location vector of the multivariate normal distribution is investigated. The value of the wavelet threshold based on Stein''s unbiased risk estimators is calculated for the shrinkage estimator in restricted parameter space. We suppose that covariance matrix is unknown and we find the dominant class of shrinkage estimators under Balance loss function. The performance evaluation of the proposed class of estimators is checked through a simulation study by using risk and average mean square error values.     

Sampling properties of estimators of a Weibull distribution of use in the lumber industry

We consider maximum-likelihood estimators of the three parameters in the Weibull distribution. Motivated by an application regarding the determination of a lower percentile of the strength of dimension lumber, we investigate the sampling properties of these estimators. Consistency is established when the shape parameter is greater than one, including some nonregular cases encountered in fitting lumber data. The joint distribution of the estimators is studied by a Monte Carlo approach. The maximum-likelihood estimator of the 5th percentile is compared with the sample 5th percentile. For the cases considered it was generally found that with sample size 70 the usual asymptotic normality does not hold.     

Pooling reliability functions

In this article large sample pooling procedures for reliability functions of an exponential life testing model is considered. Asymptotic properties of shrinkage estimation procedure subsequent to preliminary tests are developed. It is shown that the proposed estimator possesses substantially snakker asymptotic mean squared error than the usual estimator in a region of the lparameter space. Relative efficiencies of the purposed estimators to the usual estimators are obtained and recommendations of the level of the preliminary tests are provided. Relative dominance picture of the estimators is presented. It is shown that the proposed estimator provides a wider dominance range over usual estimator than the usual preliminary test estimator. More importantly, the size of the preliminary test is meaningful. Simulation studies is also carried out to appraise the performance of the estimators when samples are small.     

Wavelets for Nonparametric Stochastic Regression with Mixing Stochastic Process

We propose a wavelet based stochastic regression function estimator for the estimation of the regression function for a sequence of mixing stochastic process with a common one-dimensional probability density function. Some asymptotic properties of the proposed estimator are investigated. It is found that the estimators have similar properties to their counterparts studied earlier in literature.     

On simple wavelet estimators of random signals and their small-sample properties

In the paper we suggest certain nonparametric estimators of random signals based on the wavelet transform. We consider stochastic signals embedded in white noise and extractions with wavelet denoizing algorithms utilizing the non-decimated discrete wavelet transform and the idea of wavelet scaling. We evaluate properties of these estimators via extensive computer simulations and partially also analytically. Our wavelet estimators of random signals have clear advantages over parametric maximum likelihood methods as far as computational issues are concerned, while at the same time they can compete with these methods in terms of precision of estimation in small samples. An illustrative example concerning smoothing of survey data is also provided.     

Estimating Percentiles of Time-to-Failure Distribution Obtained from a Linear Degradation Model Using Kernel Density Method

In this article, we propose a nonparametric estimator for percentiles of the time-to-failure distribution obtained from a linear degradation model using the kernel density method. The properties of the proposed kernel estimator are investigated and compared with well-known maximum likelihood and ordinary least squares estimators via a simulation technique. The mean squared error and the length of the bootstrap confidence interval are used as the basis criteria of the comparisons. The simulation study shows that the performance of the kernel estimator is acceptable as a general estimator. When the distribution of the data is assumed to be known, the maximum likelihood and ordinary least squares estimators perform better than the kernel estimator, while the kernel estimator is superior when the assumption of our knowledge of the data distribution is violated. A comparison among different estimators is achieved using a real data set.