Semiparametric Efficient Estimation of the Mean of a Time Series in the Presence of Conditional Heterogeneity of Unknown Form

We obtain semiparametric efficiency bounds for estimation of a location parameter in a time series model where the innovations are stationary and ergodic conditionally symmetric martingale differences but otherwise possess general dependence and distributions of unknown form. We then describe an iterative estimator that achieves this bound when the conditional density functions of the sample are known. Finally, we develop a “semi-adaptive” estimator that achieves the bound when these densities are unknown by the investigator. This estimator employs nonparametric kernel estimates of the densities. Monte Carlo results are reported.     

Semiparametric Efficient Estimation of the Mean of a Time Series in the Presence of Conditional Heterogeneity of Unknown Form

Abstract

We obtain semiparametric efficiency bounds for estimation of a location parameter in a time series model where the innovations are stationary and ergodic conditionally symmetric martingale differences but otherwise possess general dependence and distributions of unknown form. We then describe an iterative estimator that achieves this bound when the conditional density functions of the sample are known. Finally, we develop a “semi-adaptive” estimator that achieves the bound when these densities are unknown by the investigator. This estimator employs nonparametric kernel estimates of the densities. Monte Carlo results are reported.     

Nonparametric localized bandwidth selection for Kernel density estimation

As conventional cross-validation bandwidth selection methods do not work properly in the situation where the data are serially dependent time series, alternative bandwidth selection methods are necessary. In recent years, Bayesian-based methods for global bandwidth selection have been studied. Our experience shows that a global bandwidth is however less suitable than a localized bandwidth in kernel density estimation based on serially dependent time series data. Nonetheless, a di?cult issue is how we can consistently estimate a localized bandwidth. This paper presents a nonparametric localized bandwidth estimator, for which we establish a completely new asymptotic theory. Applications of this new bandwidth estimator to the kernel density estimation of Eurodollar deposit rate and the S&P 500 daily return demonstrate the effectiveness and competitiveness of the proposed localized bandwidth.     

Transformation-Kernel Estimation of Copula Densities

ABSTRACT

The standard kernel estimator of copula densities suffers from boundary biases and inconsistency due to unbounded densities. Transforming the domain of estimation into an unbounded one remedies both problems, but also introduces an unbounded multiplier that may produce erratic boundary behaviors in the final density estimate. We propose an improved transformation-kernel estimator that employs a smooth tapering device to counter the undesirable influence of the multiplier. We establish the theoretical properties of the new estimator and its automatic higher-order improvement under Gaussian copulas. We present two practical methods of smoothing parameter selection. Extensive Monte Carlo simulations demonstrate the competence of the proposed estimator in terms of global and tail performance. Two real-world examples are provided. Supplementary materials for this article are available online.     

Testing the adequacy of a linear model VIA critical smoothing

An important problem for fitting local linear regression is the choice of the smoothing parameter. As the smoothing parameter becomes large, the estimator tends to a straight line, which is the least squares fit in the ordinary linear regression setting. This property may be used to assess the adequacy of a simple linear model. Motivated by Silverman's (1981) work in kernel density estimation, a suitable test statistic is the critical smoothing parameter where the estimate changes from nonlinear to linear, while linearity or non- linearity requires a more precise judgment. We define the critical smoothing parameter through the approximate F-tests by Hastie and Tibshirani (1990). To assess the significance, the “wild bootstrap” procedure is used to replicate the data and the proportion of bootstrap samples which give a nonlinear estimate when using the critical bandwidth is obtained as the p-value. Simulation results show that the critical smoothing test is useful in detecting a wide range of alternatives.     

Semiparametric Density Deconvolution

Abstract.  A new semiparametric method for density deconvolution is proposed, based on a model in which only the ratio of the unconvoluted to convoluted densities is specified parametrically. Deconvolution results from reweighting the terms in a standard kernel density estimator, where the weights are defined by the parametric density ratio. We propose that in practice, the density ratio be modelled on the log-scale as a cubic spline with a fixed number of knots. Parameter estimation is based on maximization of a type of semiparametric likelihood. The resulting asymptotic properties for our deconvolution estimator mirror the convergence rates in standard density estimation without measurement error when attention is restricted to our semiparametric class of densities. Furthermore, numerical studies indicate that for practical sample sizes our weighted kernel estimator can provide better results than the classical non-parametric kernel estimator for a range of densities outside the specified semiparametric class.     

Stabilizing the Performance of Kurtosis Estimator of Multivariate Data

The estimation of the kurtosis parameter of the underlying distribution plays a central role in many statistical applications. The central theme of the article is to improve the estimation of the kurtosis parameter using a priori information. More specifically, we consider the problem of estimating kurtosis parameter of a multivariate population when some prior information regarding the the parameter is available. The rationale is that the sample estimator of the kurtosis parameter has a large estimation error. In this situation we consider shrinkage and pretest estimation methodologies and reappraise their statistical properties. The estimation based on these strategies yield relatively smaller estimation error in comparison with the sample estimator in the candidate subspace. A large sample theory of the suggested estimators are developed and compared. The results demonstrate that suggested estimators outperform the estimator based on the sample data only in the candidate subspace. In an effort to appreciate the relative behavior of the estimators in a finite sample scenario, a Monte-carlo simulation study is planned and performed. The result of simulation study strongly corroborates the asymptotic result. To illustrate the application of the estimators, some example are showcased based on recently published data.     

Symmetric maximum kernel likelihood estimation

We introduce an estimator for the population mean based on maximizing likelihoods formed from a symmetric kernel density estimate. Due to these origins, we have dubbed the estimator the symmetric maximum kernel likelihood estimate (smkle). A speedy computational method to compute the smkle based on binning is implemented in a simulation study which shows that the smkle at an optimal bandwidth is decidedly superior in terms of efficiency to the sample mean and other measures of location for heavy-tailed symmetric distributions. An empirical rule and a computational method to estimate this optimal bandwidth are developed and used to construct bootstrap confidence intervals for the population mean. We show that the intervals have approximately nominal coverage and have significantly smaller average width than the corresponding intervals for other measures of location.     

Discrete triangular associated kernel and bandwidth choices in semiparametric estimation for count data

This work deals with semiparametric kernel estimator of probability mass functions which are assumed to be modified Poisson distributions. This semiparametric approach is based on discrete associated kernel method appropriated for modelling count data; in particular, the famous discrete symmetric triangular kernels are used. Two data-driven bandwidth selection procedures are investigated and an explicit expression of optimal bandwidth not available until now is provided. Moreover, some asymptotic properties of the cross-validation criterion adapted for discrete semiparametric kernel estimation are studied. Finally, to measure the performance of semiparametric estimator according to each type of bandwidth parameter, some applications are realized on three real count data-sets from sociology and biology.     

A simple,automatic and adaptive bivariate density estimator based on conditional densities

The standard approach to non-parametric bivariate density estimation is to use a kernel density estimator. Practical performance of this estimator is hindered by the fact that the estimator is not adaptive (in the sense that the level of smoothing is not sensitive to local properties of the density). In this paper a simple, automatic and adaptive bivariate density estimator is proposed based on the estimation of marginal and conditional densities. Asymptotic properties of the estimator are examined, and guidance to practical application of the method is given. Application to two examples illustrates the usefulness of the estimator as an exploratory tool, particularly in situations where the local behaviour of the density varies widely. The proposed estimator is also appropriate for use as a pilot estimate for an adaptive kernel estimate, since it is relatively inexpensive to calculate.     

A locally adaptive transformation method of boundary correction in kernel density estimation

Kernel smoothing methods are widely used in many research areas in statistics. However, kernel estimators suffer from boundary effects when the support of the function to be estimated has finite endpoints. Boundary effects seriously affect the overall performance of the estimator. In this article, we propose a new method of boundary correction for univariate kernel density estimation. Our technique is based on a data transformation that depends on the point of estimation. The proposed method possesses desirable properties such as local adaptivity and non-negativity. Furthermore, unlike many other transformation methods available, the proposed estimator is easy to implement. In a Monte Carlo study, the accuracy of the proposed estimator is numerically analyzed and compared with the existing methods of boundary correction. We find that it performs well for most shapes of densities. The theory behind the new methodology, along with the bias and variance of the proposed estimator, are presented. Results of a data analysis are also given.     

Local Estimation of the Second-Order Parameter in Extreme Value Statistics and Local Unbiased Estimation of the Tail Index

We develop and study in the framework of Pareto-type distributions a class of nonparametric kernel estimators for the conditional second order tail parameter. The estimators are obtained by local estimation of the conditional second order parameter using a moving window approach. Asymptotic normality of the proposed class of kernel estimators is proven under some suitable conditions on the kernel function and the conditional tail quantile function. The nonparametric estimators for the second order parameter are subsequently used to obtain a class of bias-corrected kernel estimators for the conditional tail index. In particular it is shown how for a given kernel function one obtains a bias-corrected kernel function, and that replacing the second order parameter in the latter with a consistent estimator does not change the limiting distribution of the bias-corrected estimator for the conditional tail index. The finite sample behavior of some specific estimators is illustrated with a simulation experiment. The developed methodology is also illustrated on fire insurance claim data.     

Approaches to regression analysis with multiple measurements from individual sampling units

Application of ordinary least-squares regression to data sets which contain multiple measurements from individual sampling units produces an unbiased estimator of the parameters but a biased estimator of the covariance matrix of the parameter estimates. The present work considers a random coefficient, linear model to deal with such data sets: this model permits many senses in which multiple measurements are taken from a sampling unit, not just when it is measured at several times. Three procedures to estimate the covariance matrix of the error term of the model are considered. Given these, three procedures to estimate the parameters of the model and their covariance matrix are considered; these are ordinary least-squares, generalized least-squares, and an adjusted ordinary least-squares procedure which produces an unbiased estimator of the covariance matrix of the parameters with small samples. These various procedures are compared in simulation studies using three examples from the biological literature. The possibility of testing hypotheses about the vector of parameters is also considered. It is found that all three procedures for regression estimation produce estimators of the parameters with bias of no practical consequence, Both generalized least-squares and adjusted ordinary least-squares generally produce estimators of the covariance matrix of the parameter estimates with bias of no practical consequence, while ordinary least-squares produces a negatively biased estimator. Neither ordinary nor generalized least-squares provide satisfactory hypothesis tests of the vector of parameter estimates. It is concluded that adjusted ordinary least-squares, when applied with either of two of the procedures used to estimate the error coveriance matrix, shows promise for practical application with data sets of the nature considered here.     

Skewness by Splitting the Scale Parameter

This article describes a recursive nonparametric estimation for the local partial first derivative of an arbitrary function satisfied some regularity conditions and establishes its consistency and asymptotic normality under the assumption of strong mixing sequence. The proposed estimator is a variable window width version of the Watson-Nadaraya type of derivative estimator. The window width varied as more data points become available enables a recursive algorithm that reduce computational complexity from order N ³ normally required by batch methods for kernel regression to order N ². This approach is computationally simple and attractive from practical viewpoint especially when the situation call for frequent updating of first derivative estimates. For example, maintaining a delta-hedged position of a portfolio of equities with index options is one of many applications of such estimation.     

Efficient estimation in a regression model with missing responses

This article examines methods to efficiently estimate the mean response in a linear model with an unknown error distribution under the assumption that the responses are missing at random. We show how the asymptotic variance is affected by the estimator of the regression parameter, and by the imputation method. To estimate the regression parameter, the ordinary least squares is efficient only if the error distribution happens to be normal. If the errors are not normal, then we propose a one step improvement estimator or a maximum empirical likelihood estimator to efficiently estimate the parameter.To investigate the imputation’s impact on the estimation of the mean response, we compare the listwise deletion method and the propensity score method (which do not use imputation at all), and two imputation methods. We demonstrate that listwise deletion and the propensity score method are inefficient. Partial imputation, where only the missing responses are imputed, is compared to full imputation, where both missing and non-missing responses are imputed. Our results reveal that, in general, full imputation is better than partial imputation. However, when the regression parameter is estimated very poorly, the partial imputation will outperform full imputation. The efficient estimator for the mean response is the full imputation estimator that utilizes an efficient estimator of the parameter.     

Nonparametric density deconvolution by weighted kernel estimators

Nonparametric density estimation in the presence of measurement error is considered. The usual kernel deconvolution estimator seeks to account for the contamination in the data by employing a modified kernel. In this paper a new approach based on a weighted kernel density estimator is proposed. Theoretical motivation is provided by the existence of a weight vector that perfectly counteracts the bias in density estimation without generating an excessive increase in variance. In practice a data driven method of weight selection is required. Our strategy is to minimize the discrepancy between a standard kernel estimate from the contaminated data on the one hand, and the convolution of the weighted deconvolution estimate with the measurement error density on the other hand. We consider a direct implementation of this approach, in which the weights are optimized subject to sum and non-negativity constraints, and a regularized version in which the objective function includes a ridge-type penalty. Numerical tests suggest that the weighted kernel estimation can lead to tangible improvements in performance over the usual kernel deconvolution estimator. Furthermore, weighted kernel estimates are free from the problem of negative estimation in the tails that can occur when using modified kernels. The weighted kernel approach generalizes to the case of multivariate deconvolution density estimation in a very straightforward manner.     

Robustness of three power transformation estimation procedures

Maximum likelihood, goodness-of-fit, and symmetric percentile estimators of the power transformation parameterp, are considered. The comparative robustness of each estimation procedure is evaluated when the transformed data can be made symmetric, but may not necessarily be normal. Seven types of symmetric distributions are considered as well as four contaminated normal distributions over a range of six p values for samples of size 25, 50, and 100. The results indicate that the maximum likelihood estimator was slightly better than the goodness-of-fit estimator, but both were greatly superior to the percentile estimator. In general, the procedures were robust to distributional symmetric departures from normality, but increasing kurtosis caused appreciable increases in variation for estimated p values. The variability of p was found to decrease more than exponentially with decreases in the underlying normal distribution coefficient of variation. The standard likelihood ratio confidence interval procedure was found not to be generally useful.     

Comparison of Kernel Density Estimators with Assumption on Number of Modes

A data-driven bandwidth choice for a kernel density estimator called critical bandwidth is investigated. This procedure allows the estimation to have as many modes as assumed for the density to estimate. Both Gaussian and uniform kernels are considered. For the Gaussian kernel, asymptotic results are given. For the uniform kernel, an argument against these properties is mentioned. These theoretical results are illustrated with a simulation study that compares the kernel estimators that rely on critical bandwidth with another one that uses a plug-in method to select its bandwidth. An estimator that consists in estimates of density contour clusters and takes assumptions on number of modes into account is also considered. Finally, the methodology is illustrated using environment monitoring data.     

Root n consistent and optimal density estimators for moving average processes

Abstract.  The marginal density of a first order moving average process can be written as a convolution of two innovation densities. Saavedra & Cao [Can. J. Statist. (2000), 28, 799] propose to estimate the marginal density by plugging in kernel density estimators for the innovation densities, based on estimated innovations. They obtain that for an appropriate choice of bandwidth the variance of their estimator decreases at the rate 1/ n . Their estimator can be interpreted as a specific U -statistic. We suggest a slightly simplified U -statistic as estimator of the marginal density, prove that it is asymptotically normal at the same rate, and describe the asymptotic variance explicitly. We show that the estimator is asymptotically efficient if no structural assumptions are made on the innovation density. For innovation densities known to have mean zero or to be symmetric, we describe improvements of our estimator which are again asymptotically efficient.     

Shape constrained kernel density estimation

In this paper, a method for estimating monotone, convex and log-concave densities is proposed. The estimation procedure consists of an unconstrained kernel estimator which is modified in a second step with respect to the desired shape constraint by using monotone rearrangements. It is shown that the resulting estimate is a density itself and shares the asymptotic properties of the unconstrained estimate. A short simulation study shows the finite sample behavior.