Robust estimation of multivariate regression model

This paper studies robust estimation of multivariate regression model using kernel weighted local linear regression. A robust estimation procedure is proposed for estimating the regression function and its partial derivatives. The proposed estimators are jointly asymptotically normal and attain nonparametric optimal convergence rate. One-step approximations to the robust estimators are introduced to reduce computational burden. The one-step local M-estimators are shown to achieve the same efficiency as the fully iterative local M-estimators as long as the initial estimators are good enough. The proposed estimators inherit the excellent edge-effect behavior of the local polynomial methods in the univariate case and at the same time overcome the disadvantages of the local least-squares based smoothers. Simulations are conducted to demonstrate the performance of the proposed estimators. Real data sets are analyzed to illustrate the practical utility of the proposed methodology. This work was supported by the National Natural Science Foundation of China (Grant No. 10471006).     

Two-step estimation of functional linear models with applications to longitudinal data

Functional linear models are useful in longitudinal data analysis. They include many classical and recently proposed statistical models for longitudinal data and other functional data. Recently, smoothing spline and kernel methods have been proposed for estimating their coefficient functions nonparametrically but these methods are either intensive in computation or inefficient in performance. To overcome these drawbacks, in this paper, a simple and powerful two-step alternative is proposed. In particular, the implementation of the proposed approach via local polynomial smoothing is discussed. Methods for estimating standard deviations of estimated coefficient functions are also proposed. Some asymptotic results for the local polynomial estimators are established. Two longitudinal data sets, one of which involves time-dependent covariates, are used to demonstrate the approach proposed. Simulation studies show that our two-step approach improves the kernel method proposed by Hoover and co-workers in several aspects such as accuracy, computational time and visual appeal of the estimators.     

Efficient inference for parameters of unobservable periodic autoregressive time series

This paper considers estimating the model coefficients when the observed periodic autoregressive time series is contaminated by a trend. The proposed Yule–Walker estimators are obtained by a two-step procedure. In the first step, the trend is estimated by a weighted local polynomial, and the residuals are obtained by subtracting the trend estimates from the observations; in the second step, the model coefficients are estimated by the well-known Yule–Walker method via the residuals. It is shown that under certain conditions such Yule–Walker estimators are oracally efficient, i.e., they are asymptotically equivalent to those obtained from periodic autoregressive time series without a trend. An easy-to-use implementation procedure is provided. The performance of the estimators is illustrated by simulation studies and real data analysis. In particular, the simulation studies show that the proposed estimator outperforms that obtained from the residuals when the trend is estimated by kernel smoothing without taking the heteroscedasticity into consideration.     

On mantel-haenszel type estimators in simple nested case-control studies

We are concerned with nested case-control studies in this article. For proportional hazards model, a class of over-all estimators of hazard ratios is presented when simple samples are drawn from risk sets. These estimators have the form of the Mantel-Haenszel estimator of odds ratio, and are consistent not only for large strata, but also for sparse data. Consistent estimators of the variances of the proposed hazard ratio estimators are also developed. An example is given to illustrate the proposed estimators.     

On the Local Polynomial Estimators of the Counting Process Intensity Function and its Derivatives

Abstract. We consider the properties of the local polynomial estimators of a counting process intensity function and its derivatives. By expressing the local polynomial estimators in a kernel smoothing form via effective kernels, we show that the bias and variance of the estimators at boundary points are of the same magnitude as at interior points and therefore the local polynomial estimators in the context of intensity estimation also enjoy the automatic boundary correction property as they do in other contexts such as regression. The asymptotically optimal bandwidths and optimal kernel functions are obtained through the asymptotic expressions of the mean square error of the estimators. For practical purpose, we suggest an effective and easy‐to‐calculate data‐driven bandwidth selector. Simulation studies are carried out to assess the performance of the local polynomial estimators and the proposed bandwidth selector. The estimators and the bandwidth selector are applied to estimate the rate of aftershocks of the Sichuan earthquake and the rate of the Personal Emergency Link calls in Hong Kong.     

Estimating Mixing Densities in Exponential Family Models for Discrete Variables

This paper is concerned with estimating a mixing density g using a random sample from the mixture distribution f(x)=∫f x | θ)g(θ)dθ where f(· | θ) is a known discrete exponen tial family of density functions. Recently two techniques for estimating g have been proposed. The first uses Fourier analysis and the method of kernels and the second uses orthogonal polynomials. It is known that the first technique is capable of yielding estimators that achieve (or almost achieve) the minimax convergence rate. We show that this is true for the technique based on orthogonal polynomials as well. The practical implementation of these estimators is also addressed. Computer experiments indicate that the kernel estimators give somewhat disappoint ing finite sample results. However, the orthogonal polynomial estimators appear to do much better. To improve on the finite sample performance of the orthogonal polynomial estimators, a way of estimating the optimal truncation parameter is proposed. The resultant estimators retain the convergence rates of the previous estimators and a Monte Carlo finite sample study reveals that they perform well relative to the ones based on the optimal truncation parameter.     

Efficient Estimation of an Additive Quantile Regression Model

Abstract. In this paper, two non‐parametric estimators are proposed for estimating the components of an additive quantile regression model. The first estimator is a computationally convenient approach which can be viewed as a more viable alternative to existing kernel‐based approaches. The second estimator involves sequential fitting by univariate local polynomial quantile regressions for each additive component with the other additive components replaced by the corresponding estimates from the first estimator. The purpose of the extra local averaging is to reduce the variance of the first estimator. We show that the second estimator achieves oracle efficiency in the sense that each estimated additive component has the same variance as in the case when all other additive components were known. Asymptotic properties are derived for both estimators under dependent processes that are strictly stationary and absolutely regular. We also provide a demonstrative empirical application of additive quantile models to ambulance travel times.     

Estimation of parameters of SDE driven by fractional Brownian motion with polynomial drift

ABSTRACT

Strongly consistent and asymptotically normal estimators of the Hurst index and volatility parameters of solutions of stochastic differential equations with polynomial drift are proposed. The estimators are based on discrete observations of the underlying processes.     

Estimation and variable selection in single-index composite quantile regression

In this article, a new composite quantile regression estimation approach is proposed for estimating the parametric part of single-index model. We use local linear composite quantile regression (CQR) for estimating the nonparametric part of single-index model (SIM) when the error distribution is symmetrical. The weighted local linear CQR is proposed for estimating the nonparametric part of SIM when the error distribution is asymmetrical. Moreover, a new variable selection procedure is proposed for SIM. Under some regularity conditions, we establish the large sample properties of the proposed estimators. Simulation studies and a real data analysis are presented to illustrate the behavior of the proposed estimators.     

Sparse alternatives to ridge regression: a random effects approach

In a calibration of near-infrared (NIR) instrument, we regress some chemical compositions of interest as a function of their NIR spectra. In this process, we have two immediate challenges: first, the number of variables exceeds the number of observations and, second, the multicollinearity between variables are extremely high. To deal with the challenges, prediction models that produce sparse solutions have recently been proposed. The term ‘sparse’ means that some model parameters are zero estimated and the other parameters are estimated naturally away from zero. In effect, a variable selection is embedded in the model to potentially achieve a better prediction. Many studies have investigated sparse solutions for latent variable models, such as partial least squares and principal component regression, and for direct regression models such as ridge regression (RR). However, in the latter, it mainly involves an L₁ norm penalty to the objective function such as lasso regression. In this study, we investigate new sparse alternative models for RR within a random effects model framework, where we consider Cauchy and mixture-of-normals distributions on the random effects. The results indicate that the mixture-of-normals model produces a sparse solution with good prediction and better interpretation. We illustrate the methods using NIR spectra datasets from milk and corn specimens.     

Semiparametric statistical inferences for longitudinal data with nonparametric covariance modelling

In this paper, semiparametric modelling for longitudinal data with an unstructured error process is considered. We propose a partially linear additive regression model for longitudinal data in which within-subject variances and covariances of the error process are described by unknown univariate and bivariate functions, respectively. We provide an estimating approach in which polynomial splines are used to approximate the additive nonparametric components and the within-subject variance and covariance functions are estimated nonparametrically. Both the asymptotic normality of the resulting parametric component estimators and optimal convergence rate of the resulting nonparametric component estimators are established. In addition, we develop a variable selection procedure to identify significant parametric and nonparametric components simultaneously. We show that the proposed SCAD penalty-based estimators of non-zero components have an oracle property. Some simulation studies are conducted to examine the finite-sample performance of the proposed estimation and variable selection procedures. A real data set is also analysed to demonstrate the usefulness of the proposed method.     

On the nonparametric smooth estimation of the reversed Hazard Rate function

The Reversed Hazard Rate (RHR) function is an important measure as a tool in the analysis of the reliability of both natural and man-made systems. In this paper, we present several new estimators of the RHR function using nonparametric techniques. These estimators are obtained by incorporating different binning techniques with fixed design local polynomial regression. We show that these estimators are asymptotically unbiased and consistent and, to determine the bandwidth, we propose two simple yet efficient plug-in bandwidth selection methods for even and odd order local polynomial estimators. Simulated and real life data are subsequently used to evaluate the performances of these estimators.     

Componentwise B-spline estimation for varying coefficient models with longitudinal data

A componentwise B-spline method is proposed for estimating the unknown functions in the varying-coefficient models with longitudinal data. Different amounts of smoothing are used for different individual coefficient functions and the estimators of different coefficient functions are obtained by different minimization operations. The local asymptotic bias and variance of the estimators are derived. It is shown that our estimators achieve the local and global optimal convergence rates even if the coefficient functions belong to different smoothness families. The asymptotic distributions of the estimators are also established and are used to construct approximate pointwise confidence intervals for coefficient functions. Finite sample properties of our procedures are studied through Monte Carlo simulations.     

On boundary correction in kernel density estimation

It is well known now that kernel density estimators are not consistent when estimating a density near the finite end points of the support of the density to be estimated. This is due to boundary effects that occur in nonparametric curve estimation problems. A number of proposals have been made in the kernel density estimation context with some success. As of yet there appears to be no single dominating solution that corrects the boundary problem for all shapes of densities. In this paper, we propose a new general method of boundary correction for univariate kernel density estimation. The proposed method generates a class of boundary corrected estimators. They all possess desirable properties such as local adaptivity and non-negativity. In simulation, it is observed that the proposed method perform quite well when compared with other existing methods available in the literature for most shapes of densities, showing a very important robustness property of the method. The theory behind the new approach and the bias and variance of the proposed estimators are given. Results of a data analysis are also given.     

Smoothing for discrete-valued time series

We deal with smoothed estimators for conditional probability functions of discrete-valued time series { Y_t } under two different settings. When the conditional distribution of Y_t given its lagged values falls in a parametric family and depends on exogenous random variables, a smoothed maximum (partial) likelihood estimator for the unknown parameter is proposed. While there is no prior information on the distribution, various nonparametric estimation methods have been compared and the adjusted Nadaraya–Watson estimator stands out as it shares the advantages of both Nadaraya–Watson and local linear regression estimators. The asymptotic normality of the estimators proposed has been established in the manner of sparse asymptotics, which shows that the smoothed methods proposed outperform their conventional, unsmoothed, parametric counterparts under very mild conditions. Simulation results lend further support to this assertion. Finally, the new method is illustrated via a real data set concerning the relationship between the number of daily hospital admissions and the levels of pollutants in Hong Kong in 1994–1995. An ad hoc model selection procedure based on a local Akaike information criterion is proposed to select the significant pollutant indices.     

Understanding Estimators of Treatment Effects in Regression Discontinuity Designs

In this paper, we propose two new estimators of treatment effects in regression discontinuity designs. These estimators can aid understanding of the existing estimators such as the local polynomial estimator and the partially linear estimator. The first estimator is the partially polynomial estimator which extends the partially linear estimator by further incorporating derivative differences of the conditional mean of the outcome on the two sides of the discontinuity point. This estimator is related to the local polynomial estimator by a relocalization effect. Unlike the partially linear estimator, this estimator can achieve the optimal rate of convergence even under broader regularity conditions. The second estimator is an instrumental variable estimator in the fuzzy design. This estimator will reduce to the local polynomial estimator if higher order endogeneities are neglected. We study the asymptotic properties of these two estimators and conduct simulation studies to confirm the theoretical analysis.     

Covariance Matrix Estimation via Network Structure

In this article, we employ a regression formulation to estimate the high-dimensional covariance matrix for a given network structure. Using prior information contained in the network relationships, we model the covariance as a polynomial function of the symmetric adjacency matrix. Accordingly, the problem of estimating a high-dimensional covariance matrix is converted to one of estimating low dimensional coefficients of the polynomial regression function, which we can accomplish using ordinary least squares or maximum likelihood. The resulting covariance matrix estimator based on the maximum likelihood approach is guaranteed to be positive definite even in finite samples. Under mild conditions, we obtain the theoretical properties of the resulting estimators. A Bayesian information criterion is also developed to select the order of the polynomial function. Simulation studies and empirical examples illustrate the usefulness of the proposed methods.     

Orthogonal Locally Ancillary Estimating Functions with Application to Stratified Clustered Data

We propose an orthogonal locally ancillary estimating function that provides first-order bias correction of inferences. It requires the specification of merely the first two moments of the observations when applying to analysis of stratified clustered (continuous or binary) data with the parameters of interest in both the first and second joint moments of dependent data. Simulation results confirm that the estimators obtained using the proposed method are substantially improved over those using regular profile estimating functions.     

Local polynomial regression and simulation–extrapolation

Summary.  The paper introduces a new local polynomial estimator and develops supporting asymptotic theory for nonparametric regression in the presence of covariate measurement error. We address the measurement error with Cook and Stefanski's simulation–extrapolation (SIMEX) algorithm. Our method improves on previous local polynomial estimators for this problem by using a bandwidth selection procedure that addresses SIMEX's particular estimation method and considers higher degree local polynomial estimators. We illustrate the accuracy of our asymptotic expressions with a Monte Carlo study, compare our method with other estimators with a second set of Monte Carlo simulations and apply our method to a data set from nutritional epidemiology. SIMEX was originally developed for parametric models. Although SIMEX is, in principle, applicable to nonparametric models, a serious problem arises with SIMEX in nonparametric situations. The problem is that smoothing parameter selectors that are developed for data without measurement error are no longer appropriate and can result in considerable undersmoothing. We believe that this is the first paper to address this difficulty.     

Robust modal estimation and variable selection for single-index varying-coefficient models

In this article, we present a new efficient iteration estimation approach based on local modal regression for single-index varying-coefficient models. The resulted estimators are shown to be robust with regardless of outliers and error distributions. The asymptotic properties of the estimators are established under some regularity conditions and a practical modified EM algorithm is proposed for the new method. Moreover, to achieve sparse estimator when there exists irrelevant variables in the index parameters, a variable selection procedure based on SCAD penalty is developed to select significant parametric covariates and the well-known oracle properties are also derived. Finally, some numerical examples with various distributed errors and a real data analysis are conducted to illustrate the validity and feasibility of our proposed method.