Mean response estimation with missing response in the presence of high-dimensional covariates

This paper studies the problem of mean response estimation where missingness occurs to the response but multiple-dimensional covariates are observable. Two main challenges occur in this situation: curse of dimensionality and model specification. The non parametric imputation method relieves model specification but suffers curse of dimensionality, while some model-based methods such as inverse probability weighting (IPW) and augmented inverse probability weighting (AIPW) methods are the opposite. We propose a unified non parametric method to overcome the two challenges with the aiding of sufficient dimension reduction. It imposes no parametric structure on propensity score or conditional mean response, and thus retains the non parametric flavor. Moreover, the estimator achieves the optimal efficiency that a double robust estimator can attain. Simulations were conducted and it demonstrates the excellent performances of our method in various situations.     

A Copula‐Based Non‐parametric Measure of Regression Dependence

Abstract. This article presents a framework for comparing bivariate distributions according to their degree of regression dependence. We introduce the general concept of a regression dependence order (RDO). In addition, we define a new non‐parametric measure of regression dependence and study its properties. Besides being monotone in the new RDOs, the measure takes on its extreme values precisely at independence and almost sure functional dependence, respectively. A consistent non‐parametric estimator of the new measure is constructed and its asymptotic properties are investigated. Finally, the finite sample properties of the estimate are studied by means of a small simulation study.     

An oracle property of the Nadaraya–Watson kernel estimator for high‐dimensional nonparametric regression

The Nadaraya–Watson estimator is among the most studied nonparametric regression methods. A classical result is that its convergence rate depends on the number of covariates and deteriorates quickly as the dimension grows. This underscores the “curse of dimensionality” and has limited its use in high‐dimensional settings. In this paper, however, we show that the Nadaraya–Watson estimator has an oracle property such that when the true regression function is single‐ or multi‐index, it discovers the low‐rank dependence structure between the response and the covariates, mitigating the curse of dimensionality. Specifically, we prove that, using K‐fold cross‐validation and a positive‐semidefinite bandwidth matrix, the Nadaraya–Watson estimator has a convergence rate that depends on the number of indices rather than on the number of covariates. This result follows by allowing the bandwidths to diverge to infinity rather than restricting them all to converge to zero at certain rates, as in previous theoretical studies.     

Guided Censored Regression

Parametrically guided non‐parametric regression is an appealing method that can reduce the bias of a non‐parametric regression function estimator without increasing the variance. In this paper, we adapt this method to the censored data case using an unbiased transformation of the data and a local linear fit. The asymptotic properties of the proposed estimator are established, and its performance is evaluated via finite sample simulations.     

Series Estimation in Partially Linear In‐Slide Regression Models

Abstract. The partially linear in‐slide model (PLIM) is a useful tool to make econometric analyses and to normalize microarray data. In this article, by using series approximations and a least squares procedure, we propose a semiparametric least squares estimator (SLSE) for the parametric component and a series estimator for the non‐parametric component. Under weaker conditions than those imposed in the literature, we show that the SLSE is asymptotically normal and that the series estimator attains the optimal convergence rate of non‐parametric regression. We also investigate the estimating problem of the error variance. In addition, we propose a wild block bootstrap‐based test for the form of the non‐parametric component. Some simulation studies are conducted to illustrate the finite sample performance of the proposed procedure. An example of application on a set of economical data is also illustrated.     

Averaging estimation for conditional covariance models

Estimating conditional covariance matrices is important in statistics and finance. In this paper, we propose an averaging estimator for the conditional covariance, which combines the estimates of marginal conditional covariance matrices by Model Averaging MArginal Regression of Li, Linton, and Lu. This estimator avoids the “curse of dimensionality” problem that the local constant estimator of Yin et al. suffered from. We establish the asymptotic properties of the averaging weights and that of the proposed conditional covariance estimator. The finite sample performances are augmented by simulation. An application to portfolio allocation illustrates the practical superiority of the averaging estimator.     

Enhancements of Non‐parametric Generalized Likelihood Ratio Test: Bias Correction and Dimension Reduction

Non‐parametric generalized likelihood ratio test is a popular method of model checking for regressions. However, there are two issues that may be the barriers for its powerfulness: existing bias term and curse of dimensionality. The purpose of this paper is thus twofold: a bias reduction is suggested and a dimension reduction‐based adaptive‐to‐model enhancement is recommended to promote the power performance. The proposed test statistic still possesses the Wilks phenomenon and behaves like a test with only one covariate. Thus, it converges to its limit at a much faster rate and is much more sensitive to alternative models than the classical non‐parametric generalized likelihood ratio test. As a by‐product, we also prove that the bias‐corrected test is more efficient than the one without bias reduction in the sense that its asymptotic variance is smaller. Simulation studies and a real data analysis are conducted to evaluate of proposed tests.     

A comparison of dependence function estimators in multivariate extremes

Various nonparametric and parametric estimators of extremal dependence have been proposed in the literature. Nonparametric methods commonly suffer from the curse of dimensionality and have been mostly implemented in extreme-value studies up to three dimensions, whereas parametric models can tackle higher-dimensional settings. In this paper, we assess, through a vast and systematic simulation study, the performance of classical and recently proposed estimators in multivariate settings. In particular, we first investigate the performance of nonparametric methods and then compare them with classical parametric approaches under symmetric and asymmetric dependence structures within the commonly used logistic family. We also explore two different ways to make nonparametric estimators satisfy the necessary dependence function shape constraints, finding a general improvement in estimator performance either (i) by substituting the estimator with its greatest convex minorant, developing a computational tool to implement this method for dimensions \(D\ge 2\) or (ii) by projecting the estimator onto a subspace of dependence functions satisfying such constraints and taking advantage of Bernstein–Bézier polynomials. Implementing the convex minorant method leads to better estimator performance as the dimensionality increases.     

Kernel‐based Generalized Cross‐validation in Non‐parametric Mixed‐effect Models

Abstract. Although generalized cross‐validation (GCV) has been frequently applied to select bandwidth when kernel methods are used to estimate non‐parametric mixed‐effect models in which non‐parametric mean functions are used to model covariate effects, and additive random effects are applied to account for overdispersion and correlation, the optimality of the GCV has not yet been explored. In this article, we construct a kernel estimator of the non‐parametric mean function. An equivalence between the kernel estimator and a weighted least square type estimator is provided, and the optimality of the GCV‐based bandwidth is investigated. The theoretical derivations also show that kernel‐based and spline‐based GCV give very similar asymptotic results. This provides us with a solid base to use kernel estimation for mixed‐effect models. Simulation studies are undertaken to investigate the empirical performance of the GCV. A real data example is analysed for illustration.     

Semi‐parametric Estimation in a Single‐index Model with Endogenous Variables

We consider a semiparametric single‐index model and suppose that endogeneity is present in the explanatory variables. The presence of an instrument is assumed, that is, non‐correlated with the error term. We propose an estimator of the parametric component of the model, which is the solution of an ill‐posed inverse problem. The estimator is shown to be asymptotically normal under certain regularity conditions. A simulation study is conducted to illustrate the finite sample performance of the proposed estimator.     

Testing the Goodness of Fit of Parametric Regression Models with Random Toeplitz Forms

ABSTRACT: We introduce a class of Toeplitz‐band matrices for simple goodness of fit tests for parametric regression models. For a given length r of the band matrix the asymptotic optimal solution is derived. Asymptotic normality of the corresponding test statistic is established under a fixed and random design assumption as well as for linear and non‐linear models, respectively. This allows testing at any parametric assumption as well as the computation of confidence intervals for a quadratic measure of discrepancy between the parametric model and the true signal g;. Furthermore, the connection between testing the parametric goodness of fit and estimating the error variance is highlighted. As a by‐product we obtain a much simpler proof of a result of 34 ) concerning the optimality of an estimator for the variance. Our results unify and generalize recent results by 9 ) and 15 , 16 ) in several directions. Extensions to multivariate predictors and unbounded signals are discussed. A simulation study shows that a simple jacknife correction of the proposed test statistics leads to reasonable finite sample approximations.     

Shape Constrained Non‐parametric Estimators of the Baseline Distribution in Cox Proportional Hazards Model

Abstract. We investigate non‐parametric estimation of a monotone baseline hazard and a decreasing baseline density within the Cox model. Two estimators of a non‐decreasing baseline hazard function are proposed. We derive the non‐parametric maximum likelihood estimator and consider a Grenander type estimator, defined as the left‐hand slope of the greatest convex minorant of the Breslow estimator. We demonstrate that the two estimators are strongly consistent and asymptotically equivalent and derive their common limit distribution at a fixed point. Both estimators of a non‐increasing baseline hazard and their asymptotic properties are obtained in a similar manner. Furthermore, we introduce a Grenander type estimator for a non‐increasing baseline density, defined as the left‐hand slope of the least concave majorant of an estimator of the baseline cumulative distribution function, derived from the Breslow estimator. We show that this estimator is strongly consistent and derive its asymptotic distribution at a fixed point.     

Dimension reduced kernel estimation for distribution function with incomplete data

This work focuses on the estimation of distribution functions with incomplete data, where the variable of interest Y has ignorable missingness but the covariate X is always observed. When X is high dimensional, parametric approaches to incorporate X—information is encumbered by the risk of model misspecification and nonparametric approaches by the curse of dimensionality. We propose a semiparametric approach, which is developed under a nonparametric kernel regression framework, but with a parametric working index to condense the high dimensional X—information for reduced dimension. This kernel dimension reduction estimator has double robustness to model misspecification and is most efficient if the working index adequately conveys the X—information about the distribution of Y. Numerical studies indicate better performance of the semiparametric estimator over its parametric and nonparametric counterparts. We apply the kernel dimension reduction estimation to an HIV study for the effect of antiretroviral therapy on HIV virologic suppression.     

Model‐Based Non‐parametric Variance Estimation for Systematic Sampling

Abstract. Systematic sampling is frequently used in surveys, because of its ease of implementation and its design efficiency. An important drawback of systematic sampling, however, is that no direct estimator of the design variance is available. We describe a new estimator of the model‐based expectation of the design variance, under a non‐parametric model for the population. The non‐parametric model is sufficiently flexible that it can be expected to hold at least approximately in many situations with continuous auxiliary variables observed at the population level. We prove the model consistency of the estimator for both the anticipated variance and the design variance under a non‐parametric model with a univariate covariate. The broad applicability of the approach is demonstrated on a dataset from a forestry survey.     

Mixtures of Gaussian copula factor analyzers for clustering high dimensional data

Mixtures of factor analyzers is a useful model-based clustering method which can avoid the curse of dimensionality in high-dimensional clustering. However, this approach is sensitive to both diverse non-normalities of marginal variables and outliers, which are commonly observed in multivariate experiments. We propose mixtures of Gaussian copula factor analyzers (MGCFA) for clustering high-dimensional clustering. This model has two advantages; (1) it allows different marginal distributions to facilitate fitting flexibility of the mixture model, (2) it can avoid the curse of dimensionality by embedding the factor-analytic structure in the component-correlation matrices of the mixture distribution.An EM algorithm is developed for the fitting of MGCFA. The proposed method is free of the curse of dimensionality and allows any parametric marginal distribution which fits best to the data. It is applied to both synthetic data and a microarray gene expression data for clustering and shows its better performance over several existing methods.     

An Extended Single‐index Model with Missing Response at Random

An extended single‐index model is considered when responses are missing at random. A three‐step estimation procedure is developed to define an estimator for the single‐index parameter vector by a joint estimating equation. The proposed estimator is shown to be asymptotically normal. An algorithm for computing this estimator is proposed. This algorithm only involves one‐dimensional nonparametric smoothers, thereby avoiding the data sparsity problem caused by high model dimensionality. Some simulation studies are conducted to investigate the finite sample performances of the proposed estimators.     

Non‐parametric Threshold Estimation for Models with Stochastic Diffusion Coefficient and Jumps

Abstract. We consider a stochastic process driven by diffusions and jumps. Given a discrete record of observations, we devise a technique for identifying the times when jumps larger than a suitably defined threshold occurred. This allows us to determine a consistent non‐parametric estimator of the integrated volatility when the infinite activity jump component is Lévy. Jump size estimation and central limit results are proved in the case of finite activity jumps. Some simulations illustrate the applicability of the methodology in finite samples and its superiority on the multipower variations especially when it is not possible to use high frequency data.     

Fitting Cox Models with Doubly Censored Data Using Spline‐Based Sieve Marginal Likelihood

In some applications, the failure time of interest is the time from an originating event to a failure event while both event times are interval censored. We propose fitting Cox proportional hazards models to this type of data using a spline‐based sieve maximum marginal likelihood, where the time to the originating event is integrated out in the empirical likelihood function of the failure time of interest. This greatly reduces the complexity of the objective function compared with the fully semiparametric likelihood. The dependence of the time of interest on time to the originating event is induced by including the latter as a covariate in the proportional hazards model for the failure time of interest. The use of splines results in a higher rate of convergence of the estimator of the baseline hazard function compared with the usual non‐parametric estimator. The computation of the estimator is facilitated by a multiple imputation approach. Asymptotic theory is established and a simulation study is conducted to assess its finite sample performance. It is also applied to analyzing a real data set on AIDS incubation time.     

Goodness‐of‐Fit Test for Monotone Functions

Abstract. In this article, we develop a test for the null hypothesis that a real‐valued function belongs to a given parametric set against the non‐parametric alternative that it is monotone, say decreasing. The method is described in a general model that covers the monotone density model, the monotone regression and the right‐censoring model with monotone hazard rate. The criterion for testing is an ‐distance between a Grenander‐type non‐parametric estimator and a parametric estimator computed under the null hypothesis. A normalized version of this distance is shown to have an asymptotic normal distribution under the null, whence a test can be developed. Moreover, a bootstrap procedure is shown to be consistent to calibrate the test.     

A new local estimation method for single index models for longitudinal data

Single index models are natural extensions of linear models and overcome the so-called curse of dimensionality. They are very useful for longitudinal data analysis. In this paper, we develop a new efficient estimation procedure for single index models with longitudinal data, based on Cholesky decomposition and local linear smoothing method. Asymptotic normality for the proposed estimators of both the parametric and nonparametric parts will be established. Monte Carlo simulation studies show excellent finite sample performance. Furthermore, we illustrate our methods with a real data example.