Estimating the error variance in nonparametric regression by a covariate-matched u-statistic

For nonparametric regression models with fixed and random design, two classes of estimators for the error variance have been introduced: second sample moments based on residuals from a nonparametric fit, and difference-based estimators. The former are asymptotically optimal but require estimating the regression function; the latter are simple but have larger asymptotic variance. For nonparametric regression models with random covariates, we introduce a class of estimators for the error variance that are related to difference-based estimators: covariate-matched U-statistics. We give conditions on the random weights involved that lead to asymptotically optimal estimators of the error variance. Our explicit construction of the weights uses a kernel estimator for the covariate density.     

Asymptotic optimality of estimators of a linear functional eeiation if the ratio of the error variances is known 2

For the problem of estimating a linear functional relation when the ratio of the error variances is known a general class of estimators is introduced. They include as special cases the instrumental variable and replication cases and some others. Conditions are given for consistency, asymptotic normality and asymptotic optimality within this class based on the variance of the limit distribution. Fisheb's lower bound for asymptotic variances is established, and under normality the asymptotically optimal estimators are shown to be best asymptotically normal. For an inhomogeneous linear relation only estimators which are invariant with respect to a translation of the origin are considered, and asymptotically optimal invariant and, under normality, best asymptotically normal invariant estimators are obtained. Several special cases are discussed.     

A Class of Estimators Using Auxiliary Information for Estimating Finite Population Variance in Presence of Measurement Errors

This article addresses the problem of estimating the population variance using auxiliary information in the presence of measurement errors. When the measurement error variance associated with study variable is known, a class of estimators of the population variance using auxiliary information has been proposed. We obtain the bias and mean squared errors of the suggested class of estimators upto the terms of order n ^?1, and also optimum estimators in asymptotic sense of the class with approximate mean squared error formula.     

On difference-based variance estimation in nonparametric regression when the covariate is high dimensional

Summary.  We consider the problem of estimating the noise variance in homoscedastic nonparametric regression models. For low dimensional covariates t  ∈  R ^d ,  d =1, 2, difference-based estimators have been investigated in a series of papers. For a given length of such an estimator, difference schemes which minimize the asymptotic mean-squared error can be computed for d =1 and d =2. However, from numerical studies it is known that for finite sample sizes the performance of these estimators may be deficient owing to a large finite sample bias. We provide theoretical support for these findings. In particular, we show that with increasing dimension d this becomes more drastic. If d 4, these estimators even fail to be consistent. A different class of estimators is discussed which allow better control of the bias and remain consistent when d 4. These estimators are compared numerically with kernel-type estimators (which are asymptotically efficient), and some guidance is given about when their use becomes necessary.     

Adaptive composite quantile regressions and their asymptotic relative efficiency

The composite quantile regression (CQR for short) provides an efficient and robust estimation for regression coefficients. In this paper we introduce two adaptive CQR methods. We make two contributions to the quantile regression literature. The first is that, both adaptive estimators treat the quantile levels as realizations of a random variable. This is quite different from the classic CQR in which the quantile levels are typically equally spaced, or generally, are treated as fixed values. Because the asymptotic variances of the adaptive estimators depend upon the generic distribution of the quantile levels, it has the potential to enhance estimation efficiency of the classic CQR. We compare the asymptotic variance of the estimator obtained by the CQR with that obtained by quantile regressions at each single quantile level. The second contribution is that, in terms of relative efficiency, the two adaptive estimators can be asymptotically equivalent to the CQR method as long as we choose the generic distribution of the quantile levels properly. This observation is useful in that it allows to perform parallel distributed computing when the computational complexity issue arises for the CQR method. We compare the relative efficiency of the adaptive methods with respect to some existing approaches through comprehensive simulations and an application to a real-world problem.     

Double sampling for exact values in the normal discriminant model with application to binary regression

Increasing attention is being given to problems involving binary outcomes with covariates subject to measurement error. Here, we consider the two group normal discriminant model where a subset of the continuous variates are subject to error and will typically be replaced by a vector of surrogates, perhaps of different dimension. Correcting for the measurement error is made possible by a double sampling scheme in which the surrogates are collected on all units and true values are obtained on a random subset of units. Such a scheme allows us to consider a rich set of measurement error models which extend the traditional additive error model. Maximum likelihood estimators and their asymptotic properties are derived under a variety of models for the relationship between true values and the surrogates. Specific attention is given to the coefficients in the resulting logistic regression model. Optimal allocations are derived which minimize the variance of the estimated slope subject to cost constraints for the case where there is a univariate covariate but a possibly multivariate surrogate.     

A comparison of regression calibration approaches for designs with internal validation data

We compare the asymptotic relative efficiency of several regression calibration methods of correcting for measurement error in studies with internal validation data, when a single covariate is measured with error. The estimators we consider are appropriate in main study/hybrid validation study designs, where the latter study includes internal validation and may include external validation data. Although all of the methods we consider produce consistent estimates, the method proposed by Spiegelman et al. (Statistics in Medicine, 20 (2001) 139) has an asymptotically smaller variance than the other methods. The methods for measurement error correction are illustrated using a study of the effect of in utero lead exposure on infant birth weight.     

Improved estimation in Lognormal regression models

Lognormal regression model with unknown error variance is considered. We give a class of estimators of the regression coefficients vector improving upon traditional estimator when the number of independent variables is at least three. The relationship between these estimators on one hand and James-Stein type estimators of the normal mean and improved estimators of the normal variance on another hand is discussed.     

Estimation of regression parameters in generalized linear models for cluster correlated data with measurement error

Liang and Zeger (1986) introduced a class of estimating equations that gives consistent estimates of regression parameters and of their asymptotic variances in the class of generalized linear models for cluster correlated data. When the independent variables or covariates in such models are subject to measurement errors, the parameter estimates obtained from these estimating equations are no longer consistent. To correct for the effect of measurement errors, an estimator with smaller asymptotic bias is constructed along the lines of Stefanski (1985), assuming that the measurement error variance is either known or estimable. The asymptotic distribution of the bias-corrected estimator and a consistent estimator of its asymptotic variance are also given. The special case of a binary logistic regression model is studied in detail. For this case, methods based on conditional scores and quasilikelihood are also extended to cluster correlated data. Results of a small simulation study on the performance of the proposed estimators and associated tests of hypotheses are reported.     

Arbitrage Pricing Theory as a Restricted Nonlinear Multivariate Regression Model Iterated Nonlinear Seemingly Unrelated Regression Estimates

By replacing the unknown random factors of factor analysis with observed macroeconomic variables, the arbitrage pricing theory (APT) is recast as a multivariate nonlinear regression model with across-equation restrictions. An explicit theoretical justification for the inclusion of an arbitrary, well-diversified market index is given. Using monthly returns on 70 stocks, iterated nonlinear seemingly unrelated regression techniques are employed to obtain joint estimates of asset sensitivities and their associated APT risk “prices.” Without the assumption of normally distributed errors, these estimators are strongly consistent and asymptotically normal. With the additional assumption of normal errors, they are also full-information maximum likelihood estimators. Classical asymptotic nonlinear nested hypothesis tests are supportive of the APT with measured macroeconomic factors.     

Bootstrap adaptive estimation: The trimmed-mean example

We consider the problem of choosing among a class of possible estimators by selecting the estimator with the smallest bootstrap estimate of finite sample variance. This is an alternative to using cross-validation to choose an estimator adaptively. The problem of a confidence interval based on such an adaptive estimator is considered. We illustrate the ideas by applying the method to the problem of choosing the trimming proportion of an adaptive trimmed mean. It is shown that a bootstrap adaptive trimmed mean is asymptotically normal with an asymptotic variance equal to the smallest among trimmed means. The asymptotic coverage probability of a bootstrap confidence interval based on such adaptive estimators is shown to have the nominal level. The intervals based on the asymptotic normality of the estimator share the same asymptotic result, but have poor small-sample properties compared to the bootstrap intervals. A small-sample simulation demonstrates that bootstrap adaptive trimmed means adapt themselves rather well even for samples of size 10.     

Probability density estimation with data missing at random when covariables are present

This paper addresses the problem of the probability density estimation in the presence of covariates when data are missing at random (MAR). The inverse probability weighted method is used to define a nonparametric and a semiparametric weighted probability density estimators. A regression calibration technique is also used to define an imputed estimator. It is shown that all the estimators are asymptotically normal with the same asymptotic variance as that of the inverse probability weighted estimator with known selection probability function and weights. Also, we establish the mean squared error (MSE) bounds and obtain the MSE convergence rates. A simulation is carried out to assess the proposed estimators in terms of the bias and standard error.     

Estimation in Varying-Coefficient Errors-in-Variables Models with Missing Response Variables

This article is concerned with the estimation of a varying-coefficient regression model when the response variable is sometimes missing and some of the covariates are measured with additive errors. We propose a class of estimators for the coefficient functions, as well as for the population mean and the error variance. The resulting estimators are shown to be asymptotically normal. Simulation studies are conducted to illustrate our approach.     

Optimal sample size allocation for multi-level stress testing with Weibull regression under Type-II censoring

We discuss the optimal allocation problem in a multi-level stress test with Type-II censoring and Weibull (extreme value) regression model. We derive the maximum-likelihood estimators and their asymptotic variance–covariance matrix through the Fisher information. Four optimality criteria are used to discuss the optimal allocation problem. Optimal allocation of units, both exactly for small sample sizes and asymptotically for large sample sizes, for two- and four-stress-level situations are determined numerically. Conclusions and discussions are provided based on the numerical studies.     

The effects of model mis-specifications in linear measurement error models

In the literature, there are many results on the consequences of mis-specified models for linear models with error in the response only, see, e.g., Seber(1977). There are also discussions of estimation for the model writh errors both in the response and in the predictor variables (called measurement error models; see, e.g., Fuller(1987)). In this paper, we consider the problem of model mis-specification for measurement error models. Only a few special cases have been tackled in the past (Edland, 1996; Carroll and Ruppert, 1996 and Lakshminarayanan Amp; Gunst, 1984); we deal with the situation here in some generality. Results have been obtained as follows: (a) When a model is under-fitted, the estimate of the variance of the measurement error will be asymptotically biased, as will the regression coefficients, and the asymptotic biases in the estimates of the regression coefficients will always exist for under-fitted models. Even orthogonality of the variables in the model will not make the biases vanish. (b)For over-fitting, the estimates of the variances of measurement errors and of the regression coefficients are asymptotically unbiased. However, the variance of the estimated regression coefficients will increase. Over-fitting will cause larger changes in the variances of the estimated parameters in measurement error models than in no measurement error models.     

Analysis of cointegrated models with measurement errors

We study the asymptotic properties of the reduced-rank estimator of error correction models of vector processes observed with measurement errors. Although it is well known that there is no asymptotic measurement error bias when predictor variables are integrated processes in regression models [Phillips BCB, Durlauf SN. Multiple time series regression with integrated processes. Rev Econom Stud. 1986;53:473–495], we systematically investigate the effects of the measurement errors (in the dependent variables as well as in the predictor variables) on the estimation of not only cointegrating vectors but also the speed of the adjustment matrix. Furthermore, we present the asymptotic properties of the estimators. We also obtain the asymptotic distribution of the likelihood ratio test for the cointegrating ranks. We investigate the effects of the measurement errors on estimation and test through a Monte Carlo simulation study.     

SEPARATE VERSUS SYSTEM METHODS OF STEIN-RULE ESTIMATION IN SEEMINGLY UNRELATED REGRESSION MODELS

ABSTRACT

Despite the sizeable literature associated with the seemingly unrelated regression models, not much is known about the use of Stein-rule estimators in these models. This gap is remedied in this paper, in which two families of Stein-rule estimators in seemingly unrelated regression equations are presented and their large sample asymptotic properties explored and evaluated. One family of estimators uses a shrinkage factor obtained solely from the equation under study while the other has a shrinkage factor based on all the equations of the model. Using a quadratic loss measure and Monte-Carlo sampling experiments, the finite sample risk performance of these estimators is also evaluated and compared with the traditional feasible generalized least squares estimator.     

Asymptotic normality of estimators in heteroscedastic errors-in-variables model

This article is concerned with the estimating problem of heteroscedastic partially linear errors-in-variables models. We derive the asymptotic normality for estimators of the slope parameter and the nonparametric component in the case of known error variance with stationary $\alpha $ -mixing random errors. Also, when the error variance is unknown, the asymptotic normality for the estimators of the slope parameter and the nonparametric component as well as variance function is considered under independent assumptions. Finite sample behavior of the estimators is investigated via simulations too.     

Conditional variance estimation via nonparametric generalized additive models

Generalized additive models provide a way of circumventing curse of dimension in a wide range of nonparametric regression problem. In this paper, we present a multiplicative model for conditional variance functions where one can apply a generalized additive regression method. This approach extends Fan and Yao (1998) to multivariate cases with a multiplicative structure. In this approach, we use squared residuals instead of using log-transformed squared residuals. This idea gives a smaller variance than Yu (2017) when the variance of squared error is smaller than the variance of log-transformed squared error. We provide estimators based on quasi-likelihood and an iterative algorithm based on smooth backfitting for generalized additive models. We also provide some asymptotic properties of estimators and the convergence of proposed algorithm. A numerical study shows the empirical evidence of the theory.     

Cox Regression with Covariate Measurement Error

This article deals with parameter estimation in the Cox proportional hazards model when covariates are measured with error. We consider both the classical additive measurement error model and a more general model which represents the mis-measured version of the covariate as an arbitrary linear function of the true covariate plus random noise. Only moment conditions are imposed on the distributions of the covariates and measurement error. Under the assumption that the covariates are measured precisely for a validation set, we develop a class of estimating equations for the vector-valued regression parameter by correcting the partial likelihood score function. The resultant estimators are proven to be consistent and asymptotically normal with easily estimated variances. Furthermore, a corrected version of the Breslow estimator for the cumulative hazard function is developed, which is shown to be uniformly consistent and, upon proper normalization, converges weakly to a zero-mean Gaussian process. Simulation studies indicate that the asymptotic approximations work well for practical sample sizes. The situation in which replicate measurements (instead of a validation set) are available is also studied.