A comparison of regression calibration approaches for designs with internal validation data

We compare the asymptotic relative efficiency of several regression calibration methods of correcting for measurement error in studies with internal validation data, when a single covariate is measured with error. The estimators we consider are appropriate in main study/hybrid validation study designs, where the latter study includes internal validation and may include external validation data. Although all of the methods we consider produce consistent estimates, the method proposed by Spiegelman et al. (Statistics in Medicine, 20 (2001) 139) has an asymptotically smaller variance than the other methods. The methods for measurement error correction are illustrated using a study of the effect of in utero lead exposure on infant birth weight.     

The effects of the proxy information on the iterative Stein-rule estimator of the disturbance variance

In a regression model with proxy variables, we consider the iterative estimator of the disturbance variance to obtain more precise estimates. In the formula of the estimator of the disturbance variance, the estimator is obtained by using Stein-rule (SR) estimator instead of OLS (ordinary least squares) estimator is called Iterative estimator of the disturbance variance. It is shown that, in a regression model with proxy variables the mean square error (MSE) of the iterative estimator of the disturbance variance is greater than the MSE of the disturbance variance related to the OLS estimator under certain conditions.     

Adaptive estimation of vector autoregressive models with time-varying variance: Application to testing linear causality in mean

Linear vector autoregressive (VAR) models where the innovations could be unconditionally heteroscedastic are considered. The volatility structure is deterministic and quite general, including breaks or trending variances as special cases. In this framework we propose ordinary least squares (OLS), generalized least squares (GLS) and adaptive least squares (ALS) procedures. The GLS estimator requires the knowledge of the time-varying variance structure while in the ALS approach the unknown variance is estimated by kernel smoothing with the outer product of the OLS residual vectors. Different bandwidths for the different cells of the time-varying variance matrix are also allowed. We derive the asymptotic distribution of the proposed estimators for the VAR model coefficients and compare their properties. In particular we show that the ALS estimator is asymptotically equivalent to the infeasible GLS estimator. This asymptotic equivalence is obtained uniformly with respect to the bandwidth(s) in a given range and hence justifies data-driven bandwidth rules. Using these results we build Wald tests for the linear Granger causality in mean which are adapted to VAR processes driven by errors with a nonstationary volatility. It is also shown that the commonly used standard Wald test for the linear Granger causality in mean is potentially unreliable in our framework (incorrect level and lower asymptotic power). Monte Carlo experiments illustrate the use of the different estimation approaches for the analysis of VAR models with time-varying variance innovations.     

Risk comparison of the Stein-rule estimator in a linear regression model with omitted relevant regressors and multivariate<Emphasis Type="Italic">t</Emphasis> errors under the Pitman nearness criterion

In this paper we consider a linear regression model with omitted relevant regressors and multivariatet error terms. The explicit formula for the Pitman nearness criterion of the Stein-rule (SR) estimator relative to the ordinary least squares (OLS) estimator is derived. It is shown numerically that the dominance of the SR estimator over the OLS estimator under the Pitman nearness criterion can be extended to the case of the multivariatet error distribution when the specification error is not severe. It is also shown that the dominance of the SR estimator over the OLS estimator cannot be extended to the case of the multivariatet error distribution when the specification error is severe. This research is partially supported by the Grants-in-Aid for 21st Century COE program.     

A Better Alternative to Non-parametric Approaches for Adjusting for Covariate Measurement Errors in Logistic Regression

In this article, we propose a flexible parametric (FP) approach for adjusting for covariate measurement errors in regression that can accommodate replicated measurements on the surrogate (mismeasured) version of the unobserved true covariate on all the study subjects or on a sub-sample of the study subjects as error assessment data. We utilize the general framework of the FP approach proposed by Hossain and Gustafson in 2009 for adjusting for covariate measurement errors in regression. The FP approach is then compared with the existing non-parametric approaches when error assessment data are available on the entire sample of the study subjects (complete error assessment data) considering covariate measurement error in a multiple logistic regression model. We also developed the FP approach when error assessment data are available on a sub-sample of the study subjects (partial error assessment data) and investigated its performance using both simulated and real life data. Simulation results reveal that, in comparable situations, the FP approach performs as good as or better than the competing non-parametric approaches in eliminating the bias that arises in the estimated regression parameters due to covariate measurement errors. Also, it results in better efficiency of the estimated parameters. Finally, the FP approach is found to perform adequately well in terms of bias correction, confidence coverage, and in achieving appropriate statistical power under partial error assessment data.     

A monte carlo study of collinearity in linear simultaneous equation models ∗

Although the collinearity issue has been studied in previous simulation studies with a simultaneous system of equations, alternative estimators to circumvent this problem have received little attention. Monte Carlo techniques are used to examine the performance of several estimators under a squared error loss criterion. In particular, this study considers the Vinod–Ullah ridge-type estimators at the first and/or second stage of 2SLS. Ridge regression in the second stage only of 2SLS but not the first stage only, seems to be a practical alternative to 2SLS, especially in situations of strong collinearity. The OLS estimator and the ordinary ridge regression estimator also yield favorable results in situations of moderate to strong collinearity.     

Errors-in-Variables Regression Using Stein Estimates

A method is proposed for estimating regression parameters from data containing covariate measurement errors by using Stein estimates of the unobserved true covariates. The method produces consistent estimates for the slope parameter in the classical linear errors-in-variables model and applies to a broad range of nonlinear regression problems, provided the measurement error is Gaussian with known variance. Simulations are used to examine the performance of the estimates in a nonlinear regression problem and to compare them with the usual naive ones obtained by ignoring error and with other estimates proposed recently in the literature.     

The error components regression model: conditional relative efficiency comparisons

This paper considers a simple linear regression with two-way error component disturbances and derives the conditional relative efficiency ofany feasible GLS estimator with respect to OLS, true GLS, orany other feasible GLS estimator, conditional on the estimated variance components. This is done at two crucial choices of the x variable. The first choice is where OLS is least efficient with respect to GLS and the second choice is where an arbitrary feasible GLS estimator is least efficient with respect to GLS. Our findings indicate that a better guess of a certain ‘variance components ratio’ leads to better estimates of the regression coefficients.     

Support vector regression based residual control charts

Control charts for residuals, based on the regression model, require a robust fitting technique for minimizing the error resulting from the fitted model. However, in the multivariate case, when the number of variables is high and data become complex, traditional fitting techniques, such as ordinary least squares (OLS), lose efficiency. In this paper, support vector regression (SVR) is used to construct robust control charts for residuals, called SVR-chart. This choice is based on the fact that the SVR is designed to minimize the structural error whereas other techniques minimize the empirical error. An application shows that SVR methods gives competitive results in comparison with the OLS and the partial least squares method, in terms of standard deviation of the error prediction and the standard error of performance. A sensitivity study is conducted to evaluate the SVR-chart performance based on the average run length (ARL) and showed that the SVR-chart has the best ARL behaviour in comparison with the other residuals control charts.     

The relative efficiency of the restricted estimators in linear regression models

This paper deals with the problem of multicollinearity in a multiple linear regression model with linear equality restrictions. The restricted two parameter estimator which was proposed in case of multicollinearity satisfies the restrictions. The performance of the restricted two parameter estimator over the restricted least squares (RLS) estimator and the ordinary least squares (OLS) estimator is examined under the mean square error (MSE) matrix criterion when the restrictions are correct and not correct. The necessary and sufficient conditions for the restricted ridge regression, restricted Liu and restricted shrunken estimators, which are the special cases of the restricted two parameter estimator, to have a smaller MSE matrix than the RLS and the OLS estimators are derived when the restrictions hold true and do not hold true. Theoretical results are illustrated with numerical examples based on Webster, Gunst and Mason data and Gorman and Toman data. We conduct a final demonstration of the performance of the estimators by running a Monte Carlo simulation which shows that when the variance of the error term and the correlation between the explanatory variables are large, the restricted two parameter estimator performs better than the RLS estimator and the OLS estimator under the configurations examined.     

THE EXAMINATION AND ANALYSIS OF RESIDUALS FOR SOME BIASED ESTIMATORS IN LINEAR REGRESSION

In the presence of collinearity certain biased estimation procedures like ridge regression, generalized inverse estimator, principal component regression, Liu estimator, or improved ridge and Liu estimators are used to improve the ordinary least squares (OLS) estimates in the linear regression model. In this paper new biased estimator (Liu estimator), almost unbiased (improved) Liu estimator and their residuals will be analyzed and compared with OLS residuals in terms of mean-squared error.     

Bayesian estimation of logistic regression with misclassified covariates and response

Measurement error is a commonly addressed problem in psychometrics and the behavioral sciences, particularly where gold standard data either does not exist or are too expensive. The Bayesian approach can be utilized to adjust for the bias that results from measurement error in tests. Bayesian methods offer other practical advantages for the analysis of epidemiological data including the possibility of incorporating relevant prior scientific information and the ability to make inferences that do not rely on large sample assumptions. In this paper we consider a logistic regression model where both the response and a binary covariate are subject to misclassification. We assume both a continuous measure and a binary diagnostic test are available for the response variable but no gold standard test is assumed available. We consider a fully Bayesian analysis that affords such adjustments, accounting for the sources of error and correcting estimates of the regression parameters. Based on the results from our example and simulations, the models that account for misclassification produce more statistically significant results, than the models that ignore misclassification. A real data example on math disorders is considered.     

Estimation of dependence between paired correlated failure times in the presence of covariate measurement error

Summary. In many biomedical studies, covariates are subject to measurement error. Although it is well known that the regression coefficients estimators can be substantially biased if the measurement error is not accommodated, there has been little study of the effect of covariate measurement error on the estimation of the dependence between bivariate failure times. We show that the dependence parameter estimator in the Clayton–Oakes model can be considerably biased if the measurement error in the covariate is not accommodated. In contrast with the typical bias towards the null for marginal regression coefficients, the dependence parameter can be biased in either direction. We introduce a bias reduction technique for the bivariate survival function in copula models while assuming an additive measurement error model and replicated measurement for the covariates, and we study the large and small sample properties of the dependence parameter estimator proposed.     

Classification trees aided mixed regression model

This paper introduces a novel hybrid regression method (MixReg) combining two linear regression methods, ordinary least square (OLS) and least squares ratio (LSR) regression. LSR regression is a method to find the regression coefficients minimizing the sum of squared error rate while OLS minimizes the sum of squared error itself. The goal of this study is to combine two methods in a way that the proposed method superior both OLS and LSR regression methods in terms of R² statistics and relative error rate. Applications of MixReg, on both simulated and real data, show that MixReg method outperforms both OLS and LSR regression.     

Cox Regression with Covariate Measurement Error

This article deals with parameter estimation in the Cox proportional hazards model when covariates are measured with error. We consider both the classical additive measurement error model and a more general model which represents the mis-measured version of the covariate as an arbitrary linear function of the true covariate plus random noise. Only moment conditions are imposed on the distributions of the covariates and measurement error. Under the assumption that the covariates are measured precisely for a validation set, we develop a class of estimating equations for the vector-valued regression parameter by correcting the partial likelihood score function. The resultant estimators are proven to be consistent and asymptotically normal with easily estimated variances. Furthermore, a corrected version of the Breslow estimator for the cumulative hazard function is developed, which is shown to be uniformly consistent and, upon proper normalization, converges weakly to a zero-mean Gaussian process. Simulation studies indicate that the asymptotic approximations work well for practical sample sizes. The situation in which replicate measurements (instead of a validation set) are available is also studied.     

Improving survey-weighted least squares regression

The weighted least squares (WLS) estimator is often employed in linear regression using complex survey data to deal with the bias in ordinary least squares (OLS) arising from informative sampling. In this paper a 'quasi-Aitken WLS' (QWLS) estimator is proposed. QWLS modifies WLS in the same way that Cragg's quasi-Aitken estimator modifies OLS. It weights by the usual inverse sample inclusion probability weights multiplied by a parameterized function of covariates, where the parameters are chosen to minimize a variance criterion. The resulting estimator is consistent for the superpopulation regression coefficient under fairly mild conditions and has a smaller asymptotic variance than WLS.     

Deletion residuals in the detection of heterogeneity of variances in linear regression

The heterogeneity of error variance often causes a huge interpretive problem in linear regression analysis. Before taking any remedial measures we first need to detect this problem. A large number of diagnostic plots are now available in the literature for detecting heteroscedasticity of error variances. Among them the ‘residuals’ and ‘fits’ (R–F) plot is very popular and commonly used. In the R–F plot residuals are plotted against the fitted responses, where both these components are obtained using the ordinary least squares (OLS) method. It is now evident that the OLS fits and residuals suffer a huge setback in the presence of unusual observations and hence the R–F plot may not exhibit the real scenario. The deletion residuals based on a data set free from all unusual cases should estimate the true errors in a better way than the OLS residuals. In this paper we propose ‘deletion residuals’ and the ‘deletion fits’ (DR–DF) plot for the detection of the heterogeneity of error variances in a linear regression model to get a more convincing and reliable graphical display. Examples show that this plot locates unusual observations more clearly than the R–F plot. The advantage of using deletion residuals in the detection of heteroscedasticity of error variance is investigated through Monte Carlo simulations under a variety of situations.     

Asymptotics for Panel Models with Common Shocks

This article develops a novel asymptotic theory for panel models with common shocks. We assume that contemporaneous correlation can be generated by both the presence of common regressors among units and weak spatial dependence among the error terms. Several characteristics of the panel are considered: cross-sectional and time-series dimensions can either be fixed or large; factors can either be observable or unobservable; the factor model can describe either a cointegration relationship or a spurious regression, and we also consider the stationary case. We derive the rate of convergence and the limit distributions for the ordinary least square (OLS) estimates of the model parameters under all the aforementioned cases.     

Double sampling for exact values in the normal discriminant model with application to binary regression

Increasing attention is being given to problems involving binary outcomes with covariates subject to measurement error. Here, we consider the two group normal discriminant model where a subset of the continuous variates are subject to error and will typically be replaced by a vector of surrogates, perhaps of different dimension. Correcting for the measurement error is made possible by a double sampling scheme in which the surrogates are collected on all units and true values are obtained on a random subset of units. Such a scheme allows us to consider a rich set of measurement error models which extend the traditional additive error model. Maximum likelihood estimators and their asymptotic properties are derived under a variety of models for the relationship between true values and the surrogates. Specific attention is given to the coefficients in the resulting logistic regression model. Optimal allocations are derived which minimize the variance of the estimated slope subject to cost constraints for the case where there is a univariate covariate but a possibly multivariate surrogate.