Estimation of regression parameters in generalized linear models for cluster correlated data with measurement error

Liang and Zeger (1986) introduced a class of estimating equations that gives consistent estimates of regression parameters and of their asymptotic variances in the class of generalized linear models for cluster correlated data. When the independent variables or covariates in such models are subject to measurement errors, the parameter estimates obtained from these estimating equations are no longer consistent. To correct for the effect of measurement errors, an estimator with smaller asymptotic bias is constructed along the lines of Stefanski (1985), assuming that the measurement error variance is either known or estimable. The asymptotic distribution of the bias-corrected estimator and a consistent estimator of its asymptotic variance are also given. The special case of a binary logistic regression model is studied in detail. For this case, methods based on conditional scores and quasilikelihood are also extended to cluster correlated data. Results of a small simulation study on the performance of the proposed estimators and associated tests of hypotheses are reported.     

Uniformly adaptive estimation for models with arma errors

A semiparametric estimator based on an unknown density isuniformly adaptive if the expected loss of the estimator converges to the asymptotic expected loss of the maximum liklihood estimator based on teh true density (MLE), and if convergence does not depend on either the parameter values or the form of the unknown density. Without uniform adaptivity, the asymptotic expected loss of the MLE need not approximate the expected loss of a semiparametric estimator for any finite sample I show that a two step semiparametric estimator is uniformly adaptive for the parameters of nonlinear regression models with autoregressive moving average errors.     

A Convolution Estimator for the Density of Nonlinear Regression Observations

Abstract. The problem of estimating an unknown density function has been widely studied. In this article, we present a convolution estimator for the density of the responses in a nonlinear heterogenous regression model. The rate of convergence for the mean square error of the convolution estimator is of order n ^?1 under certain regularity conditions. This is faster than the rate for the kernel density method. We derive explicit expressions for the asymptotic variance and the bias of the new estimator, and further a data‐driven bandwidth selector is proposed. We conduct simulation experiments to check the finite sample properties, and the convolution estimator performs substantially better than the kernel density estimator for well‐behaved noise densities.     

A New Regression Model: Modal Linear Regression

The mode of a distribution provides an important summary of data and is often estimated on the basis of some non‐parametric kernel density estimator. This article develops a new data analysis tool called modal linear regression in order to explore high‐dimensional data. Modal linear regression models the conditional mode of a response Y given a set of predictors x as a linear function of x . Modal linear regression differs from standard linear regression in that standard linear regression models the conditional mean (as opposed to mode) of Y as a linear function of x . We propose an expectation–maximization algorithm in order to estimate the regression coefficients of modal linear regression. We also provide asymptotic properties for the proposed estimator without the symmetric assumption of the error density. Our empirical studies with simulated data and real data demonstrate that the proposed modal regression gives shorter predictive intervals than mean linear regression, median linear regression and MM‐estimators.     

Methods of Statistical Inference for Median Regression Models with Doubly Censored Data

Recently, least absolute deviations (LAD) estimator for median regression models with doubly censored data was proposed and the asymptotic normality of the estimator was established. However, it is invalid to make inference on the regression parameter vectors, because the asymptotic covariance matrices are difficult to estimate reliably since they involve conditional densities of error terms. In this article, three methods, which are based on bootstrap, random weighting, and empirical likelihood, respectively, and do not require density estimation, are proposed for making inference for the doubly censored median regression models. Simulations are also done to assess the performance of the proposed methods.     

Uniformly adaptive estimation for models with arma errors

A semiparametric estimator based on an unknown density isuniformly adaptive if the expected loss of the estimator converges to the asymptotic expected loss of the maximum liklihood estimator based on teh true density (MLE), and if convergence does not depend on either the parameter values or the form of the unknown density. Without uniform adaptivity, the asymptotic expected loss of the MLE need not approximate the expected loss of a semiparametric estimator for any finite sample I show that a two step semiparametric estimator is uniformly adaptive for the parameters of nonlinear regression models with autoregressive moving average errors.     

Optimal and Robust Designs for Estimating the Concentration Curve and the AUC

The problem of interest is to estimate the concentration curve and the area under the curve (AUC) by estimating the parameters of a linear regression model with an autocorrelated error process. We introduce a simple linear unbiased estimator of the concentration curve and the AUC. We show that this estimator constructed from a sampling design generated by an appropriate density is asymptotically optimal in the sense that it has exactly the same asymptotic performance as the best linear unbiased estimator. Moreover, we prove that the optimal design is robust with respect to a minimax criterion. When repeated observations are available, this estimator is consistent and has an asymptotic normal distribution. Finally, a simulated annealing algorithm is applied to a pharmacokinetic model with correlated errors.     

Minimum distance estimation in linear regression with strong mixing errors

Abstract

Minimum distance estimation on the linear regression model with independent errors is known to yield an efficient and robust estimator. We extend the method to the model with strong mixing errors and obtain an estimator of the vector of the regression parameters. The goal of this article is to demonstrate the proposed estimator still retains efficiency and robustness. To that end, this article investigates asymptotic distributional properties of the proposed estimator and compares it with other estimators. The efficiency and the robustness of the proposed estimator are empirically shown, and its superiority over the other estimators is established.     

Non linear parametric mode regression

In this article, we propose a semi-parametric mode regression for a non linear model. We use an expectation-maximization algorithm in order to estimate the regression coefficients of modal non linear regression. We also establish asymptotic properties for the proposed estimator under assumptions of the error density. We investigate the performance through a simulation study.     

SEMIPARAMETRIC COMPARISON OF REGRESSION CURVES VIA NORMAL LIKELIHOODS

Härdle & Marron (1990) treated the problem of semiparametric comparison of nonparametric regression curves by proposing a kernel-based estimator derived by minimizing a version of weighted integrated squared error. The resulting estimators of unknown transformation parameters are n-consistent, which prompts a consideration of issues. of optimality. We show that when the unknown mean function is periodic, an optimal nonparametric estimator may be motivated by an elegantly simple argument based on maximum likelihood estimation in a parametric model with normal errors. Strikingly, the asymptotic variance of an optimal estimator of θ does not depend at all on the manner of estimating error variances, provided they are estimated n-consistently. The optimal kernel-based estimator derived via these considerations is asymptotically equivalent to a periodic version of that suggested by Härdle & Marron, and so the latter technique is in fact optimal in this sense. We discuss the implications of these conclusions for the aperiodic case.     

Asymptotic distributions of error density and distribution function estimators in nonparametric regression

This paper considers the problem of estimating the error density and distribution functions in nonparametric regression models. The asymptotic distribution of a suitably standardized density estimator at a fixed point is shown to be normal while that of the maximum of a suitably normalized deviation of the density estimator from the true density function is the same as in the case of the one sample set up. Finally, the standardized residual empirical process is shown to be uniformly close to the similarly standardized empirical process of the errors. This paper thus generalizes some of the well known results about the residual density estimators and the empirical process in parametric regression models to nonparametric regression models, thereby enhancing the domain of their applications.     

Asymptotic Properties and Variance Estimators of the M-quantile Regression Coefficients Estimators

M-quantile regression is defined as a “quantile-like” generalization of robust regression based on influence functions. This article outlines asymptotic properties for the M-quantile regression coefficients estimators in the case of i.i.d. data with stochastic regressors, paying attention to adjustments due to the first-step scale estimation. A variance estimator of the M-quantile regression coefficients based on the sandwich approach is proposed. Empirical results show that this estimator appears to perform well under different simulated scenarios. The sandwich estimator is applied in the small area estimation context for the estimation of the mean squared error of an estimator for the small area means. The results obtained improve previous findings, especially in the case of heteroskedastic data.     

Estimation in autoregressive models with surrogate data and validation data

Time-series data are often subject to measurement error, usually the result of needing to estimate the variable of interest. Generally, however, the relationship between the surrogate variables and the true variables can be rather complicated compared to the classical additive error structure usually assumed. In this article, we address the estimation of the parameters in autoregressive models in the presence of function measurement errors. We first develop a parameter estimation method with the help of validation data; this estimation method does not depend on functional form and the distribution of the measurement error. The proposed estimator is proved to be consistent. Moreover, the asymptotic representation and the asymptotic normality of the estimator are also derived, respectively. Simulation results indicate that the proposed method works well for practical situation.     

Analysis of cointegrated models with measurement errors

We study the asymptotic properties of the reduced-rank estimator of error correction models of vector processes observed with measurement errors. Although it is well known that there is no asymptotic measurement error bias when predictor variables are integrated processes in regression models [Phillips BCB, Durlauf SN. Multiple time series regression with integrated processes. Rev Econom Stud. 1986;53:473–495], we systematically investigate the effects of the measurement errors (in the dependent variables as well as in the predictor variables) on the estimation of not only cointegrating vectors but also the speed of the adjustment matrix. Furthermore, we present the asymptotic properties of the estimators. We also obtain the asymptotic distribution of the likelihood ratio test for the cointegrating ranks. We investigate the effects of the measurement errors on estimation and test through a Monte Carlo simulation study.     

Estimating the density of a possibly missing response variable in nonlinear regression

This paper considers linear and nonlinear regression with a response variable that is allowed to be “missing at random”. The only structural assumptions on the distribution of the variables are that the errors have mean zero and are independent of the covariates. The independence assumption is important. It enables us to construct an estimator for the response density that uses all the observed data, in contrast to the usual local smoothing techniques, and which therefore permits a faster rate of convergence. The idea is to write the response density as a convolution integral which can be estimated by an empirical version, with a weighted residual-based kernel estimator plugged in for the error density. For an appropriate class of regression functions, and a suitably chosen bandwidth, this estimator is consistent and converges with the optimal parametric rate n^1/2. Moreover, the estimator is proved to be efficient (in the sense of Hájek and Le Cam) if an efficient estimator is used for the regression parameter.     

Artificial neural networks for some macroeconomic series: A first report

There is a tendency for the true variability of feasible GLS estimators to be understated by asymptotic standard errors. For estimation of SUR models, this tendency becomes more severe in large equation systems when estimation of the error covariance matrix, C, becomes problematic. We explore a number of potential solutions involving the use of improved estimators for the disturbance covariance matrix and bootstrapping. In particular, Ullah and Racine (1992) have recently introduced a new class of estimators for SUR models that use nonparametric kernel density estimation techniques. The proposed estimators have the same structure as the feasible GLS estimator of Zellner (1962) differing only in the choice of estimator for C. Ullah and Racine (1992) prove that their nonparametric density estimator of C can be expressed as Zellner's original estimator plus a positive definite matrix that depends on the smoothing parameter chosen for the density estimation. It is this structure of the estimator that most interests us as it has the potential to be especially useful in large equation systems.

Atkinson and Wilson (1992) investigated the bias in the conventional and bootstrap estimators of coefficient standard errors in SUR models. They demonstrated that under certain conditions the former were superior, but they caution that neither estimator uniformly dominated and hence bootstrapping provides little improvement in the estimation of standard errors for the regression coefficients. Rilstone and Veal1 (1996) argue that an important qualification needs to be made to this somewhat negative conclusion. They demonstrated that bootstrapping can result in improvements in inferences if the procedures are applied to the t-ratios rather than to the standard errors. These issues are explored for the case of large equation systems and when bootstrapping is combined with improved covariance estimation.     

AMEMIYA'S FORM OF THE WEIGHTED LEAST SQUARES ESTIMATOR

Amemiya's estimator is a weighted least squares estimator of the regression coefficients in a linear model with heteroscedastic errors. It is attractive because the heteroscedasticity is not parametrized and the weights (which depend on the error covariance matrix) are estimated nonparametrically. This paper derives an asymptotic expansion for Amemiya's form of the weighted least squares estimator, and uses it to discuss the effects of estimating the weights, of the number of iterations, and of the choice of the initial estimate. The paper also discusses the special case of normally distributed errors and clarifies the particular consequences of assuming normality.     

ORDER SELECTION IN ARMA MODELS USING THE FOCUSED INFORMATION CRITERION

This paper develops a new approach for order selection in autoregressive moving average models using the focused information criterion. This criterion minimizes the asymptotic mean squared error of the estimator of a parameter of interest. Simulation studies indicate that the suggested criterion is quite effective and comparable to the Akaike information criterion, the corrected Akaike information criterion and the Bayesian information criterion in autoregressive moving average order selection. The use of the focused information criterion for the simultaneous selection of regression variables and order of the error process in a linear regression model with autoregressive moving average errors is also considered.     

Adaptive wavelet estimation of a function from an m-dependent process with possibly unbounded m

The estimation of a multivariate function from a stationary m-dependent process is investigated, with a special focus on the case where m is large or unbounded. We develop an adaptive estimator based on wavelet methods. Under flexible assumptions on the nonparametric model, we prove the good performances of our estimator by determining sharp rates of convergence under two kinds of errors: the pointwise mean squared error and the mean integrated squared error. We illustrate our theoretical result by considering the multivariate density estimation problem, the derivatives density estimation problem, the density estimation problem in a GARCH-type model and the multivariate regression function estimation problem. The performance of proposed estimator has been shown by a numerical study for a simulated and real data sets.     

Robust Linear Calibration

We regard the simple linear calibration problem where only the response y of the regression line y = β₀ + β₁ t is observed with errors. The experimental conditions t are observed without error. For the errors of the observations y we assume that there may be some gross errors providing outlying observations. This situation can be modeled by a conditionally contaminated regression model. In this model the classical calibration estimator based on the least squares estimator has an unbounded asymptotic bias. Therefore we introduce calibration estimators based on robust one-step-M-estimators which have a bounded asymptotic bias. For this class of estimators we discuss two problems: The optimal estimators and their corresponding optimal designs. We derive the locally optimal solutions and show that the maximin efficient designs for non-robust estimation and robust estimation coincide.