Optimum percentile estimating equations for nonlinear random coefficient models

In nonlinear random coefficients models, the means or variances of response variables may not exist. In such cases, commonly used estimation procedures, e.g., (extended) least-squares (LS) and quasi-likelihood methods, are not applicable. This article solves this problem by proposing an estimate based on percentile estimating equations (PEE). This method does not require full distribution assumptions and leads to efficient estimates within the class of unbiased estimating equations. By minimizing the asymptotic variance of the PEE estimates, the optimum percentile estimating equations (OPEE) are derived. Several examples including Weibull regression show the flexibility of the PEE estimates. Under certain regularity conditions, the PEE estimates are shown to be strongly consistent and asymptotic normal, and the OPEE estimates have the minimal asymptotic variance. Compared with the parametric maximum likelihood estimates (MLE), the asymptotic efficiency of the OPEE estimates is more than 98%, while the LS-type of procedures can have infinite variances. When the observations have outliers or do not follow the distributions considered in model assumptions, the article shows that OPEE is more robust than the MLE, and the asymptotic efficiency in the model misspecification cases can be above 150%.     

Bayesian shrinkage estimates for regression coefficients in m populations

For a linear regression model over m populations with separate regression coefficients but a common error variance, a Bayesian model is employed to obtain regression coefficient estimates which are shrunk toward an overall value. The formulation uses Normal priors on the coefficients and diffuse priors on the grand mean vectors, the error variance, and the between-to-error variance ratios. The posterior density of the parameters which were given diffuse priors is obtained. From this the posterior means and variances of regression coefficients and the predictive mean and variance of a future observation are obtained directly by numerical integration in the balanced case, and with the aid of series expansions in the approximately balanced case. An example is presented and worked out for the case of one predictor variable. The method is an extension of Box & Tiao's Bayesian estimation of means in the balanced one-way random effects model.     

Robust estimation and design procedures for the random effects model

The authors discuss two robust estimators for estimating variance components in the random effects model, and they obtain finite‐sample breakdown points for the estimators. Based on the finite‐sample breakdown point, they propose a criterion for selecting robust designs. With robust designs, one can get efficient and reliable estimates for variance components regardless of outliers which may happen in the experiment. The authors give examples to show the performance of robust estimators and to compare robust designs with optimal designs based on the traditional analysis of variance estimation method.     

Re-weighting estimation of the coefficients in the varying coefficient model with heteroscedastic errors

ABSTRACT

This article explores the estimation problem of the coefficients in the varying coefficient model with heteroscedastic errors. Specifically, we first present a method for estimating the variance function of the error term and the resulting estimator is proved to be consistent. Then, motivated by the generalized least-squares procedure for dealing with heteroscedasticity in the linear regression literature, we re-weight each squared residual term in the local linear smoother with the inverse of the corresponding estimated error variance to construct estimates of the coefficients. Simulation experiments and practical data analysis conducted demonstrate that the re-weighting approach can improve the accuracy of the coefficient estimates under a finite sample size, especially when the error heteroscedasticity is strong.     

Comparison of designs for the three-fold nested random model

The quality of estimation of variance components depends on the design used as well as on the unknown values of the variance components. In this article, three designs are compared, namely, the balanced, staggered, and inverted nested designs for the three-fold nested random model. The comparison is based on the so-called quantile dispersion graphs using analysis of variance (ANOVA) and maximum likelihood (ML) estimates of the variance components. It is demonstrated that the staggered nested design gives more stable estimates of the variance component for the highest nesting factor than the balanced design. The reverse, however, is true in case of lower nested factors. A comparison between ANOVA and ML estimation of the variance components is also made using each of the aforementioned designs.     

Dependence Estimation for Marginal Models of Multivariate Survival Data

We have previously(Segal and Neuhaus, 1993) devised methods for obtaining marginal regression coefficients and associated variance estimates for multivariate survival data, using a synthesis of the Poisson regression formulation for univariate censored survival analysis and generalized estimating equations (GEE's). The method is parametric in that a baseline survival distribution is specified. Analogous semiparametric models, with unspecified baseline survival, have also been developed (Wei, Lin and Weissfeld, 1989; Lin, 1994).Common to both these approaches is the provision of robust variances for the regression parameters. However, none of this work has addressed the more difficult area of dependence estimation. While GEE approaches ostensibly provide such estimates, we show that there are problems adopting these with multivariate survival data. Further, we demonstrate that these problems can affect estimation of the regression coefficients themselves. An alternate, ad hoc approach to dependence estimation, based on design effects, is proposed and evaluated via simulation and illustrative examples. This revised version was published online in July 2006 with corrections to the Cover Date.     

Robust inference for bivariate point processes

In the analysis of recurrent events where the primary interest lies in studying covariate effects on the expected number of events occurring over a period of time, it is appealing to base models on the cumulative mean function (CMF) of the processes (Lawless & Nadeau 1995). In many chronic diseases, however, more than one type of event is manifested. Here we develop a robust inference procedure for joint regression models for the CMFs arising from a bivariate point process. Consistent parameter estimates with robust variance estimates are obtained via unbiased estimating functions for the CMFs. In most situations, the covariance structure of the bivariate point processes is difficult to specify correctly, but when it is known, an optimal estimating function for the CMFs can be obtained. As a convenient model for more general settings, we suggest the use of the estimating functions arising from bivariate mixed Poisson processes. Simulation studies demonstrate that the estimators based on this working model are practically unbiased with robust variance estimates. Furthermore, hypothesis tests may be based on the generalized Wald or generalized score tests. Data from a trial of patients with bronchial asthma are analyzed to illustrate the estimation and inference procedures.     

A Numerical Study of PQL Estimation Biases in Generalized Linear Mixed Models Under Heterogeneity of Random Effects

The penalized quasi-likelihood (PQL) approach is the most common estimation procedure for the generalized linear mixed model (GLMM). However, it has been noticed that the PQL tends to underestimate variance components as well as regression coefficients in the previous literature. In this article, we numerically show that the biases of variance component estimates by PQL are systematically related to the biases of regression coefficient estimates by PQL, and also show that the biases of variance component estimates by PQL increase as random effects become more heterogeneous.     

Statistical inference for heteroscedastic semi-varying coefficient EV models

This paper proposes an estimation procedure for a class of semi-varying coefficient regression models when the covariates of the linear part are subject to measurement errors. Initial estimates for the regression and varying coefficients are first constructed by the profile least-squares procedure without input from heteroscedasticity, a bias-corrected kernel estimate for the variance function then is proposed, which in turn is used to define re-weighted bias-corrected estimates of the regression and varying coefficients. Large sample properties of the proposed estimates are thoroughly investigated. The finite-sample performance of the proposed estimates is assessed by an extensive simulation study and an application to the Boston housing data set. The simulation results show that the re-weighted bias-corrected estimates outperform the initial estimates and the naive estimates.     

Estimation using penalized quasilikelihood and quasi-pseudo-likelihood in Poisson mixed models

We consider two estimation schemes based on penalized quasilikelihood and quasi-pseudo-likelihood in Poisson mixed models. The asymptotic bias in regression coefficients and variance components estimated by penalized quasilikelihood (PQL) is studied for small values of the variance components. We show the PQL estimators of both regression coefficients and variance components in Poisson mixed models have a smaller order of bias compared to those for binomial data. Unbiased estimating equations based on quasi-pseudo-likelihood are proposed and are shown to yield consistent estimators under some regularity conditions. The finite sample performance of these two methods is compared through a simulation study.     

Cocaine Dependence Treatment Data: Methods for Measurement Error Problems With Predictors Derived From Stationary Stochastic Processes

In a cocaine dependence treatment study, we use linear and nonlinear regression models to model posttreatment cocaine craving scores and first cocaine relapse time. A subset of the covariates are summary statistics derived from baseline daily cocaine use trajectories, such as baseline cocaine use frequency and average daily use amount. These summary statistics are subject to estimation error and can therefore cause biased estimators for the regression coefficients. Unlike classical measurement error problems, the error we encounter here is heteroscedastic with an unknown distribution, and there are no replicates for the error-prone variables or instrumental variables. We propose two robust methods to correct for the bias: a computationally efficient method-of-moments-based method for linear regression models and a subsampling extrapolation method that is generally applicable to both linear and nonlinear regression models. Simulations and an application to the cocaine dependence treatment data are used to illustrate the efficacy of the proposed methods. Asymptotic theory and variance estimation for the proposed subsampling extrapolation method and some additional simulation results are described in the online supplementary material.     

Semiparametric estimation for weighted average derivatives with responses missing at random

When responses are missing at random, we propose a semiparametric direct estimator for the missing probability and density-weighted average derivatives of a general nonparametric multiple regression function. An estimator for the normalized version of the weighted average derivatives is constructed as well using instrumental variables regression. The proposed estimators are computationally simple and asymptotically normal, and provide a solution to the problem of estimating index coefficients of single-index models with responses missing at random. The developed theory generalizes the method of the density-weighted average derivatives estimation of Powell et al. (1989) for the non-missing data case. Monte Carlo simulation studies are conducted to study the performance of the methods.     

Prediction in a class of mixed models with two variance components

A best unbiased predictor (BUP) of an arbitrary linear combination of fixed and random effects in mixed linear models is available when the true values of the variance ratios are known. When the true values are unknown, a two-stage predictor, obtained from the BUP by replacing the true values by estimated values, can be used. In this article, exact mean squared errors of two-stage predictors are obtained for a class of mixed models with two variance components that includes the balanced one-way random model and other analysis-of-variance models with proportional frequencies and one balanced random factor.     

Inference for linear and nonlinear stable error processes via estimating functions

This paper describes an estimating function approach for parameter estimation in linear and nonlinear times series models with infinite variance stable errors. Joint estimates of location and scale parameters are derived for classes of autoregressive (AR) models and random coefficient autoregressive (RCA) models with stable errors, as well as for AR models with stable autoregressive conditionally heteroscedastic (ARCH) errors. Fast, on-line, recursive parametric estimation for the location parameter based on estimating functions is discussed using simulation studies. A real financial time series is also discussed in some detail.     

Asymptotic efficiency of a competing risks model: A two-sample case

A generalized Cox regression model is studied for the covariance analysis of competing risks data subject to independent random censoring. The information of the maximum partial likelihood estimates is compared with that of maximum likelihood estimates assuming a log linear hazard function.The method of generalized variance is used to define the efficiency of estimation between the two models. This is then applied to two-sample problems with two exponentially censoring rates. Numerical results are summarized ane presented graphically.The detailed results indicate that the semi-parametric model wrks well for a higher rate of censoring. A method of generalizing the result to type 1 censoring and the efficiency of estimating the coefficient of the covariate are discussecd. A brief account of using the results to help design experiments is also given.     

A Jackknife Method for Estimation of Variance Components

This paper concerns a method of estimation of variance components in a random effect linear model. It is mainly a resampling method and relies on the Jackknife principle. The derived estimators are presented as least squares estimators in an appropriate linear model, and one of them appears as a MINQUE (Minimum Norm Quadratic Unbiased Estimation) estimator. Our resampling method is illustrated by an example given by C. R. Rao [7] and some optimal properties of our estimator are derived for this example. In the last part, this method is used to derive an estimation of variance components in a random effect linear model when one of the components is assumed to be known.     

Restricted likelihood ratio lack-of-fit tests using mixed spline models

Summary.  Penalized regression spline models afford a simple mixed model representation in which variance components control the degree of non-linearity in the smooth function estimates. This motivates the study of lack-of-fit tests based on the restricted maximum likelihood ratio statistic which tests whether variance components are 0 against the alternative of taking on positive values. For this one-sided testing problem a further complication is that the variance component belongs to the boundary of the parameter space under the null hypothesis. Conditions are obtained on the design of the regression spline models under which asymptotic distribution theory applies, and finite sample approximations to the asymptotic distribution are provided. Test statistics are studied for simple as well as multiple-regression models.     

Mixtures of regression models with incomplete and noisy data

The estimation of the mixtures of regression models is usually based on the normal assumption of components and maximum likelihood estimation of the normal components is sensitive to noise, outliers, or high-leverage points. Missing values are inevitable in many situations and parameter estimates could be biased if the missing values are not handled properly. In this article, we propose the mixtures of regression models for contaminated incomplete heterogeneous data. The proposed models provide robust estimates of regression coefficients varying across latent subgroups even under the presence of missing values. The methodology is illustrated through simulation studies and a real data analysis.     

Random Coefficients Estimation of Average Total Product Costs for Multiproduct Firms

The problem of estimating the mean average total cost of each output for multiproduct firms in an industry is addressed. The identity that defines total cost for each firm as the product of output levels multiplied by their respective average total costs is viewed as a random-coefficients model. A random coefficients regression estimator is used to estimate mean average total output costs. Solutions to problems arising with this method in empirical studies are discussed. An application of the approach to data from cash grain farms in Illinois shows that the method gives reliable estimates for per-unit output production costs with considerably fewer data requirements than current methods of cost estimation.     

Robust sampling strategies for regression estimation

For probability linear regression estimation, conditions are derived where sampling will be robust against violations of the commonly assumed heterogeneous variance model. Stratified pps (spps) and stratified random sampling (spscx) are shown to satisfy these conditions approximately and are more efficient generally than restricted simple random sampling (RSRS) for some real populations and for artificial populations with weights of k = 0, 0.5, 1.0, 1.5 and 2.0. The criteria needs some additional refinement to better predict relative efficiency of spps and spscx.