Estimation using penalized quasilikelihood and quasi-pseudo-likelihood in Poisson mixed models

We consider two estimation schemes based on penalized quasilikelihood and quasi-pseudo-likelihood in Poisson mixed models. The asymptotic bias in regression coefficients and variance components estimated by penalized quasilikelihood (PQL) is studied for small values of the variance components. We show the PQL estimators of both regression coefficients and variance components in Poisson mixed models have a smaller order of bias compared to those for binomial data. Unbiased estimating equations based on quasi-pseudo-likelihood are proposed and are shown to yield consistent estimators under some regularity conditions. The finite sample performance of these two methods is compared through a simulation study.     

Editorial collaborators

A simulation study of the binomial-logit model with correlated random effects is carried out based on the generalized linear mixed model (GLMM) methodology. Simulated data with various numbers of regression parameters and different values of the variance component are considered. The performance of approximate maximum likelihood (ML) and residual maximum likelihood (REML) estimators is evaluated. For a range of true parameter values, we report the average biases of estimators, the standard error of the average bias and the standard error of estimates over the simulations. In general, in terms of bias, the two methods do not show significant differences in estimating regression parameters. The REML estimation method is slightly better in reducing the bias of variance component estimates.     

Random effects selection in generalized linear mixed models via shrinkage penalty function

In this paper, we discuss the selection of random effects within the framework of generalized linear mixed models (GLMMs). Based on a reparametrization of the covariance matrix of random effects in terms of modified Cholesky decomposition, we propose to add a shrinkage penalty term to the penalized quasi-likelihood (PQL) function of the variance components for selecting effective random effects. The shrinkage penalty term is taken as a function of the variance of random effects, initiated by the fact that if the variance is zero then the corresponding variable is no longer random (with probability one). The proposed method takes the advantage of a convenient computation for the PQL estimation and appealing properties for certain shrinkage penalty functions such as LASSO and SCAD. We propose to use a backfitting algorithm to estimate the fixed effects and variance components in GLMMs, which also selects effective random effects simultaneously. Simulation studies show that the proposed approach performs quite well in selecting effective random effects in GLMMs. Real data analysis is made using the proposed approach, too.     

On the biases of change point and change magnitude estimation after CUSUM test

The present paper investigates the biases of estimates of change point and change magnitude after CUSUM test. By assuming that the change point is far from the beginning and the in-control average run length of samples is large, second order approximations for the biases of both estimates are obtained by conditioning on detection, and biases of both estimates are very significant. Simulation studies show the approximations to be quite accurate in the case of detecting an increase in mean or variance when sampling from a normal distribution. The results demonstrate the fundamental differences between fixed sample size test and sequential test.     

The effects of model mis-specifications in linear measurement error models

In the literature, there are many results on the consequences of mis-specified models for linear models with error in the response only, see, e.g., Seber(1977). There are also discussions of estimation for the model writh errors both in the response and in the predictor variables (called measurement error models; see, e.g., Fuller(1987)). In this paper, we consider the problem of model mis-specification for measurement error models. Only a few special cases have been tackled in the past (Edland, 1996; Carroll and Ruppert, 1996 and Lakshminarayanan Amp; Gunst, 1984); we deal with the situation here in some generality. Results have been obtained as follows: (a) When a model is under-fitted, the estimate of the variance of the measurement error will be asymptotically biased, as will the regression coefficients, and the asymptotic biases in the estimates of the regression coefficients will always exist for under-fitted models. Even orthogonality of the variables in the model will not make the biases vanish. (b)For over-fitting, the estimates of the variances of measurement errors and of the regression coefficients are asymptotically unbiased. However, the variance of the estimated regression coefficients will increase. Over-fitting will cause larger changes in the variances of the estimated parameters in measurement error models than in no measurement error models.     

C239. The continued fraction for lnr( z): a stieljes conjecture

Estimation in logistic-normal models for correlated and overdispersed binomial data is complicated by the numerical evaluation of often intractable likelihood functions. Penalized quasilikelihood (PQL) estimators of fixed effects and variance components are known to be seriously biased for binary data. A simple correction procedure has been proposed to improve the performance of the PQL estimators. The proposed method is illustrated by analyzing infectious disease data. Its performance is compared, by means of simulations, with that of the Bayes approach using the Gibbs sampler.     

Efficient Estimation of an Additive Quantile Regression Model

Abstract. In this paper, two non‐parametric estimators are proposed for estimating the components of an additive quantile regression model. The first estimator is a computationally convenient approach which can be viewed as a more viable alternative to existing kernel‐based approaches. The second estimator involves sequential fitting by univariate local polynomial quantile regressions for each additive component with the other additive components replaced by the corresponding estimates from the first estimator. The purpose of the extra local averaging is to reduce the variance of the first estimator. We show that the second estimator achieves oracle efficiency in the sense that each estimated additive component has the same variance as in the case when all other additive components were known. Asymptotic properties are derived for both estimators under dependent processes that are strictly stationary and absolutely regular. We also provide a demonstrative empirical application of additive quantile models to ambulance travel times.     

Bias corrected estimates in multivariate student t regression models

The t distribution has proved to be a useful alternative to the normal distribution especially When robust estimation is desired. We consider the multivariate nonlinear Student-t regression model and show that the biased of the estimates of the regression coefficients can be computed from an auxiliary generalized linear regression. We give a formula for the biases of the estimates of the parameters in the scale matrix, which also can be computed by means of a generalized linear regression. We briefly discuss some important special cases and present simulation results which indicate that our bias-corrected estimates outperform the uncorrected ones in small samples.     

Performance of likelihood-based estimation methods for multilevel binary regression models

By means of a fractional factorial simulation experiment, we compare the performance of penalised quasi-likelihood (PQL), non-adaptive Gaussian quadrature and adaptive Gaussian quadrature in estimating parameters for multilevel logistic regression models. The comparison is done in terms of bias, mean-squared error (MSE), numerical convergence and computational efficiency. It turns out that in terms of MSE, standard versions of the quadrature methods perform relatively poorly in comparison with PQL.     

Graphical methods for evaluating ridge regression estimator in mixture experiments

When the component proportions in mixture experiments are restricted by lower and upper bounds, multicollinearity appears all too frequently. Thus, we can suggest the use of ridge regression as a mean for stabilizing the coefficient estimates in the fitted model. We propose graphical methods for evaluating the effect of ridge regression estimator with respect to the predicted response value and the prediction variance.     

A revised Cholesky decomposition to combat multicollinearity in multiple regression models

As known, the ordinary least-squares estimator (OLSE) is unbiased and also, has the minimum variance among all the linear unbiased estimators. However, under multicollinearity the estimator is generally unstable and poor in the sense that variance of the regression coefficients may be inflated and absolute values of the estimates may be too large. There are several classes of biased estimators in statistical literature to decrease the effect of multicollinearity in the design matrix. Here, based on the Cholesky decomposition, we propose such an estimator which makes the data to be slightly distorted. The exact risk expressions as well as the biases are derived for the proposed estimator. Also, some results demonstrating superiority of the suggested estimator over OLSE are obtained. Finally, a Monté-Carlo simulation study and a real data application related to acetylene data are presented to support our theoretical discussions.     

Restricted likelihood ratio lack-of-fit tests using mixed spline models

Summary.  Penalized regression spline models afford a simple mixed model representation in which variance components control the degree of non-linearity in the smooth function estimates. This motivates the study of lack-of-fit tests based on the restricted maximum likelihood ratio statistic which tests whether variance components are 0 against the alternative of taking on positive values. For this one-sided testing problem a further complication is that the variance component belongs to the boundary of the parameter space under the null hypothesis. Conditions are obtained on the design of the regression spline models under which asymptotic distribution theory applies, and finite sample approximations to the asymptotic distribution are provided. Test statistics are studied for simple as well as multiple-regression models.     

Second-order biases of the maximum likelihood estimates in von Mises regression models

This paper discusses issues related to the improvement of maximum likelihood estimates in von Mises regression models. It obtains general matrix expressions for the second-order biases of maximum likelihood estimates of the mean parameters and concentration parameters. The formulae are simple to compute, and give the biases by means of weighted linear regressions. Simulation results are presented assessing the performance of corrected maximum likelihood estimates in these models.     

Robust estimation of variance components

New robust estimates for variance components are introduced. Two simple models are considered: the balanced one-way classification model with a random factor and the balanced mixed model with one random factor and one fixed factor. However, the method of estimation proposed can be extended to more complex models. The new method of estimation we propose is based on the relationship between the variance components and the coefficients of the least-mean-squared-error predictor between two observations of the same group. This relationship enables us to transform the problem of estimating the variance components into the problem of estimating the coefficients of a simple linear regression model. The variance-component estimators derived from the least-squares regression estimates are shown to coincide with the maximum-likelihood estimates. Robust estimates of the variance components can be obtained by replacing the least-squares estimates by robust regression estimates. In particular, a Monte Carlo study shows that for outlier-contaminated normal samples, the estimates of variance components derived from GM regression estimates and the derived test outperform other robust procedures.     

Bounds on minimum mean squared error in ridge regression

The minimum MSE (mean squared error) of ridge regression coefficient estimates (for a given set of eigenvalues and variance) is a function of the transformed coefficient vector. In this paper, the authors prove that the minimum MSE is bounded, for a given coefficient vector length, by the two cases corresponding to the signal completely contained in the component associated with the smallest or largest eigenvalue. The implication for evaluating proposed estimators of the ridge regression biasing parameter is discussed.     

Statistical inference for heteroscedastic semi-varying coefficient EV models

This paper proposes an estimation procedure for a class of semi-varying coefficient regression models when the covariates of the linear part are subject to measurement errors. Initial estimates for the regression and varying coefficients are first constructed by the profile least-squares procedure without input from heteroscedasticity, a bias-corrected kernel estimate for the variance function then is proposed, which in turn is used to define re-weighted bias-corrected estimates of the regression and varying coefficients. Large sample properties of the proposed estimates are thoroughly investigated. The finite-sample performance of the proposed estimates is assessed by an extensive simulation study and an application to the Boston housing data set. The simulation results show that the re-weighted bias-corrected estimates outperform the initial estimates and the naive estimates.     

Bias correction and improved residuals for non-exponential family nonlinear models

In this paper we derive general formulae for the biases to order n ^?1 of the parameter estimates in a general class of nonlinear regression models, where n is the sample size. The formulae are related to those of Cordeiro and McCullagh (1991) and Paula (1992) and may be viewed as extensions of their results, Correction factors are derived for the score and deviance component residuals in these models. The practical use of such corrections is illustrated for the log-gamma model.     

Some asymptotic behaviour of the bootstrap estimates on a finite sample

Bootstrapping the mean, variance, standard error of the mean, regression coefficient and its standard error is considered. It is shown that at a fixed sample size bootstrap estimates converge to classical sample estimates as the number of bootstrap replications tends to infinity. For the mean, variance and regression coefficient, convergence almost everywhere is proven; for the standard error of the mean and standard error of the regression coefficient, weak convergence is proven. The speed of convergence is illustrated by simulation results.     

Smoothing the difference-based estimates of variance using variational mode decomposition

We propose a variational mode decomposition approach to estimate the variance function in a nonparametric heteroscedastic fixed design regression model. A data-driven estimator is constructed by applying variational mode decomposition technique to the difference-based initial estimates. The numerical results show that the proposed estimator performs better than the existing variance estimation procedures in the mean square sense.     

Covariate adjustment and estimation of mean response in randomised trials

Analyses of randomised trials are often based on regression models which adjust for baseline covariates, in addition to randomised group. Based on such models, one can obtain estimates of the marginal mean outcome for the population under assignment to each treatment, by averaging the model‐based predictions across the empirical distribution of the baseline covariates in the trial. We identify under what conditions such estimates are consistent, and in particular show that for canonical generalised linear models, the resulting estimates are always consistent. We show that a recently proposed variance estimator underestimates the variance of the estimator around the true marginal population mean when the baseline covariates are not fixed in repeated sampling and provide a simple adjustment to remedy this. We also describe an alternative semiparametric estimator, which is consistent even when the outcome regression model used is misspecified. The different estimators are compared through simulations and application to a recently conducted trial in asthma.