Confidence intervals for the regression coefficient in a simple regression model with a balanced two-fold nested error structure

ABSTRACT

In applications using a simple regression model with a balanced two-fold nested error structure, interest focuses on inferences concerning the regression coefficient. This article derives exact and approximate confidence intervals on the regression coefficient in the simple regression model with a balanced two-fold nested error structure. Eleven methods are considered for constructing the confidence intervals on the regression coefficient. Computer simulation is performed to compare the proposed confidence intervals. Recommendations are suggested for selecting an appropriate method.     

Performance of Confidence Intervals in Regression Models with Unbalanced One-Fold Nested Error Structures

Abstract

In this article we consider the problem of constructing confidence intervals for a linear regression model with unbalanced nested error structure. A popular approach is the likelihood-based method employed by PROC MIXED of SAS. In this article, we examine the ability of MIXED to produce confidence intervals that maintain the stated confidence coefficient. Our results suggest that intervals for the regression coefficients work well, but intervals for the variance component associated with the primary level cannot be recommended. Accordingly, we propose alternative methods for constructing confidence intervals on the primary level variance component. Computer simulation is used to compare the proposed methods. A numerical example and SAS code are provided to demonstrate the methods.     

Empirical likelihood for generalized linear models with missing responses

The paper uses the empirical likelihood method to study the construction of confidence intervals and regions for regression coefficients and response mean in generalized linear models with missing response. By using the inverse selection probability weighted imputation technique, the proposed empirical likelihood ratios are asymptotically chi-squared. Our approach is to directly calibrate the empirical likelihood ratio, which is called as a bias-correction method. Also, a class of estimators for the parameters of interest is constructed, and the asymptotic distributions of the proposed estimators are obtained. A simulation study indicates that the proposed methods are comparable in terms of coverage probabilities and average lengths/areas of confidence intervals/regions. An example of a real data set is used for illustrating our methods.     

Empirical likelihood for the varying‐coefficient single‐index model

In this article the author investigates the application of the empirical‐likelihood‐based inference for the parameters of varying‐coefficient single‐index model (VCSIM). Unlike the usual cases, if there is no bias correction the asymptotic distribution of the empirical likelihood ratio cannot achieve the standard chi‐squared distribution. To this end, a bias‐corrected empirical likelihood method is employed to construct the confidence regions (intervals) of regression parameters, which have two advantages, compared with those based on normal approximation, that is, (1) they do not impose prior constraints on the shape of the regions; (2) they do not require the construction of a pivotal quantity and the regions are range preserving and transformation respecting. A simulation study is undertaken to compare the empirical likelihood with the normal approximation in terms of coverage accuracies and average areas/lengths of confidence regions/intervals. A real data example is given to illustrate the proposed approach. The Canadian Journal of Statistics 38: 434–452; 2010 © 2010 Statistical Society of Canada     

Alternative Confidence Intervals on Variance Components in a Simple Regression Model with a Balanced Two-Fold Nested Error Structure

In applications using a linear regression model with a balanced two-fold nested error structure, interest focuses on inferences concerning variability of the effects associated with the levels of nesting. This article proposes confidence intervals on the variance components associated with the primary and secondary levels in the model. In order to construct the confidence intervals we use a modified large sample method, generalized inference method, and Satterthwaite approximation. Computer simulation is performed to compare the proposed confidence intervals. A numerical example is provided to demonstrate the intervals.     

Empirical Likelihood Based Rank Regression Inference

Rank regression procedures have been proposed and studied for numerous research applications that do not satisfy the underlying assumptions of the more common linear regression models. This article develops confidence regions for the slope parameter of rank regression using an empirical likelihood (EL) ratio method. It has the advantage of not requiring variance estimation which is required for the normal approximation method. The EL method is also range respecting and results in asymmetric confidence intervals. Simulation studies are used to compare and evaluate normal approximation versus EL inference methods for various conditions such as different sample size or error distribution. The simulation study demonstrates our proposed EL method almost outperforms the traditional method in terms of coverage probability, lower-tail side error, and upper-tail side error. An application of stability analysis also shows the EL method results in shorter confidence intervals for real life data.     

Confidence intervals for the between group variance in the unbalanced one-way random effects model of anaylsis of variance

A confidence interval for the between group variance is proposed which is deduced from Wald'sexact confidence interval for the rtio of the two variance components in the one-way random effects model and the exact confidence interval for the error variance resp.an unbiased estimator of the error variance. In a simulation study the confidence coeffecients for these two intervals are compared with the confidence coefficients of two other commonly used confidence intervals. There the confidence interval derived here yields confidence coefficiends which are always greater than the prescriped level.     

Using adaptive weighted least squares to reduce the lengths of confidence intervals

The author proposes an adaptive method which produces confidence intervals that are often narrower than those obtained by the traditional procedures. The proposed methods use both a weighted least squares approach to reduce the length of the confidence interval and a permutation technique to insure that its coverage probability is near the nominal level. The author reports simulations comparing the adaptive intervals to the traditional ones for the difference between two population means, for the slope in a simple linear regression, and for the slope in a multiple linear regression having two correlated exogenous variables. He is led to recommend adaptive intervals for sample sizes superior to 40 when the error distribution is not known to be Gaussian.     

Model averaging procedure for varying-coefficient partially linear models with missing responses

This paper is concerned with model averaging procedure for varying-coefficient partially linear models with missing responses. The profile least-squares estimation process and inverse probability weighted method are employed to estimate regression coefficients of the partially restricted models, in which the propensity score is estimated by the covariate balancing propensity score method. The estimators of the linear parameters are shown to be asymptotically normal. Then we develop the focused information criterion, formulate the frequentist model averaging estimators and construct the corresponding confidence intervals. Some simulation studies are conducted to examine the finite sample performance of the proposed methods. We find that the covariate balancing propensity score improves the performance of the inverse probability weighted estimator. We also demonstrate the superiority of the proposed model averaging estimators over those of existing strategies in terms of mean squared error and coverage probability. Finally, our approach is further applied to a real data example.     

A comparison of confidence regions and designs in estimation of a ratio

A computer simulation is performed to compare confidence regions arising from Fie1ler's theorem and approximate large sample intervals in estimating the point of extremum in quadratic regression. In addition, two designs, one of which is motivated by optimal design theory, are compared. These comparisons are made by examining the confidence level and accuracy of the regions, as well as the concentration of the standard point estimator, in a variety of settings. The results have implications for the more general problem of estimating a ratio of linear combinations in the general linear model.     

Simultaneous Small Sample Inference for Linear Combinations of Generalized Linear Model Parameters

A method is proposed to construct simultaneous confidence intervals for multiple linear combinations of generalized linear model parameters, that uses a multivariate normal- or t-distribution together with the signed likelihood root statistic. In an application to a case study simultaneous confidence bands for logistic regression are calculated. A simulation study based on the example evaluation suggests superior performance compared to the common Wald-type approaches. The proposed methods are readily implemented in the R extension package mcprofile.     

Constructing narrower confidence intervals by inverting adaptive tests

We begin by describing how to find the limits of confidence intervals by using a few permutation tests of significance. Next, we demonstrate how the adaptive permutation test, which maintains its level of significance, produces confidence intervals that maintain their coverage probabilities. By inverting adaptive tests, adaptive confidence intervals can be found for any single parameter in a multiple regression model. These adaptive confidence intervals are often narrower than the traditional confidence intervals when the error distributions are long‐tailed or skewed. We show how much reduction in width can be achieved for the slopes in several multiple regression models and for the interaction effect in a two‐way design. An R function that can compute these adaptive confidence intervals is described and instructions are provided for its use with real data.     

Reducing Bias and Mean Squared Error Associated With Regression-Based Odds Ratio Estimators

Ratio estimators of effect are ordinarily obtained by exponentiating maximum-likelihood estimators (MLEs) of log-linear or logistic regression coefficients. These estimators can display marked positive finite-sample bias, however. We propose a simple correction that removes a substantial portion of the bias due to exponentiation. By combining this correction with bias correction on the log scale, we demonstrate that one achieves complete removal of second-order bias in odds ratio estimators in important special cases. We show how this approach extends to address bias in odds or risk ratio estimators in many common regression settings. We also propose a class of estimators that provide reduced mean bias and squared error, while allowing the investigator to control the risk of underestimating the true ratio parameter. We present simulation studies in which the proposed estimators are shown to exhibit considerable reduction in bias, variance, and mean squared error compared to MLEs. Bootstrapping provides further improvement, including narrower confidence intervals without sacrificing coverage.     

Nonlinear unbiased estimation in the linear regression model with nonnormal disturbances

In the application of the linear regression model there continues to be wide-spread use of the Least Squares Estimator (LSE) due to its theoretical optimality. For example, it is well known that the LSE is the best unbiased estimator under normality while it remains best linear unbiased estimator (BLUE) when the normality assumption is dropped. In this paper we extend an approach given in Knautz (1993) that allows improvement of the LSE in the context of nonnormal and nonsymmetric error distributions. It will be shown that there exist linear plus quadratic (LPQ) estimators, consisting of linear and quadratic terms in the dependent variable, which dominate the LS estimator, depending on second, third and fourth moments of the error distribution. A simulation study illustrates that this remains true if the moments have to be estimated from the data. Computation of confidence intervals using bootstrap methods reveal significant improvement compared with inference based on the LS especially for nonsymmetric distributions of the error term.     

Permutation methods in relative risk regression models

In this paper, we develop a weighted permutation (WP) method to construct confidence intervals for regression parameters in relative risk regression models. The WP method is a generalized permutation approach. It constructs a resampled history which mimics the observed history for individuals under study. Inference procedures are based on studentized score statistics that are insensitive to the forms of the relative risk function. This makes the WP method appealing in the general framework of the relative risk regression model. First-order accuracy of the WP method is established using counting process approach with a partial likelihood filtration. A simulation study indicates that the method typically improves accuracy over asymptotic confidence intervals.     

On confidence intervals for the among-group variance in the one-way random effects model with unequal error variances

Based on a quadratic form of the group means, we consider two different approaches to construct a confidence interval for the among-group variance in the one-way random effects model with unequal error variances. In one approach the limits of the interval are determined by solving non-linear equations whereas in the second approach the bounds are given explicitly. By using correction terms for convexity in both approaches, we improve the primary intervals and obtain intervals whose actual confidence coefficients are closer to the nominal confidence coefficient. By means of a simulation study, we show that an improved confidence interval from each approach can be recommended for the practical use.     

Empirical Likelihood Local Polynomial Regression Analysis of Clustered Data

Abstract. In this article, a naive empirical likelihood ratio is constructed for a non‐parametric regression model with clustered data, by combining the empirical likelihood method and local polynomial fitting. The maximum empirical likelihood estimates for the regression functions and their derivatives are obtained. The asymptotic distributions for the proposed ratio and estimators are established. A bias‐corrected empirical likelihood approach to inference for the parameters of interest is developed, and the residual‐adjusted empirical log‐likelihood ratio is shown to be asymptotically chi‐squared. These results can be used to construct a class of approximate pointwise confidence intervals and simultaneous bands for the regression functions and their derivatives. Owing to our bias correction for the empirical likelihood ratio, the accuracy of the obtained confidence region is not only improved, but also a data‐driven algorithm can be used for selecting an optimal bandwidth to estimate the regression functions and their derivatives. A simulation study is conducted to compare the empirical likelihood method with the normal approximation‐based method in terms of coverage accuracies and average widths of the confidence intervals/bands. An application of this method is illustrated using a real data set.     

Estimation and inference for generalized semi-varying coefficient models

This paper is concerned with the estimation and inference in generalized semi-varying coefficient models. An orthogonal projection local quasi-likelihood estimation is investigated, which can easily be used to estimate the model parametric and nonparametric parts. Then an empirical likelihood logarithmic approach to construct the confidence regions/intervals of the nonparametric parts is developed. Under some mild conditions, the asymptotic properties of the resulting estimators are studied explicitly, respectively. Some simulation studies are carried out to examine the finite sample performance of the proposed methods. Finally, the methodologies are illustrated by a real data set.     

Two-sample empirical likelihood method for difference between coefficients in linear regression model

The empirical likelihood method is proposed to construct the confidence regions for the difference in value between coefficients of two-sample linear regression model. Unlike existing empirical likelihood procedures for one-sample linear regression models, as the empirical likelihood ratio function is not concave, the usual maximum empirical likelihood estimation cannot be obtained directly. To overcome this problem, we propose to incorporate a natural and well-explained restriction into likelihood function and obtain a restricted empirical likelihood ratio statistic (RELR). It is shown that RELR has an asymptotic chi-squared distribution. Furthermore, to improve the coverage accuracy of the confidence regions, a Bartlett correction is applied. The effectiveness of the proposed approach is demonstrated by a simulation study.     

Bayesian Estimation of the Log–linear Exponential Regression Model with Censorship and Collinearity

In this work, a simulation study is conducted to evaluate the performance of Bayesian estimators for the log–linear exponential regression model under different levels of censoring and degrees of collinearity for two covariates. The diffuse normal, independent Student-t and multivariate Student-t distributions are considered as prior distributions and to draw from the posterior distributions, the Metropolis algorithm is implemented. Also, the results are compared with the maximum likelihood estimators in terms of the mean squared error, coverages and length of the credibility and confidence intervals.