Jackknife inference for heteroscedastic linear regression models

Inference on the regression parameters in a heteroscedastic linear regression model with replication is considered, using either the ordinary least-squares (OLS) or the weighted least-squares (WLS) estimator. A delete-group jackknife method is shown to produce consistent variance estimators irrespective of within-group correlations, unlike the delete-one jackknife variance estimators or those based on the customary δ-method assuming within-group independence. Finite-sample properties of the delete-group variance estimators and associated confidence intervals are also studied through simulation.     

A note on biases of jackknife estimators of third central moments

In this paper we study the biases of jackknife estimators of central third moments which play an important role in improving the accuracy of the normal approximation. It has been found in simulation studies that the jackknife estimator of the skewness coefficient, into which the jackknife variance and third moment estimators are substituted, have downward biases. For the jackknife variance estimators, their asymptotic properties are precisely studied and their biases are discussed theoretically, Here we study the biases of the jackknife estimators of the central third moments for U-statistics theoretically, The results show that the biases are not always downward.     

Estimating the variances of ratio estimates in complex sample surveys with two primary sampling units per stratum—a comparison of balanced replication and jackknife techniques

The balanced half-sample and jackknife variance estimation techniques are used to estimate the variance of the combined ratio estimate. An empirical sampling study is conducted using computer-generated populations to investigate the variance, bias and mean square error of these variance estimators and results are compared to theoretical results derived elsewhere for the linear case. Results indicate that either the balanced half-sample or jackknife method may be used effectively for estimating the variance of the combined ratio estimate.     

On the stability of the jackknife variance estimator in ratio estimation

The stability of a slightly modified version of the usual jackknife variance estimator is evaluated exactly in small samples under a suitable linear regression model and compared with that of two different linearization variance estimators. Depending on the degree of heteroscedasticity of the error variance in the model, the stability of the jackknife variance estimator is found to be somewhat comparable to that of one or the other of the linearization variance estimators under conditions especially favorable to ratio estimation (i.e., regression approximately through the origin with a relatively small coefficient of variation in the x population). When these conditions do not hold, however, the jackknife variance estimator is found to be less stable than either of the linearization variance estimators.     

Jackknife empirical likelihood for the error variance in linear models

Variance estimation is a fundamental yet important problem in statistical modelling. In this paper, we propose jackknife empirical likelihood (JEL) methods for the error variance in a linear regression model. We prove that the JEL ratio converges to the standard chi-squared distribution. The asymptotic chi-squared properties for the adjusted JEL and extended JEL estimators are also established. Extensive simulation studies to compare the new JEL methods with the standard method in terms of coverage probability and interval length are conducted, and the simulation results show that our proposed JEL methods perform better than the standard method. We also illustrate the proposed methods using two real data sets.     

On Mean Squared Prediction Error Estimation in Small Area Estimation Problems

For a general linear mixed normal model, a new linearized weighted jackknife method is proposed to estimate the mean squared prediction error (MSPE) of an empirical best linear unbiased predictor (EBLUP) of a general mixed effect. Different MSPE estimators are compared using a Monte Carlo simulation study.     

Weighted empirical adaptive variance estimators for correlated data regression

Estimating equations based on marginal generalized linear models are useful for regression modelling of correlated data, but inference and testing require reliable estimates of standard errors. We introduce a class of variance estimators based on the weighted empirical variance of the estimating functions and show that an adaptive choice of weights allows reliable estimation both asymptotically and by simulation in finite samples. Connections with previous bootstrap and jackknife methods are explored. The effect of reliable variance estimation is illustrated in data on health effects of air pollution in King County, Washington.     

Invited Discussion Paper Resampling Methods in Sample Surveys

This article reviews the applications of three resampling methods, the jackknife, the balanced repeated replication, and the bootstrap, in sample surveys. The sampling design under consideration is a stratified multistage sampling design. We discuss the implementation of the resampling methods; for example, the construction of balanced repeated replications and approximated balanced repeated replication estimators; four modified bootstrap algorithms to generate bootstrap samples; and three different ways of applying the resampling methods in the presence of imputed missing values. Asymptotic properties of the resampling estimators are discussed for two types of important survey estimators, functions of weighted averages and sample quantiles.     

Jackknife Estimation for Smooth Functions of the Parametric Component in Partially Linear Regression Models

Abstract

It is known that due to the existence of the nonparametric component, the usual estimators for the parametric component or its function in partially linear regression models are biased. Sometimes this bias is severe. To reduce the bias, we propose two jackknife estimators and compare them with the naive estimator. All three estimators are shown to be asymptotically equivalent and asymptotically normally distributed under some regularity conditions. However, through simulation we demonstrate that the jackknife estimators perform better than the naive estimator in terms of bias when the sample size is small to moderate. To make our results more useful, we also construct consistent estimators of the asymptotic variance, which are robust against heterogeneity of the error variances.     

JACKKNIFING THE GENERAL LINEAR MODEL

The aim of this paper is to investigate the problems of estimating a smooth function of the parameters in a general linear model and to clarify some of the points raised by Hinkley (1977) in connection with this problem. An example of the type of problem at hand is that of estimating the maximum (or minimum) mean value in a quadratic regression model. The estimator based on the least squares estimator of the parameters in the linear model is compared to the jackknife estimator and the weighted jackknife estimator proposed by Hinkley (1977). The asymptotic properties of the estimators are examined and their small sample properties are compared through simulation studies.     

Efficient computation of the performance of bootstrap and jackknife estimators of the variance of L-statistics

Calculation of the bootstrap and the jackknife estimators of the variance of a statistic often relies on approximation techniques because the exact values are difficult if not impossible to obtain analytically. For the special case where the statistic is a linear combination of order statistics we propose to calculate the exact values combinatorically, thus completely eliminating the second-stage simulation error.     

Bias in Small-Sample Inference With Count-Data Models

Abstract

Both Poisson and negative binomial regression can provide quasi-likelihood estimates for coefficients in exponential-mean models that are consistent in the presence of distributional misspecification. It has generally been recommended, however, that inference be carried out using asymptotically robust estimators for the parameter covariance matrix. As with linear models, such robust inference tends to lead to over-rejection of null hypotheses in small samples. Alternative methods for estimating coefficient estimator variances are considered. No one approach seems to remove all test bias, but the results do suggest that the use of the jackknife with Poisson regression tends to be least biased for inference.     

Doubly robust point and variance estimation in the presence of imputed survey data

Imputation is often used in surveys to treat item nonresponse. It is well known that treating the imputed values as observed values may lead to substantial underestimation of the variance of the point estimators. To overcome the problem, a number of variance estimation methods have been proposed in the literature, including resampling methods such as the jackknife and the bootstrap. In this paper, we consider the problem of doubly robust inference in the presence of imputed survey data. In the doubly robust literature, point estimation has been the main focus. In this paper, using the reverse framework for variance estimation, we derive doubly robust linearization variance estimators in the case of deterministic and random regression imputation within imputation classes. Also, we study the properties of several jackknife variance estimators under both negligible and nonnegligible sampling fractions. A limited simulation study investigates the performance of various variance estimators in terms of relative bias and relative stability. Finally, the asymptotic normality of imputed estimators is established for stratified multistage designs under both deterministic and random regression imputation. The Canadian Journal of Statistics 40: 259–281; 2012 © 2012 Statistical Society of Canada     

Second-order variance estimation in poststratified two-stage sampling

The linearization or Taylor series variance estimator and jackknife linearization variance estimator are popular for poststratified point estimators. In this note we propose a simple second-order linearization variance estimator for the poststratified estimator of the population total in two-stage sampling, using the second-order Taylor series expansion. We investigate the properties of the proposed variance estimator and its modified version and their empirical performance through some simulation studies in comparison to the standard and jackknife linearization variance estimators. Simulation studies are carried out on both artificially generated data and real data.     

Mem squared error estimation in finite populations under nonlinear models

Four estimators of the prediction mean squared error (MSB) of an estimated finite population total for a zero-one characteristic are examined. The characteristic associated with each population unit is modeled as the realization of a Bernoulli random variable whose expected value is a nonlinear function of a parameter vector and a set of known auxiliary variables. To compare the estimators, a simulation study is conducted using a population of hospitals. The MSB estimator Implied by the form of the assumed model underestimates the mean squared error in each of the cases studied and produces confidence lntervals with less than the nominal coverage probabilities. Of the three alternative MSE estimators presented, a linear approximation to the jackknife produces the best results and improves upon the model-specific estimator.     

MODEL-BASED VARIANCE ESTIMATION IN SURVEYS WITH STRATIFIED CLUSTERED DESIGN

A model-based method for estimating the sampling variances of estimators of (sub-)population means, proportions, quantiles, and regression parameters in surveys with stratified clustered design is described and applied to a survey of US secondary education. The method is compared with the jackknife by a simulation study. The model-based estimators of the sampling variances have much smaller mean squared errors than their jackknife counterparts. In addition, they can be improved by incorporating information about the unknown parameters (variances) from external sources. A regression-based smoothing method for estimating the sampling variances of the estimators for a large number of subpopulation means is proposed. Such smoothing may be invaluable when subpopulations are represented in the sample by only few subjects.     

A jackknife variance estimator for unequal probability sampling

Summary.  The jackknife method is often used for variance estimation in sample surveys but has only been developed for a limited class of sampling designs. We propose a jackknife variance estimator which is defined for any without-replacement unequal probability sampling design. We demonstrate design consistency of this estimator for a broad class of point estimators. A Monte Carlo study shows how the proposed estimator may improve on existing estimators.     

Evaluating the Effective Degrees of Freedom of the Delete-a-Group Jackknife

The delete-a-group jackknife is sometimes used when estimating the variances of statistics based on a large sample. We investigate heavily poststratified estimators for a population mean and a simple regression coefficient, where both full-sample and domain estimates are of interest. The delete-a-group (DAG) jackknife employing 30, 60, and 100 replicates is found to be highly unstable, even for large sample sizes. The empirical degrees of freedom of these DAG jackknives are usually much less than their nominal degrees of freedom. This analysis calls into question whether coverage intervals derived from replication-based variance estimators can be trusted for highly calibrated estimates.     

SMALL AREA ESTIMATION USING SURVEY WEIGHTS WITH FUNCTIONAL MEASUREMENT ERROR IN THE COVARIATE

Nested error linear regression models using survey weights have been studied in small area estimation to obtain efficient model‐based and design‐consistent estimators of small area means. The covariates in these nested error linear regression models are not subject to measurement errors. In practical applications, however, there are many situations in which the covariates are subject to measurement errors. In this paper, we develop a nested error linear regression model with an area‐level covariate subject to functional measurement error. In particular, we propose a pseudo‐empirical Bayes (PEB) predictor to estimate small area means. This predictor borrows strength across areas through the model and makes use of the survey weights to preserve the design consistency as the area sample size increases. We also employ a jackknife method to estimate the mean squared prediction error (MSPE) of the PEB predictor. Finally, we report the results of a simulation study on the performance of our PEB predictor and associated jackknife MSPE estimator.     

A BETA-BINOMIAL MODEL FOR ESTIMATING THE SIZE OF A HETEROGENEOUS POPULATION

This paper compares the properties of various estimators for a beta‐binomial model for estimating the size of a heterogeneous population. It is found that maximum likelihood and conditional maximum likelihood estimators perform well for a large population with a large capture proportion. The jackknife and the sample coverage estimators are biased for low capture probabilities. The performance of the martingale estimator is satisfactory, but it requires full capture histories. The Gibbs sampler and Metropolis‐Hastings algorithm provide reasonable posterior estimates for informative priors.