Penalized calibration in survey sampling: Design-based estimation assisted by mixed models

Calibration techniques in survey sampling, such as generalized regression estimation (GREG), were formalized in the 1990s to produce efficient estimators of linear combinations of study variables, such as totals or means. They implicitly lie on the assumption of a linear regression model between the variable of interest and some auxiliary variables in order to yield estimates with lower variance if the model is true and remaining approximately design-unbiased even if the model does not hold. We propose a new class of model-assisted estimators obtained by releasing a few calibration constraints and replacing them with a penalty term. This penalization is added to the distance criterion to minimize. By introducing the concept of penalized calibration, combining usual calibration and this ‘relaxed’ calibration, we are able to adjust the weight given to the available auxiliary information. We obtain a more flexible estimation procedure giving better estimates particularly when the auxiliary information is overly abundant or not fully appropriate to be completely used. Such an approach can also be seen as a design-based alternative to the estimation procedures based on the more general class of mixed models, presenting new prospects in some scopes of application such as inference on small domains.     

Asymptotic results for estimation and testing variances in regression models

Usually the variance of independent observations resulting from a linear or a nonlinear relationship is estimated by the Least-Squares residual estimator. In this paper its asymptotic properties are investigated. Further the asymptotic behaviour of tests for one-sided hypotheses on the variance is studied. The paper splits into two parts, the first one concerned with linear and the second one with nonlinear models.     

Shrinkage estimation for linear regression with ARMA errors

In this paper, we extend the modified lasso of Wang et al. (2007) to the linear regression model with autoregressive moving average (ARMA) errors. Such an extension is far from trivial because new devices need to be called for to establish the asymptotics due to the existence of the moving average component. A shrinkage procedure is proposed to simultaneously estimate the parameters and select the informative variables in the regression, autoregressive, and moving average components. We show that the resulting estimator is consistent in both parameter estimation and variable selection, and enjoys the oracle properties. To overcome the complexity in numerical computation caused by the existence of the moving average component, we propose a procedure based on a least squares approximation to implement estimation. The ordinary least squares formulation with the use of the modified lasso makes the computation very efficient. Simulation studies are conducted to evaluate the finite sample performance of the procedure. An empirical example of ground-level ozone is also provided.     

An improved and efficient biased estimation technique in logistic regression model

Abstract

In this article, we propose a new improved and efficient biased estimation method which is a modified restricted Liu-type estimator satisfying some sub-space linear restrictions in the binary logistic regression model. We study the properties of the new estimator under the mean squared error matrix criterion and our results show that under certain conditions the new estimator is superior to some other estimators. Moreover, a Monte Carlo simulation study is conducted to show the performance of the new estimator in the simulated mean squared error and predictive median squared errors sense. Finally, a real application is considered.     

Complete consistency for the estimator of nonparametric regression model based on martingale difference errors

Abstract

In this paper, we study the complete consistency for the estimator of nonparametric regression model based on martingale difference errors, and obtain the convergence rates of the complete consistency by using the inequalities for martingale difference sequence. Finally, some simulations are illustrated.     

Implications of survey design for generalized regression estimation of linear functions

The paper presents a general randomization theory approach to point and interval estimation of Q linear functions T_q = Σ^N₁c_kqY_k(q = 1,…,Q), where Y₁,…,Y_N are values of a variable of interest Y in a finite population. Such linear functions include population and domain means and totals, population regression coefficients, etc. We assume that some auxiliary information can be exploited. This suggests the generalized regression technique based on the fit of a linear model, whereby is created approximately design unbiased estimators T?_q. The paper focuses on estimation of the variance-covariance matrix of the T?_q for single stage and two stage designs. Two techniques based on Taylor expansions are compared. Results of Monte-Carlo experiments (not reported here) show that the coverage properties are good of normal-theory confidence intervals flowing from one or the other variance estimate.     

The relationship between the improvement on the point estimation and the improvement on the interval estimation for the disturbance variance in a linear regression model

In this paper the relationship between the improvement on the point estimation and the improvement on the interval estimation for the disturbance variance in a linear regression model is discussed It is shown that substituting the Stein-type estimatoi for the usual estimatoi in the confidence interval leads to the improvement on the interval estimation The equal-tailed and the shoitest unbiased intervals are dealt with Some appealing relationship is also found in the unbiased case.     

Error variance estimation via least squares for small sample nonparametric regression

In this paper we explore statistical properties of some difference-based approaches to estimate an error variance for small sample based on nonparametric regression which satisfies Lipschitz condition. Our study is motivated by Tong and Wang (2005), who estimated error variance using a least squares approach. They considered the error variance as the intercept in a simple linear regression which was obtained from the expectation of their lag-k Rice estimator. Their variance estimators are highly dependent on the setting of a regressor and weight of their simple linear regression. Although this regressor and weight can be varied based on the characteristic of an unknown nonparametric mean function, Tong and Wang (2005) have used a fixed regressor and weight in a large sample and gave no indication of how to determine the regressor and the weight. In this paper, we propose a new approach via local quadratic approximation to determine this regressor and weight. Using our proposed regressor and weight, we estimate the error variance as the intercept of simple linear regression using both ordinary least squares and weighted least squares. Our approach applies to both small and large samples, while most existing difference-based methods are appropriate solely for large samples. We compare the performance of our approach with other existing approaches using extensive simulation study. The advantage of our approach is demonstrated using a real data set.     

A two-way classification of regression estimation strategies in probability sampling

This paper examines strategies for estimating the mean of a finite population in the following situation: A linear regression model is assumed to describe the population scatter. Various estimators β for the vector of regression parameters β are considered. Several ways of transforming each estimator β into a model-based estimator for the population mean are considered. Some estimators constructed in this way become sensitive to correctness of the assumed model. The estimators favoured in this paper are the ones in which the observations are weighted to reflect the sampling design, so that asymptotic design unbiasedness is achieved. For these estimators, the randomization distribution gives protection against model breakdown.     

Robust estimation and selection for single-index regression model

In this paper, we consider a single-index regression model for which we propose a robust estimation procedure for the model parameters and an efficient variable selection of relevant predictors. The proposed method is known as the penalized generalized signed-rank procedure. Asymptotic properties of the proposed estimator are established under mild regularity conditions. Extensive Monte Carlo simulation experiments are carried out to study the finite sample performance of the proposed approach. The simulation results demonstrate that the proposed method dominates many of the existing ones in terms of robustness of estimation and efficiency of variable selection. Finally, a real data example is given to illustrate the method.     

A new stochastic mixed Liu estimator in linear regression model

Abstract

To overcome multicollinearity, a new stochastic mixed Liu estimator is presented and its efficiency is considered. We also compare the proposed estimators in the sense of matrix mean squared error criteria. Finally a numerical example and a simulation study are given to show the performance of the estimators.     

On generalised estimating equations for vector regression

Generalised estimating equations (GEE) for regression problems with vector‐valued responses are examined. When the response vectors are of mixed type (e.g. continuous–binary response pairs), the GEE approach is a semiparametric alternative to full‐likelihood copula methods, and is closely related to Prentice & Zhao's mean‐covariance estimation equations approach. When the response vectors are of the same type (e.g. measurements on left and right eyes), the GEE approach can be viewed as a ‘plug‐in’ to existing methods, such as the vglm function from the state‐of‐the‐art VGAM package in R. In either scenario, the GEE approach offers asymptotically correct inferences on model parameters regardless of whether the working variance–covariance model is correctly or incorrectly specified. The finite‐sample performance of the method is assessed using simulation studies based on a burn injury dataset and a sorbinil eye trial dataset. The method is applied to data analysis examples using the same two datasets, as well as to a trivariate binary dataset on three plant species in the Hunua ranges of Auckland.     

Variance estimation of DEE and regression estimator when a first-phase cluster sample is restratified

We consider a variance estimation when a stratified single stage cluster sample is selected in the first phase and a stratified simple random element sample is selected in the second phase. We propose explicit formulas of (asymptotically), we propose explicit formulas of (asymptotically) unbiased variance estimators for the double expansion estimator and regression estimator. We perform a small simulation study to investigate the performance of the proposed variance estimators. In our simulation study, the proposed variance estimator showed better or comparable performance to the Jackknife variance estimator. We also extend the results to a two-phase sampling design in which a stratified pps with replacement cluster sample is selected in the first phase.     

A note on non-negative mean square error estimation of regression estimators in randomized response surveys

Generalized regression estimators are considered for the survey population total of a quantitative sensitive variable based on randomized responses. Formulae are presented for ‘non-negative’ estimators of approximate mean square errors of these biased estimators when population and sample sizes are large.     

Robust estimation of variance components

New robust estimates for variance components are introduced. Two simple models are considered: the balanced one-way classification model with a random factor and the balanced mixed model with one random factor and one fixed factor. However, the method of estimation proposed can be extended to more complex models. The new method of estimation we propose is based on the relationship between the variance components and the coefficients of the least-mean-squared-error predictor between two observations of the same group. This relationship enables us to transform the problem of estimating the variance components into the problem of estimating the coefficients of a simple linear regression model. The variance-component estimators derived from the least-squares regression estimates are shown to coincide with the maximum-likelihood estimates. Robust estimates of the variance components can be obtained by replacing the least-squares estimates by robust regression estimates. In particular, a Monte Carlo study shows that for outlier-contaminated normal samples, the estimates of variance components derived from GM regression estimates and the derived test outperform other robust procedures.     

On the coefficient of determination for mixed regression models

For mixed regression models, we define a variance decomposition including three terms, explained individual variance, unexplained individual variance and noise variance. In contrast to traditional variance decomposition, it is thus the unexplained  , not the explained, variance that is split. It gives rise to a coefficient of individual determination (CID) defined as the estimated fraction of explained individual variance. We argue that in many applications CID is a valuable complement to R²$R^2$, since it excludes noise variance (which can never be explained) and thus has one as a natural upper bound.     

Variance estimation for the finite population distribution function with complete auxiliary information

The authors develop jackknife and analytical variance estimators for the estimator of Chambers & Dunstan (1986) and Rao, Kovar & Mantel (1990) of the finite population distribution function, using complete auxiliary information. They also describe the associated model and show the design consistency of the variance estimators, whose small‐sample performance is examined through a limited simulation study. They highlight the operational advantages of the jackknife in the model‐based setting of Chambers & Dunstan (1986) and its better conditional performance in the design‐based setting of Rao, Kovar & Mantel (1990).