Variance estimation with Chao's sampling scheme

We show that the Hájek (Ann. Math Statist. (1964) 1491) variance estimator can be used to estimate the variance of the Horvitz–Thompson estimator when the Chao sampling scheme (Chao, Biometrika 69 (1982) 653) is implemented. This estimator is simple and can be implemented with any statistical packages. We consider a numerical and an analytic method to show that this estimator can be used. A series of simulations supports our findings.     

Rate of convergence for asymptotic variance of the Horvitz–Thompson estimator

Drawing distinct units without replacement and with unequal probabilities from a population is a problem often considered in the literature (e.g. Hanif and Brewer, 1980, Int. Statist. Rev. 48, 317–355). In such a case, the sample mean is a biased estimator of the population mean. For this reason, we use the unbiased Horvitz–Thompson estimator (1951). In this work, we focus our interest on the variance of this estimator. The variance is cumbersome to compute because it requires the calculation of a large number of second-order inclusion probabilities. It would be helpful to use an approximation that does not need heavy calculations. The Hájek (1964) variance approximation provides this advantage as it is free of second-order inclusion probabilities. Hájek (1964) proved that this approximation is valid under restrictive conditions that are usually not fulfilled in practice. In this paper, we give more general conditions and we show that this approximation remains acceptable for most practical problems.     

A Class of Sampling Two Units with Probability Proportional to Size

A class of sampling two units without replacement with inclusion probability proportional to size is proposed in this article. Many different well known probability proportional to size sampling designs are special cases from this class. The first and second inclusion probabilities of this class satisfy important properties and provide a non-negative variance estimator of the Horvitz and Thompson estimator for the population total. Suitable choice for the first and second inclusion probabilities from this class can be used to reduce the variance estimator of the Horvitz and Thompson estimator. Comparisons between different proportional to size sampling designs through real data and artificial examples are given. Examples show that the minimum variance of the Horvitz and Thompson estimator obtained from the proposed design is not attainable for the most cases at any of the well known designs.     

How to Select Representative Samples

We give a formal definition of a representative sample, but roughly speaking, it is a scaled‐down version of the population, capturing its characteristics. New methods for selecting representative probability samples in the presence of auxiliary variables are introduced. Representative samples are needed for multipurpose surveys, when several target variables are of interest. Such samples also enable estimation of parameters in subspaces and improved estimation of target variable distributions. We describe how two recently proposed sampling designs can be used to produce representative samples. Both designs use distance between population units when producing a sample. We propose a distance function that can calculate distances between units in general auxiliary spaces. We also propose a variance estimator for the commonly used Horvitz–Thompson estimator. Real data as well as illustrative examples show that representative samples are obtained and that the variance of the Horvitz–Thompson estimator is reduced compared with simple random sampling.     

An extension of Horvitz–Thompson estimator used in adaptive cluster sampling to continuous universe

In this paper, an extension of Horvitz–Thompson estimator used in adaptive cluster sampling to continuous universe is developed. Main new results are presented in theorems. The primary notions of discrete population are transferred to continuous population. First and second order inclusion probabilities for networks are delivered. Horvitz–Thompson estimator for adaptive cluster sampling in continuous universe is constructed. The unbiasedness of the estimator is proven. Variance and unbiased variance estimator are delivered. Finally, the theory is illustrated with an example.     

Spatially Balanced Sampling of Continuous Populations

When sampling from a continuous population (or distribution), we often want a rather small sample due to some cost attached to processing the sample or to collecting information in the field. Moreover, a probability sample that allows for design‐based statistical inference is often desired. Given these requirements, we want to reduce the sampling variance of the Horvitz–Thompson estimator as much as possible. To achieve this, we introduce different approaches to using the local pivotal method for selecting well‐spread samples from multidimensional continuous populations. The results of a simulation study clearly indicate that we succeed in selecting spatially balanced samples and improve the efficiency of the Horvitz–Thompson estimator.     

A Simple Variance Estimator for Unequal Probability Sampling without Replacement

Survey sampling textbooks often refer to the Sen–Yates–Grundy variance estimator for use with without-replacement unequal probability designs. This estimator is rarely implemented because of the complexity of determining joint inclusion probabilities. In practice, the variance is usually estimated by simpler variance estimators such as the Hansen–Hurwitz with replacement variance estimator; which often leads to overestimation of the variance for large sampling fractions that are common in business surveys. We will consider an alternative estimator: the Hájek (1964 Hájek J 1981 Sampling from a Finite Population New York: Marcel Dekker  [Google Scholar]) variance estimator that depends on the first-order inclusion probabilities only and is usually more accurate than the Hansen–Hurwitz estimator. We review this estimator and show its practical value. We propose a simple alternative expression; which is as simple as the Hansen–Hurwitz estimator. We also show how the Hájek estimator can be easily implemented with standard statistical packages.     

Functional calibration estimation by the maximum entropy on the mean principle

We extend the problem of obtaining an estimator for the finite population mean parameter incorporating complete auxiliary information through calibration estimation in survey sampling under a functional data framework. The functional calibration sampling weights of the estimator are obtained by matching the calibration estimation problem with the maximum entropy on the mean – MEM – principle. In particular, the calibration estimation is viewed as an infinite-dimensional linear inverse problem following the structure of the MEM approach. We give a precise theoretical setting and estimate the functional calibration weights assuming, as prior measures, the centred Gaussian and compound Poisson random measures. Additionally, through a simple simulation study, we show that the proposed functional calibration estimator improves its accuracy compared with the Horvitz–Thompson one.     

Double bootstrap estimation of variance under systematic sampling with probability proportional to size

Variance estimation under systematic sampling with probability proportional to size is known to be a difficult problem. We attempt to tackle this problem by the bootstrap resampling method. It is shown that the usual way to bootstrap fails to give satisfactory variance estimates. As a remedy, we propose a double bootstrap method which is based on certain working models and involves two levels of resampling. Unlike existing methods which deal exclusively with the Horvitz–Thompson estimator, the double bootstrap method can be used to estimate the variance of any statistic. We illustrate this within the context of both mean and median estimation. Empirical results based on five natural populations are encouraging.     

Variance approximation under balanced sampling

A balanced sampling design has the interesting property that Horvitz–Thompson estimators of totals for a set of balancing variables are equal to the totals we want to estimate, therefore the variance of Horvitz–Thompson estimators of variables of interest are reduced in function of their correlations with the balancing variables. Since it is hard to derive an analytic expression for the joint inclusion probabilities, we derive a general approximation of variance based on a residual technique. This approximation is useful even in the particular case of unequal probability sampling with fixed sample size. Finally, a set of numerical studies with an original methodology allows to validate this approximation.     

On a sampling scheme with inclusion probability proportional to size

A sampling scheme for selection of a sample of two units with inclusion probability proportionalto size is suggested which provides a non–negative variance estimator of the variance of Horvitz–Thompson estimator. The suggested sampling scheme is shown to perform better than many of the existing unequal probability and inclusion probability proportional to size sampling Achemes for a number of natural populations.     

Estimating prediction error for complex samples

With a growing interest in using non-representative samples to train prediction models for numerous outcomes it is necessary to account for the sampling design that gives rise to the data in order to assess the generalized predictive utility of a proposed prediction rule. After learning a prediction rule based on a non-uniform sample, it is of interest to estimate the rule's error rate when applied to unobserved members of the population. Efron (1986) proposed a general class of covariance penalty inflated prediction error estimators that assume the available training data are representative of the target population for which the prediction rule is to be applied. We extend Efron's estimator to the complex sample context by incorporating Horvitz–Thompson sampling weights and show that it is consistent for the true generalization error rate when applied to the underlying superpopulation. The resulting Horvitz–Thompson–Efron estimator is equivalent to dAIC, a recent extension of Akaike's information criteria to survey sampling data, but is more widely applicable. The proposed methodology is assessed with simulations and is applied to models predicting renal function obtained from the large-scale National Health and Nutrition Examination Study survey. The Canadian Journal of Statistics 48: 204–221; 2020 © 2019 Statistical Society of Canada     

VARIANCE ESTIMATION IN TWO-PHASE SAMPLING

Two‐phase sampling is often used for estimating a population total or mean when the cost per unit of collecting auxiliary variables, x, is much smaller than the cost per unit of measuring a characteristic of interest, y. In the first phase, a large sample s₁ is drawn according to a specific sampling design p(s₁) , and auxiliary data x are observed for the units i∈s₁ . Given the first‐phase sample s₁ , a second‐phase sample s₂ is selected from s₁ according to a specified sampling design {p(s₂∣s₁) } , and (y, x) is observed for the units i∈s₂ . In some cases, the population totals of some components of x may also be known. Two‐phase sampling is used for stratification at the second phase or both phases and for regression estimation. Horvitz–Thompson‐type variance estimators are used for variance estimation. However, the Horvitz–Thompson ( Horvitz & Thompson, J. Amer. Statist. Assoc. 1952 ) variance estimator in uni‐phase sampling is known to be highly unstable and may take negative values when the units are selected with unequal probabilities. On the other hand, the Sen–Yates–Grundy variance estimator is relatively stable and non‐negative for several unequal probability sampling designs with fixed sample sizes. In this paper, we extend the Sen–Yates–Grundy ( Sen , J. Ind. Soc. Agric. Statist. 1953; Yates & Grundy , J. Roy. Statist. Soc. Ser. B 1953) variance estimator to two‐phase sampling, assuming fixed first‐phase sample size and fixed second‐phase sample size given the first‐phase sample. We apply the new variance estimators to two‐phase sampling designs with stratification at the second phase or both phases. We also develop Sen–Yates–Grundy‐type variance estimators of the two‐phase regression estimators that make use of the first‐phase auxiliary data and known population totals of some of the auxiliary variables.     

Efficient estimators for adaptive stratified sequential sampling

In stratified sampling, methods for the allocation of effort among strata usually rely on some measure of within-stratum variance. If we do not have enough information about these variances, adaptive allocation can be used. In adaptive allocation designs, surveys are conducted in two phases. Information from the first phase is used to allocate the remaining units among the strata in the second phase. Brown et al. [Adaptive two-stage sequential sampling, Popul. Ecol. 50 (2008), pp. 239–245] introduced an adaptive allocation sampling design – where the final sample size was random – and an unbiased estimator. Here, we derive an unbiased variance estimator for the design, and consider a related design where the final sample size is fixed. Having a fixed final sample size can make survey-planning easier. We introduce a biased Horvitz–Thompson type estimator and a biased sample mean type estimator for the sampling designs. We conduct two simulation studies on honey producers in Kurdistan and synthetic zirconium distribution in a region on the moon. Results show that the introduced estimators are more efficient than the available estimators for both variable and fixed sample size designs, and the conventional unbiased estimator of stratified simple random sampling design. In order to evaluate efficiencies of the introduced designs and their estimator furthermore, we first review some well-known adaptive allocation designs and compare their estimator with the introduced estimators. Simulation results show that the introduced estimators are more efficient than available estimators of these well-known adaptive allocation designs.     

Location-shifts for improved estimation under Midzuno–Sen sampling scheme

Simple location-shifts for the study or auxiliary character are proposed under Midzuno–Sen sampling from a finite population. These aim at improving the efficiency of the classical Horvitz–Thompson estimator or the unbiased ratio estimator of a population total. It is demonstrated that the choice of the translation parameters is flexible. A few methods for assessing these parameters are outlined. The gain in efficiency of estimation is illustrated.     

A Monte Carlo comparison of some variance estimators of the Horvitz–Thompson estimator

At the design and estimation stage of a survey, large survey organization often uses auxiliary information. This article discusses various procedures for improving variance estimation of the Horvitz–Thompson estimator of a finite population total with the aid of auxiliary information. To study the design-based properties of the proposed variance estimators relative to the standard one, a small scale Monte Carlo study is performed.     

Performance of distance sampling estimators: a simulation study for designs based on footpaths

ABSTRACT

Recently, distance sampling emerged as an advantageous technique to estimate the abundance of many animal populations, including ungulates. Its basic design involves the random selection of several samplers (transects or points) within the population range, and a Horvitz–Thompson-like estimator is then applied to estimate the population abundance while correcting for animal detectability. Ensuring even coverage probability is essential for subsequent inference on the population size, but it may not be achievable because of limited access to parts of the population range. Moreover, in several environmental conditions, a random selection of samplers may induce very high survey costs because it does not minimize the displacement time of the observer(s) between successive samplers. We thus tested whether two-stage designs – based on the random selection of points and then of nearby samplers – could be more cost-effective, for a given population size and when even area coverage cannot be guaranteed. Here, we further extend our analyses to assess the performance of two-stage designs under varying animal densities.     

A dual problem of calibration of design weights

In this note, a dual problem to the calibration of design weights of the Deville and Särndal [Calibration estimators in survey sampling, J. Amer. Statist. Assoc. 87 (1992), pp. 376–382] method has been considered. We conclude that the chi-squared distance between the design weights and the calibrated weights equals the square of the standardized Z-score obtained by the difference between the known population total of the auxiliary variable and its corresponding Horvitz and Thompson [A generalization of sampling without replacement from a finite universe, J. Amer. Statist. Assoc. 47 (1952), pp. 663–685] estimator divided by the sample standard deviation of the auxiliary variable to obtain the linear regression estimator in survey sampling.     

Estimation of contingency tables in complex survey sampling using probabilistic expert systems

In this paper we explore the possibility to use a particular class of models, known as probabilistic expert systems, to define two classes of estimators of a contingency table in case of stratified sampling designs. The two classes are characterized by the different role of the sampling design: in the first, the sampling design is treated as an additional variable; in the second, it is used only for estimation purposes by means of the survey weights. The bias/variance trade off of these estimators is analyzed and the consequences of model misspecification are illustrated. Furthermore, it is shown that the Horvitz–Thompson estimator belongs to both classes of estimators. It comes out that the Horvitz–Thompson estimator is almost always inefficient but robust. Monte Carlo simulations illustrate the efficiency of the proposed estimators.     

An Optimal Procedure for Sample Coordination Using Multiple Objective Functions and Nearest Proportional to IPPS Size Sampling Designs

The sample coordination problem involves maximization or minimization of overlap of sampling units in different/repeated surveys. Several optimal techniques using transportation theory, controlled rounding, and controlled selection have been suggested in literature to solve the sample coordination problem. In this article, using the multiple objective programming, we propose a method for sample coordination which facilitates variance estimation using the Horvitz–Thompson estimator. The proposed procedure can be applied to any two-sample surveys having identical universe and stratification. Some examples are discussed to demonstrate the utility of the proposed procedure.