Small area estimation of mean price of habitation transaction using time-series and cross-sectional area-level models

In this paper, a new small domain estimator for area-level data is proposed. The proposed estimator is driven by a real problem of estimating the mean price of habitation transaction at a regional level in a European country, using data collected from a longitudinal survey conducted by a national statistical office. At the desired level of inference, it is not possible to provide accurate direct estimates because the sample sizes in these domains are very small. An area-level model with a heterogeneous covariance structure of random effects assists the proposed combined estimator. This model is an extension of a model due to Fay and Herriot [5], but it integrates information across domains and over several periods of time. In addition, a modified method of estimation of variance components for time-series and cross-sectional area-level models is proposed by including the design weights. A Monte Carlo simulation, based on real data, is conducted to investigate the performance of the proposed estimators in comparison with other estimators frequently used in small area estimation problems. In particular, we compare the performance of these estimators with the estimator based on the Rao–Yu model [23]. The simulation study also accesses the performance of the modified variance component estimators in comparison with the traditional ANOVA method. Simulation results show that the estimators proposed perform better than the other estimators in terms of both precision and bias.     

Exploring spatial dependence in area-level random effect model for disaggregate-level crop yield estimation

This paper describes an application of small area estimation (SAE) techniques under area-level spatial random effect models when only area (or district or aggregated) level data are available. In particular, the SAE approach is applied to produce district-level model-based estimates of crop yield for paddy in the state of Uttar Pradesh in India using the data on crop-cutting experiments supervised under the Improvement of Crop Statistics scheme and the secondary data from the Population Census. The diagnostic measures are illustrated to examine the model assumptions as well as reliability and validity of the generated model-based small area estimates. The results show a considerable gain in precision in model-based estimates produced applying SAE. Furthermore, the model-based estimates obtained by exploiting spatial information are more efficient than the one obtained by ignoring this information. However, both of these model-based estimates are more efficient than the direct survey estimate. In many districts, there is no survey data and therefore it is not possible to produce direct survey estimates for these districts. The model-based estimates generated using SAE are still reliable for such districts. These estimates produced by using SAE will provide invaluable information to policy-analysts and decision-makers.     

A Small Area Predictor under Area-Level Linear Mixed Models with Restrictions

A calibrated small area predictor based on an area-level linear mixed model with restrictions is proposed. It is showed that such restricted predictor, which guarantees the concordance between the small area estimates and a known estimate at the aggregate level, is the best linear unbiased predictor. The mean squared prediction error of the calibrated predictor is discussed. Further, a restricted predictor under a particular time-series and cross-sectional model is presented. Within a simulation study based on real data collected from a longitudinal survey conducted by a national statistical office, the proposed estimator is compared with other competitive restricted and non-restricted predictors.     

问卷分割设计的模拟研究——小域估计的一种应用

针对长问卷存在无回答率和回答负担,从而导致统计调查精度降低的问题,采用问卷分割法解决该问题,并且通过小域估计的方法进行参数估计。模拟研究表明,利用小域估计方法对分割问卷进行参数估计显然优于用多重插补法进行参数估计。研究结果表明,运用小域估计方法对分割问卷进行参数估计,能显著提高统计调查的精度。     

Some recent developments for regression analysis of multivariate failure time data

Cox's seminal 1972 paper on regression methods for possibly censored failure time data popularized the use of time to an event as a primary response in prospective studies. But one key assumption of this and other regression methods is that observations are independent of one another. In many problems, failure times are clustered into small groups where outcomes within a group are correlated. Examples include failure times for two eyes from one person or for members of the same family.This paper presents a survey of models for multivariate failure time data. Two distinct classes of models are considered: frailty and marginal models. In a frailty model, the correlation is assumed to derive from latent variables (frailties) common to observations from the same cluster. Regression models are formulated for the conditional failure time distribution given the frailties. Alternatively, marginal models describe the marginal failure time distribution of each response while separately modelling the association among responses from the same cluster.We focus on recent extensions of the proportional hazards model for multivariate failure time data. Model formulation, parameter interpretation and estimation procedures are considered.     

The forward search: Theory and data analysis

The Forward Search is a powerful general method, incorporating flexible data-driven trimming, for the detection of outliers and unsuspected structure in data and so for building robust models. Starting from small subsets of data, observations that are close to the fitted model are added to the observations used in parameter estimation. As this subset grows we monitor parameter estimates, test statistics and measures of fit such as residuals. The paper surveys theoretical development in work on the Forward Search over the last decade. The main illustration is a regression example with 330 observations and 9 potential explanatory variables. Mention is also made of procedures for multivariate data, including clustering, time series analysis and fraud detection.     

遥感辅助的农作物种植面积小域估计方法研究

遥感影像是大数据的一种,利用遥感对农作物播种面积进行估算常采用回归估计量或校准估计量,通常都需要将地面样本数据与遥感分类信息相结合。但对于大多数回归估计量,对省级总体的农作物面积估算只能满足对省级总体的精度要求而不能分解到更小区域,比如县和乡级。本文利用黑龙江省2011年的地面实测样本数据结合遥感分类结果,构建了单元层次的多响应变量的多元回归形式的小域模型,并将小域效应设定为固定形式。这样基于回归估计方法,既可以估算分县的主要作物播种面积,也可以使得各县播种面积估计结果相加就等于回归模型含义下的省级总体的总量估计。对黑龙江省玉米、水稻、大豆分县小域估计结果的精度评价（变异系数C.V）,平均而言均可以满足县级精度要求。本文的结果表明小域估计方法在解决省级总体对全省和分县的农作物种植面积多级估算问题中具有很好的应用。     

MODEL‐BASED DIRECT ESTIMATION OF SMALL‐AREA DISTRIBUTIONS

Much of the small‐area estimation literature focuses on population totals and means. However, users of survey data are often interested in the finite‐population distribution of a survey variable and in the measures (e.g. medians, quartiles, percentiles) that characterize the shape of this distribution at the small‐area level. In this paper we propose a model‐based direct estimator (MBDE, Chandra and Chambers) of the small‐area distribution function. The MBDE is defined as a weighted sum of sample data from the area of interest, with weights derived from the calibrated spline‐based estimate of the finite‐population distribution function introduced by Harms and Duchesne, under an appropriately specified regression model with random area effects. We also discuss the mean squared error estimation of the MBDE. Monte Carlo simulations based on both simulated and real data sets show that the proposed MBDE and its associated mean squared error estimator perform well when compared with alternative estimators of the area‐specific finite‐population distribution function.     

Empirical best linear unbiased and empirical Bayes prediction in multivariate small area estimation

Small area estimation plays a prominent role in survey sampling due to a growing demand for reliable small area estimates from both public and private sectors. Popularity of model-based inference is increasing in survey sampling, particularly, in small area estimation. The estimates of the small area parameters can profitably ‘borrow strength’ from data on related multiple characteristics and/or auxiliary variables from other neighboring areas through appropriate models. Fay (1987, Small Area Statistics, Wiley, New York, pp. 91–102) proposed multivariate regression for small area estimation of multiple characteristics. The success of this modeling rests essentially on the strength of correlation of these dependent variables. To estimate small area mean vectors of multiple characteristics, multivariate modeling has been proposed in the literature via a multivariate variance components model. We use this approach to empirical best linear unbiased and empirical Bayes prediction of small area mean vectors. We use data from Battese et al. (1988, J. Amer. Statist. Assoc. 83, 28 –36) to conduct a simulation which shows that the multivariate approach may achieve substantial improvement over the usual univariate approach.     

An Applied Comparison of Area-Level Linear Mixed Models in Small Area Estimation

This article reviews four area-level linear mixed models that borrow strength by exploiting the possible correlation among the neighboring areas or/and past time periods. Its main goal is to study if there are efficiency gains when a spatial dependence or/and a temporal autocorrelation among random-area effects are included into the models. The Fay–Herriot estimator is used as benchmark. A design-based simulation study based on real data collected from a longitudinal survey conducted by a statistical office is presented. Our results show that models that explore both spatial and chronological association considerably improve the efficiency of small area estimates.     

Two-stage hierarchical modeling for analysis of subpopulations in conditional distributions

In this work, we develop modeling and estimation approach for the analysis of cross-sectional clustered data with multimodal conditional distributions where the main interest is in analysis of subpopulations. It is proposed to model such data in a hierarchical model with conditional distributions viewed as finite mixtures of normal components. With a large number of observations in the lowest level clusters, a two-stage estimation approach is used. In the first stage, the normal mixture parameters in each lowest level cluster are estimated using robust methods. Robust alternatives to the maximum likelihood estimation are used to provide stable results even for data with conditional distributions such that their components may not quite meet normality assumptions. Then the lowest level cluster-specific means and standard deviations are modeled in a mixed effects model in the second stage. A small simulation study was conducted to compare performance of finite normal mixture population parameter estimates based on robust and maximum likelihood estimation in stage 1. The proposed modeling approach is illustrated through the analysis of mice tendon fibril diameters data. Analyses results address genotype differences between corresponding components in the mixtures and demonstrate advantages of robust estimation in stage 1.     

Local model uncertainty and incomplete-data bias (with discussion)

Summary.  Problems of the analysis of data with incomplete observations are all too familiar in statistics. They are doubly difficult if we are also uncertain about the choice of model. We propose a general formulation for the discussion of such problems and develop approximations to the resulting bias of maximum likelihood estimates on the assumption that model departures are small. Loss of efficiency in parameter estimation due to incompleteness in the data has a dual interpretation: the increase in variance when an assumed model is correct; the bias in estimation when the model is incorrect. Examples include non-ignorable missing data, hidden confounders in observational studies and publication bias in meta-analysis. Doubling variances before calculating confidence intervals or test statistics is suggested as a crude way of addressing the possibility of undetectably small departures from the model. The problem of assessing the risk of lung cancer from passive smoking is used as a motivating example.     

基于分层模型的缺失数据插补方法研究

大规模抽样调查多采用复杂抽样设计,得到具有分层嵌套结构的调查数据集,其中不可避免会遇到数据缺失问题,针对分层结构含缺失数据集的插补策略目前鲜有研究。本文将Gibbs算法应用到分层含缺失数据集的多重插补过程中,分别研究了固定效应模型插补法和随机效应模型插补法,进而通过理论推导和数值模拟,在不同组内相关系数、群组规模、数据缺失比例等情形下,从参数估计结果的无偏性和有效性两方面,比较不同方法的插补效果,给出插补模型的选择建议。研究结果表明,采用随机效应模型作为插补模型时,得到的参数估计结果更准确,而固定效应模型作为插补模型操作相对简便,在数据缺失比例较小、组内相关系数较大、群组规模较大等情形下,可以采用固定效应插补模型,否则建议采用随机效应插补模型。     

Small area estimates for cross-classifications

Summary.  We develop a class of log-linear structural models that is suited to estimation of small area cross-classified counts based on survey data. This allows us to account for various associ- ation structures within the data and includes as a special case the restricted log-linear model underlying structure preserving estimation. The effect of survey design can be incorporated into estimation through the specification of an unbiased direct estimator and its associated covariance structure. We illustrate our approach by applying it to estimation of small area labour force characteristics in Norway.     

Evaluation of a Frequentist Hierarchical Model to Estimate Prevalence When Sampling from a Large Geographic Area Using Pool Screening

We present a frequentist Bernoulli-Beta hierarchical model to relax the constant prevalence assumption underlying the traditional prevalence estimation approach based on pooled data. This assumption is called into question when sampling from a large geographic area. Pool screening is a method that combines individual items into pools. Each pool will either test positive (at least one of the items is positive) or negative (all items are negative). Pool screening is commonly applied to the study of tropical diseases where pools consist of vectors (e.g., black flies) that can transmit the disease. The goal is to estimate the proportion of infected vectors.

Intermediate estimators (model parameters) and estimators of ultimate interest (pertaining to prevalence) are evaluated by standard measures of merit, such as bias, variance, and mean squared error making extensive use of expansions. Using the hierarchical model an investigator can determine the probability of the prevalence being below a pre-specified threshold value, a value at which no reemergence of the disease is expected. An investigation into the least biased choice of the α parameter in the Beta (α, β) prevalence distribution leads to the choice of α = 1.     

Properties of the beta regression model for small area estimation of proportions and application to estimation of poverty rates

Abstract

Linear mixed effects models have been popular in small area estimation problems for modeling survey data when the sample size in one or more areas is too small for reliable inference. However, when the data are restricted to a bounded interval, the linear model may be inappropriate, particularly if the data are near the boundary. Nonlinear sampling models are becoming increasingly popular for small area estimation problems when the normal model is inadequate. This paper studies the use of a beta distribution as an alternative to the normal distribution as a sampling model for survey estimates of proportions which take values in (0, 1). Inference for small area proportions based on the posterior distribution of a beta regression model ensures that point estimates and credible intervals take values in (0, 1). Properties of a hierarchical Bayesian small area model with a beta sampling distribution and logistic link function are presented and compared to those of the linear mixed effect model. Propriety of the posterior distribution using certain noninformative priors is shown, and behavior of the posterior mean as a function of the sampling variance and the model variance is described. An example using 2010 Small Area Income and Poverty Estimates (SAIPE) data is given, and a numerical example studying small sample properties of the model is presented.     

Small area estimation of proportions in business surveys

Binary data are often of interest in business surveys, particularly when the aim is to characterize grouping in the businesses making up the survey population. When small area estimates are required for such binary data, use of standard estimation methods based on linear mixed models (LMMs) becomes problematic. We explore two model-based techniques of small area estimation for small area proportions, the empirical best predictor (EBP) under a generalized linear mixed model and the model-based direct estimator (MBDE) under a population-level LMM. Our empirical results show that both the MBDE and the EBP perform well. The EBP is a computationally intensive method, whereas the MBDE is easy to implement. In case of model misspecification, the MBDE also appears to be more robust. The mean-squared error (MSE) estimation of MBDE is simple and straightforward, which is in contrast to the complicated MSE estimation for the EBP.     

Spatial generalized linear mixed models in small area estimation

In survey sampling, policy decisions regarding the allocation of resources to sub‐groups of a population depend on reliable predictors of their underlying parameters. However, in some sub‐groups, called small areas due to small sample sizes relative to the population, the information needed for reliable estimation is typically not available. Consequently, data on a coarser scale are used to predict the characteristics of small areas. Mixed models are the primary tools in small area estimation (SAE) and also borrow information from alternative sources (e.g., previous surveys and administrative and census data sets). In many circumstances, small area predictors are associated with location. For instance, in the case of chronic disease or cancer, it is important for policy makers to understand spatial patterns of disease in order to determine small areas with high risk of disease and establish prevention strategies. The literature considering SAE with spatial random effects is sparse and mostly in the context of spatial linear mixed models. In this article, small area models are proposed for the class of spatial generalized linear mixed models to obtain small area predictors and corresponding second‐order unbiased mean squared prediction errors via Taylor expansion and a parametric bootstrap approach. The performance of the proposed approach is evaluated through simulation studies and application of the models to a real esophageal cancer data set from Minnesota, U.S.A. The Canadian Journal of Statistics 47: 426–437; 2019 © 2019 Statistical Society of Canada     

ROBUST ESTIMATION OF SMALL‐AREA MEANS AND QUANTILES

Small‐area estimation techniques have typically relied on plug‐in estimation based on models containing random area effects. More recently, regression M‐quantiles have been suggested for this purpose, thus avoiding conventional Gaussian assumptions, as well as problems associated with the specification of random effects. However, the plug‐in M‐quantile estimator for the small‐area mean can be shown to be the expected value of this mean with respect to a generally biased estimator of the small‐area cumulative distribution function of the characteristic of interest. To correct this problem, we propose a general framework for robust small‐area estimation, based on representing a small‐area estimator as a functional of a predictor of this small‐area cumulative distribution function. Key advantages of this framework are that it naturally leads to integrated estimation of small‐area means and quantiles and is not restricted to M‐quantile models. We also discuss mean squared error estimation for the resulting estimators, and demonstrate the advantages of our approach through model‐based and design‐based simulations, with the latter using economic data collected in an Australian farm survey.     

Bayesian non-parametric models for regional prevalence estimation

We developed a flexible non-parametric Bayesian model for regional disease-prevalence estimation based on cross-sectional data that are obtained from several subpopulations or clusters such as villages, cities, or herds. The subpopulation prevalences are modeled with a mixture distribution that allows for zero prevalence. The distribution of prevalences among diseased subpopulations is modeled as a mixture of finite Polya trees. Inferences can be obtained for (1) the proportion of diseased subpopulations in a region, (2) the distribution of regional prevalences, (3) the mean and median prevalence in the region, (4) the prevalence of any sampled subpopulation, and (5) predictive distributions of prevalences for regional subpopulations not included in the study, including the predictive probability of zero prevalence. We focus on prevalence estimation using data from a single diagnostic test, but we also briefly discuss the scenario where two conditionally dependent (or independent) diagnostic tests are used. Simulated data demonstrate the utility of our non-parametric model over parametric analysis. An example involving brucellosis in cattle is presented.