Survey non-response and the duration of unemployment

Summary.  Social surveys are often used to estimate unemployment duration distributions. Survey non-response may then cause a bias. We study this by using a data set that combines survey information of individual workers with administrative records of the same workers. The latter provide information on durations of unemployment and personal characteristics of all survey respondents and non-respondents. We develop a method to distinguish empirically between two explanations for a bias in results based on only survey data: selectivity due to related unobserved determinants of durations of unemployment and non-response and a causal effect of a job exit on non-response. The latter may occur even in fully homogeneous populations. The methodology exploits variation in the timing of the duration outcome relative to the survey moment. The results show evidence for both explanations. We discuss implications for standard methods to deal with non-response bias.     

Model-based inference for categorical survey data subject to non-ignorable non-response

We consider non-response models for a single categorical response with categorical covariates whose values are always observed. We present Bayesian methods for ignorable models and a particular non-ignorable model, and we argue that standard methods of model comparison are inappropriate for comparing ignorable and non-ignorable models. Uncertainty about ignorability of non-response is incorporated by introducing parameters describing the extent of non-ignorability into a pattern mixture specification and integrating over the prior uncertainty associated with these parameters. Our approach is illustrated using polling data from the 1992 British general election panel survey. We suggest sample size adjustments for surveys when non-ignorable non-response is expected.     

The long-term effectiveness of refusal conversion procedures on longitudinal surveys

Summary.  Survey organizations often attempt to 'convert' sample members who refuse to take part in a survey. Persuasive techniques are used in an effort to change the refusers' minds and to agree to an interview. This is done to improve the response rate and, possibly, to reduce non-response bias. However, refusal conversion attempts are expensive and must be justified. Previous studies of the effects of refusal conversion attempts are few and have been restricted to cross-sectional surveys. The criteria for 'success' of a refusal conversion attempt are different in a longitudinal survey, where for many purposes the researcher requires complete data over multiple waves. The paper uses data from the British Household Panel Survey from 1994 to 2003 to assess the long-term effectiveness of refusal conversion procedures in terms of sample sizes, sample composition and data quality.     

Information on sexual behaviour when some data are missing

When tables contain missing values, statistical models that allow the non-response probability to be a function of the intended response have been proposed by several researchers. We investigate the properties of these methods in the context of a survey of the sexual behaviour of university students. Profile likelihoods can be computed, even when models are not identified and saturated profile likelihoods (making no assumptions about the non-response mechanism) are derived. Bayesian approaches are investigated and it is shown that their results may be highly sensitive to the prior specification. The proportion of responders answering 'yes' to the question 'have you ever had sexual intercourse?' was 73%. However, different assumptions about the non-responders gave proportions as low as 46% or as high as 83%. Our preferred estimate, derived from the response-saturated profile likelihood, is 67% with a 95% confidence interval of 58–74%. This is in line with other studies on response bias in the reports of young people's sexual behaviour that suggest that the respondents overrepresent the sexually active.     

Symmetric pattern models: a latent variable approach to item non-response in attitude scales

This paper proposes a new approach to the treatment of item non-response in attitude scales. It combines the ideas of latent variable identification with the issues of non-response adjustment in sample surveys. The latent variable approach allows missing values to be included in the analysis and, equally importantly, allows information about attitude to be inferred from non-response. We present a symmetric pattern methodology for handling item non-response in attitude scales. The methodology is symmetric in that all the variables are given equivalent status in the analysis (none is designated a 'dependent' variable) and is pattern based in that the pattern of responses and non-responses across individuals is a key element in the analysis. Our approach to the problem is through a latent variable model with two latent dimensions: one to summarize response propensity and the other to summarize attitude, ability or belief. The methodology presented here can handle binary, metric and mixed (binary and metric) manifest items with missing values. Examples using both artificial data sets and two real data sets are used to illustrate the mechanism and the advantages of the methodology proposed.     

The sensitivity of estimates of the change in population behaviour to realistic changes in bias in repeated surveys

Summary.  The first British National Survey of Sexual Attitudes and Lifestyles (NATSAL) was conducted in 1990–1991 and the second in 1999–2001. When surveys are repeated, the changes in population parameters are of interest and are generally estimated from a comparison of the data between surveys. However, since all surveys may be subject to bias, such comparisons may partly reflect a change in bias. Typically limited external data are available to estimate the change in bias directly. However, one approach, which is often possible, is to define in each survey a sample of participants who are eligible for both surveys, and then to compare the reporting of selected events that occurred before the earlier survey time point. A difference in reporting suggests a change in overall survey bias between time points, although other explanations are possible. In NATSAL, changes in bias are likely to be similar for groups of sexual experiences. The grouping of experiences allows the information that is derived from the selected events to be incorporated into inference concerning population changes in other sexual experiences. We use generalized estimating equations, which incorporate weighting for differential probabilities of sampling and non-response in a relatively straightforward manner. The results, combined with estimates of the change in reporting, are used to derive minimum established population changes, based on NATSAL data. For some key population parameters, the change in reporting is seen to be consistent with a change in bias alone. Recommendations are made for the design of future surveys.     

Estimation of an indicator of the representativeness of survey response

Nonresponse is a major source of estimation error in sample surveys. The response rate is widely used to measure survey quality associated with nonresponse, but is inadequate as an indicator because of its limited relation with nonresponse bias. Schouten et al. (2009) proposed an alternative indicator, which they refer to as an indicator of representativeness or R-indicator. This indicator measures the variability of the probabilities of response for units in the population. This paper develops methods for the estimation of this R-indicator assuming that values of a set of auxiliary variables are observed for both respondents and nonrespondents. We propose bias adjustments to the point estimator proposed by Schouten et al. (2009) and demonstrate the effectiveness of this adjustment in a simulation study where it is shown that the method is valid, especially for smaller sample sizes. We also propose linearization variance estimators which avoid the need for computer-intensive replication methods and show good coverage in the simulation study even when models are not fully specified. The use of the proposed procedures is also illustrated in an application to two business surveys at Statistics Netherlands.     

Methodology of the National Survey of Sexual Attitudes and Lifestyles

"The preparation and fieldwork for the National Survey of Sexual Attitudes and Lifestyles was carried out between 1987-1991. The survey was a combination of a face-to-face interview and self-completion questionnaire and comprised 18,876 individuals aged 16-59 years in Great Britain. The response rate was similar to other sample surveys on less sensitive topics. Women were overrepresented but the distributions of other demographic characteristics were similar to other data sets. The external validity and internal consistency of the data were found to be adequate. The likely sources and effects of bias are discussed."     

Fitting disposition codes to mobile phone surveys: experiences from studies in Finland, Slovenia and the USA

Summary.  Using mobile phones to conduct survey interviews has gathered momentum recently. However, using mobile telephones in surveys poses many new challenges. One important challenge involves properly classifying final case dispositions to understand response rates and non-response error and to implement responsive survey designs. Both purposes demand accurate assessments of the outcomes of individual call attempts. By looking at actual practices across three countries, we suggest how the disposition codes of the American Association for Public Opinion Research, which have been developed for telephone surveys, can be modified to fit mobile phones. Adding an international dimension to these standard definitions will improve survey methods by making systematic comparisons across different contexts possible.     

A Second-order Adjustment to the Profile Likelihood in the Case of a Multidimensional Parameter of Interest

Inference in the presence of nuisance parameters is often carried out by using the χ²-approximation to the profile likelihood ratio statistic. However, in small samples, the accuracy of such procedures may be poor, in part because the profile likelihood does not behave as a true likelihood, in particular having a profile score bias and information bias which do not vanish. To account better for nuisance parameters, various researchers have suggested that inference be based on an additively adjusted version of the profile likelihood function. Each of these adjustments to the profile likelihood generally has the effect of reducing the bias of the associated profile score statistic. However, these adjustments are not applicable outside the specific parametric framework for which they were developed. In particular, it is often difficult or even impossible to apply them where the parameter about which inference is desired is multidimensional. In this paper, we propose a new adjustment function which leads to an adjusted profile likelihood having reduced score and information biases and is readily applicable to a general parametric framework, including the case of vector-valued parameters of interest. Examples are given to examine the performance of the new adjusted profile likelihood in small samples, and also to compare its performance with other adjusted profile likelihoods.     

Multilevel modelling of refusal and non-contact in household surveys: evidence from six UK Government surveys

Summary.  We analyse household unit non-response in six major UK Government surveys by using a multilevel multinomial modelling approach. The models are guided by current conceptual frameworks and theories of survey participation. One key feature of the analysis is the investigation of the extent to which effects of household characteristics are survey specific. The analysis is based on the 2001 UK Census Link Study, which is a unique source of data containing an unusually rich set of auxiliary variables. The study contains the response outcome of six surveys, linked to census data and interviewer observations for both respondents and non-respondents.     

A simulation study: estimation of population mean using two auxiliary variables in stratified random sampling

ABSTRACT

This paper deals with the problem of estimating the finite population mean in stratified random sampling by using two auxiliary variables. This paper proposed a ratio-cum-product exponential type estimator of population mean under different situations: (i) when there is presence of non-response and measurement errors on the study as well as auxiliary variables; (ii) when there is non-response on the study and auxiliary variables but with no measurement error; (iii) when there is complete response on study variable but there is presence of non-response and measurement error on the auxiliary variables and (iv) when there are complete response and measurement error on study as well as auxiliary variables. The expressions of the bias and mean square error of the proposed estimator have been obtained up to the first degree of approximation. The proposed estimator has been compared with usual unbiased estimator, ratio estimator and other existing estimators and the conditions obtained to show the efficacy of the proposed estimator over other considered estimators. Simulation study is carried out to support the theoretical findings.     

Estimation of population product in presence of non-response in successive sampling

The design of this paper is to examine the problem of estimation of finite population product in mail surveys for the current occasion in the context of sampling on two occasions is attempted when there is non-response on both the occasions. Estimators for the current occasion are derived as a particular case when there is non-response on first and second occasion only. The gain in efficiency of the proposed estimate over the direct estimate using no information gathered on the first occasion is computed. The proposed strategy has been compared with other estimators. An empirical study is carried out to study the performance of the proposed strategy.     

An Application of Non–Ignorable Non–Response Models for Gross Flows Estimation in the British Labour Force Survey

In previous work, non–response adjustments based on calibration weighting have been proposed for estimating gross flows in economic activity status from the quarterly Labour Force Survey. However, even after adjustment there may be residual non–response bias. The weighting is based on estimates of cross–sectional distributions and so cannot adjust for bias if non–response is associated with individual flows between quarters. To investigate this possibility, it was decided to apply models for estimating gross flows when non–response depends on the flows. This paper has two aims: first to describe the many problems encountered when attempting to implement these models; and second to outline a solution to the major problem that arose, namely, that comparing the model results directly with the weighting results was not possible. A simulation study was used to compare the results indirectly and it was tentatively concluded that non–response is not strongly associated with the flows and that the weighting provides an adequate adjustment.     

Estimation of mean vector in presence of non-response

Simultaneous estimation of means of several variables is considered for finite population in presence of non-response. Two types of nonresponses (partial and complete) are considered using the technique of sampling and subsampling with equal probabilities without replacement. The optimum sample size and the optimum value of subsampling fraction to be repeated from the nonresponding units of the sample have been obtained for fixed survey budget.     

Latent class models for longitudinal studies of the elderly with data missing at random

The elderly population in the USA is expected to double in size by the year 2025, making longitudinal health studies of this population of increasing importance. The degree of loss to follow-up in studies of the elderly, which is often because elderly people cannot remain in the study, enter a nursing home or die, make longitudinal studies of this population problematic. We propose a latent class model for analysing multiple longitudinal binary health outcomes with multiple-cause non-response when the data are missing at random and a non-likelihood-based analysis is performed. We extend the estimating equations approach of Robins and co-workers to latent class models by reweighting the multiple binary longitudinal outcomes by the inverse probability of being observed. This results in consistent parameter estimates when the probability of non-response depends on observed outcomes and covariates (missing at random) assuming that the model for non-response is correctly specified. We extend the non-response model so that institutionalization, death and missingness due to failure to locate, refusal or incomplete data each have their own set of non-response probabilities. Robust variance estimates are derived which account for the use of a possibly misspecified covariance matrix, estimation of missing data weights and estimation of latent class measurement parameters. This approach is then applied to a study of lower body function among a subsample of the elderly participating in the 6-year Longitudinal Study of Aging.     

The Use of Sample Weights in Hot Deck Imputation

A common strategy for handling item nonresponse in survey sampling is hot deck imputation, where each missing value is replaced with an observed response from a "similar" unit. We discuss here the use of sampling weights in the hot deck. The naive approach is to ignore sample weights in creation of adjustment cells, which effectively imputes the unweighted sample distribution of respondents in an adjustment cell, potentially causing bias. Alternative approaches have been proposed that use weights in the imputation by incorporating them into the probabilities of selection for each donor. We show by simulation that these weighted hot decks do not correct for bias when the outcome is related to the sampling weight and the response propensity. The correct approach is to use the sampling weight as a stratifying variable alongside additional adjustment variables when forming adjustment cells.     

Comments on the paper “Bias-adjustment and calibration of jackknife variance estimator in the presence of non-response”

Singh and Arnab (2010) presented a bias adjustment to the jackknife variance estimator of Rao and Sitter (1995) in the presence of non-response. In their paper, they obtained a second-order approximation of the bias of the Rao-Sitter variance estimator and then proposed a bias-adjusted estimator based on this approximation. To compare their proposed variance estimator to various other variance estimators, they performed a simulation study and showed that their variance estimator is superior to the Rao-Sitter variance estimator. In fact they showed that the Rao-Sitter variance estimator suffers from severe underestimation. These results contradict those in the literature, which indicate that the Rao-Sitter variance estimator suffers from a positive bias if the sampling fractions are not negligible; see Rao and Sitter (1995), Lee et al. (1995) and Haziza and Picard (2011). Because of this contradiction, we felt that a further investigation was warranted. In this paper, we attempt to recreate the results of Singh and Arnab (2010) and, in fact, show that their second order approximation to the bias of the Rao-Sitter variance estimator is incorrect and that their simulation results are also questionable.