On randomized response surveys for estimating a proportion

In studies about sensitive characteristics, randomized response (RR) methods are useful for generating reliable data, protecting respondents’ privacy. It is shown that all RR surveys for estimating a proportion can be encompassed in a common model and some general results for statistical inferences can be used for any given survey. The concepts of design and scheme are introduced for characterizing RR surveys. Some consequences of comparing RR designs based on statistical measures of efficiency and respondent’ protection are discussed. In particular, such comparisons lead to the designs that may not be suitable in practice. It is suggested that one should consider other criteria and the scheme parameters for planning a RR survey.     

A unified framework for analysis and comparison of randomized response surveys of binary characteristics

The randomized response (RR) procedures for estimating the proportion (π)$(\pi )$ of a population belonging to a sensitive or stigmatized group ask each respondent to report a response by randomly transforming his/her true attribute into one of several response categories. In this paper, we present a common framework for discussing various RR surveys of dichotomous populations with polychotomous responses. The unified approach is focused on the substantive issues relating to respondents’ privacy and statistical efficiency and is helpful for fair comparison of various procedures. We describe a general technique for constructing unbiased estimators of π$\pi$ based on arbitrary RR procedures, from unbiased estimators based on an open survey with the same sampling design. The technique works well for any sampling design p(s)$p(s)$ and also for variance estimation. We develop an approach for comparing RR procedures, taking both respondents’ protection and statistical efficiency into account. For any given RR procedure with three or more response categories, we present a method for designing an RR procedure with a binary response variable which provides the same respondents’ protection and at least as much statistical information. This result suggests that RR surveys of dichotomous populations should use only binary response variables.     

Item Count Technique in estimating the proportion of people with a sensitive feature

In assessing the prevalence of a sensitive attribute like habitual heroin consumption in a community of people, indirect questioning is a necessity to extract truth on ensuring protection of privacy. The current literature seems to need supplementary specification of a relevant practical and theoretical justification for one possibility by what is called an Item Count Technique. This method can be easily incorporated in large scale sample surveys where the medium of collecting information is a structured questionnaire. This feature will make this technique attractive to social survey researchers. In this article we present an amendment to the currently available technique rendering it well-equipped with a provision to protect privacy and also a sound theoretical foundation.     

An extension of the standardized randomized response technique to a multi-stage setup

If nonresponse and/or untruthful answering mechanisms occur, analyzing only the available cases may substantially weaken the validity of sample results. The paper starts with a reference to strategies of empirical social researchers related to respondent cooperation in surveys embedding the statistical techniques of randomized response in this framework. Further, multi-stage randomized response techniques are incorporated into the standardized randomized response technique for estimating proportions. In addition to already existing questioning designs of this family of methods, this generalization includes also several (in particular: two-stage) techniques that have not been published before. The statistical properties of this generalized design are discussed for all probability sampling designs. Further, the efficiency of the model is presented as a function of privacy protection. Hence, it can be shown that not one multi-stage design of this family at the same level of privacy protection can theoretically be more efficient than its one-stage basic version.     

Privacy protection measures for randomized response surveys on stigmatizing continuous variables

We consider the situation where sample surveys are to be undertaken on sensitive or stigmatizing issues. For such surveys, direct questioning methods usually lead to non-compliance or incorrect responses and so, the randomized response technique, where the responses are collected through some randomization device, is found to be useful. A majority of the literature on these techniques focus on dichotomous sensitive variables, while some techniques are also available for continuous sensitive variables. In this article, we focus on the extent of privacy protection available in sample surveys to respondents for continuous response variables. We also propose two measures of privacy protection. We demonstrate that the parameters of our randomization scheme can be so chosen as to achieve a pre-assigned level of privacy protection while at the same time yielding efficient estimates. We also show some numerical comparisons.     

Variance estimation in model assisted survey sampling

Two versions of Yates-Grundy type variance estimators are usually employed for large samples when estimating a survey population total by a generalized regression (Greg, in brief) predictor motivated by consideration of a linear regression model. Their two alternative modifications are developed so that the limiting values of the design expectations of the model expectations of variance estimators 'match' respectively the (I) model expectations of the Taylor approximation of the design variance of the Greg predictor and the (II) limiting value of the design expectation of the model expectation of the squared difference between the Greg predictor and the population total. The exercise is extended to yield modifications needed when randomized response (RR) is only available rather than direct response (DR) when one encounters sensitive issues demanding protection of privacy. A comparative study based on simulation is presented for illustration..     

Protection of privacy in efficient application of randomized response techniques

In estimating the proportion of people bearing a sensitive attribute A, say, in a given community, following Warner’s (J Am Stat Assoc 60:63–69, 1965) pioneering work, certain randomized response (RR) techniques are available for application. These are intended to ensure efficient and unbiased estimation protecting a respondent’s privacy when it touches a person’s socially stigmatizing feature like rash driving, tax evasion, induced abortion, testing HIV positive, etc. Lanke (Int Stat Rev 44:197–203, 1976), Leysieffer and Warner (J Am Stat Assoc 71:649–656, 1976), Anderson (Int Stat Rev 44:213–217, 1976, Scand J Stat 4:11–19, 1977) and Nayak (Commun Stat Theor Method 23:3303–3321, 1994) among others have discussed how maintenance of efficiency is in conflict with protection of privacy. In their RR-related activities the sample selection is traditionally by simple random sampling (SRS) with replacement (WR). In this paper, an extension of an essential similarity in case of general unequal probability sample selection even without replacement is reported. Large scale surveys overwhelmingly employ complex designs other than SRSWR. So extension of RR techniques to complex designs is essential and hence this paper principally refers to them. New jeopardy measures to protect revelation of secrecy presented here are needed as modifications of those in the literature covering SRSWR alone. Observing that multiple responses are feasible in addressing such a dichotomous situation especially with Kuk’s (Biometrika 77:436–438, 1990) and Christofides’ (Metrika 57:195–200, 2003) RR devices, an average of the response-specific jeopardizing measures is proposed. This measure which is device dependent, could be regarded as a technical characteristic of the device and it should be made known to the participants before they agree to use the randomization device. The views expressed are the authors’, not of the organizations they work for. Prof Chaudhuri’s research is partially supported by CSIR Grant No. 21(0539)/02/EMR-II.     

Optional versus compulsory randomized response techniques in complex surveys

In estimating the proportion of people bearing a sensitive attribute in a community, to mitigate possible evasive answer biases, Warner (J. Amer. Statist. Assoc. 60 (1965) 63) introduced a technique of randomized response (RR) in human surveys, by way of protecting individual privacy. Chaudhuri and Mukerjee (Calcutta Statist. Assoc. Bull. 34 (1985) 225; Randomized Response: Theory and Techniques, Marcel Dekker, New York) presented a modification allowing a direct response (DR) option to whom the attribute does not appear to be stigmatizing enough. Warner himself and many of his followers restrict the application of their RR devices to surveys with selection exclusively by ‘simple random sampling with replacement’. Chaudhuri (J. Statist. Plann. Inference 34 (2001a) 37; Pakistan J. Statist. 17 (3) (2001b) 259; Calcutta Statist. Assoc. Bull. 52 (205–208) (2002) 315) showed the efficacy of some of these devices when sample selection is by general unequal probabilities possibly even without replacement. Here, we present theories for unbiased estimation of the proportion along with unbiased estimation of the variances of the estimators when ‘compulsory’ or ‘optional’ RR's are gathered from persons sampled with varying probabilities. Gains in efficiency by allowing DR option rather than RR compulsion are illustrated numerically through simulation from data.     

Item count technique with no floor and ceiling effects

Abstract

Conclusions of sample surveys dealing with matters of privacy are often disputed. Many people either refuse to participate or provide false answers. Indirect questioning techniques are designed so that reliable estimates can be produced while the privacy of the participants is protected. One such method, the Item Count Technique is gaining increasing popularity. However, the original version of the method does not fully protect the privacy of the participants. In this paper we propose a new version of the technique which better protects the privacy of the respondents and does not have the so-called floor and ceiling effects.     

Efficiency comparison of unrelated question models based on same privacy protection degree

In this study, the problem of estimating the proportion π _A of people bearing a sensitive attribute A is considered. Three dichotomous unrelated question mechanisms which are alternative to the well-known Simmons’ model are discussed and their performance is evaluated taking into account both efficiency and respondent privacy protection. The variance of the estimators of π _A is compared under equal levels of confidentiality measures introduced by Lanke (1976) and Leysieffer and Warner (1976).     

Model selection in randomized response techniques for binary responses

Randomized response techniques (RRTs) have been proposed in survey sampling literature as a solution to the problem of social desirability bias (SDB) while dealing with sensitive questions. All RRTs reduce the SDB by introducing privacy protection for the respondents, but the variances of the estimates become larger compared with the ones obtained from direct questioning technique (DQT). The success of RRTs heavily depends on the assumption that the variable of interest is in fact sensitive for the population under study. There might be situations, however, where a presumably sensitive question is not considered to be sensitive in some populations, in which case using an RRT instead of the DQT would inflate the variance of the estimates unreasonably. In this study, we propose a two-stage sampling procedure for a binary response, which enables one to accurately estimate both the prevalence of the sensitive characteristic and the probability of cheating in a population. The proposed model allows one to choose between an RRT and the DQT. We support our theoretical results with numerous simulations.     

Valid estimates for repeated randomized response methods

Surveys with sensitive characteristics (e.g. cheating in exams, fiscal evasion, social fraud, insurance fraud, discrimination, political views, financial situation) need special concepts, because normal direct questioning causes answer refusal and lies. One well-established concept is the randomized response (RR) approach. RRs protect the interviewees' privacy and facilitate their cooperation. Based on the RRs of many persons, inference is possible. A recently published article suggests two repeated RR methods. That is, each interviewee must give more than one answer. Repeated RRs are a good idea to improve the estimation efficiency of RR techniques. However, this recently published article contains serious mistakes and derives invalid estimates. For this reason, we correct these mistakes and develop valid estimates in the first part of our article. Subsequently, in the second part, we present generalized considerations that cover many more repeated RR schemes.     

On using non identical and independent Bernoulli trials in human surveys

Unlike the usual randomized response techniques, as a pioneering attempt, this article focuses on using non identical independent Bernoulli trials in sensitive surveys. For this purpose, a general class of randomized response techniques is considered. The usual randomized response techniques are based on a fixed probability of having a yes answer. Contrary to usual techniques, in the proposed technique every respondent has a different probability of reporting a yes answer. With this setting, in most of the situations, the proposed technique is observed performing better in terms of variability. To illustrate and support the superiority of the proposed technique it is compared with models such as Warner (1965), Greenberg et al. (1969), Mangat and Singh (1990), and Mangat (1994) using identical Bernoulli trials. Relative efficiency and privacy protection are studied in detail using Warner (1965) and Mangat (1994) models.     

Using randomized response from a complex survey to estimate a sensitive proportion in a dichotomous finite population

In estimating the proportion θ_A of people in a given community bearing a sensitive characteristic A, in order to protect the respondent's privacy, various techniques of generating randomized response (RR) rather than direct response are available in the literature. But the theory concerning them is developed only for samples selected by ‘simple random sampling’ (SRS) ‘with replacement’ or at most by SRS without replacement method. Illustrating two such RR devices we show how an estimator along with an estimated measure of its error may be developed if the sample of persons may be drawn adopting a complex survey design involving unequal selection probabilities with or without replacement.     

An alternative to unrelated randomized response techniques with logistic regression analysis

The randomized response technique (RRT) is an important tool that is commonly used to protect a respondent’s privacy and avoid biased answers in surveys on sensitive issues. In this work, we consider the joint use of the unrelated-question RRT of Greenberg et al. (J Am Stat Assoc 64:520–539, 1969) and the related-question RRT of Warner (J Am Stat Assoc 60:63–69, 1965) dealing with the issue of an innocuous question from the unrelated-question RRT. Unlike the existing unrelated-question RRT of Greenberg et al. (1969), the approach can provide more information on the innocuous question by using the related-question RRT of Warner (1965) to effectively improve the efficiency of the maximum likelihood estimator of Scheers and Dayton (J Am Stat Assoc 83:969–974, 1988). We can then estimate the prevalence of the sensitive characteristic by using logistic regression. In this new design, we propose the transformation method and provide large-sample properties. From the case of two survey studies, an extramarital relationship study and a cable TV study, we develop the joint conditional likelihood method. As part of this research, we conduct a simulation study of the relative efficiencies of the proposed methods. Furthermore, we use the two survey studies to compare the analysis results under different scenarios.     

Some New Results on the Multinomial Randomized Response Model

ABSTRACT

The randomized response technique is an effective survey method designed to elicit sensitive information while ensuring the privacy of the respondents. In this article, we present some new results on the randomization response model in situations wherein one or two response variables are assumed to follow a multinomial distribution. For a single sensitive question, we use the well-known Hopkins randomization device to derive estimates, both under the assumption of truthful and untruthful responses, and present a technique for making pairwise comparisons. When there are two sensitive questions of interest, we derive a Pearson product moment correlation estimator based on the multinomial model assumption. This estimator may be used to quantify the linear relationship between two variables when multinomial response data are observed according to a randomized-response protocol.     

Mean and sensitivity estimation of a sensitive variable through additive scrambling

Abstract

This article focuses on reducing the additional variance due to randomization of the responses. The idea of additive scrambling and its inverse has been used along with (i) split sample approach and (ii) double response approach. Specifically, our proposal is based on Gupta et al. (2006) randomized response model. We selected this model for improvement because it provides estimator of mean and sensitivity level of a sensitive variable and is better than all of its competitors proposed earlier to it and even Gupta et al. (2006) sensitivity estimator is better than that of Gupta et al. (2010). Our suggested estimators are unbiased estimators and perform better than Gupta et al. (2006) estimator. The issue of privacy protection is also discussed.     

Estimating a sensitive proportion through randomized response procedures based on auxiliary information

Randomized response techniques are widely employed in surveys dealing with sensitive questions to ensure interviewee anonymity and reduce nonrespondents rates and biased responses. Since Warner’s (J Am Stat Assoc 60:63–69, 1965) pioneering work, many ingenious devices have been suggested to increase respondent’s privacy protection and to better estimate the proportion of people, π _A , bearing a sensitive attribute. In spite of the massive use of auxiliary information in the estimation of non-sensitive parameters, very few attempts have been made to improve randomization strategy performance when auxiliary variables are available. Moving from Zaizai’s (Model Assist Stat Appl 1:125–130, 2006) recent work, in this paper we provide a class of estimators for π _A , for a generic randomization scheme, when the mean of a supplementary non-sensitive variable is known. The minimum attainable variance bound of the class is obtained and the best estimator is also identified. We prove that the best estimator acts as a regression-type estimator which is at least as efficient as the corresponding estimator evaluated without allowing for the auxiliary variable. The general results are then applied to Warner and Simmons’ model.     

On comutativity of design and model expectations in randomized response surveys

Interchangability of the order of design expectation ( E_p ) and model expectation ( E_R ) under randomized response ( RR ) surveys for finding expectation and mean-square error of a linear estimator e found to be permissible for a non-informative sampling design. The non-comutative property established by Chaudhuri & Adhikary ( 1990 ) under probability proportional to size with replacement ( PPSWR ) sampling is disproved.     

A modified unrelated question randomized response device for complex surveys

Singh et al. (Stat Trans 6(4):515–522, 2003) proposed a modified unrelated question procedure and they also demonstrated that the modified procedure is capable of producing a more efficient estimator of the population parameter π _A , namely, the proportion of persons in a community bearing a sensitive character A when π _A  < 0.50. The development of Singh et al. (Stat Trans 6(4):515–522, 2003) is based on simple random samples with replacement and on the assumption that π _B , namely, the proportion of individuals bearing an unrelated innocuous character B is known. Due to these limitations, Singh et al.’s (Stat Trans 6(4):515–522, 2003) procedure cannot be used in practical surveys where usually the sample units are chosen with varying selection probabilities. In this article, following Singh et al. (Stat Trans 6(4):515–522, 2003) we propose an alternative RR procedure assuming that the population units are sampled with unequal selection probabilities and that the value of π _B is unknown. A numerical example comparing the performance of the proposed RR procedure under alternative sampling designs is also reported.