Estimation bias of different design and analytical strategies in dual-frame telephone surveys: an empirical evaluation

Dual-frame survey designs have become increasingly popular in large-scale telephone surveys. This is due to the lack of coverage of the traditional landline survey design and the escalating use of cell phones in recent years. Several estimation strategies have been proposed and their properties have been discussed under ideal scenarios, including pseudo-maximum-likelihood estimation, single-frame estimation, and simple composite estimation C.J. Skinner and J.N.K. Rao, Estimation in dual frame surveys with complex designs, J. Am. Statist. Assoc. 91 (1996), pp. 349–356; S.L. Lohr and J.N.K. Rao, Inference from dual frame surveys, J. Am. Statist. Assoc. 95 (2000), pp. 271–280]. In practice, estimation in dual-frame telephone surveys is vulnerable to biases and errors (e.g. inaccessibility, topic/mode salience, and measurement error). The investigation of the performance of popular dual-frame estimation methods is scarce in real and less ideal scenarios. Through an innovatively designed simulation study, we compare the estimation bias under different sampling designs with various estimation strategies. To reduce bias, different raking strategies are compared. Simulated scenarios incorporating sampling costs are examined for practical considerations. Overall, the cell phone-only design yields results with the least bias and variance. When accurate covariate information is available for post-stratification, raking estimates from the cell phone-any design also perform very well. We also provide SAS macros for this simulation evaluation upon request. Survey practitioners can fine-tune the parameters based on their prior knowledge of the target population and run the simulation under different scenarios to gain more insights into how to optimally design and analyse telephone surveys.