Tests of forecast accuracy and bias for county population projections

"This article deals with the forecast accuracy and bias of population projections for 2,971 counties in the United States. It uses three different projection techniques and data from 1950, 1960, 1970, and 1980 to make two sets of 10-year projections and one set of 20-year projections. These projections are compared with census counts to determine forecast errors. The size, direction, and distribution of forecast errors are analyzed by size of place, rate of growth, and length of projection horizon. A number of consistent patterns are noted, and an extension of the empirical results to the production of confidence intervals for population projections is considered." A comment by Paul M. Beaumont and Andrew M. Isserman is included (pp. 1,004-9) together with a rejoinder by the author (pp. 1,009-12). This is a revised version of a paper presented at the 1986 Annual Meeting of the Population Association of America (see Population Index, Vol. 52, No. 3, Fall 1986, p. 456).     

Uncertain population forecasting

"Errors in population forecasts arise from errors in the jump-off population and errors in the predictions of future vital rates. The propagation of these errors through the linear (Leslie) growth model is studied, and prediction intervals for future population are developed. For U.S. national forecasts, the prediction intervals are compared with the U.S. Census Bureau's high-low intervals." In order to assess the accuracy of the predictions of vital rates, the authors "derive the predictions from a parametric statistical model and estimate the extent of model misspecification and errors in parameter estimates. Subjective, expert opinion, so important in real forecasting, is incorporated with the technique of mixed estimation. A robust regression model is used to assess the effects of model misspecification."     

An evaluation of population projection errors for census tracts

"In this article we evaluate the accuracy and bias of projections of total population and population by age group for census tracts in three counties in Florida. We use [U.S. census] data from 1970 and 1980 and several simple extrapolation techniques to produce projections for 1990; we then compare these projections with 1990 census counts and evaluate the differences. For the total sample, we find mean absolute errors of 17%-20% for the three most accurate techniques for projecting total population and find no indication of overall bias. For individual age groups, mean absolute errors range from 20%-29%." This is a revised version of a paper presented at the 1993 Annual Meeting of the Population Association of America.     

Empirical Likelihood for Linear Models Under Linear Process Errors

In this article, we study the construction of confidence intervals for regression parameters in a linear model under linear process errors by using the blockwise technique. It is shown that the blockwise empirical likelihood (EL) ratio statistic is asymptotically χ²-type distributed. The result is used to obtain EL based confidence regions for regression parameters. The finite-sample performance of the method is evaluated through a simulation study.     

Nonparametric standard errors and confidence intervals

We investigate several nonparametric methods; the bootstrap, the jackknife, the delta method, and other related techniques. The first and simplest goal is the assignment of nonparametric standard errors to a real-valued statistic. More ambitiously, we consider setting nonparametric confidence intervals for a real-valued parameter. Building on the well understood case of confidence intervals for the median, some hopeful evidence is presented that such a theory may be possible.     

Semi-empirical likelihood inference for the ROC curve with missing data

The receiver operating characteristic (ROC) curve is one of the most commonly used methods to compare the diagnostic performance of two or more laboratory or diagnostic tests. In this paper, we propose semi-empirical likelihood based confidence intervals for ROC curves of two populations, where one population is parametric and the other one is non-parametric and both have missing data. After imputing missing values, we derive the semi-empirical likelihood ratio statistic and the corresponding likelihood equations. It is shown that the log-semi-empirical likelihood ratio statistic is asymptotically scaled chi-squared. The estimating equations are solved simultaneously to obtain the estimated lower and upper bounds of semi-empirical likelihood confidence intervals. We conduct extensive simulation studies to evaluate the finite sample performance of the proposed empirical likelihood confidence intervals with various sample sizes and different missing probabilities.     

Confidence intervals based on the deviance statistic for the hyperparameters in state space models

The main objective of this work is to evaluate the performance of confidence intervals, built using the deviance statistic, for the hyperparameters of state space models. The first procedure is a marginal approximation to confidence regions, based on the likelihood test, and the second one is based on the signed root deviance profile. Those methods are computationally efficient and are not affected by problems such as intervals with limits outside the parameter space, which can be the case when the focus is on the variances of the errors. The procedures are compared to the usual approaches existing in the literature, which includes the method based on the asymptotic distribution of the maximum likelihood estimator, as well as bootstrap confidence intervals. The comparison is performed via a Monte Carlo study, in order to establish empirically the advantages and disadvantages of each method. The results show that the methods based on the deviance statistic possess a better coverage rate than the asymptotic and bootstrap procedures.     

Pseudo‐empirical likelihood ratio confidence intervals for complex surveys

The authors show how an adjusted pseudo‐empirical likelihood ratio statistic that is asymptotically distributed as a chi‐square random variable can be used to construct confidence intervals for a finite population mean or a finite population distribution function from complex survey samples. They consider both non‐stratified and stratified sampling designs, with or without auxiliary information. They examine the behaviour of estimates of the mean and the distribution function at specific points using simulations calling on the Rao‐Sampford method of unequal probability sampling without replacement. They conclude that the pseudo‐empirical likelihood ratio confidence intervals are superior to those based on the normal approximation, whether in terms of coverage probability, tail error rates or average length of the intervals.     

Adjusted Wilcoxon signed rank test tables for ratio of percentiles

The ordinary Wilcoxon signed rank test table provides confidence intervals for the median of one population. Adjusted Wilcoxon signed rank test tables which can provide confidence intervals for the median and the 10th percentile of one population are created in this paper. Base-(n + 1) number system and theorems about property of symmetry of the adjusted Wilcoxon signed rank test statistic are derived for programming. Theorem 1 states that the adjusted Wilcoxon signed rank test statistic are symmetric around n(n + 1)/4. Theorem 2 states that the adjusted Wilcoxon signed rank test statistic with the same number of negative ranks m are symmetric around m(n+1)/2. 87.5% and 85% confidence intervals of the median are given in the table for n = 12, 13,…, 29 to create approximated 95% confidence intervals of the ratio of medians for two independent populations. 95% and 92.5% confidence intervals of the 10th percentile are given in the table for n = 26, 27, 28, 29 to create approximated 95% confidence regions of the ratio of the 10th percentiles for two independent populations. Finally two large datasets from wood industry will be partitioned to verify the correctness of adjusted Wilcoxon signed rank test tables for small samples.     

Quantile inference based on partially rank-ordered set samples

This paper develops statistical inference for population quantiles based on a partially rank-ordered set (PROS) sample design. A PROS sample design is similar to a ranked set sample with some clear differences. This design first creates partially rank-ordered subsets by allowing ties whenever the units in a set cannot be ranked with high confidence. It then selects a unit for full measurement at random from one of these partially rank-ordered subsets. The paper develops a point estimator, confidence interval and hypothesis testing procedure for the population quantile of order p. Exact, as well as asymptotic, distribution of the test statistic is derived. It is shown that the null distribution of the test statistic is distribution-free, and statistical inference is reasonably robust against possible ranking errors in ranking process.     

Empirical Likelihood for Nonparametric Models Under Linear Process Errors

In this paper, we study the construction of confidence intervals for a nonparametric regression function under linear process errors by using the blockwise technique. It is shown that the blockwise empirical likelihood (EL) ratio statistic is asymptotically distributed. The result is used to obtain EL based confidence intervals for the nonparametric regression function. The finite‐sample performance of the method is evaluated through a simulation study.     

Stochastic population forecasts for the United States: beyond high,medium, and low

"This article presents and implements a new method for making stochastic population forecasts that provide consistent probability intervals. We blend mathematical demography and statistical time series methods to estimate stochastic models of fertility and mortality based on U.S. data back to 1900 and then use the theory of random-matrix products to forecast various demographic measures and their associated probability intervals to the year 2065. Our expected total population sizes agree quite closely with the Census medium projections, and our 95 percent probability intervals are close to the Census high and low scenarios. But Census intervals in 2065 for ages 65+ are nearly three times as broad as ours, and for 85+ are nearly twice as broad. In contrast, our intervals for the total dependency and youth dependency ratios are more than twice as broad as theirs, and our ratio for the elderly dependency ratio is 12 times as great as theirs. These items have major implications for policy, and these contrasting indications of uncertainty clearly show the limitations of the conventional scenario-based methods."     

Empirical Likelihood Confidence Region for the Parameter in a Partially Linear Errors-in-Variables Model

In this article, a partially linear errors-in-variables model is considered, and empirical log-likelihood ratio statistic for the unknown parameter in the model is suggested. It is proved that the proposed statistic is asymptotically standard chi-square distribution under some suitable conditions, and hence it can be used to construct the confidence region of the parameter. A simulation study indicates that, in terms of coverage probabilities and average lengths of the confidence intervals, the proposed method performs better than the least-squares method.     

One-sided confidence intervals for population variances of skewed distributions

In this article we study the coverage accuracy of one-sided bootstrap-t confidence intervals for the population variances combined with Hall's and Johnson's transformation. We compare the coverage accuracy of all suggested intervals and intervals based on the Chi-square statistic for variances of positively skewed distributions. In addition, we describe and discuss an application of the presented methods for measuring and analyzing revenue variability within the food retail industry. The results show that both Hall's transformation and Johnson's transformation approaches yield good coverage accuracy of the lower endpoint confidence intervals, better than method based on the Chi-square statistic. For the upper endpoint confidence intervals Hall's bootstrap-t method yields the best coverage accuracy when compared with other methods.     

A Primer on Visualizations for Comparing Populations,Including the Issue of Overlapping Confidence Intervals

In comparing a collection of K populations, it is common practice to display in one visualization confidence intervals for the corresponding population parameters θ₁, θ₂, …, θ_K. For a pair of confidence intervals that do (or do not) overlap, viewers of the visualization are cognitively compelled to declare that there is not (or there is) a statistically significant difference between the two corresponding population parameters. It is generally well known that the method of examining overlap of pairs of confidence intervals should not be used for formal hypothesis testing. However, use of a single visualization with overlapping and nonoverlapping confidence intervals leads many to draw such conclusions, despite the best efforts of statisticians toward preventing users from reaching such conclusions. In this article, we summarize some alternative visualizations from the literature that can be used to properly test equality between a pair of population parameters. We recommend that these visualizations be used with caution to avoid incorrect statistical inference. The methods presented require only that we have K sample estimates and their associated standard errors. We also assume that the sample estimators are independent, unbiased, and normally distributed.     

Coverage-adjusted estimators for mark-recapture in heterogeneous populations

Consideration of coverage yields a new class of estimators of population size for the standard mark-recapture model which permits heterogeneity of capture probabilities. Real data and simulation studies are used to assess these coverage-adjusted estimators. The simulations highlight the need for estimators that perform well for a wide range of values of the mean and coefficient of variation of the capture probabilities. When judged for this type of robustness, the simulations provide good grounds for preferring the new estimators to earlier ones for this model, except when the number of sampling occasions is large. A bootstrapping approach is used to estimate the standard errors of the new estimators, and to obtain confidence intervals for the population size.     

Robust likelihood inference for public policy

Issues of public policy are typically decided by non‐specialists who are increasingly informed by statistical methods. In order to be influential, inferential techniques must be widely understood and accepted. This motivates the author to propose likelihood‐based methods that prove relatively insensitive to the choice of underlying distribution because they exploit a peculiarly stable relation between two standard errors and a 95% coverage probability. The author also notes that bootstrap and jackknife estimates of variance can sometimes be strongly biased. In fact, symbolic computations in R suggest that they are reliable only for statistics that are well approximated by averages whose distributions are roughly symmetric. The author thus proposes to transform the classical likelihood ratio into a statistic whose variance can be estimated robustly. He shows that the signed root of the log‐likelihood is well approximated by an average with a roughly symmetric distribution. This leads to Cox‐Tukey intervals for a Student‐like statistic and to simple confidence intervals for most models used in public policy.     

ON THE ANALYSIS AND APPLICATION OF MEASURES OF LINKAGE DISEQUILIBRIUM

The maximum likelihood, jackknife and bootstrap estimators of linkage disequilibrium, a measure of association in population genetics, are derived and compared. It is found that for point estimation, the resampling methods generate almost identical mean square errors. The maximum likelihood estimator could have bigger or smaller mean square errors depending on the parameters of the underlying population. However the bootstrap confidence interval is superior to the other two as the length of the intervals is shorter or the probability that the 95% confidence intervals include the true parameter is closer to 0.95. Although the standardised measure of linkage disequilibrium has a range from -1 to 1 regardless of marginal frequencies, it is shown that the distribution of this standardised measure is still not allele frequency independent under the multinomial sampling scheme.     

Empirical Likelihood for Nonparametric Components in Additive Partially Linear Models

Empirical likelihood-based inference for the nonparametric components in additive partially linear models is investigated. An empirical likelihood approach to construct the confidence intervals of the nonparametric components is proposed when the linear covariate is measured with and without errors. We show that the proposed empirical log-likelihood ratio is asymptotically standard chi-squared without requiring the undersmoothing of the nonparametric components. Then, it can be directly used to construct the confidence intervals for the nonparametric functions. A simulation study indicates that, compared with a normal approximation-based approach, the proposed method works better in terms of coverage probabilities and widths of the pointwise confidence intervals.