Insights from the Evaluation of Past Local Area Population Forecasts

Local area population forecasts have a wide variety of uses in the public and private sectors. But not enough is known about the errors of such forecasts, particularly over the longer term (20 years or more). Understanding past errors is valuable for both forecast producers and users. This paper (i) evaluates the forecast accuracy of past local area population forecasts published by Australian State and Territory Governments over the last 30 years and (ii) illustrates the ways in which past error distributions can be employed to quantify the uncertainty of current forecasts. Population forecasts from the past 30 years were sourced from State and Territory Governments. Estimated resident populations to which the projections were compared were created for the geographical regions of the past projections. The key features of past forecast error patterns are described. Forecast errors mostly confirm earlier findings with regard to the relationship between error and length of projection horizon and population size. The paper then introduces the concept of a forecast ‘shelf life’, which indicates how far into the future a forecast is likely to remain reliable. It also illustrates how past error distributions can be used to create empirical prediction intervals for current forecasts. These two complementary measures provide a simple way of communicating the likely magnitude of error that can be expected with current local area population forecasts.     

Accounting for migration in cohort-component projections of state and local populations

Migration is the most difficult component of state and local population growth to forecast accurately because it is more volatile than either births or deaths, and subject to much larger fluctuations within a short period of time. In addition, migration rates can be based on several different measures of migration and the base population. The choice of the appropriate base population has received little attention from demographic researchers, but can have a tremendous impact on population projections. In this article, I develop three different models for projecting migration, each using a different denominator for migration rates. Population projections for ten states are made, using identical data and cohort component techniques, except for the different formulations of migration rates. Differences among the three sets of projections are noted, and conclusions are drawn regarding their usefulness as forecasts of population growth.     

Development and accuracy of projections of population and households in the united states

The history of the official U. S. projections of population and house-holds in recent decades is briefly reviewed, with particular attention to methodology and the relation of the methodology to the accuracy of the projections. The introduction of the cohort-component procedure in the 1930’s opened the way for separate analysis of the trend of the components of births, deaths, and net immigration in connection with making population projections. As a further development, the period-fertility method of projecting births gave way in the 1960’s to the cohort-fertility method. Consideration is now being given to various methods involving parity-progression procedures. Some alternative methods and problems of measuring the accuracy of population projections are then considered. The percent “error” in the projected population growth, by components and age, and the range from high to low expressed as a percent should also be examined in addition to the percent “error” in the total population. However accuracy is measured, the projections made in the 1930’s and 1940’s were often wide of the mark, and those made in the 1950’s and 1960’s failed to anticipate the sharp changes which occurred, even though the actual figures usually fell within the range projected. Elaboration of projection methodology has not resulted in any great increase in the precision of the projections, largely because birth rates have fluctuated widely, and the fluctuations have proven difficult, if not impossible, to predict. The projections of households have had a roughly similar history, and the methods and problems of evaluation are somewhat similar. Their development has been characterized by the introduction of alternative and changing “headship” rates and increasing disaggregation of the data and procedures. The paper concludes with some generalizations based on U. S. experience. Although refinement of methods may contribute little to accuracy, accuracy is only one aspect of the usefulness of projections. The need for conditional projections and their analytical usefulness are such that there is no question that we should confidently continue to make them.     

Evaluation of Alternative Cohort-Component Models for Local Area Population Forecasts

It is generally accepted by demographers that cohort-component projection models which incorporate directional migration are conceptually preferable to those using net migration. Yet net migration cohort-component models, and other simplified variations, remain in common use by both academics and practitioners because of their simplicity and low data requirements. While many arguments have been presented in favour of using one or other type of model, surprisingly little analysis has been undertaken to assess which tend to give the most accurate forecasts. This paper evaluates five cohort-component models which differ in the way they handle migration, four of which are well known, with one—a composite net migration model—being proposed here for the first time. The paper evaluates the performance of these five models in their unconstrained form, and then in a constrained form in which age–sex-specific forecasts are constrained to independent total populations from an extrapolative model shown to produce accurate forecasts in earlier research. Retrospective forecasts for 67 local government areas of New South Wales were produced for the period 1991–2011 and then compared to population estimates. Assessments of both total and age-specific population forecasts were made. The results demonstrate the superior performance of the forecasts constrained to total populations from the extrapolative model, with the constrained bi-regional model giving the lowest errors. The findings should be of use to practitioners in selecting appropriate models for local area population forecasts.     

Empirical Prediction Intervals for County Population Forecasts

Population forecasts entail a significant amount of uncertainty, especially for long-range horizons and for places with small or rapidly changing populations. This uncertainty can be dealt with by presenting a range of projections or by developing statistical prediction intervals. The latter can be based on models that incorporate the stochastic nature of the forecasting process, on empirical analyses of past forecast errors, or on a combination of the two. In this article, we develop and test prediction intervals based on empirical analyses of past forecast errors for counties in the United States. Using decennial census data from 1900 to 2000, we apply trend extrapolation techniques to develop a set of county population forecasts; calculate forecast errors by comparing forecasts to subsequent census counts; and use the distribution of errors to construct empirical prediction intervals. We find that empirically-based prediction intervals provide reasonably accurate predictions of the precision of population forecasts, but provide little guidance regarding their tendency to be too high or too low. We believe the construction of empirically-based prediction intervals will help users of small-area population forecasts measure and evaluate the uncertainty inherent in population forecasts and plan more effectively for the future.     

Forecast accuracy of australian subnational population projections

Despite the considerable resources devoted to making demographic projections in Australia over the past two decades, there have been few attempts to evaluate the performance of these projections in terms of forecast accuracy. This paper first considers the role of accuracy amongst other objectives of projection activity. Accepting accuracy as a legitimate goal, we then assess the performance of 48 sets of population projections and forecasts for states and territories of Australia prepared since 1970. Projection accuracy is assessed by reference to length of forecast horizon, population size and rate of growth. We also examine the main sources of forecast error in selected projections for each state and compare the performance of past projections with alternatives based on simple extrapolation of contemporary population trends.     

Effects of targets and aggregation on the propagation of error in mortality forecasts

"Official forecasts of mortality depend on assumptions about target values for the future rates of decline in mortality rates. Smooth functions connect the jump-off (base-year) mortality to the level implied by the targets. Three alternative sets of targets are assumed, leading to high, middle, and low forecasts. We show that this process can be closely modeled using simple linear statistical models. These explicit models allow us to analyze the error structure of the forecasts. We show that the current assumption of perfect correlation between errors in different ages, at different forecast years, and for different causes of death, is erroneous. An alternative correlation structure is suggested, and we show how its parameters can be estimated from the past data. The effect of the level of aggregation on the accuracy of mortality forecasts is considered." The geographical focus is on the United States. (SUMMARY IN FRE)     

Effect of census errors on the measurement of net migration

This paper traces the history of the use of vital statistics, survival rates, and ratios in the estimation of net migration from one decade to another. Net migration studies by Hart (1921); Baker (1933) ; Hamilton (1934); Thornthwaite (1934); Lively and Taeuber (1939) ; Henderson (1943); Hamilton and Henderson (1943); Hamilton (1951); Siegel and Hamilton (1952); Lee and Bowles (1954); Price (1955); Lee, Miller, and others (1957); Hamilton (1959); Zachariah (1962); Tarver (1962); Shryock (1964); Eldridge (1965); Hamilton (1965); and the United States Census Bureau are cited as the principal users of various residual methods of estimating net migration. All these demographers have either implicitly or explicitly recognized that errors in census enumeration and in the registration of births and deaths have been reflected in errors of estimated net migration.The underlying characteristic of all the methods used by these demographers has been the estimation of net migration as a residual obtained by subtracting natural increase in an area during a decade from the population change during the same decade. This method has been most generally stated in the classic formula {fx394-1} This formula has been used both with total populations and with aging cohorts. The principal variations of the basic formula have involved the use of life table and census survival ratios as a means of measuring natural increase (B - D), or of estimating "expected" populations assuming no migration. The main points of controversy have involved life table v. censm survival ratios, assumptions regarding the similarity in national and state census enumeration errors, and ways and means of estimating the errors involved in estimates of migration and of migration rates by the various methods.Daniel O. Price (1955) and Zachariah (1962) made important mathematical contributions and attempted to evaluate the errors involved in the me of census survival rates. Eldridge (1965) discovered that, in the United States between 1950 and 1960, the use of the census survival rate method usually gave much lower estimates of net migration than did the classic vital statistic method. Hamilton (1965), using some suggestions by Hope T. Eldridge, developed a mathematical theory or explanation of not only why the CSR estimates were usually lower than the VS estimates but also why the CSR estimates would usually give closer estimates on the true net migration than would the EVS method, which itself is subject to errors of census enumeration and of underregistration of births and deaths. The author also discusses the effect of improvement in census enumeration between 1950 and 1960 on estimates of net migration and derives a generalized formula which takes the timing of migration into consideration.The author acknowledges with sincere appreciation important constructive suggestions made by Dr. Hope T. Eldridge, Population Studies Center, University of Pennsylvania, and the authors of the many papers used as original material. This paper is a revision of a paper read before the annual meeting ot the Population Association of America, Hotel Roosevelt, New York, New York, April 29-30, 1966. Contribution from the Departments of Sociology and Experimental Statistics, North Carolina Agricultural Experiment Station, North Carolina State University. Published with the approval of the Director of Research as Paper No. 2227 of the Journal series.     

MAPE-R: a rescaled measure of accuracy for cross-sectional subnational population forecasts

Accurately measuring a population and its attributes at past, present, and future points in time has been of great interest to demographers. Within discussions of forecast accuracy, demographers have often been criticized for their inaccurate prognostications of the future. Discussions of methods and data are usually at the centre of these criticisms, along with suggestions for providing an idea of forecast uncertainty. The measures used to evaluate the accuracy of forecasts also have received attention and while accuracy is not the only criterion advocated for evaluating demographic forecasts, it is generally acknowledged to be the most important. In this paper, we continue the discussion of measures of forecast accuracy by concentrating on a rescaled version of a measure that is arguably the one used most often in evaluating cross-sectional, subnational forecasts, Mean Absolute Percent Error (MAPE). The rescaled version, MAPE-R, has not had the benefit of a major empirical test, which is the central focus of this paper. We do this by comparing 10-year population forecasts for U.S. counties to 2000 census counts. We find that the MAPE-R offers a significantly more meaningful representation of average error than MAPE in the presence of substantial outlying errors, and we provide guidelines for its implementation.     

Population projections: states, 1995 - 2025

All states will have more people in the future, especially in the south and west, while population aging occurs as the baby boomers age. This report identifies population changes projected to affect the US's 50 states and District of Columbia during 1995-2015. Basic assumptions for state population projections are presented with regard to population, births, deaths, net international migration, and net internal migration. The methodology used to produce the report is also described. Total population and net change is presented in tabular format for each state over the period. These data are used as the basic input to many federal, state, and local projection models which produce detailed statistics on education, economic factors, labor force, health care, voting, and other subjects. State differentials in fertility and mortality are also projected to widen, reflecting the concentration of race and ethnic groups with high fertility in some states and differential migration patterns.     

The role of population size in the determination and prediction of population forecast errors: An evaluation using conifidence intervals for subcounty areas

Producers of population forecasts acknowledge the uncertainty inherent in trying to predict the future and should warn about the likely error of their forecasts. Confidence intervals represent one way of quantifying population forecast error. Most of the work in this area relates to national forecasts; although, confidence intervals have been developed for state and county forecasts. A few studies have examined subcounty forecast error, however, they only measured point estimates of error. This paper describes a technique for making subcounty population forecasts and for generating confidence intervals around their forecast error. It also develops statistical equations for calculating point estimates and confidence intervals for areas with different population sizes. A non-linear, inverse relationship between population size and forecast accuracy was found and we demonstrate the ability to accurately predict average forecast error and confidence intervals based on this relationship.     

Stability over time in the distribution of population forecast errors

A number of studies in recent years have investigated empirical approaches to the production of confidence intervals for population projections. The critical assumption underlying these approaches is that the distribution of forecast errors remains stable over time. In this article, we evaluate this assumption by making population projections for states for a number of time periods during the 20th century, comparing these projections with census enumerations to determine forecast errors, and analyzing the stability of the resulting error distributions over time. These data are then used to construct and test empirical confidence limits. We find that in this sample the distribution of absolute percentage errors remained relatively stable over time and data on past forecast errors provided very useful predictions of future forecast errors.     

Population forecast accuracy: does the choice of summary measure of error matter?

Population projections are judged primarily by their accuracy. The most commonly used measure for the precision component of accuracy is the mean absolute percent error (MAPE). Recently, the MAPE has been criticized for overstating forecast error and other error measures have been proposed. This study compares the MAPE with two alternative measures of forecast error, the Median APE and an M-estimator. In addition, the paper also investigates forecast bias. The analysis extends previous studies of forecast error by examining a wide range of trend extrapolation techniques using a dataset that spans a century for a large sample of counties in the US. The main objective is to determine whether the choice of summary measure of error makes a difference from a practitioner’s standpoint. The paper finds that the MAPE indeed produces error values that exceed the robust measures. However, except for situations where extreme outliers rendered the MAPE meaningless, and which are rare in real world applications, there was not a single instance where using an alternative summary measure of error would have led to a fundamentally different evaluation of the projections. Moreover, where differences existed, it was not always clear that the values and patterns provided by the robust measures were necessarily more correct than those obtained with the MAPE. While research into refinements and alternatives to the MAPE and mean algebraic percent error are worthwhile, consideration of additional evaluation procedures that go beyond a single criterion might provide more benefits to producers and users of population forecasts.

Lessons from stochastic small-area population projections: the case of Waikato subregions in New Zealand

Subnational population projections in New Zealand by means of the conventional deterministic cohort-component method have had a tendency to be conservative: underprojecting fast-growing populations and overprojecting slow-growing ones. In this paper we use a stochastic population projection method as an alternative. We generate population projections for five demographically distinct administrative areas within the Waikato region of New Zealand: Hamilton City, Franklin District, Thames-Coromandel District, Otorohanga District and South Waikato District. The results are compared to official subnational deterministic projections. The accuracy of subnational population projections in New Zealand is strongly affected by the instability of migration as a component of population change. Differently from the standard cohort-component method, in which net migration levels are projected, the key parameters of our method are age-gender-area specific probabilistic net migration rates. Generally, the identified and modelled uncertainty makes the traditional ‘mid-range’ scenario of subnational deterministic projections of limited use for policy analysis or planning beyond a relatively short projection horizon. We find that the projected range of rates of population growth is wider for smaller regions and/or regions more strongly affected by net migration. Directions for further development of the methodology are suggested.     

Consistent correction of data for aboriginal populations

A consistent correction procedure is used to determine improved, consistent estimates by sex of census age distributions, intercensal births, intercensal deaths and net migration by age for the Aboriginal populations of the Northern Territory, South Australia and Western Australia during the period 1986–91. Undercount estimates and life tables show the Aboriginal populations to have lower coverage in statistical collections and much higher death risks than the total Australian population. Inter-regional net migration estimates show that component of change can no longer be ignored.     

On the validity of MAPE as a measure of population forecast accuracy

The mean absolute percent error (MAPE) is the summary measure most often used for evaluating the accuracy of population forecasts. While MAPE has many desirable criteria, we argue from both normative and relative standpoints that the widespread practice of exclusively using it for evaluating population forecasts should be changed. Normatively, we argue that MAPE does not meet the criterion of validity because as a summary measure it overstates the error found in a population forecast. We base this argument on logical grounds and support it empirically, using a sample of population forecasts for counties. From a relative standpoint, we examine two alternatives to MAPE, both sharing with it, the important conceptual feature of using most of the information about error. These alternatives are symmetrical MAPE (SMAPE) and a class of measures known as M-estimators. The empirical evaluation suggests M-estimators do not overstate forecast error as much as either MAPE or SMAPE and are, therefore, more valid measures of accuracy. We consequently recommend incorporating M-estimators into the evaluation toolkit. Because M-estimators do not meet the desired criterion of interpretative ease as well as MAPE, we also suggest another approach that focuses on nonlinear transformations of the error distribution.     

Some fundamental problems in population projection]

When family planning work in China developed, during the 1970's, the work of population projection also expanded. Population projections were done for China and its regions beginning in 1974 and remains a relatively new activity. Some question its validity, while others speculate about its methods and beleive only higher mathematics can be used, but this is all due to a lack of understanding of the nature of population forecasting. It is possible to predict population because if a current population situation and its changes are known, population of a particular future period can be projected e.g. for each year that is lived, a person will be 1 year older. And, population changes are primarily based on changes in births and deaths. These changes in turn are influenced by social and economic factors. Population projection is basically a forecasting of a certain period's total population, age and sex structure, the number of births and deaths, and migration. Different methods and formulas can be used to measure different population indicators, but all methods utilize comparisons. There are basically 2 methods for projecting total population: 1) the "direct method" regards total population as a quantity that itself changes and 2) the "separate factor method" breaks down total population into births, deaths, and migration. In the past, population projection has focused on the natural development of population which can be called "uncontrolled" because it makes "passive measurements" of possible population developments. In China, however, population projection is "controlled." Although it too measures future population developments, China's projections are not based on natural developments, but on definite population policies and estimates of results of family planning efforts.     

Diverging Demography: Hispanic and Non-Hispanic Contributions to U.S. Population Redistribution and Diversity

The substantial growth and geographic dispersion of Hispanics is among the most important demographic trends in recent U.S. demographic history. Our county-level study examines how widespread Hispanic natural increase and net migration has combined with the demographic change among non-Hispanics to produce an increasingly diverse population. This paper uses U.S. Census Bureau data and special tabulations of race/ethnic specific births and deaths from NCHS to highlight the demographic role of Hispanics as an engine of new county population growth and ethnoracial diversity across the U.S. landscape. It highlights key demographic processes—natural increase and net migration—that accounted for 1990–2010 changes in the absolute and relative sizes of the Hispanic and non-Hispanic populations. Hispanics accounted for the majority of all U.S. population growth between 2000 and 2010. Yet, Hispanics represented only 16 % of the U.S. population in 2010. Most previous research has focused on Hispanic immigration; here, we examine how natural increase and net migration among both the Hispanic and non-Hispanic population contribute to the nation’s growing diversity. Indeed, the demographic impact of rapid Hispanic growth has been reinforced by minimal white population growth due to low fertility, fewer women of reproductive age and growing mortality among the aging white population America’s burgeoning Hispanic population has left a large demographic footprint that is magnified by low and declining fertility and increasing mortality among America’s aging non-Hispanic population.     

The Vital Statistics Method of Estimating Net Migration By Age Cohorts

The focus of this paper is the development and testing of a method of estimating deaths which occur during a decade to aging birth and death cohorts, so that it may be possible to estimate net migration by the vital statistics (VS) method for age cohorts. Until now the VS method has been used only in making estimates of total net migration.The results obtained by using the VS method for age cohorts show that (1) the average census survival rate (CSR) method generally yields algebraically lower estimates of net migration than does the VS method; but (2) there are some striking exceptions which are apparently associated with errors in census enumeration by age, sex, and color. Comparisons between the average CSR and the VS methods are shown, by age, for both the North Carolina and the coterminous United States populations.A cursory examination of these comparisons suggests that the exclusive use of the VS method in estimating net migration for age cohorts may lead to substantial error. Finally, the magnitude of these errors in estimating net migration, as well as in census enumeration, can be roughly approximated if it is assumed that the use of the CSR method yields reasonably accurate estimates of net migration.     

Estimation of local demographic variation in a flexible framework for population projections

A cohort component projection of local populations based on sex and single year of age offers great value for planning local services, but demands data beyond the detail available. Local fertility, mortality and migration schedules by age and sex must be estimated sensitively to local variation if the results are to be of greater value than simpler methods of projection. Two approaches are compared, using data for the recent past: (a) direct estimation of local area age-specific schedules of fertility, mortality and migration based on data available to the national statistical agency; (b) graduation of national schedules using only local area population estimates by age, total numbers of births, and total numbers of deaths; age-specific migration is indirectly estimated from successive population estimates. These two approaches are compared with a projection using the same rates for each area. The three projections have been implemented for electoral wards in the Fife local government area of Scotland, using the flexible framework provided by POPGROUP software. Persuasive local population projections based on standard data for standard areas are feasible without the regular publication of migration flows.