An evaluation of the American Community Survey: results from the Oregon test site

The American Community Survey (ACS) is a Census Bureau product designed to provide accurate and timely demographic and economic indicators on an annual basis for both large and small geographic areas within the United States. Operational plans for Census 2010 call for ACS to replace the decennial census long form (Census LF), pending the results of evaluation studies. This plan represents a major change in that variables that traditionally have been collected on a “snapshot” basis once every 10 years would be collected on a “rolling” annual basis. Using a loss function analysis and other tools, this paper reports preliminary findings from a comparison of ACS and Census 2000 results in Multnomah County, Oregon, one of five national “local expert” test sites set up to compare ACS data collected at the time of Census 2000. The preliminary findings suggest that there are notable differences between some of the corresponding variables found in the ACS and Census LF that require more detailed examination. For example, the loss function analysis reveals notable differences for race and disability variables. In other comparisons of corresponding variables between ACS and Census 2000, differences are found within each of the four major areas of interest: (1) demographic characteristics, (2) social characteristics, (3) economic characteristics, and (4) housing characteristics, with housing characteristics showing the least similarity overall. These results also suggest that more detailed examinations are needed to understand differences between corresponding variables collected by ACS and the Census LF.     

Moving from a decennial census to a continuous measurement survey: factors affecting nonresponse at the neighborhood level

The Census Bureau is testing a continuous measurement program, known as the American Community Survey (ACS), which will provide census “long form” data annually, though with slightly higher levels of sampling variability. This paper focuses on the 1999–2001 ACS in the Bronx, 1 of 31 ACS test sites. It examines whether the quality of ACS data in the Bronx varies across neighborhoods, focusing specifically on how neighborhood sociodemographic factors influence nonresponse, as measured by mail return and allocation rates. It also examines whether these neighborhood factors have a differential impact on nonresponse in the ACS and the 2000 decennial census, and discusses reasons why this may be so.The ACS mail return rates are not only lower than those of the census, but are highly sensitive to race and socioeconomic distress. Despite this initial disadvantage, the ACS has lower levels of allocation on key variables, relative to the 2000 Census. Moreover, the effect of neighborhood socioeconomic distress on allocation rates in the ACS was minimal, compared to its effect on census allocation. We find that the overall quality of ACS data in the Bronx is superior to that of the decennial census. Our analysis of Bronx data suggests that the proposed elimination of the decennial long form and its replacement with the ACS is a reasonable tradeoff for users of small area data.     

The American Community Survey in counties with “seasonal” populations

The U.S. Census Bureau designed the American Community Survey (ACS) to provide annual estimates of social and economic characteristics for states, counties, municipalities, census tracts, and block groups. Because of its April 1 reference date, in northern nonmetropolitan counties with substantial seasonal population fluctuations the decennial census provides a statistical representation of the demographic and social characteristics of the population at a time when the population is close to its annual minimum. The year-round monthly ACS sample survey has the potential to provide local communities with an unprecedented understanding of the average population characteristics over the course of a year. In the future, the ACS even has the potential for providing social and economic characteristics of the population by season. This paper examines four ACS pilot data collection counties, Oneida and Vilas Counties in northern Wisconsin, and Lake and Flathead Counties in northwest Montana. We hypothesize that the ACS will reflect a resident population over the course of the year that is different from the traditional April 1 decennial census population. While the ACS holds much promise, our research uncovered some sampling problems that are not yet fully resolved. In addition, our analysis was not able to examine ACS estimates for minor civil divisions (MCDs), which are functioning governmental units in many states. The fact that these MCDs often have very small populations, together with the fact that estimated standard errors at the much larger census tract level in these counties are disconcertingly large, raises (currently unanswerable) questions concerning the eventual statistical quality of ACS estimates for small MCDs. Consequently, the adequacy of the ACS as a replacement for the census long form may depend on the ability of the Census Bureau to effectively address the concerns presented in this analysis.     

The American Community Survey: general commentary on the findings from external evaluations

As part of the Census Bureau’s effort to prove the operational feasibility of the American Community Survey (ACS), the Bureau contracted with local experts to conduct comparative analyses of the 1999–2001 ACS estimates with the 2000 Census data for various test counties. One of the goals of the ACS is to replace the decennial census long form. The resulting research papers analyzed various quality measures (response rates, allocation rates, and so on), conceptual differences in the two instruments, and statistical reliability in an effort to add to our assessment of the ACS as an adequate replacement instrument for collecting long form data. This paper discusses the results of these four research efforts and presents conclusions and recommendations for further Census Bureau action and research. The quality of traditional long form data, the importance of accurate population estimates and an accurate Master Address File (MAF), and continued research on the quality of small area data are discussed.     

Population Trends in Metropolitan and Nonmetropolitan America: Selective Deconcentration and the Rural Rebound

This study uses revised annual population estimates that incorporate adjustments from the 2000 Census to backcast demographic change for U.S. counties during the 1990s. These data are supplemented with new post-censal population estimates for 2001–2003. We use these data to examine demographic trends in the late 1990s and first years of the new century. Our findings are consistent with a model suggesting that a selective deconcentration of the U.S. population is underway. Our findings also confirm the occurrence of the rural rebound in the first half of the 1990s and a waning of this rebound in the late 1990s. Post-censal data also suggest a modest upturn in nonmetropolitan population growth rates in 2001–2003.     

Residential expansion as a continental threat to U.S. coastal ecosystems

Spatially extensive analysis of satellite, climate, and census data reveals human-environment interactions of regional or continental concern in the United States. A grid-based principal components analysis of Bureau of Census variables revealed two independent demographic phenomena, α-settlement reflecting traditional human settlement patterns and β-settlement describing relative population growth correlated with recent construction in non-agricultural areas, notably in coastal, desert, and “recreational” counties and around expanding metropolitan areas. Regression tree analysis showed that β-settlement was differentially associated with five distinct combinations of seasonality, summer heat or cool, intensity of agriculture, and extent of “barren” land. Beta-settlement was greatest in coastal and desert areas, and coincided with national concentrations of threatened and endangered species.     

Coverage of the population in Census 2000: Results from Demographic Analysis

Since the 1960 Census, Demographic Analysis (DA) has been used by the Census Bureau to evaluate the coverage of the population. Administrative statistics on births, deaths, immigration and Medicare enrollments as well as estimates of legal emigration and net undocumented immigration are used to produce demographic analysis estimates of the population for the census date. These estimates are compared to the Census 2000 data to evaluate coverage in the census. The results are also compared to measures of undercount obtained from dual system estimation. The DA measures substantial reduction in net undercount in Census 2000 compared to 1990. The reductions occur among all demographic categories: all broad age groups, males and females, Blacks and Non-Blacks.     

Reconstructing the size of the African American population by age and sex, 1930–1990

We estimate the size of the African American population in five-year age groups at census dates from 1930 to 1990 using a three-part strategy. For cohorts born after 1935, we follow the U.S. Census Bureau in using classical demographic analysis. To estimate the size of cohorts born before 1895, we use extinct-generation estimates. For remaining cohorts, we implement an age/period/cohort model of census counts. All approaches are applied to a data set in which the age distribution of deaths has been corrected for age misreporting. Results provide strong confirmation of the basic validity of Census Bureau estimates of census undercounts for African Americans while extending estimates to new cohorts and periods. Our estimates are less consistent with an historical series prepared by Coale and Rives (1973).     

An Application of Bayesian Methods to Small Area Poverty Rate Estimates

Efforts to estimate various sociodemographic variables in small geographic areas are proving difficult with the replacement of the Census long-form with the American Community Survey (ACS). Researchers interested in subnational demographic processes have previously relied on Census 2000 long-form data products in order to answer research questions. ACS data products promise to begin providing up-to-date profiles of the nation’s population and economy; however, unit- and item-level nonresponse in the ACS have left researchers with gaps in subnational coverage resulting in unstable and unreliable estimates for basic demographic measures. Borrowing information from neighboring areas and across time with a spatiotemporal smoothing process based on Bayesian statistical methods, it is possible to generate more stable and accurate estimates of rates for geographic areas not represented in the ACS. This research evaluates this spatiotemporal smoothing process in its ability to derive estimates of poverty rates at the county level for the contiguous United States. These estimates are then compared to more traditional estimates produced by the US Census Bureau, and comparisons between the two methods of estimation are carried out to evaluate the practical application of this smoothing method. Our findings suggest that by using available data from the ACS only, we are able to recreate temporal and spatial patterns of poverty in US counties even in years where data are sparse. Results show that the Bayesian methodology strongly agrees with the estimates produced by the SAIPE program, even in years with little data. This methodology can be expanded to other demographic and socioeconomic data with ease.     

American Community Survey overview and the role of external evaluations

In 2010 the U.S. Census Bureau will achieve its goal of eliminating the long form sample from the decennial census and will produce its first set of five-year data products from the full sample American Community Survey (ACS). This paper provides an overview of the call for change that prompted the Census Bureau to pursue the development of a new approach to collecting socioeconomic and housing data. The paper details the evolution of the ACS from its earliest origins to its current design and describes that design in detail. The current design has benefited from external debate and consultation. Work such as that described later in this journal exemplifies the key role that external users and advisors have played, and will continue to play, in the evolution of the ACS. Over the past 10 years, the Census Bureau has undertaken research and testing to demonstrate operational feasibility and to assess survey quality. Research has also compared ACS and Census 2000 data. ACS staff are involved in survey improvement efforts and continue to confront survey challenges. In the next few years the ACS will give priority to developing user tools to aid all users in the correct interpretation of multi-year estimates. The ultimate validation of the ACS is, however, in the hands of users. Continued input from the people who are responsible for administering and evaluating programs, identifying local needs, and planning for the future will allow the ACS to grow in value and utility.     

A Comparative Evaluation of Error and Bias in Census Tract-Level Age/Sex-Specific Population Estimates: Component I (Net-Migration) vs Component III (Hamilton–Perry)

While the housing-unit method continues to be the preferred method nationwide for producing small-area population estimates, this procedures lacks a method for making age/sex-specific estimates. This paper reports evaluation research on implementation of component-based methods for estimating census tract populations with age/sex detail. Two alternatives are explored: (1) the Component I method relying upon estimates of births, deaths, and net-migration and (2) the Component III method relying solely upon 1990 and 2000 Census counts. From an April 1, 2000 base, each method is used to make estimates moving forward to an April 1, 2010 estimate that is compared to the results of the 2010 Census. The two methods are compared in terms of accuracy and bias using both absolute and algebraic mean and median percentage errors. Results are reviewed and discussed in light of their implications for applied demographers tasked with making small-area demographic estimates.     

Comparison of Census 2000 and American Community Survey 1999–2001 Estimates: San Francisco and Tulare Counties, California

Data from the United States 2000 decennial census long form sample is compared to the U.S. Census Bureau’s fledgling American Community Survey (ACS) that was designed to replace the census long form in 2010. This article concentrates on two California counties, San Francisco and Tulare, which were part of the demonstration phase of the ACS. These counties are described and an overall comparison of the demographic, social, economic, and housing variables is presented. The project data and measures of census and survey quality such as self-response rates and nonresponse rates are displayed and discussed. Differences in the census and survey results are noted in the context of statistically significant and meaningful differences. Finally, strategies for analyzing and using ACS data are suggested.     

The business of demographics

The emergence of "demographics" in the past 15 years is a vital tool for American business research and planning. Tracing demographic trends became important for businesses when traditional consumer markets splintered with the enormous changes since the 1960s in US population growth, age structure, geographic distribution, income, education, living arrangements, and life-styles. The mass of reliable, small-area demographic data needed for market estimates and projections became available with the electronic census--public release of Census Bureau census and survey data on computer tape, beginning with the 1970 census. Census Bureau tapes as well as printed reports and microfiche are now widely accessible at low cost through summary tape processing centers designated by the bureau and its 12 regional offices and State Data Center Program. Data accessibility, plummeting computer costs, and businessess' unfamiliarity with demographics spawned the private data industry. By 1984, 70 private companies were offering demographic services to business clients--customized information repackaged from public data or drawn from proprietary data bases created from such data. Critics protest the for-profit use of public data by companies able to afford expensive mainframe computer technology. Business people defend their rights to public data as taxpaying ceitzens, but they must ensure that the data are indeed used for the public good. They must also question the quality of demographic data generated by private companies. Business' demographic expertise will improve when business schools offer training in demography, as few now do, though 40 of 88 graduate-level demographic programs now include business-oriented courses. Lower cost, easier access to business demographics is growing as more census data become available on microcomputer diskettes and through on-line linkages with large data bases--from private data companies and the Census Bureau itself. A directory of private and public demographic resources is appended, including forecasting, consulting and research services available.     

An overview of demographic analysis as a method for evaluating census coverage in the United States

"Since the 1950 U.S. census, demographic methods based on the fundamental balancing equation of demography have played an important role in the evaluation of the census net undercount. Application of this set of methods, called demographic analysis, results in national estimates of the net undercount for age-race-sex groups. Although results of demographic analysis are readily available in Bureau of the Census publications, the procedures used to estimate each of the components of population change are less well-known. In this paper we review the historical foundation of demographic analysis, beginning with Coale's 1950 census evaluation project and concluding with the recent evaluation of the 1990 census. We examine each of the components of the method, how their estimation has changed over time, and how they were estimated for the 1990 census."     

Integrating Census Data to Support a Motion for Change of Venue

This applied demography case study illustrates the practical application of demographic concepts and methods to an issue facing the court. We show how census data can be used to support a legal motion for a change of venue. “Change of venue,” the legal term for moving a trial to a new location, usually is sought to avoid prejudice against one of the parties to a lawsuit. The case study will interest demographic practitioners, and it can be used as an instructional case in teaching applied demography: students can replicate it using data for any particular pair of populous metropolitan communities. By doing so, students would gain proficiency working with Public Use Microdata Sample (PUMS) household records—and the person records within household records—to identify and categorize family and nonfamily relationships among household members, and practical experience translating legal issues into questions that can be answered empirically using American Community Survey (ACS) data.     

Demographic comparison between self-response and personal visit interview in Census 2000

During Census 2000, over 95 percent of the housing units were in mailback areas. In these areas either the United States Postal Service or Census Bureau staff delivered the census questionnaire. In urban and suburban areas of the country, the United States Postal Service delivered the census questionnaires between March 13 and 15 of 2000. The addresses in these areas are predominately city style; house number and street name. In more rural areas of the country with predominately non-city style addresses, Census Bureau staff delivered the questionnaire during March of 2000. Respondents completed their census questionnaires and returned them through the mail. Returns from these housing units are classified as self-response. Respondents which did not complete and return their census questionnaire by April 18, 2000 were interviewed during the nonresponse followup operation (personal visit interview). This paper will examine the demographic characteristics of persons enumerated on the mail return questionnaire (self-response) and the persons enumerated during the nonresponse followup operation (personal visit interview).     

“Lingering effects” of discrimination: tracing persistence over time in local populations

The broad principle that historical injustice may call for corrective remedies in today’s world poses new and interesting challenges for applied demographers. I illustrate applications of demographic analysis to examine how former policies and practices produced effects that persist (or linger on) among members of a contemporary population. Such effects involve populations at different times and places and posit causal mechanisms that can be examined and evaluated. Applying standard demographic concepts and thinking to these issues can clarify and sharpen public understanding of whether past experiences still matter and precisely for whom, and whether proposed corrective remedies under the law are feasible.Revision of paper presented at the 2003 meetings of the Population Association of America, Session 1103, “Beyond Basics: Estimating and Projecting Characteristics Other Than Age, Sex, and Race.”     

The Contemporary Marriage and Fertility Revolutions in the West the Explanations Provided by Australian Participants

The last half century has witnessed both in Australia, and elsewhere in the West, first a “baby boom” (and marriage boom) and then a “baby bust” (and marriage bust). Economists, demographers, sociologists and social historians, especially in America, have developed theories to explain these phenomena. Presented to the Annual Meeting of the Population Association of America, New Orleans, 21–23 April 1988.     

Selective Out-Migration from Florida

I test if selective out-migration of unhealthy seniors explains why disability rates are so much lower for Florida, as compared to the national average. This particular area of research is timely given the significant demographic changes relating to aging. Moreover, this study contributes to the body of literature examining migration with respect to disability and widowhood. Using State Federal Information Processing Standard (FIPS) and Public Use Microdata Areas (PUMA), I create national maps showing disability rates for the following age-groups: 50–59, 60–69, and 70+. After creating maps in ARCGIS and conducting univariate and clustering analysis on mobility disability and personal care limitation, I employ multinomial logit (MNL) analysis to test if individuals with disability are more likely to out-migrate from Florida. The regression analyses lend support to the relaxed Litwak and Longino (The Gerontologist, 27(3): 266–272, 1987) second-move hypothesis, which claims individuals with progressively worse health are more likely to undertake another move to be closer to family and friends. I state “relaxed” because the data does not allow one to determine the reason for migration—only that migration occurred during the past year. This research informs policy-makers to recognize that elderly in better health may migrate to places such as Arizona and Florida due to amenity-seeking behavior, but unhealthy elderly are more likely to leave these states due to assistance-seeking behavior. This out-migration can place excess demand on health services for the incoming regions, which requires state and local government to ensure resources are in place. Also noteworthy, my results are less likely to be flawed by erroneous age and sex data in the public use microdata samples (IPUMS) since I stack the 2006 and 2007 American Community Survey (ACS). A recent working studies by Alexander et al. (Inaccurate age and sex data in the Census PUMS files: Evidence and implications. Munich: CESifo, 2010) shows inaccuracies in the IPUMS for the 1 and 5% 2000 Census, the 2003–2006 ACS, the 2005–2007 3-year ACS, and the 2004–2009 current population survey (CPS) files.     

Research Note: What Counts as a House? Comparing 2010 Census Counts and Administrative Records

As a tool for fulfilling data needs for small area (subcounty) analysis, demographers are increasingly turning to administrative records such as building permits and tax assessor records as a source of data for use in producing small area demographic estimates. While Census counts are considered to be the “gold standard,” administrative records provide a fine level of spatial detail and a valuable source of information for intercensal years. However, there is uncertainty about the quality of administrative records data for use in estimation. This analysis builds upon earlier research by comparing administrative records-based housing unit estimates developed during the 2000s decade with housing counts from the 2010 Census in San Diego County. Results show that both administrative records and Census counts have strengths and weaknesses that should be understood by the data user.