Theory & Methods: Residual Diagnostic Plots for Checking for Model Mis‐specification in Time Series Regression

This paper considers residuals for time series regression. Despite much literature on visual diagnostics for uncorrelated data, there is little on the autocorrelated case. To examine various aspects of the fitted time series regression model, three residuals are considered. The fitted regression model can be checked using orthogonal residuals; the time series error model can be analysed using marginal residuals; and the white noise error component can be tested using conditional residuals. When used together, these residuals allow identification of outliers, model mis‐specification and mean shifts. Due to the sensitivity of conditional residuals to model mis‐specification, it is suggested that the orthogonal and marginal residuals be examined first.     

Model Diagnostics for Remote Access Regression Servers

To protect public-use microdata, one approach is not to allow users access to the microdata. Instead, users submit analyses to a remote computer that reports back basic output from the fitted model, such as coefficients and standard errors. To be most useful, this remote server also should provide some way for users to check the fit of their models, without disclosing actual data values. This paper discusses regression diagnostics for remote servers. The proposal is to release synthetic diagnostics—i.e. simulated values of residuals and dependent and independent variables–constructed to mimic the relationships among the real-data residuals and independent variables. Using simulations, it is shown that the proposed synthetic diagnostics can reveal model inadequacies without substantial increase in the risk of disclosures. This approach also can be used to develop remote server diagnostics for generalized linear models.     

Variance Estimation in Spatial Regression Using a Non-parametric Semivariogram Based on Residuals

Abstract.  The empirical semivariogram of residuals from a regression model with stationary errors may be used to estimate the covariance structure of the underlying process. For prediction (kriging) the bias of the semivariogram estimate induced by using residuals instead of errors has only a minor effect because the bias is small for small lags. However, for estimating the variance of estimated regression coefficients and of predictions, the bias due to using residuals can be quite substantial. Thus we propose a method for reducing this bias. The adjusted empirical semivariogram is then isotonized and made conditionally negative-definite and used to estimate the variance of estimated regression coefficients in a general estimating equations setup. Simulation results for least squares and robust regression show that the proposed method works well in linear models with stationary correlated errors.     

Reviews and Notices

Abstract

Longstaff's Studies in Statistics: Studies in Statistics. Social, Political, and Medical. By George Blundell Longstaff, M.A., M.B., etc. London: Edward Stanford. 1891. 8vo. Pp. 455. Maps and diagrams. Reviewed by S. W. Abbott.

Keynes on Statistics: The Scope and Method of Political Economy. By John Neville Keynes, M.A., London. Macmillan and Company. 1891. Pp. xiv, 359. Reviewed by Davis R. Dewey.

United States Census Bulletins: 1No. 39. March 16, 1891. Wealth and Resources of Alaska. By Ivan Petroff. Pp. 15.

United States Census Bulletins: No. 40. March 17. Population by Counties, North Central Division. Pp. 9.

United States Census Bulletins: No. 41. March 19. Agriculture, Truck Farming. By J. H. Hale. Pp. 12.

United States Census Bulletins: No. 42. March 20. Population by Counties, South Central and Western Divisions. Pp. 9.

United States Census Bulletins: No. 43. March 21. Coal Product West of the Mississippi River. By John H. Jones. Pp. 8.

United States Census Bulletins: No. 44. March 25. Distribution of Population in Accordance with Mean Relative Humidity of the Atmosphere. By Henry Gannett. Pp. 3.

United States Census Bulletins: No. 45. March 26. Granite. By William C. Day. Pp. 41.

United States Census Bulletins: No. 46. March 27. Railway Statistics of the New England States. By Henry C. Adams. Pp. 18.

United States Census Bulletins: No. 47. March 28. Distribution of Population by Drainage Basins. By Henry Gannett. Pp. 5.

United States Census Bulletins: No. 48. April 7. The White and Colored Population of the South. 1890. Pp. 27.

United States Census Bulletins: No. 49. April 14. Precious and Ornamental Stones and Diamond Cutting. By George Frederick Kunz. Pp. 8.

United States Census Bulletins: No. 50. April 15. Population of Rhode Island by Minor Civil Divisions. Pp. 3.

United States Census Bulletins: No. 51. April 16. Population of Vermont by Minor Civil Divisions. Pp. 4.

United States Census Bulletins: No. 52. April 17. Urban Population in 1890. Cities Containing 8000 Inhabitants or more. Pp. 9.

United States Census Bulletins: No. 53. April 20. Statistics of Education. Alaska, Arkansas, Delaware, Missouri, Iowa, Michigan, Minnesota, Mississippi, New Mexico, New York, North Dakota, Oregon, Texas, Utah, Washington, West Virginia, and forty-two Cities. By James H. Blodgett. Pp. 34.

United States Census Bulletins: No. 54. April 23. Public School Finances. By J. K. Upton. Pp. 12.

United States Census Bulletins: No. 55. April 24. The Relative Economy of Cable, Electric, and Animal Motive Power for Street Railways. By Charles H. Cooley. Pp. 17.

United States Census Bulletins: No. 56. April 25. Population of Maine by Minor Civil Divisions. Pp. 7.

United States Census Bulletins: No. 57. April 27. Population of Delaware by Minor Civil Divisions. Pp. 3.

United States Census Bulletins: No. 58. April 28. Population of Connecticut by Minor Civil Divisions. Pp. 3.

United States Census Bulletins: No. 59. April 29. Commercial Floriculture. By J. H. Hale. Pp. 11.

United States Census Bulletins: No. 60. April 30. Irrigation in New Mexico. By F. H. Newell. Pp. 14.

United States Census Bulletins: No. 61. May 8. The Production of Mica. By L. J. Childs. Pp. 6.

United States Census Bulletins: No. 62. May 9. Asylums for the Insane in the United States. By Dr. John S. Billings and W. H. Olcott. Pp. 32.

United States Census Bulletins: No. 63. May 11. Distribution of Population in Accordance with Latitude and Longitude. With diagrams. By Henry Gannett. Pp. 7.

United States Census Bulletins: No. 64. May 12. Foreign, National, State, and County Indebtedness. By J. K. Upton. Pp. 52.

United States Census Bulletins: No. 65. May 13. Distribution of Population in Accordance with Topographic Features. By Henry Gannett. Pp. 7.

United States Census Bulletins: No. 66. May 14. Floating Equipment on the Great Lakes. By Henry C. Adams. Pp. 11.

United States Census Bulletins: No. 68. May 16. Production of Manganese Ores. By Joseph D. Weeks. Pp. 5.

United States Census Bulletins: Mo. 69. May 18. Population of New Jersey by Minor Civil Divisions. Pp. 6.

United States Census Bulletins: No. 70. May 22. Statistics of Churches. By Henry K. Carroll. Pp. 27.

United States Census Bulletins: No. 71. May 23. Production of Bluestone. By William C. Day. Pp. 6.

Congress of Demography.

Old Age and Pauperism in England.

Mortality of English Clergymen.

Fire Statistics.     

Deletion diagnostics for marginal mean and correlation model parameters in estimating equations

Regression diagnostics are introduced for parameters in marginal association models for clustered binary outcomes in an implementation of generalized estimating equations. Estimating equations for intracluster correlations facilitate computational formulae for one-step deletion diagnostics in an extension of earlier work on diagnostics for parameters in the marginal mean model. The proposed diagnostics measure the influence of an observation or a cluster of observations on the estimated regression parameters and on the overall fit of the model. The diagnostics are applied to data from four research studies from public health and medicine.     

Nonparametric Nonlinear Cotrending Analysis,With an Application to Interest and Inflation in the United States

Given the assumption that the components of a vector time series are stationary around nonlinear deterministic time trends, nonlinear cotrending is the phenomenon that one or more linear combinations of the time series are stationary around a linear trend or a constant; hence, the series have common nonlinear deterministic time trends. In this article, I develop nonparametric tests for nonlinear cotrending, and I derive nonparametric estimators of the cotrending vectors. I apply this approach to the federal funds rate and the consumer price index inflation rate in the United States, using monthly data, to analyze the price puzzle.     

Residuals for the linear model with general covariance structure

A general theory is presented for residuals from the general linear model with correlated errors. It is demonstrated that there are two fundamental types of residual associated with this model, referred to here as the marginal and the conditional residual. These measure respectively the distance to the global aspects of the model as represented by the expected value and the local aspects as represented by the conditional expected value. These residuals may be multivariate. Some important dualities are developed which have simple implications for diagnostics. The results are illustrated by reference to model diagnostics in time series and in classical multivariate analysis with independent cases.     

Bayes Estimates for the Price and Income Elasticities of Alcoholic Beverages in Finland From 1955 to 1980

The Lindley–Smith theory of Bayes estimates for multiple regression equations with exchangeability between the regression coefficients of the individual equations is extended to the case in which a first-order autoregressive process generates the regression coefficients. The ensuing formulas are applied to study monthly Finnish consumption of alcohol. The point of this application is that exchangeability between the regression coefficients is less than we can assess beforehand when there is a natural ordering, in this case according to chronological time, of the equations. Still, the general task of the Lindley–Smith estimators, to consider the combined data when estimating individual regression coefficients, is a relevant one.     

Bootstrapping regression models with BLUS residuals

To bootstrap a regression problem, pairs of response and explanatory variables or residuals can be resam‐pled, according to whether we believe that the explanatory variables are random or fixed. In the latter case, different residuals have been proposed in the literature, including the ordinary residuals (Efron 1979), standardized residuals (Bickel & Freedman 1983) and Studentized residuals (Weber 1984). Freedman (1981) has shown that the bootstrap from ordinary residuals is asymptotically valid when the number of cases increases and the number of variables is fixed. Bickel & Freedman (1983) have shown the asymptotic validity for ordinary residuals when the number of variables and the number of cases both increase, provided that the ratio of the two converges to zero at an appropriate rate. In this paper, the authors introduce the use of BLUS (Best Linear Unbiased with Scalar covariance matrix) residuals in bootstrapping regression models. The main advantage of the BLUS residuals, introduced in Theil (1965), is that they are uncorrelated. The main disadvantage is that only n —p residuals can be computed for a regression problem with n cases and p variables. The asymptotic results of Freedman (1981) and Bickel & Freedman (1983) for the ordinary (and standardized) residuals are generalized to the BLUS residuals. A small simulation study shows that even though only n — p residuals are available, in small samples bootstrapping BLUS residuals can be as good as, and sometimes better than, bootstrapping from standardized or Studentized residuals.     

Influence diagnostics in the Gamma regression model with adjusted deviance residuals

In this study, we develop the adjusted deviance residuals for the gamma regression model (GRM) by following Cordeiro's (2004) method. These adjusted deviance residuals under the GRM are used for influence diagnostics. A comparative analysis has been sorted out between our proposed method of the adjusted deviance residuals and an existing method for influence diagnostics. These results are illustrated by a simulation study and using a real data set. They are presented for different values of dispersion and sample sizes and indicate the significant role of the GRM inferences.     

A Multivariate Intervention Model for the Dutch Mint Circulation: Estimation and Monte Carlo Simulation

For each of the five Dutch coinage denominations, a transfer-function model is estimated. The output variables are monthly observations of coins in circulation. Two input variables represent transaction flows; all other inputs are step functions, representing the occurrence of interventions. Using the method of cross-correlating the residuals of the individual equations, a multivariate transfer-function model is constructed and estimated. Next Monte Carlo simulation is applied to derive expectations and variances of the yearly addition to the stock of coins until 1996. Our results enlighten on some aspects of a problem situation faced by the Dutch State Mint.     

Efficient inference for parameters of unobservable periodic autoregressive time series

This paper considers estimating the model coefficients when the observed periodic autoregressive time series is contaminated by a trend. The proposed Yule–Walker estimators are obtained by a two-step procedure. In the first step, the trend is estimated by a weighted local polynomial, and the residuals are obtained by subtracting the trend estimates from the observations; in the second step, the model coefficients are estimated by the well-known Yule–Walker method via the residuals. It is shown that under certain conditions such Yule–Walker estimators are oracally efficient, i.e., they are asymptotically equivalent to those obtained from periodic autoregressive time series without a trend. An easy-to-use implementation procedure is provided. The performance of the estimators is illustrated by simulation studies and real data analysis. In particular, the simulation studies show that the proposed estimator outperforms that obtained from the residuals when the trend is estimated by kernel smoothing without taking the heteroscedasticity into consideration.     

Regression diagnostics for removing an observation with animating graphics

Recently, Cook and Weisberg (1989) presented dynamic graphics for regression diagnostics. They suggested animating graphics which could aid to understanding the effects of adding a variable to a model. In this paper, using the Cook and Weisberg's idea of animation, we propose a dynamic graphical method for residuals to display the effects of removing an observation from a model. Based on the information obtained from these animating graphics, it is possible to see the influence of observations for regression diagnostics.     

Influence Diagnostics for Simultaneous Equations Models

This paper presents influence diagnostics for simultaneous equations models. It proposes residuals, leverage and other influence measures. A missing data method is adopted to minimize the masking effect due to case deletions. The assessment of local influence is also considered. The paper shows how to evaluate the effects that perturbations to the endogenous variables, predetermined variables and case weights may have on the parameter estimates. The diagnostics are illustrated with two examples.     

Using large data sets to forecast sectoral employment

We use several models using classical and Bayesian methods to forecast employment for eight sectors of the US economy. In addition to using standard vector-autoregressive and Bayesian vector autoregressive models, we also augment these models to include the information content of 143 additional monthly series in some models. Several approaches exist for incorporating information from a large number of series. We consider two multivariate approaches—extracting common factors (principal components) and Bayesian shrinkage. After extracting the common factors, we use Bayesian factor-augmented vector autoregressive and vector error-correction models, as well as Bayesian shrinkage in a large-scale Bayesian vector autoregressive models. For an in-sample period of January 1972 to December 1989 and an out-of-sample period of January 1990 to March 2010, we compare the forecast performance of the alternative models. More specifically, we perform ex-post and ex-ante out-of-sample forecasts from January 1990 through March 2009 and from April 2009 through March 2010, respectively. We find that factor augmented models, especially error-correction versions, generally prove the best in out-of-sample forecast performance, implying that in addition to macroeconomic variables, incorporating long-run relationships along with short-run dynamics play an important role in forecasting employment. Forecast combination models, however, based on the simple average forecasts of the various models used, outperform the best performing individual models for six of the eight sectoral employment series.     

A semiparametric approach for modelling multivariate nonlinear time series

In this article, a semiparametric time‐varying nonlinear vector autoregressive (NVAR) model is proposed to model nonlinear vector time series data. We consider a combination of parametric and nonparametric estimation approaches to estimate the NVAR function for both independent and dependent errors. We use the multivariate Taylor series expansion of the link function up to the second order which has a parametric framework as a representation of the nonlinear vector regression function. After the unknown parameters are estimated by the maximum likelihood estimation procedure, the obtained NVAR function is adjusted by a nonparametric diagonal matrix, where the proposed adjusted matrix is estimated by the nonparametric kernel estimator. The asymptotic consistency properties of the proposed estimators are established. Simulation studies are conducted to evaluate the performance of the proposed semiparametric method. A real data example on short‐run interest rates and long‐run interest rates of United States Treasury securities is analyzed to demonstrate the application of the proposed approach. The Canadian Journal of Statistics 47: 668–687; 2019 © 2019 Statistical Society of Canada     

Testing Monotonicity of Regression Functions – An Empirical Process Approach

We propose several new tests for monotonicity of regression functions based on different empirical processes of residuals and pseudo‐residuals. The residuals are obtained from an unconstrained kernel regression estimator whereas the pseudo‐residuals are obtained from an increasing regression estimator. Here, in particular, we consider a recently developed simple kernel‐based estimator for increasing regression functions based on increasing rearrangements of unconstrained non‐parametric estimators. The test statistics are estimated distance measures between the regression function and its increasing rearrangement. We discuss the asymptotic distributions, consistency and small sample performances of the tests.     

On detecting influential data and selecting regression variables

The analysis of residuals may reveal various functional forms suitable for the regression model. In this paper, we investigate some selection criteria for selecting important regression variables. In doing so, we use statistical selection and ranking procedures. Thus, we derive an appropriate criterion to measure the influence and bias for the reduced models. We show that the reduced models are based on some noncentrality parameters which provide a measure of goodness of fit for the fitted models. In this paper, we also discuss the relationships of influence diagnostics and the statistic proposed earlier by Gupta and Huang (J. Statist. Plann. Inference 20 (1988) 155–167). We introduce a new measure for detecting influential data as an alternative to Cook's measure.     

A Reexamination of Friedman's Consumption Puzzle

The different average and marginal consumption propensities estimated from time series data constitute a classic puzzle of the theory of consumption. This article argues that if consumption and income possess a common stochastic trend (and thus are cointegrated), both the average propensity to consume (APC) and the marginal propensity to consume (MPC) will be consistent but biased in small samples. Upon correcting for this small sample bias, the puzzling discrepancies between the APC and the MPC estimated using annual data for the United States from 1897 to 1949 become substantially smaller. This supports an alternative resolution of the puzzle based on the theory of cointegration.     

Nonstationarity and Level Shifts With an Application to Purchasing Power Parity

This study considers testing for a unit root in a time series characterized by a structural change in its mean. The analysis is in the spirit of Perron (1990a), who showed that the existence of such a shift in a stationary time series biases the usual tests for a unit root toward nonrejection. The approach is, however, different given that we suppose the date of the change to be unknown. The statistic of interest is then the minimal t statistic over all possible breakpoints in regressions similar to those proposed by Perron (1990a). Other related statistics are also discussed. We derive and tabulate the asymptotic distributions of interest. Most of the emphasis, however, is given to the tabulation of finite-sample critical values using simulation experiments. Particular attention is given to the effect, on the finite-sample critical values, of various procedures to select the appropriate order of the estimated autoregressions. We apply the tests to analyze the issue of purchasing power parity between the United States and the United Kingdom and also between the United States and Finland, whose real exchange rates are characterized by apparent shifts in level when using particular price indexes.