Penalized least squares smoothing of two-dimensional mortality tables with imposed smoothness

This paper presents a method to estimate mortality trends of two-dimensional mortality tables. Comparability of mortality trends for two or more of such tables is enhanced by applying penalized least squares and imposing a desired percentage of smoothness to be attained by the trends. The smoothing procedure is basically determined by the smoothing parameters that are related to the percentage of smoothness. To quantify smoothness, we employ an index defined first for the one-dimensional case and then generalized to the two-dimensional one. The proposed method is applied to data from member countries of the OECD. We establish as goal the smoothed mortality surface for one of those countries and compare it with some other mortality surfaces smoothed with the same percentage of two-dimensional smoothness. Our aim is to be able to see whether convergence exists in the mortality trends of the countries under study, in both year and age dimensions.     

Smoothing a Time Series by Segments of the Data Range

We consider the problem of estimating a trend with different amounts of smoothness for segments of a time series subjected to different variability regimes. We propose using an unobserved components model to consider the existence of at least two data segments. We first fix some desired percentages of smoothness for the trend segments and deduce the corresponding smoothing parameters involved. Once the size of each segment is chosen, the smoothing formulas here derived produce trend estimates for all segments with the desired smoothness as well as their corresponding estimated variances. Empirical examples from demography and economics illustrate our proposal.     

Effect of autocorrelation when estimating the trend of a time series via penalized least squares with controlled smoothness

This paper studies the effect of autocorrelation on the smoothness of the trend of a univariate time series estimated by means of penalized least squares. An index of smoothness is deduced for the case of a time series represented by a signal-plus-noise model, where the noise follows an autoregressive process of order one. This index is useful for measuring the distortion of the amount of smoothness by incorporating the effect of autocorrelation. Different autocorrelation values are used to appreciate the numerical effect on smoothness for estimated trends of time series with different sample sizes. For comparative purposes, several graphs of two simulated time series are presented, where the estimated trend is compared with and without autocorrelation in the noise. Some findings are as follows, on the one hand, when the autocorrelation is negative (no matter how large) or positive but small, the estimated trend gets very close to the true trend. Even in this case, the estimation is improved by fixing the index of smoothness according to the sample size. On the other hand, when the autocorrelation is positive and large the simulated and estimated trends lie far away from the true trend. This situation is mitigated by fixing an appropriate index of smoothness for the estimated trend in accordance to the sample size at hand. Finally, an empirical example serves to illustrate the use of the smoothness index when estimating the trend of Mexico’s quarterly GDP.     

Smoothing Time Series with Local Polynomial Regression on Time

Time series smoothers estimate the level of a time series at time t as its conditional expectation given present, past and future observations, with the smoothed value depending on the estimated time series model. Alternatively, local polynomial regressions on time can be used to estimate the level, with the implied smoothed value depending on the weight function and the bandwidth in the local linear least squares fit. In this article we compare the two smoothing approaches and describe their similarities. Through simulations, we assess the increase in the mean square error that results when approximating the estimated optimal time series smoother with the local regression estimate of the level.     

Estimated Generalized Least Squares for Random Coefficient Regression Models

ABSTRACT. This paper considers a general class of random coefficient regression (RCR) models to represent pooled cross-sectional and time series data. A new method is given to estimate the covariance matrix of the error component in these RCR models. Also, the asymptotic and small sample properties of the estimated generalized least squares estimator of the regression coefficient vector are established. Procedures for testing a linear restriction on the mean vector of the random coefficients are derived. Finally, a test for non-randomness in the RCR model is devised, and the asymptotic distribution of the test statistic is obtained.     

Smoothing spline ANOPOW

This paper is motivated by the pioneering work of Emanuel Parzen wherein he advanced the estimation of (spectral) densities via kernel smoothing and established the role of reproducing kernel Hilbert spaces (RKHS) in field of time series analysis. Here, we consider analysis of power (ANOPOW) for replicated time series collected in an experimental design where the main goals are to estimate, and to detect differences among, group spectra. To accomplish these goals, we obtain smooth estimators of the group spectra by assuming that each spectral density is in some RKHS; we then apply penalized least squares in a smoothing spline ANOPOW. For inference, we obtain simultaneous confidence intervals for the estimated group spectra via bootstrapping.     

Minimax Linear Smoothers

We consider the problem of estimating the mean of a multivariate distribution. As a general alternative to penalized least squares estimators, we consider minimax estimators for squared error over a restricted parameter space where the restriction is determined by the penalization term. For a quadratic penalty term, the minimax estimator among linear estimators can be found explicitly. It is shown that all symmetric linear smoothers with eigenvalues in the unit interval can be characterized as minimax linear estimators over a certain parameter space where the bias is bounded. The minimax linear estimator depends on smoothing parameters that must be estimated in practice. Using results in Kneip (1994), this can be done using Mallows' C _L -statistic and the resulting adaptive estimator is now asymptotically minimax linear. The minimax estimator is compared to the penalized least squares estimator both in finite samples and asymptotically.     

Trend smoothness achieved by penalized least squares with the smoothing parameter chosen by optimality criteria

This work presents a study about the smoothness attained by the methods more frequently used to choose the smoothing parameter in the context of splines: Cross Validation, Generalized Cross Validation, and corrected Akaike and Bayesian Information Criteria, implemented with Penalized Least Squares. It is concluded that the amount of smoothness strongly depends on the length of the series and on the type of underlying trend, while the presence of seasonality even though statistically significant is less relevant. The intrinsic variability of the series is not statistically significant and its effect is taken into account only through the smoothing parameter.     

Effects of the estimation of covariance matrix parameters in the generalized multivariate linear model

We Consider the generalized multivariate linear model and assume the covariance matrix of the p x 1 vector of responses on a given individual can be represented in the general linear structure form described by Anderson (1973). The effects of the use of estimates of the parameters of the covariance matrix on the generalized least squares estimator of the regression coefficients and on the prediction of a portion of a future vector, when only the first portion of the vector has been observed, are investigated. Approximations are derived for the covariance matrix of the generalized least squares estimator and for the mean square error matrix of the usual predictor, for the practical case where estimated parameters are used.     

Generalized least squares with misspecified serial correlation structures

Summary. The regression literature contains hundreds of studies on serially correlated disturbances. Most of these studies assume that the structure of the error covariance matrix Ω is known or can be estimated consistently from data. Surprisingly, few studies investigate the properties of estimated generalized least squares (GLS) procedures when the structure of Ω is incorrectly identified and the parameters are inefficiently estimated. We compare the finite sample efficiencies of ordinary least squares (OLS), GLS and incorrect GLS (IGLS) estimators. We also prove new theorems establishing theoretical efficiency bounds for IGLS relative to GLS and OLS. Results from an exhaustive simulation study are used to evaluate the finite sample performance and to demonstrate the robustness of IGLS estimates vis-à-vis OLS and GLS estimates constructed for models with known and estimated (but correctly identified) Ω. Some of our conclusions for finite samples differ from established asymptotic results.     

Parametric scalp mapping and inference via spatially smooth linear models for mismatch negativity studies

Mismatch negativity (MMN) is a neurophysiological tool that can be used to investigate various facets of comprehension. Subjects are presented with different stimuli to elicit the MMN response, which is derived from electroencephalography (EEG) signals recorded at electrodes across the brain. We propose a methodology to extend single electrode analyses of MMN data by generating smooth scalp maps of estimated experimental effects. It is shown that penalized least squares estimates of effect maps can be produced using a two step procedure involving (a) ANOVA at each electrode and (b) spatial smoothing across electrodes. A Fisher von-Mises kernel is used for smoothing scalp maps with cross-validated bandwidth selection. The methodology is applied to a case control study involving aphasics (language disordered individuals). Analysis of residuals shows possible heteroscedasticity and non-Gaussian tail behavior. For robust inference, a semiparametric multivariate approach is proposed to determine the significance of parametric maps. A variety of global and regional test statistics are developed to investigate the significance of spatial patterns in treatment effects. The methodology is seen to confirm previous findings from single electrode analysis and identifies some new significant spatial patterns of difference between controls and aphasics.     

Nonparametric smoothing using state space techniques

The authors examine the equivalence between penalized least squares and state space smoothing using random vectors with infinite variance. They show that despite infinite variance, many time series techniques for estimation, significance testing, and diagnostics can be used. The Kalman filter can be used to fit penalized least squares models, computing the smoothed quantities and related values. Infinite variance is equivalent to differencing to stationarity, and to adding explanatory variables. The authors examine constructs called “smoothations” which they show to be fundamental in smoothing. Applications illustrate concepts and methods.     

Estimation of parameters in a generalized GMANOVA model based on an outer product analogy and least squares

This paper investigates estimation of parameters in a combination of the multivariate linear model and growth curve model, called a generalized GMANOVA model. Making analogy between the outer product of data vectors and covariance yields an approach to directly do least squares to covariance. An outer product least squares estimator of covariance (COPLS estimator) is obtained and its distribution is presented if a normal assumption is imposed on the error matrix. Based on the COPLS estimator, two-stage generalized least squares estimators of the regression coefficients are derived. In addition, asymptotic normalities of these estimators are investigated. Simulation studies have shown that the COPLS estimator and two-stage GLS estimators are alternative competitors with more efficiency in the sense of sample mean, standard deviations and mean of the variance estimates to the existing ML estimator in finite samples. An example of application is also illustrated.     

Regression Shrinkage Methods and Autoregressive Time Series Prediction

The pros and cons of applying regression shrinkage prediction arguments and methods to autoregressive time series forecasting are discussed. Simulation evidence of the performance of a Stein regression prediction formula suggests that the overall dominance of the shrunken predictor over least squares in regression no longer holds in time series samples of a reasonable length. Rather, shrinkage appears the better of the two, with respect to prediction mean squared error, only for weaker relationships and seems to be inferior to the least squares predictor when the autoregressive relationship is strong.     

Estimation of linear models for field experiments

General mixed linear models for experiments conducted over a series of sltes and/or years are described. The ordinary least squares (OLS) estlmator is simple to compute, but is not the best unbiased estimator. Also, the usuaL formula for the varlance of the OLS estimator is not correct and seriously underestimates the true variance. The best linear unbiased estimator is the generalized least squares (GLS) estimator. However, t requires an inversion of the variance-covariance matrix V, whlch is usually of large dimension. Also, in practice, V is unknown.

We presented an estlmator [Vcirc] of the matrix V using the estimators of variance components [for sites, blocks (sites), etc.]. We also presented a simple transformation of the data, such that an ordinary least squares regression of the transformed data gives the estimated generalized least squares (EGLS) estimator. The standard errors obtained from the transformed regression serve as asymptotic standard errors of the EGLS estimators. We also established that the EGLS estlmator is unbiased.

An example of fitting a linear model to data for 18 sites (environments) located in Brazil is given. One of the site variables (soil test phosphorus) was measured by plot rather than by site and this established the need for a covariance model such as the one used rather than the usual analysis of variance model. It is for this variable that the resulting parameter estimates did not correspond well between the OLS and EGLS estimators. Regression statistics and the analysis of variance for the example are presented and summarized.     

The collected works of john w. tukey

We describe novel, analytical, data-analysis, and Monte-Carlo-simulation studies of strongly heteroscedastic data of both small and wide range.Many different types of heteroscedasticity and fixed or variable weighting are incorporated through error-variance models.Attention is given to parameter bias determinations, evaluations of their significances, and to new ways to correct for bias.The error-variance models allow for both additive and independent power-law errors, and the power exponent is shown to be able to be well determined for typical physicalsciences data by the rapidly-converging, general-purpose, extended-least-squares program we use.The fitting and error-variance models are applied to both low-and high-heteroscedasticity situations, including single-response data from radioactive decay.Monte-Carlo simulations of data with similar parameters are used to evaluate the analytical models developed and the various minimization methods em-ployed, such as extended and generalized least squares.Logarithmic and inversion transformations are investigated in detail, and it is shown analytically and by simulations that exponential data with constant percentage errors can be logarithmically transformed to allow a simple parameter-bias-removal procedure.A more-general bias-reduction approach combining direct and inversion fitting is also developed.Distributions of fitting-model and error-variance-model parameters are shown to be typically non-normal, thus invalidating the usual estimates of parameter bias and precision.Errors in conventional confidence-interval estimates are quantified by comparison with accurate simulation results.     

On distribution-weighted partial least squares with diverging number of highly correlated predictors

Summary.  Because highly correlated data arise from many scientific fields, we investigate parameter estimation in a semiparametric regression model with diverging number of predictors that are highly correlated. For this, we first develop a distribution-weighted least squares estimator that can recover directions in the central subspace, then use the distribution-weighted least squares estimator as a seed vector and project it onto a Krylov space by partial least squares to avoid computing the inverse of the covariance of predictors. Thus, distrbution-weighted partial least squares can handle the cases with high dimensional and highly correlated predictors. Furthermore, we also suggest an iterative algorithm for obtaining a better initial value before implementing partial least squares. For theoretical investigation, we obtain strong consistency and asymptotic normality when the dimension p of predictors is of convergence rate O { n ^1/2/ log ( n )} and o ( n ^1/3) respectively where n is the sample size. When there are no other constraints on the covariance of predictors, the rates n ^1/2 and n ^1/3 are optimal. We also propose a Bayesian information criterion type of criterion to estimate the dimension of the Krylov space in the partial least squares procedure. Illustrative examples with a real data set and comprehensive simulations demonstrate that the method is robust to non-ellipticity and works well even in 'small n –large p ' problems.     

Least squares parameter estimation in multiplicative noise models

A simple multiplicative noise model with a constant signal has become a basic mathematical model in processing synthetic aperture radar images. The purpose of this paper is to examine a general multiplicative noise model with linear signals represented by a number of unknown parameters. The ordinary least squares (LS) and weighted LS methods are used to estimate the model parameters. The biases of the weighted LS estimates of the parameters are derived. The biases are then corrected to obtain a second-order unbiased estimator, which is shown to be exactly equivalent to the maximum log quasi-likelihood estimation, though the quasi-likelihood function is founded on a completely different theoretical consideration and is known, at the present time, to be a uniquely acceptable theory for multiplicative noise models. Synthetic simulations are carried out to confirm theoretical results and to illustrate problems in processing data contaminated by multiplicative noises. The sensitivity of the LS and weighted LS methods to extremely noisy data is analysed through the simulated examples.     

Adaptive estimation of vector autoregressive models with time-varying variance: Application to testing linear causality in mean

Linear vector autoregressive (VAR) models where the innovations could be unconditionally heteroscedastic are considered. The volatility structure is deterministic and quite general, including breaks or trending variances as special cases. In this framework we propose ordinary least squares (OLS), generalized least squares (GLS) and adaptive least squares (ALS) procedures. The GLS estimator requires the knowledge of the time-varying variance structure while in the ALS approach the unknown variance is estimated by kernel smoothing with the outer product of the OLS residual vectors. Different bandwidths for the different cells of the time-varying variance matrix are also allowed. We derive the asymptotic distribution of the proposed estimators for the VAR model coefficients and compare their properties. In particular we show that the ALS estimator is asymptotically equivalent to the infeasible GLS estimator. This asymptotic equivalence is obtained uniformly with respect to the bandwidth(s) in a given range and hence justifies data-driven bandwidth rules. Using these results we build Wald tests for the linear Granger causality in mean which are adapted to VAR processes driven by errors with a nonstationary volatility. It is also shown that the commonly used standard Wald test for the linear Granger causality in mean is potentially unreliable in our framework (incorrect level and lower asymptotic power). Monte Carlo experiments illustrate the use of the different estimation approaches for the analysis of VAR models with time-varying variance innovations.