A Comparison of Partially Adaptive and Reweighted Least Squares Estimation

The small sample performance of least median of squares, reweighted least squares, least squares, least absolute deviations, and three partially adaptive estimators are compared using Monte Carlo simulations. Two data problems are addressed in the paper: (1) data generated from non-normal error distributions and (2) contaminated data. Breakdown plots are used to investigate the sensitivity of partially adaptive estimators to data contamination relative to RLS. One partially adaptive estimator performs especially well when the errors are skewed, while another partially adaptive estimator and RLS perform particularly well when the errors are extremely leptokur-totic. In comparison with RLS, partially adaptive estimators are only moderately effective in resisting data contamination; however, they outperform least squares and least absolute deviation estimators.     

Forecast Evaluation in the Presence of Unobserved Volatility

A number of volatility forecasting studies have led to the perception that the ARCH- and Stochastic Volatility-type models provide poor out-of-sample forecasts of volatility. This is primarily based on the use of traditional forecast evaluation criteria concerning the accuracy and the unbiasedness of forecasts. In this paper we provide an analytical assessment of volatility forecasting performance. We use the volatility and log volatility framework to prove how the inherent noise in the approximation of the true- and unobservable-volatility by the squared return, results in a misleading forecast evaluation, inflating the observed mean squared forecast error and invalidating the Diebold-Mariano statistic. We analytically characterize this noise and explicitly quantify its effects assuming normal errors. We extend our results using more general error structures such as the Compound Normal and the Gram-Charlier classes of distributions. We argue that evaluation problems are likely to be exacerbated by non-normality of the shocks and that non-linear and utility-based criteria can be more suitable for the evaluation of volatility forecasts.     

Semiparametric Ridge Regression Approach in Partially Linear Models

In this article, we introduce a semiparametric ridge regression estimator for the vector-parameter in a partial linear model. It is also assumed that some additional artificial linear restrictions are imposed to the whole parameter space and the errors are dependent. This estimator is a generalization of the well-known restricted least-squares estimator and is confined to the (affine) subspace which is generated by the restrictions. Asymptotic distributional bias and risk are also derived and the comparison result is then given.     

Ridge regression methodology in partial linear models with correlated errors

In this article, a generalized restricted difference-based ridge estimator is defined for the vector parameter in a partial linear model when the errors are dependent. It is suspected that some additional linear constraints may hold on to the whole parameter space. The estimator is a generalization of the well-known restricted least-squares estimator and is confined to the (affine) subspace which is generated by the restrictions. The risk functions of the proposed estimators are derived under balanced loss function. Finally, the performance of the new estimators is evaluated by a simulated data set.     

On preliminary test and shrinkage estimation in linear models with long-memory errors

This paper discusses the problem of estimating a subset of parameters when the complementary subset is possibly redundant, in a linear regression model when the errors are generated from a long-memory process. Such a model arises due to the overmodelling of a situation involving long-memory data. Along with the classical least-squares estimator and restricted least-squares estimator, preliminary test least-squares estimator and shrinkage least-squares estimator are investigated in an asymptotic set-up and their relative performances are studied under contiguous alternatives. The contiguous alternatives under such dependence are fundamentally different from those under the independent errors case.     

The feasible generalized restricted ridge regression estimator

The presence of autocorrelation in errors and multicollinearity among the regressors have undesirable effects on the least-squares regression. There are a wide range of methods which are proposed to overcome the usefulness of the ordinary least-squares estimator or the generalized least-squares estimator, such as the Stein-rule, restricted least-squares or ridge estimator. Therefore, we introduce a new feasible generalized restricted ridge regression (FGRR) estimator to examine multicollinearity and autocorrelation problems simultaneously for the general linear regression model. We also derive some statistical properties of the FGRR estimator and comparisons have been conducted using matrix mean-square error. Moreover, a Monte Carlo simulation experiment is performed to investigate the performance of the proposed estimator over the others.     

Bootstrapping an Econometric Model: Some Empirical Results

The bootstrap, like the jackknife, is a technique for estimating standard errors. The idea is to use Monte Carlo simulation, based on a nonparametric estimate of the underlying error distribution. The bootstrap will be applied to an econometric model describing the demand for capital, labor, energy, and materials. The model is fitted by three-stage least squares. In sharp contrast with previous results, the coefficient estimates and the estimated standard errors perform very well. However, the model's forecasts show serious bias and large random errors, significantly understated by the conventional standard error of forecast.     

Multiple-response repeated measurement or multivariate growth curve model with distribution-free errors

In this article, we propose a new modeling approach for the multivariate growth curve model with distribution-free errors, which is a useful tool for analyzing multiple-response repeated measurements. We first use the outer product least-squares technique to directly estimate covariance and then explore the feasible generalized least-squares technique to derive the estimator of regression coefficients. Large-sample properties are investigated for these estimators. Moreover, the above estimations for covariance and regression coefficients are extended to the situation under certain null hypothesis tests and the best subset BIC is used for variable selection. A real dataset is analyzed to demonstrate the usefulness and competency of the proposed methodology for model specification (identification) and model fitting (parameter estimation) in multiple-response repeated measurements.     

Ratios of linear combinations in the general linear model

We consider the problem of estimating. the ratio of two linear combinations of thevector of parameters in the general linear model. The nonexistence of an unbiased estimator under normal errorsisdiscussed. Properties of an often used estimator, the maximum likelihood estimator under normal errors, are presented, This is done both for fixed sample size and asymptotically, in the

presence of normal and non-normal errors.     

Ordinary and weighted least-squares estimators

We propose a method of estimating the asymptotic relative efficiency (ARE) of the weighted least-squares estimator (WLSE) with respect to the ordinary least-squares estimator (OLSE) in a heteroscedastic linear regression model with a large number of observations but a small number of replicates at each value of the regressors. The weights used in the WLSE are the reciprocals of the (within-group) average of squared residuals. It is shown that the OLSE is more efficient than the WLSE if the maximum number of replicates is not larger than two. The proposed estimator of the ARE is consistent as the number of observations tends to infinity. Finite-sample performance of this estimator is examined in a simulation study. An adaptive estimator, which is asymptotically more efficient than the OLSE and the WLSE, is proposed.     

The exact density of nonparametric regression estimators: fixed design case

This paper studies the exact density of a general nonparametric regression estimator when the errors are non-normal. The fixed design case is considered. The density function is derived by an application of the technique of Davis (1976)     

A bootstrap procedure in linear regression with nonstationary errors

In the context of linear regression with dependent and nonstationary errors, the classical moving-block bootstrap (MBB) fails to capture the nonstationarity of the errors. A new bootstrap procedure called the blocking external bootstrap (BEB) is proposed to overcome the problem. The consistency of the BEB in estimating the variance of the least-squares estimator is studied in the case of α-mixing and nonstationary sequence of errors. It is shown that the BEB only achieves partial correction if the block size is fixed. Complete consistency is achieved by the BEB when the block size is allowed to go to infinity. We also study the first-order consistency of the least squares estimator based on the BEB. A simulation study is carried out to assess the performance of the BEB versus the MBB in estimating the variance of the least-squares estimator. Finally, some open problems are discussed.     

Asymptotic and Bootstrap Inference for AR(∞) Processes with Conditional Heteroskedasticity

The main contribution of this paper is a proof of the asymptotic validity of the application of the bootstrap to AR(∞) processes with unmodelled conditional heteroskedasticity. We first derive the asymptotic properties of the least-squares estimator of the autoregressive sieve parameters when the data are generated by a stationary linear process with martingale difference errors that are possibly subject to conditional heteroskedasticity of unknown form. These results are then used in establishing that a suitably constructed bootstrap estimator will have the same limit distribution as the least-squares estimator. Our results provide theoretical justification for the use of either the conventional asymptotic approximation based on robust standard errors or the bootstrap approximation of the distribution of autoregressive parameters. A simulation study suggests that the bootstrap approach tends to be more accurate in small samples.     

Modelling functional additive quantile regression using support vector machines approach

This work deals with conditional quantiles estimation when several functional covariates are involved, via a support vector machines nonparametric methodology. We establish weak consistency of this estimator. To fit the additive components, we use an ordinary backfitting procedure combined with an iterative reweighted least-squares procedure to solve the penalised minimisation problem. This procedure makes it possible to derive a split sample method for choosing the hyper-parameters of the model. The performances of the proposed technique, in terms of forecast accuracy, are evaluated through simulation and a real dataset study.     

Jackknife inference for heteroscedastic linear regression models

Inference on the regression parameters in a heteroscedastic linear regression model with replication is considered, using either the ordinary least-squares (OLS) or the weighted least-squares (WLS) estimator. A delete-group jackknife method is shown to produce consistent variance estimators irrespective of within-group correlations, unlike the delete-one jackknife variance estimators or those based on the customary δ-method assuming within-group independence. Finite-sample properties of the delete-group variance estimators and associated confidence intervals are also studied through simulation.     

Asymptotic and Bootstrap Inference for AR(∞) Processes with Conditional Heteroskedasticity

The main contribution of this paper is a proof of the asymptotic validity of the application of the bootstrap to AR(∞) processes with unmodelled conditional heteroskedasticity. We first derive the asymptotic properties of the least-squares estimator of the autoregressive sieve parameters when the data are generated by a stationary linear process with martingale difference errors that are possibly subject to conditional heteroskedasticity of unknown form. These results are then used in establishing that a suitably constructed bootstrap estimator will have the same limit distribution as the least-squares estimator. Our results provide theoretical justification for the use of either the conventional asymptotic approximation based on robust standard errors or the bootstrap approximation of the distribution of autoregressive parameters. A simulation study suggests that the bootstrap approach tends to be more accurate in small samples.     

Bootstrap variance and bias estimation in linear models

Let θ be a nonlinear function of the regression parameters and θ be its estimator based on the least-squares method. This paper studies the bootstrap estimators of the variance and bias of θ. The bootstrap estimators are shown to be consistent and asymptotically unbiased under some conditions. Asymptotic orders of the mean squared errors of the bootstrap estimators are also obtained. The bootstrap and the classical linearization method are compared in a simulation study. Discussions about when to use the bootstrap are given.     

Using Bayesian Techniques for Data Pooling in Regional Payroll Forecasting

This article adapts to the regional level a multicountry technique recently used by Garcia-Ferrer, Highfield, Palm, and Zellner (1987) and extended by Zellner and Hong (1987) to forecast the growth rates in gross national product across nine countries. This forecasting methodology is applied to the regional level by modeling payroll formation in seven Ohio metropolitan areas. We compare the forecasting performance of these procedures with that of a ridge estimator and find that the ridge estimator produces forecasts equal to or better than those from the newly proposed estimators. We conclude that the ridge estimator, which does not reference the pooled data information introduced by the newly proposed techniques, may serve as a benchmark against which to judge the relative importance of this kind of information in improving forecasts.     

Hits-and-misses for the evaluation and combination of forecasts

Error measures for the evaluation of forecasts are usually based on the size of the forecast errors. Common measures are, e.g. the mean squared error (MSE), the mean absolute deviation (MAD) or the mean absolute percentage error (MAPE). Alternative measures for the comparison of forecasts are turning points or hits-and-misses, where an indicator loss function is used to decide if a forecast is of high quality or not. Here, we discuss the latter to obtain reliable combined forecasts. We apply several combination techniques to a set of German macroeconomic data. Furthermore, we perform a small simulation study for the combination of two biased forecasts.     

AMEMIYA'S FORM OF THE WEIGHTED LEAST SQUARES ESTIMATOR

Amemiya's estimator is a weighted least squares estimator of the regression coefficients in a linear model with heteroscedastic errors. It is attractive because the heteroscedasticity is not parametrized and the weights (which depend on the error covariance matrix) are estimated nonparametrically. This paper derives an asymptotic expansion for Amemiya's form of the weighted least squares estimator, and uses it to discuss the effects of estimating the weights, of the number of iterations, and of the choice of the initial estimate. The paper also discusses the special case of normally distributed errors and clarifies the particular consequences of assuming normality.