General linear estimators under the prediction error sum of squares criterion in a linear regression model

In this paper, the notion of the general linear estimator and its modified version are introduced using the singular value decomposition theorem in the linear regression model y=X β+e to improve some classical linear estimators. The optimal selections of the biasing parameters involved are theoretically given under the prediction error sum of squares criterion. A numerical example and a simulation study are finally conducted to illustrate the superiority of the proposed estimators.     

On necessary and sufficient condition for superiority of ridge estimator over least squares estimator

The necessary and sufficient condition is obtained such that ridge estimator is better than the least squares estimator relative to the matrix mean square error.     

On the equality of usual and amemiya's partially generalized least squares estimator

Equivalent conditions are derived for the equality of GLSE (generalized least squares estimator) and partially GLSE (PGLSE), the latter introduced by Amemiya (1983). By adopting a more general approach the ordinary least squares estimator (OLSE) can shown to be a special PGLSE. Furthcrmore, linearly restricted estimators proposed by Balestra (1983) are investigated in this context. To facilitate the comparison of estimators extensive use of oblique and orthogonal projectors is made.     

Improvement of the Liu estimator in linear regression model

In the presence of stochastic prior information, in addition to the sample, Theil and Goldberger (1961) introduced a Mixed Estimator for the parameter vector β in the standard multiple linear regression model (T,Xβ,σ² I). Recently, the Liu estimator which is an alternative biased estimator for β has been proposed by Liu (1993). In this paper we introduce another new Liu type biased estimator called Stochastic restricted Liu estimator for β, and discuss its efficiency. The necessary and sufficient conditions for mean squared error matrix of the Stochastic restricted Liu estimator to exceed the mean squared error matrix of the mixed estimator will be derived for the two cases in which the parametric restrictions are correct and are not correct. In particular we show that this new biased estimator is superior in the mean squared error matrix sense to both the Mixed estimator and to the biased estimator introduced by Liu (1993).     

On the inadmssibelity of ridge estimator in a linear model

The necessary and sufficient conditions for the inadmissibility of the ridge regression is discussed under two different criteria, namely, average loss and Pitman nearness. Although the two criteria are very different, same conclusions are obtained. The loss functions considered in this article are th likelihood loss function and the Mahalanobis loss function. The two loss functions are motivated from the point of view of classification of two normal populations. Under the Mahalanobis loss it is demonstrated that the ridge regression is always inadmissible as long as the errors are assumed to be symmetrically distributed about the origin.     

Effects of the estimation of covariance matrix parameters in the generalized multivariate linear model

We Consider the generalized multivariate linear model and assume the covariance matrix of the p x 1 vector of responses on a given individual can be represented in the general linear structure form described by Anderson (1973). The effects of the use of estimates of the parameters of the covariance matrix on the generalized least squares estimator of the regression coefficients and on the prediction of a portion of a future vector, when only the first portion of the vector has been observed, are investigated. Approximations are derived for the covariance matrix of the generalized least squares estimator and for the mean square error matrix of the usual predictor, for the practical case where estimated parameters are used.     

A new estimator combining the ridge regression and the restricted least squares methods of estimation

It is well-known in the literature on multicollinearity that one of the major consequences of multicollinearity on the ordinary least squares estimator is that the estimator produces large sampling variances, which in turn might inappropriately lead to exclusion of otherwise significant coefficients from the model. To circumvent this problem, two accepted estimation procedures which are often suggested are the restricted least squares method and the ridge regression method. While the former leads to a reduction in the sampling variance of the estimator, the later ensures a smaller mean square error value for the estimator. In this paper we have proposed a new estimator which is based on a criterion that combines the ideas underlying these two estimators. The standard properties of this new estimator have been studied in the paper. It has also been shown that this estimator is superior to both the restricted least squares as well as the ordinary ridge regression estimators by the criterion of mean sauare error of the estimator of the regression coefficients when the restrictions are indeed correct. The conditions for superiority of this estimator over the other two have also been derived for the situation when the restrictions are not correct.     

Error misspecification and properties of the simple ridge estimator

This paper gives necessary and sufficient conditions for the ridge estimator applied to an error misspecified regression model to dominate the generalised least squares estimator. It is shown that the requirements for mean square error dominance are more stringent than in the correct specification case.     

Measuring the degree of severity of heteroskedasticity and the choice between the ols estimator and the 2sae

This paper dwells on the choice between the ordinary least squares and the estimated generalized least squares estimators when the presence of heteroskedasticity is suspected. Since the estimated generalized least squares estimator does not dominate the ordinary least squares estimator completely over the whole parameter space, it is of interest to the researcher to know in advance whether the degree of severity of heteroskedasticity is such that OLS estimator outperforms the estimated generalized least squares (or 2SAE). Casting the problem in the non-spherical error mold and exploiting the principle underlying the Bayesian pretest estimator, an intuitive non-mathematical procedure is proposed to serve as an aid to the researcher in deciding when to use either the ordinary least squares (OLS) or the estimated generalized least squares (2SAE) estimators.     

Forecast mean squared error reductionin the VAR(1) process

When VAR models are used to predict future outcomes, the forecast error can be substantial. Through imposition of restrictions on the off-diagonal elements of the parameter matrix, however, the information in the process may be condensed to the marginal processes. In particular, if the cross-autocorrelations in the system are small and only a small sample is available, then such a restriction may reduce the forecast mean squared error considerably.

In this paper, we propose three different techniques to decide whether to use the restricted or unrestricted model, i.e. the full VAR(1) model or only marginal AR(1) models. In a Monte Carlo simulation study, all three proposed tests have been found to behave quite differently depending on the parameter setting. One of the proposed tests stands out, however, as the preferred one and is shown to outperform other estimators for a wide range of parameter settings.     

On the calculations of the maximum likelihood estimates for the polynomial spline regression model with unknown knots and AR(1) errors

Asymptotic distributions of the maximum likelihood estimators of the regression coefficients and knot points for the polynomial spline regression models with unknown knots and AR(1) errors have been derived by Chan (1989). Chan showed that under some mild conditions the maximum likelihood estimators, after suitable standardization, asymptotically follow normal distributions as n diverges to infinity. For the calculations of the maximum likelihood estimators, iterative methods must be applied. But this is not easy to implement for the model considered. In this paper, we suggested an alternative method to compute the estimates of the regression parameters and knots. It is shown that the estimates obtained by this method are asymptotically equivalent to the maximum likelihood estimates considered by Chan.     

On the efficiency of regression analysis with AR(p) errors

In this paper we will consider a linear regression model with the sequence of error terms following an autoregressive stationary process. The statistical properties of the maximum likelihood and least squares estimators of the regression parameters will be summarized. Then, it will be proved that, for some typical cases of the design matrix, both methods produce asymptotically equivalent estimators. These estimators are also asymptotically efficient. Such cases include the most commonly used models to describe trend and seasonality like polynomial trends, dummy variables and trigonometric polynomials. Further, a very convenient asymptotic formula for the covariance matrix will be derived. It will be illustrated through a brief simulation study that, for the simple linear trend model, the result applies even for sample sizes as small as 20.     

On the correct regression function (in L2) and its applications when the dimension of the covariate vector is random

We derive the optimal regression function (i.e., the best approximation in the L₂ sense) when the vector of covariates has a random dimension. Furthermore, we consider applications of these results to problems in statistical regression and classification with missing covariates. It will be seen, perhaps surprisingly, that the correct regression function for the case with missing covariates can sometimes perform better than the usual regression function corresponding to the case with no missing covariates. This is because even if some of the covariates are missing, an indicator random variable δ$\delta$, which is always observable, and is equal to 1 if there are no missing values (and 0 otherwise), may have far more information and predictive power about the response variable Y than the missing covariates do. We also propose kernel-based procedures for estimating the correct regression function nonparametrically. As an alternative estimation procedure, we also consider the least-squares method.     

Bayesian inference and prediction of the Pareto distribution based on ordered ranked set sampling

In this paper, order statistics from independent and non identically distributed random variables is used to obtain ordered ranked set sampling (ORSS). Bayesian inference of unknown parameters under a squared error loss function of the Pareto distribution is determined. We compute the minimum posterior expected loss (the posterior risk) of the derived estimates and compare them with those based on the corresponding simple random sample (SRS) to assess the efficiency of the obtained estimates. Two-sample Bayesian prediction for future observations is introduced by using SRS and ORSS for one- and m-cycle. A simulation study and real data are applied to show the proposed results.     

On Small Sample Properties of the Wald,LR and LM Tests in a Linear Model with AR(1) Errors

When the error terms are autocorrelated, the conventional t-tests for individual regression coefficients mislead us to over-rejection of the null hypothesis. We examine, by Monte Carlo experiments, the small sample properties of the unrestricted estimator of ρ and of the estimator of ρ restricted by the null hypothesis. We compare the small sample properties of the Wald, likelihood ratio and Lagrange multiplier test statistics for individual regression coefficients. It is shown that when the null hypothesis is true, the unrestricted estimator of ρ is biased. It is also shown that the Lagrange multiplier test using the maximum likelihood estimator of ρ performs better than the Wald and likelihood ratio tests.