GENERALIZED INVERSES USED IN RECURSIVE ESTIMATION OF THE GENERAL LINEAR MODEL1

Recursive methods in regression have proved useful in providing diagnostic tools for checking the model as well as checking the stability of the model over time. Such methods are now extended to deal with the problems of singularity that arise when one variable is completely confounded with previously fitted variables up to a particular time point. The problem is solved by setting it in the framework of the general linear model with dependent errors.     

MEDIAN ESTIMATES IN SIMPLE LINEAR REGRESSION

Summary
Estimates in simple linear regression which minimize the sum of absolute values of residuals are discussed. These estimates, called median-estimates , are an analogue of the sample median, and their properties reflect this. A simple graphical procedure, related to an early (1923) one of Edgeworth, is outlined for computing median estimates. This method works well in small or moderate samples, and is demonstrated on two examples. The small sample distribution of the median estimates is examined by conditioning on the pattern of signs of the residuals, and the resulting conditional distribution gives qualitative information on the adequacy of the approximation of the exact distribution by the (known) asymptotic distibution.     

RANDOM PERMUTATION TESTING IN MULTIPLE LINEAR REGRESSION

We developed an alternative random permutation testing method for multiple linear regression, which is an improvement over the existing one proposed by [1] Kennedy, P. E. 1995. Randomization tests in econometrics. Journal of Business and Economic Statistics, 13: 85–94. [Taylor & Francis Online], [Web of Science ®] , [Google Scholar] or [2] Freedman, D. and Lane, D. 1983. A nonstochastic interpretation of reported significance levels. Journal of Business and Economic Statistics, 1: 292–298. [Taylor & Francis Online] , [Google Scholar].     

SKEWNESS FOR PARAMETERS IN GENERALIZED LINEAR MODELS

In this paper we give an asymptotic formula of order n ^?1/2, where n is the sample size, for the skewness of the distribution of the maximum likelihood estimates of the linear parameters in generalized linear models. The formula is given in matrix notation and is very suitable for computer implementation. Several special cases are discussed. We also give asymptotic formulae for the skewness of the distribution of the maximum likelihood estimates of the dispersion and precision parameters.     

INCOMPLETE DATA IN GENERALIZED LINEAR MODELS WITH CONTINUOUS COVARIATES

This paper proposes a method for estimating the parameters in a generalized linear model with missing covariates. The missing covariates are assumed to come from a continuous distribution, and are assumed to be missing at random. In particular, Gaussian quadrature methods are used on the E-step of the EM algorithm, leading to an approximate EM algorithm. The parameters are then estimated using the weighted EM procedure given in Ibrahim (1990). This approximate EM procedure leads to approximate maximum likelihood estimates, whose standard errors and asymptotic properties are given. The proposed procedure is illustrated on a data set.     

A MORE GENERAL CRITERION FOR SUBSET SELECTION IN MULTIPLE LINEAR REGRESSION

ABSTRACT

In this article, we propose a more general criterion called S_p -criterion, for subset selection in the multiple linear regression Model. Many subset selection methods are based on the Least Squares (LS) estimator of β, but whenever the data contain an influential observation or the distribution of the error variable deviates from normality, the LS estimator performs ‘poorly’ and hence a method based on this estimator (for example, Mallows’ C_p -criterion) tends to select a ‘wrong’ subset. The proposed method overcomes this drawback and its main feature is that it can be used with any type of estimator (either the LS estimator or any robust estimator) of β without any need for modification of the proposed criterion. Moreover, this technique is operationally simple to implement as compared to other existing criteria. The method is illustrated with examples.     

JAMES-STEIN RULE ESTIMATORS IN LINEAR REGRESSION MODELS WITH MULTIVARIATE-t DISTRIBUTED ERROR

This paper considers estimation of β in the regression model y =Xβ+μ, where the error components in μ have the jointly multivariate Student-t distribution. A family of James-Stein type estimators (characterised by nonstochastic scalars) is presented. Sufficient conditions involving only X are given, under which these estimators are better (with respect to the risk under a general quadratic loss function) than the usual minimum variance unbiased estimator (MVUE) of β. Approximate expressions for the bias, the risk, the mean square error matrix and the variance-covariance matrix for the estimators in this family are obtained. A necessary and sufficient condition for the dominance of this family over MVUE is also given.     

A GEOMETRIC CHARACTERIZATION OF LINEAR REGRESSION

This paper concerns the asymptotic properties of the maximum likelihood estimators of the parameters in a non regular Cox model involving a change-point in the regression on time-dependent covariates. The global consistency derives from the uniform convergence of the partial log-likelihood. We prove that the estimator of the change-point is n -consistent and the estimator of the regression parameter n 1/2 -consistent, and their asymptotic distributions are established.     

BOOTSTRAP TESTS FOR THE ERROR DISTRIBUTION IN LINEAR AND NONPARAMETRIC REGRESSION MODELS

In this paper we investigate several tests for the hypothesis of a parametric form of the error distribution in the common linear and non‐parametric regression model, which are based on empirical processes of residuals. It is well known that tests in this context are not asymptotically distribution‐free and the parametric bootstrap is applied to deal with this problem. The performance of the resulting bootstrap test is investigated from an asymptotic point of view and by means of a simulation study. The results demonstrate that even for moderate sample sizes the parametric bootstrap provides a reliable and easy accessible solution to the problem of goodness‐of‐fit testing of assumptions regarding the error distribution in linear and non‐parametric regression models.     

THE EXAMINATION AND ANALYSIS OF RESIDUALS FOR SOME BIASED ESTIMATORS IN LINEAR REGRESSION

In the presence of collinearity certain biased estimation procedures like ridge regression, generalized inverse estimator, principal component regression, Liu estimator, or improved ridge and Liu estimators are used to improve the ordinary least squares (OLS) estimates in the linear regression model. In this paper new biased estimator (Liu estimator), almost unbiased (improved) Liu estimator and their residuals will be analyzed and compared with OLS residuals in terms of mean-squared error.     

GROUPING PROBLEMS IN DISTRIBUTION-FREE REGRESSION1

In regression models having symmetric errors, exact distribution-free inference about individual parameters may be carried out by grouping observations, eliminating unwanted parameters within groups, and applying distribution free techniques for the symmetric location parameter problem. Models whose errors have identical but not symmetric distributions may obtain symmetry by taking differences between pairs of observations. Both grouping and differencing involve potential efficiency loss. The choice of an optimal scheme to minimize efficiency loss is expressible as a multi–assignment type of problem, whose solutions, exact and approximate, are discussed.     

INFERENCE IN DIRICHLET PROCESS MIXED GENERALIZED LINEAR MODELS BY USING MONTE CARLO EM

Generalized linear mixed models are widely used for describing overdispersed and correlated data. Such data arise frequently in studies involving clustered and hierarchical designs. A more flexible class of models has been developed here through the Dirichlet process mixture. An additional advantage of using such mixture models is that the observations can be grouped together on the basis of the overdispersion present in the data. This paper proposes a partial empirical Bayes method for estimating all the model parameters by adopting a version of the EM algorithm. An augmented model that helps to implement an efficient Gibbs sampling scheme, under the non‐conjugate Dirichlet process generalized linear model, generates observations from the conditional predictive distribution of unobserved random effects and provides an estimate of the average number of mixing components in the Dirichlet process mixture. A simulation study has been carried out to demonstrate the consistency of the proposed method. The approach is also applied to a study on outdoor bacteria concentration in the air and to data from 14 retrospective lung‐cancer studies.     

FAST DOUBLE BOOTSTRAP TESTS OF NONNESTED LINEAR REGRESSION MODELS

It has been shown in previous work that bootstrapping the J test for nonnested linear regression models dramatically improves its finite-sample performance. We provide evidence that a more sophisticated bootstrap procedure, which we call the fast double bootstrap, produces a very substantial further improvement in cases where the ordinary bootstrap does not work as well as it might. This FDB procedure is only about twice as expensive as the usual single bootstrap.     

A NOTE ON ESTIMATING LINEAR TREND IN A REGRESSION MODEL WITH SERIALLY CORRELATED ERROR COMPONENTS

ABSTRACT

We consider a linear trend regression model when the disturbances follow a serially correlated one-way error component model. In this model, we investigate the performance of the Ordinary Least Squares Esitmator (OLSE), First Difference Estimator (FDE), Generalized Least Squares Estimator (GLSE) and the Cochrane-Orcutt-Transformation Estimator (COTE) of slope coefficient in terms of efficiency. The main findings are as follows: (1) when the autocorrelation is close to unity, then the FDE is approximately the GLSE; (2) the OLSE is better than the COTE; and (3) when the value of the autocorrelation is kept constant and T → ∞, the OLSE, COTE and GLSE are asymptotically equivalent whereas the FDE is worse than the other estimators in terms of efficiency.     

CRITICAL VALUE APPROXIMATIONS FOR TESTS OF LINEAR REGRESSION DISTURBANCES

Two important classes of tests for non-spherical disturbances in the linear regression model involve test statistics whose null distributions and hence critical values depend on the regressors. This paper investigates the accuracy of the normal, two moment beta and four moment beta approximations to the critical values of such tests. An empirical experiment aimed at evaluating the accuracy of the approximations for a variety of tests against autocorrelation and heteroscedasticity is conducted. Overall the approximations are found to provide reasonably accurate critical values with skewness being a factor determining the degree of accuracy.     

THE CONSTRUCTION OF EQUILEVERAGE DESIGNS FOR MULTIPLE LINEAR REGRESSION

The hat matrix is widely used as a diagnostic tool in linear regression because it contains the leverages which the independent variables exert on the fitted values. In some experiments, cases with high leverage may be avoided by judicious choice of design for the independent variables. A variety of methods for constructing equileverage designs for linear regression are discussed. Such designs remove one of the factors, namely large leverage points, which can lead to nonrobust estimators and tests. In addition, a method is given for combining equileverage designs to test for lack of fit of the linear model.     

FAST DOUBLE BOOTSTRAP TESTS OF NONNESTED LINEAR REGRESSION MODELS

ABSTRACT

It has been shown in previous work that bootstrapping the J test for nonnested linear regression models dramatically improves its finite-sample performance. We provide evidence that a more sophisticated bootstrap procedure, which we call the fast double bootstrap, produces a very substantial further improvement in cases where the ordinary bootstrap does not work as well as it might. This FDB procedure is only about twice as expensive as the usual single bootstrap.     

A NOTE ON VARIANCE ESTIMATION FOR THE GENERALIZED REGRESSION PREDICTOR

The generalized regression (GREG) predictor is used for estimating a finite population total when the study variable is well‐related to the auxiliary variable. In 1997, Chaudhuri & Roy provided an optimal estimator for the variance of the GREG predictor within a class of non‐homogeneous quadratic estimators (H) under a certain superpopulation model M. They also found an inequality concerning the expected variances of the estimators of the variance of the GREG predictor belonging to the class H under the model M. This paper shows that the derivation of the optimal estimator and relevant inequality, presented by Chaudhuri & Roy, are incorrect.     

TESTING THE STABILITY OF REGRESSION COEFFICIENTS USING GENERALIZED RECURSIVE RESIDUALS

Simple but flexible methods to detect deviations from the assumption of constant coefficients in linear regression are presented. Based on recursive residuals a runs test is developed as an alternative to CUSUM- and MOSUM-techniques. Finally a simulation study gives insight into the new method.