Applied regression analysis bibliography update 1990-91

The 25-page Bibliography in Applied Regression Analysis, 2nd edition, by N.R. Draper and H. Smith, published by Wiley in 1981, was previously extended by a list of selected references available during 1988-89, plus a few older references. See Communications in Statistics - Theory and Methods, 19(4), 1205-1229 (1990). Here is a further update covering the years 1990-91, and following the same format. As before, items were chosen on the basis of their perceived relevance to practical applications (sometimes rather widely interpreted). The classification system used is that of the book.

The references were selected mostly from the issues of these journals: Annals of Statistics; Applied Statistics; Biometrika; Canadian Journal of Statistics; Bulletin of the International Statistical Institute; Journal of the American Statistical Association; Journal of Quality Technology; Journal of the Royal Statistical Society, Series A and B; and Technometrics.     

Least absolute value estimation in regression models: an annotated bibliography

This paper presents a comprehensive listing of articles on least absolute value (LAV) estimation as applied to linear and non-linear regression models and in systems of equations. References to the LAV method as applied in approximation theory are also included. Annotations describing the content of each article follow each reference.     

Factor analysis regression

The paper describes two regression models—principal components and maximum-likelihood factor analysis—which may be used when the stochastic predictor varibles are highly intereorrelated and/or contain measurement error. The two problems can occur jointly, for example in social-survey data where the true (but unobserved) covariance matrix can be singular. Departure from singularity of the sample dispersion matrix is then due to measurement error. We first consider the more elementary principal components regression model, where it is shown that it can be derived as a special case of (i) canonical correlation, and (ii) restricted least squares. The second part consists of the more general maximum-likelihood factor-analysis regression model, which is derived from the generalized inverse of the product of two singular matrices. Also, it is proved that factor-analysis regression can be considered as an instrumental variables estimator and therefore does not depend on whether factors have been “properly” identified in terms of substantive behaviour. Consequently the additional task of rotating factors to “simple structure” does not arise.     

Factor analysis regression

In the presence of multicollinearity the literature points to principal component regression (PCR) as an estimation method for the regression coefficients of a multiple regression model. Due to ambiguities in the interpretation, involved by the orthogonal transformation of the set of explanatory variables, the method could not yet gain wide acceptance. Factor analysis regression (FAR) provides a model-based estimation method which is particularly tailored to overcome multicollinearity in an errors-in-variables setting. In this paper two feasible versions of a FAR estimator are compared with the OLS estimator and the PCR estimator by means of Monte Carlo simulation. While the PCR estimator performs best in cases of strong and high multicollinearity, the Thomson-based FAR estimator proves to be superior when the regressors are moderately correlated.     

Multilevel regression mixture analysis

Summary.  A two-level regression mixture model is discussed and contrasted with the conventional two-level regression model. Simulated and real data shed light on the modelling alternatives. The real data analyses investigate gender differences in mathematics achievement from the US National Education Longitudinal Survey. The two-level regression mixture analyses show that unobserved heterogeneity should not be presupposed to exist only at level 2 at the expense of level 1. Both the simulated and the real data analyses show that level 1 heterogeneity in the form of latent classes can be mistaken for level 2 heterogeneity in the form of the random effects that are used in conventional two-level regression analysis. Because of this, mixture models have an important role to play in multilevel regression analyses. Mixture models allow heterogeneity to be investigated more fully, more correctly attributing different portions of the heterogeneity to the different levels.     

On multivariate quantile regression analysis

This paper investigates the estimation of parameters in a multivariate quantile regression model when the investigator wants to evaluate the associated distribution function. It proposes a new directional quantile estimator with the following properties: (1) it applies to an arbitrary number of random variables; (2) it is equivalent to estimating the distribution function allowing for non-convex distribution contours; (3) it satisfies nice equivariance properties; (4) it has desirable statistical properties (i.e., consistency and asymptotic normality); and (5) its implementation involves a modest computational burden: our proposed estimator can be obtained by solving parametric linear programming problems. As such, this paper expands the range of applications of quantile estimation for multivariate regression models.     

Measurement error in regression analysis

Consider the linear regression model Y = Xθ+ ε where Y denotes a vector of n observations on the dependent variable, X is a known matrix, θ is a vector of parameters to be estimated and e is a random vector of uncorrelated errors. If X'X is nearly singular, that is if the smallest characteristic root of X'X s small then a small perurbation in the elements of X, such as due to measurement errors, induces considerable variation in the least squares estimate of θ. In this paper we examine for the asymptotic case when n is large the effect of perturbation with regard to the bias and mean squared error of the estimate.     

Functional sliced inverse regression analysis

Most of the usual multivariate methods have been extended to the context of functional data analysis. Our contribution concerns the study of sliced inverse regression (SIR) when the response variable is real but the regressor is a function. In the first part, we show how the relevant properties of SIR remain essentially the same in the functional context under suitable conditions. Unfortunately, the estimation procedure used in the multivariate case cannot be directly transposed to the functional one. Then, we propose a solution that overcomes this difficulty and we show the consistency of the estimates of the parameters of the model.     

Bayesian outlier analysis in binary regression

We propose alternative approaches to analyze residuals in binary regression models based on random effect components. Our preferred model does not depend upon any tuning parameter, being completely automatic. Although the focus is mainly on accommodation of outliers, the proposed methodology is also able to detect them. Our approach consists of evaluating the posterior distribution of random effects included in the linear predictor. The evaluation of the posterior distributions of interest involves cumbersome integration, which is easily dealt with through stochastic simulation methods. We also discuss different specifications of prior distributions for the random effects. The potential of these strategies is compared in a real data set. The main finding is that the inclusion of extra variability accommodates the outliers, improving the adjustment of the model substantially, besides correctly indicating the possible outliers.     

Multivariate regression analysis and canonical variates

A method called FICYREG of estimating regression coefficients is introduced. This is a generalization to the multivariate regression problem of the James-Stein estimator. When suitably représentés FICYREG emerges as a rule in which the canonical variates and canonical correlations have an intrinsic role to play. By exploiting these objects FICYREG is able to achieve stability against the influence of the “noise” present in problems where the responses are correlated so that some of the response vector's canonical variates will be essentially independent of all others including the predictors. The least squares (LS) estimator is, by contrast, highly sensitive to this noise. The use of FICYREG is illustrated in terms of an example, and its peformance is compared to the LS estimator when a quadratic loss function is assumed. The cases of both fixed and random predictors are considered. Overall, FICYREG outperforms the LS estimator.     

Using quantile regression for duration analysis

Summary Quantile regression methods are emerging as a popular technique in econometrics and biometrics for exploring the distribution of duration data. This paper discusses quantile regression for duration analysis allowing for a flexible specification of the functional relationship and of the error distribution. Censored quantile regression addresses the issue of right censoring of the response variable which is common in duration analysis. We compare quantile regression to standard duration models. Quantile regression does not impose a proportional effect of the covariates on the hazard over the duration time. However, the method cannot take account of time-varying covariates and it has not been extended so far to allow for unobserved heterogeneity and competing risks. We also discuss how hazard rates can be estimated using quantile regression methods. This paper benefitted from the helpful comments by an anonymous referee. Due to space constraints, we had to omit the details of the empirical application. These can be found in the long version of this paper, Fitzenberger and Wilke (2005). We gratefully acknowledge financial support by the German Research Foundation (DFG) through the research project ‘Microeconometric modelling of unemployment durations under consideration of the macroeconomic situation’. Thanks are due to Xuan Zhang for excellent research assistance. All errors are our sole responsibility.     

Bayesian analysis of zero-inflated regression models

In modeling defect counts collected from an established manufacturing processes, there are usually a relatively large number of zeros (non-defects). The commonly used models such as Poisson or Geometric distributions can underestimate the zero-defect probability and hence make it difficult to identify significant covariate effects to improve production quality. This article introduces a flexible class of zero inflated models which includes other familiar models such as the Zero Inflated Poisson (ZIP) models, as special cases. A Bayesian estimation method is developed as an alternative to traditionally used maximum likelihood based methods to analyze such data. Simulation studies show that the proposed method has better finite sample performance than the classical method with tighter interval estimates and better coverage probabilities. A real-life data set is analyzed to illustrate the practicability of the proposed method easily implemented using WinBUGS.     

On hsu's model in regression analysis

The paper examplifies with Hsu’s model a general pattern as how to derive results of variance component estimation from well known results on mean estimation, as far as linear model theory is concerned. This ’ dispersion-mean-correspondence‘provides new and short proofs for various theorems from the literature, concerning unbiased invariant quadratic estimators with minimum BAYES risk or minimum variance. For pure variance component models, unbiased non-negative quadratic estimability is characterized in terms of the design matrices.     

A procedure for stepwise regression analysis

A procedure for stepwise regression analysis for the non-experimental case is suggested. Regarding the problem as a multiple inference one, the procedure picks out the relevant regressors and, based on a slightly new approach, estimates the structure of dependencies among the variables involved.     

Linear regression analysis with inequality constraints on the regression parameters via empirical likelihood

An empirical likelihood ratio test is developed for testing for or against inequality constraints on regression parameters in linear regression analysis. The proposed approach imposes no parametric model nor identically distributing assumption on the random errors. The asymptotic distribution of the proposed test statistic under null hypothesis is shown to be of chi-bar-squared type. The asymptotic power under contiguous alternatives is also briefly discussed. Moreover, an adjusted empirical likelihood method is adopted to improve the small sample size behaviour of the proposed test. Several simulation studies are carried out to assess the finite sample performance of the proposed tests. The results reveal that the proposed tests could be valuable for improving inference efficiency. A real-life example is discussed to illustrate the theoretical results.     

Bayesian assessment of assumptions of regression analysis

Many statistical procedures are based on the models which specify the conditions under which the data are generated. Many applications of linear regression, for example, assume that:(i) the observations are independent; (ii) the errors in the observations are identically distributed; (iii) each error has a normal distribution with mean zero and unknown variance σ²> 0. Previous works have examined individual departures from these assumptions. Here we examine composite departures. It is assumed that the error distribution in a linear model is power-exponential and that the observations are generated via a first order autoregressive model with the possibility of spurious observations. The consequences are illustrated via an example.