On a Relation between Principal Components and Regression Analysis

A regression approach to principal component analysis is presented in this note. We provide an alternative interpretation of principal components that illustrates the relation between the extra sum of squares in regression analysis and the eigenvalues associated with the principal components.     

Application of the Bootstrap Approach to the Choice of Dimension and the α Parameter in the SIRα Method

To reduce the dimensionality of regression problems, sliced inverse regression approaches make it possible to determine linear combinations of a set of explanatory variables X related to the response variable Y in general semiparametric regression context. From a practical point of view, the determination of a suitable dimension (number of the linear combination of X) is important. In the literature, statistical tests based on the nullity of some eigenvalues have been proposed. Another approach is to consider the quality of the estimation of the effective dimension reduction (EDR) space. The square trace correlation between the true EDR space and its estimate can be used as goodness of estimation. In this article, we focus on the SIR_α method and propose a naïve bootstrap estimation of the square trace correlation criterion. Moreover, this criterion could also select the α parameter in the SIR_α method. We indicate how it can be used in practice. A simulation study is performed to illustrate the behavior of this approach.     

Two Moments of the Logitnormal Distribution

We display the first two moment functions of the Logitnormal(μ, σ²) family of distributions, conveniently described in terms of the Normal mean, μ, and the Normal signal-to-noise ratio, μ/σ, parameters that generate the family. Long neglected on account of the numerical integrations required to compute them, awareness of these moment functions should aid the sensible interpretation of logistic regression statistics and the specification of “diffuse” prior distributions in hierarchical models, which can be deceiving. We also use numerical integration to compare the correlation between bivariate Logitnormal variables with the correlation between the bivariate Normal variables from which they are transformed.     

A multiple-case deletion approach for detecting influential points in high-dimensional regression

ABSTRACT

In high-dimensional regression, the presence of influential observations may lead to inaccurate analysis results so that it is a prime and important issue to detect these unusual points before statistical regression analysis. Most of the traditional approaches are, however, based on single-case diagnostics, and they may fail due to the presence of multiple influential observations that suffer from masking effects. In this paper, an adaptive multiple-case deletion approach is proposed for detecting multiple influential observations in the presence of masking effects in high-dimensional regression. The procedure contains two stages. Firstly, we propose a multiple-case deletion technique, and obtain an approximate clean subset of the data that is presumably free of influential observations. To enhance efficiency, in the second stage, we refine the detection rule. Monte Carlo simulation studies and a real-life data analysis investigate the effective performance of the proposed procedure.     

Chain pooling model selection for two-level fixed effects factorial experiments

As many as three iterated statistical model deletion procedures are considered for an experiment.Population model coeff cients were chosen to simulate a saturated 2⁴experiment having an unfavorable distribution of parameter values.Using random number studies, three model selection strategies were developed, namely, (1) a strategy to be used in anticipation of large coefficients of variation (neighborhood of 65 percent), (2) strategy to be used in anticipation of small coefficients of variation (4 percent or less), and (3) a security regret strategy to be used in the absence of such prior knowledge     

The best subset of parameters in min-max linear regression

These Fortran-77 subroutines provide building blocks for Generalized Cross-Validation (GCV) (Craven and Wahba, 1979) calculations in data analysis and data smoothing including ridge regression (Golub, Heath, and Wahba, 1979), thin plate smoothing splines (Wahba and Wendelberger, 1980), deconvolution (Wahba, 1982d), smoothing of generalized linear models (O'sullivan, Yandell and Raynor 1986, Green 1984 and Green and Yandell 1985), and ill-posed problems (Nychka et al., 1984, O'sullivan and Wahba, 1985). We present some of the types of problems for which GCV is a useful method of choosing a smoothing or regularization parameter and we describe the structure of the subroutines.Ridge Regression: A familiar example of a smoothing parameter is the ridge parameter X in the ridge regression problem which we write.     

On the asymptotic properties of parameter estimates in a regression model with non-normally distributed errors

As pointed out in a recent paper by Amirkhalkhali and Rao (1986) (henceforth referred to as A&R), the usual assumption of normality for the error terms of a regression model isoften untenable. However, when this assumption is dropped, it may be difficult to characterize parameter estimates for the model. For example, A&R (p. 189) state that “if the regression errors are non-normal, we are not even sure of their [e.g., the generalized least squares parameter estimates1] asymptotic properties.” A partial answer, however, is given by Spall and Wall (1984), which presents an asymptotic distribution theory for Kalman filter estimates for cases where the random terms of the state space model are not necessarily Gaussian. Certain of these asymptotic distribution results are also discussed in Spall (1985) in the context of model validation (diagnostic checking)     

Ridge Regression: Degrees of Freedom in the Analysis of Variance

It appears to be common practice with ridge regression to obtain a decomposition of the total sum of squares, and assign degrees of freedom, according to established least squares theory. This discussion notes the obvious fallacies of such an approach, and introduces a decomposition based on orthogonality, and degrees of freedom based on expected mean squares, for non-stochastic k.     

An efficient cross-validation algorithm for window width selection for nonparametric kernel regression

This paper presents an approach to cross-validated window width choice which greatly reduces computation time, which can be used regardless of the nature of the kernel function, and which avoids the use of the Fast Fourier Transform. This approach is developed for window width selection in the context of kernel estimation of an unknown conditional mean.     

A boundary kernel for local polynomial regression

We define and compute a boundary kernel for local polynomial regression, We prove that the new kernel provides improvement over the existing kernels, Simulations show the improvement in finite samples.