A New STATISTIC FOR DETECTING INFLUENTIAL OBSERVATIONS IN A SCHEFFE' TYPE CALIBRATION CURVE

A statistic for identifying influential observations in calibration is given. The statistic is easy to interpret, and provides a useful measure of influence for Scheffé type calibration curves.     

A GRAPHICAL TECHNIQUE FOR DETECTING INFLUENTIAL CASES IN REGRESSION ANALYSIS

This paper presents a graphical technique for detecting influential cases in regression analysis. The idea is to decompose a diagnostic problem involving higher order dimensional regression problems, into a series of two-dimensional diagnostic sub-problems, such that the diagnoses of influential cases is undertaken by visually inspecting two-dimensional diagnostic plots of these sub-problems. An algorithm for the graphical procedure is proposed to reduce the computational effort. Practical examples are used to illustrate this graphical technique.     

Influence Functions for Sliced Inverse Regression

Abstract.  Sliced inverse regression (SIR) is a dimension reduction technique that is both efficient and simple to implement. The procedure itself relies heavily on estimates that are known to be highly non-robust and, as such, the issue of robustness is often raised. This paper looks at the robustness of SIR by deriving and plotting the influence function for a variety of contamination structures. The sample influence function is also considered and used to highlight that common outlier detection and deletion methods may not be entirely useful to SIR. The asymptotic variance of the estimates is also derived for the single index model when the explanatory variable is known to be normally distributed. The asymptotic variance is then compared for varying choices of the number of slices for a simple model example.     

Influence Functions for Dimension Reduction Methods: An Example Influence Study of Principal Hessian Direction Analysis

Abstract. The first goal of this article is to consider influence analysis of principal Hessian directions (pHd) and highlight how such an analysis can provide valuable insight into its behaviour. Such insight includes reasons as to why pHd can sometimes return informative results when it is not expected to do so, and why many prefer a residuals‐based pHd method over its response‐based counterpart. The secondary goal of this article is to introduce a new influence measure applicable to many dimension reduction methods based on average squared canonical correlations. A general form of this measure is also given, allowing for application to dimension reduction methods other than pHd. A sample version of the measure is considered, with respect to pHd, with two example data sets.     

Special Invited Paper: Dimension Reduction and Visualization in Discriminant Analysis (with discussion)

This paper discusses visualization methods for discriminant analysis. It does not address numerical methods for classification per se, but rather focuses on graphical methods that can be viewed as pre-processors, aiding the analyst's understanding of the data and the choice of a final classifier. The methods are adaptations of recent results in dimension reduction for regression, including sliced inverse regression and sliced average variance estimation. A permutation test is suggested as a means of determining dimension, and examples are given throughout the discussion.     

K-medoids inverse regression

Abstract

K-means inverse regression was developed as an easy-to-use dimension reduction procedure for multivariate regression. This approach is similar to the original sliced inverse regression method, with the exception that the slices are explicitly produced by a K-means clustering of the response vectors. In this article, we propose K-medoids clustering as an alternative clustering approach for slicing and compare its performance to K-means in a simulation study. Although the two methods often produce comparable results, K-medoids tends to yield better performance in the presence of outliers. In addition to isolation of outliers, K-medoids clustering also has the advantage of accommodating a broader range of dissimilarity measures, which could prove useful in other graphical regression applications where slicing is required.     

An Empirical Process View of Inverse Regression

A common approach taken in high‐dimensional regression analysis is sliced inverse regression, which separates the range of the response variable into non‐overlapping regions, called ‘slices’. Asymptotic results are usually shown assuming that the slices are fixed, while in practice, estimators are computed with random slices containing the same number of observations. Based on empirical process theory, we present a unified theoretical framework to study these techniques, and revisit popular inverse regression estimators. Furthermore, we introduce a bootstrap methodology that reproduces the laws of Cramér–von Mises test statistics of interest to model dimension, effects of specified covariates and whether or not a sliced inverse regression estimator is appropriate. Finally, we investigate the accuracy of different bootstrap procedures by means of simulations.     

A new reproducing kernel-based nonlinear dimension reduction method for survival data

Based on the theories of sliced inverse regression (SIR) and reproducing kernel Hilbert space (RKHS), a new approach RDSIR (RKHS-based Double SIR) to nonlinear dimension reduction for survival data is proposed. An isometric isomorphism is constructed based on the RKHS property, then the nonlinear function in the RKHS can be represented by the inner product of two elements that reside in the isomorphic feature space. Due to the censorship of survival data, double slicing is used to estimate the weight function to adjust for the censoring bias. The nonlinear sufficient dimension reduction (SDR) subspace is estimated by a generalized eigen-decomposition problem. The asymptotic property of the estimator is established based on the perturbation theory. Finally, the performance of RDSIR is illustrated on simulated and real data. The numerical results show that RDSIR is comparable with the linear SDR method. Most importantly, RDSIR can also effectively extract nonlinearity from survival data.     

On Sensitivity of Inverse Response Plot Estimation and the Benefits of a Robust Estimation Approach

Abstract. Inverse response plots are a useful tool in determining a response transformation function for response linearization in regression. Under some mild conditions it is possible to seek such transformations by plotting ordinary least squares fits versus the responses. A common approach is then to use nonlinear least squares to estimate a transformation by modelling the fits on the transformed response where the transformation function depends on an unknown parameter to be estimated. We provide insight into this approach by considering sensitivity of the estimation via the influence function. For example, estimation is insensitive to the method chosen to estimate the fits in the initial step. Additionally, the inverse response plot does not provide direct information on how well the transformation parameter is being estimated and poor inverse response plots may still result in good estimates. We also introduce a simple robustified process that can vastly improve estimation.     

An Inverse‐regression Method of Dependent Variable Transformation for Dimension Reduction with Non‐linear Confounding

Many model‐free dimension reduction methods have been developed for high‐dimensional regression data but have not paid much attention on problems with non‐linear confounding. In this paper, we propose an inverse‐regression method of dependent variable transformation for detecting the presence of non‐linear confounding. The benefit of using geometrical information from our method is highlighted. A ratio estimation strategy is incorporated in our approach to enhance the interpretation of variable selection. This approach can be implemented not only in principal Hessian directions (PHD) but also in other recently developed dimension reduction methods. Several simulation examples that are reported for illustration and comparisons are made with sliced inverse regression and PHD in ignorance of non‐linear confounding. An illustrative application to one real data is also presented.     

DO NOT WEIGHT FOR HETEROSCEDASTICITY IN NONPARAMETRIC REGRESSION

The potential role of weighting in kernel regression is examined. The concept that weighting has something to do with heteroscedastic errors is shown to be false. However, weighting does affect bias, and ways in which this might be exploited are indicated.     

Sensitivity analysis in maximum likelihood factor analysis

The present paper deals with sensitivity analysis in maximum likelihood factor analysis. To investigate the influence of a small change of data we derive theoretical influence functions I(x; LL^T ) and I(x; Δ) for a common variance matrix T= LL^T and a unique variance matrix Δ respectively. Numerical examples are shown to illustrate our procedure.     

Dimension reduction boosting

L₂Boosting is an effective method for constructing model. In the case of high-dimensional setting, Bühlmann and Yu (2003 Bühlmann, P., Yu, B. (2003). Boosting with the L₂-loss: regression and classification. J. Amer. Stat. Assoc. 98:324–339.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]) proposed the componentwise L₂Boosting, but componentwise L₂Boosting can only fit a special limited model. In this paper, by combining a boosting and sufficient dimension reduction method, e.g., sliced inverse regression (SIR), we propose a new method for regression, called dimension reduction boosting (DRBoosting). Compared with L₂Boosting, the computation of DRBoosting is less intensive and its prediction is better, especially for high-dimensional data. Simulations confirm the advantage of the new method.     

LOCATION OF PARTICULAR POINTS IN NON-PARAMETRIC REGRESSION ANALYSIS

In the random-design non-parametric regression model, the locations of particular values of the regression function or its derivatives are estimated. This paper investigates several stochastic modes of convergence and finds their rate of convergence under regularity assumptions, for a wide class of non-parametric estimators. The approach finds two natural fields of application: estimation of zeros/extrema and non-parametric absolute calibration.     

CASE-DELETION DIAGNOSTICS FOR TEST STATISTICS IN MULTIVARIATE REGRESSION

Influence measures in multivariate regression analysis have been widely developed, especially through use of the case-deletion approach. However, there seem to be few accounts of the influence of observations on test statistics in hypothesis testing. This paper examines four common multivariate tests, namely the Wilks' ratio, Lawley-Hotelling trace, Pillai's trace and Roy's greatest root for testing a general linear hypothesis of the regression coefficients in multivariate regression. The influence of observations is measured using the case-deletion approach. The proposed diagnostic measures, except that of Roy's greatest root, can be expressed in terms of statistics without involving the actual deletion of observations. An illustrative example is given with satisfactory results.     

A mixture model for dimension reduction

The existence of a dimension reduction (DR) subspace is a common assumption in regression analysis when dealing with high-dimensional predictors. The estimation of such a DR subspace has received considerable attention in the past few years, the most popular method being undoubtedly the sliced inverse regression. In this paper, we propose a new estimation procedure of the DR subspace by assuming that the joint distribution of the predictor and the response variables is a finite mixture of distributions. The new method is compared through a simulation study to some classical methods.     

Pairwise directions estimation for multivariate response regression data

This article concerns the analysis of multivariate response data with multi-dimensional covariates. Based on local linear smoothing techniques, we propose an iteratively adaptive estimation method to reduce the dimensions of response variables and covariates. Two weighted estimation strategies are incorporated in our approach to provide initial estimates. Our proposal is also extended to curve response data for a data-adaptive basis function searching. Instead of focusing on goodness of fit, we shift the problem to reveal the data structure and basis patterns. Simulation studies with multivariate response and curve data are conducted for our pairwise directions estimation (PDE) approach in comparison with sliced inverse regression of Li et al. [Dimension reduction for multivariate response data. J Amer Statist Assoc. 2003;98:99–109]. The results demonstrate that the proposed PDE method is useful for data with responses approximating linear or bending structures. Illustrative applications to two real datasets are also presented.     

DELETE-2 AND DELETE-3 JACKKNIFE PROCEDURES FOR UNMASKING IN REGRESSION

Single-case deletion regression diagnostics have been used widely to discover unusual data points, but such approaches can fail in the presence of multiple unusual data points and as a result of masking. We propose a new approach to the use of single-case deletion diagnostics that involves applying these diagnostics to delete-2 and delete-3 jackknife replicates of the data, and considering the percentage of times among these replicates that points are flagged as unusual as an indicator of their influence. By considering replicates that exclude certain collections of points, subtle masking effects can be uncovered.