Sensitivity measures of influence on the loading matrix in exploratory factor analysis

For the detection of influential observations on the loading matrix of the factor analysis model, we propose to use the infinitesimal version of two matrix coefficients, including Escoufier (1973)'s also discussed the application in factor analysis of some sensitivity measures used for similar purposes in principal component analysis.     

Sensitivity analysis in principal component analysis:influence on the subspace spanned by principal components.

The problem of detecting influential observations in principalcomponent analysis was discussed by several authors. Radhakrishnan and kshirsagar ( 1981 ), Critchley ( 1985 ), jolliffe ( 1986 )among others discussed this topicby using the influence functions I(X;θ_s)and I(X;V_s)of eigenvalues and eigenvectors, which wwere derived under the assumption that the eigenvalues of interest were simple. In this paper we propose the influence functionsI(X;∑^q _s=1θ_sV_sV_s ^T)and I(x;∑^q _s=1V_sV_s ^t)(q<p;p:number of variables) to investigate the influence onthe subspace spanned by principal components. These influence functions are applicable not only to the case where the edigenvalues of interst are all simple but also to the case where there are some multiple eigenvalues among those of interest.     

Quadratic perturbation expansions of certain functions of eigenvalues and eigenvectors and their application to sensitivity analysis in multivariate methods

Tanaka (1988) lias derived the influence functions, which are equivalent to the perturbation expansions up to linear terms, of two functions of eigenvalues and eigenvectors of a real symmetric matrix, and applied them to principal component analysis. The present paper deals with the perturbation expansions up to quadratic terms of the same functions and discusses their application to sensitivity analysis in multivariate methods, in particular, principal component analysis and principal factor analysis. Numerical examples are given to show how the approximation improves with the quadratic terms.     

Some comments on escoufier's RV- coefficient as a sensitivity measure in principal component analysis

The RV-coefficient (Escoufier, 1973; Robert and Escoufier, 1976) is studied as a sensitivity coefficient of the subspace spanned by dominant eigenvectors in principal component analysis. We use the perturbation expansion up to second order term of the corresponding projection matrix. The relationship with the measures by Benasseni (1990) and Krzanowski (1979) is also discussed.     

On Sensitivity of Inverse Response Plot Estimation and the Benefits of a Robust Estimation Approach

Abstract. Inverse response plots are a useful tool in determining a response transformation function for response linearization in regression. Under some mild conditions it is possible to seek such transformations by plotting ordinary least squares fits versus the responses. A common approach is then to use nonlinear least squares to estimate a transformation by modelling the fits on the transformed response where the transformation function depends on an unknown parameter to be estimated. We provide insight into this approach by considering sensitivity of the estimation via the influence function. For example, estimation is insensitive to the method chosen to estimate the fits in the initial step. Additionally, the inverse response plot does not provide direct information on how well the transformation parameter is being estimated and poor inverse response plots may still result in good estimates. We also introduce a simple robustified process that can vastly improve estimation.     

Stepwise local influence in generalized autoregressive conditional heteroskedasticity models

Detection of outliers or influential observations is an important work in statistical modeling, especially for the correlated time series data. In this paper we propose a new procedure to detect patch of influential observations in the generalized autoregressive conditional heteroskedasticity (GARCH) model. Firstly we compare the performance of innovative perturbation scheme, additive perturbation scheme and data perturbation scheme in local influence analysis. We find that the innovative perturbation scheme give better result than other two schemes although this perturbation scheme may suffer from masking effects. Then we use the stepwise local influence method under innovative perturbation scheme to detect patch of influential observations and uncover the masking effects. The simulated studies show that the new technique can successfully detect a patch of influential observations or outliers under innovative perturbation scheme. The analysis based on simulation studies and two real data sets show that the stepwise local influence method under innovative perturbation scheme is efficient for detecting multiple influential observations and dealing with masking effects in the GARCH model.     

A Diagnostic Method for Principal Components Tests

The local influence method is adapted to testing hypotheses about principal components for investigating the influence of observations on the test statistic. Simultaneous perturbations on all observations are considered. The main diagnostic is the direction vector of the maximum slope of the surface formed by the perturbed test statistic. A perturbation is constructed whose result is the same as that of the influence function method. An example is given for illustration.     

Efficient iteration procedures for maximum likelihood factor analysis

A class of fast convergent iteration procedures for ML factor analysis is presented in this paper. It includes a further development of Jöreskog’s (1971 with van Thillo, 1977) Newton-Raphson-like procedure which is widely available in statistical program packages but which is inclined to fail when solving difficult problems. In a comparison of efficiency, besides these two algorithms, our own versions of two quasi-Newton methods, namely the Davidon-Fletcher-Powell (DFP) and the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method, are tested.     

DETECTING INFLUENTIAL OBSERVATIONS IN SLICED INVERSE REGRESSION ANALYSIS

The detection of influential observations on the estimation of the dimension reduction subspace returned by Sliced Inverse Regression (SIR) is considered. Although there are many measures to detect influential observations in related methods such as multiple linear regression, there has been little development in this area with respect to dimension reduction. One particular influence measure for a version of SIR is examined and it is shown, via simulation and example, how this may be used to detect influential observations in practice.     

Influence Diagnostics for Simultaneous Equations Models

This paper presents influence diagnostics for simultaneous equations models. It proposes residuals, leverage and other influence measures. A missing data method is adopted to minimize the masking effect due to case deletions. The assessment of local influence is also considered. The paper shows how to evaluate the effects that perturbations to the endogenous variables, predetermined variables and case weights may have on the parameter estimates. The diagnostics are illustrated with two examples.     

Influence Functions for Dimension Reduction Methods: An Example Influence Study of Principal Hessian Direction Analysis

Abstract. The first goal of this article is to consider influence analysis of principal Hessian directions (pHd) and highlight how such an analysis can provide valuable insight into its behaviour. Such insight includes reasons as to why pHd can sometimes return informative results when it is not expected to do so, and why many prefer a residuals‐based pHd method over its response‐based counterpart. The secondary goal of this article is to introduce a new influence measure applicable to many dimension reduction methods based on average squared canonical correlations. A general form of this measure is also given, allowing for application to dimension reduction methods other than pHd. A sample version of the measure is considered, with respect to pHd, with two example data sets.     

Further influence measures in factor analysis

Various influence measures are discussed for sensitivity analysis in factor analysis. The internal norm is used to characterize the vector-valued influence curves in factor analysis. Influence curves for the chi-square goodness of fit test, and the determinants of the model covariance matrix and unique variance matrix are derived. They are found to have simple formulas which are easy to be interpreted and have nice distributions for calibration. The likelihood displacement is also applied to sensitivity analysis in maximum likelihood factor analysis.     

Exploring Cook's Statistic Graphically

An index plot of Cook's statistic is frequently used to highlight influential observations. In this article we illustrate how enhanced higher dimensional plots of Cook's statistic can provide further useful information about sets of influential observations. We provide examples using normal and generalized linear models.     

Influence functions related to eigenvalue problems which appear in multivariate analysis

Tanaka(1988) derived two influence functions related to an ordinary eigenvalue problem (A–λ_s I)v_s = 0 of a real symmetric matrix A and used them for sensitivity analysis in principal component analysis. One of these influence functions was used to develop sensitivity analysis in factor analysis (see e.g. Tanaka and Odaka, 1988a). The present paper derives some additional influence functions related to an ordinary eigenvalue problem and also several influence functions related to a generalized eigenvalue problem (A–θ_s A)u_s = 0, where A and B are real symmetric and real symmetric positive definite matrices, respectively. These influence functions are applicable not only to the case where the eigenvalues of interest are all simple but also to the case where there are some multiple eigenvalues among those of interest.     

Influential observations in the analysis of variance

The effect of influentia lob servations on t h e parameter estimates of ordinary l e a s t squares regression models has received considerable attentio n fn the last decade. However, very little attention has been given t o the problem of in fluent ia lobserva- tions in the analysis of variance . The purpose of t h i s paper is t o show by way of examples that influential observations can alter the conclusions of tests of hypotheses in the analysis of variance . Regression diagnostics for identif y in g both extreme points and outliers can be used to reveal potential data and design problems.     

Sensitivity analysis in covariance structure analysis with equality constraints

Influence functions are derived for covariance structure analysis with equality constraints, where the parameters are estimated by minimizing a discrepancy function between the assumed covariance matrix and the sample covariance matrix. As a special case maximum likelihood exploratory factor analysis is studied precisely with a numerical example. Comparison is made with the the results of Tanaka and Odaka (1989), who have proposed a sensitivity analysis procedure in maximum likelihood exploratory factor analysis using the perturbation expansion of a certain function of eigenvalues and eigenvectors of a real symmetric matrix. Also the present paper gives a generalization of Tanaka, Watadani and Moon (1991) to the case with equality constraints.     

The influence of observations on misclassification probability in multiple discriminant analysis

The influence of observations in estimating the misclassification probability in multiple discriminant analysis is studied using the common omission approach. An empirical influence function for the misclassification probability is also derived, It can give a very good approximation to the omission approach, but the computational load is much reduced, Various extensions of the measures are suggested. The proposed measures are applied to the famous Iris data set. The same three observations are identified as having the most influence under different measures.     

Deletion diagnostics for generalized linear models using the adjusted Poisson likelihood function

In this article, we propose two novel diagnostic measures for the deletion of influential observations for regression parameters in the setting of generalized linear models. The proposed diagnostic methods are capable for detecting the influential observations under model misspecification, as long as the true underlying distributions have finite second moments.More specifically, it is demonstrated that the Poisson likelihood function can be properly adjusted to become asymptotically valid for practically all underlying discrete distributions. The adjusted Poisson regression model that achieves the robustness property is presented. Simulation studies and an illustration are performed to demonstrate the efficacy of the two novel diagnostic procedures.     

Further theoretical results and a comparison between two methods for approximating eigenvalues of perturbed covariance matrices

Covariance matrices, or in general matrices of sums of squares and cross-products, are used as input in many multivariate analyses techniques. The eigenvalues of these matrices play an important role in the statistical analysis of data including estimation and hypotheses testing. It has been recognized that one or few observations can exert an undue influence on the eigenvalues of a covariance matrix. The relationship between the eigenvalues of the covariance matrix computed from all data and the eigenvalues of the perturbed covariance matrix (a covariance matrix computed after a small subset of the observations has been deleted) cannot in general be written in closed-form. Two methods for approximating the eigenvalues of a perturbed covariance matrix have been suggested by Hadi (1988) and Wang and Nyquist (1991) for the case of a perturbation by a single observation. In this paper we improve on these two methods and give some additional theoretical results that may give further insight into the problem. We also compare the two improved approximations in terms of their accuracies.     

An implicit function based procedure for analyzing maximum likelihood estimates from nonidentically distributed data

A methodology is presented for gaining insight into properties — such as outlier influence, bias, and width of confidence intervals — of maximum likelihood estimates from nonidentically distributed Gaussian data. The methodology is based on an application of the implicit function theorem to derive an approximation to the maximum likelihood estimator. This approximation, unlike the maximum likelihood estimator, is expressed in closed form and thus it can be used in lieu of costly Monte Carlo simulation to study the properties of the maximum likelihood estimator.