Sensitivity analysis in covariance structure analysis with equality constraints

Influence functions are derived for covariance structure analysis with equality constraints, where the parameters are estimated by minimizing a discrepancy function between the assumed covariance matrix and the sample covariance matrix. As a special case maximum likelihood exploratory factor analysis is studied precisely with a numerical example. Comparison is made with the the results of Tanaka and Odaka (1989), who have proposed a sensitivity analysis procedure in maximum likelihood exploratory factor analysis using the perturbation expansion of a certain function of eigenvalues and eigenvectors of a real symmetric matrix. Also the present paper gives a generalization of Tanaka, Watadani and Moon (1991) to the case with equality constraints.     

Sensitivity analysis in multivariate methods: Decomposition of an arbitrary influence into a finite number of components

The influence function of the covariance matrix is decomposed into a finite number of components. This decomposition provides a useful tool to develop efficient methods for computing empirical influence curves related to various multivariate methods. It can also be used to characterize multivariate methods from the sensitivity perspective. A numerical example is given to demonstrate efficient computing and to characterize some procedures of exploratory factor analysis.     

Robustness of bayesian factor analysis estimates

This paper presents the result of a study of the robustness of posterior estimators of the factor loading matrix, the factor scores, and the disturbance covariance matrix (the main model parameters) in a Bayesian factor analysis with respect to variations in the values of the parameters of their prior distributions (the hyperparameter). We adopt the ε - contamination model of Berger and Berliner(1986) to generate prior distributions whose hyper-paramters reflects small variations in the elements of the uncontaminated hyperparameters, and we use directional derivatives to examine the variation of the uncontaminated estimators with respect to changes in the values of the hyperparameters, in the directions of the main model parameters. Several matrix norms are used to measure the closeness of the resulting values. We illustrate the results with a numerical example.     

Accent on Teaching Materials

Estimation of covariance components in the multivariate random-effect model with nested covariance structure is discussed. There are two covariance matrices to be estimated, namely, the between-group and the within-group covariance matrices. These two covariance matrices are most often estimated by forming a multivariate analysis of variance and equating mean square matrices to their expectations. Such a procedure involves taking the difference between the between-group mean square and the within-group mean square matrices, and often produces an estimated between-group covariance matrix that is not nonnegative definite. We present estimators of the two covariance matrices that are always proper covariance matrices. The estimators are the restricted maximum likelihood estimators if the random effects are normally distributed. The estimation procedure is extended to more complicated models, including the twofold nested and the mixed-effect models. A numerical example is presented to illustrate the use of the estimation procedure.     

Bayesian tests on components of the compound symmetry covariance matrix

Complex dependency structures are often conditionally modeled, where random effects parameters are used to specify the natural heterogeneity in the population. When interest is focused on the dependency structure, inferences can be made from a complex covariance matrix using a marginal modeling approach. In this marginal modeling framework, testing covariance parameters is not a boundary problem. Bayesian tests on covariance parameter(s) of the compound symmetry structure are proposed assuming multivariate normally distributed observations. Innovative proper prior distributions are introduced for the covariance components such that the positive definiteness of the (compound symmetry) covariance matrix is ensured. Furthermore, it is shown that the proposed priors on the covariance parameters lead to a balanced Bayes factor, in case of testing an inequality constrained hypothesis. As an illustration, the proposed Bayes factor is used for testing (non-)invariant intra-class correlations across different group types (public and Catholic schools), using the 1982 High School and Beyond survey data.     

Bayesian analysis of joint mean and covariance models for longitudinal data

Efficient estimation of the regression coefficients in longitudinal data analysis requires a correct specification of the covariance structure. If misspecification occurs, it may lead to inefficient or biased estimators of parameters in the mean. One of the most commonly used methods for handling the covariance matrix is based on simultaneous modeling of the Cholesky decomposition. Therefore, in this paper, we reparameterize covariance structures in longitudinal data analysis through the modified Cholesky decomposition of itself. Based on this modified Cholesky decomposition, the within-subject covariance matrix is decomposed into a unit lower triangular matrix involving moving average coefficients and a diagonal matrix involving innovation variances, which are modeled as linear functions of covariates. Then, we propose a fully Bayesian inference for joint mean and covariance models based on this decomposition. A computational efficient Markov chain Monte Carlo method which combines the Gibbs sampler and Metropolis–Hastings algorithm is implemented to simultaneously obtain the Bayesian estimates of unknown parameters, as well as their standard deviation estimates. Finally, several simulation studies and a real example are presented to illustrate the proposed methodology.     

A Selection Procedure for the Number of Signals in Presence of Colored Noise

Ranking and selection theory is used to estimate the number of signals present in colored noise. The data structure follows the well-known MUSIC (MUltiple SIgnal Classification) model. We deal with the eigenvalues of a covariance matrix, using the MUSIC model and colored noise. The data matrix can be written as the product of two matrices. The first matrix is the sample covariance matrix of the observed vectors. The second matrix is the inverse of the sample covariance matrix of reference vectors. We propose a multi-step selection procedure to construct a confidence interval on the number of signals present in a data set. Properties of this procedure will be stated and proved. Those properties will be used to compute the required parameters (procedure constants). Numerical examples are given to illustrate our theory.     

General class of covariance structures for two or more repeated factors in longitudinal data analysis

The main difficulty in parametric analysis of longitudinal data lies in specifying covariance structure. Several covariance structures, which usually reflect one series of measurements collected over time, have been presented in the literature. However there is a lack of literature on covariance structures designed for repeated measures specified by more than one repeated factor. In this paper a new, general method of modelling covariance structure based on the Kronecker product of underlying factor specific covariance profiles is presented. The method has an attractive interpretation in terms of independent factor specific contribution to overall within subject covariance structure and can be easily adapted to standard software.     

Simultaneous variable and factor selection via sparse group lasso in factor analysis

This paper considers variable and factor selection in factor analysis. We treat the factor loadings for each observable variable as a group, and introduce a weighted sparse group lasso penalty to the complete log-likelihood. The proposal simultaneously selects observable variables and latent factors of a factor analysis model in a data-driven fashion; it produces a more flexible and sparse factor loading structure than existing methods. For parameter estimation, we derive an expectation-maximization algorithm that optimizes the penalized log-likelihood. The tuning parameters of the procedure are selected by a likelihood cross-validation criterion that yields satisfactory results in various simulation settings. Simulation results reveal that the proposed method can better identify the possibly sparse structure of the true factor loading matrix with higher estimation accuracy than existing methods. A real data example is also presented to demonstrate its performance in practice.     

Computational algorithms for the factor model

Algorithms for computing the maximum likelihood estimators and the estimated covariance matrix of the estimators of the factor model are derived. The algorithms are particularly suitable for large matrices and for samples that give zero estimates of some error variances. A method of constructing estimators for reduced models is presented. The algorithms can also be used for the multivariate errors-in-variables model with known error covariance matrix.     

Factor analysis regression

The paper describes two regression models—principal components and maximum-likelihood factor analysis—which may be used when the stochastic predictor varibles are highly intereorrelated and/or contain measurement error. The two problems can occur jointly, for example in social-survey data where the true (but unobserved) covariance matrix can be singular. Departure from singularity of the sample dispersion matrix is then due to measurement error. We first consider the more elementary principal components regression model, where it is shown that it can be derived as a special case of (i) canonical correlation, and (ii) restricted least squares. The second part consists of the more general maximum-likelihood factor-analysis regression model, which is derived from the generalized inverse of the product of two singular matrices. Also, it is proved that factor-analysis regression can be considered as an instrumental variables estimator and therefore does not depend on whether factors have been “properly” identified in terms of substantive behaviour. Consequently the additional task of rotating factors to “simple structure” does not arise.     

a bayesian analysis of the multivariate mixed-linear model

The Bayesian analysis of the multivariate mixed linear model is considered. The exact posterior distribution for the fixed effects matrix and the error covariance matrix are obtained. The exact posterior means and variances of the Bayesian estimators for the covariance matrices of random effects are also derived. These posterior moments are computed without constrained optimization and numerical integration. The calculations are feasible for arbitrary models. Reasonable approximations for the posterior distributions for the covariance matrices associated with the random effects are obtained also. Results are illustrated with a numerical example.     

Multivariate analysis for the two-period repeated measures crossover design with application to clinical trials

Multivariate analysis techniques are applied to the two-period repeated measures crossover design. The approach considered in this paper has the advantage over the univariate analysis approach proposed recently by Wallenstein and Fisher (1977) that the former does not require any specific structure on the variance-covariance matrix of the repeated measures factor. (It should be noted that sums and differences of observations over periods are used for all tests. Therefore, there are two matrices under consideration, one for sums and one for differences.) Tests of significance are derived using the Wilks? criterion, and the procedure is illustrated with a numerical example from the area of clinical trials.     

A skew-normal factor model for the analysis of student satisfaction towards university courses

Classical factor analysis relies on the assumption of normally distributed factors that guarantees the model to be estimated via the maximum likelihood method. Even when the assumption of Gaussian factors is not explicitly formulated and estimation is performed via the iterated principal factors’ method, the interest is actually mainly focussed on the linear structure of the data, since only moments up to the second ones are involved. In many real situations, the factors could not be adequately described by the first two moments only. For example, skewness characterizing most latent variables in social analysis can be properly measured by the third moment: the factors are not normally distributed and covariance is no longer a sufficient statistic. In this work we propose a factor model characterized by skew-normally distributed factors. Skew-normal refers to a parametric class of probability distributions, that extends the normal distribution by an additional shape parameter regulating the skewness. The model estimation can be solved by the generalized EM algorithm, in which the iterative Newthon–Raphson procedure is needed in the M-step to estimate the factor shape parameter. The proposed skew-normal factor analysis is applied to the study of student satisfaction towards university courses, in order to identify the factors representing different aspects of the latent overall satisfaction.     

Fast,closed-form,and efficient estimators for hierarchical models with AR(1) covariance and unequal cluster sizes

This article is concerned with statistically and computationally efficient estimation in a hierarchical data setting with unequal cluster sizes and an AR(1) covariance structure. Maximum likelihood estimation for AR(1) requires numerical iteration when cluster sizes are unequal. A near optimal non-iterative procedure is proposed. Pseudo-likelihood and split-sample methods are used, resulting in computing weights to combine cluster size specific parameter estimates. Results show that the method is statistically nearly as efficient as maximum likelihood, but shows great savings in computation time.     

Cox Regression with Incomplete Covariate Measurements using the EM-algorithm

Ibrahim (1990) used the EM-algorithm to obtain maximum likelihood estimates of the regression parameters in generalized linear models with partially missing covariates. The technique was termed EM by the method of weights. In this paper, we generalize this technique to Cox regression analysis with missing values in the covariates. We specify a full model letting the unobserved covariate values be random and then maximize the observed likelihood. The asymptotic covariance matrix is estimated by the inverse information matrix. The missing data are allowed to be missing at random but also the non-ignorable non-response situation may in principle be considered. Simulation studies indicate that the proposed method is more efficient than the method suggested by Paik & Tsai (1997). We apply the procedure to a clinical trials example with six covariates with three of them having missing values.     

A new per-field classification method using mixture discriminant analysis

In this study, a new per-field classification method is proposed for supervised classification of remotely sensed multispectral image data of an agricultural area using Gaussian mixture discriminant analysis (MDA). For the proposed per-field classification method, multivariate Gaussian mixture models constructed for control and test fields can have fixed or different number of components and each component can have different or common covariance matrix structure. The discrimination function and the decision rule of this method are established according to the average Bhattacharyya distance and the minimum values of the average Bhattacharyya distances, respectively. The proposed per-field classification method is analyzed for different structures of a covariance matrix with fixed and different number of components. Also, we classify the remotely sensed multispectral image data using the per-pixel classification method based on Gaussian MDA.     

Choosing appropriate covariance matrices in a nonparametric analysis of factorials in block designs

The standard nonparametric, rank-based approach to the analysis of dependent data from factorial designs is based on an estimated unstructured (UN) variance–covariance matrix, but the large number of variance–covariance terms in many designs can seriously affect test performance. In a simulation study for a factorial arranged in blocks, we compared estimates of type-I error probability and power based on the UN structure with the estimates obtained with a more parsimonious heterogeneous-compound-symmetry structure (CSH). Although tests based on the UN structure were anti-conservative with small number of factor levels, especially with four or six blocks, they became conservative at higher number of factor levels. Tests based on the CSH structure were anti-conservative, and results did not depend on the number of factor levels. When both tests were anti-conservative, tests based on the CSH structure were less so. Although use of the CSH structure is concluded to be more suitable than use of the UN structure for the small number of blocks typical in agricultural experiments, results suggest that further improvement of test statistics is needed for such situations.     

Supervised classifiers for high-dimensional higher-order data with locally doubly exchangeable covariance structure

We explore the performance accuracy of the linear and quadratic classifiers for high-dimensional higher-order data, assuming that the class conditional distributions are multivariate normal with locally doubly exchangeable covariance structure. We derive a two-stage procedure for estimating the covariance matrix: at the first stage, the Lasso-based structure learning is applied to sparsifying the block components within the covariance matrix. At the second stage, the maximum-likelihood estimators of all block-wise parameters are derived assuming the doubly exchangeable within block covariance structure and a Kronecker product structured mean vector. We also study the effect of the block size on the classification performance in the high-dimensional setting and derive a class of asymptotically equivalent block structure approximations, in a sense that the choice of the block size is asymptotically negligible.     

Two Graphical Techniques Useful in Detecting Correlation Structure in Repeated Measures Data

Analysis of repeated measures data using a mixed model includes specifying a form for the covariance matrix of the within-subject observations. This reduction in the number of estimated parameters from the unspecified structure may improve the efficiency of inferences made. An implementation of this technique has been incorporated in the MIXED procedure of the SAS® statistical package, and includes a wide range of options for the structure of the covariance matrix. It is demonstrated that draftman's display plots and/or plots in a coordinate system with parallel axes can aid in visualizing the dispersion structure.