Nonlinear structural equation modeling: is partial least squares an alternative?

Nonlinear structural equation modeling provides many advantages over analyses based on manifest variables only. Several approaches for the analysis of latent interaction effects have been developed within the last 15 years, including the partial least squares product indicator approach (PLS-PI), the constrained product indicator approach using the LISREL software (LISREL-PI), and the distribution-analytic latent moderated structural equations approach (LMS) using the Mplus program. An assumed advantage of PLS-PI is that it is able to deal with very large numbers of indicators, while LISREL-PI and LMS have not been investigated under such conditions. In a Monte Carlo study, the performance of LISREL-PI and LMS was compared to PLS-PI results previously reported in Chin et al. (2003) and Goodhue et al. (2007) for identical conditions. The latent interaction model included six indicator variables for the measurement of each latent predictor variable and the latent criterion, and sample size was N=100. The results showed that PLS-PI’s linear and interaction parameter estimates were downward biased, while parameter estimates were unbiased for LISREL-PI and LMS. True standard errors were smallest for PLS-PI, while the power to detect the latent interaction effect was higher for LISREL-PI and LMS. Compared to the symmetric distributions of interaction parameter estimates for LISREL-PI and LMS, PLS-PI showed a distribution that was symmetric for positive values, but included outlying negative estimates. Possible explanations for these findings are discussed.     

Multivariate Latent Growth Models for Mixed Data with Covariate Effects

The paper presents an extension of a new class of multivariate latent growth models (Bianconcini and Cagnone, 2012) to allow for covariate effects on manifest, latent variables and random effects. The new class of models combines: (i) multivariate latent curves that describe the temporal behavior of the responses, and (ii) a factor model that specifies the relationship between manifest and latent variables. Based on the Generalized Linear and Latent Variable Model framework (Bartholomew and Knott, 1999), the response variables are assumed to follow different distributions of the exponential family, with item-specific linear predictors depending on both latent variables and measurement errors. A full maximum likelihood method is used to estimate all the model parameters simultaneously. Data coming from the Data WareHouse of the University of Bologna are used to illustrate the methodology.     

The partial least squares-fix point method of estimating interdependent systems with latent variables

This paper describes a method for estimating the unknown parameters of an interdependent simultaneous equations model with latent variables. For each latent variable there may be single or multiple indicators. Estimation proceeds in three stages: first, estimates of the latent variables are constructed from the associated manifest indicators; second, treating the estimates as directly observed, fix-point estimates of the structural form parameters are obtained; third, the location parameters are estimated. The method involves only repeated application of ordinary least squares and no distributional assumptions are needed. The paper concludes with an empirical application of the method.     

Latent variable models with mixed continuous and polytomous data

Owing to the nature of the problems and the design of questionnaires, discrete polytomous data are very common in behavioural, medical and social research. Analysing the relationships between the manifest and the latent variables based on mixed polytomous and continuous data has proven to be difficult. A general structural equation model is investigated for these mixed outcomes. Maximum likelihood (ML) estimates of the unknown thresholds and the structural parameters in the covariance structure are obtained. A Monte Carlo–EM algorithm is implemented to produce the ML estimates. It is shown that closed form solutions can be obtained for the M-step, and estimates of the latent variables are produced as a by-product of the analysis. The method is illustrated with a real example.     

A mixture latent variable model for modeling mixed data in heterogeneous populations and its applications

Latent variable models are widely used for jointly modeling of mixed data including nominal, ordinal, count and continuous data. In this paper, we consider a latent variable model for jointly modeling relationships between mixed binary, count and continuous variables with some observed covariates. We assume that, given a latent variable, mixed variables of interest are independent and count and continuous variables have Poisson distribution and normal distribution, respectively. As such data may be extracted from different subpopulations, consideration of an unobserved heterogeneity has to be taken into account. A mixture distribution is considered (for the distribution of the latent variable) which accounts the heterogeneity. The generalized EM algorithm which uses the Newton–Raphson algorithm inside the EM algorithm is used to compute the maximum likelihood estimates of parameters. The standard errors of the maximum likelihood estimates are computed by using the supplemented EM algorithm. Analysis of the primary biliary cirrhosis data is presented as an application of the proposed model.     

Max-Linear Competing Factor Models

Models incorporating “latent” variables have been commonplace in financial, social, and behavioral sciences. Factor model, the most popular latent model, explains the continuous observed variables in a smaller set of latent variables (factors) in a matter of linear relationship. However, complex data often simultaneously display asymmetric dependence, asymptotic dependence, and positive (negative) dependence between random variables, which linearity and Gaussian distributions and many other extant distributions are not capable of modeling. This article proposes a nonlinear factor model that can model the above-mentioned variable dependence features but still possesses a simple form of factor structure. The random variables, marginally distributed as unit Fréchet distributions, are decomposed into max linear functions of underlying Fréchet idiosyncratic risks, transformed from Gaussian copula, and independent shared external Fréchet risks. By allowing the random variables to share underlying (latent) pervasive risks with random impact parameters, various dependence structures are created. This innovates a new promising technique to generate families of distributions with simple interpretations. We dive in the multivariate extreme value properties of the proposed model and investigate maximum composite likelihood methods for the impact parameters of the latent risks. The estimates are shown to be consistent. The estimation schemes are illustrated on several sets of simulated data, where comparisons of performance are addressed. We employ a bootstrap method to obtain standard errors in real data analysis. Real application to financial data reveals inherent dependencies that previous work has not disclosed and demonstrates the model’s interpretability to real data. Supplementary materials for this article are available online.     

Modeling Longitudinal Obesity Data with Intermittent Missingness Using a New Latent Variable Model

We propose a latent variable model for informative missingness in longitudinal studies which is an extension of latent dropout class model. In our model, the value of the latent variable is affected by the missingness pattern and it is also used as a covariate in modeling the longitudinal response. So the latent variable links the longitudinal response and the missingness process. In our model, the latent variable is continuous instead of categorical and we assume that it is from a normal distribution. The EM algorithm is used to obtain the estimates of the parameter we are interested in and Gauss–Hermite quadrature is used to approximate the integration of the latent variable. The standard errors of the parameter estimates can be obtained from the bootstrap method or from the inverse of the Fisher information matrix of the final marginal likelihood. Comparisons are made to the mixed model and complete-case analysis in terms of a clinical trial dataset, which is Weight Gain Prevention among Women (WGPW) study. We use the generalized Pearson residuals to assess the fit of the proposed latent variable model.     

Structural equation models for area health outcomes with model selection

Recent analyses seeking to explain variation in area health outcomes often consider the impact on them of latent measures (i.e. unobserved constructs) of population health risk. The latter are typically obtained by forms of multivariate analysis, with a small set of latent constructs derived from a collection of observed indicators, and a few recent area studies take such constructs to be spatially structured rather than independent over areas. A confirmatory approach is often applicable to the model linking indicators to constructs, based on substantive knowledge of relevant risks for particular diseases or outcomes. In this paper, population constructs relevant to a particular set of health outcomes are derived using an integrated model containing all the manifest variables, namely health outcome variables, as well as indicator variables underlying the latent constructs. A further feature of the approach is the use of variable selection techniques to select significant loadings and factors (especially in terms of effects of constructs on health outcomes), so ensuring parsimonious models are selected. A case study considers suicide mortality and self-harm contrasts in the East of England in relation to three latent constructs: deprivation, fragmentation and urbanicity.     

Estimation of generalized linear latent variable models

Summary.  Generalized linear latent variable models (GLLVMs), as defined by Bartholomew and Knott, enable modelling of relationships between manifest and latent variables. They extend structural equation modelling techniques, which are powerful tools in the social sciences. However, because of the complexity of the log-likelihood function of a GLLVM, an approximation such as numerical integration must be used for inference. This can limit drastically the number of variables in the model and can lead to biased estimators. We propose a new estimator for the parameters of a GLLVM, based on a Laplace approximation to the likelihood function and which can be computed even for models with a large number of variables. The new estimator can be viewed as an M -estimator, leading to readily available asymptotic properties and correct inference. A simulation study shows its excellent finite sample properties, in particular when compared with a well-established approach such as LISREL. A real data example on the measurement of wealth for the computation of multidimensional inequality is analysed to highlight the importance of the methodology.     

Latent Variable Models for Mixed Discrete and Continuous Outcomes

We propose a latent variable model for mixed discrete and continuous outcomes. The model accommodates any mixture of outcomes from an exponential family and allows for arbitrary covariate effects, as well as direct modelling of covariates on the latent variable. An EM algorithm is proposed for parameter estimation and estimates of the latent variables are produced as a by-product of the analysis. A generalized likelihood ratio test can be used to test the significance of covariates affecting the latent outcomes. This method is applied to birth defects data, where the outcomes of interest are continuous measures of size and binary indicators of minor physical anomalies. Infants who were exposed in utero to anticonvulsant medications are compared with controls.     

Robust multivariate mixture regression models with incomplete data

Multivariate mixture regression models can be used to investigate the relationships between two or more response variables and a set of predictor variables by taking into consideration unobserved population heterogeneity. It is common to take multivariate normal distributions as mixing components, but this mixing model is sensitive to heavy-tailed errors and outliers. Although normal mixture models can approximate any distribution in principle, the number of components needed to account for heavy-tailed distributions can be very large. Mixture regression models based on the multivariate t distributions can be considered as a robust alternative approach. Missing data are inevitable in many situations and parameter estimates could be biased if the missing values are not handled properly. In this paper, we propose a multivariate t mixture regression model with missing information to model heterogeneity in regression function in the presence of outliers and missing values. Along with the robust parameter estimation, our proposed method can be used for (i) visualization of the partial correlation between response variables across latent classes and heterogeneous regressions, and (ii) outlier detection and robust clustering even under the presence of missing values. We also propose a multivariate t mixture regression model using MM-estimation with missing information that is robust to high-leverage outliers. The proposed methodologies are illustrated through simulation studies and real data analysis.     

Selection of Latent Variables for Multiple Mixed‐outcome Models

Latent variable models have been widely used for modelling the dependence structure of multiple outcomes data. However, the formulation of a latent variable model is often unknown a priori, the misspecification will distort the dependence structure and lead to unreliable model inference. Moreover, multiple outcomes with varying types present enormous analytical challenges. In this paper, we present a class of general latent variable models that can accommodate mixed types of outcomes. We propose a novel selection approach that simultaneously selects latent variables and estimates parameters. We show that the proposed estimator is consistent, asymptotically normal and has the oracle property. The practical utility of the methods is confirmed via simulations as well as an application to the analysis of the World Values Survey, a global research project that explores peoples’ values and beliefs and the social and personal characteristics that might influence them.     

Modelling survival data using mixtures of frailties

Frailty models are often used to describe the extra heterogeneity in survival data by introducing an individual random, unobserved effect. The frailty term is usually assumed to act multiplicatively on a baseline hazard function common to all individuals. In order to apply the frailty model, a specific frailty distribution has to be assumed. If at least one of the latent variables is continuous, the frailty must follow a continuous distribution. In this paper, a finite mixture of continuous frailty distributions is used in order to describe situations in which one (or more) of the latent variables separates the population in study into two (or more) subpopulations. Closure properties of the unobserved quantity are given along with the maximum-likelihood estimates under the most common choices of frailty distributions. The model is illustrated on a set of lifetime data.     

有监督Group MCP方法的稳健性研究

采用模拟研究的方法,分别在回归预测和分类判别两种环境中讨论有监督Group MCP方法在不同结构错误率下进行变量选择和结果预测的稳健性,并通过实例分析讨论本研究的实用价值。研究结果显示:忽略解释变量的内部结构进行变量选择会导致很多重要解释变量被疏漏,而有监督Group MCP方法考虑了解释变量的内部结构,在结构错误率低于5%时会以不低于98%的概率选出有效解释变量,并尽量降低冗余变量被选择的可能性。此研究成果为有监督Group MCP方法的合理使用奠定了基础。     

Bayesian analysis for confirmatory factor model with finite-dimensional Dirichlet prior mixing

Confirmatory factor analysis (CFA) model is a useful multivariate statistical tool for interpreting relationships between latent variables and manifest variables. Often statistical results based on a single CFA are seriously distorted when data set takes on heterogeneity. To address the heterogeneity resulting from the multivariate responses, we propose a Bayesian semiparametric modeling for CFA. The approach relies on using a prior over the space of mixing distributions with finite components. Blocked Gibbs sampler is implemented to cope with the posterior analysis. Results obtained from a simulation study and a real data set are presented to illustrate the methodology.     

Modified Wynn's Sequential Algorithm for Constructing D-Optimal Designs: Adding Two Points at a Time

Partial least squares (PLS) is a class of methods for modeling relations between sets of observed variables by using the latent components where the predictors are highly collinear. SIMPLS is a commonly used PLS algorithm that calculates the latent components directly as linear combinations of the original variables. However, SIMPLS is known to be very sensible to outliers since it is based on the empirical cross-covariance matrix. RoPLS is a recently proposed iterative method for robust SIMPLS. In this article, the influence function for the RoPLS coefficient estimator is derived. It is demonstrated that under certain conditions, the RoPLS estimator has infinitesimal robustness.     

Latent Variable Modelling: A Survey*

Abstract. Latent variable modelling has gradually become an integral part of mainstream statistics and is currently used for a multitude of applications in different subject areas. Examples of ‘traditional’ latent variable models include latent class models, item–response models, common factor models, structural equation models, mixed or random effects models and covariate measurement error models. Although latent variables have widely different interpretations in different settings, the models have a very similar mathematical structure. This has been the impetus for the formulation of general modelling frameworks which accommodate a wide range of models. Recent developments include multilevel structural equation models with both continuous and discrete latent variables, multiprocess models and nonlinear latent variable models.     

BAYESIAN PREDICTION FOR SPATIAL GENERALISED LINEAR MIXED MODELS WITH CLOSED SKEW NORMAL LATENT VARIABLES

Spatial generalised linear mixed models are used commonly for modelling non‐Gaussian discrete spatial responses. In these models, the spatial correlation structure of data is modelled by spatial latent variables. Most users are satisfied with using a normal distribution for these variables, but in many applications it is unclear whether or not the normal assumption holds. This assumption is relaxed in the present work, using a closed skew normal distribution for the spatial latent variables, which is more flexible and includes normal and skew normal distributions. The parameter estimates and spatial predictions are calculated using the Markov Chain Monte Carlo method. Finally, the performance of the proposed model is analysed via two simulation studies, followed by a case study in which practical aspects are dealt with. The proposed model appears to give a smaller cross‐validation mean square error of the spatial prediction than the normal prior in modelling the temperature data set.     

Estimation in Truncated GLG Model for Ordered Categorical Spatial Data Using the SAEM Algorithm

In this article, we utilize a scale mixture of Gaussian random field as a tool for modeling spatial ordered categorical data with non-Gaussian latent variables. In fact, we assume a categorical random field is created by truncating a Gaussian Log-Gaussian latent variable model to accommodate heavy tails. Since the traditional likelihood approach for the considered model involves high-dimensional integrations which are computationally intensive, the maximum likelihood estimates are obtained using a stochastic approximation expectation–maximization algorithm. For this purpose, Markov chain Monte Carlo methods are employed to draw from the posterior distribution of latent variables. A numerical example illustrates the methodology.     

Bayesian latent variable models for clustered mixed outcomes

A general framework is proposed for modelling clustered mixed outcomes. A mixture of generalized linear models is used to describe the joint distribution of a set of underlying variables, and an arbitrary function relates the underlying variables to be observed outcomes. The model accommodates multilevel data structures, general covariate effects and distinct link functions and error distributions for each underlying variable. Within the framework proposed, novel models are developed for clustered multiple binary, unordered categorical and joint discrete and continuous outcomes. A Markov chain Monte Carlo sampling algorithm is described for estimating the posterior distributions of the parameters and latent variables. Because of the flexibility of the modelling framework and estimation procedure, extensions to ordered categorical outcomes and more complex data structures are straightforward. The methods are illustrated by using data from a reproductive toxicity study.