A mixture latent variable model for modeling mixed data in heterogeneous populations and its applications

Latent variable models are widely used for jointly modeling of mixed data including nominal, ordinal, count and continuous data. In this paper, we consider a latent variable model for jointly modeling relationships between mixed binary, count and continuous variables with some observed covariates. We assume that, given a latent variable, mixed variables of interest are independent and count and continuous variables have Poisson distribution and normal distribution, respectively. As such data may be extracted from different subpopulations, consideration of an unobserved heterogeneity has to be taken into account. A mixture distribution is considered (for the distribution of the latent variable) which accounts the heterogeneity. The generalized EM algorithm which uses the Newton–Raphson algorithm inside the EM algorithm is used to compute the maximum likelihood estimates of parameters. The standard errors of the maximum likelihood estimates are computed by using the supplemented EM algorithm. Analysis of the primary biliary cirrhosis data is presented as an application of the proposed model.     

Evaluation of long-term health care services through a latent Markov model with covariates

We focus on the evaluation of the long-term health care services provided to elderly patients by nursing homes of four different health districts in the Umbria region (Italy). To this end, we analyze data coming from a longitudinal survey aimed at assessing several aspects of patient health conditions and develop an extended version of the latent Markov model with covariates, which allows us to deal with dropout and intermittent missing data patterns that are common in longitudinal studies. Maximum likelihood estimates are obtained by a two-step approach that allows for fast estimation of model parameters and prevents some drawbacks of the standard maximum likelihood method encountered in the presence of many response variables and covariates. In the application to the observed data, we show how to obtain indicators of the effectiveness of the health care services delivered by each health district, by means of a resampling procedure.     

Latent Variable Models for Mixed Discrete and Continuous Outcomes

We propose a latent variable model for mixed discrete and continuous outcomes. The model accommodates any mixture of outcomes from an exponential family and allows for arbitrary covariate effects, as well as direct modelling of covariates on the latent variable. An EM algorithm is proposed for parameter estimation and estimates of the latent variables are produced as a by-product of the analysis. A generalized likelihood ratio test can be used to test the significance of covariates affecting the latent outcomes. This method is applied to birth defects data, where the outcomes of interest are continuous measures of size and binary indicators of minor physical anomalies. Infants who were exposed in utero to anticonvulsant medications are compared with controls.     

Max-Linear Competing Factor Models

Models incorporating “latent” variables have been commonplace in financial, social, and behavioral sciences. Factor model, the most popular latent model, explains the continuous observed variables in a smaller set of latent variables (factors) in a matter of linear relationship. However, complex data often simultaneously display asymmetric dependence, asymptotic dependence, and positive (negative) dependence between random variables, which linearity and Gaussian distributions and many other extant distributions are not capable of modeling. This article proposes a nonlinear factor model that can model the above-mentioned variable dependence features but still possesses a simple form of factor structure. The random variables, marginally distributed as unit Fréchet distributions, are decomposed into max linear functions of underlying Fréchet idiosyncratic risks, transformed from Gaussian copula, and independent shared external Fréchet risks. By allowing the random variables to share underlying (latent) pervasive risks with random impact parameters, various dependence structures are created. This innovates a new promising technique to generate families of distributions with simple interpretations. We dive in the multivariate extreme value properties of the proposed model and investigate maximum composite likelihood methods for the impact parameters of the latent risks. The estimates are shown to be consistent. The estimation schemes are illustrated on several sets of simulated data, where comparisons of performance are addressed. We employ a bootstrap method to obtain standard errors in real data analysis. Real application to financial data reveals inherent dependencies that previous work has not disclosed and demonstrates the model’s interpretability to real data. Supplementary materials for this article are available online.     

Latent variable models with mixed continuous and polytomous data

Owing to the nature of the problems and the design of questionnaires, discrete polytomous data are very common in behavioural, medical and social research. Analysing the relationships between the manifest and the latent variables based on mixed polytomous and continuous data has proven to be difficult. A general structural equation model is investigated for these mixed outcomes. Maximum likelihood (ML) estimates of the unknown thresholds and the structural parameters in the covariance structure are obtained. A Monte Carlo–EM algorithm is implemented to produce the ML estimates. It is shown that closed form solutions can be obtained for the M-step, and estimates of the latent variables are produced as a by-product of the analysis. The method is illustrated with a real example.     

Selection of Latent Variables for Multiple Mixed‐outcome Models

Latent variable models have been widely used for modelling the dependence structure of multiple outcomes data. However, the formulation of a latent variable model is often unknown a priori, the misspecification will distort the dependence structure and lead to unreliable model inference. Moreover, multiple outcomes with varying types present enormous analytical challenges. In this paper, we present a class of general latent variable models that can accommodate mixed types of outcomes. We propose a novel selection approach that simultaneously selects latent variables and estimates parameters. We show that the proposed estimator is consistent, asymptotically normal and has the oracle property. The practical utility of the methods is confirmed via simulations as well as an application to the analysis of the World Values Survey, a global research project that explores peoples’ values and beliefs and the social and personal characteristics that might influence them.     

Nonlinear structural equation modeling: is partial least squares an alternative?

Nonlinear structural equation modeling provides many advantages over analyses based on manifest variables only. Several approaches for the analysis of latent interaction effects have been developed within the last 15 years, including the partial least squares product indicator approach (PLS-PI), the constrained product indicator approach using the LISREL software (LISREL-PI), and the distribution-analytic latent moderated structural equations approach (LMS) using the Mplus program. An assumed advantage of PLS-PI is that it is able to deal with very large numbers of indicators, while LISREL-PI and LMS have not been investigated under such conditions. In a Monte Carlo study, the performance of LISREL-PI and LMS was compared to PLS-PI results previously reported in Chin et al. (2003) and Goodhue et al. (2007) for identical conditions. The latent interaction model included six indicator variables for the measurement of each latent predictor variable and the latent criterion, and sample size was N=100. The results showed that PLS-PI’s linear and interaction parameter estimates were downward biased, while parameter estimates were unbiased for LISREL-PI and LMS. True standard errors were smallest for PLS-PI, while the power to detect the latent interaction effect was higher for LISREL-PI and LMS. Compared to the symmetric distributions of interaction parameter estimates for LISREL-PI and LMS, PLS-PI showed a distribution that was symmetric for positive values, but included outlying negative estimates. Possible explanations for these findings are discussed.     

A deflated indicators approach for estimating second-order reflective models through PLS-PM: an empirical illustration

The paper provides a procedure aimed at obtaining more interpretable second-order models estimated with the partial least squares-path modeling. Advantages in interpretation stem from the separation of the two sources of influence on the data. As a matter of fact, in hierarchical models effects on manifest variables (MVs) are assigned to both first-order (specific) factors and second-order (general) factors. In order to separate these overlapping contributions, MVs are deflated from the effect of the specific latent variables (LVs) and used as indicators of the second-order LV. A case study is presented in order to illustrate the application of the proposed method.     

On the parameters of Zenga distribution

In 2010 Zenga introduced a new three-parameter model for distributions by size that can be used to represent income, wealth, financial and actuarial variables. This paper proposes a summary of its main properties, followed by a focus on the interpretation of the parameters in terms of inequality. The scale parameter μ is equal to the expectation, and it does not affect the inequality, while the two shape parameters α and θ are inverse and direct inequality indicators respectively. This result is obtained through stochastic orders based on inequality curves. A procedure to generate a random sample from Zenga distribution is also proposed. The second part of this article looks at the parameter estimation. Analytical solution of method of moments is obtained. This result is used as a starting point of numerical procedures to obtain maximum likelihood estimates both on ungrouped and grouped data. In the application, three empirical income distributions are considered and the aforementioned estimates are evaluated. A comparison with other well-known models is provided, by the evaluation of three goodness-of-fit indexes.     

Multivariate Latent Growth Models for Mixed Data with Covariate Effects

The paper presents an extension of a new class of multivariate latent growth models (Bianconcini and Cagnone, 2012) to allow for covariate effects on manifest, latent variables and random effects. The new class of models combines: (i) multivariate latent curves that describe the temporal behavior of the responses, and (ii) a factor model that specifies the relationship between manifest and latent variables. Based on the Generalized Linear and Latent Variable Model framework (Bartholomew and Knott, 1999), the response variables are assumed to follow different distributions of the exponential family, with item-specific linear predictors depending on both latent variables and measurement errors. A full maximum likelihood method is used to estimate all the model parameters simultaneously. Data coming from the Data WareHouse of the University of Bologna are used to illustrate the methodology.     

Nonlinear mixed‐effects models with misspecified random‐effects distribution

Nonlinear mixed‐effects models are being widely used for the analysis of longitudinal data, especially from pharmaceutical research. They use random effects which are latent and unobservable variables so the random‐effects distribution is subject to misspecification in practice. In this paper, we first study the consequences of misspecifying the random‐effects distribution in nonlinear mixed‐effects models. Our study is focused on Gauss‐Hermite quadrature, which is now the routine method for calculation of the marginal likelihood in mixed models. We then present a formal diagnostic test to check the appropriateness of the assumed random‐effects distribution in nonlinear mixed‐effects models, which is very useful for real data analysis. Our findings show that the estimates of fixed‐effects parameters in nonlinear mixed‐effects models are generally robust to deviations from normality of the random‐effects distribution, but the estimates of variance components are very sensitive to the distributional assumption of random effects. Furthermore, a misspecified random‐effects distribution will either overestimate or underestimate the predictions of random effects. We illustrate the results using a real data application from an intensive pharmacokinetic study.     

Embedding latent class regression and latent class distal outcome models into cluster-weighted latent class analysis: a detailed simulation experiment

Usually in latent class (LC) analysis, external predictors are taken to be cluster conditional probability predictors (LC models with external predictors), and/or score conditional probability predictors (LC regression models). In such cases, their distribution is not of interest. Class-specific distribution is of interest in the distal outcome model, when the distribution of the external variables is assumed to depend on LC membership. In this paper, we consider a more general formulation, that embeds both the LC regression and the distal outcome models, as is typically done in cluster-weighted modelling. This allows us to investigate (1) whether the distribution of the external variables differs across classes, (2) whether there are significant direct effects of the external variables on the indicators, by modelling jointly the relationship between the external and the latent variables. We show the advantages of the proposed modelling approach through a set of artificial examples, an extensive simulation study and an empirical application about psychological contracts among employees and employers in Belgium and the Netherlands.     

Robustness of partial least-squares method for estimating latent variable quality structures

Latent variable structural models and the partial least-squares (PLS) estimation procedure have found increased interest since being used in the context of customer satisfaction measurement. The well-known property that the estimates of the inner structure model are inconsistent implies biased estimates for finite sample sizes. A simplified version of the structural model that is used for the Swedish Customer Satisfaction Index (SCSI) system has been used to generate simulated data and to study the PLS algorithm in the presence of three inadequacies: (i) skew instead of symmetric distributions for manifest variables; (ii) multi-collinearity within blocks of manifest and between latent variables; and (iii) misspecification of the structural model (omission of regressors). The simulation results show that the PLS method is quite robust against these inadequacies. The bias that is caused by the inconsistency of PLS estimates is substantially increased only for extremely skewed distributions and for the erroneous omission of a highly relevant latent regressor variable. The estimated scores of the latent variables are always in very good agreement with the true values and seem to be unaffected by the inadequacies under investigation.     

Structural equation models for area health outcomes with model selection

Recent analyses seeking to explain variation in area health outcomes often consider the impact on them of latent measures (i.e. unobserved constructs) of population health risk. The latter are typically obtained by forms of multivariate analysis, with a small set of latent constructs derived from a collection of observed indicators, and a few recent area studies take such constructs to be spatially structured rather than independent over areas. A confirmatory approach is often applicable to the model linking indicators to constructs, based on substantive knowledge of relevant risks for particular diseases or outcomes. In this paper, population constructs relevant to a particular set of health outcomes are derived using an integrated model containing all the manifest variables, namely health outcome variables, as well as indicator variables underlying the latent constructs. A further feature of the approach is the use of variable selection techniques to select significant loadings and factors (especially in terms of effects of constructs on health outcomes), so ensuring parsimonious models are selected. A case study considers suicide mortality and self-harm contrasts in the East of England in relation to three latent constructs: deprivation, fragmentation and urbanicity.     

Reliability analysis of multicomponent stress–strength reliability from a bathtub-shaped distribution

In this paper, inference for a multicomponent stress–strength model is studied. When latent strength and stress random variables follow a bathtub-shaped distribution and the failure times are Type-II censored, the maximum likelihood estimate of the multicomponent stress–strength reliability (MSR) is established when there are common strength and stress parameters. Approximate confidence interval is also constructed by using the asymptotic distribution theory and delta method. Furthermore, another alternative generalized point and confidence interval estimators for the MSR are constructed based on pivotal quantities. Moreover, the likelihood and the pivotal quantities-based estimates for the MSR are also provided under unequal strength and stress parameter case. To compare the equivalence of the stress and strength parameters, the likelihood ratio test for hypothesis of interest is also provided. Finally, simulation studies and a real data example are given for illustration.     

Regression Analysis of Longitudinal Data with Time‐Dependent Covariates and Informative Observation Times

Abstract. Longitudinal data frequently occur in many studies, and longitudinal responses may be correlated with observation times. In this paper, we propose a new joint modelling for the analysis of longitudinal data with time‐dependent covariates and possibly informative observation times via two latent variables. For inference about regression parameters, estimating equation approaches are developed and asymptotic properties of the proposed estimators are established. In addition, a lack‐of‐fit test is presented for assessing the adequacy of the model. The proposed method performs well in finite‐sample simulation studies, and an application to a bladder tumour study is provided.     

Maximum likelihood estimation of bivariate circular hidden Markov models from incomplete data

In this paper, we propose a hidden Markov model for the analysis of the time series of bivariate circular observations, by assuming that the data are sampled from bivariate circular densities, whose parameters are driven by the evolution of a latent Markov chain. The model segments the data by accounting for redundancies due to correlations along time and across variables. A computationally feasible expectation maximization (EM) algorithm is provided for the maximum likelihood estimation of the model from incomplete data, by treating the missing values and the states of the latent chain as two different sources of incomplete information. Importance-sampling methods facilitate the computation of bootstrap standard errors of the estimates. The methodology is illustrated on a bivariate time series of wind and wave directions and compared with popular segmentation models for bivariate circular data, which ignore correlations across variables and/or along time.     

Bayesian inference for zero-and-one-inflated geometric distribution regression model using Pólya-Gamma latent variables

Abstract

In the fields of internet financial transactions and reliability engineering, there could be more zero and one observations simultaneously. In this paper, considering that it is beyond the range where the conventional model can fit, zero-and-one-inflated geometric distribution regression model is proposed. Ingeniously introducing Pólya-Gamma latent variables in the Bayesian inference, posterior sampling with high-dimensional parameters is converted to latent variables sampling and posterior sampling with lower-dimensional parameters, respectively. Circumventing the need for Metropolis-Hastings sampling, the sample with higher sampling efficiency is obtained. A simulation study is conducted to assess the performance of the proposed estimation for various sample sizes. Finally, a doctoral dissertation data set is analyzed to illustrate the practicability of the proposed method, research shows that zero-and-one-inflated geometric distribution regression model using Pólya-Gamma latent variables can achieve better fitting results.     

Distributional aspects in latent variable models

For observable indicators with ordered categories one can assume underlying latent variables following certain marginal distributions. Transforming the latent variables changes its marginal distributions but not the observable qualitative indicators. The joint distribution of the latent variables can be constructed from the marginal distributions. There is a broad class of multivariate distributions for which the observable indicators are equivalent. By choosing the multivariate normal distribution from this class we can analyse a linear relationship between the transformed latent variables. This leads to latent structural equation models. Estimation of these latter models is therefore more general than the distributional assumption might initially suggest. Robustness of the estimation procedure is also discussed for deviations from this distribution family. Using ordinal business survey data of the German Ifo-institute we test the efficiency of firms' price expectations implied by the rational expectation hypothesis.     

Generalized Linear Factor Models: A New Local EM Estimation Algorithm

In this article, a general approach to latent variable models based on an underlying generalized linear model (GLM) with factor analysis observation process is introduced. We call these models Generalized Linear Factor Models (GLFM). The observations are produced from a general model framework that involves observed and latent variables that are assumed to be distributed in the exponential family. More specifically, we concentrate on situations where the observed variables are both discretely measured (e.g., binomial, Poisson) and continuously distributed (e.g., gamma). The common latent factors are assumed to be independent with a standard multivariate normal distribution. Practical details of training such models with a new local expectation-maximization (EM) algorithm, which can be considered as a generalized EM-type algorithm, are also discussed. In conjunction with an approximated version of the Fisher score algorithm (FSA), we show how to calculate maximum likelihood estimates of the model parameters, and to yield inferences about the unobservable path of the common factors. The methodology is illustrated by an extensive Monte Carlo simulation study and the results show promising performance.