Latent Variable Modelling: A Survey*

Abstract. Latent variable modelling has gradually become an integral part of mainstream statistics and is currently used for a multitude of applications in different subject areas. Examples of ‘traditional’ latent variable models include latent class models, item–response models, common factor models, structural equation models, mixed or random effects models and covariate measurement error models. Although latent variables have widely different interpretations in different settings, the models have a very similar mathematical structure. This has been the impetus for the formulation of general modelling frameworks which accommodate a wide range of models. Recent developments include multilevel structural equation models with both continuous and discrete latent variables, multiprocess models and nonlinear latent variable models.     

Parameter estimation approaches to tackling measurement error and multicollinearity in ordinal probit models

Abstract

The regression model with ordinal outcome has been widely used in a lot of fields because of its significant effect. Moreover, predictors measured with error and multicollinearity are long-standing problems and often occur in regression analysis. However there are not many studies on dealing with measurement error models with generally ordinal response, even fewer when they suffer from multicollinearity. The purpose of this article is to estimate parameters of ordinal probit models with measurement error and multicollinearity. First, we propose to use regression calibration and refined regression calibration to estimate parameters in ordinal probit models with measurement error. Second, we develop new methods to obtain estimators of parameters in the presence of multicollinearity and measurement error in ordinal probit model. Furthermore we also extend all the methods to quadratic ordinal probit models and talk about the situation in ordinal logistic models. These estimators are consistent and asymptotically normally distributed under general conditions. They are easy to compute, perform well and are robust against the normality assumption for the predictor variables in our simulation studies. The proposed methods are applied to some real datasets.     

Bayesian structural equation modeling for the health index

There are many factors which could influence the level of health of an individual. These factors are interactive and their overall effects on health are usually measured by an index which is called as health index. The health index could also be used as an indicator to describe the health level of a community. Since the health index is important, many research have been done to study its determinant. The main purpose of this study is to model the health index of an individual based on classical structural equation modeling (SEM) and Bayesian SEM. For estimation of the parameters in the measurement and structural equation models, the classical SEM applies the robust-weighted least-square approach, while the Bayesian SEM implements the Gibbs sampler algorithm. The Bayesian SEM approach allows the user to use the prior information for updating the current information on the parameter. Both methods are applied to the data gathered from a survey conducted in Hulu Langat, a district in Malaysia. Based on the classical and the Bayesian SEM, it is found that demographic status and lifestyle are significantly related to the health index. However, mental health has no significant relation to the health index.     

Stochastic EM algorithm for generalized exponential cure rate model and an empirical study

In this paper, we consider two well-known parametric long-term survival models, namely, the Bernoulli cure rate model and the promotion time (or Poisson) cure rate model. Assuming the long-term survival probability to depend on a set of risk factors, the main contribution is in the development of the stochastic expectation maximization (SEM) algorithm to determine the maximum likelihood estimates of the model parameters. We carry out a detailed simulation study to demonstrate the performance of the proposed SEM algorithm. For this purpose, we assume the lifetimes due to each competing cause to follow a two-parameter generalized exponential distribution. We also compare the results obtained from the SEM algorithm with those obtained from the well-known expectation maximization (EM) algorithm. Furthermore, we investigate a simplified estimation procedure for both SEM and EM algorithms that allow the objective function to be maximized to split into simpler functions with lower dimensions with respect to model parameters. Moreover, we present examples where the EM algorithm fails to converge but the SEM algorithm still works. For illustrative purposes, we analyze a breast cancer survival data. Finally, we use a graphical method to assess the goodness-of-fit of the model with generalized exponential lifetimes.     

Correcting data for measurement error in generalized linear models

This paper discusses a general strategy for reducing measurement-error-induced bias in statistical models. It is assumed that the measurement error is unbiased with a known variance although no other distributional assumptions on the measurement-error are employed,

Using a preliminary fit of the model to the observed data, a transformation of the variable measured with error is estimated. The transformation is constructed so that the estimates obtained by refitting the model to the ‘corrected’ data have smaller bias,

Whereas the general strategy can be applied in a number of settings, this paper focuses on the problem of covariate measurement error in generalized linear models, Two estimators are derived and their effectiveness at reducing bias is demonstrated in a Monte Carlo study.     

A Bayesian approach of joint models for clustered zero-inflated count data with skewness and measurement errors

Count data with excess zeros are widely encountered in the fields of biomedical, medical, public health and social survey, etc. Zero-inflated Poisson (ZIP) regression models with mixed effects are useful tools for analyzing such data, in which covariates are usually incorporated in the model to explain inter-subject variation and normal distribution is assumed for both random effects and random errors. However, in many practical applications, such assumptions may be violated as the data often exhibit skewness and some covariates may be measured with measurement errors. In this paper, we deal with these issues simultaneously by developing a Bayesian joint hierarchical modeling approach. Specifically, by treating intercepts and slopes in logistic and Poisson regression as random, a flexible two-level ZIP regression model is proposed, where a covariate process with measurement errors is established and a skew-t-distribution is considered for both random errors and random effects. Under the Bayesian framework, model selection is carried out using deviance information criterion (DIC) and a goodness-of-fit statistics is also developed for assessing the plausibility of the posited model. The main advantage of our method is that it allows for more robustness and correctness for investigating heterogeneity from different levels, while accommodating the skewness and measurement errors simultaneously. An application to Shanghai Youth Fitness Survey is used as an illustrate example. Through this real example, it is showed that our approach is of interest and usefulness for applications.     

变权重组合预测模型的局部加权最小二乘解法

随着科学技术的不断进步，预测方法也得到了很大的发展，常见的预测方法就有数十种之多。而组合预测是将不同的预测方法组合起来，综合利用各个方法所提供的信息，其效果往往优于单一的预测方法，故得到了广泛的应用。而基于变系数模型的思想研究了组合预测模型，将变权重的求取转化为变系数模型中系数函数的估计问题，从而可以基于局部加权最小二乘方法求解，利用交叉证实法选取光滑参数。其结果表明所提方法预测精度很高，效果优于其他方法。     

Statistical analysis of controlled calibration model with replicates

In this work, we generalize the controlled calibration model by assuming replication on both variables. Likelihood-based methodology is used to estimate the model parameters and the Fisher information matrix is used to construct confidence intervals for the unknown value of the regressor variable. Further, we study the local influence diagnostic method which is based on the conditional expectation of the complete-data log-likelihood function related to the EM algorithm. Some useful perturbation schemes are discussed. A simulation study is carried out to assess the effect of the measurement error on the estimation of the parameter of interest. This new approach is illustrated with a real data set.     

Statistical Modeling for the Evaluation of Consumers’ Preferences

In 2009 a survey was performed in Veneto, a region in the north-east of Italy, to study the demand for wine and specifically for Passito, a typical Italian wine. The main goal of the study consisted in analyzing how the preferences and consumption habits of Passito vary depending on consumers’ characteristics. Specifically two kinds of statistical methods were applied: Covariate Uniform Binomial (CUB) model, a statistical approach for ordinal data to study the feeling toward Passito and the uncertainty of the respondents; classical logistic regression analysis, to describe how the attitude toward passito can be modeled as function of consumers’ covariates. Gender and residence were the most important covariates, useful in defining segments of consumers with significant differences in terms of Passito's preferences and consumption behavior. The logistic regression analysis allowed to complete the statistical analysis based on CUB models validating the results of the CUB model and estimating a model useful to predict the attitude toward the considered product for specific sub-groups of consumers.     

中国城市居民婴幼儿奶粉品牌选购行为研究——以北京市为例

以北京市城市消费者为调查对象,运用统计描述法和Logit回归模型,分析消费者对不同品牌婴幼儿奶粉的选购行为及其影响因素。研究结果表明:与国内婴幼儿奶粉品牌相比,国外品牌占据明显优势,82.7%的消费者购买国外品牌;从消费者自身的认知出发,质量安全是其购买国外品牌的主要原因。Logit模型回归结果表明,风险感知、收入、家中小孩数和性别是影响消费者品牌选购的主要影响因素。     

The effects of model mis-specifications in linear measurement error models

In the literature, there are many results on the consequences of mis-specified models for linear models with error in the response only, see, e.g., Seber(1977). There are also discussions of estimation for the model writh errors both in the response and in the predictor variables (called measurement error models; see, e.g., Fuller(1987)). In this paper, we consider the problem of model mis-specification for measurement error models. Only a few special cases have been tackled in the past (Edland, 1996; Carroll and Ruppert, 1996 and Lakshminarayanan Amp; Gunst, 1984); we deal with the situation here in some generality. Results have been obtained as follows: (a) When a model is under-fitted, the estimate of the variance of the measurement error will be asymptotically biased, as will the regression coefficients, and the asymptotic biases in the estimates of the regression coefficients will always exist for under-fitted models. Even orthogonality of the variables in the model will not make the biases vanish. (b)For over-fitting, the estimates of the variances of measurement errors and of the regression coefficients are asymptotically unbiased. However, the variance of the estimated regression coefficients will increase. Over-fitting will cause larger changes in the variances of the estimated parameters in measurement error models than in no measurement error models.     

A methodology for quantifying the effect of missing data on decision quality in classification problems

Decision making is often supported by decision models. This study suggests that the negative impact of poor data quality (DQ) on decision making is often mediated by biased model estimation. To highlight this perspective, we develop an analytical framework that links three quality levels – data, model, and decision. The general framework is first developed at a high-level, and then extended further toward understanding the effect of incomplete datasets on Linear Discriminant Analysis (LDA) classifiers. The interplay between the three quality levels is evaluated analytically – initially for a one-dimensional case, and then for multiple dimensions. The impact is then further analyzed through several simulative experiments with artificial and real-world datasets. The experiment results support the analytical development and reveal nearly-exponential decline in the decision error as the completeness level increases. To conclude, we discuss the framework and the empirical findings, elaborate on the implications of our model on the data quality management, and the use of data for decision-models estimation.     

The Cusum Test for Parameter Change in Time Series Models

Abstract.  In this paper, we consider the problem of testing for parameter changes in time series models based on a cusum test. Although the test procedure is well established for the mean and variance in time series models, a general parameter case has not been discussed in the literature. Therefore, here we develop a cusum test for parameter change in a more general framework. As an example, we consider the change of the parameters in a random coeefficient autoregressive (1) model and that of the autocovariances of a linear process. Simulation results are reported for illustration.     

Structural identification and variable selection in high-dimensional varying-coefficient models

Varying-coefficient models have been widely used to investigate the possible time-dependent effects of covariates when the response variable comes from normal distribution. Much progress has been made for inference and variable selection in the framework of such models. However, the identification of model structure, that is how to identify which covariates have time-varying effects and which have fixed effects, remains a challenging and unsolved problem especially when the dimension of covariates is much larger than the sample size. In this article, we consider the structural identification and variable selection problems in varying-coefficient models for high-dimensional data. Using a modified basis expansion approach and group variable selection methods, we propose a unified procedure to simultaneously identify the model structure, select important variables and estimate the coefficient curves. The unique feature of the proposed approach is that we do not have to specify the model structure in advance, therefore, it is more realistic and appropriate for real data analysis. Asymptotic properties of the proposed estimators have been derived under regular conditions. Furthermore, we evaluate the finite sample performance of the proposed methods with Monte Carlo simulation studies and a real data analysis.     

Enriching Surveys with Supplementary Data and its Application to Studying Wage Regression

We consider the problem of supplementing survey data with additional information from a population. The framework we use is very general; examples are missing data problems, measurement error models and combining data from multiple surveys. We do not require the survey data to be a simple random sample of the population of interest. The key assumption we make is that there exists a set of common variables between the survey and the supplementary data. Thus, the supplementary data serve the dual role of providing adjustments to the survey data for model consistencies and also enriching the survey data for improved efficiency. We propose a semi‐parametric approach using empirical likelihood to combine data from the two sources. The method possesses favourable large and moderate sample properties. We use the method to investigate wage regression using data from the National Longitudinal Survey of Youth Study.     

Optimality of quasi-score in the multivariate mean–variance model with an application to the zero-inflated Poisson model with measurement errors

In a multivariate mean–variance model, the class of linear score (LS) estimators based on an unbiased linear estimating function is introduced. A special member of this class is the (extended) quasi-score (QS) estimator. It is ‘extended’ in the sense that it comprises the parameters describing the distribution of the regressor variables. It is shown that QS is (asymptotically) most efficient within the class of LS estimators. An application is the multivariate measurement error model, where the parameters describing the regressor distribution are nuisance parameters. A special case is the zero-inflated Poisson model with measurement errors, which can be treated within this framework.     

The penalized biclustering model and related algorithms

Biclustering is the simultaneous clustering of two related dimensions, for example, of individuals and features, or genes and experimental conditions. Very few statistical models for biclustering have been proposed in the literature. Instead, most of the research has focused on algorithms to find biclusters. The models underlying them have not received much attention. Hence, very little is known about the adequacy and limitations of the models and the efficiency of the algorithms. In this work, we shed light on associated statistical models behind the algorithms. This allows us to generalize most of the known popular biclustering techniques, and to justify, and many times improve on, the algorithms used to find the biclusters. It turns out that most of the known techniques have a hidden Bayesian flavor. Therefore, we adopt a Bayesian framework to model biclustering. We propose a measure of biclustering complexity (number of biclusters and overlapping) through a penalized plaid model, and present a suitable version of the deviance information criterion to choose the number of biclusters, a problem that has not been adequately addressed yet. Our ideas are motivated by the analysis of gene expression data.     

A Bayesian analysis on time series structural equation models

Structural equation models (SEM) have been extensively used in behavioral, social, and psychological research to model relations between the latent variables and the observations. Most software packages for the fitting of SEM rely on frequentist methods. Traditional models and software are not appropriate for analysis of the dependent observations such as time-series data. In this study, a structural equation model with a time series feature is introduced. A Bayesian approach is used to solve the model with the aid of the Markov chain Monte Carlo method. Bayesian inferences as well as prediction with the proposed time series structural equation model can also reveal certain unobserved relationships among the observations. The approach is successfully employed using real Asian, American and European stock return data.     

SMALL AREA ESTIMATION USING SURVEY WEIGHTS WITH FUNCTIONAL MEASUREMENT ERROR IN THE COVARIATE

Nested error linear regression models using survey weights have been studied in small area estimation to obtain efficient model‐based and design‐consistent estimators of small area means. The covariates in these nested error linear regression models are not subject to measurement errors. In practical applications, however, there are many situations in which the covariates are subject to measurement errors. In this paper, we develop a nested error linear regression model with an area‐level covariate subject to functional measurement error. In particular, we propose a pseudo‐empirical Bayes (PEB) predictor to estimate small area means. This predictor borrows strength across areas through the model and makes use of the survey weights to preserve the design consistency as the area sample size increases. We also employ a jackknife method to estimate the mean squared prediction error (MSPE) of the PEB predictor. Finally, we report the results of a simulation study on the performance of our PEB predictor and associated jackknife MSPE estimator.     

Inference of Seasonal Long‐memory Time Series with Measurement Error

We consider the Whittle likelihood estimation of seasonal autoregressive fractionally integrated moving‐average models in the presence of an additional measurement error and show that the spectral maximum Whittle likelihood estimator is asymptotically normal. We illustrate by simulation that ignoring measurement errors may result in incorrect inference. Hence, it is pertinent to test for the presence of measurement errors, which we do by developing a likelihood ratio (LR) test within the framework of Whittle likelihood. We derive the non‐standard asymptotic null distribution of this LR test and the limiting distribution of LR test under a sequence of local alternatives. Because in practice, we do not know the order of the seasonal autoregressive fractionally integrated moving‐average model, we consider three modifications of the LR test that takes model uncertainty into account. We study the finite sample properties of the size and the power of the LR test and its modifications. The efficacy of the proposed approach is illustrated by a real‐life example.