Latent Variable Modelling: A Survey*

Abstract. Latent variable modelling has gradually become an integral part of mainstream statistics and is currently used for a multitude of applications in different subject areas. Examples of ‘traditional’ latent variable models include latent class models, item–response models, common factor models, structural equation models, mixed or random effects models and covariate measurement error models. Although latent variables have widely different interpretations in different settings, the models have a very similar mathematical structure. This has been the impetus for the formulation of general modelling frameworks which accommodate a wide range of models. Recent developments include multilevel structural equation models with both continuous and discrete latent variables, multiprocess models and nonlinear latent variable models.     

Semiparametric Regression Models with Applications to Scoring: A Review

Semiparametric regression models have been proposed in the econometric literature as a trade-off between the simple but easily implementable and interpretable parametric models and the flexible but structure free smoothing techniques. Some semiparametric models for binary response with possible application to scoring data are reviewed: single-index models, generalized partially linear models, generalized partially linear single-index models, and multiple-index models. All these models are extensions of the classical logistic regression.     

Hierarchical likelihood approach to non-Gaussian factor analysis

Factor models, structural equation models (SEMs) and random-effect models share the common feature that they assume latent or unobserved random variables. Factor models and SEMs allow well developed procedures for a rich class of covariance models with many parameters, while random-effect models allow well developed procedures for non-normal models including heavy-tailed distributions for responses and random effects. In this paper, we show how these two developments can be combined to result in an extremely rich class of models, which can be beneficial to both areas. A new fitting procedures for binary factor models and a robust estimation approach for continuous factor models are proposed.     

Some algebra and geometry for hierarchical models, applied to diagnostics

Recent advances in computing make it practical to use complex hierarchical models. However, the complexity makes it difficult to see how features of the data determine the fitted model. This paper describes an approach to diagnostics for hierarchical models, specifically linear hierarchical models with additive normal or t -errors. The key is to express hierarchical models in the form of ordinary linear models by adding artificial `cases' to the data set corresponding to the higher levels of the hierarchy. The error term of this linear model is not homoscedastic, but its covariance structure is much simpler than that usually used in variance component or random effects models. The re-expression has several advantages. First, it is extremely general, covering dynamic linear models, random effect and mixed effect models, and pairwise difference models, among others. Second, it makes more explicit the geometry of hierarchical models, by analogy with the geometry of linear models. Third, the analogy with linear models provides a rich source of ideas for diagnostics for all the parts of hierarchical models. This paper gives diagnostics to examine candidate added variables, transformations, collinearity, case influence and residuals.     

Generalized Least Squares Estimation in Contingency Tables Analysis: Asymptotic Properties and Applications

Different sorts of bilinear models (models with bilinear interaction terms) are currently used when analyzing contingency tables: association models, correlation models... All these can be included in a general family of bilinear models: power models. In this framework, Maximum Likelihood (ML) estimation is not always possible, as explained in an introductory example. Thus, Generalized Least Squares (GLS) estimation is sometimes needed in order to estimate parameters. A subclass of power models is then considered in this paper: separable reduced-rank (SRR) models. They allow an optimal choice of weights for GLS estimation and simplifications in asymptotic studies concerning GLS estimators. Power 2 models belong to the subclass of SRR models and the asymptotic properties of GLS estimators are established. Similar results are also established for association models which are not SRR models. However, these results are more difficult to prove. Finally, 2 examples are considered to illustrate our results.     

Parsimonious Gaussian mixture models

Parsimonious Gaussian mixture models are developed using a latent Gaussian model which is closely related to the factor analysis model. These models provide a unified modeling framework which includes the mixtures of probabilistic principal component analyzers and mixtures of factor of analyzers models as special cases. In particular, a class of eight parsimonious Gaussian mixture models which are based on the mixtures of factor analyzers model are introduced and the maximum likelihood estimates for the parameters in these models are found using an AECM algorithm. The class of models includes parsimonious models that have not previously been developed. These models are applied to the analysis of chemical and physical properties of Italian wines and the chemical properties of coffee; the models are shown to give excellent clustering performance.     

Using compositional and Dirichlet models for market share regression

When the aim is to model market shares, the marketing literature proposes some regression models which can be qualified as attraction models. They are generally derived from an aggregated version of the multinomial logit model. But aggregated multinomial logit models (MNL) and the so-called generalized multiplicative competitive interaction models (GMCI) present some limitations: in their simpler version they do not specify brand-specific and cross effect parameters. In this paper, we consider alternative models: the Dirichlet model (DIR) and the compositional model (CODA). DIR allows to introduce brand-specific parameters and CODA allows additionally to consider cross effect parameters. We show that these two models can be written in a similar fashion, called attraction form, as the MNL and the GMCI models. As market share models are usually interpreted in terms of elasticities, we also use this notion to interpret the DIR and CODA models. We compare the properties of the models in order to explain why CODA and DIR models can outperform traditional market share models. An application to the automobile market is presented where we model brands market shares as a function of media investments, controlling for the brands price and scrapping incentive. We compare the quality of the models using measures adapted to shares.     

二元选择面板模型的设定检验

 本文将RESET检验扩展到二元选择面板数据模型的设定,考察了固定效应Probit模型和Logit模型的设定检验,包括异方差、遗漏变量和分布误设的检验。模拟结果表明Logit模型的RESET设定检验显示良好的水平和功效,而Probit模型的RESET检验可能由于估计方法的选择导致在某些方面的功效表现不好。但总体说来,在二元选择面板数据模型的设定检验上,RESET检验仍然是一个较好的选择。     

Generalized choice models for categorical time series

The NDARMA models of Jacobs and Lewis (1983) allow the modeling of categorical processes with an ARMA-like serial dependence structure. These models can be represented through a backshift mechanism, and we analyze marginal and bivariate properties of the resulting backshift process. Motivated by this backshift mechanism, we define the new class of generalized choice (GC) models, which include the usual NDARMA models as a special case, and we derive results describing the marginal and bivariate distribution of the GC model. We discuss implications concerning DMA(∞) models and the serial dependence structure of NDARMA models. Examples show that the family of GC models allows creating sparsely parametrized models for categorical processes with different types of serial dependence structure.     

The Equivalence of Parameter Estimates from Growth Curve Models and Seemingly Unrelated Regression Models

Seemingly unrelated regression models and growth curve models are examples of multivariate models that require special estimation techniques. Parameters in seemingly unrelated regression models can be estimated by using two-stage Aitken estimation based on unrestricted residuals; parameters in growth curve models can be estimated by using a Potthoff-Roy (1964) transformation based on an estimate of the dispersion. With proper choice of the seemingly unrelated regression model, the two multivariate models and corresponding parameter estimates are shown to be equivalent. Recognition of the equivalence simplifies the presentation of these more complicated multivariate models. The connection is also of interest for more flexible growth curve models.     

具有不同转换机制的非线性模型理论与应用研究

近几年来,非线性时间序列模型被广泛地应用于经济学领域,尤其是其中的STAR模型能很好的描述很多经济变量的运行轨迹。本文正是对于STAR模型中转换函数的不同函数形式形成的模型进行讨论,除了常见的LSTAR和ESTAR模型外,本文还提出了令转换函数为三角函数的的TSTAR模型,并且对于这三种STAR模型的检验和模型的选择作了一些探讨。     

未决赔款准备金评估模型的比较研究

近10多年来,关于未决赔款准备金评估模型的研究取得了较大进展,其中虽然也包含对各种评估模型相互关系的探讨,如关于各种随机模型的比较、以及基于B-F法对各种准备金评估模型的比较等,但仍然不够全面和系统。在对准备金评估模型从不同角度进行了较为系统的分类和综述的同时,首次以最基本的链梯模型为基础,建立了一个统一的框架,并对常见的一些准备金评估模型进行了综合比较和分析,揭示了它们之间的一些重要关系,给出了在实务中选择准备金评估模型的一些建议。     

Transformed generalized linear models

The estimation of data transformation is very useful to yield response variables satisfying closely a normal linear model. Generalized linear models enable the fitting of models to a wide range of data types. These models are based on exponential dispersion models. We propose a new class of transformed generalized linear models to extend the Box and Cox models and the generalized linear models. We use the generalized linear model framework to fit these models and discuss maximum likelihood estimation and inference. We give a simple formula to estimate the parameter that index the transformation of the response variable for a subclass of models. We also give a simple formula to estimate the r$r$th moment of the original dependent variable. We explore the possibility of using these models to time series data to extend the generalized autoregressive moving average models discussed by Benjamin et al. [Generalized autoregressive moving average models. J. Amer. Statist. Assoc. 98, 214–223]. The usefulness of these models is illustrated in a simulation study and in applications to three real data sets.     

贝叶斯非线性混合效应模型及其应用研究

由于常用的线性混合效应模型对具有非线性关系的纵向数据建模具有一定的局限性,因此对线性混合效应模型进行扩展,根据变量间的非线性关系建立不同的非线性混合效应模型,并根据因变量的分布特征建立混合分布模型。基于一组实际的保险损失数据,建立多项式混合效应模型、截断多项式混合效应模型和B样条混合效应模型。研究结果表明,非线性混合效应模型能够显著改进对保险损失数据的建模效果,对非寿险费率厘定具有重要参考价值。     

广义Gamma分布簇广义线性混合模型的构建

 本文在广义Gamma分布簇基础上引入异质性来构建广义线性混合模型。本文构建的广义Gamma分布簇广义线性混合模型在广义线性混合模型的框架下分析,通过参数重整技术把广义Gamma分布簇变量的建模问题与指数分布簇变量的建模问题联系起来,模型推断可以方便地利用广义线性混合模型和广义线性模型的研究成果,同时也可以方便地推广到其他模型。三参数广义Gamma分布可以收缩到两参数的Gamma分布、Weibull分布或指数分布,能降低模型误设的风险,还能便利地分析误差结构。     

Discussion of “Nonparametric Bayesian Inference in Applications”: Bayesian nonparametric methods in econometrics

The use of Bayesian nonparametrics models has increased rapidly over the last few decades driven by increasing computational power and the development of efficient Markov chain Monte Carlo algorithms. We review some applications of these models in economic applications including: volatility modelling (using both stochastic volatility models and GARCH-type models) with Dirichlet process mixture models, uses in portfolio allocation problems, long memory models with flexible forms of time-dependence, flexible extension of the dynamic Nelson-Siegel model for interest rate yields and multivariate time series models used in macroeconometrics.     

Random partition models with regression on covariates

Many recent applications of nonparametric Bayesian inference use random partition models, i.e. probability models for clustering a set of experimental units. We review the popular basic constructions. We then focus on an interesting extension of such models. In many applications covariates are available that could be used to a priori inform the clustering. This leads to random clustering models indexed by covariates, i.e., regression models with the outcome being a partition of the experimental units. We discuss some alternative approaches that have been used in the recent literature to implement such models, with an emphasis on a recently proposed extension of product partition models. Several of the reviewed approaches were not originally intended as covariate-based random partition models, but can be used for such inference.     

Overview of multinomial models for ordinal data

The purpose of this paper is to relate a number of multinomial models currently in use for ordinal response data in a unified manner. By studying generalized logit models, proportional generalized odds ratio models and proportional generalized hazard models under different parameterizations, we conclude that there are only four different models and they can be specified genericaUy in a uniform way. These four models all possess the same stochastic ordering property and we compare them graphically in a simple case. Data from the NHLBI TYPE II study (Brensike et al (1984)) is used to illustrate these models. We show that the BMDP programs LE and PR can be employed in computing maximum likelihood estimators for these four models.     

Logistic与分类树模型变量筛选的比较——基于信用卡邮寄业务响应率分析

基于信用卡邮寄业务响应率分析来讨论Logistic模型和分类树模型在变量选取上的区别,并尝试从几个不同角度去解释两类模型变量筛选差异的原因。笔者认为没有绝对占优势的方法,需要结合具体场景和模型的特点来选择合适的模型。分类树模型在训练集上容易过度拟合,对单个变量的影响很敏感,在进行危险因素分析时结果更能强调危险因素,对孤立点的识别率很高。Logistic模型容易受到解释变量依存关系的影响,加上分类变量的影响容易过多地选入变量或者因子,对孤立点敏感,对噪点不敏感。判别函数的差异是变量筛选差异的关键因素。     

Optimal designs for experiments with mixtures: a survey

This is a survey article on known results about analytic solutions and numerical solutions of optimal designs for various regression models for experiments with mixtures. The regression models include polynomial models, models containing homogeneous functions, models containing inverse terms and ratios, log contrast models, models with quantitative variables, and mod els containing the amount of mixture, Optimality criteria considered include D-, A-, E-,φ_p- and I_λ-Optimalities. Uniform design and uniform optimal design for mixture components, and efficiencies of the {q,2} simplex-controid design are briefly discussed.