Mixtures of Gaussian copula factor analyzers for clustering high dimensional data

Mixtures of factor analyzers is a useful model-based clustering method which can avoid the curse of dimensionality in high-dimensional clustering. However, this approach is sensitive to both diverse non-normalities of marginal variables and outliers, which are commonly observed in multivariate experiments. We propose mixtures of Gaussian copula factor analyzers (MGCFA) for clustering high-dimensional clustering. This model has two advantages; (1) it allows different marginal distributions to facilitate fitting flexibility of the mixture model, (2) it can avoid the curse of dimensionality by embedding the factor-analytic structure in the component-correlation matrices of the mixture distribution.An EM algorithm is developed for the fitting of MGCFA. The proposed method is free of the curse of dimensionality and allows any parametric marginal distribution which fits best to the data. It is applied to both synthetic data and a microarray gene expression data for clustering and shows its better performance over several existing methods.     

Campylobacter QMRA: A Bayesian Estimation of Prevalence and Concentration in Retail Foods Under Clustering and Heavy Censoring

A Bayesian statistical temporal‐prevalence‐concentration model (TPCM) was built to assess the prevalence and concentration of pathogenic campylobacter species in batches of fresh chicken and turkey meat at retail. The data set was collected from Finnish grocery stores in all the seasons of the year. Observations at low concentration levels are often censored due to the limit of determination of the microbiological methods. This model utilized the potential of Bayesian methods to borrow strength from related samples in order to perform under heavy censoring. In this extreme case the majority of the observed batch‐specific concentrations was below the limit of determination. The hierarchical structure was included in the model in order to take into account the within‐batch and between‐batch variability, which may have a significant impact on the sample outcome depending on the sampling plan. Temporal changes in the prevalence of campylobacter were modeled using a Markovian time series. The proposed model is adaptable for other pathogens if the same type of data set is available. The computation of the model was performed using OpenBUGS software.     

中国金融对经济增长贡献的效率研究——金融发展的阶段划分及评价

通过聚类分析将改革开放以来的中国金融-经济系统分为传统金融时期、金融转型初期、金融转型中期三个阶段,然后选取资本存量、劳动力、金融机构人民币资金来源、M2、贷款余额作为投入变量,GDP作为产出变量,采用非径向超效率分析模型测度中国金融效率。研究表明:中国的金融-经济系统存在结构变化,金融的绝对效率较低,但相对效率水平较高,中国已经顺利完成了金融体系的转型。     

A general class of test statistics for Van Valen's Red Queen hypothesis

Van Valen's Red Queen hypothesis states that within a homogeneous taxonomic group the age is statistically independent of the rate of extinction. The case of the Red Queen hypothesis being addressed here is when the homogeneous taxonomic group is a group of similar species. Since Van Valen's work, various statistical approaches have been used to address the relationship between taxon age and the rate of extinction. We propose a general class of test statistics that can be used to test for the effect of age on the rate of extinction. These test statistics allow for a varying background rate of extinction and attempt to remove the effects of other covariates when assessing the effect of age on extinction. No model is assumed for the covariate effects. Instead we control for covariate effects by pairing or grouping together similar species. Simulations are used to compare the power of the statistics. We apply the test statistics to data on Foram extinctions and find that age has a positive effect on the rate of extinction. A derivation of the null distribution of one of the test statistics is provided in the supplementary material.     

Comparison between method of moments and entropy regularization algorithm applied to parameter estimation for mixed-Weibull distribution

Mixed-Weibull distribution has been used to model a wide range of failure data sets, and in many practical situations the number of components in a mixture model is unknown. Thus, the parameter estimation of a mixed-Weibull distribution is considered and the important issue of how to determine the number of components is discussed. Two approaches are proposed to solve this problem. One is the method of moments and the other is a regularization type of fuzzy clustering algorithm. Finally, numerical examples and two real data sets are given to illustrate the features of the proposed approaches.     

A Method of Finding Predictor Genes for a Particular Disease Using a Clustering Algorithm

Clustering Algorithms are nowadays really important tools in microarray data analysis. The different clustering algorithm generally used in biological science does not take into consideration the underlying probability distribution of the data. In this sense, they are heuristic in nature. In this work we proposed a clustering algorithm based on EM Algorithm. It gives 28% less misclassification than the K-means algorithm (which is mostly use in Bio science). We have also shown on a real data set that this algorithm can be efficiently used for detecting the genes which are responsible for a particular disease.     

A note on approximate confidence limits for the negative binomial model

Four methods of approximating confidence limits for the single negative binomial parameter, P, are outlined and an empirical study is presented. Some remarks on prediction intervals are also included.     

基于多元词组和数据流聚类的热点话题动态发现

本文主要通过改进的TF-IDF算法和多元词组动态构建来选择特征关键词,并利用CluStream数据流聚类方法,实现文本主题的动态发现.实验表明,该方法可以较好地发现海量文本信息中不断变化的主题信息,从而达到推荐关联主题、动态监测舆情等目的.     

Studying Complexity of Model-based Clustering

Cluster analysis is a popular statistics and computer science technique commonly used in various areas of research. In this article, we investigate factors that can influence clustering performance in the model-based clustering framework. The four factors considered are the level of overlap, number of clusters, number of dimensions, and sample size. Through a comprehensive simulation study, we investigate model-based clustering in different settings. As a measure of clustering performance, we employ three popular classification indices capable of reflecting the degree of agreement in two partitioning vectors, thus making the comparison between the true and estimated classification vectors possible. In addition to studying clustering complexity, the performance of the three classification measures is evaluated.     

Measuring Social Tension from Income Class Segregation

We develop an index that effectively measures the level of social tension generated by income class segregation. We adopt the basic concepts of between-group difference (or alienation) and within-group similarity (or identification) from the income [bi]polarization literature; but we allow for asymmetric degrees of between-group antagonism in the alienation function, and construct a more effective identification function using both the relative degree of within-group clustering and the group size. To facilitate statistical inference, we derive the asymptotic distribution of the proposed measure using results from U-statistics. As the new measure is general enough to include existing income polarization indices as well as the Gini index as special cases, the asymptotic result can be readily applied to these popular indices. Evidence from the Panel Study of Income Dynamics data suggests that, while the level of social tension shows an upward trend over the sample period of 1981 to 2005, government’s taxes and transfers have been effective in reducing the level of social tension significantly.