A data-driven selection of the number of clusters in the Dirichlet allocation model via Bayesian mixture modelling

In this paper, we consider a Bayesian mixture model that allows us to integrate out the weights of the mixture in order to obtain a procedure in which the number of clusters is an unknown quantity. To determine clusters and estimate parameters of interest, we develop an MCMC algorithm denominated by sequential data-driven allocation sampler. In this algorithm, a single observation has a non-null probability to create a new cluster and a set of observations may create a new cluster through the split-merge movements. The split-merge movements are developed using a sequential allocation procedure based in allocation probabilities that are calculated according to the Kullback–Leibler divergence between the posterior distribution using the observations previously allocated and the posterior distribution including a ‘new’ observation. We verified the performance of the proposed algorithm on the simulated data and then we illustrate its use on three publicly available real data sets.     

A new approach to classification

Clustering is a common and important issue, and finite mixture models based on the normal distribution are frequently used to address the problem. In this article, we consider a classification model and build a mixture model around it. A good assessment of the allocation of observations and number of clusters is easily obtained from this approach.     

Inferences on the difference between future observations for comparing two treatments

The comparison of two treatments with normally distributed data is considered. Inferences are considered based upon the difference between single potential future observations from each of the two treatments, which provides a useful and easily interpretable assessment of the difference between the two treatments. These methodologies combine information from a standard confidence interval analysis of the difference between the two treatment means, with information available from standard prediction intervals of future observations. Win-probabilities, which are the probabilities that a future observation from one treatment will be superior to a future observation from the other treatment, are a special case of these methodologies. The theoretical derivation of these methodologies is based upon inferences about the non-centrality parameter of a non-central t-distribution. Equal and unequal variance situations are addressed, and extensions to groups of future observations from the two treatments are also considered. Some examples and discussions of the methodologies are presented.     

Selecting the better binomial population under a budget constraint

The problem of optimal non-sequential allocation of observations for the selection of the better binomial population is considered in the case of fixed sampling costs and budget. With the appropriate choice of selection rule it is shown that a 70% reduction in the probability of incorrect selection is possible by using an unequal rather than equal allocation. Simple formulae are given for the appropriate selection rule and unequal allocation in large samples.     

塔里木河流域绿洲生态水权流域分配调整研究

塔里木河流域没有绿洲生态水权流域分配调整,无法保证干流生态水权主体权责与客体分配,导致干流绿洲生态用水被挤占,生态环境退化。为此,界定了绿洲生态水权流域分配调整的内涵,确定了其计量方法和模型,并在此基础上对塔河流域绿洲生态水权流域分配调整进行了计量。结果表明:塔河流域绿洲生态水权流域分配调整,能落实干流绿洲生态水权主体权责和客体分配。绿洲生态水权流域分配调整使源流绿洲生态水权的流域分配增加,干流减少;年内分配主要在3⁵月和65月和6⁹月,其中,适宜绿洲生态水权流域分配调整与绿洲维持恢复生态水权流域分配调整较高;叶尔羌河流域分配调整增加最多,干流中下游减少最多。这可作为未来塔河流域绿洲生态水权流域分配调整的参考依据。     

技术进步偏向、要素配置偏向与我国TFP的增长

本文在要素增强型CES生产函数的假定下,分析了技术进步偏向、要素投入偏向与我国全要素生产率增长之间的关系。结果表明：在1978-2012年期间,我国全要素生产率平均每年增加约2.10%,技术进步总体上呈现偏向资本的趋势,要素配置总体上也同样呈现出偏向资本的趋势,两者的方向基本一致。技术进步偏向对全要素生产率增长的影响由期初的促进转变为抑制,要素配置偏向对全要素生产率增长的影响在2004年以前为正值,2004年以后变为负值,即抑制了全要素生产率的增长,但是技术进步偏向与要素投入偏向两者对全要素生产率增长的交互效应却由抑制逐渐转变为促进,表明我国技术进步与要素配置慢慢由不匹配转为匹配,我国的技术进步正朝着适宜的方向发展。     

Win-probabilities for regression models

This paper considers inferences concerning future observations for regression models. Specifically, the differences between future observations at two designated sets of input values are considered. Win-probabilities, which are the probabilities that one of the future observations will exceed the other, constitute a special case of this analysis. These win-probabilities, together with the more general inferences on the difference between the future observations, provide a useful and easily interpretable tool with which a practitioner can assess the information provided by the regression model, and can make decisions regarding which of the two designated sets of input values would be optimal. A multiple-linear-regression model is considered in detail, although the results can be applied to any regression model with normally distributed errors. Central and non-central $t$-distributions are used for the analysis, and several examples of the methodologies are presented.     

An adaptive randomized design with application to estimation

When allocating observations to two populations for estimation or testing, the optimal proportion of the data that should be allocated to the first population, if it exists, often depends on unknown parameters. Adaptive designs have thus been proposed, in which allocation of the next observation is based on an estimate of the optimal proportion computed from the data already gathered. The authors introduce a simple randomized adaptive design and give some of its properties. Applications are given to estimating the difference of two success probabilities, and the difference of two normal means.     

Forecasting future values of changing sequences

Sequences of independent random variables are observed and on the basis of these observations future values of the process are forecast. The Bayesian predictive density of k future observations for normal, exponential, and binomial sequences which change exactly once are analyzed for several cases. It is seen that the Bayesian predictive densities are mixtures of standard probability distributions. For example, with normal sequences the Bayesian predictive density is a mixture of either normal or t-distributions, depending on whether or not the common variance is known. The mixing probabilities are the same as those occurring in the corresponding posterior distribution of the mean(s) of the sequence. The predictive mass function of the number of future successes that will occur in a changing Bernoulli sequence is computed and point and interval predictors are illustrated.     

Challenges and Opportunities for Statistics and Statistical Education: Looking Back,Looking Forward

The 175th anniversary of the ASA provides an opportunity to look back into the past and peer into the future. What led our forebears to found the association? What commonalities do we still see? What insights might we glean from their experiences and observations? I will use the anniversary as a chance to reflect on where we are now and where we are headed in terms of statistical education amidst the growth of data science. Statistics is the science of learning from data. By fostering more multivariable thinking, building data-related skills, and developing simulation-based problem solving, we can help to ensure that statisticians are fully engaged in data science and the analysis of the abundance of data now available to us.     

Bayesian estimation of the parameters of the generalized exponential distribution from doubly censored samples

This article deals with the problem of estimating parameters and reliability function of the generalized exponential distribution, based on type II doubly censored sample using Bayesian viewpoints. We also consider the problem of predicting an independent future sample from the same distribution. Importance sampling is used to estimate the parameters, as well as the reliability function, and the Gibbs and Metropolis samplers are used to predict the behavior of future observations from the same distribution.     

Non-parametric predictive inference for future order statistics

This article presents non-parametric predictive inference for future order statistics. Given the data consisting of n real-valued observations, m future observations are considered and predictive probabilities are presented for the rth-ordered future observation. In addition, joint and conditional probabilities for events involving multiple future order statistics are presented. The article further presents the use of such predictive probabilities for order statistics in statistical inference, in particular considering pairwise and multiple comparisons based on two or more independent groups of data.     

Optimal sign test for quantiles in ranked set samples

This paper considers the one-sample sign test for population quantiles in general ranked set sampling, and proposes a weighted sign test because observations with different ranks are not identically distributed. It is shown analytically that optimal weight always improves the Pitman efficiency for all distributions. For each quantile, the sampling allocation that maximizes the sign test efficacy is identified and shown to not depend on the population distribution. Moreover, distribution-free confidence intervals for quantiles based on ordered values of optimal ranked set samples are discussed.     

A general algorithm for computing simultaneous prediction intervals for the (log)-location-scale family of distributions

Making predictions of future realized values of random variables based on currently available data is a frequent task in statistical applications. In some applications, the interest is to obtain a two-sided simultaneous prediction interval (SPI) to contain at least k out of m future observations with a certain confidence level based on n previous observations from the same distribution. A closely related problem is to obtain a one-sided upper (or lower) simultaneous prediction bound (SPB) to exceed (or be exceeded) by at least k out of m future observations. In this paper, we provide a general approach for computing SPIs and SPBs based on data from a particular member of the (log)-location-scale family of distributions with complete or right censored data. The proposed simulation-based procedure can provide exact coverage probability for complete and Type II censored data. For Type I censored data, our simulation results show that our procedure provides satisfactory results in small samples. We use three applications to illustrate the proposed simultaneous prediction intervals and bounds.     

INDEX TO VOLUME 5 (1987)

Real-time estimates of output gaps and inflation gaps differ from the values that are obtained using data available long after the event. Part of the problem is that the data on which the real-time estimates are based is subsequently revised. We show that vector-autoregressive models of data vintages provide forecasts of post-revision values of future observations and of already-released observations capable of improving estimates of output and inflation gaps in real time. Our findings indicate that annual revisions to output and inflation data are in part predictable based on their past vintages. This article has online supplementary materials.     

Asymptotic design and estimation using linear splines

The estimation of a regression function g using linear splines is considered. The integrated mean square error is minimized using choice of estimator, allocation of observations and displacement of knots.     

Statistical consequences of an adaptive treatment allocation in a clinical trial

An examination of the likelihood arising from data in experiments comparing two therapies for neonatal respiratory failure illustrates a difficulty in the use of a design with adaptive treatment allocation as opposed to the use of the classical fixed-sample-size design. The requirement of obtaining objective scientific evidence as to which treatment is better for use on future patients conflicts with attempts to give the treatment thought to be the best to the patient at hand.     

Partition models

Product partition models assume that observations in different components of a random partition of the data are independent given the partition. If the probability distribution of random partitions is in a certain product form prior to making the observations, it is also in product form given the observations. The product model thus provides a convenient machinery for allowing the data to weight the partitions likely to hold; and inference about particular future observations may then be made by first conditioning on the partition, and then averaging over all partitions. This model is applied to fatalities in manned rocket launches, using data from the SOYUZ, APOLLO, SHUTTLE, and post-Challenger SHUTTLE programs in the Soviet Union and the United States. The combination of these data suggest that the chance of a fatality in the next shuttle, launch is about .03, after allowing for the possibility that the older programs are of slight relevance to the present shuttle program.     

Optimal Allocation of Observations for Inference on k Ordered Normal Population Means

The optimal allocation of observations when there is a natural ordering in the k normal population means is discussed. It is shown that the design which minimizes the total mean square error of the maximum likelihood estimators in the null case allocates half the observations to each of the two extreme populations. The design is obviously optimal for testing the homogeneity of means against the simple ordered alternative. It is, however, hardly acceptable for the estimation in the nonnull case. It is, therefore, shown that the observations could be allocated to the non-extreme populations according to weights which are proportional to the absolute values of the Abelson and Tukey scores at the same time keeping the minimum local power for testing the simple ordered alternative to be maximal. The design gives also the maximum minimum power, not local, for the alternative in the class of linear tests. It, of course, suffers from a small loss of efficiency for the estimation under the null case but is much better under the nonnull case than the extreme design which allocates half the observations to each of the two extreme populations. Some numerical comparisons of the mean square errors are given.     

Prediction bounds for the burr model

Given a type 2 censored sample from the Burr life time distribution, Bayesian prediction bounds are derived for future observations. An approximate Bayesian method has been used to simplify the computation of the prediction bounds. Numerical examples are used to illustrate the procedures.