Semiparametric estimation for count data through weighted distributions

This paper is concerned with semiparametric discrete kernel estimators when the unknown count distribution can be considered to have a general weighted Poisson form. The estimator is constructed by multiplying the Poisson estimate with a nonparametric discrete kernel-type estimate of the Poisson weight function. Comparisons are then carried out with the ordinary discrete kernel probability mass function estimators. The Poisson weight function is thus a local multiplicative correction factor, and is considered as the uniform measure to detect departures from the equidispersed Poisson distribution. In this way, the effects of dispersion and zero-proportion with respect to the standard Poisson distribution are also minimized. This method of estimation is also applied to the weighted binomial form for the count distribution having a finite support. The proposed estimators, in addition to being simple, easy-to-implement and effective, also outperform the competing nonparametric and parametric estimators in finite-sample situations. Two examples illustrate this new semiparametric estimation.     

Analysis of all-zero binomial outcomes with borderline and equilibrium priors

This article is concerned with the analysis of a random sample from a binomial distribution when all the outcomes are zero (or unity). We discuss how elicitation of the prior can be reduced to asking the expert whether (and which of) the so-called borderline or equilibrium priors are plausible.     

A clustering approach to interpretable principal components

A new method for constructing interpretable principal components is proposed. The method first clusters the variables, and then interpretable (sparse) components are constructed from the correlation matrices of the clustered variables. For the first step of the method, a new weighted-variances method for clustering variables is proposed. It reflects the nature of the problem that the interpretable components should maximize the explained variance and thus provide sparse dimension reduction. An important feature of the new clustering procedure is that the optimal number of clusters (and components) can be determined in a non-subjective manner. The new method is illustrated using well-known simulated and real data sets. It clearly outperforms many existing methods for sparse principal component analysis in terms of both explained variance and sparseness.     

Exploratory factor and principal component analyses: some new aspects

Exploratory Factor Analysis (EFA) and Principal Component Analysis (PCA) are popular techniques for simplifying the presentation of, and investigating the structure of, an (n×p) data matrix. However, these fundamentally different techniques are frequently confused, and the differences between them are obscured, because they give similar results in some practical cases. We therefore investigate conditions under which they are expected to be close to each other, by considering EFA as a matrix decomposition so that it can be directly compared with the data matrix decomposition underlying PCA. Correspondingly, we propose an extended version of PCA, called the EFA-like PCA, which mimics the EFA matrix decomposition in the sense that they contain the same unknowns. We provide iterative algorithms for estimating the EFA-like PCA parameters, and derive conditions that have to be satisfied for the two techniques to give similar results. Throughout, we consider separately the cases n>p and p≥n. All derived algorithms and matrix conditions are illustrated on two data sets, one for each of these two cases.     

Resolving the Mixed Models Controversy

Two standard mixed models with interactions are discussed. When each is viewed in the context of superpopulation models, the mixed models controversy is resolved. The tests suggested by the expected mean squares under the constrained-parameters model are correct for testing the main effects and interactions under both the unconstrained-and constrained-parameters models.     

k-POD: A Method for k-Means Clustering of Missing Data

The k-means algorithm is often used in clustering applications but its usage requires a complete data matrix. Missing data, however, are common in many applications. Mainstream approaches to clustering missing data reduce the missing data problem to a complete data formulation through either deletion or imputation but these solutions may incur significant costs. Our k-POD method presents a simple extension of k-means clustering for missing data that works even when the missingness mechanism is unknown, when external information is unavailable, and when there is significant missingness in the data.

[Received November 2014. Revised August 2015.]     

Automatic locally adaptive smoothing for tree-based set estimation

Tree-based methods similar to CART have recently been utilized for problems in which the main goal is to estimate some set of interest. It is often the case that the boundary of the true set is smooth in some sense, however tree-based estimates will not be smooth, as they will be a union of ‘boxes’. We propose a general methodology for smoothing such sets that allows for varying levels of smoothness on the boundary automatically. The method is similar to the idea underlying support vector machines, which is applying a computationally simple technique to data after a non-linear mapping to produce smooth estimates in the original space. In particular, we consider the problem of level-set estimation for regression functions and the dyadic tree-based method of Willett and Nowak [Minimax optimal level-set estimation, IEEE Trans. Image Process. 16 (2007), pp. 2965–2979].     

Maximum likelihood estimation of the bimodal failure rate for censored and tied observations

DIMITROV, RACHEV and YAKOVLEV ( 1985 ) have obtained the isotonic maximum likelihood estimator for the bimodal failure rate function. The authors considered only the complete failure time data. The generalization of this estimator for the case of censored and tied observations is now proposed.