A Characterization of the Normal Family of Distributions Based on the Bias of the Maximum Likelihood Estimator

This paper characterizes the family of Normal distributions within the class of exponential families of distributions, via the structure of the bias of the maximum likelihood estimator Θ _n of the canonical parameter Θ . More specifically, when E _θ ( Θ n ) – Θ = (1/ n ) Q ( Θ ) + o (1/ n ), the equality Q ( Θ ) = 0 proves to be a property of the Normal distribution only. The same conclusion is obtained for the one-dimensional case bt assuming that Q ( Θ ) is a polynomial of Θ .     

Collinearity, Bias, and Effect Size: Modeling “the” Effect of Track on Achievement

The estimated effect of any factor can be highly dependent on both the model and the data used for the analyses. This article presents an example of the estimated effect of one factor in two different data sets under three different forms of the standard linear model using the effect of track placement on achievement as an example. Some relative advantages and disadvantages of each model are considered. The analyses demonstrate that, given collinearity among the predictor variables, a model with a poorer statistical fit may be useful for some interpretive purposes.     

A pragmatic approach to experimental design in industry

The importance of statistically designed experiments in industry has been well recognized. However, the use of 'design of experiments' is still not pervasive, owing in part to the inefficient learning process experienced by many non-statisticians. In this paper, the nature of design of experiments, in contrast to the usual statistical process control techniques, is discussed. It is then pointed out that for design of experiments to be appreciated and applied, appropriate approaches should be taken in training, learning and application. Perspectives based on the concepts of objective setting and design under constraints can be used to facilitate the experimenters' formulation of plans for collection, analysis and interpretation of empirical information. A review is made of the expanding role of design of experiments in the past several decades, with comparisons made of the various formats and contexts of experimental design applications, such as Taguchi methods and Six Sigma. The trend of development shows that, from the realm of scientific research to business improvement, the competitive advantage offered by design of experiments is being increasingly felt.     

Boosting with Bayesian stumps

Boosting is a new, powerful method for classification. It is an iterative procedure which successively classifies a weighted version of the sample, and then reweights this sample dependent on how successful the classification was. In this paper we review some of the commonly used methods for performing boosting and show how they can be fit into a Bayesian setup at each iteration of the algorithm. We demonstrate how this formulation gives rise to a new splitting criterion when using a domain-partitioning classification method such as a decision tree. Further we can improve the predictive performance of simple decision trees, known as stumps, by using a posterior weighted average of them to classify at each step of the algorithm, rather than just a single stump. The main advantage of this approach is to reduce the number of boosting iterations required to produce a good classifier with only a minimal increase in the computational complexity of the algorithm.     

Heteroscedastic Regression Models and Applications to Off-line Quality Control

We discuss in the present paper the analysis of heteroscedastic regression models and their applications to off-line quality control problems. It is well known that the method of pseudo-likelihood is usually preferred to full maximum likelihood since estimators of the parameters in the regression function obtained are more robust to misspecification of the variance function. Despite its popularity, however, existing theoretical results are difficult to apply and are of limited use in many applications. Using more recent results in estimating equations, we obtain an efficient algorithm for computing the pseudo-likelihood estimator with desirable convergence properties and also derive simple, explicit and easy to apply asymptotic results. These results are used to look in detail at variance minimization in off-line quality control, yielding techniques of inferences for the optimized design parameter. In application of some existing approaches to off-line quality control, such as the dual response methodology, rigorous statistical inference techniques are scarce and difficult to obtain. An example of off-line quality control is presented to discuss the practical aspects involved in the application of the results obtained and to address issues such as data transformation, model building and the optimization of design parameters. The analysis shows very encouraging results, and is seen to be able to unveil some important information not found in previous analyses.     

A note on kernel density estimation for non-negative random variables

Kernel-based density estimation algorithms are inefficient in presence of discontinuities at support endpoints. This is substantially due to the fact that classic kernel density estimators lead to positive estimates beyond the endopoints. If a nonparametric estimate of a density functional is required in determining the bandwidth, then the problem also affects the bandwidth selection procedure. In this paper algorithms for bandwidth selection and kernel density estimation are proposed for non-negative random variables. Furthermore, the methods we propose are compared with some of the principal solutions in the literature through a simulation study.     

A clustering approach to interpretable principal components

A new method for constructing interpretable principal components is proposed. The method first clusters the variables, and then interpretable (sparse) components are constructed from the correlation matrices of the clustered variables. For the first step of the method, a new weighted-variances method for clustering variables is proposed. It reflects the nature of the problem that the interpretable components should maximize the explained variance and thus provide sparse dimension reduction. An important feature of the new clustering procedure is that the optimal number of clusters (and components) can be determined in a non-subjective manner. The new method is illustrated using well-known simulated and real data sets. It clearly outperforms many existing methods for sparse principal component analysis in terms of both explained variance and sparseness.     

Central Limit Theorem approximations for the number of runs in Markov-dependent binary sequences

We consider Markov-dependent binary sequences and study various types of success runs (overlapping, non-overlapping, exact, etc.) by examining additive functionals based on state visits and transitions in an appropriate Markov chain. We establish a multivariate Central Limit Theorem for the number of these types of runs and obtain its covariance matrix by means of the recurrent potential matrix of the Markov chain. Explicit expressions for the covariance matrix are given in the Bernoulli and a simple Markov-dependent case by expressing the recurrent potential matrix in terms of the stationary distribution and the mean transition times in the chain. We also obtain a multivariate Central Limit Theorem for the joint number of non-overlapping runs of various sizes and give its covariance matrix in explicit form for Markov dependent trials.     

Exploratory factor and principal component analyses: some new aspects

Exploratory Factor Analysis (EFA) and Principal Component Analysis (PCA) are popular techniques for simplifying the presentation of, and investigating the structure of, an (n×p) data matrix. However, these fundamentally different techniques are frequently confused, and the differences between them are obscured, because they give similar results in some practical cases. We therefore investigate conditions under which they are expected to be close to each other, by considering EFA as a matrix decomposition so that it can be directly compared with the data matrix decomposition underlying PCA. Correspondingly, we propose an extended version of PCA, called the EFA-like PCA, which mimics the EFA matrix decomposition in the sense that they contain the same unknowns. We provide iterative algorithms for estimating the EFA-like PCA parameters, and derive conditions that have to be satisfied for the two techniques to give similar results. Throughout, we consider separately the cases n>p and p≥n. All derived algorithms and matrix conditions are illustrated on two data sets, one for each of these two cases.