Identifying the time of a step change with multivariate single control charts

Change point estimation procedures simplify the efforts to search for and identify special causes in multivariate statistical process monitoring. After a signal is generated by the simultaneously used control charts or a single control chart, add-on change point procedure estimates the time of the change. In this study, multivariate joint change point estimation performance for simultaneous monitoring of both location and dispersion is compared under the assumption that various single charts are used to monitor the process. The change detection performance for several structural changes for the mean vector and covariance matrix is also discussed. It is concluded that choice of the control chart to obtain a signal may affect the change point detection performance.     

Another look at linear programming for feature selection via methods of regularization

We consider statistical procedures for feature selection defined by a family of regularization problems with convex piecewise linear loss functions and penalties of l ₁ nature. Many known statistical procedures (e.g. quantile regression and support vector machines with l ₁-norm penalty) are subsumed under this category. Computationally, the regularization problems are linear programming (LP) problems indexed by a single parameter, which are known as ‘parametric cost LP’ or ‘parametric right-hand-side LP’ in the optimization theory. Exploiting the connection with the LP theory, we lay out general algorithms, namely, the simplex algorithm and its variant for generating regularized solution paths for the feature selection problems. The significance of such algorithms is that they allow a complete exploration of the model space along the paths and provide a broad view of persistent features in the data. The implications of the general path-finding algorithms are outlined for several statistical procedures, and they are illustrated with numerical examples.     

A Comparative Study of Different Methodologies for Fault Diagnosis in Multivariate Quality Control

Different methodologies for fault diagnosis in multivariate quality control have been proposed in recent years. These methods work in the space of the original measured variables and have performed reasonably well when there is a reduced number of mildly correlated quality and/or process variables with a well-conditioned covariance matrix. These approaches have been introduced by emphasizing their positive or negative virtues, generally on an individual basis, so it is not clear for the practitioner the best method to be used. This article provides a comprehensive study of the performance of diverse methodological approaches when tested on a large number of distinct simulated scenarios. Our primary aim is to highlight key weaknesses and strengths in these methods as well as clarifying their relationships and the requirements for their implementation in practice.     

Minimizing the expected time to detect a randomly located lost target using 3-dimensional search technique

Abstract

This paper considers a new model in search theory to find a randomly located target in the 3-dimensional space. An approximation algorithm that facilitates searching procedures for searchers or robots is presented. The expected time to detect the target is also proved. The statistical analysis by calculating the optimal search strategy which minimizes the time to detect the target, assuming trivariate standard normal distribution is provided, and the technique by flowcharts is designed as well. The effectiveness of this strategy is illustrated by introducing an application from real world.     

Bayesian t-tests for correlations and partial correlations

In this paper, we develop Bayes factor based testing procedures for the presence of a correlation or a partial correlation. The proposed Bayesian tests are obtained by restricting the class of the alternative hypotheses to maximize the probability of rejecting the null hypothesis when the Bayes factor is larger than a specified threshold. It turns out that they depend simply on the frequentist t-statistics with the associated critical values and can thus be easily calculated by using a spreadsheet in Excel and in fact by just adding one more step after one has performed the frequentist correlation tests. In addition, they are able to yield an identical decision with the frequentist paradigm, provided that the evidence threshold of the Bayesian tests is determined by the significance level of the frequentist paradigm. We illustrate the performance of the proposed procedures through simulated and real-data examples.     

A new Markov chain approach for the economic statistical design of the VSS T2 control chart

An economic statistical design model for a T² chart which uses a variable sample size (VSS) feature is developed in this article. This study mainly differs from the others conducted in the field. In that a new approach is offered to achieve closed form of some statistical criteria. In other words, the proposed formulas can be considered as a better alternative approach in designing the VSS control charts in terms of simplicity and yet providing the users with better optimal solutions.     

Remedying the Neyman–Scott phenomenon in model discrimination

The objective of this paper is to investigate through simulation the possible presence of the incidental parameters problem when performing frequentist model discrimination with stratified data. In this context, model discrimination amounts to considering a structural parameter taking values in a finite space, with k points, k≥2. This setting seems to have not yet been considered in the literature about the Neyman–Scott phenomenon. Here we provide Monte Carlo evidence of the severity of the incidental parameters problem also in the model discrimination setting and propose a remedy for a special class of models. In particular, we focus on models that are scale families in each stratum. We consider traditional model selection procedures, such as the Akaike and Takeuchi information criteria, together with the best frequentist selection procedure based on maximization of the marginal likelihood induced by the maximal invariant, or of its Laplace approximation. Results of two Monte Carlo experiments indicate that when the sample size in each stratum is fixed and the number of strata increases, correct selection probabilities for traditional model selection criteria may approach zero, unlike what happens for model discrimination based on exact or approximate marginal likelihoods. Finally, two examples with real data sets are given.     

A Comparison of Two Tests for Equality of Two Proportions

Two procedures for testing equality of two proportions are compared in terms of asymptotic efficiency. The comparison favors use of a statistic equivalent to Goodman's Y ² over the usual X ² statistic in some cases including that of equal sample sizes. Numerical comparisons indicate that the asymptotic results have some relevance for moderate sample sizes.     

Exact Statistical Tests for Heterogeneity of Frequencies Based on Extreme Values

Sophisticated statistical analyses of incidence frequencies are often required for various epidemiologic and biomedical applications. Among the most commonly applied methods is the Pearson's χ² test, which is structured to detect non specific anomalous patterns of frequencies and is useful for testing the significance for incidence heterogeneity. However, the Pearson's χ² test is not efficient for assessing the significance of frequency in a particular cell (or class) to be attributed to chance alone. We recently developed statistical tests for detecting temporal anomalies of disease cases based on maximum and minimum frequencies; these tests are actually designed to test of significance for a particular high or low frequency. The purpose of this article is to demonstrate merits of these tests in epidemiologic and biomedical studies. We show that our proposed methods are more sensitive and powerful for testing extreme cell counts than is the Pearson's χ² test. This feature could provide important and valuable information in epidemiologic or biomeidcal studies. We elucidated and illustrated the differences in sensitivity among our tests and the Pearson's χ² test by analyzing a data set of Langerhans cell histiocytosis cases and its hypothetical sets. We also computed and compared the statistical power of these methods using various sets of cell numbers and alternative frequencies. The investigation of statistical sensitivity and power presented in this work will provide investigators with useful guidelines for selecting the appropriate tests for their studies.     

Empirical Bayes estimation in directional data

Empirical Bayes procedures have been developed extensively in the literature, under the assumption that the underlying parameter space (or the sample space) is Euclidean in nature. However, there has been almost no research carried out into when the data comes from a different space. We develop empirical Bayes techniques to estimate the mean direction of the Fisher-von Mises distribution. In this case, the underlying space is non-Euclidean. The special case when the data are angles on the unit circle is illustrated with an example.     

Hierarchical likelihood approach to non-Gaussian factor analysis

Factor models, structural equation models (SEMs) and random-effect models share the common feature that they assume latent or unobserved random variables. Factor models and SEMs allow well developed procedures for a rich class of covariance models with many parameters, while random-effect models allow well developed procedures for non-normal models including heavy-tailed distributions for responses and random effects. In this paper, we show how these two developments can be combined to result in an extremely rich class of models, which can be beneficial to both areas. A new fitting procedures for binary factor models and a robust estimation approach for continuous factor models are proposed.     

A cell means analysis of mixed linear models

The purpose of this paper is to describe a simple procedure for the estima-tion of parameters in the unbalanced mixed linear model. There are implications for hypothesis testing and interval estimation, A feature of these estimators is that they are expressed in terms of simple formulas. This has obvious advantages for computations and small sample analysis. In addition, the formulas suggest useful diagnostic procedures for assessing the quality of the data as well as possible defects in the model assumptions. The concepts are illustrated with several examples. Evidence is presented to indicate that, in cases of modest imbalance, these estimators are highly efficient and dominate AOV estimates over most of the parameter space. In cases of more extreme imbalance, the results are qualitatively the same but the estimators are less efficient than the AOV estimators for small values of the parameters. The extension of this method to factorial models with missing cells is not complete.     

Classification of an observation into linear manifolds,their closed subsets and closed convex subsets

In this paper, we generalize the notion of classification of an observation (sample), into one of the given n populations to the case where some or all of the populations into which the new observation is to be classified may be new but related in a simple way to the given n populations. The discussion is in the frame-work of the given set of observations obeying the usual multivariate general linear hypothesis model. The set ofpopulations into which the new observation may be classified could be linear manifolds of the parameter space or their closed subsets or closed convex subsets or a combination of them or simply t subsets of the parameter space each of which has a finite number of elements. In the last case alikelihood ratio procedure can be obtained easily. Classification procedures given here are based on Mahalanobis distance. Bonferroni lower bound estimate of the probability of correctly classifying an observation is given for the case when the covariance matrix is known or is estimated from a large sample. A numerical example relating to the classification procedures suggested her is given.     

Exhaustive k-nearest-neighbour subspace clustering

Cluster analysis is an important technique of explorative data mining. It refers to a collection of statistical methods for learning the structure of data by solely exploring pairwise distances or similarities. Often meaningful structures are not detectable in these high-dimensional feature spaces. Relevant features can be obfuscated by noise from irrelevant measurements. These observations led to the design of subspace clustering algorithms, which can identify clusters that originate from different subsets of features. Hunting for clusters in arbitrary subspaces is intractable due to the infinite search space spanned by all feature combinations. In this work, we present a subspace clustering algorithm that can be applied for exhaustively screening all feature combinations of small- or medium-sized datasets (approximately 30 features). Based on a robustness analysis via subsampling we are able to identify a set of stable candidate subspace cluster solutions.     

Estimation in noncentral distributions

This paper investigates alternatives to MIU estimators in noncentral X ² and F distributions. Two directions are pursued. In the first, a general approach for uniformly improving on MVU estimators is described and illustrated. In the second, Bayesian, procedures are characterized and illustrated as well. This effort extends earlier work of Perlman and Rasmussen and of Neff and Strawderman.     

Sparse discriminant analysis based on estimation of posterior probabilities

ABSTRACT

Fisher's linear discriminant analysis (FLDA) is known as a method to find a discriminative feature space for multi-class classification. As a theory of extending FLDA to an ultimate nonlinear form, optimal nonlinear discriminant analysis (ONDA) has been proposed. ONDA indicates that the best theoretical nonlinear map for maximizing the Fisher's discriminant criterion is formulated by using the Bayesian a posterior probabilities. In addition, the theory proves that FLDA is equivalent to ONDA when the Bayesian a posterior probabilities are approximated by linear regression (LR). Due to some limitations of the linear model, there is room to modify FLDA by using stronger approximation/estimation methods. For the purpose of probability estimation, multi-nominal logistic regression (MLR) is more suitable than LR. Along this line, in this paper, we develop a nonlinear discriminant analysis (NDA) in which the posterior probabilities in ONDA are estimated by MLR. In addition, in this paper, we develop a way to introduce sparseness into discriminant analysis. By applying L1 or L2 regularization to LR or MLR, we can incorporate sparseness in FLDA and our NDA to increase generalization performance. The performance of these methods is evaluated by benchmark experiments using last_exam17 standard datasets and a face classification experiment.     

Identification of out of control quality characteristics in a multivariate manufacturing environment

There are many instances in which the quality of a product or constancy of a process is determined by the joint levels of several attributes or properties. During the conduct of such a process or the production of such a product, one wishes to detect as quickly as possible any departure from a satisfactory state, while at the same time identifying which attributes are responsible for the deviation. In most cases of practical interest, however, there exist correlations among the several properties of interest; this makes it advisable to monitor certain aggregate characteristics of the process, rather than observing its various components separately. When the mean vector of the quality attributes is the major concern, this aggregate monitoring function is most commonly implemented via a T ² chart. The dependencies among attributes, however, complicate the determination of which are responsible when a deviation occurs. This paper presents an approach to help identify aberrant variables when Shewhart type multivariate control charts based on Hotelling's T ² are in use.     

Multivariate analysis of variance with additional data from an unknown subset of the populations

In this paper we consider the problem of testing the means of k multivariate normal populations with additional data from an unknown subset of the k populations. The purpose of this research is to offer test procedures utilizing all the available data for the multivariate analysis of variance problem because the additional data may contain valuable information about the parameters of the k populations. The standard procedure uses only the data from identified populations. We provide a test using all available data based upon Hotelling' s generalized T²statistic. The power of this test is computed using Betz's approximation of Hotelling' s generalized T²statistic by an F-distribution. A comparison of the power of the test and the standard test procedure is also given.     

Testing Equality of Means Under Order Restrictions Using Multiple Contrasts

The likelihood ratio test for equality of ordered means is known to have power characteristics that are generally superior to those of competing procedures. Difficulties in implementing this test have led to the development of alternative approaches, most of which are based on contrasts. While orthogonal contrasts can be chosen to simplify the distribution theory, we propose a class of tests that is easy to implement even if the contrasts used are not orthogonal. An overall measure of significance may be obtained by using Fisher's combination statistic to combine the dependent p-values arising from these contrasts. This method can be easily implemented for testing problems involving unequal sample sizes and any partial order, and has power properties that compare well with those of the likelihood ratio test and other contrast-based tests.     

Goodness-of-fit tests of hypothesized canonical variables

In the literature goodness-of-fit tests for canonical variables are available either in the x space or in the y space, see e.g., Bartlett (1951), Kshirsagar (1971, 1972), Radcliffe (1968), and Williams (1952). Here we present goodness-of-fit tests for canonical variables in both the x and y spaces. The results appear as extensions of the results of the above authors.