Modular-transform based clustering

Spectral clustering uses eigenvectors of the Laplacian of the similarity matrix. It is convenient to solve binary clustering problems. When applied to multi-way clustering, either the binary spectral clustering is recursively applied or an embedding to spectral space is done and some other methods, such as K-means clustering, are used to cluster the points. Here we propose and study a K-way clustering algorithm – spectral modular transformation, based on the fact that the graph Laplacian has an equivalent representation, which has a diagonal modular structure. The method first transforms the original similarity matrix into a new one, which is nearly disconnected and reveals a cluster structure clearly, then we apply linearized cluster assignment algorithm to split the clusters. In this way, we can find some samples for each cluster recursively using the divide and conquer method. To get the overall clustering results, we apply the cluster assignment obtained in the previous step as the initialization of multiplicative update method for spectral clustering. Examples show that our method outperforms spectral clustering using other initializations.     

Generalized estimating equations for mixtures with varying concentrations

A finite mixture model is considered in which the mixing probabilities vary from observation to observation. A parametric model is assumed for one mixture component distribution, while the others are nonparametric nuisance parameters. Generalized estimating equations (GEE) are proposed for the semi‐parametric estimation. Asymptotic normality of the GEE estimates is demonstrated and the lower bound for their dispersion (asymptotic covariance) matrix is derived. An adaptive technique is developed to derive estimates with nearly optimal small dispersion. An application to the sociological analysis of voting results is discussed. The Canadian Journal of Statistics 41: 217–236; 2013 © 2013 Statistical Society of Canada     

Fisher's Hypergeometric Test for a Comparison in a Finite Population

Graphical methods have played a central role in the development of statistical theory and practice. This presentation briefly reviews some of the highlights in the historical development of statistical graphics and gives a simple taxonomy that can be used to characterize the current use of graphical methods. This taxonomy is used to describe the evolution of the use of graphics in some major statistical and related scientific journals.

Some recent advances in the use of graphical methods for statistical analysis are reviewed, and several graphical methods for the statistical presentation of data are illustrated, including the use of multicolor maps.     

Developing “A Framework for Planning U.S. Federal Statistics, 1978–1989”

The Statistical Policy Division of the Office of Management and Budget has the overall responsibility for the planning and coordination of U.S. government statistics. The present staff of the Statistical Policy Division is attempting, through an integrated publication entitled “A Framework for Planning U.S. Federal Statistics, 1978–1989,” to state its perspective on necessary developments in the coming years. This article illustrates the character of the Framework materials and outlines the process for public review and comment on this undertaking.     

The Consistent System

The Consistent System (CS) is an interactive computer system for researchers in the behavioral and policy sciences and in fields with similar requirements for data management and statistical analysis. The researcher is not expected to be a programmer. The system offers a wide range of facilities and permits the user to combine them in novel ways. In particular, tools for statistical analysis may be used in combination with a powerful relational subsystem for data base management. This paper gives an overview of the objectives, capabilities, status, and availability of the system.     

The Planning Stage in Statistical Reasoning

An important step in the statistical problem-solving process is the selection of the appropriate statistical procedure for the real-world situation under analysis. A decision-tree term project has been found to be an effective teaching device to help MBA students understand this step. The project requires the students to construct a decision-tree structure, which, through a series of questions and responses, will lead from the statement of a statistical question to the appropriate sampling distribution to use in addressing the question.     

Nenparametric Two-Group Classification: Concepts and a SAS-Based Software Package

This article introduces BestClass, a set of SAS macros, available in the mainframe and workstation environment, designed for solving two-group classification problems using a class of recently developed nonparametric classification methods. The criteria used to estimate the classification function are based on either minimizing a function of the absolute deviations from the surface which separates the groups, or directly minimizing a function of the number of misclassified entities in the training sample. The solution techniques used by BestClass to estimate the classification rule use the mathematical programming routines of the SAS/OR software. Recently, a number of research studies have reported that under certain data conditions this class of classification methods can provide more accurate classification results than existing methods, such as Fisher's linear discriminant function and logistic regression. However, these robust classification methods have not yet been implemented in the major statistical packages, and hence are beyond the reach of those statistical analysts who are unfamiliar with mathematical programming techniques. We use a limited simulation experiment and an example to compare and contrast properties of the methods included in Best-Class with existing parametric and nonparametric methods. We believe that BestClass contributes significantly to the field of nonparametric classification analysis, in that it provides the statistical community with convenient access to this recently developed class of methods. BestClass is available from the authors.     

The performance of multivariate CUSUM control charts with estimated parameters

In this paper, we study the effect of estimating the vector of means and the variance–covariance matrix on the performance of two of the most widely used multivariate cumulative sum (CUSUM) control charts, the MCUSUM chart proposed by Crosier [Multivariate generalizations of cumulative sum quality-control schemes, Technometrics 30 (1988), pp. 291–303] and the MC1 chart proposed by Pignatiello and Runger [Comparisons of multivariate CUSUM charts, J. Qual. Technol. 22 (1990), pp. 173–186]. Using simulation, we investigate and compare the in-control and out-of-control performances of the competing charts in terms of the average run length measure. The in-control and out-of-control performances of the competing charts deteriorate significantly if the estimated parameters are used with control limits intended for known parameters, especially when only a few Phase I samples are used to estimate the parameters. We recommend the use of the MC1 chart over that of the MCUSUM chart if the parameters are estimated from a small number of Phase I samples.     

数据挖掘中关联规则的规范化描述

一、引言数据挖掘(Data Mining)是近年来随着人工智能和数据库技术的发展而出现的一门新兴学科。它是从大量的数据中筛选出隐含的、可信的、新颖的、有效的信息的高级处理过程。关联规则(Association Rule)是其中重要的研究课题,是数据挖掘的主要技术之一,也是在无指导学习系统     

The RL2 chart versus the np chart for detecting upward shifts in fraction defective

The use of the np chart for monitoring fraction-defective is well-established, but there are a number of relatively simple alternatives based on run-lengths of conforming items. Here, the RL₂ chart, based on the moving sum of two successive conforming run-lengths, is investigated in order to provide SPC practitioners with clear-cut guidance on the comparative performance of these competing charts. Both sampling inspection and 100% inspection are considered here, and it is shown that the RL₂ chart can often be considerably more efficient than the np chart, but the comparative performance depends on the false-alarm rate used for the comparison. Graphs to aid parameter-choice for the RL₂ chart are also provided.