Applying State Space to SPC: Monitoring Multivariate Time Series

Monitoring cross-sectional and serially interdependent processes has become a new issue in statistical process control (SPC). In up-to-date SPC literature, Kalman filtering was reported to monitor univariate autocorrelated processes. This paper applies a Kalman filter or state-space method for SPC to monitoring multivariate time series. We use Aoki's approach to estimate the parameter matrices of a state-space model. Multivariate Hotelling T ² control charts are employed to monitor the residuals of the state-space. Examples of this approach are illustrated.     

Likelihood Ratio Tests for High‐Dimensional Normal Distributions

In their recent work, Jiang and Yang studied six classical Likelihood Ratio Test statistics under high‐dimensional setting. Assuming that a random sample of size n is observed from a p‐dimensional normal population, they derive the central limit theorems (CLTs) when p and n are proportional to each other, which are different from the classical chi‐square limits as n goes to infinity, while p remains fixed. In this paper, by developing a new tool, we prove that the mentioned six CLTs hold in a more applicable setting: p goes to infinity, and p can be very close to n. This is an almost sufficient and necessary condition for the CLTs. Simulations of histograms, comparisons on sizes and powers with those in the classical chi‐square approximations and discussions are presented afterwards.     

Strictly monotone and smooth nonparametric regression for two or more variables

The authors propose a new monotone nonparametric estimate for a regression function of two or more variables. Their method consists in applying successively one‐dimensional isotonization procedures on an initial, unconstrained nonparametric regression estimate. In the case of a strictly monotone regression function, they show that the new estimate and the initial one are first‐order asymptotic equivalent; they also establish asymptotic normality of an appropriate standardization of the new estimate. In addition, they show that if the regression function is not monotone in one of its arguments, the new estimate and the initial one have approximately the same L^p‐norm. They illustrate their approach by means of a simulation study, and two data examples are analyzed.     

Small-sample intervals for regression

For the general linear regression model Y = Xη + e, we construct small-sample exponentially tilted empirical confidence intervals for a linear parameter 6 = a^Tη and for nonlinear functions of η. The coverage error for the intervals is O_p(1/n), as shown in Tingley and Field (1990). The technique, though sample-based, does not require bootstrap resampling. The first step is calculation of an estimate for η. We have used a Mallows estimate. The algorithm applies whenever η is estimated as the solution of a system of equations having expected value 0. We include calculations of the relative efficiency of the estimator (compared with the classical least-squares estimate). The intervals are compared with asymptotic intervals as found, for example, in Hampel et at. (1986). We demonstrate that the procedure gives sensible intervals for small samples.     

Estimation of the location of a 0-type or ∞-type singularity by Poisson observations

We consider an inhomogeneous Poisson process X on [0, T]. The intensity function of X is supposed to be strictly positive and smooth on [0, T] except at the point θ, in which it has either a 0-type singularity (tends to 0 like |x| ^p , p∈(0, 1)), or an ∞-type singularity (tends to ∞ like |x| ^p , p∈(?1, 0)). We suppose that we know the shape of the intensity function, but not the location of the singularity. We consider the problem of estimation of this location (shift) parameter θ based on n observations of the process X. We study the Bayesian estimators and, in the case p>0, the maximum-likelihood estimator. We show that these estimators are consistent, their rate of convergence is n ^1/(p+1), they have different limit distributions, and the Bayesian estimators are asymptotically efficient.     

Regularized proportional odds models

The proportional odds model (POM) is commonly used in regression analysis to predict the outcome for an ordinal response variable. The maximum likelihood estimation (MLE) approach is typically used to obtain the parameter estimates. The likelihood estimates do not exist when the number of parameters, p, is greater than the number of observations n. The MLE also does not exist if there are no overlapping observations in the data. In a situation where the number of parameters is less than the sample size but p is approaching to n, the likelihood estimates may not exist, and if they exist they may have quite large standard errors. An estimation method is proposed to address the last two issues, i.e. complete separation and the case when p approaches n, but not the case when p>n. The proposed method does not use any penalty term but uses pseudo-observations to regularize the observed responses by downgrading their effect so that they become close to the underlying probabilities. The estimates can be computed easily with all commonly used statistical packages supporting the fitting of POMs with weights. Estimates are compared with MLE in a simulation study and an application to the real data.     

Control variates and importance sampling for efficient bootstrap simulations

Importance sampling and control variates have been used as variance reduction techniques for estimating bootstrap tail quantiles and moments, respectively. We adapt each method to apply to both quantiles and moments, and combine the methods to obtain variance reductions by factors from 4 to 30 in simulation examples.We use two innovations in control variates—interpreting control variates as a re-weighting method, and the implementation of control variates using the saddlepoint; the combination requires only the linear saddlepoint but applies to general statistics, and produces estimates with accuracy of order n ^-1/2 B ^-1, where n is the sample size and B is the bootstrap sample size.We discuss two modifications to classical importance sampling—a weighted average estimate and a mixture design distribution. These modifications make importance sampling robust and allow moments to be estimated from the same bootstrap simulation used to estimate quantiles.     

Skewness Reduction Approach in Multi-Attribute Process Monitoring

Since the product quality of many industrial processes depends upon more than one dependent variable or attribute, they are either multivariate or multi-attribute in nature. Although multivariate statistical process control is receiving increased attention in the literature, little work has been done to deal with multi-attribute processes. In this article, we develop a new methodology to monitor multi-attribute processes. To do this, first we transform multi-attribute data in a way that their marginal probability distributions have almost zero skewness. Then, we estimate the transformed covariance matrix and apply the well-known T ² control chart. In order to illustrate the proposed method and evaluate its performance, we use two simulation experiments and compare the results with the ones from both MNP chart and the χ² control chart.     

A variable selection method for detecting abnormality based on the T2 test

This paper proposes a variable selection method for detecting abnormal items based on the T² test when the observations on abnormal items are available. Based on the unbiased estimates of the powers for all subsets of variables, the variable selection method selects the subset of variables that maximizes the power estimate. Since more than one subsets of variables maximize the power estimate frequently, the averaged p-value of the rejected items is used as a second criterion. Although the performance of the method depends on the sample size for the abnormal items and the true power values for all subsets of variables, numerical experiments show the effectiveness of the proposed method. Since normal and abnormal items are simulated using one-factor and two-factor models, basic properties of the power functions for the models are investigated.     

Bootstrap-Based T 2 Multivariate Control Charts

Control charts have been used effectively for years to monitor processes and detect abnormal behaviors. However, most control charts require a specific distribution to establish their control limits. The bootstrap method is a nonparametric technique that does not rely on the assumption of a parametric distribution of the observed data. Although the bootstrap technique has been used to develop univariate control charts to monitor a single process, no effort has been made to integrate the effectiveness of the bootstrap technique with multivariate control charts. In the present study, we propose a bootstrap-based multivariate T ² control chart that can efficiently monitor a process when the distribution of observed data is nonnormal or unknown. A simulation study was conducted to evaluate the performance of the proposed control chart and compare it with a traditional Hotelling's T ² control chart and the kernel density estimation (KDE)-based T ² control chart. The results showed that the proposed chart performed better than the traditional T ² control chart and performed comparably with the KDE-based T ² control chart. Furthermore, we present a case study to demonstrate the applicability of the proposed control chart to real situations.     

Interpreting the Principal Component Analysis of Multivariate Density Functions

Functional principal component analysis (FPCA) as a reduction data technique of a finite number T of functions can be used to identify the dominant modes of variation of numeric three-way data.

We carry out the FPCA on multidimensional probability density functions, relate this method to other standard methods and define its centered or standardized versions. Grounded on the relationship between FPCA of densities, FPCA of their corresponding characteristic functions, PCA of the MacLaurin expansions of these characteristic functions and dual STATIS method applied to their variance matrices, we propose a method for interpreting the results of the FPCA of densities. This method is based on the investigations of the relationships between the scores of the FPCA and the moments associated to the densities.

The method is illustrated using known Gaussian densities. In practice, FPCA of densities deals with observations of multidimensional variables on T occasions. These observations can be used to estimate the T associated densities (i) by estimating the parameters of these densities, assuming that they are Gaussian, or (ii) by using the Gaussian kernel method and choosing the matrix bandwidth by the normal reference rule. Thereafter, FPCA estimate is derived from these estimates and the interpretation method is carried out to explore the dominant modes of variation of the types of three-way data encountered in sensory analysis and archaeology.     

Central limit theorems for functionals of large sample covariance matrix and mean vector in matrix‐variate location mixture of normal distributions

In this paper, we consider the asymptotic distributions of functionals of the sample covariance matrix and the sample mean vector obtained under the assumption that the matrix of observations has a matrix‐variate location mixture of normal distributions. The central limit theorem is derived for the product of the sample covariance matrix and the sample mean vector. Moreover, we consider the product of the inverse sample covariance matrix and the mean vector for which the central limit theorem is established as well. All results are obtained under the large‐dimensional asymptotic regime, where the dimension p and the sample size n approach infinity such that p/n→c ∈ [0, + ∞) when the sample covariance matrix does not need to be invertible and p/n→c ∈ [0,1) otherwise.     

Jackknifing L-estimates

Linear functions of order statistics (“L-estimates”) of the form T_n =under jackknifing are investigated. This paper proves that with suitable conditions on the function J, the jackknifed version T_n of the L-estimate T_n has the same limit distribution as T_n. It is also shown that the jackknife estimate of the asymptotic variance of n^1/2 is consistent. Furthermore, the Berry-Esséen rate associated with asymptotic normality, and a law of the iterated logarithm of a class of jackknife L-estimates, are characterized.     

Estimation and Inference for Linear Panel Data Models Under Misspecification When Both n and T are Large

This article considers fixed effects (FE) estimation for linear panel data models under possible model misspecification when both the number of individuals, n, and the number of time periods, T, are large. We first clarify the probability limit of the FE estimator and argue that this probability limit can be regarded as a pseudo-true parameter. We then establish the asymptotic distributional properties of the FE estimator around the pseudo-true parameter when n and T jointly go to infinity. Notably, we show that the FE estimator suffers from the incidental parameters bias of which the top order is O(T^{? 1}), and even after the incidental parameters bias is completely removed, the rate of convergence of the FE estimator depends on the degree of model misspecification and is either (nT)^{? 1/2} or n^{? 1/2}. Second, we establish asymptotically valid inference on the (pseudo-true) parameter. Specifically, we derive the asymptotic properties of the clustered covariance matrix (CCM) estimator and the cross-section bootstrap, and show that they are robust to model misspecification. This establishes a rigorous theoretical ground for the use of the CCM estimator and the cross-section bootstrap when model misspecification and the incidental parameters bias (in the coefficient estimate) are present. We conduct Monte Carlo simulations to evaluate the finite sample performance of the estimators and inference methods, together with a simple application to the unemployment dynamics in the U.S.     

Optimal Nonparametric Quantile Estimators. Towards a General Theory. A Survey

“Nonparametric” in the title is used to say that observations X ₁,…,X _n come from an unknown distribution F ∈ ? with ? being the class of all continuous and strictly increasing distribution functions. The problem is to estimate the quantile of a given order q ∈ (0,1) of the distribution F. The class ? of distributions is very large; it is so large that even X _nq:n , where nq is an integer, may be very poor estimator of the qth quantile. To assess the performance of estimators no properties based on moments may be used: expected values of estimators should be replaced by their medians, their variances—by some characteristics of concentration of distributions around the median. If an estimator is median-biased for one of distributions, the bias of the estimator may be infinitely large for other distributions. In the note optimal estimators with respect to various criteria of optimality are presented. The pivotal function F(T) of the estimator T is introduced which enables us to apply the classical statistical approach.     

The Design and Performance of the Multivariate Synthetic-T 2 Control Chart

One of the objectives of research in statistical process control is to obtain control charts that show few false alarms but, at the same time, are able to detect quickly the shifts in the distribution of the quality variables employed to monitor a productive process. In this article, the synthetic-T ² control chart is developed, which consists of the simultaneous use of a CRL chart and a Hotelling's T ² control chart. The ARL is calculated employing Markov chains for steady and zero-state scenarios. A procedure of optimization has been developed to obtain the optimum parameters of the synthetic-T ², for zero and steady cases, given the values of in-control ARL and magnitude of shift which needs to be detected rapidly. A comparison between (standard T ², MEWMA, T ² with variable sample size, and T ² with double sampling) charts reveals that the synthetic-T ² chart always performs better than the standard T ² chart. The comparison with the remaining charts demonstrate in which cases the performance of this new chart makes it interesting to employ in real applications.     

A Bayesian wavelet approach to estimation of a change-point in a nonlinear multivariate time series

ABSTRACT

We propose a semiparametric approach to estimate the existence and location of a statistical change-point to a nonlinear multivariate time series contaminated with an additive noise component. In particular, we consider a p-dimensional stochastic process of independent multivariate normal observations where the mean function varies smoothly except at a single change-point. Our approach involves conducting a Bayesian analysis on the empirical detail coefficients of the original time series after a wavelet transform. If the mean function of our time series can be expressed as a multivariate step function, we find our Bayesian-wavelet method performs comparably with classical parametric methods such as maximum likelihood estimation. The advantage of our multivariate change-point method is seen in how it applies to a much larger class of mean functions that require only general smoothness conditions.     

Consistency of the Subsample Bootstrap empirical process

In the classical Bootstrap approach the number of distinct observation in the resample is random. To overcome this hitch Rao et al. [Bootstrap by sequential resampling, J. Statist. Plan. Inference 64 (1997), pp. 257–281] have proposed a modified resampling procedure – the so-called Sequential Bootstrap or 0.632-Bootstrap – in which each resample has exactly the same number m ～eq ?0.632 n? of distinct observations. Motivated by this idea we introduce an akin procedure, the Subsample Bootstrap, where additionally even the size of each resample is equal. It will turn out that the Subsample Bootstrap empirical process is consistent for a wide class of Donsker classes.     

Diagnosis of Multivariate Control Chart Signal Based on Dummy Variable Regression Technique

Abstract

It is common to monitor several correlated quality characteristics using the Hotelling's T ² statistic. However, T ² confounds the location shift with scale shift and consequently it is often difficult to determine the factors responsible for out of control signal in terms of the process mean vector and/or process covariance matrix. In this paper, we propose a diagnostic procedure called ‘D-technique’ to detect the nature of shift. For this purpose, two sets of regression equations, each consisting of regression of a variable on the remaining variables, are used to characterize the ‘structure’ of the ‘in control’ process and that of ‘current’ process. To determine the sources responsible for an out of control state, it is shown that it is enough to compare these two structures using the dummy variable multiple regression equation. The proposed method is operationally simpler and computationally advantageous over existing diagnostic tools. The technique is illustrated with various examples.     

Maximum Entropy Approximations for Asymptotic Distributions of Smooth Functions of Sample Means

Abstract. We propose an information‐theoretic approach to approximate asymptotic distributions of statistics using the maximum entropy (ME) densities. Conventional ME densities are typically defined on a bounded support. For distributions defined on unbounded supports, we use an asymptotically negligible dampening function for the ME approximation such that it is well defined on the real line. We establish order n^?1 asymptotic equivalence between the proposed method and the classical Edgeworth approximation for general statistics that are smooth functions of sample means. Numerical examples are provided to demonstrate the efficacy of the proposed method.