Improving the detection of unusual observations in high‐dimensional settings

Multivariate control charts are used to monitor stochastic processes for changes and unusual observations. Hotelling's T² statistic is calculated for each new observation and an out‐of‐control signal is issued if it goes beyond the control limits. However, this classical approach becomes unreliable as the number of variables p approaches the number of observations n, and impossible when p exceeds n. In this paper, we devise an improvement to the monitoring procedure in high‐dimensional settings. We regularise the covariance matrix to estimate the baseline parameter and incorporate a leave‐one‐out re‐sampling approach to estimate the empirical distribution of future observations. An extensive simulation study demonstrates that the new method outperforms the classical Hotelling T² approach in power, and maintains appropriate false positive rates. We demonstrate the utility of the method using a set of quality control samples collected to monitor a gas chromatography–mass spectrometry apparatus over a period of 67 days.     

Improving the Efficacy of Administrative Data for Evaluation of Holistic Defense

A college of social work developed a partnership with a state commission on indigent defense to examine existing data-collection procedures, potential case outcomes, and practical implications of implementing holistic defense programs. The holistic defense model responds to the complex challenges of justice-system-involved defendants by providing social services in public defense offices. Using chi-square and logistic regression analyses of administrative data for a sample of 15,994 public defendants from a single judicial circuit, this research study examined case outcomes before and after implementation of a holistic defense program. Results were mixed regarding effectiveness of holistic defense in mitigating the effect of justice involvement for indigent defendants. Implementation of holistic defense was associated with a decrease in case dismissals. However, defendants receiving holistic representation were less likely to be indicted than defendants prior to program implementation. Once indicted, there were no changes in levels of diversions, and more defendants were held on bond, convicted as guilty, and incarcerated. After program implementation, fewer defendants were sentenced to alternatives to incarceration; more defendants were sentenced to time served, avoiding further incarcerative penalty. These findings suggested the need for further research to determine whether holistic defense practices are producing desired legal and social service outcomes.     

Analysis of robust design experiments with time-dependent ordinal response characteristics: a quality improvement study from the horticulture industry

An approach to the analysis of time-dependent ordinal quality score data from robust design experiments is developed and applied to an experiment from commercial horticultural research, using concepts of product robustness and longevity that are familiar to analysts in engineering research. A two-stage analysis is used to develop models describing the effects of a number of experimental treatments on the rate of post-sales product quality decline. The first stage uses a polynomial function on a transformed scale to approximate the quality decline for an individual experimental unit using derived coefficients and the second stage uses a joint mean and dispersion model to investigate the effects of the experimental treatments on these derived coefficients. The approach, developed specifically for an application in horticulture, is exemplified with data from a trial testing ornamental plants that are subjected to a range of treatments during production and home-life. The results of the analysis show how a number of control and noise factors affect the rate of post-production quality decline. Although the model is used to analyse quality data from a trial on ornamental plants, the approach developed is expected to be more generally applicable to a wide range of other complex production systems.     

Order statistics from trivariate normal and -distributions in terms of generalized skew-normal and skew- distributions

We consider here a generalization of the skew-normal distribution, GSN(λ₁,λ₂,ρ), defined through a standard bivariate normal distribution with correlation ρ, which is a special case of the unified multivariate skew-normal distribution studied recently by Arellano-Valle and Azzalini [2006. On the unification of families of skew-normal distributions. Scand. J. Statist. 33, 561–574]. We then present some simple and useful properties of this distribution and also derive its moment generating function in an explicit form. Next, we show that distributions of order statistics from the trivariate normal distribution are mixtures of these generalized skew-normal distributions; thence, using the established properties of the generalized skew-normal distribution, we derive the moment generating functions of order statistics, and also present expressions for means and variances of these order statistics.Next, we introduce a generalized skew-t_ν distribution, which is a special case of the unified multivariate skew-elliptical distribution presented by Arellano-Valle and Azzalini [2006. On the unification of families of skew-normal distributions. Scand. J. Statist. 33, 561–574] and is in fact a three-parameter generalization of Azzalini and Capitanio's [2003. Distributions generated by perturbation of symmetry with emphasis on a multivariate skew t distribution. J. Roy. Statist. Soc. Ser. B 65, 367–389] univariate skew-t_ν form. We then use the relationship between the generalized skew-normal and skew-t_ν distributions to discuss some properties of generalized skew-t_ν as well as distributions of order statistics from bivariate and trivariate t_ν distributions. We show that these distributions of order statistics are indeed mixtures of generalized skew-t_ν distributions, and then use this property to derive explicit expressions for means and variances of these order statistics.     

Semiparametric estimation for count data through weighted distributions

This paper is concerned with semiparametric discrete kernel estimators when the unknown count distribution can be considered to have a general weighted Poisson form. The estimator is constructed by multiplying the Poisson estimate with a nonparametric discrete kernel-type estimate of the Poisson weight function. Comparisons are then carried out with the ordinary discrete kernel probability mass function estimators. The Poisson weight function is thus a local multiplicative correction factor, and is considered as the uniform measure to detect departures from the equidispersed Poisson distribution. In this way, the effects of dispersion and zero-proportion with respect to the standard Poisson distribution are also minimized. This method of estimation is also applied to the weighted binomial form for the count distribution having a finite support. The proposed estimators, in addition to being simple, easy-to-implement and effective, also outperform the competing nonparametric and parametric estimators in finite-sample situations. Two examples illustrate this new semiparametric estimation.     

Nonparametric control charts based on runs and Wilcoxon-type rank-sum statistics

In this article, we introduce three new distribution-free Shewhart-type control charts that exploit run and Wilcoxon-type rank-sum statistics to detect possible shifts of a monitored process. Exact formulae for the alarm rate, the run length distribution, and the average run length (ARL) are all derived. A key advantage of these charts is that, due to their nonparametric nature, the false alarm rate (FAR) and in-control run length distribution is the same for all continuous process distributions. Tables are provided for the implementation of the charts for some typical FAR values. Furthermore, a numerical study carried out reveals that the new charts are quite flexible and efficient in detecting shifts to Lehmann-type out-of-control situations.     

THE USE OF AGGREGATE DATA TO ESTIMATE GOMPERTZ‐TYPE OLD‐AGE MORTALITY IN HETEROGENEOUS POPULATIONS

We consider two related aspects of the study of old‐age mortality. One is the estimation of a parameterized hazard function from grouped data, and the other is its possible deceleration at extreme old age owing to heterogeneity described by a mixture of distinct sub‐populations. The first is treated by half of a logistic transform, which is known to be free of discretization bias at older ages, and also preserves the increasing slope of the log hazard in the Gompertz case. It is assumed that data are available in the form published by official statistical agencies, that is, as aggregated frequencies in discrete time. Local polynomial modelling and weighted least squares are applied to cause‐of‐death mortality counts. The second, related, problem is to discover what conditions are necessary for population mortality to exhibit deceleration for a mixture of Gompertz sub‐populations. The general problem remains open but, in the case of three groups, we demonstrate that heterogeneity may be such that it is possible for a population to show decelerating mortality and then return to a Gompertz‐like increase at a later age. This implies that there are situations, depending on the extent of heterogeneity, in which there is at least one age interval in which the hazard function decreases before increasing again.     

Comment: there may be compositional effects,but they do not work that way*

This analysis joins the debate on how declines in marriage have shifted the composition of the unmarried and married populations in the United States, and how compositional shifts have affected nonmarital birth rates. Gray, Stockard, and Stone (2006) presented one model for compositional effects that Ermisch (2009) challenged with alternative statistical tests. I propose an alternative model for compositional shifts based not on theory but on observed marriage and fertility patterns. The results from this alternative model are consistent with Ermisch’s findings yet support Gray et al.’s general case that compositional effects have had an important infiuence on nonmarital birth rates.