Revisiting retesting in the estimation of proportions by group testing

In the estimation of a proportion p by group testing (pooled testing), retesting of units within positive groups has received little attention due to the minimal gain in precision compared to testing additional units. If acquisition of additional units is impractical or too expensive, and testing is not destructive, we show that retesting can be a useful option. We propose the retesting of a random grouping of units from positive groups, and compare it with nested halving procedures suggested by others. We develop an estimator of p for our proposed method, and examine its variance properties. Using simulation we compare retesting methods across a range of group testing situations, and show that for most realistic scenarios, our method is more efficient.     

Another look at life testing

In life testing, n identical testing items are placed on test. Instead of doing a complete life testing with all n outcomes, a Type II censored life testing, consisting of the first m outcomes, is usually employed. Although statistical analysis for the life testing based on censored data is less efficient than the complete life testing, the expected length of the censored life testing is less than that of the complete life testing. In this paper, we compare censored and complete life testing and suggest ways to improve time saving and efficiency. Instead of doing a complete life testing with all n outcomes, we put N>n items on test, which continues until we observe the nth outcome. With both the censored life testing and the complete life testing containing the same number of observations, we show that the expected length of the censored life testing is less than that of the complete life testing and that the censored life testing may be also more efficient than the complete life testing with the same number of observations.     

Control charts for attributes with maxima nominated samples

We develop quality control charts for attributes using the maxima nomination sampling (MNS) method and compare them with the usual control charts based on simple random sampling (SRS) method, using average run length (ARL) performance, the required sample size in detecting quality improvement, and non-existence region for control limits. We study the effect of the sample size, the set size, and nonconformity proportion on the performance of MNS control charts using ARL curve. We show that MNS control chart can be used as a better benchmark for indicating quality improvement or quality deterioration relative to its SRS counterpart. We consider MNS charts from a cost perspective. We also develop MNS attribute control charts using randomized tests. A computer program is designed to determine the optimal control limits for an MNS p-chart such that, assuming known parameter values, the absolute deviation between the ARL and a specific nominal value is minimized. We provide good approximations for the optimal MNS control limits using regression analysis. Theoretical results are augmented with numerical evaluations. These show that MNS based control charts can yield substantial improvement over the usual control charts based on SRS.     

Optimal Viterbi Bayesian predictive classification for data from finite alphabets

A family of Viterbi Bayesian predictive classifiers has been recently popularized for speech recognition applications with continuous acoustic signals modeled by finite mixture densities embedded in a hidden Markov framework. Here we generalize such classifiers to sequentially observed data from multiple finite alphabets and derive the optimal predictive classifier under exchangeability of the emitted symbols. We demonstrate that the optimal predictive classifier which learns from unlabelled test items improves considerably upon marginal maximum a posteriori rule in the presence of sparse training data. It is shown that the learning process saturates when the amount of test data tends to infinity, such that no further gain in classification accuracy is possible upon arrival of new test items in the long run.     

OPTIMAL DESIGNS FOR 2 k PAIRED COMPARISON EXPERIMENTS

In this paper we establish the form of the optimal paired comparison design when there are k attributes, each with two levels, for testing for main effects, for main effects and two factor interactions and for main effects and two and three factor interactions. In all cases we assume that all pairs with the same number of attributes different appear equally often. In this setting the D and A optimal designs for main effects are the foldover pairs and those for main effects and two factor interactions have pairs in which about half the attributes are different.     

Optimal Designs for 2 k Choice Experiments

Abstract

In this article we establish the choice sets in the D-optimal design for a choice experiment for testing main effects and for testing main effects and two-factor interactions, when there are k attributes, each with two levels, for choice set size m. We also give a method to construct optimal and near-optimal designs with small numbers of choice sets.     

On the use of the t-statistic in preliminary testing procedures for the Behrens-Fisher problem

Preliminary testing procedures for the two means problem traditionally employ the pooled variance t-statistic. In this paper we show that bias of the t-statistic under conditions of heterogeneity of variance may be increased if use of the t-statistic is conditional on an affirmative F-test. For this reason we conclude that use of the t-statistic in preliminary testing procedures is inappropriate.     

Improved bounds for group testing designs

Group testing designs (GTDs), both adaptive and nonadaptive, are useful in reducing the number of tests needed to identify the defective items from a given set of at least six items. In this paper, we obtain improved bounds on the number of group tests necessary for both adaptive and nonadaptive GTDs. It is established that any nonadaptive GTD needs at least $2n$ group tests for identifying all the defective items from a group of $2^n$ items having at most 2 defective items. In the same context, an adaptive multistage GTD with a maximum of $2n$ group tests is presented here. It is further shown that under restrictions on group size, optimal nonadaptive GTDs can be constructed using Generalized Petersen Graphs. Also presented is the construction of a family of two-stage adaptive GTDs that are useful under certain conditions.     

An optimal search procedure

Consider the following problem. There are exactly two defective (unknown) elements in the set X={x₁, x₂,…,x_n}, all possibilities occuring with equal probabilities. We want to identify the unknown (defective) elements by testing some subsets A of X, and for each such set A determining whether A contains any of them. The test on an individual subset A informs us that either all elements of the tested set A are good, or that at least one of them is defective (but we do not know which ones or how many). A set containing at least one defective element is said to be defective. Our aim is to minimize the maximal number of tests. For the optimal strategy, let the maximal test length be denoted by l₂(n). We obtain the value of this function for an infinite sequence of values of n.     

Scalable module detection for attributed networks with applications to breast cancer

The objective of network module detection is to identify groups of nodes within a network structure that are tightly connected. Nodes in a network often have attributes (aka metadata) associated with them. It is often desirable to identify groups of nodes that are tightly connected in the network structure, but also have strong similarity in their attributes. Utilizing attribute information in module detection is a major challenge because it requires bridging the structural network with attribute data. A Weighted Fast Greedy (WFG) algorithm for attribute-based module detection is proposed. WFG utilizes logistic regression to bridge the structural and attribute spaces. The logistic function naturally emphasizes associations between attributes and network structure accordingly, and can be easily interpreted. A breast cancer application is presented that connects a protein–protein interaction network gene expression data and a survival outcome. This application demonstrates the importance of embedding attribute information into the community detection framework on a breast cancer dataset. Five modules were significant for survival and they contained known pathways and markers for cancer, including cell cycle, p53 pathway, BRCA1, BRCA2, and AURKB, among others. Whereas, neither the gene expression data nor the network structure alone gave rise to these cancer biomarkers and signatures.     

Existence theorems for some group testing strategies

Group testing problems are considered as examples of discrete search problems. Existence theorems for optimal nonsequential designs developed for the general discrete search problems in O'Geran et al. (Acta Appl. Math. 25 (1991) 241–276) are applied for construction of upper bounds for the length of optimal group testing strategies in the case of additive model. The key point in the study is derivation of analytic expressions for the so-called Renyi coefficients. In addition, some asymptotic results are obtained and an asymptotic design problem is considered. The results particularly imply that if the number of significant factors is relatively small compared to the total number of factors then the choice of the test collections all containing a half of the total number of factors is asymptotically optimal in a proper sense.     

Group testing with test error as a function of concentration

A common assumption in group testing applications is that there is no test error, i.e., misclassification of a single item or a group of items cannot occur. Graff and Roeloffs ( 1972 ) have proposed a procedure applicable when there is a known probability of misclassification. We generalize their results to the situation where the probability of misclassification depends on the proportion of defective items in the group.     

FITTING FINITE MIXTURE MODELS IN A REGRESSION CONTEXT

Suppose data are collected in a three-mode fashion (individuals x items X attributes), and it is sought to cluster the individuals into groups on the basis of lineat relations between scores on the attributes for each item and auxiliary measurements made on the same items. A mixture model is pro posed and the EM algorithm is used to fit it to the data by simultaneously estimating the group parameters and allocating individuals to groups. The method is illustrated by a simulation study and a real example in which consumers are clustered on the basis of product scores that are related to a sensory laboratory measurement.     

Optimal Bayesian estimators for latent variable cluster models

In cluster analysis interest lies in probabilistically capturing partitions of individuals, items or observations into groups, such that those belonging to the same group share similar attributes or relational profiles. Bayesian posterior samples for the latent allocation variables can be effectively obtained in a wide range of clustering models, including finite mixtures, infinite mixtures, hidden Markov models and block models for networks. However, due to the categorical nature of the clustering variables and the lack of scalable algorithms, summary tools that can interpret such samples are not available. We adopt a Bayesian decision theoretical approach to define an optimality criterion for clusterings and propose a fast and context-independent greedy algorithm to find the best allocations. One important facet of our approach is that the optimal number of groups is automatically selected, thereby solving the clustering and the model-choice problems at the same time. We consider several loss functions to compare partitions and show that our approach can accommodate a wide range of cases. Finally, we illustrate our approach on both artificial and real datasets for three different clustering models: Gaussian mixtures, stochastic block models and latent block models for networks.     

Nonadaptive group testing with lies: Probabilistic existence theorems

We consider a wide range of combinatorial group testing problems with lies including binary, additive and multiaccess channel group testing problems. We derive upper bounds for the number of tests in the optimal nonadaptive algorithms. The derivation is probabilistic and is therefore non-constructive; it does not provide the way of constructing optimal algorithms. In the asymptotic setting, we show that the leading term for the number of tests does not depend on the number of lies and it is thus the same as for the zero-lie case. However, the other terms in the asymptotic upper bounds depend on the number of lies and substantially influence the upper bounds in the non-asymptotic situation.     

Exact calculation of the Dodge-Romig LTPD single sampling plans for inspection by variables

In this paper we shall deal with the acceptance sampling plans when the remainder of rejected lots is inspected. We shall consider two types of LTPD plans- for inspection by variables and for inspection by variables and attributes (all items from the sample are inspected by variables, remainder of rejected lots is inspected by attributes). We shall report on an algorithm allowing the calculation of these plans when the non-central t distribution is used for the operating characteristic. The calculation is considerably difficult, algorithm for non-central t distribution takes several minutes. For the calculation we shall use an original method.     

A computer program for the design of group testing experiments

This paper describes a computer program GTEST for designing group testing experiments for classifying each member of a population of items as “good” or “defective”. The outcome of a test on a group of items is either “negative” (if all items in the group are good) or “positive” (if at least one of the items is defective, but it is not known which). GTEST is based on a Bayesian approach. At each stage, it attempts to maximize (nearly) the expected reduction in the “entropy”, which is a quantitative measure of the amount of uncertainty about the state of the items. The user controls the procedure through specification of the prior probabilities of being defective, restrictions on the construction of the test group, and priorities that are assigned to the items. The nominal prior probabilities can be modified adaptively, to reduce the sensitivity of the procedure to the proportion of defectives in the population.     

Pairwise comparisons for proportions estimated by pooled testing

When estimating the prevalence of a rare trait, pooled testing can confer substantial benefits when compared to individual testing. In addition to screening experiments for infectious diseases in humans, pooled testing has also been exploited in other applications such as drug testing, epidemiological studies involving animal disease, plant disease assessment, and screening for rare genetic mutations. Within a pooled-testing context, we consider situations wherein different strata or treatments are to be compared with the goals of assessing significant and practical differences between strata and ranking strata in terms of prevalence. To achieve these goals, we first present two simultaneous pairwise interval estimation procedures for use with pooled data. Our procedures rely on asymptotic results, so we investigate small-sample behavior and compare the two procedures in terms of simultaneous coverage probability and mean interval length. We then present a unified approach to determine pool sizes which deliver desired coverage properties while taking testing costs and interval precision into account. We illustrate our methods using data from an observational HIV study involving heterosexual males who use intravenous drugs.     

Dorfman-sterrett screening (group testing) schemes and the effects of faulty inspection

The Dorfman screening procedure is based on first testing a group of items as a whole, proceeding to individual testing only if the group-test indicates existence of at least one nonconforming item. A modification suggested by Sterrett allows for reintroduct-ion of group testing of all items, not yet tested individually, when an item is classified as nonconforming by an individual test. Effects of faulty test inspection on the properties of the modified procedures are studied.     

Bias correction of estimated proportions using inverse binomial group testing

Group testing, in which individuals are pooled together and tested as a group, can be combined with inverse sampling to estimate the prevalence of a disease. Alternatives to the MLE are desirable because of its severe bias. We propose an estimator based on the bias correction method of Firth (1993), which is almost unbiased across the range of prevalences consistent with the group testing design. For equal group sizes, this estimator is shown to be equivalent to that derived by applying the correction method of Burrows (1987), and better than existing methods. For unequal group sizes, the problem has some intractable elements, but under some circumstances our proposed estimator can be found, and we show it to be almost unbiased. Calculation of the bias requires computer‐intensive approximation because of the infinite number of possible outcomes.