An empirical bayes approach to estimating the probability of correct selection

In the problem of selecting the best of k populations, Olkin, Sobel, and Tong (1976) have introduced the idea of estimating the probability of correct selection. In an attempt to improve on their estimator we consider anempirical Bayes approach. We compare the two estimators via analytic results and a simulation study.     

Conditions under which F. {N } = for tong's adaptive solution to ranking and selection problems

Tong ‘1978’ proposed an adaptive approach as an alternative to the classical indifference-zone formulation of the problems of ranking and selection. With a fixed pre-selected y*‘1/k < y* < 1’ his procedure calls for the termination of vector-at-a-time sampling when the estimated probability of a correct selection exceeds Y* for the first time. The purpose of this note is to show that for the case of two normal populations with common known variance, the expected number of vector-observations required by Tong's procedure to terminate sampling approaches infinity as the two population means approach equality for Y* ≥ 0.8413.It is conjectured that this phenomenon also persists if the two largest of K ≥3 population means approach equality. Since in the typical ranking and selection setting it usually is assumed that the experimenter has no knowledge concerning the differences between the population means, the experimenter who uses Tong's procedure clearly does so at his own risk.     

Selection and ranking procedures: a brief introduction

This paper presents a brief introduction to selection and ranking methodology. Both indifference zone and subset selection approaches are discussed along with some modifications and generalizations. Two examples are provided to illustate the use of subset selection and the indifference zone approaches. The paper concludes with the remark that selection and ranking methodology is a realistic approach in statistical analyses involving comparisons among two or more treatments.     

A novel bagging approach for variable ranking and selection via a mixed importance measure

At present, ensemble learning has exhibited its great power in stabilizing and enhancing the performance of some traditional variable selection methods such as lasso and genetic algorithm. In this paper, a novel bagging ensemble method called BSSW is developed to implement variable ranking and selection in linear regression models. Its main idea is to execute stepwise search algorithm on multiple bootstrap samples. In each trial, a mixed importance measure is assigned to each variable according to the order that it is selected into final model as well as the improvement of model fitting resulted from its inclusion. Based on the importance measure averaged across some bootstrapping trials, all candidate variables are ranked and then decided to be important or not. To extend the scope of application, BSSW is extended to the situation of generalized linear models. Experiments carried out with some simulated and real data indicate that BSSW achieves better performance in most studied cases when compared with several other existing methods.     

Equal probability of correct selection for bernoulli selection procedures

The problem of selecting the Bernoulli population which has the highest "success" probability is considered. It has been noted in several articles that the probability of a correct selection is the same, uniformly in the Bernoulli p-vector (P1,P2,….,Pk), for two or more different selection procedures. We give a general theorem which explains this phenomenon.

An application of particular interest arises when "strong" curtailment of a single-stage procedure (as introduced by Bechhofer and Kulkarni (1982a) )is employed; the corresponding result for "weak" curtailment of a single-stage procedure needs no proof. The use of strong curtailment in place of weak curtailment requires no more (and usually many less) observations to achieve the same.     

Bayesian subset selection approach to ranking normal means

In this, article we consider a Bayesian approach to the problem of ranking the means of normal distributed populations, which is a common problem in the biological sciences. We use a decision-theoretic approach with a straightforward loss function to determine a set of candidate rankings. This loss function allows the researcher to balance the risk of not including the correct ranking with the risk of increasing the number of rankings selected. We apply our new procedure to an example regarding the effect of zinc on the diversity of diatom species.     

On two-stage selection procedures and related probability-inequalities

In this paper we discuss a modification of the Dudewicz-Dalal procedure for the problem of selecting the population with the largest mean from k normal populations with unknown variances. We derive some inequalities and use them to lower-bound the probability of correct selection. These bounds are applied to the determination of the second-stage sample size which is required in order to achieve a prescribed probability of correct selection. We discuss the resulting procedure and compare it to that of Dudewicz and Dalai (1975).     

On selecting the treatment with the largest probability of survival

Suppose there are k(>= 2) treatments and each treatment is a Bernoulli process with binomial sampling. The problem of selecting a random-sized subset which contains the treatment with the largest survival probability (reliability or probability of success) is considered. Based on the ideas from both classical approaches and general Bayesian statistical decision approach, a new subset selection procedure is proposed to solve this kind of problem in both balanced and unbalanced designs. Comparing with the classical procedures, the proposed procedure has a significantly smaller selected subset. The optimal properties and performance of it were examined. The methods of selecting and fitting the priors and the results of Monte Carlo simulations on selected important cases are also studied.     

Inequalities for the partial sums of elliptical order statistics related to genetic selection

Let X₁,…,X_n be exchangeable normal variables with a common correlation p, and let X₍₁₎ > … > X_(n) denote their order statistics. The random variable σⁿ_i=_n—_k+1x_i, called the selection differential by geneticists, is of particular interest in genetic selection and related areas. In this paper we give results concerning a conjecture of Tong (1982) on the distribution of this random variable as a function of ρ. The same technique used can be applied to yield more general results for linear combinations of order statistics from elliptical distributions.     

A predictive approach for the selection of a fixed number of good treatments

This paper offers a predictive approach for the selection of a fixed number (= t) of treatments from k treatments with the goal of controlling for predictive losses. For the ith treatment, independent observations X _ij (j = 1,2,…,n) can be observed where X _ij ’s are normally distributed N(θ _i ; σ ²). The ranked values of θ _i ’s and X _i ’s are θ ₍₁₎ ≤ … ≤ θ _(k) and X _[1] ≤ … ≤ X _[k] and the selected subset S = {[k], [k? 1], … , [k ? t+1]} will be considered. This paper distinguishes between two types of loss functions. A type I loss function associated with a selected subset S is the loss in utility from the selector’s view point and is a function of θ _i with i ? S. A type II loss function associated with S measures the unfairness in the selection from candidates’ viewpoint and is a function of θ _i with i ? S. This paper shows that under mild assumptions on the loss functions S is optimal and provides the necessary formulae for choosing n so that the two types of loss can be controlled individually or simultaneously with a high probability. Predictive bounds for the losses are provided, Numerical examples support the usefulness of the predictive approach over the design of experiment approach.     

Distribution-free subset selection for incompletely ranked data

In market research and some other areas, it is common that a sample of n judges (consumers, evaluators, etc.) are asked to independently rank a series of k objects or candidates. It is usually difficult to obtain the judges' full cooperation to completely rank all k objects. A practical way to overcome this difficulty is to give each judge the freedom to choose the number of top candidates he is willing to rank. A frequently encountered question in this type of survey is how to select the best object or candidate from the incompletely ranked data. This paper proposes a subset selection procedure which constructs a random subset of all the k objects involved in the survey such that the best object is included in the subset with a prespecified confidence. It is shown that the proposed subset selection procedure is distribution-free over a very broad class of underlying distributions. An example from a market research study is used to illustrate the proposed procedure.     

On parallel policies for ranking and selection problems

In this paper we develop and test experimental methodologies for selection of the best alternative among a discrete number of available treatments. We consider a scenario where a researcher sequentially decides which treatments are assigned to experimental units. This problem is particularly challenging if a single measurement of the response to a treatment is time-consuming and there is a limited time for experimentation. This time can be decreased if it is possible to perform measurements in parallel. In this work we propose and discuss asynchronous extensions of two well-known Ranking & Selection policies, namely, Optimal Computing Budget Allocation (OCBA) and Knowledge Gradient (KG) policy. Our extensions (Asynchronous Optimal Computing Budget Allocation (AOCBA) and Asynchronous Knowledge Gradient (AKG), respectively) allow for parallel asynchronous allocation of measurements. Additionally, since the standard KG method is sequential (it can only allocate one experiment at a time) we propose a parallel synchronous extension of KG policy – Synchronous Knowledge Gradient (SKG). Computer simulations of our algorithms indicate that our parallel KG-based policies (AKG, SKG) outperform the standard OCBA method as well as AOCBA, if the number of evaluated alternatives is small or the computing/experimental budget is limited. For experimentations with large budgets and big sets of alternatives, both the OCBA and AOCBA policies are more efficient.     

A restricted subset selection procedure for Weibull populations

Consider k(k ≥ 2) two-parameter Weibull populations. We want to select a subset of the populations not exceeding m in size such that the subset contains at least ? of the t best populations. We have proposed a procedure which uses either the maximum likelihood estimators or ‘simplified’ linear estimators of the parameters. The estimators are based on type II censored data. The ranking of the populations is done by comparing their reliabilities at a certain fixed time. In selected cases the constants for the procedure are tabulated using Monte Carlo methods.     

Two new goals for selection based on proportions

Procedures are derived for selecting, with controlled probability of error, (1) a subset of populations which contains all populations better than a dual probability/proportion standard and (2) a subset of populations which both contains all populations better than an upper probability/proportion standard and also contains no populations worse than a lower probability/proportion standard. The procedures are motivated by current investigations in the area of computer performance evaluation.     

Bootstrap method in ranking and slippage problems

In this paper, the bootstrap method of Efron (1979) is given for a ranking and a slippage problem, where the ranking (or slippage) is with respect to the mean of the distributions. The method is also applied to obtain a confidence interval for the largest mean.     

Comments on ‘Application of an adaptive design to a randomized phase II selection trial in gastric cancer: a report of the study design’ by Satoshi Morita and Junichi Sakamoto. Pharmaceutical Statistics

In this commentary, we show that the treatment selection probabilities in Morita and Sakamoto [1] could be different if safety information is considered. Copyright © 2012 John Wiley & Sons, Ltd.     

Inference for probability of selection with dependently truncated data using a Cox model

A truncated sample consists of realizations of two variables L and T subject to the constraint L < T. One simple solution to dependently truncated data is to take L as a covariate of T in the Cox model. We aimed at studying the probability of selection, P(L < T), in this framework. We proposed the point estimator and derived its asymptotic distribution. Both truncated-only data and censored and truncated data were generated in the simulation study. The proposed point and variance estimators showed good performance in various simulated settings. The bone marrow transplant registry data were analyzed as the illustrative example.     

Classifying weibull populations with respect to a control

This paper is concerned with classifying k Weibull populations by their reliabilities with respect to a control population. He have proposed procedures which can be used for maximum likelihood estimators or simplified linear estimators of the parameters. Estimators are based on type II censored data. The cases considered include unknown shape parameters being equal or unequal. In selected cases, the constants needed for the procedures are tabulated using Monte Carlo methods.     

A bayesian approach to the comparison of two means

A hierarchical Bayesian approach to the problem of comparison of two means is considered. Hypothesis testing, ranking and selection, and estimation (after selection) are treated. Under the hypothesis that two means are different, it is desired to select the population which has the larger mean. Expressions for the ranking probability of each mean being the larger and the corresponding estimate of each mean are given. For certain priors, it is possible to express the quantities of interest in closed form. A simulation study has been done to compare mean square errors of a hierarchical Bayesian estimator and some of the existing estimators of the selected mean. The case of comparing two means in the presence of block effects has also been considered and an example is presented to illustrate the methodology.