Relation of modified power series distributions to lagrangian probability distributions

The class of Lagrangian probability distributions ‘LPD’, given by the expansion of a probability generating function f‘t’ under the transformation u = t/g‘t’ where g ‘t’ is also a p.g.f., has been substantially widened by removing the restriction that the defining functions g ‘t’ and f‘t’ be probability generating functions. The class of modified power series distributions defined by Gupta ‘1974’ has been shown to be a sub-class of the wider class of LPDs     

Ultimate Extinction of the Promiscuous Bisexual Galton—Watson Metapopulation

A variant of a sexual Gallon–Watson process is considered. At each generation the population is partitioned among n‘hosts’ (population patches) and individual members mate at random only with others within the same host. This is appropriate for many macroparasite systems, and at low parasite loads it gives rise to a depressed rate of reproduction relative to an asexual system, due to the possibility that females are unmated. It is shown that stochasticity mitigates against this effect, so that for small initial populations the probability of ultimate extinction (the complement of an ‘epidemic’) displays a tradeoff as a function of n between the strength of fluctuations which overcome this ‘mating’ probability, and the probability of the subpopulation in one host being ‘rescued’ by that in another. Complementary approximations are developed for the extinction probability: an asymptotically exact approximation at large n, and for small n a short‐time probability that is exact in the limit where the mean number of offspring per parent is large.     

Probability models of chance fluctuations in spectra of astronomical sources with applications to X-ray absorption lines

The search for faint emission or absorption lines in astronomical spectra has received considerable attention in recent years, especially in the X-ray wavelength range. These features usually appear as a deficit or excess of counts in a single-resolution element of the detector, and as such they are referred to as unresolved fluctuations. The general problem under investigation is the probability of occurrence of chance fluctuations. A quantitative answer is necessary to determine whether detected fluctuations are a real (astronomical, in this case) signal, or if they can be attributed to chance. This application note provides a new comprehensive method to answer this question as function of the instrument's resolution, the wavelength coverage of the spectrum, the number of fluctuations of interest, and the confidence level chosen. The method is based on the binomial distribution and addresses also the probability of multiple simultaneous fluctuations. A critical aspect of the model is the a priori knowledge of the location of possible fluctuations, which significantly affects the probability of detection. In fact, a wider wavelength range for the ‘blind’ search of possible fluctuations results in a larger number of ‘tries’ for the detection of a fluctuation, lowering the overall significance of a specific feature. The method is illustrated with a case study and examples using X-ray data.     

Learning causal structure from mixed data with missing values using Gaussian copula models

We consider the problem of causal structure learning from data with missing values, assumed to be drawn from a Gaussian copula model. First, we extend the ‘Rank PC’ algorithm, designed for Gaussian copula models with purely continuous data (so-called nonparanormal models), to incomplete data by applying rank correlation to pairwise complete observations and replacing the sample size with an effective sample size in the conditional independence tests to account for the information loss from missing values. When the data are missing completely at random (MCAR), we provide an error bound on the accuracy of ‘Rank PC’ and show its high-dimensional consistency. However, when the data are missing at random (MAR), ‘Rank PC’ fails dramatically. Therefore, we propose a Gibbs sampling procedure to draw correlation matrix samples from mixed data that still works correctly under MAR. These samples are translated into an average correlation matrix and an effective sample size, resulting in the ‘Copula PC’ algorithm for incomplete data. Simulation study shows that: (1) ‘Copula PC’ estimates a more accurate correlation matrix and causal structure than ‘Rank PC’ under MCAR and, even more so, under MAR and (2) the usage of the effective sample size significantly improves the performance of ‘Rank PC’ and ‘Copula PC.’ We illustrate our methods on two real-world datasets: riboflavin production data and chronic fatigue syndrome data.

     

Cytometry inference through adaptive atomic deconvolution

In this paper, we consider a statistical estimation problem known as atomic deconvolution. Introduced in reliability, this model has a direct application when considering biological data produced by flow cytometers. From a statistical point of view, we aim at inferring the percentage of cells expressing the selected molecule and the probability distribution function associated with its fluorescence emission. We propose here an adaptive estimation procedure based on a previous deconvolution procedure introduced by Es, Gugushvili, and Spreij [(2008), ‘Deconvolution for an atomic distribution’, Electronic Journal of Statistics, 2, 265–297] and Gugushvili, Es, and Spreij [(2011), ‘Deconvolution for an atomic distribution: rates of convergence’, Journal of Nonparametric Statistics, 23, 1003–1029]. For both estimating the mixing parameter and the mixing density automatically, we use the Lepskii method based on the optimal choice of a bandwidth using a bias-variance decomposition. We then derive some convergence rates that are shown to be minimax optimal (up to some log terms) in Sobolev classes. Finally, we apply our algorithm on the simulated and real biological data.     

Image estimators based on marked bins

The problem of approximating an ‘image’ S?? ^d from a random sample of points is considered. If S is included in a grid of square bins, a plausible estimator of S is defined as the union of the ‘marked’ bins (those containing a sample point). We obtain convergence rates for this estimator and study its performance in the approximation of the border of S. The practical aspects of implementation are discussed, including some technical improvements on the estimator, whose performance is checked through a real data example.     

A Bayesian analysis for the Wilcoxon signed-rank statistic

A Bayesian analysis is provided for the Wilcoxon signed-rank statistic (T⁺). The Bayesian analysis is based on a sign-bias parameter φ on the (0, 1) interval. For the case of a uniform prior probability distribution for φ and for small sample sizes (i.e., 6 ? n ? 25), values for the statistic T⁺ are computed that enable probabilistic statements about φ. For larger sample sizes, approximations are provided for the asymptotic likelihood function P(T⁺|φ) as well as for the posterior distribution P(φ|T⁺). Power analyses are examined both for properly specified Gaussian sampling and for misspecified non Gaussian models. The new Bayesian metric has high power efficiency in the range of 0.9–1 relative to a standard t test when there is Gaussian sampling. But if the sampling is from an unknown and misspecified distribution, then the new statistic still has high power; in some cases, the power can be higher than the t test (especially for probability mixtures and heavy-tailed distributions). The new Bayesian analysis is thus a useful and robust method for applications where the usual parametric assumptions are questionable. These properties further enable a way to do a generic Bayesian analysis for many non Gaussian distributions that currently lack a formal Bayesian model.     

On the estimation of extropy

Recently, Lad, Sanfilippo, and Agro [(2015), ‘Extropy: Complementary Dual of Entropy’, Statistical Science, 30, 40–58.] showed the measure of entropy has a complementary dual, which is termed extropy. The present article introduces some estimators of the extropy of a continuous random variable. Properties of the proposed estimators are stated, and comparisons are made with Qiu and Jia’s estimators [(2018a), ‘Extropy Estimators with Applications in Testing uniformity’, Journal of Nonparametric Statistics, 30, 182–196]. The results indicate that the proposed estimators have a smaller mean squared error than competing estimators. A real example is presented and analysed.     

A new family of generalized distributions

Kumaraswamy [Generalized probability density-function for double-bounded random-processes, J. Hydrol. 462 (1980), pp. 79–88] introduced a distribution for double-bounded random processes with hydrological applications. For the first time, based on this distribution, we describe a new family of generalized distributions (denoted with the prefix ‘Kw’) to extend the normal, Weibull, gamma, Gumbel, inverse Gaussian distributions, among several well-known distributions. Some special distributions in the new family such as the Kw-normal, Kw-Weibull, Kw-gamma, Kw-Gumbel and Kw-inverse Gaussian distribution are discussed. We express the ordinary moments of any Kw generalized distribution as linear functions of probability weighted moments (PWMs) of the parent distribution. We also obtain the ordinary moments of order statistics as functions of PWMs of the baseline distribution. We use the method of maximum likelihood to fit the distributions in the new class and illustrate the potentiality of the new model with an application to real data.     

A sequential design for maximizing the probability of a favourable response

Consider the situation in which subjects arrive sequentially for a treatment and in which there are two distinct ways in which the treatment may fail. Treatments are given at different dosages, and the probabilities of the two failure types vary with dose. Assuming that decreasing the chances of one failure type increases the chances of the other, we say the failures oppose each other. Also assume that one failure type is primary in that, if it occurs, it censors the trial, so that observation of the secondary failure type is contingent on no failure of the primary type. We are interested in designs that provide information about the dose that maximizes the probability of success, i.e., the optimal dose, while treating very few subjects at dosages that have high risks of failure. Assuming that dosages belong to a discrete set, we show that a randomized version of the Pólya urn scheme causes dose selection to be progressively biased so as to favour those doses that produce success with higher probability.     

A modified LGB method for detecting significant effects based on a half-normal probability plot

In analyzing data from unreplicated factorial designs, the half-normal probability plot is commonly used to screen for the ‘vital few’ effects. Recently, many formal methods have been proposed to overcome the subjectivity of this plot. Lawson (1998) (hereafter denoted as LGB) suggested a hybrid method based on the half-normal probability plot, which is a blend of Lenth (1989) and Loh (1992) method. The method consists of fitting a simple least squares line to the inliers, which are determined by the Lenth method. The effects exceeding the prediction limits based on the fitted line are candidates for the vital few effects. To improve the accuracy of partitioning the effects into inliers and outliers, we propose a modified LGB method (hereafter denoted as the Mod_LGB method), in which more outliers can be classified by using both the Carling’s modification of the box plot (Carling, 2000) and Lenth method. If no outlier exists or there is a wide range in the inliers as determined by the Lenth method, more outliers can be found by the Carling method. A simulation study is conducted in unreplicated $2^4$ designs with the number of active effects ranging from 1 to 6 to compare the efficiency of the Lenth method, original LGB methods, and the proposed modified version of the LGB method.     

A Note on Dirichlet One-Armed Bandits

One of the two independent stochastic processes (or ‘arms’) is selected and observed sequentially at each of n(≤ ∝) stages. Arm 1 yields observations identically distributed with unknown probability measure P with a Dirichlet process prior whereas observations from arm 2 have known probability measure Q. Future observations are discounted and at stage m, the payoff is a _m(≥0) times the observation Z _m at that stage. The objective is to maximize the total expected payoff. Clayton and Berry (1985) consider this problem when a _m equals 1 for m ≤ n and 0 for m > n(< ∝) In this paper, the Clayton and Berry (1985) results are extended to the case of regular discount sequences of horizon n, which may also be infinite. The results are illustrated with numerical examples. In case of geometric discounting, the results apply to a bandit with many independent unknown Dirichlet arms.     

Some new factor screening designs using the search linear model

Factor screening designs for searching two and three effective factors using the search linear model are discussed. The construction of such factor screening designs involved finding a fraction with small number of treatments of a 2^m factorial experiment having the property P_2t (no 2t columns are linearly dependent) for t=2 and 3. A ‘Packing Problem’ is introduced in this connection. A complete solution of the problem in one case and partial solutions for the other cases are presented. Many practically useful new designs are listed.     

ESTIMATION OF THE CONDITIONAL PROBABILITY OF TUBERCULOSIS GIVEN THE RESPONSE OF A MANTOUX TEST

For the planning of community tuberculosis, prophylaxis one must know the prevalence of tuberculosis (TB) infection as a function of response to the Mantoux intradermal tuberculin test; the standard test for indicating Mycobacterium TB infection. The skin induration size used to select individuals for prophylaxis must be chosen carefully, in view of the costs associated with carrying out and supervising such prophylaxis. The Mantoux test was used to obtain measurements on adolescents in metropolitan Victoria and on a small sample of adolescents with clinical TB. These data are employed to obtain estimates and to construct upper confidence bounds for the conditional probability of TB infection, given the level of Mantoux response. Two conservative methods are presented; one is ‘nonparametric’, the other ‘semiparametric’. The analyses indicate that for responses up to and including 13 mm, the probability of TB infection is less than.07 with ninety-five percent confidence.     

A series of single array 2m factorial search designs for even m

By means of a search design one is able to search for and estimate a small set of non‐zero elements from the set of higher order factorial interactions in addition to estimating the lower order factorial effects. One may be interested in estimating the general mean and main effects, in addition to searching for and estimating a non‐negligible effect in the set of 2‐ and 3‐factor interactions, assuming 4‐ and higher‐order interactions are all zero. Such a search design is called a ‘main effect plus one plan’ and is denoted by MEP.1. Construction of such a plan, for 2^m factorial experiments, has been considered and developed by several authors and leads to MEP.1 plans for an odd number m of factors. These designs are generally determined by two arrays, one specifying a main effect plan and the other specifying a follow‐up. In this paper we develop the construction of search designs for an even number of factors m, m≠6. The new series of MEP.1 plans is a set of single array designs with a well structured form. Such a structure allows for flexibility in arriving at an appropriate design with optimum properties for search and estimation.     

Nonparametric Estimation of the Average Availability

The average availability of a repairable system is the expected proportion of time that the system is operating in the interval [0, t]. The present article discusses the nonparametric estimation of the average availability when (i) the data on ‘n’ complete cycles of system operation are available, (ii) the data are subject to right censorship, and (iii) the process is observed upto a specified time ‘T’. In each case, a nonparametric confidence interval for the average availability is also constructed. Simulations are conducted to assess the performance of the estimators.     

Towards Enhancement of the Economy of a Thermal Power Generating System through Prediction of Plant Efficiency

The plant ‘Heat Rate’ (HR) is a measure of overall efficiency of a thermal power generating system. It depends on a large number of factors, some of which are non-measurable, while data relating to others are seldom available and recorded. However, coal quality (expressed in terms of ‘effective heat value’ (EHV) as kcal/kg) transpires to be one of the important factors that influences HR values and data on EHV are available in any thermal power generating system. In the present work, we propose a prediction interval of the HR values on the basis of only EHV, keeping in mind that coal quality is one of the important (but not the only) factors that have a pronounced effect on the combustion process and hence on HR. The underlying theory borrows the idea of providing simultaneous confidence interval (SCI) to the coefficients of a p-th p(≥1) order autoregressive model (AR(p)). The theory has been substantiated with the help of real life data from a power utility (after suitable base and scale transformation of the data to maintain the confidentiality of the classified document). Scope for formulating strategies to enhance the economy of a thermal power generating system has also been explored.     

On improving convergence rate of Bernstein polynomial density estimator

This paper is concerned with the Bernstein estimator [Vitale, R.A. (1975), ‘A Bernstein Polynomial Approach to Density Function Estimation’, in Statistical Inference and Related Topics, ed. M.L. Puri, 2, New York: Academic Press, pp. 87–99] to estimate a density with support [0, 1]. One of the major contributions of this paper is an application of a multiplicative bias correction [Terrell, G.R., and Scott, D.W. (1980), ‘On Improving Convergence Rates for Nonnegative Kernel Density Estimators’, The Annals of Statistics, 8, 1160–1163], which was originally developed for the standard kernel estimator. Moreover, the renormalised multiplicative bias corrected Bernstein estimator is studied rigorously. The mean squared error (MSE) in the interior and mean integrated squared error of the resulting bias corrected Bernstein estimators as well as the additive bias corrected Bernstein estimator [Leblanc, A. (2010), ‘A Bias-reduced Approach to Density Estimation Using Bernstein Polynomials’, Journal of Nonparametric Statistics, 22, 459–475] are shown to be O(n^?8/9) when the underlying density has a fourth-order derivative, where n is the sample size. The condition under which the MSE near the boundary is O(n^?8/9) is also discussed. Finally, numerical studies based on both simulated and real data sets are presented.     

Analyzing Hybrid Randomized Response Data with a Binomial Selection Procedure

The operating characteristics (OCs) of an indifference-zone ranking and selection procedure are derived for randomized response binomial data. The OCs include tables and figures to facilitate tradeoffs between sample size and a stated probability of a correct selection, i.e., correctly identifying the binomial population (out of k ≥ 2) characterized by the largest probability of success. Measures of efficiency are provided to assist the analyst in selection of an appropriate randomized response design for the collection of the data. A hybrid randomized response model, which includes the Warner model and the Greenberg et al. model, is introduced to facilitate comparisons among a wider range of statistical designs than previously available. An example comparing failure rates of contraceptive methods is used to illustrate the use of these new results.     

A Nonrandomized,Nonconservative Version of the Fisher Exact Test

The Fisher exact test has been unjustly dismissed by some as ‘only conditional,’ whereas it is unconditionally the uniform most powerful test among all unbiased tests, tests of size α and with power greater than its nominal level of significance α. The problem with this truly optimal test is that it requires randomization at the critical value(s) to be of size α. Obviously, in practice, one does not want to conclude that ‘with probability x the we have a statistical significant result.’ Usually, the hypothesis is rejected only if the test statistic's outcome is more extreme than the critical value, reducing the actual size considerably.

The randomized unconditional Fisher exact is constructed (using Neyman–structure arguments) by deriving a conditional randomized test randomizing at critical values c(t) by probabilities γ(t), that both depend on the total number of successes T (the complete-sufficient statistic for the nuisance parameter—the common success probability) conditioned upon.

In this paper, the Fisher exact is approximated by deriving nonrandomized conditional tests with critical region including the critical value only if γ (t) > γ₀, for a fixed threshold value γ₀, such that the size of the unconditional modified test is for all value of the nuisance parameter—the common success probability—smaller, but as close as possible to α. It will be seen that this greatly improves the size of the test as compared with the conservative nonrandomized Fisher exact test.

Size, power, and p value comparison with the (virtual) randomized Fisher exact test, and the conservative nonrandomized Fisher exact, Pearson's chi-square test, with the more competitive mid-p value, the McDonald's modification, and Boschloo's modifications are performed under the assumption of two binomial samples.