Discriminating between distributions using feed-forward neural networks

A relevant problem in many applicatory contexts is to test whether some given observations follow one of two possible probability distributions. The vast literature produced over the years on this topic does not identify a tool which can be easily adopted to any situation but only finds solutions to specific comparisons. Recently, an easy to implement procedure for discrimination between two distributions based on feed-forward neural networks has been proposed giving interesting results. In this work this procedure is further investigated in terms of power, neural network architecture and expected statistical properties of the test statistic for small, moderate and large sample sizes, in a wide range of symmetric and skewed alternatives.     

A method for constructing exact tests from test statistics that have unknown null distributions

A randomized procedure is described for constructing an exact test from a test statistic F for which the null distribution is unknown. The procedure is restricted to cases where F is a function of a random element U that has a known distribution under the null hypothesis. The power of the exact randomized test is shown to be greater in some cases than the power of the exact nonrandomized test that could be constructed if the null distribution of Fwere known.     

Assessing the fit of finite mixture distributions

Mixture distributions have become a very flexible and common class of distributions, used in many different applications, but hardly any literature can be found on tests for assessing their goodness of fit. We propose two types of smooth tests of goodness of fit for mixture distributions. The first test is a genuine smooth test, and the second test makes explicit use of the mixture structure. In a simulation study the tests are compared to some traditional goodness of fit tests that, however, are not customised for mixture distributions. The first smooth test has overall good power and generally outperforms the other tests. The second smooth test is particularly suitable for assessing the fit of each component distribution separately. The tests are applicable to both continuous and discrete distributions and they are illustrated on three medical data sets.     

Portmanteau tests based on quadratic forms in the autocorrelations

Many white noise and goodness-of-fit tests are (asymptotically) written as quadratic forms in the ordinary autocorrelation estimates. The properties of such tests are studied by investigating the structure of the matrix of the quadratic form. We suggest to choose the matrix of the quadratic form in such a way that the power is maximized according to the information available about the alternative hypothesis. A simulation study sheds some light on the behavior of the test in finite samples. It is generally found more powerful than the most popular portmanteau tests, i.e., the Box and Pierce and the Ljung and Box tests.     

Deriving a lower bound for the proportion of perceivers in replicated difference tests by means of multiple test theory

Analyzing repeated difference tests aims in significance testing for differences as well as in estimating the mean discrimination ability of the consumers. In addition to the average success probability, the proportion of consumers that may detect the difference between two products and therefore account for any increase of this probability is of interest. While some authors address the first two goals, for the latter one only an estimator directly linked to the average probability seems to be used. However, this may lead to unreasonable results. Therefore we propose a new approach based on multiple test theory. We define a suitable set of hypotheses that is closed under intersection. From this, we derive a series of hypotheses that may be sequentially tested while the overall significance level will not be violated. By means of this procedure we may determine a minimal number of assessors that must have perceived the difference between the products at least once in a while. From this, we can find a conservative lower bound for the proportion of perceivers within the consumers. In several examples, we give some insight into the properties of this new method and show that the knowledge about this lower bound might indeed be valuable for the investigator. Finally, an adaption of this approach for similarity tests will be proposed.     

Adaptive choice of scale tests in flexible two-stage designs with applications in experimental ecology and clinical trials

In this paper, the two-sample scale problem is addressed within the rank framework which does not require to specify the underlying continuous distribution. However, since the power of a rank test depends on the underlying distribution, it would be very useful for the researcher to have some information on it in order to use the possibly most suitable test. A two-stage adaptive design is used with adaptive tests where the data from the first stage are used to compute a selector statistic to select the test statistic for stage 2. More precisely, an adaptive scale test due to Hall and Padmanabhan and its components are considered in one-stage and several adaptive and non-adaptive two-stage procedures. A simulation study shows that the two-stage test with the adaptive choice in the second stage and with Liptak combination, when it is not more powerful than the corresponding one-stage test, shows, however, a quite similar power behavior. The test procedures are illustrated using two ecological applications and a clinical trial.     

An alternative method for global and partial comparison of two diagnostic systems based on ROC curves

In this paper, an alternative method for the comparison of two diagnostic systems based on receiver operating characteristic (ROC) curves is presented. ROC curve analysis is often used as a statistical tool for the evaluation of diagnostic systems. However, in general, the comparison of ROC curves is not straightforward, in particular, when they cross each other. A similar difficulty is also observed in the multi-objective optimization field where sets of solutions defining fronts must be compared with a multi-dimensional space. Thus, the proposed methodology is based on a procedure used to compare the performance of distinct multi-objective optimization algorithms. In general, methods based on the area under the ROC curves are not sensitive to the existence of crossing points between the curves. The new approach can deal with this situation and also allows the comparison of partial portions of ROC curves according to particular values of sensitivity and specificity of practical interest. Simulations results are presented. For illustration purposes, considering real data from newborns with very low birthweight, the new method was applied in order to discriminate the better index for evaluating the risk of death.     

New invariant and consistent chi-squared type goodness-of-fit tests for multivariate normality and a related comparative simulation study

ABSTRACT

New invariant and consistent goodness-of-fit tests for multivariate normality are introduced. Tests are based on the Karhunen–Loève transformation of a multidimensional sample from a population. A comparison of simulated powers of tests and other well-known tests with respect to some alternatives is given. The simulation study demonstrates that power of the proposed McCull test almost does not depend on the number of grouping cells. The test shows an advantage over other chi-squared type tests. However, averaged over all of the simulated conditions examined in this article, the Anderson–Darling type and the Cramer–von Mises type tests seem to be the best.     

Permuiation tesis for k-sample binomial data with comparisons of exact and approximate P-levels

Exact ksample permutation tests for binary data for three commonly encountered hypotheses tests are presented,, The tests are derived both under the population and randomization models . The generating function for the number of cases in the null distribution is obtained, The asymptotic distributions of the test statistics are derived . Actual significance levels are computed for the asymptotic test versions , Random sampling of the null distribution is suggested as a superior alternative to the asymptotics and an efficient computer technique for implementing the random sampling is described., finally, some numerical examples are presented and sample size guidelines given for computer implementation of the exact tests.     

Simulated Power Curves for some Location Tests

In this paper we compare the power properties of some location tests. The most widely used such test is Student's t. Recently bootstrap-based tests have received much attention in the literature. A bootstrap version of the t-test will be included in our comparison. Finally, the nonparametric tests based on the idea of permuting the signs will be represented in our comparison. Again, we will initially concentrate on a version of that test based on the mean. The permutation tests predate the bootstrap by about fourty years. Theoretical results of Pitman (1937) and Bickel & Freedman (1981) show that these three methods are asymptotically equivalent if the underlying distribution is symmetric and has finite second moment. In the modern literature, the use of the nonparametric techniques is advocated on the grounds that the size of the test would be either exact, or more nearly exact. In this paper we report on a simulation study that compares the power curves and we show that it is not necessary to use resampling tests with a statistic based on the mean of the sample.     

On the sample size of curtailed tests

Formulas are given for the asymptotic distribution, mean, and variance of m^-1N_m,where NN_m is the random sample size of the curtailed version of a fixed-sample most powerful test based on sample size m. The adequacy of the formulas is numerically investigated in some important applications where exact formulas can also be derived     

More powerful permutation test based on multistage ranked set sampling

Many researches have used ranked set sampling (RSS) method instead of simple random sampling (SRS) to improve power of some nonparametric tests. In this study, the two-sample permutation test within multistage ranked set sampling (MSRSS) is proposed and investigated. The power of this test is compared with the SRS permutation test for some symmetric and asymmetric distributions through Monte Carlo simulations. It has been found that this test is more powerful than the SRS permutation test; its power increased by set size and/or number of cycles and/or number of stages. Symmetric distributions power increased better than asymmetric distributions power.     

Constructing optimal univariate tests

Lehmann (1959) derives various univariate optimal tests and gives a technique that can be used to construct some of them. Although the technique is correct, the proof is not. Here the proof is corrected and an alternative derivation given. The alternative derivation is generalised to situations not covered by the Lehmann technique.     

Updating input–output matrices: assessing alternatives through simulation

A problem that frequently arises in economics, demography, statistics, transportation planning and stochastic modelling is how to adjust the entries of a matrix to fulfil row and column aggregation constraints. Biproportional methods in general and the so-called RAS algorithm in particular, have been used for decades to find solutions to this type of problem. Although alternatives exist, the RAS algorithm and its extensions are still the most popular. Apart from some interesting empirical and theoretical properties, tradition, simplicity and very low computational costs are among the reasons behind the great success of RAS. Nowadays computer hardware and software have made alternative procedures equally attractive. This work analyses, through simulation, the performance of RAS and some minimands when matrix coefficients vary following different schemes of change. Results suggest RAS algorithm as the best option when variations in coefficients are proportional to their size, while the method based on minimizing squared differences is seen to be the best alternative when the standard deviations of variations are either constant, variable, or an inverse function of matrix entries.     

THE POWER AND SIZE OF NONPARAMETRIC TESTS FOR COMMON DISTRIBUTIONAL CHARACTERISTICS

This paper considers the power and size properties of some well known nonparametric linear rank tests for location and scale as well as the Kolmogorov-Smirnov omnibus test and proposed alternatives to it. Independence between some classes of linear rank tests is established facilitating their joint application. Monte Carlo study confirms the asymptotic power properties of the linear rank tests but raises concerns about their application in more general and practically relevant circumstances. It also indicates that the new omnibus tests constitute viable alternatives with superior properties to the Kolmogorov-Smirnov test in certain circumstances.     

Analysis of ordered categorical data with score averaging: with applications to case-control genetic associations

The trend test is often used for the analysis of 2×K ordered categorical data, in which K pre-specified increasing scores are used. There have been discussions on how to assign these scores and the impact of the outcomes on different scores. The scores are often assigned based on the data-generating model. When this model is unknown, using the trend test is not robust. We discuss the weighted average of a trend test over all scientifically plausible choices of scores or models. This approach is more computationally efficient than a commonly used robust test MAX when K is large. Our discussion is for any ordered 2×K table, but simulation and applications to real data are focused on case-control genetic association studies. Although there is no single test optimal for all choices of scores, our numerical results show that some score averaging tests can achieve the performance of MAX.     

Dependency and truncated forms of combinations in multivariate combination-based permutation tests and ordered categorical variables

ABSTRACT

Quite an important problem usually occurs in several multi-dimensional hypotheses testing problems when variables are correlated. In this framework the non-parametric combination (NPC) of a finite number of dependent permutation tests is suitable to cover almost all real situations of practical interest since the dependence relations among partial tests are implicitly captured by the combining procedure itself without the need to specify them [Pesarin F, Salmaso L. Permutation tests for complex data: theory, applications and software. Chichester: Wiley; 2010a]. An open problem related to NPC-based tests is the impact of the dependency structure on combined tests, especially in the presence of categorical variables. This paper’s goal is firstly to investigate the impact of the dependency structure on the possible significance of combined tests in cases of ordered categorical responses using Monte Carlo simulations, then to propose some specific procedures aimed at improving the power of multivariate combination-based permutation tests. The results show that an increasing level of correlation/association among responses negatively affects the power of combination-based multivariate permutation tests. The application of special forms of combination functions based on the truncated product method [Zaykin DV, Zhivotovsky LA, Westfall PH, Weir BS. Truncated product method for combining p-values. Genet Epidemiol. 2002;22:170–185; Dudbridge F, Koeleman BPC. Rank truncated product of p-values, with application to genomewide association scans. Genet Epidemiol. 2003;25:360–366] or on Liptak combination allowed us, using Monte Carlo simulations, to demonstrate the possibility of mitigating the negative effect on power of combination-based multivariate permutation tests produced by an increasing level of correlation/association among responses.     

Empirical behavior of some tests for normality

The behavior of a range of tests assessing the normality of a sequence of independent and identically distributed random variables is investigated.An examination of the empirical significance level of the tests is undertaken for different sample sizes. The empirical power associated with these tests is also calculated under some alternative distributions.     

Accurate detection of excess over a standard based on a random sample

A known number N of packages each contain, in differing unknown amounts, both substances of no particular import and some substance of interest, the total weight of the latter substance for all N of the packages being an unknown quantity T. Based on the amounts of the substance of interest found in each of n (n ≦= N) randomly sampled packages one is to decide, with a very small probability of the error of wrongly deciding that T exceeds L, whether or not the quantity T exceeds a given amount L. An optimal way of doing this is presented in which the probability of the error of wrongly deciding that T exceeds L can be precisely bounded above as desired.     

On Multi-Sided Permutation Tests

In fields such as clinical trials, the environment, epidemiology, genetics, pharmacology, social experiments, etc., situations in which the effect of a treatment can be positive on some individuals and negative on the rest may sometimes occur. In a two-sample design, this situation can be expressed by a response model where a random effect Δ in the alternative is such that , and . Such a situation is essentially different from the traditional two-sided test, in which the alternative is assumed to be active only on one of two directions. We consider alternatives in which two sub-alternatives (Δ < 0) and (Δ > 0) can be jointly active. In order to deal with such an atypical situation, we suggest to firstly apply two goodness-of-fit tests, one for the positive deviations from the empirical distribution function of control sample and the other for the negative, and then proceed with their nonparametric combination within a permutation framework. A simulation study inspects on properties of such a solution.