LOPSIDED REASONING ON LOPSIDED TESTS AND MULTIPLE COMPARISONS

For those who have not recognized the disparate natures of tests of statistical hypotheses and tests of scientific hypotheses, one‐tailed statistical tests of null hypotheses such as ?≤ 0 or ?≥ 0 have often seemed a reasonable procedure. We earlier reviewed the many grounds for not regarding them as such. To have at least some power for detection of effects in the unpredicted direction, several authors have independently proposed the use of lopsided (also termed split‐tailed, directed or one‐and‐a‐half‐tailed) tests, two‐tailed tests with α partitioned unequally between the two tails of the test statistic distribution. We review the history of these proposals and conclude that lopsided tests are never justified. They are based on the same misunderstandings that have led to massive misuse of one‐tailed tests as well as to much needless worry, for more than half a century, over the various so‐called ‘multiplicity problems’. We discuss from a neo‐Fisherian point of view the undesirable properties of multiple comparison procedures based on either (i) maximum potential set‐wise (or family‐wise) type I error rates (SWERs), or (ii) the increasingly fashionable, maximum potential false discovery rates (FDRs). Neither the classical nor the newer multiple comparison procedures based on fixed maximum potential set‐wise error rates are helpful to the cogent analysis and interpretation of scientific data.     

Non parametric multiple comparisons of non nested rival models

The practice for testing homogeneity of several rival models is of interest. In this article, we consider a non parametric multiple test for non nested distributions in the context of the model selection. Based on the linear sign rank test, and the known union–intersection principle, we let the magnitude of the data to give a better performance to the test statistic. We consider the sample and the non nested rival models as blocks and treatments, respectively, and introduce the extended Friedman test version to compare with the results of the test based on the linear sign rank test. A real dataset based on the waiting time to earthquake is considered to illustrate the results.     

A statistical software procedure for exact parametric and nonparametric likelihood-ratio tests for two-sample comparisons

Two-sample comparisons belonging to basic class of statistical inference are extensively applied in practice. There is a rich statistical literature regarding different parametric methods to address these problems. In this context, most of the powerful techniques are assumed to be based on normally distributed populations. In practice, the alternative distributions of compared samples are commonly unknown. In this case, one can propose a combined test based on the following decision rules: (a) the likelihood-ratio test (LRT) for equality of two normal populations and (b) the Shapiro–Wilk (S-W) test for normality. The rules (a) and (b) can be merged by, e.g., using the Bonferroni correction technique to offer the correct comparison of the samples distribution. Alternatively, we propose the exact density-based empirical likelihood (DBEL) ratio test. We develop the tsc package as the first R package available to perform the two-sample comparisons using the exact test procedures: the LRT; the LRT combined with the S-W test; as well as the newly developed DBEL ratio test. We demonstrate Monte Carlo (MC) results and a real data example to show an efficiency and excellent applicability of the developed procedure.     

Comparing the performance of normality tests with ROC analysis and confidence intervals

There are several statistical hypothesis tests available for assessing normality assumptions, which is an a priori requirement for most parametric statistical procedures. The usual method for comparing the performances of normality tests is to use Monte Carlo simulations to obtain point estimates for the corresponding powers. The aim of this work is to improve the assessment of 9 normality hypothesis tests. For that purpose, random samples were drawn from several symmetric and asymmetric nonnormal distributions and Monte Carlo simulations were carried out to compute confidence intervals for the power achieved, for each distribution, by two of the most usual normality tests, Kolmogorov–Smirnov with Lilliefors correction and Shapiro–Wilk. In addition, the specificity was computed for each test, again resorting to Monte Carlo simulations, taking samples from standard normal distributions. The analysis was then additionally extended to the Anderson–Darling, Cramer-Von Mises, Pearson chi-square Shapiro–Francia, Jarque–Bera, D'Agostino and uncorrected Kolmogorov–Smirnov tests by determining confidence intervals for the areas under the receiver operating characteristic curves. Simulations were performed to this end, wherein for each sample from a nonnormal distribution an equal-sized sample was taken from a normal distribution. The Shapiro–Wilk test was seen to have the best global performance overall, though in some circumstances the Shapiro–Francia or the D'Agostino tests offered better results. The differences between the tests were not as clear for smaller sample sizes. Also to be noted, the SW and KS tests performed generally quite poorly in distinguishing between samples drawn from normal distributions and t Student distributions.     

Simulated Power Curves for some Location Tests

In this paper we compare the power properties of some location tests. The most widely used such test is Student's t. Recently bootstrap-based tests have received much attention in the literature. A bootstrap version of the t-test will be included in our comparison. Finally, the nonparametric tests based on the idea of permuting the signs will be represented in our comparison. Again, we will initially concentrate on a version of that test based on the mean. The permutation tests predate the bootstrap by about fourty years. Theoretical results of Pitman (1937) and Bickel & Freedman (1981) show that these three methods are asymptotically equivalent if the underlying distribution is symmetric and has finite second moment. In the modern literature, the use of the nonparametric techniques is advocated on the grounds that the size of the test would be either exact, or more nearly exact. In this paper we report on a simulation study that compares the power curves and we show that it is not necessary to use resampling tests with a statistic based on the mean of the sample.     

Non-parametric tests for several sample location problem based on subsample extrema

In this paper non-parametric tests for homogeneity of several populations against locationtype alternatives are proposed. For this all possible subsamples of fixed size are drawn from each sample and their maxima and minima are computed One class of tests is obtained using these subsample minima whereas other class of tests involves use of sub sample maxima. Tests belonging t o these two classes have been compared with many of the presently available tests in terms of their Pitman asymptotic relative efficiency . Some of the members of these proposed classes of tests prove to robust in terms of efficiency.     

Exact tests and confidence regions in nonlinear regression

Nonlinear regression models with spherically symmetric error vectors and a single nonlinear parameter are considered. On the basis of a new geometric approach, exact one- and two-sided tests and confidence regions for the nonlinear parameter are derived in the cases of known and unknown error variances. A geometric measure representation formula is used to determine the power functions of the tests if the error variance is known and to derive different lower bounds for the power function of a one-sided test in the case of an unknown error variance. The latter can be done quite effectively by constructing and measuring several balls inside the critical region. A numerical study compares the results for different density generating functions of the error distribution.     

A modified approximate method for analysis of degradation data

Estimation of the lifetime distribution of industrial components and systems yields very important information for manufacturers and consumers. However, obtaining reliability data is time consuming and costly. In this context, degradation tests are a useful alternative approach to lifetime and accelerated life tests in reliability studies. The approximate method is one of the most used techniques for degradation data analysis. It is very simple to understand and easy to implement numerically in any statistical software package. This paper uses time series techniques in order to propose a modified approximate method (MAM). The MAM improves the standard one in two aspects: (1) it uses previous observations in the degradation path as a Markov process for future prediction and (2) it is not necessary to specify a parametric form for the degradation path. Characteristics of interest such as mean or median time to failure and percentiles, among others, are obtained by using the modified method. A simulation study is performed in order to show the improved properties of the modified method over the standard one. Both methods are also used to estimate the failure time distribution of the fatigue-crack-growth data set.     

Linear Signed Rank Test for Model Selection

In this article, we consider a linear signed rank test for non-nested distributions in the context of the model selection. Introducing a new test, we show that, it is asymptotically more efficient than the Vuong test and the test statistic based on B statistic introduced by Clarke. However, here, we let the magnitude of the data give a better performance to the test statistic. We have shown that this test is an unbiased one. The results of simulations show that the rank test has the greater statistical power than the Vuong test where the underline distributions is symmetric.     

Simple pilot procedures for the avoidance of disconnected experimental designs

Summary.  Two simple pilot procedures are proposed for avoiding the problem of dealing with a disconnected experimental design. Both procedures should be carried out on the selected design before any experimentation is considered. The first procedure is a check that the suggested design is connected with respect to treatments. This makes use of the information matrix for the model and provides feed-back on a disconnected design. The second procedure specifies which observations are influential in causing a connected design to become disconnected, with respect to any set of parameter effects, if these observations are lost during the experimental period. This specification is found by examining the projection matrix for the model. These pilot procedures are illustrated by several examples.