On the use of the t-statistic in preliminary testing procedures for the Behrens-Fisher problem

Preliminary testing procedures for the two means problem traditionally employ the pooled variance t-statistic. In this paper we show that bias of the t-statistic under conditions of heterogeneity of variance may be increased if use of the t-statistic is conditional on an affirmative F-test. For this reason we conclude that use of the t-statistic in preliminary testing procedures is inappropriate.     

Bayesian hypothesis testing for selected regression coefficients

ABSTRACT

We develop new Bayesian regression tests for prespecified regression coefficients. Simple, closed forms of the Bayes factors are derived that depend only on the regression t-statistic and F-statistic and the usual associated t and F distributions. The priors that allow those forms are simple and also meaningful, requiring minimal but practically important subjective inputs.     

On a multiple endpoints testing problem

In a clinical trial comparing drug with placebo, where there are multiple primary endpoints, we consider testing problems where an efficacious drug effect can be claimed only if statistical significance is demonstrated at the nominal level for all endpoints. Under the assumption that the data are multivariate normal, the multiple endpoint-testing problem is formulated. The usual testing procedure involves testing each endpoint separately at the same significance level using two-sample t-tests, and claiming drug efficacy only if each t-statistic is significant. In this paper we investigate properties of this procedure. We show that it is identical to both an intersection union test and the likelihood ratio test. A simple expression for the p-value is given. The level and power function are studied; it is shown that the test may be conservative and that it is biased. Computable bounds for the power function are established.     

Confidence intervals for the ratio of two variances

In this paper we consider confidence intervals for the ratio of two population variances. We propose a confidence interval for the ratio of two variances based on the t-statistic by deriving its Edgeworth expansion and considering Hall's and Johnson's transformations. Then, we consider the coverage accuracy of suggested intervals and intervals based on the F-statistic for some distributions.     

The limiting distribution of the F-statistic from nonnormal universes

We consider a linear regression model with an unbalanced 1-fold nested error structure, where group effect and error are from nonnormal universes. The limiting distribution of the F-statistic in this model is derived, as the sample size is large and group sizes take values from a finite set of distinct integers. The result is used to approximate the F-distribution quantile and to test the significance of the random effect variance component. Results are also applicable to the F-statistic in the one-way random-effects model. The effects of departure from normality on the F-statistic distribution are given.     

Coverage accuracy for a mean without third moment

For constructing a confidence interval for the mean of a random variable with a known variance, one may prefer the sample mean standardized by the true standard deviation to the Student's t-statistic since the information of knowing the variance is used in the former way. In this paper, by comparing the leading error term in the expansion of the coverage probability, we show that the above statement is not true when the third moment is infinite. Our theory prefers the Student's t-statistic either when one-sided confidence intervals are considered for a heavier tail distribution or when two-sided confidence intervals are considered. Unlike other existing expansions for the Student's t-statistic, the derived explicit expansion for the case of infinite third moment can be used to estimate the coverage error so that bias correction becomes possible.     

Asymptotic Expansion of the Non Null Distribution of the Two-Sample t-Statistic Under Non Normality with Application in Power Comparison

This article studies the non null distribution of the two-sample t-statistic, or Welch statistic, under non normality. The asymptotic expansion of the non null distribution is derived up to n ^?1, where n is the pooled sample size, under general conditions. It is used to compare the power with that obtained by normal theory method. A simple technique is recommended to achieve more power through a monotone transformation in practice.     

Weak Identification in Fuzzy Regression Discontinuity Designs

In fuzzy regression discontinuity (FRD) designs, the treatment effect is identified through a discontinuity in the conditional probability of treatment assignment. We show that when identification is weak (i.e., when the discontinuity is of a small magnitude), the usual t-test based on the FRD estimator and its standard error suffers from asymptotic size distortions as in a standard instrumental variables setting. This problem can be especially severe in the FRD setting since only observations close to the discontinuity are useful for estimating the treatment effect. To eliminate those size distortions, we propose a modified t-statistic that uses a null-restricted version of the standard error of the FRD estimator. Simple and asymptotically valid confidence sets for the treatment effect can be also constructed using this null-restricted standard error. An extension to testing for constancy of the regression discontinuity effect across covariates is also discussed. Supplementary materials for this article are available online.     

Inflated statistical significance of student's t test associated with small intersubject correlation

The independence assumption in statistical significance testing becomes increasingly crucial and unforgiving as sample size increases. Seemingly, inconsequential violations of this assumption can substantially increase the probability of a Type I error if sample sizes are large. In the case of Student's t test, it is found that correlations within samples in a range from 0.01 to 0.05 can lead to rejection of a true null hypothesis with high probability, if N is 50, 100 or larger.     

SINGULAR ELLIPTICAL DISTRIBUTION: DENSITY AND APPLICATIONS

ABSTRACT

The paper present an explicit expression for the density of a n-dimensional random vector with a singular Elliptical distribution. Based on this, the densities of the generalized Chi-squared and generalized t distributions are derived, examining the Pearson Type VII distribution and Kotz Type distribution (as specific Elliptical distributions). Finally, the results are applied to the study of the distribution of the residuals of an Elliptical linear model and the distribution of the t-statistic, based on a sample from an Elliptical population.     

Approximate tests in unbalanced two-way random models without interaction

In the presence of non-normality, we consider testing for the significance of the variance components in the unbalanced two-way random model without interaction. The approximate test is based on the F-statistic for this model. The asymptotic distribution of the F-statistic is derived as the number of treatments tends to infinity while the number of observations for a treatment in any block takes value from a finite set of positive integers. Robustness of the approximate test is given.     

Efficiency of t-Test and Hotelling's T 2-Test After Box-Cox Transformation

Early investigations of the effects of non-normality indicated that skewness has a greater effect on the distribution of t-statistic than does kurtosis. When the distribution is skewed, the actual p-values can be larger than the values calculated from the t-tables. Transformation of data to normality has shown good results in the case of univariate t-test. In order to reduce the effect of skewness of the distribution on normal-based t-test, one can transform the data and perform the t-test on the transformed scale. This method is not only a remedy for satisfying the distributional assumption, but it also turns out that one can achieve greater efficiency of the test. We investigate the efficiency of tests after a Box-Cox transformation. In particular, we consider the one sample test of location and study the gains in efficiency for one-sample t-test following a Box-Cox transformation. Under some conditions, we prove that the asymptotic relative efficiency of transformed t-test and Hotelling's T ²-test of multivariate location with respect to the same statistic based on untransformed data is at least one.     

A new test for high-dimensional regression coefficients in partially linear models

Partially linear regression models are semiparametric models that contain both linear and nonlinear components. They are extensively used in many scientific fields for their flexibility and convenient interpretability. In such analyses, testing the significance of the regression coefficients in the linear component is typically a key focus. Under the high-dimensional setting, i.e., “large p, small n,” the conventional F-test strategy does not apply because the coefficients need to be estimated through regularization techniques. In this article, we develop a new test using a U-statistic of order two, relying on a pseudo-estimate of the nonlinear component from the classical kernel method. Using the martingale central limit theorem, we prove the asymptotic normality of the proposed test statistic under some regularity conditions. We further demonstrate our proposed test's finite-sample performance by simulation studies and by analyzing some breast cancer gene expression data.     

A simple unit root testing methodology that does not require knowledge regarding the presence of a break

We develop a simple methodology that allows practitioners to test for the presence of a unit root without a priori knowledge regarding the occurrence of a break under the null hypothesis. We use a pre-test that is readily available in the estimated regression used to calculate the unit root statistics, and so our methodology is very easy to implement. The t-statistic corresponding to the impulse dummy variables evaluated at break date estimator is used as a pre-test to ascertain whether a break exists under the null hypothesis. Finite sample simulations show that our methodology yields tests that maintain their size.     

Evidence of spurious regression driven by heavy-tailed observations with structural changes

This article extends the theoretical analysis of spurious relationship and considers the situation where the deterministic components of the processes generating individual series are independent heavy-tailed with structural changes. It shows when those sequences are used in ordinary least-square regression, the convenient t-statistic procedures wrongly indicate that (i) the spurious significance is established when regressing mean-stationary and trend-stationary series with structural changes, (ii) the spurious relationship occurs under broken mean-stationary and difference-stationary sequences, and (iii) the extent of spurious regression becomes stronger between difference-stationary and trend-stationary series in the presence of breaks. The spurious phenomenon is present regardless of the sample size and structural breaks taking place at the same points or not. Simulation experiments confirm our asymptotic results and reveal that the spurious effects are not only sensitive to the relative location of structural changes with the sample, but seriously depend on the stable indexes.     

Comparison of permutation methods for the partial correlation and partial mantel tests

This study compares empirical type I error and power of different permutation techniques that can be used for partial correlation analysis involving three data vectors and for partial Mantel tests. The partial Mantel test is a form of first-order partial correlation analysis involving three distance matrices which is widely used in such fields as population genetics, ecology, anthropology, psychometry and sociology. The methods compared are the following: (1) permute the objects in one of the vectors (or matrices); (2) permute the residuals of a null model; (3) correlate residualized vector 1 (or matrix A) to residualized vector 2 (or matrix B); permute one of the residualized vectors (or matrices); (4) permute the residuals of a full model. In the partial correlation study, the results were compared to those of the parametric t-test which provides a reference under normality. Simulations were carried out to measure the type I error and power of these permutatio methods, using normal and non-normal data, without and with an outlier. There were 10 000 simulations for each situation (100 000 when n = 5); 999 permutations were produced per test where permutations were used. The recommended testing procedures are the following:(a) In partial correlation analysis, most methods can be used most of the time. The parametric t-test should not be used with highly skewed data. Permutation of the raw data should be avoided only when highly skewed data are combined with outliers in the covariable. Methods implying permutation of residuals, which are known to only have asymptotically exact significance levels, should not be used when highly skewed data are combined with small sample size. (b) In partial Mantel tests, method 2 can always be used, except when highly skewed data are combined with small sample size. (c) With small sample sizes, one should carefully examine the data before partial correlation or partial Mantel analysis. For highly skewed data, permutation of the raw data has correct type I error in the absence of outliers. When highly skewed data are combined with outliers in the covariable vector or matrix, it is still recommended to use the permutation of raw data. (d) Method 3 should never be used.     

The Berry-Esseen bound for Studentized U-statistics

Callaert and Veraverbeke (1981) recently obtained a Berry-Esseen-type bound of order n^–1/2 for Studentized nondegenerate U-statistics of degree two. The condition these authors need to obtain this order bound is the finiteness of the 4.5th absolute moment of the kernel h. In this note it is shown that this assumption can be weakened to that of a finite (4 + ?)th absolute moment of the kernel h, for some ? > 0. Our proof resembles part of Helmers and van Zwet (1982), where an analogous result is obtained for the Student t-statistic. The present note extends this to Studentized U-statistics.     

Testing for random effect in the Fuller–Battese model

We consider the Fuller–Battese model where random effects are allowed to be from non-normal universes. The asymptotic distribution of the F-statistic in this model is derived as the number of groups tends to infinity (is large) and sample size from any group is either fixed or large. The result is used to establish an approximate test for the significance of the random effect variance component. Robustness of the established approximate test is given.     

Monitoring prespecified changes in linear profiles using control charts with supplementary runs rules

In profile monitoring, some methods have been developed to detect the unspecified changes in the profiles. However, detecting changes away from the “normal” profile toward one of several prespecified “bad” profiles is one possible and challenging purpose. In this article, control charts with supplementary runs rules are developed to detect the prespecified changes in linear profiles. A control chart is first developed based on the Student's t-statistic in t test, and two runs rules are then supplemented to this chart, respectively. Simulation studies show that the proposed control schemes are effective and stable. Moreover, the control schemes are better than the existing alternative charts when the number of observations per sample profile is large. Finally, two illustrative examples indicate that our proposed schemes are effective and easy to be implemented.     

Abandon Statistical Significance

ABSTRACT

We discuss problems the null hypothesis significance testing (NHST) paradigm poses for replication and more broadly in the biomedical and social sciences as well as how these problems remain unresolved by proposals involving modified p-value thresholds, confidence intervals, and Bayes factors. We then discuss our own proposal, which is to abandon statistical significance. We recommend dropping the NHST paradigm—and the p-value thresholds intrinsic to it—as the default statistical paradigm for research, publication, and discovery in the biomedical and social sciences. Specifically, we propose that the p-value be demoted from its threshold screening role and instead, treated continuously, be considered along with currently subordinate factors (e.g., related prior evidence, plausibility of mechanism, study design and data quality, real world costs and benefits, novelty of finding, and other factors that vary by research domain) as just one among many pieces of evidence. We have no desire to “ban” p-values or other purely statistical measures. Rather, we believe that such measures should not be thresholded and that, thresholded or not, they should not take priority over the currently subordinate factors. We also argue that it seldom makes sense to calibrate evidence as a function of p-values or other purely statistical measures. We offer recommendations for how our proposal can be implemented in the scientific publication process as well as in statistical decision making more broadly.