Full Bayesian Analysis for a Class of Jump-Diffusion Models

The Full Bayesian Significance Test (FBST) is adjusted for jump detection in a diffusion process. Under a natural parameterization, pure diffusion can be seen as a precise hypothesis. The evidence measure defined by FBST deals with absolutely continuous posterior distributions, when posterior rates for precise hypotheses are not appropriate. Applications to simulated and real data are shown.     

Unit Roots: Bayesian Significance Test

The unit root problem plays a central role in empirical applications in the time series econometric literature. However, significance tests developed under the frequentist tradition present various conceptual problems that jeopardize the power of these tests, especially for small samples. Bayesian alternatives, although having interesting interpretations and being precisely defined, experience problems due to the fact that that the hypothesis of interest in this case is sharp or precise. The Bayesian significance test used in this article, for the unit root hypothesis, is based solely on the posterior density function, without the need of imposing positive probabilities to sets of zero Lebesgue measure. Furthermore, it is conducted under strict observance of the likelihood principle. It was designed mainly for testing sharp null hypotheses and it is called FBST for Full Bayesian Significance Test.     

Convergence of posterior odds

The problem of approximating an interval null or imprecise hypothesis test by a point null or precise hypothesis test under a Bayesian framework is considered. In the literature, some of the methods for solving this problem have used the Bayes factor for testing a point null and justified it as an approximation to the interval null. However, many authors recommend evaluating tests through the posterior odds, a Bayesian measure of evidence against the null hypothesis. It is of interest then to determine whether similar results hold when using the posterior odds as the primary measure of evidence. For the prior distributions under which the approximation holds with respect to the Bayes factor, it is shown that the posterior odds for testing the point null hypothesis does not approximate the posterior odds for testing the interval null hypothesis. In fact, in order to obtain convergence of the posterior odds, a number of restrictive conditions need to be placed on the prior structure. Furthermore, under a non-symmetrical prior setup, neither the Bayes factor nor the posterior odds for testing the imprecise hypothesis converges to the Bayes factor or posterior odds respectively for testing the precise hypothesis. To rectify this dilemma, it is shown that constraints need to be placed on the priors. In both situations, the class of priors constructed to ensure convergence of the posterior odds are not practically useful, thus questioning, from a Bayesian perspective, the appropriateness of point null testing in a problem better represented by an interval null. The theories developed are also applied to an epidemiological data set from White et al. (Can. Veterinary J. 30 (1989) 147–149.) in order to illustrate and study priors for which the point null hypothesis test approximates the interval null hypothesis test. AMS Classification: Primary 62F15; Secondary 62A15     

On the Use of Bayes Factor in Frequentist Testing of a Precise Hypothesis

This article deals with Bayes factors as useful Bayesian tools in frequentist testing of a precise hypothesis. A result and several examples are included to justify the definition of Bayes factor for point null hypotheses, without merging the initial distribution with a degenerate distribution on the null hypothesis. Of special interest is the problem of testing a proportion (joint with a natural criterion to compare different tests), the possible presence of nuisance parameters, or the influence of Bayesian sufficiency on this problem. The problem of testing a precise hypothesis under a Bayesian perspective is also considered and two alternative methods to deal with are given.     

Bayesian testing of an exponential point null hypothesis

The problem of testing a point null hypothesis involving an exponential mean is The problem of testing a point null hypothesis involving an exponential mean is usual interpretation of P-values as evidence against precise hypotheses is faulty. As in Berger and Delampady (1986) and Berger and Sellke (1987), lower bounds on Bayesian measures of evidence over wide classes of priors are found emphasizing the conflict between posterior probabilities and P-values. A hierarchical Bayes approach is also considered as an alternative to computing lower bounds and “automatic” Bayesian significance tests which further illustrates the point that P-values are highly misleading measures of evidence for tests of point null hypotheses.     

Model fusion and multiple testing in the likelihood paradigm: shrinkage and evidence supporting a point null hypothesis

ABSTRACT

According to the general law of likelihood, the strength of statistical evidence for a hypothesis as opposed to its alternative is the ratio of their likelihoods, each maximized over the parameter of interest. Consider the problem of assessing the weight of evidence for each of several hypotheses. Under a realistic model with a free parameter for each alternative hypothesis, this leads to weighing evidence without any shrinkage toward a presumption of the truth of each null hypothesis. That lack of shrinkage can lead to many false positives in settings with large numbers of hypotheses. A related problem is that point hypotheses cannot have more support than their alternatives. Both problems may be solved by fusing the realistic model with a model of a more restricted parameter space for use with the general law of likelihood. Applying the proposed framework of model fusion to data sets from genomics and education yields intuitively reasonable weights of evidence.     

Confidence intervals for secondary parameters following a sequential test

In sequential studies, formal interim analyses are usually restricted to a consideration of a single null hypothesis concerning a single parameter of interest. Valid frequentist methods of hypothesis testing and of point and interval estimation for the primary parameter have already been devised for use at the end of such a study. However, the completed data set may warrant a more detailed analysis, involving the estimation of parameters corresponding to effects that were not used to determine when to stop, and yet correlated with those that were. This paper describes methods for setting confidence intervals for secondary parameters in a way which provides the correct coverage probability in repeated frequentist realizations of the sequential design used. The method assumes that information accumulates on the primary and secondary parameters at proportional rates. This requirement will be valid in many potential applications, but only in limited situations in survival analysis.     

The Use of Permutation Tests for Variance Components in Linear Mixed Models

Standard asymptotic chi-square distribution of the likelihood ratio and score statistics under the null hypothesis does not hold when the parameter value is on the boundary of the parameter space. In mixed models it is of interest to test for a zero random effect variance component. Some available tests for the variance component are reviewed and a new test within the permutation framework is presented. The power and significance level of the different tests are investigated by means of a Monte Carlo simulation study. The proposed test has a significance level closer to the nominal one and it is more powerful.     

A predictive approach to measuring the strength of statistical evidence for single and multiple comparisons

The normalized maximum likelihood (NML) is a recent penalized likelihood that has properties that justify defining the amount of discrimination information (DI) in the data supporting an alternative hypothesis over a null hypothesis as the logarithm of an NML ratio, namely, the alternative hypothesis NML divided by the null hypothesis NML. The resulting DI, like the Bayes factor but unlike the P‐value, measures the strength of evidence for an alternative hypothesis over a null hypothesis such that the probability of misleading evidence vanishes asymptotically under weak regularity conditions and such that evidence can support a simple null hypothesis. Instead of requiring a prior distribution, the DI satisfies a worst‐case minimax prediction criterion. Replacing a (possibly pseudo‐) likelihood function with its weighted counterpart extends the scope of the DI to models for which the unweighted NML is undefined. The likelihood weights leverage side information, either in data associated with comparisons other than the comparison at hand or in the parameter value of a simple null hypothesis. Two case studies, one involving multiple populations and the other involving multiple biological features, indicate that the DI is robust to the type of side information used when that information is assigned the weight of a single observation. Such robustness suggests that very little adjustment for multiple comparisons is warranted if the sample size is at least moderate. The Canadian Journal of Statistics 39: 610–631; 2011. © 2011 Statistical Society of Canada     

A Note on the Use of Confidence Limits following Rejection of a Null Hypothesis

The equivalence of some tests of hypothesis and confidence limits is well known. When, however, the confidence limits are computed only after rejection of a null hypothesis, the usual unconditional confidence limits are no longer valid. This refers to a strict two-stage inference procedure: first test the hypothesis of interest and if the test results in a rejection decision, then proceed with estimating the relevant parameter. Under such a situation, confidence limits should be computed conditionally on the specified outcome of the test under which estimation proceeds. Conditional confidence sets will be longer than unconditional confidence sets and may even contain values of the parameter previously rejected by the test of hypothesis. Conditional confidence limits for the mean of a normal population with known variance are used to illustrate these results. In many applications, these results indicate that conditional estimation is probably not good practice.     

Weighing the Evidence for Hypotheses with Small Samples of Right-censored Exponential Data

The p-value evidence for an alternative to a null hypothesis regarding the mean lifetime can be unreliable if based on asymptotic approximations when there is only a small sample of right-censored exponential data. However, a guarded weight of evidence for the alternative can always be obtained without approximation, no matter how small the sample, and has some other advantages over p-values. Weights of evidence are defined as estimators of 0 when the null hypothesis is true and 1 when the alternative is true, and they are judged on the basis of the ensuing risks, where risk is mean squared error of estimation. The evidence is guarded in that a preassigned bound is placed on the risk under the hypothesis. Practical suggestions are given for choosing the bound and for interpreting the magnitude of the weight of evidence. Acceptability profiles are obtained by inversion of a family of guarded weights of evidence for two-sided alternatives to point hypotheses, just as confidence intervals are obtained from tests; these profiles are arguably more informative than confidence intervals, and are easily determined for any level and any sample size, however small. They can help understand the effects of different amounts of censoring. They are found for several small size data sets, including a sample of size 12 for post-operative cancer patients. Both singly Type I and Type II censored examples are included. An examination of the risk functions of these guarded weights of evidence suggests that if the censoring time is of the same magnitude as the mean lifetime, or larger, then the risks in using a guarded weight of evidence based on a likelihood ratio are not much larger than they would be if the parameter were known.     

A Correction Note on Improvement in Variance Estimation Using Auxiliary Information

ABSTRACT

In this paper, the testing problem for homogeneity in the mixture exponential family is considered. The model is irregular in the sense that each interest parameter forms a part of the null hypothesis (sub-null hypothesis) and the null hypothesis is the union of the sub-null hypotheses. The generalized likelihood ratio test does not distinguish between the sub-null hypotheses. The Supplementary Score Test is proposed by combining two orthogonalized score tests obtained corresponding to the two sub-null hypotheses after proper reparameterization. The test is easy to design and performs better than the generalized likelihood ratio test and other alternative tests by numerical comparisons.     

Simple and accurate one‐sided inference from signed roots of likelihood ratios

The authors propose two methods based on the signed root of the likelihood ratio statistic for one‐sided testing of a simple null hypothesis about a scalar parameter in the présence of nuisance parameters. Both methods are third‐order accurate and utilise simulation to avoid the need for onerous analytical calculations characteristic of competing saddlepoint procedures. Moreover, the new methods do not require specification of ancillary statistics. The methods respect the conditioning associated with similar tests up to an error of third order, and conditioning on ancillary statistics to an error of second order.     

Testing the linearity of negative binomial regression models

The negative binomial (NB) is frequently used to model overdispersed Poisson count data. To study the effect of a continuous covariate of interest in an NB model, a flexible procedure is used to model the covariate effect by fixed-knot cubic basis-splines or B-splines with a second-order difference penalty on the adjacent B-spline coefficients to avoid undersmoothing. A penalized likelihood is used to estimate parameters of the model. A penalized likelihood ratio test statistic is constructed for the null hypothesis of the linearity of the continuous covariate effect. When the number of knots is fixed, its limiting null distribution is the distribution of a linear combination of independent chi-squared random variables, each with one degree of freedom. The smoothing parameter value is determined by setting a specified value equal to the asymptotic expectation of the test statistic under the null hypothesis. The power performance of the proposed test is studied with simulation experiments.     

On the existence of a unique most probable category

When analyzing multinomial data interest often centers on the category with the largest probability. We examine a likelihood ratio test relevant for assessing the evidence for the existence of a unique most probable category. We provide the asymptotic null hypothesis distribution of the likelihood ratio statistic and the asymptotically least favorable configuration.     

Modified p-Value of Two-Sided Test for Normal Distribution with Restricted Parameter Space

This article proposes a modified p-value for the two-sided test of the location of the normal distribution when the parameter space is restricted. A commonly used test for the two-sided test of the normal distribution is the uniformly most powerful unbiased (UMPU) test, which is also the likelihood ratio test. The p-value of the test is used as evidence against the null hypothesis. Note that the usual p-value does not depend on the parameter space but only on the observation and the assumption of the null hypothesis. When the parameter space is known to be restricted, the usual p-value cannot sufficiently utilize this information to make a more accurate decision. In this paper, a modified p-value (also called the rp-value) dependent on the parameter space is proposed, and the test derived from the modified p-value is also shown to be the UMPU test.     

Score tests when a nuisance parameter is unidentified under the null hypothesis

Rao's score test normally replaces nuisance parameters by their maximum likelihood estimates under the null hypothesis about the parameter of interest. In some models, however, a nuisance parameter is not identified under the null, so that this approach cannot be followed. This paper suggests replacing the nuisance parameter by its maximum likelihood estimate from the unrestricted model and making the appropriate adjustment to the variance of the estimated score. This leads to a rather natural modification of Rao's test, which is examined in detail for a regression-type model. It is compared with the approach, which has featured most frequently in the literature on this problem, where a test statistic appropriate to a known value of the nuisance parameter is treated as a function of that parameter and maximised over its range. It is argued that the modified score test has considerable advantages, including robustness to a crucial assumption required by the rival approach.     

Robust Tests in Semiparametric Partly Linear Models

Abstract.  This paper focuses on the problem of testing the null hypothesis that the regression parameter equals a fixed value under a semiparametric partly linear regression model by using a three-step robust estimate for the regression parameter and the regression function. Two families of tests statistics are considered and their asymptotic distributions are studied under the null hypothesis and under contiguous alternatives. A Monte Carlo study is performed to compare the finite sample behaviour of the proposed tests with the classical one.     

On similarity and independence properties of composite edf goodness-of-fit tests

Many test statistics for classical simple goodness-of-fit hypothesis testing problems are distancemeasures between the distribution function of the null hypothesis distributipn and the empirical distribution function sometimes called EDF tests. If a composite parametric null hypothesis is considered in place of the simple null hypothesis, then a test statistic can be obtained from each EDF test by replacing the known distribution function of the simple problem by the Rao-Blackwell estimating distribution function. In this note we use known results to show that these Rao-Blackwell-EDF test statistics have distributions that do not depend upon parameter values, and hence that these tests are independent of a complete sufficient statistic for the parameters.     

Advantages of Variance Stabilization

Abstract. Variance stabilization is a simple device for normalizing a statistic. Even though its large sample properties are similar to those of studentizing, many simulation studies of confidence interval procedures show that variance stabilization works better for small samples. We investigated this question in the context of testing a null hypothesis involving a single parameter. We provide support for a measure of evidence for an alternative hypothesis that is simple to compute, calibrate and interpret. It has applications in most routine problems in statistics, and leads to more accurate confidence intervals, estimated power and hence sample size calculations than standard asymptotic methods. Such evidence is readily combined when obtained from different studies. Connections to other approaches to statistical evidence are described, with a notable link to Kullback–Leibler symmetrized divergence.