A modified combination test for the analysis of a clinical trial when a protocol amendment changed the inclusion criteria

Protocol amendments are often necessary in clinical trials. They can change the entry criteria and, therefore, the population. Simply analysing the pooled data is not acceptable. Instead, each phase should be analysed separately and a combination test such as Fisher's test should be applied to the resulting p-values. In this situation, an asymmetric decision rule is not appropriate. Therefore, we propose a modification of Bauer and Köhne's test. We compare this new test with the tests of Liptak, Fisher, Bauer/Köhne and Edgington. In case of differences in variance only or only small differences in mean, Liptak's Z-score approach is the best, and the new test keeps up with the rest and is in most cases slightly superior. In other situations, the new test and the Z-score approach are not preferable. But no big differences in populations are usually to be expected due to amendments. Then, the new method is a recommendable alternative.     

A Starting Point for Navigating the World of Daily Fantasy Basketball

Fantasy sports, particularly the daily variety in which new lineups are selected each day, are a rapidly growing industry. The two largest companies in the daily fantasy business, DraftKings and Fanduel, have been valued as high as $2 billion. This research focuses on the development of a complete system for daily fantasy basketball, including both the prediction of player performance and the construction of a team. First, a Bayesian random effects model is used to predict an aggregate measure of daily NBA player performance. The predictions are then used to construct teams under the constraints of the game, typically related to a fictional salary cap and player positions. Permutation based and K-nearest neighbors approaches are compared in terms of the identification of “successful” teams—those who would be competitive more often than not based on historical data. We demonstrate the efficacy of our system by comparing our predictions to those from a well-known analytics website, and by simulating daily competitions over the course of the 2015–2016 season. Our results show an expected profit of approximately $9,000 on an initial $500 investment using the K-nearest neighbors approach, a 36% increase relative to using the permutation-based approach alone. Supplementary materials for this article are available online.     

A Permutation Based Procedure for Classification Assessment

This article proposes a permutation procedure for evaluating the performance of different classification methods. In particular, we focus on two of the most widespread and used classification methodologies: latent class analysis and k-means clustering. The classification performance is assessed by means of a permutation procedure which allows for a direct comparison of the methods, the development of a statistical test, and points out better potential solutions. Our proposal provides an innovative framework for the validation of the data partitioning and offers a guide in the choice of which classification procedure should be used     

Detecting Outliers in Gamma Distribution

Zerbet and Nikulin presented the new statistic Z _k for detecting outliers in exponential distribution. They also compared this statistic with Dixon's statistic D _k . In this article, we extend this approach to gamma distribution and compare the result with Dixon's statistic. The results show that the test based on statistic Z _k is more powerful than the test based on the Dixon's statistic.     

Ideal bootstrap estimation of expected prediction error for k-nearest neighbor classifiers: Applications for classification and error assessment

Euclidean distance k-nearest neighbor (k-NN) classifiers are simple nonparametric classification rules. Bootstrap methods, widely used for estimating the expected prediction error of classification rules, are motivated by the objective of calculating the ideal bootstrap estimate of expected prediction error. In practice, bootstrap methods use Monte Carlo resampling to estimate the ideal bootstrap estimate because exact calculation is generally intractable. In this article, we present analytical formulae for exact calculation of the ideal bootstrap estimate of expected prediction error for k-NN classifiers and propose a new weighted k-NN classifier based on resampling ideas. The resampling-weighted k-NN classifier replaces the k-NN posterior probability estimates by their expectations under resampling and predicts an unclassified covariate as belonging to the group with the largest resampling expectation. A simulation study and an application involving remotely sensed data show that the resampling-weighted k-NN classifier compares favorably to unweighted and distance-weighted k-NN classifiers.     

Estimation of prediction error by using <Emphasis Type="Italic">K</Emphasis>-fold cross-validation

Estimation of prediction accuracy is important when our aim is prediction. The training error is an easy estimate of prediction error, but it has a downward bias. On the other hand, K-fold cross-validation has an upward bias. The upward bias may be negligible in leave-one-out cross-validation, but it sometimes cannot be neglected in 5-fold or 10-fold cross-validation, which are favored from a computational standpoint. Since the training error has a downward bias and K-fold cross-validation has an upward bias, there will be an appropriate estimate in a family that connects the two estimates. In this paper, we investigate two families that connect the training error and K-fold cross-validation.     

A general algorithm for computing simultaneous prediction intervals for the (log)-location-scale family of distributions

Making predictions of future realized values of random variables based on currently available data is a frequent task in statistical applications. In some applications, the interest is to obtain a two-sided simultaneous prediction interval (SPI) to contain at least k out of m future observations with a certain confidence level based on n previous observations from the same distribution. A closely related problem is to obtain a one-sided upper (or lower) simultaneous prediction bound (SPB) to exceed (or be exceeded) by at least k out of m future observations. In this paper, we provide a general approach for computing SPIs and SPBs based on data from a particular member of the (log)-location-scale family of distributions with complete or right censored data. The proposed simulation-based procedure can provide exact coverage probability for complete and Type II censored data. For Type I censored data, our simulation results show that our procedure provides satisfactory results in small samples. We use three applications to illustrate the proposed simultaneous prediction intervals and bounds.     

Permutation Methods for Comparing the Accuracy of Nested Prediction Models in Survival Analysis

When making patient-specific prediction, it is important to compare prediction models to evaluate the gain in prediction accuracy for including additional covariates. We propose two statistical testing methods, the complete data permutation (CDP) and the permutation cross-validation (PCV) for comparing prediction models. We simulate clinical trial settings extensively and show that both methods are robust and achieve almost correct test sizes; the methods have comparable power in moderate to large sample situations, while the CDP is more efficient in computation. The methods are also applied to ovarian cancer clinical trial data.     

Test for the equality of several correlation coefficients

We derive two C(α) statistics and the likelihood-ratio statistic for testing the equality of several correlation coefficients, from k ≥ 2 independent random samples from bivariate normal populations. The asymptotic relationship of the C(α) tests, the likelihood-ratio test, and a statistic based on the normality assumption of Fisher's Z-transform of the sample correlation coefficient is established. A comparative performance study, in terms of size and power, is then conducted by Monte Carlo simulations. The likelihood-ratio statistic is often too liberal, and the statistic based on Fisher's Z-transform is conservative. The performance of the two C(α) statistics is identical. They maintain significance level well and have almost the same power as the other statistics when empirically calculated critical values of the same size are used. The C(α) statistic based on a noniterative estimate of the common correlation coefficient (based on Fisher's Z-transform) is recommended.     

Exact moments of the symmetric cubic assignment statistic

Hubert (1987Assignment Methods in Combinatorial Data Analysis) presented a class of permutation, or random assignment, techniques for assessing correspondence between general k-dimensional proximity measures on a set of “objects.” A major problem in higher-order assignment models is the prohibitive level of computation that is required. We present the first three exact moments of a test statistic for the symmetric cubic assignment model. Efficient computational formulas for the first three moments have been derived, thereby permitting approximation of the permutation distribution using well-known methods.     

Concentration inequalities of the cross-validation estimator for empirical risk minimizer

ABSTRACT

We derive concentration inequalities for the cross-validation estimate of the generalization error for empirical risk minimizers. In the general setting, we show that the worst-case error of this estimate is not much worse that of training error estimate see Kearns M, Ron D. [Algorithmic stability and sanity-check bounds for leave-one-out cross-validation. Neural Comput. 1999;11:1427–1453]. General loss functions and class of predictors with finite VC-dimension are considered. Our focus is on proving the consistency of the various cross-validation procedures. We point out the interest of each cross-validation procedure in terms of rates of convergence. An interesting consequence is that the size of the test sample is not required to grow to infinity for the consistency of the cross-validation procedure.     

Robust Lagrange multiplier test for detecting ARCH/GARCH effect using permutation and bootstrap

The Lagrange Multiplier (LM) test is one of the principal tools to detect ARCH and GARCH effects in financial data analysis. However, when the underlying data are non‐normal, which is often the case in practice, the asymptotic LM test, based on the χ²‐approximation of critical values, is known to perform poorly, particularly for small and moderate sample sizes. In this paper we propose to employ two re‐sampling techniques to find critical values of the LM test, namely permutation and bootstrap. We derive the properties of exactness and asymptotically correctness for the permutation and bootstrap LM tests, respectively. Our numerical studies indicate that the proposed re‐sampled algorithms significantly improve size and power of the LM test in both skewed and heavy‐tailed processes. We also illustrate our new approaches with an application to the analysis of the Euro/USD currency exchange rates and the German stock index. The Canadian Journal of Statistics 40: 405–426; 2012 © 2012 Statistical Society of Canada     

A model-averaging approach for smoothing spline regression

ABSTRACT

This article considers nonparametric regression problems and develops a model-averaging procedure for smoothing spline regression problems. Unlike most smoothing parameter selection studies determining an optimum smoothing parameter, our focus here is on the prediction accuracy for the true conditional mean of Y given a predictor X. Our method consists of two steps. The first step is to construct a class of smoothing spline regression models based on nonparametric bootstrap samples, each with an appropriate smoothing parameter. The second step is to average bootstrap smoothing spline estimates of different smoothness to form a final improved estimate. To minimize the prediction error, we estimate the model weights using a delete-one-out cross-validation procedure. A simulation study has been performed by using a program written in R. The simulation study provides a comparison of the most well known cross-validation (CV), generalized cross-validation (GCV), and the proposed method. This new method is straightforward to implement, and gives reliable performances in simulations.     

Chi-squared tests in kth-moment sufficient dimension reduction

We present a novel approach to sufficient dimension reduction for the conditional kth moments in regression. The approach provides a computationally feasible test for the dimension of the central kth-moment subspace. In addition, we can test predictor effects without assuming any models. All test statistics proposed in the novel approach have asymptotic chi-squared distributions.     

Paired permutation testing in 2<Superscript><Emphasis Type="Italic">k</Emphasis></Superscript> unreplicated factorials

Exact permutation testing of effects in unreplicated two-level multifactorial designs is developed based on the notion of realigning observations and on paired permutations. This approach preserves the exchangeability of error components for testing up tok effects. Advantages and limitations of exact permutation procedures for unreplicated factorials are discussed and a simulation study on paired permutation testing is presented.     

Permuiation tesis for k-sample binomial data with comparisons of exact and approximate P-levels

Exact ksample permutation tests for binary data for three commonly encountered hypotheses tests are presented,, The tests are derived both under the population and randomization models . The generating function for the number of cases in the null distribution is obtained, The asymptotic distributions of the test statistics are derived . Actual significance levels are computed for the asymptotic test versions , Random sampling of the null distribution is suggested as a superior alternative to the asymptotics and an efficient computer technique for implementing the random sampling is described., finally, some numerical examples are presented and sample size guidelines given for computer implementation of the exact tests.     

Concomitants of multivariate order statistics from multivariate elliptical distributions

ABSTRACT

In this article, we consider a (k + 1)n-dimensional elliptically contoured random vector (X^T₁, X₂^T, …, X^T_k, Z^T)^T = (X₁₁, …, X_1n, …, X_k1, …, X_kn, Z₁, …, Z_n)^T and derive the distribution of concomitant of multivariate order statistics arising from X₁, X₂, …, X_k. Specially, we derive a mixture representation for concomitant of bivariate order statistics. The joint distribution of the concomitant of bivariate order statistics is also obtained. Finally, the usefulness of our result is illustrated by a real-life data.     

Une application des méthodes séquentielles aux tests de permutations

To carry out a permutation test we have to examine the n! permutations of the observations. In order to make the permutation test feasible, Dwass (1957) proposed to examine only a sample of these permutations. With the help of sequential methods, we obtain a test which is never less efficient than that proposed by Dwass or the permutation test itself, in the sense that it is as powerful and never requires more permutations to make a decision. In practice, we can expect to gain much efficiency.     

A note on the minimum size of an orthogonal array

It is an elementary fact that the size of an orthogonal array of strength t on k factors must be a multiple of a certain number, say L_t, that depends on the orders of the factors. Thus L_t is a lower bound on the size of arrays of strength t on those factors, and is no larger than L_k, the size of the complete factorial design. We investigate the relationship between the numbers L_t, and two questions in particular: For what t is L_t < L_k? And when L_t = L_k, is the complete factorial design the only array of that size and strength t? Arrays are assumed to be mixed-level.

We refer to an array of size less than L_k as a proper fraction. Guided by our main result, we construct a variety of mixed-level proper fractions of strength k ? 1 that also satisfy a certain group-theoretic condition.     

A comparative study of tests for paired lifetime data

In this paper, we investigate different procedures for testing the equality of two mean survival times in paired lifetime studies. We consider Owen’s M-test and Q-test, a likelihood ratio test, the paired t-test, the Wilcoxon signed rank test and a permutation test based on log-transformed survival times in the comparative study. We also consider the paired t-test, the Wilcoxon signed rank test and a permutation test based on original survival times for the sake of comparison. The size and power characteristics of these tests are studied by means of Monte Carlo simulations under a frailty Weibull model. For less skewed marginal distributions, the Wilcoxon signed rank test based on original survival times is found to be desirable. Otherwise, the M-test and the likelihood ratio test are the best choices in terms of power. In general, one can choose a test procedure based on information about the correlation between the two survival times and the skewness of the marginal survival distributions.