An empirical study of five multivariate tests for the single-factor repeated measures model

A Monte Carlo study was used to examine the Type I error and power values of five multivariate tests for the single-factor repeated measures model The performance of Hotelling's T2 and four nonparametric tests, including a chi-square and an F-test version of a rank-transform procedure, were investigated for different distributions, sample sizes, and numbers of repeated measures. The results indicated that both Hotellings T* and the F-test version of the rank-transform performed well, producing Type I error rates which were close to the nominal value. The chi-square version of the rank-transform test, on the other hand, produced inflated Type I error rates for every condition studied. The Hotelling and F-test version of the rank-transform procedure showed similar power for moderately-skewed distributions, but for strongly skewed distributions the F-test showed much better power. The performance of the other nonparametric tests depended heavily on sample size. Based on these results, the F-test version of the rank-transform procedure is recommended for the single-factor repeated measures model.     

Minimum average partial correlation and parallel analysis: The influence of oblique structures

ABSTRACT

Parallel analysis (Horn 1965) and the minimum average partial correlation (MAP; Velicer 1976) have been widely spread as optimal solutions to identify the correct number of axes in principal component analysis. Previous results showed, however, that they become inefficient when variables belonging to different components strongly correlate. Simulations are used to assess their power to detect the dimensionality of data sets with oblique structures. Overall, MAP had the best performances as it was more powerful and accurate than PA when the component structure was modestly oblique. However, both stopping rules performed poorly in the presence of highly oblique factors.     

Robust parametric tests of constant conditional correlation in a MGARCH model

This article provides a rigorous asymptotic treatment of new and existing asymptotically valid conditional moment (CM) testing procedures of the constant conditional correlation (CCC) assumption in a multivariate GARCH model. Full and partial quasi maximum likelihood estimation (QMLE) frameworks are considered, as is the robustness of these tests to non-normality. In particular, the asymptotic validity of the LM procedure proposed by Tse (2000 Tse, Y. K. (2000). A test for constant correlations in a multivariate GARCH model. Journal of Econometrics 98 (1):107–127.[Crossref], [Web of Science ®] , [Google Scholar]) is analyzed, and new asymptotically robust versions of this test are proposed for both estimation frameworks. A Monte Carlo study suggests that a robust Tse test procedure exhibits good size and power properties, unlike the original variant which exhibits size distortion under non-normality.     

Permutational tests for correlation matrices

Permutational tests are proposed for the hypotheses that two population correlation matrices have common eigenvectors, and that two population correlation matrices are equal. The only assumption made in these tests is that the distributional form is the same in the two populations; they should be useful as a prelude either to tests of mean differences in grouped standardised data or to principal component investigation of such data.The performance of the permutational tests is subjected to Monte Carlo investigation, and a comparison is made with the performance of the likelihood-ratio test for equality of covariance matrices applied to standardised data. Bootstrapping is considered as an alternative to permutation, but no particular advantages are found for it. The various tests are applied to several data sets.     

Performance of unit-root tests for non linear unit-root and partial unit-root processes

ABSTRACT

This paper investigates the finite-sample performance of the augmented Dickey–Fuller (ADF), Phillips–Perron (PP), momentum threshold autoregressive (M-TAR), Kapetanios–Shin–Snell (KSS), and the inf-t unit-root tests. Simulation results show that the ADF and KSS tests have better size, whereas other tests generate severe size distortions when the date-generating processes are non linear unit-root processes. In general, with regard to the combination of test powers with test sizes, the ADF and KSS tests are comparatively better than the PP, M-TAR, and inf-t tests; moreover, the inf-t test exhibits the poorest performance even for larger sample sizes.     

Test for the equality of several correlation coefficients

We derive two C(α) statistics and the likelihood-ratio statistic for testing the equality of several correlation coefficients, from k ≥ 2 independent random samples from bivariate normal populations. The asymptotic relationship of the C(α) tests, the likelihood-ratio test, and a statistic based on the normality assumption of Fisher's Z-transform of the sample correlation coefficient is established. A comparative performance study, in terms of size and power, is then conducted by Monte Carlo simulations. The likelihood-ratio statistic is often too liberal, and the statistic based on Fisher's Z-transform is conservative. The performance of the two C(α) statistics is identical. They maintain significance level well and have almost the same power as the other statistics when empirically calculated critical values of the same size are used. The C(α) statistic based on a noniterative estimate of the common correlation coefficient (based on Fisher's Z-transform) is recommended.     

A monte carlo study on the robustness of four MANOVA criterion tests

Four MANOVA tests (Wilk's Lambda, Roy's Largest Root Test, the Hotelling-Lawley Trace and the Pillai-Bartlett Trace) were studied when restricted sample data were drawn from normal populations. Robustness was compared by examining bias at critical points and fluctuations in the standard error of the empirical distributions. The Wilk's Lambda statistic was found to be the least affected by the restricted sampling.     

The performance of alternative estimators for the probit model with first-order serial correlation 1

The purpose of the paper is to evaluate the relative performance of two generalized conditional moment (GCM) estimators in terms of their mean squared errors, for the Probit model with first-order serial correlation. The first estimator is a linearized one-step estimator described by Poirier and Ruud (1988). The second one is defined in the present paper. Monte Car10 experiments suggest that the GCM estimators outperform the ordinary Probit estimator. The two GCM estimators do almost equally well, except that the second one may be easier to calculate, especially in large samples.     

Improved estimators of regression coefficients in measurement error models

This paper is concerned with methods of reducing variability and computer time in a simulation study. The Monte Carlo swindle, through mathematical manipulations, has been shown to yield more precise estimates than the “naive” approach. In this study computer time is considered in conjunction with the variance estimates. It is shown that by this measure the naive method is often a viable alternative to the swindle. This study concentrates on the problem of estimating the variance of an estimator of location. The advantage of one technique over another depends upon the location estimator, the sample size, and the underlying distribution. For a fixed number of samples, while the naive method gives a less precise estimate than the swindle, it requires fewer computations. In addition, for certain location estimators and distributions, the naive method is able to take advantage of certain shortcuts in the generation of each sample. The small amount of time required by this “enlightened” naive method often more than compensates for its relative lack of precision.     

Clustering of correlation matrices: A step-wise procedure

An alternative to the criteria proposed by King [9] for clustering correlation matrices is explored. Several simple examples examined by total enumeration employing our criterion indicate that the criterion yields clusters with high within group correlation. A step-wise routine is suggested for problems of realistic size and is applied to the examples presented in King's article. Additional evidence in the form of several Monte Carlo experiments indicates that the routine performs satisfactorily in determining the optimal separation of variables into two groups.     

Estimates of the quantiles of kendall's partial rank correlation coefficient

The sampling distribution of kendall's partial rank correlation coefficient, J_xy?z, is not known for N>4, where N is the number of subjectts. Moran (1951) used a direcr conbinatorial method to obtain the distribution of J_xy?z forN=4; however, ten minor computationa; errors in his Table 2apparently resulted in how erroneous entries for his frequency table. Since the parctial limits of the direct combinatorial approach have been reached once N>4, the first main objective of this paper was to obtain the exact distribution of J_xy?z for N=f, 6, and 7 using an electronic computer. The second was to use the Monte Carlo method to obtain reliable estimates of the quantiles of J_xy?z for N=8,9,...,30     

Combined estimators for the correlation coefficient

Three combined estimators for the bivariate normal correlation parameter are considered. The data consist of k independent sample correlation coefficients and it is assumed that the underlying correlation parameters are all equal to ρ. Based upon the joint density function of the sample correlations a combined estimator of ρ is obtained as an approximation to the maximum likelihood solution. Two linearly combined estimators are also considered. One of them is based on Fisher's z-transformation of the sample correlations and the other on an unbiased estimator of ρ. The comparison of these three estimators indicates that the combined (approximate) MLE has a slightly smaller estimated mean squared error relative to the other two combined methods of estimation, but it does so at the expense of a relatively larger bias.     

An empirical power study of multi-sample tests for exponentiality

Results are given of an empirical power study of three statistical procedures for testing for exponentiality of several independent samples. The test procedures are the Tiku (1974) test, a multi-sample Durbin (1975) test, and a multi-sample Shapiro–Wilk (1972) test. The alternative distributions considered in the study were selected from the gamma, Weibull, Lomax, lognormal, inverse Gaussian, and Burr families of positively skewed distributions. The general behavior of the conditional mean exceedance function is used to classify each alternative distribution. It is shown that Tiku's test generally exhibits overall greater power than either of the other two test procedures. For certain alternative distributions, Shapiro–Wilk's test is superior when the sample sizes are small.     

An empirical comparison of several methods for testing the equality of dependent proportions

Three methods for testing the equality of nonindependent proportions were compared with, the use of Monte Carlo techniques. The three methods included Cochran's test, an ANOVA F test, and Hotelling's T² test. With respect to empirical significance levels, the ANOVA F test is recommended as the preferred method of analysis.

Oftentimes an experimenter is interested in testing the equality of several proportions. When the proportions are independent Kemp and Butcher (1972) and Butcher and Kemp (1974) compared several methods for analysing large sample binomial data for the case of a 3 x 3 factorial design without replication. In addition, Levy and Narula (1977) compared many of the same methods for analyzing binomial data; however, Levy and Narula investigated the relative utility of the methods for small sample sizes.     

Some empirical results on the two-way analysis of variance by ranks

Iman (1974) and Conover and Iman (1976) have shown by means of simulation studies that a rank transform, whereby the ordinary F-tests are applied to the ranks of the original observations in two-way experimental designs, presents a remarkably powerful method of analysis. In the present study it is shown that this rank transform is closely related to the procedure proposed by Lemmer and Stoker (1967) for the two-way analysis of variance. The main conclusions are that the Iman tests are generally slightly more powerful than those of Lemmer and Stoker, but that in the presence of interaction the latter tests for main effects are safer to use than the former because interaction tends to make the Iman tests for main effects significant even if no main effects exist.     

Power studies of tests for uniformity,II

Results from a power study of six statistics for testing that a sample is from a uniform distribution on the unit interval (0,1) are reported. The test statistics are all well-known and each of them was originally proposed because they should have high power against some alternative distributions. The tests considered are the Pearson probability product test, the Neyman smooth test, the Sukhatme test, the Durbin-Kolmogorov test, the Kuiper test, and the Sherman test. Results are given for each of these tests against each of four classes of alternatives. Also, the most powerful test against each member of the first three alternatives is obtained, and the powers of these tests are given for the same sample sizes as for the six general "omnibus" test statistics. These values constitute a "power envelope" against which all tests can be compared. The Neyman smooth tests with 2nd and 4th degree polynomials are found to have good power and are recommended as general tests for uniformity.     

Exact monte carlo tests using several statistics

Concepts of ranking and boundary of multivariate statistics are discussed and applied to the simultaneous use of several test statistics calculated for data and simulated replicates. An example of residual analysis in regression is given using layer ranks and supplementary simulation with a stopping rule.     

A comparative study of tests for paired lifetime data

In this paper, we investigate different procedures for testing the equality of two mean survival times in paired lifetime studies. We consider Owen’s M-test and Q-test, a likelihood ratio test, the paired t-test, the Wilcoxon signed rank test and a permutation test based on log-transformed survival times in the comparative study. We also consider the paired t-test, the Wilcoxon signed rank test and a permutation test based on original survival times for the sake of comparison. The size and power characteristics of these tests are studied by means of Monte Carlo simulations under a frailty Weibull model. For less skewed marginal distributions, the Wilcoxon signed rank test based on original survival times is found to be desirable. Otherwise, the M-test and the likelihood ratio test are the best choices in terms of power. In general, one can choose a test procedure based on information about the correlation between the two survival times and the skewness of the marginal survival distributions.     

Comparison of permutation methods for the partial correlation and partial mantel tests

This study compares empirical type I error and power of different permutation techniques that can be used for partial correlation analysis involving three data vectors and for partial Mantel tests. The partial Mantel test is a form of first-order partial correlation analysis involving three distance matrices which is widely used in such fields as population genetics, ecology, anthropology, psychometry and sociology. The methods compared are the following: (1) permute the objects in one of the vectors (or matrices); (2) permute the residuals of a null model; (3) correlate residualized vector 1 (or matrix A) to residualized vector 2 (or matrix B); permute one of the residualized vectors (or matrices); (4) permute the residuals of a full model. In the partial correlation study, the results were compared to those of the parametric t-test which provides a reference under normality. Simulations were carried out to measure the type I error and power of these permutatio methods, using normal and non-normal data, without and with an outlier. There were 10 000 simulations for each situation (100 000 when n = 5); 999 permutations were produced per test where permutations were used. The recommended testing procedures are the following:(a) In partial correlation analysis, most methods can be used most of the time. The parametric t-test should not be used with highly skewed data. Permutation of the raw data should be avoided only when highly skewed data are combined with outliers in the covariable. Methods implying permutation of residuals, which are known to only have asymptotically exact significance levels, should not be used when highly skewed data are combined with small sample size. (b) In partial Mantel tests, method 2 can always be used, except when highly skewed data are combined with small sample size. (c) With small sample sizes, one should carefully examine the data before partial correlation or partial Mantel analysis. For highly skewed data, permutation of the raw data has correct type I error in the absence of outliers. When highly skewed data are combined with outliers in the covariable vector or matrix, it is still recommended to use the permutation of raw data. (d) Method 3 should never be used.