Comparison of Goodness-of-Fit Measures in Probit Regression Model

This article examines several goodness-of-fit measures in the binary probit regression model. Existing pseudo-R ² measures are reviewed, two modified and one new pseudo-R ² measure are proposed. For the probit regression model, empirical comparisons are made for different goodness-of-fit measures with the squared sample correlation coefficient of the observed response and the predicted probabilities. As an illustration, the goodness-of-fit measures are applied to a “paid labor force” data set.     

Variability explained by covariates in linear mixed‐effect models for longitudinal data

Variability explained by covariates or explained variance is a well‐known concept in assessing the importance of covariates for dependent outcomes. In this paper we study R² statistics of explained variance pertinent to longitudinal data under linear mixed‐effect models, where the R² statistics are computed at two different levels to measure, respectively, within‐ and between‐subject variabilities explained by the covariates. By deriving the limits of R² statistics, we find that the interpretation of explained variance for the existing R² statistics is clear only in the case where the covariance matrix of the outcome vector is compound symmetric. Two new R² statistics are proposed to address the effect of time‐dependent covariate means. In the general case where the outcome covariance matrix is not compound symmetric, we introduce the concept of compound symmetry projection and use it to define level‐one and level‐two R² statistics. Numerical results are provided to support the theoretical findings and demonstrate the performance of the R² statistics. The Canadian Journal of Statistics 38: 352–368; 2010 © 2010 Statistical Society of Canada     

A new approach to modeling positive random variables with repeated measures

In many situations, it is common to have more than one observation per experimental unit, thus generating the experiments with repeated measures. In the modeling of such experiments, it is necessary to consider and model the intra-unit dependency structure. In the literature, there are several proposals to model positive continuous data with repeated measures. In this paper, we propose one more with the generalization of the beta prime regression model. We consider the possibility of dependence between observations of the same unit. Residuals and diagnostic tools also are discussed. To evaluate the finite-sample performance of the estimators, using different correlation matrices and distributions, we conducted a Monte Carlo simulation study. The methodology proposed is illustrated with an analysis of a real data set. Finally, we create an R package for easy access to publicly available the methodology described in this paper.     

R 2 Measures Based on Wald and Likelihood Ratio Joint Significance Tests

Two methods are suggested for generating R ² measures for a wide class of models. These measures are linked to the R ² of the standard linear regression model through Wald and likelihood ratio statistics for testing the joint significance of the explanatory variables. Some currently used R ²'s are shown to be special cases of these methods.     

Diagnostic Measures for Generalized Linear Models with Missing Covariates

Abstract.  In this paper, we carry out an in-depth investigation of diagnostic measures for assessing the influence of observations and model misspecification in the presence of missing covariate data for generalized linear models. Our diagnostic measures include case-deletion measures and conditional residuals. We use the conditional residuals to construct goodness-of-fit statistics for testing possible misspecifications in model assumptions, including the sampling distribution. We develop specific strategies for incorporating missing data into goodness-of-fit statistics in order to increase the power of detecting model misspecification. A resampling method is proposed to approximate the p -value of the goodness-of-fit statistics. Simulation studies are conducted to evaluate our methods and a real data set is analysed to illustrate the use of our various diagnostic measures.     

Higher-order asymptotics under model misspecification

Most of the higher-order asymptotic results in statistical inference available in the literature assume model correctness. The aim of this paper is to develop higher-order results under model misspecification. The density functions to O(n^?3/2) of the robust score test statistic and the robust Wald test statistic are derived under the null hypothesis, for the scalar as well as the multiparameter case. Alternate statistics which are robust to O(n^?3/2) are also proposed.     

A comparison of the Hosmer–Lemeshow,Pigeon–Heyse,and Tsiatis goodness-of-fit tests for binary logistic regression under two grouping methods

Algebraic relationships between Hosmer–Lemeshow (HL), Pigeon–Heyse (J²), and Tsiatis (T) goodness-of-fit statistics for binary logistic regression models with continuous covariates were investigated, and their distributional properties and performances studied using simulations. Groups were formed under deciles-of-risk (DOR) and partition-covariate-space (PCS) methods. Under DOR, HL and T followed reported null distributions, while J² did not. Under PCS, only T followed its reported null distribution, with HL and J² dependent on model covariate number and partitioning. Generally, all had similar power. Of the three, T performed best, maintaining Type-I error rates and having a distribution invariant to covariate characteristics, number, and partitioning.     

Goodness-of-fit tests for location–scale families based on random distance

For location–scale families, we consider a random distance between the sample order statistics and the quasi sample order statistics derived from the null distribution as a measure of discrepancy. The conditional qth quantile and expectation of the random discrepancy on the given sample are chosen as test statistics. Simulation results of powers against various alternatives are illustrated under the normal and exponential hypotheses for moderate sample size. The proposed tests, especially the qth quantile tests with a small or large q, are shown to be more powerful than other prominent goodness-of-fit tests in most cases.     

Goodness-of-fit tests for progressively Type-II censored data from location–scale distributions

In this article, we propose several goodness-of-fit methods for location–scale families of distributions under progressively Type-II censored data. The new tests are based on order statistics and sample spacings. We assess the performance of the proposed tests for the normal and Gumbel models against several alternatives by means of Monte Carlo simulations. It has been observed that the proposed tests are quite powerful in comparison with an existing goodness-of-fit test proposed for progressively Type-II censored data by Balakrishnan et al. [Goodness-of-fit tests based on spacings for progressively Type-II censored data from a general location–scale distribution, IEEE Trans. Reliab. 53 (2004), pp. 349–356]. Finally, we illustrate the proposed goodness-of-fit tests using two real data from reliability literature.     

Tests for the multivariate two-sample problem based on empirical probability measures

Nonparametric tests are proposed for the equality of two unknown p-variate distributions. Empirical probability measures are defined from samples from the two distributions and used to construct test statistics as the supremum of the absolute differences between empirical probabilities, the supremum being taken over all possible events. The test statistics are truly multivariate in not requiring the artificial ranking of multivariate observations, and they are distribution-free in the general p-variate case. Asymptotic null distributions are obtained. Powers of the proposed tests and a competitor are examined by Monte Carlo techniques.     

Approximations of the Distributions of Test Statistics for Homogeneity of a Product Multinomial Model

Statistics R ^a based on power divergence can be used for testing the homogeneity of a product multinomial model. All R ^a have the same chi-square limiting distribution under the null hypothesis of homogeneity. R ⁰ is the log likelihood ratio statistic and R ¹ is Pearson's X ² statistic. In this article, we consider improvement of approximation of the distribution of R ^a under the homogeneity hypothesis. The expression of the asymptotic expansion of distribution of R ^a under the homogeneity hypothesis is investigated. The expression consists of continuous and discontinuous terms. Using the continuous term of the expression, a new approximation of the distribution of R ^a is proposed. A moment-corrected type of chi-square approximation is also derived. By numerical comparison, we show that both of the approximations perform much better than that of usual chi-square approximation for the statistics R ^a when a ≤ 0, which include the log likelihood ratio statistic.     

Testing for spherical symmetry via the empirical characteristic function

Kolmogorov–Smirnov-type and Cramér–von Mises-type goodness-of-fit tests are proposed for the null hypothesis that the distribution of a random vector X is spherically symmetric. The test statistics utilize the fact that X has a spherical symmetric distribution if, and only if, the characteristic function of X is constant over surfaces of spheres centred at the origin. Both tests come in convenient forms that are straightforwardly applicable with the computer. The asymptotic null distribution of the test statistics as well as the consistency of the tests is investigated under general conditions. Since both the finite sample and the asymptotic null distribution depend on the unknown distribution of the Euclidean norm of X, a conditional Monte Carlo procedure is used to actually carry out the tests. Results on the behaviour of the test in finite-samples are included along with a real-data example.     

Goodness‐of‐fit Test for Directional Data

In this paper, we study the problem of testing the hypothesis on whether the density f of a random variable on a sphere belongs to a given parametric class of densities. We propose two test statistics based on the L² and L¹ distances between a non‐parametric density estimator adapted to circular data and a smoothed version of the specified density. The asymptotic distribution of the L² test statistic is provided under the null hypothesis and contiguous alternatives. We also consider a bootstrap method to approximate the distribution of both test statistics. Through a simulation study, we explore the moderate sample performance of the proposed tests under the null hypothesis and under different alternatives. Finally, the procedure is illustrated by analysing a real data set based on wind direction measurements.     

A New Measure of Fit for Equations With Dichotomous Dependent Variables

The econometrics literature contains many alternative measures of goodness of fit, roughly analogous to R ², for use with equations with dichotomous dependent variables. There is, however, no consensus as to the measures' relative merits or about which ones should be reported in empirical work. This article proposes a new measure that possesses several useful properties that the other measures lack. The new measure may be interpreted intuitively in a similar way to R ² in the linear regression context.     

Hypothesis testing in smoothing spline models

Nonparametric regression models are often used to check or suggest a parametric model. Several methods have been proposed to test the hypothesis of a parametric regression function against an alternative smoothing spline model. Some tests such as the locally most powerful (LMP) test by Cox et al. (Cox, D., Koh, E., Wahba, G. and Yandell, B. (1988). Testing the (parametric) null model hypothesis in (semiparametric) partial and generalized spline models. Ann. Stat., 16, 113–119.), the generalized maximum likelihood (GML) ratio test and the generalized cross validation (GCV) test by Wahba (Wahba, G. (1990). Spline models for observational data. CBMS-NSF Regional Conference Series in Applied Mathematics, SIAM.) were developed from the corresponding Bayesian models. Their frequentist properties have not been studied. We conduct simulations to evaluate and compare finite sample performances. Simulation results show that the performances of these tests depend on the shape of the true function. The LMP and GML tests are more powerful for low frequency functions while the GCV test is more powerful for high frequency functions. For all test statistics, distributions under the null hypothesis are complicated. Computationally intensive Monte Carlo methods can be used to calculate null distributions. We also propose approximations to these null distributions and evaluate their performances by simulations.     

Negative exponential disparity based family of goodness-of-fit tests for multinomial models

This paper investigates a new family of goodness-of-fit tests based on the negative exponential disparities. This family includes the popular Pearson's chi-square as a member and is a subclass of the general class of disparity tests (Basu and Sarkar, 1994) which also contains the family of power divergence statistics. Pitman efficiency and finite sample power comparisons between different members of this new family are made. Three asymptotic approximations of the exact null distributions of the negative exponential disparity famiiy of tests are discussed. Some numerical results on the small sample perfomance of this family of tests are presented for the symmetric null hypothesis. It is shown that the negative exponential disparity famiiy, Like the power divergence family, produces a new goodness-of-fit test statistic that can be a very attractive alternative to the Pearson's chi-square. Some numerical results suggest that, application of this test statistic, as an alternative to Pearson's chi-square, could be preferable to the I ^2/3 statistic of Cressie and Read (1984) under the use of chi-square critical values.     

A generalized Q–Q plot for longitudinal data

Most biomedical research is carried out using longitudinal studies. The method of generalized estimating equations (GEEs) introduced by Liang and Zeger [Longitudinal data analysis using generalized linear models, Biometrika 73 (1986), pp. 13–22] and Zeger and Liang [Longitudinal data analysis for discrete and continuous outcomes, Biometrics 42 (1986), pp. 121–130] has become a standard method for analyzing non-normal longitudinal data. Since then, a large variety of GEEs have been proposed. However, the model diagnostic problem has not been explored intensively. Oh et al. [Modeldiagnostic plots for repeated measures data using the generalized estimating equations approach, Comput. Statist. Data Anal. 53 (2008), pp. 222–232] proposed residual plots based on the quantile–quantile (Q–Q) plots of the χ²-distribution for repeated-measures data using the GEE methodology. They considered the Pearson, Anscombe and deviance residuals. In this work, we propose to extend this graphical diagnostic using a generalized residual. A simulation study is presented as well as two examples illustrating the proposed generalized Q–Q plots.     

Testing Serial Correlation in Partially Linear Single-Index Errors-in-Variables Models

In this article, we propose two test statistics for testing the underlying serial correlation in a partially linear single-index model Y = η(Z ^τα) + X ^τβ + ? when X is measured with additive error. The proposed test statistics are shown to have asymptotic normal or chi-squared distributions under the null hypothesis of no serial correlation. Monte Carlo experiments are also conducted to illustrate the finite sample performance of the proposed test statistics. The simulation results confirm that these statistics perform satisfactorily in both estimated sizes and powers.     

Default Bayesian goodness-of-fit tests for the skew-normal model

In this paper we propose a series of goodness-of-fit tests for the family of skew-normal models when all parameters are unknown. As the null distributions of the considered test statistics depend only on asymmetry parameter, we used a default and proper prior on skewness parameter leading to the prior predictive p-value advocated by G. Box. Goodness-of-fit tests, here proposed, depend only on sample size and exhibit full agreement between nominal and actual size. They also have good power against local alternative models which also account for asymmetry in the data.     

Comparing mean ranks for repeated measures data

Rank tests are considered that compare t treatments in repeated measures designs. A statistic is given that contains as special cases several that have been proposed for this problem, including one that corresponds to the randomized block ANOVA statistic applied to the rank transformed data. Another statistic is proposed, having a null distribution holding under more general conditions, that is the rank transform of the Hotelling statistic for repeated measures. A statistic of this type is also given for data that are ordered categorical rather than fully rankedo Unlike the Friedman statistic, the statistics discussed in this article utilize a single ranking of the entire sample. Power calculations for an underlying normal distribution indicate that the rank transformed ANOVA test can be substantially more powerful than the Friedman test.