Comparison of Type I error rates and statistical power of different propensity score methods

Propensity score analysis (PSA) is a technique to correct for potential confounding in observational studies. Covariate adjustment, matching, stratification, and inverse weighting are the four most commonly used methods involving propensity scores. The main goal of this research is to determine which PSA method performs the best in terms of protecting against spurious association detection, as measured by Type I error rate, while maintaining sufficient power to detect a true association, if one exists. An examination of these PSA methods along with ordinary least squares regression was conducted under two cases: correct PSA model specification and incorrect PSA model specification. PSA covariate adjustment and PSA matching maintain the nominal Type I error rate, when the PSA model is correctly specified, but only PSA covariate adjustment achieves adequate power levels. Other methods produced conservative Type I Errors in some scenarios, while liberal Type I error rates were observed in other scenarios.     

Structural vector autoregressive analysis for cointegrated variables

Summary Vector autoregressive (VAR) models are capable of capturing the dynamic structure of many time series variables. Impulse response functions are typically used to investigate the relationships between the variables included in such models. In this context the relevant impulses or innovations or shocks to be traced out in an impulse response analysis have to be specified by imposing appropriate identifying restrictions. Taking into account the cointegration structure of the variables offers interesting possibilities for imposing identifying restrictions. Therefore VAR models which explicitly take into account the cointegration structure of the variables, so-called vector error correction models, are considered. Specification, estimation and validation of reduced form vector error correction models is briefly outlined and imposing structural short- and long-run restrictions within these models is discussed. I thank an anonymous reader for comments on an earlier draft of this paper that helped me to improve the exposition.     

空间面板杜宾模型的Bootstrap Wald-COMFAC检验研究

将共同因子约束(COMFAC)的Wald检验问题引入到空间面板模型中,讨论空间面板杜宾模型与空间面板误差模型的识别问题。蒙特卡洛模拟表明:在有限样本下,基于渐近临界值的Wald检验有着良好的检验功效,但存在着较为严重的尺度扭曲。进一步采用残差Bootstrap方法,在不损失检验功效的前提下,能够显著地降低检验的尺度扭曲。因此,残差Bootstrap方法是更为有效的检验方法。     

Parametric simultaneous robust inferences for regression coefficient under generalized linear models

In this article, the parametric robust regression approaches are proposed for making inferences about regression parameters in the setting of generalized linear models (GLMs). The proposed methods are able to test hypotheses on the regression coefficients in the misspecified GLMs. More specifically, it is demonstrated that with large samples, the normal and gamma regression models can be properly adjusted to become asymptotically valid for inferences about regression parameters under model misspecification. These adjusted regression models can provide the correct type I and II error probabilities and the correct coverage probability for continuous data, as long as the true underlying distributions have finite second moments.     

A shared component model for detecting joint and selective clustering of two diseases

The study of spatial variations in disease rates is a common epidemiological approach used to describe the geographical clustering of diseases and to generate hypotheses about the possible 'causes' which could explain apparent differences in risk. Recent statistical and computational developments have led to the use of realistically complex models to account for overdispersion and spatial correlation. However, these developments have focused almost exclusively on spatial modelling of a single disease. Many diseases share common risk factors (smoking being an obvious example) and, if similar patterns of geographical variation of related diseases can be identified, this may provide more convincing evidence of real clustering in the underlying risk surface. We propose a shared component model for the joint spatial analysis of two diseases. The key idea is to separate the underlying risk surface for each disease into a shared and a disease-specific component. The various components of this formulation are modelled simultaneously by using spatial cluster models implemented via reversible jump Markov chain Monte Carlo methods. We illustrate the methodology through an analysis of oral and oesophageal cancer mortality in the 544 districts of Germany, 1986–1990.     

Power and sample size when multiple endpoints are considered

A common approach to analysing clinical trials with multiple outcomes is to control the probability for the trial as a whole of making at least one incorrect positive finding under any configuration of true and false null hypotheses. Popular approaches are to use Bonferroni corrections or structured approaches such as, for example, closed-test procedures. As is well known, such strategies, which control the family-wise error rate, typically reduce the type I error for some or all the tests of the various null hypotheses to below the nominal level. In consequence, there is generally a loss of power for individual tests. What is less well appreciated, perhaps, is that depending on approach and circumstances, the test-wise loss of power does not necessarily lead to a family wise loss of power. In fact, it may be possible to increase the overall power of a trial by carrying out tests on multiple outcomes without increasing the probability of making at least one type I error when all null hypotheses are true. We examine two types of problems to illustrate this. Unstructured testing problems arise typically (but not exclusively) when many outcomes are being measured. We consider the case of more than two hypotheses when a Bonferroni approach is being applied while for illustration we assume compound symmetry to hold for the correlation of all variables. Using the device of a latent variable it is easy to show that power is not reduced as the number of variables tested increases, provided that the common correlation coefficient is not too high (say less than 0.75). Afterwards, we will consider structured testing problems. Here, multiplicity problems arising from the comparison of more than two treatments, as opposed to more than one measurement, are typical. We conduct a numerical study and conclude again that power is not reduced as the number of tested variables increases.     

The relative efficiency of the restricted estimators in linear regression models

This paper deals with the problem of multicollinearity in a multiple linear regression model with linear equality restrictions. The restricted two parameter estimator which was proposed in case of multicollinearity satisfies the restrictions. The performance of the restricted two parameter estimator over the restricted least squares (RLS) estimator and the ordinary least squares (OLS) estimator is examined under the mean square error (MSE) matrix criterion when the restrictions are correct and not correct. The necessary and sufficient conditions for the restricted ridge regression, restricted Liu and restricted shrunken estimators, which are the special cases of the restricted two parameter estimator, to have a smaller MSE matrix than the RLS and the OLS estimators are derived when the restrictions hold true and do not hold true. Theoretical results are illustrated with numerical examples based on Webster, Gunst and Mason data and Gorman and Toman data. We conduct a final demonstration of the performance of the estimators by running a Monte Carlo simulation which shows that when the variance of the error term and the correlation between the explanatory variables are large, the restricted two parameter estimator performs better than the RLS estimator and the OLS estimator under the configurations examined.     

The probability to select the correct model using likelihood-ratio based criteria in choosing between two nested models of which the more extended one is true

The probability to select the correct model is calculated for likelihood-ratio-based criteria to compare two nested models. If the more extended of the two models is true, the difference between twice the maximised log-likelihoods is approximately noncentral chi-square distributed with d.f. the difference in the number of parameters. The noncentrality parameter of this noncentral chi-square distribution can be approximated by twice the minimum Kullback–Leibler divergence (MKLD) of the best-fitting simple model to the true version of the extended model.The MKLD, and therefore the probability to select the correct model increases approximately proportionally to the number of observations if all observations are performed under the same conditions. If a new set of observations can only be performed under different conditions, the model parameters may depend on the conditions, and therefore have to be estimated for each set of observations separately. An increase in observations will then go together with an increase in the number of model parameters. In this case, the power of the likelihood-ratio test will increase with an increasing number of observations. However, the probability to choose the correct model with the AIC will only increase if for each set of observations the MKLD is more than 0.5. If the MKLD is less than 0.5, that probability will decrease. The probability to choose the correct model with the BIC will always decrease, sometimes after an initial increase for a small number of observation sets. The results are illustrated by a simulation study with a set of five nested nonlinear models for binary data.     

Utilizing the Flexibility of the Epsilon-Skew-Normal Distribution for Common Regression Problems

In this paper we illustrate the properties of the epsilon-skew-normal (ESN) distribution with respect to developing more flexible regression models. The ESN model is a simple one-parameter extension of the standard normal model. The additional parameter ~ corresponds to the degree of skewness in the model. In the fitting process we take advantage of relatively new powerful routines that are now available in standard software packages such as SAS. It is illustrated that even if the true underlying error distribution is exactly normal there is no practical loss n power with respect to testing for non-zero regression coefficients. If the true underlying error distribution is slightly skewed, the ESN model is superior in terms of statistical power for tests about the regression coefficient. This model has good asymptotic properties for samples of size n>50.     

Controlling type I error in the reference‐scaled bioequivalence evaluation of highly variable drugs

Reference‐scaled average bioequivalence (RSABE) approaches for highly variable drugs are based on linearly scaling the bioequivalence limits according to the reference formulation within‐subject variability. RSABE methods have type I error control problems around the value where the limits change from constant to scaled. In all these methods, the probability of type I error has only one absolute maximum at this switching variability value. This allows adjusting the significance level to obtain statistically correct procedures (that is, those in which the probability of type I error remains below the nominal significance level), at the expense of some potential power loss. In this paper, we explore adjustments to the EMA and FDA regulatory RSABE approaches, and to a possible improvement of the original EMA method, designated as HoweEMA. The resulting adjusted methods are completely correct with respect to type I error probability. The power loss is generally small and tends to become irrelevant for moderately large (affordable in real studies) sample sizes.     

Selecting the Optimal Transformation of a Continuous Covariate in Cox's Regression: Implications for Hypothesis Testing

ABSTRACT

Background: Many exposures in epidemiological studies have nonlinear effects and the problem is to choose an appropriate functional relationship between such exposures and the outcome. One common approach is to investigate several parametric transformations of the covariate of interest, and to select a posteriori the function that fits the data the best. However, such approach may result in an inflated Type I error. Methods: Through a simulation study, we generated data from Cox's models with different transformations of a single continuous covariate. We investigated the Type I error rate and the power of the likelihood ratio test (LRT) corresponding to three different procedures that considered the same set of parametric dose-response functions. The first unconditional approach did not involve any model selection, while the second conditional approach was based on a posteriori selection of the parametric function. The proposed third approach was similar to the second except that it used a corrected critical value for the LRT to ensure a correct Type I error. Results: The Type I error rate of the second approach was two times higher than the nominal size. For simple monotone dose-response, the corrected test had similar power as the unconditional approach, while for non monotone, dose-response, it had a higher power. A real-life application that focused on the effect of body mass index on the risk of coronary heart disease death, illustrated the advantage of the proposed approach. Conclusion: Our results confirm that a posteriori selecting the functional form of the dose-response induces a Type I error inflation. The corrected procedure, which can be applied in a wide range of situations, may provide a good trade-off between Type I error and power.     

Impact of unbalanced DIF item proportions on group-specific DIF identification

Differences in type I error and power rates for majority and minority groups are investigated when differential item functioning (DIF) contamination in a test is unbalanced. Typically, type I error and power rates are aggregated across groups, however cumulative results can be misleading if subgroups are affected differently by study conditions. With unbalanced DIF contamination, type I error and power rates are reduced for groups with more DIF items favoring them, and increased for groups with less DIF contamination. Even when aggregated impacts appear small, differing subgroup impacts can result in a larger proportional bias than in the original data.     

Using marginal structural models to analyze the impact of subsequent therapy on the treatment effect in survival data: Simulations and clinical trial examples

We explore the impact of time-varying subsequent therapy on the statistical power and treatment effects in survival analysis. The marginal structural model (MSM) with stabilized inverse probability treatment weights (sIPTW) was used to account for the effects due to the subsequent therapy. Simulations were performed to compare the MSM-sIPTW method with the conventional method without accounting for the time-varying covariate such as subsequent therapy that is dependent on the initial response of the treatment effect. The results of the simulations indicated that the statistical power, thereby the Type I error, of the trials to detect the frontline treatment effect could be inflated if no appropriate adjustment was made for the impact due to the add-on effects of the subsequent therapy. Correspondingly, the hazard ratio between the treatment groups may be overestimated by the conventional analysis methods. In contrast, MSM-sIPTW can maintain the Type I error rate and gave unbiased estimates of the hazard ratio for the treatment. Two real examples were used to discuss the potential clinical implications. The study demonstrated the importance of accounting for time-varying subsequent therapy for obtaining unbiased interpretation of data.     

Statistical inference for a semiparametric measurement error regression model with heteroscedastic errors

Efficient inference for regression models requires that the heteroscedasticity be taken into account. We consider statistical inference under heteroscedasticity in a semiparametric measurement error regression model, in which some covariates are measured with errors. This paper has multiple components. First, we propose a new method for testing the heteroscedasticity. The advantages of the proposed method over the existing ones are that it does not need any nonparametric estimation and does not involve any mismeasured variables. Second, we propose a new two-step estimator for the error variances if there is heteroscedasticity. Finally, we propose a weighted estimating equation-based estimator (WEEBE) for the regression coefficients and establish its asymptotic properties. Compared with existing estimators, the proposed WEEBE is asymptotically more efficient, avoids undersmoothing the regressor functions and requires less restrictions on the observed regressors. Simulation studies show that the proposed test procedure and estimators have nice finite sample performance. A real data set is used to illustrate the utility of our proposed methods.     

Testing exogeneity in overidentified models

Summary In this paper we analyse the consequences of model overidentification on testing exogeneity, when maximum likelihood techniques for estimation and inference are used. This situation is viewed as a particular case of the more general problem of considering how restrictions on nuisance parameters could help in making inference on the parameters of interest. At first a general model is considered. A suitable likelihood function factorization is used which allows a simple derivation of the information matrix and others tools useful for building up joint tests of exogeneity and overidentifying restrictions both of Wald and Lagrange Multiplier type. The asymptotic local power of the exogeneity test in the justidentified model is compared with that in the overidentified one, when we assume that the latter is the true model. Then the pseudo-likelihood framework is used to derive the consequences of working with a model where overidentifying restrictions are erroneously imposed. The inconsistency introduced by imposing false restrictions is analysed and the consequences of the misspecification on the exogeneity test are carefully examined.     

Robust Small Sample Inference for Generalised Estimating Equations: An Application of the Anova‐type Test

Asymptotically, the Wald‐type test for generalised estimating equations (GEE) models can control the type I error rate at the nominal level. However in small sample studies, it may lead to inflated type I error rates. Even with currently available small sample corrections for the GEE Wald‐type test, the type I error rate inflation is still serious when the tested contrast is multidimensional. This paper extends the ANOVA‐type test for heteroscedastic factorial designs to GEE and shows that the proposed ANOVA‐type test can also control the type I error rate at the nominal level in small sample studies while still maintaining robustness with respect to mis‐specification of the working correlation matrix. Differences of inference between the Wald‐type test and the proposed test are observed in a two‐way repeated measures ANOVA model for a diet‐induced obesity study and a two‐way repeated measures logistic regression for a collagen‐induced arthritis study. Simulation studies confirm that the proposed test has better control of the type I error rate than the Wald‐type test in small sample repeated measures models. Additional simulation studies further show that the proposed test can even achieve larger power than the Wald‐type test in some cases of the large sample repeated measures ANOVA models that were investigated.     

A generalisation of T‐optimality for discriminating between competing models with an application to pharmacokinetic studies

The T‐optimality criterion is used in optimal design to derive designs for model selection. To set up the method, it is required that one of the models is considered to be true. We term this local T‐optimality. In this work, we propose a generalisation of T‐optimality (termed robust T‐optimality) that relaxes the requirement that one of the candidate models is set as true. We then show an application to a nonlinear mixed effects model with two candidate non‐nested models and combine robust T‐optimality with robust D‐optimality. Optimal design under local T‐optimality was found to provide adequate power when the a priori assumed true model was the true model but poor power if the a priori assumed true model was not the true model. The robust T‐optimality method provided adequate power irrespective of which model was true. The robust T‐optimality method appears to have useful properties for nonlinear models, where both the parameter values and model structure are required to be known a priori, and the most likely model that would be applied to any new experiment is not known with certainty. Copyright © 2012 John Wiley & Sons, Ltd.     

空间经济计量滞后模型Bootstrap Moran检验功效的模拟分析

 当误差项不服从独立同分布时,利用Moran’s I统计量的渐近检验,无法有效判断空间经济计量滞后模型2SLS估计残差间存在空间关系与否。本文采用两种基于残差的Bootstrap方法,诊断空间经济计量滞后模型残差中的空间相关关系。大量Monte Carlo模拟结果显示,从功效角度看,无论误差项服从独立同分布与否,与渐近检验相比,Bootstrap Moran检验都具有更好的有限样本性质,能够更有效地进行空间相关性检验。尤其是,在样本量较小和空间衔接密度较高情况下,Bootstrap Moran检验的功效显著大于渐近检验。     

Near exogeneity,weak identification and specification testing: Some asymptotic results

The paper studies the asymptotic size property of various specification tests in linear structural models where instrumental variables may locally violate the exclusion restrictions. Our results provide some new insights and extensions of earlier studies. In particular, we derive an explicit formula of the asymptotic size of the tests which shows clearly the factors that influence their size under instrument endogeneity. We show that all tests have correct asymptotic size when the usual orthogonality condition holds, but their asymptotic size can be arbitrary large even if only one instrument is slightly correlated with the error term. We present a Monte Carlo experiment that confirms our theoretical findings.     

NEIGHBOUR ANALYSIS WITH ADJUSTMENT FOR INTERPLOT COMPETITION

In field experiments involving a large number of experimental plots, a neighbour analysis can be used to control environmental variation by estimating the trend within blocks. The effect of interplot competition is another important source of variation which has an influence on the estimation of treatment contrasts. To reduce the effect of the variation from these sources and to improve the precision of comparison between treatments, a spatial model is proposed for incorporating both trend effect and interplot competition. It is a modification to the residual maximum likelihood neighbour analysis of Gleeson & Cullis (1987) using the two neighbouring treatment effects to estimate interplot competition. A real example is used to illustrate this methodology. The results indicate that the extended model gives no appreciable difference in standard error of mean differences compared with the model taking into account the trend effect only. However, the rankings of estimated treatment means do differ. More research using both real and simulated data is required before such models that incorporate trend and competition effects can be confidently recommended.