On making causal claims: A review and recommendations

Social scientists often estimate models from correlational data, where the independent variable has not been exogenously manipulated; they also make implicit or explicit causal claims based on these models. When can these claims be made? We answer this question by first discussing design and estimation conditions under which model estimates can be interpreted, using the randomized experiment as the gold standard. We show how endogeneity – which includes omitted variables, omitted selection, simultaneity, common-method variance, and measurement error – renders estimates causally uninterpretable. Second, we present methods that allow researchers to test causal claims in situations where randomization is not possible or when causal interpretation could be confounded; these methods include fixed-effects panel, sample selection, instrumental variable, regression discontinuity, and difference-in-differences models. Third, we take stock of the methodological rigor with which causal claims are being made in a social sciences discipline by reviewing a representative sample of 110 articles on leadership published in the previous 10 years in top-tier journals. Our key finding is that researchers fail to address at least 66% and up to 90% of design and estimation conditions that make causal claims invalid. We conclude by offering 10 suggestions on how to improve non-experimental research.     

On Causal Inferences for Personalized Medicine: How Hidden Causal Assumptions Led to Erroneous Causal Claims About the D-Value

Abstract

Personalized medicine asks if a new treatment will help a particular patient, rather than if it improves the average response in a population. Without a causal model to distinguish these questions, interpretational mistakes arise. These mistakes are seen in an article by Demidenko that recommends the “D-value,” which is the probability that a randomly chosen person from the new-treatment group has a higher value for the outcome than a randomly chosen person from the control-treatment group. The abstract states “The D-value has a clear interpretation as the proportion of patients who get worse after the treatment” with similar assertions appearing later. We show these statements are incorrect because they require assumptions about the potential outcomes which are neither testable in randomized experiments nor plausible in general. The D-value will not equal the proportion of patients who get worse after treatment if (as expected) those outcomes are correlated. Independence of potential outcomes is unrealistic and eliminates any personalized treatment effects; with dependence, the D-value can even imply treatment is better than control even though most patients are harmed by the treatment. Thus, D-values are misleading for personalized medicine. To prevent misunderstandings, we advise incorporating causal models into basic statistics education.     

A test of the normality assumption in ordered probit model

This paper presents a Lagrange multiplier test of the normality assumption underlying the ordered probit model. The test is presented both for the standard ordered probit model and a version in which censoring is present in the dependent variable. The test is then compared to normality tests proposed here compares favorably to tests based on artificial regression techinques.     

Separating trend and cyclical dynamics in state space models with exogenous inputs

In the present study we compare three state rotation methods in modelling the impact of the US economy on the Finnish economy, i.e. Schur decomposition, eigenvalue analysis and singular value decomposition. Singular value decomposition is seen to provide a robust approximation of the state rotation in most cases studied, irrespective of whether the characteristic roots of the state transition matrix are complex. Thus, singular value decomposition seems to be a viable computational device not only in estimating the system matrices of the state space model, but also in state rotation, as compared to the more involved techniques based on eigenvalue analysis or Schur decomposition.     

A monte carlo comparison of tests based on the durbin-watson statistic with other autocorrelation tests in dynamic models

This paper examines the sampling properties of a number of serial correlation tests in dynamic linear models which include one or two lags of the dependent variable. Among the tests considered are the Durbin-Watson (DW) bounds test, modified versions of the DW proposed recently by King and Wu and Inder, Durbin's m test, Inder's point optimal test and a Hausman type test. Sampling designs include models with one or two lags of the dependent variable. The m, Hausman, and Inder's tests have the best performance, while Inder's modified DW test appears to be better than the other DW based tests. Results also suggest that tests are less powerful and more sensitive to design parameters in models with higher dynamics, with the DW-based tests being the most sensitive.     

Bootstrapping a consistent nonparametric goodness-of-fit test

In this paper, we employ the parametric bootstrap to approximate the finite sample distribution of a goodness-of-fit test statistic in Fan (1994). We show that the proposed bootstrap procedure works in that the bootstrap distribution conditional on the random sample tends to the asymptotic distribution of the test statistic in probability. A simulation study demonstrates that the bootstrap approximation works extremely well in small samples with only 25 observations and is very robust to the value of the smoothing parameter in the kernel density estimation.     

地方保护主义和地区间贸易壁垒的检验性分析

随着市场经济的发展,贸易作为经济运行中的一个环节,其重要性越来越凸现出来,而阻碍贸易正常进行的地方保护主义和贸易壁垒对宏观经济的良性发展起到了消极的作用。准确把握我国地方保护主义和地区间贸易壁垒的现状意义重大。本文基于国务院发展研究中心发展战略和区域经济研究部与北京大学光华管理学院所作的“地区间经济联系障碍的调查问卷”,对前期回收的有效样本,运用统计检验分析的方法,就当前地方保护和贸易壁垒的存在性以及企业的不同类型是否与其面临的地方保护和贸易壁垒有程度上的关联等问题做了相关研究,认为企业的规模和所有制性质对企业面临的地方保护和贸易壁垒基本没有程度上的影响;同时,统计分析结果还表明目前国内地区间贸易壁垒偏重于无形和隐形的形式。     

The application of the durbin-watson test to the dynamic regression model under normal and non-normal errors

Until recently, a difficulty with applying the Durbin-Watson (DW) test to the dynamic linear regression model has been the lack of appropriate critical values. Inder (1986) used a modified small-disturbance distribution (SDD) to find approximate critical values. King and Wu (1991) showed that the exact SDD of the DW statistic is equivalent to the distribution of the DW statistic from the regression with the lagged dependent variables replaced by their means. Unfortunately, these means are unknown although they could be estimated by the actual variable values. This provides a justification for using the exact critical values of the DW statistic from the regression with the lagged dependent variables treated as non-stochastic regressors. Extensive Monte Carlo experiments are reported in this paper. They show that this approach leads to reasonably accurate critical values, particularly when two lags of the dependent variable are present. Robustness to non-normality is also investigated.     

Extending the Barnard’s test to non-inferiority

In 1945, George Alfred Barnard presented an unconditional exact test to compare two independent proportions. Critical regions for this test, by construction accomplish the very useful property of being Barnard convex sets. Besides, there are empirical findings suggesting that Barnard’s test is the most generally powerful. For Barnard’s test, calculation of critical regions is complicated due that they are constructed in an iterative form until is obtained a test size, as close as possible to the nominal significance level and less than or equal to it. In this article we present an extension to non-inferiority of this very leading test. This extension was contructed for any dissimilarity measure and tables were constructed for the difference between proportions. Also we calculate the critical regions for this extended test for sample sizes less or equal than 30, nominal significance level 0.01, 0.025, 0.05, and 0.10 and for non-inferiority margins 0.05, 0.10, 0.15, and 0.20. Additionally, we computed test sizes for the mentioned configurations. To do this calculations, we have written a program in the R environment.     

Testing hypotheses about covariance matrices using bootstrap methods

The usual chi-squared approximation to test statistics based on normal theory for testing covariance structures of multivariate populations is very sensitive to the normality assumption. Two general bootstrap procedures are developed in this paper to obtain approximately valid critical values for these test statistics when the data are not normally distributed. The first is based on separate sampling from individual samples, and the second is based on sampling from pooled samples. Although the second method requires more assumptions, its small sample properties are better.