Estimator Choice and Fisher's Paradox: A Monte Carlo Study

Abstract

This paper argues that Fisher's paradox can be explained away in terms of estimator choice. We analyse by means of Monte Carlo experiments the small sample properties of a large set of estimators (including virtually all available single-equation estimators), and compute the critical values based on the empirical distributions of the t-statistics, for a variety of Data Generation Processes (DGPs), allowing for structural breaks, ARCH effects etc. We show that precisely the estimators most commonly used in the literature, namely OLS, Dynamic OLS (DOLS) and non-prewhitened FMLS, have the worst performance in small samples, and produce rejections of the Fisher hypothesis. If one employs the estimators with the most desirable properties (i.e., the smallest downward bias and the minimum shift in the distribution of the associated t-statistics), or if one uses the empirical critical values, the evidence based on US data is strongly supportive of the Fisher relation, consistently with many theoretical models.     

A comparison of some robust, adaptive, and partially adaptive estimators of regression models

Numerous estimation techniques for regression models have been proposed. These procedures differ in how sample information is used in the estimation procedure. The efficiency of least squares (OLS) estimators implicity assumes normally distributed residuals and is very sensitive to departures from normality, particularly to "outliers" and thick-tailed distributions. Lead absolute deviation (LAD) estimators are less sensitive to outliers and are optimal for laplace random disturbances, but not for normal errors. This paper reports monte carlo comparisons of OLS,LAD, two robust estimators discussed by huber, three partially adaptiveestimators, newey's generalized method of moments estimator, and an adaptive maximum likelihood estimator based on a normal kernal studied by manski. This paper is the first to compare the relative performance of some adaptive robust estimators (partially adaptive and adaptive procedures) with some common nonadaptive robust estimators. The partially adaptive estimators are based on three flxible parametric distributions for the errors. These include the power exponential (Box-Tiao) and generalized t distributions, as well as a distribution for the errors, which is not necessarily symmetric. The adaptive procedures are "fully iterative" rather than one step estimators. The adaptive estimators have desirable large sample properties, but these properties do not necessarily carry over to the small sample case.

The monte carlo comparisons of the alternative estimators are based on four different specifications for the error distribution: a normal, a mixture of normals (or variance-contaminated normal), a bimodal mixture of normals, and a lognormal. Five hundred samples of 50 are used. The adaptive and partially adaptive estimators perform very well relative to the other estimation procedures considered, and preliminary results suggest that in some important cases they can perform much better than OLS with 50 to 80% reductions in standard errors.     

A Note on Bayesian Analyses of Capture–Recapture Data with Perfect Recaptures

The present article deals with the problem of misspecifying the disturbance-covariance matrix as scalar, when it is locally non scalar. We consider a family of shrinkage estimators based on OLS estimator and compare its asymptotic properties with the properties of OLS estimator. We proposed a similar family of estimators based on FGLS and compared its asymptotic properties with the shrinkage estimator based on OLS under a Pitman's drift process. The effect of misspecifying the disturbances covariance matrix was analyzed with the help of a numerical simulation.     

A comparison of some robust,adaptive, and partially adaptive estimators of regression models

Numerous estimation techniques for regression models have been proposed. These procedures differ in how sample information is used in the estimation procedure. The efficiency of least squares (OLS) estimators implicity assumes normally distributed residuals and is very sensitive to departures from normality, particularly to "outliers" and thick-tailed distributions. Lead absolute deviation (LAD) estimators are less sensitive to outliers and are optimal for laplace random disturbances, but not for normal errors. This paper reports monte carlo comparisons of OLS,LAD, two robust estimators discussed by huber, three partially adaptiveestimators, newey's generalized method of moments estimator, and an adaptive maximum likelihood estimator based on a normal kernal studied by manski. This paper is the first to compare the relative performance of some adaptive robust estimators (partially adaptive and adaptive procedures) with some common nonadaptive robust estimators. The partially adaptive estimators are based on three flxible parametric distributions for the errors. These include the power exponential (Box-Tiao) and generalized t distributions, as well as a distribution for the errors, which is not necessarily symmetric. The adaptive procedures are "fully iterative" rather than one step estimators. The adaptive estimators have desirable large sample properties, but these properties do not necessarily carry over to the small sample case.

The monte carlo comparisons of the alternative estimators are based on four different specifications for the error distribution: a normal, a mixture of normals (or variance-contaminated normal), a bimodal mixture of normals, and a lognormal. Five hundred samples of 50 are used. The adaptive and partially adaptive estimators perform very well relative to the other estimation procedures considered, and preliminary results suggest that in some important cases they can perform much better than OLS with 50 to 80% reductions in standard errors.

     

Modified ratio estimators using robust regression methods

When there is an outlier in the data set, the efficiency of traditional methods decreases. In order to solve this problem, Kadilar et al. (2007) adapted Huber-M method which is only one of robust regression methods to ratio-type estimators and decreased the effect of outlier problem. In this study, new ratio-type estimators are proposed by considering Tukey-M, Hampel M, Huber MM, LTS, LMS and LAD robust methods based on the Kadilar et al. (2007). Theoretically, we obtain the mean square error (MSE) for these estimators. We compared with MSE values of proposed estimators and MSE values of estimators based on Huber-M and OLS methods. As a result of these comparisons, we observed that our proposed estimators give more efficient results than both Huber M approach which was proposed by Kadilar et al. (2007) and OLS approach. Also, under all conditions, all of the other proposed estimators except Lad method are more efficient than robust estimators proposed by Kadilar et al. (2007). And, these theoretical results are supported with the aid of a numerical example and simulation by basing on data that includes an outlier.     

Aalen's linear model for doubly censored data

Double censoring often occurs in registry studies when left censoring is present in addition to right censoring. In this work, we examine estimation of Aalen's nonparametric regression coefficients based on doubly censored data. We propose two estimation techniques. The first type of estimators, including ordinary least squared (OLS) estimator and weighted least squared (WLS) estimators, are obtained using martingale arguments. The second type of estimator, the maximum likelihood estimator (MLE), is obtained via expectation-maximization (EM) algorithms that treat the survival times of left censored observations as missing. Asymptotic properties, including the uniform consistency and weak convergence, are established for the MLE. Simulation results demonstrate that the MLE is more efficient than the OLS and WLS estimators.     

Robust Regression Using Data Partitioning and M-Estimation

We propose a new robust regression estimator using data partition technique and M estimation (DPM). The data partition technique is designed to define a small fixed number of subsets of the partitioned data set and to produce corresponding ordinary least square (OLS) fits in each subset, contrary to the resampling technique of existing robust estimators such as the least trimmed squares estimator. The proposed estimator shares a common strategy with the median ball algorithm estimator that is obtained from the OLS trial fits only on a fixed number of subsets of the data. We examine performance of the DPM estimator in the eleven challenging data sets and simulation studies. We also compare the DPM with the five commonly used robust estimators using empirical convergence rates relative to the OLS for clean data, robustness through mean squared error and bias, masking and swamping probabilities, the ability of detecting the known outliers, and the regression and affine equivariances.     

Pitfalls in Estimating Cointegrating Vector when Cointegration Relationship has Nonlinear Adjustment

This article investigates the properties of the estimators of the cointegrating vector when the cointegration error has a nonlinear adjustment. We investigate the properties of three estimators, namely, ordinary least squares (OLS), dynamic OLS (DOLS), and autoregressive distributed lag (ADL) models. Monte Carlo simulation results demonstrate that although all the estimators have consistency under cointegration with a nonlinear adjustment, they suffer from severe size distortions for the t-statistics of the cointegrating vector when the cointegration error has a highly persistent nonlinear adjustment and endogeneity. The results imply that the use of DOLS and ADL for cointegration with nonlinear adjustment cannot sufficiently improve the estimates and size performances.     

Bootstrap-based inferential improvements in beta autoregressive moving average model

We consider the issue of performing accurate small sample inference in beta autoregressive moving average model, which is useful for modeling and forecasting continuous variables that assume values in the interval (0,?1). The inferences based on conditional maximum likelihood estimation have good asymptotic properties, but their performances in small samples may be poor. This way, we propose bootstrap bias corrections of the point estimators and different bootstrap strategies for confidence interval improvements. Our Monte Carlo simulations show that finite sample inference based on bootstrap corrections is much more reliable than the usual inferences. We also presented an empirical application.     

MSE performance of the weighted average estimators consisting of shrinkage estimators when each individual regression coefficient is estimated

In this paper, we analytically derive the exact formula for the mean squared error (MSE) of two weighted average (WA) estimators for each individual regression coefficient. Further, we execute numerical evaluations to investigate small sample properties of the WA estimators, and compare the MSE performance of the WA estimators with the other shrinkage estimators and the usual OLS estimator. Our numerical results show that (1) the WA estimators have smaller MSE than the other shrinkage estimators and the OLS estimator over a wide region of parameter space; (2) the range where the relative MSE of the WA estimator is smaller than that of the OLS estimator gets narrower as the number of explanatory variables k increases.     

Performance of threshold cointegration tests

In this paper, we examine and compare the performance of ordinary least squares (OLS) and instrumental variables (IVs) based single-equation Engle–Granger, error correction model (ECM), and autoregressive-distributed lag (ADL) threshold cointegration tests. Results are additionally compared with the system-equation OLS–ECM test. Finite sample properties of size and power are examined in simulations. The single-equation ECM test performs well, but has certain limitations to resolve. The system-equation ECM test has good power, but suffers from mild size distortions that lead to over-rejections. Overall, the single-equation OLS–ADL test has the most desirable properties among the OLS-based tests and the IV–ADL test has the most desirable properties among the IV-based tests. The IV-based tests are invariant to nuisance parameters and have a standard normal distribution in each case, while the OLS-based tests are non-standard so that critical values must be simulated in different models.     

Estimation of Variance Components for a Linear Toeplitz Model

The linear Toeplitz covariance structure model of order one is considered. We give some elegant explicit expressions of the Locally Minimum Variance Quadratic Unbiased Estimators of its covariance parameters. We deduce from a Monte Carlo method some properties of their Gaussian maximum likelihood estimators. Finally, for small sample sizes, these two types of estimators are compared with the intuitive empirical estimators and it is shown that the empirical biased estimators should be used.     

Difference-in-difference estimation by FE and OLS when there is panel non-response

We show that the ordinary least squares (OLS) and fixed-effects (FE) estimators of the popular difference-in-differences model may deviate when there is time-varying panel non-response. If such non-response does not affect the common-trend assumption, then OLS and FE are consistent, but OLS is more precise. However, if non-response is affecting the common-trend assumption, then FE estimation may still be consistent, while OLS will be inconsistent. We provide simulation as well as empirical evidence for this phenomenon to occur. We conclude that in case of unbalanced panels, deviating OLS and FE estimates should be considered as evidence that non-response is not ignorable for the differences-in-differences estimation.     

基尼加权回归分析：概念、方法及应用

普通最小二乘法是进行回归分析最常用的基本方法,但该方法要求满足若干经典假设,对于小样本或在与收入相关回归分析的参数估计中易受奇异值、高收入群体的影响。本文试图利用基尼加权回归弥补以上不足。基尼加权回归可分为参数方法与非参数方法两类,参数方法基于样本残差的基尼平均差最小原则对参数进行估计;非参数方法则是直接由两点间的斜率加权得到。基尼加权回归分析可以进行参数假设检验并定义拟合优度,其中的假设检验在实际应用中采用Jackknife重抽样方法估计方差。文中提出的样本拓展基尼平均差算法,弥补了现有算法对样本数据只能提供近似计算的不足,极大简化相应的计算公式。利用我国2015年省域截面数据、1994至2015年总量时间序列数据分别讨论入境旅游收入对收入基尼系数的影响,发现使用基尼加权回归的结果不仅符合理论预期,而且可以通过不平等厌恶参数的变化反映入境旅游收入对不同群体收入公平性的影响。     

Does sequential augmenting of simple linear heteroscedastic regression reduce variances of ordinary least-squares estimators?

If uncorrelated random variables have a common expected value and decreasing variances, then the variance of a sample mean is decreasing with the number of observations. Unfortunately, this natural and desirable variance reduction property (VRP) by augmenting data is not automatically inherited by ordinary least-squares (OLS) estimators of parameters. We derive a new decomposition for updating the covariance matrices of the OLS which implies conditions for the OLS to have the VRP. In particular, in the case of a straight-line regression, we show that the OLS estimators of intercept and slope have the VRP if the values of the explanatory variable are increasing. This also holds true for alternating two-point experimental designs.     

Estimating utility functions using generalized maximum entropy

This paper estimates von Neumann and Morgenstern utility functions using the generalized maximum entropy (GME), applied to data obtained by utility elicitation methods. Given the statistical advantages of this approach, we provide a comparison of the performance of the GME estimator with ordinary least square (OLS) in a real data small sample setup. The results confirm the ones obtained for small samples through Monte Carlo simulations. The difference between the two estimators is small and it decreases as the width of the parameter support vector increases. Moreover, the GME estimator is more precise than the OLS one. Overall, the results suggest that GME is an interesting alternative to OLS in the estimation of utility functions when data are generated by utility elicitation methods.     

Estimation of mean in presence of non-response using two phase sampling scheme

This paper considers the problem of estimating the population mean using information on an auxiliary variable in presence of non-response. Some modified ratio, product and regression estimators in double sampling have been suggested and their properties are studied. It is shown that to the first degree of approximation, estimators based on estimated optimum values have the same variance as that of the optimum estimators. An empirical study is carried to judge the merits of the suggested estimators over conventional unbiased estimator and other known estimators. Both theoretical and empirical study results present the soundness and usefulness of the suggested estimators in practice.     

Finite sample properties of an HPT estimator when each individual regression coefficient is estimated in a misspecified linear regression model

ABSTRACT

In this paper, assuming that there exist omitted variables in the specified model, we analytically derive the exact formula for the mean squared error (MSE) of a heterogeneous pre-test (HPT) estimator whose components are the ordinary least squares (OLS) and feasible ridge regression (FRR) estimators. Since we cannot examine the MSE performance analytically, we execute numerical evaluations to investigate small sample properties of the HPT estimator, and compare the MSE performance of the HPT estimator with those of the FRR estimator and the usual OLS estimator. Our numerical results show that (1) the HPT estimator is more efficient when the model misspecification is severe; (2) the HPT estimator with the optimal critical value obtained under the correctly specified model can be safely used even when there exist omitted variables in the specified model.     

Predicting integrals of diffusion processes

Consider predicting the integral of a diffusion process Z in a bounded interval A, based on the observations Z(t_1n),…,Z(t_nn), where t_1n,…,t_nn is a dense triangular array of points (the step of discretization tends to zero as n increases) in the bounded interval. The best linear predictor is generally not asymptotically optimal. Instead, we predict using the conditional expectation of the integral of the diffusion process, the optimal predictor in terms of minimizing the mean squared error, given the observed values of the process. We obtain that, conditioning on the observed values, the order of convergence in probability to zero of the mean squared prediction error is O_p(n⁻²). We prove that the standardized conditional prediction error is approximately Gaussian with mean zero and unit variance, even though the underlying diffusion is generally non-Gaussian. Because the optimal predictor is hard to calculate exactly for most diffusions, we present an easily computed approximation that is asymptotically optimal. This approximation is a function of the diffusion coefficient.     

Efficient M-estimators with auxiliary information

This paper introduces a new class of M-estimators based on generalised empirical likelihood (GEL) estimation with some auxiliary information available in the sample. The resulting class of estimators is efficient in the sense that it achieves the same asymptotic lower bound as that of the efficient generalised method of moment (GMM) estimator with the same auxiliary information. The paper also shows that in case of smooth estimating equations the proposed estimators enjoy a small second order bias property compared to both efficient GMM and full GEL estimators. Analytical formulae to obtain bias corrected estimators are also provided. Simulations show that with correctly specified auxiliary information the proposed estimators and in particular those based on empirical likelihood outperform standard M and efficient GMM estimators both in terms of finite sample bias and efficiency. On the other hand with moderately misspecified auxiliary information estimators based on the nonparametric tilting method are typically characterised by the best finite sample properties.