Estimator Choice and Fisher's Paradox: A Monte Carlo Study

This paper argues that Fisher's paradox can be explained away in terms of estimator choice. We analyse by means of Monte Carlo experiments the small sample properties of a large set of estimators (including virtually all available single-equation estimators), and compute the critical values based on the empirical distributions of the t-statistics, for a variety of Data Generation Processes (DGPs), allowing for structural breaks, ARCH effects etc. We show that precisely the estimators most commonly used in the literature, namely OLS, Dynamic OLS (DOLS) and non-prewhitened FMLS, have the worst performance in small samples, and produce rejections of the Fisher hypothesis. If one employs the estimators with the most desirable properties (i.e., the smallest downward bias and the minimum shift in the distribution of the associated t-statistics), or if one uses the empirical critical values, the evidence based on US data is strongly supportive of the Fisher relation, consistently with many theoretical models.     

Pitfalls in Estimating Cointegrating Vector when Cointegration Relationship has Nonlinear Adjustment

This article investigates the properties of the estimators of the cointegrating vector when the cointegration error has a nonlinear adjustment. We investigate the properties of three estimators, namely, ordinary least squares (OLS), dynamic OLS (DOLS), and autoregressive distributed lag (ADL) models. Monte Carlo simulation results demonstrate that although all the estimators have consistency under cointegration with a nonlinear adjustment, they suffer from severe size distortions for the t-statistics of the cointegrating vector when the cointegration error has a highly persistent nonlinear adjustment and endogeneity. The results imply that the use of DOLS and ADL for cointegration with nonlinear adjustment cannot sufficiently improve the estimates and size performances.     

Stepwise Regression in Mixed Quantitative Linear Models with Autocorrelated Errors

ABSTRACT

In the stepwise procedure of selection of a fixed or a random explanatory variable in a mixed quantitative linear model with errors following a Gaussian stationary autocorrelated process, we have studied the efficiency of five estimators relative to Generalized Least Squares (GLS): Ordinary Least Squares (OLS), Maximum Likelihood (ML), Restricted Maximum Likelihood (REML), First Differences (FD), and First-Difference Ratios (FDR). We have also studied the validity and power of seven derived testing procedures, to assess the significance of the slope of the candidate explanatory variable x ₂ to enter the model in which there is already one regressor x ₁. In addition to five testing procedures of the literature, we considered the FDR t-test with n ? 3 df and the modified t-test with n? ? 3 df for partial correlations, where n? is Dutilleul's effective sample size. Efficiency, validity, and power were analyzed by Monte Carlo simulations, as functions of the nature, fixed vs. random (purely random or autocorrelated), of x ₁ and x ₂, the sample size and the autocorrelation of random terms in the regression model. We report extensive results for the autocorrelation structure of first-order autoregressive [AR(1)] type, and discuss results we obtained for other autocorrelation structures, such as spherical semivariogram, first-order moving average [MA(1)] and ARMA(1,1), but we could not present because of space constraints. Overall, we found that:

1. the efficiency of slope estimators and the validity of testing procedures depend primarily on the nature of x ₂, but not on that of x ₁;

2. FDR is the most inefficient slope estimator, regardless of the nature of x ₁ and x ₂;

3. REML is the most efficient of the slope estimators compared relative to GLS, provided the specified autocorrelation structure is correct and the sample size is large enough to ensure the convergence of its optimization algorithm;

4. the FDR t-test, the modified t-test and the REML t-test are the most valid of the testing procedures compared, despite the inefficiency of the FDR and OLS slope estimators for the former two;

5. the FDR t-test, however, suffers from a lack of power that varies with the nature of x ₁ and x ₂; and

6. the modified t-test for partial correlations, which does not require the specification of an autocorrelation structure, can be recommended when x ₁ is fixed or random and x ₂ is random, whether purely random or autocorrelated. Our results are illustrated by the environmental data that motivated our work.

     

Robust Estimators for the Parameters of Multivariate Lognormal Distribution

Abstract

In this paper, we introduce a class of location and scale estimators for the p-variate lognormal distribution. These estimators are obtained by applying a log transform to the data, computing robust Fisher consistent estimators for the obtained Gaussian data and transforming those estimators for the lognormal using the relationship between the parameters of both distributions. We prove some of the properties of these estimators, such as Fisher consistency, robustness and asymptotic normality.     

Partly linear models on Riemannian manifolds

In partly linear models, the dependence of the response y on (x ^T, t) is modeled through the relationship y=x ^T β+g(t)+?, where ? is independent of (x ^T, t). We are interested in developing an estimation procedure that allows us to combine the flexibility of the partly linear models, studied by several authors, but including some variables that belong to a non-Euclidean space. The motivating application of this paper deals with the explanation of the atmospheric SO₂ pollution incidents using these models when some of the predictive variables belong in a cylinder. In this paper, the estimators of β and g are constructed when the explanatory variables t take values on a Riemannian manifold and the asymptotic properties of the proposed estimators are obtained under suitable conditions. We illustrate the use of this estimation approach using an environmental data set and we explore the performance of the estimators through a simulation study.     

A Nonrandomized,Nonconservative Version of the Fisher Exact Test

The Fisher exact test has been unjustly dismissed by some as ‘only conditional,’ whereas it is unconditionally the uniform most powerful test among all unbiased tests, tests of size α and with power greater than its nominal level of significance α. The problem with this truly optimal test is that it requires randomization at the critical value(s) to be of size α. Obviously, in practice, one does not want to conclude that ‘with probability x the we have a statistical significant result.’ Usually, the hypothesis is rejected only if the test statistic's outcome is more extreme than the critical value, reducing the actual size considerably.

The randomized unconditional Fisher exact is constructed (using Neyman–structure arguments) by deriving a conditional randomized test randomizing at critical values c(t) by probabilities γ(t), that both depend on the total number of successes T (the complete-sufficient statistic for the nuisance parameter—the common success probability) conditioned upon.

In this paper, the Fisher exact is approximated by deriving nonrandomized conditional tests with critical region including the critical value only if γ (t) > γ₀, for a fixed threshold value γ₀, such that the size of the unconditional modified test is for all value of the nuisance parameter—the common success probability—smaller, but as close as possible to α. It will be seen that this greatly improves the size of the test as compared with the conservative nonrandomized Fisher exact test.

Size, power, and p value comparison with the (virtual) randomized Fisher exact test, and the conservative nonrandomized Fisher exact, Pearson's chi-square test, with the more competitive mid-p value, the McDonald's modification, and Boschloo's modifications are performed under the assumption of two binomial samples.     

MSE performance of the weighted average estimators consisting of shrinkage estimators when each individual regression coefficient is estimated

In this paper, we analytically derive the exact formula for the mean squared error (MSE) of two weighted average (WA) estimators for each individual regression coefficient. Further, we execute numerical evaluations to investigate small sample properties of the WA estimators, and compare the MSE performance of the WA estimators with the other shrinkage estimators and the usual OLS estimator. Our numerical results show that (1) the WA estimators have smaller MSE than the other shrinkage estimators and the OLS estimator over a wide region of parameter space; (2) the range where the relative MSE of the WA estimator is smaller than that of the OLS estimator gets narrower as the number of explanatory variables k increases.     

Efficiency and Validity Analyses of Two-Stage Estimation Procedures and Derived Testing Procedures in Quantitative Linear Models with AR(1) Errors

Abstract

In a quantitative linear model with errors following a stationary Gaussian, first-order autoregressive or AR(1) process, Generalized Least Squares (GLS) on raw data and Ordinary Least Squares (OLS) on prewhitened data are efficient methods of estimation of the slope parameters when the autocorrelation parameter of the error AR(1) process, ρ, is known. In practice, ρ is generally unknown. In the so-called two-stage estimation procedures, ρ is then estimated first before using the estimate of ρ to transform the data and estimate the slope parameters by OLS on the transformed data. Different estimators of ρ have been considered in previous studies. In this article, we study nine two-stage estimation procedures for their efficiency in estimating the slope parameters. Six of them (i.e., three noniterative, three iterative) are based on three estimators of ρ that have been considered previously. Two more (i.e., one noniterative, one iterative) are based on a new estimator of ρ that we propose: it is provided by the sample autocorrelation coefficient of the OLS residuals at lag 1, denoted r(1). Lastly, REstricted Maximum Likelihood (REML) represents a different type of two-stage estimation procedure whose efficiency has not been compared to the others yet. We also study the validity of the testing procedures derived from GLS and the nine two-stage estimation procedures. Efficiency and validity are analyzed in a Monte Carlo study. Three types of explanatory variable x in a simple quantitative linear model with AR(1) errors are considered in the time domain: Case 1, x is fixed; Case 2, x is purely random; and Case 3, x follows an AR(1) process with the same autocorrelation parameter value as the error AR(1) process. In a preliminary step, the number of inadmissible estimates and the efficiency of the different estimators of ρ are compared empirically, whereas their approximate expected value in finite samples and their asymptotic variance are derived theoretically. Thereafter, the efficiency of the estimation procedures and the validity of the derived testing procedures are discussed in terms of the sample size and the magnitude and sign of ρ. The noniterative two-stage estimation procedure based on the new estimator of ρ is shown to be more efficient for moderate values of ρ at small sample sizes. With the exception of small sample sizes, REML and its derived F-test perform the best overall. The asymptotic equivalence of two-stage estimation procedures, besides REML, is observed empirically. Differences related to the nature, fixed or random (uncorrelated or autocorrelated), of the explanatory variable are also discussed.     

Finite sample properties of an HPT estimator when each individual regression coefficient is estimated in a misspecified linear regression model

ABSTRACT

In this paper, assuming that there exist omitted variables in the specified model, we analytically derive the exact formula for the mean squared error (MSE) of a heterogeneous pre-test (HPT) estimator whose components are the ordinary least squares (OLS) and feasible ridge regression (FRR) estimators. Since we cannot examine the MSE performance analytically, we execute numerical evaluations to investigate small sample properties of the HPT estimator, and compare the MSE performance of the HPT estimator with those of the FRR estimator and the usual OLS estimator. Our numerical results show that (1) the HPT estimator is more efficient when the model misspecification is severe; (2) the HPT estimator with the optimal critical value obtained under the correctly specified model can be safely used even when there exist omitted variables in the specified model.     

A comparison of some robust,adaptive, and partially adaptive estimators of regression models

Numerous estimation techniques for regression models have been proposed. These procedures differ in how sample information is used in the estimation procedure. The efficiency of least squares (OLS) estimators implicity assumes normally distributed residuals and is very sensitive to departures from normality, particularly to "outliers" and thick-tailed distributions. Lead absolute deviation (LAD) estimators are less sensitive to outliers and are optimal for laplace random disturbances, but not for normal errors. This paper reports monte carlo comparisons of OLS,LAD, two robust estimators discussed by huber, three partially adaptiveestimators, newey's generalized method of moments estimator, and an adaptive maximum likelihood estimator based on a normal kernal studied by manski. This paper is the first to compare the relative performance of some adaptive robust estimators (partially adaptive and adaptive procedures) with some common nonadaptive robust estimators. The partially adaptive estimators are based on three flxible parametric distributions for the errors. These include the power exponential (Box-Tiao) and generalized t distributions, as well as a distribution for the errors, which is not necessarily symmetric. The adaptive procedures are "fully iterative" rather than one step estimators. The adaptive estimators have desirable large sample properties, but these properties do not necessarily carry over to the small sample case.

The monte carlo comparisons of the alternative estimators are based on four different specifications for the error distribution: a normal, a mixture of normals (or variance-contaminated normal), a bimodal mixture of normals, and a lognormal. Five hundred samples of 50 are used. The adaptive and partially adaptive estimators perform very well relative to the other estimation procedures considered, and preliminary results suggest that in some important cases they can perform much better than OLS with 50 to 80% reductions in standard errors.

     

Second-order and resampling approximation of finite population U-statistics based on stratified samples

We construct one-term Edgeworth expansions to distributions of U statistics and Studentized U-statistics, based on stratified samples drawn without replacement. Replacing the cumulants defining the expansions by consistent jackknife estimators, we obtain empirical Edgeworth expansions. The expansions provide second-order approximations that improve upon the normal approximation. Theoretical results are illustrated by a simulation study where we compare various approximations to the distribution of the commonly used Gini's mean difference estimator.     

Combining the unrestricted estimators into a single estimator and a simulation study on the unrestricted estimators

The purpose of this paper is to combine several regression estimators (ordinary least squares (OLS), ridge, contraction, principal components regression (PCR), Liu, r?k and r?d class estimators) into a single estimator. The conditions for the superiority of this new estimator over the PCR, the r?k class, the r?d class, β?(k, d), OLS, ridge, Liu and contraction estimators are derived by the scalar mean square error criterion and the estimators of the biasing parameters for this new estimator are examined. Also, a numerical example based on Hald data and a simulation study are used to illustrate the results.     

Finite sample properties of ml and reml estimators in time series regression models with long memory noise

Finite sample properties of ML and REML estimators in time series regression models with fractional ARIMA noise are examined. In particular, theoretical approximations for bias of ML and REML estimators of the noise parameters are developed and their accuracy is assessed through simulations. The impact of noise parameter estimation on performance of t -statistics and likelihood ratio statistics for testing regression parameters is also investigated.     

A Note on Robustness of Normal Variance Estimators Under t-Distributions

ABSTRACT

In many real life problems one assumes a normal model because the sample histogram looks unimodal, symmetric, and/or the standard tests like the Shapiro-Wilk test favor such a model. However, in reality, the assumption of normality may be misplaced since the normality tests often fail to detect departure from normality (especially for small sample sizes) when the data actually comes from slightly heavier tail symmetric unimodal distributions. For this reason it is important to see how the existing normal variance estimators perform when the actual distribution is a t-distribution with k degrees of freedom (d.f.) (t _k -distribution). This note deals with the performance of standard normal variance estimators under the t _k -distributions. It is shown that the relative ordering of the estimators is preserved for both the quadratic loss as well as the entropy loss irrespective of the d.f. and the sample size (provided the risks exist).     

PMSE performance of two different types of preliminary test estimators under a multivariate t error term

Abstract

In this paper, assuming that the error terms follow a multivariate t distribution, we derive the exact formula for the predictive mean squared error (PMSE) of two different types of pretest estimators. It is shown analytically that one of the pretest estimator dominates the SR estimator if a critical value of the pretest is chosen appropriately. Also, we compare the PMSE of the pretest estimators with the MMSE, AMMSE, SR and PSR estimators by numerical evaluations. Our results show that the pretest estimators dominate the OLS estimator for all combinations when the degrees of freedom is not more than 5.     

On mitigating collinearity through mixtures

In linear models having near collinear columns of X, ridge and surrogate estimators often are used to mitigate collinearity. A new class of estimators is based on mixtures, either of X and a design minimal in an ordered class or of the Fisher information and a scalar matrix. Comparisons are drawn among choices for the mixing parameter, and the estimators are found to be admissible relative to ordinary least squares. Case studies demonstrate that selected mixture designs are perturbed from the original design to a lesser extent than are those of the surrogate method, while retaining reasonable efficiency characteristics.     

Evaluation of the predictive performance of the r-k and r-d class estimators

Multiple linear regression models are frequently used in predicting unknown values of the response variable y. In this case, a regression model's ability to produce an adequate prediction equation is of prime importance. This paper discusses the predictive performance of the r-k and r-d class estimators compared to ordinary least squares (OLS), principal components, ridge regression and Liu estimators and between each other. The theoretical results are illustrated using Portland cement data and a region is established where the r-k and the r-d class estimators are uniformly superior to the other mentioned estimators.     

Inference for IZAWA’S Bivariate Gamma Distribution

Abstract

In this article, we aim to establish some theoretical properties of Izawa’s bivariate gamma distribution having equal shape parameters. First, we propose a procedure to obtain the maximum likelihood estimates and derive an expression for the Fisher information. Simulation studies illuminate the properties of maximum likelihood estimators. We also establish an asymptotic test for independence based on the limiting distribution of maximum likelihood estimators.     

Two-way ANOVA when the distribution of the error terms is skew t

In this article, we assume that the distribution of the error terms is skew t in two-way analysis of variance (ANOVA). Skew t distribution is very flexible for modeling the symmetric and the skew datasets, since it reduces to the well-known normal, skew normal, and Student's t distributions. We obtain the estimators of the model parameters by using the maximum likelihood (ML) and the modified maximum likelihood (MML) methodologies. We also propose new test statistics based on these estimators for testing the equality of the treatment and the block means and also the interaction effect. The efficiencies of the ML and the MML estimators and the power values of the test statistics based on them are compared with the corresponding normal theory results via Monte Carlo simulation study. Simulation results show that the proposed methodologies are more preferable. We also show that the test statistics based on the ML estimators are more powerful than the test statistics based on the MML estimators as expected. However, power values of the test statistics based on the MML estimators are very close to the corresponding test statistics based on the ML estimators. At the end of the study, a real life example is given to show the implementation of the proposed methodologies.     

Peculiar bias properties of the ols estimator when applied to a dynamic model with autocorrelated disturbances

This study reveals that contrary to the conventional wisdom among econometricians, the bias of the OLS estimator can be quite small when the estimator is applied to a geometrically distributed lag model, y_t<ce:glyph name="dbnd6"/> α + βx _t+ λy _t-₁. + u_t, with autocorrelated disturbances, be they AR(1), MA(1), MA(2), AR(2), and ARMA(1,1). This happens when λ is large and x_tis smoothly trended (e.g., a real GNP series). In fact, the bias of the OLS estimator becomes zero at one parameter combination, and the OLS estimator performs well over a wide range around this parameter combination. By decomposing the disturbance term into two parts, the paper also explains why OLS shows such an unexpected property. These findings have both pedagogical and practical significance.