Improving the bandwidth-free inference methods by prewhitening

In this paper we consider inference of parameters in time series regression models. In the traditional inference approach, the heteroskedasticity and autocorrelation consistent (HAC) estimation is often involved to consistently estimate the asymptotic covariance matrix of regression parameter estimator. Since the bandwidth parameter in the HAC estimation is difficult to choose in practice, there has been a recent surge of interest in developing bandwidth-free inference methods. However, existing simulation studies show that these new methods suffer from severe size distortion in the presence of strong temporal dependence for a medium sample size. To remedy the problem, we propose to apply the prewhitening to the inconsistent long-run variance estimator in these methods to reduce the size distortion. The asymptotic distribution of the prewhitened Wald statistic is obtained and the general effectiveness of prewhitening is shown through simulations.     

Nonstationary nonlinear quantile regression

This study examines estimation and inference based on quantile regression for parametric nonlinear models with an integrated time series covariate. We first derive the limiting distribution of the nonlinear quantile regression estimator and then consider testing for parameter restrictions, when the regression function is specified as an asymptotically homogeneous function. We also study linear-in-parameter regression models when the regression function is given by integrable regression functions as well as asymptotically homogeneous regression functions. We, furthermore, propose a fully modified estimator to reduce the bias in the original estimator under a certain set of conditions. Finally, simulation studies show that the estimators behave well, especially when the regression error term has a fat-tailed distribution.     

A Simple Approach to Inference in Random Coefficient Models

Random coefficient regression models have been used to analyze cross-sectional and longitudinal data in economics and growth-curve data from biological and agricultural experiments. In the literature several estimators, including the ordinary least squares and the estimated generalized least squares (EGLS), have been considered for estimating the parameters of the mean model. Based on the asymptotic properties of the EGLS estimators, test statistics have been proposed for testing linear hypotheses involving the parameters of the mean model. An alternative estimator, the simple mean of the individual regression coefficients, provides estimation and hypothesis-testing procedures that are simple to compute and teach. The large sample properties of this simple estimator are shown to be similar to that of the EGLS estimator. The performance of the proposed estimator is compared with that of the existing estimators by Monte Carlo simulation.     

Estimation and inference in regression discontinuity designs with asymmetric kernels

We study the behaviour of the Wald estimator of causal effects in regression discontinuity design when local linear regression (LLR) methods are combined with an asymmetric gamma kernel. We show that the resulting statistic is no more complex to implement than existing methods, remains consistent at the usual non-parametric rate, and maintains an asymptotic normal distribution but, crucially, has bias and variance that do not depend on kernel-related constants. As a result, the new estimator is more efficient and yields more reliable inference. A limited Monte Carlo experiment is used to illustrate the efficiency gains. As a by product of the main discussion, we extend previous published work by establishing the asymptotic normality of the LLR estimator with a gamma kernel. Finally, the new method is used in a substantive application.     

Bias in Small-Sample Inference With Count-Data Models

Abstract

Both Poisson and negative binomial regression can provide quasi-likelihood estimates for coefficients in exponential-mean models that are consistent in the presence of distributional misspecification. It has generally been recommended, however, that inference be carried out using asymptotically robust estimators for the parameter covariance matrix. As with linear models, such robust inference tends to lead to over-rejection of null hypotheses in small samples. Alternative methods for estimating coefficient estimator variances are considered. No one approach seems to remove all test bias, but the results do suggest that the use of the jackknife with Poisson regression tends to be least biased for inference.     

Saddlepoint-Based Bootstrap Inference for Quadratic Estimating Equations

Abstract.  We propose an easy to implement method for making small sample parametric inference about the root of an estimating equation expressible as a quadratic form in normal random variables. It is based on saddlepoint approximations to the distribution of the estimating equation whose unique root is a parameter's maximum likelihood estimator (MLE), while substituting conditional MLEs for the remaining (nuisance) parameters. Monotoncity of the estimating equation in its parameter argument enables us to relate these approximations to those for the estimator of interest. The proposed method is equivalent to a parametric bootstrap percentile approach where Monte Carlo simulation is replaced by saddlepoint approximation. It finds applications in many areas of statistics including, nonlinear regression, time series analysis, inference on ratios of regression parameters in linear models and calibration. We demonstrate the method in the context of some classical examples from nonlinear regression models and ratios of regression parameter problems. Simulation results for these show that the proposed method, apart from being generally easier to implement, yields confidence intervals with lengths and coverage probabilities that compare favourably with those obtained from several competing methods proposed in the literature over the past half-century.     

Semiparametric Regression with Kernel Error Model

Abstract.  We propose and study a class of regression models, in which the mean function is specified parametrically as in the existing regression methods, but the residual distribution is modelled non-parametrically by a kernel estimator, without imposing any assumption on its distribution. This specification is different from the existing semiparametric regression models. The asymptotic properties of such likelihood and the maximum likelihood estimate (MLE) under this semiparametric model are studied. We show that under some regularity conditions, the MLE under this model is consistent (when compared with the possibly pseudo-consistency of the parameter estimation under the existing parametric regression model), is asymptotically normal with rate and efficient. The non-parametric pseudo-likelihood ratio has the Wilks property as the true likelihood ratio does. Simulated examples are presented to evaluate the accuracy of the proposed semiparametric MLE method.     

Methods of Statistical Inference for Median Regression Models with Doubly Censored Data

Recently, least absolute deviations (LAD) estimator for median regression models with doubly censored data was proposed and the asymptotic normality of the estimator was established. However, it is invalid to make inference on the regression parameter vectors, because the asymptotic covariance matrices are difficult to estimate reliably since they involve conditional densities of error terms. In this article, three methods, which are based on bootstrap, random weighting, and empirical likelihood, respectively, and do not require density estimation, are proposed for making inference for the doubly censored median regression models. Simulations are also done to assess the performance of the proposed methods.     

A robust and efficient estimation method for nonparametric models with jump points

Nonparametric models with jump points have been considered by many researchers. However, most existing methods based on least squares or likelihood are sensitive when there are outliers or the error distribution is heavy tailed. In this article, a local piecewise-modal method is proposed to estimate the regression function with jump points in nonparametric models, and a piecewise-modal EM algorithm is introduced to estimate the proposed estimator. Under some regular conditions, the large-sample theory is established for the proposed estimators. Several simulations are presented to evaluate the performances of the proposed method, which shows that the proposed estimator is more efficient than the local piecewise-polynomial regression estimator in the presence of outliers or heavy tail error distribution. What is more, the proposed procedure is asymptotically equivalent to the local piecewise-polynomial regression estimator under the assumption that the error distribution is a Gaussian distribution. The proposed method is further illustrated via the sea-level pressures.     

Using Liu-Type Estimator to Combat Collinearity

Abstract

Linear regression model and least squares method are widely used in many fields of natural and social sciences. In the presence of collinearity, the least squares estimator is unstable and often gives misleading information. Ridge regression is the most common method to overcome this problem. We find that when there exists severe collinearity, the shrinkage parameter selected by existing methods for ridge regression may not fully address the ill conditioning problem. To solve this problem, we propose a new two-parameter estimator. We show using both theoretic results and simulation that our new estimator has two advantages over ridge regression. First, our estimator has less mean squared error (MSE). Second, our estimator can fully address the ill conditioning problem. A numerical example from literature is used to illustrate the results.     

Estimation and Inference for Linear Panel Data Models Under Misspecification When Both n and T are Large

This article considers fixed effects (FE) estimation for linear panel data models under possible model misspecification when both the number of individuals, n, and the number of time periods, T, are large. We first clarify the probability limit of the FE estimator and argue that this probability limit can be regarded as a pseudo-true parameter. We then establish the asymptotic distributional properties of the FE estimator around the pseudo-true parameter when n and T jointly go to infinity. Notably, we show that the FE estimator suffers from the incidental parameters bias of which the top order is O(T^{? 1}), and even after the incidental parameters bias is completely removed, the rate of convergence of the FE estimator depends on the degree of model misspecification and is either (nT)^{? 1/2} or n^{? 1/2}. Second, we establish asymptotically valid inference on the (pseudo-true) parameter. Specifically, we derive the asymptotic properties of the clustered covariance matrix (CCM) estimator and the cross-section bootstrap, and show that they are robust to model misspecification. This establishes a rigorous theoretical ground for the use of the CCM estimator and the cross-section bootstrap when model misspecification and the incidental parameters bias (in the coefficient estimate) are present. We conduct Monte Carlo simulations to evaluate the finite sample performance of the estimators and inference methods, together with a simple application to the unemployment dynamics in the U.S.     

Inference in Approximately Sparse Correlated Random Effects Probit Models With Panel Data

Abstract

We propose a simple procedure based on an existing “debiased” l₁-regularized method for inference of the average partial effects (APEs) in approximately sparse probit and fractional probit models with panel data, where the number of time periods is fixed and small relative to the number of cross-sectional observations. Our method is computationally simple and does not suffer from the incidental parameters problems that come from attempting to estimate as a parameter the unobserved heterogeneity for each cross-sectional unit. Furthermore, it is robust to arbitrary serial dependence in underlying idiosyncratic errors. Our theoretical results illustrate that inference concerning APEs is more challenging than inference about fixed and low-dimensional parameters, as the former concerns deriving the asymptotic normality for sample averages of linear functions of a potentially large set of components in our estimator when a series approximation for the conditional mean of the unobserved heterogeneity is considered. Insights on the applicability and implications of other existing Lasso-based inference procedures for our problem are provided. We apply the debiasing method to estimate the effects of spending on test pass rates. Our results show that spending has a positive and statistically significant average partial effect; moreover, the effect is comparable to found using standard parametric methods.     

On Semiparametric EV Models with Serially Correlated Errors in Both Regression Models and Mismeasured Covariates

Abstract.  We consider inference for a semiparametric regression model where some covariates are measured with errors, and the errors in both the regression model and the mismeasured covariates are serially correlated. We propose a weighted estimating equations-based estimator (WEEBE) for the regression coefficients. We show that the WEEBE is asymptotically more efficient than the estimators that neglect the serial correlations. This is an interesting new finding since earlier results in the statistical literature have shown that the weighted estimation is not as efficient as the unweighted estimation when the measurement errors and serially correlated errors of the regression models exist simultaneously (Biometrics, 49, 1993, 1262; Technometrics, 42, 2000, 137). The proposed WEEBE does not require undersmoothing the regressor functions in order to make it attain the root- n consistency. Simulation studies show that the proposed estimator has nice finite sample properties. A real data set is used to illustrate the proposed method.     

A note on the efficiency of composite quantile regression

Composite quantile regression (CQR) is motivated by the desire to have an estimator for linear regression models that avoids the breakdown of the least-squares estimator when the error variance is infinite, while having high relative efficiency even when the least-squares estimator is fully efficient. Here, we study two weighting schemes to further improve the efficiency of CQR, motivated by Jiang et al. [Oracle model selection for nonlinear models based on weighted composite quantile regression. Statist Sin. 2012;22:1479–1506]. In theory the two weighting schemes are asymptotically equivalent to each other and always result in more efficient estimators compared with CQR. Although the first weighting scheme is hard to implement, it sheds light on in what situations the improvement is expected to be large. A main contribution is to theoretically and empirically identify that standard CQR has good performance compared with weighted CQR only when the error density is logistic or close to logistic in shape, which was not noted in the literature.     

Developing a restricted two-parameter Liu-type estimator: A comparison of restricted estimators in the binary logistic regression model

In the context of estimating regression coefficients of an ill-conditioned binary logistic regression model, we develop a new biased estimator having two parameters for estimating the regression vector parameter β when it is subjected to lie in the linear subspace restriction Hβ = h. The matrix mean squared error and mean squared error (MSE) functions of these newly defined estimators are derived. Moreover, a method to choose the two parameters is proposed. Then, the performance of the proposed estimator is compared to that of the restricted maximum likelihood estimator and some other existing estimators in the sense of MSE via a Monte Carlo simulation study. According to the simulation results, the performance of the estimators depends on the sample size, number of explanatory variables, and degree of correlation. The superiority region of our proposed estimator is identified based on the biasing parameters, numerically. It is concluded that the new estimator is superior to the others in most of the situations considered and it is recommended to the researchers.     

The effect of small sample on optimal designs for logistic regression models

Asymptotic methods are commonly used in statistical inference for unknown parameters in binary data models. These methods are based on large sample theory, a condition which may be in conflict with small sample size and hence leads to poor results in the optimal designs theory. In this paper, we apply the second order expansions of the maximum likelihood estimator and derive a matrix formula for the mean square error (MSE) to obtain more precise optimal designs based on the MSE. Numerical results indicate the new optimal designs are more efficient than the optimal designs based on the information matrix.     

Inverse censoring weighted median regression

We implement semiparametric random censorship model aided inference for censored median regression models. This is based on the idea that, when the censoring is specified by a common distribution, a semiparametric survival function estimator acts as an improved weight in the so-called inverse censoring weighted estimating function. We show that the proposed method will always produce estimates of the model parameters that are as good as or better than an existing estimator based on the traditional Kaplan–Meier weights. We also provide an illustration of the method through an analysis of a lung cancer data set.     

Constrained penalized splines

The penalized spline is a popular method for function estimation when the assumption of “smoothness” is valid. In this paper, methods for estimation and inference are proposed using penalized splines under additional constraints of shape, such as monotonicity or convexity. The constrained penalized spline estimator is shown to have the same convergence rates as the corresponding unconstrained penalized spline, although in practice the squared error loss is typically smaller for the constrained versions. The penalty parameter may be chosen with generalized cross‐validation, which also provides a method for determining if the shape restrictions hold. The method is not a formal hypothesis test, but is shown to have nice large‐sample properties, and simulations show that it compares well with existing tests for monotonicity. Extensions to the partial linear model, the generalized regression model, and the varying coefficient model are given, and examples demonstrate the utility of the methods. The Canadian Journal of Statistics 40: 190–206; 2012 © 2012 Statistical Society of Canada     

Empirical likelihood inference for a common mean in the presence of heteroscedasticity

The authors develop empirical likelihood (EL) based methods of inference for a common mean using data from several independent but nonhomogeneous populations. For point estimation, they propose a maximum empirical likelihood (MEL) estimator and show that it is n‐consistent and asymptotically optimal. For confidence intervals, they consider two EL based methods and show that both intervals have approximately correct coverage probabilities under large samples. Finite‐sample performances of the MEL estimator and the EL based confidence intervals are evaluated through a simulation study. The results indicate that overall the MEL estimator and the weighted EL confidence interval are superior alternatives to the existing methods.     

Heteroscedastic Regression Models and Applications to Off-line Quality Control

We discuss in the present paper the analysis of heteroscedastic regression models and their applications to off-line quality control problems. It is well known that the method of pseudo-likelihood is usually preferred to full maximum likelihood since estimators of the parameters in the regression function obtained are more robust to misspecification of the variance function. Despite its popularity, however, existing theoretical results are difficult to apply and are of limited use in many applications. Using more recent results in estimating equations, we obtain an efficient algorithm for computing the pseudo-likelihood estimator with desirable convergence properties and also derive simple, explicit and easy to apply asymptotic results. These results are used to look in detail at variance minimization in off-line quality control, yielding techniques of inferences for the optimized design parameter. In application of some existing approaches to off-line quality control, such as the dual response methodology, rigorous statistical inference techniques are scarce and difficult to obtain. An example of off-line quality control is presented to discuss the practical aspects involved in the application of the results obtained and to address issues such as data transformation, model building and the optimization of design parameters. The analysis shows very encouraging results, and is seen to be able to unveil some important information not found in previous analyses.