Iterative weighted estimation based on variance modelling in linear regression models

ABSTRACT

The estimation of variance function plays an extremely important role in statistical inference of the regression models. In this paper we propose a variance modelling method for constructing the variance structure via combining the exponential polynomial modelling method and the kernel smoothing technique. A simple estimation method for the parameters in heteroscedastic linear regression models is developed when the covariance matrix is unknown diagonal and the variance function is a positive function of the mean. The consistency and asymptotic normality of the resulting estimators are established under some mild assumptions. In particular, a simple version of bootstrap test is adapted to test misspecification of the variance function. Some Monte Carlo simulation studies are carried out to examine the finite sample performance of the proposed methods. Finally, the methodologies are illustrated by the ozone concentration dataset.     

Estimation of rational transfer function models

In the estimation of rational transfer function models, it has been recommended that starting values of a transfer function component be assumed to be zero (or a constant) in the recursive computation of the transfer function response. It is demonstrated that such algorithms may lead to serious bias in the estimation of moving average parameters. This paper discusses several other algorithms that may rectify this problem. It is found that the starting-value-free (SVF) method is a more reliable algorithm. For computer programs using the traditional algorithm, i.e., the zero-starting-value (ZSV) method, the bias problem can be easily remedied using a short-cut method that omits appropriate number of values at the beginning of the residual series.     

Superiority comparisons of heterogeneous linear estimators

In this paper conditions for strong and weak superiority of a heterogeneous linear estimator over another are derived. The general results are applied to some special cases: in particular, two restricted least squares estimators are compared using the superiority conditions obtained. The weak superiority criterion is used as a basis in forming a generalization of an optimal se-quence of tests (Anderson, 1962) for searching for the best estimator when the alternative linear restrictions form a nested se-quence of hypotheses. An application of this is the determination of the correct length of lag and appropriate degree of polynomial in the estimation of polynomial distributed lag models.     

Indirect inference for fractional time series models

In this paper we present an indirect estimation procedure for (ARFIMA) fractional time series models.The estimation method is based on an ‘incorrect’criterion which does not directly provide a consistent estimator of the parameters of interest,but leads to correct inference by using simulations.

The main steps are the following. First,we consider an auxiliary model which can be easily estimated.Specifically,we choose the finite lag Autoregressive model.Then, this is estimated on the observations and simulated values drawn from the ARFIMA model associated with a given value of the parameters of interest.Finally,the latter is calibrated in order to obtain close values of the two estimators of the auxiliary parameters.

In this article,we describe the estimation procedure and compare the performance of the indirect estimator with some alternative estimators based on the likelihood function by a Monte Carlo study.     

Inferences in dynamic logit models in semi-parametric setup for repeated binary data

Binary dynamic fixed and mixed logit models are extensively studied in the literature. These models are developed to examine the effects of certain fixed covariates through a parametric regression function as a part of the models. However, there are situations where one may like to consider more covariates in the model but their direct effect is not of interest. In this paper we propose a generalization of the existing binary dynamic logit (BDL) models to the semi-parametric longitudinal setup to address this issue of additional covariates. The regression function involved in such a semi-parametric BDL model contains (i) a parametric linear regression function in some primary covariates, and (ii) a non-parametric function in certain secondary covariates. We use a simple semi-parametric conditional quasi-likelihood approach for consistent estimation of the non-parametric function, and a semi-parametric likelihood approach for the joint estimation of the main regression and dynamic dependence parameters of the model. The finite sample performance of the estimation approaches is examined through a simulation study. The asymptotic properties of the estimators are also discussed. The proposed model and the estimation approaches are illustrated by reanalysing a longitudinal infectious disease data.     

Empirical likelihood estimation for linear regression models with AR(p) error terms with numerical examples

Linear regression models are useful statistical tools to analyze data sets in different fields. There are several methods to estimate the parameters of a linear regression model. These methods usually perform under normally distributed and uncorrelated errors. If error terms are correlated the Conditional Maximum Likelihood (CML) estimation method under normality assumption is often used to estimate the parameters of interest. The CML estimation method is required a distributional assumption on error terms. However, in practice, such distributional assumptions on error terms may not be plausible. In this paper, we propose to estimate the parameters of a linear regression model with autoregressive error term using Empirical Likelihood (EL) method, which is a distribution free estimation method. A small simulation study is provided to evaluate the performance of the proposed estimation method over the CML method. The results of the simulation study show that the proposed estimators based on EL method are remarkably better than the estimators obtained from CML method in terms of mean squared errors (MSE) and bias in almost all the simulation configurations. These findings are also confirmed by the results of the numerical and real data examples.     

Efficient Estimation of the Partly Linear Additive Hazards Model with Current Status Data

This paper focuses on efficient estimation, optimal rates of convergence and effective algorithms in the partly linear additive hazards regression model with current status data. We use polynomial splines to estimate both cumulative baseline hazard function with monotonicity constraint and nonparametric regression functions with no such constraint. We propose a simultaneous sieve maximum likelihood estimation for regression parameters and nuisance parameters and show that the resultant estimator of regression parameter vector is asymptotically normal and achieves the semiparametric information bound. In addition, we show that rates of convergence for the estimators of nonparametric functions are optimal. We implement the proposed estimation through a backfitting algorithm on generalized linear models. We conduct simulation studies to examine the finite‐sample performance of the proposed estimation method and present an analysis of renal function recovery data for illustration.     

Quantile regression estimation of partially linear additive models

In this paper, we consider the estimation of partially linear additive quantile regression models where the conditional quantile function comprises a linear parametric component and a nonparametric additive component. We propose a two-step estimation approach: in the first step, we approximate the conditional quantile function using a series estimation method. In the second step, the nonparametric additive component is recovered using either a local polynomial estimator or a weighted Nadaraya–Watson estimator. Both consistency and asymptotic normality of the proposed estimators are established. Particularly, we show that the first-stage estimator for the finite-dimensional parameters attains the semiparametric efficiency bound under homoskedasticity, and that the second-stage estimators for the nonparametric additive component have an oracle efficiency property. Monte Carlo experiments are conducted to assess the finite sample performance of the proposed estimators. An application to a real data set is also illustrated.     

MIDAS Regressions: Further Results and New Directions

We explore mixed data sampling (henceforth MIDAS) regression models. The regressions involve time series data sampled at different frequencies. Volatility and related processes are our prime focus, though the regression method has wider applications in macroeconomics and finance, among other areas. The regressions combine recent developments regarding estimation of volatility and a not-so-recent literature on distributed lag models. We study various lag structures to parameterize parsimoniously the regressions and relate them to existing models. We also propose several new extensions of the MIDAS framework. The paper concludes with an empirical section where we provide further evidence and new results on the risk-return trade-off. We also report empirical evidence on microstructure noise and volatility forecasting.     

Bayesian quantile regression for hierarchical linear models

The paper proposes a Bayesian quantile regression method for hierarchical linear models. Existing approaches of hierarchical linear quantile regression models are scarce and most of them were not from the perspective of Bayesian thoughts, which is important for hierarchical models. In this paper, based on Bayesian theories and Markov Chain Monte Carlo methods, we introduce Asymmetric Laplace distributed errors to simulate joint posterior distributions of population parameters and across-unit parameters and then derive their posterior quantile inferences. We run a simulation as the proposed method to examine the effects on parameters induced by units and quantile levels; the method is also applied to study the relationship between Chinese rural residents' family annual income and their cultivated areas. Both the simulation and real data analysis indicate that the method is effective and accurate.     

Fast and robust bootstrap

In this paper we review recent developments on a bootstrap method for robust estimators which is computationally faster and more resistant to outliers than the classical bootstrap. This fast and robust bootstrap method is, under reasonable regularity conditions, asymptotically consistent. We describe the method in general and then consider its application to perform inference based on robust estimators for the linear regression and multivariate location-scatter models. In particular, we study confidence and prediction intervals and tests of hypotheses for linear regression models, inference for location-scatter parameters and principal components, and classification error estimation for discriminant analysis.     

A Bayesian estimation of lag lengths in distributed lag models

Dynamic regression models are widely used because they express and model the behaviour of a system over time. In this article, two dynamic regression models, the distributed lag (DL) model and the autoregressive distributed lag model, are evaluated focusing on their lag lengths. From a classical statistics point of view, there are various methods to determine the number of lags, but none of them are the best in all situations. This is a serious issue since wrong choices will provide bad estimates for the effects of the regressors on the response variable. We present an alternative for the aforementioned problems by considering a Bayesian approach. The posterior distributions of the numbers of lags are derived under an improper prior for the model parameters. The fractional Bayes factor technique [A. O'Hagan, Fractional Bayes factors for model comparison (with discussion), J. R. Statist. Soc. B 57 (1995), pp. 99–138] is used to handle the indeterminacy in the likelihood function caused by the improper prior. The zero-one loss function is used to penalize wrong decisions. A naive method using the specified maximum number of DLs is also presented. The proposed and the naive methods are verified using simulation data. The results are promising for the method we proposed. An illustrative example with a real data set is provided.     

Identification of non-linear additive autoregressive models

Summary.  We propose a lag selection method for non-linear additive autoregressive models that is based on spline estimation and the Bayes information criterion. The additive structure of the autoregression function is used to overcome the 'curse of dimensionality', whereas the spline estimators effectively take into account such a structure in estimation. A stepwise procedure is suggested to implement the method proposed. A comprehensive Monte Carlo study demonstrates good performance of the method proposed and a substantial computational advantage over existing local-polynomial-based methods. Consistency of the lag selection method based on the Bayes information criterion is established under the assumption that the observations are from a stochastic process that is strictly stationary and strongly mixing, which provides the first theoretical result of this kind for spline smoothing of weakly dependent data.     

Partially Linear Additive Models with Unknown Link Functions

In this paper, we consider partially linear additive models with an unknown link function, which include single‐index models and additive models as special cases. We use polynomial spline method for estimating the unknown link function as well as the component functions in the additive part. We establish that convergence rates for all nonparametric functions are the same as in one‐dimensional nonparametric regression. For a faster rate of the parametric part, we need to define appropriate ‘projection’ that is more complicated than that defined previously for partially linear additive models. Compared to previous approaches, a distinct advantage of our estimation approach in implementation is that estimation directly reduces estimation in the single‐index model and can thus deal with much larger dimensional problems than previous approaches for additive models with unknown link functions. Simulations and a real dataset are used to illustrate the proposed model.     

On biased linear estimators in models with arbitrary rank

In this paper we define a class of biased linear estimators for the unknown parameters in linear models with arbitrary rank. The feature of our approach is to reduce the estimation problem in arbitrary rank models to the one in full-rank models. Some important properties are discussed. As special cases of our class, we extend to deficient-rank models six known biased linear estimators.     

Ordered Regressions

There are often situations where two or more regression functions are ordered over a range of covariate values. In this paper, we develop efficient constrained estimation and testing procedures for such models. Specifically, necessary and sufficient conditions for ordering generalized linear regressions are given and shown to unify previous results obtained for simple linear regression, for polynomial regression and in the analysis of covariance models. We show that estimating the parameters of ordered linear regressions requires either quadratic programming or semi‐infinite programming, depending on the shape of the covariate space. A distance‐type test for order is proposed. Simulations demonstrate that the proposed methodology improves the mean square error and power compared with the usual, unconstrained, estimation and testing procedures. Improvements are often substantial. The methodology is extended to order generalized linear models where convex semi‐infinite programming plays a role. The methodology is motivated by, and applied to, a hearing loss study.     

The convolution theorem for estimating linear functionals in indirect nonparametric regression models

Nonparametric regression—directly or indirectly observed—is one of the important statistical models. On one hand it contains two infinite dimensional parameters (the regression function and the error density), and on the other it is of rather simple structure. Therefore, it may serve as an interesting paradigm for illustrating or developing abstract statistical theory for non-Euclidean parameters. In this paper estimation of a linear functional of the indirectly observed regression function is considered, when a deterministic design is used. It should be noted that any Fourier coefficient of an expansion of the regression function in an orthonormal basis is such a functional. Because the design is deterministic the observables are independent but not identically distributed. Local asymptotic normality is established and applied to prove Hájek's convolution theorem for this functional. Pertinent references are Beran [1977. Robust location estimates. Ann. Statist. 5, 431–444] and McNeney and Wellner [2000. Application of convolution theorems in semiparametric models with non-i.i.d. data. J. Statist. Plann. Inference 91, 441–480]. For purposes explained above, however, the paper is kept self-contained and full proofs are provided.     

异质性大数据的分布式估计

随着物联网技术的进步,大数据给网络带宽和计算机存储能力带来巨大挑战,传统的集中式数据处理难以实现,客观上促进了分布式统计学习的发展。在无迭代算法研究中,Zhang等(2013)证明了当数据集个数s=O(N) 时,基于局部经验风险最小化的分治(DC)简单平均估计量具有O(N-1)均方误差收敛速度,Huang和Huo(2019)在M估计框架下进一步提出分布式一步估计量,但上述方法均未考虑海量数据可能存在的异质性对分治估计效果的影响。本文在线性模型框架下提出海量异质数据的分治一步加权估计,证明了估计量的渐近性质并考虑了异质性检验问题。将本文提出的方法应用于美国医疗保险实际数据分析,结果表明该方法能更好地拟合数据的线性趋势且显著提高了计算效率。     

Keep it simple: estimation strategies for ordered response models with fixed effects

By running Monte Carlo simulations, we compare different estimation strategies of ordered response models in the presence of non-random unobserved heterogeneity. We find that very simple binary recoding schemes deliver parameter estimates with very low bias and high efficiency. Furthermore, if the researcher is interested in the relative size of parameters the simple linear fixed effects model is the method of choice.