Multiple Linear Regression Model Under Nonnormality

Abstract

We consider multiple linear regression models under nonnormality. We derive modified maximum likelihood estimators (MMLEs) of the parameters and show that they are efficient and robust. We show that the least squares esimators are considerably less efficient. We compare the efficiencies of the MMLEs and the M estimators for symmetric distributions and show that, for plausible alternatives to an assumed distribution, the former are more efficient. We provide real-life examples.     

A General Class of Estimators for the Linear Regression Model Affected by Collinearity and Outliers

In this article, a general class of estimators for the linear regression model affected by outliers and collinearity is introduced and studied in some detail. This class of estimators combines the theory of light, maximum entropy, and robust regression techniques. Our theoretical findings are illustrated through a Monte Carlo simulation study.     

Linearized Ridge Regression Estimator Under the Mean Squared Error Criterion in a Linear Regression Model

This article mainly aims to study the superiority of the notion of linearized ridge regression estimator (LRRE) under the mean squared error criterion in a linear regression model. Firstly, we derive uniform lower bound of MSE for the class of the generalized shrinkage estimator (GSE), based on which it is shown that the optimal LRRE is the best estimator in the class of GSE's. Secondly, we propose the notion of the almost unbiased completeness and show that LRRE possesses such a property. Thirdly, the simulation study is given, from which it indicates that the LRRE performs desirably. Finally, the main results are applied to the well known Hald data.     

Simultaneous Two-Sided Tolerance Intervals for a Univariate Linear Regression Model

In this article we deal with simultaneous two-sided tolerance intervals for a univariate linear regression model with independent normally distributed errors. We present a method for determining the intervals derived by the general confidence-set approach (GCSA), i.e. the intervals are constructed based on a specified confidence set for unknown parameters of the model. The confidence set used in the new method is formed based on a suggested hypothesis test about all parameters of the model. The simultaneous two-sided tolerance intervals determined by the presented method are found to be efficient and fast to compute based on a preliminary numerical comparison of all the existing methods based on GCSA.     

计量经济学和统计学视角下的线性回归模型——再议线性回归模型的设定

从属性、构建方法及意义等方面,分析研究线性回归模型在计量经济学和统计学两学科视角下的差异,并根据这种差异进一步提出回归模型的基本设定思路。研究表明:识别这种差异是完成模型设定工作的基础性和必要性举措,有助于实现线性回归模型的正确设定。以经典例证对计量经济学和统计学回归模型在应用中的区别以及模型设定问题进行进一步展示和分析。     

Heteroscedasticity and Distributional Assumptions in the Censored Regression Model

Data censoring causes ordinary least-square estimators of linear models to be biased and inconsistent. The Tobit estimator yields consistent estimators in the presence of data censoring if the errors are normally distributed. However, nonnormality or heteroscedasticity results in the Tobit estimators being inconsistent. Various estimators have been proposed for circumventing the normality assumption. Some of these estimators include censored least absolute deviations (CLAD), symmetrically censored least-square (SCLS), and partially adaptive estimators. CLAD and SCLS will be consistent in the presence of heteroscedasticity; however, SCLS performs poorly in the presence of asymmetric errors. This article extends the partially adaptive estimation approach to accommodate possible heteroscedasticity as well as nonnormality. A simulation study is used to investigate the estimators’ relative performance in these settings. The partially adaptive censored regression estimators have little efficiency loss for censored normal errors and appear to outperform the Tobit and semiparametric estimators for nonnormal error distributions and be less sensitive to the presence of heteroscedasticity. An empirical example is considered, which supports these results.     

Statistical Inference in Orthogonal Regression for Three-Part Compositional Data Using a Linear Model with Type-II Constraints

Orthogonal regression is a proper tool to analyze relations between two variables when three-part compositional data, i.e., three-part observations carrying relative information (like proportions or percentages), are under examination. When linear statistical models with type-II constraints (constraints involving other parameters besides the ones of the unknown model) are employed for estimating the parameters of the regression line, approximate variances and covariances of the estimated line coefficients can be determined. Moreover, the additional assumption of normality enables to construct confidence domains and perform hypotheses testing. The theoretical results are applied to a real-world example.     

Partially Adaptive Estimation of the Censored Regression Model

Data censoring causes ordinary least squares estimates of linear models to be biased and inconsistent. Tobit, semiparametric, and partially adaptive estimators have been considered as possible solutions. This paper proposes several new partially adaptive estimators that cover a wide range of distributional characteristics. A simulation study is used to investigate the estimators’ relative efficiency in these settings. The partially adaptive censored regression estimators have little efficiency loss for censored normal errors and may outperform Tobit and semiparametric estimators considered for non-normal distributions. An empirical example of out-of-pocket expenditures for a health insurance plan provides an example, which supports these results.     

A Logspline Estimation for a Linear Regression Model with an Interval-Censored Continuous Covariate

The study of a linear regression model with an interval-censored covariate, which was motivated by an acquired immunodeficiency syndrome (AIDS) clinical trial, was first proposed by Gómez et al. They developed a likelihood approach, together with a two-step conditional algorithm, to estimate the regression coefficients in the model. However, their method is inapplicable when the interval-censored covariate is continuous. In this article, we propose a novel and fast method to treat the continuous interval-censored covariate. By using logspline density estimation, we impute the interval-censored covariate with a conditional expectation. Then, the ordinary least-squares method is applied to the linear regression model with the imputed covariate. To assess the performance of the proposed method, we compare our imputation with the midpoint imputation and the semiparametric hierarchical method via simulations. Furthermore, an application to the AIDS clinical trial is presented.     

Model Selection and Regression t-Statistics

It is shown that dropping quantitative variables from a linear regression, based on t-statistics, is mathematically equivalent to dropping variables based on commonly used information criteria.     

Empirical Likelihood Ratio Test for a Change-Point in Linear Regression Model

A nonparametric method based on the empirical likelihood is proposed to detect the change-point in the coefficient of linear regression models. The empirical likelihood ratio test statistic is proved to have the same asymptotic null distribution as that with classical parametric likelihood. Under some mild conditions, the maximum empirical likelihood change-point estimator is also shown to be consistent. The simulation results show the sensitivity and robustness of the proposed approach. The method is applied to some real datasets to illustrate the effectiveness.     

Variational Inference of Linear Regression with Nonzero Prior Means

In this article, we employ the variational Bayesian method to study the parameter estimation problems of linear regression model, wherein some regressors are of Gaussian distribution with nonzero prior means. We obtain an analytical expression of the posterior parameter distribution, and then propose an iterative algorithm for the model. Simulations are carried out to test the performance of the proposed algorithm, and the simulation results confirm both the effectiveness and the reliability of the proposed algorithm.     

Multivariate Linear Regression Model with Elliptically Contoured Distributed Errors and Monotone Missing Dependent Variables

In this article, the multivariate linear regression model is studied under the assumptions that the error term of this model is described by the elliptically contoured distribution and the observations on the response variables are of a monotone missing pattern. It is primarily concerned with estimation of the model parameters, as well as with the development of the likelihood ratio test in order to examine the existence of linear constraints on the regression coefficients. An illustrative example is presented for the explanation of the results.     

Shrinkage and Penalty Estimators of a Poisson Regression Model

In this paper we propose Stein‐type shrinkage estimators for the parameter vector of a Poisson regression model when it is suspected that some of the parameters may be restricted to a subspace. We develop the properties of these estimators using the notion of asymptotic distributional risk. The shrinkage estimators are shown to have higher efficiency than the classical estimators for a wide class of models. Furthermore, we consider three different penalty estimators: the LASSO, adaptive LASSO, and SCAD estimators and compare their relative performance with that of the shrinkage estimators. Monte Carlo simulation studies reveal that the shrinkage strategy compares favorably to the use of penalty estimators, in terms of relative mean squared error, when the number of inactive predictors in the model is moderate to large. The shrinkage and penalty strategies are applied to two real data sets to illustrate the usefulness of the procedures in practice.     

Bayesian Estimation of the Number of Change Points in Simple Linear Regression Models

A Bayesian approach is considered to detect the number of change points in simple linear regression models. A normal-gamma empirical prior for the regression parameters based on maximum likelihood estimator (MLE) is employed in the analysis. Under mild conditions, consistency for the number of change points and boundedness between the estimated location and the true location of the change points are established. The Bayesian approach to the detection of the number of change points is suitable whether the switching simple regression is continuous or discontinuous. Some simulation results are given to confirm the accuracy of the proposed estimator.     

带线性约束的多元线性回归模型参数估计

本文首先构造线性约束条件下的多元线性回归模型的样本似然函数,利用Lagrange法证明其合理性。其次,从似然函数的角度讨论线性约束条件对模型参数的影响,对由传统理论得出的参数估计作出贝叶斯与经验贝叶斯的改进。做贝叶斯改进时,将矩阵正态-Wishart分布作为模型参数和精度阵的联合共轭先验分布,结合构造的似然函数得出参数的后验分布,计算出参数的贝叶斯估计;做经验贝叶斯改进时,将样本分组,从方差的角度讨论由子样得出的参数估计对总样本的参数估计的影响,计算出经验贝叶斯估计。最后,利用Matlab软件生成的随机矩阵做模拟。结果表明,这两种改进后的参数估计均较由传统理论得出的参数估计更精确,拟合结果的误差比更小,可信度更高,在大数据的情况下,这种计算方法的速度更快。     

Estimation of Slope for Linear Regression Model with Uncertain Prior Information and Student-t Error

This article considers estimation of the slope parameter of the linear regression model with Student-t errors in the presence of uncertain prior information on the value of the unknown slope. Incorporating uncertain non sample prior information with the sample data the unrestricted, restricted, preliminary test, and shrinkage estimators are defined. The performances of the estimators are compared based on the criteria of unbiasedness and mean squared errors. Both analytical and graphical methods are explored. Although none of the estimators is uniformly superior to the others, if the non sample information is close to its true value, the shrinkage estimator over performs the rest of the estimators.     

The Use of Polynomials to Shift Coefficients in Linear Regression Models

Polynomials are commonly used in linear regression models to capture nonlinearities in explanatory variables. It is less common, however, that polynomials are used to shift the regression coefficients, an exception being the use of polynomially distributed lag coefficients. This note recommends the technique for a wider range of applications and suggests the Lagrangian interpolation representation as the most convenient for practitioners.     

Quasi-Minimax Estimation in the Linear Model with Measurement Errors

In this article, we consider quasi-minimax estimation in the linear regression model where some covariates are measured with additive errors. When measurement errors are directly ignored the minimax risk of the resulting estimator can be large. By correcting the attenuation we propose a penalized quadratic risk function. A simulation study is conducted to illustrate the performance of the proposed estimators.     

隐非齐次马尔可夫多元线性回归模型及其贝叶斯估计

隐马尔可夫模型对于异质纵向数据的处理有良好的效果,因此被广泛应用于工程技术、生物医学、经济管理等领域。文章引入了一种特殊的非齐次隐马尔可夫状态转移方式,并将其与经典的多元线性回归相结合,提出了隐非齐次马尔可夫多元线性回归模型,介绍了对该模型进行贝叶斯推断的方法原理和技术细节。最后,通过两个模拟实验说明了推断方法的结果是可靠的。