The small sample properties of the restricted principal component regression estimator in linear regression model

In regression analysis, to deal with the problem of multicollinearity, the restricted principal components regression estimator is proposed. In this paper, we compared the restricted principal components regression estimator, the principal components regression estimator, and the ordinary least-squares estimator with each other under the Pitman's closeness criterion. We showed that the restricted principal components regression estimator is always superior to the principal components regression estimator, under certain conditions the restricted principal components regression estimator is superior to the ordinary least-squares estimator under the Pitman's closeness criterion and under certain conditions the principal components regression estimator is superior to the ordinary least-squares estimator under the Pitman's closeness criterion.     

基于半参数方法的模型辅助抽样估计研究

在总结现有模型辅助估计方法的基础上,本文通过构造一种半参数超总体模型,同时结合广义差分估计思想提出一种新型的模型辅助估计量。该估计量比传统的非参数和半参数回归估计利用更少、更易得到的辅助信息,即只需利用和广义回归估计相同的辅助信息,但一般会比广义回归估计拥有更高的估计精度。理论证明了该估计量是渐近设计无偏和设计一致的,其渐近设计均方误差为广义差分估计量的方差。模拟结果显示：其至少与广义回归估计一样好;对于线性程度越低的超总体模型,其估计精度比广义回归估计有越明显的提高;就本文模拟而言,光滑参数在0.04～0.12间适当取值时其会取到相对较好的估计效果。     

THE EXAMINATION AND ANALYSIS OF RESIDUALS FOR SOME BIASED ESTIMATORS IN LINEAR REGRESSION

In the presence of collinearity certain biased estimation procedures like ridge regression, generalized inverse estimator, principal component regression, Liu estimator, or improved ridge and Liu estimators are used to improve the ordinary least squares (OLS) estimates in the linear regression model. In this paper new biased estimator (Liu estimator), almost unbiased (improved) Liu estimator and their residuals will be analyzed and compared with OLS residuals in terms of mean-squared error.     

More on the restricted ridge regression estimation

Several alternative methods for derivation of the restricted ridge regression estimator (RRRE) are provided. Theoretical comparison and relationship of RRRE with related methods for regression with the multicollinearity problem are described. We also find inter-connections among RRRE, ordinary ridge regression estimator (ORRE), restricted least squares estimator (RLSE), modified ridge regression estimator (MRRE) and restricted modified generalized ridge estimator (RMGRE). Finally, numerical comparison, in addition to theoretical derivation, is also conducted with a Monte Carlo simulation and a real data example.     

EXACT FINITE-SAMPLE PROPERTIES OF A PRE-TEST ESTIMATOR IN RIDGE REGRESSION

This paper discusses a pre-test regression estimator which uses the least squares estimate when it is “large” and a ridge regression estimate for “small” regression coefficients, where the preliminary test is applied separately to each regression coefficient in turn to determine whether it is “large” or “small.” For orthogonal regressors, the exact finite-sample bias and mean squared error of the pre-test estimator are derived. The latter is less biased than a ridge estimator, and over much of the parameter space the pre-test estimator has smaller mean squared error than least squares. A ridge estimator is found to be inferior to the pre-test estimator in terms of mean squared error in many situations, and at worst the latter estimator is only slightly less efficient than the former at commonly used significance levels.     

r − k Class estimator in the linear regression model with correlated errors

Autocorrelation in errors and multicollinearity among the regressors are serious problems in regression analysis. The aim of this paper is to examine multicollinearity and autocorrelation problems concurrently and to compare the r ? k class estimator to the generalized least squares estimator, the principal components regression estimator and the ridge regression estimator by the scalar and matrix mean square error criteria in the linear regression model with correlated errors.     

On the Restricted Liu Estimator in the Logistic Regression Model

The logistic regression model is used when the response variables are dichotomous. In the presence of multicollinearity, the variance of the maximum likelihood estimator (MLE) becomes inflated. The Liu estimator for the linear regression model is proposed by Liu to remedy this problem. Urgan and Tez and Mansson et al. examined the Liu estimator (LE) for the logistic regression model. We introduced the restricted Liu estimator (RLE) for the logistic regression model. Moreover, a Monte Carlo simulation study is conducted for comparing the performances of the MLE, restricted maximum likelihood estimator (RMLE), LE, and RLE for the logistic regression model.     

Rank-based ridge estimation in multiple linear regression

Multicollinearity and model misspecification are frequently encountered problems in practice that produce undesirable effects on classical ordinary least squares (OLS) regression estimator. The ridge regression estimator is an important tool to reduce the effects of multicollinearity, but it is still sensitive to a model misspecification of error distribution. Although rank-based statistical inference has desirable robustness properties compared to the OLS procedures, it can be unstable in the presence of multicollinearity. This paper introduces a rank regression estimator for regression parameters and develops tests for general linear hypotheses in a multiple linear regression model. The proposed estimator and the tests have desirable robustness features against the multicollinearity and model misspecification of error distribution. Asymptotic behaviours of the proposed estimator and the test statistics are investigated. Real and simulated data sets are used to demonstrate the feasibility and the performance of the estimator and the tests.     

Local Likelihood Estimation in Generalized Additive Models

ABSTRACT.  Generalized additive models are a popular class of multivariate non-parametric regression models, due in large part to the ease of use of the local scoring estimation algorithm. However, the theoretical properties of the local scoring estimator are poorly understood. In this article, we propose a local likelihood estimator for generalized additive models that is closely related to the local scoring estimator fitted by local polynomial regression. We derive the statistical properties of the estimator and show that it achieves the same asymptotic convergence rate as a one-dimensional local polynomial regression estimator. We also propose a wild bootstrap estimator for calculating point-wise confidence intervals for the additive component functions. The practical behaviour of the proposed estimator is illustrated through a simulation experiment.     

A Wald-type test statistic based on robust modified median estimator in logistic regression models

In this paper a new robust estimator, modified median estimator, is introduced and studied for the logistic regression model. This estimator is based on the median estimator considered in Hobza et al. [Robust median estimator in logistic regression. J Stat Plan Inference. 2008;138:3822–3840]. Its asymptotic distribution is obtained. Using the modified median estimator, we also consider a Wald-type test statistic for testing linear hypotheses in the logistic regression model and we obtain its asymptotic distribution under the assumption of random regressors. An extensive simulation study is presented in order to analyse the efficiency as well as the robustness of the modified median estimator and Wald-type test based on it.     

多辅助变量线性组合的回归估计

以提高估计量的精度为目的,定义了一种新的多个辅助变量的回归估计法,从理论上研究了该方法下权的选取方法,并将该方法下的估计量与Raj多辅助变量回归估计量、Ghosh多元线性回归估计量在精度上进行了数值比较,结果表明：这种新的多辅助变量的回归估计法在精度上优于Raj多辅助变量回归估计和Ghosh多元线性回归估计。     

The mixed model for survey regression estimation

Estimation of the population mean under the regression model with random components is considered. Conditions under which the random components regression estimator is design consistent are given. It is shown that consistency holds when incorrect values are used for the variance components. The regression estimator constructed with model parameters that differ considerably from the true parameters performed well in a Monte Carlo study. Variance estimators for the regression predictor are suggested. A variance estimator appropriate for estimators constructed with a biased estimator for the between-group variance component performed well in the Monte Carlo study.     

Principal components regression estimator and a test for the restrictions

In this article, we introduce restricted principal components regression (RPCR) estimator by combining the approaches followed in obtaining the restricted least squares estimator and the principal components regression estimator. The performance of the RPCR estimator with respect to the matrix and the generalized mean square error are examined. We also suggest a testing procedure for linear restrictions in principal components regression by using singly and doubly non-central F distribution.     

Restricted ridge estimator in the logistic regression model

It is known that when the multicollinearity exists in the logistic regression model, variance of maximum likelihood estimator is unstable. As a remedy, Schaefer et al. presented a ridge estimator in the logistic regression model. Making use of the ridge estimator, when some linear restrictions are also present, we introduce a restricted ridge estimator in the logistic regression model. Statistical properties of this newly defined estimator will be studied and comparisons are done in the simulation study in the sense of mean squared error criterion. A real-data example and a simulation study are introduced to discuss the performance of this estimator.     

The feasible generalized restricted ridge regression estimator

The presence of autocorrelation in errors and multicollinearity among the regressors have undesirable effects on the least-squares regression. There are a wide range of methods which are proposed to overcome the usefulness of the ordinary least-squares estimator or the generalized least-squares estimator, such as the Stein-rule, restricted least-squares or ridge estimator. Therefore, we introduce a new feasible generalized restricted ridge regression (FGRR) estimator to examine multicollinearity and autocorrelation problems simultaneously for the general linear regression model. We also derive some statistical properties of the FGRR estimator and comparisons have been conducted using matrix mean-square error. Moreover, a Monte Carlo simulation experiment is performed to investigate the performance of the proposed estimator over the others.     

Mixed Discrete and Continuous Cox Regression Model

The Cox (1972) regression model is extended to include discrete and mixed continuous/discrete failure time data by retaining the multiplicative hazard rate form of the absolutely continuous model. Application of martingale arguments to the regression parameter estimating function show the Breslow (1974) estimator to be consistent and asymptotically Gaussian under this model. A computationally convenient estimator of the variance of the score function can be developed, again using martingale arguments. This estimator reduces to the usual hypergeometric form in the special case of testing equality of several survival curves, and it leads more generally to a convenient consistent variance estimator for the regression parameter. A small simulation study is carried out to study the regression parameter estimator and its variance estimator under the discrete Cox model special case and an application to a bladder cancer recurrence dataset is provided.     

On the predictive performance of the almost unbiased Liu estimator

ABSTRACT

Regression models are usually used in forecasting (predicting) unknown values of the response variable y. This article considers the predictive performance of the almost unbiased Liu estimator compared to the ordinary least-squares estimator, principal component regression estimator, and Liu estimator. Finally, we present a numerical example to explain the theoretical results and we obtain a region where the almost unbiased Liu estimator is uniformly superior to the ordinary least-squares estimator, principal component regression estimator, and Liu estimator.     

Automated selection of post‐strata using a model‐assisted regression tree estimator

Despite having desirable properties, model‐assisted estimators are rarely used in anything but their simplest form to produce official statistics. This is due to the fact that the more complicated models are often ill suited to the available auxiliary data. Under a model‐assisted framework, we propose a regression tree estimator for a finite‐population total. Regression tree models are adept at handling the type of auxiliary data usually available in the sampling frame and provide a model that is easy to explain and justify. The estimator can be viewed as a post‐stratification estimator where the post‐strata are automatically selected by the recursive partitioning algorithm of the regression tree. We establish consistency of the regression tree estimator and a variance estimator, along with asymptotic normality of the regression tree estimator. We compare the performance of our estimator to other survey estimators using the United States Bureau of Labor Statistics Occupational Employment Statistics Survey data.     

Estimation in a linear regression model with stochastic linear restrictions: a new two-parameter-weighted mixed estimator

The present paper considers the weighted mixed regression estimation of the coefficient vector in a linear regression model with stochastic linear restrictions binding the regression coefficients. We introduce a new two-parameter-weighted mixed estimator (TPWME) by unifying the weighted mixed estimator of Schaffrin and Toutenburg [1] and the two-parameter estimator (TPE) of Özkale and Kaç?ranlar [2]. This new estimator is a general estimator which includes the weighted mixed estimator, the TPE and the restricted two-parameter estimator (RTPE) proposed by Özkale and Kaç?ranlar [2] as special cases. Furthermore, we compare the TPWME with the weighted mixed estimator and the TPE with respect to the matrix mean square error criterion. A numerical example and a Monte Carlo simulation experiment are presented by using different estimators of the biasing parameters to illustrate some of the theoretical results.     

数据随机缺失下分位数回归模型的诱导光滑估计法

在数据随机缺失的分位数回归模型中,运用诱导光滑思想构造光滑的估计方程,得到了回归参数的诱导光滑估计及渐近协方差估计。接着证明了诱导光滑估计的渐近正态性质,并给出诱导光滑估计及其渐近协方差估计的算法。模拟研究表明新方法在有限样本中表现出色。