Linearized Ridge Regression Estimator in Linear Regression

In this article, we aim to study the linearized ridge regression (LRR) estimator in a linear regression model motivated by the work of Liu (1993). The LRR estimator and the two types of generalized Liu estimators are investigated under the PRESS criterion. The method of obtaining the optimal generalized ridge regression (GRR) estimator is derived from the optimal LRR estimator. We apply the Hald data as a numerical example and then make a simulation study to show the main results. It is concluded that the idea of transforming the GRR estimator as a complicated function of the biasing parameters to a linearized version should be paid more attention in the future.     

Combining Unbiased Ridge and Principal Component Regression Estimators

In the presence of multicollinearity problem, ordinary least squares (OLS) estimation is inadequate. To circumvent this problem, two well-known estimation procedures often suggested are the unbiased ridge regression (URR) estimator given by Crouse et al. (1995 Crouse , R. , Jin , C. , Hanumara , R. ( 1995 ). Unbiased ridge estimation with prior information and ridge trace . Commun. Statist. Theor. Meth. 24 : 2341 – 2354 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) and the (r, k) class estimator given by Baye and Parker (1984 Baye , M. , Parker , D. ( 1984 ). Combining ridge and principal component regression: a money demand illustration . Commun. Statist. Theor. Meth. 13 : 197 – 205 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]). In this article, we proposed a new class of estimators, namely modified (r, k) class ridge regression (MCRR) which includes the OLS, the URR, the (r, k) class, and the principal components regression (PCR) estimators. It is based on a criterion that combines the ideas underlying the URR and the PCR estimators. The standard properties of this new class estimator have been investigated and a numerical illustration is done. The conditions under which the MCRR estimator is better than the other two estimators have been investigated.     

Improved two-parameter estimators for the negative binomial and Poisson regression models

Negative binomial regression (NBR) and Poisson regression (PR) applications have become very popular in the analysis of count data in recent years. However, if there is a high degree of relationship between the independent variables, the problem of multicollinearity arises in these models. We introduce new two-parameter estimators (TPEs) for the NBR and the PR models by unifying the two-parameter estimator (TPE) of Özkale and Kaç?ranlar [The restricted and unrestricted two-parameter estimators. Commun Stat Theory Methods. 2007;36:2707–2725]. These new estimators are general estimators which include maximum likelihood (ML) estimator, ridge estimator (RE), Liu estimator (LE) and contraction estimator (CE) as special cases. Furthermore, biasing parameters of these estimators are given and a Monte Carlo simulation is done to evaluate the performance of these estimators using mean square error (MSE) criterion. The benefits of the new TPEs are also illustrated in an empirical application. The results show that the new proposed TPEs for the NBR and the PR models are better than the ML estimator, the RE and the LE.     

Linearized Restricted Ridge Regression Estimator in Linear Regression

This article primarily aims to put forward the linearized restricted ridge regression (LRRR) estimator in linear regression models. Two types of LRRR estimators are investigated under the PRESS criterion and the optimal LRRR estimators and the optimal restricted generalized ridge regression estimator are obtained. We apply the results to the Hald data and finally make a simulation study by using the method of McDonald and Galarneau.     

A new estimator combining the ridge regression and the restricted least squares methods of estimation

It is well-known in the literature on multicollinearity that one of the major consequences of multicollinearity on the ordinary least squares estimator is that the estimator produces large sampling variances, which in turn might inappropriately lead to exclusion of otherwise significant coefficients from the model. To circumvent this problem, two accepted estimation procedures which are often suggested are the restricted least squares method and the ridge regression method. While the former leads to a reduction in the sampling variance of the estimator, the later ensures a smaller mean square error value for the estimator. In this paper we have proposed a new estimator which is based on a criterion that combines the ideas underlying these two estimators. The standard properties of this new estimator have been studied in the paper. It has also been shown that this estimator is superior to both the restricted least squares as well as the ordinary ridge regression estimators by the criterion of mean sauare error of the estimator of the regression coefficients when the restrictions are indeed correct. The conditions for superiority of this estimator over the other two have also been derived for the situation when the restrictions are not correct.     

Restricted Two Parameter Ridge Estimator

In 2005 Lipovetsky and Conklin proposed an estimator, the two parameter ridge estimator (TRE), as an alternative to the ordinary least squares estimator (OLSE) and the ordinary ridge estimator (RE) in the presence of multicollinearity, and in 2006 Lipovetsky improved the two parameter model. In this paper, we introduce two new estimators, one of which is the modified two parameter ridge estimator (MTRE) defined by following Swindel's paper of 1976. The other one is the restricted two parameter ridge estimator (RTRE) which is derived by setting additional linear restrictions on the parameter vectors. This estimator is a generalization of the restricted least squares estimator (RLSE) and includes the restricted ridge estimator (RRE) proposed by Groß in 2003. A numerical example is provided and a simulation study is conducted for the comparisons of the RTRE with the OLSE, RLSE, RE, RRE and TRE.     

Empirical Likelihood Ratio Test for a Change-Point in Linear Regression Model

A nonparametric method based on the empirical likelihood is proposed to detect the change-point in the coefficient of linear regression models. The empirical likelihood ratio test statistic is proved to have the same asymptotic null distribution as that with classical parametric likelihood. Under some mild conditions, the maximum empirical likelihood change-point estimator is also shown to be consistent. The simulation results show the sensitivity and robustness of the proposed approach. The method is applied to some real datasets to illustrate the effectiveness.     

A two-parameter estimator in the negative binomial regression model

In this article, a two-parameter estimator is proposed to combat multicollinearity in the negative binomial regression model. The proposed two-parameter estimator is a general estimator which includes the maximum likelihood (ML) estimator, the ridge estimator (RE) and the Liu estimator as special cases. Some properties on the asymptotic mean-squared error (MSE) are derived and necessary and sufficient conditions for the superiority of the two-parameter estimator over the ML estimator and sufficient conditions for the superiority of the two-parameter estimator over the RE and the Liu estimator in the asymptotic mean-squared error (MSE) matrix sense are obtained. Furthermore, several methods and three rules for choosing appropriate shrinkage parameters are proposed. Finally, a Monte Carlo simulation study is given to illustrate some of the theoretical results.     

Two Stochastic Restricted Principal Components Regression Estimator in Linear Regression

In this article, we propose two stochastic restricted principal components regression estimator by combining the approach followed in obtaining the ordinary mixed estimator and the principal components regression estimator in linear regression model. The performance of the two new estimators in terms of matrix MSE criterion is studied. We also give an example and a Monte Carlo simulation to show the theoretical results.     

Modified and Restricted r-k Class Estimators

In this article, we introduce the modified r-k class estimator and the restricted r-k class estimator. We compare the performances of the new estimators to the r-k class estimator with respect to the matrix mean square error (MSE) criterion. As a special case of the restricted r-k class estimator, we obtain the restricted principal components regression (RPCR) estimator. Finally, we conduct a Monte Carlo simulation study and a numerical example to investigate the performances of the proposed estimators by the scalar mean square error (mse) criterion.     

Restricted ridge estimator in the logistic regression model

It is known that when the multicollinearity exists in the logistic regression model, variance of maximum likelihood estimator is unstable. As a remedy, Schaefer et al. presented a ridge estimator in the logistic regression model. Making use of the ridge estimator, when some linear restrictions are also present, we introduce a restricted ridge estimator in the logistic regression model. Statistical properties of this newly defined estimator will be studied and comparisons are done in the simulation study in the sense of mean squared error criterion. A real-data example and a simulation study are introduced to discuss the performance of this estimator.     

Performance analysis of the preliminary test estimator with series of stochastic restrictions

In this paper, the problem of estimation of the regression coefficients in a multiple regression model is considered under the multicollinearity situation when there are series of stochastic linear restrictions available on the regression parameter vector. We have considered the preliminary test ridge regression estimators (PTRREs) based on the Wald, likelihood ratio, and lagrangian multiplier tests. Tables for the maximum and minimum guaranteed efficiency of the PTRREs are obtained, which allow us to determine the optimum choice of the level of significance corresponding to the optimum estimator. Some numerical results support the findings.     

Approximating bayesian inference by weighted likelihood

The author proposes to use weighted likelihood to approximate Bayesian inference when no external or prior information is available. He proposes a weighted likelihood estimator that minimizes the empirical Bayes risk under relative entropy loss. He discusses connections among the weighted likelihood, empirical Bayes and James‐Stein estimators. Both simulated and real data sets are used for illustration purposes.     

A modified Liu-type estimator with an intercept term under mixture experiments

We consider ridge regression with an intercept term under mixture experiments. We propose a new estimator which is shown to be a modified version of the Liu-type estimator. The so-called compound covariate estimator is applied to modify the Liu-type estimator. We then derive a formula of the total mean squared error (TMSE) of the proposed estimator. It is shown that the new estimator improves upon existing estimators in terms of the TMSE, and the performance of the new estimator is invariant under the change of the intercept term. We demonstrate the new estimator using a real dataset on mixture experiments.     

Combining information from multiple surveys through the empirical likelihood method

It is often desirable to combine information collected in compatible multiple surveys in order to improve estimation and meet consistency requirements. Zieschang (1990) and Renssen & Nieuwenbroek (1997) suggested to this end the use of the generalized regression estimator with enlarged number of auxiliary variables. Unfortunately, adjusted weights associated with their approach can be negative. The author uses the notion of pseudo empirical likelihood to construct new estimators that are consistent, efficient and possess other attractive properties. The proposed approach is asymptotically equivalent to the earlier one, but it has clear maximum likelihood interpretations and its adjusted weights are always positive. The author also provides efficient algorithms for computing his estimators.     

On the small sample properties of norm-restricted maximum likelihood estimators for logistic regression models

This paper develops alternatives to maximum likelihood estimators (MLE) for logistic regression models and compares the mean squared error (MSE) of the estimators. The MLE for the vector of underlying success probabilities has low MSE only when the true probabilities are extreme (i.e., near 0 or 1). Extreme probabilities correspond to logistic regression parameter vectors which are large in norm. A competing “restricted” MLE and an empirical version of it are suggested as estimators with better performance than the MLE for central probabilities. An approximate EM-algorithm for estimating the restriction is described. As in the case of normal theory ridge estimators, the proposed estimators are shown to be formally derivable by Bayes and empirical Bayes arguments. The small sample operating characteristics of the proposed estimators are compared to the MLE via a simulation study; both the estimation of individual probabilities and of logistic parameters are considered.     

Empirical likelihood for linear regression models under imputation for missing responses

The authors study the empirical likelihood method for linear regression models. They show that when missing responses are imputed using least squares predictors, the empirical log‐likelihood ratio is asymptotically a weighted sum of chi‐square variables with unknown weights. They obtain an adjusted empirical log‐likelihood ratio which is asymptotically standard chi‐square and hence can be used to construct confidence regions. They also obtain a bootstrap empirical log‐likelihood ratio and use its distribution to approximate that of the empirical log‐likelihood ratio. A simulation study indicates that the proposed methods are comparable in terms of coverage probabilities and average lengths of confidence intervals, and perform better than a normal approximation based method.     

Performance of ridge estimator in inverse Gaussian regression model

The presence of multicollinearity among the explanatory variables has undesirable effects on the maximum likelihood estimator (MLE). Ridge estimator (RE) is a widely used estimator in overcoming this issue. The RE enjoys the advantage that its mean squared error (MSE) is less than that of MLE. The inverse Gaussian regression (IGR) model is a well-known model in the application when the response variable positively skewed. The purpose of this paper is to derive the RE of the IGR under multicollinearity problem. In addition, the performance of this estimator is investigated under numerous methods for estimating the ridge parameter. Monte Carlo simulation results indicate that the suggested estimator performs better than the MLE estimator in terms of MSE. Furthermore, a real chemometrics dataset application is utilized and the results demonstrate the excellent performance of the suggested estimator when the multicollinearity is present in IGR model.     

Empirical Likelihood Inference for the Rao‐Hartley‐Cochran Sampling Design

The Hartley‐Rao‐Cochran sampling design is an unequal probability sampling design which can be used to select samples from finite populations. We propose to adjust the empirical likelihood approach for the Hartley‐Rao‐Cochran sampling design. The approach proposed intrinsically incorporates sampling weights, auxiliary information and allows for large sampling fractions. It can be used to construct confidence intervals. In a simulation study, we show that the coverage may be better for the empirical likelihood confidence interval than for standard confidence intervals based on variance estimates. The approach proposed is simple to implement and less computer intensive than bootstrap. The confidence interval proposed does not rely on re‐sampling, linearization, variance estimation, design‐effects or joint inclusion probabilities.     

MORE ON THE PRE-TEST ESTIMATOR IN RIDGE REGRESSION

ABSTRACT

The problem of estimation of the regression coefficients in a multiple regression model is considered under a multicollinearity situation when it is suspected that the regression coefficients may be restricted to a subspace. The objective of this paper is to compare the usual preliminary test estimator and the preliminary test ridge regression estimator in the sense of the dispersion matrix of one dominating that of the other. In particular we proved two results giving necessary and sufficient conditions for the superiority of the preliminary test ridge regression estimator over the preliminary test estimator associated with the δ = 0 (or Δ = 0) and δ ≠ 0 (or Δ ≠ 0).