EXACT FINITE-SAMPLE PROPERTIES OF A PRE-TEST ESTIMATOR IN RIDGE REGRESSION

This paper discusses a pre-test regression estimator which uses the least squares estimate when it is “large” and a ridge regression estimate for “small” regression coefficients, where the preliminary test is applied separately to each regression coefficient in turn to determine whether it is “large” or “small.” For orthogonal regressors, the exact finite-sample bias and mean squared error of the pre-test estimator are derived. The latter is less biased than a ridge estimator, and over much of the parameter space the pre-test estimator has smaller mean squared error than least squares. A ridge estimator is found to be inferior to the pre-test estimator in terms of mean squared error in many situations, and at worst the latter estimator is only slightly less efficient than the former at commonly used significance levels.     

Selection of the Ridge Parameter Using Mathematical Programming

Ridge regression solves multicollinearity problems by introducing a biasing parameter that is called ridge parameter; it shrinks the estimates and their standard errors in order to reach acceptable results. Selection of the ridge parameter was done using several subjective and objective techniques that are concerned with certain criteria. In this study, selection of the ridge parameter depends on other important statistical measures to reach a better value of the ridge parameter. The proposed ridge parameter selection technique depends on a mathematical programming model and the results are evaluated using a simulation study. The performance of the proposed method is good when the error variance is greater than or equal to one; the sample consists of 20 observations, the number of explanatory variables in the model is 2, and there is a very strong correlation between the two explanatory variables.     

A new linearized ridge Poisson estimator in the presence of multicollinearity

Poisson regression is a very commonly used technique for modeling the count data in applied sciences, in which the model parameters are usually estimated by the maximum likelihood method. However, the presence of multicollinearity inflates the variance of maximum likelihood (ML) estimator and the estimated parameters give unstable results. In this article, a new linearized ridge Poisson estimator is introduced to deal with the problem of multicollinearity. Based on the asymptotic properties of ML estimator, the bias, covariance and mean squared error of the proposed estimator are obtained and the optimal choice of shrinkage parameter is derived. The performance of the existing estimators and proposed estimator is evaluated through Monte Carlo simulations and two real data applications. The results clearly reveal that the proposed estimator outperforms the existing estimators in the mean squared error sense.KEYWORDS: Poisson regression, multicollinearity, ridge Poisson estimator, linearized ridge regression estimator, mean squared errorMathematics Subject Classifications: 62J07, 62F10     

A new modified Jackknifed estimator for the Poisson regression model

The Poisson regression is very popular in applied researches when analyzing the count data. However, multicollinearity problem arises for the Poisson regression model when the independent variables are highly intercorrelated. Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators and some methods for estimating the ridge parameter k in the Poisson regression have been proposed. It has been found that some estimators are better than the commonly used maximum-likelihood (ML) estimator and some other RR estimators. In this study, the modified Jackknifed Poisson ridge regression (MJPR) estimator is proposed to remedy the multicollinearity. A simulation study and a real data example are provided to evaluate the performance of estimators. Both mean-squared error and the percentage relative error are considered as the performance criteria. The simulation study and the real data example results show that the proposed MJPR method outperforms the Poisson ridge regression, Jackknifed Poisson ridge regression and the ML in all of the different situations evaluated in this paper.     

ESTIMATING A RATIO OF MEANS FROM BIVARIATE COUNTS WITH APPLICATIONS IN STEREOLOGY

In survey sampling and in stereology, it is often desirable to estimate the ratio of means θ= E(Y)/E(X) from bivariate count data (X, Y) with unknown joint distribution. We review methods that are available for this problem, with particular reference to stereological applications. We also develop new methods based on explicit statistical models for the data, and associated model diagnostics. The methods are tested on a stereological dataset. For point‐count data, binomial regression and bivariate binomial models are generally adequate. Intercept‐count data are often overdispersed relative to Poisson regression models, but adequately fitted by negative binomial regression.     

THE EXAMINATION AND ANALYSIS OF RESIDUALS FOR SOME BIASED ESTIMATORS IN LINEAR REGRESSION

In the presence of collinearity certain biased estimation procedures like ridge regression, generalized inverse estimator, principal component regression, Liu estimator, or improved ridge and Liu estimators are used to improve the ordinary least squares (OLS) estimates in the linear regression model. In this paper new biased estimator (Liu estimator), almost unbiased (improved) Liu estimator and their residuals will be analyzed and compared with OLS residuals in terms of mean-squared error.     

零浮动Poisson项目计数法在敏感数据抽样调查中的应用

文章将Poisson-Poisson项目计数法进行推广,提出零浮动Poisson项目计数法,其中,非敏感辅助变量来自于一个参数已知的零浮动Poisson分布。并给出了该模型下敏感参数极大似然估计的EM算法以及构造其置信区间的bootstrap方法。此外,还对该模型保护受访者隐私的能力加以讨论,发现该模型的隐私保护要优于Poisson-Poisson项目计数法。最后,从随机模拟的结果表明在该模型下利用本文所介绍的分析方法可以得到敏感参数的较为准确的估计。     

SOME STATISTICAL DISTRIBUTIONS OF USE IN ANALYSING BIRD COUNT DATA1

The following report arose from an enquiry by a zoologist, J. M. Cullen, concerning various possible ways of analysing bird count data. He was particularly interested in two questions: (i) What is the best way to estimate abundances of various species?, and (ii) What evidence is there that birds occur randomly (in particular, as a Poisson process) as opposed to in flocks or according to some mutual repulsion process?     

ESTIMATION OF A PROPORTION USING SEVERAL INDEPENDENT SAMPLES OF BINOMIAL MIXTURES

This paper considers a sequence of independent counts, with each count arising from a mixture of binomial distributions; the mixing distribution is fixed but the number of trials varies from count to count. In this common situation, an estimate of the underlying mean binomial proportion is needed. Two estimators are in general use: the arithmetic average and a weighted average of the observed proportions. Variances of the two estimators are compared and used to decide which estimator is preferred in a given context. The relative merits depend on the distribution of the proportions and the numbers of trials used.     

Modified ridge-type for the Poisson regression model: simulation and application

The Poisson regression model (PRM) is employed in modelling the relationship between a count variable (y) and one or more explanatory variables. The parameters of PRM are popularly estimated using the Poisson maximum likelihood estimator (PMLE). There is a tendency that the explanatory variables grow together, which results in the problem of multicollinearity. The variance of the PMLE becomes inflated in the presence of multicollinearity. The Poisson ridge regression (PRRE) and Liu estimator (PLE) have been suggested as an alternative to the PMLE. However, in this study, we propose a new estimator to estimate the regression coefficients for the PRM when multicollinearity is a challenge. We perform a simulation study under different specifications to assess the performance of the new estimator and the existing ones. The performance was evaluated using the scalar mean square error criterion and the mean squared error prediction error. The aircraft damage data was adopted for the application study and the estimators’ performance judged by the SMSE and the mean squared prediction error. The theoretical comparison shows that the proposed estimator outperforms other estimators. This is further supported by the simulation study and the application result.KEYWORDS: Poisson regression model, Poisson maximum likelihood estimator, multicollinearity, Poisson ridge regression, Liu estimator, simulation     

A class of generalized ridge estimators

Presence of collinearity among the explanatory variables results in larger standard errors of parameters estimated. When multicollinearity is present among the explanatory variables, the ordinary least-square (OLS) estimators tend to be unstable due to larger variance of the estimators of the regression coefficients. As alternatives to OLS estimators few ridge estimators are available in the literature. This article presents some of the popular ridge estimators and attempts to provide (i) a generalized class of ridge estimators and (ii) a modified ridge estimator. The performance of the proposed estimators is investigated with the help of Monte Carlo simulation technique. Simulation results indicate that the suggested estimators perform better than the ordinary least-square (OLS) estimators and other estimators considered in this article.     

Comment on hoerl and kennard's ridge regression simulation methodology

Ridge regression has received strong support in several Monte carlo studies, leading some authors to advocate its general use. It is argued, however, that these studies were strongly biased in favor of ridge regression by simulating regression coefficient vectors centered at the origin; a condition well suited for a shrinkage technique. Studies which modeled some non-zero regression coefficients and which showed only qualified support for ridge regression are cited in support of this argument. It is concluded that only to the extent that ridge regression type coefficient vectors actually underlie real data sets -a poorly understood phenomenon - will ridge regression be of use.     

Using Bayesian Techniques for Data Pooling in Regional Payroll Forecasting

This article adapts to the regional level a multicountry technique recently used by Garcia-Ferrer, Highfield, Palm, and Zellner (1987) and extended by Zellner and Hong (1987) to forecast the growth rates in gross national product across nine countries. This forecasting methodology is applied to the regional level by modeling payroll formation in seven Ohio metropolitan areas. We compare the forecasting performance of these procedures with that of a ridge estimator and find that the ridge estimator produces forecasts equal to or better than those from the newly proposed estimators. We conclude that the ridge estimator, which does not reference the pooled data information introduced by the newly proposed techniques, may serve as a benchmark against which to judge the relative importance of this kind of information in improving forecasts.     

Confidence Interval for Shrinkage Parameters in Ridge Regression

In ridge regression, the estimation of ridge parameter k is an important problem. There are several methods available in the literature to do this job some what efficiently. However, no attempts were made to suggest a confidence interval for the ridge parameter using the knwoledge from the data. In this article, we propose a data dependent confidence interval for the ridge parameter k. The method of obtaining the confidence interval is illustrated with the help of a data set. A simulation study indicates that the empirical coverage probability of the suggested confidence intervals are quite high.     

Ridge Analysis 25 Years Later

The response surface technique called ridge analysis was originally introduced by Hoerl (1959) more than 25 years ago. Despite tremendous advantages over more conventional response surface procedures when more than two independent variables are present, ridge analysis has received little attention in the statistical literature since then, although numerous applications have appeared in engineering journals. This situation may be partially due to the fact that this procedure led to the discovery of ridge regression, which has completely overshadowed ridge analysis in the literature since. This discussion will briefly review the mathematics of ridge analysis, its literature, practical advantages, and relationship to ridge regression.     

MORE ON THE PRE-TEST ESTIMATOR IN RIDGE REGRESSION

ABSTRACT

The problem of estimation of the regression coefficients in a multiple regression model is considered under a multicollinearity situation when it is suspected that the regression coefficients may be restricted to a subspace. The objective of this paper is to compare the usual preliminary test estimator and the preliminary test ridge regression estimator in the sense of the dispersion matrix of one dominating that of the other. In particular we proved two results giving necessary and sufficient conditions for the superiority of the preliminary test ridge regression estimator over the preliminary test estimator associated with the δ = 0 (or Δ = 0) and δ ≠ 0 (or Δ ≠ 0).     

Computational schema on ridge analysis

The purpose of this paper is to revisit the response surface technique ridge analysis within the context of the “trust region” problem in numerical analysis. It is found that these two approaches inherently solve the same problem. We introduce the computational difficulty, termed the “hard case”, which originates in the trust region methods, also exists in ridge analysis but has never been formally discussed in response surface methodology (RSM). The dual response global optimization algorithm (DRSALG) based on the trust region method is applied (with a certain modification) to solving the ridge analysis problem. Some numerical comparisons against a general-purpose nonlinear optimization algorithm are illustrated in terms of examples appearing in the literature     

An application of the local influence approach to ridge regression

In this study, the method of local influence, which was introduced by Cook as a general tool for assessing the influence of local departures from the underlying assumptions, is applied to ridge regression, by defining the maximum pseudo-likelihood ridge estimator obtained using the augmentation approach, because this method is suitable for likelihood-based models. In addition, an alternative local influence approach suggested by Billor and Loynes is applied to ridge regression. A comparison of these approaches and an example are given.     

Diagnostic measures for kernel ridge regression on reproducing kernel Hilbert space

The aim of this paper is to define and develop diagnostic measures with respect to kernel ridge regression in a reproducing kernel Hilbert space (RKHS). To identify influential observations, we define a particular version of Cook’s distance for the kernel ridge regression model in RKHS, which is conceptually consistent with Cook’s distance in a classical regression model. Then, by using the perturbation formula for the regularized conditional expectation of the outcome in RKHS, we develop an approximate version of Cook”s distance in RKHS because the original definition requires intensive computations. Such an approximated Cook”s distance is represented in terms of basic building blocks such as residuals and leverages of the kernel ridge regression. The results of the simulation and real application demonstrate that our diagnostic measure successfully detects potentially influential observations on estimators in kernel ridge regression.     

Local Influence Analysis for The Ridge Regression Under Stochastic Linear Restrictions

In this article, we assess the local influence for the ridge regression of linear models with stochastic linear restrictions in the spirit of Cook by using the log-likelihood of the stochastic restricted ridge regression estimator. The diagnostics under the perturbations of constant variance, responses and individual explanatory variables are derived. We also assess the local influence of the stochastic restricted ridge regression estimator under the approach suggested by Billor and Loynes. At the end, a numerical example on the Longley data is given to illustrate the theoretic results.