On a class of shrinkage estimators of vector of regression coefficients the

This paper studies a class of shrinkage estimators of the vector of regression coefficients. The small disturbance approximations for the bias and the mean squared error matrix of the estimator are derived. In the sense of mean squared error, these estimators dominate the least squares estimator and the generalized Stein estimator developed by Hosmane (1988).     

Parametrically Guided Non-parametric Regression

We present a new approach to regression function estimation in which a non-parametric regression estimator is guided by a parametric pilot estimate with the aim of reducing the bias. New classes of parametrically guided kernel weighted local polynomial estimators are introduced and formulae for asymptotic expectation and variance, hence approximated mean squared error and mean integrated squared error, are derived. It is shown that the new classes of estimators have the very same large sample variance as the estimators in the standard non-parametric setting, while there is substantial room for reducing the bias if the chosen parametric pilot function belongs to a wide neighbourhood around the true regression line. Bias reduction is discussed in light of examples and simulations.     

Nonparametric hazard rate estimation by orthogonal wavelet methods

We propose linear and nonlinear wavelet-based hazard rate estimators where the linear estimator is equivalent to a generalized kernel estimator. An asymptotic formula for the mean integrated squared error (MISE) of the nonlinear wavelet-based hazard rate estimator is provided. It is shown that the MISE formula for the nonlinear estimator is available for hazard rates which are smooth only in a piecewise sense, a feature not available for the kernel estimators.     

On exact small sample properties of the minimax generalized ridge regression estimators

The purpose of this paper is to compare sampling performance of the minimax generalized ridge regression estimators considered by Casella (1985) with that of ordinary least squares estimator by numerical calculations of exact mean squared error of these estimators.     

Prediction Error Property of the Lasso Estimator and its Generalization

The lasso procedure is an estimator‐shrinkage and variable selection method. This paper shows that there always exists an interval of tuning parameter values such that the corresponding mean squared prediction error for the lasso estimator is smaller than for the ordinary least squares estimator. For an estimator satisfying some condition such as unbiasedness, the paper defines the corresponding generalized lasso estimator. Its mean squared prediction error is shown to be smaller than that of the estimator for values of the tuning parameter in some interval. This implies that all unbiased estimators are not admissible. Simulation results for five models support the theoretical results.     

An improved and efficient biased estimation technique in logistic regression model

Abstract

In this article, we propose a new improved and efficient biased estimation method which is a modified restricted Liu-type estimator satisfying some sub-space linear restrictions in the binary logistic regression model. We study the properties of the new estimator under the mean squared error matrix criterion and our results show that under certain conditions the new estimator is superior to some other estimators. Moreover, a Monte Carlo simulation study is conducted to show the performance of the new estimator in the simulated mean squared error and predictive median squared errors sense. Finally, a real application is considered.     

Multivariate generalized Birnbaum—Saunders kernel density estimators

In this article, we first propose the classical multivariate generalized Birnbaum–Saunders kernel estimator for probability density function estimation in the context of multivariate non negative data. Then, we apply two multiplicative bias correction (MBC) techniques for multivariate kernel density estimator. Some properties (bias, variance, and mean integrated squared error) of the corresponding estimators are also investigated. Finally, the performances of the classical and MBC estimators based on family of generalized Birnbaum–Saunders kernels are illustrated by a simulation study.     

Developing a restricted two-parameter Liu-type estimator: A comparison of restricted estimators in the binary logistic regression model

In the context of estimating regression coefficients of an ill-conditioned binary logistic regression model, we develop a new biased estimator having two parameters for estimating the regression vector parameter β when it is subjected to lie in the linear subspace restriction Hβ = h. The matrix mean squared error and mean squared error (MSE) functions of these newly defined estimators are derived. Moreover, a method to choose the two parameters is proposed. Then, the performance of the proposed estimator is compared to that of the restricted maximum likelihood estimator and some other existing estimators in the sense of MSE via a Monte Carlo simulation study. According to the simulation results, the performance of the estimators depends on the sample size, number of explanatory variables, and degree of correlation. The superiority region of our proposed estimator is identified based on the biasing parameters, numerically. It is concluded that the new estimator is superior to the others in most of the situations considered and it is recommended to the researchers.     

Improvement of generalized difference-based mixed Liu estimator in partially linear model

In this paper, a generalized difference-based mixed Liu estimator in partially linear model is presented, when it is supposed that the regression parameter may be restricted to a subspace and compare the proposed estimators in the sense of matrix mean squared error criteria. Finally a simulation study is presented to show the performance of the estimators.     

Efficient improved estimation of the parameters in two seemingly unrelated regression models

For a system of two seemingly unrelated regression equations, this paper proposes a two-stage covariance improved estimator of the regression coefficients. The new estimator is shown to uniformly dominate the present estimators in terms of generalized mean square error criterion. In addition, we also propose the exact generalized mean square error of new estimator.     

A Jackknifed estimators for the negative binomial regression model

Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators for estimating the ridge parameter k in the negative binomial (NB) regression have been proposed. The Jackknifed estimators are obtained to remedy the multicollinearity and reduce the bias. A simulation study is provided to evaluate the performance of estimators. Both mean squared error (MSE) and the percentage relative error (PRE) are considered as the performance criteria. The simulated result indicated that some of proposed Jackknifed estimators should be preferred to the ML method and ridge estimators to reduce MSE and bias.     

Restricted ridge estimator in generalized linear models: Monte Carlo simulation studies on Poisson and binomial distributed responses

It is known that collinearity among the explanatory variables in generalized linear models (GLMs) inflates the variance of maximum likelihood estimators. To overcome multicollinearity in GLMs, ordinary ridge estimator and restricted estimator were proposed. In this study, a restricted ridge estimator is introduced by unifying the ordinary ridge estimator and the restricted estimator in GLMs and its mean squared error (MSE) properties are discussed. The MSE comparisons are done in the context of first-order approximated estimators. The results are illustrated by a numerical example and two simulation studies are conducted with Poisson and binomial responses.     

A modified Liu-type estimator with an intercept term under mixture experiments

We consider ridge regression with an intercept term under mixture experiments. We propose a new estimator which is shown to be a modified version of the Liu-type estimator. The so-called compound covariate estimator is applied to modify the Liu-type estimator. We then derive a formula of the total mean squared error (TMSE) of the proposed estimator. It is shown that the new estimator improves upon existing estimators in terms of the TMSE, and the performance of the new estimator is invariant under the change of the intercept term. We demonstrate the new estimator using a real dataset on mixture experiments.     

On the finite-sample properties of conditional empirical likelihood estimators

We provide Monte Carlo evidence on the finite-sample behavior of the conditional empirical likelihood (CEL) estimator of Kitamura, Tripathi, and Ahn and the conditional Euclidean empirical likelihood (CEEL) estimator of Antoine, Bonnal, and Renault in the context of a heteroscedastic linear model with an endogenous regressor. We compare these estimators with three heteroscedasticity-consistent instrument-based estimators and the Donald, Imbens, and Newey estimator in terms of various performance measures. Our results suggest that the CEL and CEEL with fixed bandwidths may suffer from the no-moment problem, similarly to the unconditional generalized empirical likelihood estimators studied by Guggenberger. We also study the CEL and CEEL estimators with automatic bandwidths selected through cross-validation. We do not find evidence that these suffer from the no-moment problem. When the instruments are weak, we find CEL and CEEL to have finite-sample properties—in terms of mean squared error and coverage probability of confidence intervals—poorer than the heteroscedasticity-consistent Fuller (HFUL) estimator. In the strong instruments case, the CEL and CEEL estimators with automatic bandwidths tend to outperform HFUL in terms of mean squared error, while the reverse holds in terms of the coverage probability, although the differences in numerical performance are rather small.     

On the almost unbiased ridge regression estimator

The purpose of this paper is two-fold. One is to compare the almost unbiased generalized ridge regression (AUGRR) estimator proposed by Singh, Chaubey and Dwivedi (1986) with the generalized ridge regression (GRR) estimator and with the ordinary least squares (OLS) estimator in terms of the mean squared error criterion. Second is to examine small sample properties of the operational almost unbiased ordinary ridge regression (AUORR) estimator by Monte Carlo experiments.     

A minimum variance kernel estimator and a discrete frequency polygon estimator for ordinal contingency tables

This paper introduces two estimators, a boundary corrected minimum variance kernel estimator based on a uniform kernel and a discrete frequency polygon estimator, for the cell probabilities of ordinal contingency tables. Simulation results show that the minimum variance boundary kernel estimator has a smaller average sum of squared error than the existing boundary kernel estimators. The discrete frequency polygon estimator is simple and easy to interpret, and it is competitive with the minimum variance boundary kernel estimator. It is proved that both estimators have an optimal rate of convergence in terms of mean sum of squared error, The estimators are also defined for high-dimensional tables.     

The Mean Squared Error of the Generalized Ridge Regression Estimator and the Orientation of β

Newhouse and Oman (1971) identified the orientations with respect to the eigenvectors of X'X of the true coefficient vector of the linear regression model for which the ordinary ridge regression estimator performs best and performs worse when mean squared error is the measure of performance. In this paper the corresponding result is derived for generalized ridge regression for two risk functions: mean squared error and mean squared error of prediction.     

A generalized inverse regression estimator in multi-univariate linear calibration

A multivariate linear calibration problem, in which response variable is multivariate and explanatory variable is univariate, is considered. In this paper a class of generalized inverse regression estimators is proposed in multi-univariate linear calibration. It includes the classical estimator and the inverse regression one (or Krutchkoff estimator). For the proposed estimator we derive the expressions of bias and mean square error (MSE). Furthermore the behavior of these characteristics is investigated through an analytical method. In addition through a numerical study we confirm the existence of a generalized inverse regression estimator to improve both the classical and the inverse regression estimators on the MSE criterion.     

Estimation in linear models using gradient descent with early stopping

A new shrinkage estimator of the coefficients of a linear model is derived. The estimator is motivated by the gradient-descent algorithm used to minimize the sum of squared errors and results from early stopping of the algorithm. The statistical properties of the estimator are examined and compared with other well-established methods such as least squares and ridge regression, both analytically and through a simulation study. An important result is that the new estimator is shown to be comparable to other shrinkage estimators in terms of mean squared error of parameters and of predictions, and superior under certain circumstances.Supported by the Greek State Scholarships Foundation     

A comparison of estimators for the mean in the inverse gaussian distribution with a known coefficient of variation

In this paper, we discuss an estimation problem of the mean in the inverse Gaussian distribution with a known coefficient of variation. Two types of linear estimators for the mean, the linear minimum variance unbiased estimator and the linear minimum mean squared error estimator, are constructed by using the squared error loss function and their properties are examined. It is observed that, for small samples the performance of the proposed estimators is better than that of the maximum likelihood estimator, when the coefficient of variation is large.