An improved and efficient biased estimation technique in logistic regression model

Abstract

In this article, we propose a new improved and efficient biased estimation method which is a modified restricted Liu-type estimator satisfying some sub-space linear restrictions in the binary logistic regression model. We study the properties of the new estimator under the mean squared error matrix criterion and our results show that under certain conditions the new estimator is superior to some other estimators. Moreover, a Monte Carlo simulation study is conducted to show the performance of the new estimator in the simulated mean squared error and predictive median squared errors sense. Finally, a real application is considered.     

Prediction Error of Small Area Predictors Shrinking Both Means and Variances

The article considers a new approach for small area estimation based on a joint modelling of mean and variances. Model parameters are estimated via expectation–maximization algorithm. The conditional mean squared error is used to evaluate the prediction error. Analytical expressions are obtained for the conditional mean squared error and its estimator. Our approximations are second‐order correct, an unwritten standardization in the small area literature. Simulation studies indicate that the proposed method outperforms the existing methods in terms of prediction errors and their estimated values.     

Two-parameter ridge estimator in the binary logistic regression

The binary logistic regression is a commonly used statistical method when the outcome variable is dichotomous or binary. The explanatory variables are correlated in some situations of the logit model. This problem is called multicollinearity. It is known that the variance of the maximum likelihood estimator (MLE) is inflated in the presence of multicollinearity. Therefore, in this study, we define a new two-parameter ridge estimator for the logistic regression model to decrease the variance and overcome multicollinearity problem. We compare the new estimator to the other well-known estimators by studying their mean squared error (MSE) properties. Moreover, a Monte Carlo simulation is designed to evaluate the performances of the estimators. Finally, a real data application is illustrated to show the applicability of the new method. According to the results of the simulation and real application, the new estimator outperforms the other estimators for all of the situations considered.     

ON THE USE OF THE STEIN VARIANCE ESTIMATOR IN THE DOUBLE k-CLASS ESTIMATOR IN REGRESSION

This paper investigates the predictive mean squared error performance of a modified double k-class estimator by incorporating the Stein variance estimator. Recent studies show that the performance of the Stein rule estimator can be improved by using the Stein variance estimator. However, as we demonstrate below, this conclusion does not hold in general for all members of the double k-class estimators. On the other hand, an estimator is found to have smaller predictive mean squared error than the Stein variance-Stein rule estimator, over quite large parts of the parameter space.     

Modified ridge-type for the Poisson regression model: simulation and application

The Poisson regression model (PRM) is employed in modelling the relationship between a count variable (y) and one or more explanatory variables. The parameters of PRM are popularly estimated using the Poisson maximum likelihood estimator (PMLE). There is a tendency that the explanatory variables grow together, which results in the problem of multicollinearity. The variance of the PMLE becomes inflated in the presence of multicollinearity. The Poisson ridge regression (PRRE) and Liu estimator (PLE) have been suggested as an alternative to the PMLE. However, in this study, we propose a new estimator to estimate the regression coefficients for the PRM when multicollinearity is a challenge. We perform a simulation study under different specifications to assess the performance of the new estimator and the existing ones. The performance was evaluated using the scalar mean square error criterion and the mean squared error prediction error. The aircraft damage data was adopted for the application study and the estimators’ performance judged by the SMSE and the mean squared prediction error. The theoretical comparison shows that the proposed estimator outperforms other estimators. This is further supported by the simulation study and the application result.KEYWORDS: Poisson regression model, Poisson maximum likelihood estimator, multicollinearity, Poisson ridge regression, Liu estimator, simulation     

On Variance-Stabilizing Multivariate Non Parametric Regression Estimation

The mean squared error (MSE)-minimizing local variable bandwidth for the univariate local linear estimator (the LL) is well-known. This bandwidth does not stabilize variance over the domain. Moreover, in regions where a regression function has zero curvature, the LL estimator is discontinuous. In this paper, we propose a variance-stabilizing (VS) local variable diagonal bandwidth matrix for the multivariate LL estimator. Theoretically, the VS bandwidth can outperform the multivariate extension of the MSE-minimizing local variable scalar bandwidth in terms of asymptotic mean integrated squared error and can avoid discontinuity created by the MSE-minimizing bandwidth. We present an algorithm for estimating the VS bandwidth and simulation studies.     

Developing a restricted two-parameter Liu-type estimator: A comparison of restricted estimators in the binary logistic regression model

In the context of estimating regression coefficients of an ill-conditioned binary logistic regression model, we develop a new biased estimator having two parameters for estimating the regression vector parameter β when it is subjected to lie in the linear subspace restriction Hβ = h. The matrix mean squared error and mean squared error (MSE) functions of these newly defined estimators are derived. Moreover, a method to choose the two parameters is proposed. Then, the performance of the proposed estimator is compared to that of the restricted maximum likelihood estimator and some other existing estimators in the sense of MSE via a Monte Carlo simulation study. According to the simulation results, the performance of the estimators depends on the sample size, number of explanatory variables, and degree of correlation. The superiority region of our proposed estimator is identified based on the biasing parameters, numerically. It is concluded that the new estimator is superior to the others in most of the situations considered and it is recommended to the researchers.     

On the mean squared error of nonparametric quantile estimators under random right-censorship

For randomly right-censored data, new asymptotic expressions for the mean squared errors of the product-limit quantile estimator and a kernel-type quantile estimator are presented in this paper. From these results a comparison of the two quantile estimators with respect to their mean squared errors is given.     

A family of shrinkage estimators for Weibull shape parameter in censored sampling

In this paper a new class of shrinkage estimators has been introduced for the shape parameter in an independently identically distributed two-parameterWeibull model under censored sampling. The main idea is to incorporate the prior guessed value by correcting the standard estimator, which is essentially an unbiased estimator, with optimally weighted ratios of the guessed value and the standard estimator, instead of considering a convex combination of the standard estimator and the difference of the guessed value and the standard estimator. The resulting estimator dominates the standard estimator in a surprisingly large neighborhood of the guessed value. The suggested estimator has also been compared with the minimum mean squared error estimator and a class of estimators suggested by Singh and Shukla in IAPQR Trans 25(2), 107–118, 2000. It is found that the suggested class of estimators has lesser bias as well as lesser mean squared error than its competitors subject to certain conditions.      

One-sided trimming in small samples with asymmetric contamination

An estimator for location, given a sample of only four or five observations, is proposed. The underlying distribution on of the sample may (with probability p) be contaminated by an outlier from a rightly-skewed distribution. The estimator minimizes the maximum mean squared error over all values of p. In fact, there exists an estimator which is unbiased in both the outlier - free and extreme-outlier cases, but its mean square error is substantially higher than the mean squared error for the minimax estimator. Mean squared errors for various underlying distributional situations are calculated and compared with those of other location estimators such as the mean and the median.     

A new linearized ridge Poisson estimator in the presence of multicollinearity

Poisson regression is a very commonly used technique for modeling the count data in applied sciences, in which the model parameters are usually estimated by the maximum likelihood method. However, the presence of multicollinearity inflates the variance of maximum likelihood (ML) estimator and the estimated parameters give unstable results. In this article, a new linearized ridge Poisson estimator is introduced to deal with the problem of multicollinearity. Based on the asymptotic properties of ML estimator, the bias, covariance and mean squared error of the proposed estimator are obtained and the optimal choice of shrinkage parameter is derived. The performance of the existing estimators and proposed estimator is evaluated through Monte Carlo simulations and two real data applications. The results clearly reveal that the proposed estimator outperforms the existing estimators in the mean squared error sense.KEYWORDS: Poisson regression, multicollinearity, ridge Poisson estimator, linearized ridge regression estimator, mean squared errorMathematics Subject Classifications: 62J07, 62F10     

A modified Liu-type estimator with an intercept term under mixture experiments

We consider ridge regression with an intercept term under mixture experiments. We propose a new estimator which is shown to be a modified version of the Liu-type estimator. The so-called compound covariate estimator is applied to modify the Liu-type estimator. We then derive a formula of the total mean squared error (TMSE) of the proposed estimator. It is shown that the new estimator improves upon existing estimators in terms of the TMSE, and the performance of the new estimator is invariant under the change of the intercept term. We demonstrate the new estimator using a real dataset on mixture experiments.     

On parameter estimation in Bernoulli trials with dependence

A simple estimator is proposed for the dependence parameter for the Klotz model of Bernoulli trials with Markov dependence and it is compared with the ratio estimator given by Price and the approximate maximum likelihood estimator given by Klotz. The proposed estimator is shown to have considerably smaller bias than the other two estimators with comparable mean squared errors, and has all the large sample optimal properties that the other two estimators have.     

Robust Regression Using Data Partitioning and M-Estimation

We propose a new robust regression estimator using data partition technique and M estimation (DPM). The data partition technique is designed to define a small fixed number of subsets of the partitioned data set and to produce corresponding ordinary least square (OLS) fits in each subset, contrary to the resampling technique of existing robust estimators such as the least trimmed squares estimator. The proposed estimator shares a common strategy with the median ball algorithm estimator that is obtained from the OLS trial fits only on a fixed number of subsets of the data. We examine performance of the DPM estimator in the eleven challenging data sets and simulation studies. We also compare the DPM with the five commonly used robust estimators using empirical convergence rates relative to the OLS for clean data, robustness through mean squared error and bias, masking and swamping probabilities, the ability of detecting the known outliers, and the regression and affine equivariances.     

On the Principal Component Liu-type Estimator in Linear Regression

In this article, we present a principal component Liu-type estimator (LTE) by combining the principal component regression (PCR) and LTE to deal with the multicollinearity problem. The superiority of the new estimator over the PCR estimator, the ordinary least squares estimator (OLSE) and the LTE are studied under the mean squared error matrix. The selection of the tuning parameter in the proposed estimator is also discussed. Finally, a numerical example is given to explain our theoretical results.     

Linear-representation Based Estimation of Stochastic Volatility Models

Abstract.  A new way of estimating stochastic volatility models is developed. The method is based on the existence of autoregressive moving average (ARMA) representations for powers of the log-squared observations. These representations allow to build a criterion obtained by weighting the sums of squared innovations corresponding to the different ARMA models. The estimator obtained by minimizing the criterion with respect to the parameters of interest is shown to be consistent and asymptotically normal. Monte-Carlo experiments illustrate the finite sample properties of the estimator. The method has potential applications to other non-linear time-series models.     

A Bezier curve method in interval-censored data

In this paper we propose a Bezier curve method to estimate the survival function and the median survival time in interval-censored data. We compare the proposed estimator with other existing methods such as the parametric method, the single point imputation method, and the nonparametric maximum likelihood estimator through extensive numerical studies, and it is shown that the proposed estimator performs better than others in the sense of mean squared error and mean integrated squared error. An illustrative example based on a real data set is given.     

Estimation in Singular Linear Models with Stochastic Linear Restrictions and Linear Equality Restrictions

This article is concerned with the parameter estimation in a singular linear regression model with stochastic linear restrictions and linear equality restrictions simultaneously. A new estimator is introduced and it is proved that the proposed estimator is superior to the least squares estimator and singular mixed estimator in the mean squared error sense under certain conditions.     

Recursive distribution estimator defined by stochastic approximation method using Bernstein polynomials

We propose a recursive distribution estimator using Robbins-Monro's algorithm and Bernstein polynomials. We study the properties of the recursive estimator, as a competitor of Vitale's distribution estimator. We show that, with optimal parameters, our proposal dominates Vitale's estimator in terms of the mean integrated squared error. Finally, we confirm theoretical result throught a simulation study.     

Heteroskedasticity–robust tests with minimum size distortion

Asymptotically valid inference in linear regression models is easily achieved under mild conditions using the well-known Eicker–White heteroskedasticity–robust covariance matrix estimator or one of its variant. In finite sample however, such inferences can suffer from substantial size distortion. Indeed, it is well established in the literature that the finite sample accuracy of a test may depend on which variant of the Eicker–White estimator is used, on the underlying data generating process (DGP) and on the desired level of the test.

This paper develops a new variant of the Eicker–White estimator which explicitly aims to minimize the finite sample null error in rejection probability (ERP) of the test. This is made possible by selecting the transformation of the squared residuals which results in the smallest possible ERP through a numerical algorithm based on the wild bootstrap. Monte Carlo evidence indicates that this new procedure achieves a level of robustness to the DGP, sample size and nominal testing level unequaled by any other Eicker–White estimator based asymptotic test.