The minimum L2 distance estimator for Poisson mixture models

A robust estimator is developed for Poisson mixture models with a known number of components. The proposed estimator minimizes the L₂ distance between a sample of data and the model. When the component distributions are completely known, the estimators for the mixing proportions are in closed form. When the parameters for the component Poisson distributions are unknown, numerical methods are needed to calculate the estimators. Compared to the minimum Hellinger distance estimator, the minimum L₂ estimator can be less robust to extreme outliers, and often more robust to moderate outliers.     

A new modified Jackknifed estimator for the Poisson regression model

The Poisson regression is very popular in applied researches when analyzing the count data. However, multicollinearity problem arises for the Poisson regression model when the independent variables are highly intercorrelated. Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators and some methods for estimating the ridge parameter k in the Poisson regression have been proposed. It has been found that some estimators are better than the commonly used maximum-likelihood (ML) estimator and some other RR estimators. In this study, the modified Jackknifed Poisson ridge regression (MJPR) estimator is proposed to remedy the multicollinearity. A simulation study and a real data example are provided to evaluate the performance of estimators. Both mean-squared error and the percentage relative error are considered as the performance criteria. The simulation study and the real data example results show that the proposed MJPR method outperforms the Poisson ridge regression, Jackknifed Poisson ridge regression and the ML in all of the different situations evaluated in this paper.     

Robust Estimation for Zero‐Inflated Poisson Regression

Abstract. The zero‐inflated Poisson regression model is a special case of finite mixture models that is useful for count data containing many zeros. Typically, maximum likelihood (ML) estimation is used for fitting such models. However, it is well known that the ML estimator is highly sensitive to the presence of outliers and can become unstable when mixture components are poorly separated. In this paper, we propose an alternative robust estimation approach, robust expectation‐solution (RES) estimation. We compare the RES approach with an existing robust approach, minimum Hellinger distance (MHD) estimation. Simulation results indicate that both methods improve on ML when outliers are present and/or when the mixture components are poorly separated. However, the RES approach is more efficient in all the scenarios we considered. In addition, the RES method is shown to yield consistent and asymptotically normal estimators and, in contrast to MHD, can be applied quite generally.     

On least absolute values estimation

The resistance of least absolute values (L₁) estimators to outliers and their robustness to heavy-tailed distributions make these estimators useful alternatives to the usual least squares estimators. The recent development of efficient algorithms for L₁ estimation in linear models has permitted their use in practical data analysis. Although in general the L₁ estimators are not unique, there are a number of properties they all share. The set of all L₁ estimators for a given model and data set can be characterized as the convex hull of some extreme estimators. Properties of the extreme estimators and of the L₁-estimate set are considered.     

Negative moments of a modified power series distribution and bias of the maximum likelihood estimator

The class of Modified Power Series distributions (MPSD) containing Lagrangian Poisson (LPD) (Consul and Jain, 1973) and Lagrangian binomial distributions (LBD) (Jain and Consul, 1971) was studied by Gupta (1974). We investigate the problem of finding the negative momentsE[X^-r ], of displaced and decapitated Modified Power Series Distributions. We derive the relationship between rand (r-1) negative moments. The negative moments of the decapitated and displaced LPD are obtained. These results are, then, used to find the exact amount of bias in the ML estimators of the parameters in the LPD and the LBD. We have also given the variances of the ML estimator and the minimum variance unbiased estimator of the parameter in the LPD.     

The optimal Lp norm estimator in linear regression models

The least squares estimator is usually applied when estimating the parameters in linear regression models. As this estimator is sensitive to departures from normality in the residual distribution, several alternatives have been proposed. The L_p norm estimators is one class of such alternatives. It has been proposed that the kurtosis of the residual distribution be taken into account when a choice of estimator in the L_p norm class is made (i.e. the choice of p). In this paper, the asymtotic variance of the estimators is used as the criterion in the choice of p. It is shown that when this criterion is applied, other characteristics of the residual distribution than the kurtosis (namely moments of order p-2 and 2p-2) are important.     

Three estimators for the poisson regression model with measurement errors

We consider two consistent estimators for the parameters of the linear predictor in the Poisson regression model, where the covariate is measured with errors. The measurement errors are assumed to be normally distributed with known error variance σ _u ² . The SQS estimator, based on a conditional mean-variance model, takes the distribution of the latent covariate into account, and this is here assumed to be a normal distribution. The CS estimator, based on a corrected score function, does not use the distribution of the latent covariate. Nevertheless, for small σ _u ² , both estimators have identical asymptotic covariance matrices up to the order of σ _u ² . We also compare the consistent estimators to the naive estimator, which is based on replacing the latent covariate with its (erroneously) measured counterpart. The naive estimator is biased, but has a smaller covariance matrix than the consistent estimators (at least up to the order of σ _u ² ).     

On the asymptotic non‐equivalence of efficient‐GMM and MEL estimators in models with missing data

The generalized method of moments (GMM) and empirical likelihood (EL) are popular methods for combining sample and auxiliary information. These methods are used in very diverse fields of research, where competing theories often suggest variables satisfying different moment conditions. Results in the literature have shown that the efficient‐GMM (GMM_E) and maximum empirical likelihood (MEL) estimators have the same asymptotic distribution to order n^?1/2 and that both estimators are asymptotically semiparametric efficient. In this paper, we demonstrate that when data are missing at random from the sample, the utilization of some well‐known missing‐data handling approaches proposed in the literature can yield GMM_E and MEL estimators with nonidentical properties; in particular, it is shown that the GMM_E estimator is semiparametric efficient under all the missing‐data handling approaches considered but that the MEL estimator is not always efficient. A thorough examination of the reason for the nonequivalence of the two estimators is presented. A particularly strong feature of our analysis is that we do not assume smoothness in the underlying moment conditions. Our results are thus relevant to situations involving nonsmooth estimating functions, including quantile and rank regressions, robust estimation, the estimation of receiver operating characteristic (ROC) curves, and so on.     

Improved two-parameter estimators for the negative binomial and Poisson regression models

Negative binomial regression (NBR) and Poisson regression (PR) applications have become very popular in the analysis of count data in recent years. However, if there is a high degree of relationship between the independent variables, the problem of multicollinearity arises in these models. We introduce new two-parameter estimators (TPEs) for the NBR and the PR models by unifying the two-parameter estimator (TPE) of Özkale and Kaç?ranlar [The restricted and unrestricted two-parameter estimators. Commun Stat Theory Methods. 2007;36:2707–2725]. These new estimators are general estimators which include maximum likelihood (ML) estimator, ridge estimator (RE), Liu estimator (LE) and contraction estimator (CE) as special cases. Furthermore, biasing parameters of these estimators are given and a Monte Carlo simulation is done to evaluate the performance of these estimators using mean square error (MSE) criterion. The benefits of the new TPEs are also illustrated in an empirical application. The results show that the new proposed TPEs for the NBR and the PR models are better than the ML estimator, the RE and the LE.     

PORT Hill and Moment Estimators for Heavy-Tailed Models

In this article, we use the peaks over random threshold (PORT)-methodology, and consider Hill and moment PORT-classes of extreme value index estimators. These classes of estimators are invariant not only to changes in scale, like the classical Hill and moment estimators, but also to changes in location. They are based on the sample of excesses over a random threshold, the order statistic X _[np]+1:n , 0 ≤ p < 1, being p a tuning parameter, which makes them highly flexible. Under convenient restrictions on the underlying model, these classes of estimators are consistent and asymptotically normal for adequate values of k, the number of top order statistics used in the semi-parametric estimation of the extreme value index γ. In practice, there may however appear a stability around a value distant from the target γ when the minimum is chosen for the random threshold, and attention is drawn for the danger of transforming the original data through the subtraction of the minimum. A new bias-corrected moment estimator is also introduced. The exact performance of the new extreme value index PORT-estimators is compared, through a large-scale Monte-Carlo simulation study, with the original Hill and moment estimators, the bias-corrected moment estimator, and one of the minimum-variance reduced-bias (MVRB) extreme value index estimators recently introduced in the literature. As an empirical example we estimate the tail index associated to a set of real data from the field of finance.     

Comparative Study of Blu and ML Estimators of Location and Scale Parameters of an Extreme Value Distribution for Small Samples and Type I Censorship

The Maximum Likelihood (ML) and Best Linear Unbiased (BLU) estimators of the location and scale parameters of an extreme value distribution (Lawless [1982]) are compared under conditions of small sample sizes and Type I censorship. The comparisons were made in terms of the mean square error criterion. According to this criterion, the ML estimator of σ in the case of very small sample sizes (n < 10) and heavy censorship (low censoring time) proved to be more efficient than the corresponding BLU estimator. However, the BLU estimator for σ attains parity with the corresponding ML estimator when the censoring time increases even for sample sizes as low as 10. The BLU estimator of σ attains equivalence with the ML estimator when the sample size increases above 10, particularly when the censoring time is also increased. The situation is reversed when it came to estimating the location parameter μ, as the BLU estimator was found to be consistently more efficient than the ML estimator despite the improved performance of the ML estimator when the sample size increases. However, computational ease and convenience favor the ML estimators.     

Choosing a robustness tuning parameter

A novel method is proposed for choosing the tuning parameter associated with a family of robust estimators. It consists of minimising estimated mean squared error, an approach that requires pilot estimation of model parameters. The method is explored for the family of minimum distance estimators proposed by [Basu, A., Harris, I.R., Hjort, N.L. and Jones, M.C., 1998, Robust and efficient estimation by minimising a density power divergence. Biometrika, 85, 549–559.] Our preference in that context is for a version of the method using the L ₂ distance estimator [Scott, D.W., 2001, Parametric statistical modeling by minimum integrated squared error. Technometrics, 43, 274–285.] as pilot estimator.     

New types of shrinkage estimators of Poisson means under the normalized squared error loss

In estimating p( ? 2) independent Poisson means, Clevenson and Zidek (1975) have proposed a class of estimators that shrink the unbiased estimator to the origin and dominate the unbiased one under the normalized squared error loss. This class of estimators was subsequently enlarged in several directions. This article deals with the problem and proposes new classes of dominating estimators using prior information pertinently. Dominance is shown by partitioning the sample space into disjoint subsets and averaging the loss difference over each subset. Estimation of several Poisson mean vectors is also discussed. Further, simultaneous estimation of Poisson means under order restriction is treated and estimators which dominate the isotonic regression estimator are proposed for some types of order restrictions.     

Combining poisson means

The problem of estimating the Poisson mean is considered based on the two samples in the presence of uncertain prior information (not in the form of distribution) that two independent random samples taken from two possibly identical Poisson populations. The parameter of interest is λ₁ from population I. Three estimators, i.e. the unrestricted estimator, restricted estimator and preliminary test estimator are proposed. Their asymptotic mean squared errors are derived and compared; parameter regions have been found for which restricted and preliminary test estimators are always asymptotically more efficient than the classical estimator. The relative dominance picture of the estimators is presented. Maximum and minimum asymptotic efficiencies of the estimators relative to the classical estimator are tabulated. A max-min rule for the size of the preliminary test is also discussed. A Monte Carlo study is presented to compare the performance of the estimator with that of Kale and Bancroft (1967).     

Estimating the parameter of selected uniform population under the squared log error loss function

Let π₁, …, π_k be k (? 2) independent populations, where π_i denotes the uniform distribution over the interval (0, θ_i) and θ_i > 0 (i = 1, …, k) is an unknown scale parameter. The population associated with the largest scale parameter is called the best population. For selecting the best population, We use a selection rule based on the natural estimators of θ_i, i = 1, …, k, for the case of unequal sample sizes. Consider the problem of estimating the scale parameter θ_L of the selected uniform population when sample sizes are unequal and the loss is measured by the squared log error (SLE) loss function. We derive the uniformly minimum risk unbiased (UMRU) estimator of θ_L under the SLE loss function and two natural estimators of θ_L are also studied. For k = 2, we derive a sufficient condition for inadmissibility of an estimator of θ_L. Using these condition, we conclude that the UMRU estimator and natural estimator are inadmissible. Finally, the risk functions of various competing estimators of θ_L are compared through simulation.     

Improved minimax estimation of a normal precision matrix

Let S_{p × p} have a Wishart distribution with parameter matrix Σ and n degrees of freedom. We consider here the problem of estimating the precision matrix Σ^?1 under the loss functions L₁(σ) tr (σ) - log |σ| and L₂(σ) = tr (σ). James-Stein-type estimators have been derived for an arbitrary p. We also obtain an orthogonal invariant and a diagonal invariant minimax estimator under both loss functions. A Monte-Carlo simulation study indicates that the risk improvement of the orthogonal invariant estimators over the James-Stein type estimators, the Haff (1979) estimator, and the “testimator” given by Sinha and Ghosh (1987) is substantial.     

Asymptotic distribution of regression correlation coefficient for Poisson regression model

This study treats an asymptotic distribution for measures of predictive power for generalized linear models (GLMs). We focus on the regression correlation coefficient (RCC) that is one of the measures of predictive power. The RCC, proposed by Zheng and Agresti is a population value and a generalization of the population value for the coefficient of determination. Therefore, the RCC is easy to interpret and familiar. Recently, Takahashi and Kurosawa provided an explicit form of the RCC and proposed a new RCC estimator for a Poisson regression model. They also showed the validity of the new estimator compared with other estimators. This study discusses the new statistical properties of the RCC for the Poisson regression model. Furthermore, we show an asymptotic normality of the RCC estimator.     

Density Estimation by Total Variation Penalized Likelihood Driven by the Sparsity ℓ1 Information Criterion

Abstract. We propose a non‐linear density estimator, which is locally adaptive, like wavelet estimators, and positive everywhere, without a log‐ or root‐transform. This estimator is based on maximizing a non‐parametric log‐likelihood function regularized by a total variation penalty. The smoothness is driven by a single penalty parameter, and to avoid cross‐validation, we derive an information criterion based on the idea of universal penalty. The penalized log‐likelihood maximization is reformulated as an ?₁‐penalized strictly convex programme whose unique solution is the density estimate. A Newton‐type method cannot be applied to calculate the estimate because the ?₁‐penalty is non‐differentiable. Instead, we use a dual block coordinate relaxation method that exploits the problem structure. By comparing with kernel, spline and taut string estimators on a Monte Carlo simulation, and by investigating the sensitivity to ties on two real data sets, we observe that the new estimator achieves good L ₁ and L ₂ risk for densities with sharp features, and behaves well with ties.     

Location adjustment for the minimum volume ellipsoid estimator

Estimating multivariate location and scatter with both affine equivariance and positive breakdown has always been difficult. A well-known estimator which satisfies both properties is the Minimum Volume Ellipsoid Estimator (MVE). Computing the exact MVE is often not feasible, so one usually resorts to an approximate algorithm. In the regression setup, algorithms for positive-breakdown estimators like Least Median of Squares typically recompute the intercept at each step, to improve the result. This approach is called intercept adjustment. In this paper we show that a similar technique, called location adjustment, can be applied to the MVE. For this purpose we use the Minimum Volume Ball (MVB), in order to lower the MVE objective function. An exact algorithm for calculating the MVB is presented. As an alternative to MVB location adjustment we propose L ₁ location adjustment, which does not necessarily lower the MVE objective function but yields more efficient estimates for the location part. Simulations compare the two types of location adjustment. We also obtain the maxbias curves of L ₁ and the MVB in the multivariate setting, revealing the superiority of L ₁.     

Goodness‐of‐fit tests for linear regression models with missing response data

The authors show how to test the goodness‐of‐fit of a linear regression model when there are missing data in the response variable. Their statistics are based on the L₂ distance between nonparametric estimators of the regression function and a ‐consistent estimator of the same function under the parametric model. They obtain the limit distribution of the statistics and check the validity of their bootstrap version. Finally, a simulation study allows them to examine the behaviour of their tests, whether the samples are complete or not.