A comparison of estimation techniques for the three parameter pareto distribution

This paper compares minimum distance estimation with best linear unbiased estimation to determine which technique provides the most accurate estimates for location and scale parameters as applied to the three parameter Pareto distribution. Two minimum distance estimators are developed for each of the three distance measures used (Kolmogorov, Cramer‐von Mises, and Anderson‐Darling) resulting in six new estimators. For a given sample size 6 or 18 and shape parameter 1(1)4, the location and scale parameters are estimated. A Monte Carlo technique is used to generate the sample sets. The best linear unbiased estimator and the six minimum distance estimators provide parameter estimates based on each sample set. These estimates are compared using mean square error as the evaluation tool. Results show that the best linear unbaised estimator provided more accurate estimates of location and scale than did the minimum estimators tested.     

Choosing a robustness tuning parameter

A novel method is proposed for choosing the tuning parameter associated with a family of robust estimators. It consists of minimising estimated mean squared error, an approach that requires pilot estimation of model parameters. The method is explored for the family of minimum distance estimators proposed by [Basu, A., Harris, I.R., Hjort, N.L. and Jones, M.C., 1998, Robust and efficient estimation by minimising a density power divergence. Biometrika, 85, 549–559.] Our preference in that context is for a version of the method using the L ₂ distance estimator [Scott, D.W., 2001, Parametric statistical modeling by minimum integrated squared error. Technometrics, 43, 274–285.] as pilot estimator.     

Minimum-Distance Estimator for Stable Exponent

Assume that X ₁, X ₂,…, X _n is a sequence of i.i.d. random variables with α-stable distribution (α ∈ (0,2], the stable exponent, is the unknown parameter). We construct minimum distance estimators for α by minimizing the Kolmogorov distance or the Cramér–von-Mises distance between the empirical distribution function G _n , and a class of distributions defined based on the sum-preserving property of stable random variables. The minimum distance estimators can also be obtained by minimizing a U-statistic estimate of an empirical distribution function involving the stable exponent. They share the same invariance property with the maximum likelihood estimates. In this article, we prove the strong consistency of the minimum distance estimators. We prove the asymptotic normality of our estimators. Simulation study shows that the new estimators are competitive to the existing ones and perform very closely even to the maximum likelihood estimator.     

Minimum hellinger distance estimation for normal models

A robust estimator introduced by Beran (1977a, 1977b), which is based on the minimum Hellinger distance between a projection model density and a nonparametric sample density, is studied empirically. An extensive simulation provides an estimate of the small sample distribution and supplies empirical evidence of the estimator performance for a normal location-scale model. While the performance of the minimum Hellinger distance estimator is seen to be competitive with the maximum likelihood estimator at the true model, its robustness to deviations from normality is shown to be competitive in this setting with that obtained from the M-estimator and the Cramér-von Mises minimum distance estimator. Beran also introduced a goodness-of-fit statisticH ₂, based on the minimized Hellinger distance between a member of a parametric family of densities and a nonparametric density estimate. We investigate the statistic H (the square root of H ₂) as a test for normality when both location and scale are unspecified. Empirically derived critical values are given which do not require extensive tables. The power of the statistic H compares favorably with four other widely used tests for normality.     

PORT Hill and Moment Estimators for Heavy-Tailed Models

In this article, we use the peaks over random threshold (PORT)-methodology, and consider Hill and moment PORT-classes of extreme value index estimators. These classes of estimators are invariant not only to changes in scale, like the classical Hill and moment estimators, but also to changes in location. They are based on the sample of excesses over a random threshold, the order statistic X _[np]+1:n , 0 ≤ p < 1, being p a tuning parameter, which makes them highly flexible. Under convenient restrictions on the underlying model, these classes of estimators are consistent and asymptotically normal for adequate values of k, the number of top order statistics used in the semi-parametric estimation of the extreme value index γ. In practice, there may however appear a stability around a value distant from the target γ when the minimum is chosen for the random threshold, and attention is drawn for the danger of transforming the original data through the subtraction of the minimum. A new bias-corrected moment estimator is also introduced. The exact performance of the new extreme value index PORT-estimators is compared, through a large-scale Monte-Carlo simulation study, with the original Hill and moment estimators, the bias-corrected moment estimator, and one of the minimum-variance reduced-bias (MVRB) extreme value index estimators recently introduced in the literature. As an empirical example we estimate the tail index associated to a set of real data from the field of finance.     

FDA: strong consistency of the kNN local linear estimation of the functional conditional density and mode

In this paper we present a new estimator of the conditional density and mode when the co-variables are of functional kind. This estimator is a combination of both, the k-Nearest Neighbours procedure and the functional local linear estimation. Then, for each statistical parameter (conditional density or mode), results concerning the strong consistency and rate of convergence of the estimators are presented. Finally, their performances, for finite sample sizes, are illustrated by using simulated data.     

Profile Hellinger distance estimation

The successful application of the Hellinger distance approach to fully parametric models is well known. The corresponding optimal estimators, known as minimum Hellinger distance (MHD) estimators, are efficient and have excellent robustness properties [Beran R. Minimum Hellinger distance estimators for parametric models. Ann Statist. 1977;5:445–463]. This combination of efficiency and robustness makes MHD estimators appealing in practice. However, their application to semiparametric statistical models, which have a nuisance parameter (typically of infinite dimension), has not been fully studied. In this paper, we investigate a methodology to extend the MHD approach to general semiparametric models. We introduce the profile Hellinger distance and use it to construct a minimum profile Hellinger distance estimator of the finite-dimensional parameter of interest. This approach is analogous in some sense to the profile likelihood approach. We investigate the asymptotic properties such as the asymptotic normality, efficiency, and adaptivity of the proposed estimator. We also investigate its robustness properties. We present its small-sample properties using a Monte Carlo study.     

Bias reduction of a tail index estimator through an external estimation of the second-order parameter

In this paper, we first consider a class of consistent semi-parametric estimators of a positive tail index γ, parameterised in a tuning or control parameter α. Such a control parameter enables us to have access, for any available sample, to an estimator of the tail index γ with a null dominant component of asymptotic bias, and consequently with a reasonably flat mean squared error pattern, as a function of k, the number of top-order statistics considered. Such a control parameter depends on a second-order parameter ρ, which will be adequately estimated so that we may achieve a high efficiency relative to the classical Hill estimator, provided we use a number of top-order statistics larger than the one usually required for the estimation through the Hill estimator. An illustration of the behaviour of the estimators is provided, through the analysis of the daily log-returns on the Euro–US$ exchange rates.     

The minimum L2 distance estimator for Poisson mixture models

A robust estimator is developed for Poisson mixture models with a known number of components. The proposed estimator minimizes the L₂ distance between a sample of data and the model. When the component distributions are completely known, the estimators for the mixing proportions are in closed form. When the parameters for the component Poisson distributions are unknown, numerical methods are needed to calculate the estimators. Compared to the minimum Hellinger distance estimator, the minimum L₂ estimator can be less robust to extreme outliers, and often more robust to moderate outliers.     

Estimation and Inference for Linear Panel Data Models Under Misspecification When Both n and T are Large

This article considers fixed effects (FE) estimation for linear panel data models under possible model misspecification when both the number of individuals, n, and the number of time periods, T, are large. We first clarify the probability limit of the FE estimator and argue that this probability limit can be regarded as a pseudo-true parameter. We then establish the asymptotic distributional properties of the FE estimator around the pseudo-true parameter when n and T jointly go to infinity. Notably, we show that the FE estimator suffers from the incidental parameters bias of which the top order is O(T^{? 1}), and even after the incidental parameters bias is completely removed, the rate of convergence of the FE estimator depends on the degree of model misspecification and is either (nT)^{? 1/2} or n^{? 1/2}. Second, we establish asymptotically valid inference on the (pseudo-true) parameter. Specifically, we derive the asymptotic properties of the clustered covariance matrix (CCM) estimator and the cross-section bootstrap, and show that they are robust to model misspecification. This establishes a rigorous theoretical ground for the use of the CCM estimator and the cross-section bootstrap when model misspecification and the incidental parameters bias (in the coefficient estimate) are present. We conduct Monte Carlo simulations to evaluate the finite sample performance of the estimators and inference methods, together with a simple application to the unemployment dynamics in the U.S.     

Estimation of Pr( x<y ) in the exponential case with common location parameter

This paper considers the problem of estimating the probability P = Pr(X < Y) when X and Y are independent exponential random variables with unequal scale parameters and a common location parameter. Uniformly minimum variance unbiased estimator of P is obtained. The asymptotic distribution of the maximum likelihood estimator is obtained and then the asymptotic equivalence of the two estimators is established. Performance of the two estimators for moderate sample sizes is studied by Monte Carlo simulation. An approximate interval estimator is also obtained.     

Kernel Estimators for Distribution Functions on Dependent Random Fields

Consider observations (representing lifelengths) taken on a random field indexed by lattice points. Estimating the distribution function F(x) = P(X _i  ≤ x) is an important problem in survival analysis. We propose to estimate F(x) by kernel estimators, which take into account the smoothness of the distribution function. Under some general mixing conditions, our estimators are shown to be asymptotically unbiased and consistent. In addition, the proposed estimator is shown to be strongly consistent and sharp rates of convergence are obtained.     

Efficiency and Validity Analyses of Two-Stage Estimation Procedures and Derived Testing Procedures in Quantitative Linear Models with AR(1) Errors

Abstract

In a quantitative linear model with errors following a stationary Gaussian, first-order autoregressive or AR(1) process, Generalized Least Squares (GLS) on raw data and Ordinary Least Squares (OLS) on prewhitened data are efficient methods of estimation of the slope parameters when the autocorrelation parameter of the error AR(1) process, ρ, is known. In practice, ρ is generally unknown. In the so-called two-stage estimation procedures, ρ is then estimated first before using the estimate of ρ to transform the data and estimate the slope parameters by OLS on the transformed data. Different estimators of ρ have been considered in previous studies. In this article, we study nine two-stage estimation procedures for their efficiency in estimating the slope parameters. Six of them (i.e., three noniterative, three iterative) are based on three estimators of ρ that have been considered previously. Two more (i.e., one noniterative, one iterative) are based on a new estimator of ρ that we propose: it is provided by the sample autocorrelation coefficient of the OLS residuals at lag 1, denoted r(1). Lastly, REstricted Maximum Likelihood (REML) represents a different type of two-stage estimation procedure whose efficiency has not been compared to the others yet. We also study the validity of the testing procedures derived from GLS and the nine two-stage estimation procedures. Efficiency and validity are analyzed in a Monte Carlo study. Three types of explanatory variable x in a simple quantitative linear model with AR(1) errors are considered in the time domain: Case 1, x is fixed; Case 2, x is purely random; and Case 3, x follows an AR(1) process with the same autocorrelation parameter value as the error AR(1) process. In a preliminary step, the number of inadmissible estimates and the efficiency of the different estimators of ρ are compared empirically, whereas their approximate expected value in finite samples and their asymptotic variance are derived theoretically. Thereafter, the efficiency of the estimation procedures and the validity of the derived testing procedures are discussed in terms of the sample size and the magnitude and sign of ρ. The noniterative two-stage estimation procedure based on the new estimator of ρ is shown to be more efficient for moderate values of ρ at small sample sizes. With the exception of small sample sizes, REML and its derived F-test perform the best overall. The asymptotic equivalence of two-stage estimation procedures, besides REML, is observed empirically. Differences related to the nature, fixed or random (uncorrelated or autocorrelated), of the explanatory variable are also discussed.     

Modified minimum covariance determinant estimator and its application to outlier detection of chemical process data

To overcome the main flaw of minimum covariance determinant (MCD) estimator, i.e. difficulty to determine its main parameter h, a modified-MCD (M-MCD) algorithm is proposed. In M-MCD, the self-adaptive iteration is proposed to minimize the deflection between the standard deviation of robust mahalanobis distance square, which is calculated by MCD with the parameter h based on the sample, and the standard deviation of theoretical mahalanobis distance square by adjusting the parameter h of MCD. Thus, the optimal parameter h of M-MCD is determined when the minimum deflection is obtained. The results of convergence analysis demonstrate that M-MCD has good convergence property. Further, M-MCD and MCD were applied to detect outliers for two typical data and chemical process data, respectively. The results show that M-MCD can get the optimal parameter h by using the self-adaptive iteration and thus its performances of outlier detection are better than MCD.     

Cross-validation Revisited

Data-based choice of the bandwidth is an important problem in kernel density estimation. The pseudo-likelihood and the least-squares cross-validation bandwidth selectors are well known, but widely criticized in the literature. For heavy-tailed distributions, the L₁ distance between the pseudo-likelihood-based estimator and the density does not seem to converge in probability to zero with increasing sample size. Even for normal-tailed densities, the rate of L₁ convergence is disappointingly slow. In this article, we report an interesting finding that with minor modifications both the cross-validation methods can be implemented effectively, even for heavy-tailed densities. For both these estimators, the L₁ distance (from the density) are shown to converge completely to zero irrespective of the tail of the density. The expected L₁ distance also goes to zero. These results hold even in the presence of a strongly mixing-type dependence. Monte Carlo simulations and analysis of the Old Faithful geyser data suggest that if implemented appropriately, contrary to the traditional belief, the cross-validation estimators compare well with the sophisticated plug-in and bootstrap-based estimators.     

Hellinger distance as a penalized log likelihood

The present paper studies the minimum Hellinger distance estimator by recasting it as the maximum likelihood estimator in a data driven modification of the model density. In the process, the Hellinger distance itself is expressed as a penalized log likelihood function. The penalty is the sum of the model probabilities over the non-observed values of the sample space. A comparison of the modified model density with the original data provides insights into the robustness of the minimum Hellinger distance estimator. Adjustments of the amount of penalty leads to a class of minimum penalized Hellinger distance estimators, some members of which perform substantially better than the minimum Hellinger distance estimator at the model for small samples, without compromising the robustness properties of the latter.     

Simultaneous robust estimation of location and scale parameters: A minimum-distance approach

Simultaneous robust estimates of location and scale parameters are derived from minimizing a minimum-distance criterion function. The criterion function measures the squared distance between the pth power (p > 0) of the empirical distribution function and the pth power of the imperfectly determined model distribution function over the real line. We show that the estimator is uniquely defined, is asymptotically bivariate normal and for p > 0.3 has positive breakdown. If the scale parameter is known, when p = 0.9 the asymptotic variance (1.0436) of the location estimator for the normal model is smaller than the asymptotic variance of the Hodges-Lehmann (HL)estimator (1.0472). Efficiencies with respect to HL and maximum-likelihood estimators (MLE) are 1.0034 and 0.9582, respectively. Similarly, if the location parameter is known, when p = 0.97 the asymptotic variance (0.6158) of the scale estimator is minimum. The efficiency with respect to the MLE is 0.8119. We show that the estimator can tolerate more corrupted observations at oo than at – for p < 1, and vice versa for p > 1.     

A bias-corrected histogram estimator for line transect sampling

The classical histogram method has already been applied in line transect sampling to estimate the parameter f(0), which in turns is used to estimate the population abundance D or the population size N. It is well know that the bias convergence rate for histogram estimator of f(0) is o(h²) as h → 0, under the shoulder condition assumption. If the shoulder condition is not true, then the bias convergence rate is only o(h). This paper proposed two new estimators for f(0), which can be considered as modifications of the classical histogram estimator. The first estimator is derived when the shoulder condition is assumed to be valid and it reduces the bias convergence rate from o(h²) to o(h³). The other one is constructed without using the shoulder condition assumption and it reduces the bias convergence rate from o(h) to o(h²). The asymptotic properties of the proposed estimators are derived and formulas for bin width are also given. The finite properties based on a real data set and an extensive simulation study demonstrated the potential practical use of the proposed estimators.     

Recursive kernel estimate of the conditional quantile for functional ergodic data

ABSTRACT

In this article, we study the recursive kernel estimator of the conditional quantile of a scalar response variable Y given a random variable (rv) X taking values in a semi-metric space. Two estimators are considered. While the first one is given by inverting the double-kernel estimate of the conditional distribution function, the second estimator is obtained by using the robust approach. We establish the almost complete consistency of these estimates when the observations are sampled from a functional ergodic process. Finally, a simulation study is carried out to illustrate the finite sample performance of these estimators.     

Multiparameter estimation of discrete exponential distributions

Let X₁, …, X_p be independent random variables, all having the same distribution up to a possibly varying unspecified parameter, where each of the p distributions belongs to the family of one parameter discrete exponential distributions. The problem is to estimate the unknown parameters simultaneously. Hudson (1978) shows that the minimum variance unbiased estimator (MVUE) of the parameters is inadmissible under squared error loss, and estimators better than the MVUE are proposed. Essentially, these estimators shrink the MVUE towards the origin. In this paper, we indicate that estimators shifting the MVUE towards a point different from the origin or a point determined by the observations can be obtained.