Divergence based robust estimation of the tail index through an exponential regression model

The extreme value theory is very popular in applied sciences including finance, economics, hydrology and many other disciplines. In univariate extreme value theory, we model the data by a suitable distribution from the general max-domain of attraction characterized by its tail index; there are three broad classes of tails—the Pareto type, the Weibull type and the Gumbel type. The simplest and most common estimator of the tail index is the Hill estimator that works only for Pareto type tails and has a high bias; it is also highly non-robust in presence of outliers with respect to the assumed model. There have been some recent attempts to produce asymptotically unbiased or robust alternative to the Hill estimator; however all the robust alternatives work for any one type of tail. This paper proposes a new general estimator of the tail index that is both robust and has smaller bias under all the three tail types compared to the existing robust estimators. This essentially produces a robust generalization of the estimator proposed by Matthys and Beirlant (Stat Sin 13:853–880, 2003) under the same model approximation through a suitable exponential regression framework using the density power divergence. The robustness properties of the estimator are derived in the paper along with an extensive simulation study. A method for bias correction is also proposed with application to some real data examples.     

Tail exponent estimation via broadband log density-quantile regression

Heavy tail probability distributions are important in many scientific disciplines such as hydrology, geology, and physics and therefore feature heavily in statistical practice. Rather than specifying a family of heavy-tailed distributions for a given application, it is more common to use a nonparametric approach, where the distributions are classified according to the tail behavior. Through the use of the logarithm of Parzen's density-quantile function, this work proposes a consistent, flexible estimator of the tail exponent. The approach we develop is based on a Fourier series estimator and allows for separate estimates of the left and right tail exponents. The theoretical properties for the tail exponent estimator are determined, and we also provide some results of independent interest that may be used to establish weak convergence of stochastic processes. We assess the practical performance of the method by exploring its finite sample properties in simulation studies. The overall performance is competitive with classical tail index estimators, and, in contrast, with these our method obtains somewhat better results in the case of lighter heavy-tailed distributions.     

Hill estimator of projections of functional data on principal components

Functional principal component scores are commonly used to reduce mathematically infinitely dimensional functional data to finite dimensional vectors. In certain applications, most notably in finance, these scores exhibit tail behaviour consistent with the assumption of regular variation. Knowledge of the index of the regular variation, α, is needed to apply methods of extreme value theory. The most commonly used method of the estimation of α is the Hill estimator. We derive conditions under which the Hill estimator computed from the sample scores is consistent for the tail index of the unobservable population scores.     

Semiparametric lower bounds for tail index estimation

We consider estimation of the tail index parameter from i.i.d. observations in Pareto and Weibull type models, using a local and asymptotic approach. The slowly varying function describing the non-tail behavior of the distribution is considered as an infinite dimensional nuisance parameter. Without further regularity conditions, we derive a local asymptotic normality (LAN) result for suitably chosen parametric submodels of the full semiparametric model. From this result, we immediately obtain the optimal rate of convergence of tail index parameter estimators for more specific models previously studied. On top of the optimal rate of convergence, our LAN result also gives the minimal limiting variance of estimators (regular for our parametric model) through the convolution theorem. We show that the classical Hill estimator is regular for the submodels introduced with limiting variance equal to the induced convolution theorem bound. We also discuss the Weibull model in this respect.     

Moment-based tail index estimation

A general method of tail index estimation for heavy-tailed time series, based on examining the growth rate of the logged sample second moment of the data was proposed and studied in Meerschaert and Scheffler (1998. A simple robust estimator for the thickness of heavy tails. J. Statist. Plann. Inference 71, 19–34) as well as Politis (2002. A new approach on estimation of the tail index. C. R. Acad. Sci. Paris, Ser. I 335, 279–282). To improve upon the basic estimator, we introduce a scale-invariant estimator that is computed over subsets of the whole data set. We show that the new estimator, under some stronger conditions on the data, has a polynomial rate of consistency for the tail index. Empirical studies explore how the new method compares with the Hill, Pickands, and DEdH estimators.     

A robust prediction error criterion for pareto modelling of upper tails

Estimation of the Pareto tail index from extreme order statistics is an important problem in many settings. The upper tail of the distribution, where data are sparse, is typically fitted with a model, such as the Pareto model, from which quantities such as probabilities associated with extreme events are deduced. The success of this procedure relies heavily not only on the choice of the estimator for the Pareto tail index but also on the procedure used to determine the number k of extreme order statistics that are used for the estimation. The authors develop a robust prediction error criterion for choosing k and estimating the Pareto index. A Monte Carlo study shows the good performance of the new estimator and the analysis of real data sets illustrates that a robust procedure for selection, and not just for estimation, is needed.     

Optimal choice of sample fraction in univariate financial tail index estimation

This study introduces a technique to estimate the Pareto distribution of the stock exchange index by using the maximum-likelihood Hill estimator. Recursive procedures based on the goodness-of-fit statistics are used to determine the optimal threshold fraction of extreme values to be included in tail estimation. These procedures are applied to three indices in the Malaysian stock market which included the consideration of a drastic economic event such as the Asian financial crisis. The empirical results evidenced alternating varying behavior of heavy-tailed distributions in the regimes for both upper and lower tails.     

Fat tail distributions and local thin tail alternatives

The behaviour of the Hill estimator for the tail index of fat tailed distributions in the presence of local alternatives which have a thin tail is investigated. The converse problem is also briefly addressed. A local thin tail alternative can severely bias the Hill statistic. The relevance of this issue for the class of stable distributions is discussed. We conduct a small simulation study to support the analysis. In the conclusion it is argued that for moderate out of sample quantile analysis the problem of local alternatives may be less pressing.     

Bias reduction of a tail index estimator through an external estimation of the second-order parameter

In this paper, we first consider a class of consistent semi-parametric estimators of a positive tail index γ, parameterised in a tuning or control parameter α. Such a control parameter enables us to have access, for any available sample, to an estimator of the tail index γ with a null dominant component of asymptotic bias, and consequently with a reasonably flat mean squared error pattern, as a function of k, the number of top-order statistics considered. Such a control parameter depends on a second-order parameter ρ, which will be adequately estimated so that we may achieve a high efficiency relative to the classical Hill estimator, provided we use a number of top-order statistics larger than the one usually required for the estimation through the Hill estimator. An illustration of the behaviour of the estimators is provided, through the analysis of the daily log-returns on the Euro–US$ exchange rates.     

Tail index estimation in small smaples Simulation results for independent and ARCH-type financial return models

Estimation of the tail index of stationary, fat-tailed return distributions is non-trivial since the well-known Hill estimator is optimal only under iid draws from an exact Pareto model. We provide a small sample simulation study of recently suggested adaptive estimators under ARCH-type dependence. The Hill estimator’s performance is found to be dominated by a ratio estimator. Dependence increases estimation error which can remain substantial even in larger data sets. As small sample bias is related to the magnitude of the tail index, recent standard applications may have overestimated (underestimated) the risk of assets with low (high) degrees of fat-tailedness. This paper is a shortened version of the Berkeley Research Program in Finance Working Paper RPF-295. Thanks are to the Center for Mathematical Sciences at Munich University of Technology for generously providing access to computer facilities and to participants at the IAFE 2001 Budapest, OR 2002 Klagenfurt, EIR 2002 London, DGF 2002 Cologne, FBI 2002 Karlsruhe conferences and the 2001 Wallis Workshop for helpful comments. Two anonymous referees provided helpful suggestions in streamlining the material. Niklas Wagner acknowledges a Maple program by Klaus Kiefersbeck and financial support by Deutsche Forschungsgemeinschaft (DFG).     

Generalised Smooth Tests of Goodness of Fit Utilising L‐moments

In this paper we present a semiparametric test of goodness of fit which is based on the method of L‐moments for the estimation of the nuisance parameters. This test is particularly useful for any distribution that has a convenient expression for its quantile function. The test proceeds by investigating equality of the first few L‐moments of the true and the hypothesised distributions. We provide details and undertake simulation studies for the logistic and the generalised Pareto distributions. Although for some distributions the method of L‐moments estimator is less efficient than the maximum likelihood estimator, the former method has the advantage that it may be used in semiparametric settings and that it requires weaker existence conditions. The new test is often more powerful than competitor tests for goodness of fit of the logistic and generalised Pareto distributions.     

On tail index estimation based on multivariate data

This article is devoted to the study of tail index estimation based on i.i.d. multivariate observations, drawn from a standard heavy-tailed distribution, that is, of which Pareto-like marginals share the same tail index. A multivariate central limit theorem for a random vector, whose components correspond to (possibly dependent) Hill estimators of the common tail index α, is established under mild conditions. We introduce the concept of (standard) heavy-tailed random vector of tail index α and show how this limit result can be used in order to build an estimator of α with small asymptotic mean squared error, through a proper convex linear combination of the coordinates. Beyond asymptotic results, simulation experiments illustrating the relevance of the approach promoted are also presented.     

α-stable laws for noncoding regions in DNA sequences

In this work, we analyze the long-range dependence parameter for a nucleotide sequence in several different transformations. The long-range dependence parameter is estimated by the approximated maximum likelihood method, by a novel estimator based on the spectral envelope theory, by a regression method based on the periodogram function, and also by the detrended fluctuation analysis method. We study the length distribution of coding and noncoding regions for all Homo sapiens chromosomes available from the European Bioinformatics Institute. The parameter of the tail rate decay is estimated by the Hill estimator ?α. We show that the tail rate decay is greater than 2 for coding regions, while for almost all noncoding regions it is less than 2.     

The failure of meta-analytic asymptotics for the seemingly efficient estimator of the common risk difference

We consider the case of a multicenter trial in which the center specific sample sizes are potentially small. Under homogeneity, the conventional procedure is to pool information using a weighted estimator where the weights used are inverse estimated center-specific variances. Whereas this procedure is efficient for conventional asymptotics (e. g. center-specific sample sizes become large, number of center fixed), it is commonly believed that the efficiency of this estimator holds true also for meta-analytic asymptotics (e.g. center-specific sample size bounded, potentially small, and number of centers large). In this contribution we demonstrate that this estimator fails to be efficient. In fact, it shows a persistent bias with increasing number of centers showing that it isnot meta-consistent. In addition, we show that the Cochran and Mantel-Haenszel weighted estimators are meta-consistent and, in more generality, provide conditions on the weights such that the associated weighted estimator is meta-consistent.     

Extreme value index estimator using maximum likelihood and moment estimation

ABSTRACT

When a distribution function is in the max domain of attraction of an extreme value distribution, its tail can be well approximated by a generalized Pareto distribution. Based on this fact we use a moment estimation idea to propose an adapted maximum likelihood estimator for the extreme value index, which can be understood as a combination of the maximum likelihood estimation and moment estimation. Under certain regularity conditions, we derive the asymptotic normality of the new estimator and investigate its finite sample behavior by comparing with several classical or competitive estimators. A simulation study shows that the new estimator is competitive with other estimators in view of average bias, average MSE, and coefficient of variance of the new device for the optimal selection of the threshold.     

A method of moments estimator of tail dependence in meta-elliptical models

A meta-elliptical model is a distribution function whose copula is that of an elliptical distribution. The tail dependence function in such a bivariate model has a parametric representation with two parameters: a tail parameter and a correlation parameter. The correlation parameter can be estimated by robust methods based on the whole sample. Using the estimated correlation parameter as plug-in estimator, we then estimate the tail parameter applying a modification of the method of moments approach proposed in the paper by Einmahl et al. (2008). We show that such an estimator is consistent and asymptotically normal. Further, we derive the joint limit distribution of the estimators of the two parameters. We illustrate the small sample behavior of the estimator of the tail parameter by a simulation study and on real data, and we compare its performance to that of the competitive estimators.     

On the identification of extreme outliers and dragon-kings mechanisms in the upper tail of income distribution

The presence of extreme outliers in the upper tail data of income distribution affects the Pareto tail modeling. A simulation study is carried out to compare the performance of three types of boxplot in the detection of extreme outliers for Pareto data, including standard boxplot, adjusted boxplot and generalized boxplot. It is found that the generalized boxplot is the best method for determining extreme outliers for Pareto distributed data. For the application, the generalized boxplot is utilized for determining the exreme outliers in the upper tail of Malaysian income distribution. In addition, for this data set, the confidence interval method is applied for examining the presence of dragon-kings, extreme outliers which are beyond the Pareto or power-laws distribution.     

Two-sample inference procedures under nonproportional hazards

We introduce a new two-sample inference procedure to assess the relative performance of two groups over time. Our model-free method does not assume proportional hazards, making it suitable for scenarios where nonproportional hazards may exist. Our procedure includes a diagnostic tau plot to identify changes in hazard timing and a formal inference procedure. The tau-based measures we develop are clinically meaningful and provide interpretable estimands to summarize the treatment effect over time. Our proposed statistic is a U-statistic and exhibits a martingale structure, allowing us to construct confidence intervals and perform hypothesis testing. Our approach is robust with respect to the censoring distribution. We also demonstrate how our method can be applied for sensitivity analysis in scenarios with missing tail information due to insufficient follow-up. Without censoring, Kendall's tau estimator we propose reduces to the Wilcoxon-Mann–Whitney statistic. We evaluate our method using simulations to compare its performance with the restricted mean survival time and log-rank statistics. We also apply our approach to data from several published oncology clinical trials where nonproportional hazards may exist.     

Optimal estimation for the Pareto distribution

This paper proposes an optimal estimation method for the shape parameter, probability density function and upper tail probability of the Pareto distribution. The new method is based on a weighted empirical distribution function. The exact efficiency functions of the estimators relative to the existing estimators are derived. The paper gives L ₁-optimal and L ₂-optimal weights for the new weighted estimator. Monte Carlo simulation results confirm the theoretical conclusions. Both theoretical and simulation results show that the new estimation method is more efficient relative to several existing methods in many situations.     

Generalized Pickands estimators for the extreme value index

The Pickands estimator for the extreme value index is generalized in a way that includes all of its previously known variants. A detailed study of the asymptotic behavior of the estimators in the family serves to determine its optimally performing members. These are given by simple, explicit formulas, have the same asymptotic variance as the maximum likelihood estimator in the generalized Pareto model, and are robust to departures from the limiting generalized Pareto model in case the convergence of the excess distribution to its limit is slow. A simulation study involving a wide range of distributions shows the new estimators to compare favorably with the maximum likelihood estimator.