Additive Transformation Models for Multivariate Interval-Censored Data

ABSTRACT

We present a new estimator of extreme quantiles dedicated to Weibull tail distributions. This estimate is based on a consistent estimator of the Weibull tail coefficient. This parameter is defined as the regular variation coefficient of the inverse cumulative hazard function. We give conditions in order to obtain the weak consistency and the asymptotic distribution of the extreme quantiles estimator. Its asymptotic as well as its finite sample performances are compared to classical ones.     

Hill estimator of projections of functional data on principal components

Functional principal component scores are commonly used to reduce mathematically infinitely dimensional functional data to finite dimensional vectors. In certain applications, most notably in finance, these scores exhibit tail behaviour consistent with the assumption of regular variation. Knowledge of the index of the regular variation, α, is needed to apply methods of extreme value theory. The most commonly used method of the estimation of α is the Hill estimator. We derive conditions under which the Hill estimator computed from the sample scores is consistent for the tail index of the unobservable population scores.     

Does bias reduction with external estimator of second order parameter work for endpoint?

Bias reduction estimation for tail index has been studied in the literature. One method is to reduce bias with an external estimator of the second order regular variation parameter; see Gomes and Martins [2002. Asymptotically unbiased estimators of the tail index based on external estimation of the second order parameter. Extremes 5(1), 5–31]. It is known that negative extreme value index implies that the underlying distribution has a finite right endpoint. As far as we know, there exists no bias reduction estimator for the endpoint of a distribution. In this paper, we study the bias reduction method with an external estimator of the second order parameter for both the negative extreme value index and endpoint simultaneously. Surprisingly, we find that this bias reduction method for negative extreme value index requires a larger order of sample fraction than that for positive extreme value index. This finding implies that this bias reduction method for endpoint is less attractive than that for positive extreme value index. Nevertheless, our simulation study prefers the proposed bias reduction estimators to the biased estimators in Hall [1982. On estimating the endpoint of a distribution. Ann. Statist. 10, 556–568].     

On Smooth Statistical Tail Functionals

Many estimators of the extreme value index of a distribution function F that are based on a certain number k _n of largest order statistics can be represented as a statistical tail function al, that is a functional T applied to the empirical tail quantile function Q _n. We study the asymptotic behaviour of such estimators with a scale and location invariant functional T under weak second order conditions on F . For that purpose first a new approximation of the empirical tail quantile function is established. As a consequence we obtain weak consistency and asymptotic normality of T ( Q _n) if T is continuous and Hadamard differentiable, respectively, at the upper quantile function of a generalized Pareto distribution and k _pn tends to infinity sufficiently slowly. Then we investigate the asymptotic variance and bias. In particular, those functionals T re characterized that lead to an estimator with minimal asymptotic variance. Finally, we introduce a method to construct estimators of the extreme value index with a made-to-order asymptotic behaviour     

Semiparametric lower bounds for tail index estimation

We consider estimation of the tail index parameter from i.i.d. observations in Pareto and Weibull type models, using a local and asymptotic approach. The slowly varying function describing the non-tail behavior of the distribution is considered as an infinite dimensional nuisance parameter. Without further regularity conditions, we derive a local asymptotic normality (LAN) result for suitably chosen parametric submodels of the full semiparametric model. From this result, we immediately obtain the optimal rate of convergence of tail index parameter estimators for more specific models previously studied. On top of the optimal rate of convergence, our LAN result also gives the minimal limiting variance of estimators (regular for our parametric model) through the convolution theorem. We show that the classical Hill estimator is regular for the submodels introduced with limiting variance equal to the induced convolution theorem bound. We also discuss the Weibull model in this respect.     

Extreme value index estimator using maximum likelihood and moment estimation

ABSTRACT

When a distribution function is in the max domain of attraction of an extreme value distribution, its tail can be well approximated by a generalized Pareto distribution. Based on this fact we use a moment estimation idea to propose an adapted maximum likelihood estimator for the extreme value index, which can be understood as a combination of the maximum likelihood estimation and moment estimation. Under certain regularity conditions, we derive the asymptotic normality of the new estimator and investigate its finite sample behavior by comparing with several classical or competitive estimators. A simulation study shows that the new estimator is competitive with other estimators in view of average bias, average MSE, and coefficient of variance of the new device for the optimal selection of the threshold.     

On the Optimality of Estimating the Tail Index and a Naive Estimator

Using a straightforward estimator for estimating the tail index of a distribution we illustrate the inherent difficulties of this problem. We prove strong and weak consistencies and central limit theorems for our naive estimator, and discuss its various rates of convergence under different conditions. We argue that, while optimal rates of convergence do exist under various conditions for a number of estimators of the tail index, the notion of an optimal sequence for this problem is bound to run into unsurmountable difficulties.     

Nonparametric regression estimation of conditional tails: the random covariate case

We present families of nonparametric estimators for the conditional tail index of a Pareto-type distribution in the presence of random covariates. These families are constructed from locally weighted sums of power transformations of excesses over a high threshold. The asymptotic properties of the proposed estimators are derived under some assumptions on the conditional response distribution, the weight function and the density function of the covariates. We also introduce bias-corrected versions of the estimators for the conditional tail index, and propose in this context a consistent estimator for the second-order tail parameter. The finite sample performance of some specific examples from our classes of estimators is illustrated with a small simulation experiment.     

Divergence based robust estimation of the tail index through an exponential regression model

The extreme value theory is very popular in applied sciences including finance, economics, hydrology and many other disciplines. In univariate extreme value theory, we model the data by a suitable distribution from the general max-domain of attraction characterized by its tail index; there are three broad classes of tails—the Pareto type, the Weibull type and the Gumbel type. The simplest and most common estimator of the tail index is the Hill estimator that works only for Pareto type tails and has a high bias; it is also highly non-robust in presence of outliers with respect to the assumed model. There have been some recent attempts to produce asymptotically unbiased or robust alternative to the Hill estimator; however all the robust alternatives work for any one type of tail. This paper proposes a new general estimator of the tail index that is both robust and has smaller bias under all the three tail types compared to the existing robust estimators. This essentially produces a robust generalization of the estimator proposed by Matthys and Beirlant (Stat Sin 13:853–880, 2003) under the same model approximation through a suitable exponential regression framework using the density power divergence. The robustness properties of the estimator are derived in the paper along with an extensive simulation study. A method for bias correction is also proposed with application to some real data examples.     

ON THE USE OF THE EMPIRICAL DISTRIBUTION AND CHARACTERISTIC FUNCTION TO ESTIMATE PARAMETERS OF REGULAR VARIATION

In this paper we present two new classes of estimators of parameters of regular variation, one based on the empirical distribution function and the other on the empirical characteristic function. They achieve the same rates of mean square error convergence as the estimators proposed by Hall (1982). The estimator based on the empirical characteristic function, unlike the other estimators, utilises the whole sample and not just a few extreme order statistics.     

Asymptotic Expansions of Estimators for the Tail Index with Applications

We present asymptotic expansions for two well-known estimators of the tail index of a distribution—the Hill's estimator and the simplified Pickands' estimator. We then use the expansions to get more accurate interval estimates. Comparisons between the two estimators are also discussed.     

A diagnostic for selecting the threshold in extreme value analysis

A new approach is suggested for choosing the threshold when fitting the Hill estimator of a tail exponent to extreme value data. Our method is based on an easily computed diagnostic, which in turn is founded directly on the Hill estimator itself, 'symmetrized' to remove the effect of the tail exponent but designed to emphasize biases in estimates of that exponent. The attractions of the method are its accuracy, its simplicity and the generality with which it applies. This generality implies that the technique has somewhat different goals from more conventional approaches, which are designed to accommodate the minor component of a postulated two-component Pareto mixture. Our approach does not rely on the second component being Pareto distributed. Nevertheless, in the conventional setting it performs competitively with recently proposed methods, and in more general cases it achieves optimal rates of convergence. A by-product of our development is a very simple and practicable exponential approximation to the distribution of the Hill estimator under departures from the Pareto distribution.     

A robust prediction error criterion for pareto modelling of upper tails

Estimation of the Pareto tail index from extreme order statistics is an important problem in many settings. The upper tail of the distribution, where data are sparse, is typically fitted with a model, such as the Pareto model, from which quantities such as probabilities associated with extreme events are deduced. The success of this procedure relies heavily not only on the choice of the estimator for the Pareto tail index but also on the procedure used to determine the number k of extreme order statistics that are used for the estimation. The authors develop a robust prediction error criterion for choosing k and estimating the Pareto index. A Monte Carlo study shows the good performance of the new estimator and the analysis of real data sets illustrates that a robust procedure for selection, and not just for estimation, is needed.     

Generalized Pickands estimators for the extreme value index

The Pickands estimator for the extreme value index is generalized in a way that includes all of its previously known variants. A detailed study of the asymptotic behavior of the estimators in the family serves to determine its optimally performing members. These are given by simple, explicit formulas, have the same asymptotic variance as the maximum likelihood estimator in the generalized Pareto model, and are robust to departures from the limiting generalized Pareto model in case the convergence of the excess distribution to its limit is slow. A simulation study involving a wide range of distributions shows the new estimators to compare favorably with the maximum likelihood estimator.     

Tail index and second-order parameters’ semi-parametric estimation based on the log-excesses

In this paper we are interested in the derivation of the asymptotic and finite-sample distributional properties of a ‘quasi-maximum likelihood’ estimator of a ‘scale’ second-order parameter β, directly based on the log-excesses of an available sample. Such estimation is of primordial importance for the adaptive selection of the optimal sample fraction to be used in the classical semi-parametric tail index estimation as well as for the reduced-bias estimation of the tail index, high quantiles and other parameters of extreme or even rare events. An application in the area of survival analysis is provided, on the basis of a data set on males diagnosed with cancer of the tongue.     

Local Estimation of the Second-Order Parameter in Extreme Value Statistics and Local Unbiased Estimation of the Tail Index

We develop and study in the framework of Pareto-type distributions a class of nonparametric kernel estimators for the conditional second order tail parameter. The estimators are obtained by local estimation of the conditional second order parameter using a moving window approach. Asymptotic normality of the proposed class of kernel estimators is proven under some suitable conditions on the kernel function and the conditional tail quantile function. The nonparametric estimators for the second order parameter are subsequently used to obtain a class of bias-corrected kernel estimators for the conditional tail index. In particular it is shown how for a given kernel function one obtains a bias-corrected kernel function, and that replacing the second order parameter in the latter with a consistent estimator does not change the limiting distribution of the bias-corrected estimator for the conditional tail index. The finite sample behavior of some specific estimators is illustrated with a simulation experiment. The developed methodology is also illustrated on fire insurance claim data.     

Approximations to the tail empirical distribution function with application to testing extreme value conditions

Weighted approximations to the tail of the distribution function and its empirical counterpart are derived which are suitable for applications in extreme value statistics. The approximation of the tail empirical distribution function is then used to develop an Anderson–Darling type test of the null hypothesis that the distribution function belongs to the domain of attraction of an extreme value distribution.     

Pitfalls in using Weibull tailed distributions

By assuming that the underlying distribution belongs to the domain of attraction of an extreme value distribution, one can extrapolate the data to a far tail region so that a rare event can be predicted. However, when the distribution is in the domain of attraction of a Gumbel distribution, the extrapolation is quite limited generally in comparison with a heavy tailed distribution. In view of this drawback, a Weibull tailed distribution has been studied recently. Some methods for choosing the sample fraction in estimating the Weibull tail coefficient and some bias reduction estimators have been proposed in the literature. In this paper, we show that the theoretical optimal sample fraction does not exist and a bias reduction estimator does not always produce a smaller mean squared error than a biased estimator. These are different from using a heavy tailed distribution. Further we propose a refined class of Weibull tailed distributions which are more useful in estimating high quantiles and extreme tail probabilities.     

Extreme Quantile Estimation for Autoregressive Models

ABSTRACT

A quantile autoregresive model is a useful extension of classical autoregresive models as it can capture the influences of conditioning variables on the location, scale, and shape of the response distribution. However, at the extreme tails, standard quantile autoregression estimator is often unstable due to data sparsity. In this article, assuming quantile autoregresive models, we develop a new estimator for extreme conditional quantiles of time series data based on extreme value theory. We build the connection between the second-order conditions for the autoregression coefficients and for the conditional quantile functions, and establish the asymptotic properties of the proposed estimator. The finite sample performance of the proposed method is illustrated through a simulation study and the analysis of U.S. retail gasoline price.     

A robust and efficient estimation method for nonparametric models with jump points

Nonparametric models with jump points have been considered by many researchers. However, most existing methods based on least squares or likelihood are sensitive when there are outliers or the error distribution is heavy tailed. In this article, a local piecewise-modal method is proposed to estimate the regression function with jump points in nonparametric models, and a piecewise-modal EM algorithm is introduced to estimate the proposed estimator. Under some regular conditions, the large-sample theory is established for the proposed estimators. Several simulations are presented to evaluate the performances of the proposed method, which shows that the proposed estimator is more efficient than the local piecewise-polynomial regression estimator in the presence of outliers or heavy tail error distribution. What is more, the proposed procedure is asymptotically equivalent to the local piecewise-polynomial regression estimator under the assumption that the error distribution is a Gaussian distribution. The proposed method is further illustrated via the sea-level pressures.