Reducing bias of the maximum-likelihood estimation for the truncated Pareto distribution

The Pareto distribution is an important distribution in statistics, which has been widely used in finance, physics, hydrology, geology, astronomy, and so on. Even though the parameter estimation for the Pareto distribution has been well established in the literature, the estimation problem for the truncated Pareto distribution becomes complex. This article investigates the bias and mean-squared error of the maximum-likelihood estimation for the truncated Pareto distribution, and some useful results are obtained.     

Inference for a Hidden Truncated (Both-Sided) Bivariate Pareto (II) Distribution

The Pareto distribution is a simple model for non negative data with a power law probability tail. Income and wealth data are typically modeled using some variant of the classical Pareto distribution. In practice, it is frequently likely that the observed data have been truncated with respect to some unobserved covariable. In this paper, a hidden truncation formulation of this scenario is proposed and analyzed. A bivariate Pareto (II) distribution is assumed for the variable of interest and the unobserved covariable. Distributional properties of the resulting model are investigated. A variety of parameter estimation strategies (under the classical set up) are investigated.     

A Sieve model for extreme values

From the class of extreme value distributions, we focus on the set of heavy-tailed distributions which produce low-frequency, high-cost events. The regular Pareto distribution is the basic model of choice, being the simplest heavy-tailed distribution. Real data suggest that modifications of the Pareto distribution may be a better fit; an alternative model is the truncated Pareto distribution (TPD). For further study, this paper proposed a TPD Sieve class of distributions. The properties and estimation on the Sieve class are also discussed. We fit the models to the largest Black Sea bass caught in Buzzard's Bay, MA, USA and the costliest Atlantic hurricanes from 1900 to 2005. Using measures of model adequacy, the TPD Sieve model is generally found to be the best-fitting model.     

Discrete Burr and discrete Pareto distributions

In this paper we obtain discrete Burr and Pareto distributions using the general approach of discretizing a continuous distribution and propose them as suitable lifetime models. It may be worth exploring the possibility of developing discrete versions of the Burr and Pareto distributions, so that, the same can be used for modeling discrete data. The equivalence of continuous and discrete Burr distributions has been established. Some important distributional properties and estimation of reliability characteristics are discussed. An application in reliability estimation in series system and a real data example on dentistry using this distribution is also discussed.     

A robust prediction error criterion for pareto modelling of upper tails

Estimation of the Pareto tail index from extreme order statistics is an important problem in many settings. The upper tail of the distribution, where data are sparse, is typically fitted with a model, such as the Pareto model, from which quantities such as probabilities associated with extreme events are deduced. The success of this procedure relies heavily not only on the choice of the estimator for the Pareto tail index but also on the procedure used to determine the number k of extreme order statistics that are used for the estimation. The authors develop a robust prediction error criterion for choosing k and estimating the Pareto index. A Monte Carlo study shows the good performance of the new estimator and the analysis of real data sets illustrates that a robust procedure for selection, and not just for estimation, is needed.     

Statistical regularity of firm size distribution: the Pareto IV and truncated Yule for Italian SCI manufacturing

In this paper we model the firm size distribution (FSD) of Italian manufacturing firms of SCI, the GDP survey of ISTAT, by a continuous and a discrete distribution: the Pareto IV distribution on total assets and the Yule distribution on Number of Employees. The Pareto IV distribution is characterized by four parameters and shows a better fit than both the Lognormal and Pareto I, which are the distributions more frequently applied to model firm size. The Pareto IV is inconsistent with Gibrat’s Law according to which the different segments of an Industry are characterized by proportionate growth and the distribution of size is Lognormal. A truncation of the Yule distribution has been necessary because the dataset is characterized by firms with at least 20 employees. The truncated Yule distribution shows a good fit for medium–large firms (firms with more than 50 employees). The partition of the dataset in innovative and non-innovative firms – both of which are well described by the Pareto IV – reveals a beneficial effect of scale on innovation. Finally, the good fit of both distributions holds not only for the composite industry, but for the single sectors too. The present work is part of a more general research project: “Industry evolution: innovation, profitability and firm’s growth”, conducted within the Department of Economic and Social Sciences of the Università Cattolica del Sacro Cuore (UCSC), Piacenza, coordinated by Professor Maurizio Baussola in cooperation with ISTAT (Italian Statistical Office, regional office for Lombardy). Part of this research was done when Lisa Crosato was a visiting research student at the LSE Statistics Department, during her Ph.D program in “Quantitative Models for Policy Analysis” at the UCSC of Piacenza.     

Estimation of parameters for the truncated exponential distribution

We consider the right truncated exponential distribution where the truncation point is unknown and show that the ML equation has a unique solution over an extended parameter space. In the case of the estimation of the truncation point T we show that the asymptotic distribution of the MLE is not centered at T. A modified MLE is introduced which outperforms all other considered estimators including the minimum variance unbiased estimator. Asymptotic as well as small sample properties of different estimators are investigated and compared. The truncated exponential distribution has an increasing failure rate, ideally suited for use as a survival distribution for biological and industrial data.     

Generalised Smooth Tests of Goodness of Fit Utilising L‐moments

In this paper we present a semiparametric test of goodness of fit which is based on the method of L‐moments for the estimation of the nuisance parameters. This test is particularly useful for any distribution that has a convenient expression for its quantile function. The test proceeds by investigating equality of the first few L‐moments of the true and the hypothesised distributions. We provide details and undertake simulation studies for the logistic and the generalised Pareto distributions. Although for some distributions the method of L‐moments estimator is less efficient than the maximum likelihood estimator, the former method has the advantage that it may be used in semiparametric settings and that it requires weaker existence conditions. The new test is often more powerful than competitor tests for goodness of fit of the logistic and generalised Pareto distributions.     

Estimation of the Lognormal-Pareto Distribution Using Probability Weighted Moments and Maximum Likelihood

This article deals with the estimation of the lognormal-Pareto and the lognormal-generalized Pareto distributions, for which a general result concerning asymptotic optimality of maximum likelihood estimation cannot be proved. We develop a method based on probability weighted moments, showing that it can be applied straightforwardly to the first distribution only. In the lognormal-generalized Pareto case, we propose a mixed approach combining maximum likelihood and probability weighted moments. Extensive simulations analyze the relative efficiencies of the methods in various setups. Finally, the techniques are applied to two real datasets in the actuarial and operational risk management fields.     

Goodness-of-fit tests for the pareto and lognormal distributions based on multiply truncated samples

This article considers the situation where, in the most general case, each observation in a sample has been “truncated” below at a different, but known value. Each observation is truncated in the sense that, had it been 1ess than the truncati on point, it would not have appeared in the sample. A goodness-of-fit test based on Gnedenko’ F statistic is developed to test the hypothesis that the underlying distribution is Pareto against the alternative of lognormality. The Chi-square and Kolmogorov tests are adapted to test the hypothesi s of lognormality with unspecified alternative. The application of these techni ques to the analysis of insurance claim data is discussed.     

Efficient estimation in the Pareto distribution

The maximum likelihood estimation (MLE) of the probability density function (pdf) and cumulative distribution function (CDF) are derived for the Pareto distribution. It has been shown that MLEs are more efficient than uniform minimum variance unbiased estimators of pdf and CDF.     

Interval estimation for the Pareto distribution based on the progressive Type II censored sample

For the complete sample and the right Type II censored sample, Chen [Joint confidence region for the parameters of Pareto distribution. Metrika 44 (1996), pp. 191–197] proposed the interval estimation of the parameter θ and the joint confidence region of the two parameters of Pareto distribution. This paper proposed two methods to construct the confidence region of the two parameters of the Pareto distribution for the progressive Type II censored sample. A simulation study comparing the performance of the two methods is done and concludes that Method 1 is superior to Method 2 by obtaining a smaller confidence area. The interval estimation of parameter ν is also given under progressive Type II censoring. In addition, the predictive intervals of the future observation and the ratio of the two future consecutive failure times based on the progressive Type II censored sample are also proposed. Finally, one example is given to illustrate all interval estimations in this paper.     

On sampling with probability proportional to size

One of the vehicles for utilization of auxiliary information is to use a sampling scheme with inclusion probabilities proportional to given size measures, a πps scheme. The paper addresses the following πps problem: Exhibit a πps scheme with prescribed sample size, which leads to good estimation precision and has good variance estimation properties.Rosén (1997) presented a novel general class of sampling schemes, called order sampling schemes, which here are shown to provide interesting contributions to the πps problem. A notion ‘order sampling with fixed distribution shape’ (OSFS) is introduced, and employed to construct a general class of πps schemes, called OSFSπps schemes. A particular scheme, Pareto πps, is shown to be optimal among OSFSπps schemes, in the sense that it minimizes estimator variances. Comparisons are made of three OSFSπps schemes and three other πps schemes; Sunter πps and systematic πps with frame ordered at random respectively by the sizes. The main conclusion is as follows. Pareto πps is superior among πps schemes which admit objective assessment of sampling errors.     

Statistical Inference for a New Class of Multivariate Pareto Distributions

Various solutions to the parameter estimation problem of a recently introduced multivariate Pareto distribution are developed and exemplified numerically. Namely, a density of the aforementioned multivariate Pareto distribution with respect to a dominating measure, rather than the corresponding Lebesgue measure, is specified and then employed to investigate the maximum likelihood estimation (MLE) approach. Also, in an attempt to fully enjoy the common shock origins of the multivariate model of interest, an adapted variant of the expectation-maximization (EM) algorithm is formulated and studied. The method of moments is discussed as a convenient way to obtain starting values for the numerical optimization procedures associated with the MLE and EM methods.     

On the hidden truncated bivariate Pareto (IV) model and associated inferential issues

Income and wealth data are typically modelled by some variant of the classical Pareto distribution. Often, in practice, the observed data are truncated with respect to some unobserved covariate. In this paper, a hidden truncation formulation of this scenario is proposed and analysed. For this purpose, a bivariate Pareto (IV) distribution is assumed for the variable of interest and the unobserved covariate. Some important distributional properties of the resulting model as well as associated inferential methods are studied. An example is used finally to illustrate the results developed here. In this case, it is noted that hidden truncation on the left does not result in any new model, but the hidden truncation on the right does. The properties and fit of such a model pose a challenging problem and that is what is focused here in this work.     

A matching prior for extreme quantile estimation of the generalized Pareto distribution

Extreme quantile estimation plays an important role in risk management and environmental statistics among other applications. A popular method is the peaks-over-threshold (POT) model that approximate the distribution of excesses over a high threshold through generalized Pareto distribution (GPD). Motivated by a practical financial risk management problem, we look for an appropriate prior choice for Bayesian estimation of the GPD parameters that results in better quantile estimation. Specifically, we propose a noninformative matching prior for the parameters of a GPD so that a specific quantile of the Bayesian predictive distribution matches the true quantile in the sense of Datta et al. (2000).     

Statistical Analysis of the Hirsch Index

Abstract. The Hirsch index (commonly referred to as h‐index) is a bibliometric indicator which is widely recognized as effective for measuring the scientific production of a scholar since it summarizes size and impact of the research output. In a formal setting, the h‐index is actually an empirical functional of the distribution of the citation counts received by the scholar. Under this approach, the asymptotic theory for the empirical h‐index has been recently exploited when the citation counts follow a continuous distribution and, in particular, variance estimation has been considered for the Pareto‐type and the Weibull‐type distribution families. However, in bibliometric applications, citation counts display a distribution supported by the integers. Thus, we provide general properties for the empirical h‐index under the small‐ and large‐sample settings. In addition, we also introduce consistent non‐parametric variance estimation, which allows for the implementation of large‐sample set estimation for the theoretical h‐index.     

Extreme value index estimator using maximum likelihood and moment estimation

ABSTRACT

When a distribution function is in the max domain of attraction of an extreme value distribution, its tail can be well approximated by a generalized Pareto distribution. Based on this fact we use a moment estimation idea to propose an adapted maximum likelihood estimator for the extreme value index, which can be understood as a combination of the maximum likelihood estimation and moment estimation. Under certain regularity conditions, we derive the asymptotic normality of the new estimator and investigate its finite sample behavior by comparing with several classical or competitive estimators. A simulation study shows that the new estimator is competitive with other estimators in view of average bias, average MSE, and coefficient of variance of the new device for the optimal selection of the threshold.     

The E-Bayesian and hierarchical Bayesian estimations of Pareto distribution parameter under different loss functions

In this paper, we study the E-Bayesian and hierarchical Bayesian estimations of the parameter derived from Pareto distribution under different loss functions. The definition of the E-Bayesian estimation of the parameter is provided. Moreover, for Pareto distribution, under the condition of the scale parameter is known, based on the different loss functions, formulas of the E-Bayesian estimation and hierarchical Bayesian estimations for the shape parameter are given, respectively, properties of the E-Bayesian estimation – (i) the relationship between of E-Bayesian estimations under different loss functions are provided, (ii) the relationship between of E-Bayesian and hierarchical Bayesian estimations under the same loss function are also provided, and using the Monte Carlo method simulation example is given. Finally, combined with the golfers income data practical problem are calculated, the results show that the proposed method is feasible and convenient for application.