On detecting outliers in the Pareto distribution

In this paper, we introduce two new statistics for detecting outliers in the Pareto distribution. These new statistics are the extension of the statistics for detecting outliers in exponential and gamma distributions. In fact, we compare the power of our test statistics with the other statistics and select the best test statistic for detecting outliers in the Pareto distribution. Finally, numerical examples of different insurance claims are used to see the performance of the test.     

On the identification of extreme outliers and dragon-kings mechanisms in the upper tail of income distribution

The presence of extreme outliers in the upper tail data of income distribution affects the Pareto tail modeling. A simulation study is carried out to compare the performance of three types of boxplot in the detection of extreme outliers for Pareto data, including standard boxplot, adjusted boxplot and generalized boxplot. It is found that the generalized boxplot is the best method for determining extreme outliers for Pareto distributed data. For the application, the generalized boxplot is utilized for determining the exreme outliers in the upper tail of Malaysian income distribution. In addition, for this data set, the confidence interval method is applied for examining the presence of dragon-kings, extreme outliers which are beyond the Pareto or power-laws distribution.     

Classical estimation of hazard rate and mean residual life functions of Pareto distribution

Abstract

In this paper we find the maximum likelihood estimates (MLEs) of hazard rate and mean residual life functions (MRLF) of Pareto distribution, their asymptotic non degenerate distribution, exact distribution and moments. We also discuss the uniformly minimum variance unbiased estimate (UMVUE) of hazard rate function and MRLF. Finally, two numerical examples with simulated data and real data set, are presented to illustrate the proposed estimates.     

Mean-shift outliers model in skew scale-mixtures of normal distributions

ABSTRACT

Asymmetric models have been discussed quite extensively in recent years, in situations where the normality assumption is suspected due to lack of symmetry in the data. Techniques for assessing the quality of fit and diagnostic analysis are important for model validation. This paper presents a study of the mean-shift method for the detection of outliers in regression models under skew scale-mixtures of normal distributions. Analytical solutions for the estimators of the parameters are obtained through the use of Expectation–Maximization algorithm. The observed information matrix for the calculation of standard errors is obtained for each distribution. Simulation studies and an application to the analysis of a data have been carried out, showing the efficiency of the proposed method in detecting outliers.     

Bayesian estimation for non zero inflated modified power series distribution under linex and generalized entropy loss functions

Abstract

In this paper, we derive Bayesian estimators of the parameters of modified power series distributions inflated at any of a support point under linex and general entropy loss function. We assume that the prior information can be summarized by a uniform, Beta, two-sided power, Gamma or generalized Pareto distributions. The obtained results are demonstrated on the generalized Poisson and the generalized negative binomial distribution inflated at a given point.     

A note on determining the number of outliers in an exponential sample by least squares procedure

In this paper, we suggest a least squares procedure for the determination of the number of upper outliers in an exponential sample by minimizing sample mean squared error. Moreover, the method can reduce the masking or “swamping” effects. In addition, we have also found that the least squares procedure is easy and simple to compute than test test procedure T _k suggested by Zhang (1998) for determining the number of upper outliers, since Zhang (1998) need to use the complicated null distribution of T _k . Moreover, we give three practical examples and a simulated example to illustrate the procedures. Further, simulation studies are given to show the advantages of the proposed method. Finally, the proposed least squares procedure can also determine the number of upper outliers in other continuous univariate distributions (for example, Pareto, Gumbel, Weibull, etc.). Received: May 10, 1999; revised version: June 5, 2000     

Inferences Based on a Skipped Correlation Coefficient

The most popular method for trying to detect an association between two random variables is to test H ₀ ?:?ρ=0, the hypothesis that Pearson's correlation is equal to zero. It is well known, however, that Pearson's correlation is not robust, roughly meaning that small changes in any distribution, including any bivariate normal distribution as a special case, can alter its value. Moreover, the usual estimate of ρ, r, is sensitive to only a few outliers which can mask a true association. A simple alternative to testing H ₀ ?:?ρ =0 is to switch to a measure of association that guards against outliers among the marginal distributions such as Kendall's tau, Spearman's rho, a Winsorized correlation, or a so-called percentage bend correlation. But it is known that these methods fail to take into account the overall structure of the data. Many measures of association that do take into account the overall structure of the data have been proposed, but it seems that nothing is known about how they might be used to detect dependence. One such measure of association is selected, which is designed so that under bivariate normality, its estimator gives a reasonably accurate estimate of ρ. Then methods for testing the hypothesis of a zero correlation are studied.     

Inference based on progressively censored sample from Pareto population

Abstract

In this paper, we assume that the lifetimes have a two-parameter Pareto distribution and discuss some results of progressive Type-II censored sample. We obtain maximum likelihood estimators and Bayes estimators of the unknown parameters under squared error loss and a precautionary loss functions in progressively Type-II censored sample. Robust Bayes estimation of unknown parameters over three different classes of priors under progressively Type-II censored sample, squared error loss, and precautionary loss functions are obtained. We discuss estimation of unknown parameters on competing risks progressive Type-II censoring. Finally, we consider the problem of estimating the common scale parameter of two Pareto distributions when samples are progressively Type-II censored.     

Irreconcilability of P-value and Bayesian measure in two-sided hypothesis: Pareto distribution with the presence of nuisance parameter

Abstract

This article is concerned with the comparison of Bayesian and classical testing of a point null hypothesis for the Pareto distribution when there is a nuisance parameter. In the first stage, using a fixed prior distribution, the posterior probability is obtained and compared with the P-value. In the second case, lower bounds of the posterior probability of H₀, under a reasonable class of prior distributions, are compared with the P-value. It has been shown that even in the presence of nuisance parameters for the model, these two approaches can lead to different results in statistical inference.     

Influence diagnostics in the capital asset pricing model under elliptical distributions

In this paper we consider the Capital Asset Pricing Model under Elliptical (symmetric) Distributions. This class of distributions, which contains the normal distribution, t, contaminated normal and power exponential, among others, offers a more flexible framework for modelling asset prices or returns. In order to analyze the sensibility to possible outliers and/or atypical returns of the maximum likelihood estimators, the local influence method was implemented. The results are illustrated by using a set of shares from companies who trade in the Chilean Stock Market. Our main conclusion is that symmetric distributions having heavier tails than those of the normal distribution, especially the t distribution with small degrees of freedom, show a better fit and allow the reduction of the influence of atypical returns in the maximum likelihood estimators.     

Information approach for a lifetime change-point model based on the exponential-logarithmic distribution

ABSTRACT

The exponential-logarithmic distribution is a distribution which has a decreasing failure function and various applications such as in biological and engineering fields. In this paper, we study a change-point problem of this distribution. A procedure based on Schwarz information criterion is proposed to detect changes in parameters of this distribution. Simulations are conducted to indicate the performance of the proposed procedure under different scenarios. Applications on two real data are provided to illustrate the detection procedure.     

A new hybrid estimation method for the generalized pareto distribution

ABSTRACT

The generalized Pareto distribution (GPD) is important in the analysis of extreme values, especially in modeling exceedances over thresholds. Most of the existing methods for estimating the scale and shape parameters of the GPD suffer from theoretical and/or computational problems. A new hybrid estimation method is proposed in this article, which minimizes a goodness-of-fit measure and incorporates some useful likelihood information. Compared with the maximum likelihood method and other leading methods, our new hybrid estimation method retains high efficiency, reduces the estimation bias, and is computation friendly.     

NEYMAN SMOOTH TESTS FOR THE GENERALIZED PARETO DISTRIBUTION

ABSTRACT

The generalized Pareto distribution (GPD) is commonly used as extreme values's distribution. We present goodness of fit tests for the GPD based on Neyman's smooth tests statistics. The methods of maximum likelihood, moments and probability-weighted moments are used for estimating the GPD's parameters. Simulations are done to study the power of these tests.     

A New Weibull–Pareto Distribution: Properties and Applications

Many distributions have been used as lifetime models. In this article, we propose a new three-parameter Weibull–Pareto distribution, which can produce the most important hazard rate shapes, namely, constant, increasing, decreasing, bathtub, and upsidedown bathtub. Various structural properties of the new distribution are derived including explicit expressions for the moments and incomplete moments, Bonferroni and Lorenz curves, mean deviations, mean residual life, mean waiting time, and generating and quantile functions. The Rényi and q entropies are also derived. We obtain the density function of the order statistics and their moments. The model parameters are estimated by maximum likelihood and the observed information matrix is determined. The usefulness of the new model is illustrated by means of two real datasets on Wheaton river flood and bladder cancer. In the two applications, the new model provides better fits than the Kumaraswamy–Pareto, beta-exponentiated Pareto, beta-Pareto, exponentiated Pareto, and Pareto models.     

Robust variable selection and parametric component identification in varying coefficient models

ABSTRACT

In this paper, we study a novelly robust variable selection and parametric component identification simultaneously in varying coefficient models. The proposed estimator is based on spline approximation and two smoothly clipped absolute deviation (SCAD) penalties through rank regression, which is robust with respect to heavy-tailed errors or outliers in the response. Furthermore, when the tuning parameter is chosen by modified BIC criterion, we show that the proposed procedure is consistent both in variable selection and the separation of varying and constant coefficients. In addition, the estimators of varying coefficients possess the optimal convergence rate under some assumptions, and the estimators of constant coefficients have the same asymptotic distribution as their counterparts obtained when the true model is known. Simulation studies and a real data example are undertaken to assess the finite sample performance of the proposed variable selection procedure.     

Bayesian estimation and prediction based on generalized Type-II hybrid censored sample

ABSTRACT

In this paper, we consider a general form for the underlying distribution and a general conjugate prior, and develop a general procedure for deriving the maximum likelihood and Bayesian estimators based on an observed generalized Type-II hybrid censored sample. The problems of predicting the future order statistics from the same sample and that from a future sample are also discussed from a Bayesian viewpoint. For the illustration of the developed results, the exponential and Pareto distributions are used as examples. Finally, two numerical examples are presented for illustrating all the inferential procedures developed here.     

Robust multivariate diagnostics for PLSR and application on high dimensional spectrally overlapped drug systems

ABSTRACT

Statistical methods are effectively used in the evaluation of pharmaceutical formulations instead of laborious liquid chromatography. However, signal overlapping, nonlinearity, multicollinearity and presence of outliers deteriorate the performance of statistical methods. The Partial Least Squares Regression (PLSR) is a very popular method in the quantification of high dimensional spectrally overlapped drug formulations. The SIMPLS is the mostly used PLSR algorithm, but it is highly sensitive to outliers that also effect the diagnostics. In this paper, we propose new robust multivariate diagnostics to identify outliers, influential observations and points causing non-normality for a PLSR model. We study performances of the proposed diagnostics on two everyday use highly overlapping drug systems: Paracetamol–Caffeine and Doxylamine Succinate–Pyridoxine Hydrochloride.     

Robust bootstrap densities for dynamic conditional correlations: implications for portfolio selection and Value-at-Risk

ABSTRACT

Many financial decisions such as portfolio allocation, risk management, option pricing and hedge strategies are based on the forecast of the conditional variances, covariances and correlations of financial returns. Although the decisions depend on the forecasts covariance matrix little is known about effects of outliers on the uncertainty associated with these forecasts. In this paper we analyse these effects on the context of dynamic conditional correlation models when the uncertainty is measured using bootstrap methods. We also propose a bootstrap procedure to obtain forecast densities for return, volatilities, conditional correlation and Value-at-Risk that is robust to outliers. The results are illustrated with simulated and real data.     

A Marshall–Olkin family of heavy-tailed distributions which includes the lognormal one

Abstract

A new class of heavy-tailed distribution functions,, containing the lognormal distribution as a particular case is introduced. The class thus obtained depends on a set of three parameters, incorporating an additional distribution to the classical lognormal one. This new class of heavy-tailed distribution is presented as an alternative to other useful heavy-tailed distributions, such as the lognormal, Weibull, and Pareto distributions. The density and distribution functions of this new class are given by a closed expression which allows us to easily compute probabilities, quantiles, moments, and related measurements. Finally, some applications are shown as examples.     

Detection of outliers in mixed regressive-spatial autoregressive models

ABSTRACT

This article studies the outlier detection problem in mixed regressive-spatial autoregressive model. The formulae for testing outliers and their approximate distributions are derived under the mean-shift model and the variance-weight model, respectively. The simulation studies are conducted for examining the power and size of the test, as well as for the detection of outliers when a simulated data contains several outliers. A real data is analyzed to illustrate the proposed method, and modified models based on mean-shift and variance-weight models in which detected outliers are taken into account are suggested to deal with the outliers and confirm theconclusions.