Statistical estimation of quadratic Rényi entropy for a stationary m-dependent sequence

The Rényi entropy is a generalisation of the Shannon entropy and is widely used in mathematical statistics and applied sciences for quantifying the uncertainty in a probability distribution. We consider estimation of the quadratic Rényi entropy and related functionals for the marginal distribution of a stationary m-dependent sequence. The U-statistic estimators under study are based on the number of ε-close vector observations in the corresponding sample. A variety of asymptotic properties for these estimators are obtained (e.g. consistency, asymptotic normality, and Poisson convergence). The results can be used in diverse statistical and computer science problems whenever the conventional independence assumption is too strong (e.g. ε-keys in time series databases and distribution identification problems for dependent samples).     

商业银行操作风险的度量与资本需求模型

目前,商业银行操作风险的度量大都是在操作风险损失数据的分布假定下、根据VaR风险度量方法给出资本需求（风险准备金）,这一理论方法的基础是假定分布。然而商业银行操作风险的准备金往往又是一个基本确定的数值或需求区间,这就给风险准备金提出了比较严格的要求,否则将为商业银行操作带来一定的风险隐患。故根据分区多目标风险方法度量操作风险,并在此基础上根据信息熵的理论给出最优的资本需求（风险准备金）及其模型,其方法的优点是灵活简单,但要求初始密度函数的极值分布收敛于耿贝尔类型。为此给出实证分析,以说明两者之间的关系,这一理论方法可以为监管部门的管理提供一定程度的参考。     

Likelihood ratio-based distribution-free sequential change-point detection

Most existing control charts are for monitoring location or scale parameters, rather than any change in process distribution such as shift in shape. Goodness-of-fit (GOF) test can detect any change in distribution. This paper develops a new distribution-free control chart by integrating a powerful two-sample nonparametric likelihood ratio GOF test into the effective change-point model. Our proposed chart is easy in computation, convenient to use, and very efficient in detecting any change in process distribution, including shifts in location, scale, and shape. It is also robust in detecting various magnitudes of shifts and especially powerful in monitoring any distributional change involving a decrease in scale.     

A comment on “testing the lucas critique: A review ”

We derive general distribution tests based on the method of maximum entropy (ME) density. The proposed tests are derived from maximizing the differential entropy subject to given moment constraints. By exploiting the equivalence between the ME and maximum likelihood (ML) estimates for the general exponential family, we can use the conventional likelihood ratio (LR), Wald, and Lagrange multiplier (LM) testing principles in the maximum entropy framework. In particular, we use the LM approach to derive tests for normality. Monte Carlo evidence suggests that the proposed tests are compatible with and sometimes outperform some commonly used normality tests. We show that the proposed tests can be extended to tests based on regression residuals and non-i.i.d. data in a straightforward manner. An empirical example on production function estimation is presented.     

Maximum entropy sampling and optimal Bayesian experimental design

When Shannon entropy is used as a criterion in the optimal design of experiments, advantage can be taken of the classical identity representing the joint entropy of parameters and observations as the sum of the marginal entropy of the observations and the preposterior conditional entropy of the parameters. Following previous work in which this idea was used in spatial sampling, the method is applied to standard parameterized Bayesian optimal experimental design. Under suitable conditions, which include non-linear as well as linear regression models, it is shown in a few steps that maximizing the marginal entropy of the sample is equivalent to minimizing the preposterior entropy, the usual Bayesian criterion, thus avoiding the use of conditional distributions. It is shown using this marginal formulation that under normality assumptions every standard model which has a two-point prior distribution on the parameters gives an optimal design supported on a single point. Other results include a new asymptotic formula which applies as the error variance is large and bounds on support size.     

Asymptotic normality for plug-in estimators of diversity indices on countable alphabets

The plug-in estimator is one of the most popular approaches to the estimation of diversity indices. In this paper, we study its asymptotic distribution for a large class of diversity indices on countable alphabets. In particular, we give conditions for the plug-in estimator to be asymptotically normal, and in the case of uniform distributions, where asymptotic normality fails, we give conditions for the asymptotic distribution to be chi-squared. Our results cover some of the most commonly used indices, including Simpson's index, Reńyi's entropy and Shannon's entropy.     

Some new results on the cumulative residual entropy

The residual entropy function is a relevant dynamic measure of uncertainty in reliability and survival studies. Recently, Rao et al. [2004. Cumulative residual entropy: a new measure of information. IEEE Transactions on Information Theory 50, 1220–1228] and Asadi and Zohrevand [2007. On the dynamic cumulative residual entropy. Journal of Statistical Planning and Inference 137, 1931–1941] define the cumulative residual entropy and the dynamic cumulative residual entropy, respectively, as some new measures of uncertainty. They study some properties and applications of these measures showing how the cumulative residual entropy and the dynamic cumulative residual entropy are connected with the mean residual life function. In this paper, we obtain some new results on these functions. We also define and study the dynamic cumulative past entropy function. Some results are given connecting these measures of a lifetime distribution and that of the associated weighted distribution.     

Some computational aspects of the generalized von Mises distribution

This article deals with some important computational aspects of the generalized von Mises distribution in relation with parameter estimation, model selection and simulation. The generalized von Mises distribution provides a flexible model for circular data allowing for symmetry, asymmetry, unimodality and bimodality. For this model, we show the equivalence between the trigonometric method of moments and the maximum likelihood estimators, we give their asymptotic distribution, we provide bias-corrected estimators of the entropy, the Akaike information criterion and the measured entropy for model selection, and we implement the ratio-of-uniforms method of simulation.     

Rényi entropy properties of records

We provide bounds for Rényi entropy of records. We also show that the Rényi entropy ordering of random variables determines the Rényi entropy ordering of their respective records. We characterize exponential distribution by maximization of Rényi entropy under some conditions. We show that Rényi distance between distribution of records and parent distribution is distribution free.     

Residual Renyi entropy of k-record values

ABSTRACT

In this article, the residual Renyi entropy (RRE) of k-record values arising from an absolutely continuous distribution is considered. A representation of RRE of k-records arising from an arbitrary distribution in terms of RRE of k-record values arising from uniform distribution is given. Some properties for RRE of k-records are also discussed.     

Uniform-Geometric distribution

In this paper, a new discrete distribution called Uniform-Geometric distribution is proposed. Several distributional properties including survival function, moments, skewness, kurtosis, entropy and hazard rate function are discussed. Estimation of distribution parameter is studied by methods of moments, proportions and maximum likelihood. A simulation study is performed to compare the performance of the different estimates in terms of bias and mean square error. Two real data applications are also presented to see that new distribution is useful in modelling data.     

A complementary generalized linear failure rate-geometric distribution

In this article, we shall attempt to introduce a new class of lifetime distributions, which enfolds several known distributions such as the generalized linear failure rate distribution and covers both positive as well as negative skewed data. This new four-parameter distribution allows for flexible hazard rate behavior. Indeed, the hazard rate function here can be increasing, decreasing, bathtub-shaped, or upside-down bathtub-shaped. We shall first study some basic distributional properties of the new model such as the cumulative distribution function, the density of the order statistics, their moments, and Rényi entropy. Estimation of the stress-strength parameter as an important reliability property is also studied. The maximum likelihood estimation procedure for complete and censored data and Bayesian method are used for estimating the parameters involved. Finally, application of the new model to three real datasets is illustrated to show the flexibility and potential of the new model compared to rival models.     

An entropic structure in capability indices

Abstract

Analysis capability indices for symmetric process in normal case is obtained via maximum entropy approach of distribution function of the data. In view of it, we have perused on production processes to be in statistical control. Generally a process is capable based on capability indices when its reasonable index was more than a known threshold value. Thus by conditioning on indices, the most general distribution is found out whose parameters can be approximated by using the data of process. Also analysis via Kullback-Leibler information measure based on the above arguments is obtained in the last part of the paper.     

On the Choice of Nonparametric Entropy Estimator in Entropy-Based Goodness-of-Fit Test Statistics

Entropy-based goodness-of-fit test statistics can be established by estimating the entropy difference or Kullback–Leibler information, and several entropy-based test statistics based on various entropy estimators have been proposed. In this article, we first give comments on some problems resulting from not satisfying the moment constraints. We then study the choice of the entropy estimator by noting the reason why a test based on a better entropy estimator does not necessarily provide better powers.     

Bootstrapping one way analysis of rao's quadratic entropy

We establish a bootstrap approximation for the one way analysis of quadratic entropy in this paper. The quadratic entropy was introduced by Rao in 1982. It generalizes the concepts of variance and diversity measures and is useful in statistical inference. Our results in this paper provide a computational method to get rid of the nuisance parameters in the asymptotic distribution of the analysis of quadratic entropy (ANOQE) statistic.     

Ecological Bias: Use of Maximum-Entropy Approximations

The focus of geographical studies in epidemiology has recently moved towards looking for effects of exposures based on data taken at local levels of aggregation (i.e. small areas). This paper investigates how regression coefficients measuring covariate effects at the point level are modified under aggregation. Changing the level of aggregation can lead to completely different conclusions about exposure–effect relationships, a phenomenon often referred to as ecological bias. With partial knowledge of the within‐area distribution of the exposure variable, the notion of maximum entropy can be used to approximate that part of the distribution that is unknown. From the approximation, an expression for the ecological bias is obtained; simulations and an example show that the maximum‐entropy approximation is often better than other commonly used approximations.     

Characterization based on generalized entropy of order statistics

ABSTRACT

In the present study, several characterizations of order statistics are obtained on the basis of the generalized entropy. Under some conditions, it is shown that the parent distribution can be uniquely determined by equality of generalized entropy of order statistics.     

基于最大熵原理的条件回收率建模分析

通常使用的数据拟合只是对回收率分布的一个描述,为有效对回收率进行研究,建模时将抵押担保和企业的信用等级这两个因素加以考虑,应用最大熵原理,对违约损失率估计出最佳的条件概率密度。利用该模型不仅能够估计出违约损失率的均值和方差,还可以得出违约损失率的分布密度;另外,该模型也具有更明确的经济学意义。返回检验表明,该模型的估计效果优于单因素模型、动态多元回归模型以及非参数核密度估计法。     

Improvement of goodness-of-fit test for normal distribution based on entropy and power comparison

Vasicek's entropy test for normality is based on sample entropy and a parametric entropy estimator. These estimators are known to have bias in small samples. The use of Vasicek's test could affect the capability of detecting non-normality to some extent. This paper presents an improved entropy test, which uses bias-corrected entropy estimators. A Monte Carlo simulation study is performed to compare the power of the proposed test under several alternative distributions with some other tests. The results report that as anticipated, the improved entropy test has consistently higher power than the ordinary entropy test in nearly all sample sizes and alternatives considered, and compares favorably with other tests.     

Entropy measures associated with j-divergences

This paper shows the different entropy measures which originate from the distance, in the sense of the J-divergences, between one distribution and the reference distribution.

Here are studied several conditions for the concavity and non-negativity of the proposed measure, and the reference distribution is characterized by the Dalton–Pielou condition.

At last, an interpretation of the and ψ-entropies is gotten, from the distance between one distribution and the equilibrium distribution.