Statistical estimation of quadratic Rényi entropy for a stationary m-dependent sequence

The Rényi entropy is a generalisation of the Shannon entropy and is widely used in mathematical statistics and applied sciences for quantifying the uncertainty in a probability distribution. We consider estimation of the quadratic Rényi entropy and related functionals for the marginal distribution of a stationary m-dependent sequence. The U-statistic estimators under study are based on the number of ε-close vector observations in the corresponding sample. A variety of asymptotic properties for these estimators are obtained (e.g. consistency, asymptotic normality, and Poisson convergence). The results can be used in diverse statistical and computer science problems whenever the conventional independence assumption is too strong (e.g. ε-keys in time series databases and distribution identification problems for dependent samples).     

Parameter Estimation in Conditional Heteroscedastic Models

Abstract

We study asymptotics of parameter estimates in conditional heteroscedastic models. The estimators considered are those obtained by minimizing certain functionals and those obtained by solving estimation equations. We establish consistency and derive asymptotic limit laws of the estimators. Condition under which the limit law is normal is studied. Further, bootstrap for these estimators is discussed. The limiting distribution of the estimators is not necessary always normal, and we present a real data example to illustrate this.     

The beta Weibull-geometric distribution

A new five-parameter distribution called the beta Weibull-geometric (BWG) distribution is proposed. The new distribution is generated from the logit of a beta random variable and includes the Weibull-geometric distribution of Barreto-Souza et al. [The Weibull-geometric distribution, J. Stat. Comput. Simul. 81 (2011), pp. 645–657], beta Weibull (BW), beta exponential, exponentiated Weibull, and some other lifetime distributions as special cases. A comprehensive mathematical treatment of this distribution is provided. The density function can be expressed as an infinite mixture of BW densities and then we derive some mathematical properties of the new distribution from the corresponding properties of the BW distribution. The density function of the order statistics and also estimation of the stress–strength parameter are obtained using two general expressions. To estimate the model parameters, we use the maximum likelihood method and the asymptotic distribution of the estimators is also discussed. The capacity of the new distribution are examined by various tools, using two real data sets.     

NONPARAMETRIC TWO SAMPLE TESTS BASED ON AREA STATISTICS

ABSTRACT

Area statistics are sample versions of areas occurring in a probability plot of two distribution functions F and G. This paper presents a unified basis for five statistics of this type. They can be used for various testing problems in the framework of the two sample problem for independent observations, such as testing equality of distributions against inequality or testing stochastic dominance of distributions in one or either direction against nondominance. Though three of the statistics considered have already been suggested in literature, two of them are new and deserve our interest. The finite sample distributions of the statistics (under F=G) can be calculated via recursion formulae. Two tables with critical values of the new statistics are included. The asymptotic distribution of the properly normalized versions of the area statistics are functionals of the Brownian bridge. The distribution functions and quantiles thereof are obtained by Monte Carlo simulation. Finally, the power functions of the two new tests based on area statistics are compared to the power functions of the tests based on the corresponding supremum statistics, i.e., statistics of the Kolmogorov–Smirnov type.     

Expansion and estimation of Lévy process functionals in nonlinear and nonstationary time series regression

In this article, we develop a series estimation method for unknown time-inhomogeneous functionals of Lévy processes involved in econometric time series models. To obtain an asymptotic distribution for the proposed estimators, we establish a general asymptotic theory for partial sums of bivariate functionals of time and nonstationary variables. These results show that the proposed estimators in different situations converge to quite different random variables. In addition, the rates of convergence depend on various factors rather than just the sample size. Finite sample simulations are provided to evaluate the finite sample performance of the proposed model and estimation method.     

Polynomial Histograms for Multivariate Density and Mode Estimation

Abstract. We consider the problem of efficiently estimating multivariate densities and their modes for moderate dimensions and an abundance of data. We propose polynomial histograms to solve this estimation problem. We present first‐ and second‐order polynomial histogram estimators for a general d‐dimensional setting. Our theoretical results include pointwise bias and variance of these estimators, their asymptotic mean integrated square error (AMISE), and optimal binwidth. The asymptotic performance of the first‐order estimator matches that of the kernel density estimator, while the second order has the faster rate of O(n^?6/(d+6)). For a bivariate normal setting, we present explicit expressions for the AMISE constants which show the much larger binwidths of the second order estimator and hence also more efficient computations of multivariate densities. We apply polynomial histogram estimators to real data from biotechnology and find the number and location of modes in such data.     

The Finite Sample Performance of Inference Methods for Propensity Score Matching and Weighting Estimators

ABSTRACT

This article investigates the finite sample properties of a range of inference methods for propensity score-based matching and weighting estimators frequently applied to evaluate the average treatment effect on the treated. We analyze both asymptotic approximations and bootstrap methods for computing variances and confidence intervals in our simulation designs, which are based on German register data and U.S. survey data. We vary the design w.r.t. treatment selectivity, effect heterogeneity, share of treated, and sample size. The results suggest that in general, theoretically justified bootstrap procedures (i.e., wild bootstrapping for pair matching and standard bootstrapping for “smoother” treatment effect estimators) dominate the asymptotic approximations in terms of coverage rates for both matching and weighting estimators. Most findings are robust across simulation designs and estimators.     

Minimum local distance density estimation

We present a local density estimator based on first-order statistics. To estimate the density at a point, x, the original sample is divided into subsets and the average minimum sample distance to x over all such subsets is used to define the density estimate at x. The tuning parameter is thus the number of subsets instead of the typical bandwidth of kernel or histogram-based density estimators. The proposed method is similar to nearest-neighbor density estimators but it provides smoother estimates. We derive the asymptotic distribution of this minimum sample distance statistic to study globally optimal values for the number and size of the subsets. Simulations are used to illustrate and compare the convergence properties of the estimator. The results show that the method provides good estimates of a wide variety of densities without changes of the tuning parameter, and that it offers competitive convergence performance.     

On tests of symmetry,marginal homogeneity and quasi-symmetry in two-way contingency tables based on minimum φ-divergence estimator with constraints

The restricted minimum φ-divergence estimator, [Pardo, J.A., Pardo, L. and Zografos, K., 2002, Minimum φ-divergence estimators with constraints in multinomial populations. Journal of Statistical Planning and Inference, 104, 221–237], is employed to obtain estimates of the cell frequencies of an I×I contingency table under hypotheses of symmetry, marginal homogeneity or quasi-symmetry. The associated φ-divergence statistics are distributed asymptotically as chi-squared distributions under the null hypothesis. The new estimators and test statistics contain, as particular cases, the classical estimators and test statistics previously presented in the literature for the cited problems. A simulation study is presented, for the symmetry problem, to choose the best function φ₂ for estimation and the best function φ₁ for testing.     

Estimators of location based on Kolmogorov-Smirnov-type statistics

Two classes of estimators of a location parameter ø₀ are proposed, based on a nonnegative functional H₁^* of the pair (D₁^ø_N, G^ø_N), where and where F_N is the sample distribution function. The estimators of the first class are defined as a value of ø minimizing H₁^*; the estimators of the second class are linearized versions of those of the first. The asymptotic distribution of the estimators is derived, and it is shown that the Kolmogorov-Smirnov statistic, the signed linear rank statistics, and the Cramérvon Mises statistics are special cases of such functionals H₁^*;. These estimators are closely related to the estimators of a shift in the two-sample case, proposed and studied by Boulanger in B2 (pp. 271–284).     

Non-parametric estimation of copula based mutual information

Abstract

Mutual information is a measure for investigating the dependence between two random variables. The copula based estimation of mutual information reduces the complexity because it is depend only on the copula density. We propose two estimators and discuss the asymptotic properties. To compare the performance of the estimators a simulation study is carried out. The methods are illustrated using real data sets.     

Bernstein-von Mises theorem and Bayes estimation from single server queues

Abstract

In this paper, we prove the Bernstein-von Mises theorem for the GI∕G∕1 queueing system which is observed over a continuous time interval (0, T], where T is a suitable stopping time. And also the asymptotic properties of Bayes estimators of the parameters are investigated.     

Uniform-in-bandwidth kernel estimation for censored data

We present a sharp uniform-in-bandwidth functional limit law for the increments of the Kaplan–Meier empirical process based upon right-censored random data. We apply this result to obtain limit laws for nonparametric kernel estimators of local functionals of lifetime densities, which are uniform with respect to the choices of bandwidth and kernel. These are established in the framework of convergence in probability, and we allow the bandwidth to vary within the complete range for which the estimators are consistent. We provide explicit values for the asymptotic limiting constant for the sup-norm of the estimation random error.     

Statistical Inference for Damaged Poisson Distribution

Abstract

For non-negative integer-valued random variables, the concept of “damaged” observations was introduced, for the first time, by Rao and Rubin [Rao, C. R., Rubin, H. (1964). On a characterization of the Poisson distribution. Sankhya 26:295–298] in 1964 on a paper concerning the characterization of Poisson distribution. In 1965, Rao [Rao, C. R. (1965). On discrete distribution arising out of methods of ascertainment. Sankhya Ser. A. 27:311–324] discusses some results related with inferences for parameters of a Poisson Model when it has occurred partial destruction of observations. A random variable is said to be damaged if it is unobservable, due to a damage mechanism which randomly reduces its magnitude. In subsequent years, considerable attention has been given to characterizations of distributions of such random variables that satisfy the “Rao–Rubin” condition. This article presents some inference aspects of a damaged Poisson distribution, under reasonable assumption that, when an observation on the random variable is made, it is also possible to determine whether or not some damage has occurred. In other words, we do not know how many items are damaged, but we can identify the existence of damage. Particularly it is illustrated the situation in which it is possible to identify the occurrence of some damage although it is not possible to determine the amount of items damaged. Maximum likelihood estimators of the underlying parameters and their asymptotic covariance matrix are obtained. Convergence of the estimates of parameters to the asymptotic values are studied through Monte Carlo simulations.     

Estimators of shift based on statistics of the Kolmogorov-Smirnov type

This paper is concerned with the estimation of a shift parameter δ_o, based on some nonnegative functional Hg₁ of the pair (D^δ_N(x), f?^δ_N(x)), where D^δ_N(x) = K_N/b {F_2,n(x)—F_1,m (x + δ)}, +^δ_N(x) = {mF_1,m (x + δ) + nF_2,n(x)}/N, where F_1,m and F_2,n are the empirical distribution functions of two independent random samples (N = m + n), and where K²_N = mn/N. First an estimator δ_N, is defined as a value of δ minimizing a functional H of the type of H₁. A second estimator δ¹_N is also defined which is a linearized version of the first. Finite and asymptotic properties of these estimators are considered. It is also shown that most well-known test statistics of the Kolmogorov-Smirnov type are particular cases of such functionals H₁. The asymptotic distribution and the asymptotic efficiency of some estimators are given.     

On approximations to the inverse student’s-t distribution function

The inverse of the Student's t-distribution is often needed in computer simulation and applied statistics, e.g, in generating random variates from t-distributions and in computing tables needed for statistical procedures which do not assume known variances. The t-distribution algorithm of Dudewicz and Dalal (1972) can be used to approximate the inverset t distribution function. The author notes an algorithm for evaluation of this inverse d.f. which can be implemented in a fast, accurate and short computer program. The error analysis is also reported. An application is considered for the problem of testing the hypothesis that a sequence of random variates follows Student's-t distribution.     

Asymptotic behaviour of binned kernel density estimators for locally non-stationary random fields

We investigate the asymptotic behaviour of binned kernel density estimators for dependent and locally non-stationary random fields converging to stationary random fields. We focus on the study of the bias and the asymptotic normality of the estimators. A simulation experiment conducted shows that both the kernel density estimator and the binned kernel density estimator have the same behavior and both estimate accurately the true density when the number of fields increases. We apply our results to the 2002 incidence rates of tuberculosis in the departments of France.     

Efficiency and Validity Analyses of Two-Stage Estimation Procedures and Derived Testing Procedures in Quantitative Linear Models with AR(1) Errors

Abstract

In a quantitative linear model with errors following a stationary Gaussian, first-order autoregressive or AR(1) process, Generalized Least Squares (GLS) on raw data and Ordinary Least Squares (OLS) on prewhitened data are efficient methods of estimation of the slope parameters when the autocorrelation parameter of the error AR(1) process, ρ, is known. In practice, ρ is generally unknown. In the so-called two-stage estimation procedures, ρ is then estimated first before using the estimate of ρ to transform the data and estimate the slope parameters by OLS on the transformed data. Different estimators of ρ have been considered in previous studies. In this article, we study nine two-stage estimation procedures for their efficiency in estimating the slope parameters. Six of them (i.e., three noniterative, three iterative) are based on three estimators of ρ that have been considered previously. Two more (i.e., one noniterative, one iterative) are based on a new estimator of ρ that we propose: it is provided by the sample autocorrelation coefficient of the OLS residuals at lag 1, denoted r(1). Lastly, REstricted Maximum Likelihood (REML) represents a different type of two-stage estimation procedure whose efficiency has not been compared to the others yet. We also study the validity of the testing procedures derived from GLS and the nine two-stage estimation procedures. Efficiency and validity are analyzed in a Monte Carlo study. Three types of explanatory variable x in a simple quantitative linear model with AR(1) errors are considered in the time domain: Case 1, x is fixed; Case 2, x is purely random; and Case 3, x follows an AR(1) process with the same autocorrelation parameter value as the error AR(1) process. In a preliminary step, the number of inadmissible estimates and the efficiency of the different estimators of ρ are compared empirically, whereas their approximate expected value in finite samples and their asymptotic variance are derived theoretically. Thereafter, the efficiency of the estimation procedures and the validity of the derived testing procedures are discussed in terms of the sample size and the magnitude and sign of ρ. The noniterative two-stage estimation procedure based on the new estimator of ρ is shown to be more efficient for moderate values of ρ at small sample sizes. With the exception of small sample sizes, REML and its derived F-test perform the best overall. The asymptotic equivalence of two-stage estimation procedures, besides REML, is observed empirically. Differences related to the nature, fixed or random (uncorrelated or autocorrelated), of the explanatory variable are also discussed.     

Bias reduction in kernel density estimation

In this paper, we propose two kernel density estimators based on a bias reduction technique. We study the properties of these estimators and compare them with Parzen–Rosenblatt's density estimator and Mokkadem, A., Pelletier, M., and Slaoui, Y. (2009, ‘The stochastic approximation method for the estimation of a multivariate probability density’, J. Statist. Plann. Inference, 139, 2459–2478) is density estimators. It turns out that, with an adequate choice of the parameters of the two proposed estimators, the rate of convergence of two estimators will be faster than the two classical estimators and the asymptotic MISE (Mean Integrated Squared Error) will be smaller than the two classical estimators. We corroborate these theoretical results through simulations.     

PORT Hill and Moment Estimators for Heavy-Tailed Models

In this article, we use the peaks over random threshold (PORT)-methodology, and consider Hill and moment PORT-classes of extreme value index estimators. These classes of estimators are invariant not only to changes in scale, like the classical Hill and moment estimators, but also to changes in location. They are based on the sample of excesses over a random threshold, the order statistic X _[np]+1:n , 0 ≤ p < 1, being p a tuning parameter, which makes them highly flexible. Under convenient restrictions on the underlying model, these classes of estimators are consistent and asymptotically normal for adequate values of k, the number of top order statistics used in the semi-parametric estimation of the extreme value index γ. In practice, there may however appear a stability around a value distant from the target γ when the minimum is chosen for the random threshold, and attention is drawn for the danger of transforming the original data through the subtraction of the minimum. A new bias-corrected moment estimator is also introduced. The exact performance of the new extreme value index PORT-estimators is compared, through a large-scale Monte-Carlo simulation study, with the original Hill and moment estimators, the bias-corrected moment estimator, and one of the minimum-variance reduced-bias (MVRB) extreme value index estimators recently introduced in the literature. As an empirical example we estimate the tail index associated to a set of real data from the field of finance.