Asymptotic distributions of weighted divergence between discrete distributions

A divergence measure between discrete probability distributions introduced by Csiszar (1967) generalizes the Kullback-Leibler information and several other information measures considered in the literature. We introduce a weighted divergence which generalizes the weighted Kullback-Leibler information considered by Taneja (1985). The weighted divergence between an empirical distribution and a fixed distribution and the weighted divergence between two independent empirical distributions are here investigated for large simple random samples, and the asymptotic distributions are shown to be either normal or equal to the distribution of a linear combination of independent X²-variables     

An emprical bayes approach to two-sample problems with censored data

Empirical Bayes methods are used in estimating the probability based on randomly right-censored samples. The estimator is shown to be asymptotically optimal. Thus, in a way, this uork is similar to the results of Hollander and Korwar (1976) who used a similar approach in estimating A in the case of non-censored data. We also give hero a shorter proof to their rate result. In addition, a. resting procedure is obtained to test the hypothesis against on the basis of censored data. It is shown that this procedure is asymptotically optimal with rate of convergence n . Tnis result is analogous to our earlier result for the uncensored case (1970) The empirical Hayes procedure has been illustrated by means of a practical example.     

Parameter Estimation for the Discrete Stable Family

The discrete stable family constitutes an interesting two-parameter model of distributions on the non-negative integers with a Paretian tail. The practical use of the discrete stable distribution is inhibited by the lack of an explicit expression for its probability function. Moreover, the distribution does not possess moments of any order. Therefore, the usual tools—such as the maximum-likelihood method or even the moment method—are not feasible for parameter estimation. However, the probability generating function of the discrete stable distribution is available in a simple form. Hence, we initially explore the application of some existing estimation procedures based on the empirical probability generating function. Subsequently, we propose a new estimation method by minimizing a suitable weighted L ²-distance between the empirical and the theoretical probability generating functions. In addition, we provide a goodness-of-fit statistic based on the same distance.     

Nonparametric estimator for mean residual life and vitality function

In this paper, we study the estimation of the vitality function(v(x)=E(X|X>x) and mean residual life function(e(x)=E(X-x|X>x) from a sample ofX using the empirical estimator and kernel estimator. Under suitable conditions of regularity, the asymptotic normality of the kernel estimator is obtained. Partially supported by Consejeria de Cultura y Ed. (C.A.R.M.), under Grant PIB 95/90.     

General characterization theorems via the mean absolute deviation

The Chernoff–Borovkov–Utev inequality resulted owing to earlier inequalities established by Chernoff (1981) and Borovkov and Utev (1983), respectively, giving bounds for the variance of functions of normal r.v.’s and leading to characterizations of normality. Subsequently, several analytic properties of variance bounds and other relevant results were established by others. Defining the mean absolute deviation (about a median) as E|X−med(X)| where med(X) is a median of the distribution of the random variable X, Freimer and Mudholkar (1991) gave a bound for the mean absolute deviation of a certain real-valued function of an absolutely continuous random variable (w.r.t. Lebesgue measure) and Korwar (1991) presented an analogue of this in the discrete case; these authors, also, characterized the Laplace and a mixture of two Waring distributions via the respective bounds.We extend these latter results theorems to the case where the distributions are not necessarily purely discrete or absolutely continuous, via the approach of Alharbi and Shanbhag (1996). The results in Freimer and Mudholkar (1991) and Korwar (1991) are now corollaries to our findings. Also, following Alharbi and Shanbhag (1996), we relate these results to the modified version of Cox’s representation for a survival function in terms of the hazard measure, given in Kotz and Shanbhag (1980). (The original version of the representation mentioned had appeared in Cox (1972).)     

On invariance and randomization in factorial designs with applications to D-optimal main effect designs of the symmetrical factorial

Under the setting of the columnwise orthogonal polynomial model in the context of the general factorial it is shown that (i) the determinant of the information matrix of a design relative to an admissible vector of effects is invariant under a permutation of levels; (ii) the unbiased estimation of a linear function of an admissible vector of effects can be obtained under equal probability randomization. These results extend the work on invariance and randomization carried out under the more restrictive assumption of the orthonormal polynomial model by Srivastava, Raktoe and Pesotan (1976) and Pesotan and Raktoe (1981). Moreover, the problem of the construction of D-optimal main effect designs in the symmetrical factorial is reduced to a study of a special class of (0,1)-matrices using the Helmat matrix model. Using this class of (0,1)-matrices and the determinant invariance result, some classes of D-optimal main effect designs of the s² and s³ factorial respectively are presented.     

EMPIRICAL LIKELIHOOD-BASED KERNEL DENSITY ESTIMATION

This paper considers the estimation of a probability density function when extra distributional information is available (e.g. the mean of the distribution is known or the variance is a known function of the mean). The standard kernel method cannot exploit such extra information systematically as it uses an equal probability weight n^-1 at each data point. The paper suggests using empirical likelihood to choose the probability weights under constraints formulated from the extra distributional information. An empirical likelihood-based kernel density estimator is given by replacing n^-1 by the empirical likelihood weights, and has these advantages: it makes systematic use of the extra information, it is able to reflect the extra characteristics of the density function, and its variance is smaller than that of the standard kernel density estimator.     

Weighted bivariate logarithmic series distributions

In this paper, we are interested in the weighted distributions of a bivariate three parameter logarithmic series distribution studied by Kocherlakota and Kocherlakota (1990). The weighted versions of the model are derived with weight W(x,y) = x^[r] y^[s]. Explicit expressions for the probability mass function and probability generating functions are derived in the case r = s = l. The marginal and conditional distributions are derived in the general case. The maximum likelihood estimation of the parameters, in both two parameter and three parameter cases, is studied. A procedure for computer generation of bivariate data from a discrete distribution is described. This enables us to present two examples, in order to illustrate the methods developed, for finding the maximum likelihood estimates.     

On the rate of convergence of recursive kernel estimates of probability densities

Recursive estimates f_n^r(x)of the rth derivative f^r(x)(r=0,1)of the univariate probability density f(x) for strictly stationary processes {X_j,} are considered. The asymptotic variance-covariance of f_n^r(x)is established for stationary triangular arrays of random variables satisfying various asymptotic independence-uncorrelatedness conditions.     

Slow convergence of the Gibbs sampler

We consider the Gibbs sampler as a tool for generating an absolutely continuous probability measure ≥ on R^d. When an appropriate irreducibility condition is satisfied, the Gibbs Markov chain (X_n;n ≥ 0) converges in total variation to its target distribution ≥. Sufficient conditions for geometric convergence have been given by various authors. Here we illustrate, by means of simple examples, how slow the convergence can be. In particular, we show that given a sequence of positive numbers decreasing to zero, say (b_n;n ≥ 1), one can construct an absolutely continuous probability measure ≥ on R^d which is such that the total variation distance between ≥ and the distribution of X_n, converges to 0 at a rate slower than that of the sequence (b_n;n ≥ 1). This can even be done in such a way that ≥ is the uniform distribution over a bounded connected open subset of R^d. Our results extend to hit-and-run samplers with direction distributions having supports with symmetric gaps.     

Empirical likelihood confidence bands for distribution functions with missing responses

Distribution function estimation plays a significant role of foundation in statistics since the population distribution is always involved in statistical inference and is usually unknown. In this paper, we consider the estimation of the distribution function of a response variable Y with missing responses in the regression problems. It is proved that the augmented inverse probability weighted estimator converges weakly to a zero mean Gaussian process. A augmented inverse probability weighted empirical log-likelihood function is also defined. It is shown that the empirical log-likelihood converges weakly to the square of a Gaussian process with mean zero and variance one. We apply these results to the construction of Gaussian process approximation based confidence bands and empirical likelihood based confidence bands of the distribution function of Y. A simulation is conducted to evaluate the confidence bands.     

Estimating population size with truncated sampling

Let X₁, X₂,…,X_n be independent, indentically distributed random variables with density f(x,θ) with respect to a σ-finite measure μ. Let R be a measurable set in the sample space X. The value of X is observable if X ? (X?R) and not otherwise. The number J of observable X’s is binomial, N, Q, Q = 1?P(X ? R). On the basis of J observations, it is desired to estimate N and θ. Estimators considered are conditional and unconditional maximum likelihood and modified maximum likelihood using a prior weight function to modify the likelihood before maximizing. Asymptotic expansions are developed for the [Ncirc]’s of the form [Ncirc] = N + α√N + β + o_p(1), where α and β are random variables. All estimators have the same α, which has mean 0, variance σ² (a function of θ) and is asymptotically normal. Hence all are asymptotically equivalent by the usual limit distributional theory. The β’s differ and Eβ can be considered an “asymptotic bias”. Formulas are developed to compare the asymptotic biases of the various estimators. For a scale parameter family of absolutely continuous distributions with X = (0,∞) and R = (T,∞), special formuli are developed and a best estimator is found.     

Convergence Rates of Empirical Bayes Estimation for the Parameter of the Uniform Distribution U(0, θ) Under Random Censorship

ABSTRACT

This article considers the empirical Bayes estimation problem in the uniform distribution U(0, θ) with censored data. For the parameter θ, using the empirical Bayes (EB) approach, we propose an EB estimation of θ which possesses a rate of convergence can be arbitrarily close to O(n ^?1/2) when the historical samples are randomly censored from the right, where n is the number of historical sample. A sample and some simulation results are also presented.     

Multiplier bootstrap methods for conditional distributions

The multiplier bootstrap is a fast and easy-to-implement alternative to the standard bootstrap; it has been used successfully in many statistical contexts. In this paper, resampling methods based on multipliers are proposed in a general framework where one investigates the stochastic behavior of a random vector \(\mathbf {Y}\in \mathbb {R}^d\) conditional on a covariate \(X \in \mathbb {R}\). Specifically, two versions of the multiplier bootstrap adapted to empirical conditional distributions are introduced as alternatives to the conditional bootstrap and their asymptotic validity is formally established. As the method walks hand-in-hand with the functional delta method, theory around the estimation of statistical functionals is developed accordingly; this includes the interval estimation of conditional mean and variance, conditional correlation coefficient, Kendall’s dependence measure and copula. Composite inference about univariate and joint conditional distributions is also considered. The sample behavior of the new bootstrap schemes and related estimation methods are investigated via simulations and an illustration on real data is provided.     

An adjusted cumulative Kullback-Leibler information with application to test of exponentiality

Abstract

In order to discriminate between two probability distributions extensions of Kullback–Leibler (KL) information have been proposed in the literature. In recent years, an extension called cumulative Kullback–Leibler (CKL) information is considered by authors which is closely related to equilibrium distributions. In this paper, we propose an adjusted version of CKL based on equilibrium distributions. Some properties of the proposed measure of divergence are investigated. A test of exponentiality based on the adjusted measure, is proposed. The empirical power of the presented test is calculated and compared with some existing standard tests of exponentiality. The results show that our proposed test, for some important alternative distributions, has better performance than some of the existing tests.     

On the asymptotic non‐equivalence of efficient‐GMM and MEL estimators in models with missing data

The generalized method of moments (GMM) and empirical likelihood (EL) are popular methods for combining sample and auxiliary information. These methods are used in very diverse fields of research, where competing theories often suggest variables satisfying different moment conditions. Results in the literature have shown that the efficient‐GMM (GMM_E) and maximum empirical likelihood (MEL) estimators have the same asymptotic distribution to order n^?1/2 and that both estimators are asymptotically semiparametric efficient. In this paper, we demonstrate that when data are missing at random from the sample, the utilization of some well‐known missing‐data handling approaches proposed in the literature can yield GMM_E and MEL estimators with nonidentical properties; in particular, it is shown that the GMM_E estimator is semiparametric efficient under all the missing‐data handling approaches considered but that the MEL estimator is not always efficient. A thorough examination of the reason for the nonequivalence of the two estimators is presented. A particularly strong feature of our analysis is that we do not assume smoothness in the underlying moment conditions. Our results are thus relevant to situations involving nonsmooth estimating functions, including quantile and rank regressions, robust estimation, the estimation of receiver operating characteristic (ROC) curves, and so on.     

Estimators of shift based on statistics of the Kolmogorov-Smirnov type

This paper is concerned with the estimation of a shift parameter δ_o, based on some nonnegative functional Hg₁ of the pair (D^δ_N(x), f?^δ_N(x)), where D^δ_N(x) = K_N/b {F_2,n(x)—F_1,m (x + δ)}, +^δ_N(x) = {mF_1,m (x + δ) + nF_2,n(x)}/N, where F_1,m and F_2,n are the empirical distribution functions of two independent random samples (N = m + n), and where K²_N = mn/N. First an estimator δ_N, is defined as a value of δ minimizing a functional H of the type of H₁. A second estimator δ¹_N is also defined which is a linearized version of the first. Finite and asymptotic properties of these estimators are considered. It is also shown that most well-known test statistics of the Kolmogorov-Smirnov type are particular cases of such functionals H₁. The asymptotic distribution and the asymptotic efficiency of some estimators are given.     

Empirical likelihood inference for probability density functions under association

In this paper, we consider to apply the empirical likelihood method to a probability density function under an associated sample. It is shown that the empirical likelihood ratio statistic is asymptotically χ²-type distributed under some mild conditions. The result is used to construct empirical likelihood-based confidence intervals on the probability density function.     

Biased discriminant analysis: Evaluation of the optimum probability of misclassification

This article extends the work of DiPillo (1976) on the Biased Minimum x² Rule. The optimum value of k (the biasing factor) Is determined and the true probability of misclassification is found. The proportion improvements reported in the 1976 paper are shown to be conservative. Some suggestions for algorithms to determine the optimal value of k are presented.     

A class of fixed-width confidence intervals for a ranked normal mean

Suppose that we are given k(≥ 2) independent and normally distributed populations π₁, …, π_k where π_i has unknown mean μ_i and unknown variance σ² _i (i = 1, …, k). Let μ_[i] (i = 1, …, k) denote the i^th smallest one of μ₁, …, μ_k. A two-stage procedure is used to construct lower and upper confidence intervals for μ_[i] and then use these to obtain a class of two-sided confidence intervals on μ_[i] with fixed width. For i = k, the interval given by Chen and Dudewicz (1976) is a special case. Comparison is made between the class of two-sided intervals and a symmetric interval proposed by Chen and Dudewicz (1976) for the largest mean, and it is found that for large values of k at least one of the former intervals requires a smaller total sample size. The tables needed to actually apply the procedure are provided.