Estimation of the Moment Generating Function

A number of statistical problems use the moment generating function (mgf) for purposes other than determining the moments of a distribution. If the distribution is not completely specified, then the mgf must be estimated from available data. The empirical mgf makes no assumptions concerning the underlying distribution except for the existence of the mgf. In contrast to the nonparametric approach provided by the empirical mgf, alternative estimators can be formed based on an assumed parametric model. Comparison of these approaches is considered for two parametric models; the normal and a one parameter gamma. Comparison criteria are efficiency and empirical confidence interval coverage. In general the parametric estimators outperform the empirical mgf when the model is correct. The comparisons are extended to underlying models which are two component mixtures from the distributional family assumed by the parametric estimators. Under the mixture models the superiority of the parametric estimator depends upon the model, value of the argument of the mgf, and the comparison criterion. The empirical mgf is the better estimator in some cases.     

Robust Mean Estimation Under a Possibly Incorrect Log-Normality Assumption

Nonparametric and parametric estimators are combined to minimize the mean squared error among their linear combinations. The combined estimator is consistent and for large sample sizes has a smaller mean squared error than the nonparametric estimator when the parametric assumption is violated. If the parametric assumption holds, the combined estimator has a smaller MSE than the parametric estimator. Our simulation examples focus on mean estimation when data may follow a lognormal distribution, or can be a mixture with an exponential or a uniform distribution. Motivating examples illustrate possible application areas.     

Adaptive estimators for parameters of the autoregression function of a Markov chain

Suppose we observe an ergodic Markov chain on the real line, with a parametric model for the autoregression function, i.e. the conditional mean of the transition distribution. If one specifies, in addition, a parametric model for the conditional variance, one can define a simple estimator for the parameter, the maximum quasi-likelihood estimator. It is robust against misspecification of the conditional variance, but not efficient. We construct an estimator which is adaptive in the sense that it is efficient if the conditional variance is misspecified, and asymptotically as good as the maximum quasi-likelihood estimator if the conditional variance is correctly specified. The adaptive estimator is a weighted nonlinear least-squares estimator, with weights given by predictors for the conditional variance.     

Comparison of renewal function estimators for censored data

The computation of the renewal function when the distribution function is completely known has received much attention in the literature. However, in many cases the form of the distribution function is unknown and has to be estimated nonparametrically. A nonparametric estimator for the renewal function for complete data was suggested by Frees (1986). In many cases, however, censoring of the lifetime might occur. We shall present parametric and nonparametric estimators of the renewal function based on censored data. In a simulation study we compare the nonparametric estimators with parametric estimators for the Weibull and lognormal distribution. The study suggests that the nonparametric estimator is a viable alternative to the parametric estimators when the lifetime distribution is unknown. Also, the nonparametric estimator is computationally simpler than the parametric estimator.     

Efficient estimation of auto-regression parameters and innovation distributions for semiparametric integer-valued AR(p) models

Summary.  Integer-valued auto-regressive (INAR) processes have been introduced to model non-negative integer-valued phenomena that evolve over time. The distribution of an INAR( p ) process is essentially described by two parameters: a vector of auto-regression coefficients and a probability distribution on the non-negative integers, called an immigration or innovation distribution. Traditionally, parametric models are considered where the innovation distribution is assumed to belong to a parametric family. The paper instead considers a more realistic semiparametric INAR( p ) model where there are essentially no restrictions on the innovation distribution. We provide an (semiparametrically) efficient estimator of both the auto-regression parameters and the innovation distribution.     

Statistical inference for restricted partially linear varying coefficient errors-in-variables models

As a useful extension of partially linear models and varying coefficient models, the partially linear varying coefficient model is useful in statistical modelling. This paper considers statistical inference for the semiparametric model when the covariates in the linear part are measured with additive error and some additional linear restrictions on the parametric component are available. We propose a restricted modified profile least-squares estimator for the parametric component, and prove the asymptotic normality of the proposed estimator. To test hypotheses on the parametric component, we propose a test statistic based on the difference between the corrected residual sums of squares under the null and alterative hypotheses, and show that its limiting distribution is a weighted sum of independent chi-square distributions. We also develop an adjusted test statistic, which has an asymptotically standard chi-squared distribution. Some simulation studies are conducted to illustrate our approaches.     

Importance Sampling for Sums of Lognormal Distributions with Applications to Operational Risk

In this paper we use Importance Sampling to estimate tail probabilities for a finite sum of lognormal distributions. We use a defensive mixture, and develop a method of choosing the parameters via the EM algorithm; we also consider the technique which assumes the importance sampling density to belong to the same parametric family of the random variables to be summed. In both cases, the instrumental density is found by minimizing Cross-Entropy. A comparison based on several simulation experiments shows that the defensive mixture has the best performance. Finally, we study the Poisson-lognormal compound distribution framework and present a real-data application.     

Asymptotic Normality in Mixtures of Power Series Distributions

Abstract.  The problem of estimating the individual probabilities of a discrete distribution is considered. The true distribution of the independent observations is a mixture of a family of power series distributions. First, we ensure identifiability of the mixing distribution assuming mild conditions. Next, the mixing distribution is estimated by non-parametric maximum likelihood and an estimator for individual probabilities is obtained from the corresponding marginal mixture density. We establish asymptotic normality for the estimator of individual probabilities by showing that, under certain conditions, the difference between this estimator and the empirical proportions is asymptotically negligible. Our framework includes Poisson, negative binomial and logarithmic series as well as binomial mixture models. Simulations highlight the benefit in achieving normality when using the proposed marginal mixture density approach instead of the empirical one, especially for small sample sizes and/or when interest is in the tail areas. A real data example is given to illustrate the use of the methodology.     

Statistical Inference in Partially Linear Varying-Coefficient Models with Missing Responses at Random

This article considers statistical inference for partially linear varying-coefficient models when the responses are missing at random. We propose a profile least-squares estimator for the parametric component with complete-case data and show that the resulting estimator is asymptotically normal. To avoid to estimate the asymptotic covariance in establishing confidence region of the parametric component with the normal-approximation method, we define an empirical likelihood based statistic and show that its limiting distribution is chi-squared distribution. Then, the confidence regions of the parametric component with asymptotically correct coverage probabilities can be constructed by the result. To check the validity of the linear constraints on the parametric component, we construct a modified generalized likelihood ratio test statistic and demonstrate that it follows asymptotically chi-squared distribution under the null hypothesis. Then, we extend the generalized likelihood ratio technique to the context of missing data. Finally, some simulations are conducted to illustrate the proposed methods.     

Local Box–Cox transformation on time-varying parametric models for smoothing estimation of conditional CDF with longitudinal data

Nonparametric estimation and inferences of conditional distribution functions with longitudinal data have important applications in biomedical studies, such as epidemiological studies and longitudinal clinical trials. Estimation approaches without any structural assumptions may lead to inadequate and numerically unstable estimators in practice. We propose in this paper a nonparametric approach based on time-varying parametric models for estimating the conditional distribution functions with a longitudinal sample. Our model assumes that the conditional distribution of the outcome variable at each given time point can be approximated by a parametric model after local Box–Cox transformation. Our estimation is based on a two-step smoothing method, in which we first obtain the raw estimators of the conditional distribution functions at a set of disjoint time points, and then compute the final estimators at any time by smoothing the raw estimators. Applications of our two-step estimation method have been demonstrated through a large epidemiological study of childhood growth and blood pressure. Finite sample properties of our procedures are investigated through a simulation study. Application and simulation results show that smoothing estimation from time-variant parametric models outperforms the existing kernel smoothing estimator by producing narrower pointwise bootstrap confidence band and smaller root mean squared error.     

Two-sample high-dimensional empirical likelihood

In this paper, we apply empirical likelihood for two-sample problems with growing high dimensionality. Our results are demonstrated for constructing confidence regions for the difference of the means of two p-dimensional samples and the difference in value between coefficients of two p-dimensional sample linear model. We show that empirical likelihood based estimator has the efficient property. That is, as p → ∞ for high-dimensional data, the limit distribution of the EL ratio statistic for the difference of the means of two samples and the difference in value between coefficients of two-sample linear model is asymptotic normal distribution. Furthermore, empirical likelihood (EL) gives efficient estimator for regression coefficients in linear models, and can be as efficient as a parametric approach. The performance of the proposed method is illustrated via numerical simulations.     

Inference based on adaptive grid selection of probability transforms

Probability transform-based inference, for example, characteristic function-based inference, is a good alternative to likelihood methods when the probability density function is unavailable or intractable. However, a set of grids needs to be determined to provide an effective estimator based on probability transforms. This paper is concerned with parametric inference based on adaptive selection of grids. By employing a closeness measure to evaluate the asymptotic variance of the transform-based estimator, we propose a statistical inference procedure, accompanied with adaptive grid selection. The selection algorithm aims for a small set of grids, and yet the resulting estimator can be highly efficient. Generally, the asymptotic variance is very close to that of the maximum likelihood estimator.     

Pairwise likelihood estimation for multivariate mixed Poisson models generated by Gamma intensities

Estimating the parameters of multivariate mixed Poisson models is an important problem in image processing applications, especially for active imaging or astronomy. The classical maximum likelihood approach cannot be used for these models since the corresponding masses cannot be expressed in a simple closed form. This paper studies a maximum pairwise likelihood approach to estimate the parameters of multivariate mixed Poisson models when the mixing distribution is a multivariate Gamma distribution. The consistency and asymptotic normality of this estimator are derived. Simulations conducted on synthetic data illustrate these results and show that the proposed estimator outperforms classical estimators based on the method of moments. An application to change detection in low-flux images is also investigated.     

Estimating the density of a possibly missing response variable in nonlinear regression

This paper considers linear and nonlinear regression with a response variable that is allowed to be “missing at random”. The only structural assumptions on the distribution of the variables are that the errors have mean zero and are independent of the covariates. The independence assumption is important. It enables us to construct an estimator for the response density that uses all the observed data, in contrast to the usual local smoothing techniques, and which therefore permits a faster rate of convergence. The idea is to write the response density as a convolution integral which can be estimated by an empirical version, with a weighted residual-based kernel estimator plugged in for the error density. For an appropriate class of regression functions, and a suitably chosen bandwidth, this estimator is consistent and converges with the optimal parametric rate n^1/2. Moreover, the estimator is proved to be efficient (in the sense of Hájek and Le Cam) if an efficient estimator is used for the regression parameter.     

Estimation of the population mean when the coefficient of variation is known

Theory has been developed to provide an optimum estimator of the population mean based on a “mean per unit” estimator and the estimated standard deviation, assuming that the form of the distribution as well as its coefficient of variation (c.v.) are known. Theory has been extended to the case when an estimate of c.v. is available from an independent sample drawn in the past; the case when the form of the distribution is not known is also discussed. It is shown that the relative efficiency of the estimator with respect to “mean per unit estimator” is generally high for normal or near normal populations. For log-normal populations, an increase in efficiency of about 17 percent can be achieved. The results have been illustrated with data from biological populations.     

A Note on the Hodges–Lehmann Estimator and the Logistic Distribution

ABSTRACT

It is well known that the Hodges–Lehmann estimator is asymptotically efficient for the location parameter of the logistic distribution. In this article we give a simple and direct proof that this property also characterizes the logistic between all the symmetric location distributions under mild conditions. Using pseudolikelihood, we also show how to find from the Hodges–Lehmann estimator an asymptotically efficient estimator of the scale parameter of the logistic distribution.     

Estimation and Inference Based on Neumann Series Approximation to Locally Efficient Score in Missing Data Problems

Abstract.  Theory on semi-parametric efficient estimation in missing data problems has been systematically developed by Robins and his coauthors. Except in relatively simple problems, semi-parametric efficient scores cannot be expressed in closed forms. Instead, the efficient scores are often expressed as solutions to integral equations. Neumann series was proposed in the form of successive approximation to the efficient scores in those situations. Statistical properties of the estimator based on the Neumann series approximation are difficult to obtain and as a result, have not been clearly studied. In this paper, we reformulate the successive approximation in a simple iterative form and study the statistical properties of the estimator based on the reformulation. We show that a doubly robust locally efficient estimator can be obtained following the algorithm in robustifying the likelihood score. The results can be applied to, among others, parametric regression, marginal regression and Cox regression when data are subject to missing values and the data are missing at random. A simulation study is conducted to evaluate the performance of the approach and a real data example is analysed to demonstrate the use of the approach.     

The Logistic-Uniform Distribution and Its Applications

In this article, we give a new family of univariate distributions generated by the Logistic random variable. A special case of this family is the Logistic-Uniform distribution. We show that the Logistic-Uniform distribution provides great flexibility in modeling for symmetric, negatively and positively skewed, bathtub-shaped, “J”-shaped, and reverse “J”-shaped distributions. We discuss simulation issues, estimation by the methods of moments, maximum likelihood, and the new method of minimum spacing distance estimator. We also derive Shannon entropy and asymptotic distribution of the extreme order statistics of this distribution. The new distribution can be used effectively in the analysis of survival data since the hazard function of the distribution can be “J,” bathtub, and concave-convex shaped. The usefulness of the new distribution is illustrated through two real datasets by showing that it is more flexible in analyzing the data than the Beta Generalized-Exponential, Beta-Exponential, Beta-Normal, Beta-Laplace, Beta Generalized half-Normal, β-Birnbaum-Saunders, Gamma-Uniform, Beta Generalized Pareto, Beta Modified Weibull, Beta-Pareto, Generalized Modified Weibull, Beta-Weibull, and Modified-Weibull distributions.     

On the asymptotic bias of grenander’s mode estimator

Grenander introduced a direct estimator of the mode for a large class of densities. This note considers a large subclass of these densities for which Grenander’s estimator is asymptotically biased. Some of the distributions from this subclass include the F, gamma, and beta for which asymptotic expressions for the bias are given. To reduce the bias, it is recommended to choose larger values for one of the parameters of the estimator when the underlying distribution is nonsymmetric.     

Generating data from improper distributions: application to Cox proportional hazards models with cure

Cure rate models are survival models characterized by improper survivor distributions which occur when the cumulative distribution function, say F, of the survival times does not sum up to 1 (i.e. F(+∞)<1). The first objective of this paper is to provide a general approach to generate data from any improper distribution. An application to times to event data randomly drawn from improper distributions with proportional hazards is investigated using the semi-parametric proportional hazards model with cure obtained as a special case of the nonlinear transformation models in [Tsodikov, Semiparametric models: A generalized self-consistency approach, J. R. Stat. Soc. Ser. B 65 (2003), pp. 759–774]. The second objective of this paper is to show by simulations that the bias, the standard error and the mean square error of the maximum partial likelihood (PL) estimator of the hazard ratio as well as the statistical power based on the PL estimator strongly depend on the proportion of subjects in the whole population who will never experience the event of interest.