Comparison of dispersions of mixtures of ordered distributions

In the paper we consider mixtures of unknown stochastically ordered distribution functions according to known mixing distribution functions. We provide optimal lower and upper bounds on ratios of general dispersion measures of such mixtures. The bounds do not depend on the particular form of dispersion measure. We present applications of the results in reliability theory, insurance mathematics, Bayesian statistics, and regression analysis.     

Asymptotically optimal bandwidth selection rules for the kernel density estimator with dependent observations

With mild restrictions placed on the kernel, kernel estimates of an unknown multivariate density are investigated when the observed data are dependent. A modified cross validation rule, the simple ‘leave-(2P + 1)-out’ version of simple cross validation, is considered for bandwidth selection. Under the mild assumption that the unknown density is bounded, this rule is shown to be asymptotically optimal under a geometric strong mixing condition. This strengthens recent results of Hart and Vieu (Ann. Statist., 18 (1990)). The results are then extended to bandwidth selection problems associated to the Gibbs sampler.     

On approximations via convolution-defined mixture models

An often-cited fact regarding mixing or mixture distributions is that their density functions are able to approximate the density function of any unknown distribution to arbitrary degrees of accuracy, provided that the mixing or mixture distribution is sufficiently complex. This fact is often not made concrete. We investigate and review theorems that provide approximation bounds for mixing distributions. Connections between the approximation bounds of mixing distributions and estimation bounds for the maximum likelihood estimator of finite mixtures of location-scale distributions are reviewed.     

Plug-in bandwidth choice for estimation of nonparametric part in partial linear regression models with strong mixing errors

Consider a regression model where the regression function is the sum of a linear and a nonparametric component. Assuming that the errors of the model follow a stationary strong mixing process with mean zero, the problem of bandwidth selection for a kernel estimator of the nonparametric component is addressed here. We obtain an asymptotic expression for an optimal band-width and we propose to use a plug-in methodology in order to estimate this bandwidth through preliminary estimates of the unknown quantities. Asymptotic optimality for the plug-in bandwidth is established.     

Some Recent Developments in Econometric Inference

Recent results in information theory, see Soofi (1996; 2001) for a review, include derivations of optimal information processing rules, including Bayes' theorem, for learning from data based on minimizing a criterion functional, namely output information minus input information as shown in Zellner (1988; 1991; 1997; 2002). Herein, solution post data densities for parameters are obtained and studied for cases in which the input information is that in (1) a likelihood function and a prior density; (2) only a likelihood function; and (3) neither a prior nor a likelihood function but only input information in the form of post data moments of parameters, as in the Bayesian method of moments approach. Then it is shown how optimal output densities can be employed to obtain predictive densities and optimal, finite sample structural coefficient estimates using three alternative loss functions. Such optimal estimates are compared with usual estimates, e.g., maximum likelihood, two-stage least squares, ordinary least squares, etc. Some Monte Carlo experimental results in the literature are discussed and implications for the future are provided.     

Some Recent Developments in Econometric Inference

Abstract

Recent results in information theory, see Soofi (1996; 2001) for a review, include derivations of optimal information processing rules, including Bayes' theorem, for learning from data based on minimizing a criterion functional, namely output information minus input information as shown in Zellner (1988; 1991; 1997; 2002). Herein, solution post data densities for parameters are obtained and studied for cases in which the input information is that in (1) a likelihood function and a prior density; (2) only a likelihood function; and (3) neither a prior nor a likelihood function but only input information in the form of post data moments of parameters, as in the Bayesian method of moments approach. Then it is shown how optimal output densities can be employed to obtain predictive densities and optimal, finite sample structural coefficient estimates using three alternative loss functions. Such optimal estimates are compared with usual estimates, e.g., maximum likelihood, two‐stage least squares, ordinary least squares, etc. Some Monte Carlo experimental results in the literature are discussed and implications for the future are provided.     

Grouping normal distributions with unknown parameters

Optimal group definitions, maximizing retained information, are well known for grouping normal distributions with known parameters. When these group definitions are used with sample estimates substituted for unknown parameters, there is a shrinkage of retained information. If there are k groups, and if the size of the sample yielding parameter estimates is n, then this shrinkage is less than 5% for n <= 30 and minimization of shrinkage yields less than 1 % improvement for n <= 5, when k >= 12.     

Bayes estimates of mixing proportions in finite mixture distributions

With reference to the problem of estimating the mixing proportions in a finite mixture distribution with known components, employing Dirichlet prior, closed form expressions for the posterior means and variances are obtained. To avoid the difficulties in computing the estimates, an approximation procedure is introduced. Numerical studies carried out for normal mixtures indicate the closeness of the approximations and their superiority over the maximum likelihood estimates at least in the case of small samples.     

Semiparametric Efficient Estimation of the Mean of a Time Series in the Presence of Conditional Heterogeneity of Unknown Form

We obtain semiparametric efficiency bounds for estimation of a location parameter in a time series model where the innovations are stationary and ergodic conditionally symmetric martingale differences but otherwise possess general dependence and distributions of unknown form. We then describe an iterative estimator that achieves this bound when the conditional density functions of the sample are known. Finally, we develop a “semi-adaptive” estimator that achieves the bound when these densities are unknown by the investigator. This estimator employs nonparametric kernel estimates of the densities. Monte Carlo results are reported.     

A general maximum likelihood analysis of overdispersion in generalized linear models

This paper presents an EM algorithm for maximum likelihood estimation in generalized linear models with overdispersion. The algorithm is initially derived as a form of Gaussian quadrature assuming a normal mixing distribution, but with only slight variation it can be used for a completely unknown mixing distribution, giving a straightforward method for the fully non-parametric ML estimation of this distribution. This is of value because the ML estimates of the GLM parameters may be sensitive to the specification of a parametric form for the mixing distribution. A listing of a GLIM4 algorithm for fitting the overdispersed binomial logit model is given in an appendix.A simple method is given for obtaining correct standard errors for parameter estimates when using the EM algorithm.Several examples are discussed.     

On estimating parameters of a progressively censored lognormal distribution

We consider the problem of making statistical inference on unknown parameters of a lognormal distribution under the assumption that samples are progressively censored. The maximum likelihood estimates (MLEs) are obtained by using the expectation-maximization algorithm. The observed and expected Fisher information matrices are provided as well. Approximate MLEs of unknown parameters are also obtained. Bayes and generalized estimates are derived under squared error loss function. We compute these estimates using Lindley's method as well as importance sampling method. Highest posterior density interval and asymptotic interval estimates are constructed for unknown parameters. A simulation study is conducted to compare proposed estimates. Further, a data set is analysed for illustrative purposes. Finally, optimal progressive censoring plans are discussed under different optimality criteria and results are presented.     

Conditional Prior Proposals in Dynamic Models

ABSTRACT. Dynamic models extend state space models to non-normal observations. This paper suggests a specific hybrid Metropolis–Hastings algorithm as a simple device for Bayesian inference via Markov chain Monte Carlo in dynamic models. Hastings proposals from the (conditional) prior distribution of the unknown, time-varying parameters are used to update the corresponding full conditional distributions. It is shown through simulated examples that the methodology has optimal performance in situations where the prior is relatively strong compared to the likelihood. Typical examples include smoothing priors for categorical data. A specific blocking strategy is proposed to ensure good mixing and convergence properties of the simulated Markov chain. It is also shown that the methodology is easily extended to robust transition models using mixtures of normals. The applicability is illustrated with an analysis of a binomial and a binary time series, known in the literature.     

Semiparametric Efficient Estimation of the Mean of a Time Series in the Presence of Conditional Heterogeneity of Unknown Form

Abstract

We obtain semiparametric efficiency bounds for estimation of a location parameter in a time series model where the innovations are stationary and ergodic conditionally symmetric martingale differences but otherwise possess general dependence and distributions of unknown form. We then describe an iterative estimator that achieves this bound when the conditional density functions of the sample are known. Finally, we develop a “semi-adaptive” estimator that achieves the bound when these densities are unknown by the investigator. This estimator employs nonparametric kernel estimates of the densities. Monte Carlo results are reported.     

Statistical Inference and Applications of Mixture of Varying Coefficient Models

In this paper, we consider a new mixture of varying coefficient models, in which each mixture component follows a varying coefficient model and the mixing proportions and dispersion parameters are also allowed to be unknown smooth functions. We systematically study the identifiability, estimation and inference for the new mixture model. The proposed new mixture model is rather general, encompassing many mixture models as its special cases such as mixtures of linear regression models, mixtures of generalized linear models, mixtures of partially linear models and mixtures of generalized additive models, some of which are new mixture models by themselves and have not been investigated before. The new mixture of varying coefficient model is shown to be identifiable under mild conditions. We develop a local likelihood procedure and a modified expectation–maximization algorithm for the estimation of the unknown non‐parametric functions. Asymptotic normality is established for the proposed estimator. A generalized likelihood ratio test is further developed for testing whether some of the unknown functions are constants. We derive the asymptotic distribution of the proposed generalized likelihood ratio test statistics and prove that the Wilks phenomenon holds. The proposed methodology is illustrated by Monte Carlo simulations and an analysis of a CO₂‐GDP data set.     

Semiparametric estimation in the normal variance-mean mixture model

In this paper we study the problem of statistical inference on the parameters of the semiparametric variance-mean mixtures. This class of mixtures has recently become rather popular in statistical and financial modelling. We design a semiparametric estimation procedure that first estimates the mean of the underlying normal distribution and then recovers non-parametrically the density of the corresponding mixing distribution. We illustrate the performance of our procedure on simulated and real data.     

Generalized EM estimation for semi-parametric mixture distributions with discretized non-parametric component

We consider independent sampling from a two-component mixture distribution, where one component (called the parametric component) is from a known distributional family and the other component (called the non-parametric component) is unknown. This is a semi-parametric mixture distribution. We discretize the non-parametric component and estimate the parameters of this mixture model, namely the mixing proportion, the unknown parameters of the parametric component and the discretized non-parametric component. We define the maximum penalized likelihood (MPL) estimates of the mixture model parameters and then develop a generalized EM (GEM) iterative scheme to compute the MPL estimates. A simulation study and an example from biology are presented.     

Mixture experiments with mixing errors

In mixture experiments the properties of mixtures are usually studied by mixing the amounts of the mixture components that are required to obtain the necessary proportions. This paper considers the impact of inaccuracies in discharging the required amounts of the mixture components on the statistical analysis of the data. It shows how the regression calibration approach can be used to minimize the resulting bias in the model and in the estimates of the model parameters, as well as to find correct estimates of the corresponding variances. Its application is made difficult by the complex structure of these errors. We also show how knowledge of the form of the model bias allows for choosing a manufacturing setting for a mixture product that is not biased and has smaller signal to noise ratio.     

Kernel Likelihood Inference for Time Series

Abstract.  This paper develops non-parametric techniques for dynamic models whose data have unknown probability distributions. Point estimators are obtained from the maximization of a semiparametric likelihood function built on the kernel density of the disturbances. This approach can also provide Kullback–Leibler cross-validation estimates of the bandwidth of the kernel densities. Confidence regions are derived from the dual-empirical likelihood method based on non-parametric estimates of the scores. Limit theorems for martingale difference sequences support the statistical theory; moreover, simulation experiments and a real case study show the validity of the methods.     

Error reduction in density estimation under shape restrictions

For the problems of nonparametric estimation of nonincreasing and symmetric unimodal density functions with bounded supports we determine the projections of estimates onto the convex families of possible parent densities with respect to the weighted integrated squared error. We also describe the method of approximating the analogous projections onto the respective density classes satisfying some general moment conditions. The method of projections reduces the estimation errors for all possible values of observations of a given finite sample size in a uniformly optimal way and provides estimates sharing the properties of the parent densities.     

An adaptive rule for stopping a discovery process under time censorship

Consider a finite population of large but unknown size of hidden objects. Consider searching for these objects for a period of time, at a certain cost, and receiving a reward depending on the sizes of the objects found. Suppose that the size and discovery time of the objects both have unknown distributions, but the conditional distribution of time given size is exponential with an unknown non-negative and non-decreasing function of the size as failure rate. The goal is to find an optimal way to stop the discovery process. Assuming that the above parameters are known, an optimal stopping time is derived and its asymptotic properties are studied. Then, an adaptive rule based on order restricted estimates of the distributions from truncated data is presented. This adaptive rule is shown to perform nearly as well as the optimal stopping time for large population size.