On the Convergence of the Monte Carlo Maximum Likelihood Method for Latent Variable Models

While much used in practice, latent variable models raise challenging estimation problems due to the intractability of their likelihood. Monte Carlo maximum likelihood (MCML), as proposed by Geyer & Thompson (1992 ), is a simulation-based approach to maximum likelihood approximation applicable to general latent variable models. MCML can be described as an importance sampling method in which the likelihood ratio is approximated by Monte Carlo averages of importance ratios simulated from the complete data model corresponding to an arbitrary value of the unknown parameter. This paper studies the asymptotic (in the number of observations) performance of the MCML method in the case of latent variable models with independent observations. This is in contrast with previous works on the same topic which only considered conditional convergence to the maximum likelihood estimator, for a fixed set of observations. A first important result is that when is fixed, the MCML method can only be consistent if the number of simulations grows exponentially fast with the number of observations. If on the other hand, is obtained from a consistent sequence of estimates of the unknown parameter, then the requirements on the number of simulations are shown to be much weaker.     

Approximation of density functions by orthogonal series with grouped data

Summary Letg(x) andf(x) be continuous density function on (a, b) and let {ϕ_j} be a complete orthonormal sequence of functions onL ₂(g), which is the set of squared integrable functions weighted byg on (a, b). Suppose that over (a, b). Given a grouped sample of sizen fromf(x), the paper investigates the asymptotic properties of the restricted maximum likelihood estimator of density, obtained by setting all but the firstm of the ϑ_j’s equal to0. Practical suggestions are given for performing estimation via the use of Fourier and Legendre polynomial series. Research partially supported by: CNR grant, n. 93. 00837. CT10.     

A Unified Approach to the Approximation of Multivariate Densities

Approximation of a density by another density is considered in the case of different dimensionalities of the distributions. The results have been derived by inverting expansions of characteristic functions with the help of matrix techniques. The approximations obtained are all functions of cumulant differences and derivatives of the approximating density. The multivariate Edgeworth expansion follows from the results as a special case. Furthermore, the density functions of the trace and eigenvalues of the sample covariance matrix are approximated by the multivariate normal density and a numerical example is given     

ASSESSING THE RELATIONSHIP BETWEEN SYMPTOMS OF ALLERGIC RHINOCONJUNCTIVITIS AND POLLEN COUNTS

The relationship between daily pollen counts during the peak pollen season and hay fever symptoms in known sufferers of pollen allergy was investigated in a Sydney hospital‐based study. This paper develops statistical models for both the short term (day to day) associations and the longer term relationships between these time series. Possible effects of asthma status are investigated. The analyses illustrate how different relationships between time series may be explored in a simple way by working on different time scales with suitably transformed variables.     

Statistical Analysis of Marginal Count Failure Data

Manufacturers want to assess the quality andreliability of their products. Specifically, they want to knowthe exact number of failures from the sales transacted duringa particular month. Information available today is sometimesincomplete as many companies analyze their failure data simplycomparing sales for a total month from a particular departmentwith the total number of claims registered for that given month.This information—called marginal count data—is, thus,incomplete as it does not give the exact number of failures ofthe specific products that were sold in a particular month. Inthis paper we discuss nonparametric estimation of the mean numbersof failures for repairable products and the failure probabilitiesfor nonrepairable products. We present a nonhomogeneous Poissonprocess model for repairable products and a multinomial modeland its Poisson approximation for nonrepairable products. A numericalexample is given and a simulation is carried out to evaluatethe proposed methods of estimating failure probabilities undera number of possible situations.     

On Non-parametric Estimation of the Survival Function with Competing Risks

The non-parametric maximum likelihood estimators (MLEs) are derived for survival functions associated with individual risks or system components in a reliability framework. Lifetimes are observed for systems that contain one or more of those components. Analogous to a competing risks model, the system is assumed to fail upon the first instance of any component failure; i.e. the system is configured in series. For any given risk or component type, the asymptotic distribution is shown to depend explicitly on the unknown survival function of the other risks, as well as the censoring distribution. Survival functions with increasing failure rate are investigated as a special case. The order restricted MLE is shown to be consistent under mild assumptions of the underlying component lifetime distributions.     

Smoothing for discrete-valued time series

We deal with smoothed estimators for conditional probability functions of discrete-valued time series { Y_t } under two different settings. When the conditional distribution of Y_t given its lagged values falls in a parametric family and depends on exogenous random variables, a smoothed maximum (partial) likelihood estimator for the unknown parameter is proposed. While there is no prior information on the distribution, various nonparametric estimation methods have been compared and the adjusted Nadaraya–Watson estimator stands out as it shares the advantages of both Nadaraya–Watson and local linear regression estimators. The asymptotic normality of the estimators proposed has been established in the manner of sparse asymptotics, which shows that the smoothed methods proposed outperform their conventional, unsmoothed, parametric counterparts under very mild conditions. Simulation results lend further support to this assertion. Finally, the new method is illustrated via a real data set concerning the relationship between the number of daily hospital admissions and the levels of pollutants in Hong Kong in 1994–1995. An ad hoc model selection procedure based on a local Akaike information criterion is proposed to select the significant pollutant indices.     

SEMIPARAMETRIC REGRESSION AND GRAPHICAL MODELS

Semiparametric regression models that use spline basis functions with penalization have graphical model representations. This link is more powerful than previously established mixed model representations of semiparametric regression, as a larger class of models can be accommodated. Complications such as missingness and measurement error are more naturally handled within the graphical model architecture. Directed acyclic graphs, also known as Bayesian networks, play a prominent role. Graphical model-based Bayesian 'inference engines', such as bugs and vibes , facilitate fitting and inference. Underlying these are Markov chain Monte Carlo schemes and recent developments in variational approximation theory and methodology.     

Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations

Summary.  Structured additive regression models are perhaps the most commonly used class of models in statistical applications. It includes, among others, (generalized) linear models, (generalized) additive models, smoothing spline models, state space models, semiparametric regression, spatial and spatiotemporal models, log-Gaussian Cox processes and geostatistical and geoadditive models. We consider approximate Bayesian inference in a popular subset of structured additive regression models, latent Gaussian models , where the latent field is Gaussian, controlled by a few hyperparameters and with non-Gaussian response variables. The posterior marginals are not available in closed form owing to the non-Gaussian response variables. For such models, Markov chain Monte Carlo methods can be implemented, but they are not without problems, in terms of both convergence and computational time. In some practical applications, the extent of these problems is such that Markov chain Monte Carlo sampling is simply not an appropriate tool for routine analysis. We show that, by using an integrated nested Laplace approximation and its simplified version, we can directly compute very accurate approximations to the posterior marginals. The main benefit of these approximations is computational: where Markov chain Monte Carlo algorithms need hours or days to run, our approximations provide more precise estimates in seconds or minutes. Another advantage with our approach is its generality, which makes it possible to perform Bayesian analysis in an automatic, streamlined way, and to compute model comparison criteria and various predictive measures so that models can be compared and the model under study can be challenged.     

Forecasting obesity trends in England

Summary.  Forecasts of trends in obesity in England for 2010 are produced by treating the available data, which contain the proportions of the population, categorized by age and sex, falling into different body mass index ranges, as compositional data sets, so that the implicit simplex restrictions are automatically satisfied. Forecasts are calculated by using linear trend models for the log-ratio transformations and are accompanied by prediction regions. The advantages of treating data on proportions compositionally are emphasized and compared with forecasts that have been obtained by ignoring this restriction.