Properties and Applications of the Generalized Likelihood as a Summary Function for Prediction Problems

The generalized likelihood plays an important role in parametric inference for prediction and empirical Bayesian models. This paper emphasizes the utility of the generalized likelihood as a summarization procedure in general prediction models. Properties of the generalized likelihood when used in this setting, and examples of its use as a data analytic tool are given in a series of numerical examples.     

Numerical Comparison Between Different Empirical Prediction Intervals Under the Fay-Herriot Model

Recently, an empirical best linear unbiased predictor is widely used as a practical approach to small area inference. It is also of interest to construct empirical prediction intervals. However, we do not know which method should be used from among the several existing prediction intervals. In this article, we first obtain an empirical prediction interval by using the residual maximum likelihood method for estimating unknown model variance parameters. Then we compare the later with other intervals with the residual maximum likelihood method. Additionally, some different parametric bootstrap methods for constructing empirical prediction intervals are also compared in a simulation study.     

Approximate maximum likelihood predictors of future failure times of shifted exponential distributions under multiple type II censoring

Suppose we consider a general multiple type II censored sample (some middle observations being censored) from a shifted exponential distribution. The maximum likelihood prediction method does not admit explicit solutions. We introduce a simple approximation to one of prediction likelihood equations and derive approximate predictors of missing failure times. We compute their mean square prediction errors by simulation and compare them with the best linear predictors. Further, we present two real examples to illustrate this method of prediction.AMS Subject Classification (2000): 62G30, 62M20, 62F99     

Approximate composite marginal likelihood inference in spatial generalized linear mixed models

Non-Gaussian spatial responses are usually modeled using spatial generalized linear mixed model with spatial random effects. The likelihood function of this model cannot usually be given in a closed form, thus the maximum likelihood approach is very challenging. There are numerical ways to maximize the likelihood function, such as Monte Carlo Expectation Maximization and Quadrature Pairwise Expectation Maximization algorithms. They can be applied but may in such cases be computationally very slow or even prohibitive. Gauss–Hermite quadrature approximation only suitable for low-dimensional latent variables and its accuracy depends on the number of quadrature points. Here, we propose a new approximate pairwise maximum likelihood method to the inference of the spatial generalized linear mixed model. This approximate method is fast and deterministic, using no sampling-based strategies. The performance of the proposed method is illustrated through two simulation examples and practical aspects are investigated through a case study on a rainfall data set.     

Linear approximate ML estimation in scaled Type I generalized logistic distribution based on Type-II censored samples

The scaled (two-parameter) Type I generalized logistic distribution (GLD) is considered with the known shape parameter. The ML method does not yield an explicit estimator for the scale parameter even in complete samples. In this article, we therefore construct a new linear estimator for scale parameter, based on complete and doubly Type-II censored samples, by making linear approximations to the intractable terms of the likelihood equation using least-squares (LS) method, a new approach of linearization. We call this as linear approximate maximum likelihood estimator (LAMLE). We also construct LAMLE based on Taylor series method of linear approximation and found that this estimator is slightly biased than that based on the LS method. A Monte Carlo simulation is used to investigate the performance of LAMLE and found that it is almost as efficient as MLE, though biased than MLE. We also compare unbiased LAMLE with BLUE based on the exact variances of the estimators and interestingly this new unbiased LAMLE is found just as efficient as the BLUE in both complete and Type-II censored samples. Since MLE is known as asymptotically unbiased, in large samples we compare unbiased LAMLE with MLE and found that this estimator is almost as efficient as MLE. We have also discussed interval estimation of the scale parameter from complete and Type-II censored samples. Finally, we present some numerical examples to illustrate the construction of the new estimators developed here.     

Measurement error in the generalised linear model

This paper considers the problem of estimating the linear parameters of a Generalised Linear Model (GLM) when the explanatory variable is subject to measurement error. In this situation the induced model for dependence on the approximate explanatory variable is not usually of GLM form. However, when the distribution of measurement error is known or estimated from replicated measurements, application of the GLIM iteratively reweighted least squares algorithm with transformed data and weighting is shown to produce maximum quasi likelihood estimates in many cases. Details of this approach are given for two particular generalized linear models; simulation results illustrate the usefulness of the theory for these models.     

A profile likelihood method for normal mixture with unequal variance

It is well known that the normal mixture with unequal variance has unbounded likelihood and thus the corresponding global maximum likelihood estimator (MLE) is undefined. One of the commonly used solutions is to put a constraint on the parameter space so that the likelihood is bounded and then one can run the EM algorithm on this constrained parameter space to find the constrained global MLE. However, choosing the constraint parameter is a difficult issue and in many cases different choices may give different constrained global MLE. In this article, we propose a profile log likelihood method and a graphical way to find the maximum interior mode. Based on our proposed method, we can also see how the constraint parameter, used in the constrained EM algorithm, affects the constrained global MLE. Using two simulation examples and a real data application, we demonstrate the success of our new method in solving the unboundness of the mixture likelihood and locating the maximum interior mode.     

Mixed effects smoothing spline analysis of variance

We propose a general family of nonparametric mixed effects models. Smoothing splines are used to model the fixed effects and are estimated by maximizing the penalized likelihood function. The random effects are generic and are modelled parametrically by assuming that the covariance function depends on a parsimonious set of parameters. These parameters and the smoothing parameter are estimated simultaneously by the generalized maximum likelihood method. We derive a connection between a nonparametric mixed effects model and a linear mixed effects model. This connection suggests a way of fitting a nonparametric mixed effects model by using existing programs. The classical two-way mixed models and growth curve models are used as examples to demonstrate how to use smoothing spline analysis-of-variance decompositions to build nonparametric mixed effects models. Similarly to the classical analysis of variance, components of these nonparametric mixed effects models can be interpreted as main effects and interactions. The penalized likelihood estimates of the fixed effects in a two-way mixed model are extensions of James–Stein shrinkage estimates to correlated observations. In an example three nested nonparametric mixed effects models are fitted to a longitudinal data set.     

Predictive inference for linear and multivariate linear models with ma(1) error processes

The prediction distributions of future responses from the linear and multivariate linear models with errors having a first order moving average (MA(1)) process have been derived. First, we obtained the marginal likelihood function for the moving average parameter 6 and from this likelihood function we estimate the maximum likelihood estimates (MLE) of θ. Using the estimated value θ, we have derived the prediction distributions as well as prediction regions for the future responses. An example has been included.     

Structural change tests for GEL criteria

This article examines structural change tests based on generalized empirical likelihood methods in the time series context, allowing for dependent data. Standard structural change tests for the Generalized method of moments (GMM) are adapted to the generalized empirical likelihood (GEL) context. We show that when moment conditions are properly smoothed, these test statistics converge to the same asymptotic distribution as in the GMM, in cases with known and unknown breakpoints. New test statistics specific to GEL methods, and that are robust to weak identification, are also introduced. A simulation study examines the small sample properties of the tests and reveals that GEL-based robust tests performed well, both in terms of the presence and location of a structural change and in terms of the nature of identification.     

A general maximum likelihood analysis of measurement error in generalized linear models

This paper describes an EM algorithm for maximum likelihood estimation in generalized linear models (GLMs) with continuous measurement error in the explanatory variables. The algorithm is an adaptation of that for nonparametric maximum likelihood (NPML) estimation in overdispersed GLMs described in Aitkin (Statistics and Computing 6: 251–262, 1996). The measurement error distribution can be of any specified form, though the implementation described assumes normal measurement error. Neither the reliability nor the distribution of the true score of the variables with measurement error has to be known, nor are instrumental variables or replication required.Standard errors can be obtained by omitting individual variables from the model, as in Aitkin (1996).Several examples are given, of normal and Bernoulli response variables.     

Two Adjusted Empirical-Likelihood-Based Methods in Generalized Varying-Coefficient Partially Linear Model

An empirical likelihood method was proposed by Owen and has been extended to many semiparametric and nonparametric models with a continuous response variable. However, there has been less attention focused on the generalized regression model. This article systematically studies two adjusted empirical-likelihood-based methods in the generalized varying-coefficient partially linear models. Based on the popular profile likelihood estimation procedure, the new adjusted empirical likelihood technology for the parameter is established and the resulting statistics are shown to be asymptotically standard chi-square distributed. Further, the adjusted empirical-likelihood-based confidence regions are established, and an efficient adjusted profile empirical-likelihood-based confidence intervals/regions for any components of the parameter, which are of primary interest, is also constructed. Their asymptotic properties are also derived. Some numerical studies are carried out to illustrate the performance of the proposed inference procedures.     

LEVERAGE ADJUSTMENTS FOR DISPERSION MODELLING IN GENERALIZED NONLINEAR MODELS

For normal linear models, it is generally accepted that residual maximum likelihood estimation is appropriate when covariance components require estimation. This paper considers generalized linear models in which both the mean and the dispersion are allowed to depend on unknown parameters and on covariates. For these models there is no closed form equivalent to residual maximum likelihood except in very special cases. Using a modified profile likelihood for the dispersion parameters, an adjusted score vector and adjusted information matrix are found under an asymptotic development that holds as the leverages in the mean model become small. Subsequently, the expectation of the fitted deviances is obtained directly to show that the adjusted score vector is unbiased at least to O(1/n) . Exact results are obtained in the single‐sample case. The results reduce to residual maximum likelihood estimation in the normal linear case.     

Composite Likelihood Estimation for Multivariate Probit Latent Traits Models

Inference in generalized linear mixed models with multivariate random effects is often made cumbersome by the high-dimensional intractable integrals involved in the marginal likelihood. This article presents an inferential methodology based on the marginal composite likelihood approach for the probit latent traits models. This method belonging to the broad class of pseudo-likelihood involves marginal pairs probabilities of the responses which has analytical expression. The different results are illustrated with a simulation study and with an analysis of real data from health related quality of life.     

Learning from a lot: Empirical Bayes for high‐dimensional model‐based prediction

Empirical Bayes is a versatile approach to “learn from a lot” in two ways: first, from a large number of variables and, second, from a potentially large amount of prior information, for example, stored in public repositories. We review applications of a variety of empirical Bayes methods to several well‐known model‐based prediction methods, including penalized regression, linear discriminant analysis, and Bayesian models with sparse or dense priors. We discuss “formal” empirical Bayes methods that maximize the marginal likelihood but also more informal approaches based on other data summaries. We contrast empirical Bayes to cross‐validation and full Bayes and discuss hybrid approaches. To study the relation between the quality of an empirical Bayes estimator and p, the number of variables, we consider a simple empirical Bayes estimator in a linear model setting. We argue that empirical Bayes is particularly useful when the prior contains multiple parameters, which model a priori information on variables termed “co‐data”. In particular, we present two novel examples that allow for co‐data: first, a Bayesian spike‐and‐slab setting that facilitates inclusion of multiple co‐data sources and types and, second, a hybrid empirical Bayes–full Bayes ridge regression approach for estimation of the posterior predictive interval.     

Improved Prediction Intervals and Distribution Functions

Abstract.  The plug-in solution is usually not entirely adequate for computing prediction intervals, as their coverage probability may differ substantially from the nominal value. Prediction intervals with improved coverage probability can be defined by adjusting the plug-in ones, using rather complicated asymptotic procedures or suitable simulation techniques. Other approaches are based on the concept of predictive likelihood for a future random variable. The contribution of this paper is the definition of a relatively simple predictive distribution function giving improved prediction intervals. This distribution function is specified as a first-order unbiased modification of the plug-in predictive distribution function based on the constrained maximum likelihood estimator. Applications of the results to the Gaussian and the generalized extreme-value distributions are presented.     

Composite Marginal Likelihoods to the Normal Bradley-Terry Model

Inference in generalized linear mixed models with crossed random effects is often made cumbersome by the high-dimensional intractable integrals involved in the marginal likelihood. This article presents two inferential approaches based on the marginal composite likelihood for the normal Bradley-Terry model. The two approaches are illustrated by a simulation study to evaluate their performance. Thereafter, the asymptotic variances of the estimated variance component are compared.     

Statistical Inference in Partially Linear Varying-Coefficient Models with Missing Responses at Random

This article considers statistical inference for partially linear varying-coefficient models when the responses are missing at random. We propose a profile least-squares estimator for the parametric component with complete-case data and show that the resulting estimator is asymptotically normal. To avoid to estimate the asymptotic covariance in establishing confidence region of the parametric component with the normal-approximation method, we define an empirical likelihood based statistic and show that its limiting distribution is chi-squared distribution. Then, the confidence regions of the parametric component with asymptotically correct coverage probabilities can be constructed by the result. To check the validity of the linear constraints on the parametric component, we construct a modified generalized likelihood ratio test statistic and demonstrate that it follows asymptotically chi-squared distribution under the null hypothesis. Then, we extend the generalized likelihood ratio technique to the context of missing data. Finally, some simulations are conducted to illustrate the proposed methods.     

Extending conditional likelihood in models for stratified binary data

The conditional likelihood is widely used in logistic regression models with stratified binary data. In particular, it leads to accurate inference for the parameters of interest, which are common to all strata, eliminating stratum-specific nuisance parameters. The modified profile likelihood is an accurate approximation to the conditional likelihood, but has the advantage of being available for general parametric models. Here, we propose the modified profile likelihood as an ideal extension of the conditional likelihood in generalized linear models for binary data, with generic link function. An important feature is that for the implementation we only need standard outputs of routines for generalized linear models. The accuracy of the method is supported by theoretical properties and is confirmed by simulation results.This research was supported by MIUR COFIN 2001-2003.     

Series evaluation of Tweedie exponential dispersion model densities

Exponential dispersion models, which are linear exponential families with a dispersion parameter, are the prototype response distributions for generalized linear models. The Tweedie family comprises those exponential dispersion models with power mean-variance relationships. The normal, Poisson, gamma and inverse Gaussian distributions belong to theTweedie family. Apart from these special cases, Tweedie distributions do not have density functions which can be written in closed form. Instead, the densities can be represented as infinite summations derived from series expansions. This article describes how the series expansions can be summed in an numerically efficient fashion. The usefulness of the approach is demonstrated, but full machine accuracy is shown not to be obtainable using the series expansion method for all parameter values. Derivatives of the density with respect to the dispersion parameter are also derived to facilitate maximum likelihood estimation. The methods are demonstrated on two data examples and compared with with Box-Cox transformations and extended quasi-likelihoood.