Likelihood Prediction for Generalized Linear Mixed Models under Covariate Uncertainty

This article presents the techniques of likelihood prediction for the generalized linear mixed models. Methods of likelihood prediction are explained through a series of examples; from a classical one to more complicated ones. The examples show, in simple cases, that the likelihood prediction (LP) coincides with already known best frequentist practice such as the best linear unbiased predictor. This article outlines a way to deal with the covariate uncertainty while producing predictive inference. Using a Poisson errors-in-variable generalized linear model, it has been shown in certain cases that LP produces better results than already known methods.     

The Gaussian Mixture Dynamic Conditional Correlation Model: Parameter Estimation,Value at Risk Calculation,and Portfolio Selection

A multivariate generalized autoregressive conditional heteroscedasticity model with dynamic conditional correlations is proposed, in which the individual conditional volatilities follow exponential generalized autoregressive conditional heteroscedasticity models and the standardized innovations follow a mixture of Gaussian distributions. Inference on the model parameters and prediction of future volatilities are addressed by both maximum likelihood and Bayesian estimation methods. Estimation of the Value at Risk of a given portfolio and selection of optimal portfolios under the proposed specification are addressed. The good performance of the proposed methodology is illustrated via Monte Carlo experiments and the analysis of the daily closing prices of the Dow Jones and NASDAQ indexes.     

Quasi Bayesian likelihood

If the form of the distribution of data is unknown, the Bayesian method fails in the parametric inference because there is no posterior distribution of the parameter. In this paper, a theoretical framework of Bayesian likelihood is introduced via the Hilbert space method, which is free of the distributions of data and the parameter. The posterior distribution and posterior score function based on given inner products are defined and, consequently, the quasi posterior distribution and quasi posterior score function are derived, respectively, as the projections of the posterior distribution and posterior score function onto the space spanned by given estimating functions. In the space spanned by data, particularly, an explicit representation for the quasi posterior score function is obtained, which can be derived as a projection of the true posterior score function onto this space. The methods of constructing conservative quasi posterior score and quasi posterior log-likelihood are proposed. Some examples are given to illustrate the theoretical results. As an application, the quasi posterior distribution functions are used to select variables for generalized linear models. It is proved that, for linear models, the variable selections via quasi posterior distribution functions are equivalent to the variable selections via the penalized residual sum of squares or regression sum of squares.     

Mixed effects smoothing spline analysis of variance

We propose a general family of nonparametric mixed effects models. Smoothing splines are used to model the fixed effects and are estimated by maximizing the penalized likelihood function. The random effects are generic and are modelled parametrically by assuming that the covariance function depends on a parsimonious set of parameters. These parameters and the smoothing parameter are estimated simultaneously by the generalized maximum likelihood method. We derive a connection between a nonparametric mixed effects model and a linear mixed effects model. This connection suggests a way of fitting a nonparametric mixed effects model by using existing programs. The classical two-way mixed models and growth curve models are used as examples to demonstrate how to use smoothing spline analysis-of-variance decompositions to build nonparametric mixed effects models. Similarly to the classical analysis of variance, components of these nonparametric mixed effects models can be interpreted as main effects and interactions. The penalized likelihood estimates of the fixed effects in a two-way mixed model are extensions of James–Stein shrinkage estimates to correlated observations. In an example three nested nonparametric mixed effects models are fitted to a longitudinal data set.     

Extending conditional likelihood in models for stratified binary data

The conditional likelihood is widely used in logistic regression models with stratified binary data. In particular, it leads to accurate inference for the parameters of interest, which are common to all strata, eliminating stratum-specific nuisance parameters. The modified profile likelihood is an accurate approximation to the conditional likelihood, but has the advantage of being available for general parametric models. Here, we propose the modified profile likelihood as an ideal extension of the conditional likelihood in generalized linear models for binary data, with generic link function. An important feature is that for the implementation we only need standard outputs of routines for generalized linear models. The accuracy of the method is supported by theoretical properties and is confirmed by simulation results.This research was supported by MIUR COFIN 2001-2003.     

Reliability analysis for stress-strength model from a general family of truncated distributions under censored data

Abstract

Under progressive Type-II censoring, inference of stress-strength reliability (SSR) is studied for a general family of lower truncated distributions. When the lifetime models of the strength and stress variables have arbitrary and common parameters, maximum likelihood and pivotal quantities based generalized estimators of SSR are established, respectively. Confidence intervals are constructed based on generalized pivotal quantities and bootstrap technique under different parameter cases as well. In addition, to compare the equivalence of the strength and stress parameters, likelihood ratio testing of interested parameters is provided as a complementary. Simulation studies and two real-life data examples are provided to investigate the performance of proposed methods.     

Profile likelihood in directed graphical models from BUGS output

We present a method for using posterior samples produced by the computer program BUGS (Bayesian inference Using Gibbs Sampling) to obtain approximate profile likelihood functions of parameters or functions of parameters in directed graphical models with incomplete data. The method can also be used to approximate integrated likelihood functions. It is easily implemented and it performs a good approximation. The profile likelihood represents an aspect of the parameter uncertainty which does not depend on the specification of prior distributions, and it can be used as a worthwhile supplement to BUGS that enable us to do both Bayesian and likelihood based analyses in directed graphical models.     

Statistical Inference and Applications of Mixture of Varying Coefficient Models

In this paper, we consider a new mixture of varying coefficient models, in which each mixture component follows a varying coefficient model and the mixing proportions and dispersion parameters are also allowed to be unknown smooth functions. We systematically study the identifiability, estimation and inference for the new mixture model. The proposed new mixture model is rather general, encompassing many mixture models as its special cases such as mixtures of linear regression models, mixtures of generalized linear models, mixtures of partially linear models and mixtures of generalized additive models, some of which are new mixture models by themselves and have not been investigated before. The new mixture of varying coefficient model is shown to be identifiable under mild conditions. We develop a local likelihood procedure and a modified expectation–maximization algorithm for the estimation of the unknown non‐parametric functions. Asymptotic normality is established for the proposed estimator. A generalized likelihood ratio test is further developed for testing whether some of the unknown functions are constants. We derive the asymptotic distribution of the proposed generalized likelihood ratio test statistics and prove that the Wilks phenomenon holds. The proposed methodology is illustrated by Monte Carlo simulations and an analysis of a CO₂‐GDP data set.     

Estimating the variance for heterogeneity in arm‐based network meta‐analysis

Network meta‐analysis can be implemented by using arm‐based or contrast‐based models. Here we focus on arm‐based models and fit them using generalized linear mixed model procedures. Full maximum likelihood (ML) estimation leads to biased trial‐by‐treatment interaction variance estimates for heterogeneity. Thus, our objective is to investigate alternative approaches to variance estimation that reduce bias compared with full ML. Specifically, we use penalized quasi‐likelihood/pseudo‐likelihood and hierarchical (h) likelihood approaches. In addition, we consider a novel model modification that yields estimators akin to the residual maximum likelihood estimator for linear mixed models. The proposed methods are compared by simulation, and 2 real datasets are used for illustration. Simulations show that penalized quasi‐likelihood/pseudo‐likelihood and h‐likelihood reduce bias and yield satisfactory coverage rates. Sum‐to‐zero restriction and baseline contrasts for random trial‐by‐treatment interaction effects, as well as a residual ML‐like adjustment, also reduce bias compared with an unconstrained model when ML is used, but coverage rates are not quite as good. Penalized quasi‐likelihood/pseudo‐likelihood and h‐likelihood are therefore recommended.     

Two Adjusted Empirical-Likelihood-Based Methods in Generalized Varying-Coefficient Partially Linear Model

An empirical likelihood method was proposed by Owen and has been extended to many semiparametric and nonparametric models with a continuous response variable. However, there has been less attention focused on the generalized regression model. This article systematically studies two adjusted empirical-likelihood-based methods in the generalized varying-coefficient partially linear models. Based on the popular profile likelihood estimation procedure, the new adjusted empirical likelihood technology for the parameter is established and the resulting statistics are shown to be asymptotically standard chi-square distributed. Further, the adjusted empirical-likelihood-based confidence regions are established, and an efficient adjusted profile empirical-likelihood-based confidence intervals/regions for any components of the parameter, which are of primary interest, is also constructed. Their asymptotic properties are also derived. Some numerical studies are carried out to illustrate the performance of the proposed inference procedures.     

Bayesian validation assessment of multivariate computational models

Multivariate model validation is a complex decision-making problem involving comparison of multiple correlated quantities, based upon the available information and prior knowledge. This paper presents a Bayesian risk-based decision method for validation assessment of multivariate predictive models under uncertainty. A generalized likelihood ratio is derived as a quantitative validation metric based on Bayes’ theorem and Gaussian distribution assumption of errors between validation data and model prediction. The multivariate model is then assessed based on the comparison of the likelihood ratio with a Bayesian decision threshold, a function of the decision costs and prior of each hypothesis. The probability density function of the likelihood ratio is constructed using the statistics of multiple response quantities and Monte Carlo simulation. The proposed methodology is implemented in the validation of a transient heat conduction model, using a multivariate data set from experiments. The Bayesian methodology provides a quantitative approach to facilitate rational decisions in multivariate model assessment under uncertainty.     

Understanding and Addressing the Unbounded “Likelihood” Problem

The joint probability density function, evaluated at the observed data, is commonly used as the likelihood function to compute maximum likelihood estimates. For some models, however, there exist paths in the parameter space along which this density-approximation likelihood goes to infinity and maximum likelihood estimation breaks down. In all applications, however, observed data are really discrete due to the round-off or grouping error of measurements. The “correct likelihood” based on interval censoring can eliminate the problem of an unbounded likelihood. This article categorizes the models leading to unbounded likelihoods into three groups and illustrates the density-approximation breakdown with specific examples. Although it is usually possible to infer how given data were rounded, when this is not possible, one must choose the width for interval censoring, so we study the effect of the round-off on estimation. We also give sufficient conditions for the joint density to provide the same maximum likelihood estimate as the correct likelihood, as the round-off error goes to zero.     

Prediction Error Estimation Under Bregman Divergence for Non-Parametric Regression and Classification

Abstract.  Prediction error is critical to assess model fit and evaluate model prediction. We propose the cross-validation (CV) and approximated CV methods for estimating prediction error under the Bregman divergence (BD), which embeds nearly all of the commonly used loss functions in the regression, classification procedures and machine learning literature. The approximated CV formulas are analytically derived, which facilitate fast estimation of prediction error under BD. We then study a data-driven optimal bandwidth selector for local-likelihood estimation that minimizes the overall prediction error or equivalently the covariance penalty. It is shown that the covariance penalty and CV methods converge to the same mean-prediction-error-criterion. We also propose a lower-bound scheme for computing the local logistic regression estimates and demonstrate that the algorithm monotonically enhances the target local likelihood and converges. The idea and methods are extended to the generalized varying-coefficient models and additive models.     

Diagnostics for generalized Poisson regression models with errors in variables

In this paper, we develop diagnostic methods for generalized Poisson regression (GPR) models with errors in variables based on the corrected likelihood. The one-step approximations of the estimates in the case-deletion model are given and case-deletion and local influence measures are presented. Meanwhile, based on a corrected score function, the testing statistics for the significance of dispersion parameters in GPR models with measurement errors are investigated. Finally, illustration of our methodology is given through numerical examples.     

LEVERAGE ADJUSTMENTS FOR DISPERSION MODELLING IN GENERALIZED NONLINEAR MODELS

For normal linear models, it is generally accepted that residual maximum likelihood estimation is appropriate when covariance components require estimation. This paper considers generalized linear models in which both the mean and the dispersion are allowed to depend on unknown parameters and on covariates. For these models there is no closed form equivalent to residual maximum likelihood except in very special cases. Using a modified profile likelihood for the dispersion parameters, an adjusted score vector and adjusted information matrix are found under an asymptotic development that holds as the leverages in the mean model become small. Subsequently, the expectation of the fitted deviances is obtained directly to show that the adjusted score vector is unbiased at least to O(1/n) . Exact results are obtained in the single‐sample case. The results reduce to residual maximum likelihood estimation in the normal linear case.     

A note on model selection using information criteria for general linear models estimated using REML

It is common practice to compare the fit of non‐nested models using the Akaike (AIC) or Bayesian (BIC) information criteria. The basis of these criteria is the log‐likelihood evaluated at the maximum likelihood estimates of the unknown parameters. For the general linear model (and the linear mixed model, which is a special case), estimation is usually carried out using residual or restricted maximum likelihood (REML). However, for models with different fixed effects, the residual likelihoods are not comparable and hence information criteria based on the residual likelihood cannot be used. For model selection, it is often suggested that the models are refitted using maximum likelihood to enable the criteria to be used. The first aim of this paper is to highlight that both the AIC and BIC can be used for the general linear model by using the full log‐likelihood evaluated at the REML estimates. The second aim is to provide a derivation of the criteria under REML estimation. This aim is achieved by noting that the full likelihood can be decomposed into a marginal (residual) and conditional likelihood and this decomposition then incorporates aspects of both the fixed effects and variance parameters. Using this decomposition, the appropriate information criteria for model selection of models which differ in their fixed effects specification can be derived. An example is presented to illustrate the results and code is available for analyses using the ASReml‐R package.     

Inflated modified power series distributions with applications

In certain applications involving discrete data, it is sometimes found that X = 0 is observed with a frequency significantly higher than predicted by the assumed model. Zero inflated Poisson, binomial and negative binomial models have been employed in some clinical trials and in some regression analysis problems.

In this paper, we study the zero inflated modified power series distributions (IMPSD) which include among others the generalized Poisson and the generalized negative binomial distributions and hence the Poisson, binomial and negative binomial distributions. The structural properties along with the distribution of the sum of independent IMPSD variables are studied. The maximum likelihood estimation of the parameters of the model is examined and the variance-covariance matrix of the estimators is obtained. Finally, examples are presented for the generalized Poisson distribution to illustrate the results.     

Penalized Maximum Likelihood Estimator for Normal Mixtures

The estimation of the parameters of a mixture of Gaussian densities is considered, within the framework of maximum likelihood. Due to unboundedness of the likelihood function, the maximum likelihood estimator fails to exist. We adopt a solution to likelihood function degeneracy which consists in penalizing the likelihood function. The resulting penalized likelihood function is then bounded over the parameter space and the existence of the penalized maximum likelihood estimator is granted. As original contribution we provide asymptotic properties, and in particular a consistency proof, for the penalized maximum likelihood estimator. Numerical examples are provided in the finite data case, showing the performances of the penalized estimator compared to the standard one.     

Empirical likelihood based variable selection

Information criteria form an important class of model/variable selection methods in statistical analysis. Parametric likelihood is a crucial part of these methods. In some applications such as the generalized linear models, the models are only specified by a set of estimating functions. To overcome the non-availability of well defined likelihood function, the information criteria under empirical likelihood are introduced. Under this setup, we successfully solve the existence problem of the profile empirical likelihood due to the over constraint in variable selection problems. The asymptotic properties of the new method are investigated. The new method is shown to be consistent at selecting the variables under mild conditions. Simulation studies find that the proposed method has comparable performance to the parametric information criteria when a suitable parametric model is available, and is superior when the parametric model assumption is violated. A real data set is also used to illustrate the usefulness of the new method.