A new algorithm to estimate monotone nonparametric link functions and a comparison with parametric approach

The generalized linear model (GLM) is a class of regression models where the means of the response variables and the linear predictors are joined through a link function. Standard GLM assumes the link function is fixed, and one can form more flexible GLM by either estimating the flexible link function from a parametric family of link functions or estimating it nonparametically. In this paper, we propose a new algorithm that uses P-spline for nonparametrically estimating the link function which is guaranteed to be monotone. It is equivalent to fit the generalized single index model with monotonicity constraint. We also conduct extensive simulation studies to compare our nonparametric approach for estimating link function with various parametric approaches, including traditional logit, probit and robit link functions, and two recently developed link functions, the generalized extreme value link and the symmetric power logit link. The simulation study shows that the link function estimated nonparametrically by our proposed algorithm performs well under a wide range of different true link functions and outperforms parametric approaches when they are misspecified. A real data example is used to illustrate the results.     

On selecting parametric link transformation families in generalized linear models

The use of parametric link transformation families in generalized linear models (GLM) has been shown to improve substantially the fit of standard analyses using a fixed link in some data sets (see Czado, 1993, for example). When link and regression parameters are globally orthogonal (Cox and Reid, 1987), then the variance inflation of the regression parameter estimates due to the additional estimation of the link is asymptotically zero. Parameter orthogonality also induces numerical stability which is seen in the reduction of computation time required for the calculation of parameter estimates. This stability remains a desirable property even for inferences which are conditional on a fixed link value. Czado and Santner (1992b), for binomial error, and Czado (1992), for GLMs have shown that only local orthogonality can be achieved in general. This paper provides conditions on the link family to extend the notion of local orthogonality at a point to orthogonality in a neighborhood asymptotically and shows that the resulting links are location and scale invariant. General concepts for the construction of such links are given, and it is shown how they relate to link families proposed in the literature. The ideas are illustrated by two examples.     

Orthogonalizing parametric link transformation families in binary regression analysis

This paper studies the application of the orthogonalization technique of Cox and Reid (1987) to parametric families of link functions used in binary regression analysis. The explicit form of Cox and Reid's condition (4), for orthogonality at a point, is derived for arbitrary link families. This condition is used to determine a transform of a family introduced by Burr (1942) and Prentice (1975, 1976) which is locally orthogonal when the regression parameter is zero. Thus the benefits of having orthogonal parameters are limited to “small” regression effects. The extent to which approximate orthogonality holds for nonzero regression coefficients is investigated for two data sets from the literature. Two specific issues considered are: (1) the ability of orthogonal reparametrization to reduce the variability of the regression parameters caused by estimation of the link parameter and (2) the improved numerical stability (and hence interpretability) of regression estimates corresponding to different link parameters.     

Parametric link modification of both tails in binary regression

Common binary regression models such as logistic or probit regression have been extended to include parametric link transformation families. These binary regression models with parametric link are designed to avoid possible link misspecification and improve fit in some data sets. One and two parameter link families have been proposed in the literature (for a review see Stukel (1988)). However in real data examples published so far only one parameter link families have found to improve the fit significantly. This paper introduces a two parameter link family involving the modification of both tails of the link. An analysis based on computationally tractable Bayesian inference involving Monte Carlo sampling algorithms is presented extending earlier work of Czado (1992, 1993b). Finally, the usefulness of the two tailed link modification will be demonstrated in an example where single tail modification can be significantly improved upon by using a two tailed modification.     

Norm restricted maximum likelihood estimators for binary regression models with parametric link

Parametric link transformation families have shown to be useful in the analysis of binary regression data since they avoid th? problem of link misspecifaction. Inference for these models are commonly based on likelihood methods. Duffy and Santner (1988, 1989) however showed that ordinary logistic maximum likelihood estimators (MLE) have poor mean square error (MSE) behavior in small samples compared to alternative norm restricted estimators. This paper extends these alternative norm restricted estimators to binary regression models with any specified parametric link family. These extended norm restricted MLE's are strongly consistent and efficient under regularity conditions. Finally a simulation study shows that an empiric version of norm restricted MLE's exhibit superior MSE behavior in small samples compared to MLE's with fixed known link.     

Hierarchical Bayes small‐area estimation with an unknown link function

Area‐level unmatched sampling and linking models have been widely used as a model‐based method for producing reliable estimates of small‐area means. However, one practical difficulty is the specification of a link function. In this paper, we relax the assumption of a known link function by not specifying its form and estimating it from the data. A penalized‐spline method is adopted for estimating the link function, and a hierarchical Bayes method of estimating area means is developed using a Markov chain Monte Carlo method for posterior computations. Results of simulation studies comparing the proposed method with a conventional approach based on a known link function are presented. In addition, the proposed method is applied to data from the Survey of Family Income and Expenditure in Japan and poverty rates in Spanish provinces.     

Choice of link and variance function for generalized linear mixed models: a case study with binomial response in proteomics

Abstract

Non-normality is a common phenomenon in data from agricultural and biological research, especially in molecular data (for example; -omics, RNAseq, flow cytometric data, etc.). For over half a century, the leading paradigm called for using analysis of variance (ANOVA) after applying a data transformation. The introduction of generalized linear mixed models (GLMM) provides a new way of analyzing non-normal data. Selecting an apt link function in GLMM can be quite influential, however, and is as critical as selecting an appropriate transformation for ANOVA. In this paper, we assess the performance of different parametric link families available in literature. Then, we propose a new estimation method for selecting an appropriate link function with a suitable variance function in a quasi-likelihood framework. We apply these methods to a proteomics data set, showing that GLMMs provide a very flexible framework for analyzing these kinds of data.     

Properties of Bayes Factors Based on Test Statistics

Abstract.  This article examines the consistency, interpretation and application of Bayes factors constructed from standard test statistics. Primary conclusions are that Bayes factors based on multinomial and normal test statistics are consistent for suitable choices of the hyperparameters used to specify alternative hypotheses, and that such constructions can be extended to obtain consistent Bayes factors based on likelihood ratio statistics. A connection between Bayes factors based on likelihood ratio statistics and the Bayesian information criterion is exposed, as is a connection between Bayes factors based on F statistics and parametric Bayes factors based on normal-inverse gamma models. Similarly, Bayes factors based on chi-squared statistics for multinomial data are shown to provide accurate approximations to Bayes factors based on multinomial/Dirichlet models. An illustration of how the simple form of these Bayes factors can be exploited to generate easily interpretable summaries of the experimental 'weight of evidence' is provided.     

Model influence functions based on mixtures

Influence functions are considered as diagnostics for model departures in parametric Bayesian inference. A baseline model density is expressed as a mixture; then the mixing distribution is perturbed. This is designed to engender perturbations which are plausible a priori. The influence of perturbations is measured for both Bayes estimates and their associated posterior expected losses. To assess the plausibility of perturbations a posteriori, an additional influence function is constructed for the Bayes factor comparing the perturbed and baseline models. The effect of perturbation on various estimands is incorporated in the analysis.     

ON CONFIDENCE REGIONS OF EMBEDDED MODELS IN REGULAR PARAMETRIC FAMILIES (A GEOMETRIC APPROACH)

Regular parametric families are commonly encountered in statistical problems (e.g. Cox & Hinkley, 1974). In this paper, we propose a differential geometric framework for the embedded models in these families. Our framework may be regarded as an extension of that presented by Bates & Watts (1980) for nonlinear regression models. As an application, we use this geometric framework to derive three kinds of improved approximate confidence regions for the parameter and parameter subsets in terms of curvatures. The results obtained by Hamilton et al. (1982) and Hamilton (1986) are extended to embedded models in regular parametric families.     

Approximate Bayes estimation of the parameters and reliability function of a mixture of two inverse Weibull distributions under Type-2 censoring

The main goal of this paper is to develop the approximate Bayes estimation of the five-dimensional vector of the parameters and reliability function of a mixture of two inverse Weibull distributions (MTIWD) under Type-2 censoring. Usually, the posterior distribution is complicated under the scheme of Type-2 censoring and the integrals that are involved cannot be obtained in a simple explicit form. In this study, we use Lindley's [Approximate Bayesian method, Trabajos Estadist. 31 (1980), pp. 223–237] approximate form of Bayes estimation in the case of an MTIWD under Type-2 censoring. Later, we calculate the estimated risks (ERs) of the Bayes estimates and compare them with the corresponding ERs of the maximum-likelihood estimates through Monte Carlo simulation. Finally, we analyse a real data set using the findings.     

Bayes estimation of Gompertz distribution parameters and acceleration factor under partially accelerated life tests with type-I censoring

In this paper, the Bayesian approach is applied to the estimation problem in the case of step stress partially accelerated life tests with two stress levels and type-I censoring. Gompertz distribution is considered as a lifetime model. The posterior means and posterior variances are derived using the squared-error loss function. The Bayes estimates cannot be obtained in explicit forms. Approximate Bayes estimates are computed using the method of Lindley [D.V. Lindley, Approximate Bayesian methods, Trabajos Estadistica 31 (1980), pp. 223–237]. The advantage of this proposed method is shown. The approximate Bayes estimates obtained under the assumption of non-informative priors are compared with their maximum likelihood counterparts using Monte Carlo simulation.     

A Bayesian estimation of lag lengths in distributed lag models

Dynamic regression models are widely used because they express and model the behaviour of a system over time. In this article, two dynamic regression models, the distributed lag (DL) model and the autoregressive distributed lag model, are evaluated focusing on their lag lengths. From a classical statistics point of view, there are various methods to determine the number of lags, but none of them are the best in all situations. This is a serious issue since wrong choices will provide bad estimates for the effects of the regressors on the response variable. We present an alternative for the aforementioned problems by considering a Bayesian approach. The posterior distributions of the numbers of lags are derived under an improper prior for the model parameters. The fractional Bayes factor technique [A. O'Hagan, Fractional Bayes factors for model comparison (with discussion), J. R. Statist. Soc. B 57 (1995), pp. 99–138] is used to handle the indeterminacy in the likelihood function caused by the improper prior. The zero-one loss function is used to penalize wrong decisions. A naive method using the specified maximum number of DLs is also presented. The proposed and the naive methods are verified using simulation data. The results are promising for the method we proposed. An illustrative example with a real data set is provided.     

Probability model choice in single samples from exponential families using Poisson log-linear modelling,and model comparison using Bayes and posterior Bayes factors

This paper describes a method due to Lindsey (1974a) for fitting different exponential family distributions for a single population to the same data, using Poisson log-linear modelling of the density or mass function. The method is extended to Efron's (1986) double exponential family, giving exact ML estimation of the two parameters not easily achievable directly. The problem of comparing the fit of the non-nested models is addressed by both Bayes and posterior Bayes factors (Aitkin, 1991). The latter allow direct comparisons of deviances from the fitted distributions.     

Sequential Monte Carlo on large binary sampling spaces

A Monte Carlo algorithm is said to be adaptive if it automatically calibrates its current proposal distribution using past simulations. The choice of the parametric family that defines the set of proposal distributions is critical for good performance. In this paper, we present such a parametric family for adaptive sampling on high dimensional binary spaces. A practical motivation for this problem is variable selection in a linear regression context. We want to sample from a Bayesian posterior distribution on the model space using an appropriate version of Sequential Monte Carlo. Raw versions of Sequential Monte Carlo are easily implemented using binary vectors with independent components. For high dimensional problems, however, these simple proposals do not yield satisfactory results. The key to an efficient adaptive algorithm are binary parametric families which take correlations into account, analogously to the multivariate normal distribution on continuous spaces. We provide a review of models for binary data and make one of them work in the context of Sequential Monte Carlo sampling. Computational studies on real life data with about a hundred covariates suggest that, on difficult instances, our Sequential Monte Carlo approach clearly outperforms standard techniques based on Markov chain exploration.     

Robust empirical Bayes tests for continuous distributions

In this paper, we study the empirical Bayes two-action problem under linear loss function. Upper bounds on the regret of empirical Bayes testing rules are investigated. Previous results on this problem construct empirical Bayes tests using kernel type estimators of nonparametric functionals. Further, they have assumed specific forms, such as the continuous one-parameter exponential family for {F_θ:θΩ}, for the family of distributions of the observations. In this paper, we present a new general approach of establishing upper bounds (in terms of rate of convergence) of empirical Bayes tests for this problem. Our results are given for any family of continuous distributions and apply to empirical Bayes tests based on any type of nonparametric method of functional estimation. We show that our bounds are very sharp in the sense that they reduce to existing optimal or nearly optimal rates of convergence when applied to specific families of distributions.     

A Hybrid Approximation Bayesian Test of Variance Components for Longitudinal Data

The test of variance components of possibly correlated random effects in generalized linear mixed models (GLMMs) can be used to examine if there exists heterogeneous effects. The Bayesian test with Bayes factors offers a flexible method. In this article, we focus on the performance of Bayesian tests under three reference priors and a conjugate prior: an approximate uniform shrinkage prior, modified approximate Jeffreys' prior, half-normal unit information prior and Wishart prior. To compute Bayes factors, we propose a hybrid approximation approach combining a simulated version of Laplace's method and importance sampling techniques to test the variance components in GLMMs.     

Generalized linear models with covariate measurement error and unknown link function

Generalized linear models (GLMs) with error-in-covariates are useful in epidemiological research due to the ubiquity of non-normal response variables and inaccurate measurements. The link function in GLMs is chosen by the user depending on the type of response variable, frequently the canonical link function. When covariates are measured with error, incorrect inference can be made, compounded by incorrect choice of link function. In this article we propose three flexible approaches for handling error-in-covariates and estimating an unknown link simultaneously. The first approach uses a fully Bayesian (FB) hierarchical framework, treating the unobserved covariate as a latent variable to be integrated over. The second and third are approximate Bayesian approach which use a Laplace approximation to marginalize the variables measured with error out of the likelihood. Our simulation results show support that the FB approach is often a better choice than the approximate Bayesian approaches for adjusting for measurement error, particularly when the measurement error distribution is misspecified. These approaches are demonstrated on an application with binary response.     

Power-linear models for incomplete contingency tables with nonignorable non-responses

For categorical data exhibiting nonignorable non-responses, it is well known that maximum likelihood (ML) estimates with a boundary solution are implausible and do not provide a perfect fit to the observed data even for saturated models. We provide the conditions under which ML estimates for the generalized linear model (GLM) with the usual log/logit link function have a boundary solution. These conditions introduce a new GLM with appropriately defined power link functions where its ML estimates resolve the problems arising from a boundary solution and offer useful statistics for identifying the non-response mechanism. This model is applied to a real dataset and compared with Bayesian models.     

Linear Models with Doubly Exchangeable Distributed Errors

We study the general linear model (GLM) with doubly exchangeable distributed error for m observed random variables. The doubly exchangeable general linear model (DEGLM) arises when the m-dimensional error vectors are “doubly exchangeable,” jointly normally distributed, which is a much weaker assumption than the independent and identically distributed error vectors as in the case of GLM or classical GLM (CGLM). We estimate the parameters in the model and also find their distributions. We show that the tests of intercept and slope are possible in DEGLM as a particular case using parametric bootstrap as well as multivariate Satterthwaite approximation.