Estimation and prediction for spatial generalized linear mixed models using high order Laplace approximation

Estimation and prediction in generalized linear mixed models are often hampered by intractable high dimensional integrals. This paper provides a framework to solve this intractability, using asymptotic expansions when the number of random effects is large. To that end, we first derive a modified Laplace approximation when the number of random effects is increasing at a lower rate than the sample size. Second, we propose an approximate likelihood method based on the asymptotic expansion of the log-likelihood using the modified Laplace approximation which is maximized using a quasi-Newton algorithm. Finally, we define the second order plug-in predictive density based on a similar expansion to the plug-in predictive density and show that it is a normal density. Our simulations show that in comparison to other approximations, our method has better performance. Our methods are readily applied to non-Gaussian spatial data and as an example, the analysis of the rhizoctonia root rot data is presented.     

广义卡方型混合分布的鞍点逼近

广义卡方型混合分布在许多非参数检验问题中有着广泛运用。通常采用正态分布近似这类分布，但是在非大样本的情况下，正态近似的效果并不理想。运用鞍点逼近技术近似广义卡方型混合随机变量的密度函数和分布函数，并且与正态近似方法以及卡方近似方法进行了比较。模拟表明鞍点逼近效果要优于其余两种方法，特别是密度函数尾部区域。     

Saddlepoint approximations to tail expectations under non-Gaussian base distributions: option pricing applications

The saddlepoint approximation formulas provide versatile tools for analytic approximation of the tail expectation of a random variable by approximating the complex Laplace integral of the tail expectation expressed in terms of the cumulant generating function of the random variable. We generalize the saddlepoint approximation formulas for calculating tail expectations from the usual Gaussian base distribution to an arbitrary base distribution. Specific discussion is presented on the criteria of choosing the base distribution that fits better the underlying distribution. Numerical performance and comparison of accuracy are made among different saddlepoint approximation formulas. Improved accuracy of the saddlepoint approximations to tail expectations is revealed when proper base distributions are chosen. We also demonstrate enhanced accuracy of the generalized saddlepoint approximation formulas under non-Gaussian base distributions in pricing European options on continuous integrated variance under the Heston stochastic volatility model.     

DISTRIBUTION OF THE LEAST SQUARES ESTIMATOR IN A FIRST-ORDER AUTOREGRESSIVE MODEL

This paper investigates the finite sample distribution of the least squares estimator of the autoregressive parameter in a first-order autoregressive model. A uniform asymptotic expansion for the distribution applicable to both stationary and nonstationary cases is obtained. Accuracy of the approximation to the distribution by a first few terms of this expansion is then investigated. It is found that the leading term of this expansion approximates well the distribution. The approximation is, in almost all cases, accurate to the second decimal place throughout the distribution. In the literature, there exist a number of approximations to this distribution which are specifically designed to apply in some special cases of this model. The present approximation compares favorably with those approximations and in fact, its accuracy is, with almost no exception, as good as or better than these other approximations. Convenience of numerical computations seems also to favor the present approximations over the others. An application of the finding is illustrated with examples.     

Asymptotic approximations to the distribution of Kendall's sample τ

This paper provides a saddlepoint approximation to the distribution of the sample version of Kendall's τ, which is a measure of association between two samples. The saddlepoint approximation is compared with the Edgeworth and the normal approximations, and with the bootstrap resampling distribution. A numerical study shows that with small sample sizes the saddlepoint approximation outperforms both the normal and the Edgeworth approximations. This paper gives also an analytical comparison between approximated and exact cumulants of the sample Kendall's τ when the two samples are independent.     

Approximate Small-Sample Tests of Fixed Effects in Nonlinear Mixed Models

Nonlinear mixed effect models have been studied extensively over several decades, particularly in pharmacokinetic and pharmacodynamic applications. Here, we focus on investigating the performance of commonly applied tests of linear hypotheses about the fixed effect parameters under different approximations to the likelihood function and to the estimated covariance matrix of the estimators. Included are the first-order approximation (FIRO), first-order conditional approximation (FOCE), and Gaussian quadrature approximation (AGQ) estimation methods. There is no straightforward way to mimic the approximations and adjustments taken in linear mixed models, such as the Kackar–Harville–Jeske–Kenward–Roger approach. By simulations, we illustrate the accuracy of p-values for the tests considered here. The observed results indicate that FOCE and AGQ estimation methods outperform FIRO. The test with an adjustment coefficient that takes into consideration the number of sampling units and the number of fixed effect parameters (Gallant-type) seems to perform closest to desirable even for small-sample sizes.     

Some asymptotic results on generalized penalized spline smoothing

Summary.  The paper discusses asymptotic properties of penalized spline smoothing if the spline basis increases with the sample size. The proof is provided in a generalized smoothing model allowing for non-normal responses. The results are extended in two ways. First, assuming the spline coefficients to be a priori normally distributed links the smoothing framework to generalized linear mixed models. We consider the asymptotic rates such that the Laplace approximation is justified and the resulting fits in the mixed model correspond to penalized spline estimates. Secondly, we make use of a fully Bayesian viewpoint by imposing an a priori distribution on all parameters and coefficients. We argue that with the postulated rates at which the spline basis dimension increases with the sample size the posterior distribution of the spline coefficients is approximately normal. The validity of this result is investigated in finite samples by comparing Markov chain Monte Carlo results with their asymptotic approximation in a simulation study.     

Laplace approximations for censored linear regression models

We consider approximate Bayesian inference about scalar parameters of linear regression models with possible censoring. A second-order expansion of their Laplace posterior is seen to have a simple and intuitive form for logconcave error densities with nondecreasing hazard functions. The accuracy of the approximations is assessed for normal and Gumbel errors when the number of regressors increases with sample size. Perturbations of the prior and the likelihood are seen to be easily accommodated within our framework. Links with the work of DiCiccio et al. (1990) and Viveros and Sprott (1987) extend the applicability of our results to conditional frequentist inference based on likelihood-ratio statistics.     

General Saddlepoint Approximation for Testing Separate Location-Scale Families of Hypotheses

For testing separate families of hypotheses, the likelihood ratio test does not have the usual asymptotic properties. This paper considers the asymptotic distribution of the ratio of maximized likelihoods (RML) statistic in the special case of testing separate scale or location-scale families of distributions. We derive saddlepoint approximations to the density and tail probabilities of the log of the RML statistic. These approximations are based on the expansion of the log of the RML statistic up to the second order, which is shown not to depend on the location and scale parameters. The resulting approximations are applied in several cases, including normal versus Laplace, normal versus Cauchy, and Weibull versus log-normal. Our results show that the saddlepoint approximations are satisfactory, even for fairly small sample sizes, and are more accurate than normal approximations and Edgeworth approximations, especially for tail probabilities that are the values of main interest in hypothesis testing problems.     

Modified Wald statistics for generalized linear models

Summary: Wald statistics in generalized linear models are asymptotically ² distributed. The asymptotic chi–squared law of the corresponding quadratic form shows disadvantages with respect to the approximation of the finite–sample distribution. It is shown by means of a comprehensive simulation study that improvements can be achieved by applying simple finite–sample size approximations to the distribution of the quadratic form in generalized linear models. These approximations are based on a ² distribution with an estimated degree of freedom that generalizes an approach by Patnaik and Pearson. Simulation studies confirm that nominal level is maintained with higher accuracy compared to the Wald statistics.     

Enhancement for two commonly-used approximations for the inverse cumulative function of the normal distribution

Two commonly used approximations for the inverse distribution function of the normal distribution are Schmeiser's and Shore's. Both approximations are based on a power transformation of either the cumulative density function (CDF) or a simple function of it. In this note we demonstrate, that if these approximations are presented in the form of the classical one-parameter Box-Cox transformation, and the exponent of the transformation is expressed as a simple function of the CDF, then the accuracy of both approximations may be considerably enhanced, without losing much in algebraic simplicity. Since both approximations are special cases of more general four-parameter systems of distributions, the results presented here indicate that the accuracy of the latter, when used to represent non-normal density functions, may also be considerably enhanced.     

Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm

Two new implementations of the EM algorithm are proposed for maximum likelihood fitting of generalized linear mixed models. Both methods use random (independent and identically distributed) sampling to construct Monte Carlo approximations at the E-step. One approach involves generating random samples from the exact conditional distribution of the random effects (given the data) by rejection sampling, using the marginal distribution as a candidate. The second method uses a multivariate t importance sampling approximation. In many applications the two methods are complementary. Rejection sampling is more efficient when sample sizes are small, whereas importance sampling is better with larger sample sizes. Monte Carlo approximation using random samples allows the Monte Carlo error at each iteration to be assessed by using standard central limit theory combined with Taylor series methods. Specifically, we construct a sandwich variance estimate for the maximizer at each approximate E-step. This suggests a rule for automatically increasing the Monte Carlo sample size after iterations in which the true EM step is swamped by Monte Carlo error. In contrast, techniques for assessing Monte Carlo error have not been developed for use with alternative implementations of Monte Carlo EM algorithms utilizing Markov chain Monte Carlo E-step approximations. Three different data sets, including the infamous salamander data of McCullagh and Nelder, are used to illustrate the techniques and to compare them with the alternatives. The results show that the methods proposed can be considerably more efficient than those based on Markov chain Monte Carlo algorithms. However, the methods proposed may break down when the intractable integrals in the likelihood function are of high dimension.     

Saddlepoint Approximation for Sample and Bootstrap Quantiles

The paper gives the saddlepoint approximation for the distribution function of the sample quantile. A comparison of the saddlepoint approximations for the distribution functions of the sample quantile and the bootstrap quantile shows that the error of the bootstrap approximation to the distribution of the sample quantile obtained by Singh (1981) as an absolute error is actually a relative error.     

Power and Sample Size for Fixed-Effects Inference in Reversible Linear Mixed Models

ABSTRACT

Despite the popularity of the general linear mixed model for data analysis, power and sample size methods and software are not generally available for commonly used test statistics and reference distributions. Statisticians resort to simulations with homegrown and uncertified programs or rough approximations which are misaligned with the data analysis. For a wide range of designs with longitudinal and clustering features, we provide accurate power and sample size approximations for inference about fixed effects in the linear models we call reversible. We show that under widely applicable conditions, the general linear mixed-model Wald test has noncentral distributions equivalent to well-studied multivariate tests. In turn, exact and approximate power and sample size results for the multivariate Hotelling–Lawley test provide exact and approximate power and sample size results for the mixed-model Wald test. The calculations are easily computed with a free, open-source product that requires only a web browser to use. Commercial software can be used for a smaller range of reversible models. Simple approximations allow accounting for modest amounts of missing data. A real-world example illustrates the methods. Sample size results are presented for a multicenter study on pregnancy. The proposed study, an extension of a funded project, has clustering within clinic. Exchangeability among the participants allows averaging across them to remove the clustering structure. The resulting simplified design is a single-level longitudinal study. Multivariate methods for power provide an approximate sample size. All proofs and inputs for the example are in the supplementary materials (available online).     

Approximate Bayesian Inference for Survival Models

Abstract. Bayesian analysis of time‐to‐event data, usually called survival analysis, has received increasing attention in the last years. In Cox‐type models it allows to use information from the full likelihood instead of from a partial likelihood, so that the baseline hazard function and the model parameters can be jointly estimated. In general, Bayesian methods permit a full and exact posterior inference for any parameter or predictive quantity of interest. On the other side, Bayesian inference often relies on Markov chain Monte Carlo (MCMC) techniques which, from the user point of view, may appear slow at delivering answers. In this article, we show how a new inferential tool named integrated nested Laplace approximations can be adapted and applied to many survival models making Bayesian analysis both fast and accurate without having to rely on MCMC‐based inference.     

The Role of Posterior Densities in Latent Variable Models for Ordinal Data

In latent variable models, problems related to the integration of the likelihood function arise since analytical solutions do not exist. Laplace and Adaptive Gauss-Hermite (AGH) approximations have been discussed as good approximating methods. Their performance relies on the assumption of normality of the posterior density of the latent variables, but, in small samples, this is not necessarily assured. Here, we analyze how the shape of the posterior densities varies as function of the model parameters, and we investigate its influence on the performance of AGH and of the Laplace approximation.     

A comparison of two approaches for power and sample size calculations in logistic regression models

Whittemore (1981) proposed an approach for calculating the sample size needed to test hypotheses with specified significance and power against a given alternative for logistic regression with small response probability. Based on the distribution of covariate, which could be either discrete or continuous, this approach first provides a simple closed-form approximation to the asymptotic covariance matrix of the maximum likelihood estimates, and then uses it to calculate the sample size needed to test a hypothesis about the parameter. Self et al. (1992) described a general approach for power and sample size calculations within the framework of generalized linear models, which include logistic regression as a special case. Their approach is based on an approximation to the distribution of the likelihood ratio statistic. Unlike the Whittemore approach, their approach is not limited to situations of small response probability. However, it is restricted to models with a finite number of covariate configurations. This study compares these two approaches to see how accurate they would be for the calculations of power and sample size in logistic regression models with various response probabilities and covariate distributions. The results indicate that the Whittemore approach has a slight advantage in achieving the nominal power only for one case with small response probability. It is outperformed for all other cases with larger response probabilities. In general, the approach proposed in Self et al. (1992) is recommended for all values of the response probability. However, its extension for logistic regression models with an infinite number of covariate configurations involves an arbitrary decision for categorization and leads to a discrete approximation. As shown in this paper, the examined discrete approximations appear to be sufficiently accurate for practical purpose.     

An approximation to the distribution of the sample variance

A simple three-moment approximation is introduced for the distribution of the sample variance. Comparisons are given with other approximations discussed by Tan and Wong (1977) and with an approximation developed very recently by Mudholkar and Trivedi (1981).     

Bias Correction in the Dynamic Panel Data Model with a Nonscalar Disturbance Covariance Matrix

Approximation formulae are developed for the bias of ordinary and generalized Least Squares Dummy Variable (LSDV) estimators in dynamic panel data models. Results from Kiviet [Kiviet, J. F. (1995), on bias, inconsistency, and efficiency of various estimators in dynamic panel data models, J. Econometrics68:53-78; Kiviet, J. F. (1999), Expectations of expansions for estimators in a dynamic panel data model: some results for weakly exogenous regressors, In: Hsiao, C., Lahiri, K., Lee, L-F., Pesaran, M. H., eds., Analysis of Panels and Limited Dependent Variables, Cambridge: Cambridge University Press, pp. 199-225] are extended to higher-order dynamic panel data models with general covariance structure. The focus is on estimation of both short- and long-run coefficients. The results show that proper modelling of the disturbance covariance structure is indispensable. The bias approximations are used to construct bias corrected estimators which are then applied to quarterly data from 14 European Union countries. Money demand functions for M1, M2 and M3 are estimated for the EU area as a whole for the period 1991: I-1995: IV. Significant spillovers between countries are found reflecting the dependence of domestic money demand on foreign developments. The empirical results show that in general plausible long-run effects are obtained by the bias corrected estimators. Moreover, finite sample bias, although of moderate magnitude, is present underlining the importance of more refined estimation techniques. Also the efficiency gains by exploiting the heteroscedasticity and cross-correlation patterns between countries are sometimes considerable.     

Pareto Sampling versus Sampford and Conditional Poisson Sampling

Abstract.  Pareto sampling was introduced by Rosén in the late 1990s. It is a simple method to get a fixed size π ps sample though with inclusion probabilities only approximately as desired. Sampford sampling, introduced by Sampford in 1967, gives the desired inclusion probabilities but it may take time to generate a sample. Using probability functions and Laplace approximations, we show that from a probabilistic point of view these two designs are very close to each other and asymptotically identical. A Sampford sample can rapidly be generated in all situations by letting a Pareto sample pass an acceptance–rejection filter. A new very efficient method to generate conditional Poisson ( CP ) samples appears as a byproduct. Further, it is shown how the inclusion probabilities of all orders for the Pareto design can be calculated from those of the CP design. A new explicit very accurate approximation of the second-order inclusion probabilities, valid for several designs, is presented and applied to get single sum type variance estimates of the Horvitz–Thompson estimator.