ASYMPTOTIC LIKELIHOOD APPROXIMATIONS USING A PARTIAL LAPLACE APPROXIMATION

Elimination of a nuisance variable is often non‐trivial and may involve the evaluation of an intractable integral. One approach to evaluate these integrals is to use the Laplace approximation. This paper concentrates on a new approximation, called the partial Laplace approximation, that is useful when the integrand can be partitioned into two multiplicative disjoint functions. The technique is applied to the linear mixed model and shows that the approximate likelihood obtained can be partitioned to provide a conditional likelihood for the location parameters and a marginal likelihood for the scale parameters equivalent to restricted maximum likelihood (REML). Similarly, the partial Laplace approximation is applied to the t‐distribution to obtain an approximate REML for the scale parameter. A simulation study reveals that, in comparison to maximum likelihood, the scale parameter estimates of the t‐distribution obtained from the approximate REML show reduced bias.     

Gauss–Christoffel quadrature for inverse regression: applications to computer experiments

Sufficient dimension reduction (SDR) provides a framework for reducing the predictor space dimension in statistical regression problems. We consider SDR in the context of dimension reduction for deterministic functions of several variables such as those arising in computer experiments. In this context, SDR can reveal low-dimensional ridge structure in functions. Two algorithms for SDR—sliced inverse regression (SIR) and sliced average variance estimation (SAVE)—approximate matrices of integrals using a sliced mapping of the response. We interpret this sliced approach as a Riemann sum approximation of the particular integrals arising in each algorithm. We employ the well-known tools from numerical analysis—namely, multivariate numerical integration and orthogonal polynomials—to produce new algorithms that improve upon the Riemann sum-based numerical integration in SIR and SAVE. We call the new algorithms Lanczos–Stieltjes inverse regression (LSIR) and Lanczos–Stieltjes average variance estimation (LSAVE) due to their connection with Stieltjes’ method—and Lanczos’ related discretization—for generating a sequence of polynomials that are orthogonal with respect to a given measure. We show that this approach approximates the desired integrals, and we study the behavior of LSIR and LSAVE with two numerical examples. The quadrature-based LSIR and LSAVE eliminate the first-order algebraic convergence rate bottleneck resulting from the Riemann sum approximation, thus enabling high-order numerical approximations of the integrals when appropriate. Moreover, LSIR and LSAVE perform as well as the best-case SIR and SAVE implementations (e.g., adaptive partitioning of the response space) when low-order numerical integration methods (e.g., simple Monte Carlo) are used.

     

BARTLETT CORRECTED LIKELIHOOD RATIO TESTS IN LOCATION-SCALE NONLINEAR MODELS

The paper derives Bartlett corrections for improving the chisquare approximation to the likelihood ratio statistics in a class of location-scale family of distributions, which encompasses the elliptical family of distributions and also asymmetric distributions such as the extreme value distributions. We present, in matrix notation, a Bartlett corrected likelihood ratio statistic for testing that a subset of the nonlinear regression coefficients in this class of models equals a given vector of constants. The formulae derived are simple enough to be used analytically to obtain several Bartlett corrections in a variety of important models. We show that these formulae generalize a number of previously published results. We also present simulation results comparing the sizes and powers of the usual likelihood ratio tests and their Bartlett corrected versions when the scale parameter is considered known and when this parameter is uncorrectly specified.     

Hypothesis testing on the common location parameter of several shifted exponential distributions: A note

This note deals with hypothesis testing on the common location parameter of several shifted exponential distributions with unknown and possibly unequal scale parameters. No exact test is available for the above mentioned problem; and one does not have the luxury of applying the asymptotic Chi-square test for the likelihood ratio test statistic since the distributions do not satisfy the usual regularity conditions. Therefore, we have proposed a few approximate tests based on the parametric bootstrap method which appear to work well even for small samples in terms of attaining the level. Powers of the proposed tests have been provided along with a recommendation of their usage.     

Approximations to the p-values of tests for a change-point under non-standard conditions

Three test statistics for a change-point in a linear model, variants of those considered by Andrews and Ploberger [Optimal tests when a nusiance parameter is present only under the alternative. Econometrica. 1994;62:1383–1414]: the sup-likelihood ratio (LR) statistic; a weighted average of the exponential of LR-statistics and a weighted average of LR-statistics, are studied. Critical values for the statistics with time trend regressors, obtained via simulation, are found to vary considerably, depending on conditions on the error terms. The performance of the bootstrap in approximating p-values of the distributions is assessed in a simulation study. A sample approximation to asymptotic analytical expressions extending those of Kim and Siegmund [The likelihood ratio test for a change-point in simple linear regression. Biometrika. 1989;76:409–423] in the case of the sup-LR test is also assessed. The approximations and bootstrap are applied to the Quandt data [The estimation of a parameter of a linear regression system obeying two separate regimes. J Amer Statist Assoc. 1958;53:873–880] and real data concerning a change-point in oxygen uptake during incremental exercise testing and the bootstrap gives reasonable results.     

LR cointegration tests when some cointegrating relations are known

This paper considers the asymptotic analysis of the likelihood ratio (LR), cointegration (CI) rank test in vector autoregressive models (VAR) when some CI vectors are known and fixed. It is shown that the limit law is free of nuisance parameters. In the case of LR tests against the alternative of completely unrestricted CI space, the limit law can be expressed as the convolution of known distributions. This deconvolution is employed to approximate the quantiles of the distribution, without resorting to new simulations.     

Likelihood Based Inference in Non-linear Regression Models Using the p* and R* Approach

We develop second order asymptotic results for likelihood-based inference in Gaussian non-linear regression models. We provide an approximation to the conditional density of the maximum likelihood estimator given an approximate ancillary statistic (the affine ancillary). From this approximation, we derive a statistic to test an hypothesis on one component of the parameter. This test statistic is an adjustment of the signed log-likelihood ratio statistic. The distributional approximations (for the maximum likelihood estimator and for the test statistic) are of second order in large deviation regions.     

Estimation of the scale parameter of the half-logistic distribution under progressively type II censored sample

Based on a progressively type II censored sample, the maximum likelihood and Bayes estimators of the scale parameter of the half-logistic distribution are derived. However, since the maximum likelihood estimator (MLE) and Bayes estimator do not exist in an explicit form for the scale parameter, we consider a simple method of deriving an explicit estimator by approximating the likelihood function and derive the asymptotic variances of MLE and approximate MLE. Also, an approximation based on the Laplace approximation (Tierney and Kadane in J Am Stat Assoc 81:82–86, 1986) and importance sampling methods are used for obtaining the Bayes estimator. In order to compare the performance of the MLE, approximate MLE and Bayes estimates of the scale parameter, we use Monte Carlo simulation.     

Empirical Likelihood for Censored Linear Regression

In this paper we investigate the empirical likelihood method in a linear regression model when the observations are subject to random censoring. An empirical likelihood ratio for the slope parameter vector is defined and it is shown that its limiting distribution is a weighted sum of independent chi-square distributions. This reduces to the empirical likelihood to the linear regression model first studied by Owen (1991) if there is no censoring present. Some simulation studies are presented to compare the empirical likelihood method with the normal approximation based method proposed in Lai et al. (1995). It was found that the empirical likelihood method performs much better than the normal approximation method.     

Laplace’s approximation for relative risk frailty models

Relative risk frailty models are used extensively in analyzing clustered and/or recurrent time-to-event data. In this paper, Laplace’s approximation for integrals is applied to marginal distributions of data arising from parametric relative risk frailty models. Under regularity conditions, the approximate maximum likelihood estimators (MLE) are consistent with a rate of convergence that depends on both the number of subjects and number of members per subject. We compare the approximate MLE against alternative estimators using limited simulation and demonstrate the utility of Laplace’s approximation approach by analyzing U.S. patient waiting time to deceased kidney transplant data.     

Exact likelihood ratio and score confidence intervals for the binomial proportion

Many methods are available for computing a confidence interval for the binomial parameter, and these methods differ in their operating characteristics. It has been suggested in the literature that the use of the exact likelihood ratio (LR) confidence interval for the binomial proportion should be considered. This paper provides an evaluation of the operating characteristics of the two‐sided exact LR and exact score confidence intervals for the binomial proportion and compares these results to those for three other methods that also strictly maintain nominal coverage: Clopper‐Pearson, Blaker, and Casella. In addition, the operating characteristics of the two‐sided exact LR method and exact score method are compared with those of the corresponding asymptotic methods to investigate the adequacy of the asymptotic approximation. Copyright © 2013 John Wiley & Sons, Ltd.     

Estimation of the scale parameter of the half-logistic distribution with multiply type II censored sample

In this paper, we consider the maximum likelihood and Bayes estimation of the scale parameter of the half-logistic distribution based on a multiply type II censored sample. However, the maximum likelihood estimator(MLE) and Bayes estimator do not exist in an explicit form for the scale parameter. We consider a simple method of deriving an explicit estimator by approximating the likelihood function and discuss the asymptotic variances of MLE and approximate MLE. Also, an approximation based on the Laplace approximation (Tierney & Kadane, 1986) is used to obtain the Bayes estimator. In order to compare the MLE, approximate MLE and Bayes estimates of the scale parameter, Monte Carlo simulation is used.     

Tests for the skewness parameter of two-piece double exponential distribution in the presence of nuisance parameters

In this paper, tests for the skewness parameter of the two-piece double exponential distribution are derived when the location parameter is unknown. Classical tests like Neyman structure test and likelihood ratio test (LRT), that are generally used to test hypotheses in the presence of nuisance parameters, are not feasible for this distribution since the exact distributions of the test statistics become very complicated. As an alternative, we identify a set of statistics that are ancillary for the location parameter. When the scale parameter is known, Neyman–Pearson's lemma is used, and when the scale parameter is unknown, the LRT is applied to the joint density function of ancillary statistics, in order to obtain a test for the skewness parameter of the distribution. Test for symmetry of the distribution can be deduced as a special case. It is found that power of the proposed tests for symmetry is only marginally less than the power of corresponding classical optimum tests when the location parameter is known, especially for moderate and large sample sizes.     

A Matching Prior for the Shape Parameter of the Skew‐Normal Distribution

Abstract. This paper deals with the issue of performing a default Bayesian analysis on the shape parameter of the skew‐normal distribution. Our approach is based on a suitable pseudo‐likelihood function and a matching prior distribution for this parameter, when location (or regression) and scale parameters are unknown. This approach is important for both theoretical and practical reasons. From a theoretical perspective, it is shown that the proposed matching prior is proper thus inducing a proper posterior distribution for the shape parameter, also when the likelihood is monotone. From the practical perspective, the proposed approach has the advantages of avoiding the elicitation on the nuisance parameters and the computation of multidimensional integrals.     

Tests for symmetry of two-piece normal distribution

In this paper, classical optimum tests for symmetry of two-piece normal distribution is derived. Uniformly most powerful one-sided test for the skewness parameter is obtained when the location and scale parameters are known and is compared with sequential probability ratio test. An ad-hoc test for symmetry and likelihood ratio test for symmetry for large samples, can be found in literature for this distribution. But in this paper, we derive exact likelihood ratio test for symmetry, when location parameter is known. The exact power of the test is evaluated for different sample sizes.     

Likelihood ratio test for testing equality of location parameters of two exponential distributions from doubly censored samples

The Likelihood Ratio (LR) test for testing equality of two exponential distributions with common unknown scale parameter is obtained. Samples are assumed to be drawn under a type II doubly censored sampling scheme. Effects of left and right censoring on the power of the test are studied. Further, the performance of the LR test is compared with the Tiku(1981) test.     

Testing equality of mean vectors in a one-way MANOVA with monotone missing data

In this study, testing the equality of mean vectors in a one-way multivariate analysis of variance (MANOVA) is considered when each dataset has a monotone pattern of missing observations. The likelihood ratio test (LRT) statistic in a one-way MANOVA with monotone missing data is given. Furthermore, the modified test (MT) statistic based on likelihood ratio (LR) and the modified LRT (MLRT) statistic with monotone missing data are proposed using the decomposition of the LR and an asymptotic expansion for each decomposed LR. The accuracy of the approximation for the Chi-square distribution is investigated using a Monte Carlo simulation. Finally, an example is given to illustrate the methods.     

Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm

Two new implementations of the EM algorithm are proposed for maximum likelihood fitting of generalized linear mixed models. Both methods use random (independent and identically distributed) sampling to construct Monte Carlo approximations at the E-step. One approach involves generating random samples from the exact conditional distribution of the random effects (given the data) by rejection sampling, using the marginal distribution as a candidate. The second method uses a multivariate t importance sampling approximation. In many applications the two methods are complementary. Rejection sampling is more efficient when sample sizes are small, whereas importance sampling is better with larger sample sizes. Monte Carlo approximation using random samples allows the Monte Carlo error at each iteration to be assessed by using standard central limit theory combined with Taylor series methods. Specifically, we construct a sandwich variance estimate for the maximizer at each approximate E-step. This suggests a rule for automatically increasing the Monte Carlo sample size after iterations in which the true EM step is swamped by Monte Carlo error. In contrast, techniques for assessing Monte Carlo error have not been developed for use with alternative implementations of Monte Carlo EM algorithms utilizing Markov chain Monte Carlo E-step approximations. Three different data sets, including the infamous salamander data of McCullagh and Nelder, are used to illustrate the techniques and to compare them with the alternatives. The results show that the methods proposed can be considerably more efficient than those based on Markov chain Monte Carlo algorithms. However, the methods proposed may break down when the intractable integrals in the likelihood function are of high dimension.     

Profile likelihood in directed graphical models from BUGS output

We present a method for using posterior samples produced by the computer program BUGS (Bayesian inference Using Gibbs Sampling) to obtain approximate profile likelihood functions of parameters or functions of parameters in directed graphical models with incomplete data. The method can also be used to approximate integrated likelihood functions. It is easily implemented and it performs a good approximation. The profile likelihood represents an aspect of the parameter uncertainty which does not depend on the specification of prior distributions, and it can be used as a worthwhile supplement to BUGS that enable us to do both Bayesian and likelihood based analyses in directed graphical models.     

More on the Preliminary Test Estimator in Almost Unbiased Liu Regression

This article is concerned with the parameter estimation in linear regression model when it is suspected that the regression coefficients are the subspace of the equality restrictions. The objective of this article is to introduce the preliminary test almost unbiased Liu estimators (PTAULE) based on the Wald (W), the likelihood ratio (LR), and the Lagrangian multiplier (LM) tests and compare the proposed estimators in the sense of the quadratic bias and mean square error (MSE) criterion.