Modified quasi-profile likelihoods from estimating functions

We discuss higher-order adjustments for a quasi-profile likelihood for a scalar parameter of interest, in order to alleviate some of the problems inherent to the presence of nuisance parameters, such as bias and inconsistency. Indeed, quasi-profile score functions for the parameter of interest have bias of order O(1)$\mathrm{O}(1)$, and such bias can lead to poor inference on the parameter of interest. The higher-order adjustments are obtained so that the adjusted quasi-profile score estimating function is unbiased and its variance is the negative expected derivative matrix of the adjusted profile estimating equation. The modified quasi-profile likelihood is then obtained as the integral of the adjusted profile estimating function. We discuss two methods for the computation of the modified quasi-profile likelihoods: a bootstrap simulation method and a first-order asymptotic expression, which can be simplified under an orthogonality assumption. Examples in the context of generalized linear models and of robust inference are provided, showing that the use of a modified quasi-profile likelihood ratio statistic may lead to coverage probabilities more accurate than those pertaining to first-order Wald-type confidence intervals.     

Inference from PseudoLikelihoods with Plug‐In Estimates

Effective implementation of likelihood inference in models for high‐dimensional data often requires a simplified treatment of nuisance parameters, with these having to be replaced by handy estimates. In addition, the likelihood function may have been simplified by means of a partial specification of the model, as is the case when composite likelihood is used. In such circumstances tests and confidence regions for the parameter of interest may be constructed using Wald type and score type statistics, defined so as to account for nuisance parameter estimation or partial specification of the likelihood. In this paper a general analytical expression for the required asymptotic covariance matrices is derived, and suggestions for obtaining Monte Carlo approximations are presented. The same matrices are involved in a rescaling adjustment of the log likelihood ratio type statistic that we propose. This adjustment restores the usual chi‐squared asymptotic distribution, which is generally invalid after the simplifications considered. The practical implication is that, for a wide variety of likelihoods and nuisance parameter estimates, confidence regions for the parameters of interest are readily computable from the rescaled log likelihood ratio type statistic as well as from the Wald type and score type statistics. Two examples, a measurement error model with full likelihood and a spatial correlation model with pairwise likelihood, illustrate and compare the procedures. Wald type and score type statistics may give rise to confidence regions with unsatisfactory shape in small and moderate samples. In addition to having satisfactory shape, regions based on the rescaled log likelihood ratio type statistic show empirical coverage in reasonable agreement with nominal confidence levels.     

Adjusted Supremum Score‐Type Statistics for Evaluating Non‐Standard Hypotheses

Supremum score test statistics are often used to evaluate hypotheses with unidentifiable nuisance parameters under the null hypothesis. Although these statistics provide an attractive framework to address non‐identifiability under the null hypothesis, little attention has been paid to their distributional properties in small to moderate sample size settings. In situations where there are identifiable nuisance parameters under the null hypothesis, these statistics may behave erratically in realistic samples as a result of a non‐negligible bias induced by substituting these nuisance parameters by their estimates under the null hypothesis. In this paper, we propose an adjustment to the supremum score statistics by subtracting the expected bias from the score processes and show that this adjustment does not alter the limiting null distribution of the supremum score statistics. Using a simple example from the class of zero‐inflated regression models for count data, we show empirically and theoretically that the adjusted tests are superior in terms of size and power. The practical utility of this methodology is illustrated using count data in HIV research.     

Bounded Estimation in the Presence of Nuisance Parameters

The aim of this paper is to extend in a natural fashion the results on the treatment of nuisance parameters from the profile likelihood theory to the field of robust statistics. Similarly to what happens when there are no nuisance parameters, the attempt is to derive a bounded estimating function for a parameter of interest in the presence of nuisance parameters. The proposed method is based on a classical truncation argument of the theory of robustness applied to a generalized profile score function. By means of comparative studies, we show that this robust procedure for inference in the presence of a nuisance parameter can be used successfully in a parametric setting.     

Extending conditional likelihood in models for stratified binary data

The conditional likelihood is widely used in logistic regression models with stratified binary data. In particular, it leads to accurate inference for the parameters of interest, which are common to all strata, eliminating stratum-specific nuisance parameters. The modified profile likelihood is an accurate approximation to the conditional likelihood, but has the advantage of being available for general parametric models. Here, we propose the modified profile likelihood as an ideal extension of the conditional likelihood in generalized linear models for binary data, with generic link function. An important feature is that for the implementation we only need standard outputs of routines for generalized linear models. The accuracy of the method is supported by theoretical properties and is confirmed by simulation results.This research was supported by MIUR COFIN 2001-2003.     

Modified profile likelihood estimation for the Weibull regression models in survival analysis

In this study, adjustment of profile likelihood function of parameter of interest in presence of many nuisance parameters is investigated for survival regression models. Our objective is to extend the Barndorff–Nielsen’s technique to Weibull regression models for estimation of shape parameter in presence of many nuisance and regression parameters. We conducted Monte-Carlo simulation studies and a real data analysis, all of which demonstrate and suggest that the modified profile likelihood estimators outperform the profile likelihood estimators in terms of three comparison criterion: mean squared errors, bias and standard errors.     

Bayesian and frequentist confidence intervals via adjusted likelihoods under prior specification on the interest parameter

Suppose a prior is specified only on the interest parameter and a posterior distribution, free from nuisance parameters, is considered on the basis of the profile likelihood or an adjusted version thereof. In this setup, we derive higher order asymptotic results on the construction of confidence intervals that have approximately correct posterior as well as frequentist coverage. Apart from meeting both Bayesian and frequentist objectives under prior specification on the interest parameter alone, these results allow a comparison with their counterpart arising when the nuisance parameters are known, and hence provide additional justification for the Cox and Reid adjustment from a Bayesian-cum-frequentist perspective, with regard to neutralization of unknown nuisance parameters.     

The Effects of Rounding on Likelihood Procedures

The aim of this paper is to investigate the robustness properties of likelihood inference with respect to rounding effects. Attention is focused on exponential families and on inference about a scalar parameter of interest, also in the presence of nuisance parameters. A summary value of the influence function of a given statistic, the local-shift sensitivity, is considered. It accounts for small fluctuations in the observations. The main result is that the local-shift sensitivity is bounded for the usual likelihood-based statistics, i.e. the directed likelihood, the Wald and score statistics. It is also bounded for the modified directed likelihood, which is a higher-order adjustment of the directed likelihood. The practical implication is that likelihood inference is expected to be robust with respect to rounding effects. Theoretical analysis is supplemented and confirmed by a number of Monte Carlo studies, performed to assess the coverage probabilities of confidence intervals based on likelihood procedures when data are rounded. In addition, simulations indicate that the directed likelihood is less sensitive to rounding effects than the Wald and score statistics. This provides another criterion for choosing among first-order equivalent likelihood procedures. The modified directed likelihood shows the same robustness as the directed likelihood, so that its gain in inferential accuracy does not come at the price of an increase in instability with respect to rounding.     

Profile Likelihood Inferences on the Partially Linear Model with a Diverging Number of Parameters

In this article, we study the profile likelihood estimation and inference on the partially linear model with a diverging number of parameters. Polynomial splines are applied to estimate the nonparametric component and we focus on constructing profile likelihood ratio statistic to examine the testing problem for the parametric component in the partially linear model. Under some regularity conditions, the asymptotic distribution of profile likelihood ratio statistic is proposed when the number of parameters grows with the sample size. Numerical studies confirm our theory.     

Score tests when a nuisance parameter is unidentified under the null hypothesis

Rao's score test normally replaces nuisance parameters by their maximum likelihood estimates under the null hypothesis about the parameter of interest. In some models, however, a nuisance parameter is not identified under the null, so that this approach cannot be followed. This paper suggests replacing the nuisance parameter by its maximum likelihood estimate from the unrestricted model and making the appropriate adjustment to the variance of the estimated score. This leads to a rather natural modification of Rao's test, which is examined in detail for a regression-type model. It is compared with the approach, which has featured most frequently in the literature on this problem, where a test statistic appropriate to a known value of the nuisance parameter is treated as a function of that parameter and maximised over its range. It is argued that the modified score test has considerable advantages, including robustness to a crucial assumption required by the rival approach.     

Parameter Orthogonality and Bias Adjustment for Estimating Functions

Abstract.  We consider an extended notion of parameter orthogonality for estimating functions, called nuisance parameter insensitivity, which allows a unified treatment of nuisance parameters for a wide range of methods, including Liang and Zeger's generalized estimating equations. Nuisance parameter insensitivity has several important properties in common with conventional parameter orthogonality, such as the nuisance parameter causing no loss of efficiency for estimating the interest parameter, and a simplified estimation algorithm. We also consider bias adjustment for profile estimating functions, and apply the results to restricted maximum likelihood estimation of dispersion parameters in generalized estimating equations.     

Varying Dispersion Diagnostics for Inverse Gaussian Regression Models

Homogeneity of dispersion parameters is a standard assumption in inverse Gaussian regression analysis. However, this assumption is not necessarily appropriate. This paper is devoted to the test for varying dispersion in general inverse Gaussian linear regression models. Based on the modified profile likelihood (Cox & Reid, 1987), the adjusted score test for varying dispersion is developed and illustrated with Consumer- Product Sales data (Whitmore, 1986) and Gas vapour data (Weisberg, 1985). The effectiveness of orthogonality transformation and the properties of a score statistic and its adjustment are investigated through Monte Carlo simulations.     

Statistical inferences for partially linear single-index models with error-prone linear covariates

We consider statistical inference for partially linear single-index models (PLSIM) when some linear covariates are not observed, but ancillary variables are available. Based on the profile least-squared estimators of the unknowns, we study the testing problems for parametric components in the proposed models. It is to see whether the generalized likelihood ratio (GLR) tests proposed by Fan et al. (2001) are applicable to testing for the parametric components. We show that under the null hypothesis the proposed GLR statistics follow asymptotically the χ²-distributions with the scale constants and the degrees of freedom being independent of the nuisance parameters or functions, which is called the Wilks phenomenon. Simulated experiments are conducted to illustrate our proposed methodology.     

Integrated likelihoods in parametric survival models for highly clustered censored data

In studies that involve censored time-to-event data, stratification is frequently encountered due to different reasons, such as stratified sampling or model adjustment due to violation of model assumptions. Often, the main interest is not in the clustering variables, and the cluster-related parameters are treated as nuisance. When inference is about a parameter of interest in presence of many nuisance parameters, standard likelihood methods often perform very poorly and may lead to severe bias. This problem is particularly evident in models for clustered data with cluster-specific nuisance parameters, when the number of clusters is relatively high with respect to the within-cluster size. However, it is still unclear how the presence of censoring would affect this issue. We consider clustered failure time data with independent censoring, and propose frequentist inference based on an integrated likelihood. We then apply the proposed approach to a stratified Weibull model. Simulation studies show that appropriately defined integrated likelihoods provide very accurate inferential results in all circumstances, such as for highly clustered data or heavy censoring, even in extreme settings where standard likelihood procedures lead to strongly misleading results. We show that the proposed method performs generally as well as the frailty model, but it is superior when the frailty distribution is seriously misspecified. An application, which concerns treatments for a frequent disease in late-stage HIV-infected people, illustrates the proposed inferential method in Weibull regression models, and compares different inferential conclusions from alternative methods.     

The Cox Proportional Hazards Model with a Partly Linear Relative Risk Function

The conventional Cox proportional hazards regression model contains a loglinear relative risk function, linking the covariate information to the hazard ratio with a finite number of parameters. A generalization, termed the partly linear Cox model, allows for both finite dimensional parameters and an infinite dimensional parameter in the relative risk function, providing a more robust specification of the relative risk function. In this work, a likelihood based inference procedure is developed for the finite dimensional parameters of the partly linear Cox model. To alleviate the problems associated with a likelihood approach in the presence of an infinite dimensional parameter, the relative risk is reparameterized such that the finite dimensional parameters of interest are orthogonal to the infinite dimensional parameter. Inference on the finite dimensional parameters is accomplished through maximization of the profile partial likelihood, profiling out the infinite dimensional nuisance parameter using a kernel function. The asymptotic distribution theory for the maximum profile partial likelihood estimate is established. It is determined that this estimate is asymptotically efficient; the orthogonal reparameterization enables employment of profile likelihood inference procedures without adjustment for estimation of the nuisance parameter. An example from a retrospective analysis in cancer demonstrates the methodology.     

Facilitating the Calculation of the Efficient Score Using Symbolic Computing

The score statistic continues to be a fundamental tool for statistical inference. In the analysis of data from high-throughput genomic assays, inference on the basis of the score usually enjoys greater stability, considerably higher computational efficiency, and lends itself more readily to the use of resampling methods than the asymptotically equivalent Wald or likelihood ratio tests. The score function often depends on a set of unknown nuisance parameters which have to be replaced by estimators, but can be improved by calculating the efficient score, which accounts for the variability induced by estimating these parameters. Manual derivation of the efficient score is tedious and error-prone, so we illustrate using computer algebra to facilitate this derivation. We demonstrate this process within the context of a standard example from genetic association analyses, though the techniques shown here could be applied to any derivation, and have a place in the toolbox of any modern statistician. We further show how the resulting symbolic expressions can be readily ported to compiled languages, to develop fast numerical algorithms for high-throughput genomic analysis. We conclude by considering extensions of this approach. The code featured in this report is available online as part of the supplementary material.     

On bias reduction estimators of skew-normal and skew-t distributions

A particular concerns of researchers in statistical inference is bias in parameters estimation. Maximum likelihood estimators are often biased and for small sample size, the first order bias of them can be large and so it may influence the efficiency of the estimator. There are different methods for reduction of this bias. In this paper, we proposed a modified maximum likelihood estimator for the shape parameter of two popular skew distributions, namely skew-normal and skew-t, by offering a new method. We show that this estimator has lower asymptotic bias than the maximum likelihood estimator and is more efficient than those based on the existing methods.     

How Often Likelihood Ratios are Misleading in Sequential Trials

How often would investigators be misled if they took advantage of the likelihood principle and used likelihood ratios—which need not be adjusted for multiple looks at the data—to frequently examine accumulating data? The answer, perhaps surprisingly, is not often. As expected, the probability of observing misleading evidence does increase with each additional examination. However, the amount by which this probability increases converges to zero as the sample size grows. As a result, the probability of observing misleading evidence remains bounded—and therefore controllable—even with an infinite number of looks at the data. Here we use boundary crossing results to detail how often misleading likelihood ratios arise in sequential designs. We find that the probability of observing a misleading likelihood ratio is often much less than its universal bound. Additionally, we find that in the presence of fixed-dimensional nuisance parameters, profile likelihoods are to be preferred over estimated likelihoods which result from replacing the nuisance parameters by their global maximum likelihood estimates.     

Asymptotic likelihood inference for nonhomogeneous observations

This paper surveys asymptotic theory of maximum likelihood estimation for not identically distributed, possibly dependent observations. Main results on consistency, asymptotic normality and efficiency are stated within a unified framework. Limiting distributions of the likelihood ratio, Wald and score statistics for composite hypotheses are obtained under the same conditions by a generalization of existing theory. Modifications for maximum likelihood estimation under misspecification, containing the results for correctly specified models, are presented, and extensions to likelihood inference in the presence of nuisance parameters are indicated.     

Profile upper Confidence Limits from Discrete Data

When the data are discrete, standard approximate confidence limits often have coverage well below nominal for some parameter values. While ad hoc adjustments may largely solve this problem for particular cases, Kabaila & Lloyd (1997) gave a more systematic method of adjustment which leads to tight upper limits, which have coverage which is never below nominal and are as small as possible within a particular class. However, their computation for all but the simplest models is infeasible. This paper suggests modifying tight upper limits by an initial replacement of the unknown nuisance parameter vector by its profile maximum likelihood estimator. While the resulting limits no longer possess the optimal properties of tight limits exactly, the paper presents both numerical and theoretical evidence that the resulting coverage function is close to optimal. Moreover these profile upper limits are much (possibly many orders of magnitude) easier to compute than tight upper limits.