Modified quasi-profile likelihoods from estimating functions

We discuss higher-order adjustments for a quasi-profile likelihood for a scalar parameter of interest, in order to alleviate some of the problems inherent to the presence of nuisance parameters, such as bias and inconsistency. Indeed, quasi-profile score functions for the parameter of interest have bias of order O(1)$\mathrm{O}(1)$, and such bias can lead to poor inference on the parameter of interest. The higher-order adjustments are obtained so that the adjusted quasi-profile score estimating function is unbiased and its variance is the negative expected derivative matrix of the adjusted profile estimating equation. The modified quasi-profile likelihood is then obtained as the integral of the adjusted profile estimating function. We discuss two methods for the computation of the modified quasi-profile likelihoods: a bootstrap simulation method and a first-order asymptotic expression, which can be simplified under an orthogonality assumption. Examples in the context of generalized linear models and of robust inference are provided, showing that the use of a modified quasi-profile likelihood ratio statistic may lead to coverage probabilities more accurate than those pertaining to first-order Wald-type confidence intervals.     

Minimum Scoring Rule Inference

Proper scoring rules are devices for encouraging honest assessment of probability distributions. Just like log‐likelihood, which is a special case, a proper scoring rule can be applied to supply an unbiased estimating equation for any statistical model, and the theory of such equations can be applied to understand the properties of the associated estimator. In this paper, we discuss some novel applications of scoring rules to parametric inference. In particular, we focus on scoring rule test statistics, and we propose suitable adjustments to allow reference to the usual asymptotic chi‐squared distribution. We further explore robustness and interval estimation properties, by both theory and simulations.     

Applications of the Bootstrap in ROC Analysis

The problem of estimating standard errors for diagnostic accuracy measures might be challenging for many complicated models. We can address such a problem by using the Bootstrap methods to blunt its technical edge with resampled empirical distributions. We consider two cases where bootstrap methods can successfully improve our knowledge of the sampling variability of the diagnostic accuracy estimators. The first application is to make inference for the area under the ROC curve resulted from a functional logistic regression model which is a sophisticated modelling device to describe the relationship between a dichotomous response and multiple covariates. We consider using this regression method to model the predictive effects of multiple independent variables on the occurrence of a disease. The accuracy measures, such as the area under the ROC curve (AUC) are developed from the functional regression. Asymptotical results for the empirical estimators are provided to facilitate inferences. The second application is to test the difference of two weighted areas under the ROC curve (WAUC) from a paired two sample study. The correlation between the two WAUC complicates the asymptotic distribution of the test statistic. We then employ the bootstrap methods to gain satisfactory inference results. Simulations and examples are supplied in this article to confirm the merits of the bootstrap methods.     

Bootstrap bias corrections for ensemble methods

This paper examines the use of a residual bootstrap for bias correction in machine learning regression methods. Accounting for bias is an important obstacle in recent efforts to develop statistical inference for machine learning. We demonstrate empirically that the proposed bootstrap bias correction can lead to substantial improvements in both bias and predictive accuracy. In the context of ensembles of trees, we show that this correction can be approximated at only double the cost of training the original ensemble. Our method is shown to improve test set accuracy over random forests by up to 70% on example problems from the UCI repository.     

A Martingale-based bootstrap inference with censored data

In this paper, me shall investigate a bootstrap method hasd on a martingale representation of the relevant statistic for inference to a class of functionals of the survival distribution. The method is similar in spirit to Efron's (1981) bootstrap, and thus in the present paper will be referred to as “martingale-based bootstrap” The method was derived from Lin,Wei and Ying (1993), who appiied the method in checking the Cox model with cumulative sums of martingale-based residuals. It is shown that this martingale-based bootstrap gives a correct first-order asymptotic approximation to the distribution function of the corresponding functional of the Kaplan-Meier estimator. As a consequence, confidence intervals constructed by the martingale-based bootstrap have asymptotially correct coverage probability. Our simulation study indicats that the martingale-based bootst strap method for a small and moderate sample sizes can be uniformly better than the usual bootstrap method in estimating the sampling distribution for a mean function and a point probability in survival analysis.     

Evaluation of methods for interval estimation of model outputs, with application to survival models

When a published statistical model is also distributed as computer software, it will usually be desirable to present the outputs as interval, as well as point, estimates. The present paper compares three methods for approximate interval estimation about a model output, for use when the model form does not permit an exact interval estimate. The methods considered are first-order asymptotics, using second derivatives of the log-likelihood to estimate variance information; higher-order asymptotics based on the signed-root transformation; and the non-parametric bootstrap. The signed-root method is Bayesian, and uses an approximation for posterior moments that has not previously been tested in a real-world application. Use of the three methods is illustrated with reference to a software project arising in medical decision-making, the UKPDS Risk Engine. Intervals from the first-order and signed-root methods are near- identical, and typically 1% wider to 7% narrower than those from the non-parametric bootstrap. The asymptotic methods are markedly faster than the bootstrap method.     

协整参数的自举推断

 针对完全修正最小二乘(full-modified ordinary least square,简称FMOLS）估计方法,给出一种协整参数的自举推断程序,证明零假设下自举统计量与检验统计量具有相同的渐近分布。关于检验功效的研究表明,虽然有约束自举的实际检验水平表现良好,但如果零假设不成立,自举统计量的分布是不确定的,因而其经验分布不能作为检验统计量精确分布的有效估计。实际应用中建议使用无约束自举,因为无论观测数据是否满足零假设,其自举统计量与零假设下检验统计量都具有相同的渐近分布。最后,利用蒙特卡洛模拟对自举推断和渐近推断的有限样本表现进行比较研究。     

Empirical likelihood for quantile regression models with longitudinal data

We develop two empirical likelihood-based inference procedures for longitudinal data under the framework of quantile regression. The proposed methods avoid estimating the unknown error density function and the intra-subject correlation involved in the asymptotic covariance matrix of the quantile estimators. By appropriately smoothing the quantile score function, the empirical likelihood approach is shown to have a higher-order accuracy through the Bartlett correction. The proposed methods exhibit finite-sample advantages over the normal approximation-based and bootstrap methods in a simulation study and the analysis of a longitudinal ophthalmology data set.     

ROUTES TO HIGHER-ORDER ACCURACY IN PARAMETRIC INFERENCE

Developments in the theory of frequentist parametric inference in recent decades have been driven largely by the desire to achieve higher-order accuracy, in particular distributional approximations that improve on first-order asymptotic theory by one or two orders of magnitude. At the same time, much methodology is specifically designed to respect key principles of parametric inference, in particular conditionality principles. Two main routes to higher-order accuracy have emerged: analytic methods based on 'small-sample asymptotics', and simulation, or 'bootstrap', approaches. It is argued here that, of these, the simulation methodology provides a simple and effective approach, which nevertheless retains finer inferential components of theory. The paper seeks to track likely developments of parametric inference, in an era dominated by the emergence of methodological problems involving complex dependences and/or high-dimensional parameters that typically exceed available data sample sizes.     

EMPIRICAL LIKELIHOOD METHODS FOR THE GINI INDEX

The Gini index and its generalizations have been used extensively for measuring inequality and poverty in the social sciences. Recently, interval estimation based on nonparametric statistics has been proposed in the literature, for example the naive bootstrap method, the iterated bootstrap method and the bootstrap method via a pivotal statistic. In this paper, we propose empirical likelihood methods to construct confidence intervals for the Gini index or the difference of two Gini indices. Simulation studies show that the proposed empirical likelihood method performs slightly worse than the bootstrap method based on a pivotal statistic in terms of coverage accuracy, but it requires less computation. However, the bootstrap calibration of the empirical likelihood method performs better than the bootstrap method based on a pivotal statistic.     

Bootstrapping p Values and Power in the First-Order Autoregression: A Monte Carlo Investigation

The small-sample behavior of the bootstrap is investigated as a method for estimating p values and power in the stationary first-order autoregressive model. Monte Carlo methods are used to examine the bootstrap and Student-t approximations to the true distribution of the test statistic frequently used for testing hypotheses on the underlying slope parameter. In contrast to Student's t, the results suggest that the bootstrap can accurately estimate p values and power in this model in sample sizes as small as 5–10.     

Fast and robust bootstrap

In this paper we review recent developments on a bootstrap method for robust estimators which is computationally faster and more resistant to outliers than the classical bootstrap. This fast and robust bootstrap method is, under reasonable regularity conditions, asymptotically consistent. We describe the method in general and then consider its application to perform inference based on robust estimators for the linear regression and multivariate location-scatter models. In particular, we study confidence and prediction intervals and tests of hypotheses for linear regression models, inference for location-scatter parameters and principal components, and classification error estimation for discriminant analysis.     

Likelihood inference for lognormal data with left truncation and right censoring with an illustration

The lognormal distribution is quite commonly used as a lifetime distribution. Data arising from life-testing and reliability studies are often left truncated and right censored. Here, the EM algorithm is used to estimate the parameters of the lognormal model based on left truncated and right censored data. The maximization step of the algorithm is carried out by two alternative methods, with one involving approximation using Taylor series expansion (leading to approximate maximum likelihood estimate) and the other based on the EM gradient algorithm (Lange, 1995). These two methods are compared based on Monte Carlo simulations. The Fisher scoring method for obtaining the maximum likelihood estimates shows a problem of convergence under this setup, except when the truncation percentage is small. The asymptotic variance-covariance matrix of the MLEs is derived by using the missing information principle (Louis, 1982), and then the asymptotic confidence intervals for scale and shape parameters are obtained and compared with corresponding bootstrap confidence intervals. Finally, some numerical examples are given to illustrate all the methods of inference developed here.     

Empirical likelihood confidence intervals under imputation for missing survey data from stratified simple random sampling

Missing observations due to non‐response are commonly encountered in data collected from sample surveys. The focus of this article is on item non‐response which is often handled by filling in (or imputing) missing values using the observed responses (donors). Random imputation (single or fractional) is used within homogeneous imputation classes that are formed on the basis of categorical auxiliary variables observed on all the sampled units. A uniform response rate within classes is assumed, but that rate is allowed to vary across classes. We construct confidence intervals (CIs) for a population parameter that is defined as the solution to a smooth estimating equation with data collected using stratified simple random sampling. The imputation classes are assumed to be formed across strata. Fractional imputation with a fixed number of random draws is used to obtain an imputed estimating function. An empirical likelihood inference method under the fractional imputation is proposed and its asymptotic properties are derived. Two asymptotically correct bootstrap methods are developed for constructing the desired CIs. In a simulation study, the proposed bootstrap methods are shown to outperform traditional bootstrap methods and some non‐bootstrap competitors under various simulation settings. The Canadian Journal of Statistics 47: 281–301; 2019 © 2019 Statistical Society of Canada     

Estimation and Asymptotic Properties in Periodic GARCH(1, 1) Models

This article studies the probabilistic structure and asymptotic inference of the first-order periodic generalized autoregressive conditional heteroscedasticity (PGARCH(1, 1)) models in which the parameters in volatility process are allowed to switch between different regimes. First, we establish necessary and sufficient conditions for a PGARCH(1, 1) process to have a unique stationary solution (in periodic sense) and for the existence of moments of any order. Second, using the representation of squared PGARCH(1, 1) model as a PARMA(1, 1) model, we then consider Yule-Walker type estimators for the parameters in PGARCH(1, 1) model and derives their consistency and asymptotic normality. The estimator can be surprisingly efficient for quite small numbers of autocorrelations and, in some cases can be more efficient than the least squares estimate (LSE). We use a residual bootstrap to define bootstrap estimators for the Yule-Walker estimates and prove the consistency of this bootstrap method. A set of numerical experiments illustrates the practical relevance of our theoretical results.     

Improved estimation procedures for multilevel models with binary response: a case-study

During recent years, analysts have been relying on approximate methods of inference to estimate multilevel models for binary or count data. In an earlier study of random-intercept models for binary outcomes we used simulated data to demonstrate that one such approximation, known as marginal quasi-likelihood, leads to a substantial attenuation bias in the estimates of both fixed and random effects whenever the random effects are non-trivial. In this paper, we fit three-level random-intercept models to actual data for two binary outcomes, to assess whether refined approximation procedures, namely penalized quasi-likelihood and second-order improvements to marginal and penalized quasi-likelihood, also underestimate the underlying parameters. The extent of the bias is assessed by two standards of comparison: exact maximum likelihood estimates, based on a Gauss–Hermite numerical quadrature procedure, and a set of Bayesian estimates, obtained from Gibbs sampling with diffuse priors. We also examine the effectiveness of a parametric bootstrap procedure for reducing the bias. The results indicate that second-order penalized quasi-likelihood estimates provide a considerable improvement over the other approximations, but all the methods of approximate inference result in a substantial underestimation of the fixed and random effects when the random effects are sizable. We also find that the parametric bootstrap method can eliminate the bias but is computationally very intensive.     

Bootstrap and Bayesian bootstrap clones for censored Markov chains

This is a study of the behaviors of the naive bootstrap and the Bayesian bootstrap clones designed to approximate the sampling distribution of the Aalen–Johansen estimator of a non-homogeneous censored Markov chain. The study shows that the approximations based on the Bayesian bootstrap clones and the naive bootstrap are first-order asymptotically equivalent. The two bootstrap methods are illustrated by a marketing example, and their performance is validated by a Monte Carlo experiment.     

Data-based interval estimation of classification error rates

Leave-one-out and 632 bootstrap are popular data-based methods of estimating the true error rate of a classification rule, but practical applications almost exclusively quote only point estimates. Interval estimation would provide better assessment of the future performance of the rule, but little has been published on this topic. We first review general-purpose jackknife and bootstrap methodology that can be used in conjunction with leave-one-out estimates to provide prediction intervals for true error rates of classification rules. Monte Carlo simulation is then used to investigate coverage rates of the resulting intervals for normal data, but the results are disappointing; standard intervals show considerable overinclusion, intervals based on Edgeworth approximations or random weighting do not perform well, and while a bootstrap approach provides intervals with coverage rates closer to the nominal ones there is still marked underinclusion. We then turn to intervals constructed from 632 bootstrap estimates, and show that much better results are obtained. Although there is now some overinclusion, particularly for large training samples, the actual coverage rates are sufficiently close to the nominal rates for the method to be recommended. An application to real data illustrates the considerable variability that can arise in practical estimation of error rates.     

Bootstrapping Noncausal Autoregressions: With Applications to Explosive Bubble Modeling

In this article, we develop new bootstrap-based inference for noncausal autoregressions with heavy-tailed innovations. This class of models is widely used for modeling bubbles and explosive dynamics in economic and financial time series. In the noncausal, heavy-tail framework, a major drawback of asymptotic inference is that it is not feasible in practice as the relevant limiting distributions depend crucially on the (unknown) decay rate of the tails of the distribution of the innovations. In addition, even in the unrealistic case where the tail behavior is known, asymptotic inference may suffer from small-sample issues. To overcome these difficulties, we propose bootstrap inference procedures using parameter estimates obtained with the null hypothesis imposed (the so-called restricted bootstrap). We discuss three different choices of bootstrap innovations: wild bootstrap, based on Rademacher errors; permutation bootstrap; a combination of the two (“permutation wild bootstrap”). Crucially, implementation of these bootstraps do not require any a priori knowledge about the distribution of the innovations, such as the tail index or the convergence rates of the estimators. We establish sufficient conditions ensuring that, under the null hypothesis, the bootstrap statistics estimate consistently particular conditionaldistributions of the original statistics. In particular, we show that validity of the permutation bootstrap holds without any restrictions on the distribution of the innovations, while the permutation wild and the standard wild bootstraps require further assumptions such as symmetry of the innovation distribution. Extensive Monte Carlo simulations show that the finite sample performance of the proposed bootstrap tests is exceptionally good, both in terms of size and of empirical rejection probabilities under the alternative hypothesis. We conclude by applying the proposed bootstrap inference to Bitcoin/USD exchange rates and to crude oil price data. We find that indeed noncausal models with heavy-tailed innovations are able to fit the data, also in periods of bubble dynamics. Supplementary materials for this article are available online.     

Semiparametric estimation in single index Poisson regression: a practical approach

In a single index Poisson regression model with unknown link function, the index parameter can be root- n consistently estimated by the method of pseudo maximum likelihood. In this paper, we study, by simulation arguments, the practical validity of the asymptotic behaviour of the pseudo maximum likelihood index estimator and of some associated cross-validation bandwidths. A robust practical rule for implementing the pseudo maximum likelihood estimation method is suggested, which uses the bootstrap for estimating the variance of the index estimator and a variant of bagging for numerically stabilizing its variance. Our method gives reasonable results even for moderate sized samples; thus, it can be used for doing statistical inference in practical situations. The procedure is illustrated through a real data example.