Estimating the First- and Second-Order Parameters of a Heavy-Tailed Distribution

This paper suggests censored maximum likelihood estimators for the first‐ and second‐order parameters of a heavy‐tailed distribution by incorporating the second‐order regular variation into the censored likelihood function. This approach is different from the bias‐reduced maximum likelihood method proposed by Feuerverger and Hall in 1999. The paper derives the joint asymptotic limit for the first‐ and second‐order parameters under a weaker assumption. The paper also demonstrates through a simulation study that the suggested estimator for the first‐order parameter is better than the estimator proposed by Feuerverger and Hall although these two estimators have the same asymptotic variances.     

Bias Reduction of Likelihood Estimators in Semiparametric Frailty Models

Abstract. Frailty models with a non‐parametric baseline hazard are widely used for the analysis of survival data. However, their maximum likelihood estimators can be substantially biased in finite samples, because the number of nuisance parameters associated with the baseline hazard increases with the sample size. The penalized partial likelihood based on a first‐order Laplace approximation still has non‐negligible bias. However, the second‐order Laplace approximation to a modified marginal likelihood for a bias reduction is infeasible because of the presence of too many complicated terms. In this article, we find adequate modifications of these likelihood‐based methods by using the hierarchical likelihood.     

Empirical likelihood for the varying‐coefficient single‐index model

In this article the author investigates the application of the empirical‐likelihood‐based inference for the parameters of varying‐coefficient single‐index model (VCSIM). Unlike the usual cases, if there is no bias correction the asymptotic distribution of the empirical likelihood ratio cannot achieve the standard chi‐squared distribution. To this end, a bias‐corrected empirical likelihood method is employed to construct the confidence regions (intervals) of regression parameters, which have two advantages, compared with those based on normal approximation, that is, (1) they do not impose prior constraints on the shape of the regions; (2) they do not require the construction of a pivotal quantity and the regions are range preserving and transformation respecting. A simulation study is undertaken to compare the empirical likelihood with the normal approximation in terms of coverage accuracies and average areas/lengths of confidence regions/intervals. A real data example is given to illustrate the proposed approach. The Canadian Journal of Statistics 38: 434–452; 2010 © 2010 Statistical Society of Canada     

Empirical phi-divergence test statistics for testing simple and composite null hypotheses

The main purpose of this paper is to introduce first a new family of empirical test statistics for testing a simple null hypothesis when the vector of parameters of interest is defined through a specific set of unbiased estimating functions. This family of test statistics is based on a distance between two probability vectors, with the first probability vector obtained by maximizing the empirical likelihood (EL) on the vector of parameters, and the second vector defined from the fixed vector of parameters under the simple null hypothesis. The distance considered for this purpose is the phi-divergence measure. The asymptotic distribution is then derived for this family of test statistics. The proposed methodology is illustrated through the well-known data of Newcomb's measurements on the passage time for light. A simulation study is carried out to compare its performance with that of the EL ratio test when confidence intervals are constructed based on the respective statistics for small sample sizes. The results suggest that the ‘empirical modified likelihood ratio test statistic’ provides a competitive alternative to the EL ratio test statistic, and is also more robust than the EL ratio test statistic in the presence of contamination in the data. Finally, we propose empirical phi-divergence test statistics for testing a composite null hypothesis and present some asymptotic as well as simulation results for evaluating the performance of these test procedures.     

First‐order bias correction for fractionally integrated time series

Most of the long memory estimators for stationary fractionally integrated time series models are known to experience non‐negligible bias in small and finite samples. Simple moment estimators are also vulnerable to such bias, but can easily be corrected. In this article, the authors propose bias reduction methods for a lag‐one sample autocorrelation‐based moment estimator. In order to reduce the bias of the moment estimator, the authors explicitly obtain the exact bias of lag‐one sample autocorrelation up to the order n⁻¹. An example where the exact first‐order bias can be noticeably more accurate than its asymptotic counterpart, even for large samples, is presented. The authors show via a simulation study that the proposed methods are promising and effective in reducing the bias of the moment estimator with minimal variance inflation. The proposed methods are applied to the northern hemisphere data. The Canadian Journal of Statistics 37: 476–493; 2009 © 2009 Statistical Society of Canada     

On bias reduction estimators of skew-normal and skew-t distributions

A particular concerns of researchers in statistical inference is bias in parameters estimation. Maximum likelihood estimators are often biased and for small sample size, the first order bias of them can be large and so it may influence the efficiency of the estimator. There are different methods for reduction of this bias. In this paper, we proposed a modified maximum likelihood estimator for the shape parameter of two popular skew distributions, namely skew-normal and skew-t, by offering a new method. We show that this estimator has lower asymptotic bias than the maximum likelihood estimator and is more efficient than those based on the existing methods.     

Empirical Likelihood Local Polynomial Regression Analysis of Clustered Data

Abstract. In this article, a naive empirical likelihood ratio is constructed for a non‐parametric regression model with clustered data, by combining the empirical likelihood method and local polynomial fitting. The maximum empirical likelihood estimates for the regression functions and their derivatives are obtained. The asymptotic distributions for the proposed ratio and estimators are established. A bias‐corrected empirical likelihood approach to inference for the parameters of interest is developed, and the residual‐adjusted empirical log‐likelihood ratio is shown to be asymptotically chi‐squared. These results can be used to construct a class of approximate pointwise confidence intervals and simultaneous bands for the regression functions and their derivatives. Owing to our bias correction for the empirical likelihood ratio, the accuracy of the obtained confidence region is not only improved, but also a data‐driven algorithm can be used for selecting an optimal bandwidth to estimate the regression functions and their derivatives. A simulation study is conducted to compare the empirical likelihood method with the normal approximation‐based method in terms of coverage accuracies and average widths of the confidence intervals/bands. An application of this method is illustrated using a real data set.     

On the estimation of the extreme value and normal distribution parameters based on progressive type-II hybrid-censored data

A progressive hybrid censoring scheme is a mixture of type-I and type-II progressive censoring schemes. In this paper, we mainly consider the analysis of progressive type-II hybrid-censored data when the lifetime distribution of the individual item is the normal and extreme value distributions. Since the maximum likelihood estimators (MLEs) of these parameters cannot be obtained in the closed form, we propose to use the expectation and maximization (EM) algorithm to compute the MLEs. Also, the Newton–Raphson method is used to estimate the model parameters. The asymptotic variance–covariance matrix of the MLEs under EM framework is obtained by Fisher information matrix using the missing information and asymptotic confidence intervals for the parameters are then constructed. This study will end up with comparing the two methods of estimation and the asymptotic confidence intervals of coverage probabilities corresponding to the missing information principle and the observed information matrix through a simulation study, illustrated examples and real data analysis.     

Bias‐reduced marginal Akaike information criteria based on a Monte Carlo method for linear mixed‐effects models

In linear mixed‐effects (LME) models, if a fitted model has more random‐effect terms than the true model, a regularity condition required in the asymptotic theory may not hold. In such cases, the marginal Akaike information criterion (AIC) is positively biased for (?2) times the expected log‐likelihood. The asymptotic bias of the maximum log‐likelihood as an estimator of the expected log‐likelihood is evaluated for LME models with balanced design in the context of parameter‐constrained models. Moreover, bias‐reduced marginal AICs for LME models based on a Monte Carlo method are proposed. The performance of the proposed criteria is compared with existing criteria by using example data and by a simulation study. It was found that the bias of the proposed criteria was smaller than that of the existing marginal AIC when a larger model was fitted and that the probability of choosing a smaller model incorrectly was decreased.     

ASYMPTOTIC DISTRIBUTIONS OF SEMIPARAMETRIC MAXIMUM LIKELIHOOD ESTIMATORS WITH ESTIMATING EQUATIONS FOR GROUP-CENSORED DATA

Semiparametric maximum likelihood estimation with estimating equations (SMLE) is more flexible than traditional methods; it has fewer restrictions on distributions and regression models. The required information about distribution and regression structures is incorporated in estimating equations of the SMLE to improve the estimation quality of non‐parametric methods. The likelihood of SMLE for censored data involves complicated implicit functions without closed‐form expressions, and the first derivatives of the log‐profile‐likelihood cannot be expressed as summations of independent and identically distributed random variables; it is challenging to derive asymptotic properties of the SMLE for censored data. For group‐censored data, the paper shows that all the implicit functions are well defined and obtains the asymptotic distributions of the SMLE for model parameters and lifetime distributions. With several examples the paper compares the SMLE, the regular non‐parametric likelihood estimation method and the parametric MLEs in terms of their asymptotic efficiencies, and illustrates application of SMLE. Various asymptotic distributions of the likelihood ratio statistics are derived for testing the adequacy of estimating equations and a partial set of parameters equal to some known values.     

Estimation in the presence of incidental parameters

This paper considers the estimation of “structural” parameters when the number of unknown parameters increases with the sample size. Neyman and Scott (1948) had demonstrated that maximum likelihood estimators (MLE) of structural parameters may be inconsistent in this case. Patefield (1977) further observed that the asymptotic covariance matrix of the MLE is not equal to the inverse of the information matrix. In this paper we establish asymptotic properties of estimators (which include in particular the MLE) obtained via the usual likelihood approach when the incidental parameters are first replaced by their estimates (which are allowed to depend on the structural parameters). Conditions for consistency and asymptotic normality together with a proper formula for the asymptotic covariance matrix are given. The results are illustrated and applied to the problem of estimating linear functional relationships, and mild conditions on the incidental parameters for the MLE (or an adjusted MLE) to be consistent and asymptotically normal are obtained. These conditions are weaker than those imposed by previous authors.     

The asymptotic distribution of MLE of treatment lag threshold

Semiparametric models with an unknown threshold parameter for hazard rates are formulated for failure time data. Maximum partial likelihood estimates for threshold and regular parameters are shown to be asymptotically consistent. The asymptotic distributions of the estimators are obtained.     

Analysis of cointegrated models with measurement errors

We study the asymptotic properties of the reduced-rank estimator of error correction models of vector processes observed with measurement errors. Although it is well known that there is no asymptotic measurement error bias when predictor variables are integrated processes in regression models [Phillips BCB, Durlauf SN. Multiple time series regression with integrated processes. Rev Econom Stud. 1986;53:473–495], we systematically investigate the effects of the measurement errors (in the dependent variables as well as in the predictor variables) on the estimation of not only cointegrating vectors but also the speed of the adjustment matrix. Furthermore, we present the asymptotic properties of the estimators. We also obtain the asymptotic distribution of the likelihood ratio test for the cointegrating ranks. We investigate the effects of the measurement errors on estimation and test through a Monte Carlo simulation study.     

Asymptotic biases of information and cross-validation criteria under canonical parametrization

An asymptotic expansion of the cross-validation criterion (CVC) using the Kullback-Leibler distance is derived when the leave-k-out method is used and when parameters are estimated by the weighted score method. By this expansion, the asymptotic bias of the Takeuchi information criterion (TIC) is derived as well as that of the CVC. Under canonical parametrization in the exponential family of distributions when maximum likelihood estimation is used, the magnitudes of the asymptotic biases of the Akaike information criterion (AIC) and CVC are shown to be smaller than that of the TIC. Examples in typical statistical distributions are shown.     

Efficient estimation for time series following generalized linear models

In this paper, we consider James–Stein shrinkage and pretest estimation methods for time series following generalized linear models when it is conjectured that some of the regression parameters may be restricted to a subspace. Efficient estimation strategies are developed when there are many covariates in the model and some of them are not statistically significant. Statistical properties of the pretest and shrinkage estimation methods including asymptotic distributional bias and risk are developed. We investigate the relative performances of shrinkage and pretest estimators with respect to the unrestricted maximum partial likelihood estimator (MPLE). We show that the shrinkage estimators have a lower relative mean squared error as compared to the unrestricted MPLE when the number of significant covariates exceeds two. Monte Carlo simulation experiments were conducted for different combinations of inactive covariates and the performance of each estimator was evaluated in terms of its mean squared error. The practical benefits of the proposed methods are illustrated using two real data sets.     

Inference for Box–Cox Transformed Threshold GARCH Models with Nuisance Parameters

Abstract. Generalized autoregressive conditional heteroscedastic (GARCH) models have been widely used for analyzing financial time series with time‐varying volatilities. To overcome the defect of the Gaussian quasi‐maximum likelihood estimator (QMLE) when the innovations follow either heavy‐tailed or skewed distributions, Berkes & Horváth (Ann. Statist., 32, 633, 2004) and Lee & Lee (Scand. J. Statist. 36, 157, 2009) considered likelihood methods that use two‐sided exponential, Cauchy and normal mixture distributions. In this paper, we extend their methods for Box–Cox transformed threshold GARCH model by allowing distributions used in the construction of likelihood functions to include parameters and employing the estimated quasi‐likelihood estimators (QELE) to handle those parameters. We also demonstrate that the proposed QMLE and QELE are consistent and asymptotically normal under regularity conditions. Simulation results are provided for illustration.     

Enhancements of Non‐parametric Generalized Likelihood Ratio Test: Bias Correction and Dimension Reduction

Non‐parametric generalized likelihood ratio test is a popular method of model checking for regressions. However, there are two issues that may be the barriers for its powerfulness: existing bias term and curse of dimensionality. The purpose of this paper is thus twofold: a bias reduction is suggested and a dimension reduction‐based adaptive‐to‐model enhancement is recommended to promote the power performance. The proposed test statistic still possesses the Wilks phenomenon and behaves like a test with only one covariate. Thus, it converges to its limit at a much faster rate and is much more sensitive to alternative models than the classical non‐parametric generalized likelihood ratio test. As a by‐product, we also prove that the bias‐corrected test is more efficient than the one without bias reduction in the sense that its asymptotic variance is smaller. Simulation studies and a real data analysis are conducted to evaluate of proposed tests.     

Maximum Likelihood Estimation for the 4-Parameter Beta Distribution

This paper addresses the problem of obtaining maximum likelihood estimates for the parameters of the Pearson Type I distribution (beta distribution with unknown end points and shape parameters). Since they do not seem to have appeared in the literature, the likelihood equations and the information matrix are derived. The regularity conditions which ensure asymptotic normality and efficiency are examined, and some apparent conflicts in the literature are noted. To ensure regularity, the shape parameters must be greater than two, giving an (assymmetrical) bell-shaped distribution with high contact in the tails. A numerical investigation was carried out to explore the bias and variance of the maximum likelihood estimates and their dependence on sample size. The numerical study indicated that only for large samples (n ≥ 1000) does the bias in the estimates become small and does the Cramér-Rao bound give a good approximation for their variance. The likelihood function has a global maximum which corresponds to parameter estimates that are inadmissable. Useful parameter estimates can be obtained at a local maximum, which is sometimes difficult to locate when the sample size is small.     

Parameter Estimation Robust to Low-Frequency Contamination

We provide methods to robustly estimate the parameters of stationary ergodic short-memory time series models in the potential presence of additive low-frequency contamination. The types of contamination covered include level shifts (changes in mean) and monotone or smooth time trends, both of which have been shown to bias parameter estimates toward regions of persistence in a variety of contexts. The estimators presented here minimize trimmed frequency domain quasi-maximum likelihood (FDQML) objective functions without requiring specification of the low-frequency contaminating component. When proper sample size-dependent trimmings are used, the FDQML estimators are consistent and asymptotically normal, asymptotically eliminating the presence of any spurious persistence. These asymptotic results also hold in the absence of additive low-frequency contamination, enabling the practitioner to robustly estimate model parameters without prior knowledge of whether contamination is present. Popular time series models that fit into the framework of this article include autoregressive moving average (ARMA), stochastic volatility, generalized autoregressive conditional heteroscedasticity (GARCH), and autoregressive conditional heteroscedasticity (ARCH) models. We explore the finite sample properties of the trimmed FDQML estimators of the parameters of some of these models, providing practical guidance on trimming choice. Empirical estimation results suggest that a large portion of the apparent persistence in certain volatility time series may indeed be spurious. Supplementary materials for this article are available online.     

Design considerations for small experiments and simple logistic regression

Inference for a generalized linear model is generally performed using asymptotic approximations for the bias and the covariance matrix of the parameter estimators. For small experiments, these approximations can be poor and result in estimators with considerable bias. We investigate the properties of designs for small experiments when the response is described by a simple logistic regression model and parameter estimators are to be obtained by the maximum penalized likelihood method of Firth [Firth, D., 1993, Bias reduction of maximum likelihood estimates. Biometrika, 80, 27–38]. Although this method achieves a reduction in bias, we illustrate that the remaining bias may be substantial for small experiments, and propose minimization of the integrated mean square error, based on Firth's estimates, as a suitable criterion for design selection. This approach is used to find locally optimal designs for two support points.