An Equivalence of Conditional and Unconditional Maximum Likelihood Estimators via Infinite Replication of Observations

Motivated by an application with complex survey data, we show that for logistic regression with a simple matched-pairs design, infinitely replicating observations and maximizing the conditional likelihood results in an estimator exactly identical to the unconditional maximum likelihood estimator based on the original sample, which is inconsistent. Therefore, applying conditional likelihood methods to a pseudosample with observations replicated a large number of times can lead to an inconsistent estimator; this casts doubt on one possible approach to conditional logistic regression with complex survey data. We speculate that for more general designs, an asymptotic equivalence holds.     

Variance component models for longitudinal count data with baseline information: epilepsy data revisited

Random effect models have often been used in longitudinal data analysis since they allow for association among repeated measurements due to unobserved heterogeneity. Various approaches have been proposed to extend mixed models for repeated count data to include dependence on baseline counts. Dependence between baseline counts and individual-specific random effects result in a complex form of the (conditional) likelihood. An approximate solution can be achieved ignoring this dependence, but this approach could result in biased parameter estimates and in wrong inferences. We propose a computationally feasible approach to overcome this problem, leaving the random effect distribution unspecified. In this context, we show how the EM algorithm for nonparametric maximum likelihood (NPML) can be extended to deal with dependence of repeated measures on baseline counts.     

A Conditional Density Approach to the Order Determination of Time Series

The study focuses on the selection of the order of a general time series process via the conditional density of the latter, a characteristic of which is that it remains constant for every order beyond the true one. Using simulated time series from various nonlinear models we illustrate how this feature can be traced from conditional density estimation. We study whether two statistics derived from the likelihood function can serve as univariate statistics to determine the order of the process. It is found that a weighted version of the log likelihood function has desirable robust properties in detecting the order of the process.     

Parametric inference based on judgment post stratified samples

In this paper, we consider a judgment post stratified (JPS) sample of set size $H$ from a location and scale family of distributions. In a JPS sample, ranks of measured units are random variables. By conditioning on these ranks, we derive the maximum likelihood (MLEs) and best linear unbiased estimators (BLUEs) of the location and scale parameters. Since ranks are random variables, by considering the conditional distributions of ranks given the measured observations we construct Rao-Blackwellized version of MLEs and BLUEs. We show that Rao-Blackwellized estimators always have smaller mean squared errors than MLEs and BLUEs in a JPS sample. In addition, the paper provides empirical evidence for the efficiency of the proposed estimators through a series of Monte Carlo simulations.     

Bootstrapping Periodic State-Space Models

This article is concerned with how the bootstrap can be applied to study conditional forecast error distributions and construct prediction regions for future observations in periodic time-varying state-space models. We derive, first, an algorithm for assessing the precision of quasi-maximum likelihood estimates of the parameters. As a result, the derived algorithm is exploited for numerically evaluating the conditional forecast accuracy of a periodic time series model expressed in state space form. We propose a method which requires the backward, or reverse-time, representation of the model for assessing conditional forecast errors. Finally, the small sample properties of the proposed procedures will be investigated by some simulation studies. Furthermore, we illustrate the results by applying the proposed method to a real time series.     

Bias in Multinomial Models

This paper investigates bias in parameter estimates and residual diagnostics for parametric multinomial models by considering the effect of deleting a cell. In particular, it describes the average changes in the standardized residuals and maximum likelihood estimates resulting from conditioning on the given cells. These changes suggest how individual cell observations affect biases. Emphasis is placed on the role of individual cell observations in determining bias and on how bias affects standard diagnostic methods. Examples from genetics and log–linear models are considered. Numerical results show that conditioning on an influential cell results in substantial changes in biases.     

Bayesian partial likelihood approach for tied observations

The Bayesian analysis based on the partial likelihood for Cox's proportional hazards model is frequently used because of its simplicity. The Bayesian partial likelihood approach is often justified by showing that it approximates the full Bayesian posterior of the regression coefficients with a diffuse prior on the baseline hazard function. This, however, may not be appropriate when ties exist among uncensored observations. In that case, the full Bayesian and Bayesian partial likelihood posteriors can be much different. In this paper, we propose a new Bayesian partial likelihood approach for many tied observations and justify its use.     

Semiparametric Likelihood Based Method for Goodness of Fit Tests and Estimation in Upgraded Mixture Models

We use Owen's (1988, 1990) empirical likelihood method in upgraded mixture models. Two groups of independent observations are available. One is z ₁, ..., z _n which is observed directly from a distribution F ( z ). The other one is x ₁, ..., x _m which is observed indirectly from F ( z ), where the x _i s have density ∫ p ( x | z ) dF ( z ) and p ( x | z ) is a conditional density function. We are interested in testing H ₀: p ( x | z ) = p ( x | z ; θ ), for some specified smooth density function. A semiparametric likelihood ratio based statistic is proposed and it is shown that it converges to a chi-squared distribution. This is a simple method for doing goodness of fit tests, especially when x is a discrete variable with finitely many values. In addition, we discuss estimation of θ and F ( z ) when H ₀ is true. The connection between upgraded mixture models and general estimating equations is pointed out.     

COVARIANCE MODELS FOR LATTICE DATA

Given observations on an m × n lattice, approximate maximum likelihood estimates are derived for a family of models including direct covariance, spatial moving average, conditional autoregressive and simultaneous autoregressive models. The approach involves expressing the (approximate) covariance matrix of the observed variables in terms of a linear combination of neighbour relationship matrices, raised to a power. The structure is such that the eigenvectors of the covariance matrix are independent of the parameters of interest. This result leads to a simple Fisher scoring type algorithm for estimating the parameters. The ideas are illustrated by fitting models to some remotely sensed data.     

On the Convergence of the Monte Carlo Maximum Likelihood Method for Latent Variable Models

While much used in practice, latent variable models raise challenging estimation problems due to the intractability of their likelihood. Monte Carlo maximum likelihood (MCML), as proposed by Geyer & Thompson (1992 ), is a simulation-based approach to maximum likelihood approximation applicable to general latent variable models. MCML can be described as an importance sampling method in which the likelihood ratio is approximated by Monte Carlo averages of importance ratios simulated from the complete data model corresponding to an arbitrary value of the unknown parameter. This paper studies the asymptotic (in the number of observations) performance of the MCML method in the case of latent variable models with independent observations. This is in contrast with previous works on the same topic which only considered conditional convergence to the maximum likelihood estimator, for a fixed set of observations. A first important result is that when is fixed, the MCML method can only be consistent if the number of simulations grows exponentially fast with the number of observations. If on the other hand, is obtained from a consistent sequence of estimates of the unknown parameter, then the requirements on the number of simulations are shown to be much weaker.     

Comparisons of the Unbiased Ridge Estimation to the Other Estimations

In this article, we employ the method of empirical likelihood to construct confidence intervals of conditional density for a left-truncation model. It is proved that the empirical likelihood ratio admits a limiting chi-square distribution with one degree of freedom when the lifetime observations with multivariate covariates form a stationary α-mixing sequence.     

Variable selection for model-based clustering using the integrated complete-data likelihood

Variable selection in cluster analysis is important yet challenging. It can be achieved by regularization methods, which realize a trade-off between the clustering accuracy and the number of selected variables by using a lasso-type penalty. However, the calibration of the penalty term can suffer from criticisms. Model selection methods are an efficient alternative, yet they require a difficult optimization of an information criterion which involves combinatorial problems. First, most of these optimization algorithms are based on a suboptimal procedure (e.g. stepwise method). Second, the algorithms are often computationally expensive because they need multiple calls of EM algorithms. Here we propose to use a new information criterion based on the integrated complete-data likelihood. It does not require the maximum likelihood estimate and its maximization appears to be simple and computationally efficient. The original contribution of our approach is to perform the model selection without requiring any parameter estimation. Then, parameter inference is needed only for the unique selected model. This approach is used for the variable selection of a Gaussian mixture model with conditional independence assumed. The numerical experiments on simulated and benchmark datasets show that the proposed method often outperforms two classical approaches for variable selection. The proposed approach is implemented in the R package VarSelLCM available on CRAN.     

Likelihood Inference for Multivariate Extreme Value Distributions Whose Spectral Vectors have known Conditional Distributions

Multivariate extreme value statistical analysis is concerned with observations on several variables which are thought to possess some degree of tail dependence. The main approaches to inference for multivariate extremes consist in approximating either the distribution of block component‐wise maxima or the distribution of the exceedances over a high threshold. Although the expressions of the asymptotic density functions of these distributions may be characterized, they cannot be computed in general. In this paper, we study the case where the spectral random vector of the multivariate max‐stable distribution has known conditional distributions. The asymptotic density functions of the multivariate extreme value distributions may then be written through univariate integrals that are easily computed or simulated. The asymptotic properties of two likelihood estimators are presented, and the utility of the method is examined via simulation.     

Can we recover information from concordant pairs in binary matched pairs?

When possible values of a response variable are limited, distributional assumptions about random effects may not be checkable. This may cause a distribution-robust estimator, such as the conditional maximum likelihood estimator to be recommended; however, it does not utilize all the information in the data. We show how, with binary matched pairs, the hierarchical likelihood can be used to recover information from concordant pairs, giving an improvement over the conditional maximum likelihood estimator without losing distribution-robustness.     

The generalized symmetric Laplace distribution on the sphere

A distribution on the unit sphere is generated by conditioning a normal mixture distribution with an inverse gamma distributed weighting function. It can be regarded as the generalized symmetric Laplace distribution on the unit sphere. The density involves a modified Bessel function of the third kind which can be approximated by other simpler functions in certain limiting cases. As a consequence, the von Mises–Fisher, cardioid and Jones–Pewsey distributions are limiting cases of the new distribution. No closed form expressions exist for the roots of the likelihood equations. However, given the normal mixture structure of the distribution, we propose an E–M-algorithm-based approach for finding the maximum-likelihood estimates of the parameters which assumes the weights in the mixture to be missing data. The modeling capabilities of the proposed distribution are illustrated by fitting it and some of its competitors to two circular data sets.     

Doubly robust empirical likelihood inference in covariate-missing data problems

Missing covariate data occurs often in regression analysis. We study methods for estimating the regression coefficients in an assumed conditional mean function when some covariates are completely observed but other covariates are missing for some subjects. We adopt the semiparametric perspective of Robins et al. [Estimation of regression coefficients when some regressors are not always observed. J Amer Statist Assoc. 1994;89:846–866] on regression analyses with missing covariates, in which they pioneered the use of two working models, the working propensity score model and the working conditional score model. A recent approach to missing covariate data analysis is the empirical likelihood method of Qin et al. [Empirical likelihood in missing data problems. J Amer Statist Assoc. 2009;104:1492–1503], which effectively combines unbiased estimating equations. In this paper, we consider an alternative likelihood approach based on the full likelihood of the observed data. This full likelihood-based method enables us to generate estimators for the vector of the regression coefficients that are (a) asymptotically equivalent to those of Qin et al. [Empirical likelihood in missing data problems. J Amer Statist Assoc. 2009;104:1492–1503] when the working propensity score model is correctly specified, and (b) doubly robust, like the augmented inverse probability weighting (AIPW) estimators of Robins et al. [Estimation of regression coefficients when some regressors are not always observed. J Am Statist Assoc. 1994;89:846–866]. Thus, the proposed full likelihood-based estimators improve on the efficiency of the AIPW estimators when the working propensity score model is correct but the working conditional score model is possibly incorrect, and also improve on the empirical likelihood estimators of Qin, Zhang and Leung [Empirical likelihood in missing data problems. J Amer Statist Assoc. 2009;104:1492–1503] when the reverse is true, that is, the working conditional score model is correct but the working propensity score model is possibly incorrect. In addition, we consider a regression method for estimation of the regression coefficients when the working conditional score model is correctly specified; the asymptotic variance of the resulting estimator is no greater than the semiparametric variance bound characterized by the theory of Robins et al. [Estimation of regression coefficients when some regressors are not always observed. J Amer Statist Assoc. 1994;89:846–866]. Finally, we compare the finite-sample performance of various estimators in a simulation study.     

Bivariate Kumaraswamy distribution: properties and a new method to generate bivariate classes

In this paper, we introduce a bivariate Kumaraswamy (BVK) distribution whose marginals are Kumaraswamy distributions. The cumulative distribution function of this bivariate model has absolutely continuous and singular parts. Representations for the cumulative and density functions are presented and properties such as marginal and conditional distributions, product moments and conditional moments are obtained. We show that the BVK model can be obtained from the Marshall and Olkin survival copula and obtain a tail dependence measure. The estimation of the parameters by maximum likelihood is discussed and the Fisher information matrix is determined. We propose an EM algorithm to estimate the parameters. Some simulations are presented to verify the performance of the direct maximum-likelihood estimation and the proposed EM algorithm. We also present a method to generate bivariate distributions from our proposed BVK distribution. Furthermore, we introduce a BVK distribution which has only an absolutely continuous part and discuss some of its properties. Finally, a real data set is analysed for illustrative purposes.     

Stepwise likelihood ratio statistics in sequential studies

Summary.  It is well known that in a sequential study the probability that the likelihood ratio for a simple alternative hypothesis H ₁ versus a simple null hypothesis H ₀ will ever be greater than a positive constant c will not exceed 1/ c under H ₀. However, for a composite alternative hypothesis, this bound of 1/ c will no longer hold when a generalized likelihood ratio statistic is used. We consider a stepwise likelihood ratio statistic which, for each new observation, is updated by cumulatively multiplying the ratio of the conditional likelihoods for the composite alternative hypothesis evaluated at an estimate of the parameter obtained from the preceding observations versus the simple null hypothesis. We show that, under the null hypothesis, the probability that this stepwise likelihood ratio will ever be greater than c will not exceed 1/ c . In contrast, under the composite alternative hypothesis, this ratio will generally converge in probability to ∞. These results suggest that a stepwise likelihood ratio statistic can be useful in a sequential study for testing a composite alternative versus a simple null hypothesis. For illustration, we conduct two simulation studies, one for a normal response and one for an exponential response, to compare the performance of a sequential test based on a stepwise likelihood ratio statistic with a constant boundary versus some existing approaches.     

Likelihood Inference for Unions of Interacting Discs

Abstract. This is probably the first paper which discusses likelihood inference for a random set using a germ‐grain model, where the individual grains are unobservable, edge effects occur and other complications appear. We consider the case where the grains form a disc process modelled by a marked point process, where the germs are the centres and the marks are the associated radii of the discs. We propose to use a recent parametric class of interacting disc process models, where the minimal sufficient statistic depends on various geometric properties of the random set, and the density is specified with respect to a given marked Poisson model (i.e. a Boolean model). We show how edge effects and other complications can be handled by considering a certain conditional likelihood. Our methodology is illustrated by analysing Peter Diggle's heather data set, where we discuss the results of simulation‐based maximum likelihood inference and the effect of specifying different reference Poisson models.     

Ascertainment-Adjusted Maximum Likelihood Estimation for the Additive Genetic Gamma Frailty Model

The additive genetic gamma frailty model has been proposed for genetic linkage analysis for complex diseases to account for variable age of onset and possible covariates effects. To avoid ascertainment biases in parameter estimates, retrospective likelihood ratio tests are often used, which may result in loss of efficiency due to conditioning. This paper considers when the sibships are ascertained by having at least two affected sibs with the disease before a given age and provides two approaches for estimating the parameters in the additive gamma frailty model. One approach is based on the likelihood function conditioning on the ascertainment event, the other is based on maximizing a full ascertainment-adjusted likelihood. Explicit forms for these likelihood functions are derived. Simulation studies indicate that when the baseline hazard function can be correctly pre-specified, both approaches give accurate estimates of the model parameters. However, when the baseline hazard function has to be estimated simultaneously, only the ascertainment-adjusted likelihood method gives an unbiased estimate of the parameters. These results imply that the ascertainment-adjusted likelihood ratio test in the context of the additive genetic gamma frailty may be used for genetic linkage analysis.