MAXIMUM LIKELIHOOD ESTIMATION IN LINEAR MODELS WITH EQUI-CORRELATED RANDOM ERRORS

Necessary and sufficient conditions for the existence of maximum likelihood estimators of unknown parameters in linear models with equi‐correlated random errors are presented. The basic technique we use is that these models are, first, orthogonally transformed into linear models with two variances, and then the maximum likelihood estimation problem is solved in the environment of transformed models. Our results generalize a result of Arnold, S. F. (1981) [The theory of linear models and multivariate analysis. Wiley, New York]. In addition, we give necessary and sufficient conditions for the existence of restricted maximum likelihood estimators of the parameters. The results of Birkes, D. & Wulff, S. (2003) [Existence of maximum likelihood estimates in normal variance‐components models. J Statist Plann. Inference. 113 , 35–47] are compared with our results and differences are pointed out.     

Simultaneous sparse model selection and coefficient estimation for heavy-tailed autoregressive processes

We propose a sparse coefficient estimation and automated model selection procedure for autoregressive processes with heavy-tailed innovations based on penalized conditional maximum likelihood. Under mild moment conditions on the innovation processes, the penalized conditional maximum likelihood estimator satisfies a strong consistency, O_P(N^?1/2) consistency, and the oracle properties, where N is the sample size. We have the freedom in choosing penalty functions based on the weak conditions on them. Two penalty functions, least absolute shrinkage and selection operator and smoothly clipped average deviation, are compared. The proposed method provides a distribution-based penalized inference to AR models, which is especially useful when the other estimation methods fail or under perform for AR processes with heavy-tailed innovations [Feigin, Resnick. Pitfalls of fitting autoregressive models for heavy-tailed time series. Extremes. 1999;1:391–422]. A simulation study confirms our theoretical results. At the end, we apply our method to a historical price data of the US Industrial Production Index for consumer goods, and obtain very promising results.     

Bayesian inference based on a jointly type-II censored sample from two exponential populations

In this paper, based on a jointly type-II censored sample from two exponential populations, the Bayesian inference for the two unknown parameters are developed with the use of squared-error, linear-exponential and general entropy loss functions. The problem of predicting the future failure times, both point and interval prediction, based on the observed joint type-II censored data, is also addressed from a Bayesian viewpoint. A Monte Carlo simulation study is conducted to compare the Bayesian estimators with the maximum likelihood estimator developed by Balakrishnan and Rasouli [Exact likelihood inference for two exponential populations under joint type-II censoring. Comput Stat Data Anal. 2008;52:2725–2738]. Finally, a numerical example is utilized for the purpose of illustration.     

Asymptotic likelihood inference for nonhomogeneous observations

This paper surveys asymptotic theory of maximum likelihood estimation for not identically distributed, possibly dependent observations. Main results on consistency, asymptotic normality and efficiency are stated within a unified framework. Limiting distributions of the likelihood ratio, Wald and score statistics for composite hypotheses are obtained under the same conditions by a generalization of existing theory. Modifications for maximum likelihood estimation under misspecification, containing the results for correctly specified models, are presented, and extensions to likelihood inference in the presence of nuisance parameters are indicated.     

Bayesian estimation for time-series regressions improved with exact likelihoods

We propose an estimation procedure for time-series regression models under the Bayesian inference framework. With the exact method of Wise [Wise, J. (1955). The autocorrelation function and spectral density function. Biometrika, 42, 151–159], an exact likelihood function can be obtained instead of the likelihood conditional on initial observations. The constraints on the parameter space arising from the stationarity conditions are handled by a reparametrization, which was not taken into consideration by Chib [Chib, S. (1993). Bayes regression with autoregressive errors: A Gibbs sampling approach. J. Econometrics, 58, 275–294] or Chib and Greenberg [Chib, S. and Greenberg, E. (1994). Bayes inference in regression model with ARMA(p, q) errors. J. Econometrics, 64, 183–206]. Simulation studies show that our method leads to better inferential results than their results.     

The gamma-Lomax distribution

For any continuous baseline G distribution, Zografos and Balakrishnan [On families of beta- and generalized gamma-generated distributions and associated inference. Statist Methodol. 2009;6:344–362] proposed a generalized gamma-generated distribution with an extra positive parameter. A new three-parameter continuous distribution called the gamma-Lomax distribution, which extends the Lomax distribution is proposed and studied. Various structural properties of the new distribution are derived including explicit expressions for the moments, generating and quantile functions, mean deviations and Rényi entropy. The estimation of the model parameters is performed by maximum likelihood. We also determine the observed information matrix. An application illustrates the usefulness of the proposed model.     

Exact likelihood inference based on an unified hybrid censored sample from the exponential distribution

Recently, exact confidence bounds and exact likelihood inference have been developed based on hybrid censored samples by Chen and Bhattacharyya [Chen, S. and Bhattacharyya, G.K. (1998). Exact confidence bounds for an exponential parameter under hybrid censoring. Communications in Statistics—Theory and Methods, 17, 1857–1870.], Childs et al. [Childs, A., Chandrasekar, B., Balakrishnan, N. and Kundu, D. (2003). Exact likelihood inference based on Type-I and Type-II hybrid censored samples from the exponential distribution. Annals of the Institute of Statistical Mathematics, 55, 319–330.], and Chandrasekar et al. [Chandrasekar, B., Childs, A. and Balakrishnan, N. (2004). Exact likelihood inference for the exponential distribution under generalized Type-I and Type-II hybrid censoring. Naval Research Logistics, 51, 994–1004.] for the case of the exponential distribution. In this article, we propose an unified hybrid censoring scheme (HCS) which includes many cases considered earlier as special cases. We then derive the exact distribution of the maximum likelihood estimator as well as exact confidence intervals for the mean of the exponential distribution under this general unified HCS. Finally, we present some examples to illustrate all the methods of inference developed here.     

Consistency of Bernstein polynomial posteriors

A Bernstein prior is a probability measure on the space of all the distribution functions on [0, 1]. Under very general assumptions, it selects absolutely continuous distribution functions, whose densities are mixtures of known beta densities. The Bernstein prior is of interest in Bayesian nonparametric inference with continuous data. We study the consistency of the posterior from a Bernstein prior. We first show that, under mild assumptions, the posterior is weakly consistent for any distribution function P ₀ on [0, 1] with continuous and bounded Lebesgue density. With slightly stronger assumptions on the prior, the posterior is also Hellinger consistent. This implies that the predictive density from a Bernstein prior, which is a Bayesian density estimate, converges in the Hellinger sense to the true density (assuming that it is continuous and bounded). We also study a sieve maximum likelihood version of the density estimator and show that it is also Hellinger consistent under weak assumptions. When the order of the Bernstein polynomial, i.e. the number of components in the beta distribution mixture, is truncated, we show that under mild restrictions the posterior concentrates on the set of pseudotrue densities. Finally, we study the behaviour of the predictive density numerically and we also study a hybrid Bayes–maximum likelihood density estimator.     

Structural inference for parameters of a power function distribution

We consider the problem of statistical inference on the parameters of the three parameter power function distribution based on a full unordered sample of observations or a type II censored ordered sample of observations. The inference philosophy used is the theory of structural inference. We state inference procedures which yield inferential statements about the three unknown parameters. A numerical example is given to illustrate these procedures. It is seen that within the context of this example the inference procedures of this paper do not encounter certain difficulties associated with classical maximum likelihood based procedures. Indeed it has been our numerical experience that this behavior is typical within the context of that subclass of the three parameter power function distribution to which this example belongs.     

Robust estimation in the normal mixture model

This paper focuses on the inference of the normal mixture model with unequal variances. A feature of the model is flexibility of density shape, but its flexibility causes the unboundedness of the likelihood function and excessive sensitivity of the maximum likelihood estimator to outliers. A modified likelihood approach suggested in Basu et al. [1998, Biometrika 85, 549–559] can overcome these drawbacks. It is shown that the modified likelihood function is bounded above under a mild condition on mixing proportions and the resultant estimator is robust to outliers. A relationship between robustness and efficiency is investigated and an adaptive method for selecting the tuning parameter of the modified likelihood is suggested, based on the robust model selection criterion and the cross-validation. An EM-like algorithm is also constructed. Numerical studies are presented to evaluate the performance. The robust method is applied to single nuleotide polymorphism typing for the purpose of outlier detection and clustering.     

A gradient-based algorithm for semiparametric models with missing covariates

In the parametric regression model, the covariate missing problem under missing at random is considered. It is often desirable to use flexible parametric or semiparametric models for the covariate distribution, which can reduce a potential misspecification problem. Recently, a completely nonparametric approach was developed by [H.Y. Chen, Nonparametric and semiparametric models for missing covariates in parameter regression, J. Amer. Statist. Assoc. 99 (2004), pp. 1176–1189; Z. Zhang and H.E. Rockette, On maximum likelihood estimation in parametric regression with missing covariates, J. Statist. Plann. Inference 47 (2005), pp. 206–223]. Although it does not require a model for the covariate distribution or the missing data mechanism, the proposed method assumes that the covariate distribution is supported only by observed values. Consequently, their estimator is a restricted maximum likelihood estimator (MLE) rather than the global MLE. In this article, we show the restricted semiparametric MLE could be very misleading in some cases. We discuss why this problem occurs and suggest an algorithm to obtain the global MLE. Then, we assess the performance of the proposed method via some simulation experiments.     

Isotonic estimation of the intensity of a nonhomogeneous Poisson process: The multiple realization setup

In the 1950s Brunk and Van Eeden each obtained maximum-likelihood estimators of a finite product of probability density functions under partial or complete ordering of their parameters. Their results play an important role in the general theory of inference under order restrictions and lead to an isotonic estimator of the intensity of a nonhomogeneous Poisson process. Here an elementary derivation of the maximum likelihood estimator (m.l.e.) for the intensity of a nonhomogeneous Poisson process is given when several (possibly censored) realizations are available. Boswell obtained the m.l.e. based on a single realization as well as a conditional m.l.e. under the same conditions. An example is given to show that in the multirealization setup a conditional m.l.e. may not exist; the proofs are, we believe, new and elementary. An illustrative application is given.     

Hypotheses testing for structural calibration model

The main objective of this paper is to discuss maximum likelihood inference for the comparative structural calibration model (Barnett, in Biometrics 25:129–142, 1969), which is frequently used in the problem of assessing the relative calibrations and relative accuracies of a set of p instruments, each designed to measure the same characteristic on a common group of n experimental units. We consider asymptotic tests to answer the outlined questions. The methodology is applied to a real data set and a small simulation study is presented.     

Exact likelihood inference for two exponential populations based on a joint generalized Type-I hybrid censored sample

Following the work of Chen and Bhattacharyya [Exact confidence bounds for an exponential parameter under hybrid censoring. Comm Statist Theory Methods. 1988;17:1857–1870], several results have been developed regarding the exact likelihood inference of exponential parameters based on different forms of censored samples. In this paper, the conditional maximum likelihood estimators (MLEs) of two exponential mean parameters are derived under joint generalized Type-I hybrid censoring on the two samples. The moment generating functions (MGFs) and the exact densities of the conditional MLEs are obtained, using which exact confidence intervals are then developed for the model parameters. We also derive the means, variances, and mean squared errors of these estimates. An efficient computational method is developed based on the joint MGF. Finally, an example is presented to illustrate the methods of inference developed here.     

Likelihood‐based inference for correlated diffusions

The authors address the problem of likelihood‐based inference for correlated diffusions. Such a task presents two issues; the positive definite constraints of the diffusion matrix and the likelihood intractability. The first issue is handled by using the Cholesky factorization on the diffusion matrix. To deal with the likelihood unavailability, a generalization of the data augmentation framework of Roberts and Stramer [Roberts and Stramer (2001) Biometrika 88(3), 603–621] to d‐dimensional correlated diffusions, including multivariate stochastic volatility models, is given. The methodology is illustrated through simulated and real data sets. The Canadian Journal of Statistics 39: 52–72; 2011 © 2011 Statistical Society of Canada     

Likelihood ratio comparisons among spacings related to both one or two samples

In this paper, we investigate some stochastic comparisons in terms of likelihood ratio ordering between spacings from independent random variables exponentially distributed with different scale parameters. We partially solve some open problems in [Wen S, Lu Q, Hu T. Likelihood ratio orderings of spacings of heterogeneous exponential random variables. J Multivariate Anal. 2007;98:743–756] for a one-sample problem and in [Hu T, Lu Q, Wen S. Stochastic comparisons and dependence of spacings from two samples of exponential random variables. Commun Stat – Theory Methods 2006;35:979–988] for a two-sample problem. Specifically, we prove that the second spacing is always smaller than the third spacing in terms of the likelihood ratio order and we provide the ordering among all spacings in the case n=4. In the two-sample case, we establish comparisons between the second spacings related to each sample under certain conditions.     

Copula representation of bivariate L-moments: a new estimation method for multiparameter two-dimensional copula models

Serfling and Xiao [A contribution to multivariate L-moments, L-comoment matrices. J Multivariate Anal. 2007;98:1765–1781] extended the L-moment theory to the multivariate setting. In the present paper, we focus on the two-dimensional random vectors to establish a link between the bivariate L-moments (BLM) and the underlying bivariate copula functions. This connection provides a new estimate of dependence parameters of bivariate statistical data. Extensive simulation study is carried out to compare estimators based on the BLM, the maximum likelihood, the minimum distance and a rank approximate Z-estimation. The obtained results show that, when the sample size increases, BLM-based estimation performs better as far as the bias and computation time are concerned. Moreover, the root-mean-squared error is quite reasonable and less sensitive in general to outliers than those of the above cited methods. Further, the proposed BLM method is an easy-to-use tool for the estimation of multiparameter copula models. A generalization of the BLM estimation method to the multivariate case is discussed.     

Likelihood ratio tests for variance components in linear mixed models

Although the asymptotic distributions of the likelihood ratio for testing hypotheses of null variance components in linear mixed models derived by Stram and Lee [1994. Variance components testing in longitudinal mixed effects model. Biometrics 50, 1171–1177] are valid, their proof is based on the work of Self and Liang [1987. Asymptotic properties of maximum likelihood estimators and likelihood tests under nonstandard conditions. J. Amer. Statist. Assoc. 82, 605–610] which requires identically distributed random variables, an assumption not always valid in longitudinal data problems. We use the less restrictive results of Vu and Zhou [1997. Generalization of likelihood ratio tests under nonstandard conditions. Ann. Statist. 25, 897–916] to prove that the proposed mixture of chi-squared distributions is the actual asymptotic distribution of such likelihood ratios used as test statistics for null variance components in models with one or two random effects. We also consider a limited simulation study to evaluate the appropriateness of the asymptotic distribution of such likelihood ratios in moderately sized samples.     

Assessing the bias of maximum likelihood estimates of contaminated garch models

It is well known that Gaussian maximum likelihood estimates of time series models are not robust. In this paper we prove this is also the case for the Generalized Autoregressive Conditional Heteroscedastic (GARCH) models. By expressing the Gaussian maximum likelihood estimates as Ψ estimates and by assuming the existence of a contaminated process, we prove they possess zero breakdown point and unbounded influence curves. By simulating GARCH processes under several proportions of contaminations we assess how much biased the maximum likelihood estimates may become and compare these results to a robust alternative. The t-student maximum likelihood estimates of GARCH models are also considered.     

Objective Bayesian Analysis of Skew‐t Distributions

Abstract. We study the Jeffreys prior and its properties for the shape parameter of univariate skew‐t distributions with linear and nonlinear Student's t skewing functions. In both cases, we show that the resulting priors for the shape parameter are symmetric around zero and proper. Moreover, we propose a Student's t approximation of the Jeffreys prior that makes an objective Bayesian analysis easy to perform. We carry out a Monte Carlo simulation study that demonstrates an overall better behaviour of the maximum a posteriori estimator compared with the maximum likelihood estimator. We also compare the frequentist coverage of the credible intervals based on the Jeffreys prior and its approximation and show that they are similar. We further discuss location‐scale models under scale mixtures of skew‐normal distributions and show some conditions for the existence of the posterior distribution and its moments. Finally, we present three numerical examples to illustrate the implications of our results on inference for skew‐t distributions.