Relative Risk Regression for Binary Outcomes: Methods and Recommendations

Relative risks are often considered preferable to odds ratios for quantifying the association between a predictor and a binary outcome. Relative risk regression is an alternative to logistic regression where the parameters are relative risks rather than odds ratios. It uses a log link binomial generalised linear model, or log‐binomial model, which requires parameter constraints to prevent probabilities from exceeding 1. This leads to numerical problems with standard approaches for finding the maximum likelihood estimate (MLE), such as Fisher scoring, and has motivated various non‐MLE approaches. In this paper we discuss the roles of the MLE and its main competitors for relative risk regression. It is argued that reliable alternatives to Fisher scoring mean that numerical issues are no longer a motivation for non‐MLE methods. Nonetheless, non‐MLE methods may be worthwhile for other reasons and we evaluate this possibility for alternatives within a class of quasi‐likelihood methods. The MLE obtained using a reliable computational method is recommended, but this approach requires bootstrapping when estimates are on the parameter space boundary. If convenience is paramount, then quasi‐likelihood estimation can be a good alternative, although parameter constraints may be violated. Sensitivity to model misspecification and outliers is also discussed along with recommendations and priorities for future research.     

A profile likelihood method for normal mixture with unequal variance

It is well known that the normal mixture with unequal variance has unbounded likelihood and thus the corresponding global maximum likelihood estimator (MLE) is undefined. One of the commonly used solutions is to put a constraint on the parameter space so that the likelihood is bounded and then one can run the EM algorithm on this constrained parameter space to find the constrained global MLE. However, choosing the constraint parameter is a difficult issue and in many cases different choices may give different constrained global MLE. In this article, we propose a profile log likelihood method and a graphical way to find the maximum interior mode. Based on our proposed method, we can also see how the constraint parameter, used in the constrained EM algorithm, affects the constrained global MLE. Using two simulation examples and a real data application, we demonstrate the success of our new method in solving the unboundness of the mixture likelihood and locating the maximum interior mode.     

Linear approximate ML estimation in scaled Type I generalized logistic distribution based on Type-II censored samples

The scaled (two-parameter) Type I generalized logistic distribution (GLD) is considered with the known shape parameter. The ML method does not yield an explicit estimator for the scale parameter even in complete samples. In this article, we therefore construct a new linear estimator for scale parameter, based on complete and doubly Type-II censored samples, by making linear approximations to the intractable terms of the likelihood equation using least-squares (LS) method, a new approach of linearization. We call this as linear approximate maximum likelihood estimator (LAMLE). We also construct LAMLE based on Taylor series method of linear approximation and found that this estimator is slightly biased than that based on the LS method. A Monte Carlo simulation is used to investigate the performance of LAMLE and found that it is almost as efficient as MLE, though biased than MLE. We also compare unbiased LAMLE with BLUE based on the exact variances of the estimators and interestingly this new unbiased LAMLE is found just as efficient as the BLUE in both complete and Type-II censored samples. Since MLE is known as asymptotically unbiased, in large samples we compare unbiased LAMLE with MLE and found that this estimator is almost as efficient as MLE. We have also discussed interval estimation of the scale parameter from complete and Type-II censored samples. Finally, we present some numerical examples to illustrate the construction of the new estimators developed here.     

Estimation of the parameters of a truncated gamma distribution

This paper deals with the estimation of the parameters of a truncated gamma distribution over (0,τ), where τ is assumed to be a real number. We obtain a necessary and sufficient condition for the existence of the maximum likelihood estimator(MLE). The probability of nonexistence of MLE is observed to be positive. A simulation study indicates that the modified maximum likelihood estimator and the mixed estimator, which exist with probability one,are to be preferred over MLE. The bias, the mean square error, and the probability of nearness form a basis of our simulation study.     

Penalized maximum likelihood estimation for Gaussian hidden Markov models

ABSTRACT

The likelihood function of a Gaussian hidden Markov model is unbounded, which is why the maximum likelihood estimator (MLE) is not consistent. A penalized MLE is introduced along with a rigorous consistency proof.     

Inference about regression parameters using highly stratified survey count data with over-dispersion and repeated measurements

We study methods to estimate regression and variance parameters for over-dispersed and correlated count data from highly stratified surveys. Our application involves counts of fish catches from stratified research surveys and we propose a novel model in fisheries science to address changes in survey protocols. A challenge with this model is the large number of nuisance parameters which leads to computational issues and biased statistical inferences. We use a computationally efficient profile generalized estimating equation method and compare it to marginal maximum likelihood (MLE) and restricted MLE (REML) methods. We use REML to address bias and inaccurate confidence intervals because of many nuisance parameters. The marginal MLE and REML approaches involve intractable integrals and we used a new R package that is designed for estimating complex nonlinear models that may include random effects. We conclude from simulation analyses that the REML method provides more reliable statistical inferences among the three methods we investigated.     

Asymptotic properties of MLE for partially observed fractional diffusion system with dependent noises

The paper studies long time asymptotic properties of the maximum likelihood estimator (MLE) for the signal drift parameter in a partially observed fractional diffusion system with dependent noise. Using the method of weak convergence of likelihoods due to Ibragimov and Khasminskii [1981. Statistics of Random Processes. Springer, New-York], consistency, asymptotic normality and convergence of the moments are established for MLE. The proof is based on Laplace transform computations which was introduced in Brouste and Kleptsyna [2008. Asymptotic properties of MLE for partially observed fractional diffusion system, preprint].     

Semiparametric Regression with Kernel Error Model

Abstract.  We propose and study a class of regression models, in which the mean function is specified parametrically as in the existing regression methods, but the residual distribution is modelled non-parametrically by a kernel estimator, without imposing any assumption on its distribution. This specification is different from the existing semiparametric regression models. The asymptotic properties of such likelihood and the maximum likelihood estimate (MLE) under this semiparametric model are studied. We show that under some regularity conditions, the MLE under this model is consistent (when compared with the possibly pseudo-consistency of the parameter estimation under the existing parametric regression model), is asymptotically normal with rate and efficient. The non-parametric pseudo-likelihood ratio has the Wilks property as the true likelihood ratio does. Simulated examples are presented to evaluate the accuracy of the proposed semiparametric MLE method.     

A simple root selection method for univariate finite normal mixture models

It is well known that there exist multiple roots of the likelihood equations for finite normal mixture models. Selecting a consistent root for finite normal mixture models has long been a challenging problem. Simply using the root with the largest likelihood will not work because of the spurious roots. In addition, the likelihood of normal mixture models with unequal variance is unbounded and thus its maximum likelihood estimate (MLE) is not well defined. In this paper, we propose a simple root selection method for univariate normal mixture models by incorporating the idea of goodness of fit test. Our new method inherits both the consistency properties of distance estimators and the efficiency of the MLE. The new method is simple to use and its computation can be easily done using existing R packages for mixture models. In addition, the proposed root selection method is very general and can be also applied to other univariate mixture models. We demonstrate the effectiveness of the proposed method and compare it with some other existing methods through simulation studies and a real data application.     

Estimation of system reliability for exponential distributions based on L ranked set sampling

Abstract

In the case where strength and stress both follow exponential distributions, this paper considers the maximum likelihood estimator (MLE) of the system reliability based on L ranked set sampling (LRSS). The proposed MLE is shown to have existence, uniqueness and asymptotic normality, and its asymptotic variance is obtained by the Fisher information matrix of LRSS. The values of asymptotic relative efficiencies show that the proposed MLE is always more efficient than the MLE using simple random sampling (SRS). However, the MLE using LRSS cannot be written in closed form. Therefore, the modified MLE is proposed using the technique replaced some terms in the maximum likelihood equations by their expectations. The newly modified MLE using LRSS is shown to be superior to the MLE using SRS. Finally, the proposed method is applied to a real data set on metastatic renal carcinoma study.     

Semi-parametric modelling and likelihood estimation with estimating equations

This paper proposes a semi-parametric modelling and estimating method for analysing censored survival data. The proposed method uses the empirical likelihood function to describe the information in data, and formulates estimating equations to incorporate knowledge of the underlying distribution and regression structure. The method is more flexible than the traditional methods such as the parametric maximum likelihood estimation (MLE), Cox's (1972) proportional hazards model, accelerated life test model, quasi-likelihood (Wedderburn, 1974) and generalized estimating equations (Liang & Zeger, 1986). This paper shows the existence and uniqueness of the proposed semi-parametric maximum likelihood estimates (SMLE) with estimating equations. The method is validated with known cases studied in the literature. Several finite sample simulation and large sample efficiency studies indicate that when the sample size is larger than 100 the SMLE is compatible with the parametric MLE; and in all case studies, the SMLE is about 15% better than the parametric MLE with a mis-specified underlying distribution.     

Estimation of the parameters in a truncated normal distribution

This paper deals with the estimation of the parameters of doubly truncated and singly truncated normal distributions when truncation points are known. We derive, for these families, a necessary and sufficient condition for the maximum likelihood estimator(MLE) to be finite. Furthermore, the probability of the MLE being infinite is positive. A simulation study for single truncation is carried out to compare the modified maximum likelihood estimator, and the mixed estimator.     

Bias reduction of maximum likelihood estimates for a modified skew-normal distribution

ABSTRACT

This paper presents a modified skew-normal (SN) model that contains the normal model as a special case. Unlike the usual SN model, the Fisher information matrix of the proposed model is always non-singular. Despite of this desirable property for the regular asymptotic inference, as with the SN model, in the considered model the divergence of the maximum likelihood estimator (MLE) of the skewness parameter may occur with positive probability in samples with moderate sizes. As a solution to this problem, a modified score function is used for the estimation of the skewness parameter. It is proved that the modified MLE is always finite. The quasi-likelihood approach is considered to build confidence intervals. When the model includes location and scale parameters, the proposed method is combined with the unmodified maximum likelihood estimates of these parameters.     

Estimates for mean and dispersion effects in unreplicated factorial designs

One important goal of experimentation in quality improvement is to minimize the variability of a product or process around a target mean value. Factors which affect variances as well as factors that affect the mean can be identified using the analysis of mean and dispersion. Box and Meyer (1986b) proposed a method of model identification and maximum likelihood estimation for mean and dispersion effects from unreplicated designs. In this article, we address two problems associated with MLE’s. First, asymptotic variance of MLE's for dispersion effects which can be used to judge the significance of factors can be misleading. A possible explanation is provided; simulation results also indicate that the asymptotic, variance underestimates.     

Implementing particle swarm optimization algorithm to estimate the mixture of two Weibull parameters with censored data

We present the maximum likelihood estimation (MLE) via particle swarm optimization (PSO) algorithm to estimate the mixture of two Weibull parameters with complete and multiple censored data. A simulation study is conducted to assess the performance of the MLE via PSO algorithm, quasi-Newton method and expectation-maximization (EM) algorithm for different parameter settings and sample sizes in both uncensored and censored cases. The simulation results showed that the PSO algorithm outperforms the quasi-Newton method and the EM algorithm in most cases regarding bias and root mean square errors. Two numerical examples are used to demonstrate the performance of our proposed method.     

Approximate MLE for the scale parameter of the generalized exponential distribution under random censoring

In this paper, we consider the maximum likelihood estimator (MLE) of the scale parameter of the generalized exponential (GE) distribution based on a random censoring model. We assume the censoring distribution also follows a GE distribution. Since the estimator does not provide an explicit solution, we propose a simple method of deriving an explicit estimator by approximating the likelihood function. In order to compare the performance of the estimators, Monte Carlo simulation is conducted. The results show that the MLE and the approximate MLE are almost identical in terms of bias and variance.     

Identity for the NPMLE in Censored Data Models

We derive an identity for nonparametric maximum likelihood estimators (NPMLE) and regularized MLEs in censored data models which expresses the standardized maximum likelihood estimator in terms of the standardized empirical process. This identity provides an effective starting point in proving both consistency and efficiency of NPMLE and regularized MLE. The identity and corresponding method for proving efficiency is illustrated for the NPMLE in the univariate right-censored data model, the regularized MLE in the current status data model and for an implicit NPMLE based on a mixture of right-censored and current status data. Furthermore, a general algorithm for estimation of the limiting variance of the NPMLE is provided. This revised version was published online in July 2006 with corrections to the Cover Date.     

Maximum likelihood estimation for a nearly random walk model

This paper proposes an effective reparameterization method for the maximum likelihood estimation of a nearly random walk ARIMA (1,1,1) model, an important case where standard method of locating the MLE is not satisfactory. This model is equivalent to the permanent and temporary components model that Fama &French (1988) and others used to capture the slow mean reversion behavior of stock prices. The reparameterization method we prppose for estimating the nearly cancelled AR and MA parameters performs satisfactorily. The exact likelihood function based on the transformed parameters is studied. We argue that the region of interest will get magnified and emphasized in the transformed space, thus making the search for MLE more thorough and effective. Substantiai simuiation evidences are provided to demonstrate the effectiveness of the method. The sample size requirement is critical and is discussed in details. For application, this method is applied to estimate a nearly random walk ARIMA (1,1,1) model for NYSE/AMEX value-weighted market return in daily and longer holding-period horizons.     

Estimation in the presence of incidental parameters

This paper considers the estimation of “structural” parameters when the number of unknown parameters increases with the sample size. Neyman and Scott (1948) had demonstrated that maximum likelihood estimators (MLE) of structural parameters may be inconsistent in this case. Patefield (1977) further observed that the asymptotic covariance matrix of the MLE is not equal to the inverse of the information matrix. In this paper we establish asymptotic properties of estimators (which include in particular the MLE) obtained via the usual likelihood approach when the incidental parameters are first replaced by their estimates (which are allowed to depend on the structural parameters). Conditions for consistency and asymptotic normality together with a proper formula for the asymptotic covariance matrix are given. The results are illustrated and applied to the problem of estimating linear functional relationships, and mild conditions on the incidental parameters for the MLE (or an adjusted MLE) to be consistent and asymptotically normal are obtained. These conditions are weaker than those imposed by previous authors.     

The Minimum Density Power Divergence Estimation for the Lognormal Density

In this article, we implement the minimum density power divergence estimation for estimating the parameters of the lognormal density. We compare the minimum density power divergence estimator (MDPDE) and the maximum likelihood estimator (MLE) in terms of robustness and asymptotic distribution. The simulations and an example indicate that the MDPDE is less biased than MLE and is as good as MLE in terms of the mean square error under various distributional situations.