Estimating the correlation coefficient in the presence of correlated observations from a bivariate normal population

Estimation of the correlation coefficient between two variates (p) in the presence of correlated observations from a bivar iate normal population is considered The estimated maximum likelihood estimator (EMLE), an estimate based on the maximum likelihood estimator (MLE), is proposed and studied for the estimation of p For the large sample case , approximate expressions foi the variance and the bias of the Pearson estimate of the correlation coefficient are derived. These expressions suggests that the Pearson’s estimator possesses high mean square error (MSE) in estimating ρ in comparison to the MLE The MSE is particularly high when the observations within clusters aie highly correlated. The Pearson’s estimate, the MLE, and the EMLE aie evaluated in a simulation study This study shows that the proposed EMLE pefoims bettei than the Pearson’s correlation coefficient except when the number of clusters is small.     

Response shrinkage estimation in multinomial logit models

A shrinkage estimation method for multinomial logit models is developed. The proposed method is based on shrinking the responses for each category towards the underlying probabilities. The estimator is also used in combination with Pregibon's resistant fitting. The resulting estimator can also be used to control the over-estimation of Pregibon's resistant estimator. The proposed method handles not only the problem of separation in multinomial logit models but estimates also exist when the number of covariates is large relative to the sample size. Estimates exist even when the MLE does not exist. Estimates can be easily computed with all commonly used statistical packages supporting the fitting procedures with weights. Estimates are compared with the usual MLE and Firth's bias reduction technique in a simulation study and an application.     

Numerical maximum likelihood estimation for the g-and-k and generalized g-and-h distributions

Continuing increases in computing power and availability mean that many maximum likelihood estimation (MLE) problems previously thought intractable or too computationally difficult can now be tackled numerically. However, ML parameter estimation for distributions whose only analytical expression is as quantile functions has received little attention. Numerical MLE procedures for parameters of new families of distributions, the g-and-k and the generalized g-and-h distributions, are presented and investigated here. Simulation studies are included, and the appropriateness of using asymptotic methods examined. Because of the generality of these distributions, the investigations are not only into numerical MLE for these distributions, but are also an initial investigation into the performance and problems for numerical MLE applied to quantile-defined distributions in general. Datasets are also fitted using the procedures here. Results indicate that sample sizes significantly larger than 100 should be used to obtain reliable estimates through maximum likelihood.     

Optimum percentile estimating equations for nonlinear random coefficient models

In nonlinear random coefficients models, the means or variances of response variables may not exist. In such cases, commonly used estimation procedures, e.g., (extended) least-squares (LS) and quasi-likelihood methods, are not applicable. This article solves this problem by proposing an estimate based on percentile estimating equations (PEE). This method does not require full distribution assumptions and leads to efficient estimates within the class of unbiased estimating equations. By minimizing the asymptotic variance of the PEE estimates, the optimum percentile estimating equations (OPEE) are derived. Several examples including Weibull regression show the flexibility of the PEE estimates. Under certain regularity conditions, the PEE estimates are shown to be strongly consistent and asymptotic normal, and the OPEE estimates have the minimal asymptotic variance. Compared with the parametric maximum likelihood estimates (MLE), the asymptotic efficiency of the OPEE estimates is more than 98%, while the LS-type of procedures can have infinite variances. When the observations have outliers or do not follow the distributions considered in model assumptions, the article shows that OPEE is more robust than the MLE, and the asymptotic efficiency in the model misspecification cases can be above 150%.     

Firth adjustment for Weibull current-status survival analysis

Abstract

Analysis of right-censored data is problematic due to infinite maximum likelihood estimates (MLE) and potentially biased estimates, especially for small numbers of events. Analyzing current-status data is especially troublesome because of the extreme loss of precision due to large failure intervals. We extend Firth’s method for regular parametric problems to current-status modeling with the Weibull distribution. Firth advocated a bias reduction method for MLE by systematically correcting the score equation. An advantage is that it is still applicable when the MLE does not exist. We present simulation studies and two illustrative analyses involving RFM mice lung tumor data.     

Estimation of the scale parameter of the half-logistic distribution with multiply type II censored sample

In this paper, we consider the maximum likelihood and Bayes estimation of the scale parameter of the half-logistic distribution based on a multiply type II censored sample. However, the maximum likelihood estimator(MLE) and Bayes estimator do not exist in an explicit form for the scale parameter. We consider a simple method of deriving an explicit estimator by approximating the likelihood function and discuss the asymptotic variances of MLE and approximate MLE. Also, an approximation based on the Laplace approximation (Tierney & Kadane, 1986) is used to obtain the Bayes estimator. In order to compare the MLE, approximate MLE and Bayes estimates of the scale parameter, Monte Carlo simulation is used.     

A stochastic approximation algorithm for maximum-likelihood estimation with incomplete data

We propose a new stochastic approximation (SA) algorithm for maximum-likelihood estimation (MLE) in the incomplete-data setting. This algorithm is most useful for problems when the EM algorithm is not possible due to an intractable E-step or M-step. Compared to other algorithm that have been proposed for intractable EM problems, such as the MCEM algorithm of Wei and Tanner (1990), our proposed algorithm appears more generally applicable and efficient. The approach we adopt is inspired by the Robbins-Monro (1951) stochastic approximation procedure, and we show that the proposed algorithm can be used to solve some of the long-standing problems in computing an MLE with incomplete data. We prove that in general O(n) simulation steps are required in computing the MLE with the SA algorithm and O(n log n) simulation steps are required in computing the MLE using the MCEM and/or the MCNR algorithm, where n is the sample size of the observations. Examples include computing the MLE in the nonlinear error-in-variable model and nonlinear regression model with random effects.     

Parameter Estimation for Bivariate Shock Models with Singular Distribution for Censored Data with Concomitant Order Statistics

When two‐component parallel systems are tested, the data consist of Type‐II censored data X_(i), i= 1, n, from one component, and their concomitants Y _[i] randomly censored at X_(r), the stopping time of the experiment. Marshall & Olkin's (1967) bivariate exponential distribution is used to illustrate statistical inference procedures developed for this data type. Although this data type is motivated practically, the likelihood is complicated, and maximum likelihood estimation is difficult, especially in the case where the parameter space is a non‐open set. An iterative algorithm is proposed for finding maximum likelihood estimates. This article derives several properties of the maximum likelihood estimator (MLE) including existence, uniqueness, strong consistency and asymptotic distribution. It also develops an alternative estimation method with closed‐form expressions based on marginal distributions, and derives its asymptotic properties. Compared with variances of the MLEs in the finite and large sample situations, the alternative estimator performs very well, especially when the correlation between X and Y is small.     

Simultaneous robust estimation of location and scale parameters: A minimum-distance approach

Simultaneous robust estimates of location and scale parameters are derived from minimizing a minimum-distance criterion function. The criterion function measures the squared distance between the pth power (p > 0) of the empirical distribution function and the pth power of the imperfectly determined model distribution function over the real line. We show that the estimator is uniquely defined, is asymptotically bivariate normal and for p > 0.3 has positive breakdown. If the scale parameter is known, when p = 0.9 the asymptotic variance (1.0436) of the location estimator for the normal model is smaller than the asymptotic variance of the Hodges-Lehmann (HL)estimator (1.0472). Efficiencies with respect to HL and maximum-likelihood estimators (MLE) are 1.0034 and 0.9582, respectively. Similarly, if the location parameter is known, when p = 0.97 the asymptotic variance (0.6158) of the scale estimator is minimum. The efficiency with respect to the MLE is 0.8119. We show that the estimator can tolerate more corrupted observations at oo than at – for p < 1, and vice versa for p > 1.     

Test for the mean matrix in a Growth Curve model for high dimensions

We consider the problem of estimating and testing a general linear hypothesis in a general multivariate linear model, the so-called Growth Curve model, when the p × N observation matrix is normally distributed.

The maximum likelihood estimator (MLE) for the mean is a weighted estimator with the inverse of the sample covariance matrix which is unstable for large p close to N and singular for p larger than N. We modify the MLE to an unweighted estimator and propose new tests which we compare with the previous likelihood ratio test (LRT) based on the weighted estimator, i.e., the MLE. We show that the performance of these new tests based on the unweighted estimator is better than the LRT based on the MLE.     

Empirical likelihood confidence regions in the single-index model with growing dimensions

This paper investigates statistical inference for the single-index model when the number of predictors grows with sample size. Empirical likelihood method for constructing confidence region for the index vector, which does not require a multivariate non parametric smoothing, is employed. However, the classical empirical likelihood ratio for this model does not remain valid because plug-in estimation of an infinite-dimensional nuisance parameter causes a non negligible bias and the diverging number of parameters/predictors makes the limit not chi-squared any more. To solve these problems, we define an empirical likelihood ratio based on newly proposed weighted estimating equations and show that it is asymptotically normal. Also we find that different weights used in the weighted residuals require, for asymptotic normality, different diverging rate of the number of predictors. However, the rate n^1/3, which is a possible fastest rate when there are no any other conditions assumed in the setting under study, is still attainable. A simulation study is carried out to assess the performance of our method.     

On maximum likelihood estimation of the long-memory parameter in fractional Gaussian noise

Approximate normality and unbiasedness of the maximum likelihood estimate (MLE) of the long-memory parameter H of a fractional Brownian motion hold reasonably well for sample sizes as small as 20 if the mean and scale parameter are known. We show in a Monte Carlo study that if the latter two parameters are unknown the bias and variance of the MLE of H both increase substantially. We also show that the bias can be reduced by using a parametric bootstrap procedure. In very large samples, maximum likelihood estimation becomes problematic because of the large dimension of the covariance matrix that must be inverted. To overcome this difficulty, we propose a maximum likelihood method based upon first differences of the data. These first differences form a short-memory process. We split the data into a number of contiguous blocks consisting of a relatively small number of observations. Computation of the likelihood function in a block then presents no computational problem. We form a pseudo-likelihood function consisting of the product of the likelihood functions in each of the blocks and provide a formula for the standard error of the resulting estimator of H. This formula is shown in a Monte Carlo study to provide a good approximation to the true standard error. The computation time required to obtain the estimate and its standard error from large data sets is an order of magnitude less than that required to obtain the widely used Whittle estimator. Application of the methodology is illustrated on two data sets.     

ARFIMA processes and outliers: a weighted likelihood approach

In this paper, we consider the problem of robust estimation of the fractional parameter, d, in long memory autoregressive fractionally integrated moving average processes, when two types of outliers, i.e. additive and innovation, are taken into account without knowing their number, position or intensity. The proposed method is a weighted likelihood estimation (WLE) approach for which needed definitions and algorithm are given. By an extensive Monte Carlo simulation study, we compare the performance of the WLE method with the performance of both the approximated maximum likelihood estimation (MLE) and the robust M-estimator proposed by Beran (Statistics for Long-Memory Processes, Chapman & Hall, London, 1994). We find that robustness against the two types of considered outliers can be achieved without loss of efficiency. Moreover, as a byproduct of the procedure, we can classify the suspicious observations in different kinds of outliers. Finally, we apply the proposed methodology to the Nile River annual minima time series.     

Ordinal ridge regression with categorical predictors

In multi-category response models, categories are often ordered. In the case of ordinal response models, the usual likelihood approach becomes unstable with ill-conditioned predictor space or when the number of parameters to be estimated is large relative to the sample size. The likelihood estimates do not exist when the number of observations is less than the number of parameters. The same problem arises if constraint on the order of intercept values is not met during the iterative procedure. Proportional odds models (POMs) are most commonly used for ordinal responses. In this paper, penalized likelihood with quadratic penalty is used to address these issues with a special focus on POMs. To avoid large differences between two parameter values corresponding to the consecutive categories of an ordinal predictor, the differences between the parameters of two adjacent categories should be penalized. The considered penalized-likelihood function penalizes the parameter estimates or differences between the parameter estimates according to the type of predictors. Mean-squared error for parameter estimates, deviance of fitted probabilities and prediction error for ridge regression are compared with usual likelihood estimates in a simulation study and an application.     

Comparison of estimation methods for the Weibull distribution

Weibull distributions have received wide ranging applications in many areas including reliability, hydrology and communication systems. Many estimation methods have been proposed for Weibull distributions. But there has not been a comprehensive comparison of these estimation methods. Most studies have focused on comparing the maximum likelihood estimation (MLE) with one of the other approaches. In this paper, we first propose an L-moment estimator for the Weibull distribution. Then, a comprehensive comparison is made of the following methods: the method of maximum likelihood estimation (MLE), the method of logarithmic moments, the percentile method, the method of moments and the method of L-moments.     

An algorithm for maximum likelihood estimation in incomplete block variance component models

In the recovery of interblock information to improve the treatment differences estimates in incomplete block designs, the parameter p is usually unknown. Many authors have worked on the problem of estimating it and of studying its properties together with the properties of the treatment differences estimates. In this paper a numerically efficient algorithm is developed which yields the maximum likelihood estimates (MLE) of all the parameters in the mixed incomplete block design model (treatment effects, ρ and variance)     

Modified p-Value of Two-Sided Test for Normal Distribution with Restricted Parameter Space

This article proposes a modified p-value for the two-sided test of the location of the normal distribution when the parameter space is restricted. A commonly used test for the two-sided test of the normal distribution is the uniformly most powerful unbiased (UMPU) test, which is also the likelihood ratio test. The p-value of the test is used as evidence against the null hypothesis. Note that the usual p-value does not depend on the parameter space but only on the observation and the assumption of the null hypothesis. When the parameter space is known to be restricted, the usual p-value cannot sufficiently utilize this information to make a more accurate decision. In this paper, a modified p-value (also called the rp-value) dependent on the parameter space is proposed, and the test derived from the modified p-value is also shown to be the UMPU test.     

A Novel Bayesian Parameter Mapping Method for Estimating the Parameters of an Underlying Scientific Model

Population-parameter mapping (PPM) is a method for estimating the parameters of latent scientific models that describe the statistical likelihood function. The PPM method involves a Bayesian inference in terms of the statistical parameters and the mapping from the statistical parameter space to the parameter space of the latent scientific parameters, and obtains a model coherence estimate, P(coh). The P(coh) statistic can be valuable for designing experiments, comparing competing models, and can be helpful in redesigning flawed models. Examples are provided where greater estimation precision was found for small sample sizes for the PPM point estimates relative to the maximum likelihood estimator (MLE).     

Influence diagnostics for censored regression models with autoregressive errors

Observations collected over time are often autocorrelated rather than independent, and sometimes include observations below or above detection limits (i.e. censored values reported as less or more than a level of detection) and/or missing data. Practitioners commonly disregard censored data cases or replace these observations with some function of the limit of detection, which often results in biased estimates. Moreover, parameter estimation can be greatly affected by the presence of influential observations in the data. In this paper we derive local influence diagnostic measures for censored regression models with autoregressive errors of order p (hereafter, AR(p)‐CR models) on the basis of the Q‐function under three useful perturbation schemes. In order to account for censoring in a likelihood‐based estimation procedure for AR(p)‐CR models, we used a stochastic approximation version of the expectation‐maximisation algorithm. The accuracy of the local influence diagnostic measure in detecting influential observations is explored through the analysis of empirical studies. The proposed methods are illustrated using data, from a study of total phosphorus concentration, that contain left‐censored observations. These methods are implemented in the R package ARCensReg.     

A comparison of estimators for the proportion in the tail of a normal distribution

Three estimators of the proportion in a tail of the normal distribution are compared using the criteria of mean squared error and mean absolute error. The estimators that we compare are the maximum likelihood estimator, the minimum variance unbiased estimator, and an intuitive estimator that is frequently used in practice. The intuitive estimator is similar to the MLE but uses the usual unbiased estimator of σ² rather than the MLE of σ². We show that the intuitive estimator has low efficiency, and for this reason it is not recommended. For very smallp and for largep the MVUE has the highest efficiency. The MLE is best for moderate values ofp.