Orthogonality of the mean and error distribution in generalized linear models

We show that the mean-model parameter is always orthogonal to the error distribution in generalized linear models. Thus, the maximum likelihood estimator of the mean-model parameter will be asymptotically efficient regardless of whether the error distribution is known completely, known up to a finite vector of parameters, or left completely unspecified, in which case the likelihood is taken to be an appropriate semiparametric likelihood. Moreover, the maximum likelihood estimator of the mean-model parameter will be asymptotically independent of the maximum likelihood estimator of the error distribution. This generalizes some well-known results for the special cases of normal, gamma, and multinomial regression models, and, perhaps more interestingly, suggests that asymptotically efficient estimation and inferences can always be obtained if the error distribution is non parametrically estimated along with the mean. In contrast, estimation and inferences using misspecified error distributions or variance functions are generally not efficient.     

Testing variance components in balanced linear growth curve models

It is well known that the testing of zero variance components is a non-standard problem since the null hypothesis is on the boundary of the parameter space. The usual asymptotic chi-square distribution of the likelihood ratio and score statistics under the null does not necessarily hold because of this null hypothesis. To circumvent this difficulty in balanced linear growth curve models, we introduce an appropriate test statistic and suggest a permutation procedure to approximate its finite-sample distribution. The proposed test alleviates the necessity of any distributional assumptions for the random effects and errors and can easily be applied for testing multiple variance components. Our simulation studies show that the proposed test has Type I error rate close to the nominal level. The power of the proposed test is also compared with the likelihood ratio test in the simulations. An application on data from an orthodontic study is presented and discussed.     

Using entropy distance measures for testing odds ratio parameters under logistic regression models based on case–control data

We propose two retrospective test statistics for testing the vector of odds ratio parameters under the logistic regression model based on case–control data by exploiting the density ratio structure under a two-sample semiparametric model, which is equivalent to the assumed logistic regression model. The proposed test statistics are based on Kullback–Leibler entropy distance and are particularly relevant to the case–control sampling plan. These two test statistics have identical asymptotic chi-squared distributions under the null hypothesis and identical asymptotic noncentral chi-squared distributions under local alternatives to the null hypothesis. Moreover, the proposed test statistics require computation of the maximum semiparametric likelihood estimators of the underlying parameters, but are otherwise easily computed. We present some results on simulation and on the analysis of two real data sets.     

Preliminary phi-divergence test estimators for linear restrictions in a logistic regression model

The problem of estimation of the parameters in a logistic regression model is considered under multicollinearity situation when it is suspected that the parameter of the logistic regression model may be restricted to a subspace. We study the properties of the preliminary test based on the minimum ϕ -divergence estimator as well as in the ϕ -divergence test statistic. The minimum ϕ -divergence estimator is a natural extension of the maximum likelihood estimator and the ϕ -divergence test statistics is a family of the test statistics for testing the hypothesis that the regression coefficients may be restricted to a subspace.     

Iso‐chi‐squared testing of 2 × k: ordered tables

The Pearson chi‐squared statistic for testing the equality of two multinomial populations when the categories are nominal is much less appropriate for ordinal categories. Test statistics typically used in this context are based on scorings of the ordinal levels, but the results of these tests are highly dependent on the choice of scores. The authors propose a test which naturally modifies the Pearson chi‐squared statistic to incorporate the ordinal information. The proposed test statistic does not depend on the scores and under the null hypothesis of equality of populations, it is asymptotically equivalent to the likelihood ratio test against the alternative of two‐sided likelihood ratio ordering.     

Properties of maximum likelihood estimates of the ratio of parameters in ordinal response regression models

The properties of a method of estimating the ratio of parameters for ordered categorical response regression models are discussed. If the link function relating the response variable to the linear combination of covariates is unknown then it is only possible to estimate the ratio of regression parameters. This ratio of parameters has a substitutability or relative importance interpretation.

The maximum likelihood estimate of the ratio of parameters, assuming a logistic function (McCullagh, 1980), is found to have very small bias for a wide variety of true link functions. Further it is shown using Monte Carlo simulations that this maximum likelihood estimate, has good coverage properties, even if the link function is incorrectly specified. It is demonstrated that combining adjacent categories to make the response binary can result in an analysis which is appreciably less efficient. The size of the efficiency loss on, among other factors, the marginal distribution in the ordered categories     

Hypotheses testing for a multidimensional parameter of inhomogeneous Poisson processes

We observe a realization of an inhomogeneous Poisson process whose intensity function depends on an unknown multidimensional parameter. We consider the asymptotic behaviour of the Rao score test for a simple null hypothesis against the multilateral alternative. By using the Edgeworth type expansion (under the null hypothesis) for a vector of stochastic integrals with respect to the Poisson process, we refine the (classic) threshold of the test (obtained by the central limit theorem), which improves the first type probability of error. The expansion allows us to describe the power of the test under the local alternative, i.e. a sequence of alternatives, which converge to the null hypothesis with a certain rate. The rates can be different for components of the parameter.     

Score tests when a nuisance parameter is unidentified under the null hypothesis

Rao's score test normally replaces nuisance parameters by their maximum likelihood estimates under the null hypothesis about the parameter of interest. In some models, however, a nuisance parameter is not identified under the null, so that this approach cannot be followed. This paper suggests replacing the nuisance parameter by its maximum likelihood estimate from the unrestricted model and making the appropriate adjustment to the variance of the estimated score. This leads to a rather natural modification of Rao's test, which is examined in detail for a regression-type model. It is compared with the approach, which has featured most frequently in the literature on this problem, where a test statistic appropriate to a known value of the nuisance parameter is treated as a function of that parameter and maximised over its range. It is argued that the modified score test has considerable advantages, including robustness to a crucial assumption required by the rival approach.     

Converting observed likelihood to levels of significance for transformation models

Inference for a scalar parameter in the pressence of nuisance parameters requires high dimensional integrations of the joint density of the pivotal quantities. Recent development in asymptotic methods provides accurate approximations for significance levels and thus confidence intervals for a scalar component parameter. In this paper, a simple, efficient and accurate numerical procedure is first developed for the location model and is then extended to the location-scale model and the linear regression model. This numerical procedure only requires a fine tabulation of the parameter and the observed log likelihood function, which can be either the full, marginal or conditional observed log likelihood function, as input and output is the corresponding significance function. Numerical results showed that this approximation is not only simple but also very accurate. It outperformed the usual approximations such as the signed likelihood ratio statistic, the maximum likelihood estimate and the score statistic.     

A likelihood ratio test of quasi-independence for sparse two-way contingency tables

We consider a likelihood ratio test of independence for large two-way contingency tables having both structural (non-random) and sampling (random) zeros in many cells. The solution of this problem is not available using standard likelihood ratio tests. One way to bypass this problem is to remove the structural zeroes from the table and implement a test on the remaining cells which incorporate the randomness in the sampling zeros; the resulting test is a test of quasi-independence of the two categorical variables. This test is based only on the positive counts in the contingency table and is valid when there is at least one sampling (random) zero. The proposed (likelihood ratio) test is an alternative to the commonly used ad hoc procedures of converting the zero cells to positive ones by adding a small constant. One practical advantage of our procedure is that there is no need to know if a zero cell is structural zero or a sampling zero. We model the positive counts using a truncated multinomial distribution. In fact, we have two truncated multinomial distributions; one for the null hypothesis of independence and the other for the unrestricted parameter space. We use Monte Carlo methods to obtain the maximum likelihood estimators of the parameters and also the p-value of our proposed test. To obtain the sampling distribution of the likelihood ratio test statistic, we use bootstrap methods. We discuss many examples, and also empirically compare the power function of the likelihood ratio test relative to those of some well-known test statistics.     

Test for Trend With a Multinomial Outcome

ABSTRACT

There is no established procedure for testing for trend with nominal outcomes that would provide both a global hypothesis test and outcome-specific inference. We derive a simple formula for such a test using a weighted sum of Cochran–Armitage test statistics evaluating the trend in each outcome separately. The test is shown to be equivalent to the score test for multinomial logistic regression, however, the new formulation enables the derivation of a sample size formula and multiplicity-adjusted inference for individual outcomes. The proposed methods are implemented in the R package multiCA.     

Simulated classical tests in multinomial probit models

This paper compares the application of different versions of the simulated counterparts of the Wald test, the score test, and the likelihood ratio test in one- and multiperiod multinomial probit models. Monte Carlo experiments show that the use of the simple form of the simulated likelihood ratio test delivers relatively robust results regarding the testing of several multinomial probit model specifications. In contrast, the inclusion of the Hessian matrix of the simulated loglikelihood function into the simulated score test and (in the multiperiod multinomial probit model) particularly the inclusion of the quasi-maximum likelihood theory into the simulated likelihood ratio test leads to substantial computational problems. The combined application of the quasi-maximum likelihood theory with the simulated Wald test or the simulated score test is not systematically superior to the application of the other versions of these two simulated classical tests either. Neither an increase in the number of observations nor in the number of random draws in the incorporated Geweke-Hajivassiliou-Keane simulator systematically lead to more precise conformities between the frequencies of type I errors and the basic significance levels. An increase in the number of observations only decreases the frequencies of type II errors, particularly regarding the simulated classical testing of multiperiod multinomial probit model specifications.     

Maximum likelihood estimation for the negative multinomial log-linear model

A log-linear model is defined for multiway contingency tables with negative multinomial frequency counts. The maximum likelihood estimator of the model parameters and the estimator covariance matrix is given. The likelihood ratio test for the general log-linear hypothesis also is presented.     

Guaranteed Local Maximum Likelihood Detection of a Change Point in Nonparametric Logistic Regression

We consider nonparametric logistic regression and propose a generalized likelihood test for detecting a threshold effect that indicates a relationship between some risk factor and a defined outcome above the threshold but none below it. One important field of application is occupational medicine and in particular, epidemiological studies. In epidemiological studies, segmented fully parametric logistic regression models are often threshold models, where it is assumed that the exposure has no influence on a response up to a possible unknown threshold, and has an effect beyond that threshold. Finding efficient methods for detection and estimation of a threshold is a very important task in these studies. This article proposes such methods in a context of nonparametric logistic regression. We use a local version of unknown likelihood functions and show that under rather common assumptions the asymptotic power of our test is one. We present a guaranteed non asymptotic upper bound for the significance level of the proposed test. If applying the test yields the acceptance of the conclusion that there was a change point (and hence a threshold limit value), we suggest using the local maximum likelihood estimator of the change point and consider the asymptotic properties of this estimator.     

Stein-type estimation in logistic regression models based on minimum ϕ-divergence estimators

In this paper we present a study of Stein-type estimators for the unknown parameters in logistic regression models when it is suspected that the parameters may be restricted to a subspace of the parameter space. The Stein-type estimators studied are based on the minimum phi-divergence estimator instead on the maximum likelihood estimator as well as on phi-divergence test statistics.     

Ordinal ridge regression with categorical predictors

In multi-category response models, categories are often ordered. In the case of ordinal response models, the usual likelihood approach becomes unstable with ill-conditioned predictor space or when the number of parameters to be estimated is large relative to the sample size. The likelihood estimates do not exist when the number of observations is less than the number of parameters. The same problem arises if constraint on the order of intercept values is not met during the iterative procedure. Proportional odds models (POMs) are most commonly used for ordinal responses. In this paper, penalized likelihood with quadratic penalty is used to address these issues with a special focus on POMs. To avoid large differences between two parameter values corresponding to the consecutive categories of an ordinal predictor, the differences between the parameters of two adjacent categories should be penalized. The considered penalized-likelihood function penalizes the parameter estimates or differences between the parameter estimates according to the type of predictors. Mean-squared error for parameter estimates, deviance of fitted probabilities and prediction error for ridge regression are compared with usual likelihood estimates in a simulation study and an application.     

The large sample coverage probability of confidence intervals in general regression models after a preliminary hypothesis test

We derive a computationally convenient formula for the large sample coverage probability of a confidence interval for a scalar parameter of interest following a preliminary hypothesis test that a specified vector parameter takes a given value in a general regression model. Previously, this large sample coverage probability could only be estimated by simulation. Our formula only requires the evaluation, by numerical integration, of either a double or a triple integral, irrespective of the dimension of this specified vector parameter. We illustrate the application of this formula to a confidence interval for the odds ratio of myocardial infarction when the exposure is recent oral contraceptive use, following a preliminary test where two specified interactions in a logistic regression model are zero. For this real‐life data, we compare this large sample coverage probability with the actual coverage probability of this confidence interval, obtained by simulation.     

A score test for overdispersion in Poisson regression based on the generalized Poisson-2 model

Overdispersion is a common phenomenon in Poisson modeling. The generalized Poisson (GP) regression model accommodates both overdispersion and underdispersion in count data modeling, and is an increasingly popular platform for modeling overdispersed count data. The Poisson model is one of the special cases in the collection of models which may be specified by GP regression. Thus, we may derive a test of overdispersion which compares the equi-dispersion Poisson model within the context of the more general GP regression model. The score test has an advantage over the likelihood ratio test (LRT) and over the Wald test in that the score test only requires that the parameter of interest be estimated under the null hypothesis (the Poisson model). Herein, we propose a score test for overdispersion based on the GP model (specifically the GP-2 model) and compare the power of the test with the LRT and Wald tests. A simulation study indicates the proposed score test based on asymptotic standard normal distribution is more appropriate in practical applications.     

A Diagnostic Test for the Multinomial Logit Model

I propose a Lagrange multiplier test for the multinomial logit model against the dogit model (Gaudry and Dagenais 1979) as the alternative hypothesis. In view of the well-known drawback of the restrictive property of independence from irrelevant alternatives implied by the multinomial logit model, a specification test has much to recommend it. Finite sample properties of the test are studied using a Monte Carlo experiment, and the test's power against the nested multinomial logit model and the multinomial probit model is investigated. The test is found to be sensitive to the values of the regression parameters of the linear random utility function.     

Change-point analysis with bathtub shape for the exponential distribution

Likelihood ratio type test statistic and Schwarz information criterion statistics are proposed for detecting possible bathtub-shaped changes in the parameter in a sequence of exponential distributions. The asymptotic distribution of likelihood ratio type statistic under the null hypothesis and the testing procedure based on Schwarz information criterion are derived. Numerical critical values and powers of two methods are tabulated for certain selected values of the parameters. The tests are applied to detect the change points for the predator data and Stanford heart transplant data.