Ordinal ridge regression with categorical predictors

In multi-category response models, categories are often ordered. In the case of ordinal response models, the usual likelihood approach becomes unstable with ill-conditioned predictor space or when the number of parameters to be estimated is large relative to the sample size. The likelihood estimates do not exist when the number of observations is less than the number of parameters. The same problem arises if constraint on the order of intercept values is not met during the iterative procedure. Proportional odds models (POMs) are most commonly used for ordinal responses. In this paper, penalized likelihood with quadratic penalty is used to address these issues with a special focus on POMs. To avoid large differences between two parameter values corresponding to the consecutive categories of an ordinal predictor, the differences between the parameters of two adjacent categories should be penalized. The considered penalized-likelihood function penalizes the parameter estimates or differences between the parameter estimates according to the type of predictors. Mean-squared error for parameter estimates, deviance of fitted probabilities and prediction error for ridge regression are compared with usual likelihood estimates in a simulation study and an application.     

Parameter estimation in multivariate logit models with many binary choices

Multivariate Logit models are convenient to describe multivariate correlated binary choices as they provide closed-form likelihood functions. However, the computation time required for calculating choice probabilities increases exponentially with the number of choices, which makes maximum likelihood-based estimation infeasible when many choices are considered. To solve this, we propose three novel estimation methods: (i) stratified importance sampling, (ii) composite conditional likelihood (CCL), and (iii) generalized method of moments, which yield consistent estimates and still have similar small-sample bias to maximum likelihood. Our simulation study shows that computation times for CCL are much smaller and that its efficiency loss is small.     

Inferences in dynamic logit models in semi-parametric setup for repeated binary data

Binary dynamic fixed and mixed logit models are extensively studied in the literature. These models are developed to examine the effects of certain fixed covariates through a parametric regression function as a part of the models. However, there are situations where one may like to consider more covariates in the model but their direct effect is not of interest. In this paper we propose a generalization of the existing binary dynamic logit (BDL) models to the semi-parametric longitudinal setup to address this issue of additional covariates. The regression function involved in such a semi-parametric BDL model contains (i) a parametric linear regression function in some primary covariates, and (ii) a non-parametric function in certain secondary covariates. We use a simple semi-parametric conditional quasi-likelihood approach for consistent estimation of the non-parametric function, and a semi-parametric likelihood approach for the joint estimation of the main regression and dynamic dependence parameters of the model. The finite sample performance of the estimation approaches is examined through a simulation study. The asymptotic properties of the estimators are also discussed. The proposed model and the estimation approaches are illustrated by reanalysing a longitudinal infectious disease data.     

Outlier detection in contingency tables using decomposable graphical models

For high-dimensional data, it is a tedious task to determine anomalies such as outliers. We present a novel outlier detection method for high-dimensional contingency tables. We use the class of decomposable graphical models to model the relationship among the variables of interest, which can be depicted by an undirected graph called the interaction graph. Given an interaction graph, we derive a closed-form expression of the likelihood ratio test (LRT) statistic and an exact distribution for efficient simulation of the test statistic. An observation is declared an outlier if it deviates significantly from the approximated distribution of the test statistic under the null hypothesis. We demonstrate the use of the LRT outlier detection framework on genetic data modeled by Chow–Liu trees.     

Two-step jackknife bias reduction for logistic regression mles

Maximum likelihood estimates (MLEs) for logistic regression coefficients are known to be biased in finite samples and consequently may produce misleading inferences. Bias adjusted estimates can be calculated using the first-order asymptotic bias derived from a Taylor series expansion of the log likelihood. Jackknifing can also be used to obtain bias corrected estimates, but the approach is computationally intensive, requiring an additional series of iterations (steps) for each observation in the dataset.Although the one-step jackknife has been shown to be useful in logistic regression diagnostics and i the estimation of classification error rates, it does not effectively reduce bias. The two-step jackknife, however, can reduce computation in moderate-sized samples, provide estimates of dispersion and classification error, and appears to be effective in bias reduction. Another alternative, a two-step closed-form approximation, is found to be similar to the Taylo series method in certain circumstances. Monte Carlo simulations indicate that all the procedures, but particularly the multi-step jackknife, may tend to over-correct in very small samples. Comparison of the various bias correction proceduresin an example from the medical literature illustrates that bias correction can have a considerable impact on inference     

Estimation for binary models generated by Gaussian autoregressive processes

Regression-type and partial likelihood models are presented for binary data obtained by clipping a Gaussian autoregressive process. Five methods for estimating parameters of the model are proposed and compared via a simulation study. A real data analysis is also presented.     

The analytic construction of D-optimal designs for the two-variable binary logistic regression model without interaction

Candidate locally D-optimal designs for the binary two-variable logistic model with no interaction, which comprise 3 and 4 support points lying in the first quadrant of the two-dimensional Euclidean space, were introduced by Haines et al. (D-optimal designs for logistic regression in two variables. In: Lopez-Fidalgo J, Rodrigez-Diaz JM, Torsney B, editors. MODA8 – advances in model-oriented designs and analysis. Heidelberg: Physica-Verlag; 2007. p. 91–98). The authors proved algebraically the global D-optimality of the 3-point design for the special case in which the intercept parameter is equal to?1.5434. However for other selected values of the intercept parameter, the global D-optimality of the proposed 3- and 4-point designs was only demonstrated numerically. In this paper, we provide analytical proofs of the D-optimality of these 3- and 4-point designs for all negative and zero intercept parameters of the binary two-variable logistic model with no interaction. The results are extended to the construction of D-optimal designs on a rectangular design space and illustrated by means of two examples of which one is a real example taken from the literature.     

Crossover design in clinical trials for binary response

In this paper, we consider a binary response model for the analysis of the two-treatment, two-period and four-sequence crossover design. We have introduced intra-patient drug dependency parameter in the model and provide two tests for the hypothesis of equality of treatment effects. We employ Monte Carlo simulation to compare our tests and a test that works under parallel design on the basis of type I error rate and power. We find that our procedures are dominant over the competitor with respect to power. Finally, we use a data set to illustrate the applicability of our procedure.     

Marginal and association regression models for longitudinal binary data with drop‐outs: A likelihood‐based approach

Longitudinal data often contain missing observations, and it is in general difficult to justify particular missing data mechanisms, whether random or not, that may be hard to distinguish. The authors describe a likelihood‐based approach to estimating both the mean response and association parameters for longitudinal binary data with drop‐outs. They specify marginal and dependence structures as regression models which link the responses to the covariates. They illustrate their approach using a data set from the Waterloo Smoking Prevention Project They also report the results of simulation studies carried out to assess the performance of their technique under various circumstances.     

Optimal designs for estimating a choice hierarchy by a general nested multinomial logit model

Abstract

In choice experiments the process of decision-making can be more complex than the proposed by the Multinomial Logit Model (MNL). In these scenarios, models such as the Nested Multinomial Logit Model (NMNL) are often employed to model a more complex decision-making. Understanding the decision-making process is important in some fields such as marketing. Achieving a precise estimation of the models is crucial to the understanding of this process. To do this, optimal experimental designs are required. To construct an optimal design, information matrix is key. A previous research by others has developed the expression for the information matrix of the two-level NMNL model with two nests: Alternatives nest (J alternatives) and No-Choice nest (1 alternative). In this paper, we developed the likelihood function for a two-stage NMNL model for M nests and we present the expression for the information matrix for 2 nests with any amount of alternatives in them. We also show alternative D-optimal designs for No-Choice scenarios with similar relative efficiency but with less complex alternatives which can help to obtain more reliable answers and one application of these designs.     

A restricted Liu estimator for binary regression models and its application to an applied demand system

In this article, we propose a restricted Liu regression estimator (RLRE) for estimating the parameter vector, β, in the presence of multicollinearity, when the dependent variable is binary and it is suspected that β may belong to a linear subspace defined by Rβ?=?r. First, we investigate the mean squared error (MSE) properties of the new estimator and compare them with those of the restricted maximum likelihood estimator (RMLE). Then we suggest some estimators of the shrinkage parameter, and a simulation study is conducted to compare the performance of the different estimators. Finally, we show the benefit of using RLRE instead of RMLE when estimating how changes in price affect consumer demand for a specific product.     

Regression models for binary longitudinal responses

Some conditional models to deal with binary longitudinal responses are proposed, extending random effects models to include serial dependence of Markovian form, and hence allowing for quite general association structures between repeated observations recorded on the same individual. The presence of both these components implies a form of dependence between them, and so a complicated expression for the resulting likelihood. To handle this problem, we introduce, as a first instance, what Follmann and Wu (1995) called, in a different setting, an approximate conditional model, which represents an optimal choice for the general framework of categorical longitudinal responses. Then we define two more formally correct models for the binary case, with no assumption about the distribution of the random effect. All of the discussed models are estimated by means of an EM algorithm for nonparametric maximum likelihood. The algorithm, an adaptation of that used by Aitkin (1996) for the analysis of overdispersed generalized linear models, is initially derived as a form of Gaussian quadrature, and then extended to a completely unknown mixing distribution. A large scale simulation work is described to explore the behaviour of the proposed approaches in a number of different situations.     

A Measure of partial association between disease and genotype for the two-parent problem

The two-parent disease-genotype association problem is studied from the point of view of a coefficient of association between the disease phenotype of the child and the disease phenotypes of the parents, in the presence of some genotypic information about the parents. This coefficient of partial association is derived, and certain tests of hypotheses are constructed. The results are shown to be useful in estimation of recurrence risks, and in understanding the nature of the association between child and parental disease phenotypes.     

Improving logistic regression on the imbalanced data by a novel penalized log-likelihood function

Logistic regression is estimated by maximizing the log-likelihood objective function formulated under the assumption of maximizing the overall accuracy. That does not apply to the imbalanced data. The resulting models tend to be biased towards the majority class (i.e. non-event), which can bring great loss in practice. One strategy for mitigating such bias is to penalize the misclassification costs of observations differently in the log-likelihood function. Existing solutions require either hard hyperparameter estimating or high computational complexity. We propose a novel penalized log-likelihood function by including penalty weights as decision variables for observations in the minority class (i.e. event) and learning them from data along with model coefficients. In the experiments, the proposed logistic regression model is compared with the existing ones on the statistics of area under receiver operating characteristics (ROC) curve from 10 public datasets and 16 simulated datasets, as well as the training time. A detailed analysis is conducted on an imbalanced credit dataset to examine the estimated probability distributions, additional performance measurements (i.e. type I error and type II error) and model coefficients. The results demonstrate that both the discrimination ability and computation efficiency of logistic regression models are improved using the proposed log-likelihood function as the learning objective.     

Pairwise- and marginal-likelihood estimation for the mixed Rasch model with binary data

A marginal–pairwise-likelihood estimation approach is examined in the mixed Rasch model with the binary response and logit link. This method belonging to the broad class of composite likelihood provides estimators with desirable asymptotic properties such as consistency and asymptotic normality. We study the performance of the proposed methodology when the random effect distribution is misspecified. A simulation study was conducted to compare this approach with the maximum marginal likelihood. The different results are also illustrated with an analysis of the real data set from a quality-of-life study.     

Likelihood inference for correlated binary data without any information about the joint distributions

We propose a universal robust likelihood that is able to accommodate correlated binary data without any information about the underlying joint distributions. This likelihood function is asymptotically valid for the regression parameter for any underlying correlation configurations, including varying under- or over-dispersion situations, which undermines one of the regularity conditions ensuring the validity of crucial large sample theories. This robust likelihood procedure can be easily implemented by using any statistical software that provides naïve and sandwich covariance matrices for regression parameter estimates. Simulations and real data analyses are used to demonstrate the efficacy of this parametric robust method.     

Using a correlated probit model approximation to estimate the variance for binary matched pairs

A correlated probit model approximation for conditional probabilities (Mendell and Elston 1974) is used to estimate the variance for binary matched pairs data by maximum likelihood. Using asymptotic data, the bias of the estimates is shown to be small for a wide range of intra-class correlations and incidences. This approximation is also compared with other recently published, or implemented, improved approximations. For the small sample examples presented, it shows a substantial advantage over other approximations. The method is extended to allow covariates for each observation, and fitting by iteratively reweighted least squares.     

Testing equality of correlation coefficients for paired binary data from multiple groups

Paired binary data arise naturally when paired body parts are investigated in clinical trials. One of the widely used models for dealing with this kind of data is the equal correlation coefficients model. Before using this model, it is necessary to test whether the correlation coefficients in each group are actually equal. In this paper, three test statistics (likelihood ratio test, Wald-type test, and Score test) are derived for this purpose. The simulation results show that the Score test statistic maintains type I error rate and has satisfactory power, and therefore is recommended among the three methods. The likelihood ratio test is over conservative in most cases, and the Wald-type statistic is not robust with respect to empirical type I error. Three real examples, including a multi-centre Phase II double-blind placebo randomized controlled trial, are given to illustrate the three proposed test statistics.