Orthogonalizing parametric link transformation families in binary regression analysis

This paper studies the application of the orthogonalization technique of Cox and Reid (1987) to parametric families of link functions used in binary regression analysis. The explicit form of Cox and Reid's condition (4), for orthogonality at a point, is derived for arbitrary link families. This condition is used to determine a transform of a family introduced by Burr (1942) and Prentice (1975, 1976) which is locally orthogonal when the regression parameter is zero. Thus the benefits of having orthogonal parameters are limited to “small” regression effects. The extent to which approximate orthogonality holds for nonzero regression coefficients is investigated for two data sets from the literature. Two specific issues considered are: (1) the ability of orthogonal reparametrization to reduce the variability of the regression parameters caused by estimation of the link parameter and (2) the improved numerical stability (and hence interpretability) of regression estimates corresponding to different link parameters.     

Parametric link modification of both tails in binary regression

Common binary regression models such as logistic or probit regression have been extended to include parametric link transformation families. These binary regression models with parametric link are designed to avoid possible link misspecification and improve fit in some data sets. One and two parameter link families have been proposed in the literature (for a review see Stukel (1988)). However in real data examples published so far only one parameter link families have found to improve the fit significantly. This paper introduces a two parameter link family involving the modification of both tails of the link. An analysis based on computationally tractable Bayesian inference involving Monte Carlo sampling algorithms is presented extending earlier work of Czado (1992, 1993b). Finally, the usefulness of the two tailed link modification will be demonstrated in an example where single tail modification can be significantly improved upon by using a two tailed modification.     

Estimation of exponential regression parameters using binary data

Exponential regression model is important in analyzing data from heterogeneous populations. In this paper we propose a simple method to estimate the regression parameters using binary data. Under certain design distributions, including ellipticaily symmetric distributions, for the explanatory variables, the estimators are shown to be consistent and asymptotically normal when sample size is large. For finite samples, the new estimates were shown to behave reasonably well. They are competitive with the maximum likelihood estimates and more importantly, according to our simulation results, the cost of CPU time for computing new estimates is only 1/7 of that required for computing the usual maximum likelihood estimates. We expect the savings in CPU time would be more dramatic with larger dimension of the regression parameter space.     

Robust nonparametric estimation with missing data

In this paper, under a nonparametric regression model, we introduce two families of robust procedures to estimate the regression function when missing data occur in the response. The first proposal is based on a local M$M$-functional applied to the conditional distribution function estimate adapted to the presence of missing data. The second proposal imputes the missing responses using the local M$M$-smoother based on the observed sample and then estimates the regression function with the completed sample. We show that the robust procedures considered are consistent and asymptotically normally distributed. A robust procedure to select the smoothing parameter is also discussed.     

Parameter Orthogonality in Mixed Regression Models for Survival Data

The implications of parameter orthogonality for the robustness of survival regression models are considered. The question of which of the proportional hazards or the accelerated life families of models would be more appropriate for analysis is usually ignored, and the proportional hazards family is applied, particularly in medicine, for convenience. Accelerated life models have conventionally been used in reliability applications. We propose a one-parameter family mixture survival model which includes both the accelerated life and the proportional hazards models. By orthogonalizing relative to the mixture parameter, we can show that, for small effects of the covariates, the regression parameters under the alternative families agree to within a constant. This recovers a known misspecification result. We use notions of parameter orthogonality to explore robustness to other types of misspecification including misspecified base-line hazards. The results hold in the presence of censoring. We also study the important question of when proportionality matters.     

Single-index composite quantile regression

In this paper, we extend the composite quantile regression (CQR) method to a single-index model. The unknown link function is estimated by local composite quantile regression and the parametric index is estimated through the linear composite quantile. It is shown that the proposed estimators are consistent and asymptotically normal. The simulation studies and real data applications are conducted to illustrate the finite sample performance of the proposed methods.     

A simple least squares method for estimating a change in mean

A simple least squares method for estimating a change in mean of a sequence of independent random variables is studied. The method first tests for a change in mean based on the regression principle of constrained and unconstrained sums of squares. Conditionally on a decision by this test that a change has occurred, least squares estimates are used to estimate the change point, the initial mean level (prior to the change point) and the change itself. The estimates of the initial level and change are functions of the change point estimate. All estimates are shown to be consistent, and those for the initial level and change are shown to be asymptotically jointly normal. The method performs well for moderately large shifts (one standard deviation or more), but the estimates of the initial level and change are biased in a predictable way for small shifts. The large sample theory is helpful in understanding this problem. The asymptotic distribution of the change point estimator is obtained for local shifts in mean, but the case of non-local shifts appears analytically intractable.     

The effects of model mis-specifications in linear measurement error models

In the literature, there are many results on the consequences of mis-specified models for linear models with error in the response only, see, e.g., Seber(1977). There are also discussions of estimation for the model writh errors both in the response and in the predictor variables (called measurement error models; see, e.g., Fuller(1987)). In this paper, we consider the problem of model mis-specification for measurement error models. Only a few special cases have been tackled in the past (Edland, 1996; Carroll and Ruppert, 1996 and Lakshminarayanan Amp; Gunst, 1984); we deal with the situation here in some generality. Results have been obtained as follows: (a) When a model is under-fitted, the estimate of the variance of the measurement error will be asymptotically biased, as will the regression coefficients, and the asymptotic biases in the estimates of the regression coefficients will always exist for under-fitted models. Even orthogonality of the variables in the model will not make the biases vanish. (b)For over-fitting, the estimates of the variances of measurement errors and of the regression coefficients are asymptotically unbiased. However, the variance of the estimated regression coefficients will increase. Over-fitting will cause larger changes in the variances of the estimated parameters in measurement error models than in no measurement error models.     

Estimation of a linear model under microaggregation by individual ranking

Summary Microaggregation by individual ranking is one of themost commonly applied disclosure control techniques for continuous microdata. The paper studies the effect of microaggregation by individual ranking on the least squares estimation of a multiple linear regression model. It is shown that the traditional least squares estimates are asymptotically unbiased. Moreover, the least squares estimates asymptotically have the same variances as the least squares estimates based on the original (non-aggregated) data. Thus, asymptotically, microaggregation by individual ranking does not result in a loss of efficiency in the least squares estimation of a multiple linear regression model. I thank Hans Schneeweiss for very helpful discussions and comments. Financial support from the Deutsche Forschungsgemeinschaft (German Science Foundation) is gratefully acknowledged.     

Norm restricted maximum likelihood estimators for binary regression models with parametric link

Parametric link transformation families have shown to be useful in the analysis of binary regression data since they avoid th? problem of link misspecifaction. Inference for these models are commonly based on likelihood methods. Duffy and Santner (1988, 1989) however showed that ordinary logistic maximum likelihood estimators (MLE) have poor mean square error (MSE) behavior in small samples compared to alternative norm restricted estimators. This paper extends these alternative norm restricted estimators to binary regression models with any specified parametric link family. These extended norm restricted MLE's are strongly consistent and efficient under regularity conditions. Finally a simulation study shows that an empiric version of norm restricted MLE's exhibit superior MSE behavior in small samples compared to MLE's with fixed known link.     

ESTIMATION FOR THE GENERAL SAMPLE SELECTION MODELS

Consider a general regression model with an arbitrary and unknown link function and a stochastic selection variable that determines whether the outcome variable is observable or missing. The paper proposes U-statistics that are based on kernel functions as estimators for the directions of the parameter vectors in the link function and the selection equation, and shows that these estimators are consistent and asymptotically normal.     

On the calculations of the maximum likelihood estimates for the polynomial spline regression model with unknown knots and AR(1) errors

Asymptotic distributions of the maximum likelihood estimators of the regression coefficients and knot points for the polynomial spline regression models with unknown knots and AR(1) errors have been derived by Chan (1989). Chan showed that under some mild conditions the maximum likelihood estimators, after suitable standardization, asymptotically follow normal distributions as n diverges to infinity. For the calculations of the maximum likelihood estimators, iterative methods must be applied. But this is not easy to implement for the model considered. In this paper, we suggested an alternative method to compute the estimates of the regression parameters and knots. It is shown that the estimates obtained by this method are asymptotically equivalent to the maximum likelihood estimates considered by Chan.     

A Semiparametric Binary Regression Model Involving Monotonicity Constraints

Abstract.  We study a binary regression model using the complementary log–log link, where the response variable Δ is the indicator of an event of interest (for example, the incidence of cancer, or the detection of a tumour) and the set of covariates can be partitioned as ( X ,  Z ) where Z (real valued) is the primary covariate and X (vector valued) denotes a set of control variables. The conditional probability of the event of interest is assumed to be monotonic in Z , for every fixed X . A finite-dimensional (regression) parameter β describes the effect of X . We show that the baseline conditional probability function (corresponding to X  =  0 ) can be estimated by isotonic regression procedures and develop an asymptotically pivotal likelihood-ratio-based method for constructing (asymptotic) confidence sets for the regression function. We also show how likelihood-ratio-based confidence intervals for the regression parameter can be constructed using the chi-square distribution. An interesting connection to the Cox proportional hazards model under current status censoring emerges. We present simulation results to illustrate the theory and apply our results to a data set involving lung tumour incidence in mice.     

Nonadditivity in loglinear models using φ-divergences and MLEs

In this paper three families of test statistics for testing nonadditivity in loglinear models are presented under the assumption of either Poisson, multinomial, or product-multinomial sampling. These new families are based on the φ-divergence measures. The standard method for testing nonadditivity is used, i.e., the two-stage tests procedure. In this procedure the parameters are first estimated using an additive model and then the estimates are treated as known constants for the second stage of the procedure. These test statistics, which are asymptotically chi-squared, generalize the likelihood ratio test for this problem given by Christensen and Utts (J. Statist. Plann. Inference 33 (1992) 333). An example and a simulation study are included.     

On parameter orthogonality in symmetric and skew models

Orthogonal and partly orthogonal reparametrisations are provided for certain wide and important families of univariate continuous distributions. First, the orthogonality of parameters in location-scale symmetric families is extended to symmetric distributions involving a third parameter. This sets the scene for consideration of the four-parameter situation in which skewness is also allowed. It turns out that one specific approach to generating such four-parameter families, that of two-piece distributions with a certain parametrisation restriction, has some attractive features with regard to parameter orthogonality which, to the best of our knowledge, are not shared with other four-parameter distributions. Our work also affords partly orthogonal parametrisations of three-parameter two-piece models.     

On maximum likelihood estimation in parametric regression with missing covariates

We consider parametric regression problems with some covariates missing at random. It is shown that the regression parameter remains identifiable under natural conditions. When the always observed covariates are discrete, we propose a semiparametric maximum likelihood method, which does not require parametric specification of the missing data mechanism or the covariate distribution. The global maximum likelihood estimator (MLE), which maximizes the likelihood over the whole parameter set, is shown to exist under simple conditions. For ease of computation, we also consider a restricted MLE which maximizes the likelihood over covariate distributions supported by the observed values. Under regularity conditions, the two MLEs are asymptotically equivalent and strongly consistent for a class of topologies on the parameter set.     

Iterative weighted least-squares estimates in a heteroscedastic linear regression model

The aim of this study is to improve the efficiency of weighted least-squares estimates for a regression parameter. An iterative procedure, starting with an unbiased estimate other than the unweighted least-squares estimate, yields estimates which are asymptotically more efficient than the feasible generalized least-squares estimate when errors are spherically distributed. The result has an application in the improvement of the Graybill–Deal estimate of the common mean of several normal populations.     

Methods for missing covariates in logistic regression

Various methods have been suggested in the literature to handle a missing covariate in the presence of surrogate covariates. These methods belong to one of two paradigms. In the imputation paradigm, Pepe and Fleming (1991) and Reilly and Pepe (1995) suggested filling in missing covariates using the empirical distribution of the covariate obtained from the observed data. We can proceed one step further by imputing the missing covariate using nonparametric maximum likelihood estimates (NPMLE) of the density of the covariate. Recently Murphy and Van der Vaart (1998a) showed that such an approach yields a consistent, asymptotically normal, and semiparametric efficient estimate for the logistic regression coefficient. In the weighting paradigm, Zhao and Lipsitz (1992) suggested an estimating function using completely observed records after weighting inversely by the probability of observation. An extension of this weighting approach designed to achieve semiparametric efficient bound is considered by Robins, Hsieh and Newey (RHN) (1995). The two ends of each paradigm (NPMLE and RHN) attain the efficiency bound and are asymptotically equivalent. However, both require a substantial amount of computation. A question arises whether and when, in practical situations, this extensive computation is worthwhile. In this paper we investigate the performance of single and multiple imputation estimates, weighting estimates, semiparametric efficient estimates, and two new imputation estimates. Simulation studies suggest that the sample size should be substantially large (e.g. n=2000) for NPMLE and RHN to be more efficient than simpler imputation estimates. When the sample size is moderately large (n≤ 1500), simpler imputation estimates have as small a variance as semiparametric efficient estimates.     

Local influence in multilevel models

The authors study the local influence of observations in multilevel regression models. To this end, they perturb simultaneously the variances, responses and design matrix. To measure the local change caused by these perturbations, they use generalized Cook statistics for the fixed and random parameter estimates. Closed form local influence measures also allow them to assess the joint influence of various observations. They suggest a simple computation method and illustrate their results using two examples.     

The asymptotic distribution of MLE of treatment lag threshold

Semiparametric models with an unknown threshold parameter for hazard rates are formulated for failure time data. Maximum partial likelihood estimates for threshold and regular parameters are shown to be asymptotically consistent. The asymptotic distributions of the estimators are obtained.