Estimation of weighted multinomial probabilities under log-convex constraints

Many problems in Statistics involve maximizing a multinomial likelihood over a restricted region. In this paper, we consider instead maximizing a weighted multinomial likelihood. We show that a dual problem always exits which is frequently more tractable and that a solution to the dual problem leads directly to a solution of the primal problem. Moreover, the form of the dual problem suggests an iterative algorithm for solving the MLE problem when the constraint region can be written as a finite intersection of cones. We show that this iterative algorithm is guaranteed to converge to the true solution and show that when the cones are isotonic, this algorithm is a version of Dykstra's algorithm (Dykstra, J. Amer. Statist. Assoc. 78 (1983) 837–842) for the special case of least squares projection onto the intersection of isotonic cones. We give several meaningful examples to illustrate our results. In particular, we obtain the nonparametric maximum likelihood estimator of a monotone density function in the presence of selection bias.     

A convergent algorithm for a generalized multivariate isotonic regression problem

Sasabuchi et al. (Biometrika 70(2):465–472, 1983) introduces a multivariate version of the well-known univariate isotonic regression which plays a key role in the field of statistical inference under order restrictions. His proposed algorithm for computing the multivariate isotonic regression, however, is guaranteed to converge only under special conditions (Sasabuchi et al., J Stat Comput Simul 73(9):619–641, 2003). In this paper, a more general framework for multivariate isotonic regression is given and an algorithm based on Dykstra’s method is used to compute the multivariate isotonic regression. Two numerical examples are given to illustrate the algorithm and to compare the result with the one published by Fernando and Kulatunga (Comput Stat Data Anal 52:702–712, 2007).     

Semiparametric additive isotonic regression

We consider the efficient estimation in the semiparametric additive isotonic regression model where each additive nonparametric component is assumed to be a monotone function. We show that the least-square estimator of the finite-dimensional regression coefficient is root-n$n$ consistent and asymptotically normal. Moreover, the isotonic estimator of each additive functional component is proved to have the oracle property, which means the additive component can be estimated with the highest asymptotic accuracy as if the other components were known. A fast algorithm is developed by iterating between a cyclic pool adjacent violators procedure and solving a standard ordinary least squares problem. Simulations are used to illustrate the performance of the proposed procedure and verify the oracle property.     

Changepoint Analysis as a Method for Isotonic Inference

Concavity and sigmoidicity hypotheses are developed as a natural extension of the simple ordered hypothesis in normal means. Those hypotheses give reasonable shape constraints for obtaining a smooth response curve in the non-parametric inputoutput analysis. The slope change and inflection point models are introduced correspondingly as the corners of the polyhedral cones defined by those isotonic hypotheses. Then a maximal contrast type test is derived systematically as the likelihood ratio test for each of those changepoint hypotheses. The test is also justified for the original isotonic hypothesis by a complete class lemma. The component variables of the resulting test statistic have second or third order Markov property which, together with an appropriate non-linear transformation, leads to an exact and very efficient algorithm for the probability calculation. Some considerations on the power of the test are given showing this to be a very promising way of approaching to the isotonic inference.     

M-estimators for isotonic regression

In this paper we propose a family of robust estimates for isotonic regression: isotonic M-estimators. We show that their asymptotic distribution is, up to an scalar factor, the same as that of Brunk's classical isotonic estimator. We also derive the influence function and the breakdown point of these estimates. Finally we perform a Monte Carlo study that shows that the proposed family includes estimators that are simultaneously highly efficient under Gaussian errors and highly robust when the error distribution has heavy tails.     

Extension of the pool-adjacent-violators algorithm

The pool-adjacent-violators algorithm (PAVA) is an efficient algorithm which converges in a finite number of steps. However, it has been applicable so far only in isotonic regression with the simple order. This report extends its applicability to other quadratic programming problems, including certain one-sided multivariate testing problems and concave regression problems.     

Isotonic estimation of survival under a misattribution of cause of death

Several authors have indicated that incorrectly classified cause of death for prostate cancer survivors may have played a role in the observed recent peak and decline of prostate cancer mortality. Motivated by the suggestion we studied a competing risks model where other cause of death may be misattributed as a death of interest. We first consider a na?ve approach using unconstrained nonparametric maximum likelihood estimation (NPMLE), and then present the constrained NPMLE where the survival function is forced to be monotonic. Surprising observations were made as we studied their small-sample and asymptotic properties in continuous and discrete situations. Contrary to the common belief that the non-monotonicity of a survival function NPMLE is a small-sample problem, the constrained NPMLE is asymptotically biased in the continuous setting. Other isotonic approaches, the supremum (SUP) method and the Pooled-Adjacent-Violators (PAV) algorithm, and the EM algorithm are also considered. We found that the EM algorithm is equivalent to the constrained NPMLE. Both SUP method and PAV algorithm deliver consistent and asymptotically unbiased estimator. All methods behave well asymptotically in the discrete time setting. Data from the Surveillance, Epidemiology and End Results (SEER) database are used to illustrate the proposed estimators.     

On the asymptotic distribution of an isotonic window estimator for the generalized failure rate function

Barlow and van Zwet (1969, 1970, 1971) have proposed the isotonic window estimators for the generalized failure rate function and established some asymptotic properties. In this paper, we provide a proof, together with a set of sufficient conditions, of the asymptotic normality of an isotonic window estimator.     

A study on the least squares estimator of multivariate isotonic regression function

Consider the problem of pointwise estimation of f in a multivariate isotonic regression model Z=f(X₁,…,X_d)+ϵ, where Z is the response variable, f is an unknown nonparametric regression function, which is isotonic with respect to each component, and ϵ is the error term. In this article, we investigate the behavior of the least squares estimator of f. We generalize the greatest convex minorant characterization of isotonic regression estimator for the multivariate case and use it to establish the asymptotic distribution of properly normalized version of the estimator. Moreover, we test whether the multivariate isotonic regression function at a fixed point is larger (or smaller) than a specified value or not based on this estimator, and the consistency of the test is established. The practicability of the estimator and the test are shown on simulated and real data as well.     

Maximum likelihood estimation of the bimodal failure rate for censored and tied observations

DIMITROV, RACHEV and YAKOVLEV ( 1985 ) have obtained the isotonic maximum likelihood estimator for the bimodal failure rate function. The authors considered only the complete failure time data. The generalization of this estimator for the case of censored and tied observations is now proposed.     

An Exact Algorithm for Projection onto a Polyhedral Cone

Projection is an operation widely used in restricted statistical inferences. A polyhedral cone restriction includes many interesting problems in linear regression, order restricted inferences etc. This paper proposes an exact algorithm for the projection of a vector onto a polyhedral cone, and presents an application to second order polynomial regression subject to a non-negative, isotonic restriction.     

Semi-non parametric smooth isotonic regression

We propose a new method for smooth isotonic regression analysis. Unlike most existing methods for isotonic regression, the proposed method is akin to parametric regression without order restriction. To account for smoothness and isotonicity simultaneously, we exploit the flexible class of semi-non parametric densities to model isotonic regression functions. Under this framework, the full range of inference techniques for parametric regression models become applicable for model estimation and model validation in isotonic regression.     

三个多元正态总体在简单半序约束下均值估计

 多元保序回归理论对统计学中研究多维参数在序约束下的估计理论起着关键性作用。本文讨论了当协方差矩阵已知,在简单半序约束下,对三个多元正态总体均值的估计问题,给出了估计的算法。并证明了在多元均方损失条件下,给出的均值估计优于无序约束的均值估计。     

Evaluation of the bias of the isotonic regression

The systematic error (bias) of the isotonic regression analysis of temporal spacings between failure events is investigated by means of numerical simulation. Spacings that are sampled from an exponential distribution with a constant failure rate (CFR) arc subjected to an isotonic regression search for a declining failure rate (DFR). The results indicate a considerable declining trend (bias) that is imposed upon these CFR-data by isotonic regression analysis. The corresponding results for an increasing trend can be readily obtained through transformation. For practical applications, the results of 100,000 simulations have been approximated by simple analytical expressions. For the evaluation of a trend in a specific set of isotonized spacings (or rates) the results of the latter analysis can be compared with the isotonic bias of a set of CFR data for the same number of events. Alternatively, the specific set of isotonized spacings can be suitably related to the corresponding isotonized CFR data to reduce the bias by largely eliminating the CFR contribution.     

Isotonic Window Estimators of the Baseline Hazard Function in Cox's Regression Model Under Order Restriction

The regression model suggested by Cox (1972) has been widely used in survival analysis with censored observations. We propose isotonic window estimators for a monotone baseline hazard function in the Cox regression model. We prove that these estimators are asymptotically normal. The simulati on results presented in the article suggest that the proposed estimator performs better than several existing estimators in the literature     

Pre-test estimation and design in the linear model

As a robust method against model deviation we consider a pre-test estimation function. To optimize a continuous design for this problem we give an asymptotic risk matrix for the quadratic loss. The risk will then be given by an isotonic criterion function of the asymptotic risk matrix. As an optimization criterion we look for a design that minimizes the maximal risk in the deviation model under the restriction that the risk in the original model does not exceed a given bound. This optimization problem will be solved for the polynomial regression, the deviation consisting in one additional regression function and the criterion function being the determinant.     

Testing the monotonicity or convexity of a function using regression splines

Methods for smoothed isotonic or convex regression are useful in many applications. Sometimes the shape assumptions constitute a priori knowledge about the regression function, but often the shape is part of the research question. The authors propose tests for monotonicity and convexity using constrained and unconstrained regression splines. The tests have good large‐sample properties and the small‐sample behaviour is illustrated through simulations. Extensions to the partial linear model and the generalized regression model are presented. The Canadian Journal of Statistics 39: 89–107; 2011 © 2011 Statistical Society of Canada     

Trimmed Mean Isotonic Regression

The trimmed mean is well‐known in literature for being more robust and for having better efficiency than the sample mean when data is generated from heavy‐tailed distributions. In this article, the trimmed mean in the isotonic regression setup is proposed, and the asymptotic as well as the robustness properties of the estimator are studied. The usefulness of the proposed estimator is illustrated using different real and simulated data. Further, the performance of the estimator is compared with that of the mean and the median isotonic regression estimators.     

Small sample properties of isotonic estimators — bias and mean squared error of linear combinations under normality

Bias and mean squared error for linear combinations of the isotonic regression estimators are computed. The case of sampling three distinct populations and the case of sampling seven or fewer populations having common mean are studied in detail. Numerical results are given, and comparisons between isotonic and unbiased estimation procedures are made.     

Estimating ordered parameters by linear programming

There are many models that require the estimation of a set of ordered parameters. For example, multivariate analysis of variance often is formulated as testing for the equality of the parameters versus an ordered alternative. This problem, referred to as isotonic inference, constrained inference, or isotonic regression, has led to the development of general solutions, not often easy to apply in special models. In this expository paper, we study the special case of a separable convex quadratic programming problem for which the optimality conditions lead to a readily solved linear complementarity problem in the Lagrange multipliers, and subsequently to an equivalent linear programming problem, whose solution can be used to recover the solution of the original isotonic problem. The method can be applied to estimating ordered correlations, ordered binomial probabilities, ordered Poisson parameters, ordered exponential scale parameters, or ordered risk differences.