Robust linear discriminant analysis using S‐estimators

The authors consider a robust linear discriminant function based on high breakdown location and covariance matrix estimators. They derive influence functions for the estimators of the parameters of the discriminant function and for the associated classification error. The most B‐robust estimator is determined within the class of multivariate S‐estimators. This estimator, which minimizes the maximal influence that an outlier can have on the classification error, is also the most B‐robust location S‐estimator. A comparison of the most B‐robust estimator with the more familiar biweight S‐estimator is made.     

Regression with outlier shrinkage

We propose a robust regression method called regression with outlier shrinkage (ROS) for the traditional n>p$n>p$ cases. It improves over the other robust regression methods such as least trimmed squares (LTS) in the sense that it can achieve maximum breakdown value and full asymptotic efficiency simultaneously. Moreover, its computational complexity is no more than that of LTS. We also propose a sparse estimator, called sparse regression with outlier shrinkage (SROS), for robust variable selection and estimation. It is proven that SROS can not only give consistent selection but also estimate the nonzero coefficients with full asymptotic efficiency under the normal model. In addition, we introduce a concept of nearly regression equivariant estimator for understanding the breakdown properties of sparse estimators, and prove that SROS achieves the maximum breakdown value of nearly regression equivariant estimators. Numerical examples are presented to illustrate our methods.     

Estimation of treatment effects for heterogeneous matched‐pairs data with probit models

Estimating the effect of medical treatments on subject responses is one of the crucial problems in medical research. Matched‐pairs designs are commonly implemented in the field of medical research to eliminate confounding and improve efficiency. In this article, new estimators of treatment effects for heterogeneous matched‐pairs data are proposed. Asymptotic properties of the proposed estimators are derived. Simulation studies show that the proposed estimators have some advantages over the famous Heckman's estimator, the conditional maximum likelihood estimator, and the inverse probability weighted estimator. We apply the proposed methodology to a data set from a study of low‐birth‐weight infants.     

A class of high-breakdown scale estimators based on subranges

We consider a new class of scale estimators with 50% breakdown point. The estimators are defined as order statistics of certain subranges. They all have a finite-sample breakdown point of [n/2]/n, which is the best possible value. (Here, [...] denotes the integer part.) One estimator in this class has the same influence function as the median absolute deviation and the least median of squares (LMS) scale estimator (i.e., the length of the shortest half), but its finite-sample efficiency is higher. If we consider the standard deviation of a subsample instead of its range, we obtain a different class of 50% breakdown estimators. This class contains the least trimmed squares (LTS) scale estimator. Simulation shows that the LTS scale estimator is nearly unbiased, so it does not need a small-sample correction factor. Surprisingly, the efficiency of the LTS scale estimator is less than that of the LMS scale estimator.     

Maximum likelihood estimation for outcome‐dependent samples

In outcome‐dependent sampling, the continuous or binary outcome variable in a regression model is available in advance to guide selection of a sample on which explanatory variables are then measured. Selection probabilities may either be a smooth function of the outcome variable or be based on a stratification of the outcome. In many cases, only data from the final sample is accessible to the analyst. A maximum likelihood approach for this data configuration is developed here for the first time. The likelihood for fully general outcome‐dependent designs is stated, then the special case of Poisson sampling is examined in more detail. The maximum likelihood estimator differs from the well‐known maximum sample likelihood estimator, and an information bound result shows that the former is asymptotically more efficient. A simulation study suggests that the efficiency difference is generally small. Maximum sample likelihood estimation is therefore recommended in practice when only sample data is available. Some new smooth sample designs show considerable promise.     

A Robust and Almost Fully Efficient M-Estimator

Simultaneous robust estimates of location and scale parameters are derived from a class of M-estimating equations. A coefficient p ( p > 0), which plays a role similar to that of a tuning constant in the theory of M-estimation, determines the estimating equations. These estimating equations may be obtained as the gradient of a strictly convex criterion function. This article shows that the estimators are uniquely defined, asymptotically bi-variate normal and have positive breakdown for some choices of p . When p = 0.12 and p = 0.3, the estimators are almost fully efficient for normal and exponential distributions: efficiencies with respect to the maximum likelihood estimators are 1.00 and 0.99, respectively. It is shown that the location estimator for known scale has the maximum breakdown point 0.5 independent of p , when the target model is symmetric. Also it is shown that the scale estimator has a positive breakdown point which depends on the choice of p . A simulation study finds that the proposed location estimator has smaller variance than the Hodges–Lehmann estimator, Huber's minimax and bisquare M-estimators.     

Delete-m Jackknife for Unequal m

In this paper, the delete-mj jackknife estimator is proposed. This estimator is based on samples obtained from the original sample by successively removing mutually exclusive groups of unequal size. In a Monte Carlo simulation study, a hierarchical linear model was used to evaluate the role of nonnormal residuals and sample size on bias and efficiency of this estimator. It is shown that bias is reduced in exchange for a minor reduction in efficiency. The accompanying jackknife variance estimator even improves on both bias and efficiency, and, moreover, this estimator is mean-squared-error consistent, whereas the maximum likelihood equivalents are not.     

Conditional estimation using prior information in 2‐stage group sequential designs assuming asymptotic normality when the trial terminated early

Two‐stage designs are widely used to determine whether a clinical trial should be terminated early. In such trials, a maximum likelihood estimate is often adopted to describe the difference in efficacy between the experimental and reference treatments; however, this method is known to display conditional bias. To reduce such bias, a conditional mean‐adjusted estimator (CMAE) has been proposed, although the remaining bias may be nonnegligible when a trial is stopped for efficacy at the interim analysis. We propose a new estimator for adjusting the conditional bias of the treatment effect by extending the idea of the CMAE. This estimator is calculated by weighting the maximum likelihood estimate obtained at the interim analysis and the effect size prespecified when calculating the sample size. We evaluate the performance of the proposed estimator through analytical and simulation studies in various settings in which a trial is stopped for efficacy or futility at the interim analysis. We find that the conditional bias of the proposed estimator is smaller than that of the CMAE when the information time at the interim analysis is small. In addition, the mean‐squared error of the proposed estimator is also smaller than that of the CMAE. In conclusion, we recommend the use of the proposed estimator for trials that are terminated early for efficacy or futility.     

Focused information criterion and model averaging based on weighted composite quantile regression

We study the focused information criterion and frequentist model averaging and their application to post‐model‐selection inference for weighted composite quantile regression (WCQR) in the context of the additive partial linear models. With the non‐parametric functions approximated by polynomial splines, we show that, under certain conditions, the asymptotic distribution of the frequentist model averaging WCQR‐estimator of a focused parameter is a non‐linear mixture of normal distributions. This asymptotic distribution is used to construct confidence intervals that achieve the nominal coverage probability. With properly chosen weights, the focused information criterion based WCQR estimators are not only robust to outliers and non‐normal residuals but also can achieve efficiency close to the maximum likelihood estimator, without assuming the true error distribution. Simulation studies and a real data analysis are used to illustrate the effectiveness of the proposed procedure.     

THE MAXIMUM RESISTANCE OF TESTS

The breakdown point of an estimator is the smallest fraction of contamination that can force the value of the estimator beyond the boundary of the parameter space. It is well known that the highest possible breakdown point, under equivariance restrictions, is 50% of the sample. However, this upper bound is not always attainable. We give an example of an estimation problem in which the highest possible attainable breakdown point is much less than 50% of the sample. For hypothesis testing, we discuss the resistance of a test and propose new definitions of resistance. The maximum resistance to rejection (acceptance) is the smallest fraction of contamination necessary to force a test to reject (fail to reject) regardless of the original sample. We derive the maximum resistances of the t-test and sign test in the one-sample problem and of the t-test and Mood test in the two-sample problem. We briefly discuss another measure known as the expected resistance.     

LIKELIHOOD MOMENT ESTIMATION FOR THE GENERALIZED PARETO DISTRIBUTION

Traditional methods for estimating parameters in the generalized Pareto distribution have theoretical and computational defects. The moment estimator and the probability‐weighted moment estimator have low asymptotic efficiencies. They may not exist or may give nonsensical estimates. The maximum likelihood estimator, which sometimes does not exist, is asymptotically efficient, but its computation is complex and has convergence problems. The likelihood moment estimator is proposed, which is computationally easy and has high asymptotic efficiency.     

Robust estimation for ordinal regression

Ordinal regression is used for modelling an ordinal response variable as a function of some explanatory variables. The classical technique for estimating the unknown parameters of this model is Maximum Likelihood (ML). The lack of robustness of this estimator is formally shown by deriving its breakdown point and its influence function. To robustify the procedure, a weighting step is added to the Maximum Likelihood estimator, yielding an estimator with bounded influence function. We also show that the loss in efficiency due to the weighting step remains limited. A diagnostic plot based on the Weighted Maximum Likelihood estimator allows to detect outliers of different types in a single plot.     

On bias reduction estimators of skew-normal and skew-t distributions

A particular concerns of researchers in statistical inference is bias in parameters estimation. Maximum likelihood estimators are often biased and for small sample size, the first order bias of them can be large and so it may influence the efficiency of the estimator. There are different methods for reduction of this bias. In this paper, we proposed a modified maximum likelihood estimator for the shape parameter of two popular skew distributions, namely skew-normal and skew-t, by offering a new method. We show that this estimator has lower asymptotic bias than the maximum likelihood estimator and is more efficient than those based on the existing methods.     

Estimation of treatment effects based on possibly misspecified Cox regression

In randomized clinical trials, a treatment effect on a time-to-event endpoint is often estimated by the Cox proportional hazards model. The maximum partial likelihood estimator does not make sense if the proportional hazard assumption is violated. Xu and O'Quigley (Biostatistics 1:423-439, 2000) proposed an estimating equation, which provides an interpretable estimator for the treatment effect under model misspecification. Namely it provides a consistent estimator for the log-hazard ratio among the treatment groups if the model is correctly specified, and it is interpreted as an average log-hazard ratio over time even if misspecified. However, the method requires the assumption that censoring is independent of treatment group, which is more restricted than that for the maximum partial likelihood estimator and is often violated in practice. In this paper, we propose an alternative estimating equation. Our method provides an estimator of the same property as that of Xu and O'Quigley under the usual assumption for the maximum partial likelihood estimation. We show that our estimator is consistent and asymptotically normal, and derive a consistent estimator of the asymptotic variance. If the proportional hazards assumption holds, the efficiency of the estimator can be improved by applying the covariate adjustment method based on the semiparametric theory proposed by Lu and Tsiatis (Biometrika 95:679-694, 2008).     

A comparison of robust estimators based on two types of trimming

The least trimmed squares (LTS) estimator and the trimmed mean (TM) are two well-known trimming-based estimators of the location parameter. Both estimates are used in practice, and they are implemented in standard statistical software (e.g., S-PLUS, R, Matlab, SAS). The breakdown point of each of these estimators increases as the trimming proportion increases, while the efficiency decreases. Here we have shown that for a wide range of distributions with exponential and polynomial tails, TM is asymptotically more efficient than LTS as an estimator of the location parameter, when they have equal breakdown points.     

A new algorithm for maximum likelihood estimation in normal scale-mixture generalized autoregressive conditional heteroskedastic models

In this paper, we propose a new generalized autoregressive conditional heteroskedastic (GARCH) model using infinite normal scale-mixtures which can suitably avoid order selection problems in the application of finite normal scale-mixtures. We discuss its theoretical properties and develop a two-stage algorithm for the maximum likelihood estimator to estimate the mixing distribution non-parametric maximum likelihood estimator (NPMLE) as well as GARCH parameters (two-stage MLE). For the estimation of a mixing distribution, we employ a fast computational algorithm proposed by Wang [On fast computation of the non-parametric maximum likelihood estimate of a mixing distribution. J R Stat Soc Ser B. 2007;69:185–198] under the gradient characterization of the non-parametric mixture likelihood. The GARCH parameters are then estimated either using the expectation-mazimization algorithm or general optimization scheme. In addition, we propose a new forecasting algorithm of value-at-risk (VaR) using the two-stage MLE and the NPMLE. Through a simulation study and real data analysis, we compare the performance of the two-stage MLE with the existing ones including quasi-maximum likelihood estimator based on the standard normal density and the finite normal mixture quasi maximum estimated-likelihood estimator (cf. Lee S, Lee T. Inference for Box–Cox transformed threshold GARCH models with nuisance parameters. Scand J Stat. 2012;39:568–589) in terms of the relative efficiency and accuracy of VaR forecasting.     

Partial breakdown in two-factor models

The usual (global) breakdown point describes the worst effect that a given number of gross errors can have. In a two-way layout, without interaction, one is frustrated by the small number of gross errors such a design can tolerate. However, neither the whole fit nor all parameter estimates need to be affected by such a breakdown. An example from molecular spectroscopy serves to illustrate such partial breakdown in a large, “sparse” two-factor model. Because the global finite sample breakdown point is zero for all usual estimators in this example, this concept does not make sense in such problems. The more appropriate concept of partial breakdown point is discussed in this paper. It also provides a crude quantification of the robustness properties of an estimator, yet for any linear combination of the estimated parameters. The maximum number of gross errors to which the linear combination of the estimated parameters can resist is related to the minimum number of observations that must be omitted to make the linear function a non-estimable function. In the example, we are mainly interested in differences of parameters. Then the maximal partial breakdown point for regression equivariant estimators is one half, and Huber-type regression M-estimators with bounded ψ-function reach this limit.     

Investigation of the performance of trimmed estimators of life time distributions with censoring

For the lifetime (or negative) exponential distribution, the trimmed likelihood estimator has been shown to be explicit in the form of a β‐trimmed mean which is representable as an estimating functional that is both weakly continuous and Fréchet differentiable and hence qualitatively robust at the parametric model. It also has high efficiency at the model. The robustness is in contrast to the maximum likelihood estimator (MLE) involving the usual mean which is not robust to contamination in the upper tail of the distribution. When there is known right censoring, it may be perceived that the MLE which is the most asymptotically efficient estimator may be protected from the effects of ‘outliers’ due to censoring. We demonstrate that this is not the case generally, and in fact, based on the functional form of the estimators, suggest a hybrid defined estimator that incorporates the best features of both the MLE and the β‐trimmed mean. Additionally, we study the pure trimmed likelihood estimator for censored data and show that it can be easily calculated and that the censored observations are not always trimmed. The different trimmed estimators are compared by a modest simulation study.     

A note on the efficiency of composite quantile regression

Composite quantile regression (CQR) is motivated by the desire to have an estimator for linear regression models that avoids the breakdown of the least-squares estimator when the error variance is infinite, while having high relative efficiency even when the least-squares estimator is fully efficient. Here, we study two weighting schemes to further improve the efficiency of CQR, motivated by Jiang et al. [Oracle model selection for nonlinear models based on weighted composite quantile regression. Statist Sin. 2012;22:1479–1506]. In theory the two weighting schemes are asymptotically equivalent to each other and always result in more efficient estimators compared with CQR. Although the first weighting scheme is hard to implement, it sheds light on in what situations the improvement is expected to be large. A main contribution is to theoretically and empirically identify that standard CQR has good performance compared with weighted CQR only when the error density is logistic or close to logistic in shape, which was not noted in the literature.     

Focused information criteria for copulas

In this paper, we extend the focused information criterion (FIC) to copula models. Copulas are often used for applications where the joint tail behavior of the variables is of particular interest, and selecting a copula that captures this well is then essential. Traditional model selection methods such as the Akaike information criterion (AIC) and the Bayesian information criterion (BIC) aim at finding the overall best‐fitting model, which is not necessarily the one best suited for the application at hand. The FIC, on the other hand, evaluates and ranks candidate models based on the precision of their point estimates of a context‐given focus parameter. This could be any quantity of particular interest, for example, the mean, a correlation, conditional probabilities, or measures of tail dependence. We derive FIC formulae for the maximum likelihood estimator, the two‐stage maximum likelihood estimator, and the so‐called pseudo‐maximum‐likelihood (PML) estimator combined with parametric margins. Furthermore, we confirm the validity of the AIC formula for the PML estimator combined with parametric margins. To study the numerical behavior of FIC, we have carried out a simulation study, and we have also analyzed a multivariate data set pertaining to abalones. The results from the study show that the FIC successfully ranks candidate models in terms of their performance, defined as how well they estimate the focus parameter. In terms of estimation precision, FIC clearly outperforms AIC, especially when the focus parameter relates to only a specific part of the model, such as the conditional upper‐tail probability.