Regularization through variable selection and conditional MLE with application to classification in high dimensions

It is often the case that high-dimensional data consist of only a few informative components. Standard statistical modeling and estimation in such a situation is prone to inaccuracies due to overfitting, unless regularization methods are practiced. In the context of classification, we propose a class of regularization methods through shrinkage estimators. The shrinkage is based on variable selection coupled with conditional maximum likelihood. Using Stein's unbiased estimator of the risk, we derive an estimator for the optimal shrinkage method within a certain class. A comparison of the optimal shrinkage methods in a classification context, with the optimal shrinkage method when estimating a mean vector under a squared loss, is given. The latter problem is extensively studied, but it seems that the results of those studies are not completely relevant for classification. We demonstrate and examine our method on simulated data and compare it to feature annealed independence rule and Fisher's rule.     

Another look at Huber's estimator: A new minimax estimator in regression with stochastically bounded noise

Huber's estimator has had a long lasting impact, particularly on robust statistics. It is well known that under certain conditions, Huber's estimator is asymptotically minimax. A moderate generalization in rederiving Huber's estimator shows that Huber's estimator is not the only choice. We develop an alternative asymptotic minimax estimator and name it regression with stochastically bounded noise (RSBN). Simulations demonstrate that RSBN is slightly better in performance, although it is unclear how to justify such an improvement theoretically. We propose two numerical solutions: an iterative numerical solution, which is extremely easy to implement and is based on the proximal point method; and a solution by applying state-of-the-art nonlinear optimization software packages, e.g., SNOPT. Contribution: the generalization of the variational approach is interesting and should be useful in deriving other asymptotic minimax estimators in other problems.     

On the incidence–prevalence relation and length‐biased sampling

For many diseases, logistic constraints render large incidence studies difficult to carry out. This becomes a drawback, particularly when a new study is needed each time the incidence rate is investigated in a new population. By carrying out a prevalent cohort study with follow‐up it is possible to estimate the incidence rate if it is constant. The authors derive the maximum likelihood estimator (MLE) of the overall incidence rate, λ, as well as age‐specific incidence rates, by exploiting the epidemiologic relationship, (prevalence odds) = (incidence rate) × (mean duration) (P/[1 ? P] = λ × µ). The authors establish the asymptotic distributions of the MLEs and provide approximate confidence intervals for the parameters. Moreover, the MLE of λ is asymptotically most efficient and is the natural estimator obtained by substituting the marginal maximum likelihood estimators for P and µ into P/[1 ? P] = λ × µ. Following‐up the subjects allows the authors to develop these widely applicable procedures. The authors apply their methods to data collected as part of the Canadian Study of Health and Ageing to estimate the incidence rate of dementia amongst elderly Canadians. The Canadian Journal of Statistics © 2009 Statistical Society of Canada     

ESTIMATING A PARAMETER WHEN IT IS KNOWN THAT THE PARAMETER EXCEEDS A GIVEN VALUE

In some statistical problems a degree of explicit, prior information is available about the value taken by the parameter of interest, θ say, although the information is much less than would be needed to place a prior density on the parameter's distribution. Often the prior information takes the form of a simple bound, ‘θ > θ₁ ’ or ‘θ < θ₁ ’, where θ₁ is determined by physical considerations or mathematical theory, such as positivity of a variance. A conventional approach to accommodating the requirement that θ > θ₁ is to replace an estimator, , of θ by the maximum of and θ₁. However, this technique is generally inadequate. For one thing, it does not respect the strictness of the inequality θ > θ₁ , which can be critical in interpreting results. For another, it produces an estimator that does not respond in a natural way to perturbations of the data. In this paper we suggest an alternative approach, in which bootstrap aggregation, or bagging, is used to overcome these difficulties. Bagging gives estimators that, when subjected to the constraint θ > θ₁ , strictly exceed θ₁ except in extreme settings in which the empirical evidence strongly contradicts the constraint. Bagging also reduces estimator variability in the important case for which is close to θ₁, and more generally produces estimators that respect the constraint in a smooth, realistic fashion.     

基于季报数据的收入公告后漂移现象研究

利用收入指标对股票超额收益率进行解释构成了理解"定价异常"的重要方面。为此,基于盈余公告后漂移的理论分析框架,以上证A股2008年1季度至2011年4季度的相关数据为基础,利用标准化预期外收入估计量(SURE)和分类检验模型方法对中国股票市场公告期内股票价格的收入公告后漂移现象进行实证检验,研究发现:在盈余公告期内,预期外收入与股票超额收益率呈现出负相关或是不显著的关系,即中国股票市场的收入公告后漂移效应不显著。之后的稳健性分析也同样证实了负相关或是不显著关系的存在,而这种异常可能与中国股市的弱有效率相关。     

Model-based confidence bands for survival functions

This paper focuses on a novel method of developing one-sample confidence bands for survival functions from right censored data. The approach is model-based, relying on a parametric model for the conditional expectation of the censoring indicator given the observed minimum, and derives its strength from easy access to a good-fitting model among a plethora of choices available for binary response data. The substantive methodological contribution is in exploiting a semiparametric estimator of the survival function to produce improved simultaneous confidence bands. To obtain critical values for computing the confidence bands, a two-stage bootstrap approach that combines the classical bootstrap with the more recent model-based regeneration of censoring indicators is proposed and a justification of its asymptotic validity is also provided. Several different confidence bands are studied using the proposed approach. Numerical studies, including robustness of the proposed bands to misspecification, are carried out to check efficacy. The method is illustrated using two lung cancer data sets.     

Detecting influential observations in Liu and modified Liu estimators

In regression, detecting anomalous observations is a significant step for model-building process. Various influence measures based on different motivational arguments are designed to measure the influence of observations through different aspects of various regression models. The presence of influential observations in the data is complicated by the existence of multicollinearity. The purpose of this paper is to assess the influence of observations in the Liu [9] and modified Liu [15] estimators by using the method of approximate case deletion formulas suggested by Walker and Birch [14]. A numerical example using a real data set used by Longley [10] and a Monte Carlo simulation are given to illustrate the theoretical results.     

An efficient sequential design of clinical trials

We propose an efficient group sequential monitoring rule for clinical trials. At each interim analysis both efficacy and futility are evaluated through a specified loss structure together with the predicted power. The proposed design is robust to a wide range of priors, and achieves the specified power with a saving of sample size compared to existing adaptive designs. A method is also proposed to obtain a reduced-bias estimator of treatment difference for the proposed design. The new approaches hold great potential for efficiently selecting a more effective treatment in comparative trials. Operating characteristics are evaluated and compared with other group sequential designs in empirical studies. An example is provided to illustrate the application of the method.     

Asymptotic normality of Huber–Dutter estimators in a linear model with AR(1) processes

The paper studies a linear regression model with first order autoregressive (AR(1)) processes. The Huber–Dutter (HD) estimators of unknown parameters are given, and the asymptotic normality of the HD estimators is investigated. An example is presented to illustrate the proposed method.     

Asymptotically minimax non–parametric function estimation with positivity constraints I

The spatially inhomogeneous smoothness of the non-parametric density or regression-function to be estimated by non-parametric methods is often modelled by Besov- and Triebel-type smoothness constraints. For such problems, Donoho and Johnstone [D.L. Donoho and I.M. Johnstone, Minimax estimation via wavelet shrinkage. Ann. Stat. 26 (1998), pp. 879–921.], Delyon and Juditsky [B. Delyon and A. Juditsky, On minimax wavelet estimators, Appl. Comput. Harmon. Anal. 3 (1996), pp. 215–228.] studied minimax rates of convergence for wavelet estimators with thresholding, while Lepski et al. [O.V. Lepski, E. Mammen, and V.G. Spokoiny, Optimal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimators with variable bandwidth selectors, Ann. Stat. 25 (1997), pp. 929–947.] proposed a variable bandwidth selection for kernel estimators that achieved optimal rates over Besov classes. However, a second challenge in many real applications of non-parametric curve estimation is that the function must be positive. Here, we show how to construct estimators under positivity constraints that satisfy these constraints and also achieve minimax rates over the appropriate smoothness class.