Regularization through variable selection and conditional MLE with application to classification in high dimensions

It is often the case that high-dimensional data consist of only a few informative components. Standard statistical modeling and estimation in such a situation is prone to inaccuracies due to overfitting, unless regularization methods are practiced. In the context of classification, we propose a class of regularization methods through shrinkage estimators. The shrinkage is based on variable selection coupled with conditional maximum likelihood. Using Stein's unbiased estimator of the risk, we derive an estimator for the optimal shrinkage method within a certain class. A comparison of the optimal shrinkage methods in a classification context, with the optimal shrinkage method when estimating a mean vector under a squared loss, is given. The latter problem is extensively studied, but it seems that the results of those studies are not completely relevant for classification. We demonstrate and examine our method on simulated data and compare it to feature annealed independence rule and Fisher's rule.     

Another look at Huber's estimator: A new minimax estimator in regression with stochastically bounded noise

Huber's estimator has had a long lasting impact, particularly on robust statistics. It is well known that under certain conditions, Huber's estimator is asymptotically minimax. A moderate generalization in rederiving Huber's estimator shows that Huber's estimator is not the only choice. We develop an alternative asymptotic minimax estimator and name it regression with stochastically bounded noise (RSBN). Simulations demonstrate that RSBN is slightly better in performance, although it is unclear how to justify such an improvement theoretically. We propose two numerical solutions: an iterative numerical solution, which is extremely easy to implement and is based on the proximal point method; and a solution by applying state-of-the-art nonlinear optimization software packages, e.g., SNOPT. Contribution: the generalization of the variational approach is interesting and should be useful in deriving other asymptotic minimax estimators in other problems.     

基于多变量动态模型的产出缺口估算

 本文运用多变量动态模型系统下的Beveridge-Nelson分解方法和贝叶斯Gibbs抽样估计,估算了1985年1季度至2008年2季度期间中国的产出缺口,并且与传统的单变量估计方法测算的结果在统计属性和对货币政策调节的预测效果方面进行了比较。实证结果表明,不同产出缺口的统计属性存在差别,并且只有基于多变量系统测算的产出缺口对货币政策具有显著预测效果。这说明多变量模型估计出的产出缺口更全面地考虑了经济产出与其他相关变量的互动效应,含有的信息更为丰富,从而对宏观政策调整具有更重要的参考价值。     

Finite normal mixture copulas for multivariate discrete data modeling

A new family of copulas is introduced that provides flexible dependence structure while being tractable and simple to use for multivariate discrete data modeling. The construction exploits finite mixtures of uncorrelated normal distributions. Accordingly, the cumulative distribution function is simply the product of univariate normal distributions. At the same time, however, the mixing operation introduces association. The properties of the new family of copulas are examined and a concrete application is used to show its applicability.     

Modified inference about the mean of the exponential distribution using moving extreme ranked set sampling

The maximum likelihood estimator (MLE) and the likelihood ratio test (LRT) will be considered for making inference about the scale parameter of the exponential distribution in case of moving extreme ranked set sampling (MERSS). The MLE and LRT can not be written in closed form. Therefore, a modification of the MLE using the technique suggested by Maharota and Nanda (Biometrika 61:601–606, 1974) will be considered and this modified estimator will be used to modify the LRT to get a test in closed form for testing a simple hypothesis against one sided alternatives. The same idea will be used to modify the most powerful test (MPT) for testing a simple hypothesis versus a simple hypothesis to get a test in closed form for testing a simple hypothesis against one sided alternatives. Then it appears that the modified estimator is a good competitor of the MLE and the modified tests are good competitors of the LRT using MERSS and simple random sampling (SRS).     

Estimating a sensitive proportion through randomized response procedures based on auxiliary information

Randomized response techniques are widely employed in surveys dealing with sensitive questions to ensure interviewee anonymity and reduce nonrespondents rates and biased responses. Since Warner’s (J Am Stat Assoc 60:63–69, 1965) pioneering work, many ingenious devices have been suggested to increase respondent’s privacy protection and to better estimate the proportion of people, π _A , bearing a sensitive attribute. In spite of the massive use of auxiliary information in the estimation of non-sensitive parameters, very few attempts have been made to improve randomization strategy performance when auxiliary variables are available. Moving from Zaizai’s (Model Assist Stat Appl 1:125–130, 2006) recent work, in this paper we provide a class of estimators for π _A , for a generic randomization scheme, when the mean of a supplementary non-sensitive variable is known. The minimum attainable variance bound of the class is obtained and the best estimator is also identified. We prove that the best estimator acts as a regression-type estimator which is at least as efficient as the corresponding estimator evaluated without allowing for the auxiliary variable. The general results are then applied to Warner and Simmons’ model.     

ESTIMATING A PARAMETER WHEN IT IS KNOWN THAT THE PARAMETER EXCEEDS A GIVEN VALUE

In some statistical problems a degree of explicit, prior information is available about the value taken by the parameter of interest, θ say, although the information is much less than would be needed to place a prior density on the parameter's distribution. Often the prior information takes the form of a simple bound, ‘θ > θ₁ ’ or ‘θ < θ₁ ’, where θ₁ is determined by physical considerations or mathematical theory, such as positivity of a variance. A conventional approach to accommodating the requirement that θ > θ₁ is to replace an estimator, , of θ by the maximum of and θ₁. However, this technique is generally inadequate. For one thing, it does not respect the strictness of the inequality θ > θ₁ , which can be critical in interpreting results. For another, it produces an estimator that does not respond in a natural way to perturbations of the data. In this paper we suggest an alternative approach, in which bootstrap aggregation, or bagging, is used to overcome these difficulties. Bagging gives estimators that, when subjected to the constraint θ > θ₁ , strictly exceed θ₁ except in extreme settings in which the empirical evidence strongly contradicts the constraint. Bagging also reduces estimator variability in the important case for which is close to θ₁, and more generally produces estimators that respect the constraint in a smooth, realistic fashion.     

Merging data from multiple sources: pretest and shrinkage perspectives

In this article, we have developed asymptotic theory for the simultaneous estimation of the k means of arbitrary populations under the common mean hypothesis and further assuming that corresponding population variances are unknown and unequal. The unrestricted estimator, the Graybill-Deal-type restricted estimator, the preliminary test, and the Stein-type shrinkage estimators are suggested. A large sample test statistic is also proposed as a pretest for testing the common mean hypothesis. Under the sequence of local alternatives and squared error loss, we have compared the asymptotic properties of the estimators by means of asymptotic distributional quadratic bias and risk. Comprehensive Monte-Carlo simulation experiments were conducted to study the relative risk performance of the estimators with reference to the unrestricted estimator in finite samples. Two real-data examples are also furnished to illustrate the application of the suggested estimation strategies.     

A backward construction and simulation of correlated Poisson processes

In this paper, we consider a generalisation of the backward simulation method of Duch et al. [New approaches to operational risk modeling. IBM J Res Develop. 2014;58:1–9] to build bivariate Poisson processes with flexible time correlation structures, and to simulate the arrival times of the processes. The proposed backward construction approach uses the Marshall–Olkin bivariate binomial distribution for the conditional law and some well-known families of bivariate copulas for the joint success probability in lieu of the typical conditional independence assumption. The resulting bivariate Poisson process can exhibit various time correlation structures which are commonly observed in real data.