Some notes on robust sure independence screening

Sure independence screening (SIS) proposed by Fan and Lv [4 J. Fan and R. Li, Variable selection via nonconcave penalized likelihood and its oracle properties, J. Amer. Statist. Assoc. 96 (2001), pp. 1348–1360. doi: 10.1198/016214501753382273[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]] uses marginal correlations to select important variables, and has proven to be an efficient method for ultrahigh-dimensional linear models. This paper provides two robust versions of SIS against outliers. The two methods, respectively, replace the sample correlation in SIS with two robust measures, and screen variables by ranking them. Like SIS, the proposed methods are simple and fast. In addition, they are highly robust against a substantial fraction of outliers in the data. These features make them applicable to large datasets which may contain outliers. Simulation results are presented to show their effectiveness.     

A Bayesian Meta-Modeling Approach for Gaussian Stochastic Process Models Using a Non Informative Prior

In this article, an efficient Bayesian meta-modeling approach is proposed for Gaussian stochastic process models in computer experiments. Different prior densities and particularly, a non informative hyper prior have been employed on the parameters involved in the correlation matrix. And the estimation of related parameters is obtained by the expectation-maximization algorithm. Compared with the recent work of Li and Sudjianto (2005 Li , R. , Sudjianto , A. ( 2005 ). Analysis of computer experiments using penalized likelihood in Kriging models . Technometrics 47 : 111 – 120 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), the proposed approach is not only of higher prediction accuracy but also of lower computational cost, due to the utilization of the non informative prior and the absence of tuning parameters. Experimental results demonstrate that our approach yields state-of-the-art performance.     

Tuning Parameter Selection and Various Good Fitting Characteristics for the Liu-Type Estimator in Linear Regression

Liu (2003 Liu , K. ( 2003 ). Using Liu-Type estimator to combat collinearity . Commun. Statist. Theor. Meth. 32 ( 5 ): 1009 – 1020 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) proposed the Liu-Type estimator (LTE) to combat the well-known multicollinearity problem in linear regression. In this article, various better fitting characteristics of the LTE than those of the ordinary ridge regression estimator (Hoerl and Kennard, 1970 Hoerl , A. E. , Kennard , R. W. ( 1970 ). Ridge regression: Biased estimation for non-orthogonal problems . Technometrics 12 : 55 – 67 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) are considered. In particular, we derived two methods to determine the parameter d for the LTE and find that the ridge parameter k could serve for regularization of an ill-conditioned design matrix, while the other parameter d could be used for tuning the fit quality. In addition, the coefficients of regression, coefficient of multiple determination, residual error variance, and generalized cross validation (GCV) of the prediction quality are very stable, and as the ridge parameter increases they eventually reach asymptotic levels, which produces robust regression models. Furthermore, a Monte Carlo evaluation of these features is also given to illustrate some of the theoretical results.     

Group Variable Selection with Oracle Property by Weight-Fused Adaptive Elastic Net Model for Strongly Correlated Data

This paper is the generalization of weight-fused elastic net (Fu and Xu, 2012 Fu, G., Xu, Q. (2012). Grouping variable selection by weight fused elastic net for multi-collinear data. Communications in Statistics-Simulation and Computation 41(2):205–221.[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), which performs group variable selection by combining weight-fused LASSO(wfLasso) and elastic net (Zou and Hastie, 2005 Zou, H., Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 67(2):301–320.[Crossref], [Web of Science ®] , [Google Scholar]) penalties. In this study, the elastic net penalty is replaced by adaptive elastic net penalty (AdaEnet) (Zou and Zhang, 2009 Zou, H., Zhang, H. (2009). On the adaptive elastic-net with a diverging number of parameters. Annals of Statistics 37(4):1733–1751.[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]), and a new group variable selection algorithm with oracle property (Fan and Li, 2001 Fan, J., Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96(456):1348–1360.[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]; Zou, 2006 Zou, H. (2006). The adaptive lasso and its oracle properties. Journal of the American Statistical Association 101(476):1418–1429.[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) is obtained.     

Comment on Ridge Estimation to the Restricted Linear Model

In this note, we show that the estimator and the following results given by Zhong and Yang (2007 Zhong , Z. , Yang , H. ( 2007 ). Ridge estimation to the restricted linear model . Commun. Statist. Theor. Meth. 36 : 2099 – 2115 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) are the same with that of Groß (2003 Groß , J. ( 2003 ). Restricted ridge estimation . Statist. Probab. Lett. 65 : 57 – 64 .[Crossref], [Web of Science ®] , [Google Scholar]).     

Estimating Prediction Error: Cross-Validation vs. Accumulated Prediction Error

We study the validation of prediction rules such as regression models and classification algorithms through two out-of-sample strategies, cross-validation and accumulated prediction error. We use the framework of Efron (1983 Efron , B. ( 1983 ). Estimating the error rate of a prediction rule: improvement on cross-validation . Journal of the American Statistical Association 78 : 316 – 331 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) where measures of prediction errors are defined as sample averages of expected errors and show through exact finite sample calculations that cross-validation and accumulated prediction error yield different smoothing parameter choices in nonparametric regression. The difference in choice does not vanish as sample size increases.     

Additive Mixed Effect Model for Clustered Doubly Censored Data

Cai and Zeng (2011 Cai, J. and Zeng, D. 2011. Additive mixed effect model for clustered failure time data. Biometrics, 67(4): 1340–1351. [Crossref], [PubMed] , [Google Scholar]) proposed an additive mixed effect model to analyze clustered right-censored data. In this article, we demonstrate that the approach of Cai and Zeng (2011 Cai, J. and Zeng, D. 2011. Additive mixed effect model for clustered failure time data. Biometrics, 67(4): 1340–1351. [Crossref], [PubMed] , [Google Scholar]) can be extended to clustered doubly censored data. Furthermore, when both left- and right-censoring variables are always observed, we propose alternative estimators using the approach of Cai and Cheng (2004 Cai, T. and Cheng, S. C. 2004. Semiparametric regression analysis for doubly censored data. Biometrika, 91: 277–290. [Crossref], [Web of Science ®] , [Google Scholar]). A simulation study is conducted to investigate the performance of the proposed estimators.     

Interval Estimation for the Scaled Half-Logistic Distribution Under Progressive Type-II Censoring

In this article, we propose an exact confidence interval and an exact test for the scale parameter of the scaled half-logistic distribution based on progressively Type-II censored sample. Using the Monte Carlo method, we compare the expected length of the proposed confidence interval with one of confidence interval proposed by Balakrishnan and Asgharzadeh (2005 Balakrishnan , N. , Asgharzadeh , A. ( 2005 ). Inference for the scaled half-logistic distribution based on progressively Type-II censored sample . Commun. Statist. Theor. Meth. 34 : 73 – 87 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]). The simulation results show that the proposed confidence interval performs better. Finally, we present a numerical example to illustrate the proposed procedures.     

A Note on Testing Homogeneity of Several Exponential Location Parameters

In this article, the problems of testing homogeneity of several exponential location parameters against simple and tree ordered alternatives are considered separately. Test procedures for both the alternatives are proposed using restricted maximum likelihood estimators (RMLE) of exponential location parameters under the respective orderings. Critical constants for the implementation of the proposed procedures are tabulated. Power comparison of the proposed test procedure under the simple ordered alternative with the procedure of Chen (1982 Chen , H. J. ( 1982 ). A new range statistic for comparisons of several exponential location parameters . Biometrika 69 ( 1 ): 257 – 260 .[Crossref], [Web of Science ®] , [Google Scholar]) and of Dhawan and Gill (1999 Dhawan , A. K. , Gill , A. N. ( 1999 ). A one-sided test for testing homogeneity of scale parameters against ordered alternative . Communication in Statistics—Theory and Methods 28 ( 10 ): 2417 – 2439 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) is carried out using Monte-Carlo simulation.     

On Testing Sample Selection Bias Under the Multicollinearity Problem

ABSTRACT

This paper reviews and extends the literature on the finite sample behavior of tests for sample selection bias. Monte Carlo results show that, when the “multicollinearity problem” identified by Nawata (1993 Nawata , K. ( 1993 ). A note on the estimation of models with sample-selection biases . Economics Letters 42 : 15 – 24 . [CSA] [CROSSREF] [Crossref], [Web of Science ®] , [Google Scholar]) is severe, (i) the t-test based on the Heckman–Greene variance estimator can be unreliable, (ii) the Likelihood Ratio test remains powerful, and (iii) nonnormality can be interpreted as severe sample selection bias by Maximum Likelihood methods, leading to negative Wald statistics. We also confirm previous findings (Leung and Yu, 1996 Leung , S. F. , Yu , S. ( 1996 ). On the choice between sample selection and two-part models . Journal of Econometrics 72 : 197 – 229 . [CSA] [CROSSREF] [Crossref], [Web of Science ®] , [Google Scholar]) that the standard regression-based t-test (Heckman, 1979 Heckman , J. J. ( 1979 ). Sample selection bias as a specification error . Econometrica 47 : 153 – 161 . [CSA] [Crossref], [Web of Science ®] , [Google Scholar]) and the asymptotically efficient Lagrange Multiplier test (Melino, 1982 Melino , A. ( 1982 ). Testing for sample selection bias . Review of Economic Studies 49 : 151 – 153 . [CSA] [Crossref], [Web of Science ®] , [Google Scholar]), are robust to nonnormality but have very little power.     

Tuning Parameter Selection in Penalized Frailty Models

The penalized likelihood approach of Fan and Li (2001 Fan, J., Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Association 96:1348–1360.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar], 2002 Fan, J., Li, R. (2002). Variable selection for Cox’s proportional hazards model and frailty model. The Annals of Statistics 30:74–99.[Crossref], [Web of Science ®] , [Google Scholar]) differs from the traditional variable selection procedures in that it deletes the non-significant variables by estimating their coefficients as zero. Nevertheless, the desirable performance of this shrinkage methodology relies heavily on an appropriate selection of the tuning parameter which is involved in the penalty functions. In this work, new estimates of the norm of the error are firstly proposed through the use of Kantorovich inequalities and, subsequently, applied to the frailty models framework. These estimates are used in order to derive a tuning parameter selection procedure for penalized frailty models and clustered data. In contrast with the standard methods, the proposed approach does not depend on resampling and therefore results in a considerable gain in computational time. Moreover, it produces improved results. Simulation studies are presented to support theoretical findings and two real medical data sets are analyzed.     

Parametric Nonlinear Regression with Endogenous Switching

Based on the insightful work of Olsen (1980 Olsen , R. J. ( 1980 ). A least squares correction for selectivity bias . Econometrica 48 : 1815 – 1820 .[Crossref], [Web of Science ®] , [Google Scholar]) for the linear context, a generic and unifying framework is developed that affords a simple extension of the classical method of Heckman (1974 Heckman , J. ( 1974 ). Shadow prices, market wages, and labor supply . Econometrica 42 : 679 – 694 .[Crossref], [Web of Science ®] , [Google Scholar], 1976 Heckman , J. ( 1976 ). The common structure of statistical models of truncation sample selection and limited dependent variables and a simple estimator for such models . Annals of Economic and Social Measurement 5 : 475 – 492 . [Google Scholar], 1978 Heckman , J. ( 1978 ). Dummy endogenous variables in a simultaneous equation system . Econometrica 46 : 931 – 959 .[Crossref], [Web of Science ®] , [Google Scholar], 1979 Heckman , J. ( 1979 ). Sample selection bias as a specification error . Econometrica 47 : 153 – 161 .[Crossref], [Web of Science ®] , [Google Scholar]) to a broad class of nonlinear regression models involving endogenous switching and its two most common incarnations, endogenous sample selection and endogenous treatment effects. The approach should be appealing to applied researchers for three reasons. First, econometric applications involving endogenous switching abound. Secondly, the approach requires neither linearity of the regression function nor full parametric specification of the model. It can, in fact, be applied under the minimal parametric assumptions—i.e., specification of only the conditional means of the outcome and switching variables. Finally, it is amenable to relatively straightforward estimation methods. Examples of applications of the method are discussed.     

Automatic Smoothing for Poisson Regression

Abstract

Adaptive choice of smoothing parameters for nonparametric Poisson regression (O'Sullivan et al., 1986 O'Sullivan , F. , Yandell , B. S. , Raynor , W. J., Jr. ( 1986 ). Automatic smoothing of regression functions in generalized linear models . J. Amer. Statist. Assoc. 81 : 96 – 103 . [CSA] [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) is considered in this article. A computable approximation of the unbiased risk estimate (AUBR) for Poisson regression is introduced. This approximation can be used to automatically tune the smoothing parameter for the penalized likelihood estimator. An alternative choice is the generalized approximate cross validation (GACV) proposed by Xiang and Wahba (1996 Xiang , D. , Wahba , G. ( 1996 ). A generalized approximate cross validation for smoothing splines with non-Gaussian data . Statist. Sinica 6 (3): 675 – 692 .[Web of Science ®] , [Google Scholar]). Although GACV enjoys a great success in practice when applying for nonparametric logisitic regression, its performance for Poisson regression is not clear. Numerical simulations have been conducted to evaluate the GACV and AUBR based tuning methods. We found that GACV has a tendency to oversmooth the data when the intensity function is small. As a consequence, we suggest tuning the smoothing parameter using AUBR in practice.     

Bayesian Inference for the Precision Matrix for Scale Mixtures of Normal Distributions

Baysian inference is considered for the precision matrix of the multivariate regression model with distribution of the random responses belonging to the multivariate scale mixtures of normal distributions. The posterior distribution and some identities involving expectations taken with respect to this posterior distribution are derived when the prior distribution of the parameters is from the conjugate family. The results are specialized to the case where the random responses have a matrix-t distribution and thus generalizing the results of Zellner (1976 Zellner , A. ( 1976 ). Bayesian and non-Bayesian analysis of the regression model with multivariate Student-t error terms . J. Amer. Statist. Assoc. 71 : 400 – 405 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) and Muirhead (1986 Muirhead , R. J. ( 1986 ). A note on some Wishart expectations . Metrika 33 : 247 – 251 .[Crossref] , [Google Scholar]).     

A Generalization to Bivariate Mittag–Leffler and Bivariate Discrete Mittag–Leffler Autoregressive Processes

A generalization of the Gaver and Lewis (1980 Gaver , D. P. , Lewis , P. A. W. ( 1980 ). First order autoregressive gamma sequences and point processes . Adv. Appl. Probab. 12 : 727 – 745 .[Crossref], [Web of Science ®] , [Google Scholar]) model of first-order autoregressive process with marginals as bivariate Mittag–Leffler distribution is obtained. A necessary and sufficient condition for stationarity of the process is established. Autoregressive process with marginals follow bivariate discrete Mittag–Leffler distribution is also developed. The unknown parameters of the processes are estimated and some numerical results of the estimations are given.     

Step-Up Procedure for Multiple Comparison with a Control for Multivariate Normal Means

In this study, we consider the multiple comparison with a control for multivariate normal means. Specifically, we construct a step-up procedure by referring to Dunnett and Tamhane (1992 Dunnett , C. W. , Tamhane , A. C. ( 1992 ). A step-up multiple test procedure . Journal of the American Statistical Association 87 : 162 – 170 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]). We derive recursive formulae for determining the critical values of the step-up procedure for a specified significance level. Then we formulate the power of the test. Finally, we compare the step-up procedure with the single-step procedure proposed by Nakamura and Imada (2005 Nakamura , T. , Imada , T. ( 2005 ). Multiple comparison procedure of Dunnett's type for multivariate normal means . Journal of the Japanese Society of Computational Statistics 18 : 21 – 32 .[Crossref] , [Google Scholar]) and the step-down procedure proposed by Imada and Douke (2007 Imada , T. , Douke , H. ( 2007 ). Step down procedure for comparing several treatments with a control based on multivariate normal response . Biometrical Journal 49 ( 1 ): 18 – 29 .[Crossref], [Web of Science ®] , [Google Scholar]) in terms of numerical examples regarding the power of the test.     

Maximum Likelihood Estimation in Compound Geometric Failure Model with Changing Parameters From Type-I Two-Stage Progressively Censored and Group Censored Samples

Boardman and Kendell (1970 Boardman , T. J. , Kendell , P. J. ( 1970 ). Estimation in compound failure models . Technometrics 12 : 891 – 908 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) considered the problem of estimation with respect to Type-I censoring when an item is subjected to only one of the two causes of failure assuming exponential model. Patel and Gajjar (1992 Patel , M. N. , Gajjar , A. V. ( 1992 ). Maximum likelihood estimation in compound exponential failure model with changing failure rates from Type-I progressively censored and group censored samples . Commun. Statist. Theor. Meth. 21 ( 10 ): 2899 – 2908 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) considered extension of the Boardman and Kendell's results in case of two-stage progressive censoring. Here we have considered geometric competing risk failure model with two independent causes of failures. Maximum likelihood estimation of the parameters is carried out using Type-I two-stage progressively censored and group censored samples. Asymptotic standard errors of the estimators are obtained for both the cases. Two illustrative examples are cited for ungroup and group competing risk models.     

A side sensitive modified group runs control chart to detect shifts in the process mean

Gadre and Rattihalli [5 Gadre, M. P. and Rattihalli, R. N. 2006. Modified group runs control charts to detect increases in fraction non-conforming and shifts in the process mean. Commun. Stat. Simul. Comput., 35: 225–240. [Taylor & Francis Online], [Web of Science ®] [Google Scholar]] have introduced the Modified Group Runs (MGR) control chart to identify the increases in fraction non-conforming and to detect shifts in the process mean. The MGR chart reduces the out-of-control average time-to-signal (ATS), as compared with most of the well-known control charts. In this article, we develop the Side Sensitive Modified Group Runs (SSMGR) chart to detect shifts in the process mean. With the help of numerical examples, it is illustrated that the SSMGR chart performs better than the Shewhart's X¯ chart, the synthetic chart [12 Wu, Z. and Spedding, T. A. 2000. A synthetic control chart for detecting small shifts in the process mean. J. Qual. Technol., 32: 32–38. [Taylor & Francis Online], [Web of Science ®] [Google Scholar]], the Group Runs chart [4 Gadre, M. P. and Rattihalli, R. N. 2004. A group runs control chart for detecting shifts in the process mean. Econ. Qual. Control, 19: 29–43. [Crossref] [Google Scholar]], the Side Sensitive Group Runs chart [6 Gadre, M. P. and Rattihalli, R. N. 2007. A side sensitive group runs control chart for detecting shifts in the process mean. Stat. Methods Appl., 16: 27–37. [Crossref], [Web of Science ®] [Google Scholar]], as well as the MGR chart [5 Gadre, M. P. and Rattihalli, R. N. 2006. Modified group runs control charts to detect increases in fraction non-conforming and shifts in the process mean. Commun. Stat. Simul. Comput., 35: 225–240. [Taylor & Francis Online], [Web of Science ®] [Google Scholar]]. In some situations it is also superior to the Cumulative Sum chart p9 Page, E. S. 1954. Continuous inspection schemes. Biometrika, 41: 100–114. [Crossref], [Web of Science ®] [Google Scholar]] and the exponentially weighed moving average chart [10 Roberts, S. W. 1959. Control chart tests based on geometric moving averages. Technometrics, 1: 239–250. [Taylor & Francis Online] [Google Scholar]]. In the steady state also, its performance is better than the above charts.     

Block–Savits Characterization and Star Ordering of Exponential Mixtures

Block and Savits (1980 Block , H. W. , Savits , T. H. ( 1980 ). Laplace transforms for classes of life distributions . Ann. Probab. 8 : 465 – 474 .[Crossref], [Web of Science ®] , [Google Scholar]) established a characterization of life distributions using the Laplace transform. In this article, we remark that one of the necessary conditions to be IFRA distribution is equivalent to the star ordering of exponential mixtures. It leads to the definition of two new classes of life distributions, called LIFR and LIFRA, and their dual classes: LDFR and LDFRA. It occurs that these classes have many useful aging properties and preserve known reliability operations. Properties of the classes are studied and relations with known classes are established.     

A Note on Importance Resampling for Multi-Dimensional Statistics

Johns (1988 Johns , M. V. (1988). Importance sampling for bootstrap confidence intervals. Journal of the American Statistical Association 83:709–714.[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), Davison (1988 Davison , A. C. ( 1988 ). Discussion of paper by D. V. Hinkley . Journal of the Royal Statistical Society Series B 50 : 356 – 357 . [Google Scholar]), and Do and Hall (1991 Do , K. A. , Hall , P. ( 1991 ). On importance sampling for the bootstrap . Biometrika 78 : 161 – 167 .[Crossref], [Web of Science ®] , [Google Scholar]) used importance sampling for calculating bootstrap distributions of one-dimensional statistics. Realizing that their methods can not be extended easily to multi-dimensional statistics, Fuh and Hu (2004 Fuh , C. D. , Hu , I. ( 2004 ). Efficient importance sampling for events of moderate deviations with applications . Biometrika 91 : 471 – 490 .[Crossref], [Web of Science ®] , [Google Scholar]) proposed an exponential tilting formula for statistics of multi-dimension, which is optimal in the sense that the asymptotic variance is minimized for estimating tail probabilities of asymptotically normal statistics. For one-dimensional statistics, Hu and Su (2008 Hu , J. , Su , Z. ( 2008 ). Adaptive resampling algorithms for estimating bootstrap distributions . Journal of Statistical Planning and Inference 138 ( 6 ): 1763 – 1777 .[Crossref], [Web of Science ®] , [Google Scholar]) proposed a multi-step variance minimization approach that can be viewed as a generalization of the two-step variance minimization approach proposed by Do and Hall (1991 Do , K. A. , Hall , P. ( 1991 ). On importance sampling for the bootstrap . Biometrika 78 : 161 – 167 .[Crossref], [Web of Science ®] , [Google Scholar]). In this article, we generalize the approach of Hu and Su (2008 Hu , J. , Su , Z. ( 2008 ). Adaptive resampling algorithms for estimating bootstrap distributions . Journal of Statistical Planning and Inference 138 ( 6 ): 1763 – 1777 .[Crossref], [Web of Science ®] , [Google Scholar]) to multi-dimensional statistics, which applies to general statistics and does not resort to asymptotics. Empirical results on a real survival data set show that the proposed algorithm provides significant computational efficiency gains.