SCAD-Penalized Least Absolute Deviation Regression in High-Dimensional Models

When outliers and/or heavy-tailed errors exist in linear models, the least absolute deviation (LAD) regression is a robust alternative to the ordinary least squares regression. Existing variable-selection methods in linear models based on LAD regression either only consider the finite number of predictors or lack the oracle property associated with the estimator. In this article, we focus on the variable selection via LAD regression with a diverging number of parameters. The rate of convergence of the LAD estimator with the smoothly clipped absolute deviation (SCAD) penalty function is established. Furthermore, we demonstrate that, under certain regularity conditions, the penalized estimator with a properly selected tuning parameter enjoys the oracle property. In addition, the rank correlation screening method originally proposed by Li et al. (2011 Li, G.R., Peng, H., Zhu, L.X. (2011). Nonconcave penalized M-estimation with a diverging number of parameters. Statistica Sinica 21:391–419.[Web of Science ®] , [Google Scholar]) is applied to deal with ultrahigh dimensional data. Simulation studies are conducted for revealing the finite sample performance of the estimator. We further illustrate the proposed methodology by a real example.     

Empirical likelihood for varying-coefficient semiparametric mixed-effects errors-in-variables models with longitudinal data

In this paper, the empirical likelihood inferences for varying-coefficient semiparametric mixed-effects errors-in-variables models with longitudinal data are investigated. We construct the empirical log-likelihood ratio function for the fixed-effects parameters and the mean parameters of random-effects. The empirical log-likelihood ratio at the true parameters is proven to be asymptotically $\chi ^2_{q+r}$ , where $q$ and $r$ are dimensions of the fixed and random effects respectively, and the corresponding confidence regions for them are then constructed. We also obtain the maximum empirical likelihood estimator of the parameters of interest, and prove it is the asymptotically normal under some suitable conditions. A simulation study and a real data application are undertaken to assess the finite sample performance of the proposed method.     

Covariance matrix of the bias-corrected maximum likelihood estimator in generalized linear models

For the first time, we obtain a general formula for the \(n^{-2}\) asymptotic covariance matrix of the bias-corrected maximum likelihood estimators of the linear parameters in generalized linear models, where \(n\) is the sample size. The usefulness of the formula is illustrated in order to obtain a better estimate of the covariance of the maximum likelihood estimators and to construct better Wald statistics. Simulation studies and an application support our theoretical results.     

Asymptotic Properties of the MLE in Nonlinear Reproductive Dispersion Models With Stochastic Regressors

Nonlinear reproductive dispersion models with stochastic regressors (NRDMWSR) includes generalized linear models with stochastic regressors (Fahrmer and Kaufmann, 1985 Fahrmer , L. , Kaufmann , H. ( 1985 ). Consistency and asymptotic normality of the maximum likelihood estimator in generalized linear models . Ann. Statist. 13 : 342 – 368 . [Google Scholar]) as a special case. This article presents some mild regularity conditions. On the basis of those mild conditions, the existence, strong consistency, and asymptotic normality of maximum likelihood estimator (MLE) are obtained in NRDMWSR.     

Some notes on robust sure independence screening

Sure independence screening (SIS) proposed by Fan and Lv [4 J. Fan and R. Li, Variable selection via nonconcave penalized likelihood and its oracle properties, J. Amer. Statist. Assoc. 96 (2001), pp. 1348–1360. doi: 10.1198/016214501753382273[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]] uses marginal correlations to select important variables, and has proven to be an efficient method for ultrahigh-dimensional linear models. This paper provides two robust versions of SIS against outliers. The two methods, respectively, replace the sample correlation in SIS with two robust measures, and screen variables by ranking them. Like SIS, the proposed methods are simple and fast. In addition, they are highly robust against a substantial fraction of outliers in the data. These features make them applicable to large datasets which may contain outliers. Simulation results are presented to show their effectiveness.     

On likelihood ratio testing for penalized splines

Penalized spline regression using a mixed effects representation is one of the most popular nonparametric regression tools to estimate an unknown regression function $f(\cdot )$ . In this context testing for polynomial regression against a general alternative is equivalent to testing for a zero variance component. In this paper, we fill the gap between different published null distributions of the corresponding restricted likelihood ratio test under different assumptions. We show that: (1) the asymptotic scenario is determined by the choice of the penalty and not by the choice of the spline basis or number of knots; (2) non-standard asymptotic results correspond to common penalized spline penalties on derivatives of $f(\cdot )$ , which ensure good power properties; and (3) standard asymptotic results correspond to penalized spline penalties on $f(\cdot )$ itself, which lead to sizeable power losses under smooth alternatives. We provide simple and easy to use guidelines for the restricted likelihood ratio test in this context.     

Robust parameter estimation for the Ornstein–Uhlenbeck process

In this paper, we derive elementary M- and optimally robust asymptotic linear (AL)-estimates for the parameters of an Ornstein–Uhlenbeck process. Simulation and estimation of the process are already well-studied, see Iacus (Simulation and inference for stochastic differential equations. Springer, New York, 2008). However, in order to protect against outliers and deviations from the ideal law the formulation of suitable neighborhood models and a corresponding robustification of the estimators are necessary. As a measure of robustness, we consider the maximum asymptotic mean square error (maxasyMSE), which is determined by the influence curve (IC) of AL estimates. The IC represents the standardized influence of an individual observation on the estimator given the past. In a first step, we extend the method of M-estimation from Huber (Robust statistics. Wiley, New York, 1981). In a second step, we apply the general theory based on local asymptotic normality, AL estimates, and shrinking neighborhoods due to Kohl et?al. (Stat Methods Appl 19:333–354, 2010), Rieder (Robust asymptotic statistics. Springer, New York, 1994), Rieder (2003), and Staab (1984). This leads to optimally robust ICs whose graph exhibits surprising behavior. In the end, we discuss the estimator construction, i.e. the problem of constructing an estimator from the family of optimal ICs. Therefore we carry out in our context the One-Step construction dating back to LeCam (Asymptotic methods in statistical decision theory. Springer, New York, 1969) and compare it by means of simulations with MLE and M-estimator.     

Equalities between OLSE,BLUE and BLUP in the linear model

We consider equalities between the ordinary least squares estimator ( $\mathrm {OLSE} $ ), the best linear unbiased estimator ( $\mathrm {BLUE} $ ) and the best linear unbiased predictor ( $\mathrm {BLUP} $ ) in the general linear model $\{ \mathbf y , \mathbf X \varvec{\beta }, \mathbf V \}$ extended with the new unobservable future value $ \mathbf y _{*}$ of the response whose expectation is $ \mathbf X _{*}\varvec{\beta }$ . Our aim is to provide some new insight and new proofs for the equalities under consideration. We also collect together various expressions, without rank assumptions, for the $\mathrm {BLUP} $ and provide new results giving upper bounds for the Euclidean norm of the difference between the $\mathrm {BLUP} ( \mathbf y _{*})$ and $\mathrm {BLUE} ( \mathbf X _{*}\varvec{\beta })$ and between the $\mathrm {BLUP} ( \mathbf y _{*})$ and $\mathrm {OLSE} ( \mathbf X _{*}\varvec{\beta })$ . A remark is made on the application to small area estimation.     

Computationally simple estimation and improved efficiency for special cases of double truncation

Doubly truncated survival data arise when event times are observed only if they occur within subject specific intervals of times. Existing iterative estimation procedures for doubly truncated data are computationally intensive (Turnbull 38:290–295, 1976; Efron and Petrosian 94:824–825, 1999; Shen 62:835–853, 2010a). These procedures assume that the event time is independent of the truncation times, in the sample space that conforms to their requisite ordering. This type of independence is referred to as quasi-independence. In this paper we identify and consider two special cases of quasi-independence: complete quasi-independence and complete truncation dependence. For the case of complete quasi-independence, we derive the nonparametric maximum likelihood estimator in closed-form. For the case of complete truncation dependence, we derive a closed-form nonparametric estimator that requires some external information, and a semi-parametric maximum likelihood estimator that achieves improved efficiency relative to the standard nonparametric maximum likelihood estimator, in the absence of external information. We demonstrate the consistency and potentially improved efficiency of the estimators in simulation studies, and illustrate their use in application to studies of AIDS incubation and Parkinson’s disease age of onset.     

Order selection in finite mixtures of linear regressions

Finite mixture models can adequately model population heterogeneity when this heterogeneity arises from a finite number of relatively homogeneous clusters. An example of such a situation is market segmentation. Order selection in mixture models, i.e. selecting the correct number of components, however, is a problem which has not been satisfactorily resolved. Existing simulation results in the literature do not completely agree with each other. Moreover, it appears that the performance of different selection methods is affected by the type of model and the parameter values. Furthermore, most existing results are based on simulations where the true generating model is identical to one of the models in the candidate set. In order to partly fill this gap we carried out a (relatively) large simulation study for finite mixture models of normal linear regressions. We included several types of model (mis)specification to study the robustness of 18 order selection methods. Furthermore, we compared the performance of these selection methods based on unpenalized and penalized estimates of the model parameters. The results indicate that order selection based on penalized estimates greatly improves the success rates of all order selection methods. The most successful methods were \(MDL2\) , \(MRC\) , \(MRC_k\) , \(ICL\) – \(BIC\) , \(ICL\) , \(CAIC\) , \(BIC\) and \(CLC\) but not one method was consistently good or best for all types of model (mis)specification.     

A new kind of stochastic restricted biased estimator for logistic regression model

In the logistic regression model, the variance of the maximum likelihood estimator is inflated and unstable when the multicollinearity exists in the data. There are several methods available in literature to overcome this problem. We propose a new stochastic restricted biased estimator. We study the statistical properties of the proposed estimator and compare its performance with some existing estimators in the sense of scalar mean squared criterion. An example and a simulation study are provided to illustrate the performance of the proposed estimator.KEYWORDS: Logistic regression, maximum likelihood estimator, mean squared error matrix, ridge regression, simulation study, stochastic restricted estimatorMathematics Subject Classifications: Primary 62J05, Secondary 62J07     

Computing the log concave NPMLE for interval censored data

In analyzing interval censored data, a non-parametric estimator is often desired due to difficulties in assessing model fits. Because of this, the non-parametric maximum likelihood estimator (NPMLE) is often the default estimator. However, the estimates for values of interest of the survival function, such as the quantiles, have very large standard errors due to the jagged form of the estimator. By forcing the estimator to be constrained to the class of log concave functions, the estimator is ensured to have a smooth survival estimate which has much better operating characteristics than the unconstrained NPMLE, without needing to specify a parametric family or smoothing parameter. In this paper, we first prove that the likelihood can be maximized under a finite set of parameters under mild conditions, although the log likelihood function is not strictly concave. We then present an efficient algorithm for computing a local maximum of the likelihood function. Using our fast new algorithm, we present evidence from simulated current status data suggesting that the rate of convergence of the log-concave estimator is faster (between \(n^{2/5}\) and \(n^{1/2}\)) than the unconstrained NPMLE (between \(n^{1/3}\) and \(n^{1/2}\)).     

Classical and Bayesian estimation of P(X<Y) using upper record values from Kumaraswamy’s distribution

In this paper, maximum likelihood and Bayesian approaches have been used to obtain the estimation of \(P(X<Y)\) based on a set of upper record values from Kumaraswamy distribution. The existence and uniqueness of the maximum likelihood estimates of the Kumaraswamy distribution parameters are obtained. Confidence intervals, exact and approximate, as well as Bayesian credible intervals are constructed. Bayes estimators have been developed under symmetric (squared error) and asymmetric (LINEX) loss functions using the conjugate and non informative prior distributions. The approximation forms of Lindley (Trabajos de Estadistica 3:281–288, 1980) and Tierney and Kadane (J Am Stat Assoc 81:82–86, 1986) are used for the Bayesian cases. Monte Carlo simulations are performed to compare the different proposed methods.     

A Note on the Comparison of the Stein Estimator and the James-Stein Estimator

The seminal work of Stein (1956 Stein, C. (1956). Inadmissibility of the usual estimator for the mean of a multivariate normal distribution. Proc. Third Berkeley Symp. Mathemat. Statist. Probab., University of California Press, 1:197–206. [Google Scholar]) showed that the maximum likelihood estimator (MLE) of the mean vector of a p-dimensional multivariate normal distribution is inadmissible under the squared error loss function when p ? 3 and proposed the Stein estimator that dominates the MLE. Later, James and Stein (1961 James, W., Stein, C. (1961). Estimation with quadratic loss. Proc. Fourth Berkeley Symp. Mathemat. Statist. Probab., University of California Press, 1:361–379. [Google Scholar]) proposed the James-Stein estimator for the same problem and received much more attention than the original Stein estimator. We re-examined the Stein estimator and conducted an analytic comparison with the James-Stein estimator. We found that the Stein estimator outperforms the James-Stein estimator under certain scenarios and derived the sufficient conditions.     

Sparse group variable selection based on quantile hierarchical Lasso

The group Lasso is a penalized regression method, used in regression problems where the covariates are partitioned into groups to promote sparsity at the group level [27 M. Yuan and Y. Lin, Model selection and estimation in regression with grouped variables, J. R. Stat. Soc. Ser. B 68 (2006), pp. 49–67. doi: 10.1111/j.1467-9868.2005.00532.x[Crossref] , [Google Scholar]]. Quantile group Lasso, a natural extension of quantile Lasso [25 Y. Wu and Y. Liu, Variable selection in quantile regression, Statist. Sinica 19 (2009), pp. 801–817.[Web of Science ®] , [Google Scholar]], is a good alternative when the data has group information and has many outliers and/or heavy tails. How to discover important features that are correlated with interest of outcomes and immune to outliers has been paid much attention. In many applications, however, we may also want to keep the flexibility of selecting variables within a group. In this paper, we develop a sparse group variable selection based on quantile methods which select important covariates at both the group level and within the group level, which penalizes the empirical check loss function by the sum of square root group-wise L₁-norm penalties. The oracle properties are established where the number of parameters diverges. We also apply our new method to varying coefficient model with categorial effect modifiers. Simulations and real data example show that the newly proposed method has robust and superior performance.     

Let us do the twist again

Krämer (Sankhy $\bar{\mathrm{a }}$ 42:130–131, 1980) posed the following problem: “Which are the $\mathbf{y}$ , given $\mathbf{X}$ and $\mathbf{V}$ , such that OLS and Gauss–Markov are equal?”. In other words, the problem aimed at identifying those vectors $\mathbf{y}$ for which the ordinary least squares (OLS) and Gauss–Markov estimates of the parameter vector $\varvec{\beta }$ coincide under the general Gauss–Markov model $\mathbf{y} = \mathbf{X} \varvec{\beta } + \mathbf{u}$ . The problem was later called a “twist” to Kruskal’s Theorem, which provides conditions necessary and sufficient for the OLS and Gauss–Markov estimates of $\varvec{\beta }$ to be equal. The present paper focuses on a similar problem to the one posed by Krämer in the aforementioned paper. However, instead of the estimation of $\varvec{\beta }$ , we consider the estimation of the systematic part $\mathbf{X} \varvec{\beta }$ , which is a natural consequence of relaxing the assumption that $\mathbf{X}$ and $\mathbf{V}$ are of full (column) rank made by Krämer. Further results, dealing with the Euclidean distance between the best linear unbiased estimator (BLUE) and the ordinary least squares estimator (OLSE) of $\mathbf{X} \varvec{\beta }$ , as well as with an equality between BLUE and OLSE are also provided. The calculations are mostly based on a joint partitioned representation of a pair of orthogonal projectors.     

Some new methods to solve multicollinearity in logistic regression

The binary logistic regression is a widely used statistical method when the dependent variable is binary or dichotomous. In some of the situations of logistic regression, independent variables are collinear which leads to the problem of multicollinearity. It is known that multicollinearity affects the variance of maximum likelihood estimator (MLE) negatively. Thus, this article introduces new methods to estimate the shrinkage parameters of Liu-type logistic estimator proposed by Inan and Erdogan (2013 Inan, D., Erdogan, B. E. (2013). Liu-type logistic estimator. Communications in Statistics-Simulation and Computation 42(7):1578–1586. [Google Scholar]) which is a generalization of the Liu-type estimator defined by Liu (2003 Liu, K. (2003). Using Liu-type estimator to combat collinearity. Communications in Statistics: Theory and Methods 32(5):1009–1020. [Google Scholar]) for the linear model. A Monte Carlo study is used to show the effectiveness of the proposed methods over MLE using the mean squared error (MSE) and mean absolute error (MAE) criteria. A real data application is illustrated to show the benefits of new methods. According to the results of the simulation and application proposed methods have better performance than MLE.     

On exact and optimal single-sampling plans by variables

We deal with sampling by variables with two-way protection in the case of a $N\>(\mu ,\sigma ^2)$ distributed characteristic with unknown $\sigma $ . The LR sampling plan proposed by Lieberman and Resnikoff (JASA 50: 457 ${-}$ 516, 1955) and the BSK sampling plan proposed by Bruhn-Suhr and Krumbholz (Stat. Papers 31: 195–207, 1990) are based on the UMVU and the plug-in estimator, respectively. For given $p_1$ (AQL), $p_2$ (RQL) and $\alpha ,\beta $ (type I and II errors) we present an algorithm allowing to determine the optimal LR and BSK plans having minimal sample size among all plans satisfying the corresponding two-point condition on the OC. An R (R: A language and environment for statistical computing, R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL http://www.R-project.org/ 2012) package, ExLiebeRes‘ (Krumbholz and Steuer ExLiebeRes: calculating exact LR- and BSK-plans, R-package version 0.9.9. http://exlieberes.r-forge.r-project.org 2012) implementing that algorithm is provided to the public.     

A Euclidean Likelihood Estimator for Bivariate Tail Dependence

The spectral measure plays a key role in the statistical modeling of multivariate extremes. Estimation of the spectral measure is a complex issue, given the need to obey a certain moment condition. We propose a Euclidean likelihood-based estimator for the spectral measure which is simple and explicitly defined, with its expression being free of Lagrange multipliers. Our estimator is shown to have the same limit distribution as the maximum empirical likelihood estimator of Einmahl and Segers (2009 Einmahl , J. H. J. , Segers , J. ( 2009 ). Maximum empirical likelihood estimation of the spectral measure of an extreme-value distribution . Ann. Statist. 37 ( 5B ): 2953 – 2989 .[Crossref], [Web of Science ®] , [Google Scholar]). Numerical experiments suggest an overall good performance and identical behavior to the maximum empirical likelihood estimator. We illustrate the method in an extreme temperature data analysis.