Efficient Robust Estimation for Linear Models with Missing Response at Random

Coefficient estimation in linear regression models with missing data is routinely carried out in the mean regression framework. However, the mean regression theory breaks down if the error variance is infinite. In addition, correct specification of the likelihood function for existing imputation approach is often challenging in practice, especially for skewed data. In this paper, we develop a novel composite quantile regression and a weighted quantile average estimation procedure for parameter estimation in linear regression models when some responses are missing at random. Instead of imputing the missing response by randomly drawing from its conditional distribution, we propose to impute both missing and observed responses by their estimated conditional quantiles given the observed data and to use the parametrically estimated propensity scores to weigh check functions that define a regression parameter. Both estimation procedures are resistant to heavy‐tailed errors or outliers in the response and can achieve nice robustness and efficiency. Moreover, we propose adaptive penalization methods to simultaneously select significant variables and estimate unknown parameters. Asymptotic properties of the proposed estimators are carefully investigated. An efficient algorithm is developed for fast implementation of the proposed methodologies. We also discuss a model selection criterion, which is based on an IC_Q ‐type statistic, to select the penalty parameters. The performance of the proposed methods is illustrated via simulated and real data sets.     

Bayesian Elastic Net Tobit Quantile Regression

In this article, the problem of parameter estimation and variable selection in the Tobit quantile regression model is considered. A Tobit quantile regression with the elastic net penalty from a Bayesian perspective is proposed. Independent gamma priors are put on the l₁ norm penalty parameters. A novel aspect of the Bayesian elastic net Tobit quantile regression is to treat the hyperparameters of the gamma priors as unknowns and let the data estimate them along with other parameters. A Bayesian Tobit quantile regression with the adaptive elastic net penalty is also proposed. The Gibbs sampling computational technique is adapted to simulate the parameters from the posterior distributions. The proposed methods are demonstrated by both simulated and real data examples.     

Non‐parametric Estimation of Extreme Risk Measures from Conditional Heavy‐tailed Distributions

In this paper, we introduce a new risk measure, the so‐called conditional tail moment. It is defined as the moment of order a ≥ 0 of the loss distribution above the upper α‐quantile where α ∈ (0,1). Estimating the conditional tail moment permits us to estimate all risk measures based on conditional moments such as conditional tail expectation, conditional value at risk or conditional tail variance. Here, we focus on the estimation of these risk measures in case of extreme losses (where α ↓0 is no longer fixed). It is moreover assumed that the loss distribution is heavy tailed and depends on a covariate. The estimation method thus combines non‐parametric kernel methods with extreme‐value statistics. The asymptotic distribution of the estimators is established, and their finite‐sample behaviour is illustrated both on simulated data and on a real data set of daily rainfalls.     

Lessons From Quantile Panel Estimation of the Environmental Kuznets Curve

We employ quantile regression fixed effects models to estimate the income-pollution relationship on NO _x (nitrogen oxide) and SO ₂ (sulfur dioxide) using U.S. data. Conditional median results suggest that conditional mean methods provide too optimistic estimates about emissions reduction for NO _x , while the opposite is found for SO ₂. Deleting outlier states reverses the absence of a turning point for SO ₂ in the conditional mean model, while the conditional median model is robust to them. We also document the relationship's sensitivity to including additional covariates for NO _x , and undertake simulations to shed light on some estimation issues of the methods employed.     

Simultaneous sparse model selection and coefficient estimation for heavy-tailed autoregressive processes

We propose a sparse coefficient estimation and automated model selection procedure for autoregressive processes with heavy-tailed innovations based on penalized conditional maximum likelihood. Under mild moment conditions on the innovation processes, the penalized conditional maximum likelihood estimator satisfies a strong consistency, O_P(N^?1/2) consistency, and the oracle properties, where N is the sample size. We have the freedom in choosing penalty functions based on the weak conditions on them. Two penalty functions, least absolute shrinkage and selection operator and smoothly clipped average deviation, are compared. The proposed method provides a distribution-based penalized inference to AR models, which is especially useful when the other estimation methods fail or under perform for AR processes with heavy-tailed innovations [Feigin, Resnick. Pitfalls of fitting autoregressive models for heavy-tailed time series. Extremes. 1999;1:391–422]. A simulation study confirms our theoretical results. At the end, we apply our method to a historical price data of the US Industrial Production Index for consumer goods, and obtain very promising results.     

An efficient model-free estimation of multiclass conditional probability

Conventional multiclass conditional probability estimation methods, such as Fisher's discriminate analysis and logistic regression, often require restrictive distributional model assumption. In this paper, a model-free estimation method is proposed to estimate multiclass conditional probability through a series of conditional quantile regression functions. Specifically, the conditional class probability is formulated as a difference of corresponding cumulative distribution functions, where the cumulative distribution functions can be converted from the estimated conditional quantile regression functions. The proposed estimation method is also efficient as its computation cost does not increase exponentially with the number of classes. The theoretical and numerical studies demonstrate that the proposed estimation method is highly competitive against the existing competitors, especially when the number of classes is relatively large.     

Structured kernel quantile regression

Quantile regression can provide more useful information on the conditional distribution of a response variable given covariates while classical regression provides informations on the conditional mean alone. In this paper, we propose a structured quantile estimation methodology in a nonparametric function estimation setup. Through the functional analysis of variance decomposition, the optimization of the proposed method can be solved using a series of quadratic and linear programmings. Our method automatically selects relevant covariates by adopting a lasso-type penalty. The performance of the proposed methodology is illustrated through numerical examples on both simulated and real data.     

Bayesian Tobit quantile regression using g-prior distribution with ridge parameter

A Bayesian approach is proposed for coefficient estimation in the Tobit quantile regression model. The proposed approach is based on placing a g-prior distribution depends on the quantile level on the regression coefficients. The prior is generalized by introducing a ridge parameter to address important challenges that may arise with censored data, such as multicollinearity and overfitting problems. Then, a stochastic search variable selection approach is proposed for Tobit quantile regression model based on g-prior. An expression for the hyperparameter g is proposed to calibrate the modified g-prior with a ridge parameter to the corresponding g-prior. Some possible extensions of the proposed approach are discussed, including the continuous and binary responses in quantile regression. The methods are illustrated using several simulation studies and a microarray study. The simulation studies and the microarray study indicate that the proposed approach performs well.     

Integrative Analysis of Cancer Diagnosis Studies with Composite Penalization

In cancer diagnosis studies, high‐throughput gene profiling has been extensively conducted, searching for genes whose expressions may serve as markers. Data generated from such studies have the ‘large d, small n’ feature, with the number of genes profiled much larger than the sample size. Penalization has been extensively adopted for simultaneous estimation and marker selection. Because of small sample sizes, markers identified from the analysis of single data sets can be unsatisfactory. A cost‐effective remedy is to conduct integrative analysis of multiple heterogeneous data sets. In this article, we investigate composite penalization methods for estimation and marker selection in integrative analysis. The proposed methods use the minimax concave penalty (MCP) as the outer penalty. Under the homogeneity model, the ridge penalty is adopted as the inner penalty. Under the heterogeneity model, the Lasso penalty and MCP are adopted as the inner penalty. Effective computational algorithms based on coordinate descent are developed. Numerical studies, including simulation and analysis of practical cancer data sets, show satisfactory performance of the proposed methods.     

Shrinkage estimation of varying covariate effects based on quantile regression

Varying covariate effects often manifest meaningful heterogeneity in covariate-response associations. In this paper, we adopt a quantile regression model that assumes linearity at a continuous range of quantile levels as a tool to explore such data dynamics. The consideration of potential non-constancy of covariate effects necessitates a new perspective for variable selection, which, under the assumed quantile regression model, is to retain variables that have effects on all quantiles of interest as well as those that influence only part of quantiles considered. Current work on l ₁-penalized quantile regression either does not concern varying covariate effects or may not produce consistent variable selection in the presence of covariates with partial effects, a practical scenario of interest. In this work, we propose a shrinkage approach by adopting a novel uniform adaptive LASSO penalty. The new approach enjoys easy implementation without requiring smoothing. Moreover, it can consistently identify the true model (uniformly across quantiles) and achieve the oracle estimation efficiency. We further extend the proposed shrinkage method to the case where responses are subject to random right censoring. Numerical studies confirm the theoretical results and support the utility of our proposals.     

Nonparametric estimation of extreme conditional quantiles

The estimation of extreme conditional quantiles is an important issue in different scientific disciplines. Up to now, the extreme value literature focused mainly on estimation procedures based on independent and identically distributed samples. Our contribution is a two-step procedure for estimating extreme conditional quantiles. In a first step nonextreme conditional quantiles are estimated nonparametrically using a local version of [Koenker, R. and Bassett, G. (1978). Regression quantiles. Econometrica, 46, 33–50.] regression quantile methodology. Next, these nonparametric quantile estimates are used as analogues of univariate order statistics in procedures for extreme quantile estimation. The performance of the method is evaluated for both heavy tailed distributions and distributions with a finite right endpoint using a small sample simulation study. A bootstrap procedure is developed to guide in the selection of an optimal local bandwidth. Finally the procedure is illustrated in two case studies.     

Random Weighting Estimation for Quantile Processes and Negatively Associated Samples

This article presents new theories of random weighting estimation for quantile processes and negatively associated samples. Under the condition that X ₁, X ₂,…, X _n are independent random variables with a common distribution, the consistency for random weighting estimation of quantile processes is rigorously proved. When X ₁, X ₂,…, X _n are not independent of each other, random weighting estimation of sample mean is established for negatively associated samples.     

A Quantile Survival Model for Censored Data

In this paper we propose a quantile survival model to analyze censored data. This approach provides a very effective way to construct a proper model for the survival time conditional on some covariates. Once a quantile survival model for the censored data is established, the survival density, survival or hazard functions of the survival time can be obtained easily. For illustration purposes, we focus on a model that is based on the generalized lambda distribution (GLD). The GLD and many other quantile function models are defined only through their quantile functions, no closed‐form expressions are available for other equivalent functions. We also develop a Bayesian Markov Chain Monte Carlo (MCMC) method for parameter estimation. Extensive simulation studies have been conducted. Both simulation study and application results show that the proposed quantile survival models can be very useful in practice.     

Estimation of a probability density function using interval aggregated data

ABSTRACT

In economics and government statistics, aggregated data instead of individual level data are usually reported for data confidentiality and for simplicity. In this paper we develop a method of flexibly estimating the probability density function of the population using aggregated data obtained as group averages when individual level data are grouped according to quantile limits. The kernel density estimator has been commonly applied to such data without taking into account the data aggregation process and has been shown to perform poorly. Our method models the quantile function as an integral of the exponential of a spline function and deduces the density function from the quantile function. We match the aggregated data to their theoretical counterpart using least squares, and regularize the estimation by using the squared second derivatives of the density function as the penalty function. A computational algorithm is developed to implement the method. Application to simulated data and US household income survey data show that our penalized spline estimator can accurately recover the density function of the underlying population while the common use of kernel density estimation is severely biased. The method is applied to study the dynamic of China's urban income distribution using published interval aggregated data of 1985–2010.     

On the asymptotic non‐equivalence of efficient‐GMM and MEL estimators in models with missing data

The generalized method of moments (GMM) and empirical likelihood (EL) are popular methods for combining sample and auxiliary information. These methods are used in very diverse fields of research, where competing theories often suggest variables satisfying different moment conditions. Results in the literature have shown that the efficient‐GMM (GMM_E) and maximum empirical likelihood (MEL) estimators have the same asymptotic distribution to order n^?1/2 and that both estimators are asymptotically semiparametric efficient. In this paper, we demonstrate that when data are missing at random from the sample, the utilization of some well‐known missing‐data handling approaches proposed in the literature can yield GMM_E and MEL estimators with nonidentical properties; in particular, it is shown that the GMM_E estimator is semiparametric efficient under all the missing‐data handling approaches considered but that the MEL estimator is not always efficient. A thorough examination of the reason for the nonequivalence of the two estimators is presented. A particularly strong feature of our analysis is that we do not assume smoothness in the underlying moment conditions. Our results are thus relevant to situations involving nonsmooth estimating functions, including quantile and rank regressions, robust estimation, the estimation of receiver operating characteristic (ROC) curves, and so on.     

Goodness-of-fit testing of a weak memoryless property of life distributions

A life distribution is said to have a weak memoryless property if its conditional probability of survival beyond a fixed time point is equal to its (unconditional) survival probability at that point. Goodness‐of‐fit testing of this notion is proposed in the current investigation, both when the fixed time point is known and when it is unknown but estimable from the data. The limiting behaviour of the proposed test statistic is obtained and the null variance is explicitly given. The empirical power of the test is evaluated for a commonly known alternative using Monte Carlo methods, showing that the test performs well. The case when the fixed time point t₀ equals a quantile of the distribution F gives a distribution‐free test procedure. The procedure works even if t₀ is unknown but is estimable.     

TWO-STAGE SUPPORT ESTIMATION

This paper presents a two‐stage procedure for estimating the conditional support curve of a random variable X, given the information of a random vector X. Quantile estimation is followed by an extremal analysis on the residuals for problems which can be written as regression models. The technique is applied to data from the National Bureau of Economic Research and US Census Bureau's Center for Economic Studies which contain all four‐digit manufacturing industries. Simulation results show that in linear regression models the proposed estimation procedure is more efficient than the extreme linear regression quantile.     

On the strong Kotz approximation of Dirichlet random vectors

Let (X ₁, X ₂) be a bivariate L _p -norm generalized symmetrized Dirichlet (LpGSD) random vector with parameters α₁,α₂. If p=α₁=α₂=2, then (X ₁, X ₂) is a spherical random vector. The estimation of the conditional distribution of Z _u *:=X ₂ | X ₁>u for u large is of some interest in statistical applications. When (X ₁, X ₂) is a spherical random vector with associated random radius in the Gumbel max-domain of attraction, the distribution of Z _u * can be approximated by a Gaussian distribution. Surprisingly, the same Gaussian approximation holds also for Z _u :=X ₂| X ₁=u. In this paper, we are interested in conditional limit results in terms of convergence of the density functions considering a d-dimensional LpGSD random vector. Stating our results for the bivariate setup, we show that the density function of Z _u * and Z _u can be approximated by the density function of a Kotz type I LpGSD distribution, provided that the associated random radius has distribution function in the Gumbel max-domain of attraction. Further, we present two applications concerning the asymptotic behaviour of concomitants of order statistics of bivariate Dirichlet samples and the estimation of the conditional quantile function.     

Extreme Quantile Estimation for Autoregressive Models

ABSTRACT

A quantile autoregresive model is a useful extension of classical autoregresive models as it can capture the influences of conditioning variables on the location, scale, and shape of the response distribution. However, at the extreme tails, standard quantile autoregression estimator is often unstable due to data sparsity. In this article, assuming quantile autoregresive models, we develop a new estimator for extreme conditional quantiles of time series data based on extreme value theory. We build the connection between the second-order conditions for the autoregression coefficients and for the conditional quantile functions, and establish the asymptotic properties of the proposed estimator. The finite sample performance of the proposed method is illustrated through a simulation study and the analysis of U.S. retail gasoline price.     

Regularized Bayesian quantile regression

A number of nonstationary models have been developed to estimate extreme events as function of covariates. A quantile regression (QR) model is a statistical approach intended to estimate and conduct inference about the conditional quantile functions. In this article, we focus on the simultaneous variable selection and parameter estimation through penalized quantile regression. We conducted a comparison of regularized Quantile Regression model with B-Splines in Bayesian framework. Regularization is based on penalty and aims to favor parsimonious model, especially in the case of large dimension space. The prior distributions related to the penalties are detailed. Five penalties (Lasso, Ridge, SCAD0, SCAD1 and SCAD2) are considered with their equivalent expressions in Bayesian framework. The regularized quantile estimates are then compared to the maximum likelihood estimates with respect to the sample size. A Markov Chain Monte Carlo (MCMC) algorithms are developed for each hierarchical model to simulate the conditional posterior distribution of the quantiles. Results indicate that the SCAD0 and Lasso have the best performance for quantile estimation according to Relative Mean Biais (RMB) and the Relative Mean-Error (RME) criteria, especially in the case of heavy distributed errors. A case study of the annual maximum precipitation at Charlo, Eastern Canada, with the Pacific North Atlantic climate index as covariate is presented.