Unbiased L1 and L∞ estimation

Sielken and Heartely 1973 have shown that the L₁ and L_∞ estimation problems may be formulated in such a way as to yield unbiased estimators of in the standard linear model y = Xβ + ε In this paper we will show that the L₁ estimation problem is closely related to the dual of the L_∞ estimation problem and vice versa. We will use this resu;t to obtain four fistiner lineat programming problems which yield unbiased L₁ and L_∞ estimators of β.     

A note on penalized minimum distance estimation in nonparametric regression

The authors introduce a penalized minimum distance regression estimator. They show the estimator to balance, among a sequence of nested models of increasing complexity, the L¹ ‐approximation error of each model class and a penalty term which reflects the richness of each model and serves as a upper bound for the estimation error.     

Standard and robust orthogonal regression

A fast routine for converting regression algorithms into corresponding orthogonal regression (OR) algorithms was introduced in Ammann and Van Ness (1988). The present paper discusses the properties of various ordinary and robust OR procedures created using this routine. OR minimizes the sum of the orthogonal distances from the regression plane to the data points. OR has three types of applications. First, L ₂ OR is the maximum likelihood solution of the Gaussian errors-in-variables (EV) regression problem. This L ₂ solution is unstable, thus the robust OR algorithms created from robust regression algorithms should prove very useful. Secondly, OR is intimately related to principal components analysis. Therefore, the routine can also be used to create L ₁, robust, etc. principal components algorithms. Thirdly, OR treats the x and y variables symmetrically which is important in many modeling problems. Using Monte Carlo studies this paper compares the performance of standard regression, robust regression, OR, and robust OR on Gaussian EV data, contaminated Gaussian EV data, heavy-tailed EV data, and contaminated heavy-tailed EV data.     

Comparison of computer programs for simple linear L 1 regression

A number of efficient computer codes are available for the simple linear L ₁ regression problem. However, a number of these codes can be made more efficient by utilizing the least squares solution. In fact, a couple of available computer programs already do so.

We report the results of a computational study comparing several openly available computer programs for solving the simple linear L ₁ regression problem with and without computing and utilizing a least squares solution.     

Penalized regression with model‐based penalties

Nonparametric regression techniques such as spline smoothing and local fitting depend implicitly on a parametric model. For instance, the cubic smoothing spline estimate of a regression function ∫ μ based on observations ti, Yi is the minimizer of Σ{Yi ‐ μ(ti)}² + λ∫(μ′′)². Since ∫(μ″)² is zero when μ is a line, the cubic smoothing spline estimate favors the parametric model μ(t) = α_o + α₁t. Here the authors consider replacing ∫(μ″)² with the more general expression ∫(Lμ)² where L is a linear differential operator with possibly nonconstant coefficients. The resulting estimate of μ performs well, particularly if Lμ is small. They present an O(n) algorithm for the computation of μ. This algorithm is applicable to a wide class of L's. They also suggest a method for the estimation of L. They study their estimates via simulation and apply them to several data sets.     

Bootstrap test of goodness of fit to a linear model when errors are correlated

Given the regression model Y_i = m(x_i) +ε_i (x_i ε C, i = l,…,n, C a compact set in R) where m is unknown and the random errors {ε_i} present an ARMA structure, we design a bootstrap method for testing the hypothesis that the regression function follows a general linear model: H_o : m ε {mθ(.) = A^t(.)θ : θ ε ? ? R^q} with A a functional from R to R^q. The criterion of the test derives from a Cramer-von-Mises type functional distance D = d²([mcirc]_n, A^t(.)θ_n), between [mcirc]_n, a Gasser-Miiller non-parametric estimator of m, and the member of the class defined in H_o that is closest to m_n in terms of this distance. The consistency of the bootstrap distribution of D and θ_n is obtained under general conditions. Finally, simulations show the good behavior of the bootstrap approximation with respect to the asymptotic distribution of D = d².     

On the L1‐consistency of wavelet density estimates

The authors analyze the L₁ performance of wavelet density estimators. They prove that under mild conditions on the family of wavelets, such estimates are universally consistent in the L₁ sense.     

Quasi-universal bandwidth selection for kernel density estimators

Let f?_n, h denote the kernel density estimate based on a sample of size n drawn from an unknown density f. Using techniques from L₂ projection density estimators, the author shows how to construct a data-driven estimator f?_n, h which satisfies This paper is inspired by work of Stone (1984), Devroye and Lugosi (1996) and Birge and Massart (1997).     

Adaptive penalized quantile regression for high dimensional data

We propose a new adaptive L₁ penalized quantile regression estimator for high-dimensional sparse regression models with heterogeneous error sequences. We show that under weaker conditions compared with alternative procedures, the adaptive L₁ quantile regression selects the true underlying model with probability converging to one, and the unique estimates of nonzero coefficients it provides have the same asymptotic normal distribution as the quantile estimator which uses only the covariates with non-zero impact on the response. Thus, the adaptive L₁ quantile regression enjoys oracle properties. We propose a completely data driven choice of the penalty level _λn${\lambda }_n$, which ensures good performance of the adaptive L₁ quantile regression. Extensive Monte Carlo simulation studies have been conducted to demonstrate the finite sample performance of the proposed method.     

On the admissibility of c[Xbar] + d with respect to the linex loss function

Let X₁, …,X_n be a random sample from a normal distribution with mean θ and variance σ² The problem is to estimate θ with loss function L(θ,e) = v(e-θ) where v(x) = b(exp(ax)-ax-l) and where a, b are constants with b>0, a¦0. Zellner (1986), showed that [Xbar] ? σ²a/2n dominates [Xbar] and hence [Xbar] is inadmissible. The question of what values of c and d render c[Xbar]+ d admissible is studied here.     

On the Usefulness or Lack Thereof of Optimality Criteria for Structural Change Tests

Elliott and Müller (2006) considered the problem of testing for general types of parameter variations, including infrequent breaks. They developed a framework that yields optimal tests, in the sense that they nearly attain some local Gaussian power envelop. The main ingredient in their setup is that the variance of the process generating the changes in the parameters must go to zero at a fast rate. They recommended the so-called qL?L test, a partial sums type test based on the residuals obtained from the restricted model. We show that for breaks that are very small, its power is indeed higher than other tests, including the popular sup-Wald (SW) test. However, the differences are very minor. When the magnitude of change is moderate to large, the power of the test is very low in the context of a regression with lagged dependent variables or when a correction is applied to account for serial correlation in the errors. In many cases, the power goes to zero as the magnitude of change increases. The power of the SW test does not show this non-monotonicity and its power is far superior to the qL?L test when the break is not very small. We claim that the optimality of the qL?L test does not come from the properties of the test statistics but the criterion adopted, which is not useful to analyze structural change tests. Instead, we use fixed-break size asymptotic approximations to assess the relative efficiency or power of the two tests. When doing so, it is shown that the SW test indeed dominates the qL?L test and, in many cases, the latter has zero relative asymptotic efficiency.     

Robust estimation of the SUR model

This paper proposes robust regression to solve the problem of outliers in seemingly unrelated regression (SUR) models. The authors present an adaptation of S‐estimators to SUR models. S‐estimators are robust, have a high breakdown point and are much more efficient than other robust regression estimators commonly used in practice. Furthermore, modifications to Ruppert's algorithm allow a fast evaluation of them in this context. The classical example of U.S. corporations is revisited, and it appears that the procedure gives an interesting insight into the problem.     

De l'unicité des estimateurs robustes en régression lorsque le paramètre d'échelle et le paramètre de la régression sont estimés simultanément

Several methods have been suggested to calculate robust M- and G-M -estimators of the regression parameter β and of the error scale parameter σ in a linear model. This paper shows that, for some data sets well known in robust statistics, the nonlinear systems of equations for the simultaneous estimation of β, with an M-estimate with a redescending ψ-function, and σ, with the residual median absolute deviation (MAD), have many solutions. This multiplicity is not caused by the possible lack of uniqueness, for redescending ψ-functions, of the solutions of the system defining β with known σ; rather, the simultaneous estimation of β and σ together creates the problem. A way to avoid these multiple solutions is to proceed in two steps. First take σ as the median absolute deviation of the residuals for a uniquely defined robust M-estimate such as Huber's Proposal 2 or the L₁-estimate. Then solve the nonlinear system for the M-estimate with σ equal to the value obtained at the first step to get the estimate of β. Analytical conditions for the uniqueness of M and G-M-estimates are also given.     

L1 for the simple linear regression model

By modifying the direct method to solve the overdetermined linear system we are able to present an algorithm for L₁ estimation which appears to be superior computationally to any other known algorithm for the simple linear regression problem.     

Influence measure for the L1 regression

Because outliers and leverage observations unduly affect the least squares regression, the identification of influential observations is considered an important and integrai part of the analysis. However, very few techniques have been developed for the residual analysis and diagnostics for the minimum sum of absolute errors, L₁ regression. Although the L₁ regression is more resistant to the outliers than the least squares regression, it appears that outliers (leverage) in the predictor variables may affect it. In this paper, our objective is to develop an influence measure for the L₁ regression based on the likelihood displacement function. We illustrate the proposed influence measure with examples.     

Improved minimax estimation of a normal precision matrix

Let S_{p × p} have a Wishart distribution with parameter matrix Σ and n degrees of freedom. We consider here the problem of estimating the precision matrix Σ^?1 under the loss functions L₁(σ) tr (σ) - log |σ| and L₂(σ) = tr (σ). James-Stein-type estimators have been derived for an arbitrary p. We also obtain an orthogonal invariant and a diagonal invariant minimax estimator under both loss functions. A Monte-Carlo simulation study indicates that the risk improvement of the orthogonal invariant estimators over the James-Stein type estimators, the Haff (1979) estimator, and the “testimator” given by Sinha and Ghosh (1987) is substantial.     

PLUG-IN ESTIMATION OF GENERAL LEVEL SETS

Given an unknown function (e.g. a probability density, a regression function, …) f and a constant c, the problem of estimating the level set L(c) ={f≥c} is considered. This problem is tackled in a very general framework, which allows f to be defined on a metric space different from . Such a degree of generality is motivated by practical considerations and, in fact, an example with astronomical data is analyzed where the domain of f is the unit sphere. A plug‐in approach is followed; that is, L(c) is estimated by L_n(c) ={f_n≥c} , where f_n is an estimator of f. Two results are obtained concerning consistency and convergence rates, with respect to the Hausdorff metric, of the boundaries ?L_n(c) towards ?L(c) . Also, the consistency of L_n(c) to L(c) is shown, under mild conditions, with respect to the L₁ distance. Special attention is paid to the particular case of spherical data.     

Robust Coordinate Descent Algorithm Robust Solution Path for High-dimensional Sparse Regression Modeling

The L₁-type regularization provides a useful tool for variable selection in high-dimensional regression modeling. Various algorithms have been proposed to solve optimization problems for L₁-type regularization. Especially the coordinate descent algorithm has been shown to be effective in sparse regression modeling. Although the algorithm shows a remarkable performance to solve optimization problems for L₁-type regularization, it suffers from outliers, since the procedure is based on the inner product of predictor variables and partial residuals obtained from a non-robust manner. To overcome this drawback, we propose a robust coordinate descent algorithm, especially focusing on the high-dimensional regression modeling based on the principal components space. We show that the proposed robust algorithm converges to the minimum value of its objective function. Monte Carlo experiments and real data analysis are conducted to examine the efficiency of the proposed robust algorithm. We observe that our robust coordinate descent algorithm effectively performs for the high-dimensional regression modeling even in the presence of outliers.     

On nonparametric regression estimators based on regression quantiles

In the ciassical regression model Y_i=h(x_i) + ? _i, i=1,…,n, Cheng (1984) introduced linear combinations of regression quantiles as a new class of estimators for the unknown regression function h(x). The asymptotic properties studied in Cheng (1984) are reconsidered. We obtain a sharper scrong consistency rate and we improve on the conditions for asymptotic normality by proving a new result on the remainder term in the Bahadur representation for regression quantiles.     

Marginally restricted sequential D‐optimal designs

In many experiments, not all explanatory variables can be controlled. When the units arise sequentially, different approaches may be used. The authors study a natural sequential procedure for “marginally restricted” D‐optimal designs. They assume that one set of explanatory variables (x₁) is observed sequentially, and that the experimenter responds by choosing an appropriate value of the explanatory variable x₂. In order to solve the sequential problem a priori, the authors consider the problem of constructing optimal designs with a prior marginal distribution for x₁. This eliminates the influence of units already observed on the next unit to be designed. They give explicit designs for various cases in which the mean response follows a linear regression model; they also consider a case study with a nonlinear logistic response. They find that the optimal strategy often consists of randomizing the assignment of the values of x₂.