Rank estimation of regression coefficients using iterated reweighted least squares

This paper is concerned with the rank estimator for the parameter vector β in a linear model which is obtained by the minimization of a rank dispersion function. The rank estimator has many advantages over the regular least squares estimator, but the inaccessibility of software to carry out its computation has limited its use. An iterated reweighted least squares algorithm is presented for the computation of the rank estimator. The method is simple in concept and can be carried out readily with a wide variety of statistical software. Details of the method are discussed along with some results on its asymptotic distribution and numerical stability. Some examples are presented to show advantages of the rank method.     

Bounded influence nonlinear signed‐rank regression

In this paper we consider weighted generalized‐signed‐rank estimators of nonlinear regression coefficients. The generalization allows us to include popular estimators such as the least squares and least absolute deviations estimators but by itself does not give bounded influence estimators. Adding weights results in estimators with bounded influence function. We establish conditions needed for the consistency and asymptotic normality of the proposed estimator and discuss how weight functions can be chosen to achieve bounded influence function of the estimator. Real life examples and Monte Carlo simulation experiments demonstrate the robustness and efficiency of the proposed estimator. An example shows that the weighted signed‐rank estimator can be useful to detect outliers in nonlinear regression. The Canadian Journal of Statistics 40: 172–189; 2012 © 2012 Statistical Society of Canada     

Measuring the degree of severity of heteroskedasticity and the choice between the ols estimator and the 2sae

This paper dwells on the choice between the ordinary least squares and the estimated generalized least squares estimators when the presence of heteroskedasticity is suspected. Since the estimated generalized least squares estimator does not dominate the ordinary least squares estimator completely over the whole parameter space, it is of interest to the researcher to know in advance whether the degree of severity of heteroskedasticity is such that OLS estimator outperforms the estimated generalized least squares (or 2SAE). Casting the problem in the non-spherical error mold and exploiting the principle underlying the Bayesian pretest estimator, an intuitive non-mathematical procedure is proposed to serve as an aid to the researcher in deciding when to use either the ordinary least squares (OLS) or the estimated generalized least squares (2SAE) estimators.     

Seemingly unrelated regressions with covariance matrix of cross-equation ridge regression residuals

Generalized least squares estimation of a system of seemingly unrelated regressions is usually a two-stage method: (1) estimation of cross-equation covariance matrix from ordinary least squares residuals for transforming data, and (2) application of least squares on transformed data. In presence of multicollinearity problem, conventionally ridge regression is applied at stage 2. We investigate the usage of ridge residuals at stage 1, and show analytically that the covariance matrix based on the least squares residuals does not always result in more efficient estimator. A simulation study and an application to a system of firms' gross investment support our finding.     

On equality of ordinary least squares estimator,best linear unbiased estimator and best linear unbiased predictor in the general linear model

The equality of ordinary least squares estimator (OLSE), best linear unbiased estimator (BLUE) and best linear unbiased predictor (BLUP) in the general linear model with new observations is investigated through matrix rank method, some new necessary and sufficient conditions are given.     

A class of high-breakdown scale estimators based on subranges

We consider a new class of scale estimators with 50% breakdown point. The estimators are defined as order statistics of certain subranges. They all have a finite-sample breakdown point of [n/2]/n, which is the best possible value. (Here, [...] denotes the integer part.) One estimator in this class has the same influence function as the median absolute deviation and the least median of squares (LMS) scale estimator (i.e., the length of the shortest half), but its finite-sample efficiency is higher. If we consider the standard deviation of a subsample instead of its range, we obtain a different class of 50% breakdown estimators. This class contains the least trimmed squares (LTS) scale estimator. Simulation shows that the LTS scale estimator is nearly unbiased, so it does not need a small-sample correction factor. Surprisingly, the efficiency of the LTS scale estimator is less than that of the LMS scale estimator.     

The density function and the MSE dominance of the pre-test estimator in a heteroscedastic linear regression model with omitted variables

In this paper, we consider a heteroscedastic linear regression model with omitted variables. We derive the density function of the pre-test estimator consisting of the two-stage Aitken estimator (2SAE) and the ordinary least squares estimator (OLSE) after the pre-test for homoscedasticity. We also derive the first two moments based on the density function and show the sufficient condition for the pre-test estimator to dominate the 2SAE in terms of the MSE. Our numerical evaluations show that when this sufficient condition does not hold and when the magnitude of the specification error is large, the pre-test estimator can be dominated by the 2SAE, and further, the 2SAE can be dominated by the OLSE.     

Least Trimmed Squares Estimator in the Errors-in-Variables Model

We propose a robust estimator in the errors-in-variables model using the least trimmed squares estimator. We call this estimator the orthogonal least trimmed squares (OLTS) estimator. We show that the OLTS estimator has the high breakdown point and appropriate equivariance properties. We develop an algorithm for the OLTS estimate. Simulations are performed to compare the efficiencies of the OLTS estimates with the total least squares (TLS) estimates and a numerical example is given to illustrate the effectiveness of the estimate.     

Robust Sparse Regression with High-Breakdown Value

Penalized least squares estimators are sensitive to the influence of outliers like the ordinary least squares estimator. We propose a sparse regression estimator for robust variable selection and estimation based on a robust initial estimator. It is proven that our estimator has at least the same breakdown value as the initial estimator. Numerical examples are presented to illustrate our method.     

Score estimation in the monotone single‐index model

We consider estimation in the single‐index model where the link function is monotone. For this model, a profile least‐squares estimator has been proposed to estimate the unknown link function and index. Although it is natural to propose this procedure, it is still unknown whether it produces index estimates that converge at the parametric rate. We show that this holds if we solve a score equation corresponding to this least‐squares problem. Using a Lagrangian formulation, we show how one can solve this score equation without any reparametrization. This makes it easy to solve the score equations in high dimensions. We also compare our method with the effective dimension reduction and the penalized least‐squares estimator methods, both available on CRAN as R packages, and compare with link‐free methods, where the covariates are elliptically symmetric.     

The Statistical Properties of the Equity Estimator

In this article we consider the Equity estimator proposed by Krishnamurthi and Rangaswamy. We show that this estimator is inconsistent and does not necessarily improve on the mean squared error (MSE) of the least squares (LS) estimator. We perform a Monte Carlo experiment based on the price-promotion model used in marketing research, with marketing data, comparing the MSE of the Equity estimator to that of two empirical Bayes estimators and the LS estimator. We find that the empirical Bayes estimators have substantially smaller MSE than the Equity estimator in almost every case.     

Estimating the Intercept in an Orthogonally Blocked Experiment when the Block Effects are Random

Abstract

For an orthogonally blocked experiment, Khuri [Khuri, A. I. (1992). Response surface models with random block effects. Technometrics 34:26–37] has shown that the ordinary least squares estimator, the generalized least squares estimator and the intra-block estimator of the factor effects in a response surface model with random block effects coincide. The ordinary least squares estimator ignores the blocks, whereas the generalized least squares and the intra-block estimators treat the block effects as random and fixed, respectively. As shown in this paper, the equivalence does not hold for the estimation of the intercept when the block sizes are heterogeneous. Practical examples are given to illustrate the theoretical results.     

Mean square error and efficiency of the least squares estimator over interval constraints

We derive the mean square error of an interval constrained least squares estimator (INCLS) for a regression model. We then show that the INCLS estimator dominates, in mean square error, the unconstrained least squares estimator provided the regression residuais are normally distri'iiuted and Ynat Yrie imposed coii-

straint is satisfied or nearly satisfied.     

Fundamental Equations of BLUE and BLUP in the Multivariate Linear Model with Applications

Ordinary least squares estimator (OLSE), best linear unbiased estimator (BLUE), and best linear unbiased predictor (BLUP) in the general linear model with new observations are generalized to the general multivariate linear model. The fundamental equations of BLUE and BLUP in the multivariate linear model are derived by two methods, including the vectorization method and projection method. By using the matrix rank method, some new results of linear BLUE-sufficiency, linear BLUP-sufficiency, and the equality of OLSE, BLUE, and BLUP are given in the multivariate linear model.     

Generalised Rank Regression Estimator with Standard Error Adjusted Lasso

One of the standard variable selection procedures in multiple linear regression is to use a penalisation technique in least‐squares (LS) analysis. In this setting, many different types of penalties have been introduced to achieve variable selection. It is well known that LS analysis is sensitive to outliers, and consequently outliers can present serious problems for the classical variable selection procedures. Since rank‐based procedures have desirable robustness properties compared to LS procedures, we propose a rank‐based adaptive lasso‐type penalised regression estimator and a corresponding variable selection procedure for linear regression models. The proposed estimator and variable selection procedure are robust against outliers in both response and predictor space. Furthermore, since rank regression can yield unstable estimators in the presence of multicollinearity, in order to provide inference that is robust against multicollinearity, we adjust the penalty term in the adaptive lasso function by incorporating the standard errors of the rank estimator. The theoretical properties of the proposed procedures are established and their performances are investigated by means of simulations. Finally, the estimator and variable selection procedure are applied to the Plasma Beta‐Carotene Level data set.     

Asymptotic robustness of least median of squares for autoregressions with additive outliers

This paper considers the robustness properties in the time series context of the least median of squares (LMS) estimator. The influence function of the LMS estimator is derived under additive outlier contamination. This influence function is redescending and bounded for fixed values of the AR parameters. The gross-error sensitivity, however, is an unbounded function of the AR parameters. In order to asses the global robustness behavior of the LMS estimator, we consider several notions of breakdown. The breakdown points of the LMS estimator depend on the value of the underlying AR parameter. Generally, the breakdown point is below one half for high values of the AR parameter. The bias curves of the LMS estimator reveal, however, that the magnitude of outliers has to be considerable in order to cause breakdown.     

The influence function of penalized regression estimators

To perform regression analysis in high dimensions, lasso or ridge estimation are a common choice. However, it has been shown that these methods are not robust to outliers. Therefore, alternatives as penalized M-estimation or the sparse least trimmed squares (LTS) estimator have been proposed. The robustness of these regression methods can be measured with the influence function. It quantifies the effect of infinitesimal perturbations in the data. Furthermore, it can be used to compute the asymptotic variance and the mean-squared error (MSE). In this paper we compute the influence function, the asymptotic variance and the MSE for penalized M-estimators and the sparse LTS estimator. The asymptotic biasedness of the estimators make the calculations non-standard. We show that only M-estimators with a loss function with a bounded derivative are robust against regression outliers. In particular, the lasso has an unbounded influence function.     

A likelihood based estimator for vector autoregressive processes

A onestep estimator, which is an approximation to the unconditional maximum likelihood estimator (MLE) of the coefficient matrices of a Gaussian vector autoregressive process is presented. The onestep estimator is easy to compute and can be computed using standard software. Unlike the computation of the unconditional MLE, the computation of the onestep estimator does not require any iterative optimization and the computation is numerically stable. In finite samples the onestep estimator generally has smaller mean square error than the ordinary least squares estimator. In a simple model, where the unconditional MLE can be computed, numerical investigation shows that the onestep estimator is slightly worse than the unconditional MLE in terms of mean square error but superior to the ordinary least squares estimator. The limiting distribution of the onestep estimator for processes with some unit roots is derived.     

Some finite sample properties of generalized ridge regression estimators

In this paper we study the mean square error properties of the generalized ridge estimator. We obtain the exact and the approximate bias and the mean square error of the operational generalized ridge estimator in terms of G( ) functions. We show, among other things, that the operational generalized ridge estimator does not dominate the ordinary least squares estimator up to a certain order of approximation. Finally, we note that the iterative procedures to obtain coverging ridge estimators should be used with caution.     

Estimating partially linear panel data models with one-way error components

We consider the problem of estimating a partially linear panel data model whenthe error follows an one-way error components structure. We propose a feasiblesemiparametric generalized least squares (GLS) type estimator for estimating the coefficient of the linear component and show that it is asymptotically more efficient than a semiparametric ordinary least squares (OLS) type estimator. We also discussed the case when the regressor of the parametric component is correlated with the error, and propose an instrumental variable GLS-type semiparametric estimator.