Developing ridge estimation method for median regression

In this paper, the ridge estimation method is generalized to the median regression. Though the least absolute deviation (LAD) estimation method is robust in the presence of non-Gaussian or asymmetric error terms, it can still deteriorate into a severe multicollinearity problem when non-orthogonal explanatory variables are involved. The proposed method increases the efficiency of the LAD estimators by reducing the variance inflation and giving more room for the bias to get a smaller mean squared error of the LAD estimators. This paper includes an application of the new methodology and a simulation study as well.     

Least absolute value estimation in regression models: an annotated bibliography

This paper presents a comprehensive listing of articles on least absolute value (LAV) estimation as applied to linear and non-linear regression models and in systems of equations. References to the LAV method as applied in approximation theory are also included. Annotations describing the content of each article follow each reference.     

Nonparametric M-regression with free knot splines

Many problems of practical interest can be formulated as the nonparametric estimation of a certain function such as a regression function, logistic or other generalized regression function, density function, conditional density function, hazard function, or conditional hazard function. Extended linear modeling provides a convenient theoretical framework for using polynomial splines and their selected tensor products in such function estimation problems and especially for obtaining rates of convergence of the resulting estimates in a unified manner. For a long time the theoretical results were restricted to fixed knot splines and to log-likelihood functions that were twice continuously differentiable. Recently, Stone and Huang extended the theory to handle free knot splines. In the present paper, the theory is further extended to handle contexts in which the log-likelihood function may not be differentiable. Specifically, we establish rates of convergence for estimation based on free knot splines in the context of nonparametric regression corresponding to M-estimates, which includes least absolute deviations (LAD) regression, quantile regression, and robust regression as special cases.     

Least absolute value regression: a special case of piecewise linear minimization

The Barrodale and Roberts algorithm for least absolute value (LAV) regression and the algorithm proposed by Bartels and Conn both have the advantage that they are often able to skip across points at which the conventional simplex-method algorithms for LAV regression would be required to carry out an (expensive) pivot operation.

We indicate here that this advantage holds in the Bartels-Conn approach for a wider class of problems: the minimization of piecewise linear functions. We show how LAV regression, restricted LAV regression, general linear programming and least maximum absolute value regression can all be easily expressed as piecewise linear minimization problems.     

Monte carlo comparison of estimation methods for additive two-way tables

We considered the problem of estimating effects in the following linear model for data arranged in a two-way table: Response = Common effect + Row effect + Column effect + Residual. This work was occasioned by a project to analyse Federal Aviation Administration (FAA) data on daily temporal deviations from flight plans for commercial US flights, with rows and columns representing origin and destination airports, respectively. We conducted a large Monte Carlo study comparing the accuracy of three methods of estimation: classical least squares, median polish and least absolute deviations (LAD). The experiments included a wide spectrum of tables of different sizes and shapes, with different levels of non-linearity, noise variance, and percentages of empty cells and outliers. We based our comparison on the accuracy of the estimates and on computational speed. We identified factors that significantly affect accuracy and speed, and compared the methods based on their sensitivity to these factors. We concluded that there is no dominant method of estimation and identified conditions under which each method is most attractive.     

Algorithms for restricted least absolute value estimation

We present a concise summary of recent progress in developing algorithms for restricted least absolute value (LAV) estimation (i. e. ?₁ approximation subject to linear constraints). The emphasis is on our own new algorithm, and we provide some numerical results obtained with it.     

Complexity of penalized likelihood estimation

We show that for a class of penalty functions, finding the global optimizer in the penalized least-squares estimation is equivalent to the ‘exact cover by 3-sets’ problem, which belongs to a class of NP-hard problems. The NP-hardness result is then extended to the cases of penalized least absolute deviations regression and a special class of penalized support vector machines. We discuss its implication in statistics. To the best of our knowledge, this is the first formal documentation on the complexity of this type of problem.     

Useful generalized properties of L1 estimators

Recent results by G. Appa and C. Smith, as well as I. Barrodale and F. D. K. Roberts, underscore several properties exhibited for fitting a linear model to a set of observation points under the criterion of least sum of absolute deviations(commonly denoted as the L₁ criterion). This paper will generalize these properties to the non-full rank case and relax in a natural way some assumptions given by Appa and Smith.     

The jackknife procedure for the probabilities of misclassification

Seven estimators for the probabilities of misclassifi-cation associated with the linear discriminant function are considered. Four of them are known in the literature. The remaining three are constructed through the Jackknife Pro-cedure. An empirical investigation is conducted to evalu-ate the relative merits of these estimators. Summary of the results is presented.     

Weighted Median of the Data in Solving Least Absolute Deviations Problems

We consider the weighted median problem for a given set of data and analyze its main properties. As an illustration, an efficient method for searching for a weighted Least Absolute Deviations (LAD)-line is given, which is used as the basis for solving various linear and nonlinear LAD-problems occurring in applications. Our method is illustrated by an example of hourly natural gas consumption forecast.     

On estimation of the PDF and CDF of the Lindley distribution

This article addresses two methods of estimation of the probability density function (PDF) and cumulative distribution function (CDF) for the Lindley distribution. Following estimation methods are considered: uniformly minimum variance unbiased estimator (UMVUE) and maximum likelihood estimator (MLE). Since the Lindley distribution is more flexible than the exponential distribution, the same estimators have been found out for the exponential distribution and compared. Monte Carlo simulations and a real data analysis are performed to compare the performances of the proposed methods of estimation.     

Conditional median absolute deviation

We introduce conditional median absolute deviation to characterize how the local variability of one quantitative random variable varies with another one. A two-step estimation procedure is proposed and the resultant estimator possesses an adaptiveness property. Simulation indicates that this estimator is much more efficient than its competitors such as the conditional semi-interquartile range.     

The effect of temporal aggregation on the estimation accuracy of ARMA models

ABSTRACT

Autoregressive Moving Average (ARMA) time series model fitting is a procedure often based on aggregate data, where parameter estimation plays a key role. Therefore, we analyze the effect of temporal aggregation on the accuracy of parameter estimation of mixed ARMA and MA models. We derive the expressions required to compute the parameter values of the aggregate models as functions of the basic model parameters in order to compare their estimation accuracy. To this end, a simulation experiment shows that aggregation causes a severe accuracy loss that increases with the order of aggregation, leading to poor accuracy.     

Best minimum bias linear estimators in gauss-markoff model

In this note we present a criterion for linear estimation which is similar to MV-MB-LE of Rao (1978) in Gauss-Markoff model (Y, XB, α²G). We call this criterion MMS-MB-LE (Minimum Mean Square Error-Minimum Bias-Linear Estimation)> Representations of solutions to such estimators similar to those of Rao (1978) are provided.     

Improved autoregressive forecasts in the presence of non-normal errors

This paper is concerned with obtaining more accurate point forecasts in the presence of non-normal errors. Specifically, we apply the residual augmented least-squares (RALS) estimator to autoregressive models to utilize the additional moment restrictions embodied in non-normal errors. Monte Carlo experiments are performed to compare our RALS forecasts to forecasts based on the ordinary least-squares estimator and the least absolute deviations (LAD) estimator. We find that the RALS approach provides superior forecasts when the data are skewed. Compared to the LAD forecast, the RALS forecast has smaller mean squared prediction errors in the baseline case with normal errors.     

Simultaneous estimation for semi-parametric multi-index models

Estimation of a general multi-index model comprises determining the number of linear combinations of predictors (structural dimension) that are related to the response, estimating the loadings of each index vector, selecting the active predictors and estimating the underlying link function. These objectives are often achieved sequentially at different stages of the estimation process. In this study, we propose a unified estimation approach under a semi-parametric model framework to attain these estimation goals simultaneously. The proposed estimation method is more efficient and stable than many existing methods where the estimation error in the structural dimension may propagate to the estimation of the index vectors and variable selection stages. A detailed algorithm is provided to implement the proposed method. Comprehensive simulations and a real data analysis illustrate the effectiveness of the proposed method.     

Estimating coefficients of single-index regression models by minimizing variation

This paper proposes a novel estimation of coefficients in single-index regression models. Unlike the traditional average derivative estimation [Powell JL, Stock JH, Stoker TM. Semiparametric estimation of index coefficients. Econometrica. 1989;57(6):1403–1430; Hardle W, Thomas M. Investigating smooth multiple regression by the method of average derivatives. J Amer Statist Assoc. 1989;84(408):986–995] and semiparametric least squares estimation [Ichimura H. Semiparametric least squares (sls) and weighted sls estimation of single-index models. J Econometrics. 1993;58(1):71–120; Hardle W, Hall P, Ichimura H. Optimal smoothing in single-index models. Ann Statist. 1993;21(1):157–178], the procedure developed in this paper is to estimate the coefficients directly by minimizing the mean variation function and does not involve estimating the link function nonparametrically. As a result, it avoids the selection of the bandwidth or the number of knots, and its implementation is more robust and easier. The resultant estimator is shown to be consistent. Numerical results and real data analysis also show that the proposed procedure is more applicable against model free assumptions.     

Robustness of confidence intervals for scale parameters based on m-estimators

The robustness of confidence intervals for a scale parameter based on M-esimators is studied, especially in small size samples. The coverage probablity is used as measure of robustness. A theorem for a lower bound of the minimum coverage probability of M-estimators is presented and it is applied in order to examine the behavior of the standard deviation and the median absolute deviation, as interval estimators. This bound can confirm the robustness of any other scale M-estimator in interval estimation. The idea of stretching is used to formulate the family of distributions that are considered as underlying. Critical values for the confidence interval are computed where it is needed, that is for the median absolute deviation in the Normal, Uniform and Cauchy distribution and for the standard deviation in the Uniform and Cauchy distribution. Simulation results have been achieved for the estimation of the coverage probabilities and the critical values.     

Estimation procedures for grouped data – a comparative study

Interval-censored data are very common in the reliability and lifetime data analysis. This paper investigates the performance of different estimation procedures for a special type of interval-censored data, i.e. grouped data, from three widely used lifetime distributions. The approaches considered here include the maximum likelihood estimation, the minimum distance estimation based on chi-square criterion, the moment estimation based on imputation (IM) method and an ad hoc estimation procedure. Although IM-based techniques are extensively used recently, we show that this method is not always effective. It is found that the ad hoc estimation procedure is equivalent to the minimum distance estimation with another distance metric and more effective in the simulation. The procedures of different approaches are presented and their performances are investigated by Monte Carlo simulation for various combinations of sample sizes and parameter settings. The numerical results provide guidelines to analyse grouped data for practitioners when they need to choose a good estimation approach.