Bounded-leverage weights for robust regression estimators

Both the least squares estimator and M-estimators of regression coefficients are susceptible to distortion when high leverage points occur among the predictor variables in a multiple linear regression model. In this article a weighting scheme which enables one to bound the leverage values of a weighted matrix of predictor variables is proposed. Bounded-leverage weighting of the predictor variables followed by M-estimation of the regression coefficients is shown to be effective in protecting against distortion due to extreme predictor-variable values, extreme response values, or outlier-induced multieollinearites. Bounded-leverage estimators can also protect against distortion by small groups of high leverage points.     

Identification of multiple influential observations in logistic regression

The identification of influential observations in logistic regression has drawn a great deal of attention in recent years. Most of the available techniques like Cook's distance and difference of fits (DFFITS) are based on single-case deletion. But there is evidence that these techniques suffer from masking and swamping problems and consequently fail to detect multiple influential observations. In this paper, we have developed a new measure for the identification of multiple influential observations in logistic regression based on a generalized version of DFFITS. The advantage of the proposed method is then investigated through several well-referred data sets and a simulation study.     

On the asymptotic distribution of Cook's distance in logistic regression models

It sometimes occurs that one or more components of the data exert a disproportionate influence on the model estimation. We need a reliable tool for identifying such troublesome cases in order to decide either eliminate from the sample, when the data collect was badly realized, or otherwise take care on the use of the model because the results could be affected by such components. Since a measure for detecting influential cases in linear regression setting was proposed by Cook [Detection of influential observations in linear regression, Technometrics 19 (1977), pp. 15–18.], apart from the same measure for other models, several new measures have been suggested as single-case diagnostics. For most of them some cutoff values have been recommended (see [D.A. Belsley, E. Kuh, and R.E. Welsch, Regression Diagnostics: Identifying Influential Data and Sources of Collinearity, 2nd ed., John Wiley & Sons, New York, Chichester, Brisban, (2004).], for instance), however the lack of a quantile type cutoff for Cook's statistics has induced the analyst to deal only with index plots as worthy diagnostic tools. Focussed on logistic regression, the aim of this paper is to provide the asymptotic distribution of Cook's distance in order to look for a meaningful cutoff point for detecting influential and leverage observations.     

Least absolute values estimation: an introduction

A brief review and bibliography of least absolute values (LAV) estimation is given. This paper serves to introduce the other articles in this special issue on the computational aspects of LAV estimation.     

Diagnostics for non-linear regression

Sensitivity analysis in regression is concerned with assessing the sensitivity of the results of a regression model (e.g., the objective function, the regression parameters, and the fitted values) to changes in the data. Sensitivity analysis in least squares linear regression has seen a great surge of research activities over the last three decades. By contrast, sensitivity analysis in non-linear regression has received very little attention. This paper deals with the problem of local sensitivity analysis in non-linear regression. Closed-form general formulas are provided for the sensitivities of three standard methods for the estimation of the parameters of a non-linear regression model based on a set of data. These methods are the least squares, the minimax, and the least absolute value methods. The effectiveness of the proposed measures is illustrated by application to several non-linear models including the ultrasonic data and the onion yield data. The proposed sensitivity measures are shown to deal effectively with the detection of influential observations in non-linear regression models.     

Coefficients of determinations for variable selection in the msae regression

Our objective is to modify a robust coefficient of determination for the minimum sum of absolute errors MSAE regression proposed by McKean and Sievers (1987) so that it satisfies all the desirable properties. We also propose an adjusted coefficient of determination that is appropriate for comparing several models with different number of variables. Further, it has the property that if it decreases with the addition of predictor variables to the model, then the contribution of these variables is statistically non-significant. We illustrate the results with an example.     

L1 for the simple linear regression model

By modifying the direct method to solve the overdetermined linear system we are able to present an algorithm for L₁ estimation which appears to be superior computationally to any other known algorithm for the simple linear regression problem.     

Robust estimation for ordinal regression

Ordinal regression is used for modelling an ordinal response variable as a function of some explanatory variables. The classical technique for estimating the unknown parameters of this model is Maximum Likelihood (ML). The lack of robustness of this estimator is formally shown by deriving its breakdown point and its influence function. To robustify the procedure, a weighting step is added to the Maximum Likelihood estimator, yielding an estimator with bounded influence function. We also show that the loss in efficiency due to the weighting step remains limited. A diagnostic plot based on the Weighted Maximum Likelihood estimator allows to detect outliers of different types in a single plot.     

Useful generalized properties of L1 estimators

Recent results by G. Appa and C. Smith, as well as I. Barrodale and F. D. K. Roberts, underscore several properties exhibited for fitting a linear model to a set of observation points under the criterion of least sum of absolute deviations(commonly denoted as the L₁ criterion). This paper will generalize these properties to the non-full rank case and relax in a natural way some assumptions given by Appa and Smith.     

Estimation and selection procedures in regression: An L1 approach

The authors consider the problem of estimating a regression function go involving several variables by the closest functional element of a prescribed class _G that is closest to it in the L₁ norm. They propose a new estimator ? based on independent observations and give explicit finite sample bounds for the L₁distance between ?g and go. They apply their estimation procedure to the problem of selecting the smoothing parameter in nonparametric regression.     

Modified profile likelihood estimation for the Weibull regression models in survival analysis

In this study, adjustment of profile likelihood function of parameter of interest in presence of many nuisance parameters is investigated for survival regression models. Our objective is to extend the Barndorff–Nielsen’s technique to Weibull regression models for estimation of shape parameter in presence of many nuisance and regression parameters. We conducted Monte-Carlo simulation studies and a real data analysis, all of which demonstrate and suggest that the modified profile likelihood estimators outperform the profile likelihood estimators in terms of three comparison criterion: mean squared errors, bias and standard errors.     

Robust multivariate diagnostics for PLSR and application on high dimensional spectrally overlapped drug systems

ABSTRACT

Statistical methods are effectively used in the evaluation of pharmaceutical formulations instead of laborious liquid chromatography. However, signal overlapping, nonlinearity, multicollinearity and presence of outliers deteriorate the performance of statistical methods. The Partial Least Squares Regression (PLSR) is a very popular method in the quantification of high dimensional spectrally overlapped drug formulations. The SIMPLS is the mostly used PLSR algorithm, but it is highly sensitive to outliers that also effect the diagnostics. In this paper, we propose new robust multivariate diagnostics to identify outliers, influential observations and points causing non-normality for a PLSR model. We study performances of the proposed diagnostics on two everyday use highly overlapping drug systems: Paracetamol–Caffeine and Doxylamine Succinate–Pyridoxine Hydrochloride.     

Pena's statistic for the Liu regression

In fitting regression model, one or more observations may have substantial effects on estimators. These unusual observations are precisely detected by a new diagnostic measure, Pena's statistic. In this article, we introduce a type of Pena's statistic for each point in Liu regression. Using the forecast change property, we simplify the Pena's statistic in a numerical sense. It is found that the simplified Pena's statistic behaves quite well as far as detection of influential observations is concerned. We express Pena's statistic in terms of the Liu leverages and residuals. The normality of this statistic is also discussed and it is demonstrated that it can identify a subset of high Liu leverage outliers. For numerical evaluation, simulated studies are given and a real data set has been analysed for illustration.     

Ridge regression estimators in the linear regression models with non-spherical errors

The present paper considers a family of ordinary ridge regression estimators in the linear regression model when the disturbances covariance matrix depends upon a few unknown parameters. An asymptotic expansion for the distribution of the ridge regression estimator is developed and under the quadratic loss function its asymptotic risk is compared with that of the feasible GLS estimator.     

A Jackknifed estimators for the negative binomial regression model

Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators for estimating the ridge parameter k in the negative binomial (NB) regression have been proposed. The Jackknifed estimators are obtained to remedy the multicollinearity and reduce the bias. A simulation study is provided to evaluate the performance of estimators. Both mean squared error (MSE) and the percentage relative error (PRE) are considered as the performance criteria. The simulated result indicated that some of proposed Jackknifed estimators should be preferred to the ML method and ridge estimators to reduce MSE and bias.     

A new modified Jackknifed estimator for the Poisson regression model

The Poisson regression is very popular in applied researches when analyzing the count data. However, multicollinearity problem arises for the Poisson regression model when the independent variables are highly intercorrelated. Shrinkage estimator is a commonly applied solution to the general problem caused by multicollinearity. Recently, the ridge regression (RR) estimators and some methods for estimating the ridge parameter k in the Poisson regression have been proposed. It has been found that some estimators are better than the commonly used maximum-likelihood (ML) estimator and some other RR estimators. In this study, the modified Jackknifed Poisson ridge regression (MJPR) estimator is proposed to remedy the multicollinearity. A simulation study and a real data example are provided to evaluate the performance of estimators. Both mean-squared error and the percentage relative error are considered as the performance criteria. The simulation study and the real data example results show that the proposed MJPR method outperforms the Poisson ridge regression, Jackknifed Poisson ridge regression and the ML in all of the different situations evaluated in this paper.     

Estimating the density of a possibly missing response variable in nonlinear regression

This paper considers linear and nonlinear regression with a response variable that is allowed to be “missing at random”. The only structural assumptions on the distribution of the variables are that the errors have mean zero and are independent of the covariates. The independence assumption is important. It enables us to construct an estimator for the response density that uses all the observed data, in contrast to the usual local smoothing techniques, and which therefore permits a faster rate of convergence. The idea is to write the response density as a convolution integral which can be estimated by an empirical version, with a weighted residual-based kernel estimator plugged in for the error density. For an appropriate class of regression functions, and a suitably chosen bandwidth, this estimator is consistent and converges with the optimal parametric rate n^1/2. Moreover, the estimator is proved to be efficient (in the sense of Hájek and Le Cam) if an efficient estimator is used for the regression parameter.     

An alternative algorithm of the empirical likelihood estimation for the parameter of a linear regression model

ABSTRACT

Empirical likelihood (EL) is a nonparametric method based on observations. EL method is defined as a constrained optimization problem. The solution of this constrained optimization problem is carried on using duality approach. In this study, we propose an alternative algorithm to solve this constrained optimization problem. The new algorithm is based on a newton-type algorithm for Lagrange multipliers for the constrained optimization problem. We provide a simulation study and a real data example to compare the performance of the proposed algorithm with the classical algorithm. Simulation and the real data results show that the performance of the proposed algorithm is comparable with the performance of the existing algorithm in terms of efficiencies and cpu-times.