Consistent estimation approach to tackling collinearity and Berkson-type measurement error in linear regression using adjusted Wald-type estimator

This article proposes a consistent estimation approach in linear regression models for the case when the predictor variables are subject to collinearities and Berkson-type measurement errors simultaneously. Our presented procedure does not rely on ridge regression (RR) methods that have been widely addressed in the literature for ill-conditioned problems resulted from multicollinearity. Instead, we review and propose new consistent estimators due to Wald (1940 Wald, A. (1940). Fitting of straight lines if both variables are subject to error. Ann. Math. Stat. 11:284–300.[Crossref] , [Google Scholar]) so that, except finite fourth moments assumptions, no prior knowledge of parametric settings on observations and errors is used, and there is no need to solve estimating equations for coefficient parameters. The performance of the estimation procedure is compared with that of RR-based estimators by using a variety of numerical experiments through Monte Carlo simulation under estimated mean squared error (EMSE) criterion.     

Performance of Wald-type estimator for parametric component in partial linear regression with a mixture of Berkson and classical error models

This article discusses a consistent and almost unbiased estimation approach in partial linear regression for parameters of interest when the regressors are contaminated with a mixture of Berkson and classical errors. Advantages of the presented procedure are: (1) random errors and observations are not necessarily to be parametric settings; (2) there is no need to use additional sample information, and to consider the estimation of nuisance parameters. We will examine the performance of our presented estimate in a variety of numerical examples through Monte Carlo simulation. The proposed approach is also illustrated in the analysis of an air pollution data.     

A Wald-type test statistic based on robust modified median estimator in logistic regression models

In this paper a new robust estimator, modified median estimator, is introduced and studied for the logistic regression model. This estimator is based on the median estimator considered in Hobza et al. [Robust median estimator in logistic regression. J Stat Plan Inference. 2008;138:3822–3840]. Its asymptotic distribution is obtained. Using the modified median estimator, we also consider a Wald-type test statistic for testing linear hypotheses in the logistic regression model and we obtain its asymptotic distribution under the assumption of random regressors. An extensive simulation study is presented in order to analyse the efficiency as well as the robustness of the modified median estimator and Wald-type test based on it.     

Trimmed least squares estimator as best trimmed linear conditional estimator for linear regression model

A class of trimmed linear conditional estimators based on regression quantiles for the linear regression model is introduced. This class serves as a robust analogue of non-robust linear unbiased estimators. Asymptotic analysis then shows that the trimmed least squares estimator based on regression quantiles ( Koenker and Bassett ( 1978 ) ) is the best in this estimator class in terms of asymptotic covariance matrices. The class of trimmed linear conditional estimators contains the Mallows-type bounded influence trimmed means ( see De Jongh et al ( 1988 ) ) and trimmed instrumental variables estimators. A large sample methodology based on trimmed instrumental variables estimator for confidence ellipsoids and hypothesis testing is also provided.     

Liu estimator in partly linear regression models with correlated errors

This article is concerned with the parameter estimation in partly linear regression models when the errors are dependent. To overcome the multicollinearity problem, a generalized Liu estimator is proposed. The theoretical properties of the proposed estimator and its relationship with some existing methods designed for partly linear models are investigated. Finally, a hypothetical data is conducted to illustrate some of the theoretical results.     

Detection of outliers in the multivariate linear regression model

In this article we suggest multivariate kurtosis as a statistic for detection of outliers in a multivariate linear regression model. The statistic has some local optimality properties.     

Variable selection in linear regression based on ridge estimator

In this article, we consider the problem of variable selection in linear regression when multicollinearity is present in the data. It is well known that in the presence of multicollinearity, performance of least square (LS) estimator of regression parameters is not satisfactory. Consequently, subset selection methods, such as Mallow's Cp, which are based on LS estimates lead to selection of inadequate subsets. To overcome the problem of multicollinearity in subset selection, a new subset selection algorithm based on the ridge estimator is proposed. It is shown that the new algorithm is a better alternative to Mallow's Cp when the data exhibit multicollinearity.     

Asymptotic properties of maximum quasi-likelihood estimator in quasi-likelihood non linear models with stochastic regression

In this paper, we establish the asymptotic properties of maximum quasi-likelihood estimator (MQLE) in quasi-likelihood non linear models (QLNMs) with stochastic regression under some mild regular conditions. We also investigate the existence, strong consistency, and asymptotic normality of MQLE in QLNMs with stochastic regression.     

Nonlinear regression models for heterogeneous data with massive outliers

The income or expenditure-related data sets are often nonlinear, heteroscedastic, skewed even after the transformation, and contain numerous outliers. We propose a class of robust nonlinear models that treat outlying observations effectively without removing them. For this purpose, case-specific parameters and a related penalty are employed to detect and modify the outliers systematically. We show how the existing nonlinear models such as smoothing splines and generalized additive models can be robustified by the case-specific parameters. Next, we extend the proposed methods to the heterogeneous models by incorporating unequal weights. The details of estimating the weights are provided. Two real data sets and simulated data sets show the potential of the proposed methods when the nature of the data is nonlinear with outlying observations.     

A generalized inverse regression estimator in multi-univariate linear calibration

A multivariate linear calibration problem, in which response variable is multivariate and explanatory variable is univariate, is considered. In this paper a class of generalized inverse regression estimators is proposed in multi-univariate linear calibration. It includes the classical estimator and the inverse regression one (or Krutchkoff estimator). For the proposed estimator we derive the expressions of bias and mean square error (MSE). Furthermore the behavior of these characteristics is investigated through an analytical method. In addition through a numerical study we confirm the existence of a generalized inverse regression estimator to improve both the classical and the inverse regression estimators on the MSE criterion.     

On the sensitivity to non-normality of a test for outliers in linear models

In this note we use the class of exponential power distributions to assess the robustness to non-normality of the test for outliers based on the maximum absolute studentized residual. We find that the significance levels can be quite markedly affected by even moderate departures from normality of the error distribution in a regression model when the sample size is moderately large.     

The optimal Lp norm estimator in linear regression models

The least squares estimator is usually applied when estimating the parameters in linear regression models. As this estimator is sensitive to departures from normality in the residual distribution, several alternatives have been proposed. The L_p norm estimators is one class of such alternatives. It has been proposed that the kurtosis of the residual distribution be taken into account when a choice of estimator in the L_p norm class is made (i.e. the choice of p). In this paper, the asymtotic variance of the estimators is used as the criterion in the choice of p. It is shown that when this criterion is applied, other characteristics of the residual distribution than the kurtosis (namely moments of order p-2 and 2p-2) are important.     

Improved Liu estimator in a linear regression model

In this paper, we mainly aim to introduce the notion of improved Liu estimator (ILE) in the linear regression model y=Xβ+e. The selection of the biasing parameters is investigated under the PRESS criterion and the optimal selection is successfully derived. We make a simulation study to show the performance of ILE compared to the ordinary least squares estimator and the Liu estimator. Finally, the main results are applied to the Hald data.     

A modified Liu-type estimator with an intercept term under mixture experiments

We consider ridge regression with an intercept term under mixture experiments. We propose a new estimator which is shown to be a modified version of the Liu-type estimator. The so-called compound covariate estimator is applied to modify the Liu-type estimator. We then derive a formula of the total mean squared error (TMSE) of the proposed estimator. It is shown that the new estimator improves upon existing estimators in terms of the TMSE, and the performance of the new estimator is invariant under the change of the intercept term. We demonstrate the new estimator using a real dataset on mixture experiments.     

Robust quasi-likelihood inference in generalized linear mixed models with outliers

It is well known that in a traditional outlier-free situation, the generalized quasi-likelihood (GQL) approach [B.C. Sutradhar, On exact quasilikelihood inference in generalized linear mixed models, Sankhya: Indian J. Statist. 66 (2004), pp. 261–289] performs very well to obtain the consistent as well as the efficient estimates for the parameters involved in the generalized linear mixed models (GLMMs). In this paper, we first examine the effect of the presence of one or more outliers on the GQL estimation for the parameters in such GLMMs, especially in two important models such as count and binary mixed models. The outliers appear to cause serious biases and hence inconsistency in the estimation. As a remedy, we then propose a robust GQL (RGQL) approach in order to obtain the consistent estimates for the parameters in the GLMMs in the presence of one or more outliers. An extensive simulation study is conducted to examine the consistency performance of the proposed RGQL approach.     

A linear regression with unobserved dependent variables

Let (?,X) be a random vector such that E(X|?) = ? and Var(x|?) a + b? + c?² for some known constants a, b and c. Assume X₁,…,X_n are independent observations which have the same distribution as X. Let t(X) be the linear regression of ? on X. The linear empirical Bayes estimator is used to approximate the linear regression function. It is shown that under appropriate conditions, the linear empirical Bayes estimator approximates the linear regression well in the sense of mean squared error.     

Estimation in a linear regression model with stochastic linear restrictions: a new two-parameter-weighted mixed estimator

The present paper considers the weighted mixed regression estimation of the coefficient vector in a linear regression model with stochastic linear restrictions binding the regression coefficients. We introduce a new two-parameter-weighted mixed estimator (TPWME) by unifying the weighted mixed estimator of Schaffrin and Toutenburg [1] and the two-parameter estimator (TPE) of Özkale and Kaç?ranlar [2]. This new estimator is a general estimator which includes the weighted mixed estimator, the TPE and the restricted two-parameter estimator (RTPE) proposed by Özkale and Kaç?ranlar [2] as special cases. Furthermore, we compare the TPWME with the weighted mixed estimator and the TPE with respect to the matrix mean square error criterion. A numerical example and a Monte Carlo simulation experiment are presented by using different estimators of the biasing parameters to illustrate some of the theoretical results.     

Estimation for the censored partially linear quantile regression models

In this article, we develop estimation procedures for partially linear quantile regression models, where some of the responses are censored by another random variable. The nonparametric function is estimated by basis function approximations. The estimation procedure is easy to implement through existing weighted quantile regression, and it requires no specification of the error distributions. We show the large-sample properties of the resulting estimates, the proposed estimator of the regression parameter is root-n consistent and asymptotically normal and the estimator of the functional component achieves the optimal convergence rate of the nonparametric function. The proposed method is studied via simulations and illustrated with the analysis of a primary biliary cirrhosis (BPC) data.     

Multiple outliers detection in sparse high-dimensional regression

The presence of outliers would inevitably lead to distorted analysis and inappropriate prediction, especially for multiple outliers in high-dimensional regression, where the high dimensionality of the data might amplify the chance of an observation or multiple observations being outlying. Noting that the detection of outliers is not only necessary but also important in high-dimensional regression analysis, we, in this paper, propose a feasible outlier detection approach in sparse high-dimensional linear regression model. Firstly, we search a clean subset by use of the sure independence screening method and the least trimmed square regression estimates. Then, we define a high-dimensional outlier detection measure and propose a multiple outliers detection approach through multiple testing procedures. In addition, to enhance efficiency, we refine the outlier detection rule after obtaining a relatively reliable non-outlier subset based on the initial detection approach. By comparison studies based on Monte Carlo simulation, it is shown that the proposed method performs well for detecting multiple outliers in sparse high-dimensional linear regression model. We further illustrate the application of the proposed method by empirical analysis of a real-life protein and gene expression data.