Rank-based ridge estimation in multiple linear regression

Multicollinearity and model misspecification are frequently encountered problems in practice that produce undesirable effects on classical ordinary least squares (OLS) regression estimator. The ridge regression estimator is an important tool to reduce the effects of multicollinearity, but it is still sensitive to a model misspecification of error distribution. Although rank-based statistical inference has desirable robustness properties compared to the OLS procedures, it can be unstable in the presence of multicollinearity. This paper introduces a rank regression estimator for regression parameters and develops tests for general linear hypotheses in a multiple linear regression model. The proposed estimator and the tests have desirable robustness features against the multicollinearity and model misspecification of error distribution. Asymptotic behaviours of the proposed estimator and the test statistics are investigated. Real and simulated data sets are used to demonstrate the feasibility and the performance of the estimator and the tests.     

Joint impact of multiple observations on a subset of variables in multiple linear regression

In multiple linear regression analysis, each observation affects the fitted regression equation differently and has varying influences on the regression coefficients of the different variables. Chatterjee & Hadi (1988) have proposed some measures such as DSSE_ij (Impact on Residual Sum of Squares of simultaneously omitting the ith observation and the jth variable), F_j (Partial F-test for the jth variable) and F_j(i) (Partial F-test for the jth variable omitting the ith observation) to show the joint impact and the interrelationship that exists among a variable and an observation. In this paper we have proposed more extended form of those measures DSSE_IJ, F_J and F_J(I) to deal with the interrelationships that exist among the multiple observations and a subset of variables by monitoring the effects of the simultaneous omission of multiple variables and multiple observations.     

A ray method of confidence band construction for multiple linear regression models

This paper addresses the problem of confidence band construction for a standard multiple linear regression model. A “ray” method of construction is developed which generalizes the method of Graybill and Bowden [1967. Linear segment confidence bands for simple linear regression models. J. Amer. Statist. Assoc. 62, 403–408] for a simple linear regression model to a multiple linear regression model. By choosing suitable directions for the rays this method requires only critical points from t-distributions so that the confidence bands are easy to construct. Both one-sided and two-sided confidence bands can be constructed using this method. An illustration of the new method is provided.     

Variable selection in linear regression based on ridge estimator

In this article, we consider the problem of variable selection in linear regression when multicollinearity is present in the data. It is well known that in the presence of multicollinearity, performance of least square (LS) estimator of regression parameters is not satisfactory. Consequently, subset selection methods, such as Mallow's Cp, which are based on LS estimates lead to selection of inadequate subsets. To overcome the problem of multicollinearity in subset selection, a new subset selection algorithm based on the ridge estimator is proposed. It is shown that the new algorithm is a better alternative to Mallow's Cp when the data exhibit multicollinearity.     

Robust two parameter ridge M-estimator for linear regression

The problem of multicollinearity and outliers in the data set produce undesirable effects on the ordinary least squares estimator. Therefore, robust two parameter ridge estimation based on M-estimator (ME) is introduced to deal with multicollinearity and outliers in the y-direction. The proposed estimator outperforms ME, two parameter ridge estimator and robust ridge M-estimator according to mean square error criterion. Moreover, a numerical example and a Monte Carlo simulation experiment are presented.     

New shrinkage-type estimators in a linear regression model when multicollinearity is severe

For the linear regression model y=Xβ+e with severe multicollinearity, we put forward three shrinkage-type estimators based on the ordinary least-squares estimator including two types of independent factor estimators and a seemingly convex combination. The simulation study shows that the new estimators are not good enough when multicollinearity is mild to moderate, but perform very well when multicollinearity is severe to very severe.     

Multivariate linear regression for heterogeneous data

The problem of multivariate regression modelling in the presence of heterogeneous data is dealt to address the relevant issue of the influence of such heterogeneity in assessing the linear relations between responses and explanatory variables. In spite of its popularity, clusterwise regression is not designed to identify the linear relationships within ‘homogeneous’ clusters exhibiting internal cohesion and external separation. A within-clusterwise regression is introduced to achieve this aim and, since the possible presence of a linear relation ‘between’ clusters should be also taken into account, a general regression model is introduced to account for both the between-cluster and the within-cluster regression variation. Some decompositions of the variance of the responses accounted for are also given, the least-squares estimation of the parameters is derived, together with an appropriate coordinate descent algorithms and the performance of the proposed methodology is evaluated in different datasets.     

Stability of linear programming solutions using regression coefficients

In at least one important application of stochastic linear programming (Lavaca-Tres Palacios Estuary:A Study of the Influence of Freshwater Inflows, 1980)constraint parameters are simultaneously estimated using multiple regression with historic data for the values of the decision variables and the right hand side of the constraint function. In this circumstance, the question immediately arises "How stable is the linear programming (LP) solution with regard to regression issues such as sample size, magnitude of the error variance, centroids of the decision variables, apd collinearity?" This paper reports a simulation designed to assess the stability of the LP solution and to compare the effectiveness of ridge as an alternative to ordinary least squares (OLS) regression. For the given scenario, the LP solution is consistently "biased." The amount of bias is exacerbated by small samples, large error variances, and collinearity among observations of the decision variables. The best regression criterion is a function not only of collinearity, but also of the magnitude of the error variance and the sum of the means of the decision variables relative to the right hand side of the stochastic constraint

In the application that motivated this research, the LP solutions were recommended fresh water inflows from Lake Texana into the estuaries of the Gulf of Mexico. The stochastic constraint estimates commercial fish harvest as a function of seasonal fresh water inflow. The historic data set used to estimate parameters of the constraint comprised rainfall data and fish harvest data prior to the construction of the Lake Texana dam, of necessity a small sample with collinear seasonal rainfall. It is not the authors' intent to solve this application, but rather to investigate through a simpler simulated systemwhether or not regression estimates in similar circumstances might introduce a systematic and predictable bias. The answer to this latter question is a qualified Yes!.     

M-estimation for functional linear regression

This paper studies M-estimation in functional linear regression in which the dependent variable is scalar while the covariate is a function. An estimator for the slope function is obtained based on the functional principal component basis. The global convergence rate of the M-estimator of unknown slope function is established. The convergence rate of the mean-squared prediction error for the proposed estimators is also established. Monte Carlo simulation studies are conducted to examine the finite-sample performance of the proposed procedure. Finally, the proposed method is applied to analyze the Berkeley growth data.     

Testing for a change point in linear regression models

In this paper, the Schwarz Information Criterion (SIC) is proposed to locate a change point in the simple linear regression model, as well as in the multiple linear regression model. The method is then applied to a financial data set, and a change point is successfully detected.     

An alternative algorithm of the empirical likelihood estimation for the parameter of a linear regression model

ABSTRACT

Empirical likelihood (EL) is a nonparametric method based on observations. EL method is defined as a constrained optimization problem. The solution of this constrained optimization problem is carried on using duality approach. In this study, we propose an alternative algorithm to solve this constrained optimization problem. The new algorithm is based on a newton-type algorithm for Lagrange multipliers for the constrained optimization problem. We provide a simulation study and a real data example to compare the performance of the proposed algorithm with the classical algorithm. Simulation and the real data results show that the performance of the proposed algorithm is comparable with the performance of the existing algorithm in terms of efficiencies and cpu-times.     

An algorithm for discrete chebychev curve fitting for the simple model using a dual linear programming approach

An algorithm, in t h e form of a Fortran subroutine TRIPLE,is given to compute statistics forthetriples test for symmetry, The 2 computational complexity of the algorithm is O(n² ) which is an 3 improvement over the straightforward method, which is O(n³).     

Interval-valued data regression using partial linear model

Semi-parametric modelling of interval-valued data is of great practical importance, as exampled by applications in economic and financial data analysis. We propose a flexible semi-parametric modelling of interval-valued data by integrating the partial linear regression model based on the Center & Range method, and investigate its estimation procedure. Furthermore, we introduce a test statistic that allows one to decide between a parametric linear model and a semi-parametric model, and approximate its null asymptotic distribution based on wild Bootstrap method to obtain the critical values. Extensive simulation studies are carried out to evaluate the performance of the proposed methodology and the new test. Moreover, several empirical data sets are analysed to document its practical applications.     

Reducing errors-in-variables bias in linear regression using compact genetic algorithms

A new technique is devised to mitigate the errors-in-variables bias in linear regression. The procedure mimics a 2-stage least squares procedure where an auxiliary regression which generates a better behaved predictor variable is derived. The generated variable is then used as a substitute for the error-prone variable in the first-stage model. The performance of the algorithm is tested by simulation and regression analyses. Simulations suggest the algorithm efficiently captures the additive error term used to contaminate the artificial variables. Regressions provide further credit to the simulations as they clearly show that the compact genetic algorithm-based estimate of the true but unobserved regressor yields considerably better results. These conclusions are robust across different sample sizes and different variance structures imposed on both the measurement error and regression disturbances.     

An algorithm for the minimum sum of weighted absolute errors regression

We propose an algorithm to estimate the unknown constants in a multiple linear regression model under the minimum sum of weighted absolute errors (MSWAE). The proposed algorithm, a generalization of an earlier algorithm, is compared to a bounded variable algorithm. Some somputational experience is reported.     

Bandwidth selection for local linear regression smoothers

Summary. The paper presents a general strategy for selecting the bandwidth of nonparametric regression estimators and specializes it to local linear regression smoothers. The procedure requires the sample to be divided into a training sample and a testing sample. Using the training sample we first compute a family of regression smoothers indexed by their bandwidths. Next we select the bandwidth by minimizing the empirical quadratic prediction error on the testing sample. The resulting bandwidth satisfies a finite sample oracle inequality which holds for all bounded regression functions. This permits asymptotically optimal estimation for nearly any regression function. The practical performance of the method is illustrated by a simulation study which shows good finite sample behaviour of our method compared with other bandwidth selection procedures.     

Estimation for semi-functional linear regression

This paper studies estimation in semi-functional linear regression. A general formulation is used to treat mean regression, median regression, quantile regression and robust mean regression in one setting. The linear slope function is estimated by the functional principal component basis and the nonparametric component is approximated by a B-spline function. The global convergence rates of the estimators of unknown slope function and nonparametric component are established under suitable norm. The convergence rate of the mean-squared prediction error for the proposed estimators is also established. Finite sample properties of our procedures are studied through Monte Carlo simulations. A real data example about Berkeley growth data is used to illustrate our proposed methodology.     

Subset selection in multiple linear regression in the presence of outlier and multicollinearity

Various subset selection methods are based on the least squares parameter estimation method. The performance of these methods is not reasonably well in the presence of outlier or multicollinearity or both. Few subset selection methods based on the $M$-estimator are available in the literature for outlier data. Very few subset selection methods account the problem of multicollinearity with ridge regression estimator.In this article, we develop a generalized version of $S_p$ statistic based on the jackknifed ridge $M$-estimator for subset selection in the presence of outlier and multicollinearity. We establish the equivalence of this statistic with the existing $C_p$, $S_p$ and $R_p$ statistics. The performance of the proposed method is illustrated through some numerical examples and the correct model selection ability is evaluated using simulation study.