A simulation study of least squares and ridge estimaors for normal and nonnormal autocorrelated disturbances

The purpose of this paper is to examine the small sample properties of various ridge estimators along with least squares, in some special settings.Specifically, we consider a first order autoregressive structuure for normal and nonnormal disturbances, and report on a Monte Carlo study the small sample behavior of these estimators according to the criteria of bias and dispersion.The results suggest that under all the examined settings and for all the criteria used the HKB estimator exhibited a superior performance compared to the other estimators, while the LS and LW estimators gave consistently poor results.Also if the error term is only moderately autocorrelated the performance of the ridge estimators that do not account for autocorrelation outperform their counterparts as well as least squares that account for autocorrelation.     

A Monte Carlo simulation study on partially adaptive estimators of linear regression models

This paper presents a comprehensive comparison of well-known partially adaptive estimators (PAEs) in terms of efficiency in estimating regression parameters. The aim is to identify the best estimators of regression parameters when error terms follow from normal, Laplace, Student's t, normal mixture, lognormal and gamma distribution via the Monte Carlo simulation. In the results of the simulation, efficient PAEs are determined in the case of symmetric leptokurtic and skewed leptokurtic regression error data. Additionally, these estimators are also compared in terms of regression applications. Regarding these applications, using certain standard error estimators, it is shown that PAEs can reduce the standard error of the slope parameter estimate relative to ordinary least squares.     

A Note on a Commonly Used Ridge Regression Monte Carlo Design

Ridge estimators are usually examined through Monte Carlo simulations since their properties are difficult to obtain analytically. In this paper we argue that a simulation design commonly used in the literature will give biased results of Monte Carlo simulations in favor of ridge regression over ordinary least square estimators. Specifically, it is argued that the properties of ridge estimators that are functions of p distinct regressor eigenvalues should not be evaluated through Monte Carlo designs using only two distinct eigenvalues.     

Bayes estimation of reliability for the two-parameter lognormal distribution

Bayes estimators of reliability for the lognormal failure distribution with two parameters (M,∑) are obtained both for informative priors of normal-gamma type and for the vague prior of Jeffreys. The estimators are in terms of the t-distribution function. The Bayes estimators are compared with the maximum likelihood and minimum variance unbiased estimators of reliabil-ity using Monte Carlo simulations.     

The relative performances of improved ridge estimators and an empirical bayes estimator: some monte carlo results

The relative 'performances of improved ridge estimators and an empirical Bayes estimator are studied by means of Monte Carlo simulations. The empirical Bayes method is seen to perform consistently better in terms of smaller MSE and more accurate empirical coverage than any of the estimators considered here. A bootstrap method is proposed to obtain more reliable estimates of the MSE of ridge esimators. Some theorems on the bootstrap for the ridge estimators are also given and they are used to provide an analytical understanding of the proposed bootstrap procedure. Empirical coverages of the ridge estimators based on the proposed procedure are generally closer to the nominal coverage when compared to their earlier counterparts. In general, except for a few cases, these coverages are still less accurate than the empirical coverages of the empirical Bayes estimator.     

A Monte Carlo study of some limited and full information simultaneous equation estimators with normal and nonnormal autocorrelated disturbances

The purpose of this paper is to examine the small sample properties of various limited and full information estimators of the structural coefficients of a system of two equations. Specifically, we consider a first-order autoregressive error structure under normal and nonnormal disturbances — for four different covariance structures — and report on a Monte Carlo study of the small sample behavior of limited and full information estimators according to the criteria of bias and dispersion. The results show that the differences in performance of the estimators for the alternative forms of the disturbance distributions are large. Moreover, none of the examined estimators is superior relative to the others, in the sense that its bias and dispersion are the smallest for at least one form of the disturbance distribution. Finally, no combination of highly or lowly autocorrelated disturbances favors some specific limited or full information estimator.     

A class of generalized ridge estimators

Presence of collinearity among the explanatory variables results in larger standard errors of parameters estimated. When multicollinearity is present among the explanatory variables, the ordinary least-square (OLS) estimators tend to be unstable due to larger variance of the estimators of the regression coefficients. As alternatives to OLS estimators few ridge estimators are available in the literature. This article presents some of the popular ridge estimators and attempts to provide (i) a generalized class of ridge estimators and (ii) a modified ridge estimator. The performance of the proposed estimators is investigated with the help of Monte Carlo simulation technique. Simulation results indicate that the suggested estimators perform better than the ordinary least-square (OLS) estimators and other estimators considered in this article.     

A consistent parameter estimation in the three-parameter lognormal distribution

In this work, we propose a consistent method of estimation for the parameters of the three-parameter lognormal distribution. We then discuss some properties of these estimators and show by means of a Monte Carlo simulation study that the proposed estimators perform better than some other prominent estimators in terms of bias and root mean squared error. Finally, we present two real-life examples to illustrate the method of estimation proposed.     

Restricted Ridge Estimators of the Parameters in Semiparametric Regression Model

In this article, we introduce a ridge estimator for the vector of parameters β in a semiparametric model when additional linear restrictions on the parameter vector are assumed to hold. We also obtain the semiparametric restricted ridge estimator for the parametric component in the semiparametric regression model. The ideas in this article are illustrated with a data set consisting of housing prices and through a comparison of the performances of the proposed and related estimators via a Monte Carlo simulation.     

Ridge estimation in regression problems with autocorrelated errors: A monte carlo study

This paper presents the results of a Monte Carlo study of OLS and GLS based adaptive ridge estimators for regression problems in which the independent variables are collinear and the errors are autocorrelated. It studies the effects of degree of collinearity, magnitude of error variance, orientation of the parameter vector and serial correlation of the independent variables on the mean squared error performance of these estimators. Results suggest that such estimators produce greatly improved performance in favorable portions of the parameter space. The GLS based methods are best when the independent variables are also serially correlated.     

Modified moment and maximum likelihood estimators for parameters of the three-parameter gamma distribution

This article is concerned with modifications of both maximum likelihood and moment estimators for parameters of the three-parameter gamma distribution. Modifications employed here are essentially the same as those previously considered by the authors (1980, 1981) in connection with the lognormal distribution. Sampling behavior of the estimates is indicated by a Monte Carlo simulation. For certain combinations of parameter values, these new estimators appear better than both maximum likelihood and moment estimators with respect to bias, variance and/or ease of calculation.     

Standard and goodness-of-fit parameter estimation methods for the three-parameter lognormal distribution

A class of goodness-of-fit estimators is found to provide a useful alternative in certain situations to the standard maximum likelihood method which has some undesirable estimation characteristics for estimation from the three-parameter lognormal distribution. The class of goodness-of-fit tests considered include the Shapiro-Wilk and Filliben tests which reduce to a weighted linear combination of the order statistics that can be maximized in estimation problems. The weighted order statistic estimators are compared to the standard procedures in Monte Carlo simulations. Robustness of the procedures are examined and example data sets analyzed.     

Analysis of Type-II hybrid censored competing risks data

Kundu and Gupta [Analysis of hybrid life-tests in presence of competing risks. Metrica. 2007;65:159–170] provided the analysis of Type-I hybrid censored competing risks data, when the lifetime distributions of the competing cause of failures follows exponential distribution. In this paper, we consider the analysis of Type-II hybrid censored competing risks data. It is assumed that latent lifetime distributions of the competing causes of failures follow independent exponential distributions with different scale parameters. It is observed that the maximum likelihood estimators of the unknown parameters do not always exist. We propose the modified estimators of the scale parameters, which coincide with the corresponding maximum likelihood estimators when they exist, and asymptotically they are equivalent. We obtain the exact distribution of the proposed estimators. Using the exact distributions of the proposed estimators, associated confidence intervals are obtained. The asymptotic and bootstrap confidence intervals of the unknown parameters are also provided. Further, Bayesian inference of some unknown parametric functions under a very flexible Beta-Gamma prior is considered. Bayes estimators and associated credible intervals of the unknown parameters are obtained using the Monte Carlo method. Extensive Monte Carlo simulations are performed to see the effectiveness of the proposed estimators and one real data set has been analysed for the illustrative purposes. It is observed that the proposed model and the method work quite well for this data set.     

Using Heteroscedasticity-Consistent Standard Errors for the Linear Regression Model with Correlated Regressors

The use of heteroscedasticity-consistent covariance matrix (HCCM) estimators is very common in practice to draw correct inference for the coefficients of a linear regression model with heteroscedastic errors. However, in addition to the problem of heteroscedasticity, linear regression models may also be plagued with some considerable degree of collinearity among the regressors when two or more regressors are considered. This situation causes many adverse effects on the least squares measures and alternatively, the ordinary ridge regression method is used as a common practice. But in the available literature, the problems of multicollinearity and heteroscedasticity have not been discussed as a combined issue especially, for the inference of the regression coefficients. The present article addresses the inference about the regression coefficients taking both the issues of multicollinearity and heteroscedasticity into account and suggests the use of HCCM estimators for the ridge regression. This article proposes t- and F-tests, based on these HCCM estimators, that perform adequately well in the numerical evaluation of the Monte Carlo simulations.     

A modified ridge m-estimator for linear regression model with multicollinearity and outliers

The ordinary least-square estimators for linear regression analysis with multicollinearity and outliers lead to unfavorable results. In this article, we propose a new robust modified ridge M-estimator (MRME) based on M-estimator (ME) to deal with the combined problem resulting from multicollinearity and outliers in the y-direction. MRME outperforms modified ridge estimator, robust ridge estimator and ME, according to mean squares error criterion. Furthermore, a numerical example and a Monte Carlo simulation experiment are given to illustrate some of the theoretical results.     

Two kinds of restricted modified estimators in linear regression model

In this paper, we introduce two kinds of new restricted estimators called restricted modified Liu estimator and restricted modified ridge estimator based on prior information for the vector of parameters in a linear regression model with linear restrictions. Furthermore, the performance of the proposed estimators in mean squares error matrix sense is derived and compared. Finally, a numerical example and a Monte Carlo simulation are given to illustrate some of the theoretical results.     

A note on some conditions for optimality of certain ridge estimators

Consider the linear regression model, y = Xβ + ε in the usual notation with X'X being in the correlation form. Galpin(1980) claimed that the ridge estimators of Hoerl, Kennard and Baldwin(1975) and Lawless and Wang(1976) give guaranteed lower mean squared error than the least squares estimator when X'X has at least two very small eigen values. We show that the arguments of Galpin(1980) leading to the above claim are incorrect, and hence the claim itself is unsubstantited. A Monte Carlo study shows that Galpin's claim is not correct in general.     

The Almon two parameter estimator for the distributed lag models

The two parameter estimator proposed by Özkale and Kaç?ranlar [The restricted and unrestricted two parameter estimators. Comm Statist Theory Methods. 2007;36(15):2707–2725] is a general estimator which includes the ordinary least squares, the ridge and the Liu estimators as special cases. In the present paper we introduce Almon two parameter estimator based on the two parameter estimation procedure to deal with the problem of multicollinearity for the distiributed lag models. This estimator outperforms the Almon estimator according to the matrix mean square error criterion. Moreover, a numerical example and a Monte Carlo simulation experiment are presented by using different estimators of the biasing parameters.     

Two methods of evaluating hoerl and kennard's ridge regression

We formulate a modified version of the Hoerl-Kennard ridge regression method to solve the problem of estimating coefficients in economic relationships. We investigate two approaches for determining the biasing parameter One approach utilizes prior information in choosing jr, the other approach estimates y from the sample data. Monte Carlo experiments are used to evaluate the relative efficiencies of alternative ridge estimators.     

A Simulation Study of Some Biasing Parameters for the Ridge Type Estimation of Poisson Regression

This article proposes several estimators for estimating the ridge parameter k based on Poisson ridge regression (RR) model. These estimators have been evaluated by means of Monte Carlo simulations. As performance criteria, we have calculated the mean squared error (MSE), the mean value, and the standard deviation of k. The first criterion is commonly used, while the other two have never been used when analyzing Poisson RR. However, these performance criteria are very informative because, if several estimators have an equal estimated MSE, then those with low average value and standard deviation of k should be preferred. Based on the simulated results, we may recommend some biasing parameters that may be useful for the practitioners in the field of health, social, and physical sciences.