A final twist on the equality of OLS and GLS

This note generalizes various previous results on the equality of OLS and GLS in the General Linear Regression Model.     

具有核化函数的部分线性模型及其应用

本文基于再生核希尔伯特空间中的再生核，将核技巧与高斯-赛责尔迭代算法相结合，提出了具有核化函数的部分线性模型（PLMKF）及其算法收敛性条件等相关内容，具体包括：（1）基于OLS的PLMKF；（2）基于岭估计的PLMKF；（3）基于GLS的PLMKF；（4）基于多核学习的PLMKF。它们构成了PLMKF家族，具有一定的相互转化关系。在数值模拟中，本文验证了各个算法的有效性，比较了基于OLS与GLS、单核与多核的PLMKF模拟结果。实际应用中，在大幅外推情景下，PLMKF仍保持了良好的泛化能力，预测精度高于PLM、GAM和SVR。     

A simulation study on SPSS ridge regression and ordinary least squares regression procedures for multicollinearity data

This study compares the SPSS ordinary least squares (OLS) regression and ridge regression procedures in dealing with multicollinearity data. The LS regression method is one of the most frequently applied statistical procedures in application. It is well documented that the LS method is extremely unreliable in parameter estimation while the independent variables are dependent (multicollinearity problem). The Ridge Regression procedure deals with the multicollinearity problem by introducing a small bias in the parameter estimation. The application of Ridge Regression involves the selection of a bias parameter and it is not clear if it works better in applications. This study uses a Monte Carlo method to compare the results of OLS procedure with the Ridge Regression procedure in SPSS.     

A Novel Regression Approach: Least Squares Ratio

Regression Analysis (RA) is one of the frequently used tool for forecasting. The Ordinary Least Squares (OLS) Technique is the basic instrument of RA and there are many regression techniques based on OLS. This paper includes a new regression approach, called Least Squares Ratio (LSR), and comparison of OLS and LSR according to mean square errors of estimation of theoretical regression parameters (mse ß) and dependent value (mse y).     

Generalized Systematic Sampling

In this paper, we introduce a new systematic sampling design, called a Generalized Systematic Sampling (GSS), for estimation of finite population mean. The proposed design is found to be better than Simple Random Sampling (SRS) and the generalization of the several existing systematic sampling schemes such as, Linear Systematic Sampling (LSS), Diagonal Systematic Sampling (DSS), and Generalized Diagonal Systematic Sampling (GDSS). All of these designs become special cases of the proposed design.     

Study of partial least squares and ridge regression methods

This article considers both Partial Least Squares (PLS) and Ridge Regression (RR) methods to combat multicollinearity problem. A simulation study has been conducted to compare their performances with respect to Ordinary Least Squares (OLS). With varying degrees of multicollinearity, it is found that both, PLS and RR, estimators produce significant reductions in the Mean Square Error (MSE) and Prediction Mean Square Error (PMSE) over OLS. However, from the simulation study it is evident that the RR performs better when the error variance is large and the PLS estimator achieves its best results when the model includes more variables. However, the advantage of the ridge regression method over PLS is that it can provide the 95% confidence interval for the regression coefficients while PLS cannot.     

The asymptotic unbiasedness of S2 in the linear regression model with AR(1)-disturbances

The OLS-estimator of the disturbance variance in the Linear Regression Model is shown to be asymptotically unbiased in the context of AR(1)-disturbances, although for any given design, E(s²/σ²) tends to zero as correlation increases.     

A New Sampling Design for Systematic Sampling

A sampling design called “Modified Systematic Sampling (MSS)” is proposed. In this design each unit has an equal probability of selection. Moreover, it works for both situations: N = nk or N ≠ nk. Consequently, the Linear Systematic Sampling (LSS) and Circular Systematic Sampling (CSS) become special cases of the proposed MSS design.     

地区与国家GDP核算总量数据衔接方法比较研究

 本文以国家数据为准,通过采用2005-2009年的数据为样本,从理论与实证上比较分析了地区与国家GDP数据衔接的三种方法即Geary和Stark的产出估算方法、线性调整法与辅助回归法,比较结果显示：（1）从理论上分析,三种方法都有其合理性,只是辅助回归法较另两种方法更具可取性。（2）从衔接效果上看,辅助回归法优于Geary和Stark的产出估算方法,Geary和Stark的产出估算方法又优于线性调整法。不过不同的方法皆有相应的适用场合与特点以及不同的衔接效果,因而只能说三种方法中有趋优的方法,但不能明确断定何种方法可以具体应用于实际数据衔接中并能达到良好的调整效果。     

Robust Liu estimator for regression based on an M-estimator

Consider the regression model y = beta 0 1 + Xbeta + epsilon. Recently, the Liu estimator, which is an alternative biased estimator beta L (d) = (X'X + I) -1 (X'X + dI)beta OLS , where 0<d<1 is a parameter, has been proposed to overcome multicollinearity . The advantage of beta L (d) over the ridge estimator beta R (k) is that beta L (d) is a linear function of d. Therefore, it is easier to choose d than to choose k in the ridge estimator. However, beta L (d) is obtained by shrinking the ordinary least squares (OLS) estimator using the matrix (X'X + I) -1 (X'X + dI) so that the presence of outliers in the y direction may affect the beta L (d) estimator. To cope with this combined problem of multicollinearity and outliers, we propose an alternative class of Liu-type M-estimators (LM-estimators) obtained by shrinking an M-estimator beta M , instead of the OLS estimator using the matrix (X'X + I) -1 (X'X + dI).     

A New Ridge Regression Causality Test in the Presence of Multicollinearity

The VAR lag structure applied for the traditional Granger causality (GC) test is always severely affected by multicollinearity due to autocorrelation among the lags. Therefore, as a remedy to this problem we introduce a new Ridge Regression Granger Causality (RRGC) test, which is compared to the GC test by means of Monte Carlo simulations. Based on the simulation study we conclude that the traditional OLS version of the GC test over-rejects the true null hypothesis when there are relatively high (but empirically normal) levels of multicollinearity, while the new RRGC test will remedy or substantially decrease this problem.     

A New Robust Regression Method Based on Particle Swarm Optimization

Regression analysis is one of methods widely used in prediction problems. Although there are many methods used for parameter estimation in regression analysis, ordinary least squares (OLS) technique is the most commonly used one among them. However, this technique is highly sensitive to outlier observation. Therefore, in literature, robust techniques are suggested when data set includes outlier observation. Besides, in prediction a problem, using the techniques that reduce the effectiveness of outlier and using the median as a target function rather than an error mean will be more successful in modeling these kinds of data. In this study, a new parameter estimation method using the median of absolute rate obtained by division of the difference between observation values and predicted values by the observation value and based on particle swarm optimization was proposed. The performance of the proposed method was evaluated with a simulation study by comparing it with OLS and some other robust methods in the literature.     

Estimation of linear models for field experiments

General mixed linear models for experiments conducted over a series of sltes and/or years are described. The ordinary least squares (OLS) estlmator is simple to compute, but is not the best unbiased estimator. Also, the usuaL formula for the varlance of the OLS estimator is not correct and seriously underestimates the true variance. The best linear unbiased estimator is the generalized least squares (GLS) estimator. However, t requires an inversion of the variance-covariance matrix V, whlch is usually of large dimension. Also, in practice, V is unknown.

We presented an estlmator [Vcirc] of the matrix V using the estimators of variance components [for sites, blocks (sites), etc.]. We also presented a simple transformation of the data, such that an ordinary least squares regression of the transformed data gives the estimated generalized least squares (EGLS) estimator. The standard errors obtained from the transformed regression serve as asymptotic standard errors of the EGLS estimators. We also established that the EGLS estlmator is unbiased.

An example of fitting a linear model to data for 18 sites (environments) located in Brazil is given. One of the site variables (soil test phosphorus) was measured by plot rather than by site and this established the need for a covariance model such as the one used rather than the usual analysis of variance model. It is for this variable that the resulting parameter estimates did not correspond well between the OLS and EGLS estimators. Regression statistics and the analysis of variance for the example are presented and summarized.     

OLS与ML：回归模型两种参数估计方法的比较研究

最小二乘法(OLS)和最大似然法(ML)是回归模型参数估计的两种最重要的方法。 但二者有着明显的差别,本文就二者之间的有关差别进行比较。     

Two-stage estimation of inequality-constrained marginal linear models with longitudinal data

Inequality-constrained regression models have received increased attention in longitudinal analysis during recent years. Regression parameters are usually obtained from iteration algorithms. An analytical formulae of the estimators cannot be provided. Therefore, the asymptotic behavior of estimators has not been fully clarified yet. This paper presents a TS estimation (TS for short) and the asymptotic distribution of the estimators. Simulations are conducted to compare constrained TS estimation, constrained ordinary least squares (OLS) estimation and TS estimation in terms of sample bias, sample mean-square error (MSE) and sample variance of the estimators.     

技术采纳对农业生产技术效率的影响效应分析——基于随机前沿分析与分位数回归分解

使用随机前沿分析方法研究了中国农业生产的技术效率,利用OLS和分位数回归及分解方法分析了技术采纳对中国农业生产技术效率的影响。结果发现:中国农业生产呈现出规模报酬递增的状态,但土地利用效率仍具提升空间;技术采纳对农业生产技术效率有一定的改善,而改善空间却局限于农业生产的规模;技术采纳所带来的农业生产技术效率提升作用会随着农业生产技术效率增加而逐渐被耗散,农业产出与技术采纳之间具有"刺猬效应"。     

Book Reviews

Book reviewed in this article:
The Correspondence between A. A. Markov and A. A. Chuprov on the Theory of Probability and Mathematical Statistics. Edited by Kh. O. Ondar. Translated by Charles & Margaret Stein.
Applied Regression Analysis, 2nd Edition. By Norman R. Draper and Harry Smith.
The Analysis of Categorical Data , 2nd Edition. By R. L. Plackett.
Survival Analysis. By Rupert G. Miller.
Interpreting Multivariate Data. Edited by Vic Barnett.
Handbook of Statistics , Volume 1; Analysis of Variance, Edited by P. R. Krishnaiah.
Sampling from a Finite Population. By Jaroslav Hájek.
Circular Statistics in Biology. By E. Batschelet.
Non-Negative Matrices and Markov Chains , 2nd Edition. By E. Seneta.
Spectral Analysis and Time Series. By M. B. Priestley.     

Classification trees aided mixed regression model

This paper introduces a novel hybrid regression method (MixReg) combining two linear regression methods, ordinary least square (OLS) and least squares ratio (LSR) regression. LSR regression is a method to find the regression coefficients minimizing the sum of squared error rate while OLS minimizes the sum of squared error itself. The goal of this study is to combine two methods in a way that the proposed method superior both OLS and LSR regression methods in terms of R² statistics and relative error rate. Applications of MixReg, on both simulated and real data, show that MixReg method outperforms both OLS and LSR regression.