A Mixed Model-assisted Regression Estimator that Uses Variables Employed at the Design Stage

The Generalized regression estimator (GREG) of a finite population mean or total has been shown to be asymptotically optimal when the working linear regression model upon which it is based includes variables related to the sampling design. In this paper a regression estimator assisted by a linear mixed superpopulation model is proposed. It accounts for the extra information coming from the design in the random component of the model and saves degrees of freedom in finite sample estimation. This procedure combines the larger asymptotic efficiency of the optimal estimator and the greater finite sample stability of the GREG. Design based properties of the proposed estimator are discussed and a small simulation study is conducted to explore its finite sample performance.     

Polynomial approximation of nonlinear regression functions

In the present paper a nonlinear regression function is approximated by a polynomial estimator according to the expectation of the quadratic L ²-distance as risk is given. For special experimental designs with repeating experimental points this estimator coincides with the estimator by the method of the reproducing kernel.

Considerations about the relation for the sample size and the degree of the approximation polynomial and about the quadratic mean are given.     

A simulation study of five biased estimators for straight line regression

Five biased estimators of the slope in straight line regression are considered. For each, the estimate of the “bias parameter”, k, is a function of N, the number of observations, and [rcirc]² , the square of the least squares estimate of the standardized slope, β. The estimators include that of Farebrother, the ridge estimator of Hoerl, Kennard, and Baldwin, Vinod's shrunken estimators., and a new modification of one of the latter. Properties of the estimators are studied for 13 combinations of N and 3. Results of simulation experiments provide empirical evidence concerning the values of means and variances of the biased estimators of the slope and estimates of the “bias parameter”, the mean square errors of the estimators, and the frequency of improvement relative to least squares. Adjustments to degrees of freedom in the biased regression analysis of variance table are also considered. An extension of the new modification to the case of p> 1 independent variables is presented in an Appendix.     

Estimates of regression coefficients based on the sign covariance matrix

Summary. A new estimator of the regression parameters is introduced in a multivariate multiple-regression model in which both the vector of explanatory variables and the vector of response variables are assumed to be random. The affine equivariant estimate matrix is constructed using the sign covariance matrix (SCM) where the sign concept is based on Oja's criterion function. The influence function and asymptotic theory are developed to consider robustness and limiting efficiencies of the SCM regression estimate. The estimate is shown to be consistent with a limiting multinormal distribution. The influence function, as a function of the length of the contamination vector, is shown to be linear in elliptic cases; for the least squares (LS) estimate it is quadratic. The asymptotic relative efficiencies with respect to the LS estimate are given in the multivariate normal as well as the t -distribution cases. The SCM regression estimate is highly efficient in the multivariate normal case and, for heavy-tailed distributions, it performs better than the LS estimate. Simulations are used to consider finite sample efficiencies with similar results. The theory is illustrated with an example.     

Testing the equality of correlation matrices

We present results that extend an existing test of equality of correlation matrices. A new test statistic is proposed and is shown to be asymptotically distributed as a linear combination of independent x ² random variables. This new formulation allows us to find the power of the existing test and our extensions by deriving the distribution under the alternative using a linear combination of independent non-central x ² random variables. We also investigate the null and the alternative distribution of two related statistics. The first one is a quadratic form in deviations from a control group with which the remaining k-1 groups are to be compared. The second test is designed for comparing adjacent groups. Several approximations for the null and the alternative distribution are considered and two illustrative examples are provided.     

Singular gauss-markov models

For the general linear model Y = X$sZ + e in which e has a singular dispersion matrix $sG²A, $sG > 0, where A is n x n and singular, Mitra [2] considers the problem of testing F$sZ, where F is a known q x q matrix and claims that the sum of squares (SS) due to hypothesis is not distributed (as a x² variate with degrees of freedom (d. f.) equal to the rank of F) independent of the SS due to error, when a generalized inverse of A is chosen as (A + X'X)^–. This claim does not hold if a pseudo-inverse of A is taken to be (A + X'X)⁺ where A⁺ denotes the unique Moore-Penrose inverse (MPI) of A.     

Instrumental variable estimation in a probit measurement error model

Probit regression is studied when normally distributed covariates are subject to normally distributed measurement errors. Under the assumption that surrogate instrumental variables are available, the parameters in the probit model are shown to be identified. The maximum likelihood estimator and an easily computed two-stage estimator are derived and studied. The two-stage estimator is shown to be asymptotically efficient. Simulation results complement the theory and provide evidence of robustness to the normality assumptions.     

Estimation of the parameters of a general student’s t distribution

We have observations for a t distribution with unknown mean, variance, and degrees of freedom, each of which we wish to estimate. The major problem lies in the estimate of the degrees of freedom. We show that a relatively efficient yet very simple estimator is a given function of the ratio of percentile estimates. We derive the appropriate estimator, provide equations for transformation and standard errors, contrast this with other estimators, and give examples.     

PMSE dominance of the positive-part shrinkage estimator in a regression model with proxy variables

Consider a linear regression model with some relevant regressors are unobservable. In such a situation, we estimate the model by using the proxy variables as regressors or by simply omitting the relevant regressors. In this paper, we derive the explicit formula of predictive mean squared error (PMSE) of a general family of shrinkage estimators of regression coefficients. It is shown analytically that the positive-part shrinkage estimator dominates the ordinary shrinkage estimator even when proxy variables are used in place of the unobserved variables. Also, as an example, our result is applied to the double k-class estimator proposed by Ullah and Ullah (Double k-class estimators of coefficients in linear regression. Econometrica. 1978;46:705–722). Our numerical results show that the positive-part double k-class estimator with proxy variables has preferable PMSE performance.     

Empirical distribution function under heteroscedasticity

Neglecting heteroscedasticity of error terms may imply the wrong identification of a regression model (see appendix). Employment of (heteroscedasticity resistent) White's estimator of covariance matrix of estimates of regression coefficients may lead to the correct decision about the significance of individual explanatory variables under heteroscedasticity. However, White's estimator of covariance matrix was established for least squares (LS)-regression analysis (in the case when error terms are normally distributed, LS- and maximum likelihood (ML)-analysis coincide and hence then White's estimate of covariance matrix is available for ML-regression analysis, tool). To establish White's-type estimate for another estimator of regression coefficients requires Bahadur representation of the estimator in question, under heteroscedasticity of error terms. The derivation of Bahadur representation for other (robust) estimators requires some tools. As the key too proved to be a tight approximation of the empirical distribution function (d.f.) of residuals by the theoretical d.f. of the error terms of the regression model. We need the approximation to be uniform in the argument of d.f. as well as in regression coefficients. The present paper offers this approximation for the situation when the error terms are heteroscedastic.     

Preliminary Test Regression Estimator When two Prior Values of Population Mean (μx) of an Auxiliary Variable (x) are Available

A regression estimator using two prior values of population mean (μ_x) of an auxiliary variable (x) is proposed after a preliminary test of closeness of these prior values to the true valueμ_x. The proposed preliminary test regression estimator has been found to be more efficient in general than the usual regression estimator when prior values are used in place of μ_xwithout preliminary test of significance. The efficiency of the proposed estimator over the usual regression estimator has also been computed for different values of Δ₀, Δ₁, n, and ρ, which showed considerable gain in precision.     

Secondary design considerations for minimum bias estimation

Several authors have suggested the method of minimum bias estimation for estimating response surfaces. The minimum bias estimation procedure achieves minimum average squared bias of the fitted model without depending on the values of the unknown parameters of the true surface. The only requirement is that the design satisfies a simple estimability condition. Subject to providing minimum average squared bias, the minimum bias estimator also provides minimum average variance of ?(x) where ?(x) is the estimate of the response at the point x.

To support the estimation of the parameters in the fitted model, very little has been suggested in the way of experimental designs except to say that a full rank matrix X of independent variables should be used. This paper presents a closer look at the estimability conditions that are required for minimum bias estimation, and from the form of the matrix X, a formula is derived which measures the amount of design flexibility available. The design flexibility is termed “the degrees of freedom” of the X matrix and it is shown how the degrees of freedom can be used to decide if other design optimality criteria might be considered along with minimum bias estimation. Several examples are provided.     

Identifying rational distributed lag models using a joint test applied to the corner table

Often a distributed lag response pattern can be usefully represented in rational polynomial form. When the impulse response function decays, the corner table may be useful for model identification if appropriate statistical tests may be done. One or more joint tests are called for since use of the corner table involves studying groups of its elements. We consider an asymptotic x2 statistic that permits joint tests. We report simulation results showing that the distribution of this statistic follows the x ² distribution, for certain sample sizes and degrees of freedom, well enough to be useful in practice. With two data sets we illustrate how this statistic can be a useful aid when using the corner table.     

On an adjustment of degrees of freedom in the minimim mean squared error ertimator

In this paper, we consider an adjustment of degrees of freedom in the minimum mean squared error (MMSE) estimator, We derive the exact MSE of the adjusted MMSE (AMMSE) estimator, and compare the MSE of the AMMSE estimator with those of the Stein-(SR), positive-part Stein-rule (PSR) and MMSE estimators by numerical evaluations. It is shown that the adjustment of degrees of freedom is effective when the noncentrality parameter is close to zero, and the MSE performance of the MMSE estimator can be improved in the wide region of the noncentrality parameter by the adjustment, ft is also shown that the AMMSE estimator can have the smaller MSE than the PSR estimator in the wide region of the noncentrality parameter     

SHRINKAGE, PRETEST AND ABSOLUTE PENALTY ESTIMATORS IN PARTIALLY LINEAR MODELS

We consider a partially linear model in which the vector of coefficients β in the linear part can be partitioned as ( β ₁, β ₂) , where β ₁ is the coefficient vector for main effects (e.g. treatment effect, genetic effects) and β ₂ is a vector for ‘nuisance’ effects (e.g. age, laboratory). In this situation, inference about β ₁ may benefit from moving the least squares estimate for the full model in the direction of the least squares estimate without the nuisance variables (Steinian shrinkage), or from dropping the nuisance variables if there is evidence that they do not provide useful information (pretesting). We investigate the asymptotic properties of Stein‐type and pretest semiparametric estimators under quadratic loss and show that, under general conditions, a Stein‐type semiparametric estimator improves on the full model conventional semiparametric least squares estimator. The relative performance of the estimators is examined using asymptotic analysis of quadratic risk functions and it is found that the Stein‐type estimator outperforms the full model estimator uniformly. By contrast, the pretest estimator dominates the least squares estimator only in a small part of the parameter space, which is consistent with the theory. We also consider an absolute penalty‐type estimator for partially linear models and give a Monte Carlo simulation comparison of shrinkage, pretest and the absolute penalty‐type estimators. The comparison shows that the shrinkage method performs better than the absolute penalty‐type estimation method when the dimension of the β ₂ parameter space is large.     

半参数纵向模型的惩罚二次推断函数估计

针对纵向数据半参数模型E(y|x,t)=XTβ+f(t),采用惩罚二次推断函数方法同时估计模型中的回归参数β和未知光滑函数f(t)。首先利用截断幂函数基对未知光滑函数进行基函数展开近似,然后利用惩罚样条的思想构造关于回归参数和基函数系数的惩罚二次推断函数,最小化惩罚二次推断函数便可得到回归参数和基函数系数的惩罚二次推断函数估计。理论结果显示,估计结果具有相合性和渐近正态性,通过数值方法也得到了较好的模拟结果。     

Variances of UMVU Estimators for Means and Variances After Using a Normalizing Transformation

Suppose that the function f is of recursive type and the random variable X is normally distributed with mean μ and variance α². We set C = f(x). Neyman & Scott (1960) and Hoyle (1968) gave the UMVU estimators for the mean E(C) and for the variance Var(C) from independent and identically distributed random variables X₁,…, X_n(n ≧ 2) having a normal distribution with mean μ and variance σ², respectively. Shimizu & Iwase (1981) gave the variance of the UMVU estimator for E(C). In this paper, the variance of the UMVU estimator for Var(C) is given.     

On the bounded regret of empirical bayes estimators

Let (θ₁,x₁),…,(θ_n,x_n) be independent and identically distributed random vectors with E(xθ) = θ and Var(x|θ) = a + bθ + cθ². Let t_i be the linear Bayes estimator of θ_i and θ~_i be the linear empirical Bayes estimator of θ_i as proposed in Robbins (1983). When Ex and Var x are unknown to the statistician. The regret of using θ~_i instead of t_i because of ignorance of the mean and the variance is r_i = E(θ_i ? θ_i)² ?E(t_i ?θ_i)². Under appropriate conditions cumulative regret R_n = r₁+…r_n is shown to have a finite limit even when n tends to infinity. The limit can be explicitly computed in terms of a,b,c and the first four moments of x.     

On the relationship between the sample size and the number of variables in a linear regression model

The problem of determining the number of variables to be included in the linear regression model is considered under the assumption that the dependent and independent variables have a joint normal distribution. It is shown that for a given sample size n there exists an optimal number k₀ (0 ≤ k₀ < n-2) of variables among all independent variables in the model, such that the expectation of the mean squared error corresponding to the prediction equation with k₀ variables is minimal.Application of this result to ustepwise procedures is discussed.     

A simple approximation of the sampling distribution of least absolute residuals regression estimates

For multivariate regression with a symmetric disturbance distribution, the error in the least absolute residuals estimator is approximately multivariate normally distributed with mean zero and variance matrix λ²(X′X)^?1, where X is the matrix of K explanatory variables and T observations, and λ ²/T is the variance of the median of a sample of size T from the disturbance distribution. The approximate sampling theory is validated by extensive Monte Carlo studies, and some directions of possible refinement emerge.