Quasi-minimax estimation in the general linear regression model

This paper is mainly concerned with minimax estimation in the general linear regression model y=Xβ+ε$y=X\beta +\epsilon$ under ellipsoidal restrictions on the parameter space and quadratic loss function. We confine ourselves to estimators that are linear in the response vector y  . The minimax estimators of the regression coefficient β$\beta$ are derived under homogeneous condition and heterogeneous condition, respectively. Furthermore, these obtained estimators are the ridge-type estimators and mean dispersion error (MDE) superior to the best linear unbiased estimator b=(X^′W^-1X)^-1X^′W^-1y$b=(X^{\prime }{W}^{-1}X{)}^{-1}{X}^{\prime }{W}^{-1}y$ under some conditions.     

Generalized minimum distance estimators of a linear model with correlated errors

R^p of a linear regression model of the type Y = Xθ + ɛ, where X is the design matrix, Y the vector of the response variable and ɛ the random error vector that follows an AR(1) correlation structure. These estimators are asymptotically analyzed, by proving their strong consistency, asymptotic normality and asymptotic efficiency. In a simulation study, a better behaviour of the Mean Squared Error of the proposed estimator with respect to that of the generalized least squares estimators is observed. Received: November 16, 1998; revised version: May 10, 2000     

General linear estimators under the prediction error sum of squares criterion in a linear regression model

In this paper, the notion of the general linear estimator and its modified version are introduced using the singular value decomposition theorem in the linear regression model y=X β+e to improve some classical linear estimators. The optimal selections of the biasing parameters involved are theoretically given under the prediction error sum of squares criterion. A numerical example and a simulation study are finally conducted to illustrate the superiority of the proposed estimators.     

Improved ridge estimators in a linear regression model

In this paper, the notion of the improved ridge estimator (IRE) is put forward in the linear regression model y=X β+e. The problem arises if augmenting the equation 0=c′α+ε instead of 0=C α+? to the model. Three special IREs are considered and studied under the mean-squared error criterion and the prediction error sum of squares criterion. The simulations demonstrate that the proposed estimators are effective and recommendable, especially when multicollinearity is severe.     

Regression model with elliptically contoured errors

For the regression model y=X β+ε where the errors follow the elliptically contoured distribution, we consider the least squares, restricted least squares, preliminary test, Stein-type shrinkage and positive-rule shrinkage estimators for the regression parameters, β.

We compare the quadratic risks of the estimators to determine the relative dominance properties of the five estimators.     

Confidence Intervals for Nonparametric Regression Functions with Missing Data

Suppose that we have a nonparametric regression model Y = m(X) + ε with X ∈ R^p, where X is a random design variable and is observed completely, and Y is the response variable and some Y-values are missing at random. Based on the “complete” data sets for Y after nonaprametric regression imputation and inverse probability weighted imputation, two estimators of the regression function m(x₀) for fixed x₀ ∈ R^p are proposed. Asymptotic normality of two estimators is established, which is used to construct normal approximation-based confidence intervals for m(x₀). We also construct an empirical likelihood (EL) statistic for m(x₀) with limiting distribution of χ²₁, which is used to construct an EL confidence interval for m(x₀).     

ROBUST RIDGE REGRESSION BASED ON AN M-ESTIMATOR

Consider the linear regression model y =β₀1 +Xβ+ in the usual notation. It is argued that the class of ordinary ridge estimators obtained by shrinking the least squares estimator by the matrix (X¹X + kI)^-1X'X is sensitive to outliers in the ^variable. To overcome this problem, we propose a new class of ridge-type M-estimators, obtained by shrinking an M-estimator (instead of the least squares estimator) by the same matrix. Since the optimal value of the ridge parameter k is unknown, we suggest a procedure for choosing it adaptively. In a reasonably large scale simulation study with a particular M-estimator, we found that if the conditions are such that the M-estimator is more efficient than the least squares estimator then the corresponding ridge-type M-estimator proposed here is better, in terms of a Mean Squared Error criteria, than the ordinary ridge estimator with k chosen suitably. An example illustrates that the estimators proposed here are less sensitive to outliers in the y-variable than ordinary ridge estimators.     

The Gauss—Markov Theorem and Random Regressors

In the standard linear regression model with independent, homoscedastic errors, the Gauss—Markov theorem asserts that = (X'X)^-1(X'y) is the best linear unbiased estimator of β and, furthermore, that is the best linear unbiased estimator of c'β for all p × 1 vectors c. In the corresponding random regressor model, X is a random sample of size n from a p-variate distribution. If attention is restricted to linear estimators of c'β that are conditionally unbiased, given X, the Gauss—Markov theorem applies. If, however, the estimator is required only to be unconditionally unbiased, the Gauss—Markov theorem may or may not hold, depending on what is known about the distribution of X. The results generalize to the case in which X is a random sample without replacement from a finite population.     

Estimation in Partially Linear Single-Index Models with Missing Covariates

In this article, we consider a partially linear single-index model Y = g(Z ^τθ₀) + X ^τβ₀ + ? when the covariate X may be missing at random. We propose weighted estimators for the unknown parametric and nonparametric part by applying weighted estimating equations. We establish normality of the estimators of the parameters and asymptotic expansion for the estimator of the nonparametric part when the selection probabilities are unknown. Simulation studies are also conducted to illustrate the finite sample properties of these estimators.     

Quantile inference for heteroscedastic regression models

Consider the nonparametric heteroscedastic regression model Y=m(X)+σ(X)?, where m(·) is an unknown conditional mean function and σ(·) is an unknown conditional scale function. In this paper, the limit distribution of the quantile estimate for the scale function σ(X) is derived. Since the limit distribution depends on the unknown density of the errors, an empirical likelihood ratio statistic based on quantile estimator is proposed. This statistics is used to construct confidence intervals for the variance function. Under certain regularity conditions, it is shown that the quantile estimate of the scale function converges to a Brownian motion and the empirical likelihood ratio statistic converges to a chi-squared random variable. Simulation results demonstrate the superiority of the proposed method over the least squares procedure when the underlying errors have heavy tails.     

New shrinkage-type estimators in a linear regression model when multicollinearity is severe

For the linear regression model y=Xβ+e with severe multicollinearity, we put forward three shrinkage-type estimators based on the ordinary least-squares estimator including two types of independent factor estimators and a seemingly convex combination. The simulation study shows that the new estimators are not good enough when multicollinearity is mild to moderate, but perform very well when multicollinearity is severe to very severe.     

Regularized robust estimation in binary regression models

In this paper, we investigate robust parameter estimation and variable selection for binary regression models with grouped data. We investigate estimation procedures based on the minimum-distance approach. In particular, we employ minimum Hellinger and minimum symmetric chi-squared distances criteria and propose regularized minimum-distance estimators. These estimators appear to possess a certain degree of automatic robustness against model misspecification and/or for potential outliers. We show that the proposed non-penalized and penalized minimum-distance estimators are efficient under the model and simultaneously have excellent robustness properties. We study their asymptotic properties such as consistency, asymptotic normality and oracle properties. Using Monte Carlo studies, we examine the small-sample and robustness properties of the proposed estimators and compare them with traditional likelihood estimators. We also study two real-data applications to illustrate our methods. The numerical studies indicate the satisfactory finite-sample performance of our procedures.     

Improving the estimators of the parameters of a probit regression model: A ridge regression approach

This paper considered the estimation of the regression parameters of a general probit regression model. Accordingly, we proposed five ridge regression (RR) estimators for the probit regression models for estimating the parameters (β)$(\beta )$ when the weighted design matrix is ill-conditioned and it is suspected that the parameter β$\beta$ may belong to a linear subspace defined by Hβ=h$H\beta =h$. Asymptotic properties of the estimators are studied with respect to quadratic biases, MSE matrices and quadratic risks. The regions of optimality of the proposed estimators are determined based on the quadratic risks. Some relative efficiency tables and risk graphs are provided to illustrate the numerical comparison of the estimators. We conclude that when q≥3$q\ge 3$, one would uses PRRRE; otherwise one uses PTRRE with some optimum size α$\alpha$. We also discuss the performance of the proposed estimators compare to the alternative ridge regression method due to Liu (1993).     

Semiparametric partially linear regression models for functional data

In the context of longitudinal data analysis, a random function typically represents a subject that is often observed at a small number of time point. For discarding this restricted condition of observation number of each subject, we consider the semiparametric partially linear regression models with mean function x^?β$x^{?}\beta$ + g(z), where x and z   are functional data. The estimations of β$\beta$ and g(z) are presented and some asymptotic results are given. It is shown that the estimator of the parametric component is asymptotically normal. The convergence rate of the estimator of the nonparametric component is also obtained. Here, the observation number of each subject is completely flexible. Some simulation study is conducted to investigate the finite sample performance of the proposed estimators.     

JAMES-STEIN RULE ESTIMATORS IN LINEAR REGRESSION MODELS WITH MULTIVARIATE-t DISTRIBUTED ERROR

This paper considers estimation of β in the regression model y =Xβ+μ, where the error components in μ have the jointly multivariate Student-t distribution. A family of James-Stein type estimators (characterised by nonstochastic scalars) is presented. Sufficient conditions involving only X are given, under which these estimators are better (with respect to the risk under a general quadratic loss function) than the usual minimum variance unbiased estimator (MVUE) of β. Approximate expressions for the bias, the risk, the mean square error matrix and the variance-covariance matrix for the estimators in this family are obtained. A necessary and sufficient condition for the dominance of this family over MVUE is also given.     

Improved Liu estimator in a linear regression model

In this paper, we mainly aim to introduce the notion of improved Liu estimator (ILE) in the linear regression model y=Xβ+e. The selection of the biasing parameters is investigated under the PRESS criterion and the optimal selection is successfully derived. We make a simulation study to show the performance of ILE compared to the ordinary least squares estimator and the Liu estimator. Finally, the main results are applied to the Hald data.     

M-Estimation for partially functional linear regression model based on splines

ABSTRACT

M-estimation is a widely used technique for robust statistical inference. In this paper, we study robust partially functional linear regression model in which a scale response variable is explained by a function-valued variable and a finite number of real-valued variables. For the estimation of the regression parameters, which include the infinite dimensional function as well as the slope parameters for the real-valued variables, we use polynomial splines to approximate the slop parameter. The estimation procedure is easy to implement, and it is resistant to heavy-tailederrors or outliers in the response. The asymptotic properties of the proposed estimators are established. Finally, we assess the finite sample performance of the proposed method by Monte Carlo simulation studies.     

Pseudo‐R2 statistics under complex sampling

Model summaries based on the ratio of fitted and null likelihoods have been proposed for generalised linear models, reducing to the familiar R² coefficient of determination in the Gaussian model with identity link. In this note I show how to define the Cox–Snell and Nagelkerke summaries under arbitrary probability sampling designs, giving a design‐consistent estimator of the population model summary. It is also shown that for logistic regression models under case–control sampling the usual Cox–Snell and Nagelkerke R² are not design‐consistent, but are systematically larger than would be obtained with a cross‐sectional or cohort sample from the same population, even in settings where the weighted and unweighted logistic regression estimators are similar or identical. Implementation of the new estimators is straightforward and code is provided in R.