Robust Liu estimator for regression based on an M-estimator

Consider the regression model y = beta 0 1 + Xbeta + epsilon. Recently, the Liu estimator, which is an alternative biased estimator beta L (d) = (X'X + I) -1 (X'X + dI)beta OLS , where 0<d<1 is a parameter, has been proposed to overcome multicollinearity . The advantage of beta L (d) over the ridge estimator beta R (k) is that beta L (d) is a linear function of d. Therefore, it is easier to choose d than to choose k in the ridge estimator. However, beta L (d) is obtained by shrinking the ordinary least squares (OLS) estimator using the matrix (X'X + I) -1 (X'X + dI) so that the presence of outliers in the y direction may affect the beta L (d) estimator. To cope with this combined problem of multicollinearity and outliers, we propose an alternative class of Liu-type M-estimators (LM-estimators) obtained by shrinking an M-estimator beta M , instead of the OLS estimator using the matrix (X'X + I) -1 (X'X + dI).     

Singular gauss-markov models

For the general linear model Y = X$sZ + e in which e has a singular dispersion matrix $sG²A, $sG > 0, where A is n x n and singular, Mitra [2] considers the problem of testing F$sZ, where F is a known q x q matrix and claims that the sum of squares (SS) due to hypothesis is not distributed (as a x² variate with degrees of freedom (d. f.) equal to the rank of F) independent of the SS due to error, when a generalized inverse of A is chosen as (A + X'X)^–. This claim does not hold if a pseudo-inverse of A is taken to be (A + X'X)⁺ where A⁺ denotes the unique Moore-Penrose inverse (MPI) of A.     

Measurement error in regression analysis

Consider the linear regression model Y = Xθ+ ε where Y denotes a vector of n observations on the dependent variable, X is a known matrix, θ is a vector of parameters to be estimated and e is a random vector of uncorrelated errors. If X'X is nearly singular, that is if the smallest characteristic root of X'X s small then a small perurbation in the elements of X, such as due to measurement errors, induces considerable variation in the least squares estimate of θ. In this paper we examine for the asymptotic case when n is large the effect of perturbation with regard to the bias and mean squared error of the estimate.     

A simpler way of obtaining non-sIngularity conditions of rotatability

In response surface designs, it is not usually easy to handle the moment matrix X'X, especially for higher orders. This paper presents a method in which the moment matrix of a response surface of any order can be standardized, i.e., X'X splits into a diagonal matrix consisting of sub-matrices of lower order. This eases the calculation of the determinant and the inverse of X'X. The method has been illustrated with applications to second, third and fourth order response surfaces.     

Finding maximum G-criterion values for central composite designs on the hypercube

For quadratic regression on the hypercube, G—efficiencies are often used in the selection process of an experimental design. To calculate a design's G—efficiency, it is necessary to maximize the prediction variance over the experimental design region. However, it is common to approximate a G—efficiency. This is achieved by calculating the prediction variances generated from a subset of points in the design space and taking the maximum to estimate the maximum prediction variance. This estimate is then applied to approximate the G—efficiency. In this paper, it will be shown that over the class of central composite designs (CCDs) on the hypercube. the prediction variance can be expressed in a closed-form. An exact value of the maximum prediction variance can then be determined by evaluating this closed-form expression over a finite subset of barycentric points. Tables of exact G—efficiencies will be presented. Design optimality criteria, quadratic regression on the hypercube, and the structures of the design matrix X, X'X, and (X'X)^?1 for any CCD will be discussed.     

A theorem on invariance of estimability of linear parametric functions in linear models

Consider the general linear model Y = Xβ + ? , where E[??'] = σ²I and rank of X is less than or equal to the number of columns of X. It is well known that the linear parametric function λ'β is estimable if and only if λ' is in the row space of X. This paper characterizes all orthogonal matrices P such that the row space of XP is equal to the row space of X, i.e. the estimability of λ'β is invariant under P. An additional property of these matrices is the invariance of the spectrum of the information matrix X'X. An application of the results is also given.     

ROBUST RIDGE REGRESSION BASED ON AN M-ESTIMATOR

Consider the linear regression model y =β₀1 +Xβ+ in the usual notation. It is argued that the class of ordinary ridge estimators obtained by shrinking the least squares estimator by the matrix (X¹X + kI)^-1X'X is sensitive to outliers in the ^variable. To overcome this problem, we propose a new class of ridge-type M-estimators, obtained by shrinking an M-estimator (instead of the least squares estimator) by the same matrix. Since the optimal value of the ridge parameter k is unknown, we suggest a procedure for choosing it adaptively. In a reasonably large scale simulation study with a particular M-estimator, we found that if the conditions are such that the M-estimator is more efficient than the least squares estimator then the corresponding ridge-type M-estimator proposed here is better, in terms of a Mean Squared Error criteria, than the ordinary ridge estimator with k chosen suitably. An example illustrates that the estimators proposed here are less sensitive to outliers in the y-variable than ordinary ridge estimators.     

The Mean Squared Error of the Generalized Ridge Regression Estimator and the Orientation of β

Newhouse and Oman (1971) identified the orientations with respect to the eigenvectors of X'X of the true coefficient vector of the linear regression model for which the ordinary ridge regression estimator performs best and performs worse when mean squared error is the measure of performance. In this paper the corresponding result is derived for generalized ridge regression for two risk functions: mean squared error and mean squared error of prediction.     

IN JUSTIFICATION OF WALD'S CRITERION OF EFFICIENCY OF EXPERIMENTAL DESIGNS IN RELATION TO THE WEIGHING PROBLEM1

Wald (1943), among others, examined the question of efficiency of experimental designs in general and suggested the criterion of maximization of the det. [X'X] for the purpose, where X denotes the design matrix. In the same context, Ehrenfeld (1955) recommended the maximization of the minimum eigenvalue of [X'X] as the desideratum. It has been shown in this note that for weighing designs in general maximization of det. [X'X] may be preferred to the latter criterion.     

ESTIMATION OF THE PARAMETER OF A POISSON DISTRIBUTION USING A LINEX LOSS FUNCTION

This paper considers estimation of the parameter of a Poisson distribution using Varian's (1975) asymmetric LINEX loss function L (δ) = b{exp(aδ) - aδ - 1}, where δ is the estimation error and b > 0, a 0. It is shown that for a < 0, the sample mean X¯ is admissible whereas for a > 0, X¯ is dominated by c*X¯, where c*= (n/a)log(1+a/n). Practical implications of this result are indicated. More general results, concerning the admissibility of estimators of the form cX¯+ d are also presented.     

Questions & Answers: The Role of Mathematical Models in Clinical Research

Given any generalized inverse (X'X)^? appropriate to normal equations X'Xb ⁰ = X'y for the linear model y = Xb + e, a procedure is given for obtaining from it a generalized inverse appropriate to a restricted model having restrictions P'b = 0 for P'b nonestimable.     

An extension of bayesian measure of information to regression

This paper extends Lindley's measure of average information to the linear model, E(Y∣ß) = Xß. An expression which quantifies the average amount of information provided by the nxl vector of observations Y about the pxl vector of coefficient parameters ß will be derived. The effect of the structure of the regressor matrix, X, on the information measure is discussed. An information theoretic optimal design is characterized. Some applications are suggested.     

Two‐stage model selection procedures in partially linear regression

The authors propose a two‐stage estimation procedure for the partially linear model Y = f_o(T) + X'β_o + ψ. They show how to estimate consistently the location of the nonzero components of β_o. Their approach turns out to be compatible with minimax adaptive estimation of f_o over Besov balls in the case of penalized least squares. Their proofs are based on a new type of oracle inequality.     

A closed form solution for tre least squares regression problem with linear inequality constraints

In this paper we consider a linear model Y = Xβ+e with linear inequality constraints R'β≥r, where X and R are known and full column rank matrices. The closed form of the inequality constrained least squares (ICLS) estimator is given. We provide two examples which illustrate the use of this closed form in the computation of estimates.     

Robustness of the EWMA and the combined X¯–EWMA control charts with variable sampling intervals to non-normality

The exponentially weighted moving average (EWMA) control charts with variable sampling intervals (VSIs) have been shown to be substantially quicker than the fixed sampling intervals (FSI) EWMA control charts in detecting process mean shifts. The usual assumption for designing a control chart is that the data or measurements are normally distributed. However, this assumption may not be true for some processes. In the present paper, the performances of the EWMA and combined X¯–EWMA control charts with VSIs are evaluated under non-normality. It is shown that adding the VSI feature to the EWMA control charts results in very substantial decreases in the expected time to detect shifts in process mean under both normality and non-normality. However, the combined X¯–EWMA chart has its false alarm rate and its detection ability is affected if the process data are not normally distributed.     

The best subset of parameters in min-max linear regression

These Fortran-77 subroutines provide building blocks for Generalized Cross-Validation (GCV) (Craven and Wahba, 1979) calculations in data analysis and data smoothing including ridge regression (Golub, Heath, and Wahba, 1979), thin plate smoothing splines (Wahba and Wendelberger, 1980), deconvolution (Wahba, 1982d), smoothing of generalized linear models (O'sullivan, Yandell and Raynor 1986, Green 1984 and Green and Yandell 1985), and ill-posed problems (Nychka et al., 1984, O'sullivan and Wahba, 1985). We present some of the types of problems for which GCV is a useful method of choosing a smoothing or regularization parameter and we describe the structure of the subroutines.Ridge Regression: A familiar example of a smoothing parameter is the ridge parameter X in the ridge regression problem which we write.     

The Gauss—Markov Theorem and Random Regressors

In the standard linear regression model with independent, homoscedastic errors, the Gauss—Markov theorem asserts that = (X'X)^-1(X'y) is the best linear unbiased estimator of β and, furthermore, that is the best linear unbiased estimator of c'β for all p × 1 vectors c. In the corresponding random regressor model, X is a random sample of size n from a p-variate distribution. If attention is restricted to linear estimators of c'β that are conditionally unbiased, given X, the Gauss—Markov theorem applies. If, however, the estimator is required only to be unconditionally unbiased, the Gauss—Markov theorem may or may not hold, depending on what is known about the distribution of X. The results generalize to the case in which X is a random sample without replacement from a finite population.     

Effect of measurement error and autocorrelation on the X¯ chart

Measurement error and autocorrelation often exist in quality control applications. Both have an adverse effect on the X¯ chart's performance. To counteract the undesired effect of autocorrelation, we build-up the samples with non-neighbouring items, according to the time they were produced. To counteract the undesired effect of measurement error, we measure the quality characteristic of each item of the sample several times. The chart's performance is assessed when multiple measurements are applied and the samples are built by taking one item from the production line and skipping one, two or more before selecting the next.     

Finite time ruin probability and structural density properties in the presence of dependence in insurance risk model

In the present paper, we consider the classical compound Poisson risk model with dependence between claim sizes and claim inter-arrival time. We attempt to analyze the approximation of finite time ruin probability. The finite time ruin probabilities are plotted for fixed threshold value associated to the claim inter-arrival time and also for fixed dependence parameter in Nelsen (2006) copula separately. Additionally, a general form for joint density of the interclaim times and claim sizes is considered. With respect to the classical Gerber-Shiu's (1998) function, first some structural density properties of dependent collective risk model is obtained. Then the ladder height probability density function of claim sizes is computed and the dependency structure investigated for Erlang interclaim time. As the application, some dependent models of the interclaim times and claim sizes are studied.     

Selection of regress or va riables when E(Y) is an unknown honlinear function

We consider the problem of deciding which of a set of p independent variables x₁ X₂J x_s we are to regard as being functionally involved in the mean of a dependent normal random variable Y and estimating E( Y) in terms of the chosen x's. This mean is an unknown function (assumed to be doubly differentiable) of some or all of the x's, so that the problem is of wide relevance. We approximate to the hypersurface in two different ways, and select within each approximation:

(a)For the situation where the mean of Y is assumed to be a linear function of the x's, we use ono of the optimum methods of selection.

(b)More generally, in the space of the X's the function will be approximately linear in a relatively small region. Accordingly this p-dimensional space is subdivided into smaller regions by a clustering procedure, and a hyperplane if fitted with in each region to aproximate to the unknown responce surface.An adaption of an optimum-regressor-selection procedure is then used to assist in the selection of the regressors

Approximate F tests are given to choose between models, including deciding how many x's to retain. Alternatively: the application of Akaike's Extended Maximum Likelihood Principle provides another way of choosing between the models and of selecting regressor variables. The methods are applied to data on glass manufacture.