An extension of bayesian measure of information to regression

This paper extends Lindley's measure of average information to the linear model, E(Y∣ß) = Xß. An expression which quantifies the average amount of information provided by the nxl vector of observations Y about the pxl vector of coefficient parameters ß will be derived. The effect of the structure of the regressor matrix, X, on the information measure is discussed. An information theoretic optimal design is characterized. Some applications are suggested.     

Singular gauss-markov models

For the general linear model Y = X$sZ + e in which e has a singular dispersion matrix $sG²A, $sG > 0, where A is n x n and singular, Mitra [2] considers the problem of testing F$sZ, where F is a known q x q matrix and claims that the sum of squares (SS) due to hypothesis is not distributed (as a x² variate with degrees of freedom (d. f.) equal to the rank of F) independent of the SS due to error, when a generalized inverse of A is chosen as (A + X'X)^–. This claim does not hold if a pseudo-inverse of A is taken to be (A + X'X)⁺ where A⁺ denotes the unique Moore-Penrose inverse (MPI) of A.     

Robust Liu estimator for regression based on an M-estimator

Consider the regression model y = beta 0 1 + Xbeta + epsilon. Recently, the Liu estimator, which is an alternative biased estimator beta L (d) = (X'X + I) -1 (X'X + dI)beta OLS , where 0<d<1 is a parameter, has been proposed to overcome multicollinearity . The advantage of beta L (d) over the ridge estimator beta R (k) is that beta L (d) is a linear function of d. Therefore, it is easier to choose d than to choose k in the ridge estimator. However, beta L (d) is obtained by shrinking the ordinary least squares (OLS) estimator using the matrix (X'X + I) -1 (X'X + dI) so that the presence of outliers in the y direction may affect the beta L (d) estimator. To cope with this combined problem of multicollinearity and outliers, we propose an alternative class of Liu-type M-estimators (LM-estimators) obtained by shrinking an M-estimator beta M , instead of the OLS estimator using the matrix (X'X + I) -1 (X'X + dI).     

A note on some conditions for optimality of certain ridge estimators

Consider the linear regression model, y = Xβ + ε in the usual notation with X'X being in the correlation form. Galpin(1980) claimed that the ridge estimators of Hoerl, Kennard and Baldwin(1975) and Lawless and Wang(1976) give guaranteed lower mean squared error than the least squares estimator when X'X has at least two very small eigen values. We show that the arguments of Galpin(1980) leading to the above claim are incorrect, and hence the claim itself is unsubstantited. A Monte Carlo study shows that Galpin's claim is not correct in general.     

A simpler way of obtaining non-sIngularity conditions of rotatability

In response surface designs, it is not usually easy to handle the moment matrix X'X, especially for higher orders. This paper presents a method in which the moment matrix of a response surface of any order can be standardized, i.e., X'X splits into a diagonal matrix consisting of sub-matrices of lower order. This eases the calculation of the determinant and the inverse of X'X. The method has been illustrated with applications to second, third and fourth order response surfaces.     

A theorem on invariance of estimability of linear parametric functions in linear models

Consider the general linear model Y = Xβ + ? , where E[??'] = σ²I and rank of X is less than or equal to the number of columns of X. It is well known that the linear parametric function λ'β is estimable if and only if λ' is in the row space of X. This paper characterizes all orthogonal matrices P such that the row space of XP is equal to the row space of X, i.e. the estimability of λ'β is invariant under P. An additional property of these matrices is the invariance of the spectrum of the information matrix X'X. An application of the results is also given.     

Finding maximum G-criterion values for central composite designs on the hypercube

For quadratic regression on the hypercube, G—efficiencies are often used in the selection process of an experimental design. To calculate a design's G—efficiency, it is necessary to maximize the prediction variance over the experimental design region. However, it is common to approximate a G—efficiency. This is achieved by calculating the prediction variances generated from a subset of points in the design space and taking the maximum to estimate the maximum prediction variance. This estimate is then applied to approximate the G—efficiency. In this paper, it will be shown that over the class of central composite designs (CCDs) on the hypercube. the prediction variance can be expressed in a closed-form. An exact value of the maximum prediction variance can then be determined by evaluating this closed-form expression over a finite subset of barycentric points. Tables of exact G—efficiencies will be presented. Design optimality criteria, quadratic regression on the hypercube, and the structures of the design matrix X, X'X, and (X'X)^?1 for any CCD will be discussed.     

Non-parametric Regression with Dependent Censored Data

Abstract.  Let ( X _i , Y _i ) ( i = 1 ,…, n ) be n replications of a random vector ( X , Y  ), where Y is supposed to be subject to random right censoring. The data ( X _i , Y _i ) are assumed to come from a stationary α -mixing process. We consider the problem of estimating the function m ( x ) = E ( φ ( Y ) |  X = x ), for some known transformation φ . This problem is approached in the following way: first, we introduce a transformed variable     , that is not subject to censoring and satisfies the relation     , and then we estimate m ( x ) by applying local linear regression techniques. As a by-product, we obtain a general result on the uniform rate of convergence of kernel type estimators of functionals of an unknown distribution function, under strong mixing assumptions.     

The Mean Squared Error of the Generalized Ridge Regression Estimator and the Orientation of β

Newhouse and Oman (1971) identified the orientations with respect to the eigenvectors of X'X of the true coefficient vector of the linear regression model for which the ordinary ridge regression estimator performs best and performs worse when mean squared error is the measure of performance. In this paper the corresponding result is derived for generalized ridge regression for two risk functions: mean squared error and mean squared error of prediction.     

Linear model estimation errors with estimated design parameters

The problem of error estimation of parameters b in a linear model,Y = Xb+ e, is considered when the elements of the design matrix X are functions of an unknown ‘design’ parameter vector c. An estimated value c is substituted in X to obtain a derived design matrix [Xtilde]. Even though the usual linear model conditions are not satisfied with [Xtilde], there are situations in physical applications where the least squares solution to the parameters is used without concern for the magnitude of the resulting error. Such a solution can suffer from serious errors.

This paper examines bias and covariance errors of such estimators. Using a first-order Taylor series expansion, we derive approximations to the bias and covariance matrix of the estimated parameters. The bias approximation is a sum of two terms:One is due to the dependence between ? and Y; the other is due to the estimation errors of ? and is proportional to b, the parameter being estimated. The covariance matrix approximation, on the other hand, is composed of three omponents:One component is due to the dependence between ? and Y; the second is the covariance matrix ∑_b corresponding to the minimum variance unbiased b, as if the design parameters were known without error; and the third is an additional component due to the errors in the design parameters. It is shown that the third error component is directly proportional to bb'. Thus, estimation of large parameters with wrong design matrix [Xtilde] will have larger errors of estimation. The results are illustrated with a simple linear example.     

On Censored Bivariate Random Variables: Copula,Characterization, and Estimation

Let (X, Y) be a bivariate random vector with joint distribution function F_{X, Y}(x, y) = C(F(x), G(y)), where C is a copula and F and G are marginal distributions of X and Y, respectively. Suppose that (X_i, Y_i), i = 1, 2, …, n is a random sample from (X, Y) but we are able to observe only the data consisting of those pairs (X_i, Y_i) for which X_i ? Y_i. We denote such pairs as (X*_i, Y_i*), i = 1, 2, …, ν, where ν is a random variable. The main problem of interest is to express the distribution function F_{X, Y}(x, y) and marginal distributions F and G with the distribution function of observed random variables X* and Y*. It is shown that if X and Y are exchangeable with marginal distribution function F, then F can be uniquely determined by the distributions of X* and Y*. It is also shown that if X and Y are independent and absolutely continuous, then F and G can be expressed through the distribution functions of X* and Y* and the stress–strength reliability P{X ? Y}. This allows also to estimate P{X ? Y} with the truncated observations (X*_i, Y_i*). The copula of bivariate random vector (X*, Y*) is also derived.     

Estimation of parameters in the singular gauss-markoff model

Consider the Gauss-Markoff model (Y, Xβ, σ² V) in the usual notation (Rao, 1973a, p. 294). If V is singular, there exists a matrix N such that N'Y has zero covariance. The minimum variance unbiased estimator of an estimable parametric function p'β is obtained in the wider class of (non-linear) unbiased estimators of the form f(N'Y) + Y'g(N'Y) where f is a scalar and g is a vector function.     

Computation of the central and noncentral f distributions

Recursion relations suitable for rapid computation are derived for the cumulative distribution of F′ = (X/m)/(Y/n) where X is χ²(λ, m) and Y is independently χ²(n). When n is even no complicated function evaluations are needed. For n odd, a special doubly noncentral t distribution is needed to start the computation. Series representations for this t distribution are given with rigorous bounds on truncation errors. Proper recursion techniques for numerical evaluation of the special functions are given.     

ESTIMATION OF THE PARAMETER OF A POISSON DISTRIBUTION USING A LINEX LOSS FUNCTION

This paper considers estimation of the parameter of a Poisson distribution using Varian's (1975) asymmetric LINEX loss function L (δ) = b{exp(aδ) - aδ - 1}, where δ is the estimation error and b > 0, a 0. It is shown that for a < 0, the sample mean X¯ is admissible whereas for a > 0, X¯ is dominated by c*X¯, where c*= (n/a)log(1+a/n). Practical implications of this result are indicated. More general results, concerning the admissibility of estimators of the form cX¯+ d are also presented.     

Joint analysis of longitudinal data comprising repeated measures and times to events

In biomedical and public health research, both repeated measures of biomarkers Y as well as times T to key clinical events are often collected for a subject. The scientific question is how the distribution of the responses [ T , Y | X ] changes with covariates X . [ T | X ] may be the focus of the estimation where Y can be used as a surrogate for T . Alternatively, T may be the time to drop-out in a study in which [ Y | X ] is the target for estimation. Also, the focus of a study might be on the effects of covariates X on both T and Y or on some underlying latent variable which is thought to be manifested in the observable outcomes. In this paper, we present a general model for the joint analysis of [ T , Y | X ] and apply the model to estimate [ T | X ] and other related functionals by using the relevant information in both T and Y . We adopt a latent variable formulation like that of Fawcett and Thomas and use it to estimate several quantities of clinical relevance to determine the efficacy of a treatment in a clinical trial setting. We use a Markov chain Monte Carlo algorithm to estimate the model's parameters. We illustrate the methodology with an analysis of data from a clinical trial comparing risperidone with a placebo for the treatment of schizophrenia.     

Uniform Representation of Product-Limit Integrals with Applications

Abstract.  Let X be a d -variate random vector that is completely observed, and let Y be a random variable that is subject to right censoring and left truncation. For arbitrary functions φ we consider expectations of the form E [ φ ( X ,  Y )], which appear in many statistical problems, and we estimate these expectations by using a product-limit estimator for censored and truncated data, extended to the context where covariates are present. An almost sure representation for these estimators is obtained, with a remainder term that is of a certain negligible order, uniformly over a class of φ -functions. This uniformity is important for the application to goodness-of-fit testing in regression and to inference for the regression depth, which we consider in more detail.     

Estimating the structural dimension of regressions via parametric inverse regression

A new estimation method for the dimension of a regression at the outset of an analysis is proposed. A linear subspace spanned by projections of the regressor vector X , which contains part or all of the modelling information for the regression of a vector Y on X , and its dimension are estimated via the means of parametric inverse regression. Smooth parametric curves are fitted to the p inverse regressions via a multivariate linear model. No restrictions are placed on the distribution of the regressors. The estimate of the dimension of the regression is based on optimal estimation procedures. A simulation study shows the method to be more powerful than sliced inverse regression in some situations.     

An Efficient Simulation Method for the Computation of a Class of Conditional Expectations

Suppose that the random vector X and the random variable Y are jointly continuous. Also suppose that an observation x of X can be easily simulated and that the probability density function of Y conditional on X = x is known. The paper presents an efficient simulation-based algorithm for estimating E{ g ( X , Y ) | h ( X , Y ) = r } where g and h are real-valued functions. This algorithm is applicable to time series problems in which X = ( X ₁, . . . , X _n−1) and Y = X_n where { x_t } is a discrete time stochastic process for which ( X₁ , . . . , X_n ) is a continuous random vector. A numerical example from time series analysis illustrates the algorithim, for prediction for an ARCH(1) process.     

Questions & Answers: The Role of Mathematical Models in Clinical Research

Given any generalized inverse (X'X)^? appropriate to normal equations X'Xb ⁰ = X'y for the linear model y = Xb + e, a procedure is given for obtaining from it a generalized inverse appropriate to a restricted model having restrictions P'b = 0 for P'b nonestimable.     

Non-linear canonical correlation analysis with a simulated annealing solution

An exploratory tool is introduced to examine potential non-linear relation-ships between two sets of variables, X andY, in a sample of multivariate data. Simulated annealing is applied to find canonical coefficient vectors a and b such that a squared non-linear correlation between a'Xand b'Y is maximiSed. A measure of non-linear correlation is developed for this optimization which utilies a nearest-neighbor regression estimate for the unknown functional relationship. In addition to examining potential relations between the canonical variables, this method can identify the important variables in each set.