A LINEAR MODEL WITH ERRORS LACKING A VARIANCE II1

Numerical results are presented for estimates of the parameters in the linear model Y =βX +ε in which X is normally distributed and ε is symmetric stable. The study complements an earlier paper of the same title and the main concern is with numerical comparisons between four estimates of β; the least squares estimate, the minimum absolute deviations estimate, and two moment estimates of the form derived in Chambers and Heathcote (1975). The generation of fifty independent sets of observations (X_j, Y_j), j = 1,2, …, n for each of n = 100, 500 and selected combinations of parameter values provided the basis of the results. It is indicated that the moment estimators and the minimum absolute deviation estimator performed comparably, and are a significant improvement on the least squares estimator. The main conclusion is that one of the moment estimates, based on a two stage adaptive procedure and denoted by β¯_n(t_a) below, is generally the most useful of the four.     

Estimated Generalized Least Squares for a Heteroscedastic Regression Model

This paper considers the general linear regression model yc = X₁β+u_t under the heteroscedastic structure E(u_t) = 0, E(u²) =σ²- (X_tβ)², E(u_t u_s) = 0, tæs, t, s= 1, T. It is shown that any estimated GLS estimator for β is asymptotically equivalent to the GLS estimator under some regularity conditions. A three-step GLS estimator, which calls upon the assumption E(u_t²) =s?²(X,β)² for the estimation of the disturbance covariance matrix, is considered.     

New difference-based estimator of parameters in semiparametric regression models

The paper introduces a new difference-based Liu estimator β?_Ldiff=([Xtilde]′[Xtilde]+I)^?1([Xtilde]′[ytilde]+η β?_diff) of the regression parameters β in the semiparametric regression model, y=Xβ+f+?. Difference-based estimator, β?_diff=([Xtilde]′[Xtilde])^?1[Xtilde]′[ytilde] and difference-based Liu estimator are analysed and compared with respect to mean-squared error (mse) criterion. Finally, the performance of the new estimator is evaluated for a real data set. Monte Carlo simulation is given to show the improvement in the scalar mse of the estimator.     

The Equality of the Ordinary Least Squares Estimator and the Best Linear Unbiased Estimator

It is well known that the ordinary least squares estimator of Xβ in the general linear model E _y = Xβ, cov y = σ² V, can be the best linear unbiased estimator even if V is not a multiple of the identity matrix. This article presents, in a historical perspective, the development of the several conditions for the ordinary least squares estimator to be best linear unbiased. Various characterizations of these conditions, using generalized inverses and orthogonal projectors, along with several examples, are also given. In addition, a complete set of references is provided.     

SOME ASYMPTOTIC PROPERTIES OF THE SIMPLE LOGLINEAR MODEL

Consistency and asymptotic normality of the maximum likelihood estimator of β in the loglinear model E(y_i) = e^α+βXi, where y_i are independent Poisson observations, 1 iaan, are proved under conditions which are near necessary and sufficient. The asymptotic distribution of the deviance test for β=β₀ is shown to be chi-squared with 1 degree of freedom under the same conditions, and a second order correction to the deviance is derived. The exponential model for censored survival data is also treated by the same methods.     

ROBUST RIDGE REGRESSION BASED ON AN M-ESTIMATOR

Consider the linear regression model y =β₀1 +Xβ+ in the usual notation. It is argued that the class of ordinary ridge estimators obtained by shrinking the least squares estimator by the matrix (X¹X + kI)^-1X'X is sensitive to outliers in the ^variable. To overcome this problem, we propose a new class of ridge-type M-estimators, obtained by shrinking an M-estimator (instead of the least squares estimator) by the same matrix. Since the optimal value of the ridge parameter k is unknown, we suggest a procedure for choosing it adaptively. In a reasonably large scale simulation study with a particular M-estimator, we found that if the conditions are such that the M-estimator is more efficient than the least squares estimator then the corresponding ridge-type M-estimator proposed here is better, in terms of a Mean Squared Error criteria, than the ordinary ridge estimator with k chosen suitably. An example illustrates that the estimators proposed here are less sensitive to outliers in the y-variable than ordinary ridge estimators.     

Efficiency of the generalized difference-based Liu estimators in semiparametric regression models with correlated errors

In this paper, a generalized difference-based estimator is introduced for the vector parameter β in the semiparametric regression model when the errors are correlated. A generalized difference-based Liu estimator is defined for the vector parameter β in the semiparametric regression model. Under the linear nonstochastic constraint Rβ=r, the generalized restricted difference-based Liu estimator is given. The risk function for the β?_GRD(η) associated with weighted balanced loss function is presented. The performance of the proposed estimators is evaluated by a simulated data set.     

Optimality of blue's in a general linear model with incorrect design matrix

We consider the Gauss-Markoff model (Y,X₀β,σ²V) and provide solutions to the following problem: What is the class of all models (Y,Xβ,σ²V) such that a specific linear representation/some linear representation/every linear representation of the BLUE of every estimable parametric functional p'β under (Y,X₀β,σ²V) is (a) an unbiased estimator, (b) a BLUE, (c) a linear minimum bias estimator and (d) best linear minimum bias estimator of p'β under (Y,Xβ,σ²V)? We also analyse the above problems, when attention is restricted to a subclass of estimable parametric functionals.     

Box-Cox transformed linear models: A parameter-based asymptotic approach

A Box-Cox transformed linear model usually has the form y(λ) = μ + β₁x₁ +… + β_px_p + oe, where y(λ) is the power transform of y. Although widely used in practice, the Fisher information matrix for the unknown parameters and, in particular, its inverse have not been studied seriously in the literature. We obtain those two important matrices to put the Box-Cox transformed linear model on a firmer ground. The question of how to make inference on β = (β₁,…,β_p)^T when λ; is estimated from the data is then discussed for large but finite sample size by studying some parameter-based asymptotics. Both unconditional and conditional inference are studied from the frequentist point of view.     

Bias of the lse estimator of the first order autoregressive model under tukey contamination

Results of an exhaustive study of the bias of the least square estimator (LSE) of an first order autoregression coefficient α in a contaminated Gaussian model are presented. The model describes the following situation. The process is defined as X_t = α X_t-1 + Y_t . Until a specified time T, Y_t are iid normal N(0, 1). At the moment T we start our observations and since then the distribution of Y_t, t≥T, is a Tukey mixture T(εσ) = (1 – ε)N(0,1) + εN(0, σ²). Bias of LSE as a function of α and ε, and σ² is considered. A rather unexpected fact is revealed: given α and ε, the bias does not change montonically with σ (“the magnitude of the contaminant”), and similarly, given α and σ, the bias is not growing with ε (“the amount of contaminants”).     

Spectral analysis with replicated time series

A doubly stochastic process {x(b,t);b?B,t?Z} is considered, with (B,β,P_β) being a probability space so that for each b, {X(b,t);t ? Z} is a stationary process with an absolutely continuous spectral distribution. The population spectrum is defined as f(ω) = E_B[Q(b,ω)] with Q(b,ω) being the spectral density function of X(b,t). The aim of this paper is to estimate f(ω) by means of a random sample b₁,…,b_r from (B,β,P_β). For each b₁? B, the processes X(b₁,t) are observed at the same times t=1,…,N. Thus, r time series (x(b₁,t)} are available in order to estimate f(ω). A model for each individual periodogram, which involves f(ω), is formulated. It has been proven that a certain family of linear stationary processes follows the above model In this context, a kernel estimator is proposed in order to estimate f(ω). The bias, variance and asymptotic distribution of this estimator are investigated under certain conditions.     

A linear regression with unobserved dependent variables

Let (?,X) be a random vector such that E(X|?) = ? and Var(x|?) a + b? + c?² for some known constants a, b and c. Assume X₁,…,X_n are independent observations which have the same distribution as X. Let t(X) be the linear regression of ? on X. The linear empirical Bayes estimator is used to approximate the linear regression function. It is shown that under appropriate conditions, the linear empirical Bayes estimator approximates the linear regression well in the sense of mean squared error.     

Testing stability in a spatial unilateral autoregressive model

ABSTRACT

Least squares estimator of the stability parameter ? ? |α| + |β| for a spatial unilateral autoregressive process X_{k, ?} = αX_{k ? 1, ?} + βX_{k, ? ? 1} + ?_{k, ?} is investigated and asymptotic normality with a scaling factor n^5/4 is shown in the unstable case ? = 1. The result is in contrast to the unit root case of the AR(p) model X_k = α₁X_{k ? 1} + ??? + α_pX_{k ? p} + ?_k, where the limiting distribution of the least squares estimator of the unit root parameter ? ? α₁ + ??? + α_p is not normal.     

√n-CONSISTENT ESTIMATION IN A RANDOM COEFFICIENT AUTOREGRESSIVE MODEL

This paper deals with √n-consistent estimation of the parameter μ in the RCAR(l) model defined by the difference equation X_j=(μ+U_j)X_j-l+ej (jε Z), where {e_j: jε Z} and {U_j: jε Z} are two independent sets of i.i.d. random variables with zero means, positive finite variances and E[(μ+U₁)²] < 1. A class of asymptotically normal estimators of μ indexed by a family of bounded measurable functions is introduced. Then an estimator is constructed which is asymptotically equivalent to the best estimator in that class. This estimator, asymptotically equivalent to the quasi-maximum likelihood estimator derived in Nicholls & Quinn (1982), is much simpler to calculate and is asymptotically normal without the additional moment conditions those authors impose.     

A Note on Stagewise Regression

Suppose one estimates the coefficient β₂ in E[Y] = β₀ + β₁ X ₁ + β₂ X ₂ by stagewise regression. That is, first the model E[Y] ≌ β₀ + β₁ X ₁ is fit using simple linear regression followed by a simple linear regression of the residuals from this model on X ₂ to yield the estimator β₂. The ratio of the squared t statistic for the estimate b ₂ from multiple regression to the squared t statistic for β₂ is greater than or equal to 1.0 and is shown to be a convenient function of correlation coefficients among Y, X ₁, and X ₂. Examination of stagewise regression can provide useful insights when introducing concepts of multiple regression.     

OPTIMAL DESIGNS FOR FITTING A PROPORTIONAL HAZARDS REGRESSION MODEL TO DATA SUBJECT TO CENSORING

Suppose the probability model for failure time data, subject to censoring, is specified by the hazard function λ(t)exp(β_T x), where x is a vector of covariates. Analytical difficulties involved in finding the optimal design are avoided by assuming that λ is completely specified and by using D-optimality based on the information matrix for β Optimal designs are found to depend on β, but some results of practical consequence are obtained. It is found that censoring does not affect the choice of design appreciably when β^T x ≥ 0 for all points of the feasible region, but may have an appreciable effect when β_ix_i 0, for all i and all points in the feasible experimental region. The nature of the effect is discussed in detail for the cases of one and two parameters. It is argued that in practical biomedical situations the optimal design is almost always the same as for uncensored data.     

Bootstrap test of goodness of fit to a linear model when errors are correlated

Given the regression model Y_i = m(x_i) +ε_i (x_i ε C, i = l,…,n, C a compact set in R) where m is unknown and the random errors {ε_i} present an ARMA structure, we design a bootstrap method for testing the hypothesis that the regression function follows a general linear model: H_o : m ε {mθ(.) = A^t(.)θ : θ ε ? ? R^q} with A a functional from R to R^q. The criterion of the test derives from a Cramer-von-Mises type functional distance D = d²([mcirc]_n, A^t(.)θ_n), between [mcirc]_n, a Gasser-Miiller non-parametric estimator of m, and the member of the class defined in H_o that is closest to m_n in terms of this distance. The consistency of the bootstrap distribution of D and θ_n is obtained under general conditions. Finally, simulations show the good behavior of the bootstrap approximation with respect to the asymptotic distribution of D = d².     

Quasi-minimax estimation in the partial linear model

This article discusses the minimax estimator in partial linear model y = Zβ + f + ε under ellipsoidal restrictions on the parameter space and quadratic loss function. The superiority of the minimax estimator over the two-step estimator is studied in the mean squared error matrix criterion.     

Expected Absolute Error of the Usual Estimator of the Binomial Parameter

For X with binomial (n, p) distribution the usual measure of the error of X/n as an estimator of p is its standard error S_n(p) = √{E(X/n – p)²} = √{p(1 – p)/n}. A somewhat more natural measure is the average absolute error D_n(p) = E‖X/n – p‖. This article considers use of D_n(p) instead of S_n(p) in a student's first introduction to statistical estimation. Exact and asymptotic values of D_n(p), and the appearance of its graph, are described in detail. The same is done for the Poisson distribution.