Approximate minimax estimation in linear regression: theoretical results

We consider to ordinary linear regression model where the parameter vector ß is constrained to a given ellipsoid. It will be shown that within the class of linear statistics for ß where exists a (sub-)sequence of approximate minimax estimators converging to an exact minimax estimator. This result is valid for an arbitrary quadratic loss function.     

Uniform Representation of Product-Limit Integrals with Applications

Abstract.  Let X be a d -variate random vector that is completely observed, and let Y be a random variable that is subject to right censoring and left truncation. For arbitrary functions φ we consider expectations of the form E [ φ ( X ,  Y )], which appear in many statistical problems, and we estimate these expectations by using a product-limit estimator for censored and truncated data, extended to the context where covariates are present. An almost sure representation for these estimators is obtained, with a remainder term that is of a certain negligible order, uniformly over a class of φ -functions. This uniformity is important for the application to goodness-of-fit testing in regression and to inference for the regression depth, which we consider in more detail.     

Decomposition for multivariate extremal processes

The probability distribution of an extremal process in R^d with independent max-increments is completely determined by its distribution function. The df of an extremal process is similar to the cdf of a random vector. It is a monotone function on (0, ∞) × R^d with values in the interval [0,1]. On the other hand the probability distribution of an extremal process is a probability measure on the space of sample functions. That is the space of all increasing right continuous functions y: (0, ∞) → R^d with the topology of weak convergence. A sequence of extremal processes converges in law if the probability distributions converge weakly. This is shown to be equivalent to weak convergence of the df's.

An extremal process Y: [0, ∞) → R^d is generated by a point process on the space [0, ∞) × [-∞, ∞)^d and has a decomposition Y = X v Z as the maximum of two independent extremal processes with the same lower curve as the original process. The process X is the continuous part and Z contains the fixed discontinuities of the process Y. For a real valued extremal process the decomposition is unique: for a multivariate extremal process uniqueness breaks down due to blotting.     

Testing for Covariate Effect in the Cox Proportional Hazards Regression Model

This article presents methods for testing covariate effect in the Cox proportional hazards model based on Kullback–Leibler divergence and Renyi's information measure. Renyi's measure is referred to as the information divergence of order γ (γ ≠ 1) between two distributions. In the limiting case γ → 1, Renyi's measure becomes Kullback–Leibler divergence. In our case, the distributions correspond to the baseline and one possibly due to a covariate effect. Our proposed statistics are simple transformations of the parameter vector in the Cox proportional hazards model, and are compared with the Wald, likelihood ratio and score tests that are widely used in practice. Finally, the methods are illustrated using two real-life data sets.     

A theorem on invariance of estimability of linear parametric functions in linear models

Consider the general linear model Y = Xβ + ? , where E[??'] = σ²I and rank of X is less than or equal to the number of columns of X. It is well known that the linear parametric function λ'β is estimable if and only if λ' is in the row space of X. This paper characterizes all orthogonal matrices P such that the row space of XP is equal to the row space of X, i.e. the estimability of λ'β is invariant under P. An additional property of these matrices is the invariance of the spectrum of the information matrix X'X. An application of the results is also given.     

Non-parametric Regression with Dependent Censored Data

Abstract.  Let ( X _i , Y _i ) ( i = 1 ,…, n ) be n replications of a random vector ( X , Y  ), where Y is supposed to be subject to random right censoring. The data ( X _i , Y _i ) are assumed to come from a stationary α -mixing process. We consider the problem of estimating the function m ( x ) = E ( φ ( Y ) |  X = x ), for some known transformation φ . This problem is approached in the following way: first, we introduce a transformed variable     , that is not subject to censoring and satisfies the relation     , and then we estimate m ( x ) by applying local linear regression techniques. As a by-product, we obtain a general result on the uniform rate of convergence of kernel type estimators of functionals of an unknown distribution function, under strong mixing assumptions.     

RATES OF CONVERGENCE IN SEMI-PARAMETRIC MODELLING OF LONGITUDINAL DATA

We consider the problem of semi-parametric regression modelling when the data consist of a collection of short time series for which measurements within series are correlated. The objective is to estimate a regression function of the form E[Y(t) | x] =x'ß+μ(t), where μ(.) is an arbitrary, smooth function of time t, and x is a vector of explanatory variables which may or may not vary with t. For the non-parametric part of the estimation we use a kernel estimator with fixed bandwidth h. When h is chosen without reference to the data we give exact expressions for the bias and variance of the estimators for β and μ(t) and an asymptotic analysis of the case in which the number of series tends to infinity whilst the number of measurements per series is held fixed. We also report the results of a small-scale simulation study to indicate the extent to which the theoretical results continue to hold when h is chosen by a data-based cross-validation method.     

Non-parametric Estimation of the Residual Distribution

Consider a heteroscedastic regression model Y = m ( X ) +σ( X )ε, where the functions m and σ are "smooth", and ε is independent of X . An estimator of the distribution of ε based on non-parametric regression residuals is proposed and its weak convergence is obtained. Applications to prediction intervals and goodness-of-fit tests are discussed.     

An alternative proof for a version of cochran's theorem under elliptical settings

For an elliptically contoured n × p random matrix Y with mean μ and covariance proportional to ∑_Y, the necessary and sufficient conditions, under which (Y?μ)′W(Y?μ) with nonnegative definite W is generalized Wishart distributed, are obtained by using the higher moments of Y. This version of Cochran's theorem is general as the assumptions on ∑_Y=A?∑ with nonnegative definite A and ∑, P(Y?μ)=0, and P(Y≠μ)<1 have been relaxed. An example on two way balanced mixed models is given for illustration ot our main results.     

Beta seasonal autoregressive moving average models

In this paper, we introduce the class of beta seasonal autoregressive moving average (βSARMA) models for modelling and forecasting time series data that assume values in the standard unit interval. It generalizes the class of beta autoregressive moving average models [Rocha AV and Cribari-Neto F. Beta autoregressive moving average models. Test. 2009;18(3):529–545] by incorporating seasonal dynamics to the model dynamic structure. Besides introducing the new class of models, we develop parameter estimation, hypothesis testing inference, and diagnostic analysis tools. We also discuss out-of-sample forecasting. In particular, we provide closed-form expressions for the conditional score vector and for the conditional Fisher information matrix. We also evaluate the finite sample performances of conditional maximum likelihood estimators and white noise tests using Monte Carlo simulations. An empirical application is presented and discussed.     

Correlation analysis of ordered symmetrically dependent observations and their concomitants of order statistics

Given two jointly observed random vectors Y and Z of the same dimension, let Y be a reordered version of Y and Z the resulting vector of concomitants of order statistics. When X is a covariate of interest, also jointly observed with Y, the authors obtain the joint covariance structure of (X, y, Z) and the related correlation parameters explicitly, under the assumption that the vector (X, Y, Z) is normal and that its joint covariance structure is permutation symmetric. They also discuss extensions to elliptically contoured distributions.     

Non-parametric tests for distributional treatment effect for randomly censored responses

Summary.  For a binary treatment ν =0, 1 and the corresponding 'potential response' Y ⁰ for the control group ( ν =0) and Y ¹ for the treatment group ( ν =1), one definition of no treatment effect is that Y ⁰ and Y ¹ follow the same distribution given a covariate vector X . Koul and Schick have provided a non-parametric test for no distributional effect when the realized response (1− ν ) Y ⁰+ ν Y ¹ is fully observed and the distribution of X is the same across the two groups. This test is thus not applicable to censored responses, nor to non-experimental (i.e. observational) studies that entail different distributions of X across the two groups. We propose ' X -matched' non-parametric tests generalizing the test of Koul and Schick following an idea of Gehan. Our tests are applicable to non-experimental data with randomly censored responses. In addition to these motivations, the tests have several advantages. First, they have the intuitive appeal of comparing all available pairs across the treatment and control groups, instead of selecting a number of matched controls (or treated) in the usual pair or multiple matching. Second, whereas most matching estimators or tests have a non-overlapping support (of X ) problem across the two groups, our tests have a built-in protection against the problem. Third, Gehan's idea allows the tests to make good use of censored observations. A simulation study is conducted, and an empirical illustration for a job training effect on the duration of unemployment is provided.     

A bayesian approach to detect informative observations in an experiment

In this article Lindley's (1956) measure of average information is used to measure the loss of information due to the unavailability of a set of observations in an experiment. This measure of loss of information may be used to detect a set of most informative observations in a given design.     

An investigation of lag identification tools for vector nonlinear time series

Exploratory methods for determining appropriate lagged vsrlables in a vector nonlinear time series model are investigated. The first is a multivariate extension of the R statistic considered by Granger and Lin (1994), which is based on an estimate of the mutual information criterion. The second method uses Kendall's ρ and partial ρ statistics for lag determination. The methods provide nonlinear analogues of the autocorrelation and partial autocorrelation matrices for a vector time series. Simulation studies indicate that the R statistic reliabiy identifies appropriate lagged nonlinear moving average terms in a vector time series, while Kendall's ρ and partial ρ statistics have some power in identifying appropirate lagged nonlinear moving average and autoregressive terms, respectively, when the nonlinear relationship between lagged variables is monotonic. For illustration, the methods are applied to set of annual temperature and tree ring measurements at Campito Mountain In California.     

Testing for lack of fit in inverse regression—with applications to biophotonic imaging

Summary.  We propose two test statistics for use in inverse regression problems Y = K θ + ɛ , where K is a given linear operator which cannot be continuously inverted. Thus, only noisy, indirect observations Y for the function θ are available. Both test statistics have a counterpart in classical hypothesis testing, where they are called the order selection test and the data-driven Neyman smooth test. We also introduce two model selection criteria which extend the classical Akaike information criterion and Bayes information criterion to inverse regression problems. In a simulation study we show that the inverse order selection and Neyman smooth tests outperform their direct counterparts in many cases. The theory is motivated by data arising in confocal fluorescence microscopy. Here, images are observed with blurring, modelled as convolution, and stochastic error at subsequent times. The aim is then to reduce the signal-to-noise ratio by averaging over the distinct images. In this context it is relevant to decide whether the images are still equal, or have changed by outside influences such as moving of the object table.     

Estimation of parameters in the singular gauss-markoff model

Consider the Gauss-Markoff model (Y, Xβ, σ² V) in the usual notation (Rao, 1973a, p. 294). If V is singular, there exists a matrix N such that N'Y has zero covariance. The minimum variance unbiased estimator of an estimable parametric function p'β is obtained in the wider class of (non-linear) unbiased estimators of the form f(N'Y) + Y'g(N'Y) where f is a scalar and g is a vector function.     

An Efficient Simulation Method for the Computation of a Class of Conditional Expectations

Suppose that the random vector X and the random variable Y are jointly continuous. Also suppose that an observation x of X can be easily simulated and that the probability density function of Y conditional on X = x is known. The paper presents an efficient simulation-based algorithm for estimating E{ g ( X , Y ) | h ( X , Y ) = r } where g and h are real-valued functions. This algorithm is applicable to time series problems in which X = ( X ₁, . . . , X _n−1) and Y = X_n where { x_t } is a discrete time stochastic process for which ( X₁ , . . . , X_n ) is a continuous random vector. A numerical example from time series analysis illustrates the algorithim, for prediction for an ARCH(1) process.     

A note on the asymptotic and exact Fisher information matrices of a Markov switching VARMA process

Statistical Methods & Applications - We study the asymptotic and exact Fisher information (FI) matrices of Markov switching vector autoregressive moving average (MS VARMA) models. In a related...     

Measurement error in regression analysis

Consider the linear regression model Y = Xθ+ ε where Y denotes a vector of n observations on the dependent variable, X is a known matrix, θ is a vector of parameters to be estimated and e is a random vector of uncorrelated errors. If X'X is nearly singular, that is if the smallest characteristic root of X'X s small then a small perurbation in the elements of X, such as due to measurement errors, induces considerable variation in the least squares estimate of θ. In this paper we examine for the asymptotic case when n is large the effect of perturbation with regard to the bias and mean squared error of the estimate.     

Estimating the structural dimension of regressions via parametric inverse regression

A new estimation method for the dimension of a regression at the outset of an analysis is proposed. A linear subspace spanned by projections of the regressor vector X , which contains part or all of the modelling information for the regression of a vector Y on X , and its dimension are estimated via the means of parametric inverse regression. Smooth parametric curves are fitted to the p inverse regressions via a multivariate linear model. No restrictions are placed on the distribution of the regressors. The estimate of the dimension of the regression is based on optimal estimation procedures. A simulation study shows the method to be more powerful than sliced inverse regression in some situations.