Stepwise Regression in Mixed Quantitative Linear Models with Autocorrelated Errors

ABSTRACT

In the stepwise procedure of selection of a fixed or a random explanatory variable in a mixed quantitative linear model with errors following a Gaussian stationary autocorrelated process, we have studied the efficiency of five estimators relative to Generalized Least Squares (GLS): Ordinary Least Squares (OLS), Maximum Likelihood (ML), Restricted Maximum Likelihood (REML), First Differences (FD), and First-Difference Ratios (FDR). We have also studied the validity and power of seven derived testing procedures, to assess the significance of the slope of the candidate explanatory variable x ₂ to enter the model in which there is already one regressor x ₁. In addition to five testing procedures of the literature, we considered the FDR t-test with n ? 3 df and the modified t-test with n? ? 3 df for partial correlations, where n? is Dutilleul's effective sample size. Efficiency, validity, and power were analyzed by Monte Carlo simulations, as functions of the nature, fixed vs. random (purely random or autocorrelated), of x ₁ and x ₂, the sample size and the autocorrelation of random terms in the regression model. We report extensive results for the autocorrelation structure of first-order autoregressive [AR(1)] type, and discuss results we obtained for other autocorrelation structures, such as spherical semivariogram, first-order moving average [MA(1)] and ARMA(1,1), but we could not present because of space constraints. Overall, we found that:

1. the efficiency of slope estimators and the validity of testing procedures depend primarily on the nature of x ₂, but not on that of x ₁;

2. FDR is the most inefficient slope estimator, regardless of the nature of x ₁ and x ₂;

3. REML is the most efficient of the slope estimators compared relative to GLS, provided the specified autocorrelation structure is correct and the sample size is large enough to ensure the convergence of its optimization algorithm;

4. the FDR t-test, the modified t-test and the REML t-test are the most valid of the testing procedures compared, despite the inefficiency of the FDR and OLS slope estimators for the former two;

5. the FDR t-test, however, suffers from a lack of power that varies with the nature of x ₁ and x ₂; and

6. the modified t-test for partial correlations, which does not require the specification of an autocorrelation structure, can be recommended when x ₁ is fixed or random and x ₂ is random, whether purely random or autocorrelated. Our results are illustrated by the environmental data that motivated our work.

     

On Non-parametric Testing, the Uniform Behaviour of the t-test, and Related Problems

Abstract.  In this article, we revisit some problems in non-parametric hypothesis testing. First, we extend the classical result of Bahadur & Savage [ Ann. Math. Statist . 25 (1956) 1115] to other testing problems, and we answer a conjecture of theirs. Other examples considered are testing whether or not the mean is rational, testing goodness-of-fit, and equivalence testing. Next, we discuss the uniform behaviour of the classical t -test. For most non-parametric models, the Bahadur–Savage result yields that the size of the t -test is one for every sample size. Even if we restrict attention to the family of symmetric distributions supported on a fixed compact set, the t -test is not even uniformly asymptotically level α . However, the convergence of the rejection probability is established uniformly over a large family with a very weak uniform integrability type of condition. Furthermore, under such a restriction, the t -test possesses an asymptotic maximin optimality property.     

Estimation of an Ergodic Diffusion from Discrete Observations

We consider a one-dimensional diffusion process X , with ergodic property, with drift b ( x , θ) and diffusion coefficient a ( x , σ) depending on unknown parameters θ and σ. We are interested in the joint estimation of (θ, σ). For that purpose, we dispose of a discretized trajectory, observed at n equidistant times tⁿ_i = ih_n , 1 ≤ i ≤ n . We assume that h_n ← 0 and nh_n ←∞. Under the condition nh_n^p ← 0 for an arbitrary integer p , we exhibit a contrast dependent on p which provides us with an asymptotically normal and efficient estimator of (θ, σ).     

To be or not to be valid in testing the significance of the slope in simple quantitative linear models with autocorrelated errors

In this article, the validity of procedures for testing the significance of the slope in quantitative linear models with one explanatory variable and first-order autoregressive [AR(1)] errors is analyzed in a Monte Carlo study conducted in the time domain. Two cases are considered for the regressor: fixed and trended versus random and AR(1). In addition to the classical t -test using the Ordinary Least Squares (OLS) estimator of the slope and its standard error, we consider seven t -tests with n-2\,\hbox{df} built on the Generalized Least Squares (GLS) estimator or an estimated GLS estimator, three variants of the classical t -test with different variances of the OLS estimator, two asymptotic tests built on the Maximum Likelihood (ML) estimator, the F -test for fixed effects based on the Restricted Maximum Likelihood (REML) estimator in the mixed-model approach, two t -tests with n - 2 df based on first differences (FD) and first-difference ratios (FDR), and four modified t -tests using various corrections of the number of degrees of freedom. The FDR t -test, the REML F -test and the modified t -test using Dutilleul's effective sample size are the most valid among the testing procedures that do not assume the complete knowledge of the covariance matrix of the errors. However, modified t -tests are not applicable and the FDR t -test suffers from a lack of power when the regressor is fixed and trended ( i.e. , FDR is the same as FD in this case when observations are equally spaced), whereas the REML algorithm fails to converge at small sample sizes. The classical t -test is valid when the regressor is fixed and trended and autocorrelation among errors is predominantly negative, and when the regressor is random and AR(1), like the errors, and autocorrelation is moderately negative or positive. We discuss the results graphically, in terms of the circularity condition defined in repeated measures ANOVA and of the effective sample size used in correlation analysis with autocorrelated sample data. An example with environmental data is presented.     

Evaluation of alternative confidence intervals to address non-inferiority through the stratified difference between proportions

The confidence interval (CI) for the difference between two proportions has been an important and active research topic, especially in the context of non-inferiority hypothesis testing. Issues concerning the Type 1 error rate, power, coverage rate and aberrations have been extensively studied for non-stratified cases. However, stratified confidence intervals are frequently used in non-inferiority trials and similar settings. In this paper, several methods for stratified confidence intervals for the difference between two proportions, including existing methods and novel extensions from unstratified CIs, are evaluated across different scenarios. When sparsity across the strata is not a concern, adding imputed observations to the stratification analysis can strengthen Type-1 error control without substantial loss of power. When sparseness of data is a concern, most of the evaluated methods fail to control Type-1 error; the modified stratified t-test CI is an exception. We recommend the modified stratified t-test CI as the most useful and flexible method across the respective scenarios; the modified stratified Wald CI may be useful in settings where sparsity is unlikely. These findings substantially contribute to the application of stratified CIs for non-inferiority testing of differences between two proportions.     

Principal points for an allometric extension model

A set of \(n\) -principal points of a \(p\) -dimensional distribution is an optimal \(n\) -point-approximation of the distribution in terms of a squared error loss. It is in general difficult to derive an explicit expression of principal points. Hence, we may have to search the whole space \(R^p\) for \(n\) -principal points. Many efforts have been devoted to establish results that specify a linear subspace in which principal points lie. However, the previous studies focused on elliptically symmetric distributions and location mixtures of spherically symmetric distributions, which may not be suitable to many practical situations. In this paper, we deal with a mixture of elliptically symmetric distributions that form an allometric extension model, which has been widely used in the context of principal component analysis. We give conditions under which principal points lie in the linear subspace spanned by the first several principal components.     

Semiparametric Likelihood Based Method for Goodness of Fit Tests and Estimation in Upgraded Mixture Models

We use Owen's (1988, 1990) empirical likelihood method in upgraded mixture models. Two groups of independent observations are available. One is z ₁, ..., z _n which is observed directly from a distribution F ( z ). The other one is x ₁, ..., x _m which is observed indirectly from F ( z ), where the x _i s have density ∫ p ( x | z ) dF ( z ) and p ( x | z ) is a conditional density function. We are interested in testing H ₀: p ( x | z ) = p ( x | z ; θ ), for some specified smooth density function. A semiparametric likelihood ratio based statistic is proposed and it is shown that it converges to a chi-squared distribution. This is a simple method for doing goodness of fit tests, especially when x is a discrete variable with finitely many values. In addition, we discuss estimation of θ and F ( z ) when H ₀ is true. The connection between upgraded mixture models and general estimating equations is pointed out.     

Testing the difference between two independent regression models

ABSTRACT

In some situations, for example, in biology or psychology studies, we wish to determine whether the linear relationship between response variable and predictor variables differs in two populations. The analysis of the covariance (ANCOVA) or, equivalently, the partial F-test approaches are the commonly used methods. In this study, the asymptotic distribution for the difference between two independent regression coefficients was established. The proposed method was used to derive the asymptotic confidence set for the difference between coefficients and hypothesis testing for the equality of the two regression models. Then a simulation study was conducted to compare the proposed method with the partial F method. The performance of the new method was comparable with that of the partial F method.     

Statistical methods for continuous outcomes in partially clustered designs

We address statistical issues involved in the partially clustered design where clusters are only employed in the intervention arm, but not in the control arm. We develop a cluster adjusted t-test to compare group treatment effects with individual treatment effects for continuous outcomes in which the individual level data are used as the unit of the analysis in both arms, we develop an approach for determining sample sizes using this cluster adjusted t-test, and use simulation to demonstrate the consistent accuracy of the proposed cluster adjusted t-test and power estimation procedures. Two real examples illustrate how to use the proposed methods.     

A Test for the Equality of Parameters for Separate Regression Models in the Presence of Heteroskedasticity

Testing for the equality of regression coefficients across two regressions is a problem considered by analysts in a variety of fields. If the variances of the errors of the two regressions are not equal, then it is known that the standard large sample F-test used to test the equality of the coefficients is compromised by the fact that its actual size can differ substantially from the stated level of significance in small samples. This article addresses this problem and borrows from the literature on the Behrens-Fisher problem to provide some simple modifications of the large sample test which allows one to better control the probability of committing a Type I error. Empirical evidence is presented which indicates that the suggested modifications provide tests which are superior to well-known alternative tests over a wide range of the parameter space.     

Simple and Explicit Estimating Functions for a Discretely Observed Diffusion Process

We consider a one-dimensional diffusion process X , with ergodic property, with drift b ( x , θ) and diffusion coefficient a ( x , θ) depending on an unknown parameter θ that may be multidimensional. We are interested in the estimation of θ and dispose, for that purpose, of a discretized trajectory, observed at n equidistant times t_i = iΔ , i = 0, ..., n . We study a particular class of estimating functions of the form ∑ f (θ, X _{t i −1}) which, under the assumption that the integral of f with respect to the invariant measure is null, provide us with a consistent and asymptotically normal estimator. We determine the choice of f that yields the estimator with minimum asymptotic variance within the class and indicate how to construct explicit estimating functions based on the generator of the diffusion. Finally the theoretical study is completed with simulations.     

Bayesian Analysis on a Noncentral Fisher–Student’s Hypersphere

Fisher succeeded early on in redefining Student’s t-distribution in geometrical terms on a central hypersphere. Intriguingly, a noncentral analytical extension for this fundamental Fisher–Student’s central hypersphere h-distribution does not exist. We therefore set to derive the noncentral h-distribution and use it to graphically illustrate the limitations of the Neyman–Pearson null hypothesis significance testing framework and the strengths of the Bayesian statistical hypothesis analysis framework on the hypersphere polar axis, a compact nontrivial one-dimensional parameter space. Using a geometrically meaningful maximal entropy prior, we requalify the apparent failure of an important psychological science reproducibility project. We proceed to show that the Bayes factor appropriately models the two-sample t-test p-value density of a gene expression profile produced by the high-throughput genomic-scale microarray technology, and provides a simple expression for a local false discovery rate addressing the multiple hypothesis testing problem brought about by such a technology.     

Distribution-free tests for parallelism and concurrence in two-sample regression problem

This paper suggests some distribution-free methods for testing hypothesis of parallelism and concurrence of two linear regressions. We assume that the independent variable x is equally spaced. The proposed procedures are compared with nonparametric competitors and the normal theory t-test.     

Likelihood ratio tests in linear mixed models with one variance component

Summary.  We consider the problem of testing null hypotheses that include restrictions on the variance component in a linear mixed model with one variance component and we derive the finite sample and asymptotic distribution of the likelihood ratio test and the restricted likelihood ratio test. The spectral representations of the likelihood ratio test and the restricted likelihood ratio test statistics are used as the basis of efficient simulation algorithms of their null distributions. The large sample χ ² mixture approximations using the usual asymptotic theory for a null hypothesis on the boundary of the parameter space have been shown to be poor in simulation studies. Our asymptotic calculations explain these empirical results. The theory of Self and Liang applies only to linear mixed models for which the data vector can be partitioned into a large number of independent and identically distributed subvectors. One-way analysis of variance and penalized splines models illustrate the results.     

On Minimum Distance Estimation Using Kolmogorov-Lévy Type Metrics

Let X ₁, . . ., X_n be independent identically distributed random variables with a common continuous (cumulative) distribution function (d.f.) F , and F^_n the empirical d.f. (e.d.f.) based on X ₁, . . ., X_n . Let G be a smooth d.f. and Gθ = G (·–θ) its translation through θ∈ R . Using a Kolmogorov-Lévy type metric ρ_α defined on the space of d.f.s. on R , the paper derives both null and non-null limiting distributions of √ n [ ρ_α ( F_n , Gθ_n ) – ρ_α ( F, G_θ )], √ n (θ _n –θ) and √ nρ_α ( Gθ , Gθ ), where θ _n and θ are the minimum ρ_α -distance parameters for F_n and F from G , respectively. These distributions are known explicitly in important particular cases; with some complementary Monte Carlo simulations, they help us clarify our understanding of estimation using minimum distance methods and supremum type metrics. We advocate use of the minimum distance method with supremum type metrics in cases of non-null models. The resulting functionals are Hadamard differentiable and efficient. For small scale parameters the minimum distance functionals are close to medians of the parent distributions. The optimal small scale models result in minimum distance estimators having asymptotic variances very competitive and comparable with best known robust estimators.     

Minimum Contrast Estimation for Fractional Diffusions

Abstract.  When the Hurst coefficient of a fractional Brownian motion     is greater than 1/2 it is possible to define a stochastic integral with respect to     , as the pathwise limit of Riemann sums, and thus to consider pathwise solutions to fractional diffusion equations. In this paper, we consider the vanishing drift case and assume that the solution X _t is parameterized by θ in a compact parameter space Θ . Our main interest is the estimation of θ based on discrete time, but with very frequent observations. It is shown that the estimation problem in this context is locally asymptotically mixed normal. The asymptotic behaviour of a certain class of minimum contrast estimators is then studied and asymptotic efficiency is discussed.     

Classification of an observation into linear manifolds,their closed subsets and closed convex subsets

In this paper, we generalize the notion of classification of an observation (sample), into one of the given n populations to the case where some or all of the populations into which the new observation is to be classified may be new but related in a simple way to the given n populations. The discussion is in the frame-work of the given set of observations obeying the usual multivariate general linear hypothesis model. The set ofpopulations into which the new observation may be classified could be linear manifolds of the parameter space or their closed subsets or closed convex subsets or a combination of them or simply t subsets of the parameter space each of which has a finite number of elements. In the last case alikelihood ratio procedure can be obtained easily. Classification procedures given here are based on Mahalanobis distance. Bonferroni lower bound estimate of the probability of correctly classifying an observation is given for the case when the covariance matrix is known or is estimated from a large sample. A numerical example relating to the classification procedures suggested her is given.     

Testing for state dependence in binary panel data with individual covariates by a modified quadratic exponential model

We propose a test for state dependence in binary panel data with individual covariates. For this aim, we rely on a quadratic exponential model in which the association between the response variables is accounted for in a different way with respect to more standard formulations. The level of association is measured by a single parameter that may be estimated by a Conditional Maximum Likelihood (CML) approach. Under the dynamic logit model, the conditional estimator of this parameter converges to zero when the hypothesis of absence of state dependence is true. Therefore, it is possible to implement a t-test for this hypothesis which may be very simply performed and attains the nominal significance level under several structures of the individual covariates. Through an extensive simulation study, we find that our test has good finite sample properties and it is more robust to the presence of (autocorrelated) covariates in the model specification in comparison with other existing testing procedures for state dependence. The proposed approach is illustrated by two empirical applications: the first is based on data coming from the Panel Study of Income Dynamics and concerns employment and fertility; the second is based on the Health and Retirement Study and concerns the self reported health status.     

Bayes designs for multiple linear regression on the unit sphere

We deal with experimental designs minimizing the mean square error of the linear BAYES estimator for the parameter vector of a multiple linear regression model where the experimental region is the k-dimensional unit sphere. After computing the uniquely determined optimum information matrix, we construct, separately for the homogeneous and the inhomogeneous model, both approximate and exact designs having such an information matrix.