A statistical uncertainty principle for estimating the time of a discrete shift in the mean of a continuous time random process

The purpose of this article is to present a statistical uncertainty principle that can be used when localizing a single change in the mean of a band-limited stationary random process. The statistical model investigated is a continuous time process that experiences a shift in its mean. This continuous time process is presumed to be sampled using an ideal low-pass filter. The least squares estimate of the location of the change in mean is asymptotically Gaussian. The standard deviation of the least squares estimate of the location of the change-point provides a physical limit to the accuracy of the estimate of the time of the mean shift which cannot be bettered.     

Estimation of parameters in a generalized GMANOVA model based on an outer product analogy and least squares

This paper investigates estimation of parameters in a combination of the multivariate linear model and growth curve model, called a generalized GMANOVA model. Making analogy between the outer product of data vectors and covariance yields an approach to directly do least squares to covariance. An outer product least squares estimator of covariance (COPLS estimator) is obtained and its distribution is presented if a normal assumption is imposed on the error matrix. Based on the COPLS estimator, two-stage generalized least squares estimators of the regression coefficients are derived. In addition, asymptotic normalities of these estimators are investigated. Simulation studies have shown that the COPLS estimator and two-stage GLS estimators are alternative competitors with more efficiency in the sense of sample mean, standard deviations and mean of the variance estimates to the existing ML estimator in finite samples. An example of application is also illustrated.     

Fitting linear regression models to censored data by least squares and maximum likelihood methods

Several approaches have been suggested for fitting linear regression models to censored data. These include Cox's propor­tional hazard models based on quasi-likelihoods. Methods of fitting based on least squares and maximum likelihoods have also been proposed. The methods proposed so far all require special purpose optimization routines. We describe an approach here which requires only a modified standard least squares routine.

We present methods for fitting a linear regression model to censored data by least squares and method of maximum likelihood. In the least squares method, the censored values are replaced by their expectations, and the residual sum of squares is minimized. Several variants are suggested in the ways in which the expect­ation is calculated. A parametric (assuming a normal error model) and two non-parametric approaches are described. We also present a method for solving the maximum likelihood equations in the estimation of the regression parameters in the censored regression situation. It is shown that the solutions can be obtained by a recursive algorithm which needs only a least squares routine for optimization. The suggested procesures gain considerably in computational officiency. The Stanford Heart Transplant data is used to illustrate the various methods.     

Testing inference in heteroskedastic linear regressions: a comparison of two alternative approaches

We consider the issue of performing testing inferences on the parameters that index the linear regression model under heteroskedasticity of unknown form. Quasi-t test statistics use asymptotically correct standard errors obtained from heteroskedasticity-consistent covariance matrix estimators. An alternative approach involves making an assumption about the functional form of the response variances and jointly modelling mean and dispersion effects. In this paper we compare the accuracy of testing inferences made using the two approaches. We consider several different quasi-t tests and also z tests performed after estimated generalized least squares estimation which was carried out using three different estimation strategies. The numerical evidence shows that some quasi-t tests are typically considerably less size distorted in small samples than the tests carried out after the jointly modelling of mean and dispersion effects. Finally, we present and discuss two empirical applications.     

Robust weighted one-way ANOVA: Improved approximation and efficiency

A robust test for the one-way ANOVA model under heteroscedasticity is developed in this paper. The data are assumed to be symmetrically distributed, apart from some outliers, although the assumption of normality may be violated. The test statistic to be used is a weighted sum of squares similar to the Welch [1951. On the comparison of several mean values: an alternative approach. Biometrika 38, 330-336.] test statistic, but any of a variety of robust measures of location and scale for the populations of interest may be used instead of the usual mean and standard deviation. Under the commonly occurring condition that the robust measures of location and scale are asymptotically normal, we derive approximations to the distribution of the test statistic under the null hypothesis and to its distribution under alternative hypotheses. An expression for relative efficiency is derived, thus allowing comparison of the efficiency of the test as a function of the choice of the location and scale estimators used in the test statistic. As an illustration of the theory presented here, we apply it to three commonly used robust location–scale estimator pairs: the trimmed mean with the Winsorized standard deviation; the Huber Proposal 2 estimator pair; and the Hampel robust location estimator with the median absolute deviation.     

Confidence regions based on inefficient estimators

We consider an unbiased estimator of a function of mean value parameters, which is not efficient. This inefficient estimator is correlated with a residual vector. Thus, if a unit dispersion is unknown, it is impossible to determine the correct confidence region for a function of mean value parameters via a standard estimator of an unknown dispersion with the exception of the case when the ordinary least squares (OLS) estimator is considered in a model with a special covariance structure such that the OLS and the generalized least squares (GLS) estimator are the same, that is the OLS estimator is efficient. Two different estimators of a unit dispersion independent of an inefficient estimator are derived in a singular linear statistical model. Their quality was verified by simulations for several types of experimental designs. Two new estimators of the unit dispersion were compared with the standard estimators based on the GLS and the OLS estimators of the function of the mean value parameters. The OLS estimator was considered in the incorrect model with a different covariance matrix such that the originally inefficient estimator became efficient. The numerical examples led to a slightly surprising result which seems to be due to data behaviour. An example from geodetic practice is presented in the paper.     

Applying Least Absolute Deviation Regression to Regression-type Estimation of the Index of a Stable Distribution Using the Characteristic Function

Least absolute deviation regression is applied using a fixed number of points for all values of the index to estimate the index and scale parameter of the stable distribution using regression methods based on the empirical characteristic function. The recognized fixed number of points estimation procedure uses ten points in the interval zero to one, and least squares estimation. It is shown that using the more robust least absolute regression based on iteratively re-weighted least squares outperforms the least squares procedure with respect to bias and also mean square error in smaller samples.     

Analysis of longitudinal data unbalanced over time

Summary.  The paper considers modelling, estimating and diagnostically verifying the response process generating longitudinal data, with emphasis on association between repeated meas-ures from unbalanced longitudinal designs. Our model is based on separate specifications of the moments for the mean, standard deviation and correlation, with different components possibly sharing common parameters. We propose a general class of correlation structures that comprise random effects, measurement errors and a serially correlated process. These three elements are combined via flexible time-varying weights, whereas the serial correlation can depend flexibly on the mean time and lag. When the measurement schedule is independent of the response process, our estimation procedure yields consistent and asymptotically normal estimates for the mean parameters even when the standard deviation and correlation are misspecified, and for the standard deviation parameters even when the correlation is misspecified. A generic diagnostic method is developed for verifying the models for the mean, standard deviation and, in particular, the correlation, which is applicable even when the data are severely unbalanced. The methodology is illustrated by an analysis of data from a longitudinal study that was designed to characterize pulmonary growth in girls.     

Variable selection in spatial regression via penalized least squares

We consider variable selection in linear regression of geostatistical data that arise often in environmental and ecological studies. A penalized least squares procedure is studied for simultaneous variable selection and parameter estimation. Various penalty functions are considered including smoothly clipped absolute deviation. Asymptotic properties of penalized least squares estimates, particularly the oracle properties, are established, under suitable regularity conditions imposed on a random field model for the error process. Moreover, computationally feasible algorithms are proposed for estimating regression coefficients and their standard errors. Finite‐sample properties of the proposed methods are investigated in a simulation study and comparison is made among different penalty functions. The methods are illustrated by an ecological dataset of landcover in Wisconsin. The Canadian Journal of Statistics 37: 607–624; 2009 © 2009 Statistical Society of Canada     

New Developments in Statistical Computing

Many users of regression methods are attracted to the notion that it would be valuable to determine the relative importance of independent variables. This article demonstrates a method based on hierarchies that builds on previous efforts to decompose R ² through incremental partitioning. The standard method of incremental partitioning has been to follow one order among the many possible orders available. By taking a hierarchical approach in which all orders of variables are used, the average independent contribution of a variable is obtained and an exact partitioning results. Much the same logic is used to divide the joint effect of a variable. The method is general and applicable to all regression methods, including ordinary least squares, logistic, probit, and log-linear regression. A validation test demonstrates that the algorithm is sensitive to the relationships in the data rather than the proportion of variability accounted for by the statistical model used.     

Adaptive Estimation Using Weighted Least Squares

This paper proposes an adaptive estimator that is more precise than the ordinary least squares estimator if the distribution of random errors is skewed or has long tails. The adaptive estimates are computed using a weighted least squares approach with weights based on the lengths of the tails of the distribution of residuals. Smaller weights are assigned to those observations that have residuals in the tails of long-tailed distributions and larger weights are assigned to observations having residuals in the tails of short-tailed distributions. Monte Carlo methods are used to compare the performance of the proposed estimator and the performance of the ordinary least squares estimator. The estimates that were studied in this simulation include the difference between the means of two populations, the mean of a symmetric distribution, and the slope of a regression line. The adaptive estimators are shown to have lower mean squared errors than those for the ordinary least squares estimators for short-tailed, long-tailed, and skewed distributions, provided the sample size is at least 20. The ordinary least squares estimator has slightly lower mean squared error for normally distributed errors. The adaptive estimator is recommended for general use for studies having sample sizes of at least 20 observations unless the random errors are known to be normally distributed.     

Nonlinear Censored Regression Using Synthetic Data

Abstract.  The problem of estimating a nonlinear regression model, when the dependent variable is randomly censored, is considered. The parameter of the model is estimated by least squares using synthetic data. Consistency and asymptotic normality of the least squares estimators are derived. The proofs are based on a novel approach that uses i.i.d. representations of synthetic data through Kaplan–Meier integrals. The asymptotic results are supported by a small simulation study.     

The Asymptotic Covariance Matrix of the Least Squares Estimator in the Stochastic Linear Regression Model: The Case of Elliptically Symmetric Distribution

This article considers the unconditional asymptotic covariance matrix of the least squares estimator in the linear regression model with stochastic explanatory variables. The asymptotic covariance matrix of the least squares estimator of regression parameters is evaluated relative to the standard asymptotic covariance matrix when the joint distribution of the dependent and explanatory variables is in the class of elliptically symmetric distributions. An empirical example using financial data is presented. Numerical examples and simulation experiments are given to illustrate the difference of the two asymptotic covariance matrices.     

A simple least squares method for estimating a change in mean

A simple least squares method for estimating a change in mean of a sequence of independent random variables is studied. The method first tests for a change in mean based on the regression principle of constrained and unconstrained sums of squares. Conditionally on a decision by this test that a change has occurred, least squares estimates are used to estimate the change point, the initial mean level (prior to the change point) and the change itself. The estimates of the initial level and change are functions of the change point estimate. All estimates are shown to be consistent, and those for the initial level and change are shown to be asymptotically jointly normal. The method performs well for moderately large shifts (one standard deviation or more), but the estimates of the initial level and change are biased in a predictable way for small shifts. The large sample theory is helpful in understanding this problem. The asymptotic distribution of the change point estimator is obtained for local shifts in mean, but the case of non-local shifts appears analytically intractable.     

Estimation of linear models for field experiments

General mixed linear models for experiments conducted over a series of sltes and/or years are described. The ordinary least squares (OLS) estlmator is simple to compute, but is not the best unbiased estimator. Also, the usuaL formula for the varlance of the OLS estimator is not correct and seriously underestimates the true variance. The best linear unbiased estimator is the generalized least squares (GLS) estimator. However, t requires an inversion of the variance-covariance matrix V, whlch is usually of large dimension. Also, in practice, V is unknown.

We presented an estlmator [Vcirc] of the matrix V using the estimators of variance components [for sites, blocks (sites), etc.]. We also presented a simple transformation of the data, such that an ordinary least squares regression of the transformed data gives the estimated generalized least squares (EGLS) estimator. The standard errors obtained from the transformed regression serve as asymptotic standard errors of the EGLS estimators. We also established that the EGLS estlmator is unbiased.

An example of fitting a linear model to data for 18 sites (environments) located in Brazil is given. One of the site variables (soil test phosphorus) was measured by plot rather than by site and this established the need for a covariance model such as the one used rather than the usual analysis of variance model. It is for this variable that the resulting parameter estimates did not correspond well between the OLS and EGLS estimators. Regression statistics and the analysis of variance for the example are presented and summarized.     

Parametric Proportional Odds Frailty Models

We define a parametric proportional odds frailty model to describe lifetime data incorporating heterogeneity between individuals. An unobserved individual random effect, called frailty, acts multiplicatively on the odds of failure by time t. We investigate fitting by maximum likelihood and by least squares. For the latter, the parametric survivor function is fitted to the nonparametric Kaplan–Meier estimate at the observed failure times. Bootstrap standard errors and confidence intervals are obtained for the least squares estimates. The models are applied successfully to simulated data and to two real data sets. Least squares estimates appear to have smaller bias than maximum likelihood.     

Error variance estimation via least squares for small sample nonparametric regression

In this paper we explore statistical properties of some difference-based approaches to estimate an error variance for small sample based on nonparametric regression which satisfies Lipschitz condition. Our study is motivated by Tong and Wang (2005), who estimated error variance using a least squares approach. They considered the error variance as the intercept in a simple linear regression which was obtained from the expectation of their lag-k Rice estimator. Their variance estimators are highly dependent on the setting of a regressor and weight of their simple linear regression. Although this regressor and weight can be varied based on the characteristic of an unknown nonparametric mean function, Tong and Wang (2005) have used a fixed regressor and weight in a large sample and gave no indication of how to determine the regressor and the weight. In this paper, we propose a new approach via local quadratic approximation to determine this regressor and weight. Using our proposed regressor and weight, we estimate the error variance as the intercept of simple linear regression using both ordinary least squares and weighted least squares. Our approach applies to both small and large samples, while most existing difference-based methods are appropriate solely for large samples. We compare the performance of our approach with other existing approaches using extensive simulation study. The advantage of our approach is demonstrated using a real data set.     

Bootstrapping estimators for the seemingly unrelated regressions model

In this paper we consider the possibility of using the bootstrap to estimate the finite sample variability of feasible generalized least squares and improved estimators applied to the seemingly unrelated regressions model. The improved estimators we employ include members of the Stein-rule family and a hierarchical Bayes estimator proposed by Blattberg and George (1991). Simulation experiments are carried out using several SUR examples as well as a very large example based on the price-promotion model, and data, from marketing research.     

Robust Estimation of Multi-response Surfaces Considering Correlation Structure

Response surfaces express the behavior of responses and can be used for both single and multi-response problems. A common approach to estimate a response surface using experimental results is the ordinary least squares (OLS) method. Since OLS is very sensitive to outliers, some robust approaches have been discussed in the literature. Although there are many methods available in the literature for multiple response optimizations, there are a few studies in model building especially robust models. Assuming correlated responses, in this paper, a robust coefficient estimation method is proposed for multi response problem based on M-estimators. In order to illustrate the performance of the proposed procedure, a contaminated experimental design using a numerical example available in the literature with some modifications is used. Both the classical multivariate least squares method and the proposed robust multivariate approach are used to estimate regression coefficients of multi-response surfaces based on this example. Moreover, a comparison of the proposed robust multi response surface (RMRS) approach with separate robust estimation of single response show that the proposed approach is more efficient.     

Evaluation of a Frequentist Hierarchical Model to Estimate Prevalence When Sampling from a Large Geographic Area Using Pool Screening

We present a frequentist Bernoulli-Beta hierarchical model to relax the constant prevalence assumption underlying the traditional prevalence estimation approach based on pooled data. This assumption is called into question when sampling from a large geographic area. Pool screening is a method that combines individual items into pools. Each pool will either test positive (at least one of the items is positive) or negative (all items are negative). Pool screening is commonly applied to the study of tropical diseases where pools consist of vectors (e.g., black flies) that can transmit the disease. The goal is to estimate the proportion of infected vectors.

Intermediate estimators (model parameters) and estimators of ultimate interest (pertaining to prevalence) are evaluated by standard measures of merit, such as bias, variance, and mean squared error making extensive use of expansions. Using the hierarchical model an investigator can determine the probability of the prevalence being below a pre-specified threshold value, a value at which no reemergence of the disease is expected. An investigation into the least biased choice of the α parameter in the Beta (α, β) prevalence distribution leads to the choice of α = 1.