A comparison of estimators for regression models with change points

We consider two problems concerning locating change points in a linear regression model. One involves jump discontinuities (change-point) in a regression model and the other involves regression lines connected at unknown points. We compare four methods for estimating single or multiple change points in a regression model, when both the error variance and regression coefficients change simultaneously at the unknown point(s): Bayesian, Julious, grid search, and the segmented methods. The proposed methods are evaluated via a simulation study and compared via some standard measures of estimation bias and precision. Finally, the methods are illustrated and compared using three real data sets. The simulation and empirical results overall favor both the segmented and Bayesian methods of estimation, which simultaneously estimate the change point and the other model parameters, though only the Bayesian method is able to handle both continuous and dis-continuous change point problems successfully. If it is known that regression lines are continuous then the segmented method ranked first among methods.     

Detecting structural change in linear models

Consider a sequence of independent observations which change their marginal distribution at most once somewhere in the sequence and one is not certain where the change has occurred. One would be interested in detecting the change and determining the two distributions which would describe the sequence. On the other hand if no change had occurred, one would want to know the common distribution of the observations. This study develops a Bayesian test for detecting a switch from one linear model to another. The test is based on the marginal posterior mass function of the switch point and the posterior probability of a stable model. This test and an informal sequential procedure of Smith are illustrated with data generated from an unstable linear regression model, which changes the linear relationship between the dependent and independent variables     

Modelling trends in groundwater levels by segmented regression with constraints

This paper provides a statistically unified method for modelling trends in groundwater levels for a national project that aims to predict areas at risk from salinity in 2020. It was necessary to characterize the trends in groundwater levels in thousands of boreholes that have been monitored by Agriculture Western Australia throughout the south-west of Western Australia over the last 10 years. The approach investigated in the present paper uses segmented regression with constraints when the number of change points is unknown. For each segment defined by change points, the trend can be described by a linear trend possibly superimposed on a periodic response. Four different types of change point are defined by constraints on the model parameters to cope with different patterns of change in groundwater levels. For a set of candidate change points provided by the user, a modified Akaike information criterion is used for model selection. Model parameters can be estimated by multiple linear regression. Some typical examples are presented to demonstrate the performance of the approach.     

Structural changes in multivariate regression models

This study generalizes the work of chin choy and Broemeling (1980) who investigated the change in the regression parameters of univariate linear models.

The marginal posterior distributions of the change point, the regression matrices,and the precision matrix are found with the use of a proper multivariate normal-Wishart distribution for the parameters of the model.

A numerical study is undertaken in order to gain some insight into the effect that changes in sample size and certain parameter values have on these marginal posterior distributions.     

Guaranteed testing for epidemic changes of a linear regression model

The objective of this paper is to propose and examine a class of generalized maximum likelihood asymptotic power one tests for detection of various types of changes in a linear regression model. The proposed retrospective tests are based on martingales structures Shiryayev–Roberts statistics. This approach is widely known in a sequential analysis of change point problems as an optimal method of detecting a change in distribution. Guaranteed non-asymptotic upper bounds for the significance levels of the considered tests are presented.Simulated data sets are used to demonstrate that the proposed tests can give good results in practice.     

Asymptotic Normality of M-Estimators for Varying Coefficient Models with Longitudinal Data

This article considers a nonparametric varying coefficient regression model with longitudinal observations. The relationship between the dependent variable and the covariates is assumed to be linear at a specific time point, but the coefficients are allowed to change over time. A general formulation is used to treat mean regression, median regression, quantile regression, and robust mean regression in one setting. The local M-estimators of the unknown coefficient functions are obtained by local linear method. The asymptotic distributions of M-estimators of unknown coefficient functions at both interior and boundary points are established. Various applications of the main results, including estimating conditional quantile coefficient functions and robustifying the mean regression coefficient functions are derived. Finite sample properties of our procedures are studied through Monte Carlo simulations.     

Smooth piecewise linear regression splines with hyperbolic covariates

We-propose the use of hyperbolas as covariates in piecewise linear regression splines to fit data exhibiting a multi-phase linear response with smooth transitions between phases. The hyperbolic regression spline model, fitted by non-linear regression, provides an intuitive and easy way to extend to multiple phases the two-phase hyperbolic response model previously proposed by others. The small additional effort required to fit non-linear, as opposed to linear, regression models is particularly worthwhile when investigators are unwilling to assume that the slope of the response changes abruptly at the join points. Furthermore, undue influence on the join point and slope estimates, resulting from points in the transition region, may be avoided by using the hyperbolic regression spline. Two examples illustrate the use of this method.     

Bent‐cable regression with autoregressive noise

Motivated by time series of atmospheric concentrations of certain pollutants the authors develop bent‐cable regression for autocorrelated errors. Bent‐cable regression extends the popular piecewise linear (broken‐stick) model, allowing for a smooth change region of any non‐negative width. Here the authors consider autoregressive noise added to a bent‐cable mean structure, with unknown regression and time series parameters. They develop asymptotic theory for conditional least‐squares estimation in a triangular array framework, wherein each segment of the bent cable contains an increasing number of observations while the autoregressive order remains constant as the sample size grows. They explore the theory in a simulation study, develop implementation details, apply the methodology to the motivating pollutant dataset, and provide a scientific interpretation of the bent‐cable change point not discussed previously. The Canadian Journal of Statistics 38: 386–407; 2010 © 2010 Statistical Society of Canada     

Adjusted variable plots for Cox's proportional hazards regression model

Adjusted variable plots are useful in linear regression for outlier detection and for qualitative evaluation of the fit of a model. In this paper, we extend adjusted variable plots to Cox's proportional hazards model for possibly censored survival data. We propose three different plots: a risk level adjusted variable (RLAV) plot in which each observation in each risk set appears, a subject level adjusted variable (SLAV) plot in which each subject is represented by one point, and an event level adjusted variable (ELAV) plot in which the entire risk set at each failure event is represented by a single point. The latter two plots are derived from the RLAV by combining multiple points. In each point, the regression coefficient and standard error from a Cox proportional hazards regression is obtained by a simple linear regression through the origin fit to the coordinates of the pictured points. The plots are illustrated with a reanalysis of a dataset of 65 patients with multiple myeloma.     

Testing for a change lit the regression matrix of a multivariate linear model

A Bayesian test procedure Is developed to test; the null hypothesis of no change In the regression matrix of a multivariate lin¬ear model against the alternative hypothesis of exactly one change The resulting test is based on the marginal posterior distribution of the change point; To illustrate the test procedure a numerical example using a bivariate regression model is considered.     

A comparison of confidence regions and designs in estimation of a ratio

A computer simulation is performed to compare confidence regions arising from Fie1ler's theorem and approximate large sample intervals in estimating the point of extremum in quadratic regression. In addition, two designs, one of which is motivated by optimal design theory, are compared. These comparisons are made by examining the confidence level and accuracy of the regions, as well as the concentration of the standard point estimator, in a variety of settings. The results have implications for the more general problem of estimating a ratio of linear combinations in the general linear model.     

Bayesian analysis of a linear model involving structural changes in either regression parameters or disturbances precision

ABSTRACT

The present paper considers the Bayesian analysis of a linear regression model involving structural change, which may occur either due to shift in disturbances precision or due to shift in regression parameters. The posterior density for the regression parameter has been derived and posterior odds ratio for testing the hypothesis that structural change is due to shift in disturbances precision against the alternative that the change is due to shift in regression parameters has been obtained. The findings of a numerical simulation have been presented. The proposed model has been applied to RBI data set on corporate sector.     

Parameter changes in a regression model with autocorrelated errors

This study is a Bayesian analysis of a regression model with autocorrelated errors which exhibits one change in the regression parameters and where the autocorrelation parameter is unknown

Using a normal-gamma prior for all the parameters except the shift point which has a uniform distribution, the marginal posterior distribution of the regression parameters, the shift point and the precision of the errors is found. It is important to know where the shift occurred thus the main emphasis is with the posterior distribution of the shift point

A numerical study assesses the effect of the values of the shift point and the magnitude of the shift on the posterior distribution of the shift point. The posterior distribution of the shift point is more sensitive to change, which occurs in the middle of the observations than to one which occurs at an extreme of the data.     

The limiting power of point optimal autocorrelation tests

This paper considers the point optimal tests for AR(1) errors in the linear regression model. It is shown that these tests have the same limiting power characteristics as the Durbin-Watson test. . The limiting power is zero or one when the regression has no intercept, but lies strictly between these values when an intercept is included.     

GENERALIZED INVERSES USED IN RECURSIVE ESTIMATION OF THE GENERAL LINEAR MODEL1

Recursive methods in regression have proved useful in providing diagnostic tools for checking the model as well as checking the stability of the model over time. Such methods are now extended to deal with the problems of singularity that arise when one variable is completely confounded with previously fitted variables up to a particular time point. The problem is solved by setting it in the framework of the general linear model with dependent errors.     

Exponential Growth Model: From Horizontal to Linear Asymptote

We present a smooth function that can be used as regression curve for modeling growth phenomena requiring an increasing curvilinear concave asymptote. This model is obtained as the product of a concave asymptotic curve and the exponential model. In addition to its increasing character with a curvilinear asymptote, including horizontal or linear increasing asymptote, the resulting model provides curves with a single inflection point. Numerical examples are presented.     

Slope modified confidence bands for a simple linear regression model

Considerable attention has been directed in the statistical literature towards the construction of confidence bands for a simple linear regression model. These confidence bands allow the experimenter to make inferences about the model over a particular region of interest. However, in practice an experimenter will usually first check the significance of the regression line before proceeding with any further inferences such as those provided by the confidence bands. From a theoretical point of view, this raises the question of what the conditional confidence level of the confidence bands might be, and from a practical point of view it is unsatisfactory if the confidence bands contain lines that are inconsistent with the directional decision on the slope. In this paper it is shown how confidence bands can be modified to alleviate these two problems.     

Orientation relationship in finite dimensional space

In the present article, we discuss the regression of a point on the surface of a unit sphere in d dimensions given a point on the surface of a unit sphere in p dimensions, where p may not be equal to d. Point projection is added to the rotation and linear transformation for regression link function. The identifiability of the model is proved. Then, parameter estimation in this set up is discussed. Simulation studies and data analyses are done to illustrate the model.     

Discrete M-robust designs for regression models

M-robust designs are defined and constructed for misspecified linear regression models with possibly autocorrelated errors on a discrete design space. These designs minimize the mean-squared errors if linear regression models are correct with uncorrelated errors, subject to two robust constraints which control the change of the bias and the change of variance under model departures. Simulated annealing algorithm is applied to construct M-robust designs. Examples are given to show M-robust designs and compare them with minimax robust designs.     

Therapeutic Hypothermia: Quantification of the Transition of Core Body Temperature Using the Flexible Mixture Bent‐Cable Model for Longitudinal Data

Hypothermia which is induced by reducing core body temperature is a therapeutic tool used to prevent brain damage resulting from physical trauma. However, all physiological systems begin to slow down due to hypothermia and this can result in increased risk of mortality. Therefore quantification of the transition of core body temperature to early hypothermia is of great clinical interest. Conceptually core body temperature may exhibit an either gradual or abrupt transition. Bent‐cable regression is an appealing statistical tool to model such data due to the model's flexibility and readily interpretable regression coefficients. It handles more flexibly models that traditionally have been handled by low‐order polynomial models (for gradual transition) or piecewise linear changepoint models (for abrupt change). We consider a rat model to quantify the temporal trend of core body temperature primarily to address the question: What is the critical time point associated with a breakdown in the compensatory mechanisms following the start of hypothermia therapy? To this end, we develop a Bayesian modelling framework for bent‐cable regression of longitudinal data to simultaneously account for gradual and abrupt transitions. Our analysis reveals that: (i) about 39% of rats exhibit a gradual transition in core body temperature; (ii) the critical time point is approximately the same regardless of transition type; and (iii) both transition types show a significant increase of core body temperature followed by a significant decrease.