Estimating the average slope

The slope is usually the parameter of primary importance in a simple linear regression. If the straight line model gives a poor fit to the data, one can consider the average slope of the non-linear response. In this paper, we show that if the response is quadratic, then the average slope can be obtained by simply using the slope from a straight line fit. In fact, if the slope of the best fitting line to a smooth non-linear function equals the average slope of the function over an arbitrary interval, then the function must be quadratic. This paper illustrates the case where intentionally fitting a wrong model (in this case, a straight line) gives the correct result (the average slope). The example which motivated this study is used to illustrate the results.     

Self‐modelling regression for longitudinal data with time‐invariant covariates

The authors propose the use of self‐modelling regression to analyze longitudinal data with time invariant covariates. They model the population time curve with a penalized regression spline and use a linear mixed model for transformation of the time and response scales to fit the individual curves. Fitting is done by an iterative algorithm using off‐the‐shelf linear and nonlinear mixed model software. Their method is demonstrated in a simulation study and in the analysis of tree swallow nestling growth from an experiment that includes an experimentally controlled treatment, an observational covariate and multi‐level sampling.     

Thin plate regression splines

Summary. I discuss the production of low rank smoothers for d  ≥ 1 dimensional data, which can be fitted by regression or penalized regression methods. The smoothers are constructed by a simple transformation and truncation of the basis that arises from the solution of the thin plate spline smoothing problem and are optimal in the sense that the truncation is designed to result in the minimum possible perturbation of the thin plate spline smoothing problem given the dimension of the basis used to construct the smoother. By making use of Lanczos iteration the basis change and truncation are computationally efficient. The smoothers allow the use of approximate thin plate spline models with large data sets, avoid the problems that are associated with 'knot placement' that usually complicate modelling with regression splines or penalized regression splines, provide a sensible way of modelling interaction terms in generalized additive models, provide low rank approximations to generalized smoothing spline models, appropriate for use with large data sets, provide a means for incorporating smooth functions of more than one variable into non-linear models and improve the computational efficiency of penalized likelihood models incorporating thin plate splines. Given that the approach produces spline-like models with a sparse basis, it also provides a natural way of incorporating unpenalized spline-like terms in linear and generalized linear models, and these can be treated just like any other model terms from the point of view of model selection, inference and diagnostics.     

Estimating the Concentration of Propagules of a Plant Pathogen from Soil Dilution Plate Counts when the Response is Non-linear

The estimation of micro-organism concentrations from dilution plate data is discussed for situations where expected counts are not proportional to the amount of sample per plate. Aspects of design and analysis are investigated in relation to an alternative non-linear model in which the concentration is given by the slope at the origin. This exponential model generally provides a good fit to available experimental data. Simulations show that estimators based on the model perform well when the response is non-linear and remain reasonably efficient when the response is linear.     

Kernel spline regression

The authors propose «kernel spline regression,» a method of combining spline regression and kernel smoothing by replacing the polynomial approximation for local polynomial kernel regression with the spline basis. The new approach retains the local weighting scheme and the use of a bandwidth to control the size of local neighborhood. The authors compute the bias and variance of the kernel linear spline estimator, which they compare with local linear regression. They show that kernel spline estimators can succeed in capturing the main features of the underlying curve more effectively than local polynomial regression when the curvature changes rapidly. They also show through simulation that kernel spline regression often performs better than ordinary spline regression and local polynomial regression.     

Flexible pair-copula estimation in D-vines using bivariate penalized splines

The paper presents a new method for flexible fitting of D-vines. Pair-copulas are estimated semi-parametrically using penalized Bernstein polynomials or constant and linear B-splines, respectively, as spline bases in each knot of the D-vine throughout each level. A penalty induce smoothness of the fit while the high dimensional spline basis guarantees flexibility. To ensure uniform univariate margins of each pair-copula, linear constraints are placed on the spline coefficients and quadratic programming is used to fit the model. The amount of penalizations for each pair-copula is driven by a penalty parameter which is selected in a numerically efficient way. Simulations and practical examples accompany the presentation.     

Optimal designs for estimating the slope of a regression

In the common linear model with quantitative predictors we consider the problem of designing experiments for estimating the slope of the expected response in a regression. We discuss locally optimal designs, where the experimenter is only interested in the slope at a particular point, and standardized minimax optimal designs, which could be used if precise estimation of the slope over a given region is required. General results on the number of support points of locally optimal designs are derived if the regression functions form a Chebyshev system. For polynomial regression and Fourier regression models of arbitrary degree the optimal designs for estimating the slope of the regression are determined explicitly for many cases of practical interest.     

Identifying Nonlinear Relationships in Regression using the ACE Algorithm

This paper introduces an alternating conditional expectation (ACE) algorithm: a non-parametric approach for estimating the transformations that lead to the maximal multiple correlation of a response and a set of independent variables in regression and correlation analysis. These transformations can give the data analyst insight into the relationships between these variables so that this can be best described and non-linear relationships uncovered. Using the Bayesian information criterion (BIC), we show how to find the best closed-form approximations for the optimal ACE transformations. By means of ACE and BIC, the model fit can be considerably improved compared with the conventional linear model as demonstrated in the two simulated and two real datasets in this paper.     

An evaluation of bootstrap methods for outlier detection in least squares regression

Outlier detection is a critical part of data analysis, and the use of Studentized residuals from regression models fit using least squares is a very common approach to identifying discordant observations in linear regression problems. In this paper we propose a bootstrap approach to constructing critical points for use in outlier detection in the context of least-squares Studentized residuals, and find that this approach allows naturally for mild departures in model assumptions such as non-Normal error distributions. We illustrate our methodology through both a real data example and simulated data.     

A Note on Generalized Functional Linear Model and Its Application

Motivated by a biomarker study for colorectal neoplasia, we consider generalized functional linear models where the functional predictors are measured with errors at discrete design points. Assuming that the true functional predictor and the slope function are smooth, we investigate a two-step estimating procedure where both the true functional predictor and the slope function are estimated through spline smoothing. The operating characteristics of the proposed method are derived; the usefulness of the proposed method is illustrated by a simulation study as well as data analysis for the motivating colorectal neoplasia study.     

A cubic smoothing spline based lack of fit test for nonlinear regression models

The use of a statistic based on cubic spline smoothing is considered for testing nonlinear regression models for lack of fit. The statistic is defined to be the Euclidean squared norm of the smoothed residual vector obtained from fitting the nonlinear model, The asymptotic distribution of the statistic is derived under suitable smooth local alternatives and a numerical example is presented.     

Adjusted variable plots for Cox's proportional hazards regression model

Adjusted variable plots are useful in linear regression for outlier detection and for qualitative evaluation of the fit of a model. In this paper, we extend adjusted variable plots to Cox's proportional hazards model for possibly censored survival data. We propose three different plots: a risk level adjusted variable (RLAV) plot in which each observation in each risk set appears, a subject level adjusted variable (SLAV) plot in which each subject is represented by one point, and an event level adjusted variable (ELAV) plot in which the entire risk set at each failure event is represented by a single point. The latter two plots are derived from the RLAV by combining multiple points. In each point, the regression coefficient and standard error from a Cox proportional hazards regression is obtained by a simple linear regression through the origin fit to the coordinates of the pictured points. The plots are illustrated with a reanalysis of a dataset of 65 patients with multiple myeloma.     

Component Selection in the Additive Regression Model

Abstract. Similar to variable selection in the linear model, selecting significant components in the additive model is of great interest. However, such components are unknown, unobservable functions of independent variables. Some approximation is needed. We suggest a combination of penalized regression spline approximation and group variable selection, called the group‐bridge‐type spline method (GBSM), to handle this component selection problem with a diverging number of correlated variables in each group. The proposed method can select significant components and estimate non‐parametric additive function components simultaneously. To make the GBSM stable in computation and adaptive to the level of smoothness of the component functions, weighted power spline bases and projected weighted power spline bases are proposed. Their performance is examined by simulation studies. The proposed method is extended to a partial linear regression model analysis with real data, and gives reliable results.     

Variance Reduction in Smoothing Splines

Abstract.  We develop a variance reduction method for smoothing splines. For a given point of estimation, we define a variance-reduced spline estimate as a linear combination of classical spline estimates at three nearby points. We first develop a variance reduction method for spline estimators in univariate regression models. We then develop an analogous variance reduction method for spline estimators in clustered/longitudinal models. Simulation studies are performed which demonstrate the efficacy of our variance reduction methods in finite sample settings. Finally, a real data analysis with the motorcycle data set is performed. Here we consider variance estimation and generate 95% pointwise confidence intervals for the unknown regression function.     

Flexible Copula Density Estimation with Penalized Hierarchical B‐splines

The paper introduces a new method for flexible spline fitting for copula density estimation. Spline coefficients are penalized to achieve a smooth fit. To weaken the curse of dimensionality, instead of a full tensor spline basis, a reduced tensor product based on so called sparse grids (Notes Numer. Fluid Mech. Multidiscip. Des., 31, 1991, 241‐251) is used. To achieve uniform margins of the copula density, linear constraints are placed on the spline coefficients, and quadratic programming is used to fit the model. Simulations and practical examples accompany the presentation.     

THE CONSTRUCTION OF EQUILEVERAGE DESIGNS FOR MULTIPLE LINEAR REGRESSION

The hat matrix is widely used as a diagnostic tool in linear regression because it contains the leverages which the independent variables exert on the fitted values. In some experiments, cases with high leverage may be avoided by judicious choice of design for the independent variables. A variety of methods for constructing equileverage designs for linear regression are discussed. Such designs remove one of the factors, namely large leverage points, which can lead to nonrobust estimators and tests. In addition, a method is given for combining equileverage designs to test for lack of fit of the linear model.     

M-Estimation for partially functional linear regression model based on splines

ABSTRACT

M-estimation is a widely used technique for robust statistical inference. In this paper, we study robust partially functional linear regression model in which a scale response variable is explained by a function-valued variable and a finite number of real-valued variables. For the estimation of the regression parameters, which include the infinite dimensional function as well as the slope parameters for the real-valued variables, we use polynomial splines to approximate the slop parameter. The estimation procedure is easy to implement, and it is resistant to heavy-tailederrors or outliers in the response. The asymptotic properties of the proposed estimators are established. Finally, we assess the finite sample performance of the proposed method by Monte Carlo simulation studies.     

Generalized bent-cable methodology for changepoint data: a Bayesian approach

The choice of the model framework in a regression setting depends on the nature of the data. The focus of this study is on changepoint data, exhibiting three phases: incoming and outgoing, both of which are linear, joined by a curved transition. Bent-cable regression is an appealing statistical tool to characterize such trajectories, quantifying the nature of the transition between the two linear phases by modeling the transition as a quadratic phase with unknown width. We demonstrate that a quadratic function may not be appropriate to adequately describe many changepoint data. We then propose a generalization of the bent-cable model by relaxing the assumption of the quadratic bend. The properties of the generalized model are discussed and a Bayesian approach for inference is proposed. The generalized model is demonstrated with applications to three data sets taken from environmental science and economics. We also consider a comparison among the quadratic bent-cable, generalized bent-cable and piecewise linear models in terms of goodness of fit in analyzing both real-world and simulated data. This study suggests that the proposed generalization of the bent-cable model can be valuable in adequately describing changepoint data that exhibit either an abrupt or gradual transition over time.     

Additive nonparametric regression with autocorrelated errors

A Bayesian approach is presented for nonparametric estimation of an additive regression model with autocorrelated errors. Each of the potentially non-linear components is modelled as a regression spline using many knots, while the errors are modelled by a high order stationary autoregressive process parameterized in terms of its autocorrelations. The distribution of significant knots and partial autocorrelations is accounted for using subset selection. Our approach also allows the selection of a suitable transformation of the dependent variable. All aspects of the model are estimated simultaneously by using the Markov chain Monte Carlo method. It is shown empirically that the approach proposed works well on several simulated and real examples.     

Mean and quantile boosting for partially linear additive models

Additive models are often applied in statistical learning which allow linear and nonlinear predictors to coexist. In this article we adapt existing boosting methods for both mean regression and quantile regression in additive models which can simultaneously identify nonlinear, linear and zero predictors. We use gradient boosting in which simple linear regression and univariate penalized spline are used as base learners. Twin boosting is applied to achieve better variable selection accuracy. Simulation studies as well as real data applications illustrate the strength of our proposed methods.