A method for choosing the smoothing parameter in a semi-parametric model for detecting change-points in blood flow

In a smoothing spline model with unknown change-points, the choice of the smoothing parameter strongly influences the estimation of the change-point locations and the function at the change-points. In a tumor biology example, where change-points in blood flow in response to treatment were of interest, choosing the smoothing parameter based on minimizing generalized cross-validation (GCV) gave unsatisfactory estimates of the change-points. We propose a new method, aGCV, that re-weights the residual sum of squares and generalized degrees of freedom terms from GCV. The weight is chosen to maximize the decrease in the generalized degrees of freedom as a function of the weight value, while simultaneously minimizing aGCV as a function of the smoothing parameter and the change-points. Compared with GCV, simulation studies suggest that the aGCV method yields improved estimates of the change-point and the value of the function at the change-point.     

An Efficient Algorithm for the Least-Squares Cross-Validation with Symmetric and Polynomial Kernels

The least-squares cross-validation is a completely automatic method for choosing the smoothing parameter in probability density estimation but this method consume large amounts of computer time. This article concerns an efficient computational algorithm for this method when the kernel is symmetric and polynomial functions.     

A model-averaging approach for smoothing spline regression

ABSTRACT

This article considers nonparametric regression problems and develops a model-averaging procedure for smoothing spline regression problems. Unlike most smoothing parameter selection studies determining an optimum smoothing parameter, our focus here is on the prediction accuracy for the true conditional mean of Y given a predictor X. Our method consists of two steps. The first step is to construct a class of smoothing spline regression models based on nonparametric bootstrap samples, each with an appropriate smoothing parameter. The second step is to average bootstrap smoothing spline estimates of different smoothness to form a final improved estimate. To minimize the prediction error, we estimate the model weights using a delete-one-out cross-validation procedure. A simulation study has been performed by using a program written in R. The simulation study provides a comparison of the most well known cross-validation (CV), generalized cross-validation (GCV), and the proposed method. This new method is straightforward to implement, and gives reliable performances in simulations.     

A frequency domain comparison of estimation methods for gaussian fields

This paper demonstrates that well-known parameter estimation methods for Gaussian fields place different emphasis on the high and low frequency components of the data. As a consequence, the relative importance of the frequencies under the objective of the analysis should be taken into account when selecting an estimation method, in addition to other considerations such as statistical and computational efficiency. The paper also shows that when noise is added to the Gaussian field, maximum pseudolikelihood automatically sets the smoothing parameter of the model equal to one. A simulation study then indicates that generalised cross-validation is more robust than maximum likelihood un-

der model misspecification in smoothing and image restoration problems. This has implications for Bayesian procedures since these use the same weightings of the frequencies as the likelihood.     

Smoothing spline Gaussian regression: more scalable computation via efficient approximation

Summary.  Smoothing splines via the penalized least squares method provide versatile and effective nonparametric models for regression with Gaussian responses. The computation of smoothing splines is generally of the order O ( n ³), n being the sample size, which severely limits its practical applicability. We study more scalable computation of smoothing spline regression via certain low dimensional approximations that are asymptotically as efficient. A simple algorithm is presented and the Bayes model that is associated with the approximations is derived, with the latter guiding the porting of Bayesian confidence intervals. The practical choice of the dimension of the approximating space is determined through simulation studies, and empirical comparisons of the approximations with the exact solution are presented. Also evaluated is a simple modification of the generalized cross-validation method for smoothing parameter selection, which to a large extent fixes the occasional undersmoothing problem that is suffered by generalized cross-validation.     

Resistant selection of the smoothing parameter for smoothing splines

Robust automatic selection techniques for the smoothing parameter of a smoothing spline are introduced. They are based on a robust predictive error criterion and can be viewed as robust versions of C _p and cross-validation. They lead to smoothing splines which are stable and reliable in terms of mean squared error over a large spectrum of model distributions.     

用概率方法确定平滑系数（α）

本文以现代概率论为基础,针对指数平滑法中平滑系数(α)的确定,提出了一种新的方法—概率法,从而较好地克服了现行的平滑系数(α)确定方法所存在的一些缺陷。而且,经过例证,证明效果良好。     

ON SPLINE SMOOTHING WITH AUTOCORRELATED ERRORS

The generalised cross-validation criterion for choosing the degree of smoothing in spline regression is extended to accommodate an autocorrelated error sequence. It is demonstrated via simulation that the minimum generalised cross-validation smoothing spline is an inconsistent estimator in the presence of autocorrelated errors and that ignoring even moderate autocorrelation structure can seriously affect the performance of the cross-validated smoothing spline. The method of penalised maximum likelihood is used to develop an efficient algorithm for the case in which the autocorrelation decays exponentially. An application of the method to a published data-set is described. The method does not require the data to be equally spaced in time.     

数据随机缺失下分位数回归模型的诱导光滑估计法

在数据随机缺失的分位数回归模型中,运用诱导光滑思想构造光滑的估计方程,得到了回归参数的诱导光滑估计及渐近协方差估计。接着证明了诱导光滑估计的渐近正态性质,并给出诱导光滑估计及其渐近协方差估计的算法。模拟研究表明新方法在有限样本中表现出色。     

Likelihood-based cross-validation score for selecting the smoothing parameter in maximum penalized likelihood estimation

Maximum penalized likelihood estimation is applied in non(semi)-para-metric regression problems, and enables us exploratory identification and diagnostics of nonlinear regression relationships. The smoothing parameter A controls trade-off between the smoothness and the goodness-of-fit of a function. The method of cross-validation is used for selecting A, but the generalized cross-validation, which is based on the squared error criterion, shows bad be¬havior in non-normal distribution and can not often select reasonable A. The purpose of this study is to propose a method which gives more suitable A and to evaluate the performance of it.

A method of simple calculation for the delete-one estimates in the likeli¬hood-based cross-validation (LCV) score is described. A score of similar form to the Akaike information criterion (AIC) is also derived. The proposed scores are compared with the ones of standard procedures by using data sets in liter¬atures. Simulations are performed to compare the patterns of selecting A and overall goodness-of-fit and to evaluate the effects of some factors. The LCV-scores by the simple calculation provide good approximation to the exact one if λ is not extremeiy smaii Furthermore the LCV scores by the simple size it possible to select X adaptively They have the effect, of reducing the bias of estimates and provide better performance in the sense of overall goodness-of fit. These scores are useful especially in the case of small sample size and in the case of binary logistic regression.     

A goodness-of-fit test for logistic-normal models using nonparametric smoothing method

Logistic-normal models can be applied for analysis of longitudinal binary data. The aim of this article is to propose a goodness-of-fit test using nonparametric smoothing techniques for checking the adequacy of logistic-normal models. Moreover, the leave-one-out cross-validation method for selecting the suitable bandwidth is developed. The quadratic form of the proposed test statistic based on smoothing residuals provides a global measure for checking the model with categorical and continuous covariates. The formulae of expectation and variance of the proposed statistics are derived, and their asymptotic distribution is approximated by a scaled chi-squared distribution. The power performance of the proposed test for detecting the interaction term or the squared term of continuous covariates is examined by simulation studies. A longitudinal dataset is utilized to illustrate the application of the proposed test.     

Modelling and smoothing parameter estimation with multiple quadratic penalties

Penalized likelihood methods provide a range of practical modelling tools, including spline smoothing, generalized additive models and variants of ridge regression. Selecting the correct weights for penalties is a critical part of using these methods and in the single-penalty case the analyst has several well-founded techniques to choose from. However, many modelling problems suggest a formulation employing multiple penalties, and here general methodology is lacking. A wide family of models with multiple penalties can be fitted to data by iterative solution of the generalized ridge regression problem minimize || W ^1/2 ( Xp − y ) ||²ρ+Σ _{i =1} ^m  θ _i p ' S _i p ( p is a parameter vector, X a design matrix, S _i a non-negative definite coefficient matrix defining the i th penalty with associated smoothing parameter θ _i , W a diagonal weight matrix, y a vector of data or pseudodata and ρ an 'overall' smoothing parameter included for computational efficiency). This paper shows how smoothing parameter selection can be performed efficiently by applying generalized cross-validation to this problem and how this allows non-linear, generalized linear and linear models to be fitted using multiple penalties, substantially increasing the scope of penalized modelling methods. Examples of non-linear modelling, generalized additive modelling and anisotropic smoothing are given.     

Numerical experiments with the thin plate histospline

The thin plate volume matching and volume smoothing histosplines are described. These histosplines are suitable for estimating densities or incidence rates as a function of position on the plane when data is aggregated by area, for example by county. We give a numerical algorithm for the volume matching histospline and for the volume smoothing histospline using generalized cross validation to estimate the smoothing parameter. Some numerical experiments were performed using synthetic data, population data and SMR's (standardized mortality ratios) aggregated by county over the state of Wisconsin. The method turns out to be not particularly suited for obtaining population density maps where the population density can vary by two orders of magnitude, because the histospline can be negative in     

Fast covariance estimation for sparse functional data

Smoothing of noisy sample covariances is an important component in functional data analysis. We propose a novel covariance smoothing method based on penalized splines and associated software. The proposed method is a bivariate spline smoother that is designed for covariance smoothing and can be used for sparse functional or longitudinal data. We propose a fast algorithm for covariance smoothing using leave-one-subject-out cross-validation. Our simulations show that the proposed method compares favorably against several commonly used methods. The method is applied to a study of child growth led by one of coauthors and to a public dataset of longitudinal CD4 counts.     

Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion

Many different methods have been proposed to construct nonparametric estimates of a smooth regression function, including local polynomial, (convolution) kernel and smoothing spline estimators. Each of these estimators uses a smoothing parameter to control the amount of smoothing performed on a given data set. In this paper an improved version of a criterion based on the Akaike information criterion (AIC), termed AIC_C, is derived and examined as a way to choose the smoothing parameter. Unlike plug-in methods, AIC_C can be used to choose smoothing parameters for any linear smoother, including local quadratic and smoothing spline estimators. The use of AIC_C avoids the large variability and tendency to undersmooth (compared with the actual minimizer of average squared error) seen when other 'classical' approaches (such as generalized cross-validation (GCV) or the AIC) are used to choose the smoothing parameter. Monte Carlo simulations demonstrate that the AIC_C-based smoothing parameter is competitive with a plug-in method (assuming that one exists) when the plug-in method works well but also performs well when the plug-in approach fails or is unavailable.     

ASYMPTOTIC STABILITY OF THE OSCV SMOOTHING PARAMETER SELECTION

The smoothing parameter selection by the one-sided cross-validation (OSCV) method is completely automatic in that it does not require extra parameters estimation. Also it reduces the variability comparable to that of plug-in rules. In this paper we derive analytically the asymptotic variance of the smoothing parameter selected by OSCV. It shows the dependency of the stability on the one-sided kerenl and tells the possibility of the optimal one-sided kernel which minimizes the asymptotic variability.     

Nonparametric estimation of density derivatives of dependent data

We study here the kernel type, nonparametric estimation of the derivatives of the density function associated with a strongly mixing time series. The consistency and asymptotic normality properties are studied and a method for the selection of the smoothing parameter by means of the modification of the least-squares cross-validation procedure is proposed.     

ON CROSS-VALIDATION FOR SMOOTHING SPLINES IN THE CASE OF DEPENDENT OBSERVATIONS

Cross-validation, as a popular tool for choosing a smoothing parameter, is generalized to the case of dependent observations. A general version of the ‘deletion theorem’ for representation and simplified calculation of cross-validatory criteria is given. Finally cross-validation is discussed in terms of penalized likelihoods as a method for model choice analogous to the Akaike information criterion.     

Estimating the Expectation of the Log-Likelihood with Censored Data for Estimator Selection

A criterion for choosing an estimator in a family of semi-parametric estimators from incomplete data is proposed. This criterion is the expected observed log-likelihood (ELL). Adapted versions of this criterion in case of censored data and in presence of explanatory variables are exhibited. We show that likelihood cross-validation (LCV) is an estimator of ELL and we exhibit three bootstrap estimators. A simulation study considering both families of kernel and penalized likelihood estimators of the hazard function (indexed on a smoothing parameter) demonstrates good results of LCV and a bootstrap estimator called ELL_bboot . We apply the ELL_bboot criterion to compare the kernel and penalized likelihood estimators to estimate the risk of developing dementia for women using data from a large cohort study.     

On Smoothing Sparse Multinomial Data

Asymptotic theory is developed for the problem of smoothing sparse multinomial data, with emphasis on the criterion of mean summed square error of estimators of the probability mass function. If the data are not too sparse, in a well-defined sense, then the optimal rate of convergence is that achieved by the unsmoothed cell proportions. Otherwise, this rate can be improved upon by smoothing. Explicit results, including formulae for the optimal smoothing parameter, are presented for a kernel-type estimator. Also for this case, a cross-validatory choice procedure is shown to be asymptotically optimal.