Convergence rate for cross-validatory bandwidth in kernel hazard estimation from dependent samples

This paper is devoted to the nonparametric estimation of hazard function by means of kernel smoothers, and more specifically to the crucial problem of bandwidth selection. We first get the convergence rate of usual cross-validated bandwidth under a general dependence assumption on the sample data, extending in several directions the results existing in the literature. In a second attempt, this rate of convergence is used to motivate the introduction of a penalized version of the cross-validation procedure. The rate of convergence is calculated, and a short simulation study, together with a practical application to real data, shows the interest of this approach for finite sample studies. Finally, as a by-product of our proofs, we state a general inequality for the moments of sums of strong dependent variables. Because of its possible interest for many other purposes apart from hazard estimation, this inequality is presented in a specific self-contained section.     

On variable bandwidth selection in local polynomial regression

The performances of data-driven bandwidth selection procedures in local polynomial regression are investigated by using asymptotic methods and simulation. The bandwidth selection procedures considered are based on minimizing 'prelimit' approximations to the (conditional) mean-squared error (MSE) when the MSE is considered as a function of the bandwidth h . We first consider approximations to the MSE that are based on Taylor expansions around h=0 of the bias part of the MSE. These approximations lead to estimators of the MSE that are accurate only for small bandwidths h . We also consider a bias estimator which instead of using small h approximations to bias naïvely estimates bias as the difference of two local polynomial estimators of different order and we show that this estimator performs well only for moderate to large h . We next define a hybrid bias estimator which equals the Taylor-expansion-based estimator for small h and the difference estimator for moderate to large h . We find that the MSE estimator based on this hybrid bias estimator leads to a bandwidth selection procedure with good asymptotic and, for our Monte Carlo examples, finite sample properties.     

For censored and truncated data

In this paper we develop nonparametric methods for regression analysis when the response variable is subject to censoring and/or truncation. The development is based on a data completion princple that enables us to apply, via an iterative scheme, nonparametric regression techniques to iteratively com¬pleted data from a given sample with censored and/or truncated observations. In particular, locally weighted regression smoothers and additive regression models are extended to left-truncated and right-censored data Nonparamet¬ric regression analysis is applied to the Stanford heart transplant data, which have been analyzed by previous authors using semiparametric regression meth¬ods. and provides new insights into the relationship between expected survival time after a heart transplant and explanatory variables.     

Estimation of Variance Function in Heteroscedastic Regression Models by Generalized Coiflets

A wavelet approach is presented to estimate the variance function in heteroscedastic nonparametric regression model. The initial variance estimates are obtained as squared weighted sums of neighboring observations. The initial estimator of a smooth variance function is improved by means of wavelet smoothers under the situation that the samples at the dyadic points are not available. Since the traditional wavelet system for the variance function estimation is not appropriate in this situation, we demonstrate that the choice of the wavelet system is significant to have better performance. This is accomplished by choosing a suitable wavelet system known as the generalized coiflets. We conduct extensive simulations to evaluate finite sample performance of our method. We also illustrate our method using a real dataset.     

M-Estimation for partially functional linear regression model based on splines

ABSTRACT

M-estimation is a widely used technique for robust statistical inference. In this paper, we study robust partially functional linear regression model in which a scale response variable is explained by a function-valued variable and a finite number of real-valued variables. For the estimation of the regression parameters, which include the infinite dimensional function as well as the slope parameters for the real-valued variables, we use polynomial splines to approximate the slop parameter. The estimation procedure is easy to implement, and it is resistant to heavy-tailederrors or outliers in the response. The asymptotic properties of the proposed estimators are established. Finally, we assess the finite sample performance of the proposed method by Monte Carlo simulation studies.     

On completely data-driven bandwidth selection for single-index models

The class of single-index models (SIMs) has become an important tool for nonparametric regression analysis. As with any other nonparametric regression models, the selection of bandwidth plays an important role in the inferences of the SIMs. However, most results in the literature either take the bandwidths as externally given, or require unpractical assumptions or very restrictive conditions for data-driven bandwidths. We examine the asymptotic properties of a popular bandwidth selection method based on cross-validation that is completely data-driven, under much weaker conditions than those assumed in the literature. And we show that the same bandwidth that is optimal for estimating the index vector, can be used for nearly optimal error variance estimation through the method of varying cross-validation. A simulation study is presented to demonstrate the finite sample performance of the proposed procedures, based on which we recommend a simple 2-step procedure for bandwidth selection, index vector estimation, as well as error variance estimation.     

Smoothing Time Series with Local Polynomial Regression on Time

Time series smoothers estimate the level of a time series at time t as its conditional expectation given present, past and future observations, with the smoothed value depending on the estimated time series model. Alternatively, local polynomial regressions on time can be used to estimate the level, with the implied smoothed value depending on the weight function and the bandwidth in the local linear least squares fit. In this article we compare the two smoothing approaches and describe their similarities. Through simulations, we assess the increase in the mean square error that results when approximating the estimated optimal time series smoother with the local regression estimate of the level.     

Heteroscedasticity diagnostics in varying-coefficient partially linear regression models and applications in analyzing Boston housing data

It is important to detect the variance heterogeneity in regression model because efficient inference requires that heteroscedasticity is taken into consideration if it really exists. For the varying-coefficient partially linear regression models, however, the problem of detecting heteroscedasticity has received very little attention. In this paper, we present two classes of tests of heteroscedasticity for varying-coefficient partially linear regression models. The first test statistic is constructed based on the residuals, in which the error term is from a normal distribution. The second one is motivated by the idea that testing heteroscedasticity is equivalent to testing pseudo-residuals for a constant mean. Asymptotic normality is established with different rates corresponding to the null hypothesis of homoscedasticity and the alternative. Some Monte Carlo simulations are conducted to investigate the finite sample performance of the proposed tests. The test methodologies are illustrated with a real data set example.     

CoSmo: A Constrained Scatterplot Smoother for Estimating Convex,Monotonic Transformations

In many of the applied sciences, it is common that the forms of empirical relationships are almost completely unknown prior to study. Scatterplot smoothers used in nonparametric regression methods have considerable potential to ease the burden of model specification that a researcher would otherwise face in this situation. Occasionally the researcher will know the sign of the first or second derivatives, or both. This article develops a smoothing method that can incorporate this kind of information. I show that cubic regression splines with bounds on the coefficients offer a simple and effective approximation to monotonic, convex or concave transformations. I also discuss methods for testing whether the constraints should be imposed. Monte Carlo results indicate that this method, dubbed CoSmo, has a lower approximation error than either locally weighted regression or two other constrained smoothing methods. CoSmo has many potential applications and should be especially useful in applied econometrics. As an illustration, I apply CoSmo in a multivariate context to estimate a hedonic price function and to test for concavity in one of the variables.     

EMPIRICAL INFLUENCE FOR ROBUST SMOOTHING

A simple graphical method is presented to display the sensitivity of a scatter plot smoother (e.g. loess, kernel smoothers) to perturbations in the data. This enables the robustness of smoothers which have been designed to be robust to be examined directly in particular examples. Graphs are shown of various robust smoothers on several standard datasets, so that the robustness of the smoothers can be compared. The method is found to be useful in revealing features of the smoothers. Related graphs for displaying the sensitivity of a smoother to k > 1 outliers are also presented.     

Guided Censored Regression

Parametrically guided non‐parametric regression is an appealing method that can reduce the bias of a non‐parametric regression function estimator without increasing the variance. In this paper, we adapt this method to the censored data case using an unbiased transformation of the data and a local linear fit. The asymptotic properties of the proposed estimator are established, and its performance is evaluated via finite sample simulations.     

On sign-based regression quantiles

A sign-based (SB) approach suggests an alternative criterion for quantile regression fit. The SB criterion is a piecewise constant function, which often leads to a non-unique solution. We compare the mid-point of this SB solution with the least absolute deviations (LAD) method and describe asymptotic properties of SB estimators under a weaker set of assumptions as compared with the assumptions often used with the generalized method of moments. Asymptotic properties of LAD and SB estimators are equivalent; however, there are finite sample differences as we show in simulation studies. At small to moderate sample sizes, the SB procedure for modelling quantiles at longer tails demonstrates a substantially lower bias, variance, and mean-squared error when compared with the LAD. In the illustrative example, we model a 0.8-level quantile of hospital charges and highlight finite sample advantage of the SB versus LAD.     

Bartlett corrections in beta regression models

We consider the issue of performing accurate small-sample testing inference in beta regression models, which are useful for modeling continuous variates that assume values in (0,1), such as rates and proportions. We derive the Bartlett correction to the likelihood ratio test statistic and also consider a bootstrap Bartlett correction. Using Monte Carlo simulations we compare the finite sample performances of the two corrected tests to that of the standard likelihood ratio test and also to its variant that employs Skovgaard's adjustment; the latter is already available in the literature. The numerical evidence favors the corrected tests we propose. We also present an empirical application.     

Testing generalized linear and semiparametric models against smooth alternatives

We propose goodness-of-fit tests for testing generalized linear models and semiparametric regression models against smooth alternatives. The focus is on models having both continous and factorial covariates. As a smooth extension of a parametric or semiparametric model we use generalized varying-coefficient models as proposed by Hastie and Tibshirani. A likelihood ratio statistic is used for testing. Asymptotic expansions allow us to write the estimates as linear smoothers which in turn guarantees simple and fast bootstrapping of the test statistic. The test is shown to have √ n -power, but in contrast with parametric tests it is powerful against smooth alternatives in general.     

Bootstrap estimation of the variance of the error term in monotonic regression models

The variance of the error term in ordinary regression models and linear smoothers is usually estimated by adjusting the average squared residual for the trace of the smoothing matrix (the degrees of freedom of the predicted response). However, other types of variance estimators are needed when using monotonic regression (MR) models, which are particularly suitable for estimating response functions with pronounced thresholds. Here, we propose a simple bootstrap estimator to compensate for the over-fitting that occurs when MR models are estimated from empirical data. Furthermore, we show that, in the case of one or two predictors, the performance of this estimator can be enhanced by introducing adjustment factors that take into account the slope of the response function and characteristics of the distribution of the explanatory variables. Extensive simulations show that our estimators perform satisfactorily for a great variety of monotonic functions and error distributions.     

Model averaging based on rank

In this paper, we investigate model selection and model averaging based on rank regression. Under mild conditions, we propose a focused information criterion and a frequentist model averaging estimator for the focused parameters in rank regression model. Compared to the least squares method, the new method is not only highly efficient but also robust. The large sample properties of the proposed procedure are established. The finite sample properties are investigated via extensive Monte Claro simulation study. Finally, we use the Boston Housing Price Dataset to illustrate the use of the proposed rank methods.     

An autocorrelation criterion for bandwidth selection in nonparametric regression ∗

In this note, we propose a new method for selecting the bandwidth parameter in non-parametric regression. While standard criteria, such as cross-validation, are based on the true regression curve about which we know little, we propose a criterion which focuses on the true errors about which assumptions may be made. Our proposal is to choose the bandwidth for which the residuals are as uncorrelated as possible. We use the Box-Pierce statistic as the objective to be minimized. In doing so, the behaviour of our residuals will be close to that of the true errors under the hypothesis of independent errors. A simulation study shows that our method succeeds in capturing the main features of the regression curve, in the sense that the number of turning-points of the curve is correctly estimated most of the time.     

A comparison of six smoothers when there are multiple predictors

The paper compares six smoothers, in terms of mean squared error and bias, when there are multiple predictors and the sample size is relatively small. The focus is on methods that use robust measures of location (primarily a 20% trimmed mean) and where there are four predictors. To add perspective, some methods designed for means are included. The smoothers include the locally weighted (loess) method derived by Cleveland and Devlin [W.S. Cleveland, S.J. Devlin, Locally-weighted regression: an approach to regression analysis by local fitting, Journal of the American Statistical Association 83 (1988) 596–610], a variation of a so-called running interval smoother where distances from a point are measured via a particular group of projections of the data, a running interval smoother where distances are measured based in part using the minimum volume ellipsoid estimator, a generalized additive model based on the running interval smoother, a generalized additive model based on the robust version of the smooth derived by Cleveland [W.S. Cleveland, Robust locally weighted regression and smoothing scatterplots, Journal of the American Statistical Association 74 (1979) 829–836], and a kernel regression method stemming from [J. Fan, Local linear smoothers and their minimax efficiencies, The Annals of Statistics 21 (1993) 196–216]. When the regression surface is a plane, the method stemming from [J. Fan, Local linear smoothers and their minimax efficiencies, The Annals of Statistics 21 (1993) 196–216] was found to dominate, and indeed offers a substantial advantage in various situations, even when the error term has a heavy-tailed distribution. However, if there is curvature, this method can perform poorly compared to the other smooths considered. Now the projection-type smoother used in conjunction with a 20% trimmed mean is recommended with the minimum volume ellipsoid method a close second.     

Variable screening for ultrahigh dimensional censored quantile regression

Quantile regression is a flexible approach to assessing covariate effects on failure time, which has attracted considerable interest in survival analysis. When the dimension of covariates is much larger than the sample size, feature screening and variable selection become extremely important and indispensable. In this article, we introduce a new feature screening method for ultrahigh dimensional censored quantile regression. The proposed method can work for a general class of survival models, allow for heterogeneity of data and enjoy desirable properties including the sure screening property and the ranking consistency property. Moreover, an iterative version of screening algorithm has also been proposed to accommodate more complex situations. Monte Carlo simulation studies are designed to evaluate the finite sample performance under different model settings. We also illustrate the proposed methods through an empirical analysis.     

An F-type small sample simultaneous test for nested linear regression models

A small sample simultaneous testing method is proposed for nested linear regression model. The methodology is based on the generalized likelihood ratio test which is the large sample simultaneous testing method for general nested models. The proposed test is also used for model identification.