Robust Likelihood Cross-Validation for Kernel Density Estimation

Likelihood cross-validation for kernel density estimation is known to be sensitive to extreme observations and heavy-tailed distributions. We propose a robust likelihood-based cross-validation method to select bandwidths in multivariate density estimations. We derive this bandwidth selector within the framework of robust maximum likelihood estimation. This method establishes a smooth transition from likelihood cross-validation for nonextreme observations to least squares cross-validation for extreme observations, thereby combining the efficiency of likelihood cross-validation and the robustness of least-squares cross-validation. We also suggest a simple rule to select the transition threshold. We demonstrate the finite sample performance and practical usefulness of the proposed method via Monte Carlo simulations and a real data application on Chinese air pollution.     

Asymptotic Normality for the Partially Linear EV Models with Longitudinal Data

In this article, we consider a partially linear EV regression model under longitudinal data. By using a weighted kernel method and modified least-squared method, the estimators of unknown parameter, the unknown function are constructed and the asymptotic normality of the estimators are derived. Simulation studies are conducted to illustrate the finite-sample performance of the proposed method.     

Orthogonal weighted empirical likelihood-based variable selection for semiparametric instrumental variable models

In this article, we consider the variable selection for a class of semiparametric instrumental variable models. By combining orthogonal weighting technology and empirical likelihood method, we propose an orthogonal weighted empirical likelihood-based variable selection procedure. Under some mild conditions, the consistency and sparsity of the variable selection procedure are studied. Furthermore, some simulation studies and a real data analysis are carried out to examine the finite-sample performance of the proposed method.     

Bandwidth selection for kernel density estimation with length-biased data

Length-biased data are a particular case of weighted data, which arise in many situations: biomedicine, quality control or epidemiology among others. In this paper we study the theoretical properties of kernel density estimation in the context of length-biased data, proposing two consistent bootstrap methods that we use for bandwidth selection. Apart from the bootstrap bandwidth selectors we suggest a rule-of-thumb. These bandwidth selection proposals are compared with a least-squares cross-validation method. A simulation study is accomplished to understand the behaviour of the procedures in finite samples.     

Inference With Dyadic Data: Asymptotic Behavior of the Dyadic-Robust t-Statistic

ABSTRACT

This article is concerned with inference in the linear model with dyadic data. Dyadic data are indexed by pairs of “units;” for example, trade data between pairs of countries. Because of the potential for observations with a unit in common to be correlated, standard inference procedures may not perform as expected. We establish a range of conditions under which a t-statistic with the dyadic-robust variance estimator of Fafchamps and Gubert is asymptotically normal. Using our theoretical results as a guide, we perform a simulation exercise to study the validity of the normal approximation, as well as the performance of a novel finite-sample correction. We conclude with guidelines for applied researchers wishing to use the dyadic-robust estimator for inference.     

Data-Based Choice of the Number of Pilot Stages for Plug-in Bandwidth Selection

The choice of the bandwidth is a crucial issue for kernel density estimation. Among all the data-dependent methods for choosing the bandwidth, the direct plug-in method has shown a particularly good performance in practice. This procedure is based on estimating an asymptotic approximation of the optimal bandwidth, using two “pilot” kernel estimation stages. Although two pilot stages seem to be enough for most densities, for a long time the problem of how to choose an appropriate number of stages has remained open. Here we propose an automatic (i.e., data-based) method for choosing the number of stages to be employed in the plug-in bandwidth selector. Asymptotic properties of the method are presented and an extensive simulation study is carried out to compare its small-sample performance with that of the most recommended bandwidth selectors in the literature.     

Model Selection,Transformations and Variance Estimation in Nonlinear Regression

The results of analyzing experimental data using a parametric model may heavily depend on the chosen model for regression and variance functions, moreover also on a possibly underlying preliminary transformation of the variables. In this paper we propose and discuss a complex procedure which consists in a simultaneous selection of parametric regression and variance models from a relatively rich model class and of Box-Cox variable transformations by minimization of a cross-validation criterion. For this it is essential to introduce modifications of the standard cross-validation criterion adapted to each of the following objectives: 1. estimation of the unknown regression function, 2. prediction of future values of the response variable, 3. calibration or 4. estimation of some parameter with a certain meaning in the corresponding field of application. Our idea of a criterion oriented combination of procedures (which usually if applied, then in an independent or sequential way) is expected to lead to more accurate results. We show how the accuracy of the parameter estimators can be assessed by a “moment oriented bootstrap procedure", which is an essential modification of the “wild bootstrap” of Härdle and Mammen by use of more accurate variance estimates. This new procedure and its refinement by a bootstrap based pivot (“double bootstrap”) is also used for the construction of confidence, prediction and calibration intervals. Programs written in Splus which realize our strategy for nonlinear regression modelling and parameter estimation are described as well. The performance of the selected model is discussed, and the behaviour of the procedures is illustrated, e.g., by an application in radioimmunological assay.     

Subsampling-extrapolation bandwidth selection in bivariate kernel density estimation

This paper focuses on bivariate kernel density estimation that bridges the gap between univariate and multivariate applications. We propose a subsampling-extrapolation bandwidth matrix selector that improves the reliability of the conventional cross-validation method. The proposed procedure combines a U-statistic expression of the mean integrated squared error and asymptotic theory, and can be used in both cases of diagonal bandwidth matrix and unconstrained bandwidth matrix. In the subsampling stage, one takes advantage of the reduced variability of estimating the bandwidth matrix at a smaller subsample size m (m < n); in the extrapolation stage, a simple linear extrapolation is used to remove the incurred bias. Simulation studies reveal that the proposed method reduces the variability of the cross-validation method by about 50% and achieves an expected integrated squared error that is up to 30% smaller than that of the benchmark cross-validation. It shows comparable or improved performance compared to other competitors across six distributions in terms of the expected integrated squared error. We prove that the components of the selected bivariate bandwidth matrix have an asymptotic multivariate normal distribution, and also present the relative rate of convergence of the proposed bandwidth selector.     

Weighted Estimator for the Linear Transformation Models with Multivariate Failure Time Data

In this article, a simple and efficient weighted method is proposed to improve the estimation efficiency for the linear transformation models with multivariate failure time data. Asymptotic properties of the estimators with a closed-form variance-covariance matrix are established. In addition, a goodness-of-fit test is developed to evaluate the adequacy of the model. The performance of proposed method and the comparison on the efficiency between the proposed method and the working independence method (Lu, 2005) are conducted in finite-sample situation by simulation studies. Finally a real data set from the Busselton Population Health Surveys is illustrated to validate the proposed methodology. The related proofs of the theorems are given in the Appendix.     

Bayesian Approach in Nonparametric Count Regression with Binomial Kernel

Recently, Kokonendji et al. have adapted the well-known Nadaraya–Watson kernel estimator for estimating the count function m in the context of nonparametric discrete regression. The authors have also investigated the bandwidth selection using the cross-validation method. In this article, we propose a Bayesian approach in the context of nonparametric count regression for estimating the bandwidth and the variance of the model error, which has not been estimated in Kokonendji et al. The model error is considered as Gaussian with mean of zero and a variance of σ². The Bayes estimates cannot be obtained in closed form and then, we use the well-known Markov chain Monte Carlo (MCMC) technique to compute the Bayes estimates under the squared errors loss function. The performance of this proposed approach and the cross-validation method are compared through simulation and real count data.     

EXACT FINITE-SAMPLE PROPERTIES OF A PRE-TEST ESTIMATOR IN RIDGE REGRESSION

This paper discusses a pre-test regression estimator which uses the least squares estimate when it is “large” and a ridge regression estimate for “small” regression coefficients, where the preliminary test is applied separately to each regression coefficient in turn to determine whether it is “large” or “small.” For orthogonal regressors, the exact finite-sample bias and mean squared error of the pre-test estimator are derived. The latter is less biased than a ridge estimator, and over much of the parameter space the pre-test estimator has smaller mean squared error than least squares. A ridge estimator is found to be inferior to the pre-test estimator in terms of mean squared error in many situations, and at worst the latter estimator is only slightly less efficient than the former at commonly used significance levels.     

Cross-validation Bandwidth Matrices for Multivariate Kernel Density Estimation

Abstract.  The performance of multivariate kernel density estimates depends crucially on the choice of bandwidth matrix, but progress towards developing good bandwidth matrix selectors has been relatively slow. In particular, previous studies of cross-validation (CV) methods have been restricted to biased and unbiased CV selection of diagonal bandwidth matrices. However, for certain types of target density the use of full (i.e. unconstrained) bandwidth matrices offers the potential for significantly improved density estimation. In this paper, we generalize earlier work from diagonal to full bandwidth matrices, and develop a smooth cross-validation (SCV) methodology for multivariate data. We consider optimization of the SCV technique with respect to a pilot bandwidth matrix. All the CV methods are studied using asymptotic analysis, simulation experiments and real data analysis. The results suggest that SCV for full bandwidth matrices is the most reliable of the CV methods. We also observe that experience from the univariate setting can sometimes be a misleading guide for understanding bandwidth selection in the multivariate case.     

Bayesian Bandwidth Estimation in Nonparametric Time-Varying Coefficient Models

Bandwidth plays an important role in determining the performance of nonparametric estimators, such as the local constant estimator. In this article, we propose a Bayesian approach to bandwidth estimation for local constant estimators of time-varying coefficients in time series models. We establish a large sample theory for the proposed bandwidth estimator and Bayesian estimators of the unknown parameters involved in the error density. A Monte Carlo simulation study shows that (i) the proposed Bayesian estimators for bandwidth and parameters in the error density have satisfactory finite sample performance; and (ii) our proposed Bayesian approach achieves better performance in estimating the bandwidths than the normal reference rule and cross-validation. Moreover, we apply our proposed Bayesian bandwidth estimation method for the time-varying coefficient models that explain Okun’s law and the relationship between consumption growth and income growth in the U.S. For each model, we also provide calibrated parametric forms of the time-varying coefficients. Supplementary materials for this article are available online.     

Partitioned cross-validation

Partitioned cross-validation is proposed as a method for overcoming the large amounts of across sample variability to which ordinary cross-validation is subject. The price for cutting down on the sample noise is that a type of bias is intriduced. A theory is presented for optimal trade-off of this variance and bias. Comparison with other bandwidth selection methods is given.     

Matching estimators and optimal bandwidth choice

Optimal bandwidth choice for matching estimators and their finite sample properties are examined. An approximation to their MSE is derived, as a basis for a plug-in bandwidth selector. In small samples, this approximation is not very accurate, though. Alternatively, conventional cross-validation bandwidth selection is considered and performs rather well in simulation studies: Compared to standard pair-matching, kernel and ridge matching achieve reductions in MSE of about 25 to 40%. Local linear matching and weighting perform poorly. Furthermore, the scope for developing better bandwidth selectors seems to be limited for ridge matching, but non-negligible for kernel and local linear matching.     

Partitioned cross-validation

Partitioned cross-validation is proposed as a method for overcoming the large amounts of across sample variability to which ordinary cross-validation is subject. The price for cutting down on the sample noise is that a type of bias is intriduced. A theory is presented for optimal trade-off of this variance and bias. Comparison with other bandwidth selection methods is given.     

Model Averaging for Prediction With Fragmentary Data

ABSTRACT

One main challenge for statistical prediction with data from multiple sources is that not all the associated covariate data are available for many sampled subjects. Consequently, we need new statistical methodology to handle this type of “fragmentary data” that has become more and more popular in recent years. In this article, we propose a novel method based on the frequentist model averaging that fits some candidate models using all available covariate data. The weights in model averaging are selected by delete-one cross-validation based on the data from complete cases. The optimality of the selected weights is rigorously proved under some conditions. The finite sample performance of the proposed method is confirmed by simulation studies. An example for personal income prediction based on real data from a leading e-community of wealth management in China is also presented for illustration.     

Further theoretical and practical insight to the do-validated bandwidth selector

Recent contributions to kernel smoothing show that the performance of cross-validated bandwidth selectors improves significantly from indirectness and that the recent do-validated method seems to provide the most practical alternative among these methods. In this paper we show step by step how classical cross-validation improves in theory, as well as in practice, from indirectness and that do-validated estimators improve in theory, but not in practice, from further indirectness. This paper therefore provides a strong support for the practical and theoretical properties of do-validated bandwidth selection. Do-validation is currently being introduced to survival analysis in a number of contexts and this paper provides evidence that this might be the immediate step forward.     

A comparison of estimators of the geographical relative risk function

The geographical relative risk function is a useful tool for investigating the spatial distribution of disease based on case and control data. The most common way of estimating this function is using the ratio of bivariate kernel density estimates constructed from the locations of cases and controls, respectively. An alternative is to use a local-linear (LL) estimator of the log-relative risk function. In both cases, the choice of bandwidth is critical. In this article, we examine the relative performance of the two estimation techniques using a variety of data-driven bandwidth selection methods, including likelihood cross-validation (CV), least-squares CV, rule-of-thumb reference methods, and a new approximate plug-in (PI) bandwidth for the LL estimator. Our analysis includes the comparison of asymptotic results; a simulation study; and application of the estimators on two real data sets. Our findings suggest that the density ratio method implemented with the least-squares CV bandwidth selector is generally best, with the LL estimator with PI bandwidth being competitive in applications with strong large-scale trends but much worse in situations with elliptical clusters.