Bernstein conditional density estimation with application to conditional distribution and regression functions

In this paper we propose a smooth nonparametric estimation for the conditional probability density function based on a Bernstein polynomial representation. Our estimator can be written as a finite mixture of beta densities with data-driven weights. Using the Bernstein estimator of the conditional density function, we derive new estimators for the distribution function and conditional mean. We establish the asymptotic properties of the proposed estimators, by proving their asymptotic normality and by providing their asymptotic bias and variance. Simulation results suggest that the proposed estimators can outperform the Nadaraya–Watson estimator and, in some specific setups, the local linear kernel estimators. Finally, we use our estimators for modeling the income in Italy, conditional on year from 1951 to 1998, and have another look at the well known Old Faithful Geyser data.     

On conditional sampling strategies

Estimation of the population average by means of a conditional strategy has been considered e.g. in [2–6,9] and [10]. Let us assume that the sampling design depends on a function of an auxiliary variable called an auxiliary statistic like: the sample mean or the sample variance. Under the conditional versions of these designs several estimators of the population mean are considered: the Horvitz-Thompson statistic, ratio and regression type estimators. The unbiasedness and accuracy of the strategies are considered. The empirical example of the accuracy comparisons of strategies is developed. The conditional strategies could improve the accuracy of estimation and protect it against outliers observations.     

Estimation in conditional first order autoregression with discrete support

We consider estimation in the class of first order conditional linear autoregressive models with discrete support that are routinely used to model time series of counts. Various groups of estimators proposed in the literature are discussed: moment-based estimators; regression-based estimators; and likelihood-based estimators. Some of these have been used previously and others not. In particular, we address the performance of new types of generalized method of moments estimators and propose an exact maximum likelihood procedure valid for a Poisson marginal model using backcasting. The small sample properties of all estimators are comprehensively analyzed using simulation. Three situations are considered using data generated with: a fixed autoregressive parameter and equidispersed Poisson innovations; negative binomial innovations; and, additionally, a random autoregressive coefficient. The first set of experiments indicates that bias correction methods, not hitherto used in this context to our knowledge, are some-times needed and that likelihood-based estimators, as might be expected, perform well. The second two scenarios are representative of overdispersion. Methods designed specifically for the Poisson context now perform uniformly badly, but simple, bias-corrected, Yule-Walker and least squares estimators perform well in all cases.     

Efficient estimation for the conditional autoregressive model

Recently, spatial regression models have been attracting a great deal of attention in areas ranging from effect of traffic congestion on accident rates to the analysis of trends in gastric cancer mortality. In this paper, we propose efficient estimators for the regression coefficients of the spatial conditional autoregressive model, when uncertain auxiliary information is available about these coefficients. We provide efficiency comparisons of the proposed estimators based on asymptotic risk analysis and Monte Carlo simulations. We apply the proposed methods to real data on Boston housing prices and illustrate how a bootstrapping approach can be employed to compute prediction errors of the estimators.     

Bootstrapping the conditional copula

This paper is concerned with inference about the dependence or association between two random variables conditionally upon the given value of a covariate. A way to describe such a conditional dependence is via a conditional copula function. Nonparametric estimators for a conditional copula then lead to nonparametric estimates of conditional association measures such as a conditional Kendall's tau. The limiting distributions of nonparametric conditional copula estimators are rather involved. In this paper we propose a bootstrap procedure for approximating these distributions and their characteristics, and establish its consistency. We apply the proposed bootstrap procedure for constructing confidence intervals for conditional association measures, such as a conditional Blomqvist beta and a conditional Kendall's tau. The performances of the proposed methods are investigated via a simulation study involving a variety of models, ranging from models in which the dependence (weak or strong) on the covariate is only through the copula and not through the marginals, to models in which this dependence appears in both the copula and the marginal distributions. As a conclusion we provide practical recommendations for constructing bootstrap-based confidence intervals for the discussed conditional association measures.     

On the finite-sample properties of conditional empirical likelihood estimators

We provide Monte Carlo evidence on the finite-sample behavior of the conditional empirical likelihood (CEL) estimator of Kitamura, Tripathi, and Ahn and the conditional Euclidean empirical likelihood (CEEL) estimator of Antoine, Bonnal, and Renault in the context of a heteroscedastic linear model with an endogenous regressor. We compare these estimators with three heteroscedasticity-consistent instrument-based estimators and the Donald, Imbens, and Newey estimator in terms of various performance measures. Our results suggest that the CEL and CEEL with fixed bandwidths may suffer from the no-moment problem, similarly to the unconditional generalized empirical likelihood estimators studied by Guggenberger. We also study the CEL and CEEL estimators with automatic bandwidths selected through cross-validation. We do not find evidence that these suffer from the no-moment problem. When the instruments are weak, we find CEL and CEEL to have finite-sample properties—in terms of mean squared error and coverage probability of confidence intervals—poorer than the heteroscedasticity-consistent Fuller (HFUL) estimator. In the strong instruments case, the CEL and CEEL estimators with automatic bandwidths tend to outperform HFUL in terms of mean squared error, while the reverse holds in terms of the coverage probability, although the differences in numerical performance are rather small.     

Estimating conditional tail expectation with actuarial applications in view

We develop statistical inferential tools for estimating and comparing conditional tail expectation (CTE) functions, which are of considerable interest in actuarial science. In particular, we construct estimators for the CTE functions, develop the necessary asymptotic theory for the estimators, and then use the theory for constructing confidence intervals and bands for the functions. Both parametric and non-parametric approaches are explored. Simulation studies illustrate the performance of estimators in various situations. Results are obtained under minimal assumptions, and the general Vervaat process plays a crucial role in achieving these goals.     

Nonparametric regression estimation of conditional tails: the random covariate case

We present families of nonparametric estimators for the conditional tail index of a Pareto-type distribution in the presence of random covariates. These families are constructed from locally weighted sums of power transformations of excesses over a high threshold. The asymptotic properties of the proposed estimators are derived under some assumptions on the conditional response distribution, the weight function and the density function of the covariates. We also introduce bias-corrected versions of the estimators for the conditional tail index, and propose in this context a consistent estimator for the second-order tail parameter. The finite sample performance of some specific examples from our classes of estimators is illustrated with a small simulation experiment.     

Berry-Esseen-type bounds for the kernel estimator of conditional distribution and conditional quantiles

Berry-Esseen bounds of order O(n^−1/2) have been obtained for several classes of statistics. In this paper, the rates of convergence in central limit theorem for conditional empirical functions and conditional sample quantiles based on kernel estimators are studied for both conditional and unconditional distributions.     

Comparison of conditional and unconditional confidence intervals for the double exponential distribution

Conditional confidence intervals for the location parameter of the double exponential distribution based on maximum likelihood estimators conditioned on a set of ancillary statistics and the corresponding unconditional confidence intervals based on the maximum likelihood estimators alone are compared in two ways. Monte Carlo techniques are used and the conditional approach appears to give slightly better results although agreement as n becomes larger is noted     

Lattice conditional independence models for seemingly unrelated regressions

Seemingly unrelated regressions (SUR) models appear frequently in econometrics and in the analyses of repeated measures designs and longitudinal data. It is known that iterative algorithms are generally required to obtain the MLEs of the regression parameters. Under a minimal set of lattice conditional independence (LCI) restrictions imposed on the covariance structure, however, closed-form MLEs can be obtained by standard linear regression techniques (Andersson and Perlman, 1993, 1994, 1998). In this paper, simulation is used to study the efficiency of these LCI model-based estimators. We also propose two possible improvements of the usual two-stage estimators for the regression parameters.     

Nonparametric estimation of the conditional mean residual life function with censored data

The conditional mean residual life (MRL) function is the expected remaining lifetime of a system given survival past a particular time point and the values of a set of predictor variables. This function is a valuable tool in reliability and actuarial studies when the right tail of the distribution is of interest, and can be more informative than the survivor function. In this paper, we identify theoretical limitations of some semi-parametric conditional MRL models, and propose two nonparametric methods of estimating the conditional MRL function. Asymptotic properties such as consistency and normality of our proposed estimators are established. We investigate via simulation study the empirical properties of the proposed estimators, including bootstrap pointwise confidence intervals. Using Monte Carlo simulations we compare the proposed nonparametric estimators to two popular semi-parametric methods of analysis, for varying types of data. The proposed estimators are demonstrated on the Veteran’s Administration lung cancer trial.     

Asymptotic normality of conditional density estimation under truncated,censored and dependent data

Abstract

In this paper, we focus on the left-truncated and right-censored model, and construct the local linear and Nadaraya-Watson type estimators of the conditional density. Under suitable conditions, we establish the asymptotic normality of the proposed estimators when the observations are assumed to be a stationary α-mixing sequence. Finite sample behavior of the estimators is investigated via simulations too.     

A conditional perspective of weighted variance estimation of the optimal regression estimator

The estimation of the variance for the GREG (general regression) estimator by weighted residuals is widely accepted as a method which yields estimators with good conditional properties. Since the optimal (regression) estimator shares the properties of GREG estimators which are used in the construction of weighted variance estimators, we introduce the weighting procedure also for estimating the variance of the optimal estimator. This method of variance estimation was originally presented in a seemingly ad hoc manner, and we shall discuss it from a conditional point of view and also look at an alternative way of utilizing the weights. Examples that stress conditional behaviour of estimators are then given for elementary sampling designs such as simple random sampling, stratified simple random sampling and Poisson sampling, where for the latter design we have conducted a small simulation study.     

Trimmed least squares estimator as best trimmed linear conditional estimator for linear regression model

A class of trimmed linear conditional estimators based on regression quantiles for the linear regression model is introduced. This class serves as a robust analogue of non-robust linear unbiased estimators. Asymptotic analysis then shows that the trimmed least squares estimator based on regression quantiles ( Koenker and Bassett ( 1978 ) ) is the best in this estimator class in terms of asymptotic covariance matrices. The class of trimmed linear conditional estimators contains the Mallows-type bounded influence trimmed means ( see De Jongh et al ( 1988 ) ) and trimmed instrumental variables estimators. A large sample methodology based on trimmed instrumental variables estimator for confidence ellipsoids and hypothesis testing is also provided.     

A comparison of parametric conditional error-rate estimators for the two-group linear discriminant function

Using mean absolute deviation, we compare the efficay of two new parametric conditional error rate estimators with six others, four of which are well known.The performance of both new estimators is found to be superior to the six competing estimators examined in this paper, especially when the ratio of the training sample size to the feature dimensionality is small.     

Unconditional pseudo-maximum likelihood and adaptive estimation in the presence of conditional heterogeneity of unknown form

We consider parametric non-linear regression models with additive innovations which are serially uncorrelated but not necessarily independent, and consider the consequences of maximum likelihood and related one-step iterative estimation when the innovations are treated as being iid from their unconditional density. We find that the estimators' asymptotic covariance matrices will generally differ from those that would obtain if the errors actually were iid, except for the special case of strictly exogenous regressors. One important application of these results is to analysis of the properties of adaptive estimators, which employ nonparametric kernel estimates of the unconditional density of the disturbances in the construction of one-step iterative estimators. In the presence of strictly exogenous regressors, adaptive estimators are found to be asymptotically equivalent to the one-step iterative estimators that use the correct unconditional density. We illustrate our results through a brief Monte Carlo study.     

Recursive kernel estimate of the conditional quantile for functional ergodic data

ABSTRACT

In this article, we study the recursive kernel estimator of the conditional quantile of a scalar response variable Y given a random variable (rv) X taking values in a semi-metric space. Two estimators are considered. While the first one is given by inverting the double-kernel estimate of the conditional distribution function, the second estimator is obtained by using the robust approach. We establish the almost complete consistency of these estimates when the observations are sampled from a functional ergodic process. Finally, a simulation study is carried out to illustrate the finite sample performance of these estimators.     

A quasi-Bayesian model averaging approach for conditional quantile models

The value at risk (VaR) is a risk measure that is widely used by financial institutions to allocate risk. VaR forecast estimation involves the evaluation of conditional quantiles based on the currently available information. Recent advances in VaR evaluation incorporate conditional variance into the quantile estimation, which yields the conditional autoregressive VaR (CAViaR) models. However, uncertainty with regard to model selection in CAViaR model estimators raises the issue of identifying the better quantile predictor via averaging. In this study, we propose a quasi-Bayesian model averaging method that generates combinations of conditional VaR estimators based on single CAViaR models. This approach provides us a basis for comparing single CAViaR models against averaged ones for their ability to forecast VaR. We illustrate this method using simulated and financial daily return data series. The results demonstrate significant findings with regard to the use of averaged conditional VaR estimates when forecasting quantile risk.     

Edgeworth expansions for the conditional distributions in logistic regression models

Edgeworth expansions are derived for conditional distributions of sufficient statistics as well as conditional maximum likelihood estimators of log odds ratios in logistic regression models assuming that the risk factors are not almost equally distanced. Expansions are given in several special cases. Similar results are obtained for models with polytomous outcomes.