Kernel density estimation for hierarchical data

Abstract

Multistage sampling is a common sampling technique employed in many studies. In this setting, observations are identically distributed but not independent, thus many traditional kernel smoothing techniques, which assume that the data are independent and identically distributed (i.i.d.), may not produce reasonable density estimates. In this paper, we sample repeatedly with replacement from each cluster, create multiple i.i.d. samples containing one observation from each cluster, and then create a kernel density estimate from each i.i.d. sample. These estimates will then be combined to form an estimate of the marginal probability density function of the population.     

Simple boundary correction for kernel density estimation

If a probability density function has bounded support, kernel density estimates often overspill the boundaries and are consequently especially biased at and near these edges. In this paper, we consider the alleviation of this boundary problem. A simple unified framework is provided which covers a number of straightforward methods and allows for their comparison: generalized jackknifing generates a variety of simple boundary kernel formulae. A well-known method of Rice (1984) is a special case. A popular linear correction method is another: it has close connections with the boundary properties of local linear fitting (Fan and Gijbels, 1992). Links with the optimal boundary kernels of Müller (1991) are investigated. Novel boundary kernels involving kernel derivatives and generalized reflection arise too. In comparisons, various generalized jackknifing methods perform rather similarly, so this, together with its existing popularity, make linear correction as good a method as any. In an as yet unsuccessful attempt to improve on generalized jackknifing, a variety of alternative approaches is considered. A further contribution is to consider generalized jackknife boundary correction for density derivative estimation. En route to all this, a natural analogue of local polynomial regression for density estimation is defined and discussed.     

Nonparametric density estimation based on beta prime kernel

Abstract

In this work, we propose beta prime kernel estimator for estimation of a probability density functions defined with nonnegative support. For the proposed estimator, beta prime probability density function used as a kernel. It is free of boundary bias and nonnegative with a natural varying shape. We obtained the optimal rate of convergence for the mean squared error (MSE) and the mean integrated squared error (MISE). Also, we use adaptive Bayesian bandwidth selection method with Lindley approximation for heavy tailed distributions and compare its performance with the global least squares cross-validation bandwidth selection method. Simulation studies are performed to evaluate the average integrated squared error (ISE) of the proposed kernel estimator against some asymmetric competitors using Monte Carlo simulations. Moreover, real data sets are presented to illustrate the findings.     

Recursive kernel estimate of the conditional quantile for functional ergodic data

ABSTRACT

In this article, we study the recursive kernel estimator of the conditional quantile of a scalar response variable Y given a random variable (rv) X taking values in a semi-metric space. Two estimators are considered. While the first one is given by inverting the double-kernel estimate of the conditional distribution function, the second estimator is obtained by using the robust approach. We establish the almost complete consistency of these estimates when the observations are sampled from a functional ergodic process. Finally, a simulation study is carried out to illustrate the finite sample performance of these estimators.     

Semiparametric Efficient Estimation of the Mean of a Time Series in the Presence of Conditional Heterogeneity of Unknown Form

Abstract

We obtain semiparametric efficiency bounds for estimation of a location parameter in a time series model where the innovations are stationary and ergodic conditionally symmetric martingale differences but otherwise possess general dependence and distributions of unknown form. We then describe an iterative estimator that achieves this bound when the conditional density functions of the sample are known. Finally, we develop a “semi-adaptive” estimator that achieves the bound when these densities are unknown by the investigator. This estimator employs nonparametric kernel estimates of the densities. Monte Carlo results are reported.     

Selection of bandwidth for kernel regression

ABSTRACT

The most important factor in kernel regression is a choice of a bandwidth. Considerable attention has been paid to extension the idea of an iterative method known for a kernel density estimate to kernel regression. Data-driven selectors of the bandwidth for kernel regression are considered. The proposed method is based on an optimally balanced relation between the integrated variance and the integrated square bias. This approach leads to an iterative quadratically convergent process. The analysis of statistical properties shows the rationale of the proposed method. In order to see statistical properties of this method the consistency is determined. The utility of the method is illustrated through a simulation study and real data applications.     

Asymptotic results in gamma kernel regression

Abstract

Based on the Gamma kernel density estimation procedure, this article constructs a nonparametric kernel estimate for the regression functions when the covariate are nonnegative. Asymptotic normality and uniform almost sure convergence results for the new estimator are systematically studied, and the finite performance of the proposed estimate is discussed via a simulation study and a comparison study with an existing method. Finally, the proposed estimation procedure is applied to the Geyser data set.     

Smoothed empirical likelihood for GARCH models with heavy-tailed errors

ABSTRACT

This paper proposes an empirical likelihood (EL) method for estimating the GARCH(p, q) models with heavy-tailed errors. Using the kernel smoothing method, we derive a smoothed EL ratio statistic, which yields a smoothed EL estimator. Moreover, we derive a profile EL for the partial parameters in the presence of nuisance parameters. Simulations and empirical results are conducted to illustrate our proposed method.     

On estimating the reliability in a multicomponent stress-strength model based on Chen distribution

Abstract

In this article, we obtain point and interval estimates of multicomponent stress-strength reliability model of an s-out-of-j system using classical and Bayesian approaches by assuming both stress and strength variables follow a Chen distribution with a common shape parameter which may be known or unknown. The uniformly minimum variance unbiased estimator of reliability is obtained analytically when the common parameter is known. The behavior of proposed reliability estimates is studied using the estimated risks through Monte Carlo simulations and comments are obtained. Finally, a data set is analyzed for illustrative purposes.     

Automatic and location-adaptive estimation in functional single-index regression

This paper develops a new automatic and location-adaptive procedure for estimating regression in a Functional Single-Index Model (FSIM). This procedure is based on k-Nearest Neighbours (kNN) ideas. The asymptotic study includes results for automatically data-driven selected number of neighbours, making the procedure directly usable in practice. The local feature of the kNN approach insures higher predictive power compared with usual kernel estimates, as illustrated in some finite sample analysis. As by-product, we state as preliminary tools some new uniform asymptotic results for kernel estimates in the FSIM model.     

Computationally efficient classes of higher-order kernel functions

Classes of higher-order kernels for estimation of a probability density are constructed by iterating the twicing procedure. Given a kernel K of order l, we build a family of kernels K_m of orders l(m + 1) with the attractive property that their Fourier transforms are simply 1 — {1 —$(.)}^m+1, where ? is the Fourier transform of K. These families of higher-order kernels are well suited when the fast Fourier transform is used to speed up the calculation of the kernel estimate or the least-squares cross-validation procedure for selection of the window width. We also compare the theoretical performance of the optimal polynomial-based kernels with that of the iterative twicing kernels constructed from some popular second-order kernels.     

Exact mean integrated squared error and bandwidth selection for kernel distribution function estimators

Abstract

An exact, closed form, and easy to compute expression for the mean integrated squared error (MISE) of a kernel estimator of a normal mixture cumulative distribution function is derived for the class of arbitrary order Gaussian-based kernels. Comparisons are made with MISE of the empirical distribution function, the infeasible minimum MISE, and the uniform kernel. A simple plug-in method of simultaneously selecting the optimal bandwidth and kernel order is proposed based on a non asymptotic approximation of the unknown distribution by a normal mixture. A simulation study shows that the method provides a viable alternative to existing bandwidth selection procedures.     

Admissible minimax estimators for the shape parameter of Topp–Leone distribution

Abstract

The shape parameter of Topp–Leone distribution is estimated in this article from the Bayesian viewpoint under the assumption of known scale parameter. Bayes and empirical Bayes estimates of the unknown parameter are proposed under non informative and suitable conjugate priors. These estimates are derived under the assumption of squared and linear-exponential error loss functions. The risk functions of the proposed estimates are derived in analytical forms. It is shown that the proposed estimates are minimax and admissible. The consistency of the proposed estimates under the squared error loss function is also proved. Numerical examples are provided.     

Shape-Constrained Kernel-Weighted Least Squares: Estimating Production Functions for Chilean Manufacturing Industries

ABSTRACT

In this article, we examine a novel way of imposing shape constraints on a local polynomial kernel estimator. The proposed approach is referred to as shape constrained kernel-weighted least squares (SCKLS). We prove uniform consistency of the SCKLS estimator with monotonicity and convexity/concavity constraints and establish its convergence rate. In addition, we propose a test to validate whether shape constraints are correctly specified. The competitiveness of SCKLS is shown in a comprehensive simulation study. Finally, we analyze Chilean manufacturing data using the SCKLS estimator and quantify production in the plastics and wood industries. The results show that exporting firms have significantly higher productivity.     

The single-index support vector regression model to address the problem of high dimensionality

ABSTRACT

The last few years, the applications of Support Vector Machine (SVM) for solving classification and regression problems have been increasing, due to its high performance and ability to transform the non-linear relationships among variables to linear form by employing the kernel idea (kernel function). In this work, we develop a semi-parametric approach to fit single-index models to deal with high-dimensional problems. To achieve this goal, we use support vector regression (SVR) for estimating the unknown nonparametric link function, while the single-index is determined by using the semi-parametric least squares method (Ichimura 1993). This development enhances the ability of SVR to solve high-dimensional problem. We design a three simulation examples with high-dimensional problems (linear and nonlinear). The simulations demonstrate the superior performance of the proposed method versus the standard SVR method. This is further illustrated by applying the real data.     

Bayesian Approach in Nonparametric Count Regression with Binomial Kernel

Recently, Kokonendji et al. have adapted the well-known Nadaraya–Watson kernel estimator for estimating the count function m in the context of nonparametric discrete regression. The authors have also investigated the bandwidth selection using the cross-validation method. In this article, we propose a Bayesian approach in the context of nonparametric count regression for estimating the bandwidth and the variance of the model error, which has not been estimated in Kokonendji et al. The model error is considered as Gaussian with mean of zero and a variance of σ². The Bayes estimates cannot be obtained in closed form and then, we use the well-known Markov chain Monte Carlo (MCMC) technique to compute the Bayes estimates under the squared errors loss function. The performance of this proposed approach and the cross-validation method are compared through simulation and real count data.     

Bias Reduction in Logistic Regression with Missing Responses When the Missing Data Mechanism is Nonignorable

ABSTRACT

In logistic regression with nonignorable missing responses, Ibrahim and Lipsitz proposed a method for estimating regression parameters. It is known that the regression estimates obtained by using this method are biased when the sample size is small. Also, another complexity arises when the iterative estimation process encounters separation in estimating regression coefficients. In this article, we propose a method to improve the estimation of regression coefficients. In our likelihood-based method, we penalize the likelihood by multiplying it by a noninformative Jeffreys prior as a penalty term. The proposed method reduces bias and is able to handle the issue of separation. Simulation results show substantial bias reduction for the proposed method as compared to the existing method. Analyses using real world data also support the simulation findings. An R package called brlrmr is developed implementing the proposed method and the Ibrahim and Lipsitz method.     

Classical estimation of hazard rate and mean residual life functions of Pareto distribution

Abstract

In this paper we find the maximum likelihood estimates (MLEs) of hazard rate and mean residual life functions (MRLF) of Pareto distribution, their asymptotic non degenerate distribution, exact distribution and moments. We also discuss the uniformly minimum variance unbiased estimate (UMVUE) of hazard rate function and MRLF. Finally, two numerical examples with simulated data and real data set, are presented to illustrate the proposed estimates.     

Oracally efficient spline-backfitted kernel smoothing of additive partial linear measurement error model

Abstract

We consider statistical inference for additive partial linear models when the linear covariate is measured with error. A bias-corrected spline-backfitted kernel smoothing method is proposed. Under mild assumptions, the proposed component function and parameter estimator are oracally efficient and fast to compute. The nonparametric function estimator’s pointwise distribution is asymptotically equivalent to an function estimator in partial linear model. Finite-sample performance of the proposed estimators is assessed by simulation experiments. The proposed methods are applied to Boston house data set.