A note on Bayesian nonparametric regression function estimation

In this note the problem of nonparametric regression function estimation in a random design regression model with Gaussian errors is considered from the Bayesian perspective. It is assumed that the regression function belongs to a class of functions with a known degree of smoothness. A prior distribution on the given class can be induced by a prior on the coefficients in a series expansion of the regression function through an orthonormal system. The rate of convergence of the resulting posterior distribution is employed to provide a measure of the accuracy of the Bayesian estimation procedure defined by the posterior expected regression function. We show that the Bayes’ estimator achieves the optimal minimax rate of convergence under mean integrated squared error over the involved class of regression functions, thus being comparable to other popular frequentist regression estimators.     

On the Uniform Strong Consistency of Local Polynomial Regression Under Dependence Conditions

Abstract

In this article, nonparametric estimators of the regression function, and its derivatives, obtained by means of weighted local polynomial fitting are studied. Consider the fixed regression model where the error random variables are coming from a stationary stochastic process satisfying a mixing condition. Uniform strong consistency, along with rates, are established for these estimators. Furthermore, when the errors follow an AR(1) correlation structure, strong consistency properties are also derived for a modified version of the local polynomial estimators proposed by Vilar-Fernández and Francisco-Fernández (Vilar-Fernández, J. M., Francisco-Fernández, M. (2002 Vilar-Fernández, J. M. and Francisco-Fernández, M. 2002. Local polynomial regression smoothers with AR-error structure. TEST, 11(2): 439–464.  [Google Scholar]). Local polynomial regression smoothers with AR-error structure. TEST 11(2):439–464).     

Consistency of Ridge Function Fields for Varying Nonparametric Regression

A nonparametric regression model proposed in Pelletier and Frouin (2006 Pelletier , B. , Frouin , R. ( 2006 ). Remote sensing of phytoplankton chlorophyll-a concentration by ridge function fields . App. Optics 45 : 784 – 798 . [Google Scholar]) as a solution to the geophysical problem of ocean color remote sensing is studied. The model, called ridge function field, combines a regression estimate in the form of a superposition of ridge functions, or which is equivalent to a neural network, with the idea pertaining to varying-coefficients models, where the parameters of a parametric family are allowed to vary with other variables. Under mild assumptions on the underlying distribution of the data, the strong universal consistency of the least-squares ridge function fields estimate is established.     

Posterior consistency of logistic Gaussian process priors in density estimation

We establish weak and strong posterior consistency of Gaussian process priors studied by Lenk [1988. The logistic normal distribution for Bayesian, nonparametric, predictive densities. J. Amer. Statist. Assoc. 83 (402), 509–516] for density estimation. Weak consistency is related to the support of a Gaussian process in the sup-norm topology which is explicitly identified for many covariance kernels. In fact we show that this support is the space of all continuous functions when the usual covariance kernels are chosen and an appropriate prior is used on the smoothing parameters of the covariance kernel. We then show that a large class of Gaussian process priors achieve weak as well as strong posterior consistency (under some regularity conditions) at true densities that are either continuous or piecewise continuous.     

A Universal Prior Distribution for Bayesian Consistency of Non parametric Procedures

The introduction of the Hausdorff α-entropy in Xing (2008a Xing, Y. (2008a). Convergence rates of posterior distributions for observations without the iid structure, 38 pages. Available at: www.arxiv.org:0811.4677v1. [Google Scholar]), Xing (2008b Xing, Y. (2008b). On adaptive Bayesian inference. Electron. J. Stat. 2:848–862.[Crossref] , [Google Scholar]), Xing (2010 Xing, Y. (2010). Rates of posterior convergence for iid Observations. Commun. Stat. Theory Methods. 39(19):3389–3398.[Taylor & Francis Online] , [Google Scholar]), Xing (2011 Xing, Y. (2011). Convergence rates of nonparametric posterior distributions. J. Stat. Plann. Inference 141:3382–3390.[Crossref], [Web of Science ®] , [Google Scholar]), and Xing and Ranneby (2009 Xing, Y., Ranneby, B. (2009). Sufficient conditions for Bayesian consistency. J. Stat. Plann. Inference. 139:2479–2489.[Crossref], [Web of Science ®] , [Google Scholar]) has lead a series of improvements of well-known results on posterior consistency. In this paper we discuss an application of the Hausdorff α-entropy. We construct a universal prior distribution such that the corresponding posterior distribution is almost surely consistent. The approach of the construction of this type of prior distribution is natural, but it works very well for all separable models. We illustrate such prior distributions by examples. In particular, we obtain that if the true density function is known to be some normal probability density function with unknown mean and unknown variance then without any additional assumption one can construct a prior distribution which leads to posterior consistency.     

A new Bayesian wavelet thresholding estimator of nonparametric regression

The methods of estimation of nonparametric regression function are quite common in statistical application. In this paper, the new Bayesian wavelet thresholding estimation is considered. The new mixture prior distributions for the estimation of nonparametric regression function by applying wavelet transformation are investigated. The reversible jump algorithm to obtain the appropriate prior distributions and value of thresholding is used. The performance of the proposed estimator is assessed with simulated data from well-known test functions by comparing the convergence rate of the proposed estimator with respect to another by evaluating the average mean square error and standard deviations. Finally by applying the developed method, density function of galaxy data is estimated.     

Prior Density Selection as a Particular Case of Bayesian Model Selection: A Predictive Approach

A Bayesian model consists of two elements: a sampling model and a prior density. The problem of selecting a prior density is nothing but the problem of selecting a Bayesian model where the sampling model is fixed. A predictive approach is used through a decision problem where the loss function is the squared L ² distance between the sampling density and the posterior predictive density, because the aim of the method is to choose the prior that provides a posterior predictive density as good as possible. An algorithm is developed for solving the problem; this algorithm is based on Lavine's linearization technique.     

A New Approach to Least Median of Squares and Regression Through the Origin

When the regression model passes through the origin, PROGRESS algorithm fails to find the exact minimum least median of squares (LMS). Therefore Barreto and Maharry (2006 Barreto , H. , Maharry , D. ( 2006 ). Least median of squares and regression through the origin . Comput. Statist. Data Anal. 50 : 1391 – 1397 .[Crossref], [Web of Science ®] , [Google Scholar]) proposed a new algorithm for finding the true LMS. In this paper, a new method is introduced to find the LMS solution, when the intercept is suppressed and regression model includes at most two unknown parameters, in the case of an odd number of data points.     

A Bayesian Approach to the Estimation of Expected Cell Counts by Using Log-Linear Models

ABSTRACT

In this article, Bayesian estimation of the expected cell counts for log-linear models is considered. The prior specified for log-linear parameters is used to determine a prior for expected cell counts, by means of the family and parameters of prior distributions. This approach is more cost-effective than working directly with cell counts because converting prior information into a prior distribution on the log-linear parameters is easier than that of on the expected cell counts. While proceeding from the prior on log-linear parameters to the prior of the expected cell counts, we faced with a singularity problem of variance matrix of the prior distribution, and added a new precision parameter to solve the problem. A numerical example is also given to illustrate the usage of the new parameter.     

A Conditional Density Approach to the Order Determination of Time Series

The study focuses on the selection of the order of a general time series process via the conditional density of the latter, a characteristic of which is that it remains constant for every order beyond the true one. Using simulated time series from various nonlinear models we illustrate how this feature can be traced from conditional density estimation. We study whether two statistics derived from the likelihood function can serve as univariate statistics to determine the order of the process. It is found that a weighted version of the log likelihood function has desirable robust properties in detecting the order of the process.