Optimal global rates of convergence for nonparametric regression with unbounded data

Estimation of regression functions from independent and identically distributed data is considered. The _L2$L_2$ error with integration with respect to the design measure is used as an error criterion. Usually in the analysis of the rate of convergence of estimates a boundedness assumption on the explanatory variable X$X$ is made besides smoothness assumptions on the regression function and moment conditions on the response variable Y$Y$. In this article we consider the kernel estimate and show that by replacing the boundedness assumption on X$X$ by a proper moment condition the same (optimal) rate of convergence can be shown as for bounded data. This answers Question 1 in Stone [1982. Optimal global rates of convergence for nonparametric regression. Ann. Statist., 10, 1040–1053].     

Inverse Regression in Binary Response LDV Model

Sliced inverse regression, a link-free and distribution-free method, is applied to binary response limited dependent variable models. An inverse regression property of binary response LDV model is found. Based on this property, if the distributions of X _j (j = 1, 2,…, p) satisfy the linearity condition, then β can be estimated up to a positive multiplicative scalar without any assumptions on the distribution of error ε. Moreover, the estimator can be proved to be asymptotically normal based on which testing hypotheses are considered. Simulations results are reported.     

Optimal quantization applied to sliced inverse regression

In this paper we consider a semiparametric regression model involving a d-dimensional quantitative explanatory variable X and including a dimension reduction of X via an index β′X. In this model, the main goal is to estimate the Euclidean parameter β and to predict the real response variable Y conditionally to X. Our approach is based on sliced inverse regression (SIR) method and optimal quantization in L^p-norm. We obtain the convergence of the proposed estimators of β and of the conditional distribution. Simulation studies show the good numerical behavior of the proposed estimators for finite sample size.     

Nonlinear orthogonal series estimates for random design regression

Let (X,Y) be a pair of random variables with supp(X)⊆[0,1] and EY²<∞. Let m be the corresponding regression function. Estimation of m from i.i.d. data is considered. The L₂ error with integration with respect to the design measure μ (i.e., the distribution of X) is used as an error criterion.Estimates are constructed by estimating the coefficients of an orthonormal expansion of the regression function. This orthonormal expansion is done with respect to a family of piecewise polynomials, which are orthonormal in L₂(μ_n), where μ_n denotes the empirical design measure.It is shown that the estimates are weakly and strongly consistent for every distribution of (X,Y). Furthermore, the estimates behave nearly as well as an ideal (but not applicable) estimate constructed by fitting a piecewise polynomial to the data, where the partition of the piecewise polynomial is chosen optimally for the underlying distribution. This implies e.g., that the estimates achieve up to a logarithmic factor the rate n^−2p/(2p+1), if the underlying regression function is piecewise p-smooth, although their definition depends neither on the smoothness nor on the location of the discontinuities of the regression function.     

Confidence Intervals for Nonparametric Regression Functions with Missing Data

Suppose that we have a nonparametric regression model Y = m(X) + ε with X ∈ R^p, where X is a random design variable and is observed completely, and Y is the response variable and some Y-values are missing at random. Based on the “complete” data sets for Y after nonaprametric regression imputation and inverse probability weighted imputation, two estimators of the regression function m(x₀) for fixed x₀ ∈ R^p are proposed. Asymptotic normality of two estimators is established, which is used to construct normal approximation-based confidence intervals for m(x₀). We also construct an empirical likelihood (EL) statistic for m(x₀) with limiting distribution of χ²₁, which is used to construct an EL confidence interval for m(x₀).     

A study on the least squares estimator of multivariate isotonic regression function

Consider the problem of pointwise estimation of f in a multivariate isotonic regression model Z=f(X₁,…,X_d)+ϵ, where Z is the response variable, f is an unknown nonparametric regression function, which is isotonic with respect to each component, and ϵ is the error term. In this article, we investigate the behavior of the least squares estimator of f. We generalize the greatest convex minorant characterization of isotonic regression estimator for the multivariate case and use it to establish the asymptotic distribution of properly normalized version of the estimator. Moreover, we test whether the multivariate isotonic regression function at a fixed point is larger (or smaller) than a specified value or not based on this estimator, and the consistency of the test is established. The practicability of the estimator and the test are shown on simulated and real data as well.     

Asymptotic tail behavior of a random sum with conditionally dependent subexponential summands

In this paper, we obtain some results for the asymptotic behavior of the tail probability of a random sum S_τ = ∑^τ_{k = 1}X_k, where the summands X_k, k = 1, 2, …, are conditionally dependent random variables with a common subexponential distribution F, and the random number τ is a non negative integer-valued random variable, independent of {X_k: k ? 1}.     

Consistency of l 1 estimates in censored linear regression models

In this paper, the regression model with a nonnegativity constraint on the dependent variable is considered. Under weak conditions, L ₁ estimates of the regression coefficients are shown to be consistent.     

On the asymptotic normality of the L1- and L2-errors in histogram density estimation

The L₁ and L₂-errors of the histogram estimate of a density f from a sample X₁,X₂,…,Xn using a cubic partition are shown to be asymptotically normal without any unnecessary conditions imposed on the density f. The asymptotic variances are shown to depend on f only through the corresponding norm of f. From this follows the asymptotic null distribution of a goodness-of-fit test based on the total variation distance, introduced by Györfi and van der Meulen (1991). This note uses the idea of partial inversion for obtaining characteristic functions of conditional distributions, which goes back at least to Bartlett (1938).     

Multivariate normal distribution approaches for dependently truncated data

Many statistical methods for truncated data rely on the independence assumption regarding the truncation variable. In many application studies, however, the dependence between a variable X of interest and its truncation variable L plays a fundamental role in modeling data structure. For truncated data, typical interest is in estimating the marginal distributions of (L, X) and often in examining the degree of the dependence between X and L. To relax the independence assumption, we present a method of fitting a parametric model on (L, X), which can easily incorporate the dependence structure on the truncation mechanisms. Focusing on a specific example for the bivariate normal distribution, the score equations and Fisher information matrix are provided. A robust procedure based on the bivariate t-distribution is also considered. Simulations are performed to examine finite-sample performances of the proposed method. Extension of the proposed method to doubly truncated data is briefly discussed.     

Confidence bounds for the joint survival function in the dependent competing risks setup

In the competing risks set up with two dependent competing risks, the joint distribution of (X₁,X₂), the latent lifetimes of the system under the two risks, is not identifiable on the basis of the distribution of the actual observation (T, δ) where T = min(X₁, X₂) and δ = I(T=X₁), Using Peterson's (1976) bounds, we have obtained conservative pointwise as well as simultaneous confidence bounds for the unidentifiable joint survival function. In an example we evaluate the confidence bounds and Indicate where the estimated joint survival function in the independent case, lies within them.     

ON CHARACTERIZING DISTRIBUTIONS VIA LINEARITY OF REGRESSION FOR ORDER STATISTICS

Let X₁X_n be a random sample from an absolutely continuous distribution with the corresponding order statistics X_1:n≤X_2:n≤X_n:n. A complete solution of the problem, posed in 1967 by T. Ferguson, of determining the distribution by linearity of regression of X_k+2:n with respect to X_k:n is given. The only possible distributions are of the exponential, power and Pareto type. A linear regression relation for exponents of order statistics is also considered.     

Bayesian quantile regression and variable selection for partial linear single-index model: Using free knot spline

Partial linear single-index model (PLSIM) has both the flexibility of nonparametric treatment and interpretability of linear term, yet existing literatures about it mainly focused on mean regression, and quantile regression analysis is scarce. Based on free knot spline approximation, we apply asymmetric Laplace distribution to implement Bayesian quantile regression, and perform variable selection in linear term and index vector via binary indicators. Our approach is exempt from regularity conditions in frequentist method, and could execute variable selection and quantile regression under mutual posterior correction, which is also the first work to implement them jointly for PLSIM in fully Bayesian framework. The numerical simulation manifests the superiority of our approach to previous methods, which embodied in better efficiency of variable selection, index vector estimates and link function approximation with different error distributions. For illustration of its application, we build a power consumption model of A₂/O process in wastewater treatment and emphatically analyze the impact of water quality factors.     

A characterization of the geometric distribution

We show that for a simple random sample from a discrete distribution on the positive integers, the regression ofX _(2∶n) onX _(1∶n) is linear with unit slope if and only if the distribution is geometric.     

Characterization of the geometric distribution by the form of a predictor

The problem of prodicting max (X₁X₂) given min(X₁X₂) is considered when Y₁and Y₂ are i.i.d random variables making positive integral values. It is provea tnat tne oest predictor is a linear Function or min(X₁,X₂); with unit slope Iff X₁, and X₂ have geometric distributions. As an extension of this result, the geometric distribution is characterized by the constancy of regression of min(X₁?X₂|, c) on inin(X₁,X₂) where c is any positive integer.     

Robust variable selection for the varying coefficient model based on composite L 1–L 2 regression

The varying coefficient model (VCM) is an important generalization of the linear regression model and many existing estimation procedures for VCM were built on L ₂ loss, which is popular for its mathematical beauty but is not robust to non-normal errors and outliers. In this paper, we address the problem of both robustness and efficiency of estimation and variable selection procedure based on the convex combined loss of L ₁ and L ₂ instead of only quadratic loss for VCM. By using local linear modeling method, the asymptotic normality of estimation is driven and a useful selection method is proposed for the weight of composite L ₁ and L ₂. Then the variable selection procedure is given by combining local kernel smoothing with adaptive group LASSO. With appropriate selection of tuning parameters by Bayesian information criterion (BIC) the theoretical properties of the new procedure, including consistency in variable selection and the oracle property in estimation, are established. The finite sample performance of the new method is investigated through simulation studies and the analysis of body fat data. Numerical studies show that the new method is better than or at least as well as the least square-based method in terms of both robustness and efficiency for variable selection.     

Dependent Right Censorship in the MOMW Distribution

In this paper, we consider paired survival data, in which pair members are subject to the same right censoring time, but they are dependent on each other. Assuming the Marshall–Olkin Multivariate Weibull distribution for the joint distribution of the lifetimes (X₁, X₂) and the censoring time X₃, we derive the joint density of the actual observed data and obtain maximum likelihood estimators, Bayes estimators and posterior regret Gamma minimax estimators of the unknown parameters under squared error loss and weighted squared error loss functions. We compare the performances of the maximum likelihood estimators and Bayes estimators numerically in terms of biases and estimated Mean Squared Error Loss.     

On the strong Kotz approximation of Dirichlet random vectors

Let (X ₁, X ₂) be a bivariate L _p -norm generalized symmetrized Dirichlet (LpGSD) random vector with parameters α₁,α₂. If p=α₁=α₂=2, then (X ₁, X ₂) is a spherical random vector. The estimation of the conditional distribution of Z _u *:=X ₂ | X ₁>u for u large is of some interest in statistical applications. When (X ₁, X ₂) is a spherical random vector with associated random radius in the Gumbel max-domain of attraction, the distribution of Z _u * can be approximated by a Gaussian distribution. Surprisingly, the same Gaussian approximation holds also for Z _u :=X ₂| X ₁=u. In this paper, we are interested in conditional limit results in terms of convergence of the density functions considering a d-dimensional LpGSD random vector. Stating our results for the bivariate setup, we show that the density function of Z _u * and Z _u can be approximated by the density function of a Kotz type I LpGSD distribution, provided that the associated random radius has distribution function in the Gumbel max-domain of attraction. Further, we present two applications concerning the asymptotic behaviour of concomitants of order statistics of bivariate Dirichlet samples and the estimation of the conditional quantile function.     

Bayesian quantile inference

The paper proposes a Bayesian interpretation of quantile regression that is shown to be equivalent to scale mixtures of normals leading to a skewed Laplace distribution. This representation of the model facilitates Bayesian analysis by means of Gibbs sampling with data augmentation, and nests regression in the L₁ norm as a special case. The new methods are applied to an analysis of the patents - R&D relationship for U.S. firms and unit root inference for the dollar-deutschemark exchange rate.     

Limiting behavior of randomly weighted averages of symmetric heavy-tailed random variables

In this paper we consider a sequence of independent continuous symmetric random variables X₁, X₂, …, with heavy-tailed distributions. Then we focus on limiting behavior of randomly weighted averages S_n = R⁽ⁿ⁾₁X₁ + ??? + R⁽ⁿ⁾_nX_n, where the random weights R⁽ⁿ⁾₁, …, R_n⁽ⁿ⁾ which are independent of X₁, X₂, …, X_n, are the cuts of (0, 1) by the n ? 1 order statistics from a uniform distribution. Indeed we prove that c_nS_n converges in distribution to a symmetric α-stable random variable with c_n = n^{1 ? 1/α}/Γ^1/α(α + 1).