Moment density estimation for positive random variables

An unknown moment-determinate cumulative distribution function or its density function can be recovered from corresponding moments and estimated from the empirical moments. This method of estimating an unknown density is natural in certain inverse estimation models like multiplicative censoring or biased sampling when the moments of unobserved distribution can be estimated via the transformed moments of the observed distribution. In this paper, we introduce a new nonparametric estimator of a probability density function defined on the positive real line, motivated by the above. Some fundamental properties of proposed estimator are studied. The comparison with traditional kernel density estimator is discussed.     

Empirical Likelihood Ratio Test for a Change-Point in Linear Regression Model

A nonparametric method based on the empirical likelihood is proposed to detect the change-point in the coefficient of linear regression models. The empirical likelihood ratio test statistic is proved to have the same asymptotic null distribution as that with classical parametric likelihood. Under some mild conditions, the maximum empirical likelihood change-point estimator is also shown to be consistent. The simulation results show the sensitivity and robustness of the proposed approach. The method is applied to some real datasets to illustrate the effectiveness.     

Estimating the density of a possibly missing response variable in nonlinear regression

This paper considers linear and nonlinear regression with a response variable that is allowed to be “missing at random”. The only structural assumptions on the distribution of the variables are that the errors have mean zero and are independent of the covariates. The independence assumption is important. It enables us to construct an estimator for the response density that uses all the observed data, in contrast to the usual local smoothing techniques, and which therefore permits a faster rate of convergence. The idea is to write the response density as a convolution integral which can be estimated by an empirical version, with a weighted residual-based kernel estimator plugged in for the error density. For an appropriate class of regression functions, and a suitably chosen bandwidth, this estimator is consistent and converges with the optimal parametric rate n^1/2. Moreover, the estimator is proved to be efficient (in the sense of Hájek and Le Cam) if an efficient estimator is used for the regression parameter.     

Weak and strong uniform consistency of a kernel error density estimator in nonparametric regression

This paper considers the problem of estimating the kernel error density function in nonparametric regression models. Sufficient conditions are given under which the kernel error density estimator based on nonparametric residuals is uniformly weakly and strongly consistent.     

Nonparametric survival regression using the beta-Stacy process

A novel class of hierarchical nonparametric Bayesian survival regression models for time-to-event data with uninformative right censoring is introduced. The survival curve is modeled as a random function whose prior distribution is defined using the beta-Stacy (BS) process. The prior mean of each survival probability and its prior variance are linked to a standard parametric survival regression model. This nonparametric survival regression can thus be anchored to any reference parametric form, such as a proportional hazards or an accelerated failure time model, allowing substantial departures of the predictive survival probabilities when the reference model is not supported by the data. Also, under this formulation the predictive survival probabilities will be close to the empirical survival distribution near the mode of the reference model and they will be shrunken towards its probability density in the tails of the empirical distribution.     

Estimating the error distribution function in semiparametric additive regression models

We consider semiparametric additive regression models with a linear parametric part and a nonparametric part, both involving multivariate covariates. For the nonparametric part we assume two models. In the first, the regression function is unspecified and smooth; in the second, the regression function is additive with smooth components. Depending on the model, the regression curve is estimated by suitable least squares methods. The resulting residual-based empirical distribution function is shown to differ from the error-based empirical distribution function by an additive expression, up to a uniformly negligible remainder term. This result implies a functional central limit theorem for the residual-based empirical distribution function. It is used to test for normal errors.     

Weighted quantile regression with missing covariates using empirical likelihood

This paper proposes an empirical likelihood-based weighted (ELW) quantile regression approach for estimating the conditional quantiles when some covariates are missing at random. The proposed ELW estimator is computationally simple and achieves semiparametric efficiency if the probability of missingness is correctly specified. The limiting covariance matrix of the ELW estimator can be estimated by a resampling technique, which does not involve nonparametric density estimation or numerical derivatives. Simulation results show that the ELW method works remarkably well in finite samples. A real data example is used to illustrate the proposed ELW method.     

Estimation of regression parameters in a semiparametric transformation model

A semiparametric approach to model skewed/heteroscedastic regression data is discussed. We work with a semiparametric transform-both-sides regression model, which contains a parametric regression function and a nonparametric transformation. This model is adequate when the relationship between the median response and the explanatory variable has been specified by a theoretical result or a previous empirical study. The transform-both-sides model with a parametric transformation has been studied extensively and applied successfully to a number data sets. Allowing a nonparametric transformation function increases the flexibility of the model. In this article, we estimate the nonparametric transformation function by the conditional kernel density approach developed by Wang and Ruppert (1995), and then use a pseudo-maximum likelihood estimator to estimate the regression parameters. This estimate of the regression parameters has not been studied previously. In this article, the asymptotic distribution of this pseudo-MLE is derived. We also show that when σ, the standard deviation of the error, goes to zero (small σ asymptotics), this estimator is adaptive. Adaptive means that the regression parameters are estimated as precisely as when the transformation is known exactly. A similar result holds in the parametric approaches of Carroll and Ruppert (1984) and Ruppert and Aldershof (1989). Simulated and real examples are provided to illustrate the performance of the proposed estimator for finite sample size.     

Empirical bayes estimation of binomial parameter with symmetric priors

This paper deals with the problem of estimating the binomial parameter via the nonparametric empirical Bayes approach. This estimation problem has the feature that estimators which are asymptotically optimal in the usual empirical Bayes sense do not exist (Robbins (1958, 1964)), However, as pointed out by Liang (1934) and Gupta and Liang (1988), it is possible to construct asymptotically optimal empirical Bayes estimators if the unknown prior is symmetric about the point 1/2, In this paper, assuming symmetric priors a monotone empirical Bayes estimator is constructed by using the isotonic regression method. This estimator is asymptotically optimal in the usual empirical Bayes sense. The corresponding rate of convergence is investigated and shown to be of order n^-1, where n is the number of past observations at hand.     

Extended Glivenko–Cantelli Theorem in Nonparametric Regression

In this paper, we consider the uniform strong consistency of the cumulative distribution function estimator in nonparametric regression. We obtain the extended Glivenko–Cantelli theorem for the residual-based empirical distribution function.     

Asymptotically efficient estimators for nonparametric heteroscedastic regression models

This paper concerns the estimation of a function at a point in nonparametric heteroscedastic regression models with Gaussian noise or noise having unknown distribution. In those cases an asymptotically efficient kernel estimator is constructed for the minimax absolute error risk.     

Penalized empirical likelihood inference for sparse additive hazards regression with a diverging number of covariates

High-dimensional sparse modeling with censored survival data is of great practical importance, as exemplified by applications in high-throughput genomic data analysis. In this paper, we propose a class of regularization methods, integrating both the penalized empirical likelihood and pseudoscore approaches, for variable selection and estimation in sparse and high-dimensional additive hazards regression models. When the number of covariates grows with the sample size, we establish asymptotic properties of the resulting estimator and the oracle property of the proposed method. It is shown that the proposed estimator is more efficient than that obtained from the non-concave penalized likelihood approach in the literature. Based on a penalized empirical likelihood ratio statistic, we further develop a nonparametric likelihood approach for testing the linear hypothesis of regression coefficients and constructing confidence regions consequently. Simulation studies are carried out to evaluate the performance of the proposed methodology and also two real data sets are analyzed.     

Estimation of a distribution function bounded by two known distribution functions

The problem of estimation of a cumulative distribution function (cdf), bounded by two known cdf's, is considered. An estimator satisfying the desired restriction has been obtained by suitably adjusting the empirical cdf. Consistency of the adjusted estimator has been established and its mean square error (MSE) has been shown to be smallerthan that of the empirical cdf. The new estimator has been comparedwith the empirical cdf for some special cases.     

A weighted bootstrap approximation for comparing the error distributions in nonparametric regression

Several procedures have been proposed for testing the equality of error distributions in two or more nonparametric regression models. Here we deal with methods based on comparing estimators of the cumulative distribution function (CDF) of the errors in each population to an estimator of the common CDF under the null hypothesis. The null distribution of the associated test statistics has been approximated by means of a smooth bootstrap (SB) estimator. This paper proposes to approximate their null distribution through a weighted bootstrap. It is shown that it produces a consistent estimator. The finite sample performance of this approximation is assessed by means of a simulation study, where it is also compared to the SB. This study reveals that, from a computational point of view, the proposed approximation is more efficient than the one provided by the SB.     

SEMIPARAMETRIC ESTIMATION OF THE ERROR DISTRIBUTION IN MULTIVARIATE REGRESSION USING COPULAS

A semiparametric method is developed to estimate the dependence parameter and the joint distribution of the error term in the multivariate linear regression model. The nonparametric part of the method treats the marginal distributions of the error term as unknown, and estimates them using suitable empirical distribution functions. Then the dependence parameter is estimated by either maximizing a pseudolikelihood or solving an estimating equation. It is shown that this estimator is asymptotically normal, and a consistent estimator of its large sample variance is given. A simulation study shows that the proposed semiparametric method is better than the parametric ones available when the error distribution is unknown, which is almost always the case in practice. It turns out that there is no loss of asymptotic efficiency as a result of the estimation of regression parameters. An empirical example on portfolio management is used to illustrate the method.     

A general approach to heteroscedastic linear regression

Our article presents a general treatment of the linear regression model, in which the error distribution is modelled nonparametrically and the error variances may be heteroscedastic, thus eliminating the need to transform the dependent variable in many data sets. The mean and variance components of the model may be either parametric or nonparametric, with parsimony achieved through variable selection and model averaging. A Bayesian approach is used for inference with priors that are data-based so that estimation can be carried out automatically with minimal input by the user. A Dirichlet process mixture prior is used to model the error distribution nonparametrically; when there are no regressors in the model, the method reduces to Bayesian density estimation, and we show that in this case the estimator compares favourably with a well-regarded plug-in density estimator. We also consider a method for checking the fit of the full model. The methodology is applied to a number of simulated and real examples and is shown to work well.     

Goodness-of-fit testing under long memory

This paper discusses the problem of fitting a distribution function to the marginal distribution of a long memory moving average process. Because of the uniform reduction principle, unlike in the i.i.d. set up, classical tests based on empirical process are relatively easy to implement. More importantly, we discuss fitting the marginal distribution of the error process in location, scale, location–scale and linear regression models. An interesting observation is that in the location model, location–scale model, or more generally in the linear regression models with non-zero intercept parameter, the null weak limit of the first order difference between the residual empirical process and the null model is degenerate at zero, and hence it cannot be used to fit an error distribution in these models for the large samples. This finding is in sharp contrast to a recent claim of Chan and Ling (2008) that the null weak limit of such a process is a continuous Gaussian process. This note also proposes some tests based on the second order difference for the location case. Another finding is that residual empirical process tests in the scale problem are robust against not knowing the scale parameter.     

Identity for the NPMLE in Censored Data Models

We derive an identity for nonparametric maximum likelihood estimators (NPMLE) and regularized MLEs in censored data models which expresses the standardized maximum likelihood estimator in terms of the standardized empirical process. This identity provides an effective starting point in proving both consistency and efficiency of NPMLE and regularized MLE. The identity and corresponding method for proving efficiency is illustrated for the NPMLE in the univariate right-censored data model, the regularized MLE in the current status data model and for an implicit NPMLE based on a mixture of right-censored and current status data. Furthermore, a general algorithm for estimation of the limiting variance of the NPMLE is provided. This revised version was published online in July 2006 with corrections to the Cover Date.     

The optimal Lp norm estimator in linear regression models

The least squares estimator is usually applied when estimating the parameters in linear regression models. As this estimator is sensitive to departures from normality in the residual distribution, several alternatives have been proposed. The L_p norm estimators is one class of such alternatives. It has been proposed that the kurtosis of the residual distribution be taken into account when a choice of estimator in the L_p norm class is made (i.e. the choice of p). In this paper, the asymtotic variance of the estimators is used as the criterion in the choice of p. It is shown that when this criterion is applied, other characteristics of the residual distribution than the kurtosis (namely moments of order p-2 and 2p-2) are important.     

Estimation of the Spectral Density with Assigned Risk

It is well known that adaptive sequential nonparametric estimation of differentiable functions with assigned mean integrated squared error and minimax expected stopping time is impossible. In other words, no sequential estimator can compete with an oracle estimator that knows how many derivatives an estimated curve has. Differentiable functions are typical in probability density and regression models but not in spectral density models, where considered functions are typically smoother. This paper shows that for a large class of spectral densities, which includes spectral densities of classical autoregressive moving average processes, an adaptive minimax sequential estimation with assigned mean integrated squared error is possible. Furthermore, a two‐stage sequential procedure is proposed, which is minimax and adaptive to smoothness of an underlying spectral density.