Robust signed-rank estimation and variable selection for semi-parametric additive partial linear models

A fully nonparametric model may not perform well or when the researcher wants to use a parametric model but the functional form with respect to a subset of the regressors or the density of the errors is not known. This becomes even more challenging when the data contain gross outliers or unusual observations. However, in practice the true covariates are not known in advance, nor is the smoothness of the functional form. A robust model selection approach through which we can choose the relevant covariates components and estimate the smoothing function may represent an appealing tool to the solution. A weighted signed-rank estimation and variable selection under the adaptive lasso for semi-parametric partial additive models is considered in this paper. B-spline is used to estimate the unknown additive nonparametric function. It is shown that despite using B-spline to estimate the unknown additive nonparametric function, the proposed estimator has an oracle property. The robustness of the weighted signed-rank approach for data with heavy-tail, contaminated errors, and data containing high-leverage points are validated via finite sample simulations. A practical application to an economic study is provided using an updated Canadian household gasoline consumption data.     

Efficient estimation in partially linear single‐index models for longitudinal data

In this paper, we consider the estimation of both the parameters and the nonparametric link function in partially linear single‐index models for longitudinal data that may be unbalanced. In particular, a new three‐stage approach is proposed to estimate the nonparametric link function using marginal kernel regression and the parametric components with generalized estimating equations. The resulting estimators properly account for the within‐subject correlation. We show that the parameter estimators are asymptotically semiparametrically efficient. We also show that the asymptotic variance of the link function estimator is minimized when the working error covariance matrices are correctly specified. The new estimators are more efficient than estimators in the existing literature. These asymptotic results are obtained without assuming normality. The finite‐sample performance of the proposed method is demonstrated by simulation studies. In addition, two real‐data examples are analyzed to illustrate the methodology.     

Monte Carlo methods for nonparametric survival model determination

In the causal analysis of survival data a time-based response is related to a set of explanatory variables. Definition of the relation between the time and the covariates may become a difficult task, particularly in the preliminary stage, when the information is limited. Through a nonparametric approach, we propose to estimate the survival function allowing to evaluate the relative importance of each potential explanatory variable, in a simple and explanatory fashion. To achieve this aim, each of the explanatory variables is used to partition the observed survival times. The observations are assumed to be partially exchangeable according to such partition. We then consider, conditionally on each partition, a hierarchical nonparametric Bayesian model on the hazard functions. We define and compare different prior distribution for the hazard functions.     

Asymptotic properties of mean cumulative function estimators from window-observation recurrence data

A variety of nonparametric and parametric methods have been used to estimate the mean cumulative function (MCF) for the recurrence data collected from the counting process. When the recurrence histories of some units are available in disconnected observation windows with gaps in between, Zuo et al. (2008) showed that both the nonparametric and parametric methods can be extended to estimate the MCF. In this article, we establish some asymptotic properties of the MCF estimators for the window-observation recurrence data.     

Specification of household engel curves by nonparametric regression

This paper demonstrates the usefulness of nonparametric regression analysis for functional specfication of houshold Engel curves.

After a brief review in section 2 of the literature on demand functions and equivalence scales and the functional specifications used, we first discuss in section 3 the issues of using income versus total expenditure, the origin and nature of the error terms in the light of utility theroy, and the interpretation of empirical demand functions. we shall reach the unorthodox view that household demand functions should be interpreted as conditional expectations relative to prices, household composition and either income or the conditional expectation of total expenditure (rather that total expenditure itself), where the latter conditional expectation is taken relative to income, prices and household composition. these two forms appear to be equivalent. this result also solves the simultaneity problem: the error variance matrix is no longer singular. Moreover, the errors are in general heteroskedastic.

In section 4 we discuss the model and the data, and in section 5 we review the nonparametric kernal regression approach.

In section 6 we derive the functional form of our household engel curves from nonparametric regression results, using the 1980 budget survey for the netherlands, in order to avoid model misspecification. thus the modl is derived directly from the data, without restricting its functional form. the nonparametric regression results are then translated to suitable parametric functional specifications, i.e., we choose parametric functional forms in accordance with the nanparametric regression results. these parametric specification are estimated by least squares, and various parameter restrictions are tested in order to simplify the models. this yields very simple final specifications of the household engel curves involved, namely linear functions of income and the number of children in two age groups.     

Strictly monotone and smooth nonparametric regression for two or more variables

The authors propose a new monotone nonparametric estimate for a regression function of two or more variables. Their method consists in applying successively one‐dimensional isotonization procedures on an initial, unconstrained nonparametric regression estimate. In the case of a strictly monotone regression function, they show that the new estimate and the initial one are first‐order asymptotic equivalent; they also establish asymptotic normality of an appropriate standardization of the new estimate. In addition, they show that if the regression function is not monotone in one of its arguments, the new estimate and the initial one have approximately the same L^p‐norm. They illustrate their approach by means of a simulation study, and two data examples are analyzed.     

Estimation of regression parameters in a semiparametric transformation model

A semiparametric approach to model skewed/heteroscedastic regression data is discussed. We work with a semiparametric transform-both-sides regression model, which contains a parametric regression function and a nonparametric transformation. This model is adequate when the relationship between the median response and the explanatory variable has been specified by a theoretical result or a previous empirical study. The transform-both-sides model with a parametric transformation has been studied extensively and applied successfully to a number data sets. Allowing a nonparametric transformation function increases the flexibility of the model. In this article, we estimate the nonparametric transformation function by the conditional kernel density approach developed by Wang and Ruppert (1995), and then use a pseudo-maximum likelihood estimator to estimate the regression parameters. This estimate of the regression parameters has not been studied previously. In this article, the asymptotic distribution of this pseudo-MLE is derived. We also show that when σ, the standard deviation of the error, goes to zero (small σ asymptotics), this estimator is adaptive. Adaptive means that the regression parameters are estimated as precisely as when the transformation is known exactly. A similar result holds in the parametric approaches of Carroll and Ruppert (1984) and Ruppert and Aldershof (1989). Simulated and real examples are provided to illustrate the performance of the proposed estimator for finite sample size.     

Inference of the Trend in a Partially Linear Model with Locally Stationary Regressors

In this article, we construct the uniform confidence band (UCB) of nonparametric trend in a partially linear model with locally stationary regressors. A two-stage semiparametric regression is employed to estimate the trend function. Based on this estimate, we develop an invariance principle to construct the UCB of the trend function. The proposed methodology is used to estimate the Non-Accelerating Inflation Rate of Unemployment (NAIRU) in the Phillips Curve and to perform inference of the parameter based on its UCB. The empirical results strongly suggest that the U.S. NAIRU is time-varying.     

Estimation of partially linear single-index additive hazards model with current status data

In this paper, we introduce a partially linear single-index additive hazards model with current status data. Both the unknown link function of the single-index term and the cumulative baseline hazard function are approximated by B-splines under a monotonicity constraint on the latter. The sieve method is applied to estimate the nonparametric and parametric components simultaneously. We show that, when the nonparametric link function is an exact B-spline, the resultant estimator of regression parameter vector is asymptotically normal and achieves the semiparametric information bound and the rate of convergence of the estimator for the cumulative baseline hazard function is optimal. Simulation studies are presented to examine the finite sample performance of the proposed estimation method. For illustration, we apply the method to a clinical dataset with current status outcome.     

Analytic computation of nonparametric Marsan–Lengliné estimates for Hawkes point processes

In 2008, Marsan and Lengliné presented a nonparametric way to estimate the triggering function of a Hawkes process. Their method requires an iterative and computationally intensive procedure which ultimately produces only approximate maximum likelihood estimates (MLEs) whose asymptotic properties are poorly understood. Here, we note a mathematical curiosity that allows one to compute, directly and extremely rapidly, exact MLEs of the nonparametric triggering function. The method here requires that the number q of intervals on which the nonparametric estimate is sought equals the number n of observed points. The resulting estimates have very high variance but may be smoothed to form more stable estimates. The performance and computational efficiency of the proposed method is verified in two disparate, highly challenging simulation scenarios: first to estimate the triggering functions, with simulation-based 95% confidence bands, for earthquakes and their aftershocks in Loma Prieta, California, and second, to characterise triggering in confirmed cases of plague in the United States over the last century. In both cases, the proposed estimator can be used to describe the rate of contagion of the processes in detail, and the computational efficiency of the estimator facilitates the construction of simulation-based confidence intervals.     

Specification of household engel curves by nonparametric regression

This paper demonstrates the usefulness of nonparametric regression analysis for functional specfication of houshold Engel curves.

After a brief review in section 2 of the literature on demand functions and equivalence scales and the functional specifications used, we first discuss in section 3 the issues of using income versus total expenditure, the origin and nature of the error terms in the light of utility theroy, and the interpretation of empirical demand functions. we shall reach the unorthodox view that household demand functions should be interpreted as conditional expectations relative to prices, household composition and either income or the conditional expectation of total expenditure (rather that total expenditure itself), where the latter conditional expectation is taken relative to income, prices and household composition. these two forms appear to be equivalent. this result also solves the simultaneity problem: the error variance matrix is no longer singular. Moreover, the errors are in general heteroskedastic.

In section 4 we discuss the model and the data, and in section 5 we review the nonparametric kernal regression approach.

In section 6 we derive the functional form of our household engel curves from nonparametric regression results, using the 1980 budget survey for the netherlands, in order to avoid model misspecification. thus the modl is derived directly from the data, without restricting its functional form. the nonparametric regression results are then translated to suitable parametric functional specifications, i.e., we choose parametric functional forms in accordance with the nanparametric regression results. these parametric specification are estimated by least squares, and various parameter restrictions are tested in order to simplify the models. this yields very simple final specifications of the household engel curves involved, namely linear functions of income and the number of children in two age groups.     

Nonparametric Panel Estimation of Labor Supply

In this article, we estimate structural labor supply with piecewise-linear budgets and nonseparable endogenous unobserved heterogeneity. We propose a two-stage method to address the endogeneity issue that comes from the correlation between the covariates and unobserved heterogeneity. In the first stage, Evdokimov’s nonparametric de-convolution method serves to identify the conditional distribution of unobserved heterogeneity from the quasi-reduced model that uses panel data. In the second stage, the conditional distribution is plugged into the original structural model to estimate labor supply. We apply this methodology to estimate the labor supply of U.S. married men in 2004 and 2005. Our empirical work demonstrates that ignoring the correlation between the covariates and unobserved heterogeneity will bias the estimates of wage elasticities upward. The labor elasticity estimated from a fixed effects model is less than half of that obtained from a random effects model.     

A fast algorithm for univariate log‐concave density estimation

A new fast algorithm for computing the nonparametric maximum likelihood estimate of a univariate log‐concave density is proposed and studied. It is an extension of the constrained Newton method for nonparametric mixture estimation. In each iteration, the newly extended algorithm includes, if necessary, new knots that are located via a special directional derivative function. The algorithm renews the changes of slope at all knots via a quadratically convergent method and removes the knots at which the changes of slope become zero. Theoretically, the characterisation of the nonparametric maximum likelihood estimate is studied and the algorithm is guaranteed to converge to the unique maximum likelihood estimate. Numerical studies show that it outperforms other algorithms that are available in the literature. Applications to some real‐world financial data are also given.     

Asymptotic results for model robust regression

Since the mid 1980's many statisticians have studied methods for combining parametric and nonparametric models to improve the quality of fits in a regression problem. Notably Einsporn (1987) proposed the Model Robust Regression 1 estimate (MRRl) in which the parametric function, f, and the nonparametric functiong were combined in a straightforward fashion via the use of a mixing parameter, λ This technique was studied extensively atsmall samples and was shown to be quite effective at modeling various unusual functions. In this paper we have asymptotic results for the MRRl estimate in the case where λ is theoretically optimal, is asymptotically optimal and data driven, and is chosen with the PRESS statistic (Allen, 1971) We demonstrate that the MRRl estimate with λchosen by the PRESS statistic is slightly inferior asymptotically to the other two estimates, but, nevertheless possesses positive asymptotic qualities.     

Plug-in bandwidth choice for estimation of nonparametric part in partial linear regression models with strong mixing errors

Consider a regression model where the regression function is the sum of a linear and a nonparametric component. Assuming that the errors of the model follow a stationary strong mixing process with mean zero, the problem of bandwidth selection for a kernel estimator of the nonparametric component is addressed here. We obtain an asymptotic expression for an optimal band-width and we propose to use a plug-in methodology in order to estimate this bandwidth through preliminary estimates of the unknown quantities. Asymptotic optimality for the plug-in bandwidth is established.     

Kernel Density-Based Linear Regression Estimate

For linear regression models with non normally distributed errors, the least squares estimate (LSE) will lose some efficiency compared to the maximum likelihood estimate (MLE). In this article, we propose a kernel density-based regression estimate (KDRE) that is adaptive to the unknown error distribution. The key idea is to approximate the likelihood function by using a nonparametric kernel density estimate of the error density based on some initial parameter estimate. The proposed estimate is shown to be asymptotically as efficient as the oracle MLE which assumes the error density were known. In addition, we propose an EM type algorithm to maximize the estimated likelihood function and show that the KDRE can be considered as an iterated weighted least squares estimate, which provides us some insights on the adaptiveness of KDRE to the unknown error distribution. Our Monte Carlo simulation studies show that, while comparable to the traditional LSE for normal errors, the proposed estimation procedure can have substantial efficiency gain for non normal errors. Moreover, the efficiency gain can be achieved even for a small sample size.     

Article: 2

Summary. Searching for an effective dimension reduction space is an important problem in regression, especially for high dimensional data. We propose an adaptive approach based on semiparametric models, which we call the (conditional) minimum average variance estimation (MAVE) method, within quite a general setting. The MAVE method has the following advantages. Most existing methods must undersmooth the nonparametric link function estimator to achieve a faster rate of consistency for the estimator of the parameters (than for that of the nonparametric function). In contrast, a faster consistency rate can be achieved by the MAVE method even without undersmoothing the nonparametric link function estimator. The MAVE method is applicable to a wide range of models, with fewer restrictions on the distribution of the covariates, to the extent that even time series can be included. Because of the faster rate of consistency for the parameter estimators, it is possible for us to estimate the dimension of the space consistently. The relationship of the MAVE method with other methods is also investigated. In particular, a simple outer product gradient estimator is proposed as an initial estimator. In addition to theoretical results, we demonstrate the efficacy of the MAVE method for high dimensional data sets through simulation. Two real data sets are analysed by using the MAVE approach.     

Nonparametric maximum likelihood estimation for shifted curves

The analysis of a sample of curves can be done by self-modelling regression methods. Within this framework we follow the ideas of nonparametric maximum likelihood estimation known from event history analysis and the counting process set-up. We derive an infinite dimensional score equation and from there we suggest an algorithm to estimate the shape function for a simple shape invariant model. The nonparametric maximum likelihood estimator that we find turns out to be a Nadaraya–Watson-like estimator, but unlike in the usual kernel smoothing situation we do not need to select a bandwidth or even a kernel function, since the score equation automatically selects the shape and the smoothing parameter for the estimation. We apply the method to a sample of electrophoretic spectra to illustrate how it works.     

Nonparametric Estimation of Distribution Functions of Nonstandard Mixtures

ABSTRACT

Nonstandard mixtures are those that result from a mixture of a discrete and a continuous random variable. They arise in practice, for example, in medical studies of exposure. Here, a random variable that models exposure might have a discrete mass point at no exposure, but otherwise may be continuous. In this article we explore estimating the distribution function associated with such a random variable from a nonparametric viewpoint. We assume that the locations of the discrete mass points are known so that we will be able to apply a classical nonparametric smoothing approach to the problem. The proposed estimator is a mixture of an empirical distribution function and a kernel estimate of a distribution function. A simple theoretical argument reveals that existing bandwidth selection algorithms can be applied to the smooth component of this estimator as well. The proposed approach is applied to two example sets of data.     

Nonparametric estimation of a log-variance function in scale-space

In a nonparametric regression setting, we consider the kernel estimation of the logarithm of the error variance function, which might be assumed to be homogeneous or heterogeneous. The objective of the present study is to discover important features in the variation of the data at multiple locations and scales based on a nonparametric kernel smoothing technique. Traditional kernel approaches estimate the function by selecting an optimal bandwidth, but it often turns out to be unsatisfying in practice. In this paper, we develop a SiZer (SIgnificant ZERo crossings of derivatives) tool based on a scale-space approach that provides a more flexible way of finding meaningful features in the variation. The proposed approach utilizes local polynomial estimators of a log-variance function using a wide range of bandwidths. We derive the theoretical quantile of confidence intervals in SiZer inference and also study the asymptotic properties of the proposed approach in scale-space. A numerical study via simulated and real examples demonstrates the usefulness of the proposed SiZer tool.