Smoothed categorical regression based on direct kernel estimates

A nonparametric method is considered which yields smoothed estimates of the response probabilities when the response variable is categorical. The method is based on Lauder's (1983) direct kernel estimates which are extended to allow for ordinal kernels. Thus one can make use of the ordinal scale of the response variable. A class of predictive loss functions is introduced on which the cross-validatory choice of smoothing parameters is based. Plots of the smoothed response probabilities may be used to uncover the form of covariate effects     

Bootstrap bandwidth selection method for local linear estimator in exponential family models

Many biological experiments involve data whose distribution belongs to the exponential family. Such data are often analysed using generalised linear models but this method requires specification of the link function which can have strong influence on the resulting estimate. Instead a local method based on quasi-likelihood can be used, but the choice of the smoothing parameter is crucial for its performance. A bootstrap bandwidth selection method is proposed and shown to be consistent. Examples of application to data from biological and psychometric experiments are given.     

ON CROSS-VALIDATION FOR SMOOTHING SPLINES IN THE CASE OF DEPENDENT OBSERVATIONS

Cross-validation, as a popular tool for choosing a smoothing parameter, is generalized to the case of dependent observations. A general version of the ‘deletion theorem’ for representation and simplified calculation of cross-validatory criteria is given. Finally cross-validation is discussed in terms of penalized likelihoods as a method for model choice analogous to the Akaike information criterion.     

On algorithms for ordinary least squares regression spline fitting: A comparative study

Regression spline smoothing is a popular approach for conducting nonparametric regression. An important issue associated with it is the choice of a "theoretically best" set of knots. Different statistical model selection methods, such as Akaike's information criterion and generalized cross-validation, have been applied to derive different "theoretically best" sets of knots. Typically these best knot sets are defined implicitly as the optimizers of some objective functions. Hence another equally important issue concerning regression spline smoothing is how to optimize such objective functions. In this article different numerical algorithms that are designed for carrying out such optimization problems are compared by means of a simulation study. Both the univariate and bivariate smoothing settings will be considered. Based on the simulation results, recommendations for choosing a suitable optimization algorithm under various settings will be provided.     

Robust estimation for functional coefficient regression models with spatial data

A global smoothing procedure is developed using B-spline function approximation for estimating the unknown functions of a functional coefficient regression model with spatial data. A general formulation is used to treat mean regression, median regression, quantile regression and robust mean regression in one setting. The global convergence rates of the estimators of unknown coefficient functions are established. Various applications of the main results, including estimating conditional quantile coefficient functions and robustifying the mean regression coefficient functions are given. Finite sample properties of our procedures are studied through Monte Carlo simulations. A housing data example is used to illustrate the proposed methodology.     

Using wavelets for data smoothing: a simulation study

Wavelet shrinkage has been proposed as a highly adaptable approach to signal smoothing, which can produce optimum results in some senses. This paper examines the performance of the method as a function of its parameters, by simulation for time series showing gradual, rapid and discontinuous variations, for a range of signal-to-noise ratios. Some general conclusions are drawn. The effects of the choice of wavelet, choice of threshold and choice of resolution cut-off are considered. The use of the residual autocorrelation as a diagnostic tool is suggested.     

Estimators based on ranks for arma models

In this paper we introduce a new family of robust estimators for ARMA models. These estimators are defined by replacing the residual sample autocovariances in the least squares equations by autocovariances based on ranks. The asymptotic normality of the proposed estimators is provided. The efficiency and robustness properties of these estimators are studied. An adequate choice of the score functions gives estimators which have high efficiency under normality and robustness in the presence of outliers. The score functions can also be chosen so that the resulting estimators are asymptotically as efficient as the maximum likelihood estimators for a given distribution.     

The Buckley–James Estimator and Induced Smoothing

The Buckley–James (BJ) estimator is known to be consistent and efficient for a linear regression model with censored data. However, its application in practice is handicapped by the lack of a reliable numerical algorithm for finding the solution. For a given data set, the iterative approach may yield multiple solutions, or no solution at all. To alleviate this problem, we modify the induced smoothing approach originally proposed in 2005 by Brown & Wang. The resulting estimating functions become smooth, thus eliminating the tendency of the iterative procedure to oscillate between different parameter values. In addition to facilitating point estimation the smoothing approach enables easy evaluation of the projection matrix, thus providing a means of calculating standard errors. Extensive simulation studies were carried out to evaluate the performance of different estimators. In general, smoothing greatly alleviates numerical issues that arise in the estimation process. In particular, the one‐step smoothing estimator eliminates non‐convergence problems and performs similarly to full iteration until convergence. The proposed estimation procedure is illustrated using a dataset from a multiple myeloma study.     

Deconvolution methods for non-parametric inference in two-level mixed models

Summary.  We develop a general non-parametric approach to the analysis of clustered data via random effects. Assuming only that the link function is known, the regression functions and the distributions of both cluster means and observation errors are treated non-parametrically. Our argument proceeds by viewing the observation error at the cluster mean level as though it were a measurement error in an errors-in-variables problem, and using a deconvolution argument to access the distribution of the cluster mean. A Fourier deconvolution approach could be used if the distribution of the error-in-variables were known. In practice it is unknown, of course, but it can be estimated from repeated measurements, and in this way deconvolution can be achieved in an approximate sense. This argument might be interpreted as implying that large numbers of replicates are necessary for each cluster mean distribution, but that is not so; we avoid this requirement by incorporating statistical smoothing over values of nearby explanatory variables. Empirical rules are developed for the choice of smoothing parameter. Numerical simulations, and an application to real data, demonstrate small sample performance for this package of methodology. We also develop theory establishing statistical consistency.     

Penalised spline estimation for generalised partially linear single-index models

Generalised linear models are frequently used in modeling the relationship of the response variable from the general exponential family with a set of predictor variables, where a linear combination of predictors is linked to the mean of the response variable. We propose a penalised spline (P-spline) estimation for generalised partially linear single-index models, which extend the generalised linear models to include nonlinear effect for some predictors. The proposed models can allow flexible dependence on some predictors while overcome the “curse of dimensionality”. We investigate the P-spline profile likelihood estimation using the readily available R package mgcv, leading to straightforward computation. Simulation studies are considered under various link functions. In addition, we examine different choices of smoothing parameters. Simulation results and real data applications show effectiveness of the proposed approach. Finally, some large sample properties are established.     

Penalized triograms: total variation regularization for bivariate smoothing

Summary.  Hansen, Kooperberg and Sardy introduced a family of continuous, piecewise linear functions defined over adaptively selected triangulations of the plane as a general approach to statistical modelling of bivariate densities and regression and hazard functions. These triograms enjoy a natural affine equivariance that offers distinct advantages over competing tensor product methods that are more commonly used in statistical applications. Triograms employ basis functions consisting of linear 'tent functions' defined with respect to a triangulation of a given planar domain. As in knot selection for univariate splines, Hansen and colleagues adopted the regression spline approach of Stone. Vertices of the triangulation are introduced or removed sequentially in an effort to balance fidelity to the data and parsimony. We explore a smoothing spline variant of the triogram model based on a roughness penalty adapted to the piecewise linear structure of the triogram model. We show that the roughness penalty proposed may be interpreted as a total variation penalty on the gradient of the fitted function. The methods are illustrated with real and artificial examples, including an application to estimated quantile surfaces of land value in the Chicago metropolitan area.     

Local Adaptivity of Kernel Estimates with Plug-in Local Bandwidth Selectors

In non-parametric function estimation selection of a smoothing parameter is one of the most important issues. The performance of smoothing techniques depends highly on the choice of this parameter. Preferably the bandwidth should be determined via a data-driven procedure. In this paper we consider kernel estimators in a white noise model, and investigate whether locally adaptive plug-in bandwidths can achieve optimal global rates of convergence. We consider various classes of functions: Sobolev classes, bounded variation function classes, classes of convex functions and classes of monotone functions. We study the situations of pilot estimation with oversmoothing and without oversmoothing. Our main finding is that simple local plug-in bandwidth selectors can adapt to spatial inhomogeneity of the regression function as long as there are no local oscillations of high frequency. We establish the pointwise asymptotic distribution of the regression estimator with local plug-in bandwidth.     

Minimum Risk Invariant Estimators of a Continuous Cumulative Distribution Function

The problem of nonparametric minimum risk invariant estimation has engaged a good deal of attention in the literature and minimum risk invariant estimators (MRIE's) have been constructed for some special statistical models. We present a new and simple method of obtaining the MRIE's of a continuous cumulative distribution function (cdf) under a general invariant loss function. All the MRIE's, which are known from the literature, can be constructed by the method presented in the article, in particular, under the weighted quadratic, LINEX and entropy loss functions. This method enables also to construct the MRIE's in nonparametric statistical models which have not been considered until now. In particular, considering a family of nonparametric precautionary loss functions, a new class of MRIE's of the cdf has been found. We also give some general remarks on obtaining the MRIE's and a review concerning minimaxity and admissibility of MRIE's.     

Reference priors for shrinkage and smoothing parameters

A reference prior and corresponding reference posteriors are derived for a basic Normal variance components model with two components. Different parameterizations are considered, in particular one in terms of a shrinkage or smoothing parameter. Earlier results for the one-way ANOVA setting are generalized and a broad range of applications of the general results is indicated. Numerical examples of application to spline smoothing are given for illustration and the results compared with other well-known techniques considered to be “non-informative” about the smoothing parameter.     

An algorithm for generalized monotonic smoothing

In this paper, an algorithm for Generalized Monotonic Smoothing (GMS) is developed as an extension to exponential family models of the monotonic smoothing techniques proposed by Ramsay (1988, 1998a,b). A two-step algorithm is used to estimate the coefficients of bases and the linear term. We show that the algorithm can be embedded into the iterative re-weighted least square algorithm that is typically used to estimate the coefficients in Generalized Linear Models. Thus, the GMS estimator can be computed using existing routines in S-plus and other statistical software. We apply the GMS model to the Down's syndrome data set and compare the results with those from Generalized Additive Model estimation. The choice of smoothing parameter and testing of monotonicity are also discussed.     

Discrete associated kernels method and extensions

Discrete kernel estimation of a probability mass function (p.m.f.), often mentioned in the literature, has been far less investigated in comparison with continuous kernel estimation of a probability density function (p.d.f.). In this paper, we are concerned with a general methodology of discrete kernels for smoothing a p.m.f. f. We give a basic of mathematical tools for further investigations. First, we point out a generalizable notion of discrete associated kernel which is defined at each point of the support of f and built from any parametric discrete probability distribution. Then, some properties of the corresponding estimators are shown, in particular pointwise and global (asymptotical) properties. Other discrete kernels are constructed from usual discrete probability distributions such as Poisson, binomial and negative binomial. For small samples sizes, underdispersed discrete kernel estimators are more interesting than the empirical estimator; thus, an importance of discrete kernels is illustrated. The choice of smoothing bandwidth is classically investigated according to cross-validation and, novelly, to excess of zeros methods. Finally, a unification way of this method concerning the general probability function is discussed.     

Local Box–Cox transformation on time-varying parametric models for smoothing estimation of conditional CDF with longitudinal data

Nonparametric estimation and inferences of conditional distribution functions with longitudinal data have important applications in biomedical studies, such as epidemiological studies and longitudinal clinical trials. Estimation approaches without any structural assumptions may lead to inadequate and numerically unstable estimators in practice. We propose in this paper a nonparametric approach based on time-varying parametric models for estimating the conditional distribution functions with a longitudinal sample. Our model assumes that the conditional distribution of the outcome variable at each given time point can be approximated by a parametric model after local Box–Cox transformation. Our estimation is based on a two-step smoothing method, in which we first obtain the raw estimators of the conditional distribution functions at a set of disjoint time points, and then compute the final estimators at any time by smoothing the raw estimators. Applications of our two-step estimation method have been demonstrated through a large epidemiological study of childhood growth and blood pressure. Finite sample properties of our procedures are investigated through a simulation study. Application and simulation results show that smoothing estimation from time-variant parametric models outperforms the existing kernel smoothing estimator by producing narrower pointwise bootstrap confidence band and smaller root mean squared error.     

Discrete smoothing

The simplification of complex models which were originally envisaged to explain some data is considered as a discrete form of smoothing. In this sense data based model selection techniques lead to minimal and unavoidable initial smoothing. The same techniques may also be used for further smoothing if this seems necessary. For deterministic data parametric models which are usually used for stochastic data also provide convenient notches in the process of smoothing. The usual discrepancies can be used to measure the degree of smoothing. The methods for tables of means and tables of frequencies are described in more detail and examples of applications are given.     

On generalized variance of product of powered components and multiple stable Tweedie models

A flexible family of multivariate models, named multiple stable Tweedie (MST) models, is introduced and produces generalized variance functions which are products of powered components of the mean. These MST models are built from a fixed univariate stable Tweedie variable having a positive value domain, and the remaining random variables given the fixed one are also real independent Tweedie variables, with the same dispersion parameter equal to the fixed component. In this huge family of MST models, generalized variance estimators are explicitly pointed out by maximum likelihood method and, moreover, computably presented for the uniform minimum variance and unbiased approach. The second estimator is brought from modified Lévy measures of MST which lead to some solutions of particular Monge–Ampère equations.     

Non-parametric smoothing of the location model in mixed variable discrimination

The location model is a familiar basis for discriminant analysis of mixtures of categorical and continuous variables. Its usual implementation involves second-order smoothing, using multivariate regression for the continuous variables and log-linear models for the categorical variables. In spite of the smoothing, these procedures still require many parameters to be estimated and this in turn restricts the categorical variables to a small number if implementation is to be feasible. In this paper we propose non-parametric smoothing procedures for both parts of the model. The number of parameters to be estimated is dramatically reduced and the range of applicability thereby greatly increased. The methods are illustrated on several data sets, and the performances are compared with a range of other popular discrimination techniques. The proposed method compares very favourably with all its competitors.