Estimation and prediction for spatial generalized linear mixed models using high order Laplace approximation

Estimation and prediction in generalized linear mixed models are often hampered by intractable high dimensional integrals. This paper provides a framework to solve this intractability, using asymptotic expansions when the number of random effects is large. To that end, we first derive a modified Laplace approximation when the number of random effects is increasing at a lower rate than the sample size. Second, we propose an approximate likelihood method based on the asymptotic expansion of the log-likelihood using the modified Laplace approximation which is maximized using a quasi-Newton algorithm. Finally, we define the second order plug-in predictive density based on a similar expansion to the plug-in predictive density and show that it is a normal density. Our simulations show that in comparison to other approximations, our method has better performance. Our methods are readily applied to non-Gaussian spatial data and as an example, the analysis of the rhizoctonia root rot data is presented.     

Some Nonparametric Asymptotic Results for a Class of Stochastic Processes

This article provides a solution of a generalized eigenvalue problem for integrated processes of order 2 in a nonparametric framework. Our analysis focuses on a pair of random matrices related to such integrated process. The matrices are constructed considering some weight functions. Under asymptotic conditions on such weights, convergence results in distribution are obtained and the generalized eigenvalue problem is solved. Differential equations and stochastic calculus theory are used.     

Nonparametric Estimation and Forecasting for Time-Varying Coefficient Realized Volatility Models

This article introduces a new specification for the heterogenous autoregressive (HAR) model for the realized volatility of S&P 500 index returns. In this modeling framework, the coefficients of the HAR are allowed to be time-varying with unspecified functional forms. The local linear method with the cross-validation (CV) bandwidth selection is applied to estimate the time-varying coefficient HAR (TVC-HAR) model, and a bootstrap method is used to construct the point-wise confidence bands for the coefficient functions. Furthermore, the asymptotic distribution of the proposed local linear estimators of the TVC-HAR model is established under some mild conditions. The results of the simulation study show that the local linear estimator with CV bandwidth selection has favorable finite sample properties. The outcomes of the conditional predictive ability test indicate that the proposed nonparametric TVC-HAR model outperforms the parametric HAR and its extension to HAR with jumps and/or GARCH in terms of multi-step out-of-sample forecasting, in particular in the post-2003 crisis and 2007 global financial crisis (GFC) periods, during which financial market volatilities were unduly high.     

Asymptotically Optimal Nonparametric Empirical Bayes Via Predictive Recursion

An empirical Bayes problem has an unknown prior to be estimated from data. The predictive recursion (PR) algorithm provides fast nonparametric estimation of mixing distributions and is ideally suited for empirical Bayes applications. This article presents a general notion of empirical Bayes asymptotic optimality, and it is shown that PR-based procedures satisfy this property under certain conditions. As an application, the problem of in-season prediction of baseball batting averages is considered. There the PR-based empirical Bayes rule performs well in terms of prediction error and ability to capture the distribution of the latent features.     

An empirical likelihood goodness-of-fit test for time series

Summary. Standard goodness-of-fit tests for a parametric regression model against a series of nonparametric alternatives are based on residuals arising from a fitted model. When a parametric regression model is compared with a nonparametric model, goodness-of-fit testing can be naturally approached by evaluating the likelihood of the parametric model within a nonparametric framework. We employ the empirical likelihood for an α -mixing process to formulate a test statistic that measures the goodness of fit of a parametric regression model. The technique is based on a comparison with kernel smoothing estimators. The empirical likelihood formulation of the test has two attractive features. One is its automatic consideration of the variation that is associated with the nonparametric fit due to empirical likelihood's ability to Studentize internally. The other is that the asymptotic distribution of the test statistic is free of unknown parameters, avoiding plug-in estimation. We apply the test to a discretized diffusion model which has recently been considered in financial market analysis.     

A comparison of nonparametric priors in hierarchical mixture modelling for AFT regression

We will pursue a Bayesian nonparametric approach in the hierarchical mixture modelling of lifetime data in two situations: density estimation, when the distribution is a mixture of parametric densities with a nonparametric mixing measure, and accelerated failure time (AFT) regression modelling, when the same type of mixture is used for the distribution of the error term. The Dirichlet process is a popular choice for the mixing measure, yielding a Dirichlet process mixture model for the error; as an alternative, we also allow the mixing measure to be equal to a normalized inverse-Gaussian prior, built from normalized inverse-Gaussian finite dimensional distributions, as recently proposed in the literature. Markov chain Monte Carlo techniques will be used to estimate the predictive distribution of the survival time, along with the posterior distribution of the regression parameters. A comparison between the two models will be carried out on the grounds of their predictive power and their ability to identify the number of components in a given mixture density.     

Efficient computation of the cdf of the maximal difference between a Brownian bridge and its concave majorant

In this paper, we describe two computational methods for calculating the cumulative distribution function and the upper quantiles of the maximal difference between a Brownian bridge and its concave majorant. The first method has two different variants that are both based on a Monte Carlo approach, whereas the second uses the Gaver–Stehfest (GS) algorithm for the numerical inversion of the Laplace transform. If the former method is straightforward to implement, it is very much outperformed by the GS algorithm, which provides a very accurate approximation of the cumulative distribution as well as its upper quantiles. Our numerical work has a direct application in statistics: the maximal difference between a Brownian bridge and its concave majorant arises in connection with a nonparametric test for monotonicity of a density or regression curve on [0,1]. Our results can be used to construct very accurate rejection region for this test at a given asymptotic level.     

Testing for changes in autocovariances of nonparametric time series models

In the literature on change-point analysis, much attention has been paid to detecting changes in certain marginal characteristics, such as mean, variance, and marginal distribution. For time series data with nonparametric time trend, we study the change-point problem for the autocovariance structure of the unobservable error process. To derive the asymptotic distribution of the cumulative sum test statistic, we develop substantial theory for uniform convergence of weighted partial sums and weighted quadratic forms. Our asymptotic results improve upon existing works in several important aspects. The performance of the test statistic is examined through simulations and an application to interest rates data.     

样本选择模型的一个简单半参数估计量

本文发展了一个针对样本选择模型的两阶段半参数估计量,其首先在第一阶段基于对数欧几里得分布差异测度估计离散选择概率,进而在第二阶段利用非参数sieve方法估计一个包含参数和非参数部分的部分线性模型以得到模型参数的估计。相对于文献中已有的半参数估计量,该估计量的计算更加简便,且计算负担相对较小。我们说明了该半参数估计量的一致性和渐近正态性,同时给出了其渐近方差的计算公式。蒙特卡洛模拟结果符合我们的理论结论。     

A goodness-of-fit test for logistic-normal models using nonparametric smoothing method

Logistic-normal models can be applied for analysis of longitudinal binary data. The aim of this article is to propose a goodness-of-fit test using nonparametric smoothing techniques for checking the adequacy of logistic-normal models. Moreover, the leave-one-out cross-validation method for selecting the suitable bandwidth is developed. The quadratic form of the proposed test statistic based on smoothing residuals provides a global measure for checking the model with categorical and continuous covariates. The formulae of expectation and variance of the proposed statistics are derived, and their asymptotic distribution is approximated by a scaled chi-squared distribution. The power performance of the proposed test for detecting the interaction term or the squared term of continuous covariates is examined by simulation studies. A longitudinal dataset is utilized to illustrate the application of the proposed test.     

A unified approach to rank tests for mixed models

The nonparametric version of the classical mixed model is considered and the common hypotheses of (parametric) main effects and interactions are reformulated in a nonparametric setup. To test these nonparametric hypotheses, the asymptotic distributions of quadratic forms of rank statistics are derived in a general framework which enables the derivation of the statistics for the nonparametric hypotheses of the fixed treatment effects and interactions in an arbitrary mixed model. The procedures given here are not restricted to semiparametric models or models with additive effects. Moreover, they are robust to outliers since only the ranks of the observations are needed. They are also applicable to pure ordinal data and since no continuity of the distribution functions is assumed, they can also be applied to data with ties. Some approximations for small sample sizes are suggested and analyzed in a simulation study. The application of the statistics and the interpretation of the results is demonstrated in several worked-out examples where some data sets given in the literature are re-analyzed.     

Optimal tests in AR (m) time series model

In the univariate framework, two problems of testing the nonlinearity are investigated in Hwang and Basawa. The first one is concerned with the testing problem for a nonlinear class contiguous to the AR(1) process. The second one is focused on the testing problem of the ARCH model contiguous to the AR(1) models. In each case, an efficient test of linearity was obtained, the local asymptotic normality (LAN) was proved, an efficient test of linearity was constructed, and the asymptotic power function was derived. All these results were obtained under the assumption where the parameter of the time series model is assumed to be known. In practical situation, this parameter is unspecified and its estimation induces an error that has an impact on the asymptotic limit distribution. A new method for the good evaluation of this error is introduced and investigated in the present article. Consequently, its application allows us to preserve the local asymptotic optimality with the estimated parameter. An extension to testing in class of ARCH models contiguous to p-order autoregressive processes is obtained. The LAN property plays a fundamental role in the present study.     

Nonparametric survival regression using the beta-Stacy process

A novel class of hierarchical nonparametric Bayesian survival regression models for time-to-event data with uninformative right censoring is introduced. The survival curve is modeled as a random function whose prior distribution is defined using the beta-Stacy (BS) process. The prior mean of each survival probability and its prior variance are linked to a standard parametric survival regression model. This nonparametric survival regression can thus be anchored to any reference parametric form, such as a proportional hazards or an accelerated failure time model, allowing substantial departures of the predictive survival probabilities when the reference model is not supported by the data. Also, under this formulation the predictive survival probabilities will be close to the empirical survival distribution near the mode of the reference model and they will be shrunken towards its probability density in the tails of the empirical distribution.     

On nonparametric density estimation for multivariate linear long-memory processes

We consider nonparametric estimation of the density function and its derivatives for multivariate linear processes with long-range dependence. In a first step, the asymptotic distribution of the multivariate empirical process is derived. In a second step, the asymptotic distribution of kernel density estimators and their derivatives is obtained.     

Discrete Nonparametric Kernel and Parametric Methods for the Modeling of Pavement Deterioration

This article is concerned with one discrete nonparametric kernel and two parametric regression approaches for providing the evolution law of pavement deterioration. The first parametric approach is a survival data analysis method; and the second is a nonlinear mixed-effects model. The nonparametric approach consists of a regression estimator using the discrete associated kernels. Some asymptotic properties of the discrete nonparametric kernel estimator are shown as, in particular, its almost sure consistency. Moreover, two data-driven bandwidth selection methods are also given, with a new theoretical explicit expression of optimal bandwidth provided for this nonparametric estimator. A comparative simulation study is realized with an application of bootstrap methods to a measure of statistical accuracy.     

New aspects of Bregman divergence in regression and classification with parametric and nonparametric estimation

In statistical learning, regression and classification concern different types of the output variables, and the predictive accuracy is quantified by different loss functions. This article explores new aspects of Bregman divergence (BD), a notion which unifies nearly all of the commonly used loss functions in regression and classification. The authors investigate the duality between BD and its generating function. They further establish, under the framework of BD, asymptotic consistency and normality of parametric and nonparametric regression estimators, derive the lower bound of their asymptotic covariance matrices, and demonstrate the role that parametric and nonparametric regression estimation play in the performance of classification procedures and related machine learning techniques. These theoretical results and new numerical evidence show that the choice of loss function affects estimation procedures, whereas has an asymptotically relatively negligible impact on classification performance. Applications of BD to statistical model building and selection with non‐Gaussian responses are also illustrated. The Canadian Journal of Statistics 37: 119‐139; 2009 © 2009 Statistical Society of Canada     

A sequential split-and-conquer approach for the analysis of big dependent data in computer experiments

Massive correlated data with many inputs are often generated from computer experiments to study complex systems. The Gaussian process (GP) model is a widely used tool for the analysis of computer experiments. Although GPs provide a simple and effective approximation to computer experiments, two critical issues remain unresolved. One is the computational issue in GP estimation and prediction where intensive manipulations of a large correlation matrix are required. For a large sample size and with a large number of variables, this task is often unstable or infeasible. The other issue is how to improve the naive plug-in predictive distribution which is known to underestimate the uncertainty. In this article, we introduce a unified framework that can tackle both issues simultaneously. It consists of a sequential split-and-conquer procedure, an information combining technique using confidence distributions (CD), and a frequentist predictive distribution based on the combined CD. It is shown that the proposed method maintains the same asymptotic efficiency as the conventional likelihood inference under mild conditions, but dramatically reduces the computation in both estimation and prediction. The predictive distribution contains comprehensive information for inference and provides a better quantification of predictive uncertainty as compared with the plug-in approach. Simulations are conducted to compare the estimation and prediction accuracy with some existing methods, and the computational advantage of the proposed method is also illustrated. The proposed method is demonstrated by a real data example based on tens of thousands of computer experiments generated from a computational fluid dynamic simulator.     

Uniform Inference on Quantile Effects under Sharp Regression Discontinuity Designs

ABSTRACT

This study develops methods for conducting uniform inference on quantile treatment effects for sharp regression discontinuity designs. We develop a score test for the treatment significance hypothesis and Wald-type tests for the hypotheses related to treatment significance, homogeneity, and unambiguity. The bias from the nonparametric estimation is studied in detail. In particular, we show that under some conditions, the asymptotic distribution of the score test is unaffected by the bias, without under-smoothing. For situations where the conditions can be restrictive, we incorporate a bias correction into the Wald tests and account for the estimation uncertainty. We also provide a procedure for constructing uniform confidence bands for quantile treatment effects. As an empirical application, we use the proposed methods to study the effect of cash-on-hand on unemployment duration. The results reveal pronounced treatment heterogeneity and also emphasize the importance of considering the long-term unemployed.     

Restricted Estimation in Partially Linear Errors-in-Variables Models

As a compromise between parametric regression and nonparametric regression, partially linear models are frequently used in statistical modelling. This article considers statistical inference for this semiparametric model when the linear covariate is measured with additive error and some additional linear restrictions on the parametric component are assumed to hold. We propose a restricted corrected profile least-squares estimator for the parametric component, and study the asymptotic normality of the estimator. To test hypothesis on the parametric component, we construct a Wald test statistic and obtain its limiting distribution. Some simulation studies are conducted to illustrate our approaches.     

Asymptotic expansion for nonparametric M-estimator in a nonlinear regression model with long-memory errors

We consider asymptotic expansion of the nonparametric M-estimator in a fixed-design nonlinear regression model when the errors are generated by long-memory linear processes. Under mild conditions, we show that the nonparametric M-estimator is first-order equivalent to the Nadaraya-Watson (NW) estimator, which implies that the nonparametric M-estimator has the same asymptotic distribution as that of the NW estimator. Furthermore, we study the second-order asymptotic expansion of the nonparametric M-estimator and show that the difference between the nonparametric M-estimator and the NW estimator has a limiting distribution after suitable standardization. The nature of the limiting distribution depends on the range of long-memory parameter α. We also compare the finite sample behavior of the two estimators through a numerical example when the errors are long-memory.