Decompounding discrete distributions: A nonparametric Bayesian approach

Suppose that a compound Poisson process is observed discretely in time and assume that its jump distribution is supported on the set of natural numbers. In this paper we propose a nonparametric Bayesian approach to estimate the intensity of the underlying Poisson process and the distribution of the jumps. We provide a Markov chain Monte Carlo scheme for obtaining samples from the posterior. We apply our method on both simulated and real data examples, and compare its performance with the frequentist plug-in estimator proposed by Buchmann and Grübel. On a theoretical side, we study the posterior from the frequentist point of view and prove that as the sample size n→∞, it contracts around the “true,” data-generating parameters at rate $1/\sqrt{n}$, up to a $\log n$ factor.     

Graphon estimation via nearest-neighbour algorithm and two-dimensional fused-lasso denoising

We propose a class of methods for graphon estimation based on exploiting connections with nonparametric regression. The idea is to construct an ordering of the nodes in the network, similar in spirit to Chan & Airoldi (2014). However, rather than considering orderings based only on the empirical degree as in Chan & Airoldi (2014), we use the nearest-neighbour algorithm which is an approximative solution to the travelling salesman problem. This algorithm in turn can handle general distances $\widehat{d}$ between the nodes, allowing us to incorporate rich information from the network. Once an ordering is constructed, we formulate a two-dimensional-grid graph-denoising problem that we solve through fused-lasso regularization. For particular choices of the metric $\widehat{d}$, we show that the corresponding two-step estimator can attain competitive rates when the true model is the stochastic block model, and when the underlying graphon is piecewise Hölder or has bounded variation.     

Stochastic functional estimates in longitudinal models with interval-censored anchoring events

Timelines of longitudinal studies are often anchored by specific events. In the absence of the fully observed anchoring event times, the study timeline becomes undefined, and the traditional longitudinal analysis loses its temporal reference. In this paper, we considered an analytical situation where the anchoring events are interval censored. We demonstrated that by expressing the regression parameter estimators as stochastic functionals of a plug-in estimate of the unknown anchoring event time distribution, the standard longitudinal models could be extended to accommodate the situation of less well-defined timelines. We showed that for a broad class of longitudinal models, the functional parameter estimates are consistent and asymptotically normally distributed with a $\sqrt{n}$ convergence rate under mild regularity conditions. Applying the developed theory to linear mixed-effects models, we further proposed a hybrid computational procedure that combines the strengths of the Fisher's scoring method and the expectation-expectation (EM) algorithm for model parameter estimation. We conducted a simulation study to validate the asymptotic properties and to assess the finite sample performance of the proposed method. A real data example was used to illustrate the proposed method. The method fills in a gap in the existing longitudinal analysis methodology for data with less well-defined timelines.     

High-dimensional covariance matrix estimation using a low-rank and diagonal decomposition

We study high-dimensional covariance/precision matrix estimation under the assumption that the covariance/precision matrix can be decomposed into a low-rank component $L$ and a diagonal component $D$. The rank of $L$ can either be chosen to be small or controlled by a penalty function. Under moderate conditions on the population covariance/precision matrix itself and on the penalty function, we prove some consistency results for our estimators. A block-wise coordinate descent algorithm, which iteratively updates $L$ and $D$, is then proposed to obtain the estimator in practice. Finally, various numerical experiments are presented; using simulated data, we show that our estimator performs quite well in terms of the Kullback–Leibler loss; using stock return data, we show that our method can be applied to obtain enhanced solutions to the Markowitz portfolio selection problem. The Canadian Journal of Statistics 48: 308–337; 2020 © 2019 Statistical Society of Canada     

Direct estimation of differential networks under high-dimensional nonparanormal graphical models

In genomics, it is often of interest to study the structural change of a genetic network between two phenotypes. Under Gaussian graphical models, the problem can be transformed to estimating the difference between two precision matrices, and several approaches have been recently developed for this task such as joint graphical lasso and fused graphical lasso. However, the multivariate Gaussian assumptions made in the existing approaches are often violated in reality. For instance, most RNA-Seq data follow non-Gaussian distributions even after log-transformation or other variance-stabilizing transformations. In this work, we consider the problem of directly estimating differential networks under a flexible semiparametric model, namely the nonparanormal graphical model, where the random variables are assumed to follow a multivariate Gaussian distribution after a set of monotonically increasing transformations. We propose to use a novel rank-based estimator to directly estimate the differential network, together with a parametric simplex algorithm for fast implementation. Theoretical properties of the new estimator are established under a high-dimensional setting where $p$ grows with $n$ almost exponentially fast. In particular, we show that the proposed estimator is consistent in both parameter estimation and support recovery. Both synthetic data and real genomic data are used to illustrate the promise of the new approach. The Canadian Journal of Statistics 48: 187–203; 2020 © 2019 Statistical Society of Canada     

Sparse concordance-based ordinal classification

Ordinal classification is an important area in statistical machine learning, where labels exhibit a natural order. One of the major goals in ordinal classification is to correctly predict the relative order of instances. We develop a novel concordance-based approach to ordinal classification, where a concordance function is introduced and a penalized smoothed method for optimization is designed. Variable selection using the $L_1$ penalty is incorporated for sparsity considerations. Within the set of classification rules that maximize the concordance function, we find optimal thresholds to predict labels by minimizing a loss function. After building the classifier, we derive nonparametric estimation of class conditional probabilities. The asymptotic properties of the estimators as well as the variable selection consistency are established. Extensive simulations and real data applications show the robustness and advantage of the proposed method in terms of classification accuracy, compared with other existing methods.     

Asymptotic approximation of the likelihood of stationary determinantal point processes

Continuous determinantal point processes (DPPs) are a class of repulsive point processes on ${ℝ}^d$ with many statistical applications. Although an explicit expression of their density is known, it is too complicated to be used directly for maximum likelihood estimation. In the stationary case, an approximation using Fourier series has been suggested, but it is limited to rectangular observation windows and no theoretical results support it. In this contribution, we investigate a different way to approximate the likelihood by looking at its asymptotic behavior when the observation window grows toward ${ℝ}^d$. This new approximation is not limited to rectangular windows, is faster to compute than the previous one, does not require any tuning parameter, and some theoretical justifications are provided. It moreover provides an explicit formula for estimating the asymptotic variance of the associated estimator. The performances are assessed in a simulation study on standard parametric models on ${ℝ}^d$ and compare favorably to common alternative estimation methods for continuous DPPs.     

Daisee: Adaptive importance sampling by balancing exploration and exploitation

We study adaptive importance sampling (AIS) as an online learning problem and argue for the importance of the trade-off between exploration and exploitation in this adaptation. Borrowing ideas from the online learning literature, we propose Daisee, a partition-based AIS algorithm. We further introduce a notion of regret for AIS and show that Daisee has $𝒪(\sqrt{T}{(\log T)}^{\frac{3}{4}})$ cumulative pseudo-regret, where $T$ is the number of iterations. We then extend Daisee to adaptively learn a hierarchical partitioning of the sample space for more efficient sampling and confirm the performance of both algorithms empirically.     

Selection of linear mixed-effects models for clustered data

We consider model selection for linear mixed-effects models with clustered structure, where conditional Kullback–Leibler (CKL) loss is applied to measure the efficiency of the selection. We estimate the CKL loss by substituting the empirical best linear unbiased predictors (EBLUPs) into random effects with model parameters estimated by maximum likelihood. Although the BLUP approach is commonly used in predicting random effects and future observations, selecting random effects to achieve asymptotic loss efficiency concerning CKL loss is challenging and has not been well studied. In this paper, we propose addressing this difficulty using a conditional generalized information criterion (CGIC) with two tuning parameters. We further consider a challenging but practically relevant situation where the number, $m$, of clusters does not go to infinity with the sample size. Hence the random-effects variances are not consistently estimable. We show that via a novel decomposition of the CKL risk, the CGIC achieves consistency and asymptotic loss efficiency, whether $m$ is fixed or increases to infinity with the sample size. We also conduct numerical experiments to illustrate the theoretical findings.     

On the robustness to outliers of the Student-t process

The theory of Bayesian robustness modeling uses heavy-tailed distributions to resolve conflicts of information by rejecting automatically the outlying information in favor of the other sources of information. In particular, the Student's-t process is a natural alternative to the Gaussian process when the data might carry atypical information. Several works attest to the robustness of the Student $t$ process, however, the studies are mostly guided by intuition and focused mostly on the computational aspects rather than the mathematical properties of the involved distributions. This work uses the theory of regular variation to address the robustness of the Student $t$ process in the context of nonlinear regression, that is, the behavior of the posterior distribution in the presence of outliers in the inputs, in the outputs, or in both sources of information. In all these cases, under certain conditions, it is shown that the posterior distribution tends to a quantity that does not depend on the atypical information, then, for every case, the limiting posterior distribution as the outliers tend to infinity is provided. The impact of outliers on the predictive posterior distribution is also addressed. The theory is illustrated with a few simulated examples.     

Rate efficient estimation of realized Laplace transform of volatility with microstructure noise

In this paper, we consider the problem of estimating the Laplace transform of volatility within a fixed time interval [0,T] using high‐frequency sampling, where we assume that the discretized observations of the latent process are contaminated by microstructure noise. We use the pre‐averaging approach to deal with the effect of microstructure noise. Under the high‐frequency scenario, we obtain a consistent estimator whose convergence rate is , which is known as the optimal convergence rate of the estimation of integrated volatility functionals under the presence of microstructure noise. The related central limit theorem is established. The simulation studies justify the finite‐sample performance of the proposed estimator.     

Asymptotic properties of the maximum smoothed partial likelihood estimator in the change-plane Cox model

The change-plane Cox model is a popular tool for the subgroup analysis of survival data. Despite the rich literature on this model, there has been limited investigation into the asymptotic properties of the estimators of the finite-dimensional parameter. Particularly, the convergence rate, not to mention the asymptotic distribution, has not been fully characterized for the general model where classification is based on multiple covariates. To bridge this theoretical gap, this study proposes a maximum smoothed partial likelihood estimator and establishes the following asymptotic properties. First, it shows that the convergence rate for the classification parameter can be arbitrarily close to $n^{-1}$ up to a logarithmic factor under a certain condition on covariates and the choice of tuning parameter. Given this convergence rate result, it also establishes the asymptotic normality for the regression parameter.     

Grenander functionals and Cauchy's formula

Let ${\widehat{f}}_n$ be the nonparametric maximum likelihood estimator of a decreasing density. Grenander characterized this as the left‐continuous slope of the least concave majorant of the empirical distribution function. For a sample from the uniform distribution, the asymptotic distribution of the L₂‐distance of the Grenander estimator to the uniform density was derived in an article by Groeneboom and Pyke by using a representation of the Grenander estimator in terms of conditioned Poisson and gamma random variables. This representation was also used in an article by Groeneboom and Lopuhaä to prove a central limit result of Sparre Andersen on the number of jumps of the Grenander estimator. Here we extend this to the proof of the main result on the L₂‐distance of the Grenander estimator to the uniform density and also prove a similar asymptotic normality results for the entropy functional. Cauchy's formula and saddle point methods are the main tools in our development.     

Wavelet Thresholding Estimation in a Poissonian Interactions Model with Application to Genomic Data

This paper deals with the study of dependencies between two given events modelled by point processes. In particular, we focus on the context of DNA to detect favoured or avoided distances between two given motifs along a genome suggesting possible interactions at a molecular level. For this, we naturally introduce a so‐called reproduction function h that allows to quantify the favoured positions of the motifs and that is considered as the intensity of a Poisson process. Our first interest is the estimation of this function h assumed to be well localized. The estimator based on random thresholds achieves an oracle inequality. Then, minimax properties of on Besov balls are established. Some simulations are provided, proving the good practical behaviour of our procedure. Finally, our method is applied to the analysis of the dependence between promoter sites and genes along the genome of the Escherichia coli bacterium.     

Exponential Family Techniques for the Lognormal Left Tail

Let X be lognormal(μ,σ²) with density f(x); let θ > 0 and define . We study properties of the exponentially tilted density (Esscher transform) f_θ(x) = e^?θxf(x)/L(θ), in particular its moments, its asymptotic form as θ→∞ and asymptotics for the saddlepoint θ(x) determined by . The asymptotic formulas involve the Lambert W function. The established relations are used to provide two different numerical methods for evaluating the left tail probability of the sum of lognormals S_n=X₁+?+X_n: a saddlepoint approximation and an exponential tilting importance sampling estimator. For the latter, we demonstrate logarithmic efficiency. Numerical examples for the cdf F_n(x) and the pdf f_n(x) of S_n are given in a range of values of σ²,n and x motivated by portfolio value‐at‐risk calculations.