A Non‐Parametric Estimator of the Spectral Density of a Continuous‐Time Gaussian Process Observed at Random Times

Abstract. In numerous applications data are observed at random times and an estimated graph of the spectral density may be relevant for characterizing and explaining phenomena. By using a wavelet analysis, one derives a non‐parametric estimator of the spectral density of a Gaussian process with stationary increments (or a stationary Gaussian process) from the observation of one path at random discrete times. For every positive frequency, this estimator is proved to satisfy a central limit theorem with a convergence rate depending on the roughness of the process and the moment of random durations between successive observations. In the case of stationary Gaussian processes, one can compare this estimator with estimators based on the empirical periodogram. Both estimators reach the same optimal rate of convergence, but the estimator based on wavelet analysis converges for a different class of random times. Simulation examples and an application to biological data are also provided.     

Equivalence of Gaussian measures for some nonstationary random fields

Gaussian random fields whose covariance structures are described by a power law model provide a simple and flexible class of models for isotropic random fields. This class includes fractional Brownian fields as a special case. Because these random fields are nonstationary, the extensive results available on equivalence of Gaussian measures for stationary models do not apply to them. This work shows that results on equivalence for two stationary Gaussian random field models extend in a natural way to the equivalence of a stationary model and a power law model. This result is used to show that if we use a power law model for predicting a random field at unobserved locations when in fact the random field is stationary, we can obtain asymptotically optimal predictions as long as the high frequency behavior of the true spectral density is sufficiently close to the high frequency behavior of the spectral density of the power law model.     

Frequency domain bootstrap for ratio statistics under long-range dependence

A frequency domain bootstrap (FDB) is a common technique to apply Efron’s independent and identically distributed resampling technique (Efron, 1979) to periodogram ordinates – especially normalized periodogram ordinates – by using spectral density estimates. The FDB method is applicable to several classes of statistics, such as estimators of the normalized spectral mean, the autocorrelation (but not autocovariance), the normalized spectral density function, and Whittle parameters. While this FDB method has been extensively studied with respect to short-range dependent time processes, there is a dearth of research on its use with long-range dependent time processes. Therefore, we propose an FDB methodology for ratio statistics under long-range dependence, using semi- and nonparametric spectral density estimates as a normalizing factor. It is shown that the FDB approximation allows for valid distribution estimation for a broad class of stationary, long-range (or short-range) dependent linear processes, without any stringent assumptions on the distribution of the underlying process. The results of a large simulation study show that the FDB approximation using a semi- or nonparametric spectral density estimator is often robust for various values of a long-memory parameter reflecting magnitude of dependence. We apply the proposed procedure to two data examples.     

Exact simulation of Gaussian Time Series from Nonparametric Spectral Estimates with Application to Bootstrapping

The circulant embedding method for generating statistically exact simulations of time series from certain Gaussian distributed stationary processes is attractive because of its advantage in computational speed over a competitive method based upon the modified Cholesky decomposition. We demonstrate that the circulant embedding method can be used to generate simulations from stationary processes whose spectral density functions are dictated by a number of popular nonparametric estimators, including all direct spectral estimators (a special case being the periodogram), certain lag window spectral estimators, all forms of Welch's overlapped segment averaging spectral estimator and all basic multitaper spectral estimators. One application for this technique is to generate time series for bootstrapping various statistics. When used with bootstrapping, our proposed technique avoids some – but not all – of the pitfalls of previously proposed frequency domain methods for simulating time series.     

On the estimation of the spectral density for continuous spatial processes

In this work, we study the asymptotic properties of smoothed nonparametric kernel spectral density estimators for the spatial spectral density. We consider the case of continuous stationary spatial processes under a shrinking asymptotic framework. Expressions for the bias and the covariance structure are obtained and the implications for the edge effect bias of the choice of the kernel, bandwidth and spacing parameter in the design are also discussed, both for tapered and untapered estimates. Results are illustrated with a simulation study.     

Spectral analysis for processes with almost periodic covariances

The structure of random processes with almost periodic covariances is described from a spectral perspective. Under appropriate conditions methods for spectral estimation are described for such processes which are neither stationary nor locally stationary. Some spectral mass is then located off the main diagonal in this spectral plane. A method for estimating the support of the spectral mass is described in the Gaussian case. A number of open questions are mentioned.     

AN EMPIRICAL LIKELIHOOD APPROACH FOR NON‐GAUSSIAN VECTOR STATIONARY PROCESSES AND ITS APPLICATION TO MINIMUM CONTRAST ESTIMATION

We develop the empirical likelihood approach for a class of vector‐valued, not necessarily Gaussian, stationary processes with unknown parameters. In time series analysis, it is known that the Whittle likelihood is one of the most fundamental tools with which to obtain a good estimator of unknown parameters, and that the score functions are asymptotically normal. Motivated by the Whittle likelihood, we apply the empirical likelihood approach to its derivative with respect to unknown parameters. We also consider the empirical likelihood approach to minimum contrast estimation based on a spectral disparity measure, and apply the approach to the derivative of the spectral disparity. This paper provides rigorous proofs on the convergence of our two empirical likelihood ratio statistics to sums of gamma distributions. Because the fitted spectral model may be different from the true spectral structure, the results enable us to construct confidence regions for various important time series parameters without assuming specified spectral structures and the Gaussianity of the process.     

On the mean square convergence of the convolution representation of linear filters

Mean square convergence is the most frequently considered mode of convergence for the infinite series convolution expressions representing filter outputs in stationary time series analysis. There is confusion, however, also in the literature, about which conditions guarantee that this convergence holds. If only general properties of the input series to the filter are known, it is appropriate to consider the class of series with these properties. For each of several classes of full rank, wide sense stationary, zero-mean, vector time series, a weakest possible condition on the frequency response function of a linear filter is given which guarantees that the time-domain convolution representation of the filter converges to the filter output in mean square, whenever the input series belongs to the class under consideration. The classes considered are (i) the purely nondeterministic series with essentially bounded spectral density matrix, (ii) all purely nondeterministic series, (iii) all series. We then show that more unified resttlts can be obtained if Cesiro sums are utilized to define the convergence of the convolution representation. The mean square convergence of infinite autoregressions is also discussed.     

Uniform Convergence of Compactly Supported Wavelet Expansions of Gaussian Random Processes

New results on uniform convergence in probability for expansions of Gaussian random processes using compactly supported wavelets are given. The main result is valid for general classes of non stationary processes. An application of the obtained results to stationary processes is also presented. It is shown that the convergence rate of the expansions is exponential.     

Weak approximations for quantile processes of stationary sequences

ABSTRACT By studying the deviations between uniform empirical and quantile processes (the so-called Bahadur-Kiefer representations) of a stationary sequence in properly weighted sup-norm metrics, we find a general approach to obtaining weighted results for uniform quantile processes of stationary sequences. Consequently we are able to obtain weak convergence for weighted uniform quantile processes of stationary mixing and associated sequences. Further, by studying the sup-norm distance of a general quantile process from its corresponding uniform quantile process, we find that information at the two end points of the uniform quantile process can be so utilized that this weighted sup-norm distance converges in probability to zero under the so-called Csörgõ-Révész conditions. This enables us to obtain weak convergence for weighted general quantile processes of stationary mixing and associated sequences.     

Hidden Second‐order Stationary Spatial Point Processes

In the existing statistical literature, the almost default choice for inference on inhomogeneous point processes is the most well‐known model class for inhomogeneous point processes: reweighted second‐order stationary processes. In particular, the K‐function related to this type of inhomogeneity is presented as the inhomogeneous K‐function. In the present paper, we put a number of inhomogeneous model classes (including the class of reweighted second‐order stationary processes) into the common general framework of hidden second‐order stationary processes, allowing for a transfer of statistical inference procedures for second‐order stationary processes based on summary statistics to each of these model classes for inhomogeneous point processes. In particular, a general method to test the hypothesis that a given point pattern can be ascribed to a specific inhomogeneous model class is developed. Using the new theoretical framework, we reanalyse three inhomogeneous point patterns that have earlier been analysed in the statistical literature and show that the conclusions concerning an appropriate model class must be revised for some of the point patterns.     

Measuring the association of stationary point processes using spectral analysis techniques

In this work we focus on relationships between stationary point process using spectral analysis techniques. The evaluation of these relationships is accomplished with the help of the product ratio of association (PRA), which is based on the cumulant densities of the point processes. The estimation procedure is obtained by smoothing the periodogram statistic, a function of the frequency domain. It is proved that the asymptotic distribution of the square root of the estimated PRA is Normal with a constant variance. Statistical tests for hypotheses concerning the independence of two point processes and the characterization of a Poisson process are proposed. Furthermore, approximate 95% pointwise confidence interval can be obtained for the estimated PRA. These results can be applied on stochastic systems involving as input and output stationary point processes. An illustrative example from the framework of neurophysiology is presented.     

Optimal window width choice in spectral density estimation

This paper deals with optimal window width choice in on-parametric lag or spectral window estimation of the spectral density of a stationary zero-mean process. Several approaches are reviewed: cross-validation-based methods as described by Hurvich(1985) BelträHo and Bloomfield (1987) and Hurvich and Belträo (1990); an iterative pro-cedure developed by Bühlmann (1996); and a bootstrap approach followed by Franke and Hardle (1992). These methods are compared in terms of the mean square error,the mean square percentage error, and a third measure of the istance between the true spectral density and its estimate. The comparison is based on a simulation study, the simulated processes being in he class of ARMA (5,5) processes. On the basis of simu-lation evidence we suggest to use a slightly modified version of Biihlmann's (1996)iterative method. This paper also makes a minor correction of the bootstrap criterion by Franke and Härdle (1992).     

AN INFINITE-PHASE QUASI-BIRTH-AND-DEATH MODEL FOR THE NON-PREEMPTIVE PRIORITY M/PH/1 QUEUE

This paper considers a single server queue that handles arrivals from N classes of customers on a non-preemptive priority basis. Each of the N classes of customers features arrivals from a Poisson process at rate λ _i and class-dependent phase type service. To analyze the queue length and waiting time processes of this queue, we derive a matrix geometric solution for the stationary distribution of the underlying Markov chain. A defining characteristic of the paper is the fact that the number of distinct states represented within the sub-level is countably infinite, rather than finite as is usually assumed. Among the results we obtain in the two-priority case are tractable algorithms for the computation of both the joint distribution for the number of customers present and the marginal distribution of low-priority customers, and an explicit solution for the marginal distribution of the number of high-priority customers. This explicit solution can be expressed completely in terms of the arrival rates and parameters of the two service time distributions. These results are followed by algorithms for the stationary waiting time distributions for high- and low-priority customers. We then address the case of an arbitrary number of priority classes, which we solve by relating it to an equivalent three-priority queue. Numerical examples are also presented.     

The Pearson Diffusions: A Class of Statistically Tractable Diffusion Processes

Abstract.  The Pearson diffusions form a flexible class of diffusions defined by having linear drift and quadratic squared diffusion coefficient. It is demonstrated that for this class explicit statistical inference is feasible. A complete model classification is presented for the ergodic Pearson diffusions. The class of stationary distributions equals the full Pearson system of distributions. Well-known instances are the Ornstein–Uhlenbeck processes and the square root (CIR) processes. Also diffusions with heavy-tailed and skew marginals are included. Explicit formulae for the conditional moments and the polynomial eigenfunctions are derived. Explicit optimal martingale estimating functions are found. The discussion covers GMM, quasi-likelihood, non-linear weighted least squares estimation and likelihood inference too. The analytical tractability is inherited by transformed Pearson diffusions, integrated Pearson diffusions, sums of Pearson diffusions and Pearson stochastic volatility models. For the non-Markov models, explicit optimal prediction-based estimating functions are found. The estimators are shown to be consistent and asymptotically normal.     

Unbiased Bayesian inference for population Markov jump processes via random truncations

We consider continuous time Markovian processes where populations of individual agents interact stochastically according to kinetic rules. Despite the increasing prominence of such models in fields ranging from biology to smart cities, Bayesian inference for such systems remains challenging, as these are continuous time, discrete state systems with potentially infinite state-space. Here we propose a novel efficient algorithm for joint state/parameter posterior sampling in population Markov Jump processes. We introduce a class of pseudo-marginal sampling algorithms based on a random truncation method which enables a principled treatment of infinite state spaces. Extensive evaluation on a number of benchmark models shows that this approach achieves considerable savings compared to state of the art methods, retaining accuracy and fast convergence. We also present results on a synthetic biology data set showing the potential for practical usefulness of our work.     

Mixed-spectra analysis for stationary random fields

We consider a, discrete time, weakly stationary bidimensional process, for which the spectral measure is the sum of an absolutely continuous measure, a discrete measure of finite order and a finite number of absolutely continuous measures on several lines. In this paper we are interested in estimating the spectral density of the absolutely continuous measure and of the density on the lines. For this aim, by using the double kernel method, we construct consistent estimators of these densities and we study their asymptotic behaviors in term of the mean squared error with rate.     

A novel and fast methodology for simultaneous multiple structural break estimation and variable selection for nonstationary time series models

A class of nonstationary time series such as locally stationary time series can be approximately modeled by piecewise stationary autoregressive (PSAR) processes. But the number and locations of the piecewise autoregressive segments, as well as the number of nonzero coefficients in each autoregressive process, are unknown. In this paper, by connecting the multiple structural break detection with a variable selection problem for a linear model with a large number of regression coefficients, a novel and fast methodology utilizing modern penalized model selection is introduced for detecting multiple structural breaks in a PSAR process. It also simultaneously performs variable selection for each autoregressive model and hence the order selection. To further its performance, an algorithm is given, which remains very fast in computation. Numerical results from simulation and a real data example show that the algorithm has excellent empirical performance.     

Distribution of the coates-diggle test statistic in case of replicated observations

A closed form expression for the distribution of a test statistic for comparing the spectral densities of stationary processes is given. This test statistic was introduced by COATES and DIGGLE ( 1986 ) for the unreplicated case and has been extended to the case of replicated observations by POTSCHER and RESCHENHOFER ( 1988 ). A simple method for computing approximate critical values in case of large numbers of replications is also provided. As a by-product an explicit expression for the distribution function of the range of independent variates each distributed as the logarithm of an F-variate i.e up to a factor of 2 each followin Fishers z-distriution is obtained     

Bayesian estimation for the M/G/1 queue using a phase-type approximation

This article deals with Bayesian inference and prediction for M/G/1 queueing systems. The general service time density is approximated with a class of Erlang mixtures which are phase-type distributions. Given this phase-type approximation, an explicit evaluation of measures such as the stationary queue size, waiting time and busy period distributions can be obtained. Given arrival and service data, a Bayesian procedure based on reversible jump Markov Chain Monte Carlo methods is proposed to estimate system parameters and predictive distributions.