On multivariate variable-kernel density estimates for time series

Let X₁ be a strictly stationary multiple time series with values in R^d and with a common density f. Let X₁,.,.,X_n, be n consecutive observations of X₁. Let k = k_n, be a sequence of positive integers, and let H_ni be the distance from X_i to its kth nearest neighbour among X_j, j i. The multivariate variable-kernel estimate f_n, of f is defined by where K is a given density. The complete convergence of f_n, to f on compact sets is established for time series satisfying a dependence condition (referred to as the strong mixing condition in the locally transitive sense) weaker than the strong mixing condition. Appropriate choices of k are explicitly given. The results apply to autoregressive processes and bilinear time-series models.     

Exponential Family Techniques for the Lognormal Left Tail

Let X be lognormal(μ,σ²) with density f(x); let θ > 0 and define . We study properties of the exponentially tilted density (Esscher transform) f_θ(x) = e^?θxf(x)/L(θ), in particular its moments, its asymptotic form as θ→∞ and asymptotics for the saddlepoint θ(x) determined by . The asymptotic formulas involve the Lambert W function. The established relations are used to provide two different numerical methods for evaluating the left tail probability of the sum of lognormals S_n=X₁+?+X_n: a saddlepoint approximation and an exponential tilting importance sampling estimator. For the latter, we demonstrate logarithmic efficiency. Numerical examples for the cdf F_n(x) and the pdf f_n(x) of S_n are given in a range of values of σ²,n and x motivated by portfolio value‐at‐risk calculations.     

An Estimator of the Exponent of Regular Variation Based on K-Record Values

Let {X _n:n ≥ 1} be an i.i.d. sequence of random variables with a continuous distribution function F. Under the assumption that the upper tail of Fis regularly varying with exponent 1/α, α > 0, we study the asymptotic properties of an estimator of α based on k-record values.     

Recursive estimation of the mode of a multivariate density

Let f be an unknown possibly multimodal density on R^d and let X₁, X₂, … be a sequence of independent random vectors with density f. Several recursive estimates of the mode of f are proposed, and sufficient conditions ensuring their weak and strong consistency are established.     

On the strong Kotz approximation of Dirichlet random vectors

Let (X ₁, X ₂) be a bivariate L _p -norm generalized symmetrized Dirichlet (LpGSD) random vector with parameters α₁,α₂. If p=α₁=α₂=2, then (X ₁, X ₂) is a spherical random vector. The estimation of the conditional distribution of Z _u *:=X ₂ | X ₁>u for u large is of some interest in statistical applications. When (X ₁, X ₂) is a spherical random vector with associated random radius in the Gumbel max-domain of attraction, the distribution of Z _u * can be approximated by a Gaussian distribution. Surprisingly, the same Gaussian approximation holds also for Z _u :=X ₂| X ₁=u. In this paper, we are interested in conditional limit results in terms of convergence of the density functions considering a d-dimensional LpGSD random vector. Stating our results for the bivariate setup, we show that the density function of Z _u * and Z _u can be approximated by the density function of a Kotz type I LpGSD distribution, provided that the associated random radius has distribution function in the Gumbel max-domain of attraction. Further, we present two applications concerning the asymptotic behaviour of concomitants of order statistics of bivariate Dirichlet samples and the estimation of the conditional quantile function.     

Weak and strong uniform consistency rates of kernel density estimates for randomly censored data

Let X₁,., X_n, be i.i.d. random variables with distribution function F, and let Y₁,.,.,Y_n be i.i.d. with distribution function G. For i = 1, 2,.,., n set δ_i, = 1 if X_i ≤ Y_i, and 0 otherwise, and X_i, = min{X_i, K_i}. A kernel-type density estimate of f, the density function of F w.r.t. Lebesgue measure on the Borel o-field, based on the censored data (δ_i, X_i), i = 1,.,.,n, is considered. Weak and strong uniform consistency properties over the whole real line are studied. Rates of convergence results are established under higher-order differentiability assumption on f. A procedure for relaxing such assumptions is also proposed.     

Theory of factorial designs of the parallel flats type. I: the coefficient matrix

Let GF(s) be the finite field with s elements.(Thus, when s=3, the elements of GF(s) are 0, 1 and 2.)Let A(r×n), of rank r, and c_i(i=1,…,f), (r×1), be matrices over GF(s). (Thus, for n=4, r=2, f=2, we could have A=[¹¹¹⁰₀₁₂₁], c₁=[¹₀], c₂=[⁰₂].) Let T_i (i=1,…,f) be the flat in EG(n, s) consisting of the set of all the s^n?r solutions of the equations At=c_i, wheret′=(t₁,…,t_n) is a vector of variables.(Thus, EG(4, 3) consists of the 3⁴=81 points of the form (t₁,t₂,t₃,t₄), where t's take the values 0,1,2 (in GF(3)). The number of solutions of the equations At=c_i is s^n?r, where r=Rank(A), and the set of such solutions is said to form an (n?r)-flat, i.e. a flat of (n?r) dimensions. In our example, both T₁ and T₂ are 2-flats consisting of 3^4?2=9 points each. The flats T₁,T₂,…,T_f are said to be parallel since, clearly, no two of them can have a common point. In the example, the points of T₁ are (1000), (0011), (2022), (0102), (2110), (1121), (2201), (1212) and (0220). Also, T₂ consists of (0002), (2010), (1021), (2101), (1112), (0120), (1200), (0211) and (2222).) Let T be the fractional design for a sⁿ symmetric factorial experiment obtained by taking T₁,T₂,…,T_f together. (Thus, in the example, 3⁴=81 treatments of the 3⁴ factorial experiment correspond one-one with the points of EG(4,3), and T will be the design (i.e. a subset of the 81 treatments) consisting of the 18 points of T₁ and T₂ enumerated above.)In this paper, we lay the foundation of the general theory of such ‘parallel’ types of designs. We define certain functions of A called the alias component matrices, and use these to partition the coefficient matrix X (n×v), occuring in the corresponding linear model, into components X._j(j=0,1,…,g), such that the information matrix X is the direct sum of the X′._jX._j. Here, v is the total number of parameters, which consist of (possibly μ), and a (general) set of (geometric) factorial effects (each carrying (s?1) degrees of freedom as usual). For j≠0, we show that the spectrum of X′._jX._j does not change if we change (in a certain important way) the usual definition of the effects. Assuming that such change has been adopted, we consider the partition of the X._j into the X_ij (i=1,…,f). Furthermore, the X_ij are in turn partitioned into smaller matrices (which we shall here call the) X_ijh. We show that each X_ijh can be factored into a product of 3 matrices J, ζ (not depending on i,j, and h) and Q(j,h,i)where both the Kronecker and ordinary product are used. We introduce a ring R using the additive groups of the rational field and GF(s), and show that the Q(j,h,i) belong to a ring isomorphic to R. When s is a prime number, we show that R is the cyclotomic field. Finally, we show that the study of the X._j and X′._jX._j can be done in a much simpler manner, in terms of certain relatively small sized matrices over R.     

Asymptotic behaviour of the mean integrated squared error of kernel density estimators for dependent observations

Let X₁, X₂, … be a strictly stationary sequence of observations, and g be the joint density of (X₁, …, X_d) for some fixed d ? 1. We consider kernel estimators of the density g. The asymptotic behaviour of the mean integrated squared error of the kernel estimators is obtained under an assumption of weak dependence between the observations.     

A study on the least squares estimator of multivariate isotonic regression function

Consider the problem of pointwise estimation of f in a multivariate isotonic regression model Z=f(X₁,…,X_d)+ϵ, where Z is the response variable, f is an unknown nonparametric regression function, which is isotonic with respect to each component, and ϵ is the error term. In this article, we investigate the behavior of the least squares estimator of f. We generalize the greatest convex minorant characterization of isotonic regression estimator for the multivariate case and use it to establish the asymptotic distribution of properly normalized version of the estimator. Moreover, we test whether the multivariate isotonic regression function at a fixed point is larger (or smaller) than a specified value or not based on this estimator, and the consistency of the test is established. The practicability of the estimator and the test are shown on simulated and real data as well.     

A stochastic Newton-Raphson method

A stochastic approximation procedure of the Robbins-Monro type is considered. The original idea behind the Newton-Raphson method is used as follows. Given n approximations X₁,…, X_n with observations Y₁,…, Y_n, a least squares line is fitted to the points (X_m, Y_m),…, (X_n, Y_n) where m<n may depend on n. The (n+1)st approximation is taken to be the intersection of the least squares line with y=0. A variation of the resulting process is studied. It is shown that this process yields a strongly consistent sequence of estimates which is asymptotically normal with minimal asymptotic variance.     

On the rate of convergence of recursive kernel estimates of probability densities

Recursive estimates f_n^r(x)of the rth derivative f^r(x)(r=0,1)of the univariate probability density f(x) for strictly stationary processes {X_j,} are considered. The asymptotic variance-covariance of f_n^r(x)is established for stationary triangular arrays of random variables satisfying various asymptotic independence-uncorrelatedness conditions.     

Moments of Order Statistics from Weibull Distribution in the Presence of Multiple Outliers

In this article, we derive exact expressions for the single and product moments of order statistics from Weibull distribution under the contamination model. We assume that X₁, X₂, …, X_{n ? p} are independent with density function f(x) while the remaining, p observations (outliers) X_{n ? p + 1}, …, X_n are independent with density function arises from some modified version of f(x), which is called g(x), in which the location and/or scale parameters have been shifted in value. Next, we investigate the effect of the outliers on the BLUE of the scale parameter. Finally, we deduce some special cases.     

A modified adaptive accept–reject algorithm for univariate densities with bounded support

The need to simulate from a univariate density arises in several settings, particularly in Bayesian analysis. An especially efficient algorithm which can be used to sample from a univariate density, f _X , is the adaptive accept–reject algorithm. To implement the adaptive accept–reject algorithm, the user has to envelope T ° f _X , where T is some transformation such that the density g(x) ∝ T ^?1 (α+β x) is easy to sample from. Successfully enveloping T ° f _X , however, requires that the user identify the number and location of T ° f _X ’s inflection points. This is not always a trivial task. In this paper, we propose an adaptive accept–reject algorithm which relieves the user of precisely identifying the location of T ° f _X ’s inflection points. This new algorithm is shown to be efficient and can be used to sample from any density such that its support is bounded and its log is three-times differentiable.     

A generalized semi-Pareto minification process

In this paper a generalization of the semi-Pareto autoregressive minification process of the first order is given. The necessary and sufficient condition for stationarity of the process is determined. It is shown that the process is ergodic and uniformly mixing. The joint survival function and the joint density function of the random variables X _n+h and X _n are determined. The extremes of the random variables X ₁, X ₂, ..., X _n and the geometric extremes of random variables X ₁, X ₂, ..., X _N are derived and their asymptotic distributions are discussed. The estimation of the parameters is discussed and some numerical results are given.     

PLUG-IN ESTIMATION OF GENERAL LEVEL SETS

Given an unknown function (e.g. a probability density, a regression function, …) f and a constant c, the problem of estimating the level set L(c) ={f≥c} is considered. This problem is tackled in a very general framework, which allows f to be defined on a metric space different from . Such a degree of generality is motivated by practical considerations and, in fact, an example with astronomical data is analyzed where the domain of f is the unit sphere. A plug‐in approach is followed; that is, L(c) is estimated by L_n(c) ={f_n≥c} , where f_n is an estimator of f. Two results are obtained concerning consistency and convergence rates, with respect to the Hausdorff metric, of the boundaries ?L_n(c) towards ?L(c) . Also, the consistency of L_n(c) to L(c) is shown, under mild conditions, with respect to the L₁ distance. Special attention is paid to the particular case of spherical data.     

Changepoint Problems Under Random Censorship

Let X ₁, …, X _[nθ], X _{[nθ] + 1}, …, X _n be a sequence of independent random variables (the “lifetimes”) such that X _j ? F ₁ for 1 ≤ j ≤ [nθ] and X _j ? F ₂ for [nθ] + 1 ≤ j ≤ n, with F ₁ ≠ F ₂ unknown. In this paper we investigate an estimator θ _n for the changepoint θ if the X's are subject to censoring. The rate of almost sure convergence of θ _n to θ is established and a test for the hypothesis θ = 0, i.e. “no change”, is proposed.     

Prediction in moving average processes

For the stationary invertible moving average process of order one with unknown innovation distribution F, we construct root-n   consistent plug-in estimators of conditional expectations E(h(_Xn+1)|_X1,…,_Xn)$E(h(X_{n+1})|X_1,\dots ,X_n)$. More specifically, we give weak conditions under which such estimators admit Bahadur-type representations, assuming some smoothness of h or of F. For fixed h it suffices that h   is locally of bounded variation and locally Lipschitz in _L2(F)$L_2(F)$, and that the convolution of h and F   is continuously differentiable. A uniform representation for the plug-in estimator of the conditional distribution function P(_Xn+1?·|_X1,…,_Xn)$P(X_{n+1}?·|X_1,\dots ,X_n)$ holds if F has a uniformly continuous density. For a smoothed version of our estimator, the Bahadur representation holds uniformly over each class of functions h that have an appropriate envelope and whose shifts are F-Donsker, assuming some smoothness of F. The proofs use empirical process arguments.     

COMPARISON OF THREE NON-PARAMETRIC TESTS FOR BIVARIATE INTERCHANGEABILITY1

Given a random sample(X₁, Y₁), …,(X_n, Y_n) from a bivariate (BV) absolutely continuous c.d.f. H (x, y), we consider rank tests for the null hypothesis of interchangeability H₀: H(x, y). Three linear rank test statistics, Wilcoxon (W_N), sum of squared ranks (SSR_N) and Savage (S_N), are described in Section 1. In Section 2, asymptotic relative efficiency (ARE) comparisons of the three types of tests are made for Morgenstern (Plackett, 1965) and Moran (1969)BV alternatives with marginal distributions satisfying G(x) = F(x/θ) for some θ≠ 1. Both gamma and lognormal marginal distributions are used.     

Selection rules based on divergences

This paper deals with a special adaptive estimation problem, namely how can one select for each set of i.i.d. data X ₁, …, X _n the better of two given estimates of the data-generating probability density. Such a problem was studied by Devroye and Lugosi [Combinatorial Methods in Density Estimation, Springer, Berlin, 2001] who proposed a feasible suboptimal selection (called the Scheffé selection) as an alternative to the optimal but nonfeasible selection which minimizes the L₁-error. In many typical situations, the L₁-error of the Scheffé selection was shown to tend to zero for n→∞ as fast as the L₁-error of the optimal estimate. This asymptotic result was based on an inequality between the total variation errors of the Scheffé and optimal selections. The present paper extends this inequality to the class of φ-divergence errors containing the L₁-error as a special case. The first extension compares the φ-divergence errors of the mentioned Scheffé and optimal selections of Devroye and Lugosi. The second extension deals with a class of generalized Scheffé selections adapted to the φ-divergence error criteria and reducing to the classical Scheffé selection for the L₁-criterion. It compares the φ-divergence errors of these feasible selections and the optimal nonfeasible selections minimizing the φ-divergence errors. Both extensions are motivated and illustrated by examples.     

A note on the asymptotic efficiency of the sobel–wald test

Let X₁,X₂, … be iid random variables with the pdf f(x,θ)=exp(θx?b(θ)) relative to a σ-finite measure μ, and consider the problem of deciding among three simple hypotheses H_i:θ=θ_i (1?i?3) subject to P(acceptH_i|θ_i)=1?α (1?i?3). A procedure similar to Sobel–Wald procedure is discussed and its asymptotic efficiency as compared with the best nonsequential test is obtained by finding the limit lim_a→0(E_iN(a)/n(a)), where N (a) is the stopping time of the proposed procedure and n(a) is the sample size of the best non-sequential test. It is shown that the same asymptotic limit holds for the original Sobel–Wald procedure. Specializing to N(θ,1) distribution it is found that lim_a→0$\text{(E}{}_{\mathrm{i}}\text{N(α)/n(α))=}\text{1}\text{4}$ (i=1,2) and lim_a→0 (E₃N(α)n(α))=δ²₁/4δ, where δ_i=(θ_i+1?θ_i) with 0<δ₁?δ₂. Also, the asymptotic efficiency evaluated when the X's have an exponential distribution.