Estimation of an Ergodic Diffusion from Discrete Observations

We consider a one-dimensional diffusion process X , with ergodic property, with drift b ( x , θ) and diffusion coefficient a ( x , σ) depending on unknown parameters θ and σ. We are interested in the joint estimation of (θ, σ). For that purpose, we dispose of a discretized trajectory, observed at n equidistant times tⁿ_i = ih_n , 1 ≤ i ≤ n . We assume that h_n ← 0 and nh_n ←∞. Under the condition nh_n^p ← 0 for an arbitrary integer p , we exhibit a contrast dependent on p which provides us with an asymptotically normal and efficient estimator of (θ, σ).     

IMPROVING UPON THE BEST INVARIANT ESTIMATOR IN MULTIVARIATE LOCATION PROBLEMS

We are concerned with estimators which improve upon the best invariant estimator, in estimating a location parameter θ. If the loss function is L(θ - a) with L convex, we give sufficient conditions for the inadmissibility of δ₀(X) = X. If the loss is a weighted sum of squared errors, we find various classes of estimators δ which are better than δ₀. In general, δ is the convolution of δ₁ (an estimator which improves upon δ₀ outside of a compact set) with a suitable probability density in R^p. The critical dimension of inadmissibility depends on the estimator δ₁ We also give several examples of estimators δ obtained in this way and state some open problems.     

On Minimum Distance Estimation Using Kolmogorov-Lévy Type Metrics

Let X ₁, . . ., X_n be independent identically distributed random variables with a common continuous (cumulative) distribution function (d.f.) F , and F^_n the empirical d.f. (e.d.f.) based on X ₁, . . ., X_n . Let G be a smooth d.f. and Gθ = G (·–θ) its translation through θ∈ R . Using a Kolmogorov-Lévy type metric ρ_α defined on the space of d.f.s. on R , the paper derives both null and non-null limiting distributions of √ n [ ρ_α ( F_n , Gθ_n ) – ρ_α ( F, G_θ )], √ n (θ _n –θ) and √ nρ_α ( Gθ , Gθ ), where θ _n and θ are the minimum ρ_α -distance parameters for F_n and F from G , respectively. These distributions are known explicitly in important particular cases; with some complementary Monte Carlo simulations, they help us clarify our understanding of estimation using minimum distance methods and supremum type metrics. We advocate use of the minimum distance method with supremum type metrics in cases of non-null models. The resulting functionals are Hadamard differentiable and efficient. For small scale parameters the minimum distance functionals are close to medians of the parent distributions. The optimal small scale models result in minimum distance estimators having asymptotic variances very competitive and comparable with best known robust estimators.     

Ordering of Sequentially Sampled Exponential Experiments

Let X ₁, X ₂, ... be a sequence of i.i.d. random variables, X _i∼ F _θ, θ∈Θ. Let N ₁ and N ₂ be two stopping rules. For a class of exponential families { F _θ: θ∈Θ} we show that the experiment Y ₁ = ( X ₁, ..., X _N1) carries more statistical information than Y ₂ = ( X ₁, ..., x _N2) only if N ₁ is stochastically larger then N ₂     

Estimation of Diffusion Processes by Simulated Moment Methods

We consider the parameter estimation of a diffusion process and we suppose that the trend and the diffusion coefficient depend on the parameter θ. The process is observed at time ( t_i ) _{i =0,..., n} with Δ = t_{i +1}− t_i fixed and we propose here to estimate θ from simulated moment methods.     

Semiparametric Likelihood Based Method for Goodness of Fit Tests and Estimation in Upgraded Mixture Models

We use Owen's (1988, 1990) empirical likelihood method in upgraded mixture models. Two groups of independent observations are available. One is z ₁, ..., z _n which is observed directly from a distribution F ( z ). The other one is x ₁, ..., x _m which is observed indirectly from F ( z ), where the x _i s have density ∫ p ( x | z ) dF ( z ) and p ( x | z ) is a conditional density function. We are interested in testing H ₀: p ( x | z ) = p ( x | z ; θ ), for some specified smooth density function. A semiparametric likelihood ratio based statistic is proposed and it is shown that it converges to a chi-squared distribution. This is a simple method for doing goodness of fit tests, especially when x is a discrete variable with finitely many values. In addition, we discuss estimation of θ and F ( z ) when H ₀ is true. The connection between upgraded mixture models and general estimating equations is pointed out.     

Non-parametric Regression with Dependent Censored Data

Abstract.  Let ( X _i , Y _i ) ( i = 1 ,…, n ) be n replications of a random vector ( X , Y  ), where Y is supposed to be subject to random right censoring. The data ( X _i , Y _i ) are assumed to come from a stationary α -mixing process. We consider the problem of estimating the function m ( x ) = E ( φ ( Y ) |  X = x ), for some known transformation φ . This problem is approached in the following way: first, we introduce a transformed variable     , that is not subject to censoring and satisfies the relation     , and then we estimate m ( x ) by applying local linear regression techniques. As a by-product, we obtain a general result on the uniform rate of convergence of kernel type estimators of functionals of an unknown distribution function, under strong mixing assumptions.     

Exact Slopes of Test Statistics for the Multivariate Exponential Family

The objective of this paper is to investigate exact slopes of test statistics { T_n } when the random vectors X ₁, ..., X_n are distributed according to an unknown member of an exponential family { P _θ; θ∈Ω. Here Ω is a parameter set. We will be concerned with the hypothesis testing problem of H ₀θ∈Ω₀ vs H ₁: θ∉Ω₀ where Ω₀ is a subset of Ω. It will be shown that for an important class of problems and test statistics the exact slope of { T_n } at η in Ω−Ω₀ is determined by the shortest Kullback–Leibler distance from {θ: T_n (λ(θ)) = T_n (λ(π))} to Ω₀, λ_θ = E _θ)( X ).     

Likelihood Ratio Tests Under Local and Fixed Alternatives in Monotone Function Problems

Abstract.  We focus on a class of non-standard problems involving non-parametric estimation of a monotone function that is characterized by n ^1/3 rate of convergence of the maximum likelihood estimator, non-Gaussian limit distributions and the non-existence of     -regular estimators. We have shown elsewhere that under a null hypothesis of the type ψ ( z ₀) =  θ ₀ ( ψ being the monotone function of interest) in non-standard problems of the above kind, the likelihood ratio statistic has a 'universal' limit distribution that is free of the underlying parameters in the model. In this paper, we illustrate its limiting behaviour under local alternatives of the form ψ _n ( z ), where ψ _n (·) and ψ (·) vary in O ( n ^−1/3) neighbourhoods around z ₀ and ψ _n converges to ψ at rate n ^1/3 in an appropriate metric. Apart from local alternatives, we also consider the behaviour of the likelihood ratio statistic under fixed alternatives and establish the convergence in probability of an appropriately scaled version of the same to a constant involving a Kullback–Leibler distance.     

A New Kind of Asymptotic Quasi-Score Estimating Function

A new definition of asymptotic quasi-score sequence of estimating functions is given and studied. The relationship between asymptotic quasi-likelihood and quasi-likelihood estimates is investigated. A new practical approach for obtaining a good estimate of θ in the model y _t = f_t (θ) + m_t without any prior knowledge on the nature of E ( m ² _t |F _{t −1}) is suggested, where f_t is a predictable process and m_t is a martingale difference process. Two examples are used to show that the approach is practicable.     

Root-n-consistent Estimation in Partial Linear Models with Long-memory Errors

We consider estimation of β in the semiparametric regression model y ( i ) - x ^T( i )β + f ( i / n ) + ε( i ) where x ( i ) = g ( i )/ n ) + e ( i , f and g are unknown smooth functions and the processes ε( i ) and e ( i ) are stationary with short- or long-range dependence. For the case of i.i.d. errors, Speckman (1988) proposed a √ n –consistent estimator of β. In this paper it is shown that, under suitable regularity conditions, this estimator is asymptotically unbiased and √ n –consistent even if the errors exhibit long-range dependence. The orders of the finite sample bias and of the required bandwidth depend on the long-memory parameters. Simulations and a data example illustrate the method     

A class of local likelihood methods and near-parametric asymptotics

The local maximum likelihood estimate θ^ _t of a parameter in a statistical model f ( x , θ) is defined by maximizing a weighted version of the likelihood function which gives more weight to observations in the neighbourhood of t . The paper studies the sense in which f ( t , θ^ _t ) is closer to the true distribution g ( t ) than the usual estimate f ( t , θ^) is. Asymptotic results are presented for the case in which the model misspecification becomes vanishingly small as the sample size tends to ∞. In this setting, the relative entropy risk of the local method is better than that of maximum likelihood. The form of optimum weights for the local likelihood is obtained and illustrated for the normal distribution.     

Estimating Mixing Densities in Exponential Family Models for Discrete Variables

This paper is concerned with estimating a mixing density g using a random sample from the mixture distribution f(x)=∫f x | θ)g(θ)dθ where f(· | θ) is a known discrete exponen tial family of density functions. Recently two techniques for estimating g have been proposed. The first uses Fourier analysis and the method of kernels and the second uses orthogonal polynomials. It is known that the first technique is capable of yielding estimators that achieve (or almost achieve) the minimax convergence rate. We show that this is true for the technique based on orthogonal polynomials as well. The practical implementation of these estimators is also addressed. Computer experiments indicate that the kernel estimators give somewhat disappoint ing finite sample results. However, the orthogonal polynomial estimators appear to do much better. To improve on the finite sample performance of the orthogonal polynomial estimators, a way of estimating the optimal truncation parameter is proposed. The resultant estimators retain the convergence rates of the previous estimators and a Monte Carlo finite sample study reveals that they perform well relative to the ones based on the optimal truncation parameter.     

MINIMAX ESTIMATION OF A MULTIVARIATE NORMAL MEAN UNDER A CONVEX LOSS FUNCTION

Let X = (X₁, - X_p)prime; ˜ N_p (μ, Σ) where μ= (μ₁, -, μ_p)' and Σ= diag (Σ²₁, -, Σ²_p) are both unknown and p3. Let (n_i - 2) w_i/Σ²_i! X²_ni, independent. of w_i (I ≠ j = 1, -, p). Assume that (w₁, -, w_p) and X are independent. Define W = diag (w₁, -, w_p) and ¶ X ¶²_w= X'W^-1Q^-1W^-1X where Q = diag (q₁, -,n q_p), q_i > 0, i = 1, -, p. In this paper, the minimax estimator of Berger & Bock (1976), given by δ (X, W) = [I_p - r(X, W) ¶ X ¶^-2_w Q^-1W^-1] X, is shown to be minimax relative to the convex loss (δ - μ)'[αQ + (1 - α) Σ^-1] δ - μ)/C, where C =α tr (Σ) + (1 - α)p and 0 α 1, under certain conditions on r(X, W). This generalizes the above mentioned result of Berger & Bock.     

A Characterization of the Normal Family of Distributions Based on the Bias of the Maximum Likelihood Estimator

This paper characterizes the family of Normal distributions within the class of exponential families of distributions, via the structure of the bias of the maximum likelihood estimator Θ _n of the canonical parameter Θ . More specifically, when E _θ ( Θ n ) – Θ = (1/ n ) Q ( Θ ) + o (1/ n ), the equality Q ( Θ ) = 0 proves to be a property of the Normal distribution only. The same conclusion is obtained for the one-dimensional case bt assuming that Q ( Θ ) is a polynomial of Θ .     

A TWO-PHASE SAMPLING ESTIMATOR IN SAMPLE SURVEYS

A two-phase sampling estimator of the ratio-type for estimating the mean of a finite population, has been considered where the value of ρC_y/C_x can be guessed or estimated in advance. Here C_y and C_x denote respectively the coefficients of variation of the characteristic under study, y, and the auxiliary characteristic x and ρ denotes the coefficient of correlation between y and x. When the value of ρC_y/C_x is guessed or estimated exactly, the estimator has a smaller large-sample variance compared with either an ordinary ratio estimator or an ordinary linear regression estimator in two-phase sampling in the case where the first-phase sample is drawn independently from the second-phase sample. If the sample at the second phase is a subsample of the first-phase sample, the estimator has variance equal to that of the linear regression estimator. The largest value of the difference between the assumed value and the actual value of ρC_y/C_x has been obtained so as not to result in the variance of the estimator being larger than the variances of either an ordinary ratio estimator or an ordinary linear regression estimator.     

Penalized Projection Estimator for Volatility Density

Abstract.  In this paper, we consider a stochastic volatility model ( Y _t , V _t ), where the volatility (V _t ) is a positive stationary Markov process. We assume that ( ln V _t ) admits a stationary density f that we want to estimate. Only the price process Y _t is observed at n discrete times with regular sampling interval Δ . We propose a non-parametric estimator for f obtained by a penalized projection method. Under mixing assumptions on ( V _t ), we derive bounds for the quadratic risk of the estimator. Assuming that Δ=Δ _n tends to 0 while the number of observations and the length of the observation time tend to infinity, we discuss the rate of convergence of the risk. Examples of models included in this framework are given.     

A Limit Theorem for Solutions of Inequalities

Let H ( p ) be the set { x ∈ X : h ( x ) ≤ p } where h is a real-valued lower semicontinuous function on a locally compact separable metric space X . This paper presents a general limit theorem for the sequence of random sets H _n ( p ) = { x ∈ X : h _n ( x ) ≤ p } n ≥ 1, where h _n , n ≥ 1, are functions that estimate h     

Penalized Pseudolikelihood Inference in Spatial Interaction Models with Covariates

Given spatially located observed random variables ( x , z = {( x _i , z _i )} _i , we propose a new method for non-parametric estimation of the potential functions of a Markov random field p ( x | z ), based on a roughness penalty approach. The new estimator maximizes the penalized log-pseudolikelihood function and is a natural cubic spline. The calculations involved do not rely on Monte Carlo simulation. We suggest the use of B-splines to stabilize the numerical procedure. An application in Bayesian image reconstruction is described.     

A TWO STAGE PROCEDURE FOR ESTIMATING THE SIZE OF A TRUNCATED EXPONENTIAL SAMPLE

Let X₁, …, X_N be i.i.d. exponential random variables with unknown scale parameter θ. If one can observe only those X_i in (0, T₀), an estimate of N based on the J observations obtained has a variance which explodes as θ→θC. Using θC based on the observations in (0, T₀) T is computed and all X_i in (0, ) are observed. An estimate of N based on all observations in (0, ) has a bounded variance where the bound can be adjusted by proper choice of .