Consistency of stochastic approximation algorithm with quasi-associated random errors

ABSTRACT

Many mathematical and physical problems are led to find a root of a real function f. This kind of equation is an inverse problem and it is difficult to solve it. Especially in engineering sciences, the analytical expression of the function f is unknown to the experimenter, but it can be measured at each point x_k with M(x_k) as expected value and induced error ξ_k. The aim is to approximate the unique root θ under some assumptions on the function f and errors ξ_k. We use a stochastic approximation algorithm that constructs a sequence (x_k)_{k ? 1}. We establish the almost complete convergence of the sequence (x_k)_k to the exact root θ by considering the errors (ξ_k)_k quasi-associated and we illustrate the method by numerical examples to show its efficiency.     

Exponential Family Techniques for the Lognormal Left Tail

Let X be lognormal(μ,σ²) with density f(x); let θ > 0 and define . We study properties of the exponentially tilted density (Esscher transform) f_θ(x) = e^?θxf(x)/L(θ), in particular its moments, its asymptotic form as θ→∞ and asymptotics for the saddlepoint θ(x) determined by . The asymptotic formulas involve the Lambert W function. The established relations are used to provide two different numerical methods for evaluating the left tail probability of the sum of lognormals S_n=X₁+?+X_n: a saddlepoint approximation and an exponential tilting importance sampling estimator. For the latter, we demonstrate logarithmic efficiency. Numerical examples for the cdf F_n(x) and the pdf f_n(x) of S_n are given in a range of values of σ²,n and x motivated by portfolio value‐at‐risk calculations.     

BAHADUR SLOPES FOR COMPARISONS OF TESTS BASED ON RESAMPLING

Let X₁,…, X_n be random variables symmetric about θ from a common unknown distribution F_θ(x) =F(x–θ). To test the null hypothesis H₀:θ= 0 against the alternative H₁:θ > 0, permutation tests can be used at the cost of computational difficulties. This paper investigates alternative tests that are computationally simpler, notably some bootstrap tests which are compared with permutation tests. Of these the symmetrical bootstrap-f test competes very favourably with the permutation test in terms of Bahadur asymptotic efficiency, so it is a very attractive alternative.     

Estimation of a symmetric density

Kraft, Lepage, and van Eeden (1985) have suggested using a symmetrized version of the kernel estimator when the true density f of the observation is known to be symmetric around a possibly unknown point θ. The effect of this symmetrization device depends on the smoothness of f * f(x) = f f(x+t)f(t) dt at zero. We show that if θ has to be estimated and if f is not absolutely continuous, symmetrization may deteriorate the estimate.     

THE LIMITING FORM OF THE NON-CENTRAL WISHART DISTRIBUTION1,2

Let S (p×p) have a Wishart distribution -with v degrees of freedom and non-centrality matrix θ= [θ_jK] (p×p). Define θ₀= min {| θ_jk |}, let θ₀→∞, and suppose that | θ_jK | = 0(θ_o). Then the limiting form of the standardized non-central distribution, as θ while n? remains fixed, is a multivariate Gaussian distribution. This result in turn is used to obtain known asymptotic properties of multivariate chi-square and Rayleigh distributions under somewhat weaker conditions.     

MINIMUM MSE ESTIMATION BY LINEAR COMBINATIONS OF ORDER STATISTICS1

In this paper we assume that in a random sample of size ndrawn from a population having the pdf f(x; θ) the smallest r₁ observations and the largest r₂ observations are censored (r₁0, r₂0). We consider the problem of estimating θ on the basis of the middle n-r₁-r₂ observations when either f(x;θ)=θ^-1f(x/θ) or f(x;θ) = (aθ)¹f(x-θ)/aθ) where f(·) is a known pdf, a (<0) is known and θ (>0) is unknown. The minimum mean square error (MSE) linear estimator of θ proposed in this paper is a “shrinkage” of the minimum variance linear unbiased estimator of θ. We obtain explicit expressions of these estimators and their mean square errors when (i) f(·) is the uniform pdf defined on an interval of length one and (ii) f(·) is the standard exponential pdf, i.e., f(x) = exp(–x), x0. Various special cases of censoring from the left (right) and no censoring are considered.     

On the bounded regret of empirical bayes estimators

Let (θ₁,x₁),…,(θ_n,x_n) be independent and identically distributed random vectors with E(xθ) = θ and Var(x|θ) = a + bθ + cθ². Let t_i be the linear Bayes estimator of θ_i and θ~_i be the linear empirical Bayes estimator of θ_i as proposed in Robbins (1983). When Ex and Var x are unknown to the statistician. The regret of using θ~_i instead of t_i because of ignorance of the mean and the variance is r_i = E(θ_i ? θ_i)² ?E(t_i ?θ_i)². Under appropriate conditions cumulative regret R_n = r₁+…r_n is shown to have a finite limit even when n tends to infinity. The limit can be explicitly computed in terms of a,b,c and the first four moments of x.     

Minimax Estimation of the Bounded Parameter of Some Discrete Distributions Under LINEX Loss Function

For a class of discrete distributions, including Poisson(θ), Generalized Poisson(θ), Borel(m, θ), etc., we consider minimax estimation of the parameter θ under the assumption it lies in a bounded interval of the form [0, m] and a LINEX loss function. Explicit conditions for the minimax estimator to be Bayes with respect to a boundary supported prior are given. Also for Bernoulli(θ)-distribution, which is not in the mentioned class of discrete distributions, we give conditions for which the Bayes estimator of θ ∈ [0, m], m < 1 with respect to a boundary supported prior is minimax under LINEX loss function. Numerical values are given for the largest values of m for which the corresponding Bayes estimators of θ are minimax.     

Local asymptotic normality of intermediate and central order statistics

Consider n independent random variables Z_i,…, Z_n on R with common distribution function F, whose upper tail belongs to a parametric family F(t) = F_θ(t),t ≥ x₀, where θ ∈ ? ? R ^d. A necessary and sufficient condition for the family F_θ, θ ∈ ?, is established such that the k-th largest order statistic Z_n?k+1:n alone constitutes the central sequence yielding local asymptotic normality ( LAN ) of the loglikelihood ratio of the vector (Z_n?i+1:n)¹ _i=kof the k largest order statistics. This is achieved for k = k(n)→_n→∞∞ with k/n→_n→∞ 0.

In the case of vectors of central order statistics ( Z_r:n, Z_r+1:n,…, Z_s:n ), with r/n and s/n both converging to q ∈ ( 0,1 ), it turns out that under fairly general conditions any order statistic Z_m:n with r ≤ m ≤s builds the central sequence in a pertaining LAN expansion.These results lead to asymptotically optimal tests and estimators of the underlying parameter, which depend on single order statistics only     

Estimation of a multivariate mean with constraints on the norm

Estimation of the mean θ of a spherical distribution with prior knowledge concerning the norm ||θ|| is considered. The best equivariant estimator is obtained for the local problem ||θ|| = λ₀, and its risk is evaluated. This yields a sharp lower bound for the risk functions of a large class of estimators. The risk functions of the best equivariant estimator and the best linear estimator are compared under departures from the assumption ||θ|| = λ₀.     

A new class of skew-symmetric distributions and related families

In this work, we investigate a new class of skew-symmetric distributions, which includes the distributions with the probability density function (pdf) given by g _α(x)=2f(x) G(α x), introduced by Azzalini [A class of distributions which includes the normal ones, Scand. J. Statist. 12 (1985), pp. 171–178]. We call this new class as the symmetric-skew-symmetric family and it has the pdf proportional to f(x) G _β(α x), where G _β(x) is the cumulative distribution function of g _β(x). We give some basic properties for the symmetric-skew-symmetric family and study the particular case obtained from the normal distribution.     

Sequential Fixed-Width Confidence Interval for a Function of the Parameters

Let X₁…, X_m and Y₁…, Y_n be two independent sequences of i.i.d. random variables with distribution functions F_x(.|θ) and F_y(. | φ) respectively. Let g(θ, φ) be a real-valued function of the unknown parameters θ and φ. The purpose of this paper is to suggest a sequential procedure which gives a fixed-width confidence interval for g(θ, φ) so that the coverage probability is approximately α (preas-signed). Certain asymptotic optimality properties of the sequential procedure are established. A Monte Carlo study is presented.     

A modified adaptive accept–reject algorithm for univariate densities with bounded support

The need to simulate from a univariate density arises in several settings, particularly in Bayesian analysis. An especially efficient algorithm which can be used to sample from a univariate density, f _X , is the adaptive accept–reject algorithm. To implement the adaptive accept–reject algorithm, the user has to envelope T ° f _X , where T is some transformation such that the density g(x) ∝ T ^?1 (α+β x) is easy to sample from. Successfully enveloping T ° f _X , however, requires that the user identify the number and location of T ° f _X ’s inflection points. This is not always a trivial task. In this paper, we propose an adaptive accept–reject algorithm which relieves the user of precisely identifying the location of T ° f _X ’s inflection points. This new algorithm is shown to be efficient and can be used to sample from any density such that its support is bounded and its log is three-times differentiable.     

Non-parametric estimation of an acceleration parameter

Let X₁,… X_m be a random sample of m failure times under normal conditions with the underlying distribution F(x) and Y₁,…,Y_n a random sample of n failure times under accelerated condititons with underlying distribution G(x);G(x)=1?[1?F(x)]^θ with θ being the unknown parameter under study.Define:U_ij=1 otherwise.The joint distribution of _ijdoes not involve the distribution F and thus can be used to estimate the acceleration parameter θ.The second approach for estimating θ is to use the ranks of the Y-observations in the combined X- and Y-samples.In this paper we establish that the rank of the Y-observations in the pooled sample form a sufficient statistic for the information contained in the U_ii 's about the parameter θ and that there does not exist an unbiassed estimator for the parameter θ.We also construct several estimators and confidence interavals for the parameter θ.     

On the validation of fiducial techniques

A structured model is essentially a family of random vectors X_θ defined on a probability space with values in a sample space. If, for a given sample value x and for each ω in the probability space, there is at most one parameter value θ for which X_θ(ω) is equal to x, then the model is called additive at x. When a certain conditional distribution exists, a frequency interpretation specific to additive structured models holds, and is summarized in a unique structured distribution for the parameter. Many of the techniques used by Fisher in deriving and handling his fiducial probability distribution are shown to be valid when dealing with a structured distribution.     

Asymptotic behavior of the grenander estimator at density flat regions

Over forty years ago, Grenander derived the MLE of a monotone decreasing density f with known mode. Prakasa Rao obtained the asymptotic distribution of this estimator at a fixed point x where f' (x) < 0. Here, we obtain the asymptotic distribution of this estimator at a fixed point x when f is constant and nonzero in some open neighborhood of x. This limiting distribution is expressible as the convolution of a closed-form density and a rescaled standard normal density. Groeneboom (1983) derived the aforementioned closed-form density and we provide an alternative, more direct derivation.