On Minimum Distance Estimation Using Kolmogorov-Lévy Type Metrics

Let X ₁, . . ., X_n be independent identically distributed random variables with a common continuous (cumulative) distribution function (d.f.) F , and F^_n the empirical d.f. (e.d.f.) based on X ₁, . . ., X_n . Let G be a smooth d.f. and Gθ = G (·–θ) its translation through θ∈ R . Using a Kolmogorov-Lévy type metric ρ_α defined on the space of d.f.s. on R , the paper derives both null and non-null limiting distributions of √ n [ ρ_α ( F_n , Gθ_n ) – ρ_α ( F, G_θ )], √ n (θ _n –θ) and √ nρ_α ( Gθ , Gθ ), where θ _n and θ are the minimum ρ_α -distance parameters for F_n and F from G , respectively. These distributions are known explicitly in important particular cases; with some complementary Monte Carlo simulations, they help us clarify our understanding of estimation using minimum distance methods and supremum type metrics. We advocate use of the minimum distance method with supremum type metrics in cases of non-null models. The resulting functionals are Hadamard differentiable and efficient. For small scale parameters the minimum distance functionals are close to medians of the parent distributions. The optimal small scale models result in minimum distance estimators having asymptotic variances very competitive and comparable with best known robust estimators.     

Ordering of Sequentially Sampled Exponential Experiments

Let X ₁, X ₂, ... be a sequence of i.i.d. random variables, X _i∼ F _θ, θ∈Θ. Let N ₁ and N ₂ be two stopping rules. For a class of exponential families { F _θ: θ∈Θ} we show that the experiment Y ₁ = ( X ₁, ..., X _N1) carries more statistical information than Y ₂ = ( X ₁, ..., x _N2) only if N ₁ is stochastically larger then N ₂     

Exact Slopes of Test Statistics for the Multivariate Exponential Family

The objective of this paper is to investigate exact slopes of test statistics { T_n } when the random vectors X ₁, ..., X_n are distributed according to an unknown member of an exponential family { P _θ; θ∈Ω. Here Ω is a parameter set. We will be concerned with the hypothesis testing problem of H ₀θ∈Ω₀ vs H ₁: θ∉Ω₀ where Ω₀ is a subset of Ω. It will be shown that for an important class of problems and test statistics the exact slope of { T_n } at η in Ω−Ω₀ is determined by the shortest Kullback–Leibler distance from {θ: T_n (λ(θ)) = T_n (λ(π))} to Ω₀, λ_θ = E _θ)( X ).     

MINIMAX ESTIMATION OF A MULTIVARIATE NORMAL MEAN UNDER A CONVEX LOSS FUNCTION

Let X = (X₁, - X_p)prime; ˜ N_p (μ, Σ) where μ= (μ₁, -, μ_p)' and Σ= diag (Σ²₁, -, Σ²_p) are both unknown and p3. Let (n_i - 2) w_i/Σ²_i! X²_ni, independent. of w_i (I ≠ j = 1, -, p). Assume that (w₁, -, w_p) and X are independent. Define W = diag (w₁, -, w_p) and ¶ X ¶²_w= X'W^-1Q^-1W^-1X where Q = diag (q₁, -,n q_p), q_i > 0, i = 1, -, p. In this paper, the minimax estimator of Berger & Bock (1976), given by δ (X, W) = [I_p - r(X, W) ¶ X ¶^-2_w Q^-1W^-1] X, is shown to be minimax relative to the convex loss (δ - μ)'[αQ + (1 - α) Σ^-1] δ - μ)/C, where C =α tr (Σ) + (1 - α)p and 0 α 1, under certain conditions on r(X, W). This generalizes the above mentioned result of Berger & Bock.     

Computation of the Generalized F Distribution

Exact expressions for the cumulative distribution function of a random variable of the form ( α ₁ X ₁+ α ₂ X ₂)/ Y are given where X ₁, X ₂ and Y are independent chi-squared random variables. The expressions are applied to the detection of joint outliers and Hotelling's mis-specified T ² distribution.     

Semiparametric Likelihood Based Method for Goodness of Fit Tests and Estimation in Upgraded Mixture Models

We use Owen's (1988, 1990) empirical likelihood method in upgraded mixture models. Two groups of independent observations are available. One is z ₁, ..., z _n which is observed directly from a distribution F ( z ). The other one is x ₁, ..., x _m which is observed indirectly from F ( z ), where the x _i s have density ∫ p ( x | z ) dF ( z ) and p ( x | z ) is a conditional density function. We are interested in testing H ₀: p ( x | z ) = p ( x | z ; θ ), for some specified smooth density function. A semiparametric likelihood ratio based statistic is proposed and it is shown that it converges to a chi-squared distribution. This is a simple method for doing goodness of fit tests, especially when x is a discrete variable with finitely many values. In addition, we discuss estimation of θ and F ( z ) when H ₀ is true. The connection between upgraded mixture models and general estimating equations is pointed out.     

ON THE CONVERGENCE OF EMPIRICAL PROCESSES OF MIXING VARIABLES1

Conditions are given for the weak convergence of (t—t²)L_N(a_N^-1( t )) to a Gaussian process where v<1/2, a _N is a cdf and L _N is the normalized weighted empirical cumulative distribution function (cdf) for an α-mixing sample of random variables in R which may be non-stationary with discontinuous marginals.     

Detecting changes in the mean of functional observations

Summary.  Principal component analysis has become a fundamental tool of functional data analysis. It represents the functional data as X _i ( t )= μ ( t )+Σ_{1≤ l <∞} η _{i ,  l} +  v _l ( t ), where μ is the common mean, v _l are the eigenfunctions of the covariance operator and the η _{i ,  l} are the scores. Inferential procedures assume that the mean function μ ( t ) is the same for all values of i . If, in fact, the observations do not come from one population, but rather their mean changes at some point(s), the results of principal component analysis are confounded by the change(s). It is therefore important to develop a methodology to test the assumption of a common functional mean. We develop such a test using quantities which can be readily computed in the R package fda. The null distribution of the test statistic is asymptotically pivotal with a well-known asymptotic distribution. The asymptotic test has excellent finite sample performance. Its application is illustrated on temperature data from England.     

Some Bounds on the Distribution of Certain Quadratic Forms in Normal Random Variables

The paper derives bounds on the distribution of the quadratic forms Z = y ^H( X Γ X ^H)⁻¹ y and W = y ^H(σ² I + X Γ X ^H)⁻¹ y , where the elements of the M × 1 vector y and the M × N matrix X are independent identically distributed (i.i.d.) complex zero mean Normal variables, Γ is some N × N diagonal matrix with positive diagonal elements, I , is the identity, σ² is a constant and ^H denotes the Hermitian transpose. The bounds are convenient for numerical work and appear to be tight for small values of M . This work has applications in digital mobile radio for a specific channel where M antennas are used to receive a signal with N interferers. Some of these applications in radio communication systems are discussed.     

Non-parametric Regression with Dependent Censored Data

Abstract.  Let ( X _i , Y _i ) ( i = 1 ,…, n ) be n replications of a random vector ( X , Y  ), where Y is supposed to be subject to random right censoring. The data ( X _i , Y _i ) are assumed to come from a stationary α -mixing process. We consider the problem of estimating the function m ( x ) = E ( φ ( Y ) |  X = x ), for some known transformation φ . This problem is approached in the following way: first, we introduce a transformed variable     , that is not subject to censoring and satisfies the relation     , and then we estimate m ( x ) by applying local linear regression techniques. As a by-product, we obtain a general result on the uniform rate of convergence of kernel type estimators of functionals of an unknown distribution function, under strong mixing assumptions.     

Consistency of the Semi-parametric MLE in Linear Regression Models with Interval-censored Data

Abstract.  Consider the model Y = β ^' X + ε . Let F ₀ be the unknown cumulative distribution function of the random variable ε . Consistency of the semi-parametric Maximum likelihood estimator of ( β , F ₀), denoted by     , has not been established under any interval censorship (IC) model. We prove in this paper that     is consistent under the mixed case IC model and some mild assumptions.     

A Non-parametric Test for Generalized First-order Autoregressive Models

We derive a non-parametric test for testing the presence of V(X_i,ε_i) in the non-parametric first-order autoregressive model X_i+1=T(X_i)+V(X_i,ε_i)+U(X_i)ε_i+1, where the function T(x) is assumed known. The test is constructed as a functional of a basic process for which we establish a weak invariance principle, under the null hypothesis and under stationarity and mixing assumptions. Bounds for the local and non-local powers are provided under a condition which ensures that the power tends to one as the sample size tends to infinity.The testing procedure can be applied, e.g. to bilinear models, ARCH models, EXPAR models and to some other uncommon models. Our results confirm the robustness of the test constructed in Ngatchou Wandji (1995) and in Diebolt & Ngatchou Wandji (1995).     

On the effect of overdispersion on exact conditional tests

Let Y ₁, . . ., Y_n denote independent random variables such that Y_j has a one-parameter exponential family distribution with canonical parameter θ _j =λ+ψ X_j ; here X ₁, . . ., X_n are known constants. Consider a test of the null hypothesis ψ=0. Under the null hypothesis, A =Σ Y_j is sufficient for λ and, hence, a test of ψ=0 may be based on the conditional distribution of T =Σ X_j Y_j given A , which is independent of λ. In this paper, the effects of overdispersion due to a mixture model on the conditional distribution of T given A are considered.     

Empirical Likelihood for Non-Smooth Criterion Functions

Abstract.  Suppose that X ₁ ,…,  X _n is a sequence of independent random vectors, identically distributed as a d -dimensional random vector X . Let     be a parameter of interest and     be some nuisance parameter. The unknown, true parameters ( μ ₀ , ν ₀ ) are uniquely determined by the system of equations E { g ( X , μ ₀ , ν ₀ )} =   0 , where g  =  ( g ₁ ,…, g _{p + q} ) is a vector of p + q functions. In this paper we develop an empirical likelihood (EL) method to do inference for the parameter μ ₀ . The results in this paper are valid under very mild conditions on the vector of criterion functions g . In particular, we do not require that g ₁ ,…, g _{p + q} are smooth in μ or ν . This offers the advantage that the criterion function may involve indicators, which are encountered when considering, e.g. differences of quantiles, copulas, ROC curves, to mention just a few examples. We prove the asymptotic limit of the empirical log-likelihood ratio, and carry out a small simulation study to test the performance of the proposed EL method for small samples.     

Estimation and testing stationarity for double-autoregressive models

Summary.  The paper considers the double-autoregressive model y _t  =  φ y _{t −1}+ ɛ _t with ɛ _t  =     . Consistency and asymptotic normality of the estimated parameters are proved under the condition E  ln | φ  +√ α η _t |<0, which includes the cases with | φ |=1 or | φ |>1 as well as     . It is well known that all kinds of estimators of φ in these cases are not normal when ɛ _t are independent and identically distributed. Our result is novel and surprising. Two tests are proposed for testing stationarity of the model and their asymptotic distributions are shown to be a function of bivariate Brownian motions. Critical values of the tests are tabulated and some simulation results are reported. An application to the US 90-day treasury bill rate series is given.     

OPTIMUM SINGULAR SPRING BALANCE WEIGHING DESIGNS WITH NON-HOMOGENEITY OF THE VARIANCES OF ERRORS FOR ESTIMATING THE TOTAL WEIGHT

The problem of estimation of the total weight of objects using a singular spring balance weighing design with non-homogeneity of the variances of errors has been dealt with in this paper. Based on a theorem by Katulska (1984) giving a lower bound for the variance of the estimated total weight, a necessary and sufficient condition for this lower bound to be attained is obtained. It is shown that weighing designs for which the the lower bound is attainable, can be constructed from the incidence matrices of (α₁,.,α_t)-resolvable block designs, α-resolvable block designs, singular group divisible designs, and semi-regular group divisible designs.     

An Analysis of Swendsen–Wang and Related Sampling Methods

Convergence rates, statistical efficiency and sampling costs are studied for the original and extended Swendsen–Wang methods of generating a sample path { S _j , j ≥1} with equilibrium distribution π , with r distinct elements, on a finite state space X of size N ₁. Given S _{j -1}, each method uses auxiliary random variables to identify the subset of X from which S _j is to be randomly sampled. Let π_min and π_max denote respectively the smallest and largest elements in π and let N_r denote the number of elements in π with value π_max. For a single auxiliary variable, uniform sampling from the subset and ( N ₁− N_r )π_min+ N_r π_max≈1, our results show rapid convergence and high statistical efficiency for large π_min/π_max or N_r / N ₁ and slow convergence and poor statistical efficiency for small π_min/π_max and N_r / N₁ . Other examples provide additional insight. For extended Swendsen–Wang methods with non-uniform subset sampling, the analysis identifies the properties of a decomposition of π( x ) that favour fast convergence and high statistical efficiency. In the absence of exploitable special structure, subset sampling can be costly regardless of which of these methods is employed.     

A Problem of Dimensionality in Normal Mixture Analysis

Suppose the p -variate random vector W , partitioned into q variables W₁ and p - q variables W₂, follows a multivariate normal mixture distribution. If the investigator is mainly interested in estimation of the parameters of the distribution of W₁, there are two possibilities: (1) use only the data on W₁ for estimation, and (2) estimate the parameters of the p -variate mixture distribution, and then extract the estimates of the marginal distribution of W₁. In this article we study the choice between these two possibilities mainly for the case of two mixture components with identical covariance matrices. We find the asymptotic distribution of the linear discriminant function coefficients using the work of Efron (1975 ) and O'Neill (1978 ), and give a Wald–test for redundancy of W₂. A simulation study gives further insights into conditions under which W₂ should be used in the analysis: in summary, the inclusion of W₂ seems justified if Δ 2.1, the Mahalanobis distance between the two component distributions based on the conditional distribution of W₂ given W₁, is at least 2.     

Approximations of distributions for some standardized partial sums in sequential analysis

In sequential analysis it is often necessary to determine the distributions of √t Y _t and/or √a Y _t , where t is a stopping time of the form t = inf{ n ≥ 1 : n+S_n+ξ_n> a }, Y _n is the sample mean of n independent and identically distributed random variables (iidrvs) Y_i with mean zero and variance one, S_n is the partial sum of iidrvs X_i with mean zero and a positive finite variance, and { ξ_n } is a sequence of random variables that converges in distribution to a random variable ξ as n →∞ and ξ_n is independent of ( X_n+1, Y_n+1), (X_n+2, Y_n+2), . . . for all n ≥ 1. Anscombe's (1952) central limit theorem asserts that both √t Y _t and √a Y _t are asymptotically normal for large a , but a normal approximation is not accurate enough for many applications. Refined approximations are available only for a few special cases of the general setting above and are often very complex. This paper provides some simple Edgeworth approximations that are numerically satisfactory for the problems it considers.     

IMPROVING UPON THE BEST INVARIANT ESTIMATOR IN MULTIVARIATE LOCATION PROBLEMS

We are concerned with estimators which improve upon the best invariant estimator, in estimating a location parameter θ. If the loss function is L(θ - a) with L convex, we give sufficient conditions for the inadmissibility of δ₀(X) = X. If the loss is a weighted sum of squared errors, we find various classes of estimators δ which are better than δ₀. In general, δ is the convolution of δ₁ (an estimator which improves upon δ₀ outside of a compact set) with a suitable probability density in R^p. The critical dimension of inadmissibility depends on the estimator δ₁ We also give several examples of estimators δ obtained in this way and state some open problems.