MINIMAX ESTIMATION OF A MULTIVARIATE NORMAL MEAN UNDER A CONVEX LOSS FUNCTION

Let X = (X₁, - X_p)prime; ˜ N_p (μ, Σ) where μ= (μ₁, -, μ_p)' and Σ= diag (Σ²₁, -, Σ²_p) are both unknown and p3. Let (n_i - 2) w_i/Σ²_i! X²_ni, independent. of w_i (I ≠ j = 1, -, p). Assume that (w₁, -, w_p) and X are independent. Define W = diag (w₁, -, w_p) and ¶ X ¶²_w= X'W^-1Q^-1W^-1X where Q = diag (q₁, -,n q_p), q_i > 0, i = 1, -, p. In this paper, the minimax estimator of Berger & Bock (1976), given by δ (X, W) = [I_p - r(X, W) ¶ X ¶^-2_w Q^-1W^-1] X, is shown to be minimax relative to the convex loss (δ - μ)'[αQ + (1 - α) Σ^-1] δ - μ)/C, where C =α tr (Σ) + (1 - α)p and 0 α 1, under certain conditions on r(X, W). This generalizes the above mentioned result of Berger & Bock.     

A Problem of Dimensionality in Normal Mixture Analysis

Suppose the p -variate random vector W , partitioned into q variables W₁ and p - q variables W₂, follows a multivariate normal mixture distribution. If the investigator is mainly interested in estimation of the parameters of the distribution of W₁, there are two possibilities: (1) use only the data on W₁ for estimation, and (2) estimate the parameters of the p -variate mixture distribution, and then extract the estimates of the marginal distribution of W₁. In this article we study the choice between these two possibilities mainly for the case of two mixture components with identical covariance matrices. We find the asymptotic distribution of the linear discriminant function coefficients using the work of Efron (1975 ) and O'Neill (1978 ), and give a Wald–test for redundancy of W₂. A simulation study gives further insights into conditions under which W₂ should be used in the analysis: in summary, the inclusion of W₂ seems justified if Δ 2.1, the Mahalanobis distance between the two component distributions based on the conditional distribution of W₂ given W₁, is at least 2.     

Semiparametric Likelihood Based Method for Goodness of Fit Tests and Estimation in Upgraded Mixture Models

We use Owen's (1988, 1990) empirical likelihood method in upgraded mixture models. Two groups of independent observations are available. One is z ₁, ..., z _n which is observed directly from a distribution F ( z ). The other one is x ₁, ..., x _m which is observed indirectly from F ( z ), where the x _i s have density ∫ p ( x | z ) dF ( z ) and p ( x | z ) is a conditional density function. We are interested in testing H ₀: p ( x | z ) = p ( x | z ; θ ), for some specified smooth density function. A semiparametric likelihood ratio based statistic is proposed and it is shown that it converges to a chi-squared distribution. This is a simple method for doing goodness of fit tests, especially when x is a discrete variable with finitely many values. In addition, we discuss estimation of θ and F ( z ) when H ₀ is true. The connection between upgraded mixture models and general estimating equations is pointed out.     

Bayesian model selection for join point regression with application to age-adjusted cancer rates

Summary.  The method of Bayesian model selection for join point regression models is developed. Given a set of K +1 join point models M ₀,  M ₁, …,  M _K with 0, 1, …,  K join points respec-tively, the posterior distributions of the parameters and competing models M _k are computed by Markov chain Monte Carlo simulations. The Bayes information criterion BIC is used to select the model M _k with the smallest value of BIC as the best model. Another approach based on the Bayes factor selects the model M _k with the largest posterior probability as the best model when the prior distribution of M _k is discrete uniform. Both methods are applied to analyse the observed US cancer incidence rates for some selected cancer sites. The graphs of the join point models fitted to the data are produced by using the methods proposed and compared with the method of Kim and co-workers that is based on a series of permutation tests. The analyses show that the Bayes factor is sensitive to the prior specification of the variance σ ², and that the model which is selected by BIC fits the data as well as the model that is selected by the permutation test and has the advantage of producing the posterior distribution for the join points. The Bayesian join point model and model selection method that are presented here will be integrated in the National Cancer Institute's join point software ( http://www.srab.cancer.gov/joinpoint/ ) and will be available to the public.     

IMPROVING UPON THE BEST INVARIANT ESTIMATOR IN MULTIVARIATE LOCATION PROBLEMS

We are concerned with estimators which improve upon the best invariant estimator, in estimating a location parameter θ. If the loss function is L(θ - a) with L convex, we give sufficient conditions for the inadmissibility of δ₀(X) = X. If the loss is a weighted sum of squared errors, we find various classes of estimators δ which are better than δ₀. In general, δ is the convolution of δ₁ (an estimator which improves upon δ₀ outside of a compact set) with a suitable probability density in R^p. The critical dimension of inadmissibility depends on the estimator δ₁ We also give several examples of estimators δ obtained in this way and state some open problems.     

OPTIMUM SINGULAR SPRING BALANCE WEIGHING DESIGNS WITH NON-HOMOGENEITY OF THE VARIANCES OF ERRORS FOR ESTIMATING THE TOTAL WEIGHT

The problem of estimation of the total weight of objects using a singular spring balance weighing design with non-homogeneity of the variances of errors has been dealt with in this paper. Based on a theorem by Katulska (1984) giving a lower bound for the variance of the estimated total weight, a necessary and sufficient condition for this lower bound to be attained is obtained. It is shown that weighing designs for which the the lower bound is attainable, can be constructed from the incidence matrices of (α₁,.,α_t)-resolvable block designs, α-resolvable block designs, singular group divisible designs, and semi-regular group divisible designs.     

Penalized Pseudolikelihood Inference in Spatial Interaction Models with Covariates

Given spatially located observed random variables ( x , z = {( x _i , z _i )} _i , we propose a new method for non-parametric estimation of the potential functions of a Markov random field p ( x | z ), based on a roughness penalty approach. The new estimator maximizes the penalized log-pseudolikelihood function and is a natural cubic spline. The calculations involved do not rely on Monte Carlo simulation. We suggest the use of B-splines to stabilize the numerical procedure. An application in Bayesian image reconstruction is described.     

ON THE CONVERGENCE OF EMPIRICAL PROCESSES OF MIXING VARIABLES1

Conditions are given for the weak convergence of (t—t²)L_N(a_N^-1( t )) to a Gaussian process where v<1/2, a _N is a cdf and L _N is the normalized weighted empirical cumulative distribution function (cdf) for an α-mixing sample of random variables in R which may be non-stationary with discontinuous marginals.     

A TWO STAGE PROCEDURE FOR ESTIMATING THE SIZE OF A TRUNCATED EXPONENTIAL SAMPLE

Let X₁, …, X_N be i.i.d. exponential random variables with unknown scale parameter θ. If one can observe only those X_i in (0, T₀), an estimate of N based on the J observations obtained has a variance which explodes as θ→θC. Using θC based on the observations in (0, T₀) T is computed and all X_i in (0, ) are observed. An estimate of N based on all observations in (0, ) has a bounded variance where the bound can be adjusted by proper choice of .     

A NOTE ON CONFIDENCE BOUNDS FOR CERTAIN RATIOS OF CHARACTERISTIC ROOTS OF COVARIANCE MATRICES

Let σ₁, …, σ_k be the covariance matrices of k p -variate normal populations. Let Λ_ij be the j th largest characteristic root of σ_i (j=1, …, p; i=1, …, k). In this note we obtain simultaneous confidence intervals on (i)Λ_{i+1, p}/Λ_ipand by using methods similar to those of Khatri (1965).     

A NOTE ON THE MULTIVARIATE LINEAR MODEL WITH CONSTRAINTS ON THE DEPENDENT VECTOR

It is shown that the least squares estimators of B and Σ in the multivariate linear model {E Y _i= X ₁ B , D ( Y _i) =Σ, 1 ≤ i ≤ n , Y ₁ Y _n uncorrelated} subject to the constraints Y _i M = X _i N are just the usual least squares estimators = ( X'X )-¹ X'Y and ΣC = 1/n( Y-X )( Y-X ) in the unconstrained model where Σ has full rank. Tests of hypotheses concerning B are discussed for situations in which each Y _i has a multivariate normal distribution, and examples of the applicability of the model reviewed.     

Set Compound Estimation Under Entropy Loss

Set compound estimation has been studied for nearly half a century. This paper explores, for the first time, set compound estimation under entropy (Kullback-Leibler information) loss for a k -dimensional standard exponential family with a compact parameter space. It makes detailed investigation of entropy loss with the exponential family and related proper-ties. Asymptotically optimal set compound estimators with rates O ( n ^−1/2) under this loss are established for some discrete exponential families by using power series, representing the Bayes estimators in terms of a mixture density and applying the Singh-Datta Lemma. Poisson, negative binomial families and a two-dimensional model serve as examples.     

Estimation of Diffusion Processes by Simulated Moment Methods

We consider the parameter estimation of a diffusion process and we suppose that the trend and the diffusion coefficient depend on the parameter θ. The process is observed at time ( t_i ) _{i =0,..., n} with Δ = t_{i +1}− t_i fixed and we propose here to estimate θ from simulated moment methods.     

Empirical Likelihood for Non-Smooth Criterion Functions

Abstract.  Suppose that X ₁ ,…,  X _n is a sequence of independent random vectors, identically distributed as a d -dimensional random vector X . Let     be a parameter of interest and     be some nuisance parameter. The unknown, true parameters ( μ ₀ , ν ₀ ) are uniquely determined by the system of equations E { g ( X , μ ₀ , ν ₀ )} =   0 , where g  =  ( g ₁ ,…, g _{p + q} ) is a vector of p + q functions. In this paper we develop an empirical likelihood (EL) method to do inference for the parameter μ ₀ . The results in this paper are valid under very mild conditions on the vector of criterion functions g . In particular, we do not require that g ₁ ,…, g _{p + q} are smooth in μ or ν . This offers the advantage that the criterion function may involve indicators, which are encountered when considering, e.g. differences of quantiles, copulas, ROC curves, to mention just a few examples. We prove the asymptotic limit of the empirical log-likelihood ratio, and carry out a small simulation study to test the performance of the proposed EL method for small samples.     

Nonparametric maximum likelihood estimation of population size based on the counting distribution

Summary.  The paper discusses the estimation of an unknown population size n . Suppose that an identification mechanism can identify n _obs cases. The Horvitz–Thompson estimator of n adjusts this number by the inverse of 1− p ₀, where the latter is the probability of not identifying a case. When repeated counts of identifying the same case are available, we can use the counting distribution for estimating p ₀ to solve the problem. Frequently, the Poisson distribution is used and, more recently, mixtures of Poisson distributions. Maximum likelihood estimation is discussed by means of the EM algorithm. For truncated Poisson mixtures, a nested EM algorithm is suggested and illustrated for several application cases. The algorithmic principles are used to show an inequality, stating that the Horvitz–Thompson estimator of n by using the mixed Poisson model is always at least as large as the estimator by using a homogeneous Poisson model. In turn, if the homogeneous Poisson model is misspecified it will, potentially strongly, underestimate the true population size. Examples from various areas illustrate this finding.     

DISTRIBUTION OF THE NUMBER OF MARKOVIAN RENEWALS IN AN ARBITRARY INTERVAL

A Markov Renewal Process (M.R.P.) is one which records at each time t , the number of times a system visits each of m states in time t , if the transitions from state to state are according to a Markov chain and if the time required for each successive move is a random variable whose distribution function (d.f.) depends on the two states between which the move is made. In this paper, the distribution of the number of times each state is visited in an arbitrary interval (t₀, t₀+t) is derived. Asymptotic expressions for the mean and variance of this distribution are also obtained.     

A Limit Theorem for Solutions of Inequalities

Let H ( p ) be the set { x ∈ X : h ( x ) ≤ p } where h is a real-valued lower semicontinuous function on a locally compact separable metric space X . This paper presents a general limit theorem for the sequence of random sets H _n ( p ) = { x ∈ X : h _n ( x ) ≤ p } n ≥ 1, where h _n , n ≥ 1, are functions that estimate h     

Estimating the proportion of true null hypotheses, with application to DNA microarray data

Summary.  We consider the problem of estimating the proportion of true null hypotheses, π ₀, in a multiple-hypothesis set-up. The tests are based on observed p -values. We first review published estimators based on the estimator that was suggested by Schweder and Spjøtvoll. Then we derive new estimators based on nonparametric maximum likelihood estimation of the p -value density, restricting to decreasing and convex decreasing densities. The estimators of π ₀ are all derived under the assumption of independent test statistics. Their performance under dependence is investigated in a simulation study. We find that the estimators are relatively robust with respect to the assumption of independence and work well also for test statistics with moderate dependence.     

Estimating smooth monotone functions

Many situations call for a smooth strictly monotone function f of arbitrary flexibility. The family of functions defined by the differential equation D  ² f  = w Df , where w is an unconstrained coefficient function comprises the strictly monotone twice differentiable functions. The solution to this equation is f = C ₀ + C ₁  D ⁻¹{exp( D ⁻¹ w )}, where C ₀ and C ₁ are arbitrary constants and D ⁻¹ is the partial integration operator. A basis for expanding w is suggested that permits explicit integration in the expression of f . In fitting data, it is also useful to regularize f by penalizing the integral of w ² since this is a measure of the relative curvature in f . Applications are discussed to monotone nonparametric regression, to the transformation of the dependent variable in non-linear regression and to density estimation.     

Likelihood Ratio Tests Under Local and Fixed Alternatives in Monotone Function Problems

Abstract.  We focus on a class of non-standard problems involving non-parametric estimation of a monotone function that is characterized by n ^1/3 rate of convergence of the maximum likelihood estimator, non-Gaussian limit distributions and the non-existence of     -regular estimators. We have shown elsewhere that under a null hypothesis of the type ψ ( z ₀) =  θ ₀ ( ψ being the monotone function of interest) in non-standard problems of the above kind, the likelihood ratio statistic has a 'universal' limit distribution that is free of the underlying parameters in the model. In this paper, we illustrate its limiting behaviour under local alternatives of the form ψ _n ( z ), where ψ _n (·) and ψ (·) vary in O ( n ^−1/3) neighbourhoods around z ₀ and ψ _n converges to ψ at rate n ^1/3 in an appropriate metric. Apart from local alternatives, we also consider the behaviour of the likelihood ratio statistic under fixed alternatives and establish the convergence in probability of an appropriately scaled version of the same to a constant involving a Kullback–Leibler distance.