Semiparametric Likelihood Based Method for Goodness of Fit Tests and Estimation in Upgraded Mixture Models

We use Owen's (1988, 1990) empirical likelihood method in upgraded mixture models. Two groups of independent observations are available. One is z ₁, ..., z _n which is observed directly from a distribution F ( z ). The other one is x ₁, ..., x _m which is observed indirectly from F ( z ), where the x _i s have density ∫ p ( x | z ) dF ( z ) and p ( x | z ) is a conditional density function. We are interested in testing H ₀: p ( x | z ) = p ( x | z ; θ ), for some specified smooth density function. A semiparametric likelihood ratio based statistic is proposed and it is shown that it converges to a chi-squared distribution. This is a simple method for doing goodness of fit tests, especially when x is a discrete variable with finitely many values. In addition, we discuss estimation of θ and F ( z ) when H ₀ is true. The connection between upgraded mixture models and general estimating equations is pointed out.     

Exact Slopes of Test Statistics for the Multivariate Exponential Family

The objective of this paper is to investigate exact slopes of test statistics { T_n } when the random vectors X ₁, ..., X_n are distributed according to an unknown member of an exponential family { P _θ; θ∈Ω. Here Ω is a parameter set. We will be concerned with the hypothesis testing problem of H ₀θ∈Ω₀ vs H ₁: θ∉Ω₀ where Ω₀ is a subset of Ω. It will be shown that for an important class of problems and test statistics the exact slope of { T_n } at η in Ω−Ω₀ is determined by the shortest Kullback–Leibler distance from {θ: T_n (λ(θ)) = T_n (λ(π))} to Ω₀, λ_θ = E _θ)( X ).     

Simple and Explicit Estimating Functions for a Discretely Observed Diffusion Process

We consider a one-dimensional diffusion process X , with ergodic property, with drift b ( x , θ) and diffusion coefficient a ( x , θ) depending on an unknown parameter θ that may be multidimensional. We are interested in the estimation of θ and dispose, for that purpose, of a discretized trajectory, observed at n equidistant times t_i = iΔ , i = 0, ..., n . We study a particular class of estimating functions of the form ∑ f (θ, X _{t i −1}) which, under the assumption that the integral of f with respect to the invariant measure is null, provide us with a consistent and asymptotically normal estimator. We determine the choice of f that yields the estimator with minimum asymptotic variance within the class and indicate how to construct explicit estimating functions based on the generator of the diffusion. Finally the theoretical study is completed with simulations.     

Ordering of Sequentially Sampled Exponential Experiments

Let X ₁, X ₂, ... be a sequence of i.i.d. random variables, X _i∼ F _θ, θ∈Θ. Let N ₁ and N ₂ be two stopping rules. For a class of exponential families { F _θ: θ∈Θ} we show that the experiment Y ₁ = ( X ₁, ..., X _N1) carries more statistical information than Y ₂ = ( X ₁, ..., x _N2) only if N ₁ is stochastically larger then N ₂     

A NORMAL APPROXIMATION TO THE FISHER DISTRIBUTION

A simple normal approximation is given for the joint probability density function of the polar co-ordinates (θ, ψ) of a random vector following the Fisher distribution with arbitrary mean direction (θ₀, ψ₀). The approximation leads to simple inference procedures which are particularly useful in regression models. Conditions for the adequacy of the approximation are investigated and summarized in tabular form.     

A Limit Theorem for Solutions of Inequalities

Let H ( p ) be the set { x ∈ X : h ( x ) ≤ p } where h is a real-valued lower semicontinuous function on a locally compact separable metric space X . This paper presents a general limit theorem for the sequence of random sets H _n ( p ) = { x ∈ X : h _n ( x ) ≤ p } n ≥ 1, where h _n , n ≥ 1, are functions that estimate h     

Modelling and smoothing parameter estimation with multiple quadratic penalties

Penalized likelihood methods provide a range of practical modelling tools, including spline smoothing, generalized additive models and variants of ridge regression. Selecting the correct weights for penalties is a critical part of using these methods and in the single-penalty case the analyst has several well-founded techniques to choose from. However, many modelling problems suggest a formulation employing multiple penalties, and here general methodology is lacking. A wide family of models with multiple penalties can be fitted to data by iterative solution of the generalized ridge regression problem minimize || W ^1/2 ( Xp − y ) ||²ρ+Σ _{i =1} ^m  θ _i p ' S _i p ( p is a parameter vector, X a design matrix, S _i a non-negative definite coefficient matrix defining the i th penalty with associated smoothing parameter θ _i , W a diagonal weight matrix, y a vector of data or pseudodata and ρ an 'overall' smoothing parameter included for computational efficiency). This paper shows how smoothing parameter selection can be performed efficiently by applying generalized cross-validation to this problem and how this allows non-linear, generalized linear and linear models to be fitted using multiple penalties, substantially increasing the scope of penalized modelling methods. Examples of non-linear modelling, generalized additive modelling and anisotropic smoothing are given.     

On Minimum Distance Estimation Using Kolmogorov-Lévy Type Metrics

Let X ₁, . . ., X_n be independent identically distributed random variables with a common continuous (cumulative) distribution function (d.f.) F , and F^_n the empirical d.f. (e.d.f.) based on X ₁, . . ., X_n . Let G be a smooth d.f. and Gθ = G (·–θ) its translation through θ∈ R . Using a Kolmogorov-Lévy type metric ρ_α defined on the space of d.f.s. on R , the paper derives both null and non-null limiting distributions of √ n [ ρ_α ( F_n , Gθ_n ) – ρ_α ( F, G_θ )], √ n (θ _n –θ) and √ nρ_α ( Gθ , Gθ ), where θ _n and θ are the minimum ρ_α -distance parameters for F_n and F from G , respectively. These distributions are known explicitly in important particular cases; with some complementary Monte Carlo simulations, they help us clarify our understanding of estimation using minimum distance methods and supremum type metrics. We advocate use of the minimum distance method with supremum type metrics in cases of non-null models. The resulting functionals are Hadamard differentiable and efficient. For small scale parameters the minimum distance functionals are close to medians of the parent distributions. The optimal small scale models result in minimum distance estimators having asymptotic variances very competitive and comparable with best known robust estimators.     

A TWO STAGE PROCEDURE FOR ESTIMATING THE SIZE OF A TRUNCATED EXPONENTIAL SAMPLE

Let X₁, …, X_N be i.i.d. exponential random variables with unknown scale parameter θ. If one can observe only those X_i in (0, T₀), an estimate of N based on the J observations obtained has a variance which explodes as θ→θC. Using θC based on the observations in (0, T₀) T is computed and all X_i in (0, ) are observed. An estimate of N based on all observations in (0, ) has a bounded variance where the bound can be adjusted by proper choice of .     

IMPROVING UPON THE BEST INVARIANT ESTIMATOR IN MULTIVARIATE LOCATION PROBLEMS

We are concerned with estimators which improve upon the best invariant estimator, in estimating a location parameter θ. If the loss function is L(θ - a) with L convex, we give sufficient conditions for the inadmissibility of δ₀(X) = X. If the loss is a weighted sum of squared errors, we find various classes of estimators δ which are better than δ₀. In general, δ is the convolution of δ₁ (an estimator which improves upon δ₀ outside of a compact set) with a suitable probability density in R^p. The critical dimension of inadmissibility depends on the estimator δ₁ We also give several examples of estimators δ obtained in this way and state some open problems.     

Likelihood Ratio Tests Under Local and Fixed Alternatives in Monotone Function Problems

Abstract.  We focus on a class of non-standard problems involving non-parametric estimation of a monotone function that is characterized by n ^1/3 rate of convergence of the maximum likelihood estimator, non-Gaussian limit distributions and the non-existence of     -regular estimators. We have shown elsewhere that under a null hypothesis of the type ψ ( z ₀) =  θ ₀ ( ψ being the monotone function of interest) in non-standard problems of the above kind, the likelihood ratio statistic has a 'universal' limit distribution that is free of the underlying parameters in the model. In this paper, we illustrate its limiting behaviour under local alternatives of the form ψ _n ( z ), where ψ _n (·) and ψ (·) vary in O ( n ^−1/3) neighbourhoods around z ₀ and ψ _n converges to ψ at rate n ^1/3 in an appropriate metric. Apart from local alternatives, we also consider the behaviour of the likelihood ratio statistic under fixed alternatives and establish the convergence in probability of an appropriately scaled version of the same to a constant involving a Kullback–Leibler distance.     

Estimation of an Ergodic Diffusion from Discrete Observations

We consider a one-dimensional diffusion process X , with ergodic property, with drift b ( x , θ) and diffusion coefficient a ( x , σ) depending on unknown parameters θ and σ. We are interested in the joint estimation of (θ, σ). For that purpose, we dispose of a discretized trajectory, observed at n equidistant times tⁿ_i = ih_n , 1 ≤ i ≤ n . We assume that h_n ← 0 and nh_n ←∞. Under the condition nh_n^p ← 0 for an arbitrary integer p , we exhibit a contrast dependent on p which provides us with an asymptotically normal and efficient estimator of (θ, σ).     

Computation of the Generalized F Distribution

Exact expressions for the cumulative distribution function of a random variable of the form ( α ₁ X ₁+ α ₂ X ₂)/ Y are given where X ₁, X ₂ and Y are independent chi-squared random variables. The expressions are applied to the detection of joint outliers and Hotelling's mis-specified T ² distribution.     

Estimation of Diffusion Processes by Simulated Moment Methods

We consider the parameter estimation of a diffusion process and we suppose that the trend and the diffusion coefficient depend on the parameter θ. The process is observed at time ( t_i ) _{i =0,..., n} with Δ = t_{i +1}− t_i fixed and we propose here to estimate θ from simulated moment methods.     

On the effect of overdispersion on exact conditional tests

Let Y ₁, . . ., Y_n denote independent random variables such that Y_j has a one-parameter exponential family distribution with canonical parameter θ _j =λ+ψ X_j ; here X ₁, . . ., X_n are known constants. Consider a test of the null hypothesis ψ=0. Under the null hypothesis, A =Σ Y_j is sufficient for λ and, hence, a test of ψ=0 may be based on the conditional distribution of T =Σ X_j Y_j given A , which is independent of λ. In this paper, the effects of overdispersion due to a mixture model on the conditional distribution of T given A are considered.     

Empirical Likelihood for Non-Smooth Criterion Functions

Abstract.  Suppose that X ₁ ,…,  X _n is a sequence of independent random vectors, identically distributed as a d -dimensional random vector X . Let     be a parameter of interest and     be some nuisance parameter. The unknown, true parameters ( μ ₀ , ν ₀ ) are uniquely determined by the system of equations E { g ( X , μ ₀ , ν ₀ )} =   0 , where g  =  ( g ₁ ,…, g _{p + q} ) is a vector of p + q functions. In this paper we develop an empirical likelihood (EL) method to do inference for the parameter μ ₀ . The results in this paper are valid under very mild conditions on the vector of criterion functions g . In particular, we do not require that g ₁ ,…, g _{p + q} are smooth in μ or ν . This offers the advantage that the criterion function may involve indicators, which are encountered when considering, e.g. differences of quantiles, copulas, ROC curves, to mention just a few examples. We prove the asymptotic limit of the empirical log-likelihood ratio, and carry out a small simulation study to test the performance of the proposed EL method for small samples.     

A New Kind of Asymptotic Quasi-Score Estimating Function

A new definition of asymptotic quasi-score sequence of estimating functions is given and studied. The relationship between asymptotic quasi-likelihood and quasi-likelihood estimates is investigated. A new practical approach for obtaining a good estimate of θ in the model y _t = f_t (θ) + m_t without any prior knowledge on the nature of E ( m ² _t |F _{t −1}) is suggested, where f_t is a predictable process and m_t is a martingale difference process. Two examples are used to show that the approach is practicable.     

Estimation and testing stationarity for double-autoregressive models

Summary.  The paper considers the double-autoregressive model y _t  =  φ y _{t −1}+ ɛ _t with ɛ _t  =     . Consistency and asymptotic normality of the estimated parameters are proved under the condition E  ln | φ  +√ α η _t |<0, which includes the cases with | φ |=1 or | φ |>1 as well as     . It is well known that all kinds of estimators of φ in these cases are not normal when ɛ _t are independent and identically distributed. Our result is novel and surprising. Two tests are proposed for testing stationarity of the model and their asymptotic distributions are shown to be a function of bivariate Brownian motions. Critical values of the tests are tabulated and some simulation results are reported. An application to the US 90-day treasury bill rate series is given.     

Nonparametric multistep-ahead prediction in time series analysis

Summary.  We consider the problem of multistep-ahead prediction in time series analysis by using nonparametric smoothing techniques. Forecasting is always one of the main objectives in time series analysis. Research has shown that non-linear time series models have certain advantages in multistep-ahead forecasting. Traditionally, nonparametric k -step-ahead least squares prediction for non-linear autoregressive AR( d ) models is done by estimating E ( X _{t + k}  | X _t , …,  X _{t − d +1}) via nonparametric smoothing of X _{t + k} on ( X _t , …,  X _{t − d +1}) directly. We propose a multistage nonparametric predictor. We show that the new predictor has smaller asymptotic mean-squared error than the direct smoother, though the convergence rate is the same. Hence, the predictor proposed is more efficient. Some simulation results, advice for practical bandwidth selection and a real data example are provided.     

A NOTE ON THE MULTIVARIATE LINEAR MODEL WITH CONSTRAINTS ON THE DEPENDENT VECTOR

It is shown that the least squares estimators of B and Σ in the multivariate linear model {E Y _i= X ₁ B , D ( Y _i) =Σ, 1 ≤ i ≤ n , Y ₁ Y _n uncorrelated} subject to the constraints Y _i M = X _i N are just the usual least squares estimators = ( X'X )-¹ X'Y and ΣC = 1/n( Y-X )( Y-X ) in the unconstrained model where Σ has full rank. Tests of hypotheses concerning B are discussed for situations in which each Y _i has a multivariate normal distribution, and examples of the applicability of the model reviewed.