Empirical Likelihood for Non-Smooth Criterion Functions

Abstract.  Suppose that X ₁ ,…,  X _n is a sequence of independent random vectors, identically distributed as a d -dimensional random vector X . Let     be a parameter of interest and     be some nuisance parameter. The unknown, true parameters ( μ ₀ , ν ₀ ) are uniquely determined by the system of equations E { g ( X , μ ₀ , ν ₀ )} =   0 , where g  =  ( g ₁ ,…, g _{p + q} ) is a vector of p + q functions. In this paper we develop an empirical likelihood (EL) method to do inference for the parameter μ ₀ . The results in this paper are valid under very mild conditions on the vector of criterion functions g . In particular, we do not require that g ₁ ,…, g _{p + q} are smooth in μ or ν . This offers the advantage that the criterion function may involve indicators, which are encountered when considering, e.g. differences of quantiles, copulas, ROC curves, to mention just a few examples. We prove the asymptotic limit of the empirical log-likelihood ratio, and carry out a small simulation study to test the performance of the proposed EL method for small samples.     

Nonparametric multistep-ahead prediction in time series analysis

Summary.  We consider the problem of multistep-ahead prediction in time series analysis by using nonparametric smoothing techniques. Forecasting is always one of the main objectives in time series analysis. Research has shown that non-linear time series models have certain advantages in multistep-ahead forecasting. Traditionally, nonparametric k -step-ahead least squares prediction for non-linear autoregressive AR( d ) models is done by estimating E ( X _{t + k}  | X _t , …,  X _{t − d +1}) via nonparametric smoothing of X _{t + k} on ( X _t , …,  X _{t − d +1}) directly. We propose a multistage nonparametric predictor. We show that the new predictor has smaller asymptotic mean-squared error than the direct smoother, though the convergence rate is the same. Hence, the predictor proposed is more efficient. Some simulation results, advice for practical bandwidth selection and a real data example are provided.     

A NOTE ON CONFIDENCE BOUNDS FOR CERTAIN RATIOS OF CHARACTERISTIC ROOTS OF COVARIANCE MATRICES

Let σ₁, …, σ_k be the covariance matrices of k p -variate normal populations. Let Λ_ij be the j th largest characteristic root of σ_i (j=1, …, p; i=1, …, k). In this note we obtain simultaneous confidence intervals on (i)Λ_{i+1, p}/Λ_ipand by using methods similar to those of Khatri (1965).     

Non-parametric Regression with Dependent Censored Data

Abstract.  Let ( X _i , Y _i ) ( i = 1 ,…, n ) be n replications of a random vector ( X , Y  ), where Y is supposed to be subject to random right censoring. The data ( X _i , Y _i ) are assumed to come from a stationary α -mixing process. We consider the problem of estimating the function m ( x ) = E ( φ ( Y ) |  X = x ), for some known transformation φ . This problem is approached in the following way: first, we introduce a transformed variable     , that is not subject to censoring and satisfies the relation     , and then we estimate m ( x ) by applying local linear regression techniques. As a by-product, we obtain a general result on the uniform rate of convergence of kernel type estimators of functionals of an unknown distribution function, under strong mixing assumptions.     

Bayesian model selection for join point regression with application to age-adjusted cancer rates

Summary.  The method of Bayesian model selection for join point regression models is developed. Given a set of K +1 join point models M ₀,  M ₁, …,  M _K with 0, 1, …,  K join points respec-tively, the posterior distributions of the parameters and competing models M _k are computed by Markov chain Monte Carlo simulations. The Bayes information criterion BIC is used to select the model M _k with the smallest value of BIC as the best model. Another approach based on the Bayes factor selects the model M _k with the largest posterior probability as the best model when the prior distribution of M _k is discrete uniform. Both methods are applied to analyse the observed US cancer incidence rates for some selected cancer sites. The graphs of the join point models fitted to the data are produced by using the methods proposed and compared with the method of Kim and co-workers that is based on a series of permutation tests. The analyses show that the Bayes factor is sensitive to the prior specification of the variance σ ², and that the model which is selected by BIC fits the data as well as the model that is selected by the permutation test and has the advantage of producing the posterior distribution for the join points. The Bayesian join point model and model selection method that are presented here will be integrated in the National Cancer Institute's join point software ( http://www.srab.cancer.gov/joinpoint/ ) and will be available to the public.     

An Analysis of Swendsen–Wang and Related Sampling Methods

Convergence rates, statistical efficiency and sampling costs are studied for the original and extended Swendsen–Wang methods of generating a sample path { S _j , j ≥1} with equilibrium distribution π , with r distinct elements, on a finite state space X of size N ₁. Given S _{j -1}, each method uses auxiliary random variables to identify the subset of X from which S _j is to be randomly sampled. Let π_min and π_max denote respectively the smallest and largest elements in π and let N_r denote the number of elements in π with value π_max. For a single auxiliary variable, uniform sampling from the subset and ( N ₁− N_r )π_min+ N_r π_max≈1, our results show rapid convergence and high statistical efficiency for large π_min/π_max or N_r / N ₁ and slow convergence and poor statistical efficiency for small π_min/π_max and N_r / N₁ . Other examples provide additional insight. For extended Swendsen–Wang methods with non-uniform subset sampling, the analysis identifies the properties of a decomposition of π( x ) that favour fast convergence and high statistical efficiency. In the absence of exploitable special structure, subset sampling can be costly regardless of which of these methods is employed.     

Rejoinder to 'Ahmed, M.S. (1998). A note on regression-type estimators using multiple auxiliary information.'

In the estimators t ₃ , t ₄ , t ₅ of Mukerjee, Rao & Vijayan (1987), b _{y x} and b _{y z} are partial regression coefficients of y on x and z , respectively, based on the smaller sample. With the above interpretation of b _{y x} and b _{y z} in t ₃ , t ₄ , t ₅ , all the calculations in Mukerjee at al. (1987) are correct. In this connection, we also wish to make it explicit that b _{x z} in t ₅ is an ordinary and not a partial regression coefficient. The 'corrected' MSEs of t ₃ , t ₄ , t ₅ , as given in Ahmed (1998 Section 3) are computed assuming that our b _{y x} and b _{y z} are ordinary and not partial regression coefficients. Indeed, we had no intention of giving estimators using the corresponding ordinary regression coefficients which would lead to estimators inferior to those given by Kiregyera (1984). We accept responsibility for any notational confusion created by us and express regret to readers who have been confused by our notation. Finally, in consideration of the above, it may be noted that Tripathi & Ahmed's (1995) estimator t ₀ , quoted also in Ahmed (1998), is no better than t ₅ of Mukerjee at al. (1987).     

Exact Slopes of Test Statistics for the Multivariate Exponential Family

The objective of this paper is to investigate exact slopes of test statistics { T_n } when the random vectors X ₁, ..., X_n are distributed according to an unknown member of an exponential family { P _θ; θ∈Ω. Here Ω is a parameter set. We will be concerned with the hypothesis testing problem of H ₀θ∈Ω₀ vs H ₁: θ∉Ω₀ where Ω₀ is a subset of Ω. It will be shown that for an important class of problems and test statistics the exact slope of { T_n } at η in Ω−Ω₀ is determined by the shortest Kullback–Leibler distance from {θ: T_n (λ(θ)) = T_n (λ(π))} to Ω₀, λ_θ = E _θ)( X ).     

On a Posterior Predictive Density Sample Size Criterion

Abstract.  Let Ω be a space of densities with respect to some σ -finite measure μ and let Π be a prior distribution having support Ω with respect to some suitable topology. Conditional on f , let X ⁿ  = ( X ₁ ,…, X _n ) be an independent and identically distributed sample of size n from f . This paper introduces a Bayesian non-parametric criterion for sample size determination which is based on the integrated squared distance between posterior predictive densities. An expression for the sample size is obtained when the prior is a Dirichlet mixture of normal densities.     

Computation of the Generalized F Distribution

Exact expressions for the cumulative distribution function of a random variable of the form ( α ₁ X ₁+ α ₂ X ₂)/ Y are given where X ₁, X ₂ and Y are independent chi-squared random variables. The expressions are applied to the detection of joint outliers and Hotelling's mis-specified T ² distribution.     

Ordering of Sequentially Sampled Exponential Experiments

Let X ₁, X ₂, ... be a sequence of i.i.d. random variables, X _i∼ F _θ, θ∈Θ. Let N ₁ and N ₂ be two stopping rules. For a class of exponential families { F _θ: θ∈Θ} we show that the experiment Y ₁ = ( X ₁, ..., X _N1) carries more statistical information than Y ₂ = ( X ₁, ..., x _N2) only if N ₁ is stochastically larger then N ₂     

A TWO STAGE PROCEDURE FOR ESTIMATING THE SIZE OF A TRUNCATED EXPONENTIAL SAMPLE

Let X₁, …, X_N be i.i.d. exponential random variables with unknown scale parameter θ. If one can observe only those X_i in (0, T₀), an estimate of N based on the J observations obtained has a variance which explodes as θ→θC. Using θC based on the observations in (0, T₀) T is computed and all X_i in (0, ) are observed. An estimate of N based on all observations in (0, ) has a bounded variance where the bound can be adjusted by proper choice of .     

Semiparametric Likelihood Based Method for Goodness of Fit Tests and Estimation in Upgraded Mixture Models

We use Owen's (1988, 1990) empirical likelihood method in upgraded mixture models. Two groups of independent observations are available. One is z ₁, ..., z _n which is observed directly from a distribution F ( z ). The other one is x ₁, ..., x _m which is observed indirectly from F ( z ), where the x _i s have density ∫ p ( x | z ) dF ( z ) and p ( x | z ) is a conditional density function. We are interested in testing H ₀: p ( x | z ) = p ( x | z ; θ ), for some specified smooth density function. A semiparametric likelihood ratio based statistic is proposed and it is shown that it converges to a chi-squared distribution. This is a simple method for doing goodness of fit tests, especially when x is a discrete variable with finitely many values. In addition, we discuss estimation of θ and F ( z ) when H ₀ is true. The connection between upgraded mixture models and general estimating equations is pointed out.     

Modelling and smoothing parameter estimation with multiple quadratic penalties

Penalized likelihood methods provide a range of practical modelling tools, including spline smoothing, generalized additive models and variants of ridge regression. Selecting the correct weights for penalties is a critical part of using these methods and in the single-penalty case the analyst has several well-founded techniques to choose from. However, many modelling problems suggest a formulation employing multiple penalties, and here general methodology is lacking. A wide family of models with multiple penalties can be fitted to data by iterative solution of the generalized ridge regression problem minimize || W ^1/2 ( Xp − y ) ||²ρ+Σ _{i =1} ^m  θ _i p ' S _i p ( p is a parameter vector, X a design matrix, S _i a non-negative definite coefficient matrix defining the i th penalty with associated smoothing parameter θ _i , W a diagonal weight matrix, y a vector of data or pseudodata and ρ an 'overall' smoothing parameter included for computational efficiency). This paper shows how smoothing parameter selection can be performed efficiently by applying generalized cross-validation to this problem and how this allows non-linear, generalized linear and linear models to be fitted using multiple penalties, substantially increasing the scope of penalized modelling methods. Examples of non-linear modelling, generalized additive modelling and anisotropic smoothing are given.     

A Generalization of the Childs-Moran Result for Orthoscheme Probabilities

van der Vaart (1953, 1955) introduced the orthoscheme probability R_n (c ₁,..., c_n−1 ), meaning the orthant probability of an n -dimensional normal random vector with zero mean and tridiagonal correlation matrix with elements c ₁,..., c_n−1 on the upper diagonal. Childs (1967) conjectured and Moran (1983) proved that the generating function of { R_n (½,...,½)} equals tan z + sin z . This paper derives the generating function of { R_n (τ,½,...,½)}.     

Characterizations of Competing Risks in Terms of Independent-Risks Proxy Models

In competing risks a failure time T and a cause C , one of p possible, are observed. A traditional representation is via a vector ( T ₁, ..., T_p ) of latent failure times such that T = min( T ₁, ..., T_p ); C is defined by T = T_C in the basic situation of failure from a single cause. There are several results in the literature to the effect that a joint distribution for ( T ₁, ..., T_p ), in which the T_j are independent, can always be constructed to yield any given bivariate distribution for ( C , T ). For this reason the prevailing wisdom is that independence cannot be assessed from competing risks data, not even with arbitrarily large sample sizes (e.g. Prentice et al. , 1978). A result was given by Crowder (1996) which shows that, under certain circumstances, independence can be assessed. The various results will be drawn together and a complete characterization can now be given in terms of independent-risks proxy models.     

Penalized Projection Estimator for Volatility Density

Abstract.  In this paper, we consider a stochastic volatility model ( Y _t , V _t ), where the volatility (V _t ) is a positive stationary Markov process. We assume that ( ln V _t ) admits a stationary density f that we want to estimate. Only the price process Y _t is observed at n discrete times with regular sampling interval Δ . We propose a non-parametric estimator for f obtained by a penalized projection method. Under mixing assumptions on ( V _t ), we derive bounds for the quadratic risk of the estimator. Assuming that Δ=Δ _n tends to 0 while the number of observations and the length of the observation time tend to infinity, we discuss the rate of convergence of the risk. Examples of models included in this framework are given.     

On the effect of overdispersion on exact conditional tests

Let Y ₁, . . ., Y_n denote independent random variables such that Y_j has a one-parameter exponential family distribution with canonical parameter θ _j =λ+ψ X_j ; here X ₁, . . ., X_n are known constants. Consider a test of the null hypothesis ψ=0. Under the null hypothesis, A =Σ Y_j is sufficient for λ and, hence, a test of ψ=0 may be based on the conditional distribution of T =Σ X_j Y_j given A , which is independent of λ. In this paper, the effects of overdispersion due to a mixture model on the conditional distribution of T given A are considered.     

A NOTE ON THE MULTIVARIATE LINEAR MODEL WITH CONSTRAINTS ON THE DEPENDENT VECTOR

It is shown that the least squares estimators of B and Σ in the multivariate linear model {E Y _i= X ₁ B , D ( Y _i) =Σ, 1 ≤ i ≤ n , Y ₁ Y _n uncorrelated} subject to the constraints Y _i M = X _i N are just the usual least squares estimators = ( X'X )-¹ X'Y and ΣC = 1/n( Y-X )( Y-X ) in the unconstrained model where Σ has full rank. Tests of hypotheses concerning B are discussed for situations in which each Y _i has a multivariate normal distribution, and examples of the applicability of the model reviewed.     

Goodness-of-fit Tests for Semi-Markov and Markov Survival Models with One Intermediate State

Survival data with one intermediate state are described by semi-Markov and Markov models for counting processes whose intensities are defined in terms of two stopping times T ₁< T ₂. Problems of goodness-of-fit for these models are studied. The test statistics are proposed by comparing Nelson–Aalen estimators for data stratified according to T ₁. Asymptotic distributions of these statistics are established in terms of the weak convergence of some random fields. Asymptotic consistency of these test statistics is also established. Simulation studies are included to indicate their numerical performance.