Nonparametric multistep-ahead prediction in time series analysis

Summary.  We consider the problem of multistep-ahead prediction in time series analysis by using nonparametric smoothing techniques. Forecasting is always one of the main objectives in time series analysis. Research has shown that non-linear time series models have certain advantages in multistep-ahead forecasting. Traditionally, nonparametric k -step-ahead least squares prediction for non-linear autoregressive AR( d ) models is done by estimating E ( X _{t + k}  | X _t , …,  X _{t − d +1}) via nonparametric smoothing of X _{t + k} on ( X _t , …,  X _{t − d +1}) directly. We propose a multistage nonparametric predictor. We show that the new predictor has smaller asymptotic mean-squared error than the direct smoother, though the convergence rate is the same. Hence, the predictor proposed is more efficient. Some simulation results, advice for practical bandwidth selection and a real data example are provided.     

Empirical Likelihood for Non-Smooth Criterion Functions

Abstract.  Suppose that X ₁ ,…,  X _n is a sequence of independent random vectors, identically distributed as a d -dimensional random vector X . Let     be a parameter of interest and     be some nuisance parameter. The unknown, true parameters ( μ ₀ , ν ₀ ) are uniquely determined by the system of equations E { g ( X , μ ₀ , ν ₀ )} =   0 , where g  =  ( g ₁ ,…, g _{p + q} ) is a vector of p + q functions. In this paper we develop an empirical likelihood (EL) method to do inference for the parameter μ ₀ . The results in this paper are valid under very mild conditions on the vector of criterion functions g . In particular, we do not require that g ₁ ,…, g _{p + q} are smooth in μ or ν . This offers the advantage that the criterion function may involve indicators, which are encountered when considering, e.g. differences of quantiles, copulas, ROC curves, to mention just a few examples. We prove the asymptotic limit of the empirical log-likelihood ratio, and carry out a small simulation study to test the performance of the proposed EL method for small samples.     

Non-parametric Regression with Dependent Censored Data

Abstract.  Let ( X _i , Y _i ) ( i = 1 ,…, n ) be n replications of a random vector ( X , Y  ), where Y is supposed to be subject to random right censoring. The data ( X _i , Y _i ) are assumed to come from a stationary α -mixing process. We consider the problem of estimating the function m ( x ) = E ( φ ( Y ) |  X = x ), for some known transformation φ . This problem is approached in the following way: first, we introduce a transformed variable     , that is not subject to censoring and satisfies the relation     , and then we estimate m ( x ) by applying local linear regression techniques. As a by-product, we obtain a general result on the uniform rate of convergence of kernel type estimators of functionals of an unknown distribution function, under strong mixing assumptions.     

Detecting changes in the mean of functional observations

Summary.  Principal component analysis has become a fundamental tool of functional data analysis. It represents the functional data as X _i ( t )= μ ( t )+Σ_{1≤ l <∞} η _{i ,  l} +  v _l ( t ), where μ is the common mean, v _l are the eigenfunctions of the covariance operator and the η _{i ,  l} are the scores. Inferential procedures assume that the mean function μ ( t ) is the same for all values of i . If, in fact, the observations do not come from one population, but rather their mean changes at some point(s), the results of principal component analysis are confounded by the change(s). It is therefore important to develop a methodology to test the assumption of a common functional mean. We develop such a test using quantities which can be readily computed in the R package fda. The null distribution of the test statistic is asymptotically pivotal with a well-known asymptotic distribution. The asymptotic test has excellent finite sample performance. Its application is illustrated on temperature data from England.     

Estimating smooth monotone functions

Many situations call for a smooth strictly monotone function f of arbitrary flexibility. The family of functions defined by the differential equation D  ² f  = w Df , where w is an unconstrained coefficient function comprises the strictly monotone twice differentiable functions. The solution to this equation is f = C ₀ + C ₁  D ⁻¹{exp( D ⁻¹ w )}, where C ₀ and C ₁ are arbitrary constants and D ⁻¹ is the partial integration operator. A basis for expanding w is suggested that permits explicit integration in the expression of f . In fitting data, it is also useful to regularize f by penalizing the integral of w ² since this is a measure of the relative curvature in f . Applications are discussed to monotone nonparametric regression, to the transformation of the dependent variable in non-linear regression and to density estimation.     

Semiparametric Likelihood Based Method for Goodness of Fit Tests and Estimation in Upgraded Mixture Models

We use Owen's (1988, 1990) empirical likelihood method in upgraded mixture models. Two groups of independent observations are available. One is z ₁, ..., z _n which is observed directly from a distribution F ( z ). The other one is x ₁, ..., x _m which is observed indirectly from F ( z ), where the x _i s have density ∫ p ( x | z ) dF ( z ) and p ( x | z ) is a conditional density function. We are interested in testing H ₀: p ( x | z ) = p ( x | z ; θ ), for some specified smooth density function. A semiparametric likelihood ratio based statistic is proposed and it is shown that it converges to a chi-squared distribution. This is a simple method for doing goodness of fit tests, especially when x is a discrete variable with finitely many values. In addition, we discuss estimation of θ and F ( z ) when H ₀ is true. The connection between upgraded mixture models and general estimating equations is pointed out.     

Penalized Projection Estimator for Volatility Density

Abstract.  In this paper, we consider a stochastic volatility model ( Y _t , V _t ), where the volatility (V _t ) is a positive stationary Markov process. We assume that ( ln V _t ) admits a stationary density f that we want to estimate. Only the price process Y _t is observed at n discrete times with regular sampling interval Δ . We propose a non-parametric estimator for f obtained by a penalized projection method. Under mixing assumptions on ( V _t ), we derive bounds for the quadratic risk of the estimator. Assuming that Δ=Δ _n tends to 0 while the number of observations and the length of the observation time tend to infinity, we discuss the rate of convergence of the risk. Examples of models included in this framework are given.     

A New Kind of Asymptotic Quasi-Score Estimating Function

A new definition of asymptotic quasi-score sequence of estimating functions is given and studied. The relationship between asymptotic quasi-likelihood and quasi-likelihood estimates is investigated. A new practical approach for obtaining a good estimate of θ in the model y _t = f_t (θ) + m_t without any prior knowledge on the nature of E ( m ² _t |F _{t −1}) is suggested, where f_t is a predictable process and m_t is a martingale difference process. Two examples are used to show that the approach is practicable.     

Rejoinder to 'Ahmed, M.S. (1998). A note on regression-type estimators using multiple auxiliary information.'

In the estimators t ₃ , t ₄ , t ₅ of Mukerjee, Rao & Vijayan (1987), b _{y x} and b _{y z} are partial regression coefficients of y on x and z , respectively, based on the smaller sample. With the above interpretation of b _{y x} and b _{y z} in t ₃ , t ₄ , t ₅ , all the calculations in Mukerjee at al. (1987) are correct. In this connection, we also wish to make it explicit that b _{x z} in t ₅ is an ordinary and not a partial regression coefficient. The 'corrected' MSEs of t ₃ , t ₄ , t ₅ , as given in Ahmed (1998 Section 3) are computed assuming that our b _{y x} and b _{y z} are ordinary and not partial regression coefficients. Indeed, we had no intention of giving estimators using the corresponding ordinary regression coefficients which would lead to estimators inferior to those given by Kiregyera (1984). We accept responsibility for any notational confusion created by us and express regret to readers who have been confused by our notation. Finally, in consideration of the above, it may be noted that Tripathi & Ahmed's (1995) estimator t ₀ , quoted also in Ahmed (1998), is no better than t ₅ of Mukerjee at al. (1987).     

Likelihood Ratio Tests Under Local and Fixed Alternatives in Monotone Function Problems

Abstract.  We focus on a class of non-standard problems involving non-parametric estimation of a monotone function that is characterized by n ^1/3 rate of convergence of the maximum likelihood estimator, non-Gaussian limit distributions and the non-existence of     -regular estimators. We have shown elsewhere that under a null hypothesis of the type ψ ( z ₀) =  θ ₀ ( ψ being the monotone function of interest) in non-standard problems of the above kind, the likelihood ratio statistic has a 'universal' limit distribution that is free of the underlying parameters in the model. In this paper, we illustrate its limiting behaviour under local alternatives of the form ψ _n ( z ), where ψ _n (·) and ψ (·) vary in O ( n ^−1/3) neighbourhoods around z ₀ and ψ _n converges to ψ at rate n ^1/3 in an appropriate metric. Apart from local alternatives, we also consider the behaviour of the likelihood ratio statistic under fixed alternatives and establish the convergence in probability of an appropriately scaled version of the same to a constant involving a Kullback–Leibler distance.     

Bayesian model selection for join point regression with application to age-adjusted cancer rates

Summary.  The method of Bayesian model selection for join point regression models is developed. Given a set of K +1 join point models M ₀,  M ₁, …,  M _K with 0, 1, …,  K join points respec-tively, the posterior distributions of the parameters and competing models M _k are computed by Markov chain Monte Carlo simulations. The Bayes information criterion BIC is used to select the model M _k with the smallest value of BIC as the best model. Another approach based on the Bayes factor selects the model M _k with the largest posterior probability as the best model when the prior distribution of M _k is discrete uniform. Both methods are applied to analyse the observed US cancer incidence rates for some selected cancer sites. The graphs of the join point models fitted to the data are produced by using the methods proposed and compared with the method of Kim and co-workers that is based on a series of permutation tests. The analyses show that the Bayes factor is sensitive to the prior specification of the variance σ ², and that the model which is selected by BIC fits the data as well as the model that is selected by the permutation test and has the advantage of producing the posterior distribution for the join points. The Bayesian join point model and model selection method that are presented here will be integrated in the National Cancer Institute's join point software ( http://www.srab.cancer.gov/joinpoint/ ) and will be available to the public.     

ε-Repetitions of the Maximum Residuals in an AR(1) Model

This paper considers record values of residuals or prediction errors in a one-parameter autoregressive process and the statistic Z _n = number of ε -repetitions of this record. When the parameter of the autoregression is unknown, the prediction errors, and therefore Z _n , are unobservable. Here an observable analogue ̂ n of Z _n is considered. It is proved that under special conditions the difference Z _n − unobservable. Here an observable analogue ̂ n converges to zero in probability and therefore that unobservable. Here an observable analogue ̂ n has the same asymptotic behaviour as Z _n .     

On a Posterior Predictive Density Sample Size Criterion

Abstract.  Let Ω be a space of densities with respect to some σ -finite measure μ and let Π be a prior distribution having support Ω with respect to some suitable topology. Conditional on f , let X ⁿ  = ( X ₁ ,…, X _n ) be an independent and identically distributed sample of size n from f . This paper introduces a Bayesian non-parametric criterion for sample size determination which is based on the integrated squared distance between posterior predictive densities. An expression for the sample size is obtained when the prior is a Dirichlet mixture of normal densities.     

On Statistical Models for d-Dimensional Stable Processes, and Some Generalizations

In statistical models where jumps of a d -dimensional stable process ( S _t ) _{t ≥0} are observed in windows with certain asymptotic properties, and where parameters appearing in the Levy measure of S are to be estimated, we have asymptotically efficient estimators. If Poisson random measure μ on (0, ∞) × ( R ^d \{0}) with intensity dt Λ( dx ) replaces the jump measure of S , where Λ is a ε-finite measure on R ^d \{0} admitting tail parameters in a suitable sense, we specify a notion of neighbourhood which allows to treat efficiency in statistical experiments of the second type by switching to accompanying sequences of the stable process type considered first.     

Ordering of Sequentially Sampled Exponential Experiments

Let X ₁, X ₂, ... be a sequence of i.i.d. random variables, X _i∼ F _θ, θ∈Θ. Let N ₁ and N ₂ be two stopping rules. For a class of exponential families { F _θ: θ∈Θ} we show that the experiment Y ₁ = ( X ₁, ..., X _N1) carries more statistical information than Y ₂ = ( X ₁, ..., x _N2) only if N ₁ is stochastically larger then N ₂     

Computation of the Generalized F Distribution

Exact expressions for the cumulative distribution function of a random variable of the form ( α ₁ X ₁+ α ₂ X ₂)/ Y are given where X ₁, X ₂ and Y are independent chi-squared random variables. The expressions are applied to the detection of joint outliers and Hotelling's mis-specified T ² distribution.     

On the effect of overdispersion on exact conditional tests

Let Y ₁, . . ., Y_n denote independent random variables such that Y_j has a one-parameter exponential family distribution with canonical parameter θ _j =λ+ψ X_j ; here X ₁, . . ., X_n are known constants. Consider a test of the null hypothesis ψ=0. Under the null hypothesis, A =Σ Y_j is sufficient for λ and, hence, a test of ψ=0 may be based on the conditional distribution of T =Σ X_j Y_j given A , which is independent of λ. In this paper, the effects of overdispersion due to a mixture model on the conditional distribution of T given A are considered.     

On Minimum Distance Estimation Using Kolmogorov-Lévy Type Metrics

Let X ₁, . . ., X_n be independent identically distributed random variables with a common continuous (cumulative) distribution function (d.f.) F , and F^_n the empirical d.f. (e.d.f.) based on X ₁, . . ., X_n . Let G be a smooth d.f. and Gθ = G (·–θ) its translation through θ∈ R . Using a Kolmogorov-Lévy type metric ρ_α defined on the space of d.f.s. on R , the paper derives both null and non-null limiting distributions of √ n [ ρ_α ( F_n , Gθ_n ) – ρ_α ( F, G_θ )], √ n (θ _n –θ) and √ nρ_α ( Gθ , Gθ ), where θ _n and θ are the minimum ρ_α -distance parameters for F_n and F from G , respectively. These distributions are known explicitly in important particular cases; with some complementary Monte Carlo simulations, they help us clarify our understanding of estimation using minimum distance methods and supremum type metrics. We advocate use of the minimum distance method with supremum type metrics in cases of non-null models. The resulting functionals are Hadamard differentiable and efficient. For small scale parameters the minimum distance functionals are close to medians of the parent distributions. The optimal small scale models result in minimum distance estimators having asymptotic variances very competitive and comparable with best known robust estimators.     

Exact Slopes of Test Statistics for the Multivariate Exponential Family

The objective of this paper is to investigate exact slopes of test statistics { T_n } when the random vectors X ₁, ..., X_n are distributed according to an unknown member of an exponential family { P _θ; θ∈Ω. Here Ω is a parameter set. We will be concerned with the hypothesis testing problem of H ₀θ∈Ω₀ vs H ₁: θ∉Ω₀ where Ω₀ is a subset of Ω. It will be shown that for an important class of problems and test statistics the exact slope of { T_n } at η in Ω−Ω₀ is determined by the shortest Kullback–Leibler distance from {θ: T_n (λ(θ)) = T_n (λ(π))} to Ω₀, λ_θ = E _θ)( X ).     

Minimum Contrast Estimation for Fractional Diffusions

Abstract.  When the Hurst coefficient of a fractional Brownian motion     is greater than 1/2 it is possible to define a stochastic integral with respect to     , as the pathwise limit of Riemann sums, and thus to consider pathwise solutions to fractional diffusion equations. In this paper, we consider the vanishing drift case and assume that the solution X _t is parameterized by θ in a compact parameter space Θ . Our main interest is the estimation of θ based on discrete time, but with very frequent observations. It is shown that the estimation problem in this context is locally asymptotically mixed normal. The asymptotic behaviour of a certain class of minimum contrast estimators is then studied and asymptotic efficiency is discussed.