Semiparametric Likelihood Based Method for Goodness of Fit Tests and Estimation in Upgraded Mixture Models

We use Owen's (1988, 1990) empirical likelihood method in upgraded mixture models. Two groups of independent observations are available. One is z ₁, ..., z _n which is observed directly from a distribution F ( z ). The other one is x ₁, ..., x _m which is observed indirectly from F ( z ), where the x _i s have density ∫ p ( x | z ) dF ( z ) and p ( x | z ) is a conditional density function. We are interested in testing H ₀: p ( x | z ) = p ( x | z ; θ ), for some specified smooth density function. A semiparametric likelihood ratio based statistic is proposed and it is shown that it converges to a chi-squared distribution. This is a simple method for doing goodness of fit tests, especially when x is a discrete variable with finitely many values. In addition, we discuss estimation of θ and F ( z ) when H ₀ is true. The connection between upgraded mixture models and general estimating equations is pointed out.     

Rejoinder to 'Ahmed, M.S. (1998). A note on regression-type estimators using multiple auxiliary information.'

In the estimators t ₃ , t ₄ , t ₅ of Mukerjee, Rao & Vijayan (1987), b _{y x} and b _{y z} are partial regression coefficients of y on x and z , respectively, based on the smaller sample. With the above interpretation of b _{y x} and b _{y z} in t ₃ , t ₄ , t ₅ , all the calculations in Mukerjee at al. (1987) are correct. In this connection, we also wish to make it explicit that b _{x z} in t ₅ is an ordinary and not a partial regression coefficient. The 'corrected' MSEs of t ₃ , t ₄ , t ₅ , as given in Ahmed (1998 Section 3) are computed assuming that our b _{y x} and b _{y z} are ordinary and not partial regression coefficients. Indeed, we had no intention of giving estimators using the corresponding ordinary regression coefficients which would lead to estimators inferior to those given by Kiregyera (1984). We accept responsibility for any notational confusion created by us and express regret to readers who have been confused by our notation. Finally, in consideration of the above, it may be noted that Tripathi & Ahmed's (1995) estimator t ₀ , quoted also in Ahmed (1998), is no better than t ₅ of Mukerjee at al. (1987).     

Ordering of Sequentially Sampled Exponential Experiments

Let X ₁, X ₂, ... be a sequence of i.i.d. random variables, X _i∼ F _θ, θ∈Θ. Let N ₁ and N ₂ be two stopping rules. For a class of exponential families { F _θ: θ∈Θ} we show that the experiment Y ₁ = ( X ₁, ..., X _N1) carries more statistical information than Y ₂ = ( X ₁, ..., x _N2) only if N ₁ is stochastically larger then N ₂     

Non-parametric Regression with Dependent Censored Data

Abstract.  Let ( X _i , Y _i ) ( i = 1 ,…, n ) be n replications of a random vector ( X , Y  ), where Y is supposed to be subject to random right censoring. The data ( X _i , Y _i ) are assumed to come from a stationary α -mixing process. We consider the problem of estimating the function m ( x ) = E ( φ ( Y ) |  X = x ), for some known transformation φ . This problem is approached in the following way: first, we introduce a transformed variable     , that is not subject to censoring and satisfies the relation     , and then we estimate m ( x ) by applying local linear regression techniques. As a by-product, we obtain a general result on the uniform rate of convergence of kernel type estimators of functionals of an unknown distribution function, under strong mixing assumptions.     

A Generalization of the Childs-Moran Result for Orthoscheme Probabilities

van der Vaart (1953, 1955) introduced the orthoscheme probability R_n (c ₁,..., c_n−1 ), meaning the orthant probability of an n -dimensional normal random vector with zero mean and tridiagonal correlation matrix with elements c ₁,..., c_n−1 on the upper diagonal. Childs (1967) conjectured and Moran (1983) proved that the generating function of { R_n (½,...,½)} equals tan z + sin z . This paper derives the generating function of { R_n (τ,½,...,½)}.     

A Limit Theorem for Solutions of Inequalities

Let H ( p ) be the set { x ∈ X : h ( x ) ≤ p } where h is a real-valued lower semicontinuous function on a locally compact separable metric space X . This paper presents a general limit theorem for the sequence of random sets H _n ( p ) = { x ∈ X : h _n ( x ) ≤ p } n ≥ 1, where h _n , n ≥ 1, are functions that estimate h     

Computation of the Generalized F Distribution

Exact expressions for the cumulative distribution function of a random variable of the form ( α ₁ X ₁+ α ₂ X ₂)/ Y are given where X ₁, X ₂ and Y are independent chi-squared random variables. The expressions are applied to the detection of joint outliers and Hotelling's mis-specified T ² distribution.     

Likelihood Ratio Tests Under Local and Fixed Alternatives in Monotone Function Problems

Abstract.  We focus on a class of non-standard problems involving non-parametric estimation of a monotone function that is characterized by n ^1/3 rate of convergence of the maximum likelihood estimator, non-Gaussian limit distributions and the non-existence of     -regular estimators. We have shown elsewhere that under a null hypothesis of the type ψ ( z ₀) =  θ ₀ ( ψ being the monotone function of interest) in non-standard problems of the above kind, the likelihood ratio statistic has a 'universal' limit distribution that is free of the underlying parameters in the model. In this paper, we illustrate its limiting behaviour under local alternatives of the form ψ _n ( z ), where ψ _n (·) and ψ (·) vary in O ( n ^−1/3) neighbourhoods around z ₀ and ψ _n converges to ψ at rate n ^1/3 in an appropriate metric. Apart from local alternatives, we also consider the behaviour of the likelihood ratio statistic under fixed alternatives and establish the convergence in probability of an appropriately scaled version of the same to a constant involving a Kullback–Leibler distance.     

Empirical Likelihood for Non-Smooth Criterion Functions

Abstract.  Suppose that X ₁ ,…,  X _n is a sequence of independent random vectors, identically distributed as a d -dimensional random vector X . Let     be a parameter of interest and     be some nuisance parameter. The unknown, true parameters ( μ ₀ , ν ₀ ) are uniquely determined by the system of equations E { g ( X , μ ₀ , ν ₀ )} =   0 , where g  =  ( g ₁ ,…, g _{p + q} ) is a vector of p + q functions. In this paper we develop an empirical likelihood (EL) method to do inference for the parameter μ ₀ . The results in this paper are valid under very mild conditions on the vector of criterion functions g . In particular, we do not require that g ₁ ,…, g _{p + q} are smooth in μ or ν . This offers the advantage that the criterion function may involve indicators, which are encountered when considering, e.g. differences of quantiles, copulas, ROC curves, to mention just a few examples. We prove the asymptotic limit of the empirical log-likelihood ratio, and carry out a small simulation study to test the performance of the proposed EL method for small samples.     

An Efficient Simulation Method for the Computation of a Class of Conditional Expectations

Suppose that the random vector X and the random variable Y are jointly continuous. Also suppose that an observation x of X can be easily simulated and that the probability density function of Y conditional on X = x is known. The paper presents an efficient simulation-based algorithm for estimating E{ g ( X , Y ) | h ( X , Y ) = r } where g and h are real-valued functions. This algorithm is applicable to time series problems in which X = ( X ₁, . . . , X _n−1) and Y = X_n where { x_t } is a discrete time stochastic process for which ( X₁ , . . . , X_n ) is a continuous random vector. A numerical example from time series analysis illustrates the algorithim, for prediction for an ARCH(1) process.     

Exact Slopes of Test Statistics for the Multivariate Exponential Family

The objective of this paper is to investigate exact slopes of test statistics { T_n } when the random vectors X ₁, ..., X_n are distributed according to an unknown member of an exponential family { P _θ; θ∈Ω. Here Ω is a parameter set. We will be concerned with the hypothesis testing problem of H ₀θ∈Ω₀ vs H ₁: θ∉Ω₀ where Ω₀ is a subset of Ω. It will be shown that for an important class of problems and test statistics the exact slope of { T_n } at η in Ω−Ω₀ is determined by the shortest Kullback–Leibler distance from {θ: T_n (λ(θ)) = T_n (λ(π))} to Ω₀, λ_θ = E _θ)( X ).     

An Analysis of Swendsen–Wang and Related Sampling Methods

Convergence rates, statistical efficiency and sampling costs are studied for the original and extended Swendsen–Wang methods of generating a sample path { S _j , j ≥1} with equilibrium distribution π , with r distinct elements, on a finite state space X of size N ₁. Given S _{j -1}, each method uses auxiliary random variables to identify the subset of X from which S _j is to be randomly sampled. Let π_min and π_max denote respectively the smallest and largest elements in π and let N_r denote the number of elements in π with value π_max. For a single auxiliary variable, uniform sampling from the subset and ( N ₁− N_r )π_min+ N_r π_max≈1, our results show rapid convergence and high statistical efficiency for large π_min/π_max or N_r / N ₁ and slow convergence and poor statistical efficiency for small π_min/π_max and N_r / N₁ . Other examples provide additional insight. For extended Swendsen–Wang methods with non-uniform subset sampling, the analysis identifies the properties of a decomposition of π( x ) that favour fast convergence and high statistical efficiency. In the absence of exploitable special structure, subset sampling can be costly regardless of which of these methods is employed.     

The Cusum of Squares Test for Scale Changes in Infinite Order Moving Average Processes

In this paper we consider the problem of testing for a scale change in the infinite order moving average process X _j = _{i =0} a _{i j i} , where _j are i.i.d. r.v.s with E ₁ < for some > 0. In performing the test, a cusum of squares test statistic analogous to Inclan & Tiao's (1994) statistic is considered. It is well-known from the literature that outliers affect test procedures leading to false conclusions. In order to remedy this, a cusum of squares test based on trimmed observations is considered. It is demonstrated that this test is robust against outliers, is valid for infinite variance processes as well. Simulation results are given for illustration.     

Detecting changes in the mean of functional observations

Summary.  Principal component analysis has become a fundamental tool of functional data analysis. It represents the functional data as X _i ( t )= μ ( t )+Σ_{1≤ l <∞} η _{i ,  l} +  v _l ( t ), where μ is the common mean, v _l are the eigenfunctions of the covariance operator and the η _{i ,  l} are the scores. Inferential procedures assume that the mean function μ ( t ) is the same for all values of i . If, in fact, the observations do not come from one population, but rather their mean changes at some point(s), the results of principal component analysis are confounded by the change(s). It is therefore important to develop a methodology to test the assumption of a common functional mean. We develop such a test using quantities which can be readily computed in the R package fda. The null distribution of the test statistic is asymptotically pivotal with a well-known asymptotic distribution. The asymptotic test has excellent finite sample performance. Its application is illustrated on temperature data from England.     

Estimation in Semiparametric Marginal Shared Gamma Frailty Models

The semiparametric marginal shared frailty models in survival analysis have the non–parametric hazard functions multiplied by a random frailty in each cluster, and the survival times conditional on frailties are assumed to be independent. In addition, the marginal hazard functions have the same form as in the usual Cox proportional hazard models. In this paper, an approach based on maximum likelihood and expectation–maximization is applied to semiparametric marginal shared gamma frailty models, where the frailties are assumed to be gamma distributed with mean 1 and variance θ. The estimates of the fixed–effect parameters and their standard errors obtained using this approach are compared in terms of both bias and efficiency with those obtained using the extended marginal approach. Similarly, the standard errors of our frailty variance estimates are found to compare favourably with those obtained using other methods. The asymptotic distribution of the frailty variance estimates is shown to be a 50–50 mixture of a point mass at zero and a truncated normal random variable on the positive axis for θ₀ = 0. Simulations demonstrate that, for θ₀ < 0, it is approximately an x −(100 − x )%, 0 ≤ x ≤ 50, mixture between a point mass at zero and a truncated normal random variable on the positive axis for small samples and small values of θ₀; otherwise, it is approximately normal.     

Estimation of Diffusion Processes by Simulated Moment Methods

We consider the parameter estimation of a diffusion process and we suppose that the trend and the diffusion coefficient depend on the parameter θ. The process is observed at time ( t_i ) _{i =0,..., n} with Δ = t_{i +1}− t_i fixed and we propose here to estimate θ from simulated moment methods.     

Modelling and smoothing parameter estimation with multiple quadratic penalties

Penalized likelihood methods provide a range of practical modelling tools, including spline smoothing, generalized additive models and variants of ridge regression. Selecting the correct weights for penalties is a critical part of using these methods and in the single-penalty case the analyst has several well-founded techniques to choose from. However, many modelling problems suggest a formulation employing multiple penalties, and here general methodology is lacking. A wide family of models with multiple penalties can be fitted to data by iterative solution of the generalized ridge regression problem minimize || W ^1/2 ( Xp − y ) ||²ρ+Σ _{i =1} ^m  θ _i p ' S _i p ( p is a parameter vector, X a design matrix, S _i a non-negative definite coefficient matrix defining the i th penalty with associated smoothing parameter θ _i , W a diagonal weight matrix, y a vector of data or pseudodata and ρ an 'overall' smoothing parameter included for computational efficiency). This paper shows how smoothing parameter selection can be performed efficiently by applying generalized cross-validation to this problem and how this allows non-linear, generalized linear and linear models to be fitted using multiple penalties, substantially increasing the scope of penalized modelling methods. Examples of non-linear modelling, generalized additive modelling and anisotropic smoothing are given.     

Nonparametric multistep-ahead prediction in time series analysis

Summary.  We consider the problem of multistep-ahead prediction in time series analysis by using nonparametric smoothing techniques. Forecasting is always one of the main objectives in time series analysis. Research has shown that non-linear time series models have certain advantages in multistep-ahead forecasting. Traditionally, nonparametric k -step-ahead least squares prediction for non-linear autoregressive AR( d ) models is done by estimating E ( X _{t + k}  | X _t , …,  X _{t − d +1}) via nonparametric smoothing of X _{t + k} on ( X _t , …,  X _{t − d +1}) directly. We propose a multistage nonparametric predictor. We show that the new predictor has smaller asymptotic mean-squared error than the direct smoother, though the convergence rate is the same. Hence, the predictor proposed is more efficient. Some simulation results, advice for practical bandwidth selection and a real data example are provided.     

Estimating Main Effects with Pareto Optimal Subsets

A subset T of S is said to be a Pareto Optimal subset of m ordered attributes (factors) if for profiles (combination of attribute levels) ( x ₁, …, x_m ) and ( y ₁, …, y_m ) ∈ T , no profile 'dominates' another; that is, there exists no pair such that x_i ≤ y_i , for i = 1, …, m . Pareto Optimal designs have specific applications in economics, cognitive psychology, and marketing research where investigators use main effects linear models to infer how respondents values level of costs and benefits from their preferences for sets of profiles offered them. In such studies, it is desirable that no profile dominates the others in a set. This paper shows how to construct a Pareto Optimal subset, proves that a single Pareto Optimal subset is not a connected main effects plan, provides subsets of two or more attributes that are connected in symmetric designs and gives corresponding results for asymmetric designs.     

Penalized Projection Estimator for Volatility Density

Abstract.  In this paper, we consider a stochastic volatility model ( Y _t , V _t ), where the volatility (V _t ) is a positive stationary Markov process. We assume that ( ln V _t ) admits a stationary density f that we want to estimate. Only the price process Y _t is observed at n discrete times with regular sampling interval Δ . We propose a non-parametric estimator for f obtained by a penalized projection method. Under mixing assumptions on ( V _t ), we derive bounds for the quadratic risk of the estimator. Assuming that Δ=Δ _n tends to 0 while the number of observations and the length of the observation time tend to infinity, we discuss the rate of convergence of the risk. Examples of models included in this framework are given.