Semiparametric Time Series Models with Log‐concave Innovations: Maximum Likelihood Estimation and its Consistency

We study semiparametric time series models with innovations following a log‐concave distribution. We propose a general maximum likelihood framework that allows us to estimate simultaneously the parameters of the model and the density of the innovations. This framework can be easily adapted to many well‐known models, including autoregressive moving average (ARMA), generalized autoregressive conditionally heteroscedastic (GARCH), and ARMA‐GARCH models. Furthermore, we show that the estimator under our new framework is consistent in both ARMA and ARMA‐GARCH settings. We demonstrate its finite sample performance via a thorough simulation study and apply it to model the daily log‐return of the FTSE 100 index.     

An Elementary Introduction to Maximum Likelihood Estimation for Multinomial Models: Birch's Theorem and the Delta Method

A fairly complete introduction to the large sample theory of parametric multinomial models, suitable for a second-year graduate course in categorical data analysis, can be based on Birch's theorem (1964) and the delta method (Bishop, Fienberg, and Holland 1975). I present an elementary derivation of a version of Birch's theorem using the implicit function theorem from advanced calculus, which allows the presentation to be relatively self-contained. The use of the delta method in deriving asymptotic distributions is illustrated by Rao's (1973) result on the distribution of standardized residuals, which complements the presentation in Bishop, Fienberg, and Holland. The asymptotic theory is illustrated by two examples.     

Maximum Likelihood Estimation of a Poisson Parameter in a Model of Equioverlapping Samples:A Simulation Study

In a model of equioverlapping samples maximum likelihood estimation of a Poisson parameter is examined and compared with two linear unbiased estimations by mean squared error. Since a likelihood estimator is not explicitly available in general, a simulation study has been performed and the results are illustrated     

Likelihood Ratio Type Two‐Sample Tests for Current Status Data

Abstract. We introduce fully non‐parametric two‐sample tests for testing the null hypothesis that the samples come from the same distribution if the values are only indirectly given via current status censoring. The tests are based on the likelihood ratio principle and allow the observation distributions to be different for the two samples, in contrast with earlier proposals for this situation. A bootstrap method is given for determining critical values and asymptotic theory is developed. A simulation study, using Weibull distributions, is presented to compare the power behaviour of the tests with the power of other non‐parametric tests in this situation.     

Central Limit Theorems for the Non‐Parametric Estimation of Time‐Changed Lévy Models

Abstract. Let {Z_t}_{t 0} be a Lévy process with Lévy measure ν and let be a random clock, where g is a non‐negative function and is an ergodic diffusion independent of Z. Time‐changed Lévy models of the form are known to incorporate several important stylized features of asset prices, such as leptokurtic distributions and volatility clustering. In this article, we prove central limit theorems for a type of estimators of the integral parameter β(?):=∫?(x)ν(dx), valid when both the sampling frequency and the observation time‐horizon of the process get larger. Our results combine the long‐run ergodic properties of the diffusion process with the short‐term ergodic properties of the Lévy process Z via central limit theorems for martingale differences. The performance of the estimators are illustrated numerically for Normal Inverse Gaussian process Z and a Cox–Ingersoll–Ross process .     

A Class of Convolution‐Based Models for Spatio‐Temporal Processes with Non‐Separable Covariance Structure

Abstract. In this article, we propose a new parametric family of models for real‐valued spatio‐temporal stochastic processes S ( x , t ) and show how low‐rank approximations can be used to overcome the computational problems that arise in fitting the proposed class of models to large datasets. Separable covariance models, in which the spatio‐temporal covariance function of S ( x , t ) factorizes into a product of purely spatial and purely temporal functions, are often used as a convenient working assumption but are too inflexible to cover the range of covariance structures encountered in applications. We define positive and negative non‐separability and show that in our proposed family we can capture positive, zero and negative non‐separability by varying the value of a single parameter.     

Estimation and Inference Procedures for Semiparametric Distribution Models with Varying Linear‐Index

More flexible semiparametric linear‐index regression models are proposed to describe the conditional distribution. Such a model formulation captures varying effects of covariates over the support of a response variable in distribution, offers an alternative perspective on dimension reduction and covers a lot of widely used parametric and semiparameteric regression models. A feasible pseudo likelihood approach, accompanied with a simple and easily implemented algorithm, is further developed for the mixed case with both varying and invariant coefficients. By justifying some theoretical properties on Banach spaces, the uniform consistency and asymptotic Gaussian process of the proposed estimator are also established in this article. In addition, under the monotonicity of distribution in linear‐index, we develop an alternative approach based on maximizing a varying accuracy measure. By virtue of the asymptotic recursion relation for the estimators, some of the achievements in this direction include showing the convergence of the iterative computation procedure and establishing the large sample properties of the resulting estimator. It is noticeable that our theoretical framework is very helpful in constructing confidence bands for the parameters of interest and tests for the hypotheses of various qualitative structures in distribution. Generally, the developed estimation and inference procedures perform quite satisfactorily in the conducted simulations and are demonstrated to be useful in reanalysing data from the Boston house price study and the World Values Survey.     

Box‐Cox transformations in linear models: Large sample theory and tests of normality

The authors provide a rigorous large sample theory for linear models whose response variable has been subjected to the Box‐Cox transformation. They provide a continuous asymptotic approximation to the distribution of estimators of natural parameters of the model. They show, in particular, that the maximum likelihood estimator of the ratio of slope to residual standard deviation is consistent and relatively stable. The authors further show the importance for inference of normality of the errors and give tests for normality based on the estimated residuals. For non‐normal errors, they give adjustments to the log‐likelihood and to asymptotic standard errors.     

Consistency of semiparametric maximum likelihood estimators for two‐phase sampling

Semiparametric maximum likelihood estimators have recently been proposed for a class of two‐phase, outcome‐dependent sampling models. All of them were “restricted” maximum likelihood estimators, in the sense that the maximization is carried out only over distributions concentrated on the observed values of the covariate vectors. In this paper, the authors give conditions for consistency of these restricted maximum likelihood estimators. They also consider the corresponding unrestricted maximization problems, in which the “absolute” maximum likelihood estimators may then have support on additional points in the covariate space. Their main consistency result also covers these unrestricted maximum likelihood estimators, when they exist for all sample sizes.     

Mixture Model Analysis of Partially Rank‐Ordered Set Samples: Age Groups of Fish from Length‐Frequency Data

We present a novel methodology for estimating the parameters of a finite mixture model (FMM) based on partially rank‐ordered set (PROS) sampling and use it in a fishery application. A PROS sampling design first selects a simple random sample of fish and creates partially rank‐ordered judgement subsets by dividing units into subsets of prespecified sizes. The final measurements are then obtained from these partially ordered judgement subsets. The traditional expectation–maximization algorithm is not directly applicable for these observations. We propose a suitable expectation–maximization algorithm to estimate the parameters of the FMMs based on PROS samples. We also study the problem of classification of the PROS sample into the components of the FMM. We show that the maximum likelihood estimators based on PROS samples perform substantially better than their simple random sample counterparts even with small samples. The results are used to classify a fish population using the length‐frequency data.     

Confidence intervals for the mean of a population containing many zero values under unequal‐probability sampling

In many applications, a finite population contains a large proportion of zero values that make the population distribution severely skewed. An unequal‐probability sampling plan compounds the problem, and as a result the normal approximation to the distribution of various estimators has poor precision. The central‐limit‐theorem‐based confidence intervals for the population mean are hence unsatisfactory. Complex designs also make it hard to pin down useful likelihood functions, hence a direct likelihood approach is not an option. In this paper, we propose a pseudo‐likelihood approach. The proposed pseudo‐log‐likelihood function is an unbiased estimator of the log‐likelihood function when the entire population is sampled. Simulations have been carried out. When the inclusion probabilities are related to the unit values, the pseudo‐likelihood intervals are superior to existing methods in terms of the coverage probability, the balance of non‐coverage rates on the lower and upper sides, and the interval length. An application with a data set from the Canadian Labour Force Survey‐2000 also shows that the pseudo‐likelihood method performs more appropriately than other methods. The Canadian Journal of Statistics 38: 582–597; 2010 © 2010 Statistical Society of Canada     

A comparison of bootstrap approaches for estimating uncertainty of parameters in linear mixed‐effects models

A version of the nonparametric bootstrap, which resamples the entire subjects from original data, called the case bootstrap, has been increasingly used for estimating uncertainty of parameters in mixed‐effects models. It is usually applied to obtain more robust estimates of the parameters and more realistic confidence intervals (CIs). Alternative bootstrap methods, such as residual bootstrap and parametric bootstrap that resample both random effects and residuals, have been proposed to better take into account the hierarchical structure of multi‐level and longitudinal data. However, few studies have been performed to compare these different approaches. In this study, we used simulation to evaluate bootstrap methods proposed for linear mixed‐effect models. We also compared the results obtained by maximum likelihood (ML) and restricted maximum likelihood (REML). Our simulation studies evidenced the good performance of the case bootstrap as well as the bootstraps of both random effects and residuals. On the other hand, the bootstrap methods that resample only the residuals and the bootstraps combining case and residuals performed poorly. REML and ML provided similar bootstrap estimates of uncertainty, but there was slightly more bias and poorer coverage rate for variance parameters with ML in the sparse design. We applied the proposed methods to a real dataset from a study investigating the natural evolution of Parkinson's disease and were able to confirm that the methods provide plausible estimates of uncertainty. Given that most real‐life datasets tend to exhibit heterogeneity in sampling schedules, the residual bootstraps would be expected to perform better than the case bootstrap. Copyright © 2013 John Wiley & Sons, Ltd.