Sampling of pairs in pairwise likelihood estimation for latent variable models with categorical observed variables

Pairwise likelihood is a limited information estimation method that has also been used for estimating the parameters of latent variable and structural equation models. Pairwise likelihood is a special case of composite likelihood methods that uses lower-order conditional or marginal log-likelihoods instead of the full log-likelihood. The composite likelihood to be maximized is a weighted sum of marginal or conditional log-likelihoods. Weighting has been proposed for increasing efficiency, but the choice of weights is not straightforward in most applications. Furthermore, the importance of leaving out higher-order scores to avoid duplicating lower-order marginal information has been pointed out. In this paper, we approach the problem of weighting from a sampling perspective. More specifically, we propose a sampling method for selecting pairs based on their contribution to the total variance from all pairs. The sampling approach does not aim to increase efficiency but to decrease the estimation time, especially in models with a large number of observed categorical variables. We demonstrate the performance of the proposed methodology using simulated examples and a real application.

     

Dynamic Prediction by Landmarking in Event History Analysis

Abstract.  This article advocates the landmarking approach that dynamically adjusts predictive models for survival data during the follow up. This updating is achieved by directly fitting models for the individuals still at risk at the landmark point. Using this approach, simple proportional hazards models are able to catch the development over time for models with time-varying effects of covariates or data with time-dependent covariates (biomarkers). To smooth the effect of the landmarking, sequences of models are considered with parametric effects of the landmark time point and fitted by maximizing appropriate pseudo log-likelihoods that extend the partial log-likelihood to cover the landmarking approach.     

Adjustment of the Profile Likelihood for a Class of Normal Regression Models

We apply the method of McCullagh & Tibshirani (1990) to a generalization of the model for variance components in which the parameter of interest can appear in both the mean and variance. We obtain the exact adjusted profile log-likelihood score function. For the variance components model, we obtain the adjusted profile log-likelihood and show that it equals the restricted log-likelihood of Patterson & Thompson (1971). We discuss an example due to Kempton (1982) of a regression model with autoregressive terms in which the parameter of interest appears in both the mean and variance.     

A procedure for testing suspected observations

The problem of multiple outliers detection in one-parameter exponential family is considered. The outlier detection procedure involves two estimates of scale parameter which are obtained by maximizing two log-likelihoods; the complete data log-likelihood and its conditional expectation given suspected observations. The procedure is also applied to the exponential and normal samples.     

Local influence analysis for regression models with scale mixtures of skew-normal distributions

The robust estimation and the local influence analysis for linear regression models with scale mixtures of multivariate skew-normal distributions have been developed in this article. The main virtue of considering the linear regression model under the class of scale mixtures of skew-normal distributions is that they have a nice hierarchical representation which allows an easy implementation of inference. Inspired by the expectation maximization algorithm, we have developed a local influence analysis based on the conditional expectation of the complete-data log-likelihood function, which is a measurement invariant under reparametrizations. This is because the observed data log-likelihood function associated with the proposed model is somewhat complex and with Cook's well-known approach it can be very difficult to obtain measures of the local influence. Some useful perturbation schemes are discussed. In order to examine the robust aspect of this flexible class against outlying and influential observations, some simulation studies have also been presented. Finally, a real data set has been analyzed, illustrating the usefulness of the proposed methodology.     

Quasi-maximum likelihood estimation and inference in dynamic models with time-varying covariances

We study the properties of the quasi-maximum likelihood estimator (QMLE) and related test statistics in dynamic models that jointly parameterize conditional means and conditional covariances, when a normal log-likelihood os maximized but the assumption of normality is violated. Because the score of the normal log-likelihood has the martingale difference property when the forst two conditional moments are correctly specified, the QMLE is generally Consistent and has a limiting normal destribution. We provide easily computable formulas for asymptotic standard errors that are valid under nonnormality. Further, we show how robust LM tests for the adequacy of the jointly parameterized mean and variance can be computed from simple auxiliary regressions. An appealing feature of these robyst inference procedures is that only first derivatives of the conditional mean and variance functions are needed. A monte Carlo study indicates that the asymptotic results carry over to finite samples. Estimation of several AR and AR-GARCH time series models reveals that in most sotuations the robust test statistics compare favorably to the two standard (nonrobust) formulations of the Wald and IM tests. Also, for the GARCH models and the sample sizes analyzed here, the bias in the QMLE appears to be relatively small. An empirical application to stock return volatility illustrates the potential imprtance of computing robust statistics in practice.     

Quasi-maximum likelihood estimation and inference in dynamic models with time-varying covariances

We study the properties of the quasi-maximum likelihood estimator (QMLE) and related test statistics in dynamic models that jointly parameterize conditional means and conditional covariances, when a normal log-likelihood os maximized but the assumption of normality is violated. Because the score of the normal log-likelihood has the martingale difference property when the forst two conditional moments are correctly specified, the QMLE is generally Consistent and has a limiting normal destribution. We provide easily computable formulas for asymptotic standard errors that are valid under nonnormality. Further, we show how robust LM tests for the adequacy of the jointly parameterized mean and variance can be computed from simple auxiliary regressions. An appealing feature of these robyst inference procedures is that only first derivatives of the conditional mean and variance functions are needed. A monte Carlo study indicates that the asymptotic results carry over to finite samples. Estimation of several AR and AR-GARCH time series models reveals that in most sotuations the robust test statistics compare favorably to the two standard (nonrobust) formulations of the Wald and IM tests. Also, for the GARCH models and the sample sizes analyzed here, the bias in the QMLE appears to be relatively small. An empirical application to stock return volatility illustrates the potential imprtance of computing robust statistics in practice.     

Applying skovgaard's modified directed likelihood statistic to mixed linear models

The introduction of software to calculate maximum likelihood estimates for mixed linear models has made likelihood estimation a practical alternative to methods based on sums of squares. Likelihood based tests and confidence intervals, however, may be misleading in problems with small sample sizes. This paper discusses an adjusted version of the directed log-likelihood statistic for mixed models that is highly accurate for testing one parameter hypotheses. Indroduced by Skovgaard (1996, Journal of the Bernoulli Society,2,145-165), we show in mixed models that the statistic has a simple conpact from that may be obtained from standard software. Simulation studies indicate that this statistic is more accurate than many of the specialized procedure that have been advocated.     

On the adequacy of variational lower bound functions for likelihood-based inference in Markovian models with missing values

Summary. Variational methods have been proposed for obtaining deterministic lower bounds for log-likelihoods within missing data problems, but with little formal justification or investigation of the worth of the lower bound surfaces as tools for inference. We provide, within a general Markovian context, sufficient conditions under which estimators from the variational approximations are asymptotically equivalent to maximum likelihood estimators, and we show empirically, for the simple example of a first-order autoregressive model with missing values, that the lower bound surface can be very similar in shape to the true log-likelihood in non-asymptotic situations.     

Estimation of nonlinear CUB models via numerical optimization and EM algorithm

The analysis of human perceptions is often carried out by resorting to surveys and questionnaires, where respondents are asked to express ratings about the objects being evaluated. A class of mixture models, called CUB (Combination of Uniform and shifted Binomial), has been recently proposed in this context. This article focuses on a model of this class, the Nonlinear CUB, and investigates some computational issues concerning parameter estimation, which is performed by Maximum Likelihood. More specifically, we consider two main approaches to optimize the log-likelihood: the classical numerical methods of optimization and the EM algorithm. The classical numerical methods comprise the widely used algorithms Nelder–Mead, Newton–Raphson, Broyden–Fletcher–Goldfarb–Shanno (BFGS), Berndt–Hall–Hall–Hausman (BHHH), Simulated Annealing, Conjugate Gradients and usually have the advantage of a fast convergence. On the other hand, the EM algorithm deserves consideration for some optimality properties in the case of mixture models, but it is slower. This article has a twofold aim: first we show how to obtain explicit formulas for the implementation of the EM algorithm in nonlinear CUB models and we formally derive the asymptotic variance–covariance matrix of the Maximum Likelihood estimator; second, we discuss and compare the performance of the two above mentioned approaches to the log-likelihood maximization.     

Influence analysis of additive mixed-effects nonlinear regression models via EM algorithm

This paper presents a unified method for influence analysis to deal with random effects appeared in additive nonlinear regression models for repeated measurement data. The basic idea is to apply the Q-function, the conditional expectation of the complete-data log-likelihood function obtained from EM algorithm, instead of the observed-data log-likelihood function as used in standard influence analysis. Diagnostic measures are derived based on the case-deletion approach and the local influence approach. Two real examples and a simulation study are examined to illustrate our methodology.     

Testing of homogeneity of variance and autocorrelation coefficients of nonlinear mixed models with AR(1) errors based on M-estimation

Homogeneity of between-individual variance and autocorrelation coefficients is one of assumptions in the study of longitudinal data. However, the assumption could be challenging due to the complexity of the dataset. In the paper we propose and analyze nonlinear mixed models with AR(1) errors for longitudinal data, intend to introduce Huber's function in the log-likelihood function and get robust estimation, which may help to reduce the influence of outliers, by Fisher scoring method. Testing of homogeneity of variance among individuals and autocorrelation coefficients on the basis of Huber's M-estimation is studied later in the paper. Simulation studies are carried to assess performance of score test we proposed. Results obtained from plasma concentrations data are reported as an illustrative example.     

A statistical model for orientation mechanism

A variance components model with response variable depending on both fixed effects of explanatory variables and random components is specified to model longitudinal circular data, in order to study the directional behaviour of small animals, as insects, crustaceans, amphipods, etc. Unknown parameter estimators are obtained using a simulated maximum likelihood approach. Issues concerning log-likelihood variability and the related problems in the optimization algorithm are also addressed. The procedure is applied to the analysis of directional choices under full natural conditions ofTalitrus saltator from Castiglione della Pescaia (Italy) beaches.     

BL-GARCH models and asymmetries in volatility

In this paper the class of Bilinear GARCH (BL-GARCH) models is proposed. BL-GARCH models allow to capture asymmetries in the conditional variance of financial and economic time series by means of interactions between past shocks and volatilities. The availability of likelihood based inference is an attractive feature of BL-GARCH models. Under the assumption of conditional normality, the log-likelihood function can be maximized by means of an EM type algorithm. The main reason for using the EM algorithm is that it allows to obtain parameter estimates which naturally guarantee the positive definiteness of the conditional variance with no need for additional parameter constraints. We also derive a robust LM test statistic which can be used for model identification. Finally, the effectiveness of BL-GARCH models in capturing asymmetric volatility patterns in financial time series is assessed by means of an application to a time series of daily returns on the NASDAQ Composite stock market index.     

CORRECTED SCORE FUNCTIONS IN CLASSICAL ERROR-IN-VARIABLES AND INCIDENTAL PARAMETER MODELS

Nakamura (1990) introduced an approach to estimation in measurement error models based on a corrected score function, and claimed that the estimators obtained are consistent for functional models. Proof of the claim essentially assumed the existence of a corrected log-likelihood for which differentiation with respect to model parameters can be interchanged with conditional expectation taken with respect to the measurement error distributions, given the response variables and true covariates. This paper deals with simple yet practical models for which the above assumption is false, i.e. a corrected score function for the model may not be obtained through differentiating a corrected log-likelihood although it exists. Alternative regularity conditions with no reference to log-likelihood are given, under which the corrected score functions yield consistent and asymptotically normal estimators. Application to functional comparative calibration yields interesting results.     

Minimum variance unbiased estimation in natural exponential families generated by stable distributions

The class of nature exponential families generated by stable distributions has been introduced in different contexts by several authors. Tweedie (1984) and Jorgensen (1987) studied this class in the context of generalized liner models and exponential dispersion models. Bar-Lev and Enis (1986) introduced this class in the context of the property of reproducibility in natural exponential families and Hougaard (1986) found the distributions in this class to be natural candidates for applications as survival distributions in life tables for heterogeneous populations. In this note, we consider such a class in the context of minimum variance unbiased estimation. For each family in this class, we obtain an explicit expression for the uniformly minimum variance unbiased estimator for the r-th cumlant, the density function, and the reliability function.     

Adaptive likelihood estimator of conditional variance function

Modelling volatility in the form of conditional variance function has been a popular method mainly due to its application in financial risk management. Among others, we distinguish the parametric GARCH models and the nonparametric local polynomial approximation using weighted least squares or gaussian likelihood function. We introduce an alternative likelihood estimate of conditional variance and we show that substitution of the error density with its estimate yields similar asymptotic properties, that is, the proposed estimate is adaptive to the error distribution. Theoretical comparison with existing estimates reveals substantial gains in efficiency, especially if error distribution has fatter tails than Gaussian distribution. Simulated data confirm the theoretical findings while an empirical example demonstrates the gains of the proposed estimate.     

Pseudo–Modelling of Outliers for Estimation of Regression And Time Series Models

This paper documents situations where the variance inflation model for outliers has undesirable properties. The model is commonly used to accommodate outliers in a Bayesian analysis of regression and time series models. The alternative approach provided here does not suffer from these undesirable properties but gives inferences similar to those of the variance inflation model when this is appropriate. It can be used with regression, time series, and regression with correlated errors in a unified way, and adheres to the scientific principle that inference should be based on the data after obvious outliers have been discarded. Only one parameter is required for outliers; it is interpretable as the a priori willingness to remove observations from the analysis.     

Multivariate Process Variability Monitoring

Understanding multivariate variability is a difficult task because there is no single measure that can be properly used. This article presents a new measure that features good properties. If this measure is simultaneously used with generalized variance, it will give a better understanding of multivariate variability. It can also efficiently be used for large data sets with high dimensions. Furthermore, when it is used for constructing a Shewhart-type chart to monitor multivariate variability, the resulting chart has a much better out-of-control ARL than the generalized variance chart. An example illustrates its advantage.     

Model selection for infinite variance time series

In this we consider the problem of model selection for infinite variance time series. We introduce a group of model selection critera based on a general loss function Ψ. This family includes various generalizations of predictive least square and AIC Parameter estimation is carried out using Ψ. We use two loss functions commonly used in robust estimation and show that certain criteria out perform the conventional approach based on least squares or Yule-Walker estima­tion for heavy tailed innovations. Our conclusions are based on a comprehensive study of the performance of competing criteria for a wide selection of AR(2) models. We also consider the performance of these techniques when the ‘true’ model is not contained in the family of candidate models.