Multivariate Poisson regression with covariance structure

In recent years the applications of multivariate Poisson models have increased, mainly because of the gradual increase in computer performance. The multivariate Poisson model used in practice is based on a common covariance term for all the pairs of variables. This is rather restrictive and does not allow for modelling the covariance structure of the data in a flexible way. In this paper we propose inference for a multivariate Poisson model with larger structure, i.e. different covariance for each pair of variables. Maximum likelihood estimation, as well as Bayesian estimation methods are proposed. Both are based on a data augmentation scheme that reflects the multivariate reduction derivation of the joint probability function. In order to enlarge the applicability of the model we allow for covariates in the specification of both the mean and the covariance parameters. Extension to models with complete structure with many multi-way covariance terms is discussed. The method is demonstrated by analyzing a real life data set.     

A permutation-based Bayesian approach for inverse covariance estimation

Abstract

Covariance estimation and selection for multivariate datasets in a high-dimensional regime is a fundamental problem in modern statistics. Gaussian graphical models are a popular class of models used for this purpose. Current Bayesian methods for inverse covariance matrix estimation under Gaussian graphical models require the underlying graph and hence the ordering of variables to be known. However, in practice, such information on the true underlying model is often unavailable. We therefore propose a novel permutation-based Bayesian approach to tackle the unknown variable ordering issue. In particular, we utilize multiple maximum a posteriori estimates under the DAG-Wishart prior for each permutation, and subsequently construct the final estimate of the inverse covariance matrix. The proposed estimator has smaller variability and yields order-invariant property. We establish posterior convergence rates under mild assumptions and illustrate that our method outperforms existing approaches in estimating the inverse covariance matrices via simulation studies.     

Robust Bayesian synthetic likelihood via a semi-parametric approach

Bayesian synthetic likelihood (BSL) is now a well-established method for performing approximate Bayesian parameter estimation for simulation-based models that do not possess a tractable likelihood function. BSL approximates an intractable likelihood function of a carefully chosen summary statistic at a parameter value with a multivariate normal distribution. The mean and covariance matrix of this normal distribution are estimated from independent simulations of the model. Due to the parametric assumption implicit in BSL, it can be preferred to its nonparametric competitor, approximate Bayesian computation, in certain applications where a high-dimensional summary statistic is of interest. However, despite several successful applications of BSL, its widespread use in scientific fields may be hindered by the strong normality assumption. In this paper, we develop a semi-parametric approach to relax this assumption to an extent and maintain the computational advantages of BSL without any additional tuning. We test our new method, semiBSL, on several challenging examples involving simulated and real data and demonstrate that semiBSL can be significantly more robust than BSL and another approach in the literature.     

Robust estimation in simultaneous equations models

In this paper we review existing work on robust estimation for simultaneous equations models. Then we sketch three strategies for obtaining estimators with a high breakdown point and a controllable efficiency: (a) robustifying three-stage least squares, (b) robustifying the full information maximum likelihood method by minimizing the determinant of a robust covariance matrix of residuals, and (c) generalizing multivariate tau-estimators (Lopuhaä, 1992, Can. J. Statist., 19, 307–321) to these models. They have the same order of computational complexity as high breakdown point multivariate estimators. The latter seems the most promising approach.     

Accent on Teaching Materials

Estimation of covariance components in the multivariate random-effect model with nested covariance structure is discussed. There are two covariance matrices to be estimated, namely, the between-group and the within-group covariance matrices. These two covariance matrices are most often estimated by forming a multivariate analysis of variance and equating mean square matrices to their expectations. Such a procedure involves taking the difference between the between-group mean square and the within-group mean square matrices, and often produces an estimated between-group covariance matrix that is not nonnegative definite. We present estimators of the two covariance matrices that are always proper covariance matrices. The estimators are the restricted maximum likelihood estimators if the random effects are normally distributed. The estimation procedure is extended to more complicated models, including the twofold nested and the mixed-effect models. A numerical example is presented to illustrate the use of the estimation procedure.     

Robust estimation of mean and covariance for longitudinal data with dropouts

In this paper, we study estimation of linear models in the framework of longitudinal data with dropouts. Under the assumptions that random errors follow an elliptical distribution and all the subjects share the same within-subject covariance matrix which does not depend on covariates, we develop a robust method for simultaneous estimation of mean and covariance. The proposed method is robust against outliers, and does not require to model the covariance and missing data process. Theoretical properties of the proposed estimator are established and simulation studies show its good performance. In the end, the proposed method is applied to a real data analysis for illustration.     

Skewness Reduction Approach in Multi-Attribute Process Monitoring

Since the product quality of many industrial processes depends upon more than one dependent variable or attribute, they are either multivariate or multi-attribute in nature. Although multivariate statistical process control is receiving increased attention in the literature, little work has been done to deal with multi-attribute processes. In this article, we develop a new methodology to monitor multi-attribute processes. To do this, first we transform multi-attribute data in a way that their marginal probability distributions have almost zero skewness. Then, we estimate the transformed covariance matrix and apply the well-known T ² control chart. In order to illustrate the proposed method and evaluate its performance, we use two simulation experiments and compare the results with the ones from both MNP chart and the χ² control chart.     

Explaining the behavior of joint and marginal Monte Carlo estimators in latent variable models with independence assumptions

In latent variable models parameter estimation can be implemented by using the joint or the marginal likelihood, based on independence or conditional independence assumptions. The same dilemma occurs within the Bayesian framework with respect to the estimation of the Bayesian marginal (or integrated) likelihood, which is the main tool for model comparison and averaging. In most cases, the Bayesian marginal likelihood is a high dimensional integral that cannot be computed analytically and a plethora of methods based on Monte Carlo integration (MCI) are used for its estimation. In this work, it is shown that the joint MCI approach makes subtle use of the properties of the adopted model, leading to increased error and bias in finite settings. The sources and the components of the error associated with estimators under the two approaches are identified here and provided in exact forms. Additionally, the effect of the sample covariation on the Monte Carlo estimators is examined. In particular, even under independence assumptions the sample covariance will be close to (but not exactly) zero which surprisingly has a severe effect on the estimated values and their variability. To address this problem, an index of the sample’s divergence from independence is introduced as a multivariate extension of covariance. The implications addressed here are important in the majority of practical problems appearing in Bayesian inference of multi-parameter models with analogous structures.     

Diagnostics in multivariate generalized Birnbaum-Saunders regression models

Birnbaum–Saunders (BS) models are receiving considerable attention in the literature. Multivariate regression models are a useful tool of the multivariate analysis, which takes into account the correlation between variables. Diagnostic analysis is an important aspect to be considered in the statistical modeling. In this paper, we formulate multivariate generalized BS regression models and carry out a diagnostic analysis for these models. We consider the Mahalanobis distance as a global influence measure to detect multivariate outliers and use it for evaluating the adequacy of the distributional assumption. We also consider the local influence approach and study how a perturbation may impact on the estimation of model parameters. We implement the obtained results in the R software, which are illustrated with real-world multivariate data to show their potential applications.     

Estimation and inference of the joint conditional distribution for multivariate longitudinal data using nonparametric copulas

In this paper we study estimating the joint conditional distributions of multivariate longitudinal outcomes using regression models and copulas. For the estimation of marginal models, we consider a class of time-varying transformation models and combine the two marginal models using nonparametric empirical copulas. Our models and estimation method can be applied in many situations where the conditional mean-based models are not good enough. Empirical copulas combined with time-varying transformation models may allow quite flexible modelling for the joint conditional distributions for multivariate longitudinal data. We derive the asymptotic properties for the copula-based estimators of the joint conditional distribution functions. For illustration we apply our estimation method to an epidemiological study of childhood growth and blood pressure.     

A General Multivariate Threshold GARCH Model With Dynamic Conditional Correlations

We introduce a new multivariate GARCH model with multivariate thresholds in conditional correlations and develop a two-step estimation procedure that is feasible in large dimensional applications. Optimal threshold functions are estimated endogenously from the data and the model conditional covariance matrix is ensured to be positive definite. We study the empirical performance of our model in two applications using U.S. stock and bond market data. In both applications our model has, in terms of statistical and economic significance, higher forecasting power than several other multivariate GARCH models for conditional correlations.     

Time-Varying Systemic Risk: Evidence From a Dynamic Copula Model of CDS Spreads

This article proposes a new class of copula-based dynamic models for high-dimensional conditional distributions, facilitating the estimation of a wide variety of measures of systemic risk. Our proposed models draw on successful ideas from the literature on modeling high-dimensional covariance matrices and on recent work on models for general time-varying distributions. Our use of copula-based models enables the estimation of the joint model in stages, greatly reducing the computational burden. We use the proposed new models to study a collection of daily credit default swap (CDS) spreads on 100 U.S. firms over the period 2006 to 2012. We find that while the probability of distress for individual firms has greatly reduced since the financial crisis of 2008–2009, the joint probability of distress (a measure of systemic risk) is substantially higher now than in the precrisis period. Supplementary materials for this article are available online.     

Model-based clustering, classification, and discriminant analysis via mixtures of multivariate t-distributions

The last decade has seen an explosion of work on the use of mixture models for clustering. The use of the Gaussian mixture model has been common practice, with constraints sometimes imposed upon the component covariance matrices to give families of mixture models. Similar approaches have also been applied, albeit with less fecundity, to classification and discriminant analysis. In this paper, we begin with an introduction to model-based clustering and a succinct account of the state-of-the-art. We then put forth a novel family of mixture models wherein each component is modeled using a multivariate t-distribution with an eigen-decomposed covariance structure. This family, which is largely a t-analogue of the well-known MCLUST family, is known as the tEIGEN family. The efficacy of this family for clustering, classification, and discriminant analysis is illustrated with both real and simulated data. The performance of this family is compared to its Gaussian counterpart on three real data sets.     

Asymmetric matrix-valued covariances for multivariate random fields on spheres

ABSTRACT

Matrix-valued covariance functions are crucial to geostatistical modelling of multivariate spatial data. The classical assumption of symmetry of a multivariate covariance function is overly restrictive and has been considered as unrealistic for most of the real data applications. Despite of that, the literature on asymmetric covariance functions has been very sparse. In particular, there is some work related to asymmetric covariances on Euclidean spaces, depending on the Euclidean distance. However, for data collected over large portions of planet Earth, the most natural spatial domain is a sphere, with the corresponding geodesic distance being the natural metric. In this work, we propose a strategy based on spatial rotations to generate asymmetric covariances for multivariate random fields on the d-dimensional unit sphere. We illustrate through simulations as well as real data analysis that our proposal allows to achieve improvements in the predictive performance in comparison to the symmetric counterpart.     

Inference in multivariate linear regression models with elliptically distributed errors

In this study we investigate the problem of estimation and testing of hypotheses in multivariate linear regression models when the errors involved are assumed to be non-normally distributed. We consider the class of heavy-tailed distributions for this purpose. Although our method is applicable for any distribution in this class, we take the multivariate t-distribution for illustration. This distribution has applications in many fields of applied research such as Economics, Business, and Finance. For estimation purpose, we use the modified maximum likelihood method in order to get the so-called modified maximum likelihood estimates that are obtained in a closed form. We show that these estimates are substantially more efficient than least-square estimates. They are also found to be robust to reasonable deviations from the assumed distribution and also many data anomalies such as the presence of outliers in the sample, etc. We further provide test statistics for testing the relevant hypothesis regarding the regression coefficients.     

New HEAVY Models for Fat-Tailed Realized Covariances and Returns

ABSTRACT

We develop a new score-driven model for the joint dynamics of fat-tailed realized covariance matrix observations and daily returns. The score dynamics for the unobserved true covariance matrix are robust to outliers and incidental large observations in both types of data by assuming a matrix-F distribution for the realized covariance measures and a multivariate Student's t distribution for the daily returns. The filter for the unknown covariance matrix has a computationally efficient matrix formulation, which proves beneficial for estimation and simulation purposes. We formulate parameter restrictions for stationarity and positive definiteness. Our simulation study shows that the new model is able to deal with high-dimensional settings (50 or more) and captures unobserved volatility dynamics even if the model is misspecified. We provide an empirical application to daily equity returns and realized covariance matrices up to 30 dimensions. The model statistically and economically outperforms competing multivariate volatility models out-of-sample. Supplementary materials for this article are available online.     

Cholesky-GARCH models with applications to finance

Instantaneous dependence among several asset returns is the main reason for the computational and statistical complexities in working with full multivariate GARCH models. Using the Cholesky decomposition of the covariance matrix of such returns, we introduce a broad class of multivariate models where univariate GARCH models are used for variances of individual assets and parsimonious models for the time-varying unit lower triangular matrices. This approach, while reducing the number of parameters and severity of the positive-definiteness constraint, has several advantages compared to the traditional orthogonal and related GARCH models. Its major drawback is the potential need for an a priori ordering or grouping of the stocks in a portfolio, which through a case study we show can be taken advantage of so far as reducing the forecast error of the volatilities and the dimension of the parameter space are concerned. Moreover, the Cholesky decomposition, unlike its competitors, decompose the normal likelihood function as a product of univariate normal likelihoods with independent parameters, resulting in fast estimation algorithms. Gaussian maximum likelihood methods of estimation of the parameters are developed. The methodology is implemented for a real financial dataset with seven assets, and its forecasting power is compared with other existing models.     

Parametric Inference in Stationary Time Series Models with Dependent Errors

This article is concerned with inference for the parameter vector in stationary time series models based on the frequency domain maximum likelihood estimator. The traditional method consistently estimates the asymptotic covariance matrix of the parameter estimator and usually assumes the independence of the innovation process. For dependent innovations, the asymptotic covariance matrix of the estimator depends on the fourth‐order cumulants of the unobserved innovation process, a consistent estimation of which is a difficult task. In this article, we propose a novel self‐normalization‐based approach to constructing a confidence region for the parameter vector in such models. The proposed procedure involves no smoothing parameter, and is widely applicable to a large class of long/short memory time series models with weakly dependent innovations. In simulation studies, we demonstrate favourable finite sample performance of our method in comparison with the traditional method and a residual block bootstrap approach.     

Orthogonal Stiefel manifold optimization for eigen-decomposed covariance parameter estimation in mixture models

Within the mixture model-based clustering literature, parsimonious models with eigen-decomposed component covariance matrices have dominated for over a decade. Although originally introduced as a fourteen-member family of models, the current state-of-the-art is to utilize just ten of these models; the rationale for not using the other four models usually centers around parameter estimation difficulties. Following close examination of these four models, we find that two are actually easily implemented using existing algorithms but that two benefit from a novel approach. We present and implement algorithms that use an accelerated line search for optimization on the orthogonal Stiefel manifold. Furthermore, we show that the ‘extra’ models that these decompositions facilitate outperform the current state-of-the art when applied to two benchmark data sets.     

Bayesian analysis of correlated mixed categorical data by incorporating historical prior information

In this article, we develop statistical models for analysis of correlated mixed categorical (binary and ordinal) response data arising in medical and epidemi-ologic studies. There is evidence in the literature to suggest that models including correlation structure can lead to substantial improvement in precision of estimation or are more appropriate (accurate). We use a very rich class of scale mixture of multivariate normal (SMMVN) iink functions to accommodate heavy tailed distributions. In order to incorporate available historical information, we propose a unified prior elicitation scheme based on SMMVN-link models. Further, simulation-based techniques are developed to assess model adequacy. Finally, a real data example from prostate cancer studies is used to illustrate the proposed methodologies.