Strict Positive Definiteness of a Product of Covariance Functions

Positive definiteness represents an admissibility condition for a function to be a covariance. Nevertheless, the more restricted condition of strict positive definiteness has received attention in literature, especially in spatial statistics, since it ensures that the kriging system has a unique solution. Most known covariance functions are isotropic but there are applications where isotropy is not appropriate, e.g., space-time covariance functions. One way to construct non-isotropic covariance functions is to use a product or a product-sum. In this article, it is given a necessary as well as a sufficient condition for a product of two covariance functions to be strictly positive definite. This result is extended to the well-known product-sum covariance model.     

Construction of non-Gaussian random fields with any given correlation structure

A positive definite function can be thought of as the covariance function of a Gaussian random field, according to the celebrated Kolmogorov existence theorem. A question of great theoretical and practical interest is: how could one construct a non-Gaussian random field with the given positive definite function as its covariance function? In this paper we demonstrate a novel and simple method for constructing many such non-Gaussian random fields, with the corresponding finite-dimensional distributions identified. Also, we show how to construct a non-Gaussian random field with a given negative definite function as its variogram.     

Bayesian tests on components of the compound symmetry covariance matrix

Complex dependency structures are often conditionally modeled, where random effects parameters are used to specify the natural heterogeneity in the population. When interest is focused on the dependency structure, inferences can be made from a complex covariance matrix using a marginal modeling approach. In this marginal modeling framework, testing covariance parameters is not a boundary problem. Bayesian tests on covariance parameter(s) of the compound symmetry structure are proposed assuming multivariate normally distributed observations. Innovative proper prior distributions are introduced for the covariance components such that the positive definiteness of the (compound symmetry) covariance matrix is ensured. Furthermore, it is shown that the proposed priors on the covariance parameters lead to a balanced Bayes factor, in case of testing an inequality constrained hypothesis. As an illustration, the proposed Bayes factor is used for testing (non-)invariant intra-class correlations across different group types (public and Catholic schools), using the 1982 High School and Beyond survey data.     

Multivariate Positive Dependence on the Simplex

The Liouville and Generalized Liouville families have been proposed as parametric models for data constrained to the simplex. These families have generated practical interest owing primarily to inadequacies, such as a completely negative covariance structure, that are inherent in the better-known Dirichlet class. Although there is some numerical evidence suggesting that the Liouville and Generalized Liouville families can produce completely positive and mixed covariance structures, no general paradigms have been developed. Research toward this end might naturally be focused on the many classical "positive dependence" concepts available in the literature, all of which imply a nonnegative covariance structure. However, in this article it is shown that no strictly positive distribution on the simplex can possess any of these classical dependence properties. The same result holds for Liouville and generalized Liouville distributions even if the condition of strict positivity is relaxed.     

Superiority comparisons of homogeneous linear estimators

In the paper homogeneous linear estimators of the parameter vector of the general linear model are compared in terms of their MSE matrices. A necessary and sufficient condition for the difference of two MSE matrices to be positive definite is obtained and its practical existence discussed. The non-negative definiteness of the difference also receives attention, and conditions for this case are discussed. The absence of any conditions of the above type is taken into consideration as well.     

The covariance of the backward and forward recurrence times in a renewal process: the stationary case and asymptotics for the ordinary case

In a Poisson process, it is well-known that the forward and backward recurrence times at a given time point t are independent random variables. In a renewal process, although the joint distribution of these quantities is known (asymptotically), it seems that very few results regarding their covariance function exist. In the present paper, we study this covariance and, in particular, we state both necessary and sufficient conditions for it to be positive, zero or negative in terms of reliability classifications and the coefficient of variation of the underlying inter-renewal and the associated equilibrium distribution. Our results apply either for an ordinary renewal process in the steady state or for a stationary process.     

Space–time correlation analysis: a comparative study

Space–time correlation modelling is one of the crucial steps of traditional structural analysis, since space–time models are used for prediction purposes. A comparative study among some classes of space–time covariance functions is proposed. The relevance of choosing a suitable model by taking into account the characteristic behaviour of the models is proved by using a space–time data set of ozone daily averages and the flexibility of the product-sum model is also highlighted through simulated data sets.     

An evaluation of a non-parametric method of estimating semi-variograms of isotropic spatial processes

Semi-variograms are useful for describing the correlation strucure of spatial random variables. Valid semi-variograms must be conditionally negative definite. To ensure this restriction when estimating these functions, a valid parametric model is typically fitted to a sample semi-variogram. Recently, a method of fitting valid semi-variograms without having to choose a parametric family has been described in the literature. The method is based on the spectral representation of positive definite functions. In this paper, the method is evaluated using simulated data. The fits obtained using the non-parametric method are compared with fits obtained by fitting four parametric models (exponential, Gaussian, rational quadratic and power) to simulated data using non-linear least squares. The comparisons are based on the integrated squared errors of the resulting fits. The non-parametric estimator always resulted in fits that were as good as those obtained using the parametric models. The non-parametric method is faster, easier to use and more objective than the parametric methods. Some examples are presented.     

Covariance Matrix Estimation via Network Structure

In this article, we employ a regression formulation to estimate the high-dimensional covariance matrix for a given network structure. Using prior information contained in the network relationships, we model the covariance as a polynomial function of the symmetric adjacency matrix. Accordingly, the problem of estimating a high-dimensional covariance matrix is converted to one of estimating low dimensional coefficients of the polynomial regression function, which we can accomplish using ordinary least squares or maximum likelihood. The resulting covariance matrix estimator based on the maximum likelihood approach is guaranteed to be positive definite even in finite samples. Under mild conditions, we obtain the theoretical properties of the resulting estimators. A Bayesian information criterion is also developed to select the order of the polynomial function. Simulation studies and empirical examples illustrate the usefulness of the proposed methods.     

Simulation of stationary Gaussian vector fields

In earlier work we described a circulant embedding approach for simulating scalar-valued stationary Gaussian random fields on a finite rectangular grid, with the covariance function prescribed. Here, we explain how the circulant embedding approach can be used to simulate Gaussian vector fields. As in the scalar case, the simulation procedure is theoretically exact if a certain non-negativity condition is satisfied. In the vector setting, this exactness condition takes the form of a nonnegative definiteness condition on a certain set of Hermitian matrices. The main computational tool used is the Fast Fourier Transform. Consequently, when implemented appropriately, the procedure is highly efficient, in terms of both CPU time and storage.     

A regression approach for estimating the parameters of the covariance function of a stationary spatial random process

We consider the problem of estimating the parameters of the covariance function of a stationary spatial random process. In spatial statistics, there are widely used parametric forms for the covariance functions, and various methods for estimating the parameters have been proposed in the literature. We develop a method for estimating the parameters of the covariance function that is based on a regression approach. Our method utilizes pairs of observations whose distances are closest to a value h>0$h>0$ which is chosen in a way that the estimated correlation at distance h is a predetermined value. We demonstrate the effectiveness of our procedure by simulation studies and an application to a water pH data set. Simulation studies show that our method outperforms all well-known least squares-based approaches to the variogram estimation and is comparable to the maximum likelihood estimation of the parameters of the covariance function. We also show that under a mixing condition on the random field, the proposed estimator is consistent for standard one parameter models for stationary correlation functions.     

Admissibilities of matrix linear estimators multivariate linear models

This article respectively provides sufficient conditions and necessary conditions of matrix linear estimators of an estimable parameter matrix linear function in multivariate linear models with and without the assumption that the underlying distribution is a normal one with completely unknown covariance matrix. In the latter model, a necessary and sufficient condition is given for matrix linear estimators to be admissible in the space of all matrix linear estimators under each of three different kinds of quadratic matrix loss functions, respectively. In the former model, a sufficient condition is first provided for matrix linear estimators to be admissible in the space of all matrix estimators having finite risks under each of the same loss functions, respectively. Furthermore in the former model, one of these sufficient conditions, correspondingly under one of the loss functions, is also proved to be necessary, if additional conditions are assumed.     

Unconstrained parametrizations for variance-covariance matrices

The estimation of variance-covariance matrices through optimization of an objective function, such as a log-likelihood function, is usually a difficult numerical problem. Since the estimates should be positive semi-definite matrices, we must use constrained optimization, or employ a parametrization that enforces this condition. We describe here five different parametrizations for variance-covariance matrices that ensure positive definiteness, thus leaving the estimation problem unconstrained. We compare the parametrizations based on their computational efficiency and statistical interpretability. The results described here are particularly useful in maximum likelihood and restricted maximum likelihood estimation in linear and non-linear mixed-effects models, but are also applicable to other areas of statistics.     

Adaptive robust estimation in joint mean–covariance regression model for bivariate longitudinal data

The estimation of the covariance matrix is important in the analysis of bivariate longitudinal data. A good estimator for the covariance matrix can improve the efficiency of the estimators of the mean regression coefficients. Furthermore, the covariance estimation itself is also of interest, but it is a challenging job to model the covariance matrix of bivariate longitudinal data due to the complex structure and positive definite constraint. In addition, most of existing approaches are based on the maximum likelihood, which is very sensitive to outliers or heavy-tail error distributions. In this article, an adaptive robust estimation method is proposed for bivariate longitudinal data. Unlike the existing likelihood-based methods, the proposed method can adapt to different error distributions. Specifically, at first, we utilize the modified Cholesky block decomposition to parameterize the covariance matrices. Secondly, we apply the bounded Huber's score function to develop a set of robust generalized estimating equations to estimate the parameters both in the mean and the covariance models simultaneously. A data-driven approach is presented to select the parameter c in the Huber's score function, which can ensure that the proposed method is robust and efficient. A simulation study and a real data analysis are conducted to illustrate the robustness and efficiency of the proposed approach.     

A Note on Sufficient Conditions for Valid Unmodified t Testing in Correlation Analysis with Autocorrelated and Heteroscedastic Sample Data

Traditionally, sphericity (i.e., independence and homoscedasticity for raw data) is put forward as the condition to be satisfied by the variance–covariance matrix of at least one of the two observation vectors analyzed for correlation, for the unmodified t test of significance to be valid under the Gaussian and constant population mean assumptions. In this article, the author proves that the sphericity condition is too strong and a weaker (i.e., more general) sufficient condition for valid unmodified t testing in correlation analysis is circularity (i.e., independence and homoscedasticity after linear transformation by orthonormal contrasts), to be satisfied by the variance–covariance matrix of one of the two observation vectors. Two other conditions (i.e., compound symmetry for one of the two observation vectors; absence of correlation between the components of one observation vector, combined with a particular pattern of joint heteroscedasticity in the two observation vectors) are also considered and discussed. When both observation vectors possess the same variance–covariance matrix up to a positive multiplicative constant, the circularity condition is shown to be necessary and sufficient. “Observation vectors” may designate partial realizations of temporal or spatial stochastic processes as well as profile vectors of repeated measures. From the proof, it follows that an effective sample size appropriately defined can measure the discrepancy from the more general sufficient condition for valid unmodified t testing in correlation analysis with autocorrelated and heteroscedastic sample data. The proof is complemented by a simulation study. Finally, the differences between the role of the circularity condition in the correlation analysis and its role in the repeated measures ANOVA (i.e., where it was first introduced) are scrutinized, and the link between the circular variance–covariance structure and the centering of observations with respect to the sample mean is emphasized.     

Analysis of the Covariance Structure in Manufactured Parts

We discuss the use of variograms for covariance modeling under the Kriging model to assess tolerances on manufactured parts. The variogram is very informative about the spatial dependence and it is favored by researchers in the choice of a correlation function. It may give evidence of anisotropy and of nugget effect too. In this paper, various variogram estimators from literature are used for comparing and evaluating their properties in assessing the surface error of three workpieces manufactured with different machining precisions. Contrary to the common engineering belief, the variograms give evidence of both technological signatures and systematic errors of the measurement machines.     

A Note on the Modified Likelihood Ratio Test for Homogeneity in Beta Mixture Models

This article presents a note on the modified likelihood ratio test for homogeneity in beta mixture models. Under consistency of the penalized maximum likelihood estimators, the limiting distribution of the test statistic converges to the chi-bar-squared distributions. The statistic degenerates to zero with a weight due to the negative definiteness of a complicated random matrix. The probability that this matrix is negative definite is related to the parameter values under the homogeneity hypothesis. The dependency pattern enables the introduction of an upper bound on the asymptotic null distribution. Simulation study is investigated to verify the accuracy of the results.     

A Class of Convolution‐Based Models for Spatio‐Temporal Processes with Non‐Separable Covariance Structure

Abstract. In this article, we propose a new parametric family of models for real‐valued spatio‐temporal stochastic processes S ( x , t ) and show how low‐rank approximations can be used to overcome the computational problems that arise in fitting the proposed class of models to large datasets. Separable covariance models, in which the spatio‐temporal covariance function of S ( x , t ) factorizes into a product of purely spatial and purely temporal functions, are often used as a convenient working assumption but are too inflexible to cover the range of covariance structures encountered in applications. We define positive and negative non‐separability and show that in our proposed family we can capture positive, zero and negative non‐separability by varying the value of a single parameter.     

On Necessary and Sufficient Conditions for Ordinary Least Squares Estimators to Be Best Linear Unbiased Estimators

Two often-quoted necessary and sufficient conditions for ordinary least squares estimators to be best linear unbiased estimators are described. Another necessary and sufficient condition is described, providing an additional tool for checking to see whether the covariance matrix of a given linear model is such that the ordinary least squares estimator is also the best linear unbiased estimator. The new condition is used to show that one of the two published conditions is only a sufficient condition.     

A multivariate volatility vine copula model

This article proposes a dynamic framework for modeling and forecasting of realized covariance matrices using vine copulas to allow for more flexible dependencies between assets. Our model automatically guarantees positive definiteness of the forecast through the use of a Cholesky decomposition of the realized covariance matrix. We explicitly account for long-memory behavior by using fractionally integrated autoregressive moving average (ARFIMA) and heterogeneous autoregressive (HAR) models for the individual elements of the decomposition. Furthermore, our model incorporates non-Gaussian innovations and GARCH effects, accounting for volatility clustering and unconditional kurtosis. The dependence structure between assets is studied using vine copula constructions, which allow for nonlinearity and asymmetry without suffering from an inflexible tail behavior or symmetry restrictions as in conventional multivariate models. Further, the copulas have a direct impact on the point forecasts of the realized covariances matrices, due to being computed as a nonlinear transformation of the forecasts for the Cholesky matrix. Beside studying in-sample properties, we assess the usefulness of our method in a one-day-ahead forecasting framework, comparing recent types of models for the realized covariance matrix based on a model confidence set approach. Additionally, we find that in Value-at-Risk (VaR) forecasting, vine models require less capital requirements due to smoother and more accurate forecasts.