A monte carlo study on two methods of calculating the mle's covariance matrix in a seemingly unrelated nonlinear regression. *

Econometric techniques to estimate output supply systems, factor demand systems and consumer demand systems have often required estimating a nonlinear system of equations that have an additive error structure when written in reduced form. To calculate the ML estimate's covariance matrix of this nonlinear system one can either invert the Hessian of the concentrated log likelihood function, or invert the matrix calculated by pre-multiplying and post multiplying the inverted MLE of the disturbance covariance matrix by the Jacobian of the reduced form model. Malinvaud has shown that the latter of these methods is the actual limiting distribution's covariance matrix, while Barnett has shown that the former is only an approximation.

In this paper, we use a Monte Carlo simulation study to determine how these two covariance matrices differ with respect to the nonlinearity of the model, the number of observations in the dataet, and the residual process. We find that the covariance matrix calculated from the Hessian of the concentrated likelihood function produces Wald statistics that are distributed above those calculated with the other covariance matrix. This difference becomes insignificant as the sample size increases to one-hundred or more observations, suggesting that the asymptotics of the two covariance matrices are quickly reached.     

A monte carlo study on two methods of calculating the mle's covariance matrix in a seemingly unrelated nonlinear regression.

Econometric techniques to estimate output supply systems, factor demand systems and consumer demand systems have often required estimating a nonlinear system of equations that have an additive error structure when written in reduced form. To calculate the ML estimate's covariance matrix of this nonlinear system one can either invert the Hessian of the concentrated log likelihood function, or invert the matrix calculated by pre-multiplying and post multiplying the inverted MLE of the disturbance covariance matrix by the Jacobian of the reduced form model. Malinvaud has shown that the latter of these methods is the actual limiting distribution's covariance matrix, while Barnett has shown that the former is only an approximation.

In this paper, we use a Monte Carlo simulation study to determine how these two covariance matrices differ with respect to the nonlinearity of the model, the number of observations in the dataet, and the residual process. We find that the covariance matrix calculated from the Hessian of the concentrated likelihood function produces Wald statistics that are distributed above those calculated with the other covariance matrix. This difference becomes insignificant as the sample size increases to one-hundred or more observations, suggesting that the asymptotics of the two covariance matrices are quickly reached.     

Modeling Conditional Covariances With Economic Information Instruments

We propose a new model for conditional covariances based on predetermined idiosyncratic shocks as well as macroeconomic and own information instruments. The specification ensures positive definiteness by construction, is unique within the class of linear functions for our covariance decomposition, and yields a simple yet rich model of covariances. We introduce a property, invariance to variate order, that assures estimation is not impacted by a simple reordering of the variates in the system. Simulation results using realized covariances show smaller mean absolute errors (MAE) and root mean square errors (RMSE) for every element of the covariance matrix relative to a comparably specified BEKK model with own information instruments. We also find a smaller mean absolute percentage error (MAPE) and root mean square percentage error (RMSPE) for the entire covariance matrix. Supplementary materials for practitioners as well as all Matlab code used in the article are available online.     

Nonparametric vector autoregression

We consider a vector conditional heteroscedastic autoregressive nonlinear (CHARN) model in which both the conditional mean and the conditional variance (volatility) matrix are unknown functions of the past. Nonparametric estimators of these functions are constructed based on local polynomial fitting. We examine the rates of convergence of these estimators and give a result on their asymptotic normality. These results are applied to estimation of volatility matrices in foreign exchange markets. Estimation of the conditional covariance surface for the Deutsche Mark/US Dollar (DEM/USD) and Deutsche Mark/British Pound (DEM/GBP) daily returns show negative correlation when the two series have opposite lagged values and positive correlation elsewhere. The relation of our findings to the capital asset pricing model is discussed.     

Selection of Multivariate Stochastic Volatility Models via Bayesian Stochastic Search

We propose a Bayesian stochastic search approach to selecting restrictions on multivariate regression models where the errors exhibit deterministic or stochastic conditional volatilities. We develop a Markov chain Monte Carlo (MCMC) algorithm that generates posterior restrictions on the regression coefficients and Cholesky decompositions of the covariance matrix of the errors. Numerical simulations with artificially generated data show that the proposed method is effective in selecting the data-generating model restrictions and improving the forecasting performance of the model. Applying the method to daily foreign exchange rate data, we conduct stochastic search on a VAR model with stochastic conditional volatilities.     

Singular Gaussian graphical models: Structure learning

ABSTRACT

The goal of this article is to introduce singular Gaussian graphical models and their conditional independence properties. In fact, we extend the concept of Gaussian Markov Random Field to the case of a multivariate normally distributed vector with a singular covariance matrix. We construct, then, the associated graph’s structure from the covariance matrix’s pseudo-inverse on the basis of a characterization of the pairwise conditional independence. The proposed approach can also be used when the covariance matrix is ill-conditioned, through projecting data on a smaller subspace. In this case, our method ensures numerical stability and consistency of the constructed graph and significantly reduces the inference problem’s complexity. These aspects are illustrated using numerical experiments.     

Basic structure of the asymptotic theory in dynamic nonlinear econometric models

This is the second of two papers that provide an expository discussion of the basic structure of the asymptotic theory of M-estimators in dynamic nonlinear models and a review of the literature. The first paper, Pötscher and Prucha(1991), deals with consistency. In the present paper we discuss asymptotic normality. As an important ingredient to the asymptotic normality proof in dynamic nonlinear models we consider central limit theorems for dependent random variables. We also discuss the estimation of the variance covariance matrix of m-estimators under heteroscedasticity and autocorrelation.     

Effect of heteroscedasticity between treatment groups on mixed‐effects models for repeated measures

Mixed‐effects models for repeated measures (MMRM) analyses using the Kenward‐Roger method for adjusting standard errors and degrees of freedom in an “unstructured” (UN) covariance structure are increasingly becoming common in primary analyses for group comparisons in longitudinal clinical trials. We evaluate the performance of an MMRM‐UN analysis using the Kenward‐Roger method when the variance of outcome between treatment groups is unequal. In addition, we provide alternative approaches for valid inferences in the MMRM analysis framework. Two simulations are conducted in cases with (1) unequal variance but equal correlation between the treatment groups and (2) unequal variance and unequal correlation between the groups. Our results in the first simulation indicate that MMRM‐UN analysis using the Kenward‐Roger method based on a common covariance matrix for the groups yields notably poor coverage probability (CP) with confidence intervals for the treatment effect when both the variance and the sample size between the groups are disparate. In addition, even when the randomization ratio is 1:1, the CP will fall seriously below the nominal confidence level if a treatment group with a large dropout proportion has a larger variance. Mixed‐effects models for repeated measures analysis with the Mancl and DeRouen covariance estimator shows relatively better performance than the traditional MMRM‐UN analysis method. In the second simulation, the traditional MMRM‐UN analysis leads to bias of the treatment effect and yields notably poor CP. Mixed‐effects models for repeated measures analysis fitting separate UN covariance structures for each group provides an unbiased estimate of the treatment effect and an acceptable CP. We do not recommend MMRM‐UN analysis using the Kenward‐Roger method based on a common covariance matrix for treatment groups, although it is frequently seen in applications, when heteroscedasticity between the groups is apparent in incomplete longitudinal data.     

Specification of household engel curves by nonparametric regression

This paper demonstrates the usefulness of nonparametric regression analysis for functional specfication of houshold Engel curves.

After a brief review in section 2 of the literature on demand functions and equivalence scales and the functional specifications used, we first discuss in section 3 the issues of using income versus total expenditure, the origin and nature of the error terms in the light of utility theroy, and the interpretation of empirical demand functions. we shall reach the unorthodox view that household demand functions should be interpreted as conditional expectations relative to prices, household composition and either income or the conditional expectation of total expenditure (rather that total expenditure itself), where the latter conditional expectation is taken relative to income, prices and household composition. these two forms appear to be equivalent. this result also solves the simultaneity problem: the error variance matrix is no longer singular. Moreover, the errors are in general heteroskedastic.

In section 4 we discuss the model and the data, and in section 5 we review the nonparametric kernal regression approach.

In section 6 we derive the functional form of our household engel curves from nonparametric regression results, using the 1980 budget survey for the netherlands, in order to avoid model misspecification. thus the modl is derived directly from the data, without restricting its functional form. the nonparametric regression results are then translated to suitable parametric functional specifications, i.e., we choose parametric functional forms in accordance with the nanparametric regression results. these parametric specification are estimated by least squares, and various parameter restrictions are tested in order to simplify the models. this yields very simple final specifications of the household engel curves involved, namely linear functions of income and the number of children in two age groups.     

Specification of household engel curves by nonparametric regression

This paper demonstrates the usefulness of nonparametric regression analysis for functional specfication of houshold Engel curves.

After a brief review in section 2 of the literature on demand functions and equivalence scales and the functional specifications used, we first discuss in section 3 the issues of using income versus total expenditure, the origin and nature of the error terms in the light of utility theroy, and the interpretation of empirical demand functions. we shall reach the unorthodox view that household demand functions should be interpreted as conditional expectations relative to prices, household composition and either income or the conditional expectation of total expenditure (rather that total expenditure itself), where the latter conditional expectation is taken relative to income, prices and household composition. these two forms appear to be equivalent. this result also solves the simultaneity problem: the error variance matrix is no longer singular. Moreover, the errors are in general heteroskedastic.

In section 4 we discuss the model and the data, and in section 5 we review the nonparametric kernal regression approach.

In section 6 we derive the functional form of our household engel curves from nonparametric regression results, using the 1980 budget survey for the netherlands, in order to avoid model misspecification. thus the modl is derived directly from the data, without restricting its functional form. the nonparametric regression results are then translated to suitable parametric functional specifications, i.e., we choose parametric functional forms in accordance with the nanparametric regression results. these parametric specification are estimated by least squares, and various parameter restrictions are tested in order to simplify the models. this yields very simple final specifications of the household engel curves involved, namely linear functions of income and the number of children in two age groups.     

Regression Properties for Asymmetric Generalized Scale Mixtures of Multivariate Gaussian Variables

Analytical properties of regression and the variance–covariance matrix of asymmetric generalized scale mixture of multivariate Gaussian variables are presented. The analysis includes an in-depth analytical investigation of the first two conditional moments of the mixing variable. Exact computable expressions for the prediction and the conditional variance are presented for the generalized hyperbolic distribution using the inversion theorem for Fourier transforms. An application to financial log returns is demonstrated via the classical Euler approximation. The methodology is illustrated by analyzing the regression of intraday log returns for CISCO against the corresponding data from S&P 500.     

A bivariate distribution with gamma and beta marginals with application to drought data

The first known bivariate distribution with gamma and beta marginals is introduced. Various representations are derived for its joint probability density function (pdf), joint cumulative distribution function (cdf), product moments, conditional pdfs, conditional cdfs, conditional moments, joint moment generating function, joint characteristic function and entropies. The method of maximum likelihood and the method of moments are used to derive the associated estimation procedures as well as the Fisher information matrix, variance–covariance matrix and the profile likelihood confidence intervals. An application to drought data from Nebraska is provided. Some other applications are also discussed. Finally, an extension of the bivariate distribution to the multivariate case is proposed.     

Probabilities that contaminated normal observations are extreme

Results are presented for the probability that a contaminated observation in a normal sample isan outlier. Univariate samples with mean-shift or variance inflation contamination were considered. Multivariate samples with inflation of the covariance matrix in which an outlier is the observation with minimum conditional predictive ordinate were also studied. All the results were obtained by numerical integration or simulation.     

Linear model estimation errors with estimated design parameters

The problem of error estimation of parameters b in a linear model,Y = Xb+ e, is considered when the elements of the design matrix X are functions of an unknown ‘design’ parameter vector c. An estimated value c is substituted in X to obtain a derived design matrix [Xtilde]. Even though the usual linear model conditions are not satisfied with [Xtilde], there are situations in physical applications where the least squares solution to the parameters is used without concern for the magnitude of the resulting error. Such a solution can suffer from serious errors.

This paper examines bias and covariance errors of such estimators. Using a first-order Taylor series expansion, we derive approximations to the bias and covariance matrix of the estimated parameters. The bias approximation is a sum of two terms:One is due to the dependence between ? and Y; the other is due to the estimation errors of ? and is proportional to b, the parameter being estimated. The covariance matrix approximation, on the other hand, is composed of three omponents:One component is due to the dependence between ? and Y; the second is the covariance matrix ∑_b corresponding to the minimum variance unbiased b, as if the design parameters were known without error; and the third is an additional component due to the errors in the design parameters. It is shown that the third error component is directly proportional to bb'. Thus, estimation of large parameters with wrong design matrix [Xtilde] will have larger errors of estimation. The results are illustrated with a simple linear example.     

Approximate inference in heteroskedastic regressions: A numerical evaluation

The commonly made assumption that all stochastic error terms in the linear regression model share the same variance (homoskedasticity) is oftentimes violated in practical applications, especially when they are based on cross-sectional data. As a precaution, a number of practitioners choose to base inference on the parameters that index the model on tests whose statistics employ asymptotically correct standard errors, i.e. standard errors that are asymptotically valid whether or not the errors are homoskedastic. In this paper, we use numerical integration methods to evaluate the finite-sample performance of tests based on different (alternative) heteroskedasticity-consistent standard errors. Emphasis is placed on a few recently proposed heteroskedasticity-consistent covariance matrix estimators. Overall, the results favor the HC4 and HC5 heteroskedasticity-robust standard errors. We also consider the use of restricted residuals when constructing asymptotically valid standard errors. Our results show that the only test that clearly benefits from such a strategy is the HC0 test.     

Bayesian analysis of vector-autoregressive models with noninformative priors

In this paper, we investigate the properties of Bayes estimators of vector autoregression (VAR) coefficients and the covariance matrix under two commonly employed loss functions. We point out that the posterior mean of the variances of the VAR errors under the Jeffreys prior is likely to have an over-estimation bias. Our Bayesian computation results indicate that estimates using the constant prior on the VAR regression coefficients and the reference prior of Yang and Berger (Ann. Statist. 22 (1994) 1195) on the covariance matrix dominate the constant-Jeffreys prior estimates commonly used in applications of VAR models in macroeconomics. We also estimate a VAR model of consumption growth using both constant-reference and constant-Jeffreys priors.     

The auto-regression and the moving-average

We explore some relationships in the second-order properties of a causal auto-regression and an invertible moving-average process with the same polynomial. We reveal that the inverse variance matrix for random variables from the auto-regression is equal to a conditional variance matrix of Gaussian random variables from the moving-average and vice versa. While the inverse variance matrix for the auto-regression can be written explicitly, we manage to write down the exact Gaussian likelihood of consecutive observations from the moving-average process, by using the properties of the auto-regression.     

A new heteroskedasticity-consistent covariance matrix estimator for the linear regression model

The assumption that all random errors in the linear regression model share the same variance (homoskedasticity) is often violated in practice. The ordinary least squares estimator of the vector of regression parameters remains unbiased, consistent and asymptotically normal under unequal error variances. Many practitioners then choose to base their inferences on such an estimator. The usual practice is to couple it with an asymptotically valid estimation of its covariance matrix, and then carry out hypothesis tests that are valid under heteroskedasticity of unknown form. We use numerical integration methods to compute the exact null distributions of some quasi-t test statistics, and propose a new covariance matrix estimator. The numerical results favor testing inference based on the estimator we propose.     

Accounting for Uncertainty in Heteroscedasticity in Nonlinear Regression

Toxicologists and pharmacologists often describe toxicity of a chemical using parameters of a nonlinear regression model. Thus estimation of parameters of a nonlinear regression model is an important problem. The estimates of the parameters and their uncertainty estimates depend upon the underlying error variance structure in the model. Typically, a priori the researcher would not know if the error variances are homoscedastic (i.e., constant across dose) or if they are heteroscedastic (i.e., the variance is a function of dose). Motivated by this concern, in this paper we introduce an estimation procedure based on preliminary test which selects an appropriate estimation procedure accounting for the underlying error variance structure. Since outliers and influential observations are common in toxicological data, the proposed methodology uses M-estimators. The asymptotic properties of the preliminary test estimator are investigated; in particular its asymptotic covariance matrix is derived. The performance of the proposed estimator is compared with several standard estimators using simulation studies. The proposed methodology is also illustrated using a data set obtained from the National Toxicology Program.     

Robust bootstrap densities for dynamic conditional correlations: implications for portfolio selection and Value-at-Risk

ABSTRACT

Many financial decisions such as portfolio allocation, risk management, option pricing and hedge strategies are based on the forecast of the conditional variances, covariances and correlations of financial returns. Although the decisions depend on the forecasts covariance matrix little is known about effects of outliers on the uncertainty associated with these forecasts. In this paper we analyse these effects on the context of dynamic conditional correlation models when the uncertainty is measured using bootstrap methods. We also propose a bootstrap procedure to obtain forecast densities for return, volatilities, conditional correlation and Value-at-Risk that is robust to outliers. The results are illustrated with simulated and real data.