Bayesian analysis in multivariate regression models with conjugate priors

In this paper, we consider the full rank multivariate regression model with matrix elliptically contoured distributed errors. We formulate a conjugate prior distribution for matrix elliptical models and derive the posterior distributions of mean and scale matrices. In the sequel, some characteristics of regression matrix parameters are also proposed.     

On local influence for elliptical linear models

The local influence method plays an important role in regression diagnostics and sensitivity analysis. To implement it, we need the Delta matrix for the underlying scheme of perturbations, in addition to the observed information matrix under the postulated model. Galea, Paula and Bolfarine (1997) has recently given the observed information matrix and the Delta matrix for a scheme of scale perturbations and has assessed of local influence for elliptical linear regression models. In the present paper, we consider the same elliptical linear regression models. We study the schemes of scale, predictor and response perturbations, and obtain their corresponding Delta matrices, respectively. To illustrate the methodology for assessment of local influence for these schemes and the implementation of the obtained results, we give an example.     

On multivariate nonlinear regression models with stationary correlated errors

In this paper we consider the statistical analysis of multivariate multiple nonlinear regression models with correlated errors, using Finite Fourier Transforms. Consistency and asymptotic normality of the weighted least squares estimates are established under various conditions on the regressor variables. These conditions involve different types of scalings, and the scaling factors are obtained explicitly for various types of nonlinear regression models including an interesting model which requires the estimation of unknown frequencies. The estimation of frequencies is a classical problem occurring in many areas like signal processing, environmental time series, astronomy and other areas of physical sciences. We illustrate our methodology using two real data sets taken from geophysics and environmental sciences. The data we consider from geophysics are polar motion (which is now widely known as “Chandlers Wobble”), where one has to estimate the drift parameters, the offset parameters and the two periodicities associated with elliptical motion. The data were first analyzed by Arato, Kolmogorov and Sinai who treat it as a bivariate time series satisfying a finite order time series model. They estimate the periodicities using the coefficients of the fitted models. Our analysis shows that the two dominant frequencies are 12 h and 410 days. The second example, we consider is the minimum/maximum monthly temperatures observed at the Antarctic Peninsula (Faraday/Vernadsky station). It is now widely believed that over the past 50 years there is a steady warming in this region, and if this is true, the warming has serious consequences on ecology, marine life, etc. as it can result in melting of ice shelves and glaciers. Our objective here is to estimate any existing temperature trend in the data, and we use the nonlinear regression methodology developed here to achieve that goal.     

Influence diagnostics in heteroscedastic and/or autoregressive nonlinear elliptical models for correlated data

In this paper, we propose nonlinear elliptical models for correlated data with heteroscedastic and/or autoregressive structures. Our aim is to extend the models proposed by Russo et al. 22 by considering a more sophisticated scale structure to deal with variations in data dispersion and/or a possible autocorrelation among measurements taken throughout the same experimental unit. Moreover, to avoid the possible influence of outlying observations or to take into account the non-normal symmetric tails of the data, we assume elliptical contours for the joint distribution of random effects and errors, which allows us to attribute different weights to the observations. We propose an iterative algorithm to obtain the maximum-likelihood estimates for the parameters and derive the local influence curvatures for some specific perturbation schemes. The motivation for this work comes from a pharmacokinetic indomethacin data set, which was analysed previously by Bocheng and Xuping 1 under normality.     

Nonlinear GCV and quasi-GCV for shrinkage models

The generalized cross-validation (GCV) method has been a popular technique for the selection of tuning parameters for smoothing and penalty, and has been a standard tool to select tuning parameters for shrinkage models in recent works. Its computational ease and robustness compared to the cross-validation method makes it competitive for model selection as well. It is well known that the GCV method performs well for linear estimators, which are linear functions of the response variable, such as ridge estimator. However, it may not perform well for nonlinear estimators since the GCV emphasizes linear characteristics by taking the trace of the projection matrix. This paper aims to explore the GCV for nonlinear estimators and to further extend the results to correlated data in longitudinal studies. We expect that the nonlinear GCV and quasi-GCV developed in this paper will provide similar tools for the selection of tuning parameters in linear penalty models and penalized GEE models.     

Modified D-optimal design for logistic model

Asymptotic theory of using the Fisher information matrix may provide poor approximation to the exact variance matrix of maximum likelihood estimation in nonlinear models. This may be due to not obtaining an efficient D-optimal design. In this article, we propose a modified D-optimality criterion, using a more accurate information matrix, based on the Bhattacharyya matrix. The proposed information matrix and its properties are given for two parameters simple logistic model. It is shown that the resulted modified locally D-optimal design is more efficient than the previous one; particularly, for small sample size experiments.     

Introduction of a new measure for detecting poor fit due to omitted nonlinear terms in SEM

The model chi-square that is used in linear structural equation modeling compares the fitted covariance matrix of a target model to an unstructured covariance matrix to assess global fit. For models with nonlinear terms, i.e., interaction or quadratic terms, this comparison is very problematic because these models are not nested within the saturated model that is represented by the unstructured covariance matrix. We propose a novel measure that quantifies the heteroscedasticity of residuals in structural equation models. It is based on a comparison of the likelihood for the residuals under the assumption of heteroscedasticity with the likelihood under the assumption of homoscedasticity. The measure is designed to respond to omitted nonlinear terms in the structural part of the model that result in heteroscedastic residual scores. In a small Monte Carlo study, we demonstrate that the measure appears to detect omitted nonlinear terms reliably when falsely a linear model is analyzed and the omitted nonlinear terms account for substantial nonlinear effects. The results also indicate that the measure did not respond when the correct model or an overparameterized model were used.     

On elliptical multilevel models

Multilevel models have been widely applied to analyze data sets which present some hierarchical structure. In this paper we propose a generalization of the normal multilevel models, named elliptical multilevel models. This proposal suggests the use of distributions in the elliptical class, thus involving all symmetric continuous distributions, including the normal distribution as a particular case. Elliptical distributions may have lighter or heavier tails than the normal ones. In the case of normal error models with the presence of outlying observations, heavy-tailed error models may be applied to accommodate such observations. In particular, we discuss some aspects of the elliptical multilevel models, such as maximum likelihood estimation and residual analysis to assess features related to the fitting and the model assumptions. Finally, two motivating examples analyzed under normal multilevel models are reanalyzed under Student-t and power exponential multilevel models. Comparisons with the normal multilevel model are performed by using residual analysis.     

Estimation in nonlinear mixed-effects models using heavy-tailed distributions

Nonlinear mixed-effects models are very useful to analyze repeated measures data and are used in a variety of applications. Normal distributions for random effects and residual errors are usually assumed, but such assumptions make inferences vulnerable to the presence of outliers. In this work, we introduce an extension of a normal nonlinear mixed-effects model considering a subclass of elliptical contoured distributions for both random effects and residual errors. This elliptical subclass, the scale mixtures of normal (SMN) distributions, includes heavy-tailed multivariate distributions, such as Student-t, the contaminated normal and slash, among others, and represents an interesting alternative to outliers accommodation maintaining the elegance and simplicity of the maximum likelihood theory. We propose an exact estimation procedure to obtain the maximum likelihood estimates of the fixed-effects and variance components, using a stochastic approximation of the EM algorithm. We compare the performance of the normal and the SMN models with two real data sets.     

Asymmetric Forecast Densities for U.S. Macroeconomic Variables from a Gaussian Copula Model of Cross-Sectional and Serial Dependence

Most existing reduced-form macroeconomic multivariate time series models employ elliptical disturbances, so that the forecast densities produced are symmetric. In this article, we use a copula model with asymmetric margins to produce forecast densities with the scope for severe departures from symmetry. Empirical and skew t distributions are employed for the margins, and a high-dimensional Gaussian copula is used to jointly capture cross-sectional and (multivariate) serial dependence. The copula parameter matrix is given by the correlation matrix of a latent stationary and Markov vector autoregression (VAR). We show that the likelihood can be evaluated efficiently using the unique partial correlations, and estimate the copula using Bayesian methods. We examine the forecasting performance of the model for four U.S. macroeconomic variables between 1975:Q1 and 2011:Q2 using quarterly real-time data. We find that the point and density forecasts from the copula model are competitive with those from a Bayesian VAR. During the recent recession the forecast densities exhibit substantial asymmetry, avoiding some of the pitfalls of the symmetric forecast densities from the Bayesian VAR. We show that the asymmetries in the predictive distributions of GDP growth and inflation are similar to those found in the probabilistic forecasts from the Survey of Professional Forecasters. Last, we find that unlike the linear VAR model, our fitted Gaussian copula models exhibit nonlinear dependencies between some macroeconomic variables. This article has online supplementary material.     

The Mean Squared Error Optimum Design Criterion for Parameter Estimation in Nonlinear Regression Models

In the context of nonlinear regression models, we propose an optimal experimental design criterion for estimating the parameters that account for the intrinsic and parameter-effects nonlinearity. The optimal design criterion proposed in this article minimizes the determinant of the mean squared error matrix of the parameter estimator that is quadratically approximated using the curvature array. The design criterion reduces to the D-optimal design criterion if there are no intrinsic and parameter-effects nonlinearity in the model, and depends on the scale parameter estimator and on the reparameterization used. Some examples, using a well known nonlinear kinetics model, demonstrate the application of the proposed criterion to nonsequential design of experiments as compared with the D-optimal criterion.     

Approximate testing in two-stage nonlinear mixed models

We investigate here small sample properties of approximate F-tests about fixed effects parameters in nonlinear mixed models. For estimation of population fixed effects parameters as well as variance components, we apply the two-stage approach. This method is useful and popular when the number of observations per sampling unit is large enough. The approximate F-test is constructed based on large-sample approximation to the distribution of nonlinear least-squares estimates of subject-specific parameters. We recommend a modified test statistic that takes into consideration approximation to the large-sample Fisher information matrix (See [Volaufova J, Burton JH. Note on hypothesis testing in mixed models. Oral presentation at: LINSTAT 2012/21st IWMS; 2012; Bedlewo, Poland]). Our main focus is on comparing finite sample properties of broadly used approximate tests (Wald test and likelihood ratio test) and the modified F-test under the null hypothesis, especially accuracy of p-values (See [Volaufova J, LaMotte L. Comparison of approximate tests of fixed effects in linear repeated measures design models with covariates. Tatra Mountains. 2008;39:17–25]). For that purpose two extensive simulation studies are conducted based on pharmacokinetic models (See [Hartford A, Davidian M. Consequences of misspecifying assumptions in nonlinear mixed effects models. Comput Stat and Data Anal. 2000;34:139–164; Pinheiro J, Bates D. Approximations to the log-likelihood function in the non-linear mixed-effects model. J Comput Graph Stat. 1995;4(1):12–35]).     

Exploratory model analysis using cdf knotting with applications to distinguishability,limiting forms,and moment ratios to the three parameter weibull family

An exploratory model analysis device we call CDF knotting is introduced. It is a technique we have found useful for exploring relationships between points in the parameter space of a model and global properties of associated distribution functions. It can be used to alert the model builder to a condition we call lack of distinguishability which is to nonlinear models what multicollinearity is to linear models. While there are simple remedial actions to deal with multicollinearity in linear models, techniques such as deleting redundant variables in those models do not have obvious parallels for nonlinear models. In some of these nonlinear situations, however, CDF knotting may lead to alternative models with fewer parameters whose distribution functions are very similar to those of the original overparameterized model. We also show how CDF knotting can be exploited as a mathematical tool for deriving limiting distributions and illustrate the technique for the 3-parameterWeibull family obtaining limiting forms and moment ratios which correct and extend previously published results. Finally, geometric insights obtained by CDF knotting are verified relative to data fitting and estimation.     

Modeling the density of US yield curve using Bayesian semiparametric dynamic Nelson-Siegel model

Abstract

This paper proposes the Bayesian semiparametric dynamic Nelson-Siegel model for estimating the density of bond yields. Specifically, we model the distribution of the yield curve factors according to an infinite Markov mixture (iMM). The model allows for time variation in the mean and covariance matrix of factors in a discrete manner, as opposed to continuous changes in these parameters such as the Time Varying Parameter (TVP) models. Estimating the number of regimes using the iMM structure endogenously leads to an adaptive process that can generate newly emerging regimes over time in response to changing economic conditions in addition to existing regimes. The potential of the proposed framework is examined using US bond yields data. The semiparametric structure of the factors can handle various forms of non-normalities including fat tails and nonlinear dependence between factors using a unified approach by generating new clusters capturing these specific characteristics. We document that modeling parameter changes in a discrete manner increases the model fit as well as forecasting performance at both short and long horizons relative to models with fixed parameters as well as the TVP model with continuous parameter changes. This is mainly due to fact that the discrete changes in parameters suit the typical low frequency monthly bond yields data characteristics better.     

Estimation and diagnostics for heteroscedastic nonlinear regression models based on scale mixtures of skew-normal distributions

An extension of some standard likelihood based procedures to heteroscedastic nonlinear regression models under scale mixtures of skew-normal (SMSN) distributions is developed. This novel class of models provides a useful generalization of the heteroscedastic symmetrical nonlinear regression models (Cysneiros et al., 2010), since the random term distributions cover both symmetric as well as asymmetric and heavy-tailed distributions such as skew-t, skew-slash, skew-contaminated normal, among others. A simple EM-type algorithm for iteratively computing maximum likelihood estimates of the parameters is presented and the observed information matrix is derived analytically. In order to examine the performance of the proposed methods, some simulation studies are presented to show the robust aspect of this flexible class against outlying and influential observations and that the maximum likelihood estimates based on the EM-type algorithm do provide good asymptotic properties. Furthermore, local influence measures and the one-step approximations of the estimates in the case-deletion model are obtained. Finally, an illustration of the methodology is given considering a data set previously analyzed under the homoscedastic skew-t nonlinear regression model.     

BARTLETT CORRECTED LIKELIHOOD RATIO TESTS IN LOCATION-SCALE NONLINEAR MODELS

The paper derives Bartlett corrections for improving the chisquare approximation to the likelihood ratio statistics in a class of location-scale family of distributions, which encompasses the elliptical family of distributions and also asymmetric distributions such as the extreme value distributions. We present, in matrix notation, a Bartlett corrected likelihood ratio statistic for testing that a subset of the nonlinear regression coefficients in this class of models equals a given vector of constants. The formulae derived are simple enough to be used analytically to obtain several Bartlett corrections in a variety of important models. We show that these formulae generalize a number of previously published results. We also present simulation results comparing the sizes and powers of the usual likelihood ratio tests and their Bartlett corrected versions when the scale parameter is considered known and when this parameter is uncorrectly specified.     

A Comparison of Error Variance Estimates in Nonparametric Mixed Models

In this article, an integral representation for the density of a matrix variate quaternion elliptical distribution is proposed. To this end, a weight function is used, based on the inverse Laplace transform of a function of a Hermitian quaternion matrix. Examples of well-known members of the family of quaternion elliptical distributions are given as well as their respective weight functions. It is shown that under some conditions, the proposed formula can be applied for the scale mixture of quaternion normal models. Applications of the proposed method are also given.     

Influence in the Elliptical Modified Ridge Regression

In this article, we present diagnostic methods for the modified ridge regression under elliptical model based on the pseudo-likelihood function. The maximum likelihood estimators of the parameters in the modified ridge elliptical model are given and local influence measures are developed. Finally, illustration of our methodology is given through a numerical example.     

Robust estimation of mean and covariance for longitudinal data with dropouts

In this paper, we study estimation of linear models in the framework of longitudinal data with dropouts. Under the assumptions that random errors follow an elliptical distribution and all the subjects share the same within-subject covariance matrix which does not depend on covariates, we develop a robust method for simultaneous estimation of mean and covariance. The proposed method is robust against outliers, and does not require to model the covariance and missing data process. Theoretical properties of the proposed estimator are established and simulation studies show its good performance. In the end, the proposed method is applied to a real data analysis for illustration.     

Conditional inference in linear versus nonlinear models for binary time series

The modelling of discrete such as binary time series, unlike the continuous time series, is not easy. This is due to the fact that there is no unique way to model the correlation structure of the repeated binary data. Some models may also provide a complicated correlation structure with narrow ranges for the correlations. In this paper, we consider a nonlinear dynamic binary time series model that provides a correlation structure which is easy to interpret and the correlations under this model satisfy the full?1 to 1 range. For the estimation of the parameters of this nonlinear model, we use a conditional generalized quasilikelihood (CGQL) approach which provides the same estimates as those of the well-known maximum likelihood approach. Furthermore, we consider a competitive linear dynamic binary time series model and examine the performance of the CGQL approach through a simulation study in estimating the parameters of this linear model. The model mis-specification effects on estimation as well as forecasting are also examined through simulations.