Joint confidence regions in the multivariate calibration problem

Consider a vector valued response variable related to a vector valued explanatory variable through a normal multivariate linear model. The multivariate calibration problem deals with statistical inference on unknown values of the explanatory variable. The problem addressed is the construction of joint confidence regions for several unknown values of the explanatory variable. The problem is investigated when the variance covariance matrix is a scalar multiple of the identity matrix and also when it is a completely unknown positive definite matrix. The problem is solved in only two cases: (i) the response and explanatory variables have the same dimensions, and (ii) the explanatory variable is a scalar. In the former case, exact joint confidence regions are derived based on a natural pivot statistic. In the latter case, the joint confidence regions are only conservative. Computational aspects and the practical implementation of the confidence regions are discussed and illustrated using an example.     

Connectivity‐informed adaptive regularization for generalized outcomes

One of the challenging problems in neuroimaging is the principled incorporation of information from different imaging modalities. Data from each modality are frequently analyzed separately using, for instance, dimensionality reduction techniques, which result in a loss of mutual information. We propose a novel regularization method, generalized ridgified Partially Empirical Eigenvectors for Regression (griPEER), to estimate associations between the brain structure features and a scalar outcome within the generalized linear regression framework. griPEER improves the regression coefficient estimation by providing a principled approach to use external information from the structural brain connectivity. Specifically, we incorporate a penalty term, derived from the structural connectivity Laplacian matrix, in the penalized generalized linear regression. In this work, we address both theoretical and computational issues and demonstrate the robustness of our method despite incomplete information about the structural brain connectivity. In addition, we also provide a significance testing procedure for performing inference on the estimated coefficients. Finally, griPEER is evaluated both in extensive simulation studies and using clinical data to classify HIV+ and HIV? individuals.     

Factor analysis regression

The paper describes two regression models—principal components and maximum-likelihood factor analysis—which may be used when the stochastic predictor varibles are highly intereorrelated and/or contain measurement error. The two problems can occur jointly, for example in social-survey data where the true (but unobserved) covariance matrix can be singular. Departure from singularity of the sample dispersion matrix is then due to measurement error. We first consider the more elementary principal components regression model, where it is shown that it can be derived as a special case of (i) canonical correlation, and (ii) restricted least squares. The second part consists of the more general maximum-likelihood factor-analysis regression model, which is derived from the generalized inverse of the product of two singular matrices. Also, it is proved that factor-analysis regression can be considered as an instrumental variables estimator and therefore does not depend on whether factors have been “properly” identified in terms of substantive behaviour. Consequently the additional task of rotating factors to “simple structure” does not arise.     

On testing for an identity covariance matrix when the dimensionality equals or exceeds the sample size

This article explores the problem of testing the hypothesis that the covariance matrix is an identity matrix when the dimensionality is equal to the sample size or larger. Two new test statistics are proposed under comparable assumptions to those statistics in the literature. The asymptotic distribution of the proposed test statistics are found and are shown to be consistent in the general asymptotic framework. An extensive simulation study shows the newly proposed tests are comparable to, and in some cases more powerful than, the tests for an identity covariance matrix currently in the literature.     

Parameter expansion for sampling a correlation matrix: an efficient GPX-RPMH algorithm

Sampling the correlation matrix (R) plays an important role in statistical inference for correlated models. There are two main constraints on a correlation matrix: positive definiteness and fixed diagonal elements. These constraints make sampling R difficult. In this paper, an efficient generalized parameter expanded re-parametrization and Metropolis-Hastings (GPX-RPMH) algorithm for sampling a correlation matrix is proposed. Drawing all components of R simultaneously from its full conditional distribution is realized by first drawing a covariance matrix from the derived parameter expanded candidate density (PXCD), and then translating it back to a correlation matrix and accepting it according to a Metropolis-Hastings (M-H) acceptance rate. The mixing rate in the M-H step can be adjusted through a class of tuning parameters embedded in the generalized candidate prior (GCP), which is chosen for R to derive the PXCD. This algorithm is illustrated using multivariate regression (MVR) models and a simulation study shows that the performance of the GPX-RPMH algorithm is more efficient than that of other methods.     

Diagonalization matrix and its application in distribution theory

Some matrix representations of diverse diagonal arrays are studied in this work; the results allow new definitions of classes of elliptical distributions indexed by kernels mixing Hadamard and usual products. A number of applications are derived in the setting of prior densities from the Bayesian multivariate regression model and families of non-elliptical distributions, such as the matrix multivariate generalized Birnbaum–Saunders density. The philosophy of the research about matrix representations of quadratic and inverse quadratic forms can be extended as a methodology for exploring possible new applications in non-standard distributions, matrix transformations and inference.     

General results for the beta Weibull distribution

In this paper, we study some mathematical properties of the beta Weibull (BW) distribution, which is a quite flexible model in analysing positive data. It contains the Weibull, exponentiated exponential, exponentiated Weibull and beta exponential distributions as special sub-models. We demonstrate that the BW density can be expressed as a mixture of Weibull densities. We provide their moments and two closed-form expressions for their moment-generating function. We examine the asymptotic distributions of the extreme values. Explicit expressions are derived for the mean deviations, Bonferroni and Lorenz curves, reliability and two entropies. The density of the BW-order statistics is a mixture of Weibull densities and two closed-form expressions are derived for their moments. The estimation of the parameters is approached by two methods: moments and maximum likelihood. We compare the performances of the estimates obtained from both the methods by simulation. The expected information matrix is derived. For the first time, we introduce a log-BW regression model to analyse censored data. The usefulness of the BW distribution is illustrated in the analysis of three real data sets.     

On analysis of binary response data in longitudinal factorial studies

Binary data are commonly used as responses to assess the effects of independent variables in longitudinal factorial studies. Such effects can be assessed in terms of the rate difference (RD), the odds ratio (OR), or the rate ratio (RR). Traditionally, the logistic regression seems always a recommended method with statistical comparisons made in terms of the OR. Statistical inference in terms of the RD and RR can then be derived using the delta method. However, this approach is hard to realize when repeated measures occur. To obtain statistical inference in longitudinal factorial studies, the current article shows that the mixed-effects model for repeated measures, the logistic regression for repeated measures, the log-transformed regression for repeated measures, and the rank-based methods are all valid methods that lead to inference in terms of the RD, OR, and RR, respectively. Asymptotic linear relationships between the estimators of the regression coefficients of these models are derived when the weight (working covariance) matrix is an identity matrix. Conditions for the Wald-type tests to be asymptotically equivalent in these models are provided and powers were compared using simulation studies. A phase III clinical trial is used to illustrate the investigated methods with corresponding SAS® code supplied.     

Spacing of information in polynomial regression:a simple solution

Consider a k polynomial regression on a single real variable. If n uncorrelated observations are to be taken in a design with support on more than k+1 points, there is an approximate experiment, ν, with support on k+1 points and n observations such that both designs have the same information matrix for the model. A proof of this result is provided. A method to obtain the approximate design ν is given and illustrated by an example. The source of disagreement between Kiefer (1959) and De La Garza (1954) in the solution of this problem is clarified.     

Error analysis of spatial representation and estimation of mobile robots

The problem of estimating the location of a mobile robot in an unstructured environment is discussed. This work extends earlier results in two important ways. First, the bias and variance of the estimation are analytically derived as functions of the angular error and distance between frames. Second, the uncertainty covariance matrix is derived and is compared to the first-order approximation previously used to estimate the result of compounding uncertain transformations to provide a framework in which the appropriateness of the first-order estimate can be formally studied. A simulation study, showing how the biases and expected distance between the estimate and true position of the robot vary as a function of measurement errors and different path plannings, is presented. Some possible improvements of the estimation method and future research topics are also given.     

Second-order least squares estimation of censored regression models

This paper proposes the second-order least squares estimation, which is an extension of the ordinary least squares method, for censored regression models where the error term has a general parametric distribution (not necessarily normal). The strong consistency and asymptotic normality of the estimator are derived under fairly general regularity conditions. We also propose a computationally simpler estimator which is consistent and asymptotically normal under the same regularity conditions. Finite sample behavior of the proposed estimators under both correctly and misspecified models are investigated through Monte Carlo simulations. The simulation results show that the proposed estimator using optimal weighting matrix performs very similar to the maximum likelihood estimator, and the estimator with the identity weight is more robust against the misspecification.     

A computationally efficient method for bootstrapping systems of demand equations: A comparison to traditional techniques

The solution to a Liapunov matrix equation (LME) has been proposed to estimate the parameters of the demand equations derived from the Translog, the Almost Ideal Demand System and the Rotterdam demand models. When compared to traditional scemingly unrelated regression (SUR) methods the LME approach saves both computer time and space, and it provides parameter estimates that are less likely to suffer from round-off error. However, the LME method is difficult to implement without the use of specially written computer programs and, unlike traditional SUR methods, it does not automatically provide an estimate of the covariance of the parameters. This paper solves these two problems, the first by providing a simplified solution to the Liapunov matrix equation which can be written in a few lines of code in computer languages such as SAS PROC MATRIX/IML^TM or GAUSS^TM; the second, by bootstrapping the parameter covariance matrix.     

The identification of logistic regression models with errors in the variables

The simple logistic regression model with normal measurement error and normal regressor is shown to be identifiable without any extra information about the measurement error. The multiple logistic regression model with more than one regressor variable measured with error is not identifiable. If the covariance matrix of the measurement error is known up to a scalar factor, the model is identified. Further we discuss why in spite of the identifiability the models cannot be estimated in a reasonable way without extra information about the measurement error.     

Added variable plots for linear regression with censored data

In linear regression the structure of the hat matrix plays an important part in regression diagnostics. In this note we investigate the properties of the hat matrix for regression with censored responses in the presence of one or more explanatory variables observed without censoring. The censored points in the scatterplot are renovated to positions had they been observed without censoring in a renovation process based on Buckley-James censored regression estimators. This allows natural links to be established with the structure of ordinary least squares estimators. In particular, we show that the renovated hat matrix may be partitioned in a manner which assists in deciding whether further explanatory variables should be added to the linear model. The added variable plot for regression with censored data is developed as a diagnostic tool for this decision process.     

A sequence of improved standard errors under heteroskedasticity of unknown form

The linear regression model is commonly used by practitioners to model the relationship between the variable of interest and a set of explanatory variables. The assumption that all error variances are the same (homoskedasticity) is oftentimes violated. Consistent regression standard errors can be computed using the heteroskedasticity-consistent covariance matrix estimator proposed by White (1980). Such standard errors, however, typically display nonnegligible systematic errors in finite samples, especially under leveraged data. Cribari-Neto et al. (2000) improved upon the White estimator by defining a sequence of bias-adjusted estimators with increasing accuracy. In this paper, we improve upon their main result by defining an alternative sequence of adjusted estimators whose biases vanish at a much faster rate. Hypothesis testing inference is also addressed. An empirical illustration is presented.     

VIF-based adaptive matrix perturbation method for heteroskedasticity-robust covariance estimators in the presence of multicollinearity

In this study, we investigate linear regression having both heteroskedasticity and collinearity problems. We discuss the properties related to the perturbation method. Important observations are summarized as theorems. We then prove the main result that states the heteroskedasticity-robust variances can be improved and that the resulting bias is minimized by using the matrix perturbation method. We analyze a practical example for validation of the method.     

Incremental modelling for compositional data streams

ABSTRACT

Incremental modelling of data streams is of great practical importance, as shown by its applications in advertising and financial data analysis. We propose two incremental covariance matrix decomposition methods for a compositional data type. The first method, exact incremental covariance decomposition of compositional data (C-EICD), gives an exact decomposition result. The second method, covariance-free incremental covariance decomposition of compositional data (C-CICD), is an approximate algorithm that can efficiently compute high-dimensional cases. Based on these two methods, many frequently used compositional statistical models can be incrementally calculated. We take multiple linear regression and principle component analysis as examples to illustrate the utility of the proposed methods via extensive simulation studies.     

Bayesian Analysis in Regression Models Using Pseudo-Likelihoods

This article deals with the issue of using a suitable pseudo-likelihood, instead of an integrated likelihood, when performing Bayesian inference about a scalar parameter of interest in the presence of nuisance parameters. The proposed approach has the advantages of avoiding the elicitation on the nuisance parameters and the computation of multidimensional integrals. Moreover, it is particularly useful when it is difficult, or even impractical, to write the full likelihood function.

We focus on Bayesian inference about a scalar regression coefficient in various regression models. First, in the context of non-normal regression-scale models, we give a theroetical result showing that there is no loss of information about the parameter of interest when using a posterior distribution derived from a pseudo-likelihood instead of the correct posterior distribution. Second, we present non trivial applications with high-dimensional, or even infinite-dimensional, nuisance parameters in the context of nonlinear normal heteroscedastic regression models, and of models for binary outcomes and count data, accounting also for possibile overdispersion. In all these situtations, we show that non Bayesian methods for eliminating nuisance parameters can be usefully incorporated into a one-parameter Bayesian analysis.     

UNCONSTRAINED PILOT SELECTORS FOR SMOOTHED CROSS‐VALIDATION

Two of the most useful multivariate bandwidth selection techniques are the plug‐in and cross‐validation methods. The smoothed version of the cross‐validation method is known to reduce the variability of its non‐smoothed counterpart; however, it shares with the plug‐in choice the need for a pilot bandwidth matrix. Owing to the mathematical difficulties encountered in the optimal pilot choice, it is common to restrict this pilot matrix to be a scalar multiple of the identity matrix, at the expense of losing the flexibility afforded by the unconstrained approach. Here we show how to overcome these difficulties and propose a smoothed cross‐validation selector using an unconstrained pilot matrix. Our numerical results indicate that the unconstrained selector outperforms the constrained one in practice, and is a viable competitor to unconstrained plug‐in selectors.