A Potts‐mixture spatiotemporal joint model for combined magnetoencephalography and electroencephalography data

We develop a new methodology for determining the location and dynamics of brain activity from combined magnetoencephalography (MEG) and electroencephalography (EEG) data. The resulting inverse problem is ill‐posed and is one of the most difficult problems in neuroimaging data analysis. In our development we propose a solution that combines the data from three different modalities, magnetic resonance imaging (MRI), MEG and EEG, together. We propose a new Bayesian spatial finite mixture model that builds on the mesostate‐space model developed by Daunizeau & Friston [Daunizeau and Friston, NeuroImage 2007; 38, 67–81]. Our new model incorporates two major extensions: (i) We combine EEG and MEG data together and formulate a joint model for dealing with the two modalities simultaneously; (ii) we incorporate the Potts model to represent the spatial dependence in an allocation process that partitions the cortical surface into a small number of latent states termed mesostates. The cortical surface is obtained from MRI. We formulate the new spatiotemporal model and derive an efficient procedure for simultaneous point estimation and model selection based on the iterated conditional modes algorithm combined with local polynomial smoothing. The proposed method results in a novel estimator for the number of mixture components and is able to select active brain regions, which correspond to active variables in a high‐dimensional dynamic linear model. The methodology is investigated using synthetic data and simulation studies and then demonstrated on an application examining the neural response to the perception of scrambled faces. R software implementing the methodology along with several sample datasets are available at the following GitHub repository https://github.com/v2south/PottsMix . The Canadian Journal of Statistics 47: 688–711; 2019 © 2019 Statistical Society of Canada     

A systematic review on model selection in high-dimensional regression

High dimensional models are getting much attention from diverse research fields involving very many parameters with a moderate size of data. Model selection is an important issue in such a high dimensional data analysis. Recent literature on theoretical understanding of high dimensional models covers a wide range of penalized methods including LASSO and SCAD. This paper presents a systematic overview of the recent development in high dimensional statistical models. We provide a brief review on the recent development of theory, methods, and guideline on applications of several penalized methods. The review includes appropriate settings to be implemented and limitations along with potential solution for each of the reviewed method. In particular, we provide a systematic review of statistical theory of the high dimensional methods by considering a unified high-dimensional modeling framework together with high level conditions. This framework includes (generalized) linear regression and quantile regression as its special cases. We hope our review helps researchers in this field to have a better understanding of the area and provides useful information to future study.     

Predictive assessment of copula models

Copulas are powerful explanatory tools for studying dependence patterns in multivariate data. While the primary use of copula models is in multivariate dependence modelling, they also offer predictive value for regression analysis. This article investigates the utility of copula models for model‐based predictions from two angles. We assess whether, where, and by how much various copula models differ in their predictions of a conditional mean and conditional quantiles. From a model selection perspective, we then evaluate the predictive discrepancy between copula models using in‐sample and out‐of‐sample predictions both in bivariate and higher‐dimensional settings. Our findings suggest that some copula models are more difficult to distinguish in terms of their overall predictive power than others, and depending on the quantity of interest, the differences in predictions can be detected only in some targeted regions. The situations where copula‐based regression approaches would be advantageous over traditional ones are discussed using simulated and real data. The Canadian Journal of Statistics 47: 8–26; 2019 © 2018 Statistical Society of Canada     

Analysis of Double Single Index Models

Motivated from problems in canonical correlation analysis, reduced rank regression and sufficient dimension reduction, we introduce a double dimension reduction model where a single index of the multivariate response is linked to the multivariate covariate through a single index of these covariates, hence the name double single index model. Because nonlinear association between two sets of multivariate variables can be arbitrarily complex and even intractable in general, we aim at seeking a principal one‐dimensional association structure where a response index is fully characterized by a single predictor index. The functional relation between the two single‐indices is left unspecified, allowing flexible exploration of any potential nonlinear association. We argue that such double single index association is meaningful and easy to interpret, and the rest of the multi‐dimensional dependence structure can be treated as nuisance in model estimation. We investigate the estimation and inference of both indices and the regression function, and derive the asymptotic properties of our procedure. We illustrate the numerical performance in finite samples and demonstrate the usefulness of the modelling and estimation procedure in a multi‐covariate multi‐response problem concerning concrete.     

IDENTIFYING TREATMENT EFFECTS IN MULTI-CHANNEL MEASUREMENTS IN ELECTROENCEPHALOGRAPHIC STUDIES: MULTIVARIATE PERMUTATION TESTS AND MULTIPLE COMPARISONS

A versatile procedure is described comprising an application of statistical techniques to the analysis of the large, multi‐dimensional data arrays produced by electroencephalographic (EEG) measurements of human brain function. Previous analytical methods have been unable to identify objectively the precise times at which statistically significant experimental effects occur, owing to the large number of variables (electrodes) and small number of subjects, or have been restricted to two‐treatment experimental designs. Many time‐points are sampled in each experimental trial, making adjustment for multiple comparisons mandatory. Given the typically large number of comparisons and the clear dependence structure among time‐points, simple Bonferroni‐type adjustments are far too conservative. A three‐step approach is proposed: (i) summing univariate statistics across variables; (ii) using permutation tests for treatment effects at each time‐point; and (iii) adjusting for multiple comparisons using permutation distributions to control family‐wise error across the whole set of time‐points. Our approach provides an exact test of the individual hypotheses while asymptotically controlling family‐wise error in the strong sense, and can provide tests of interaction and main effects in factorial designs. An application to two experimental data sets from EEG studies is described, but the approach has application to the analysis of spatio‐temporal multivariate data gathered in many other contexts.     

Strictly monotone and smooth nonparametric regression for two or more variables

The authors propose a new monotone nonparametric estimate for a regression function of two or more variables. Their method consists in applying successively one‐dimensional isotonization procedures on an initial, unconstrained nonparametric regression estimate. In the case of a strictly monotone regression function, they show that the new estimate and the initial one are first‐order asymptotic equivalent; they also establish asymptotic normality of an appropriate standardization of the new estimate. In addition, they show that if the regression function is not monotone in one of its arguments, the new estimate and the initial one have approximately the same L^p‐norm. They illustrate their approach by means of a simulation study, and two data examples are analyzed.     

Inference for Multi‐dimensional High‐frequency Data with an Application to Conditional Independence Testing

We find the asymptotic distribution of the multi‐dimensional multi‐scale and kernel estimators for high‐frequency financial data with microstructure. Sampling times are allowed to be asynchronous and endogenous. In the process, we show that the classes of multi‐scale and kernel estimators for smoothing noise perturbation are asymptotically equivalent in the sense of having the same asymptotic distribution for corresponding kernel and weight functions. The theory leads to multi‐dimensional stable central limit theorems and feasible versions. Hence, they allow to draw statistical inference for a broad class of multivariate models, which paves the way to tests and confidence intervals in risk measurement for arbitrary portfolios composed of high‐frequently observed assets. As an application, we enhance the approach to construct a test for investigating hypotheses that correlated assets are independent conditional on a common factor.     

LOCAL POLYNOMIAL QUASI-LIKELIHOOD REGRESSION ON RANDOM FIELDS

In this paper, local quasi‐likelihood regression is considered for stationary random fields of dependent variables. In the case of independent data, local polynomial quasi‐likelihood regression is known to have several appealing features such as minimax efficiency, design adaptivity and good boundary behaviour. These properties are shown to carry over to the case of random fields. The asymptotic normality of the regression estimator is established and explicit formulae for its asymptotic bias and variance are derived for strongly mixing stationary random fields. The extension to multi‐dimensional covariates is also provided in full generality. Moreover, evaluation of the finite sample performance is made through a simulation study.     

Connectivity‐informed adaptive regularization for generalized outcomes

One of the challenging problems in neuroimaging is the principled incorporation of information from different imaging modalities. Data from each modality are frequently analyzed separately using, for instance, dimensionality reduction techniques, which result in a loss of mutual information. We propose a novel regularization method, generalized ridgified Partially Empirical Eigenvectors for Regression (griPEER), to estimate associations between the brain structure features and a scalar outcome within the generalized linear regression framework. griPEER improves the regression coefficient estimation by providing a principled approach to use external information from the structural brain connectivity. Specifically, we incorporate a penalty term, derived from the structural connectivity Laplacian matrix, in the penalized generalized linear regression. In this work, we address both theoretical and computational issues and demonstrate the robustness of our method despite incomplete information about the structural brain connectivity. In addition, we also provide a significance testing procedure for performing inference on the estimated coefficients. Finally, griPEER is evaluated both in extensive simulation studies and using clinical data to classify HIV+ and HIV? individuals.     

Multivariate functional response low‐rank regression with an application to brain imaging data

We propose a multivariate functional response low‐rank regression model with possible high‐dimensional functional responses and scalar covariates. By expanding the slope functions on a set of sieve bases, we reconstruct the basis coefficients as a matrix. To estimate these coefficients, we propose an efficient procedure using nuclear norm regularization. We also derive error bounds for our estimates and evaluate our method using simulations. We further apply our method to the Human Connectome Project neuroimaging data to predict cortical surface motor task‐evoked functional magnetic resonance imaging signals using various clinical covariates to illustrate the usefulness of our results.     

A New Regression Model: Modal Linear Regression

The mode of a distribution provides an important summary of data and is often estimated on the basis of some non‐parametric kernel density estimator. This article develops a new data analysis tool called modal linear regression in order to explore high‐dimensional data. Modal linear regression models the conditional mode of a response Y given a set of predictors x as a linear function of x . Modal linear regression differs from standard linear regression in that standard linear regression models the conditional mean (as opposed to mode) of Y as a linear function of x . We propose an expectation–maximization algorithm in order to estimate the regression coefficients of modal linear regression. We also provide asymptotic properties for the proposed estimator without the symmetric assumption of the error density. Our empirical studies with simulated data and real data demonstrate that the proposed modal regression gives shorter predictive intervals than mean linear regression, median linear regression and MM‐estimators.     

New aspects of Bregman divergence in regression and classification with parametric and nonparametric estimation

In statistical learning, regression and classification concern different types of the output variables, and the predictive accuracy is quantified by different loss functions. This article explores new aspects of Bregman divergence (BD), a notion which unifies nearly all of the commonly used loss functions in regression and classification. The authors investigate the duality between BD and its generating function. They further establish, under the framework of BD, asymptotic consistency and normality of parametric and nonparametric regression estimators, derive the lower bound of their asymptotic covariance matrices, and demonstrate the role that parametric and nonparametric regression estimation play in the performance of classification procedures and related machine learning techniques. These theoretical results and new numerical evidence show that the choice of loss function affects estimation procedures, whereas has an asymptotically relatively negligible impact on classification performance. Applications of BD to statistical model building and selection with non‐Gaussian responses are also illustrated. The Canadian Journal of Statistics 37: 119‐139; 2009 © 2009 Statistical Society of Canada     

Truncated regular vines in high dimensions with application to financial data

Using only bivariate copulas as building blocks, regular vine copulas constitute a flexible class of high‐dimensional dependency models. However, the flexibility comes along with an exponentially increasing complexity in larger dimensions. In order to counteract this problem, we propose using statistical model selection techniques to either truncate or simplify a regular vine copula. As a special case, we consider the simplification of a canonical vine copula using a multivariate copula as previously treated by Heinen & Valdesogo ( 2009 ) and Valdesogo ( 2009 ). We validate the proposed approaches by extensive simulation studies and use them to investigate a 19‐dimensional financial data set of Norwegian and international market variables. The Canadian Journal of Statistics 40: 68–85; 2012 © 2012 Statistical Society of Canada     

A directional look at F‐tests

Directional testing of vector parameters, based on higher order approximations of likelihood theory, can ensure extremely accurate inference, even in high‐dimensional settings where standard first order likelihood results can perform poorly. Here we explore examples of directional inference where the calculations can be simplified, and prove that in several classical situations, the directional test reproduces exact results based on F‐tests. These findings give a new interpretation of some classical results and support the use of directional testing in general models, where exact solutions are typically not available. The Canadian Journal of Statistics 47: 619–627; 2019 © 2019 Statistical Society of Canada     

The convolution theorem for estimating linear functionals in indirect nonparametric regression models

Nonparametric regression—directly or indirectly observed—is one of the important statistical models. On one hand it contains two infinite dimensional parameters (the regression function and the error density), and on the other it is of rather simple structure. Therefore, it may serve as an interesting paradigm for illustrating or developing abstract statistical theory for non-Euclidean parameters. In this paper estimation of a linear functional of the indirectly observed regression function is considered, when a deterministic design is used. It should be noted that any Fourier coefficient of an expansion of the regression function in an orthonormal basis is such a functional. Because the design is deterministic the observables are independent but not identically distributed. Local asymptotic normality is established and applied to prove Hájek's convolution theorem for this functional. Pertinent references are Beran [1977. Robust location estimates. Ann. Statist. 5, 431–444] and McNeney and Wellner [2000. Application of convolution theorems in semiparametric models with non-i.i.d. data. J. Statist. Plann. Inference 91, 441–480]. For purposes explained above, however, the paper is kept self-contained and full proofs are provided.     

Composite quantile regression for massive datasets

Analysis of massive datasets is challenging owing to limitations of computer primary memory. Composite quantile regression (CQR) is a robust and efficient estimation method. In this paper, we extend CQR to massive datasets and propose a divide-and-conquer CQR method. The basic idea is to split the entire dataset into several blocks, applying the CQR method for data in each block, and finally combining these regression results via weighted average. The proposed approach significantly reduces the required amount of primary memory, and the resulting estimate will be as efficient as if the entire data set is analysed simultaneously. Moreover, to improve the efficiency of CQR, we propose a weighted CQR estimation approach. To achieve sparsity with high-dimensional covariates, we develop a variable selection procedure to select significant parametric components and prove the method possessing the oracle property. Both simulations and data analysis are conducted to illustrate the finite sample performance of the proposed methods.     

Statistical Modelling of Brain Morphological Measures Within Family Pedigrees

Large, family-based imaging studies can provide a better understanding of the interactions of environmental and genetic influences on brain structure and function. The interpretation of imaging data from large family studies, however, has been hindered by the paucity of well-developed statistical tools for that permit the analysis of complex imaging data together with behavioral and clinical data. In this paper, we propose to use two methods for these analyses. First, a variance components model along with score statistics is used to test linear hypotheses of unknown parameters, such as the associations of brain measures (e.g., cortical and subcortical surfaces) with their potential genetic determinants. Second, we develop a test procedure based on a resampling method to assess simultaneously the statistical significance of linear hypotheses across the entire brain. The value of these methods lies in their computational simplicity and in their applicability to a wide range of imaging data. Simulation studies show that our test procedure can accurately control the family-wise error rate. We apply our methods to the detection of statistical significance of gender-by-age interactions and of the effects of genetic variation on the thickness of the cerebral cortex in a family study of major depressive disorder.     

Feature Screening for Ultrahigh Dimensional Categorical Data With Applications

Ultrahigh dimensional data with both categorical responses and categorical covariates are frequently encountered in the analysis of big data, for which feature screening has become an indispensable statistical tool. We propose a Pearson chi-square based feature screening procedure for categorical response with ultrahigh dimensional categorical covariates. The proposed procedure can be directly applied for detection of important interaction effects. We further show that the proposed procedure possesses screening consistency property in the terminology of Fan and Lv (2008). We investigate the finite sample performance of the proposed procedure by Monte Carlo simulation studies and illustrate the proposed method by two empirical datasets.     

On the accuracy in high‐dimensional linear models and its application to genomic selection

Genomic selection is today a hot topic in genetics. It consists in predicting breeding values of selection candidates, using the large number of genetic markers now available owing to the recent progress in molecular biology. One of the most popular methods chosen by geneticists is ridge regression. We focus on some predictive aspects of ridge regression and present theoretical results regarding the accuracy criteria, that is, the correlation between predicted value and true value. We show the influence of singular values, the regularization parameter, and the projection of the signal on the space spanned by the rows of the design matrix. Asymptotic results in a high‐dimensional framework are given; in particular, we prove that the convergence to optimal accuracy highly depends on a weighted projection of the signal on each subspace. We discuss on how to improve the prediction. Last, illustrations on simulated and real data are proposed.     

Review: Reversed low-rank ANOVA model for transforming high dimensional genetic data into low dimension

A general modeling procedure for analyzing genetic data is reviewed. We review ANOVA type model that can handle both the continuous and discrete genetic variables in one modeling framework. Unlike the regression type models which typically set the phenotype variable as a response, this ANOVA model treats the phenotype variable as an explanatory variable. By reversely treating the phenotype variable, usual high dimensional problem is turned into low dimension. Instead, the ANOVA model always includes interaction term between the genetic locations and phenotype variable to find potential association between them. The interaction term is designed to be low rank with the multiplication of bilinear terms so that the required number of parameters is kept in a manageable degree. We compare the performance of the reviewed ANOVA model to the other popular methods via microarray and SNP data sets.