Variable selection in generalized estimating equations via empirical likelihood and Gaussian pseudo-likelihood

AIC and BIC based on either empirical likelihood (EAIC and EBIC) or Gaussian pseudo-likelihood (GAIC and GBIC) are proposed to select variables in longitudinal data analysis. Their performances are evaluated in the framework of the generalized estimating equations via intensive simulation studies. Our findings are: (i) GAIC and GBIC outperform other existing methods in selecting variables; (ii) EAIC and EBIC are effective in selecting covariates only when the working correlation structure is correctly specified; (iii) GAIC and GBIC perform well regardless the working correlation structure is correctly specified or not. A real dataset is also provided to illustrate the findings.     

Shrinkage tuning parameter selection in precision matrices estimation

Recent literature provides many computational and modeling approaches for covariance matrices estimation in a penalized Gaussian graphical models but relatively little study has been carried out on the choice of the tuning parameter. This paper tries to fill this gap by focusing on the problem of shrinkage parameter selection when estimating sparse precision matrices using the penalized likelihood approach. Previous approaches typically used K-fold cross-validation in this regard. In this paper, we first derived the generalized approximate cross-validation for tuning parameter selection which is not only a more computationally efficient alternative, but also achieves smaller error rate for model fitting compared to leave-one-out cross-validation. For consistency in the selection of nonzero entries in the precision matrix, we employ a Bayesian information criterion which provably can identify the nonzero conditional correlations in the Gaussian model. Our simulations demonstrate the general superiority of the two proposed selectors in comparison with leave-one-out cross-validation, 10-fold cross-validation and Akaike information criterion.     

Stratified Gaussian graphical models

Gaussian graphical models represent the backbone of the statistical toolbox for analyzing continuous multivariate systems. However, due to the intrinsic properties of the multivariate normal distribution, use of this model family may hide certain forms of context-specific independence that are natural to consider from an applied perspective. Such independencies have been earlier introduced to generalize discrete graphical models and Bayesian networks into more flexible model families. Here, we adapt the idea of context-specific independence to Gaussian graphical models by introducing a stratification of the Euclidean space such that a conditional independence may hold in certain segments but be absent elsewhere. It is shown that the stratified models define a curved exponential family, which retains considerable tractability for parameter estimation and model selection.     

Direct estimation of differential networks under high-dimensional nonparanormal graphical models

In genomics, it is often of interest to study the structural change of a genetic network between two phenotypes. Under Gaussian graphical models, the problem can be transformed to estimating the difference between two precision matrices, and several approaches have been recently developed for this task such as joint graphical lasso and fused graphical lasso. However, the multivariate Gaussian assumptions made in the existing approaches are often violated in reality. For instance, most RNA-Seq data follow non-Gaussian distributions even after log-transformation or other variance-stabilizing transformations. In this work, we consider the problem of directly estimating differential networks under a flexible semiparametric model, namely the nonparanormal graphical model, where the random variables are assumed to follow a multivariate Gaussian distribution after a set of monotonically increasing transformations. We propose to use a novel rank-based estimator to directly estimate the differential network, together with a parametric simplex algorithm for fast implementation. Theoretical properties of the new estimator are established under a high-dimensional setting where $p$ grows with $n$ almost exponentially fast. In particular, we show that the proposed estimator is consistent in both parameter estimation and support recovery. Both synthetic data and real genomic data are used to illustrate the promise of the new approach. The Canadian Journal of Statistics 48: 187–203; 2020 © 2019 Statistical Society of Canada     

Bias prevention of maximum likelihood estimates for scalar skew normal and skew t distributions

The skew normal model is a class of distributions that extends the Gaussian family by including a shape parameter. Despite its nice properties, this model presents some problems with the estimation of the shape parameter. In particular, for moderate sample sizes, the maximum likelihood estimator is infinite with positive probability. As a solution, we use a modified score function as an estimating equation for the shape parameter. It is proved that the resulting modified maximum likelihood estimator is always finite. For confidence intervals a quasi-likelihood approach is considered. When regression and scale parameters are present, the method is combined with maximum likelihood estimators for these parameters. Finally, also the skew t distribution is considered, which may be viewed as an extension of the skew normal. The same method is applied to this model, considering the degrees of freedom as known.     

Pair‐copula constructions for non‐Gaussian DAG models

We propose a new type of multivariate statistical model that permits non‐Gaussian distributions as well as the inclusion of conditional independence assumptions specified by a directed acyclic graph. These models feature a specific factorisation of the likelihood that is based on pair‐copula constructions and hence involves only univariate distributions and bivariate copulas, of which some may be conditional. We demonstrate maximum‐likelihood estimation of the parameters of such models and compare them to various competing models from the literature. A simulation study investigates the effects of model misspecification and highlights the need for non‐Gaussian conditional independence models. The proposed methods are finally applied to modeling financial return data. The Canadian Journal of Statistics 40: 86–109; 2012 © 2012 Statistical Society of Canada     

Marginal likelihood estimation from the Metropolis output: tips and tricks for efficient implementation in generalized linear latent variable models

The marginal likelihood can be notoriously difficult to compute, and particularly so in high-dimensional problems. Chib and Jeliazkov employed the local reversibility of the Metropolis–Hastings algorithm to construct an estimator in models where full conditional densities are not available analytically. The estimator is free of distributional assumptions and is directly linked to the simulation algorithm. However, it generally requires a sequence of reduced Markov chain Monte Carlo runs which makes the method computationally demanding especially in cases when the parameter space is large. In this article, we study the implementation of this estimator on latent variable models which embed independence of the responses to the observables given the latent variables (conditional or local independence). This property is employed in the construction of a multi-block Metropolis-within-Gibbs algorithm that allows to compute the estimator in a single run, regardless of the dimensionality of the parameter space. The counterpart one-block algorithm is also considered here, by pointing out the difference between the two approaches. The paper closes with the illustration of the estimator in simulated and real-life data sets.     

Singular Gaussian graphical models: Structure learning

ABSTRACT

The goal of this article is to introduce singular Gaussian graphical models and their conditional independence properties. In fact, we extend the concept of Gaussian Markov Random Field to the case of a multivariate normally distributed vector with a singular covariance matrix. We construct, then, the associated graph’s structure from the covariance matrix’s pseudo-inverse on the basis of a characterization of the pairwise conditional independence. The proposed approach can also be used when the covariance matrix is ill-conditioned, through projecting data on a smaller subspace. In this case, our method ensures numerical stability and consistency of the constructed graph and significantly reduces the inference problem’s complexity. These aspects are illustrated using numerical experiments.     

Consistent model identification of varying coefficient quantile regression with BIC tuning parameter selection

Quantile regression provides a flexible platform for evaluating covariate effects on different segments of the conditional distribution of response. As the effects of covariates may change with quantile level, contemporaneously examining a spectrum of quantiles is expected to have a better capacity to identify variables with either partial or full effects on the response distribution, as compared to focusing on a single quantile. Under this motivation, we study a general adaptively weighted LASSO penalization strategy in the quantile regression setting, where a continuum of quantile index is considered and coefficients are allowed to vary with quantile index. We establish the oracle properties of the resulting estimator of coefficient function. Furthermore, we formally investigate a Bayesian information criterion (BIC)-type uniform tuning parameter selector and show that it can ensure consistent model selection. Our numerical studies confirm the theoretical findings and illustrate an application of the new variable selection procedure.     

Bias reduction of the maximum-likelihood estimator for a conditional Gaussian MA(1) model

In this paper, we consider an estimation for the unknown parameters of a conditional Gaussian MA(1) model. In the majority of cases, a maximum-likelihood estimator is chosen because the estimator is consistent. However, for small sample sizes the error is large, because the estimator has a bias of O(n^{? 1}). Therefore, we provide a bias of O(n^{? 1}) for the maximum-likelihood estimator for the conditional Gaussian MA(1) model. Moreover, we propose new estimators for the unknown parameters of the conditional Gaussian MA(1) model based on the bias of O(n^{? 1}). We investigate the properties of the bias, as well as the asymptotical variance of the maximum-likelihood estimators for the unknown parameters, by performing some simulations. Finally, we demonstrate the validity of the new estimators through this simulation study.     

A new information criterion-based bandwidth selection method for non-parametric regressions

ABSTRACT

Local linear estimator is a popularly used method to estimate the non-parametric regression functions, and many methods have been derived to estimate the smoothing parameter, or the bandwidth in this case. In this article, we propose an information criterion-based bandwidth selection method, with the degrees of freedom originally derived for non-parametric inferences. Unlike the plug-in method, the new method does not require preliminary parameters to be chosen in advance, and is computationally efficient compared to the cross-validation (CV) method. Numerical study shows that the new method performs better or comparable to existing plug-in method or CV method in terms of the estimation of the mean functions, and has lower variability than CV selectors. Real data applications are also provided to illustrate the effectiveness of the new method.     

Graphical models for skew-normal variates

This paper explores the usefulness of the multivariate skew-normal distribution in the context of graphical models. A slight extension of the family recently discussed by Azzalini & Dalla Valle (1996 ) and Azzalini & Capitanio (1999 ) is described, the main motivation being the additional property of closure under conditioning. After considerations of the main probabilistic features, the focus of the paper is on the construction of conditional independence graphs for skew-normal variables. Necessary and sufficient conditions for conditional independence are stated, and the admissible structures of a graph under restriction on univariate marginal distribution are studied. Finally, parameter estimation is considered. It is shown how the factorization of the likelihood function according to a graph can be rearranged in order to obtain a parameter based factorization.     

Identifiable finite mixtures of location models for clustering mixed-mode data

For clustering mixed categorical and continuous data, Lawrence and Krzanowski (1996) proposed a finite mixture model in which component densities conform to the location model. In the graphical models literature the location model is known as the homogeneous Conditional Gaussian model. In this paper it is shown that their model is not identifiable without imposing additional restrictions. Specifically, for g groups and m locations, (g!)m–1 distinct sets of parameter values (not including permutations of the group mixing parameters) produce the same likelihood function. Excessive shrinkage of parameter estimates in a simulation experiment reported by Lawrence and Krzanowski (1996) is shown to be an artifact of the model's non-identifiability. Identifiable finite mixture models can be obtained by imposing restrictions on the conditional means of the continuous variables. These new identified models are assessed in simulation experiments. The conditional mean structure of the continuous variables in the restricted location mixture models is similar to that in the underlying variable mixture models proposed by Everitt (1988), but the restricted location mixture models are more computationally tractable.     

Asymptotic properties of the maximum smoothed partial likelihood estimator in the change-plane Cox model

The change-plane Cox model is a popular tool for the subgroup analysis of survival data. Despite the rich literature on this model, there has been limited investigation into the asymptotic properties of the estimators of the finite-dimensional parameter. Particularly, the convergence rate, not to mention the asymptotic distribution, has not been fully characterized for the general model where classification is based on multiple covariates. To bridge this theoretical gap, this study proposes a maximum smoothed partial likelihood estimator and establishes the following asymptotic properties. First, it shows that the convergence rate for the classification parameter can be arbitrarily close to $n^{-1}$ up to a logarithmic factor under a certain condition on covariates and the choice of tuning parameter. Given this convergence rate result, it also establishes the asymptotic normality for the regression parameter.     

The Effect of Estimation of Unknown Degrees of Freedom on Estimation of Remaining Parameters in Spanos’ Conditional t Heteroskedastic Model

This article investigates the effect of estimation of unknown degrees of freedom on efficient estimation of remaining parameters in Spanos’ conditional t heteroskedastic model. We compare by simulation three maximum likelihood estimators (MLEs) of the remaining parameters in the model: the MLE of the remaining parameters when all the parameters are estimated by the MLE, when the degrees of freedom is estimated by a method of moments estimator, and when the degrees of freedom is erroneously specified. The latter two methods are found to perform poorly compared to the former method for the inference of variance parameters in the model. Thus, efficient estimation of degrees of freedom by the MLE is important to estimate efficiently the remaining variance parameters.     

第三类Tobit模型的半参数估计方法

本文给出第三类Tobit模型的一种新的半参数估计方法。在独立性假设下,利用主方程和选择方程中可观察受限因变量的条件生存函数所满足的关系式,构造第三类Tobit模型参数的一步联立估计量。在已知选择方程中参数一致性估计量的条件下,这种方法也可用于构造主方程模型参数 的两步估计量。本文证明了所提出的一步联立估计量和两步估计量的一致性和渐近正态性。实验模拟表明,我们提出的估计量在有限样本下具有良好表现,且一步联立估计量的有限样本表现优于或接近于Chen（1997）的估计量。     

A computationally fast alternative to cross-validation in penalized Gaussian graphical models

We study the problem of selecting a regularization parameter in penalized Gaussian graphical models. When the goal is to obtain a model with good predictive power, cross-validation is the gold standard. We present a new estimator of Kullback–Leibler loss in Gaussian Graphical models which provides a computationally fast alternative to cross-validation. The estimator is obtained by approximating leave-one-out-cross-validation. Our approach is demonstrated on simulated data sets for various types of graphs. The proposed formula exhibits superior performance, especially in the typical small sample size scenario, compared to other available alternatives to cross-validation, such as Akaike's information criterion and Generalized approximate cross-validation. We also show that the estimator can be used to improve the performance of the Bayesian information criterion when the sample size is small.     

Labelled Graphical Models

A class of log‐linear models, referred to as labelled graphical models (LGMs), is introduced for multinomial distributions. These models generalize graphical models (GMs) by employing partial conditional independence restrictions which are valid only in subsets of an outcome space. Theoretical results concerning model identifiability, decomposability and estimation are derived. A decision theoretical framework and a search algorithm for the identification of plausible models are described. Real data sets are used to illustrate that LGMs may provide a simpler interpretation of a dependence structure than GMs.