Finite Mixture of Generalized Semiparametric Models: Variable Selection via Penalized Estimation

Selection of the important variables is one of the most important model selection problems in statistical applications. In this article, we address variable selection in finite mixture of generalized semiparametric models. To overcome computational burden, we introduce a class of variable selection procedures for finite mixture of generalized semiparametric models using penalized approach for variable selection. Estimation of nonparametric component will be done via multivariate kernel regression. It is shown that the new method is consistent for variable selection and the performance of proposed method will be assessed via simulation.     

Properties of maximum likelihood estimators of variance components in the one-way classification,balanced data

We studied properties of maximum likelihood estimators (MLEs) of the variance components obtained from balanced data of the one-way classification. Exact and asymptotic expected values and variances of these MLEs were derived under the usual normality assumptions. Numerical studies illustrate these expected values and variances, and also illustrate the probability of obtaining a negative solution to the maximum likelihood (ML) equation for the between-class variance component. Simulations were used to study the robustness of the ML estimators under non-normal distributions.     

Collinearity in generalized linear regression

The problem of ill-conditioning in generalized linear regression is investigated. Besides collinearity among the explanatory variables, we define another type of ill-conditioning, namely ML-collinearity, which has similar detrimental effects on the covariance matrix, e.g. inflation of some of the estimated standard errors of the regression coefficients. For either situation there is collinearity among the columns of the matrix of the weighted variables. We present both methods to detect, as well as practical examples to illustrate, the difference between these two types of ill-conditioning. Also the applicability of alternative regression methods will be reviewed.     

Bayesian estimation on semiparametric models with shape restricted B-splines

In this paper, we develop a Bayesian estimation procedure for semiparametric models under shape constrains. The approach uses a hierarchical Bayes framework and characterizations of shape-constrained B-splines. We employ Markov chain Monte Carlo methods for model fitting, using a truncated normal distribution as the prior for the coefficients of basis functions to ensure the desired shape constraints. The small sample properties of the function estimators are provided via simulation and compared with existing methods. A real data analysis is conducted to illustrate the application of the proposed method.     

Penalized empirical likelihood for quantile regression with missing covariates and auxiliary information

Based on the inverse probability weight method, we, in this article, construct the empirical likelihood (EL) and penalized empirical likelihood (PEL) ratios of the parameter in the linear quantile regression model when the covariates are missing at random, in the presence and absence of auxiliary information, respectively. It is proved that the EL ratio admits a limiting Chi-square distribution. At the same time, the asymptotic normality of the maximum EL and PEL estimators of the parameter is established. Also, the variable selection of the model in the presence and absence of auxiliary information, respectively, is discussed. Simulation study and a real data analysis are done to evaluate the performance of the proposed methods.     

Marginalized lasso in sparse regression

We propose marginalized lasso, a new nonconvex penalization for variable selection in regression problem. The marginalized lasso penalty is motivated from integrating out the penalty parameter in the original lasso penalty with a gamma prior distribution. This study provides a thresholding rule and a lasso-based iterative algorithm for parameter estimation in the marginalized lasso. We also provide a coordinate descent algorithm to efficiently optimize the marginalized lasso penalized regression. Numerical comparison studies are provided to demonstrate its competitiveness over the existing sparsity-inducing penalizations and suggest some guideline for tuning parameter selection.     

Lack-of-Fit Tests for Generalized Linear Models via Splines

Cubic B-splines are used to estimate the nonparametric component of a semiparametric generalized linear model. A penalized log-likelihood ratio test statistic is constructed for the null hypothesis of the linearity of the nonparametric function. When the number of knots is fixed, its limiting null distribution is the distribution of a linear combination of independent chi-squared random variables, each with one df. The smoothing parameter is determined by giving a specified value for its asymptotically expected value under the null hypothesis. A simulation study is conducted to evaluate its power performance; a real-life dataset is used to illustrate its practical use.