Generalized varying-coefficient single-index model

In this paper, the generalized varying-coefficient single-index model is discussed based on penalized likelihood. All the unknown functions are fitted by penalized spline. The estimates of the unknown parameters and the unknown coefficient functions are obtained and the estimation approach is rapid and computationally stable. Under some mild conditions, the consistency and the asymptotic normality of these resulting estimators are given. Two simulation studies are carried out to illustrate the performance of the estimates. An application of the model to the Hong Kong environmental data further demonstrates the potential of the proposed modelling procedures.     

Penalized Spline Varying-Coefficient Single-Index Model

In this article, the varying-coefficient single-index model (VCSIM) is discussed based on penalized spline estimation method. All the coefficient functions are fitted by P-spline and all parameters in P-spline varying-coefficient model can be estimated simultaneously by penalized nonlinear least squares. The detailed algorithm is given, including choosing smoothing parameters and knots. The approach is rapid and computationally stable. √n consistency and asymptotic normality of the estimators of all the parameters are showed. Both simulated and real data examples are given to illustrate the proposed estimation methodology.     

Estimation using penalized quasilikelihood and quasi-pseudo-likelihood in Poisson mixed models

We consider two estimation schemes based on penalized quasilikelihood and quasi-pseudo-likelihood in Poisson mixed models. The asymptotic bias in regression coefficients and variance components estimated by penalized quasilikelihood (PQL) is studied for small values of the variance components. We show the PQL estimators of both regression coefficients and variance components in Poisson mixed models have a smaller order of bias compared to those for binomial data. Unbiased estimating equations based on quasi-pseudo-likelihood are proposed and are shown to yield consistent estimators under some regularity conditions. The finite sample performance of these two methods is compared through a simulation study.     

Penalized Spline Estimation for Varying-Coefficient Models

Varying-coefficient models are useful extensions of classical linear models. They arise from multivariate nonparametric regression, nonlinear time series modeling and forecasting, longitudinal data analysis, and others. This article proposes the penalized spline estimation for the varying-coefficient models. Assuming a fixed but potentially large number of knots, the penalized spline estimators are shown to be strong consistency and asymptotic normality. A systematic optimization algorithm for the selection of multiple smoothing parameters is developed. One of the advantages of the penalized spline estimation is that it can accommodate varying degrees of smoothness among coefficient functions due to multiple smoothing parameters being used. Some simulation studies are presented to illustrate the proposed methods.     

Estimation and variable selection for partially functional linear models

In this paper, a new estimation procedure based on composite quantile regression and functional principal component analysis (PCA) method is proposed for the partially functional linear regression models (PFLRMs). The proposed estimation method can simultaneously estimate both the parametric regression coefficients and functional coefficient components without specification of the error distributions. The proposed estimation method is shown to be more efficient empirically for non-normal random error, especially for Cauchy error, and almost as efficient for normal random errors. Furthermore, based on the proposed estimation procedure, we use the penalized composite quantile regression method to study variable selection for parametric part in the PFLRMs. Under certain regularity conditions, consistency, asymptotic normality, and Oracle property of the resulting estimators are derived. Simulation studies and a real data analysis are conducted to assess the finite sample performance of the proposed methods.     

Improvements in the small sample efficiency of the minimum S-divergence estimators under discrete models

This paper considers the problem of inliers and empty cells and the resulting issue of relative inefficiency in estimation under pure samples from a discrete population when the sample size is small. Many minimum divergence estimators in the S-divergence family, although possessing very strong outlier stability properties, often have very poor small sample efficiency in the presence of inliers and some are not even defined in the presence of a single empty cell; this limits the practical applicability of these estimators, in spite of their otherwise sound robustness properties and high asymptotic efficiency. Here, we study a penalized version of the S-divergences such that the resulting minimum divergence estimators are free from these issues, without altering their robustness properties and asymptotic efficiencies. We present a general proof for the asymptotic properties of these minimum penalized S-divergence estimators. This provides a significant addition to the literature, as the asymptotics of penalized divergences which are not finitely defined are currently unavailable in the literature. The small sample advantages of the minimum penalized S-divergence estimators are examined through an extensive simulation study and some empirical suggestions regarding the choice of the relevant underlying tuning parameters are also provided.     

随机效应半参数logit模型的惩罚似然估计研究

 在颇具争议的收入差距和健康关系研究中,为了降低可能存在的模型设定和遗漏变量偏误,本文提出了随机效应半参数logit模型,其中非参数的设定还可用于数据的初探性分析。随后本文提出了模型非参数和参数部分的估计方法。这里涉及的难点是随机效应的存在导致似然函数中的积分没有解析式,而非参数的存在更加大了估计难度。本文基于惩罚样条非参数估计方法和四阶Laplace近似方法建立了惩罚对数似然函数,其最大化采用了Newton_Raphson近似方法。文章还建立了惩罚样条中重要光滑参数的选取准则。模型在收入差距和健康实例中的估计结果表明数据支持收入差距弱假说,且非参数估计结果表明其具有U型形式,与实例估计结果的比较指出本文提出的估计方法是较准确的。     

Penalized likelihood estimators for truncated data

We investigate the performance of linearly penalized likelihood estimators for estimating distributional parameters in the presence of data truncation. Truncation distorts the likelihood surface to create instabilities and high variance in the estimation of these parameters, and the penalty terms help in many cases to decrease estimation error and increase robustness. Approximate methods are provided for choosing a priori good penalty estimators, which are shown to perform well in a series of simulation experiments. The robustness of the methods is explored heuristically using both simulated and real data drawn from an operational risk context.     

Regularized robust estimation in binary regression models

In this paper, we investigate robust parameter estimation and variable selection for binary regression models with grouped data. We investigate estimation procedures based on the minimum-distance approach. In particular, we employ minimum Hellinger and minimum symmetric chi-squared distances criteria and propose regularized minimum-distance estimators. These estimators appear to possess a certain degree of automatic robustness against model misspecification and/or for potential outliers. We show that the proposed non-penalized and penalized minimum-distance estimators are efficient under the model and simultaneously have excellent robustness properties. We study their asymptotic properties such as consistency, asymptotic normality and oracle properties. Using Monte Carlo studies, we examine the small-sample and robustness properties of the proposed estimators and compare them with traditional likelihood estimators. We also study two real-data applications to illustrate our methods. The numerical studies indicate the satisfactory finite-sample performance of our procedures.     

Estimating the variance for heterogeneity in arm‐based network meta‐analysis

Network meta‐analysis can be implemented by using arm‐based or contrast‐based models. Here we focus on arm‐based models and fit them using generalized linear mixed model procedures. Full maximum likelihood (ML) estimation leads to biased trial‐by‐treatment interaction variance estimates for heterogeneity. Thus, our objective is to investigate alternative approaches to variance estimation that reduce bias compared with full ML. Specifically, we use penalized quasi‐likelihood/pseudo‐likelihood and hierarchical (h) likelihood approaches. In addition, we consider a novel model modification that yields estimators akin to the residual maximum likelihood estimator for linear mixed models. The proposed methods are compared by simulation, and 2 real datasets are used for illustration. Simulations show that penalized quasi‐likelihood/pseudo‐likelihood and h‐likelihood reduce bias and yield satisfactory coverage rates. Sum‐to‐zero restriction and baseline contrasts for random trial‐by‐treatment interaction effects, as well as a residual ML‐like adjustment, also reduce bias compared with an unconstrained model when ML is used, but coverage rates are not quite as good. Penalized quasi‐likelihood/pseudo‐likelihood and h‐likelihood are therefore recommended.     

Estimation on Varying-Coefficient Partially Linear Model With Different Smoothing Variables

The authors give the estimation on the varying-coefficient partially linear regression model with different smoothing variables. The efficient estimators of the intercept function and the coefficient functions are obtained by a one-step back-fitting technique based on their initial estimators given by local linear technique and the averaged method. Furthermore, their asymptotic normalities are given. Some simulation studies are used to illustrate the performances of the estimation.     

Penalized inverse probability weighted estimators for weighted rank regression with missing covariates

Abstract

In this article, we study the variable selection and estimation for linear regression models with missing covariates. The proposed estimation method is almost as efficient as the popular least-squares-based estimation method for normal random errors and empirically shown to be much more efficient and robust with respect to heavy tailed errors or outliers in the responses and covariates. To achieve sparsity, a variable selection procedure based on SCAD is proposed to conduct estimation and variable selection simultaneously. The procedure is shown to possess the oracle property. To deal with the covariates missing, we consider the inverse probability weighted estimators for the linear model when the selection probability is known or unknown. It is shown that the estimator by using estimated selection probability has a smaller asymptotic variance than that with true selection probability, thus is more efficient. Therefore, the important Horvitz-Thompson property is verified for penalized rank estimator with the covariates missing in the linear model. Some numerical examples are provided to demonstrate the performance of the estimators.     

Lasso with convex loss: Model selection consistency and estimation

Abstract

Variable selection is a fundamental challenge in statistical learning if one works with data sets containing huge amount of predictors. In this artical we consider procedures popular in model selection: Lasso and adaptive Lasso. Our goal is to investigate properties of estimators based on minimization of Lasso-type penalized empirical risk with a convex loss function, in particular nondifferentiable. We obtain theorems concerning rate of convergence in estimation, consistency in model selection and oracle properties for Lasso estimators if the number of predictors is fixed, i.e. it does not depend on the sample size. Moreover, we study properties of Lasso and adaptive Lasso estimators on simulated and real data sets.     

Nonparametric estimating equations based on a penalized information criterion

It has recently been observed that, given the mean‐variance relation, one can improve on the accuracy of the quasi‐likelihood estimator by the adaptive estimator based on the estimation of the higher moments. The estimation of such moments is usually unstable, however, and consequently only for large samples does the improvement become evident. The author proposes a nonparametric estimating equation that does not depend on the estimation of such moments, but instead on the penalized minimization of asymptotic variance. His method provides a strong improvement over the quasi‐likelihood estimator and the adaptive estimators, for a wide range of sample sizes.     

The influence function of penalized regression estimators

To perform regression analysis in high dimensions, lasso or ridge estimation are a common choice. However, it has been shown that these methods are not robust to outliers. Therefore, alternatives as penalized M-estimation or the sparse least trimmed squares (LTS) estimator have been proposed. The robustness of these regression methods can be measured with the influence function. It quantifies the effect of infinitesimal perturbations in the data. Furthermore, it can be used to compute the asymptotic variance and the mean-squared error (MSE). In this paper we compute the influence function, the asymptotic variance and the MSE for penalized M-estimators and the sparse LTS estimator. The asymptotic biasedness of the estimators make the calculations non-standard. We show that only M-estimators with a loss function with a bounded derivative are robust against regression outliers. In particular, the lasso has an unbounded influence function.     

Penalized calibration in survey sampling: Design-based estimation assisted by mixed models

Calibration techniques in survey sampling, such as generalized regression estimation (GREG), were formalized in the 1990s to produce efficient estimators of linear combinations of study variables, such as totals or means. They implicitly lie on the assumption of a linear regression model between the variable of interest and some auxiliary variables in order to yield estimates with lower variance if the model is true and remaining approximately design-unbiased even if the model does not hold. We propose a new class of model-assisted estimators obtained by releasing a few calibration constraints and replacing them with a penalty term. This penalization is added to the distance criterion to minimize. By introducing the concept of penalized calibration, combining usual calibration and this ‘relaxed’ calibration, we are able to adjust the weight given to the available auxiliary information. We obtain a more flexible estimation procedure giving better estimates particularly when the auxiliary information is overly abundant or not fully appropriate to be completely used. Such an approach can also be seen as a design-based alternative to the estimation procedures based on the more general class of mixed models, presenting new prospects in some scopes of application such as inference on small domains.     

ESTIMATORS BASED ON KENDALL'S TAU IN MULTIVARIATE COPULA MODELS

The estimation of a real‐valued dependence parameter in a multivariate copula model is considered. Rank‐based procedures are often used in this context to guard against possible misspecification of the marginal distributions. A standard approach consists of maximizing the pseudo‐likelihood. Here, we investigate alternative estimators based on the inversion of two multivariate extensions of Kendall's tau developed by Kendall and Babington Smith, and by Joe. The former, which amounts to the average value of tau over all pairs of variables, is often referred to as the coefficient of agreement. Existing results concerning the finite‐ and large‐sample properties of this coefficient are summarized, and new, parallel findings are provided for the multivariate version of tau due to Joe, along with illustrations. The performance of the estimators resulting from the inversion of these two versions of Kendall's tau is compared in the context of copula models through simulations.     

Nonconcave penalized estimation for partially linear models with longitudinal data

A nonconcave penalized estimation method is proposed for partially linear models with longitudinal data when the number of parameters diverges with the sample size. The proposed procedure can simultaneously estimate the parameters and select the important variables. Under some regularity conditions, the rate of convergence and asymptotic normality of the resulting estimators are established. In addition, an iterative algorithm is proposed to implement the proposed estimators. To improve efficiency for regression coefficients, the estimation of the covariance function is integrated in the iterative algorithm. Simulation studies are carried out to demonstrate that the proposed method performs well, and a real data example is analysed to illustrate the proposed procedure.     

Penalized least-squares estimation for regression coefficients in high-dimensional partially linear models

We consider a partially linear model with diverging number of groups of parameters in the parametric component. The variable selection and estimation of regression coefficients are achieved simultaneously by using the suitable penalty function for covariates in the parametric component. An MM-type algorithm for estimating parameters without inverting a high-dimensional matrix is proposed. The consistency and sparsity of penalized least-squares estimators of regression coefficients are discussed under the setting of some nonzero regression coefficients with very small values. It is found that the root p_n/n-consistency and sparsity of the penalized least-squares estimators of regression coefficients cannot be given consideration simultaneously when the number of nonzero regression coefficients with very small values is unknown, where p_n and n, respectively, denote the number of regression coefficients and sample size. The finite sample behaviors of penalized least-squares estimators of regression coefficients and the performance of the proposed algorithm are studied by simulation studies and a real data example.     

Estimation of Several Intraclass Correlation Coefficients

An intraclass correlation coefficient observed in several populations is estimated. The basis is a variance-stabilizing transformation. It is shown that the intraclass correlation coefficient from any elliptical distribution should be transformed in the same way. Four estimators are compared. An estimator where the components in a vector consisting of the transformed intraclass correlation coefficients are estimated separately, an estimator based on a weighted average of these components, a pretest estimator where the equality of the components is tested and then the outcome of the test is used in the estimation procedure, and a James-Stein estimator which shrinks toward the mean.