Estimating coefficients of single-index regression models by minimizing variation

This paper proposes a novel estimation of coefficients in single-index regression models. Unlike the traditional average derivative estimation [Powell JL, Stock JH, Stoker TM. Semiparametric estimation of index coefficients. Econometrica. 1989;57(6):1403–1430; Hardle W, Thomas M. Investigating smooth multiple regression by the method of average derivatives. J Amer Statist Assoc. 1989;84(408):986–995] and semiparametric least squares estimation [Ichimura H. Semiparametric least squares (sls) and weighted sls estimation of single-index models. J Econometrics. 1993;58(1):71–120; Hardle W, Hall P, Ichimura H. Optimal smoothing in single-index models. Ann Statist. 1993;21(1):157–178], the procedure developed in this paper is to estimate the coefficients directly by minimizing the mean variation function and does not involve estimating the link function nonparametrically. As a result, it avoids the selection of the bandwidth or the number of knots, and its implementation is more robust and easier. The resultant estimator is shown to be consistent. Numerical results and real data analysis also show that the proposed procedure is more applicable against model free assumptions.     

Quantile regression in varying coefficient models

This paper deals with the estimation of conditional quantiles in varying coefficient models by estimating the coefficients. Varying coefficient models are among popular models that have been proposed to alleviate the curse of dimensionality. Previous works on varying coefficient models deal with conditional means directly or indirectly. However, quantiles themselves can be defined without moment conditions and plotting several conditional quantiles would give us more understanding of the data than plotting just the conditional mean. Particularly, we estimate the conditional median by estimating varying coefficients by local L₁ regression.     

Quantile regression in heteroscedastic varying coefficient models

Varying coefficient models are flexible models to describe the dynamic structure in longitudinal data. Quantile regression, more than mean regression, gives partial information on the conditional distribution of the response given the covariates. In the literature, the focus has been so far mostly on homoscedastic quantile regression models, whereas there is an interest in looking into heteroscedastic modelling. This paper contributes to the area by modelling the heteroscedastic structure and estimating it from the data, together with estimating the quantile functions. The use of the proposed methods is illustrated on real-data applications. The finite-sample behaviour of the methods is investigated via a simulation study, which includes a comparison with an existing method.     

Tests of regression coefficients under ridge regression models

This paper investigates two “non-exact” t-type tests, t( k₂) and t(k₂), of the individual coefficients of a linear regression model, based on two ordinary ridge estimators. The reported results are built on a simulation study covering 84 different models. For models with large standard errors, the ridge-based t-tests have correct levels with considerable gain in powers over those of the least squares t-test, t(0). For models with small standard errors, t(k₁) is found to be liberal and is not safe to use while, t(k₂) is found to slightly exceed the nominal level in few cases. When tie two ridge tests art: not winners, the results indicate that they don't loose much against t(0).     

Testing equality of regression coefficients in heteroscedastic normal regression models

This article addresses the problem of testing whether the vectors of regression coefficients are equal for two independent normal regression models when the error variances are unknown. This problem poses severe difficulties both to the frequentist and Bayesian approaches to statistical inference. In the former approach, normal hypothesis testing theory does not apply because of the unrelated variances. In the latter, the prior distributions typically used for the parameters are improper and hence the Bayes factor-based solution cannot be used.We propose a Bayesian solution to this problem in which no subjective input is considered. We first generate “objective” proper prior distributions (intrinsic priors) for which the Bayes factor and model posterior probabilities are well defined. The posterior probability of each model is used as a model selection tool. This consistent procedure of testing hypotheses is compared with some of the frequentist approximate tests proposed in the literature.     

Estimation of multivariate frailty models with varying coefficients

The complicated structures can be modeled more efficiently and their flexibility can be increased through frailty models with varying coefficients.Therefore, such models are proposed in this article. The real challenge is to estimate varying coefficients by the penalized partial likelihood without closed form. The Laplace approximation is used to solve this problem. These varying coefficients are fitted using B-splines. Moreover, the variances of random effects are estimated by maximizing an approximate profile likelihood. The performance of the proposed methods are assessed with simulation studies and real data. The results show that the methods proposed are better than the counterpart in literature.     

General mixed Poisson regression models with varying dispersion

A general class of mixed Poisson regression models is introduced. This class is based on a mixing between the Poisson distribution and a distribution belonging to the exponential family. With this, we unified some overdispersed models which have been studied separately, such as negative binomial and Poisson inverse gaussian models. We consider a regression structure for both the mean and dispersion parameters of the mixed Poisson models, thus extending, and in some cases correcting, some previous models considered in the literature. An expectation–maximization (EM) algorithm is proposed for estimation of the parameters and some diagnostic measures, based on the EM algorithm, are considered. We also obtain an explicit expression for the observed information matrix. An empirical illustration is presented in order to show the performance of our class of mixed Poisson models. This paper contains a Supplementary Material.     

Improved estimators of regression coefficients in measurement error models

This paper is concerned with methods of reducing variability and computer time in a simulation study. The Monte Carlo swindle, through mathematical manipulations, has been shown to yield more precise estimates than the “naive” approach. In this study computer time is considered in conjunction with the variance estimates. It is shown that by this measure the naive method is often a viable alternative to the swindle. This study concentrates on the problem of estimating the variance of an estimator of location. The advantage of one technique over another depends upon the location estimator, the sample size, and the underlying distribution. For a fixed number of samples, while the naive method gives a less precise estimate than the swindle, it requires fewer computations. In addition, for certain location estimators and distributions, the naive method is able to take advantage of certain shortcuts in the generation of each sample. The small amount of time required by this “enlightened” naive method often more than compensates for its relative lack of precision.     

Random coefficients regression models. a review 1

A review is given of random regression coefficients models. The emphasis is put on the problem of estimating the mean regression coefficients and the covariance matrix of the coefficients. Prediction of the individual random coefficients is not discussed. The main purpose of the review is to point to the practical aspects of the models and the problem of statistical inference in finite samples. Some problems for future research are indicated.     

Variable selection of the quantile varying coefficient regression models

As a useful supplement to mean regression, quantile regression is a completely distribution-free approach and is more robust to heavy-tailed random errors. In this paper, a variable selection procedure for quantile varying coefficient models is proposed by combining local polynomial smoothing with adaptive group LASSO. With an appropriate selection of tuning parameters by the BIC criterion, the theoretical properties of the new procedure, including consistency in variable selection and the oracle property in estimation, are established. The finite sample performance of the newly proposed method is investigated through simulation studies and the analysis of Boston house price data. Numerical studies confirm that the newly proposed procedure (QKLASSO) has both robustness and efficiency for varying coefficient models irrespective of error distribution, which is a good alternative and necessary supplement to the KLASSO method.     

Estimating the error distribution function in semiparametric additive regression models

We consider semiparametric additive regression models with a linear parametric part and a nonparametric part, both involving multivariate covariates. For the nonparametric part we assume two models. In the first, the regression function is unspecified and smooth; in the second, the regression function is additive with smooth components. Depending on the model, the regression curve is estimated by suitable least squares methods. The resulting residual-based empirical distribution function is shown to differ from the error-based empirical distribution function by an additive expression, up to a uniformly negligible remainder term. This result implies a functional central limit theorem for the residual-based empirical distribution function. It is used to test for normal errors.     

Testing for varying zero-inflation and dispersion in generalized Poisson regression models

Homogeneity of dispersion parameters and zero-inflation parameters is a standard assumption in zero-inflated generalized Poisson regression (ZIGPR) models. However, this assumption may be not appropriate in some situations. This work develops a score test for varying dispersion and/or zero-inflation parameter in the ZIGPR models, and corresponding test statistics are obtained. Two numerical examples are given to illustrate our methodology, and the properties of score test statistics are investigated through Monte Carlo simulations.     

Penalized least-squares estimation for regression coefficients in high-dimensional partially linear models

We consider a partially linear model with diverging number of groups of parameters in the parametric component. The variable selection and estimation of regression coefficients are achieved simultaneously by using the suitable penalty function for covariates in the parametric component. An MM-type algorithm for estimating parameters without inverting a high-dimensional matrix is proposed. The consistency and sparsity of penalized least-squares estimators of regression coefficients are discussed under the setting of some nonzero regression coefficients with very small values. It is found that the root p_n/n-consistency and sparsity of the penalized least-squares estimators of regression coefficients cannot be given consideration simultaneously when the number of nonzero regression coefficients with very small values is unknown, where p_n and n, respectively, denote the number of regression coefficients and sample size. The finite sample behaviors of penalized least-squares estimators of regression coefficients and the performance of the proposed algorithm are studied by simulation studies and a real data example.     

Weighted composite quantile regression for partially linear varying coefficient models

Partially linear varying coefficient models (PLVCMs) with heteroscedasticity are considered in this article. Based on composite quantile regression, we develop a weighted composite quantile regression (WCQR) to estimate the non parametric varying coefficient functions and the parametric regression coefficients. The WCQR is augmented using a data-driven weighting scheme. Moreover, the asymptotic normality of proposed estimators for both the parametric and non parametric parts are studied explicitly. In addition, by comparing the asymptotic relative efficiency theoretically and numerically, WCQR method all outperforms the CQR method and some other estimate methods. To achieve sparsity with high-dimensional covariates, we develop a variable selection procedure to select significant parametric components for the PLVCM and prove the method possessing the oracle property. Both simulations and data analysis are conducted to illustrate the finite-sample performance of the proposed methods.     

A new test for high-dimensional regression coefficients in partially linear models

Partially linear regression models are semiparametric models that contain both linear and nonlinear components. They are extensively used in many scientific fields for their flexibility and convenient interpretability. In such analyses, testing the significance of the regression coefficients in the linear component is typically a key focus. Under the high-dimensional setting, i.e., “large p, small n,” the conventional F-test strategy does not apply because the coefficients need to be estimated through regularization techniques. In this article, we develop a new test using a U-statistic of order two, relying on a pseudo-estimate of the nonlinear component from the classical kernel method. Using the martingale central limit theorem, we prove the asymptotic normality of the proposed test statistic under some regularity conditions. We further demonstrate our proposed test's finite-sample performance by simulation studies and by analyzing some breast cancer gene expression data.     

Test for high dimensional regression coefficients of partially linear models

Abstract

Partially linear models attract much attention to investigate the association between predictors and the response variable when the dependency on some predictors may be nonlinear. However, the hypothesis test for significance of predictors is still challenging, especially when the number of predictors is larger than sample size. In this paper, we reconsider the test procedure of Zhong and Chen (2011 Zhong, P., and S. Chen. 2011. Tests for high-dimensional regression coefficients with factorial designs. Journal of the American Statistical Association 106 (493):260–74. doi:10.1198/jasa.2011.tm10284.[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) when regression models have nonlinear components, and propose a generalized U-statistic for testing the linear components of the high dimensional partially linear models. The asymptotic properties of test statistic are obtained under null and alternative hypotheses, where the effect of nonlinear components should be considered and thus is different from that in linear models. Through simulation studies, we demonstrate good finite-sample performance of the proposed test in comparison with the existing methods. The practical utility of our proposed method is illustrated by a real data example.     

Functional response models for intraclass correlation coefficients

Intraclass correlation coefficients (ICC) are employed in a wide range of behavioral, biomedical, psychosocial, and health care related research for assessing reliability of continuous outcomes. The linear mixed-effects model (LMM) is the most popular approach for inference about the ICC. However, since LMM is a normal distribution-based model and non-normal data are the norm rather than the exception in most studies, its applications to real study data always beg the question of inference validity. In this paper, we propose a distribution-free alternative to provide robust inference based on the functional response models. We illustrate the performance of the new approach using both real and simulated data.     

Quantile regression for robust inference on varying coefficient partially nonlinear models

In this paper, we propose a robust statistical inference approach for the varying coefficient partially nonlinear models based on quantile regression. A three-stage estimation procedure is developed to estimate the parameter and coefficient functions involved in the model. Under some mild regularity conditions, the asymptotic properties of the resulted estimators are established. Some simulation studies are conducted to evaluate the finite performance as well as the robustness of our proposed quantile regression method versus the well known profile least squares estimation procedure. Moreover, the Boston housing price data is given to further illustrate the application of the new method.     

B-spline estimation for partially linear varying coefficient composite quantile regression models

Abstract

In this article, a new composite quantile regression estimation (CQR) approach is proposed for partially linear varying coefficient models (PLVCM) under composite quantile loss function with B-spline approximations. The major advantage of the proposed procedures over the existing ones is easy to implement using existing software, and it requires no specification of the error distributions. Under the regularity conditions, the consistency and asymptotic normality of the estimators are also derived. Finally, a simulation study and a real data application are undertaken to assess the finite sample performance of the proposed estimation procedure.     

Estimation of smooth regression functions in monotone response models

We consider the estimation of smooth regression functions in a class of conditionally parametric co-variate-response models. Independent and identically distributed observations are available from the distribution of (Z,X)$(Z,X)$, where Z is a real-valued co-variate with some unknown distribution, and the response X conditional on Z   is distributed according to the density p(·,ψ(Z))$p(·,\psi (Z))$, where p(·,θ)$p(·,\theta )$ is a one-parameter exponential family. The function ψ$\psi$ is a smooth monotone function. Under this formulation, the regression function E(X|Z)$E(X|Z)$ is monotone in the co-variate Z   (and can be expressed as a one–one function of ψ$\psi$); hence the term “monotone response model”. Using a penalized least squares approach that incorporates both monotonicity and smoothness, we develop a scheme for producing smooth monotone estimates of the regression function and also the function ψ$\psi$ across this entire class of models. Point-wise asymptotic normality of this estimator is established, with the rate of convergence depending on the smoothing parameter. This enables construction of Wald-type (point-wise) as well as pivotal confidence sets for ψ$\psi$ and also the regression function. The methodology is extended to the general heteroscedastic model, and its asymptotic properties are discussed.