Principal components regression estimator of the parameters in partially linear models

ABSTRACT

As a compromise between parametric regression and non-parametric regression models, partially linear models are frequently used in statistical modelling. This paper is concerned with the estimation of partially linear regression model in the presence of multicollinearity. Based on the profile least-squares approach, we propose a novel principal components regression (PCR) estimator for the parametric component. When some additional linear restrictions on the parametric component are available, we construct a corresponding restricted PCR estimator. Some simulations are conducted to examine the performance of our proposed estimators and the results are satisfactory. Finally, a real data example is analysed.     

Adaptive Posterior Mode Estimation of a Sparse Sequence for Model Selection

Abstract.  For the problem of estimating a sparse sequence of coefficients of a parametric or non-parametric generalized linear model, posterior mode estimation with a Subbotin( λ , ν ) prior achieves thresholding and therefore model selection when ν   ∈    [0,1] for a class of likelihood functions. The proposed estimator also offers a continuum between the (forward/backward) best subset estimator ( ν  =  0 ), its approximate convexification called lasso ( ν  =  1 ) and ridge regression ( ν  =  2 ). Rather than fixing ν , selecting the two hyperparameters λ and ν adds flexibility for a better fit, provided both are well selected from the data. Considering first the canonical Gaussian model, we generalize the Stein unbiased risk estimate, SURE( λ , ν ), to the situation where the thresholding function is not almost differentiable (i.e. ν    1 ). We then propose a more general selection of λ and ν by deriving an information criterion that can be employed for instance for the lasso or wavelet smoothing. We investigate some asymptotic properties in parametric and non-parametric settings. Simulations and applications to real data show excellent performance.     

Polynomial Spline Estimation for a Generalized Additive Coefficient Model

Abstract.  We study a semiparametric generalized additive coefficient model (GACM), in which linear predictors in the conventional generalized linear models are generalized to unknown functions depending on certain covariates, and approximate the non-parametric functions by using polynomial spline. The asymptotic expansion with optimal rates of convergence for the estimators of the non-parametric part is established. Semiparametric generalized likelihood ratio test is also proposed to check if a non-parametric coefficient can be simplified as a parametric one. A conditional bootstrap version is suggested to approximate the distribution of the test under the null hypothesis. Extensive Monte Carlo simulation studies are conducted to examine the finite sample performance of the proposed methods. We further apply the proposed model and methods to a data set from a human visceral Leishmaniasis study conducted in Brazil from 1994 to 1997. Numerical results outperform the traditional generalized linear model and the proposed GACM is preferable.     

Empirical Likelihood-Based Inferences for Generalized Partially Linear Models

Abstract.  This paper considers generalized partially linear models. We propose empirical likelihood-based statistics to construct confidence regions for the parametric and non-parametric components. The resulting statistics are shown to be asymptotically chi-square distributed. Finite-sample performance of the proposed statistics is assessed by simulation experiments. The proposed methods are applied to a data set from an AIDS clinical trial.     

An Adaptive Two-stage Estimation Method for Additive Models

Abstract.  In this paper, a two-stage estimation method for non-parametric additive models is investigated. Differing from Horowitz and Mammen's two-stage estimation, our first-stage estimators are designed not only for dimension reduction but also as initial approximations to all of the additive components. The second-stage estimators are obtained by using one-dimensional non-parametric techniques to refine the first-stage ones. From this procedure, we can reveal a relationship between the regression function spaces and convergence rate, and then provide estimators that are optimal in the sense that, better than the usual one-dimensional mean-squared error (MSE) of the order n ^−4/5 , the MSE of the order n ^{− 1} can be achieved when the underlying models are actually parametric. This shows that our estimation procedure is adaptive in a certain sense. Also it is proved that the bandwidth that is selected by cross-validation depends only on one-dimensional kernel estimation and maintains the asymptotic optimality. Simulation studies show that the new estimators of the regression function and all components outperform the existing estimators, and their behaviours are often similar to that of the oracle estimator.     

A Semi-parametric Regression Model with Errors in Variables

Abstract.  In this paper, we consider a partial linear regression model with measurement errors in possibly all the variables. We use a method of moments and deconvolution to construct a new class of parametric estimators together with a non-parametric kernel estimator. Strong convergence, optimal rate of weak convergence and asymptotic normality of the estimators are investigated.     

Efficient Estimation in Marginal Partially Linear Models for Longitudinal/Clustered Data Using Splines

Abstract.  We consider marginal semiparametric partially linear models for longitudinal/clustered data and propose an estimation procedure based on a spline approximation of the non-parametric part of the model and an extension of the parametric marginal generalized estimating equations (GEE). Our estimates of both parametric part and non-parametric part of the model have properties parallel to those of parametric GEE, that is, the estimates are efficient if the covariance structure is correctly specified and they are still consistent and asymptotically normal even if the covariance structure is misspecified. By showing that our estimate achieves the semiparametric information bound, we actually establish the efficiency of estimating the parametric part of the model in a stronger sense than what is typically considered for GEE. The semiparametric efficiency of our estimate is obtained by assuming only conditional moment restrictions instead of the strict multivariate Gaussian error assumption.     

Goodness-of-Fit Tests for Multiplicative Models with Dependent Data

Abstract.  Several classical time series models can be written as a regression model between the components of a strictly stationary bivariate process. Some of those models, such as the ARCH models, share the property of proportionality of the regression function and the scale function, which is an interesting feature in econometric and financial models. In this article, we present a procedure to test for this feature in a non-parametric context. The test is based on the difference between two non-parametric estimators of the distribution of the regression error. Asymptotic results are proved and some simulations are shown in the paper in order to illustrate the finite sample properties of the procedure.     

Simple Transformation Techniques for Improved Non-parametric Regression

We propose and investigate two new methods for achieving less bias in non- parametric regression. We show that the new methods have bias of order h ⁴, where h is a smoothing parameter, in contrast to the basic kernel estimator's order h ². The methods are conceptually very simple. At the first stage, perform an ordinary non-parametric regression on { x_i , Y_i } to obtain m^ ( x_i ) (we use local linear fitting). In the first method, at the second stage, repeat the non-parametric regression but on the transformed dataset { m^ ( x_i , Y_i )}, taking the estimator at x to be this second stage estimator at m^ ( x ). In the second, and more appealing, method, again perform non-parametric regression on { m^ ( x_i , Y_i )}, but this time make the kernel weights depend on the original x scale rather than using the m^ ( x ) scale. We concentrate more of our effort in this paper on the latter because of its advantages over the former. Our emphasis is largely theoretical, but we also show that the latter method has practical potential through some simulated examples.     

Splice plots for generalized linear models

ABSTRACT

We develop splice plots as a diagnostic tool for parametric generalized linear models. Splice plots use the independence of the outcome and explanatory measures given the regression function. Plotting differences between the estimated parametric regression function and non-parametric estimates of the regression function computed in small neighborhoods of the fitted values from the parametric model can be used to assess model fit.     

Rao's score test with nonparametric density estimators

Traditionally, Rao's score (RS) tests are constructed under a parametric specification of the probability density function. We estimate the density function by a non-parametric estimator and consider a semi-parametric Rao's score (SPRS) test for a set of hypotheses concerning the parametric model. The asymptotic distribution of the SPRS test is analyzed. Further, for the regression model, we carry out a set of Monte Carlo experiments to analyze the size and power of the SPRS test in small samples. The robustness of SPRS test to the choice of the density estimator is also analyzed.     

Semiparametric Time-Varying Coefficients Regression Model for Longitudinal Data

Abstract.  In this paper, we consider a semiparametric time-varying coefficients regression model where the influences of some covariates vary non-parametrically with time while the effects of the remaining covariates follow certain parametric functions of time. The weighted least squares type estimators for the unknown parameters of the parametric coefficient functions as well as the estimators for the non-parametric coefficient functions are developed. We show that the kernel smoothing that avoids modelling of the sampling times is asymptotically more efficient than a single nearest neighbour smoothing that depends on the estimation of the sampling model. The asymptotic optimal bandwidth is also derived. A hypothesis testing procedure is proposed to test whether some covariate effects follow certain parametric forms. Simulation studies are conducted to compare the finite sample performances of the kernel neighbourhood smoothing and the single nearest neighbour smoothing and to check the empirical sizes and powers of the proposed testing procedures. An application to a data set from an AIDS clinical trial study is provided for illustration.     

Non-parametric Analysis of Covariance – The Case of Inhomogeneous and Heteroscedastic Noise

Abstract.  The purpose of this paper was to propose a procedure for testing the equality of several regression curves f _i in non-parametric regression models when the noise is inhomogeneous and heteroscedastic, i.e. when the variances depend on the regressor and may vary between groups. The presented approach is very natural because it transfers the maximum likelihood statistic from a heteroscedastic one-way analysis of variance to the context of non-parametric regression. The maximum likelihood estimators will be replaced by kernel estimators of the regression functions f _i . It is shown that the asymptotic distribution of the obtained test-statistic is nuisance parameter free. Asymptotic efficiency is compared with a test of Dette & Neumeyer [Annals of Statistics (2001) Vol. 29, 1361–1400] and it is shown that the new test is asymptotically uniformly more powerful. For practical purposes, a bootstrap variant is suggested. In a simulation study, level and power of this test will be briefly investigated and compared with other procedures. In summary, our theoretical findings are supported by this study. Finally, a crop yield experiment is reanalysed.     

Component Selection in the Additive Regression Model

Abstract. Similar to variable selection in the linear model, selecting significant components in the additive model is of great interest. However, such components are unknown, unobservable functions of independent variables. Some approximation is needed. We suggest a combination of penalized regression spline approximation and group variable selection, called the group‐bridge‐type spline method (GBSM), to handle this component selection problem with a diverging number of correlated variables in each group. The proposed method can select significant components and estimate non‐parametric additive function components simultaneously. To make the GBSM stable in computation and adaptive to the level of smoothness of the component functions, weighted power spline bases and projected weighted power spline bases are proposed. Their performance is examined by simulation studies. The proposed method is extended to a partial linear regression model analysis with real data, and gives reliable results.     

Empirical likelihood inference in mixture of semiparametric varying-coefficient models for longitudinal data with non-ignorable dropout

In this paper, empirical likelihood inference in mixture of semiparametric varying-coefficient models for longitudinal data with non-ignorable dropout is investigated. We estimate the non-parametric function based on the estimating equations and the local linear profile-kernel method. An empirical log-likelihood ratio statistic for parametric components is proposed to construct confidence regions and is shown to be an asymptotically chi-squared distribution. The non-parametric version of Wilk's theorem is also derived. A simulation study is undertaken to illustrate the finite sample performance of the proposed method.     

Inferences in dynamic logit models in semi-parametric setup for repeated binary data

Binary dynamic fixed and mixed logit models are extensively studied in the literature. These models are developed to examine the effects of certain fixed covariates through a parametric regression function as a part of the models. However, there are situations where one may like to consider more covariates in the model but their direct effect is not of interest. In this paper we propose a generalization of the existing binary dynamic logit (BDL) models to the semi-parametric longitudinal setup to address this issue of additional covariates. The regression function involved in such a semi-parametric BDL model contains (i) a parametric linear regression function in some primary covariates, and (ii) a non-parametric function in certain secondary covariates. We use a simple semi-parametric conditional quasi-likelihood approach for consistent estimation of the non-parametric function, and a semi-parametric likelihood approach for the joint estimation of the main regression and dynamic dependence parameters of the model. The finite sample performance of the estimation approaches is examined through a simulation study. The asymptotic properties of the estimators are also discussed. The proposed model and the estimation approaches are illustrated by reanalysing a longitudinal infectious disease data.     

On Semiparametric Mode Regression Estimation

It has been found that, for a variety of probability distributions, there is a surprising linear relation between mode, mean, and median. In this article, the relation between mode, mean, and median regression functions is assumed to follow a simple parametric model. We propose a semiparametric conditional mode (mode regression) estimation for an unknown (unimodal) conditional distribution function in the context of regression model, so that any m-step-ahead mean and median forecasts can then be substituted into the resultant model to deliver m-step-ahead mode prediction. In the semiparametric model, Least Squared Estimator (LSEs) for the model parameters and the simultaneous estimation of the unknown mean and median regression functions by the local linear kernel method are combined to infer about the parametric and nonparametric components of the proposed model. The asymptotic normality of these estimators is derived, and the asymptotic distribution of the parameter estimates is also given and is shown to follow usual parametric rates in spite of the presence of the nonparametric component in the model. These results are applied to obtain a data-based test for the dependence of mode regression over mean and median regression under a regression model.     

Regions of alternatives with high and low power for goodness-of-fit tests

In the present paper we find finite dimensional spaces W of alternatives with high power for a given class of tests and non-parametric alternatives. On the orthogonal complement of W the power function is flat. These methods can be used to reduce the dimension of interesting alternatives. We sketch a device how to calculate (approximately) an alternative with maximum power of a fixed test on a given ball of certain non-parametric alternatives.

The calculations are done within different asymptotic models specified by signal detection tests. Specific tests are Kolmogorov–Smirnov type tests, integral tests (like the Anderson and Darling test) and Rényi tests for hazard based models. The statistical meaning and interpretation of the spaces of alternatives with high power is discussed. These alternatives belong to least favorable directions of a class of statistical functionals which are linear combinations of quantile functions. For various cases their meaning is explained for parametric submodels, in particular for location alternatives.     

Semiparametric Regression with Kernel Error Model

Abstract.  We propose and study a class of regression models, in which the mean function is specified parametrically as in the existing regression methods, but the residual distribution is modelled non-parametrically by a kernel estimator, without imposing any assumption on its distribution. This specification is different from the existing semiparametric regression models. The asymptotic properties of such likelihood and the maximum likelihood estimate (MLE) under this semiparametric model are studied. We show that under some regularity conditions, the MLE under this model is consistent (when compared with the possibly pseudo-consistency of the parameter estimation under the existing parametric regression model), is asymptotically normal with rate and efficient. The non-parametric pseudo-likelihood ratio has the Wilks property as the true likelihood ratio does. Simulated examples are presented to evaluate the accuracy of the proposed semiparametric MLE method.     

局部平稳部分线性面板数据模型的统计推断

部分线性模型是一类非常重要的半参数回归模型,由于它既含有参数部分又含有非参数部分,与常规的线性模型相比具有更强的适应性和解释能力。文章研究带有局部平稳协变量的固定效应部分线性面板数据模型的统计推断。首先提出一个两阶段估计方法得到模型中未知参数和非参数函数的估计,并证明估计量的渐近性质,然后运用不变原理构造出非参数函数的一致置信带,最后通过数值模拟研究和实例分析验证了该方法的有效性。