Empirical Likelihood Inference for the Parameter in Additive Partially Linear EV Models

In this article, we consider empirical likelihood inference for the parameter in the additive partially linear models when the linear covariate is measured with error. By correcting for attenuation, a corrected-attenuation empirical log-likelihood ratio statistic for the unknown parameter β, which is of primary interest, is suggested. We show that the proposed statistic is asymptotically standard chi-square distribution without requiring the undersmoothing of the nonparametric components, and hence it can be directly used to construct the confidence region for the parameter β. Some simulations indicate that, in terms of comparison between coverage probabilities and average lengths of the confidence intervals, the proposed method performs better than the profile-based least-squares method. We also give the maximum empirical likelihood estimator (MELE) for the unknown parameter β, and prove the MELE is asymptotically normal under some mild conditions.     

Akaike Information Criterion for Selecting Variables in the Nested Error Regression Model

The Akaike Information Criterion (AIC) is developed for selecting the variables of the nested error regression model where an unobservable random effect is present. Using the idea of decomposing the likelihood into two parts of “within” and “between” analysis of variance, we derive the AIC when the number of groups is large and the ratio of the variances of the random effects and the random errors is an unknown parameter. The proposed AIC is compared, using simulation, with Mallows' C _p , Akaike's AIC, and Sugiura's exact AIC. Based on the rates of selecting the true model, it is shown that the proposed AIC performs better.     

Nonnested Testing for Competing Autoregressive Dynamic Models Estimated by Instrumental Variables

In this work we investigate nonnested tests for two competing univariate dynamic linear models with autoregressive disturbances, where the motivation for instrumental variable estimation is mainly due to the recognized presence of current endogenous variables in the regression function, either in one or both models. As the previous transformation of both models yields regression functions which are nonlinear in the parameters, the attractive Gauss-Newton regression (GNR) approach, firstly advocated by Davidson and Mackinnon (1981 Davidson , R. , Mackinnon , J. G. ( 1981 ). Several tests for model specification in the presence of alternative hypotheses . Econometrica 49 : 781 – 93 .[Crossref], [Web of Science ®] , [Google Scholar]), will be used to obtain the results.     

Kernel Density-Based Linear Regression Estimate

For linear regression models with non normally distributed errors, the least squares estimate (LSE) will lose some efficiency compared to the maximum likelihood estimate (MLE). In this article, we propose a kernel density-based regression estimate (KDRE) that is adaptive to the unknown error distribution. The key idea is to approximate the likelihood function by using a nonparametric kernel density estimate of the error density based on some initial parameter estimate. The proposed estimate is shown to be asymptotically as efficient as the oracle MLE which assumes the error density were known. In addition, we propose an EM type algorithm to maximize the estimated likelihood function and show that the KDRE can be considered as an iterated weighted least squares estimate, which provides us some insights on the adaptiveness of KDRE to the unknown error distribution. Our Monte Carlo simulation studies show that, while comparable to the traditional LSE for normal errors, the proposed estimation procedure can have substantial efficiency gain for non normal errors. Moreover, the efficiency gain can be achieved even for a small sample size.     

Optimal Simultaneous Confidence Bands in Multiple Linear Regression with Predictor Variables Constrained in an Ellipsoidal Region

A simultaneous confidence band provides useful information on the plausible range of an unknown regression model function, just as a confidence interval gives the plausible range of an unknown parameter. For a multiple linear regression model, confidence bands of different shapes, such as the hyperbolic band and the constant width band, can be constructed and the predictor variable region over which a confidence band is constructed can take various forms. One interesting but unsolved problem is to find the optimal (shape) confidence band over an ellipsoidal region χ_E under the Minimum Volume Confidence Set (MVCS) criterion of Liu and Hayter (2007 Liu, W., Hayter, A.J. (2007). Minimum area confidence set optimality for confidence bands in simple linear regression. J. Amer. Statist. Assoc. 102:181–190.[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) and Liu et al. (2009 Liu, W., Bretz, F., Hayter, A.J., Wynn, H.P. (2009). Assessing non-superiority, non-inferiority or equivalence when comparing two regression models over a restricted covariate region. Biometrics 65:1279–1287.[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]). This problem is challenging as it involves optimization over an unknown function that determines the shape of the confidence band over χ_E. As a step towards solving this difficult problem, in this paper, we introduce a family of confidence bands over χ_E, called the inner-hyperbolic bands, which includes the hyperbolic and constant-width bands as special cases. We then search for the optimal confidence band within this family under the MVCS criterion. The conclusion from this study is that the hyperbolic band is not optimal even within this family of inner-hyperbolic bands and so cannot be the overall optimal band. On the other hand, the constant width band can be optimal within the family of inner-hyperbolic bands when the region χ_E is small and so might be the overall optimal band.     

A Compressive Sensing Based Analysis of Anomalies in Generalized Linear Models

In this article, we present a compressive sensing based framework for generalized linear model regression that employs a two-component noise model and convex optimization techniques to simultaneously detect outliers and determine optimally sparse representations of noisy data from arbitrary sets of basis functions. We then extend our model to include model order reduction capabilities that can uncover inherent sparsity in regression coefficients and achieve simple, superior fits. Second, we use the mixed ?₂/?₁ norm to develop another model that can efficiently uncover block-sparsity in regression coefficients. By performing model order reduction over all independent variables and basis functions, our algorithms successfully deemphasize the effect of independent variables that become uncorrelated with dependent variables. This desirable property has various applications in real-time anomaly detection, such as faulty sensor detection and sensor jamming in wireless sensor networks. After developing our framework and inheriting a stable recovery theorem from compressive sensing theory, we present two simulation studies on sparse or block-sparse problems that demonstrate the superior performance of our algorithms with respect to (1) classic outlier-invariant regression techniques like least absolute value and iteratively reweighted least-squares and (2) classic sparse-regularized regression techniques like LASSO.     

Generalized Analysis-of-variance-type Test for the Single-index Quantile Model

In this paper, we are concerned with a test for the index parameter and index function in the single-index model. Based on the estimates obtained by the quantile regression, we extend the generalized analysis-of-variance-type test to the single-index model. We investigate the asymptotic behavior of the proposed test and demonstrate that its limiting null distribution follows an asymptotically χ²-distribution. The simulation studies and real data applications are conducted to illustrate the finite sample performance of the proposed methods.     

A-optimal designs for heteroscedastic multifactor regression models

Abstract

This paper searches for A-optimal designs for Kronecker product and additive regression models when the errors are heteroscedastic. Sufficient conditions are given so that A-optimal designs for the multifactor models can be built from A-optimal designs for their sub-models with a single factor. The results of an efficiency study carried out to check the adequacy of the products of optimal designs for uni-factor marginal models when these are used to estimate different multi-factor models are also reported.     

Selection of bandwidth for kernel regression

ABSTRACT

The most important factor in kernel regression is a choice of a bandwidth. Considerable attention has been paid to extension the idea of an iterative method known for a kernel density estimate to kernel regression. Data-driven selectors of the bandwidth for kernel regression are considered. The proposed method is based on an optimally balanced relation between the integrated variance and the integrated square bias. This approach leads to an iterative quadratically convergent process. The analysis of statistical properties shows the rationale of the proposed method. In order to see statistical properties of this method the consistency is determined. The utility of the method is illustrated through a simulation study and real data applications.     

Analysis of ordered probit model with surrogate response data and measurement error in covariates

ABSTRACT

Often in data arising out of epidemiologic studies, covariates are subject to measurement error. In addition ordinal responses may be misclassified into a category that does not reflect the true state of the respondents. The goal of the present work is to develop an ordered probit model that corrects for the classification errors in ordinal responses and/or measurement error in covariates. Maximum likelihood method of estimation is used. Simulation study reveals the effect of ignoring measurement error and/or classification errors on the estimates of the regression coefficients. The methodology developed is illustrated through a numerical example.