Efficient Penalized Estimation for Linear Regression Model

This paper develops new penalized estimation for linear regression model. We prove that the new method, which is referred to as efficient penalized estimation, is selection consistent, and more asymptotically efficient than the original one. Besides, we construct a new selector called efficient BIC Selector to tune the regularization parameter in the new estimation, which is shown to be consistent. Our simulation results suggest that the new method may bring significant improvement relative to the original penalized estimation. In addition, we employ a real data set to illustrate the application of the efficient penalized estimation.     

Paths Following Algorithm for Penalized Logistic Regression Using SCAD and MCP

In this article, we develop a generalized penalized linear unbiased selection (GPLUS) algorithm. The GPLUS is designed to compute the paths of penalized logistic regression based on the smoothly clipped absolute deviation (SCAD) and the minimax concave penalties (MCP). The main idea of the GPLUS is to compute possibly multiple local minimizers at individual penalty levels by continuously tracing the minimizers at different penalty levels. We demonstrate the feasibility of the proposed algorithm in logistic and linear regression. The simulation results favor the SCAD and MCP’s selection accuracy encompassing a suitable range of penalty levels.     

Penalized quantile regression for dynamic panel data

This paper studies penalized quantile regression for dynamic panel data with fixed effects, where the penalty involves l₁ shrinkage of the fixed effects. Using extensive Monte Carlo simulations, we present evidence that the penalty term reduces the dynamic panel bias and increases the efficiency of the estimators. The underlying intuition is that there is no need to use instrumental variables for the lagged dependent variable in the dynamic panel data model without fixed effects. This provides an additional use for the shrinkage models, other than model selection and efficiency gains. We propose a Bayesian information criterion based estimator for the parameter that controls the degree of shrinkage. We illustrate the usefulness of the novel econometric technique by estimating a “target leverage” model that includes a speed of capital structure adjustment. Using the proposed penalized quantile regression model the estimates of the adjustment speeds lie between 3% and 44% across the quantiles, showing strong evidence that there is substantial heterogeneity in the speed of adjustment among firms.     

Penalized regression with model‐based penalties

Nonparametric regression techniques such as spline smoothing and local fitting depend implicitly on a parametric model. For instance, the cubic smoothing spline estimate of a regression function ∫ μ based on observations ti, Yi is the minimizer of Σ{Yi ‐ μ(ti)}² + λ∫(μ′′)². Since ∫(μ″)² is zero when μ is a line, the cubic smoothing spline estimate favors the parametric model μ(t) = α_o + α₁t. Here the authors consider replacing ∫(μ″)² with the more general expression ∫(Lμ)² where L is a linear differential operator with possibly nonconstant coefficients. The resulting estimate of μ performs well, particularly if Lμ is small. They present an O(n) algorithm for the computation of μ. This algorithm is applicable to a wide class of L's. They also suggest a method for the estimation of L. They study their estimates via simulation and apply them to several data sets.     

Best Quadratic Unbiased Prediction in a General Linear Model with Stochastic Regression Coefficients

In this article, we discuss on how to predict a combined quadratic parametric function of the form β′ H β + hσ² in a general linear model with stochastic regression coefficients denoted by y  =  X β +  e . Firstly, the quadratic predictability of β′ H β + hσ² is investigated to obtain a quadratic unbiased predictor (QUP) via a general method of structuring an unbiased estimator. This QUP is also optimal in some situations and therefore we hope it will be a fine predictor. To show this idea, we apply the Lagrange multipliers method to this problem and finally reach the expected conclusion through permutation matrix techniques.     

A Direct Approach to Understanding Posterior Consistency of Bayesian Regression Problems

Previous approaches to establishing posterior consistency of Bayesian regression problems have used general theorems that involve verifying sufficient conditions for posterior consistency. In this article, we consider a direct approach by computing the posterior density explicitly and evaluating its asymptotic behavior. For this purpose, we deal with a sample size dependent prior based on a truncated regression function with increasing sample size, and evaluate the asymptotic properties of the resulting posterior. Based on a concept called posterior density consistency, we attempt to understand posterior consistency. As an application, we illustrate that the posterior density of an orthogonal semiparametric regression model is consistent.     

Penalized proportion estimation for non parametric mixture of regressions

Abstract

In this article, we propose a penalized local log-likelihood method to locally select the number of components in non parametric finite mixture of regression models via proportion shrinkage method. Mean functions and variance functions are estimated simultaneously. We show that the number of components can be estimated consistently, and further establish asymptotic normality of functional estimates. We use a modified EM algorithm to estimate the unknown functions. Simulations are conducted to demonstrate the performance of the proposed method. We illustrate our method via an empirical analysis of the housing price index data of United States.     

Covariate-Adjusted Partially Linear Regression Models

Motivated by covariate-adjusted regression (CAR) proposed by Sentürk and Müller (2005 Sentürk , D. , Müller , H. G. ( 2005 ). Covariate-adjusted regression . Biometrika 92 : 75 – 89 .[Crossref], [Web of Science ®] , [Google Scholar]) and an application problem, in this article we introduce and investigate a covariate-adjusted partially linear regression model (CAPLM), in which both response and predictor vector can only be observed after being distorted by some multiplicative factors, and an additional variable such as age or period is taken into account. Although our model seems to be a special case of covariate-adjusted varying coefficient model (CAVCM) given by Sentürk (2006 Sentürk , D. ( 2006 ). Covariate-adjusted varying coefficient models . Biostatistics 7 : 235 – 251 .[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]), the data types of CAPLM and CAVCM are basically different and then the methods for inferring the two models are different. In this article, the estimate method motivated by Cui et al. (2008 Cui , X. , Guo , W. S. , Lin , L. , Zhu , L. X. ( 2008 ). Covariate-adjusted nonlinear regression . Ann. Statist. 37 : 1839 – 1870 . [Google Scholar]) is employed to infer the new model. Furthermore, under some mild conditions, the asymptotic normality of estimator for the parametric component is obtained. Combined with the consistent estimate of asymptotic covariance, we obtain confidence intervals for the regression coefficients. Also, some simulations and a real data analysis are made to illustrate the new model and methods.     

Nonparametric Principal Components Regression

Principal components regression (PCR) is used in resolving the multicollinearity problem but specification bias occurs due to the selection only of the important principal components to be included resulting in the deterioration of predictive ability of the model. We propose the PCR in a nonparametric framework to address the multicollinearity problem while minimizing the specification bias that affects predictive ability of the model. The simulation study illustrated that nonparametric PCR addresses the multicollinearity problem while retaining higher predictive ability relative to parametric principal components regression model.     

Regression with outlier shrinkage

We propose a robust regression method called regression with outlier shrinkage (ROS) for the traditional n>p$n>p$ cases. It improves over the other robust regression methods such as least trimmed squares (LTS) in the sense that it can achieve maximum breakdown value and full asymptotic efficiency simultaneously. Moreover, its computational complexity is no more than that of LTS. We also propose a sparse estimator, called sparse regression with outlier shrinkage (SROS), for robust variable selection and estimation. It is proven that SROS can not only give consistent selection but also estimate the nonzero coefficients with full asymptotic efficiency under the normal model. In addition, we introduce a concept of nearly regression equivariant estimator for understanding the breakdown properties of sparse estimators, and prove that SROS achieves the maximum breakdown value of nearly regression equivariant estimators. Numerical examples are presented to illustrate our methods.     

Component selection in additive quantile regression models

Nonparametric additive models are powerful techniques for multivariate data analysis. Although many procedures have been developed for estimating additive components both in mean regression and quantile regression, the problem of selecting relevant components has not been addressed much especially in quantile regression. We present a doubly-penalized estimation procedure for component selection in additive quantile regression models that combines basis function approximation with a ridge-type penalty and a variant of the smoothly clipped absolute deviation penalty. We show that the proposed estimator identifies relevant and irrelevant components consistently and achieves the nonparametric optimal rate of convergence for the relevant components. We also provide an accurate and efficient computation algorithm to implement the estimator and demonstrate its performance through simulation studies. Finally, we illustrate our method via a real data example to identify important body measurements to predict percentage of body fat of an individual.     

Faster Convergence to the Estimation of Quadratic Variation with Microstructure Noise

The continuous quadratic variation of asset return plays a critical role for high-frequency trading. However, the microstructure noise could bias the estimation of the continuous quadratic variation. Zhang et al. (2005 Zhang, L., Mykland, P., Ait-Sahalia, Y. (2005). A tale of two time scales: determining integrated volatility with noisy high-frequency data. J. Amer. Statist. Assoc. 100(472):1394–1411.[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) proposed a batch estimator for the continuous quadratic variation of high-frequency data in the presence of microstructure noise. It gives the estimates after all the data arrive. This article proposes a recursive version of their estimator that outputs variation estimates as the data arrive. Our estimator gives excellent estimates well before all the data arrive. Both real high-frequency futures data and simulation data confirm the performance of the recursive estimator.     

Penalized Maximum Likelihood Principle for Choosing Ridge Parameter

We consider the problem of choosing the ridge parameter. Two penalized maximum likelihood (PML) criteria based on a distribution-free and a data-dependent penalty function are proposed. These PML criteria can be considered as “continuous” versions of AIC. A systematic simulation is conducted to compare the suggested criteria to several existing methods. The simulation results strongly support the use of our method. The method is also applied to two real data sets.     

Flat-Top Realized Kernel Estimation of Quadratic Covariation With Nonsynchronous and Noisy Asset Prices

This article develops a general multivariate additive noise model for synchronized asset prices and provides a multivariate extension of the generalized flat-top realized kernel estimators, analyzed earlier by Varneskov (2014), to estimate its quadratic covariation. The additive noise model allows for α-mixing dependent exogenous noise, random sampling, and an endogenous noise component that encompasses synchronization errors, lead-lag relations, and diurnal heteroscedasticity. The various components may exhibit polynomially decaying autocovariances. In this setting, the class of estimators considered is consistent, asymptotically unbiased, and mixed Gaussian at the optimal rate of convergence, n^1/4. A simple finite sample correction based on projections of symmetric matrices ensures positive definiteness without altering the asymptotic properties of the estimators. It, thereby, guarantees the existence of nonlinear transformations of the estimated covariance matrix such as correlations and realized betas, which inherit the asymptotic properties from the flat-top realized kernel estimators. An empirically motivated simulation study assesses the choice of sampling scheme and projection rule, and it shows that flat-top realized kernels have a desirable combination of robustness and efficiency relative to competing estimators. Last, an empirical analysis of signal detection and out-of-sample predictions for a portfolio of six stocks of varying size and liquidity illustrates the use and properties of the new estimators.     

A Combined Nonlinear Programming Model and Kibria Method for Choosing Ridge Parameter Regression

Ridge regression solves multicollinearity problems by introducing a biasing parameter that is called ridge parameter; it shrinks the estimates as well as their standard errors in order to reach acceptable results. Many methods are available for estimating a ridge parameter. This article has considered some of these methods and also proposed a combined nonlinear programming model and Kibria method. A simulation study has been made to evaluate the performance of the proposed estimators based on the minimum mean squared error criterion. The simulation study indicates that under certain conditions the proposed estimators outperform the least squares (LS) estimators and other popular existing estimators. Moreover, the new proposed model is applied on dataset that suffers also from the presence of heteroscedastic errors.     

Nonparametric Regression as an Example of Model Choice

Nonparametric regression can be considered as a problem of model choice. In this article, we present the results of a simulation study in which several nonparametric regression techniques including wavelets and kernel methods are compared with respect to their behavior on different test beds. We also include the taut-string method whose aim is not to minimize the distance of an estimator to some “true” generating function f but to provide a simple adequate approximation to the data. Test beds are situations where a “true” generating f exists and in this situation it is possible to compare the estimates of f with f itself. The measures of performance we use are the L²- and the L^∞-norms and the ability to identify peaks.     

Regression Tolerance Intervals

In this article, we discuss the utility of tolerance intervals for various regression models. We begin with a discussion of tolerance intervals for linear and nonlinear regression models. We then introduce a novel method for constructing nonparametric regression tolerance intervals by extending the well-established procedure for univariate data. Simulation results and application to real datasets are presented to help visualize regression tolerance intervals and to demonstrate that the methods we discuss have coverage probabilities very close to the specified nominal confidence level.     

Penalized empirical likelihood for quantile regression with missing covariates and auxiliary information

Based on the inverse probability weight method, we, in this article, construct the empirical likelihood (EL) and penalized empirical likelihood (PEL) ratios of the parameter in the linear quantile regression model when the covariates are missing at random, in the presence and absence of auxiliary information, respectively. It is proved that the EL ratio admits a limiting Chi-square distribution. At the same time, the asymptotic normality of the maximum EL and PEL estimators of the parameter is established. Also, the variable selection of the model in the presence and absence of auxiliary information, respectively, is discussed. Simulation study and a real data analysis are done to evaluate the performance of the proposed methods.     

Copula Density Estimation by Total Variation Penalized Likelihood

Copulas are full measures of dependence among random variables. They are increasingly popular among academics and practitioners in financial econometrics for modeling comovements between markets, risk factors, and other relevant variables. A copula's hidden dependence structure that couples a joint distribution with its marginals makes a parametric copula non-trivial. An approach to bivariate copula density estimation is introduced that is based on a penalized likelihood with a total variation penalty term. Adaptive choice of the amount of regularization is based on approximate Bayesian Information Criterion (BIC) type scores. Performance are evaluated through the Monte Carlo simulation.     

Adjusted Confidence Bands in Nonparametric Regression

Suppose we have {(x _i , y _i )} i = 1, 2,…, n, a sequence of independent observations. We wish to find approximate 1 ? α simultaneous confidence bands for the regression curve. Many previous confidence bands in the literature have practical difficulties. In this article, the local linear smoother is used to estimate the regression curve. The bias of the estimator is considered. Different methods of constructing confidence bands are discussed. Finally, a possible method incorporating logistic regression in an innovative way is proposed to construct the bands for random designs. Simulations are used to study the performance or properties of the methods. The procedure for constructing confidence bands is entirely data-driven. The advantage of the proposed method is that it is simple to use and can be applied to random designs. It can be considered as a practically useful and efficient method.