Lasso-type estimation for covariate-adjusted linear model

Lasso is popularly used for variable selection in recent years. In this paper, lasso-type penalty functions including lasso and adaptive lasso are employed in simultaneously variable selection and parameter estimation for covariate-adjusted linear model, where the predictors and response cannot be observed directly and distorted by some observable covariate through some unknown multiplicative smooth functions. Estimation procedures are proposed and some asymptotic properties are obtained under some mild conditions. It deserves noting that under appropriate conditions, the adaptive lasso estimator correctly select covariates with nonzero coefficients with probability converging to one and that the estimators of nonzero coefficients have the same asymptotic distribution that they would have if the zero coefficients were known in advance, i.e. the adaptive lasso estimator has the oracle property in the sense of Fan and Li [6]. Simulation studies are carried out to examine its performance in finite sample situations and the Boston Housing data is analyzed for illustration.     

Estimation of the error distribution in a varying coefficient regression model

This paper deals with the estimation of the error distribution function in a varying coefficient regression model. We propose two estimators and study their asymptotic properties by obtaining uniform stochastic expansions. The first estimator is a residual-based empirical distribution function. We study this estimator when the varying coefficients are estimated by under-smoothed local quadratic smoothers. Our second estimator which exploits the fact that the error distribution has mean zero is a weighted residual-based empirical distribution whose weights are chosen to achieve the mean zero property using empirical likelihood methods. The second estimator improves on the first estimator. Bootstrap confidence bands based on the two estimators are also discussed.     

Bootstrap adaptive estimation: The trimmed-mean example

We consider the problem of choosing among a class of possible estimators by selecting the estimator with the smallest bootstrap estimate of finite sample variance. This is an alternative to using cross-validation to choose an estimator adaptively. The problem of a confidence interval based on such an adaptive estimator is considered. We illustrate the ideas by applying the method to the problem of choosing the trimming proportion of an adaptive trimmed mean. It is shown that a bootstrap adaptive trimmed mean is asymptotically normal with an asymptotic variance equal to the smallest among trimmed means. The asymptotic coverage probability of a bootstrap confidence interval based on such adaptive estimators is shown to have the nominal level. The intervals based on the asymptotic normality of the estimator share the same asymptotic result, but have poor small-sample properties compared to the bootstrap intervals. A small-sample simulation demonstrates that bootstrap adaptive trimmed means adapt themselves rather well even for samples of size 10.     

Model Selection for Vector Autoregressive Processes via Adaptive Lasso

Determination of the best subset is an important step in vector autoregressive (VAR) modeling. Traditional methods either conduct subset selection and parameter estimation separately or compute expensively. In this article, we propose a VAR model selection procedure using adaptive Lasso, for it is computational efficient and can select subset and estimate parameters simultaneously. By proper choice of tuning parameters, we can choose the correct subset and obtain the asymptotic normality of the non zero parameters. Simulation studies and real data analysis show that adaptive Lasso performs better than existing methods in VAR model fitting and prediction.     

Lasso with convex loss: Model selection consistency and estimation

Abstract

Variable selection is a fundamental challenge in statistical learning if one works with data sets containing huge amount of predictors. In this artical we consider procedures popular in model selection: Lasso and adaptive Lasso. Our goal is to investigate properties of estimators based on minimization of Lasso-type penalized empirical risk with a convex loss function, in particular nondifferentiable. We obtain theorems concerning rate of convergence in estimation, consistency in model selection and oracle properties for Lasso estimators if the number of predictors is fixed, i.e. it does not depend on the sample size. Moreover, we study properties of Lasso and adaptive Lasso estimators on simulated and real data sets.     

Sieve Estimation of Constant and Time-Varying Coefficients in Nonlinear Ordinary Differential Equation Models by Considering Both Numerical Error and Measurement Error

This article considers estimation of constant and time-varying coefficients in nonlinear ordinary differential equation (ODE) models where analytic closed-form solutions are not available. The numerical solution-based nonlinear least squares (NLS) estimator is investigated in this study. A numerical algorithm such as the Runge-Kutta method is used to approximate the ODE solution. The asymptotic properties are established for the proposed estimators considering both numerical error and measurement error. The B-spline is used to approximate the time-varying coefficients, and the corresponding asymptotic theories in this case are investigated under the framework of the sieve approach. Our results show that if the maximum step size of the p-order numerical algorithm goes to zero at a rate faster than n(-1/(p∧4)), the numerical error is negligible compared to the measurement error. This result provides a theoretical guidance in selection of the step size for numerical evaluations of ODEs. Moreover, we have shown that the numerical solution-based NLS estimator and the sieve NLS estimator are strongly consistent. The sieve estimator of constant parameters is asymptotically normal with the same asymptotic co-variance as that of the case where the true ODE solution is exactly known, while the estimator of the time-varying parameter has the optimal convergence rate under some regularity conditions. The theoretical results are also developed for the case when the step size of the ODE numerical solver does not go to zero fast enough or the numerical error is comparable to the measurement error. We illustrate our approach with both simulation studies and clinical data on HIV viral dynamics.     

Maximum likelihood drift estimation for a threshold diffusion

We study the maximum likelihood estimator of the drift parameters of a stochastic differential equation, with both drift and diffusion coefficients constant on the positive and negative axis, yet discontinuous at zero. This threshold diffusion is called drifted oscillating Brownian motion. For this continuously observed diffusion, the maximum likelihood estimator coincides with a quasi-likelihood estimator with constant diffusion term. We show that this estimator is the limit, as observations become dense in time, of the (quasi)-maximum likelihood estimator based on discrete observations. In long time, the asymptotic behaviors of the positive and negative occupation times rule the ones of the estimators. Differently from most known results of the literature, we do not restrict ourselves to the ergodic framework: indeed, depending on the signs of the drift, the process may be ergodic, transient, or null recurrent. For each regime, we establish whether or not the estimators are consistent; if they are, we prove the convergence in long time of the properly rescaled difference of the estimators towards a normal or mixed normal distribution. These theoretical results are backed by numerical simulations.     

Some Results About Standardization for a Non Confounder in Estimators of (log) Relative Risk

Confounding is very fundamental to the design and analysis of studies of causal effects. A variable is not a confounder if it is not a risk factor to disease or if it has the same distribution in the exposed and unexposed population. Whether or not to adjust for a non confounder to improve the precision of estimation has been argued by many authors. This article shows that if C is a non confounder, the pooled and standardized (log) relative risk estimators are asymptotic normal distributions with the mean being the true (log) relative risk, and that the asymptotic variance of the pooled (log) relative risk estimator is less than that of the stratified estimator.     

Shape Constrained Non‐parametric Estimators of the Baseline Distribution in Cox Proportional Hazards Model

Abstract. We investigate non‐parametric estimation of a monotone baseline hazard and a decreasing baseline density within the Cox model. Two estimators of a non‐decreasing baseline hazard function are proposed. We derive the non‐parametric maximum likelihood estimator and consider a Grenander type estimator, defined as the left‐hand slope of the greatest convex minorant of the Breslow estimator. We demonstrate that the two estimators are strongly consistent and asymptotically equivalent and derive their common limit distribution at a fixed point. Both estimators of a non‐increasing baseline hazard and their asymptotic properties are obtained in a similar manner. Furthermore, we introduce a Grenander type estimator for a non‐increasing baseline density, defined as the left‐hand slope of the least concave majorant of an estimator of the baseline cumulative distribution function, derived from the Breslow estimator. We show that this estimator is strongly consistent and derive its asymptotic distribution at a fixed point.     

Estimation of parameters in a generalized GMANOVA model based on an outer product analogy and least squares

This paper investigates estimation of parameters in a combination of the multivariate linear model and growth curve model, called a generalized GMANOVA model. Making analogy between the outer product of data vectors and covariance yields an approach to directly do least squares to covariance. An outer product least squares estimator of covariance (COPLS estimator) is obtained and its distribution is presented if a normal assumption is imposed on the error matrix. Based on the COPLS estimator, two-stage generalized least squares estimators of the regression coefficients are derived. In addition, asymptotic normalities of these estimators are investigated. Simulation studies have shown that the COPLS estimator and two-stage GLS estimators are alternative competitors with more efficiency in the sense of sample mean, standard deviations and mean of the variance estimates to the existing ML estimator in finite samples. An example of application is also illustrated.     

A comparative study of some bias correction techniques for kernel- based density estimators

Methods for obtaining kernel-based density estimators with lower bias and mean integrated squared error than an estimator based on a standard Normal kernel function are described and discussed. Three main approaches are considered which are: firstly by using 'optimal' polynomial kernels as described, for example, by Gasser er a1 (1985); secondly by employing generalised jackknifing as proposed by Jones nd Foster (1993) and thirdly by subtracting an estimator of the principal asymptotic bias term from the original estimator. The emphasis in this initial discussion is on their asymptotic properties. The finite sample performance of those that have the best asymptotic properties are compared with two adaptive estimators, as well as the fixed Normal kernel estimator, in a simulation study.     

Statistic inference for a single-index ARCH-M model

For a single-index autoregressive conditional heteroscedastic (ARCH-M) model, estimators of the parametric and non parametric components are proposed by the profile likelihood method. The research results had shown that all the estimators have consistency and the parametric estimators have asymptotic normality. We extend this line of research by deriving the asymptotic normality of the non parametric estimator. Based on the asymptotic properties, we propose Wald statistic and generalized likelihood ratio statistic to investigate the testing problems for ARCH effect and goodness of fit, respectively. A simulation study is conducted to evaluate the finite-sample performance of the proposed estimation methodology and testing procedure.     

Asymptotically efficient estimation in response adaptive trials

We examine the issue of asymptotic efficiency of estimation for response adaptive designs of clinical trials, from which the collected data set contains a dependency structure. We establish the asymptotic lower bound of exponential rates for consistent estimators. Under certain regularity conditions, we show that the maximum likelihood estimator achieves the asymptotic lower bound for response adaptive trials with dichotomous responses. Furthermore, it is shown that the maximum likelihood estimator of the treatment effect is asymptotically efficient in the Bahadur sense for response adaptive clinical trials.     

A Note on Bayesian Analyses of Capture–Recapture Data with Perfect Recaptures

The present article deals with the problem of misspecifying the disturbance-covariance matrix as scalar, when it is locally non scalar. We consider a family of shrinkage estimators based on OLS estimator and compare its asymptotic properties with the properties of OLS estimator. We proposed a similar family of estimators based on FGLS and compared its asymptotic properties with the shrinkage estimator based on OLS under a Pitman's drift process. The effect of misspecifying the disturbances covariance matrix was analyzed with the help of a numerical simulation.     

Wavelet estimation of density for censored data with censoring indicator missing at random

In this paper, we define the nonlinear wavelet estimator of density for the right censoring model with the censoring indicator missing at random (MAR), and develop its asymptotic expression for mean integrated squared error (MISE). Unlike for kernel estimator, the MISE expression of the estimator is not affected by the presence of discontinuities in the curve. Meanwhile, asymptotic normality of the estimator is established. The proposed estimator can reduce to the estimator defined by Li [Non-linear wavelet-based density estimators under random censorship. J Statist Plann Inference. 2003;117(1):35–58] when the censoring indicator MAR does not occur and a bandwidth in non-parametric estimation is close to zero. Also, we define another two nonlinear wavelet estimators of the density. A simulation is done to show the performance of the three proposed estimators.     

Some conditions for optimality of the almost unbiased estimators of regression coefficients

The purpose of this paper is to examine the asymptotic properties of the operational almost unbiased estimator of regression coefficients which includes almost unbiased ordinary ridge estimator a s a special case. The small distrubance approximations for the bias and mean square error matrix of the estimator are derived. As a consequence, it is proved that, under certain conditions, the estimator is more efficient than a general class of estimators given by Vinod and Ullah (1981). Also it is shown that, if the ordinary ridge estimator (ORE) dominates the ordinary least squares estimator then the almost unbiased ordinary ridge estimator does not dominate ORE under the mean square error criterion.     

Quadratic Approximation via the SCAD Penalty with a Diverging Number of Parameters

The high-dimensional data arises in diverse fields of sciences, engineering and humanities. Variable selection plays an important role in dealing with high dimensional statistical modelling. In this article, we study the variable selection of quadratic approximation via the smoothly clipped absolute deviation (SCAD) penalty with a diverging number of parameters. We provide a unified method to select variables and estimate parameters for various of high dimensional models. Under appropriate conditions and with a proper regularization parameter, we show that the estimator has consistency and sparsity, and the estimators of nonzero coefficients enjoy the asymptotic normality as they would have if the zero coefficients were known in advance. In addition, under some mild conditions, we can obtain the global solution of the penalized objective function with the SCAD penalty. Numerical studies and a real data analysis are carried out to confirm the performance of the proposed method.     

A Simple Approach to Inference in Random Coefficient Models

Random coefficient regression models have been used to analyze cross-sectional and longitudinal data in economics and growth-curve data from biological and agricultural experiments. In the literature several estimators, including the ordinary least squares and the estimated generalized least squares (EGLS), have been considered for estimating the parameters of the mean model. Based on the asymptotic properties of the EGLS estimators, test statistics have been proposed for testing linear hypotheses involving the parameters of the mean model. An alternative estimator, the simple mean of the individual regression coefficients, provides estimation and hypothesis-testing procedures that are simple to compute and teach. The large sample properties of this simple estimator are shown to be similar to that of the EGLS estimator. The performance of the proposed estimator is compared with that of the existing estimators by Monte Carlo simulation.     

Sharp Threshold Detection Based on Sup-Norm Error Rates in High-Dimensional Models

We propose a new estimator, the thresholded scaled Lasso, in high-dimensional threshold regressions. First, we establish an upper bound on the ?_∞ estimation error of the scaled Lasso estimator of Lee, Seo, and Shin. This is a nontrivial task as the literature on high-dimensional models has focused almost exclusively on ?₁ and ?₂ estimation errors. We show that this sup-norm bound can be used to distinguish between zero and nonzero coefficients at a much finer scale than would have been possible using classical oracle inequalities. Thus, our sup-norm bound is tailored to consistent variable selection via thresholding. Our simulations show that thresholding the scaled Lasso yields substantial improvements in terms of variable selection. Finally, we use our estimator to shed further empirical light on the long-running debate on the relationship between the level of debt (public and private) and GDP growth. Supplementary materials for this article are available online.