Regression analysis for a summed missing data problem under an outcome‐dependent sampling scheme

In this paper, we consider a regression analysis for a missing data problem in which the variables of primary interest are unobserved under a general biased sampling scheme, an outcome‐dependent sampling (ODS) design. We propose a semiparametric empirical likelihood method for accessing the association between a continuous outcome response and unobservable interesting factors. Simulation study results show that ODS design can produce more efficient estimators than the simple random design of the same sample size. We demonstrate the proposed approach with a data set from an environmental study for the genetic effects on human lung function in COPD smokers. The Canadian Journal of Statistics 40: 282–303; 2012 © 2012 Statistical Society of Canada     

Adaptive estimation of error density in nonparametric regression with small sample size

It has been established recently in Efromovich [2005. Estimation of the density of regression errors. Ann. Statist. 33, 2194–2227] that, under a mild assumption, the error density in a nonparametric regression can be asymptotically estimated with the accuracy of an oracle that knows underlying regression errors. The asymptotic nature of the result, and in particular the used methodology of splitting data for estimating nuisance functions and the error density, does not make an asymptotic estimator, suggested in that article, feasible for practically interesting cases of small sample sizes. This article continues the research and solves two important issues. First, it shows that the asymptotic holds without splitting the data. Second, a data-driven estimator, based on the new asymptotic, is suggested and then tested on real and simulated examples.     

Robust precision matrix estimation via weighted median regression with regularization

A precision matrix is an important parameter of interests because its elements describe useful association information among multiple variables, which has a wide variety of applications. For example, it is used for inferring gene regulation networks in genomic studies and stock association networks in financial studies. However, in many cases, the precision matrix needs to be robustly estimated due to the presence of outliers. We propose estimating a sparse scaled precision matrix via weighted median regression with regularization. Our weighted median regression approach is consistent under various distributional assumptions including multivariate t‐ or contaminated Gaussian distributions. This fact is illustrated with simulation studies and a real data analysis with monthly stock return data. The Canadian Journal of Statistics 46: 265–278; 2018 © 2018 Statistical Society of Canada     

New aspects of Bregman divergence in regression and classification with parametric and nonparametric estimation

In statistical learning, regression and classification concern different types of the output variables, and the predictive accuracy is quantified by different loss functions. This article explores new aspects of Bregman divergence (BD), a notion which unifies nearly all of the commonly used loss functions in regression and classification. The authors investigate the duality between BD and its generating function. They further establish, under the framework of BD, asymptotic consistency and normality of parametric and nonparametric regression estimators, derive the lower bound of their asymptotic covariance matrices, and demonstrate the role that parametric and nonparametric regression estimation play in the performance of classification procedures and related machine learning techniques. These theoretical results and new numerical evidence show that the choice of loss function affects estimation procedures, whereas has an asymptotically relatively negligible impact on classification performance. Applications of BD to statistical model building and selection with non‐Gaussian responses are also illustrated. The Canadian Journal of Statistics 37: 119‐139; 2009 © 2009 Statistical Society of Canada     

Change detection in linear regression with time series errors

A new test for detecting a change in linear regression parameters assuming a general weakly dependent error structure is given. It extends earlier methods based on cumulative sums assuming independent errors. The novelty is in the new standardization method and in smoothing when the time series is dominated by high frequencies. Simulations show the excellent performance of the test. Examples are taken from environmental applications. The algorithm is easy to implement. Testing for multiple changes can be done by segmentation. The Canadian Journal of Statistics 38:65–79; 2010 © 2009 Statistical Society of Canada     

Small area estimation of poverty indicators

The authors propose to estimate nonlinear small area population parameters by using the empirical Bayes (best) method, based on a nested error model. They focus on poverty indicators as particular nonlinear parameters of interest, but the proposed methodology is applicable to general nonlinear parameters. They use a parametric bootstrap method to estimate the mean squared error of the empirical best estimators. They also study small sample properties of these estimators by model‐based and design‐based simulation studies. Results show large reductions in mean squared error relative to direct area‐specific estimators and other estimators obtained by “simulated” censuses. The authors also apply the proposed method to estimate poverty incidences and poverty gaps in Spanish provinces by gender with mean squared errors estimated by the mentioned parametric bootstrap method. For the Spanish data, results show a significant reduction in coefficient of variation of the proposed empirical best estimators over direct estimators for practically all domains. The Canadian Journal of Statistics 38: 369–385; 2010 © 2010 Statistical Society of Canada     

Bounded influence nonlinear signed‐rank regression

In this paper we consider weighted generalized‐signed‐rank estimators of nonlinear regression coefficients. The generalization allows us to include popular estimators such as the least squares and least absolute deviations estimators but by itself does not give bounded influence estimators. Adding weights results in estimators with bounded influence function. We establish conditions needed for the consistency and asymptotic normality of the proposed estimator and discuss how weight functions can be chosen to achieve bounded influence function of the estimator. Real life examples and Monte Carlo simulation experiments demonstrate the robustness and efficiency of the proposed estimator. An example shows that the weighted signed‐rank estimator can be useful to detect outliers in nonlinear regression. The Canadian Journal of Statistics 40: 172–189; 2012 © 2012 Statistical Society of Canada     

Robust penalized logistic regression with truncated loss functions

The penalized logistic regression (PLR) is a powerful statistical tool for classification. It has been commonly used in many practical problems. Despite its success, since the loss function of the PLR is unbounded, resulting classifiers can be sensitive to outliers. To build more robust classifiers, we propose the robust PLR (RPLR) which uses truncated logistic loss functions, and suggest three schemes to estimate conditional class probabilities. Connections of the RPLR with some other existing work on robust logistic regression have been discussed. Our theoretical results indicate that the RPLR is Fisher consistent and more robust to outliers. Moreover, we develop estimated generalized approximate cross validation (EGACV) for the tuning parameter selection. Through numerical examples, we demonstrate that truncating the loss function indeed yields better performance in terms of classification accuracy and class probability estimation.     

Inflation of Type I error rate in multiple regression when independent variables are measured with error

When independent variables are measured with error, ordinary least squares regression can yield parameter estimates that are biased and inconsistent. This article documents an inflation of Type I error rate that can also occur. In addition to analytic results, a large‐scale Monte Carlo study shows unacceptably high Type I error rates under circumstances that could easily be encountered in practice. A set of smaller‐scale simulations indicate that the problem applies to various types of regression and various types of measurement error. The Canadian Journal of Statistics 37: 33‐46; 2009 © 2009 Statistical Society of Canada     

New estimation and feature selection methods in mixture‐of‐experts models

We study estimation and feature selection problems in mixture‐of‐experts models. An $l_2$ ‐penalized maximum likelihood estimator is proposed as an alternative to the ordinary maximum likelihood estimator. The estimator is particularly advantageous when fitting a mixture‐of‐experts model to data with many correlated features. It is shown that the proposed estimator is root‐$n$ consistent, and simulations show its superior finite sample behaviour compared to that of the maximum likelihood estimator. For feature selection, two extra penalty functions are applied to the $l_2$ ‐penalized log‐likelihood function. The proposed feature selection method is computationally much more efficient than the popular all‐subset selection methods. Theoretically it is shown that the method is consistent in feature selection, and simulations support our theoretical results. A real‐data example is presented to demonstrate the method. The Canadian Journal of Statistics 38: 519–539; 2010 © 2010 Statistical Society of Canada     

Additive hazards regression with censoring indicators missing at random

In this article, the authors consider a semiparametric additive hazards regression model for right‐censored data that allows some censoring indicators to be missing at random. They develop a class of estimating equations and use an inverse probability weighted approach to estimate the regression parameters. Nonparametric smoothing techniques are employed to estimate the probability of non‐missingness and the conditional probability of an uncensored observation. The asymptotic properties of the resulting estimators are derived. Simulation studies show that the proposed estimators perform well. They motivate and illustrate their methods with data from a brain cancer clinical trial. The Canadian Journal of Statistics 38: 333–351; 2010 © 2010 Statistical Society of Canada     

Inference after variable selection using restricted permutation methods

When confronted with multiple covariates and a response variable, analysts sometimes apply a variable‐selection algorithm to the covariate‐response data to identify a subset of covariates potentially associated with the response, and then wish to make inferences about parameters in a model for the marginal association between the selected covariates and the response. If an independent data set were available, the parameters of interest could be estimated by using standard inference methods to fit the postulated marginal model to the independent data set. However, when applied to the same data set used by the variable selector, standard (“naive”) methods can lead to distorted inferences. The authors develop testing and interval estimation methods for parameters reflecting the marginal association between the selected covariates and response variable, based on the same data set used for variable selection. They provide theoretical justification for the proposed methods, present results to guide their implementation, and use simulations to assess and compare their performance to a sample‐splitting approach. The methods are illustrated with data from a recent AIDS study. The Canadian Journal of Statistics 37: 625–644; 2009 © 2009 Statistical Society of Canada