Inference for $$L_2$$L2 -Boosting

We propose a statistical inference framework for the component-wise functional gradient descent algorithm (CFGD) under normality assumption for model errors, also known as $$L_2$$-Boosting. The CFGD is one of the most versatile tools to analyze data, because it scales well to high-dimensional data sets, allows for a very flexible definition of additive regression models and incorporates inbuilt variable selection. Due to the variable selection, we build on recent proposals for post-selection inference. However, the iterative nature of component-wise boosting, which can repeatedly select the same component to update, necessitates adaptations and extensions to existing approaches. We propose tests and confidence intervals for linear, grouped and penalized additive model components selected by $$L_2$$-Boosting. Our concepts also transfer to slow-learning algorithms more generally, and to other selection techniques which restrict the response space to more complex sets than polyhedra. We apply our framework to an additive model for sales prices of residential apartments and investigate the properties of our concepts in simulation studies.     

Optimal estimators for the importance sampling method

The Monte Carlo method gives some estimators to evaluate the expectation [ILM0001] based on samples from either the true density f or from some instrumental density. In this paper, we show that the Riemann estimators introduced by Philippe (1997) can be improved by using the importance sampling method. This approach produces a class of Monte Carlo estimators such that the variance is of order O(n ^?2). The choice of an optimal estimator among this class is discussed. Some simulations illustrate the improvement brought by this method. Moreover, we give a criterion to assess the convergence of our optimal estimator to the integral of interest.     

Tests for the two-sample problem based on empirical probability measures

Tests are proposed for the equality of two unknown distributions. For empirical probability measures that are defined for samples from the two distributions, the proposed tests are based on the supremum of the absolute differences between the corresponding empirical probabilities, the supremum being taken over all possible events (Borel sets). In contrast, competing EDF tests compare only empirical probabilities of a subclass of Borel sets. The proposed tests are compared for simulated samples to the Kolmogorov-Smirnov, Cramér-von Mises, Kuiper, and Mann-Whitney-Wilcoxon tests     

A note on millers's empirical weights for heteroscedastic linear regression

For heteroscedastic simple linear regression when the variances are proportional to a power of the mean of the response variable, Miller (1986) recommends the following procedure: do a weighted least squares regression with the weights (empirical weights) estimated by the inverse of the appropriate power of the response variable. The practical appeal of this approach is its simplicity.

In this article some of the consequences of this simple procedure are considered. Specifically, the effect of this procedure on the bias of the point estimators of the regression coefficients and on the coverage probabilities of their corresponding confidence intervals is examined. It is found that the performance of the process of employing empirical weights in a weighted least squares regression depends on : (1) the particular regression parameter (slope or intercept) of interest, (2) the appropriate power of the mean of the response variable involved, and (3) the amount of variation in the data about the true regression line.     

Numerical estimation of a probability measure

An iterative solution to the problem of maximizing a concave functional ø defined on the set of all probability measures on a topological space is considered. Convergence of this procedure and a rapidly converging algorithm are studied. Computational aspects of this algorithm along with the ones developed earlier by Wynn, Fedorov, Atwood, Wu and others are provided. Examples discussed are taken from the area of mixture likehoods and optimal experimental design.     

Tests for the multivariate two-sample problem based on empirical probability measures

Nonparametric tests are proposed for the equality of two unknown p-variate distributions. Empirical probability measures are defined from samples from the two distributions and used to construct test statistics as the supremum of the absolute differences between empirical probabilities, the supremum being taken over all possible events. The test statistics are truly multivariate in not requiring the artificial ranking of multivariate observations, and they are distribution-free in the general p-variate case. Asymptotic null distributions are obtained. Powers of the proposed tests and a competitor are examined by Monte Carlo techniques.     

Characterization of the quantitative-qualitative measure of inaccuracy for discrete generalized probability distributions

A characterization of the quantitative-qualitative measure of inaccuracy depending upon the additivity postulate and the mean value property for discrete generalized probability distribution has been provided.     

Exploratory data analysis for counts using the empirical probability generating function 1

We present a graphical method based on the empirical probability generating function for preliminary statistical analysis of distributions for counts. The method is especially useful in fitting a Poisson model, or for identifying alternative models as well as possible outlying observations from general discrete distributions.     

Variance estimation for measures of change in probability sampling

Variance estimation of changes requires estimates of variances and covariances that would be relatively straightforward to make if the sample remained the same from one wave to the next, but this is rarely the case in practice as successive waves are usually different overlapping samples. The author proposes a design‐based estimator for covariance matrices that is adapted to this situation. Under certain conditions, he shows that his approach yields non‐negative definite estimates for covariance matrices and therefore positive variance estimates for a large class of measures of change.     

A moment-based empirical likelihood ratio test for exponentiality using the probability integral transformation

ABSTRACT

A simple and efficient goodness-of-fit test for exponentiality is developed by exploiting the characterization of the exponential distribution using the probability integral transformation. We adopted the empirical likelihood methodology in constructing the test statistic. The proposed test statistic has a chi-square limiting distribution. For small to moderate sample sizes Monte-Carlo simulations revealed that our proposed tests are much more superior under increasing failure rate (IFR) and bathtub decreasing-increasing failure rate (BFR) alternatives. Real data examples were used to demonstrate the robustness and applicability of our proposed tests in practice.     

Use of an empirical probability generating function for testing a Poisson model

We present a test of the fit to a Poisson model based on the empirical probability generating function (epgf). We derive the limiting distribution of the test under the Poisson hypothesis and show that a rescaling of it is approximately independent of the mean parameter in the Poisson distribution. When inspected under a simulation study over a range of alternative distributions, we find that this test shows reasonable behaviour compared to other goodness-of-fit tests like the Poisson index of dispersion and smooth test applied to the Poisson model. These results illustrate that epgf-based methods for anlyzing count data are promising.     

The probability of rapid climate change

If you look at a map of the air temperature of the surface of the Earth, you will see that North West Europe, including the UK, is warmer than Alaska, which is at the same latitude but on the Pacific rather than Atlantic Ocean. At school you were probably told that this was because of the Gulf Stream. However, there is a very similar current in the Pacific—the Kuroshio—which takes warm water north past Japan and then out into the Atlantic. Peter Challenor asks: What is the unique feature of the Atlantic that keeps us warm and could it change in the next few years?     

Selected percent points for a test for the two-sample problem based on empirical probability measures

Selected percent points are presented for a test for the two-sample problem based on empirical probability measures. The test is illustrated in two examples.     

An empirical bayes approach to estimating the probability of correct selection

In the problem of selecting the best of k populations, Olkin, Sobel, and Tong (1976) have introduced the idea of estimating the probability of correct selection. In an attempt to improve on their estimator we consider anempirical Bayes approach. We compare the two estimators via analytic results and a simulation study.     

An algorithm for computing a two-sample test based on empirical probability measures

An algorithm is presented for computing a test statistic based on empirical probability measures that is appropriate for the two-sample problem.     

Variance-based importance analysis measure for mission reliability of phased mission system

Importance measures are used to estimate the relative importance of components to system reliability. Phased mission systems (PMS) have many components working in several phases with different success criteria, and their component structural importance is distinct in different phases. Additionally, reliability parameters of components in PMS always have uncertainty in practice. Therefore, existing component importance measures based on either the partial derivative of system structure function or component structural importance may have difficulties in PMS importance analysis. This paper presents a simulation method to evaluate the component global importance for PMS based on the variance-based method and the Monte-Carlo method. To facilitate the practical use, we further discuss the correlation relationship between the component global importance and its possible influence factors, and present here a fitting model for evaluating component global importance. Finally, two examples are given to show that the fitting model displays quite reasonable component importance.     

An empirical likelihood-based CUSUM for on-line model change detection

Abstract

In change detection problem, the distribution of a series of observations can change at some unknown instant. The aim of on-line change detection rule is to detect this change, as rapidly as possible, while ensuring a low rate of false alarm. The most popular rule to treat this problem is the Page’s CUSUM rule. The use of this rule supposes that the two distributions, before and after the change, are known, which is often restrictive in practice. In this article, a nonparametric rule is proposed. Only two learning samples, characterizing the in-control and the out-of-control functioning modes of the system, are needed to implement the rule. The new detection approach is based on the use of a well-known nonparametric method, Empirical Likelihood. Some numerical studies show the relevance of our approach, especially when the size of the learning samples are quite small.     

A new variable importance measure for random forests with missing data

Random forests are widely used in many research fields for prediction and interpretation purposes. Their popularity is rooted in several appealing characteristics, such as their ability to deal with high dimensional data, complex interactions and correlations between variables. Another important feature is that random forests provide variable importance measures that can be used to identify the most important predictor variables. Though there are alternatives like complete case analysis and imputation, existing methods for the computation of such measures cannot be applied straightforward when the data contains missing values. This paper presents a solution to this pitfall by introducing a new variable importance measure that is applicable to any kind of data—whether it does or does not contain missing values. An extensive simulation study shows that the new measure meets sensible requirements and shows good variable ranking properties. An application to two real data sets also indicates that the new approach may provide a more sensible variable ranking than the widespread complete case analysis. It takes the occurrence of missing values into account which makes results also differ from those obtained under multiple imputation.     

The probability of rapid climate change: II

In Significance December 2004, Peter Challenor described the statistical problem associated with the possible collapse in the thermohaline circulation in the North Atlantic—"the shutting down of the Gulf Stream". Here he brings us up to date with the latest results.