Trimmed and Winsorized means based on a scaled deviation

Trimmed (and Winsorized) means based on a scaled deviation are introduced and studied. The influence functions of the estimators are derived and their limiting distributions are established via asymptotic representations. As a main focus of the paper, the performance of the estimators with respect to various robustness and efficiency criteria is evaluated and compared with leading competitors including the ordinary Tukey trimmed (and Winsorized) means. Unlike the Tukey trimming which always trims a fixed fraction of sample points at each end of data, the trimming scheme here only trims points at one or both ends that have a scaled deviation beyond some threshold. The resulting trimmed (and Winsorized) means are much more robust than their predecessors. Indeed they can share the best breakdown point robustness of the sample median for any common trimming thresholds. Furthermore, for appropriate trimming thresholds they are highly efficient at light-tailed symmetric models and more efficient than their predecessors at heavy-tailed or contaminated symmetric models. Detailed comparisons with leading competitors on various robustness and efficiency aspects reveal that the scaled deviation trimmed (Winsorized) means behave very well overall and consequently represent very favorable alternatives to the ordinary trimmed (Winsorized) means.     

Robust permutation tests for two samples

We consider robust permutation tests for a location shift in the two sample case based on estimating equations, comparing the test statistics based on a score function and an M-estimate. First we obtain a form for both tests so that the exact tests may be carried out using the same algorithms as used for permutation tests based on the mean. Then we obtain the Bahadur slopes of the tests in these two statistics, giving numerical results for two cases equivalent to a test based on Huber scores and a particular case of this related to a median test. We show that they have different Bahadur slopes with neither exceeding the other over the whole range. Finally, we give some numerical results illustrating the robustness properties of the tests and confirming the theoretical results on Bahadur slopes.     

Quantile regression for robust inference on varying coefficient partially nonlinear models

In this paper, we propose a robust statistical inference approach for the varying coefficient partially nonlinear models based on quantile regression. A three-stage estimation procedure is developed to estimate the parameter and coefficient functions involved in the model. Under some mild regularity conditions, the asymptotic properties of the resulted estimators are established. Some simulation studies are conducted to evaluate the finite performance as well as the robustness of our proposed quantile regression method versus the well known profile least squares estimation procedure. Moreover, the Boston housing price data is given to further illustrate the application of the new method.     

Empirical likelihood for a partially linear model with covariate data missing at random

In this paper, we consider the partial linear model with the covariables missing at random. Empirical likelihood ratios for the regression coefficients and the baseline function are investigated, the empirical log-likelihood ratios are proven to be asymptotically chi-squared and the corresponding confidence regions for the parameters of interest are then constructed. The finite sample behavior of the proposed method is evaluated with simulation and illustrated with an AIDS clinical trial dataset.     

Inferences from biased samples with a memory effect

Biased sampling occurs often in observational studies. With one biased sample, the problem of nonparametrically estimating both a target density function and a selection bias function is unidentifiable. This paper studies the nonparametric estimation problem when there are two biased samples that have some overlapping observations (i.e. recaptures) from a finite population. Since an intelligent subject sampled previously may experience a memory effect if sampled again, two general 2-stage models that incorporate both a selection bias and a possible memory effect are proposed. Nonparametric estimators of the target density, selection bias, and memory functions, as well as the population size are developed. Asymptotic properties of these estimators are studied and confidence bands for the selection function and memory function are provided. Our procedures are compared with those ignoring the memory effect or the selection bias in finite sample situations. A nonparametric model selection procedure is also given for choosing a model from the two 2-stage models and a mixture of these two models. Our procedures work well with or without a memory effect, and with or without a selection bias. The paper concludes with an application to a real survey data set.     

Efficiency of weighted averages

When combining estimates of a common parameter (of dimension d?1$d?1$) from independent data sets—as in stratified analyses and meta analyses—a weighted average, with weights ‘proportional’ to inverse variance matrices, is shown to have a minimal variance matrix (a standard fact when d=1$d=1$)—minimal in the sense that all convex combinations of the coordinates of the combined estimate have minimal variances. Minimum variance for the estimation of a single coordinate of the parameter can therefore be achieved by joint estimation of all coordinates using matrix weights. Moreover, if each estimate is asymptotically efficient within its own data set, then this optimally weighted average, with consistently estimated weights, is shown to be asymptotically efficient in the combined data set and avoids the need to merge the data sets and estimate the parameter in question afresh. This is so whatever additional non-common nuisance parameters may be in the models for the various data sets. A special case of this appeared in Fisher [1925. Theory of statistical estimation. Proc. Cambridge Philos. Soc. 22, 700–725.]: Optimal weights are ‘proportional’ to information matrices, and he argued that sample information should be used as weights rather than expected information, to maintain second-order efficiency of maximum likelihood. A number of special cases have appeared in the literature; we review several of them and give additional special cases, including stratified regression analysis—proportional-hazards, logistic or linear—, combination of independent ROC curves, and meta analysis. A test for homogeneity of the parameter across the data sets is also given.     

Estimating the distribution function using k-tuple ranked set samples

The basic assumption underlying the concept of ranked set sampling is that actual measurement of units is expensive, whereas ranking is cheap. This may not be true in reality in certain cases where ranking may be moderately expensive. In such situations, based on total cost considerations, k-tuple ranked set sampling is known to be a viable alternative, where one selects k units (instead of one) from each ranked set. In this article, we consider estimation of the distribution function based on k-tuple ranked set samples when the cost of selecting and ranking units is not ignorable. We investigate estimation both in the balanced and unbalanced data case. Properties of the estimation procedure in the presence of ranking error are also investigated. Results of simulation studies as well as an application to a real data set are presented to illustrate some of the theoretical findings.     

A class of count models and a new consistent test for the Poisson distribution

We define a class of count distributions which includes the Poisson as well as many alternative count models. Then the empirical probability generating function is utilized to construct a test for the Poisson distribution, which is consistent against this class of alternatives. The limit distribution of the test statistic is derived in case of a general underlying distribution, and efficiency considerations are addressed. A simulation study indicates that the new test is comparable in performance to more complicated omnibus tests.     

On the asymptotic behaviour of location-scale invariant Bickel–Rosenblatt tests

Location-scale invariant Bickel–Rosenblatt goodness-of-fit tests (IBR tests) are considered in this paper to test the hypothesis that f, the common density function of the observed independent d-dimensional random vectors, belongs to a null location-scale family of density functions. The asymptotic behaviour of the test procedures for fixed and non-fixed bandwidths is studied by using an unifying approach. We establish the limiting null distribution of the test statistics, the consistency of the associated tests and we derive its asymptotic power against sequences of local alternatives. These results show the asymptotic superiority, for fixed and local alternatives, of IBR tests with fixed bandwidth over IBR tests with non-fixed bandwidth.     

Two sample inference in functional linear models

We propose a method of comparing two functional linear models in which explanatory variables are functions (curves) and responses can be either scalars or functions. In such models, the role of parameter vectors (or matrices) is played by integral operators acting on a function space. We test the null hypothesis that these operators are the same in two independent samples. The complexity of the test statistics increases as we move from scalar to functional responses and relax assumptions on the covariance structure of the regressors. They all, however, have an asymptotic chi‐squared distribution with the number of degrees of freedom which depends on a specific setting. The test statistics are readily computable using the R package fda , and have good finite sample properties. The test is applied to egg‐laying curves of Mediterranean flies and to data from terrestrial magnetic observatories. The Canadian Journal of Statistics © 2009 Statistical Society of Canada     

Bayesian mixture modeling for spatial Poisson process intensities,with applications to extreme value analysis

We propose a method for the analysis of a spatial point pattern, which is assumed to arise as a set of observations from a spatial nonhomogeneous Poisson process. The spatial point pattern is observed in a bounded region, which, for most applications, is taken to be a rectangle in the space where the process is defined. The method is based on modeling a density function, defined on this bounded region, that is directly related with the intensity function of the Poisson process. We develop a flexible nonparametric mixture model for this density using a bivariate Beta distribution for the mixture kernel and a Dirichlet process prior for the mixing distribution. Using posterior simulation methods, we obtain full inference for the intensity function and any other functional of the process that might be of interest. We discuss applications to problems where inference for clustering in the spatial point pattern is of interest. Moreover, we consider applications of the methodology to extreme value analysis problems. We illustrate the modeling approach with three previously published data sets. Two of the data sets are from forestry and consist of locations of trees. The third data set consists of extremes from the Dow Jones index over a period of 1303 days.     

Estimating conditional tail expectation with actuarial applications in view

We develop statistical inferential tools for estimating and comparing conditional tail expectation (CTE) functions, which are of considerable interest in actuarial science. In particular, we construct estimators for the CTE functions, develop the necessary asymptotic theory for the estimators, and then use the theory for constructing confidence intervals and bands for the functions. Both parametric and non-parametric approaches are explored. Simulation studies illustrate the performance of estimators in various situations. Results are obtained under minimal assumptions, and the general Vervaat process plays a crucial role in achieving these goals.     

Robustness of least distances estimate in ultivariate linear models

In this paper, the robustness of the least distances (LD) estimate in multivariate linear models, as defined by Bai, Chen, Miao and Rao (1990), is discussed in terms of the influence function as well as the breakdown point. The LD estimate is shown to be more robust than the least squares (LS) estimate. The robustness of the LD is similar to that of the least absolute deviations (LAD) estimate, a well studied robust estimate in the univariate case. In particular, if there are no outliers in the design matrices, the breakdown point of the LD estimate reaches the highest value, 1/2.     

Semiparametric estimation for count data through weighted distributions

This paper is concerned with semiparametric discrete kernel estimators when the unknown count distribution can be considered to have a general weighted Poisson form. The estimator is constructed by multiplying the Poisson estimate with a nonparametric discrete kernel-type estimate of the Poisson weight function. Comparisons are then carried out with the ordinary discrete kernel probability mass function estimators. The Poisson weight function is thus a local multiplicative correction factor, and is considered as the uniform measure to detect departures from the equidispersed Poisson distribution. In this way, the effects of dispersion and zero-proportion with respect to the standard Poisson distribution are also minimized. This method of estimation is also applied to the weighted binomial form for the count distribution having a finite support. The proposed estimators, in addition to being simple, easy-to-implement and effective, also outperform the competing nonparametric and parametric estimators in finite-sample situations. Two examples illustrate this new semiparametric estimation.     

A lower-bound oracle inequality for a blockwise-shrinkage estimate

Efromovich-Pinsker and Stein blockwise-shrinkage estimates are traditionally studied via upper-bound oracle inequalities, which bound the estimate's risk from above by the oracle's risk plus a remainder term. These bounds allow one to establish sufficient conditions for attaining the oracle's risk. To explore necessary conditions, this article develops a lower-bound oracle inequality, which bounds the estimate's risk from below by the oracle's risk minus a remainder term. In particular, the lower bound implies that thresholds must vanish for attaining the oracle's risk.     

An asymptotically distribution-free test of symmetry

A procedure, based on sample spacings, is proposed for testing whether a univariate distribution is symmetric about some unknown value. The proposed test is a modification of a sign test suggested by Antille and Kersting [1977. Tests for symmetry. Z. Wahrscheinlichkeitstheorie verw. Gebiete 39, 235–255], but unlike Antille and Kersting's test, our modified test is asymptotically distribution-free and is usable in practice. A simulation study indicates that the proposed test maintains the nominal level of significance, α$\alpha$ fairly accurately even for samples of size as small as 20, and a comparison with the classical test based on sample coefficient of skewness, shows that our test has good power for detecting different asymmetric distributions.     

A bias correction and acceleration approach for the problem of regions

For testing the problem of regions in the space of distribution functions, this paper considers approaches to modify the bootstrap probability to be a second-order accurate p$p$-value based on the familiar bias correction and acceleration method. It is shown that Shimodaira's [2004a. Approximately unbiased tests of regions using multistep-multiscale bootstrap resampling. Ann. Statist. 32, 2616–2641] twostep-multiscale bootstrap method works even in the problem of regions in functional space. In this paper the bias correction quantity is estimated by his onestep-multiscale bootstrap method. Instead of using the twostep-multiscale bootstrap method, the acceleration constant is estimated by a newly proposed jackknife method which requires first-level bootstrap resamplings only. Some numerical examples are illustrated, in which an application to testing significance in model selection is included.     

Generalized partially linear varying-coefficient models

Generalized varying-coefficient models are useful extensions of generalized linear models. They arise naturally when investigating how regression coefficients change over different groups characterized by certain covariates such as age. In this paper, we extend these models to generalized partially linear varying-coefficient models, in which some coefficients are constants and the others are functions of certain covariates. Procedures for estimating the linear and non-parametric parts are developed and their associated statistical properties are studied. The methods proposed are illustrated using some simulations and real data analysis.