Multivariate outlier detection in incomplete survey data: the epidemic algorithm and transformed rank correlations

Summary.  As a part of the EUREDIT project new methods to detect multivariate outliers in incomplete survey data have been developed. These methods are the first to work with sampling weights and to be able to cope with missing values. Two of these methods are presented here. The epidemic algorithm simulates the propagation of a disease through a population and uses extreme infection times to find outlying observations. Transformed rank correlations are robust estimates of the centre and the scatter of the data. They use a geometric transformation that is based on the rank correlation matrix. The estimates are used to define a Mahalanobis distance that reveals outliers. The two methods are applied to a small data set and to one of the evaluation data sets of the EUREDIT project.     

Human Salmonellosis: Estimation of Dose-Illness from Outbreak Data

The quantification of the relationship between the amount of microbial organisms ingested and a specific outcome such as infection, illness, or mortality is a key aspect of quantitative risk assessment. A main problem in determining such dose-response models is the availability of appropriate data. Human feeding trials have been criticized because only young healthy volunteers are selected to participate and low doses, as often occurring in real life, are typically not considered. Epidemiological outbreak data are considered to be more valuable, but are more subject to data uncertainty. In this article, we model the dose-illness relationship based on data of 20 Salmonella outbreaks, as discussed by the World Health Organization. In particular, we model the dose-illness relationship using generalized linear mixed models and fractional polynomials of dose. The fractional polynomial models are modified to satisfy the properties of different types of dose-illness models as proposed by Teunis et al . Within these models, differences in host susceptibility (susceptible versus normal population) are modeled as fixed effects whereas differences in serovar type and food matrix are modeled as random effects. In addition, two bootstrap procedures are presented. A first procedure accounts for stochastic variability whereas a second procedure accounts for both stochastic variability and data uncertainty. The analyses indicate that the susceptible population has a higher probability of illness at low dose levels when the combination pathogen-food matrix is extremely virulent and at high dose levels when the combination is less virulent. Furthermore, the analyses suggest that immunity exists in the normal population but not in the susceptible population.     

Designing and integrating composite networks for monitoring multivariate gaussian pollution fields

Networks of ambient monitoring stations are used to monitor environmental pollution fields such as those for acid rain and air pollution. Such stations provide regular measurements of pollutant concentrations. The networks are established for a variety of purposes at various times so often several stations measuring different subsets of pollutant concentrations can be found in compact geographical regions. The problem of statistically combining these disparate information sources into a single 'network' then arises. Capitalizing on the efficiencies so achieved can then lead to the secondary problem of extending this network. The subject of this paper is a set of 31 air pollution monitoring stations in southern Ontario. Each of these regularly measures a particular subset of ionic sulphate, sulphite, nitrite and ozone. However, this subset varies from station to station. For example only two stations measure all four. Some measure just one. We describe a Bayesian framework for integrating the measurements of these stations to yield a spatial predictive distribution for unmonitored sites and unmeasured concentrations at existing stations. Furthermore we show how this network can be extended by using an entropy maximization criterion. The methods assume that the multivariate response field being measured has a joint Gaussian distribution conditional on its mean and covariance function. A conjugate prior is used for these parameters, some of its hyperparameters being fitted empirically.     

Providing Spatial Data for Secondary Analysis: Issues and Current Practices Relating to Confidentiality

Spatially explicit data pose a series of opportunities and challenges for all the actors involved in providing data for long-term preservation and secondary analysis—the data producer, the data archive, and the data user. We report on opportunities and challenges for each of the three players, and then turn to a summary of current thinking about how best to prepare, archive, disseminate, and make use of social science data that have spatially explicit identification. The core issue that runs through the paper is the risk of the disclosure of the identity of respondents. If we know where they live, where they work, or where they own property, it is possible to find out who they are. Those involved in collecting, archiving, and using data need to be aware of the risks of disclosure and become familiar with best practices to avoid disclosures that will be harmful to respondents.

Efficiency measures of the Chinese commercial banking system using an additive two-stage DEA

Measuring and improving the efficiency of the Chinese commercial banking system has recently attracted increasing interest. Few studies, however, have adopted the two-stage network DEA to explore this issue in the Chinese context. Because the entire operational process of the banking system could be divided into two sub-processes (deposit producing and profit earning), the evaluation of the sub-process efficiencies could be used to assist in identifying the sources of the inefficiency of the entire banking system. In this study, we utilize the network DEA approach to disaggregate, evaluate and test the efficiencies of 16 major Chinese commercial banks during the third round of the Chinese banking reform period (2003–2011) with the variable returns to scale setting and the consideration of undesirable/bad output. The main findings of this study are as follows: (i) the two-stage DEA model is more effective than the conventional black box DEA model in identifying the inefficiency of banking system, and the inefficiency of the Chinese banking system primarily results from the inefficiency of its deposit producing sub-process; (ii) the overall efficiency of the Chinese banking system improves over the study period because of the reform; (iii) the state-owned commercial banks (SOBs) appear to be more overall efficient than the joint-stock commercial banks (JSBs) only in the pre-reform period, and the efficiency difference between the SOBs and the JSBs is reduced over the post-reform period; (iv) the disposal of non-performing loans (NPLs) from the Chinese banking system in general explains its efficiency improvement, and the joint-equity reform of the SOBs specifically increases their efficiencies.     

面板数据的贝叶斯Elastic Net分位数回归方法及其应用研究

本文首次将Elastic Net这种用于高度相关变量的惩罚方法用于面板数据的贝叶斯分位数回归,并基于非对称Laplace先验分布推导所有参数的后验分布,进而构建Gibbs抽样。为了验证模型的有效性,本文将面板数据的贝叶斯Elastic Net分位数回归方法(BQR. EN)与面板数据的贝叶斯分位数回归方法(BQR)、面板数据的贝叶斯Lasso分位数回归方法(BLQR)、面板数据的贝叶斯自适应Lasso分位数回归方法(BALQR)进行了多种情形下的全方位比较,结果表明BQR. EN方法适用于具有高度相关性、数据维度很高和尖峰厚尾分布特征的数据。进一步地,本文就BQR. EN方法在不同扰动项假设、不同样本量的情形展开模拟比较,验证了新方法的稳健性和小样本特性。最后,本文选取互联网金融类上市公司经济增加值(EVA)作为实证研究对象,检验新方法在实际问题中的参数估计与变量选择能力,实证结果符合预期。     

Two preprocessing algorithms for climate time series

We propose two preprocessing algorithms suitable for climate time series. The first algorithm detects outliers based on an autoregressive cost update mechanism. The second one is based on the wavelet transform, a method from pattern recognition. In order to benchmark the algorithms'' performance we compare them to existing methods based on a synthetic data set. Eventually, for exemplary purposes, the proposed methods are applied to a data set of high-frequent temperature measurements from Novi Sad, Serbia. The results show that both methods together form a powerful tool for signal preprocessing: In case of solitary outliers the autoregressive cost update mechanism prevails, whereas the wavelet-based mechanism is the method of choice in the presence of multiple consecutive outliers.     

Estimating E-bayesian and hierarchical bayesian of scalar parameter of gompertz distribution under type II censoring schemes based on fuzzy data

In this study, the E-Bayesian and hierarchical Bayesian of the scalar parameter of a Gompertz distribution under Type II censoring schemes were estimated based on fuzzy data under the squared error (SE) loss function and the efficiency of the proposed methods was compared with each other and with the Bayesian estimator using Monte Carlo simulation.     

Toward an integrated Bayesian network approach to measurement error detection and correction

In this article, the quality of data produced by national statistical institutes and by governmental institutions is considered. In particular, the problem of measurement error is analyzed and an integrated Bayesian network decision support system based on non-parametric Bayesian networks is proposed for its detection and correction. Non-parametric Bayesian networks are graphical models expressing dependence structure via bivariate copulas associated to the edges of the graph. The network structure and the misreport probability are estimated using a validation sample. The Bayesian network model is proposed to decide: (i) which records have to be corrected; (ii) the kind and amount of correction to be adopted. The proposed correction procedure is applied to the Banca d’Italia Survey on Household Income and Wealth and, specifically, the bond amounts are analyzed. Finally, the sensitivity of the conditional distribution of the true value random variable given the observed one to different evidence configurations is studied.     

Multiple hypothesis tests based On conditional differences in means

Many hypothesis tests are univariate tests and cannot cope with multiple hypothesis without an auxiliary procedure as e. g. the Bonferroni-Holm-procedure. At the same time, there is an urgent need for testing multiple hypothesis due to the very simple existing methods as the Bonferroni-correction or the Bonferroni-Holm-procedure, which suffers from a very small local significance level to detect statistical inferences or the drawback that logical and statistical dependencies among the test statistics are not used, whereby its detection is NP-hard. In honour of this occasion, we present a multiple hypothesis test for i.i.d. random variables based on conditional differences in means, which is capable to cope with multiple hypothesis and does not suffer on such drawbacks as the Bonferroni-correction or the Bonferroni-Holm-procedure. Thereby, the computation time can be neglected.