大数据研究经典的研读与批判—读维克托·迈尔 舍恩佰格《大数据时代》有感

作者在研读维克托·迈尔-舍恩佰格《大数据时代》后,对舍恩佰格的三个主要观点展开讨论并提出质疑,认为随机样本和全体数据同样需要,精确性和混杂性各有所用,相关关系离不开因果关系.文章还介绍了国外学者关于大数据伦理的讨论.     

Two preprocessing algorithms for climate time series

We propose two preprocessing algorithms suitable for climate time series. The first algorithm detects outliers based on an autoregressive cost update mechanism. The second one is based on the wavelet transform, a method from pattern recognition. In order to benchmark the algorithms'' performance we compare them to existing methods based on a synthetic data set. Eventually, for exemplary purposes, the proposed methods are applied to a data set of high-frequent temperature measurements from Novi Sad, Serbia. The results show that both methods together form a powerful tool for signal preprocessing: In case of solitary outliers the autoregressive cost update mechanism prevails, whereas the wavelet-based mechanism is the method of choice in the presence of multiple consecutive outliers.     

A scoping review of knowledge syntheses in the field of evaluation across four decades of practice

This scoping review of 62 knowledge syntheses published in evaluation-focused journals between 1979 and May 2018 provides a portrait of synthesis practices and their evolution in the mainstream of the field of evaluation. Concerns surrounding the production of knowledge syntheses to answer policy questions are not new in the field of evaluation. However, during this last decade, knowledge synthesis methods have expanded as a means to go beyond the limits and constraints of singular evaluations. This scoping review reveals and discusses two key issues with regards to the expansion of knowledge synthesis practices within the field of evaluation: the diversity—and muddling— of methodological practices and synthesis designs, and the frequent omission of quality appraisals.     

The Role of Interorganizational and Organizational Factors on the Decision Mode for Adoption of Interorganizational Systems*

The dramatic growth of interorganizational systems (IOS) has changed the way organizations conduct their business, and has resulted in significant tangible and intangible benefits being realized by participating firms. However, the implementation of these IOS requires the cooperation and commitment of all the participating members. These members may have complex economic and business relationships among themselves that can result in a number of social, political, and economic factors influencing the adoption and implementation of IOS. This study examines the role of interorganizational and organizational factors on the decision mode for adoption of IOS, in the specific context of electronic data interchange (EDI). Four interorganizational factors, based on the socio-political framework derived from research in marketing, and five organizational factors based on research in IS were used in the study. The data for the study were collected through a large scale field survey. Two respondents, the sales/purchase manager and the IS manager, from 201 firms responded to the survey. The results of discriminant analysis of the data reveal that two interorganizational variables, competitive pressure and exercised power, and two organizational variables, internal need and top management support, are important variables to differentiate firms with proactive decision mode from firms with reactive decision mode. The study also evaluates the differences between proactive and reactive firms on three implementation outcomes. Proactive firms are found to have greater extent of adaptation, more external connectivity with trading partners, and better integration of EDI information in their internal IS applications.     

Estimating E-bayesian and hierarchical bayesian of scalar parameter of gompertz distribution under type II censoring schemes based on fuzzy data

In this study, the E-Bayesian and hierarchical Bayesian of the scalar parameter of a Gompertz distribution under Type II censoring schemes were estimated based on fuzzy data under the squared error (SE) loss function and the efficiency of the proposed methods was compared with each other and with the Bayesian estimator using Monte Carlo simulation.     

Toward an integrated Bayesian network approach to measurement error detection and correction

In this article, the quality of data produced by national statistical institutes and by governmental institutions is considered. In particular, the problem of measurement error is analyzed and an integrated Bayesian network decision support system based on non-parametric Bayesian networks is proposed for its detection and correction. Non-parametric Bayesian networks are graphical models expressing dependence structure via bivariate copulas associated to the edges of the graph. The network structure and the misreport probability are estimated using a validation sample. The Bayesian network model is proposed to decide: (i) which records have to be corrected; (ii) the kind and amount of correction to be adopted. The proposed correction procedure is applied to the Banca d’Italia Survey on Household Income and Wealth and, specifically, the bond amounts are analyzed. Finally, the sensitivity of the conditional distribution of the true value random variable given the observed one to different evidence configurations is studied.     

Multiple hypothesis tests based On conditional differences in means

Many hypothesis tests are univariate tests and cannot cope with multiple hypothesis without an auxiliary procedure as e. g. the Bonferroni-Holm-procedure. At the same time, there is an urgent need for testing multiple hypothesis due to the very simple existing methods as the Bonferroni-correction or the Bonferroni-Holm-procedure, which suffers from a very small local significance level to detect statistical inferences or the drawback that logical and statistical dependencies among the test statistics are not used, whereby its detection is NP-hard. In honour of this occasion, we present a multiple hypothesis test for i.i.d. random variables based on conditional differences in means, which is capable to cope with multiple hypothesis and does not suffer on such drawbacks as the Bonferroni-correction or the Bonferroni-Holm-procedure. Thereby, the computation time can be neglected.     

工业取用水监测奇异数据挖掘与重构方法

提高工业取用水监测数据质量是目前国家水资源监控能力建设的重要内容,而奇异值问题已成为影响监测数据质量的关键短板。本文在解析现阶段工业取用水监测数据奇异值主要类型基础上,以国家水资源管理系统数据库中工业取用水监测数据为样本,利用小波变换模极大值模型提取工业取用水监测数据时频变化特征,并利用傅里叶函数对其残差序列进行修正,进而运用相对误差控制方法挖掘监测数据奇异值。在此基础上,采用混沌粒子群优化的最小二乘支持向量机模型重构填补奇异值数据。研究结果表明：小波变换模极大值模型能够较好地提取工业取用水监测数据序列的时频变化特征,但是同时容易导致监测数据的信息损失,利用傅里叶函数对小波变换进行残差修正则可进一步提升取用水监测数据序列的特征提取效果;以小波变换模极大值特征序列为基础,通过相对误差控制可实现对监测数据奇异值的高效挖掘;对于挖掘出的奇异值重构填补问题,可选取混沌粒子群优化的最小二乘支持向量机模型,其重构精度要优于多项式曲线拟合等传统统计学方法和普通最小二乘支持向量机模型。上述工业取用水监测数据奇异值挖掘重构策略为现阶段国家水资源监控能力建设的推进提供了重要技术方法支持。     

大数据背景下网络调查样本的建模推断问题研究——以广义Boosted模型的倾向得分推断为例

随着大数据和网络的不断发展,网络调查越来越广泛,大部分网络调查样本属于非概率样本,难以采用传统的抽样推断理论进行推断,如何解决网络调查样本的推断问题是大数据背景下网络调查发展的迫切需求。本文首次从建模的角度提出了解决该问题的基本思路：一是入样概率的建模推断,可以考虑构建基于机器学习与变量选择的倾向得分模型来估计入样概率推断总体;二是目标变量的建模推断,可以考虑直接对目标变量建立参数、非参数或半参数超总体模型进行估计;三是入样概率与目标变量的双重建模推断,可以考虑进行倾向得分模型与超总体模型的加权估计与混合推断。最后,以基于广义Boosted模型的入样概率建模推断为例演示了具体解决方法。     

交通拥堵与雾霾污染的因果关系——基于收敛交叉映射技术的经验研究

交通拥堵与雾霾污染是制约现代城市发展的两大顽疾,准确识别交通拥堵与雾霾污染之间的交互影响,有助于城市管理者重新审视现行治堵与治霾政策的合理性。本文借助大数据平台采集了我国99个城市的高德拥堵延迟指数（CDI）、空气质量指数（AQI）及六种分项空气污染物浓度日报数据,首次采用收敛交叉映射（CCM）方法实证考察了交通拥堵与雾霾污染之间的因果关系。研究发现,CDI与AQI以及CDI与分项污染物组成的动态系统均呈现明显的非线性与弱耦合特征。基于CCM检验结果,大多数城市的CDI与AQI之间不存在显著的因果关系;从分项空气污染物的角度,大多数城市的CDI与主要空气污染物之间不存在显著因果关系,但与次要空气污染物之间却存在显著的单向或双向因果关系。上述结果表明,尽管交通拥堵与雾霾污染之间有一定关联,但在因果关系上现有的经验证据并不支持两者相互影响,治堵和治霾不能“一箭双雕”而必须“双管齐下”。本文的研究在经验上丰富了关于交通拥堵与雾霾污染交互影响的讨论,对城市管理者更加谨慎与合理地制定治堵政策与治霾政策有重要现实意义。