自然评价：人工智能驱动下的学术成果评价模式重构

学术成果评价作为科研项目评审、人才评价、期刊评价、机构评价等活动的基础,是学术评价体系的重中之重,也是科研管理的关键环节之一。当前,我国学术成果评价模式存在“同行评议黑箱”“引文动机模糊”“评价数据间接且片面”等弊端,阻碍了学术创新,也与当下建构自主知识体系的战略相悖。而近年来人工智能等技术在数据、算法、算力上的突破性发展,特别是判别式模型和生成式模型在机器翻译、文本分类、文本摘要、情感分析、问答系统等领域日益成熟的应用,为传统学术成果评价模式的革新带来契机,为实现更加科学、多元与智能的学术成果评价提供了充分可能。基于对现有学术成果评价模式不足与人工智能应用契合度的分析,文章提出学术成果评价新模式——自然评价。自然评价模式是人工智能与学术成果评价的深度融合,既是对以往同行评议、文献计量、网络计量、替代计量等传统评价模式的批判性继承与发展,也是一种突破与革新。具体而言,自然评价是基于学术共同体在各种学术活动中自然产生的全量化的学术痕迹大数据,将人工智能技术应用于学术成果知识内容和学术共同体学术痕迹数据的语义理解与自动分析中,从而动态形成评价判断,并服务于知识创新与学术进步的一种评价模式。...

Natural evaluation: The reconstruction of the academic Publication evaluation model driven by artificial intelligence

As the basis for the evaluation of scientific research projects, talents, journals, and institutions, academic publication evaluation is an important part of the academic evaluation system and a key step in scientific research management. The current academic publication evaluation model has some disadvantages such as "black box of peer review", "ambiguous motivation of citation" and "indirect and one-sided evaluation data", which hinders academic innovation and also runs counter to the current strategy of building an independent knowledge system. The breakthrough development of AI in data, algorithms, and arithmetic power in recent years, especially the increasingly mature application of discriminant and generative models in machine translation, text classification, text summary, dialogue system, etc. has brought a great opportunity for innovation of the traditional academic publication evaluation model and provided a possibility to realize a more scientific, pluralistic, and intelligent academic publication evaluation. From the analysis of the inadequacy of the existing evaluation models and the fit of AI applied to it, the paper proposes a new model of academic publication evaluation, namely Natural Evaluation. Natural Evaluation is an integration of AI and academic publication evaluation, which is both a critical inheritance and a breakthrough of the traditional evaluation models such as peer review, bibliometrics, webometrics, and alternative metrics. Specifically, Natural Evaluation is an evaluation model based on the fully quantified academic trace data naturally generated by the academic community in various scholarly activities, and AI technology is applied to the semantic understanding and automatic analysis of the knowledge content of academic results and academic trace data of the community, to dynamically form evaluation judgments and serve knowledge innovation. The paper discusses the technical logic, value compliance, and prospect of Natural Evaluation driven by AI from the doctrinal level. In terms of technical logic, Natural Evaluation is based on the naturally generated trace data from various scholarly activities, and the algorithm supports the intelligent extraction of semantics in the data and generates evaluation judgments, and enhances the accuracy and efficiency of evaluation through arithmetic power empowerment. In terms of value compliance, Natural Evaluation upholds the value principles of quality first, impartiality, and comprehensiveness, and seeks to break out of the trap of "digital discipline", and break the shackles of "human favor dominance" and "one-sided evaluation". In terms of the future outlook, Nature Evaluation presents a picture of responding to the trend of the open science era, optimizing the ecological environment of academic innovation, and promoting the construction of independent knowledge systems. Finally, although the technical logic, value compliance, and prospect of Natural Evaluation have been systematically discussed from the theoretical level, and it can be theoretically proved to present more scientific evaluation results, the specific difficulties of AI technology implementation, academic community evaluation incentive, human-machine relationship coordination, and evaluation empowerment of different subjects are yet to be further researched in the future.