|
|
|
| A Study of Innovation Evaluation Metrics for Interdisciplinary Studies: Take New Engineering Artificial Intelligence as an Example |
| Liu Xiang1, Li Hong2 |
1.Office of Talent Management, Hangzhou City University, Hangzhou 310012, China
2.Library, Zhejiang University, Hangzhou 310027, China |
|
|
|
|
Abstract The Ministry of Education has set up an “interdisciplinary” category and identified eight new first-level disciplines such as “intelligent science and technology”. The connotation of “Intelligent Science and Technology” covers new engineering majors such as “Artificial Intelligence” and “Robot Engineering”, and involves disciplines such as philosophy, science, engineering, and even literature, medicine, as well as related disciplines. Currently, there is a lack of indicator system for the evaluation of the interdisciplinary scientific researches and innovation capabilities. To provide reference for the evaluation of scientific research and innovation ability of the interdisciplinary “intelligent science and technology”. In the present paper，starting from the disciplinary characteristics of the interdisciplinary disciplines, an evaluation index system was established for the scientific and technological competitiveness of universities in the field of artificial intelligence corresponding to “intelligent science and technology”, which could objectively evaluate the scientific research and innovation capabilities of universities in this interdisciplinary field, timely track the development level of disciplinary competitiveness of universities in artificial intelligence, verify the feasibility of multiple indicator system and discipline characteristic indicators.Considering that universities’ scientific research and innovation capabilities are comprehensive, the indicator system cannot cover all innovation activities during the evaluation process, and a set of evaluation indicators for scientific and technological innovation capabilities cannot be applied to all disciplines. Therefore, an indicator system that conforms to the discipline development laws and development stages should be established based on the characteristics of sub disciplines. According to the disciplinary characteristics of artificial intelligence, the evaluation system for the technological innovation ability of artificial intelligence in universities includes four aspects: basic research output, research quality, technological innovation output, and cutting-edge research representing new technologies, corresponding to four indicators: paper output, high-quality papers, patents, and funds.In addition to the scientific nature of evaluation indicators, the accessibility and clarity of indicator data are also important in the evaluation of scientific research innovation. The more interdisciplinary research fields are, the more difficult it is to obtain accurate indicator data. As a branch of intelligence science, in addition to journal papers, scholars pay more attention to international conference papers. Based on the characteristics of artificial intelligence, this study uses the publication of documents at top conferences in this field as one of the parameters for evaluating research output. Based on the conference directory of CSRankings AI and the list of international conferences recommended by the China Computer Society, a list of top academic conferences on artificial intelligence was established, and the availability and comparability of conference paper data in the list was investigated and confirmed. We investigated keywords of artificial intelligence, and established an IPC classification number mapping table for artificial intelligence patents using keywords. Through expert consultation, we confirmed the feasibility and accuracy of obtaining artificial intelligence patents based on the IPC classification number of this table. Fund data of universities was acquired according to interdisciplinary classification. Artificial intelligence research branches are mapped to existing fund discipline classifications to obtain reliable fund support information. It includes funding data for the Department of Information Science’s first level discipline, Artificial Intelligence, and the second level discipline, Artificial Intelligence Driven Automation and Robotics and Robotics Technology. Fund data of American universities was mainly collected through statistics on the fund information of secondary disciplines such as artificial intelligence, computer vision and pattern recognition, and human-computer interaction under computer science in the ASJC discipline classification.In the empirical study of scientific research innovation evaluation, 5 “double first class” universities in China and 5 top universities in the United States were selected to conduct simulation evaluation and research on the AI research competitiveness of each university using the proposed indicator system and data acquisition method. Through the research on the characteristic indicators for the evaluation of artificial intelligence research innovation in the field of “intelligent science and technology”, it can be seen that in this interdisciplinary field there is a significant gap between the research competitiveness of top domestic universities and top American universities in terms of research scale, research influence, front research, and other aspects. Therefore, when comprehensively evaluating the interdisciplinary scientific research competitiveness of universities, through comprehensive comparison of highly differentiated indicators, it is not only possible to understand the comprehensive level of scientific research of each university but also to understand the differences in specific indicators of each university, helping universities find in-depth exploration and analysis directions in the evaluation of scientific research competitiveness, and improving the accuracy of scientific research evaluation and decision-making.
|
|
Received: 22 September 2022
|
|
|
|
1 NSTC， “Preparing for the future of artificial intelligence，” 2016-10-14， https：//obamawhitehouse.archives.gov/sites/default/files/whitehouse_files/microsites/ostp/NSTC/preparing_for_the_future_of_ai.pdf， 2022-09-22.
2 NARD， “National AI R&D Strategic Plan： 2019 Update，” 2019-06-21， https：//www.nitrd.gov/pubs/National-AI-RD-Strategy-2019.pdf， 2022-09-22.
3 AIFGS， “Key points for a federal government strategy on artificial intelligence，” 2018-07-18， https：//www.bmas.de/SharedDocs/Downloads/EN/Topics/Labour-Market/key-points-ai-strategy.pdf？__blob=publicationFile&v=1， 2022-09-22.
4 国务院： 《新一代人工智能发展规划》，2017年7月20日，https：//www.gov.cn/zhengce/content/2017-07/20/content_5211996.htm，2022年9月22日。
5 中华人民共和国教育部： 《高等学校人工智能创新行动计划》，2018年4月3日，http：//www.moe.gov.cn/srcsite/A16/s7062/201804/t20180410_332722.html，2022年9月22日。
6 Rob M.， “Building site identified for MIT Stephen A. Schwarzman College of Computing，” 2018-12-19， http：//news.mit.edu/2018/site-stephen-schwarzman-college-computing-1219， 2022-09-22.
7 中华人民共和国教育部： 《2022年度普通高等学校本科专业备案和审批结果》，2023年4月4日，http：//www.moe.gov.cn/srcsite/A08/moe_1034/s4930/202304/W020230419336779647503.pdf， 2023年4月20日。
8 Lyu Y. G.， “Artificial intelligence： enabling technology to empower society，” Engineering， Vol. 6， No. 3 （2020）， pp. 5-8.
9 季波、李魏、吕薇： 《人工智能本科人才培养的美国经验与启示——以卡内基梅隆大学为例》，《高等工程教育研究》2019年第6期，第194-200页。
10 吴飞、杨洋、何钦铭： 《人工智能本科专业课程设置思考：厘清内涵、促进交叉、赋能应用》，《中国大学教学》2019年第2期，第14-19页。
11 教育部、科技部： 《关于规范高等学校SCI论文相关指标使用 树立正确评价导向的若干意见》，2020年2月20日，http：//www.moe.gov.cn/srcsite/A16/moe_784/202002/t20200223_423334.html，2022年9月22日。
12 Berger A.， “CSRankings： computer science rankings，” 2020-01-01， http：//csrankings.org/#/index？all， 2022-09-22.
13 德勤中国： 《中国人工智能行业综述》，《科技中国》2019年第1期，第63-77页。
14 Anon.， “Artificial intelligence： how knowledge is created， transferred， and used，” 2019-04-04， https：//www.elsevier.com/？a=827872， 2022-11-14.
15 刘斐、张玲： 《科研评价服务常用工具及指标比较研究》，《情报探索》2019年第12期，第27-39页。
16 吴爱芝、肖珑、张春红等： 《基于文献计量的高校学科竞争力评估方法与体系》，《大学图书馆学报》2018年第1期，第62-67页。 |
|
|
|
2023, Vol. 53 
