《斯坦福年度AI报告：人工智能全面逼近人类能力(101页).pdf》由会员分享，可在线阅读，更多相关《斯坦福年度AI报告：人工智能全面逼近人类能力(101页).pdf（101页珍藏版）》请在taowenge.com淘文阁网|工程机械CAD图纸|机械工程制图|CAD装配图下载|SolidWorks_CaTia_CAD_UG_PROE_设计图分享下载上搜索。
1、?AI INDEX,NOVEMBER 2017?1?AI INDEX,NOVEMBER 2017STEERING COMMITTEE Yoav Shoham(chair)Stanford University Raymond Perrault SRI International Erik Brynjolfsson MIT Jack Clark OpenAI PROJECT MANAGER Calvin LeGassick?2?AI INDEX,NOVEMBER 2017TABLE OF CONTENTSIntroduction to the AI Index 2017 Annual Repor
2、t 5 Overview 7 Volume of Activity 9 Academia 9 Published Papers 9 Course Enrollment 11 Conference Attendance 14 Industry 16 AI-Related Startups 16 AI-Related Startup Funding 17 Job Openings 18 Robot Imports 21 Open Source Software 23 GitHub Project Statistics 23 Public Interest 25 Sentiment of Media
3、 Coverage 25 Technical Performance 26 Vision 26 Object Detection 26 Visual Question Answering 27 Natural Language Understanding 28 Parsing 28 Machine Translation 29 Question Answering 30 Speech Recognition 31?3?AI INDEX,NOVEMBER 2017Theorem Proving 32 SAT Solving 33 Derivative Measures 34 Towards Hu
4、man-Level Performance?37 Whats Missing?41 Expert Forum 44 Get Involved!68 Acknowledgements 70 Appendix A:Data Description&Collection Methodology 72?4?AI INDEX,NOVEMBER 2017INTRODUCTION TO THE AI INDEX 2017 ANNUAL REPORT Artificial Intelligence has leapt to the forefront of global discourse,garnering
5、 increased attention from practitioners,industry leaders,policymakers,and the general public.The diversity of opinions and debates gathered from news articles this year illustrates just how broadly AI is being investigated,studied,and applied.However,the field of AI is still evolving rapidly and eve
6、n experts have a hard time understanding and tracking progress across the field.?5?AI INDEX,NOVEMBER 2017Without the relevant data for reasoning about the state of AI technology,we are essentially“flying blind”in our conversations and decision-making related to AI.We are essentially“flying blind”in
7、our conversations and decision-making related to Artificial Intelligence.Created and launched as a project of the OneHundredYear Study on AI at Stanford University(AI100),the AI Index is an open,not-for-profit project to track activity and progress in AI.It aims to facilitate an informed conversatio
8、n about AI that is grounded in data.This is the inaugural annual report of the AI Index,and in this report we look at activity and progress in Artificial Intelligence through a range of perspectives.We aggregate data that exists freely on the web,contribute original data,and extract new metrics from
9、 combinations of data series.All of the data used to generate this report will be openly available on the AI Index website at aiindex.org.Providing data,however,is just the beginning.To become truly useful,the AI Index needs support from a larger community.Ultimately,this report is a call for partic
10、ipation.You have the ability to provide data,analyze collected data,and make a wish list of what data you think needs to be tracked.Whether you have answers or questions to provide,we hope this report inspires you to reach out to the AI Index and become part of the effort to ground the conversation
11、about AI.?6?AI INDEX,NOVEMBER 2017OVERVIEW The first half of the report showcases data aggregated by the AI Index team.This is followed by a discussion of key areas the report does not address,expert commentary on the trends displayed in the report,and a call to action to support our data collection
12、 efforts and join the conversation about measuring and communicating progress in AI technology.Data Sections The data in the report is broken into four primary parts:Volume of Activity Technical Performance Derivative Measures Towards Human-Level Performance?The Volume of Activity metrics capture th
13、e“how much”aspects of the field,like attendance at AI conferences and VC investments into startups developing AI systems.The Technical Performance metrics capture the“how good”aspects;for example,how well computers can understand images and prove mathematical theorems.The methodology used to collect
14、 each data set is detailed in the appendix.These first two sets of data confirm what is already well recognized:all graphs are“up and to the right,”reflecting the increased activity in AI efforts as well as the progress of the technology.In the Derivative Measures section we investigate the relation
15、ship between trends.We also introduce an exploratory measure,the AI Vibrancy Index,that combines trends across academia and industry to quantify the liveliness of AI as a field.When measuring the performance of AI systems,it is natural to look for comparisons to human performance.In the Towards Huma
16、n-Level Performance section we outline a short list of notable areas where AI systems have made significant progress towards?7?AI INDEX,NOVEMBER 2017matching or exceeding human performance.We also discuss the difficulties of such comparisons and introduce the appropriate caveats.Discussion Sections
17、Following the display of the collected data,we include some discussion of the trends this report highlights and important areas this report entirely omits.Part of this discussion centers on the limitations of the report.This report is biased towards US-centric data sources and may overestimate progr
18、ess in technical areas by only tracking well-defined benchmarks.It also lacks demographic breakdowns of data and contains no information about AI Research&Development investments by governments and corporations.These areas are deeply important and we intend to tackle them in future reports.We furthe
19、r discuss these limitations and others in the Whats Missing section of the report.As the reports limitations illustrate,the AI Index will always paint a partial picture.For this reason,we include subjective commentary from a cross-section of AI experts.This Expert Forum helps animate the story behin
20、d the data in the report and adds interpretation the report lacks.Finally,where the experts dialogue ends,your opportunity to Get Involved begins.We will need the feedback and participation of a larger community to address the issues identified in this report,uncover issues we have omitted,and build
21、 a productive process for tracking activity and progress in Artificial Intelligence.?8?AI INDEX,NOVEMBER 2017VOLUME OF ACTIVITY Academia Published Papers view more information in appendix A1 The number of Computer Science papers published and tagged with the keyword“Artificial Intelligence”in the Sc
22、opus database of academic papers.?99xThe number of AI papers produced each year has increased by more than 9x since 1996.?AI INDEX,NOVEMBER 2017A comparison of the annual publishing rates of different categories of academic papers,relative to their publishing rates in 1996.The graph displays the gro
23、wth of papers across all fields,papers within the Computer Science field,and AI papers within the Computer Science field.The data illustrates that growth in AI publishing is driven by more than a growing interest in the broader field of Computer Science.Concretely,while the number of papers within t
24、he general field of Computer Science has grown by 6x since 1996 the number of AI papers produced each year has increased by more than 9x in that same period.?10?AI INDEX,NOVEMBER 2017Course Enrollment view more information in appendix A2 The number of students enrolled in introductory Artificial Int
25、elligence&Machine Learning courses at Stanford University.ML is a subfield of AI.We highlight ML courses because of their rapid enrollment growth and because ML techniques are critical to many recent AI achievements.?11Introductory AI class enrollment at Stanford has increased 11x since 1996.Note:Th
26、e dip in Stanford ML enrollment for the 2016 academic year reflects an administrative quirk that year,not student interest.Details in appendix.11x?AI INDEX,NOVEMBER 2017We highlight Stanford because our data on other universities is limited.However,we can project that past enrollment trends at other
27、 universities are similar to Stanfords.?12Note:Many universities have offered AI courses since before the 90s.The graphs above represent the years for which we found available data.?AI INDEX,NOVEMBER 2017 It is worth noting that these graphs represent a specific sliver of the higher education landsc
28、ape,and the data is not necessarily representative of trends in the broader landscape of academic institutions.?13Note:Many universities have offered ML courses since before the 90s.The graphs above represent the years for which we found available data.?AI INDEX,NOVEMBER 2017Conference Attendance vi
29、ew more information in appendix A3 The number of attendees at a representative sample of AI conferences.The data is split into large conferences(over 1000 attendees)and small conferences(under 1000 attendees)in 2016.?14These attendance numbers show that research focus has shifted from symbolic reaso
30、ning to machine learning and deep learning.Shifting FocusNote:Most of the conferences have existed since the 1980s.The data above represents the years attendance data was recorded.?AI INDEX,NOVEMBER 2017?15Despite shifting focus,there is still a smaller research community making steady progress on s
31、ymbolic reasoning methods in AI.Steady Progress?AI INDEX,NOVEMBER 2017Industry AI-Related Startups view more information in appendix A4 The number of active venture-backed US private companies developing AI systems.?16The number of active US startups developing AI systems has increased 14x since 200
32、0.14x?AI INDEX,NOVEMBER 2017AI-Related Startup Funding view more information in appendix A5 The amount of annual funding by VCs into US AI startups across all funding stages.?17Annual VC investment into US startups developing AI systems has increased 6x since 2000.6x?AI INDEX,NOVEMBER 2017Job Openin
33、gs view more information in appendix A6 We obtained AI-related job growth data from two online job listing platforms,Indeed and Monster.AI-related jobs were identified with titles and keywords in descriptions.The growth of the share of US jobs requiring AI skills on the I platform.Growth is a multip
34、le of the share of jobs on the Indeed platform that required AI skills in the US in January 2013.?18The share of jobs requiring AI skills in the US has grown 4.5x since 2013.4.5x?AI INDEX,NOVEMBER 2017The growth of the share of jobs requiring AI skills on the I platform,by country.?19Note:Despite th
35、e rapid growth of the Canada and UK AI job markets,I reports they are respectively still 5%and 27%of the absolute size of the US AI job market.?AI INDEX,NOVEMBER 2017The total number of AI job openings posted the Monster platform in a given year,broken down by specific required skills.?20Note:A sing
36、le AI-related job may be double counted(belong to multiple categories).For example,a job may specifically require natural language processing and computer vision skills.?AI INDEX,NOVEMBER 2017Robot Imports view more information in appendix A7 The number of shipments of industrial robot units into No
37、rth America and globally.?21?AI INDEX,NOVEMBER 2017The growth of shipments of industrial robot units into North America and globally.?22?AI INDEX,NOVEMBER 2017Open Source Software GitHub Project Statistics view more information in appendix A8 The number of times the TensorFlow and Scikit-Learn softw
38、are packages have been Starred on GitHub.TensorFlow and Scikit-Learn are popular software packages for deep learning and machine learning.Software developers“Star”software projects on GitHub to indicate projects they are interested in,express appreciation for projects,and navigate to projects quickl
39、y.Stars can provide a signal for developer interest in and usage of software.?23?AI INDEX,NOVEMBER 2017The number of times various AI&ML software packages have been Starred on GitHub.?24Note:Forks of GitHub repositories follow almost identical trends(though,the absolute number of forks and stars for
40、 each repo differ).See the appendix for info on gathering Forks data.?AI INDEX,NOVEMBER 2017Public Interest Sentiment of Media Coverage view more information in appendix A9 The percentage of popular media articles that contain the term“Artificial Intelligence”and that are classified as either Positi
41、ve or Negative articles.?25?AI INDEX,NOVEMBER 2017TECHNICAL PERFORMANCE Vision Object Detection view more information in appendix A10 The performance of AI systems on the object detection task in the Large Scale Visual Recognition Challenge(LSVRC)Competition.?26Error rates for image labeling have fa
42、llen from 28.5%to below 2.5%since 2010.2.5%?AI INDEX,NOVEMBER 2017 Visual Question Answering view more information in appendix A11 The performance of AI systems on a task to give open-ended answers to questions about images.?27Note:The VQA 1.0 data set has already been surpassed by the VQA 2.0 data
43、set and it is unclear how much further attention the VQA 1.0 data set will receive.?AI INDEX,NOVEMBER 2017Natural Language Understanding Parsing view more information in appendix A12 The performance of AI systems on a task to determine the syntactic structure of sentences.?28?AI INDEX,NOVEMBER 2017M
44、achine Translation view more information in appendix A13 The performance of AI systems on a task to translate news between English and German.?29?AI INDEX,NOVEMBER 2017Question Answering view more information in appendix A14 The performance of AI systems on a task to find the answer to a question wi
45、thin a document.?30?AI INDEX,NOVEMBER 2017Speech Recognition view more information in appendix A15 The performance of AI systems on a task to recognize speech from phone call audio.?31?AI INDEX,NOVEMBER 2017Theorem Proving view more information in appendix A16 The average tractability of a large set
46、 of theorem proving problems for Automatic Theorem Provers.“Tractability”measures the fraction of state-of-the-art Automatic Theorem Provers that can solve a problem.See appendix for details about the“tractability”metric.?32Note:Average tractability can go down if state-of-the-art solvers are introd
47、uced that perform well on novel problems but poorly on problems other solvers are good at.?AI INDEX,NOVEMBER 2017SAT Solving view more information in appendix A17 The average performance of competitive SAT solvers on industry-applicable problems.?33?AI INDEX,NOVEMBER 2017DERIVATIVE MEASURES We can g
48、lean additional insights from the measurements in the previous sections by examining the relationships between trends.This section demonstrates how the data gathered by the AI Index can be used for further analysis and to spur the development of refined and wholly original metrics.As a case-study fo
49、r this demonstration,we look at trends across academia and industry to explore their dynamics.Further,we aggregate these metrics into a combined AI Vibrancy Index.Academia-Industry Dynamics To explore the relationship between AI-related activity in academia and industry,we first select a few represe
50、ntative measurements from the previous sections.In particular,we look at AI paper publishing,combined enrollment in introductory AI and ML courses at Stanford,and VC investments into AI-related startups.These metrics represent quantities that cannot be compared directly:papers published,students enr