[2007.05558] The Computational Limits of Deep Learning

[2007.05558] The Computational Limits of Deep Learning 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 

 
 
 
 
 
--> 

 
 
 Computer Science > Machine Learning

 

 
 arXiv:2007.05558 (cs)
 
 
 
 
 
 [Submitted on 10 Jul 2020 ( v1 ), last revised 27 Jul 2022 (this version, v2)] 
 Title: The Computational Limits of Deep Learning

 Authors: Neil C. Thompson , Kristjan Greenewald , Keeheon Lee , Gabriel F. Manso View a PDF of the paper titled The Computational Limits of Deep Learning, by Neil C. Thompson and 3 other authors 
 View PDF 

 
 Abstract: Deep learning's recent history has been one of achievement: from triumphing over humans in the game of Go to world-leading performance in image classification, voice recognition, translation, and other tasks. But this progress has come with a voracious appetite for computing power. This article catalogs the extent of this dependency, showing that progress across a wide variety of applications is strongly reliant on increases in computing power. Extrapolating forward this reliance reveals that progress along current lines is rapidly becoming economically, technically, and environmentally unsustainable. Thus, continued progress in these applications will require dramatically more computationally-efficient methods, which will either have to come from changes to deep learning or from moving to other machine learning methods.
 

 
 
 
 Comments: 
 33 pages, 8 figures 
 
 
 Subjects: 
 
 Machine Learning (cs.LG) ; Machine Learning (stat.ML) 
 
 Cite as: 
 arXiv:2007.05558 [cs.LG] 
 
 
 
 (or 
 arXiv:2007.05558v2 [cs.LG] for this version)
 
 
 
 
 https://doi.org/10.48550/arXiv.2007.05558 
 
 
 Focus to learn more 
 
 
 
 arXiv-issued DOI via DataCite 
 
 
 
 
 
 
 
 Submission history

 From: Neil Thompson [ view email ] 
 [v1] 
 Fri, 10 Jul 2020 18:26:17 UTC (1,871 KB)

 [v2] 
 Wed, 27 Jul 2022 17:26:18 UTC (1,717 KB)

 
 
 
 
 
 Full-text links: 
 Access Paper:

 
 
View a PDF of the paper titled The Computational Limits of Deep Learning, by Neil C. Thompson and 3 other authors View PDF 
 TeX Source
 
 
 
 
 view license 
 
 
 
 Current browse context: cs.LG 

 
 
 < prev 
 
 | 
 next > 
 

 
 new 
 | 
 recent 
 | 2020-07 
 
 Change to browse by:
 
 cs 
 stat 
 stat.ML 
 
 

 
 
 References & Citations

 
 NASA ADS 
 Google Scholar 

 Semantic Scholar 

 
 
 

 
 
 3 blog links 

 ( what is this? )
 
 
 
 DBLP - CS Bibliography

 
 listing | bibtex 
 
 Kristjan H. Greenewald 
 
 export BibTeX citation 
 Loading... 
 

 
 
 
 BibTeX formatted citation

 &times; 
 
 
 loading... 
 
 
 Data provided by: 
 
 
 
 
 Bookmark

 
 
 
 
 
 
 
 
 
 
 
 Bibliographic Tools 
 
 Bibliographic and Citation Tools

 
 
 
 
 
 
 Bibliographic Explorer Toggle 
 
 
 
 Bibliographic Explorer ( What is the Explorer? ) 
 
 
 
 
 
 
 
 Connected Papers Toggle 
 
 
 
 Connected Papers ( What is Connected Papers? ) 
 
 
 
 
 
 
 Litmaps Toggle 
 
 
 
 Litmaps ( What is Litmaps? ) 
 
 
 
 
 

... (truncated, 5 KB total)