Back
Google's Pathways architecture
webCredibility Rating
4/5
High(4)High quality. Established institution or organization with editorial oversight and accountability.
Rating inherited from publication venue: Google AI
A 2021 blog post by Google's Jeff Dean announcing Pathways, a precursor architecture to models like PaLM and Gemini; relevant to discussions of general-purpose AI capabilities and the trajectory toward more general AI systems.
Metadata
Importance: 55/100blog postprimary source
Summary
Jeff Dean introduces Google's Pathways architecture, a proposed next-generation AI system designed to overcome the limitations of narrow, task-specific models. Pathways aims to handle thousands of tasks simultaneously, learn new tasks quickly with less data, and develop a more unified understanding of the world by routing tasks through sparse, relevant subnetworks rather than activating the entire model.
Key Points
- •Current AI models are over-specialized; Pathways aims to build one model capable of generalizing across many tasks simultaneously.
- •Pathways draws inspiration from the human brain's sparse activation pattern, only engaging relevant neural pathways for each task.
- •The architecture enables multi-modal learning (text, images, video, audio) within a single unified model.
- •Pathways could dramatically reduce the energy and computational cost of training by reusing learned representations across tasks.
- •This represents a shift from training thousands of narrow models to training one adaptable, general-purpose model.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| AI-Driven Concentration of Power | Risk | 65.0 |
Cached Content Preview
HTTP 200Fetched Apr 9, 202610 KB
Introducing Pathways: A next-generation AI architecture
Introducing Pathways: A next-generation AI architecture
Oct 28, 2021
·
Share
x.com
Facebook
LinkedIn
Mail
Copy link
Too often, machine learning systems overspecialize at individual tasks, when they could excel at many. That’s why we’re building Pathways—a new AI architecture that will handle many tasks at once, learn new tasks quickly and reflect a better understanding of the world.
Jeff Dean
Google Senior Fellow and SVP, Google Research
Share
x.com
Facebook
LinkedIn
Mail
Copy link
When I reflect on the past two decades of computer science research, few things inspire me more than the remarkable progress we’ve seen in the field of artificial intelligence.
In 2001, some colleagues sitting just a few feet away from me at Google realized they could use an obscure technique called machine learning to help correct misspelled Search queries. (I remember I was amazed to see it work on everything from “ayambic pitnamiter” to “unnblevaiabel”). Today, AI augments many of the things that we do, whether that’s helping you capture a nice selfie , or providing more useful search results , or warning hundreds of millions of people when and where flooding will occur . Twenty years of advances in research have helped elevate AI from a promising idea to an indispensable aid in billions of people’s daily lives. And for all that progress, I’m still excited about its as-yet-untapped potential – AI is poised to help humanity confront some of the toughest challenges we’ve ever faced, from persistent problems like illness and inequality to emerging threats like climate change.
But matching the depth and complexity of those urgent challenges will require new, more capable AI systems – systems that can combine AI’s proven approaches with nascent research directions to be able to solve problems we are unable to solve today. To that end, teams across Google Research are working on elements of a next-generation AI architecture we think will help realize such systems.
We call this new AI architecture Pathways.
Pathways is a new way of thinking about AI that addresses many of the weaknesses of existing system
... (truncated, 10 KB total)Resource ID:
6f3d720bc62d3f5b | Stable ID: sid_LIHZFwpGHm