Introducing Meta Llama 3: The most capable openly available LLM to date
webCredibility Rating
High quality. Established institution or organization with editorial oversight and accountability.
Rating inherited from publication venue: Meta AI
Official Meta announcement of Llama 3; relevant to AI safety discussions around open-weight frontier models, dual-use risks, and the safety measures (or lack thereof) accompanying powerful openly available systems. The original tags referencing disinformation appear misassigned.
Metadata
Summary
Meta announces Llama 3, their most capable openly available large language model family, featuring 8B and 70B parameter models with improved reasoning, coding, and instruction-following capabilities. The release includes details on training data, architecture improvements, and safety measures implemented before public release. Llama 3 represents a significant milestone in open-weight frontier model development.
Key Points
- •Releases 8B and 70B parameter models pretrained and instruction-tuned, with a 400B+ model still in training at time of announcement
- •Trained on over 15 trillion tokens from publicly available sources, with improved data quality filtering and curation
- •Introduces a new 128K token vocabulary tokenizer and grouped query attention (GQA) for improved efficiency
- •Benchmarks show competitive performance with leading closed models like GPT-3.5 and approaches GPT-4 on several tasks
- •Includes Meta's safety tooling (Llama Guard 2, Code Shield, CyberSec Eval) and responsible use guidelines alongside the release
Cited by 3 pages
| Page | Type | Quality |
|---|---|---|
| AI Scaling Laws | Concept | 92.0 |
| Meta AI (FAIR) | Organization | 51.0 |
| AI Disinformation | Risk | 54.0 |
Cached Content Preview
Introducing Meta Llama 3: The most capable openly available LLM to date
Products
AI Research
The Latest
About
Get Llama
Try Meta AI
Large Language Model Introducing Meta Llama 3: The most capable openly available LLM to date
April 18, 2024
Takeaways:
RECOMMENDED READS
5 Steps to Getting Started with Llama 2
The Llama Ecosystem: Past, Present, and Future
Introducing Code Llama, a state-of-the-art large language model for coding
Meta and Microsoft Introduce the Next Generation of Llama
Today, we’re introducing Meta Llama 3, the next generation of our state-of-the-art open source large language model.
Llama 3 models will soon be available on AWS, Databricks, Google Cloud, Hugging Face, Kaggle, IBM WatsonX, Microsoft Azure, NVIDIA NIM, and Snowflake, and with support from hardware platforms offered by AMD, AWS, Dell, Intel, NVIDIA, and Qualcomm.
We’re dedicated to developing Llama 3 in a responsible way, and we’re offering various resources to help others use it responsibly as well. This includes introducing new trust and safety tools with Llama Guard 2, Code Shield, and CyberSec Eval 2.
In the coming months, we expect to introduce new capabilities, longer context windows, additional model sizes, and enhanced performance, and we’ll share the Llama 3 research paper.
Meta AI, built with Llama 3 technology, is now one of the world’s leading AI assistants that can boost your intelligence and lighten your load—helping you learn, get things done, create content, and connect to make the most out of every moment. You can try Meta AI here .
Today, we’re excited to share the first two models of the next generation of Llama, Meta Llama 3, available for broad use. This release features pretrained and instruction-fine-tuned language models with 8B and 70B parameters that can support a broad range of use cases. This next generation of Llama demonstrates state-of-the-art performance on a wide range of industry benchmarks and offers new capabilities, including improved reasoning. We believe these are the best open source models of their class, period. In support of our longstanding open approach, we’re putting Llama 3 in the hands of the community. We want to kickstart the next wave of innovation in AI across the stack—from applications to developer tools to evals to inference optimizations and more. We can’t wait to see what you build and look forward to your feedback.
Our goals for Llama 3
With Llama 3, we set out to build the best open models that are on par with the best proprietary models available today. We wanted to address developer feedback to increase the overall helpfulness of Llama 3 and are doing so while continuing to play a leading role on responsible use and deployment of LLMs. We are embracing the open source ethos of releasing early and often to enable the community to get access to these models while they are still in development. The text-based models we are releasing today
... (truncated, 18 KB total)f9616f30e8f51cb0 | Stable ID: MmI1NzhkNz