Goodfire

📋Page Status

Page Type:ContentStyle Guide →Standard knowledge base article

Quality:68 (Good)

Importance:72 (High)

Last edited:2026-02-01 (5 days ago)

Words:2.0k

Structure:

📊 3📈 0🔗 18📚 44•16%Score: 13/15

LLM Summary:Goodfire is a well-funded AI interpretability startup developing mechanistic interpretability tools like Ember API to make neural networks more transparent and steerable. While representing significant progress in interpretability research with strong backing including Anthropic's first direct investment, key uncertainties remain about scalability to advanced AI systems and effectiveness against sophisticated deception.

Quick Assessment

Dimension	Assessment
Founded	June 2024
Type	Public benefit corporation, AI interpretability research lab
Location	San Francisco, California
Funding	$57M+ (Seed: $7M Aug 2024, Series A: $50M Apr 2025)
Key Product	Ember (mechanistic interpretability API and platform)
Focus	Mechanistic interpretability, sparse autoencoders, AI safety
Notable Backers	Anthropic (first direct investment), Menlo Ventures, Lightspeed Venture Partners

Key Links

Source	Link
Official Website	goodfire.ai
Wikipedia	en.wikipedia.org

Overview

Goodfire is an AI interpretability research lab and public benefit corporation specializing in mechanistic interpretability—the science of reverse-engineering neural networks to understand and control their internal workings.¹ Founded in June 2024 by Eric Ho (CEO), Dan Balsam (CTO), and Tom McGrath (Chief Scientist), the company aims to transform opaque AI systems into transparent, steerable, and safer technologies.²

The company’s flagship product, Ember, is the first hosted mechanistic interpretability API, providing researchers and developers with programmable access to AI model internals.³ Rather than treating models as black boxes, Ember enables users to examine individual “features” (interpretable patterns of neural activation), edit model behavior without retraining, and audit for safety issues before deployment. The platform supports models like Llama 3.3 70B and processes tokens at a rate that has tripled monthly since launch in December 2024.⁴

Goodfire’s rapid ascent reflects growing industry recognition that interpretability is foundational to AI safety. The company raised $50 million in Series A funding less than one year after founding, led by Menlo Ventures with participation from Anthropic—marking Anthropic’s first direct investment in another company.⁵ Anthropic CEO Dario Amodei stated that “mechanistic interpretability is among the best bets to help us transform black-box neural networks into understandable, steerable systems.”⁶

History and Founding

The founding team brought complementary expertise from both entrepreneurship and frontier AI research. Eric Ho and Dan Balsam had previously co-founded RippleMatch in 2016, an AI-powered hiring platform that Ho scaled to over $10 million in annual recurring revenue.⁷ Ho’s work at RippleMatch earned him recognition on Forbes’s 30 Under 30 list in 2022.⁸

Tom McGrath, the company’s Chief Scientist, is recognized as a pioneering figure in mechanistic interpretability. He completed his PhD in 2016 and co-founded the Interpretability team at Google DeepMind, where he served as a Senior Research Scientist.⁹ In March 2024, McGrath left Google to join South Park Commons, a community for technologists, with the explicit goal of making interpretability “useful” by starting a company.¹⁰ He connected with Ho and Balsam shortly thereafter to launch Goodfire.

The company’s founding in June 2024 was followed quickly by a $7 million seed round in August 2024, led by Lightspeed Venture Partners with participation from Menlo Ventures, South Park Commons, Work-Bench, and others.¹¹ Less than one year later, in April 2025, Goodfire announced its $50 million Series A at a $200 million valuation.¹²

Team and Expertise

Beyond the three founders, Goodfire has assembled a team of leading researchers from OpenAI and DeepMind. The team includes:¹³

Lee Sharkey: Pioneered the use of sparse autoencoders in language models and co-founded Apollo Research
Nick Cammarata: Started the interpretability team at OpenAI and worked closely with Chris Olah, widely considered the founder of mechanistic interpretability

The team’s collective contributions include authoring the three most-cited papers in mechanistic interpretability and pioneering techniques like sparse autoencoders (SAEs) for feature discovery, auto-interpretability methods, and knowledge extraction from models like AlphaZero.¹⁴

Technology and Products

Ember Platform

Ember is Goodfire’s core product—a mechanistic interpretability API that provides direct, programmable access to AI model internals.¹⁵ Unlike traditional approaches that treat models as black boxes accessible only through prompts, Ember allows users to:

Examine features: Identify interpretable patterns in neural activations (e.g., features representing “professionalism,” “sarcasm,” or specific knowledge domains)
Steer behavior: Adjust feature activations to control model outputs without retraining or complex prompt engineering (e.g., making a model more “wise sage”-like by amplifying philosophical reasoning features)¹⁶
Debug and audit: Trace decision pathways, detect biases, identify vulnerabilities, and uncover hidden knowledge
Model diffing: Track changes across training checkpoints to understand why problematic behaviors emerge¹⁷

The platform is model-agnostic and currently supports models including Llama 3.3 70B and Llama 3.1 8B. Token processing has tripled monthly since launch, with hundreds of researchers using the platform.¹⁸

Auto Steer Method

Goodfire developed an “Auto Steer” method for automated behavioral adjustments. Independent evaluation found it effective for certain behavioral objectives (like “be professional”) but noted a coherence gap—outputs sometimes became less coherent compared to traditional prompt engineering.¹⁹ This highlights the practical challenges of translating interpretability research into production-ready tools.

Safety Applications

Goodfire emphasizes safety-first applications of interpretability:²⁰

Auditing: Probing model behaviors to identify misalignment, biases, and vulnerabilities
Conditional steering: Preventing jailbreaks by applying context-dependent behavioral controls (tested on the StrongREJECT adversarial dataset)
Model diffing: Detecting how and why unsafe behaviors emerge during training or fine-tuning
PII detection: Partnering with Rakuten to use sparse autoencoder probes to prevent personally identifiable information leakage²¹

Pre-release safety measures for Ember include feature moderation (removing harmful/explicit/malicious features), input/output filtering, and controlled access for researchers.²²

Partnerships and Impact

Goodfire has established collaborations with research institutions and industry partners:

Arc Institute: Early collaboration using Ember on Evo 2, a DNA foundation model, to uncover biological concepts and accelerate scientific discovery in genomics.²³
Mayo Clinic: Announced in September 2025, focusing on genomic medicine, reverse-engineering genomics models for insights into disease mechanisms while emphasizing data privacy and bias reduction.²⁴
Rakuten: Enhancing reliability for Rakuten AI, which serves over 44 million monthly users in Japan and 2 billion customers worldwide, focusing on preventing PII leakage using frontier interpretability techniques.²⁵
Haize Labs: Joint work on feature steering for AI safety auditing, red-teaming, and identifying failure modes in generative models.²⁶
Apollo Research: Using Goodfire tools for safety benchmarks and research.²⁷

In November 2024, Goodfire powered the “Reprogramming AI Models” hackathon in partnership with Apart Research, with over 200 researchers across 15 countries prototyping safety applications like adversarial attack detection and “unlearning” harmful capabilities while preserving beneficial behaviors.²⁸

AI Safety and Alignment

Goodfire positions mechanistic interpretability as foundational to AI alignment and safety. The company’s approach addresses several key challenges:

Alignment Without Side Effects

Traditional alignment methods like reinforcement learning from human feedback (RLHF) can produce unintended side effects, such as excessive refusal of benign requests or sycophantic behavior.²⁹ Goodfire’s feature steering offers an alternative by enabling precise, quantitative alignment of specific behaviors without degrading overall model performance.

Detecting Deception and Hidden Behaviors

One of the central challenges in AI safety is detecting deceptive or scheming behavior in advanced AI systems. Goodfire’s model diffing and auditing tools aim to identify rare, undesired behaviors—such as a model encouraging self-harm—that might emerge during training or deployment.³⁰ However, there is ongoing debate within the interpretability community about whether these techniques will scale to worst-case scenarios involving sophisticated deception.³¹

Governance and Compliance

Interpretability tools like Ember may become essential for regulatory compliance. The EU AI Act mandates transparency for high-risk AI systems, with fines up to €20 million for violations.³² Goodfire’s auditing and documentation capabilities could help organizations meet these requirements.

Fellowship Program

In October 2025, Goodfire announced a Fellowship Program for early- and mid-career researchers and engineers, matched with senior researchers to work on scientific discovery, interpretable models, and new interpretability methods.³³

Criticisms and Concerns

Despite significant progress, Goodfire’s approach faces several challenges and critiques:

Unproven Effectiveness in Worst-Case Scenarios

There is substantial debate about whether mechanistic interpretability can reliably detect deception in advanced AI systems. Researcher Neel Nanda has noted that interpretability lacks “ground truth” for what concepts AI models actually use, making it difficult to validate interpretability claims.³⁴ Some researchers favor alternative methods like linear probes.

A concrete example of interpretability’s limitations emerged with GPT-4o’s “extreme sycophancy” issue, which was detected behaviorally rather than through mechanistic analysis—no circuit was discovered, no particular weights or activations were identified as responsible, and mechanistic interpretability provided no advance warning.³⁵

Competition from In-House Development

Leading AI labs like Anthropic, OpenAI, and DeepMind have the resources to develop interpretability tools internally. Anthropic has publicly committed to investing significantly in reliably detecting AI model problems by 2027.³⁶ Additionally, open-source alternatives like Eleuther AI and InterpretML provide free interpretability frameworks, creating competitive pressure on commercial offerings.

Computational Cost and Accessibility

Goodfire’s pricing model is heavily compute-bound, with strict rate limits that may limit accessibility.³⁷ The platform enforces a 50,000 token/minute global cap shared across all API methods. Advanced interpretability functions like AutoSteer and AutoConditional are limited to just 30 requests/minute, while simpler utilities allow 1,000 requests/minute. This hierarchy suggests exponentially higher computational costs for core interpretability features, potentially creating barriers for smaller organizations and academic researchers.

Capabilities vs. Safety Framing

Third-hand reports indicate that Goodfire leadership has pitched interpretability work as “capabilities-enhancing” (improving AI performance) rather than primarily safety-focused when fundraising.³⁸ This framing raises questions about whether commercial incentives might prioritize performance improvements over safety applications—a tension common in dual-use AI research.

Funding and Business Model

Goodfire has raised approximately $57 million across two rounds:³⁹

Round	Date	Amount	Lead Investor	Key Participants	Valuation
Seed	August 2024	$7M	Lightspeed Venture Partners	Menlo Ventures, South Park Commons, Work-Bench, Juniper Ventures, Mythos Ventures, Bluebirds Capital	N/A
Series A	April 2025	$50M	Menlo Ventures	Anthropic, Lightspeed Venture Partners, B Capital, Work-Bench, Wing Ventures, South Park Commons, Metaplanet, Halcyon Ventures	$200M

The Series A round marked Anthropic’s first direct investment in another company, signaling significant industry validation.⁴⁰

Goodfire operates on a usage-based pricing model, charging per million tokens processed (input + output), with pricing tiered by model size:⁴¹

Smaller models (e.g., Llama 3.1 8B): $0.35/million tokens
Larger models (e.g., Llama 3.3 70B): $1.90/million tokens

The company is positioned to capture value in a rapidly growing market. The explainable AI market was valued at approximately $10 billion in 2025 and is projected to reach $25 billion by 2030.⁴²

Key Uncertainties

Scalability to superintelligent systems: Will mechanistic interpretability techniques that work on current models continue to provide safety guarantees as AI systems become more powerful and potentially deceptive?
Commercial viability: Can Goodfire compete with in-house interpretability teams at well-resourced AI labs and free open-source alternatives?
Capabilities vs. safety trade-offs: How will Goodfire navigate the tension between interpretability as a safety tool versus a capabilities enhancement that could accelerate AI development?
Ground truth validation: Without definitive ground truth about what concepts models represent, how can interpretability claims be rigorously validated?
Computational economics: Can the high computational costs of mechanistic interpretability be reduced sufficiently to enable widespread adoption?