Goodfire

Quick Assessment

Key Links

Overview

Goodfire is an AI interpretability research lab and public benefit corporation specializing in mechanistic interpretability—the science of reverse-engineering neural networks to understand and control their internal workings.¹ Founded in June 2024 by Eric Ho (CEO), Dan Balsam (CTO), and Tom McGrath (Chief Scientist), the company aims to transform opaque AI systems into transparent, steerable, and safer technologies.²

The company's flagship product, Ember, is the first hosted mechanistic interpretability API, providing researchers and developers with programmable access to AI model internals.³ Rather than treating models as black boxes, Ember enables users to examine individual "features" (interpretable patterns of neural activation), edit model behavior without retraining, and audit for safety issues before deployment. The platform supports models like Llama 3.3 70B and processes tokens at a rate that has tripled monthly since launch in December 2024.⁴

Goodfire's rapid ascent reflects growing industry recognition that interpretability is foundational to AI safety. The company raised $50 million in Series A funding less than one year after founding, led by Menlo Ventures with participation from AnthropicOrganizationAnthropicComprehensive reference page on Anthropic covering financials (\$380B valuation, \$19B ARR), safety research (Constitutional AI, mechanistic interpretability, model welfare), governance (LTBT struc...Quality: 74/100—marking Anthropic's first direct investment in another company.⁵ Anthropic CEO Dario AmodeiPersonDario AmodeiComprehensive biographical profile of Anthropic CEO Dario Amodei documenting his competitive safety development philosophy, 10-25% catastrophic risk estimate, 2026-2030 AGI timeline, and Constituti...Quality: 41/100 stated that "mechanistic interpretability is among the best bets to help us transform black-box neural networks into understandable, steerable systems."⁶ In February 2026, Goodfire raised a further $150 million in Series B funding at a $1.25 billion valuation, led by B Capital with participation from Salesforce Ventures, Eric Schmidt, and existing investors.⁷

History and Founding

The founding team brought complementary expertise from both entrepreneurship and frontier AI research. Eric Ho and Dan Balsam had previously co-founded RippleMatch in 2016, an AI-powered hiring platform that Ho scaled to over $10 million in annual recurring revenue.⁸ Ho's work at RippleMatch earned him recognition on Forbes's 30 Under 30 list in 2022.⁹

Tom McGrath, the company's Chief Scientist, is recognized as a pioneering figure in mechanistic interpretability. He completed his PhD in 2016 and co-founded the Interpretability team at Google DeepMind, where he served as a Senior Research Scientist.¹⁰ In March 2024, McGrath left Google to join South Park Commons, a community for technologists, with the explicit goal of making interpretability "useful" by starting a company.¹¹ He connected with Ho and Balsam shortly thereafter to launch Goodfire.

The company's founding in June 2024 was followed quickly by a $7 million seed round in August 2024, led by Lightspeed Venture Partners with participation from Menlo Ventures, South Park Commons, Work-Bench, and others.¹² Less than one year later, in April 2025, Goodfire announced its $50 million Series A at a $200 million valuation.¹³ In February 2026, the company closed a $150 million Series B at a $1.25 billion valuation, led by B Capital with participation from DFJ Growth, Salesforce Ventures, Eric Schmidt, and existing investors including Menlo Ventures and Lightspeed Venture Partners.⁷

Team and Expertise

Beyond the three founders, Goodfire has assembled a team of leading researchers from OpenAIOrganizationOpenAIComprehensive organizational profile of OpenAI documenting evolution from 2015 non-profit to Public Benefit Corporation, with detailed analysis of governance crisis, 2024-2025 ownership restructuri...Quality: 62/100 and DeepMindOrganizationGoogle DeepMindComprehensive overview of DeepMind's history, achievements (AlphaGo, AlphaFold with 200M+ protein structures), and 2023 merger with Google Brain. Documents racing dynamics with OpenAI and new Front...Quality: 37/100. The team includes:¹⁴

• Lee Sharkey: Pioneered the use of sparse autoencoders in language models and co-founded Apollo ResearchOrganizationApollo ResearchApollo Research demonstrated in December 2024 that all six tested frontier models (including o1, Claude 3.5 Sonnet, Gemini 1.5 Pro) engage in scheming behaviors, with o1 maintaining deception in ov...Quality: 58/100
• Nick Cammarata: Started the interpretability team at OpenAI and worked closely with Chris OlahPersonChris OlahBiographical overview of Chris Olah's career trajectory from self-taught researcher to Google Brain, OpenAI, and co-founding Anthropic, focusing on his work in mechanistic interpretability includin...Quality: 27/100, widely considered the founder of mechanistic interpretability

The team's collective contributions include authoring the three most-cited papers in mechanistic interpretability and pioneering techniques like sparse autoencoders (SAEs) for feature discovery, auto-interpretability methods, and knowledge extraction from models like AlphaZero.¹⁵

Technology and Products

Ember Platform

Ember is Goodfire's core product—a mechanistic interpretability API that provides direct, programmable access to AI model internals.¹⁶ Unlike traditional approaches that treat models as black boxes accessible only through prompts, Ember allows users to:

• Examine features: Identify interpretable patterns in neural activations (e.g., features representing "professionalism," "sarcasm," or specific knowledge domains)
• Steer behavior: Adjust feature activations to control model outputs without retraining or complex prompt engineering (e.g., making a model more "wise sage"-like by amplifying philosophical reasoning features)¹⁷
• Debug and audit: Trace decision pathways, detect biases, identify vulnerabilities, and uncover hidden knowledge
• Model diffing: Track changes across training checkpoints to understand why problematic behaviors emerge¹⁸

The platform is model-agnostic and currently supports models including Llama 3.3 70B and Llama 3.1 8B. Token processing has tripled monthly since launch, with hundreds of researchers using the platform.¹⁹

Auto Steer Method

Goodfire developed an "Auto Steer" method for automated behavioral adjustments. Independent evaluation found it effective for certain behavioral objectives (like "be professional") but noted a coherence gap—outputs sometimes became less coherent compared to traditional prompt engineering.²⁰ This highlights the practical challenges of translating interpretability research into production-ready tools.

Safety Applications

Goodfire emphasizes safety-first applications of interpretability:²¹

• Auditing: Probing model behaviors to identify misalignment, biases, and vulnerabilities
• Conditional steering: Preventing jailbreaks by applying context-dependent behavioral controls (tested on the StrongREJECT adversarial dataset)
• Model diffing: Detecting how and why unsafe behaviors emerge during training or fine-tuning
• PII detection: Partnering with Rakuten to use sparse autoencoder probes to prevent personally identifiable information leakage²²

Pre-release safety measures for Ember include feature moderation (removing harmful/explicit/malicious features), input/output filteringApproachAI Output FilteringComprehensive analysis of AI output filtering showing detection rates of 70-98% depending on content type, with 100% of models vulnerable to jailbreaks per UK AISI testing, though Anthropic's Const...Quality: 63/100, and controlled access for researchers.²³

Intentional Design

In February 2026, Goodfire announced a broader vision called "intentional design"—using interpretability to guide model training rather than merely analyzing models post-hoc.²⁴ The approach involves decomposing what a model learns from each datapoint into semantic components, then selectively applying or filtering these learning signals. Goodfire claims this method enabled them to cut hallucinations in half using interpretability-informed training.⁷ The approach has generated significant debate in the AI safety community (see Criticisms and Concerns).

Partnerships and Impact

Goodfire has established collaborations with research institutions and industry partners:

• Arc Institute: Early collaboration using Ember on Evo 2, a DNA foundation model, to uncover biological concepts and accelerate scientific discovery in genomics.²⁵
• Mayo Clinic: Announced in September 2025, focusing on genomic medicine, reverse-engineering genomics models for insights into disease mechanisms while emphasizing data privacy and bias reduction.²⁶
• Rakuten: Enhancing reliability for Rakuten AI, which serves over 44 million monthly users in Japan and 2 billion customers worldwide, focusing on preventing PII leakage using frontier interpretability techniques.²⁷
• Haize Labs: Joint work on feature steering for AI safety auditing, red-teaming, and identifying failure modes in generative models.²⁸
• Apollo Research: Using Goodfire tools for safety benchmarks and research.²⁹
• Microsoft: Partnership announced alongside the Series B funding round in February 2026.⁷

In November 2024, Goodfire powered the "Reprogramming AI Models" hackathon in partnership with Apart Research, with over 200 researchers across 15 countries prototyping safety applications like adversarial attack detection and "unlearning" harmful capabilities while preserving beneficial behaviors.³⁰

A notable scientific achievement came from Goodfire's partnership with Arc Institute: by reverse-engineering a biological foundation model, the team identified a novel class of Alzheimer's biomarkers—described as "the first major finding in the natural sciences obtained from reverse-engineering a foundation model."³¹

AI Safety and Alignment

Goodfire positions mechanistic interpretability as foundational to AI alignmentApproachAI AlignmentComprehensive review of AI alignment approaches finding current methods (RLHF, Constitutional AI) show 75%+ effectiveness on measurable safety metrics for existing systems but face critical scalabi...Quality: 91/100 and safety. The company's approach addresses several key challenges:

Alignment Without Side Effects

Traditional alignment methods like reinforcement learning from human feedback (RLHFCapabilityRLHFRLHF/Constitutional AI achieves 82-85% preference improvements and 40.8% adversarial attack reduction for current systems, but faces fundamental scalability limits: weak-to-strong supervision shows...Quality: 63/100) can produce unintended side effects, such as excessive refusal of benign requests or sycophantic behavior.³² Goodfire's feature steering offers an alternative by enabling precise, quantitative alignment of specific behaviors without degrading overall model performance.

Detecting Deception and Hidden Behaviors

One of the central challenges in AI safety is detecting deceptive or schemingRiskSchemingScheming—strategic AI deception during training—has transitioned from theoretical concern to observed behavior across all major frontier models (o1: 37% alignment faking, Claude: 14% harmful compli...Quality: 74/100 behavior in advanced AI systems. Goodfire's model diffing and auditing tools aim to identify rare, undesired behaviors—such as a model encouraging self-harm—that might emerge during training or deployment.³³ However, there is ongoing debate within the interpretability community about whether these techniques will scale to worst-case scenarios involving sophisticated deception.³⁴

Governance and Compliance

Interpretability tools like Ember may become essential for regulatory compliance. The EU AI ActPolicyEU AI ActComprehensive overview of the EU AI Act's risk-based regulatory framework, particularly its two-tier approach to foundation models that distinguishes between standard and systemic risk AI systems. ...Quality: 55/100 mandates transparency for high-risk AI systems, with fines up to €20 million for violations.³⁵ Goodfire's auditing and documentation capabilities could help organizations meet these requirements.

Fellowship Program

In October 2025, Goodfire announced a Fellowship Program for early- and mid-career researchers and engineers, matched with senior researchers to work on scientific discovery, interpretable models, and new interpretability methods.³⁶

Criticisms and Concerns

Despite significant progress, Goodfire's approach faces several challenges and critiques:

Unproven Effectiveness in Worst-Case Scenarios

There is substantial debate about whether mechanistic interpretability can reliably detect deception in advanced AI systems. Researcher Neel NandaPersonNeel NandaOverview of Neel Nanda's contributions to mechanistic interpretability, including the TransformerLens library and research on transformer circuits. Covers his educational content and role in making...Quality: 26/100 has noted that interpretability lacks "ground truth" for what concepts AI models actually use, making it difficult to validate interpretability claims.³⁷ Some researchers favor alternative methods like linear probes.

A concrete example of interpretability's limitations emerged with GPT-4o's "extreme sycophancyRiskSycophancySycophancy—AI systems agreeing with users over providing accurate information—affects 34-78% of interactions and represents an observable precursor to deceptive alignment. The page frames this as a...Quality: 65/100" issue, which was detected behaviorally rather than through mechanistic analysis—no circuit was discovered, no particular weights or activations were identified as responsible, and mechanistic interpretability provided no advance warning.³⁸

Competition from In-House Development

Leading AI labs like AnthropicOrganizationAnthropicComprehensive reference page on Anthropic covering financials (\$380B valuation, \$19B ARR), safety research (Constitutional AI, mechanistic interpretability, model welfare), governance (LTBT struc...Quality: 74/100, OpenAIOrganizationOpenAIComprehensive organizational profile of OpenAI documenting evolution from 2015 non-profit to Public Benefit Corporation, with detailed analysis of governance crisis, 2024-2025 ownership restructuri...Quality: 62/100, and DeepMindOrganizationGoogle DeepMindComprehensive overview of DeepMind's history, achievements (AlphaGo, AlphaFold with 200M+ protein structures), and 2023 merger with Google Brain. Documents racing dynamics with OpenAI and new Front...Quality: 37/100 have the resources to develop interpretability tools internally. Anthropic has publicly committed to investing significantly in reliably detecting AI model problems by 2027.³⁹ Additionally, open-source alternatives like Eleuther AI and InterpretML provide free interpretability frameworks, creating competitive pressure on commercial offerings.

Computational Cost and Accessibility

Goodfire's pricing model is heavily compute-bound, with strict rate limits that may limit accessibility.⁴⁰ The platform enforces a 50,000 token/minute global cap shared across all API methods. Advanced interpretability functions like AutoSteer and AutoConditional are limited to just 30 requests/minute, while simpler utilities allow 1,000 requests/minute. This hierarchy suggests exponentially higher computational costs for core interpretability features, potentially creating barriers for smaller organizations and academic researchers.

"The Most Forbidden Technique" Debate

In February 2026, Goodfire Chief Scientist Thomas McGrath published "Intentionally Designing the Future of AI," proposing the use of interpretability tools to shape model training by decomposing gradients into semantic components and selectively applying them on a per-datapoint basis.²⁴ This reignited a significant debate in the AI safety community about what Zvi Mowshowitz termed "The Most Forbidden Technique"—using interpretability techniques during training.⁴¹

Critics argue that optimizing against interpretability signals during training teaches models to obfuscate their internal representations, ultimately degrading the very tools needed to detect misalignment.⁴² As Mowshowitz summarized: if you train against technique [T], "you are training the AI to obfuscate its thinking, and defeat [T]." A LessWrong post specifically questioning Goodfire's approach noted that even structurally different methods like gradient decomposition may not escape this fundamental dynamic, since "selection pressure just goes into the parts that you don't know about or don't completely understand."⁴³

Defenders, including Neel NandaPersonNeel NandaOverview of Neel Nanda's contributions to mechanistic interpretability, including the TransformerLens library and research on transformer circuits. Covers his educational content and role in making...Quality: 26/100, argued that this research direction is both legitimate and potentially critical for safety. Nanda noted that multiple researchers including Anthropic Fellows have worked on interpretability-in-training, and that understanding its risks and benefits requires empirical research rather than blanket prohibition.⁴⁴ He acknowledged key uncertainties: "I don't know how well it will work, how much it will break interpretability tools, or which things are more or less dangerous."

The debate took a personal dimension when founding research scientist Liv Gorton departed Goodfire in early 2026, with AI safety advocate Holly Elmore publicly speculating the departure was "for reasons of conscience."⁴⁵ Gorton's departure—she had co-authored key research including the first sparse autoencoders on DeepSeek R1—highlighted the tensions between commercial applications of interpretability and its role as an independent safety tool.

Capabilities vs. Safety Framing

Third-hand reports indicate that Goodfire leadership has pitched interpretability work as "capabilities-enhancing" (improving AI performance) rather than primarily safety-focused when fundraising.⁴⁶ This framing raises questions about whether commercial incentives might prioritize performance improvements over safety applications—a tension common in dual-use AI research. The company's Series B announcement emphasized that interpretability-informed training had "cut hallucinations in half," framing the technology as a capabilities improvement.⁷

Funding and Business Model

Goodfire has raised approximately $209 million across three rounds:⁴⁷

The Series A round marked AnthropicOrganizationAnthropicComprehensive reference page on Anthropic covering financials (\$380B valuation, \$19B ARR), safety research (Constitutional AI, mechanistic interpretability, model welfare), governance (LTBT struc...Quality: 74/100's first direct investment in another company, signaling significant industry validation.⁴⁸ The Series B, closing less than a year later, valued Goodfire at $1.25 billion—making it one of the fastest AI startups to reach unicorn status.⁷

Goodfire operates on a usage-based pricing model, charging per million tokens processed (input + output), with pricing tiered by model size:⁴⁹

• Smaller models (e.g., Llama 3.1 8B): $0.35/million tokens
• Larger models (e.g., Llama 3.3 70B): $1.90/million tokens

The company is positioned to capture value in a rapidly growing market. The explainable AI market was valued at approximately $10 billion in 2025 and is projected to reach $25 billion by 2030.⁵⁰

Key Uncertainties

1. Scalability to superintelligent systems: Will mechanistic interpretability techniques that work on current models continue to provide safety guarantees as AI systems become more powerful and potentially deceptive?

2. Commercial viability: Can Goodfire compete with in-house interpretability teams at well-resourced AI labs and free open-source alternatives?

3. Capabilities vs. safety trade-offs: How will Goodfire navigate the tension between interpretability as a safety tool versus a capabilities enhancement that could accelerate AI development?

4. Ground truth validation: Without definitive ground truth about what concepts models represent, how can interpretability claims be rigorously validated?

5. Computational economics: Can the high computational costs of mechanistic interpretability be reduced sufficiently to enable widespread adoption?

6. Training on interpretability: Will using interpretability tools during model training ultimately compromise those tools' ability to serve as independent safety checks? This remains the central open question around Goodfire's "intentional design" approach.

Sources