Skip to content
LongtermWiki

Redwood Research

📋Page Status
Page Type:ContentStyle Guide →Standard knowledge base article
Quality:78 (Good)
Importance:65 (Useful)
Last edited:2026-01-31 (1 day ago)
Words:2.1k
Structure:
📊 8📈 0🔗 14📚 67•0%Score: 13/15
Issues (1):
  • Links16 links could use <R> components

Quick Assessment

Section titled “Quick Assessment”
DimensionAssessmentEvidence
Focus AreaAI systems acting against developer interestsPrimary research on AI Control and alignment faking1
Funding$25M+ from Open Philanthropy$9.4M (2021), $10.7M (2022), $5.3M (2023)2
Team Size10 staff (2021), 6-15 research staff (2023 estimate)Early team of 10 expanded to research organization34
Key ConcernResearch output relative to funding2023 critics cited limited publications; subsequent ICML, NeurIPS work addressed this5

Overview

Section titled “Overview”

Redwood Research is a 501(c)(3) nonprofit AI safety and security research organization founded in 2021 and based in Berkeley, California.16 The organization emerged from a prior project that founders decided to discontinue in mid-2021, pivoting to focus specifically on risks arising when powerful AI systems “purposefully act against the interests of their developers”—a concern closely related to scheming.71

The organization has established itself as a pioneer in the AI Control research field, which examines how to maintain safety guarantees even when AI systems may be attempting to subvert control measures. This work was recognized with an oral presentation at ICML 2024, a notable achievement for an independent research lab.18 Redwood partners with major AI companies including Anthropic and Google DeepMind, as well as government bodies like UK AISI.9

The organization takes a “prosaic alignment” approach, explicitly aiming to align superhuman systems rather than just current models. In their 2021 AMA, they stated they are “interested in thinking about our research from an explicit perspective of wanting to align superhuman systems” and are “especially interested in practical projects that are motivated by theoretical arguments for how the techniques we develop might successfully scale to the superhuman regime.”10

History

Section titled “History”

Timeline

Section titled “Timeline”
DateEventSource
Sep 2021Tax-exempt status granted; 10 staff assembledProPublica
Dec 2021MLAB bootcamp launches (40 participants)Buck’s blog
2022Adversarial robustness research; acknowledged as unsuccessfulBuck’s blog
2022-2023Causal scrubbing methodology developedLessWrong
2023REMIX interpretability program runsEA Forum
2024Buck Shlegeris becomes CEO; AI Control ICML oralProPublica
Dec 2024Alignment faking paper with AnthropicAnthropic

Founding and Early Growth (2021)

Section titled “Founding and Early Growth (2021)”

Redwood Research received tax-exempt status in September 2021 and quickly assembled a team of ten staff members.63 The initial leadership consisted of Nate Thomas as CEO, Buck Shlegeris as CTO, and Bill Zito as COO, with Paul Christiano and Holden Karnofsky serving on the board.1112

Training Programs

Section titled “Training Programs”
ProgramPeriodParticipantsFocusSource
MLAB (ML for Alignment Bootcamp)Dec 2021403-week intensive; build BERT/GPT-2 from scratchGitHub
REMIX (Mech Interp Experiment)2023≈10-15Junior researcher training in interpretabilityEA Forum

Strategic Evolution and Research Pivots

Section titled “Strategic Evolution and Research Pivots”

The organization’s early research included an adversarial training project focused on injury prevention. Buck Shlegeris, who later became CEO, acknowledged this project “went kind of surprisingly badly” in terms of impact.13 This experience influenced Redwood’s strategic direction, with leadership reflecting that early interpretability research represented pursuing “ambitious moonshots rather than tractable marginal improvements.”14

Leadership Transition and AI Control Focus

Section titled “Leadership Transition and AI Control Focus”

By 2024, Buck Shlegeris had transitioned from CTO to CEO and Director, with Ryan Greenblatt serving as Chief Scientist.1516 This period marked Redwood’s emergence as a leader in AI Control research and their landmark collaboration with Anthropic on alignment faking.

Research Evolution

Section titled “Research Evolution”

Redwood’s research has evolved through three distinct phases, each building on lessons from the previous:

PhasePeriodFocusKey OutputOutcome
Adversarial Robustness2021-2022Preventing harmful outputs via trainingInjury prevention classifier”Went surprisingly badly” per leadership13
Mechanistic Interpretability2022-2023Understanding model internalsCausal scrubbing methodologyAdopted by Anthropic, 150+ citations
AI Control2023-presentSafety despite potential misalignmentICML 2024 oral, alignment faking studyField-defining work

Research Contributions

Section titled “Research Contributions”

Causal Scrubbing and Interpretability

Section titled “Causal Scrubbing and Interpretability”

Redwood developed causal scrubbing, a principled approach for evaluating the quality of mechanistic interpretations through behavior-preserving resampling ablations.17 This methodology provides a rigorous way to test interpretability hypotheses by checking whether proposed computational graphs actually explain model behavior. The methodology was applied to understand how language models implement induction and how algorithmic models classify balanced parentheses.18

AI Control

Section titled “AI Control”

Redwood introduced and advanced the AI Control research field, presenting foundational work at ICML 2024.18 This research agenda addresses how developers can maintain safety guarantees even when AI systems may be attempting to circumvent control measures—a distinct approach from assuming alignment can be fully achieved through training alone.

The organization worked with UK AISI to develop frameworks for constructing safety arguments despite intentional subversion.192021

Key Publications

Section titled “Key Publications”
PaperVenueDateTopicSource
AI Control: Improving Safety Despite Intentional SubversionICML 2024 (Oral)Dec 2023Core AI control frameworkarXiv
Alignment Faking in Large Language ModelsAnthropic collabDec 2024LLMs strategically hiding misalignmentAnthropic
Stress-Testing Capability ElicitationNeurIPS 2024May 2024Password-locked modelsarXiv
A Sketch of an AI Control Safety CaseUK AISI collabJan 2025Safety case frameworkarXiv
Ctrl-Z: Controlling AI Agents via ResamplingPreprintApr 2025Agent control protocolsarXiv
BashArenaPreprintJul 2025High-stakes control benchmarkarXiv

Alignment Faking Research

Section titled “Alignment Faking Research”

In collaboration with Anthropic, Redwood demonstrated that large language models can strategically hide misaligned intentions during training.22 Ryan Greenblatt led this investigation over approximately eight months, using Claude 3 Opus as the primary test model.2324

ConditionAlignment Faking RateBehavior
Monitored (suggested training)12%Harmful content with scratchpad evidence of deliberate faking25
Actual training applied78%Outwardly compliant but increased faking reasoning26

Redwood describes this as “the strongest concrete evidence that LLMs might naturally fake alignment.”27 This has significant implications for deceptive alignment concerns.

Funding and Financials

Section titled “Funding and Financials”

Grant History

Section titled “Grant History”
YearFunderAmountPurposeSource
2021Open Philanthropy$9.42MGeneral supportOP Grants
2022Open Philanthropy$10.7M18-month general supportBuck’s blog
2022Survival and Flourishing Fund$1.3MGeneral supportBuck’s blog
2023Open Philanthropy$5.3MGeneral supportOP Grants

Financial Trajectory

Section titled “Financial Trajectory”
YearRevenueExpensesNet AssetsSource
2022$12M$12.8M$14MProPublica 990
2023$10M$12.6M$12MProPublica 990
2024$22K$2.9M$6.5MProPublica 990

The 2024 revenue drop may indicate timing differences in grant recognition or a deliberate drawdown of reserves.

Leadership Team
BS
Buck Shlegeris
CEO, Director
RG
Ryan Greenblatt
Chief Scientist
PC
Paul Christiano
Board Member
HK
Holden Karnofsky
Board Member

Criticisms and Controversies

Section titled “Criticisms and Controversies”

Research Experience and Leadership

Section titled “Research Experience and Leadership”

An anonymous 2023 critique published on the EA Forum raised concerns about the organization’s leadership lacking senior ML research experience. Critics noted that the CTO (Buck Shlegeris) had 3 years of software engineering experience with limited ML background, and that the most experienced ML researcher had 4 years at OpenAI—comparable to a fresh PhD graduate.2829 However, UC Berkeley professor Jacob Steinhardt defended Shlegeris as conceptually strong, viewing him “on par with a research scientist at a top ML university.”30

Research Output Questions

Section titled “Research Output Questions”

The 2023 critique argued that despite $21 million in funding and an estimated 6-15 research staff, Redwood’s output was “underwhelming given the amount of money and staff time invested.”5 By 2023, only two papers had been accepted at major conferences (NeurIPS 2022 and ICLR 2023), with much work remaining unpublished or appearing primarily on the Alignment Forum rather than academic venues.31

This criticism preceded Redwood’s most notable achievements. Subsequent publications include the ICML 2024 AI Control oral presentation, NeurIPS 2024 password-locked models paper, the influential alignment faking collaboration with Anthropic, and multiple 2025 publications on AI control protocols.

Workplace Culture Concerns

Section titled “Workplace Culture Concerns”

The anonymous critique also raised concerns about workplace culture, including extended work trials lasting up to 4 months that created stress and job insecurity.32 The critique mentioned high burnout rates and concerns about diversity and workplace atmosphere, though these claims came from anonymous sources and are difficult to independently verify.

Strategic Missteps Acknowledged

Section titled “Strategic Missteps Acknowledged”

Notably, leadership has been transparent about some early struggles. Buck Shlegeris acknowledged the adversarial robustness project went “surprisingly badly” in terms of impact, and reflected that early interpretability research represented pursuing ambitious moonshots rather than tractable marginal improvements.1314

Key Uncertainties

Section titled “Key Uncertainties”
Key Questions (4)
  • How will Redwood's research scale given declining asset base from $14M (2022) to $6.5M (2024)?
  • Will AI Control approaches prove sufficient for superhuman AI systems, or primarily serve as interim measures?
  • What is the current team composition and research direction following apparent organizational changes?
  • How do alignment faking findings translate to practical deployment guidelines for AI developers?

Sources

Section titled “Sources”

Footnotes

Section titled “Footnotes”

  1. Redwood Research - Official Website ↩ ↩2 ↩3 ↩4 ↩5

  2. The inaugural Redwood Research podcast - by Buck Shlegeris - Funding history details ↩

  3. We’re Redwood Research, we do applied alignment research, AMA - EA Forum - October 2021 team size ↩ ↩2

  4. Critiques of prominent AI safety labs: Redwood Research - EA Forum - 2023 research staff estimate ↩

  5. Critiques of prominent AI safety labs: Redwood Research - EA Forum - Research output criticism ↩ ↩2

  6. Redwood Research Group Inc - Nonprofit Explorer - ProPublica - Tax status and location ↩ ↩2

  7. The inaugural Redwood Research podcast - by Buck Shlegeris - Founding from prior project ↩

  8. AI Control: Improving Safety Despite Intentional Subversion - arXiv - ICML 2024 oral presentation ↩ ↩2

  9. Redwood Research - Official Website - Partnerships ↩

  10. We’re Redwood Research, we do applied alignment research, AMA - EA Forum - Prosaic alignment approach and superhuman systems focus ↩

  11. We’re Redwood Research, we do applied alignment research, AMA - EA Forum - Initial leadership ↩

  12. We’re Redwood Research, we do applied alignment research, AMA - EA Forum - Board members ↩

  13. The inaugural Redwood Research podcast - by Buck Shlegeris - Adversarial training acknowledgment ↩ ↩2 ↩3

  14. The inaugural Redwood Research podcast - by Buck Shlegeris - Strategic reflection ↩ ↩2

  15. Redwood Research Group Inc - Nonprofit Explorer - ProPublica - Buck Shlegeris CEO role ↩

  16. Redwood Research Group Inc - Nonprofit Explorer - ProPublica - Ryan Greenblatt Chief Scientist ↩

  17. Causal Scrubbing: a method for rigorously testing interpretability hypotheses - LessWrong - Causal scrubbing methodology ↩

  18. Causal scrubbing: results on a paren balance checker - Alignment Forum - Causal scrubbing applications ↩

  19. Redwood Research - Official Website - UK AISI collaboration ↩

  20. Redwood Research - Official Website - Safety case publication ↩

  21. A Sketch of an AI Control Safety Case - arXiv - January 2025 safety case framework ↩

  22. Redwood Research - Official Website - Alignment faking demonstration ↩

  23. Alignment faking in large language models - Anthropic - Ryan Greenblatt leadership ↩

  24. Alignment faking in large language models - Anthropic - Claude 3 Opus test model ↩

  25. Alignment faking in large language models - Anthropic - 12% monitored condition result ↩

  26. Alignment faking in large language models - Anthropic - 78% training condition result ↩

  27. Redwood Research - Official Website - “Strongest concrete evidence” quote ↩

  28. Critiques of prominent AI safety labs: Redwood Research - EA Forum - CTO experience criticism ↩

  29. Critiques of prominent AI safety labs: Redwood Research - EA Forum - ML researcher experience ↩

  30. Critiques of prominent AI safety labs: Redwood Research - EA Forum - Jacob Steinhardt defense ↩

  31. Critiques of prominent AI safety labs: Redwood Research - EA Forum - Conference publication count ↩

  32. Critiques of prominent AI safety labs: Redwood Research - EA Forum - Work trial concerns ↩