MATS ML Alignment Theory Scholars program

📋Page Status

Page Type:ContentStyle Guide →Standard knowledge base article

Quality:60 (Good)

Importance:65 (Useful)

Last edited:2026-02-01 (today)

Words:2.7k

Structure:

📊 1📈 0🔗 6📚 72•24%Score: 11/15

LLM Summary:MATS is a well-documented 12-week fellowship program that has successfully trained 213 AI safety researchers with strong career outcomes (80% in alignment work) and research impact (160+ publications, 8000+ citations). The program provides comprehensive support ($27k per scholar) and has produced notable alumni who founded organizations like Apollo Research and joined major AI labs.

Issues (1):

Links15 links could use <R> components

Quick Assessment

Aspect	Rating	Notes
Program Scale	High	98 scholars and 57 mentors in most recent cohort (MATS 8.0, Summer 2025)¹
Research Output	Strong	160+ publications, 8,000+ citations, h-index of 40 over 4 years²
Career Impact	Very High	80% of alumni work in AI alignment; placements at Anthropic, OpenAI, DeepMind³
Funding per Scholar	$27k	$15k stipend + $12k compute resources, plus housing and meals⁴
Selectivity	Very Competitive	≈15% acceptance rate; 40+ mentors with independent selection⁵

Overview

The ML Alignment & Theory Scholars (MATS) Program is a 12-week educational and research fellowship designed to develop talented researchers in AI alignment, governance, and security through intensive mentorship and independent research.⁶ Founded in 2021 and initially run as SERI MATS under the Stanford Existential Risks Initiative, the program became independent by Summer 2023 and now operates in-person cohorts in Berkeley, California and London, United Kingdom.⁷

MATS pairs scholars with leading researchers in AI safety for approximately 1-2 hours of mentorship per week, supplemented by seminars, workshops, guest lectures, and dedicated research manager support.⁸ The program provides comprehensive support including a $15,000 living stipend, $12,000 in compute resources, private housing, catered meals, and office space.⁹ Scholars develop independent research projects that culminate in presentations at a Scholar Symposium, with selected fellows invited to continue for 6-12 month extensions.

Since its first cohort in early 2022, MATS has supported 213 scholars and 47 mentors across its first five seasonal programs, expanding to 98 scholars and 57 mentors by MATS 8.0 in Summer 2025.¹⁰ The program has generated over 160 research publications with more than 8,000 citations, advancing agendas in mechanistic interpretability, sparse feature analysis, activation engineering, and AI safety evaluation.¹¹ Alumni have founded new AI safety organizations like Apollo Research and secured positions at major AI labs, with 80% remaining in alignment-related work.¹²

History

Founding and Early Development

MATS originated as SERI MATS, an initiative under the Strategic Research Institute (SERI) focused on AI safety research training, launching its first programs in early 2022.¹³ The initial program structure included a 4-week online upskilling phase (10 hours per week), a 2-week research sprint, and an 8-week intensive in-person program in Berkeley.¹⁴ Early mentors included Alex Gray, Beth Barnes, Evan Hubinger, John Wentworth, Leo Gao, and Stuart Armstrong.¹⁵

The program’s core mission from inception was to train talented individuals for AI alignment research by addressing risks from unaligned AI through mentorship, training, logistics, and community access.¹⁶ By Summer 2023, the program had evolved into an independent organization while maintaining its Berkeley and London hubs.¹⁷

Program Evolution and Growth

Over its first four years, MATS iterated significantly on its structure and curriculum:

Summer 2022: The first cohort produced notable outcomes, including scholars like Johannes Treutlein working under Evan Hubinger, who co-authored papers on predictive models that were later published at the UAI 2023 conference.¹⁸

Summer 2023 (4th Iteration): This cohort expanded to 60 scholars and 15 mentors, with 461 applicants (15% acceptance rate for the Training Phase).¹⁹ The program introduced the Scholar Research Plan (SRP) requiring a threat model, theory of change, and SMART plan, and implemented distinct phases: Training (Alignment 201), Research (Berkeley), and Extension (London/Berkeley).

Winter 2023-24 (5th Iteration): Further growth to 63 scholars and 20 mentors, with a significant curriculum change replacing Alignment 201 with custom curricula due to feedback.²⁰ This included Neel Nanda’s remote mechanistic interpretability curriculum (November 20-December 22) and AI Safety Strategy Discussions.

MATS 8.0 (Summer 2025): The program reached 98 scholars and 57 mentors, concluding with a symposium on August 22, 2025 featuring 10 spotlight talks and a poster session.²¹

By May 2024, MATS had supported 213 scholars and 47 mentors across five seasonal programs, presenting insights on talent selection and development at the TAIS 2024 conference.²²

Program Structure and Support

Core Components

MATS operates as a 12-week in-person fellowship with several key elements:

Mentorship: Scholars receive approximately 1-2 hours per week of one-on-one mentorship from established researchers via Slack or direct communication.²³ Each mentor conducts their own selection process, with some using work tasks and others conducting interviews focused on ML experience, research proposals, and conceptual alignment questions rather than behavioral assessments.²⁴

Research Development: Scholars develop a Research Plan approximately one month into the program, outlining their threat model, theory of change, and specific deliverables.²⁵ Dedicated research managers provide support for scoping projects, maintaining progress, and removing obstacles throughout the fellowship.²⁶

Educational Programming: The program includes seminars and workshops 2-3 times per week, featuring speakers from organizations like Redwood Research, FAR AI, OpenAI, CHAI, and GovAI.²⁷ Past speakers have included Buck Shlegeris, Adam Gleave, William Saunders, Andrew Critch, Lennart Heim, and Ajeya Cotra.

Research Tracks: MATS offers multiple specialization areas including technical governance, empirical research, policy & strategy, theory, and compute governance.²⁸

Financial and Logistical Support

The program provides comprehensive material support valued at approximately $35,000 per scholar:²⁹

$15,000 stipend for living expenses (provided by AI Safety Support)³⁰
$12,000 compute budget for experiments and evaluations³¹
Private housing for the full program duration in Berkeley or London³²
Office space access and catered meals³³
Travel reimbursement where applicable

Extension Opportunities

Selected scholars may continue for an additional 6-12 months through extension programs in London, Berkeley, Boston, or Washington D.C., with MATS arranging funding to cover monthly stipends, compute resources, housing, and office rent.³⁴ Historically, approximately 70% of scholars have been accepted for extensions based on research plans and mentor endorsements.³⁵

Research Impact and Outcomes

Publications and Citations

Over four years, MATS has produced significant research output, with alumni generating over 160 publications that have received more than 8,000 citations, yielding an organizational h-index of 40.³⁶ Notable publications include:

Steering Llama 2 via Contrastive Activation Addition (Outstanding Paper Award at ACL 2024)³⁷
Conditioning Predictive Models: Risks and Strategies (published at UAI 2023)³⁸
Incentivizing Honest Performative Predictions with Proper Scoring Rules (UAI 2023)
Neural Networks Learn Statistics of Increasing Complexity
Copy Suppression, Inverse Scaling, The Reasons That Agents Act

In a survey of alumni from the first four programs (46% response rate), 78% reported their key publication “possibly” or “probably” would not have happened without MATS, with 10% accelerated by more than 6 months and 14% accelerated by 1-6 months.³⁹

Research Agendas Developed

MATS scholars have advanced numerous technical agendas in AI safety:

Sparse auto-encoders for AI interpretability⁴⁰
Activation and representation engineering
Emergent misalignment detection
Inoculation prompting techniques
Developmental interpretability
Computational mechanics applications
Glitch token analysis
Situational awareness evaluations
Gradient routing methods
Externalized reasoning oversight
Formalizing natural abstractions

These research directions span mechanistic interpretability, sparse feature analysis, and studies of latent representations in AI systems.⁴¹

Career Outcomes

MATS has achieved strong career placement results for alumni:

Employment: 49% of surveyed alumni reported working or interning on AI alignment or control, with 29% conducting independent alignment research.⁴² Among earlier cohorts, 39% were hired by research organizations post-MATS, with 50% indicating MATS made them “much more likely” to be hired.⁴³ An additional 22% pursued Master’s or PhD programs.

Organizational Placements: Alumni have joined nearly every major AI safety initiative, including Anthropic, OpenAI, DeepMind, CHAI, and Redwood Research.⁴⁴ Notable examples include:

Nina (Summer 2023, mentored by Evan Hubinger): Joined Anthropic as a research scientist; won ACL 2024 Outstanding Paper Award; later mentored SPAR and MATS cohorts⁴⁵
Marius Hobbhahn (Winter 2022/23, mentored by Evan Hubinger): Founded and became CEO of Apollo Research, a London-based technical alignment organization focused on scheming evaluations and AI control⁴⁶
Johannes Treutlein (Summer 2022, mentored by Evan Hubinger): Pursued PhD at CHAI; joined Anthropic in 2024 for alignment stress-testing⁴⁷

New Organizations: Alumni have founded new AI safety initiatives including Apollo Research, Cadenza Labs, PRISM Eval, and have organized conferences on singular learning theory and developmental interpretability.⁴⁸

Skill Development: 49% of alumni reported MATS increased their research or technical skills, while 38% gained legible career capital.⁴⁹

Key People

Leadership

Ryan Kidd serves as Co-Executive Director of MATS and Co-Founder of the London Initiative for Safe AI (LISA).⁵⁰ He was a scholar in MATS’s first iteration (which had only 5 scholars total) and has since become a Manifund Regrantor and advisor to organizations including Halcyon Futures, Catalyze Impact, AI Safety ANZ, and Pivotal Research.

Christian Smith serves as Co-Executive Director and Co-Founder of LISA.⁵¹ He brings a background in particle physics and pedagogy from Stanford University, having conducted research at CERN and organized educational programs like the Uncommon Sense Seminar.

Laura Vaughan, a Thiel Fellow (2017) who studied physics at the University of Waterloo, brings experience in ML model dataset creation and training, management, entrepreneurship, full-stack software engineering, and biomedical research.⁵² She co-founded a stem cell cryogenics startup before joining MATS.

Notable Mentors

MATS mentors come from leading organizations including Anthropic, Google DeepMind, Redwood Research, OpenAI, MIRI, ARC (Alignment Research Center), CHAI, CAIS, and the Centre on Long-Term Risk.⁵³ Selected examples include:

Marius Hobbhahn: CEO and Director of Apollo Research focusing on evaluations for scheming and control; PhD in Bayesian ML; formerly worked on AI forecasting at Epoch; was a MATS Winter 2022/23 scholar under Evan Hubinger⁵⁴
Sam Bowman: Leads AI alignment and welfare research at Anthropic with focus on evaluation; Associate Professor at NYU (on leave); has studied neural network language models since 2012⁵⁵
Joe Benton: Member of Anthropic’s Alignment Science team working on scalable oversight, control, chain-of-thought monitoring, and evaluations⁵⁶
Arthur Conmy: Research Engineer at Google DeepMind on Language Model Interpretability with Neel Nanda; previously conducted influential work on automating interpretability at Redwood Research⁵⁷
Evan Hubinger: Provided mentorship for early SERI MATS trials and multiple cohorts; formerly at MIRI, now at Anthropic⁵⁸
Neel Nanda: From Google DeepMind; created custom mechanistic interpretability curriculum for MATS including sessions on sparse autoencoders and superposition toy models⁵⁹

Funding

MATS operates as a non-profit fellowship program sustained through grants and donations rather than generating revenue.⁶⁰ The program does not coordinate funding directly but relies on partner organizations:

Primary Funding Sources:

AI Safety Support: Provides the $15,000 stipend for each fellow completing the full program (prorated for partial participation)⁶¹
MATS-arranged funding: Covers extension program costs including monthly stipends, compute, housing, and office rent for 6-12 month extensions⁶²
Open Philanthropy: Provided grants to support the early SERI MATS trial program under Evan Hubinger⁶³
Other supporters (2024): Foresight Institute, Survival and Flourishing Fund, Long-Term Future Fund, Craig Falls, and several donors via Manifund⁶⁴

Per-Scholar Investment: The total cost per scholar is approximately $35,000 for the full program, based on recent cohorts of 60 scholars and 15 mentors.⁶⁵ This includes the $15k stipend, $12k compute resources, and costs for housing, meals, office space, and program administration.

Historical Funding: In the 2022 SERI MATS program, scholars received $6,000 after the training and research sprint phase, $16,000 at program completion, plus ongoing discretionary funding, with all accommodation, office, and event expenses covered.⁶⁶

Criticisms and Concerns

While MATS has achieved strong outcomes, program organizers and alumni have identified several concerns and limitations:

Field Growth Risks

Program organizers acknowledge concerns that MATS’s appeal—particularly access to scaling lab mentors—could attract aspiring AI researchers not primarily focused on existential risk reduction, potentially introducing viewpoints that dilute the field’s epistemic rigor.⁶⁷ While organizers maintain high selection pressure to prioritize x-risk-motivated scholars, they recognize this tension between growth and field quality as they plan broader advertising.

Mentorship Dependency and Deference

Critics note that scholars might overly defer to mentors, failing to critically analyze assumptions and reducing independent thinking or new viewpoints in the field.⁶⁸ This concern exists in tension with the opposite problem: insufficient mentorship could lead to excessive peer reliance among inexperienced researchers. MATS rarely accepts scholars without mentors, viewing mentorship as essential for knowledge transfer, which limits scalability and raises barriers since mentors have high entry requirements and capacity constraints.⁶⁹

Opportunity Costs for Participants

Alumni impact analysis reveals mostly positive views but highlights specific challenges:⁷⁰

Time allocation: Non-research tasks like writing proposals and preparing talks divert effort from core research
Career uncertainty: One alumnus noted MATS pushed them into technical research with less than 70% confidence it was positive; another preferred their prior ML engineering role for deeper technical challenges
Relationship strain: Some scholars reported impacts on prior commitments, such as strained relationships with PhD supervisors when pausing unrelated work
Emotional fit: Some felt out of place in the AI safety community or experienced slowed involvement
Grant stress: Short-term funding uncertainty led some to doubt their counterfactual impact when applying to AI safety roles

Selection Challenges

With approximately 15% acceptance rates and 40+ mentors conducting independent selection, even proficient researchers and engineers with AI safety experience frequently receive rejections due to mentor capacity limits rather than candidate quality.⁷¹ Application processes involve mentor-specific interviews on ML experience, research proposals, conceptual questions, and experiments, with rejections common even after strong interviews.

Alumni feedback indicates that scholars with prior research experience often rate MATS superior to alternatives like independent research or “hub-hopping,” though some note they would have preferred later participation after building more ML skills through programs like ARENA.⁷²

Key Uncertainties

Scalability: Can MATS maintain research quality while expanding beyond current mentor capacity constraints, given the program’s emphasis on apprenticeship-style learning?
Counterfactual Impact: What proportion of alumni would have entered AI safety careers through alternative pathways, and how much does MATS accelerate versus redirect talent?
Optimal Program Length: Is the 12-week duration optimal for research skill development, or would longer or shorter programs better serve different scholar populations?
Field Dilution Risks: As MATS expands and advertises more broadly, how can the program maintain epistemic standards while increasing accessibility?
Extension Selection: With ~70% of scholars historically advancing to extensions, what criteria best predict long-term research impact?
Mentor-Scholar Matching: How can the program optimize matching between mentors and scholars to balance deference concerns against knowledge transfer benefits?