What Are ARC's Current Research Projects?

The Alignment Research Center’s Theory team is starting a new hiring round for researchers with a theoretical background. Please apply here . 

 Update January 2024 : we have paused hiring and expect to reopen in the second half of 2024. We are open to expressions of interest but do not plan to process them until that time. 

 What is ARC’s Theory team?

 The Alignment Research Center (ARC) is a non-profit whose mission is to align future machine learning systems with human interests. The high-level agenda of the Theory team (not to be confused with the Evals team) is described by the report on Eliciting Latent Knowledge (ELK): roughly speaking, we’re trying to design ML training objectives that incentivize systems to honestly report their internal beliefs.

 For the last year or so, we’ve mostly been focused on an approach to ELK based on formalizing a kind of heuristic reasoning that could be used to analyze neural network behavior, as laid out in our paper on Formalizing the presumption of independence . Our research has reached a stage where we’re coming up against concrete problems in mathematics and theoretical computer science, and so we’re particularly excited about hiring researchers with relevant background, regardless of whether they have worked on AI alignment before . See below for further discussion of ARC’s current theoretical research directions.

 Who is ARC looking to hire?

 Compared to our last hiring round , we have more of a need for people with a strong theoretical background (in math, physics or computer science, for example), but we remain open to anyone who is excited about getting involved in AI alignment, even if they do not have an existing research record.

 Ultimately, we are excited to hire people who could contribute to our research agenda. The best way to figure out whether you might be able to contribute is to take a look at some of our recent research problems and directions:

 Some of our research problems are purely mathematical, such as these matrix completion problems – although note that these are unusually difficult, self-contained and well-posed (making them more appropriate for prizes).
 Some of our other research is more informal, as described in some of our recent blog posts such as Finding gliders in the game of life .
 A lot of our research occupies a middle ground between fully-formalized problems and more informal questions, such as fixing the problems with cumulant propagation described in Appendix D of Formalizing the presumption of independence .
 What is working on ARC’s Theory team like?

 ARC’s Theory team is led by Paul Christiano and currently has 2 other permanent team members, Mark Xu and Jacob Hilton , alongside a varying number of temporary team members (recently anywhere from 0–3).

 Most of the time, team members work on research problems independently, with frequent check-ins with their research advisor (e.g., twice weekly). The problems described above give a rough indication of th

... (truncated, 7 KB total)