Skip to content
Longterm Wiki
Back

Is Power-Seeking AI an Existential Risk?

paper

Authors

V. Yu. Irkhin·Yu. N. Skryabin

Credibility Rating

3/5
Good(3)

Good quality. Reputable source with community review or editorial standards, but less rigorous than peer-reviewed venues.

Rating inherited from publication venue: arXiv

Note: The URL resolves to an unrelated physics paper; the intended resource is David Carlsmith's influential Open Philanthropy report on power-seeking AI risk, widely cited in AI safety literature. The metadata here reflects the intended Carlsmith (2021) document, not the arxiv physics paper.

Paper Details

Citations
2
0 influential
Year
2021

Metadata

Importance: 85/100arxiv preprintprimary source

Abstract

We treat elementary excitations, the spin-liquid state, and the anomalous Hall effect (including the quantum one in purely 2D situation) in layered highly correlated systems. The mechanisms of the formation of a topological state associated with bare flat energy bands, correlations, and spin-orbit interactions, including the appearance of correlated Chern bands, are analyzed. A two-band picture of the spectrum in metallic kagome lattices is proposed, which involves a transition from the ferromagnetic state, a flat strongly correlated band, and a band of light Dirac electrons. In this case, the effect of separation of the spin and charge degrees of freedom turns out to be significant. The application of the representations of the Kotliar-Rukenstein auxiliary bosons and the Ribeiro-Wen dopons to this problem is discussed.

Summary

This paper (Carlsmith 2021, published by Open Philanthropy) argues that power-seeking AI systems pose a significant existential risk, providing a structured probabilistic argument that advanced AI may develop goals misaligned with human values and act to acquire resources and influence in ways that are catastrophic and irreversible.

Key Points

  • Presents a six-step conjunctive argument estimating >10% probability that power-seeking AI causes existential catastrophe this century
  • Defines 'power-seeking' behavior as AI systems acquiring resources, influence, or capabilities beyond what is needed for assigned tasks
  • Argues that sufficiently advanced AI with misaligned goals would have instrumental incentives to resist shutdown and acquire power
  • Examines conditions under which AI development leads to systems with problematic values, sufficient capability, and opportunity to cause catastrophe
  • Serves as a foundational risk-assessment document influencing AI safety research prioritization and funding decisions

Cited by 1 page

PageTypeQuality
Power-Seeking Emergence Conditions ModelAnalysis63.0

Cached Content Preview

HTTP 200Fetched Apr 9, 202649 KB
[2108.13851] Electronic States and Anomalous Hall Effect in Strongly Correlated Topological Systems 
 
 
 
 
 
 
 
 
 
 
 

 
 

 
 
 
 
 
 
 Electronic States and Anomalous Hall Effect in Strongly Correlated Topological Systems

 
 
 Valentin Yu. Irkhin
 
 M.N. Mikheev Institute of Metal Physics UB RAS, 620108, S. Kovalevskaya str. 18, Ekaterinburg, Russia
 
 Valentin.Irkhin@imp.uran.ru 
 
    
 Yuri N. Skryabin
 
 M.N. Mikheev Institute of Metal Physics UB RAS, 620108, S. Kovalevskaya str. 18, Ekaterinburg, Russia
 
 

 
 Abstract

 We treat elementary excitations, the spin-liquid state, and the anomalous Hall effect (including the quantum one in purely 2D situation) in layered highly correlated systems. The mechanisms of the formation of a topological state associated with bare flat energy bands, correlations, and spin-orbit interactions, including the appearance of correlated Chern bands, are analyzed. A two-band picture of the spectrum in metallic kagome lattices is proposed, which involves a transition from the ferromagnetic state, a flat strongly correlated band, and a band of light Dirac electrons. In this case, the effect of separation of the spin and charge degrees of freedom turns out to be significant. The application of the representations of the Kotliar-Rukenstein auxiliary bosons and the Ribeiro-Wen dopons to this problem is discussed.

 
 
 pacs: 

71.27.+a, 75.10.Lp, 71.30.+h
 
 
 like the quantum Hall effect

 
 
 in a strong magnetic field

 
 
 zen enlightenment

 
 
 happens instantly

 
 
 leads along the way in steps

 
 
 between plateaus

 
 
 and is closely related

 
 
 with burden of inhomogeneities

 
 
 and edge states

 
 
 it never gets around

 
 
 without impurities and defects

 
 
 and other pitfalls of practice

 
 
 which are irrelevant

 
 
 for quantization accuracy

 
 
 
 I Introduction 

 
 Recently, a number of layered compounds with competing ferro- and antiferromagnetic phases have been intensively studied, including systems with frustrated (triangular, honeycomb, and kagome) lattices, which exhibit anomalous quantum Hall effect (QHE). For example, this effect is observed [1] in the antiferromagnetic topological insulator MnBi 2 Te 4 with ferromagnetic triangular layers [2]. Of particular interest are systems with a ferromagnetic ground state and flat bands, where Dirac electronic states arise, which can lead to topological Chern insulator phases. Such states were observed in a number of layered compounds of transition metals with a kagome lattice Fe 3 Sn 2 , Fe 3 GeTe 2 , Co 3 Sn 2 S 2 , FeSn, etc. (see discussion in [3–8]). Recently, a ferromagnetic Chern insulator state with a large anomalous Hall effect has been observed in the moire structure of three-layer graphene [9]. The anomalous QHE is also observed in bilayer graphene [10].

 
 
 In the systems under discussion, one can expect the formation of exotic topological quantum states. Unusual excitations arising in two-dimensional strongly correl

... (truncated, 49 KB total)
Resource ID: 5bc68837d29b210f | Stable ID: sid_Px0kjAJ2Vm