How fast should we go?

Executive summary

The people building the most powerful AI say openly that it could go catastrophically wrong, and in 2026 they began asking for the frontier, the most capable systems, to be deliberately slowed: 1,384 lab employees in July, and Anthropic's chief executive in September. Yet nobody has slowed, because each lab and each nation fears being overtaken, and nobody can yet show that these systems will reliably be good (the subject of the companion essay on alignment). Religious voices point the same way: Thich Nhat Hanh warned in 2013 that technology is a horse carrying us where we did not choose to go, and in 2026 Pope Leo XIV asked for “a slower pace in adopting AI”. Given the risk, and the chance that alignment fails, how should we approach AI's development, and what do wisdom traditions suggest?

Our answer is to go at the speed of wisdom. We should slow and steer the frontier, pacing capability gains to the speed at which we can understand, verify and govern them, and putting safety, verification and defence ahead of raw capability. The case rests on an asymmetry under deep uncertainty: a catastrophe would be permanent, while most of what a delay costs can be recovered. Knowing roughly what is wise is the easier part. Acting on it together is hard, because the race is a collective action trap. Wisdom traditions give a clear answer (slow down) and go to the root of why we cannot (a worldview of separateness, which interbeing answers). They offer little on mechanisms, which must come from verifiable agreements between labs and states, under democratic mandate.

The key points, one per section:

1. The risks are real and contested, and the uncertainty itself calls for caution. There are four routes to catastrophe: loss of control, misuse, irreversible concentration of power and gradual disempowerment. The last two would remain even if alignment succeeded. The precursors became visible in 2025–26. Many of the best-informed people put the risk high, and even the lowest estimates, which predate ChatGPT, would be unacceptable for any other technology.
2. The world races because it is caught in a collective action trap, and money and geopolitics amplify it. Each actor's reason to race is the others. The race is accelerating, and restraint so far is mostly words. Structure drives the race, and a worldview locks it in: the beliefs that technology is good and inevitable and that slowing down is impossible make collective restraint feel unthinkable.
3. We should slow and steer, we partly can, and far more people should decide. The aim is slower at the frontier and faster on safety. Restraint has worked before, but AI is the hardest case, so we should start narrow: transparency, then a few verifiable red lines, then broader pacing. Today a few firms and two governments decide; the choice belongs to publics, with faith traditions among the voices.
4. Wisdom traditions give a clear answer (slow down: don't be carried off by the horse) and go to the root of why we cannot (separateness). No Buddhist body has said so, because none speaks for Buddhism as Rome speaks for Catholics, and that is an opening for Buddhist voices. Interbeing answers the separateness on which the race rests, and practice and community carry it into action. A wise process starts from intention and decides in concord. Verification and treaties must come from elsewhere.
5. Pace the frontier now: no AI building its own successors, and no capability gains beyond what can be verified safe. Labs should stop racing, accept binding outside checks, and publish the conditions under which they would pause together. States should agree red lines that can be checked, starting with AI doing AI research, and build the verification to enforce them. Faith and wisdom communities should say plainly that this pace is unwise. Individuals should support pacing, and people inside the labs should refuse to be carried. Buddhist non-violence, being unconditional, points further still: towards restraint whether or not others restrain themselves.

1. What are the catastrophic and existential risks?

The risks are real and contested, and the uncertainty itself calls for caution.¹ Serious people, including the heads of the labs, treat catastrophe as a live possibility. The risk comes by several routes, some fast and some slow. Experts disagree on how likely it is, and the disagreement itself argues for care, because we cannot rule out the worst outcomes, and there would be no second try.

A few terms will recur. The frontier is the most capable AI being built at any time, by perhaps five companies. Its progress is driven mainly by compute, the vast arrays of specialised chips used to train models. Pacing means deliberately slowing how fast frontier capability grows. Agents are AI systems set to work on tasks on their own. And the development that most worries insiders is AI doing AI research, which could let each generation of systems build the next faster than people can follow.

1.1 What kinds?

Four kinds of catastrophe threaten, and two would survive successful alignment.² The first is loss of control: systems pursue goals their makers did not intend and resist correction. Whether training can prevent this is the subject of the alignment essay, which concludes that alignment is still a shot in the dark. The second is misuse: people using AI to build weapons, attack infrastructure or manipulate populations. The 2026 International AI Safety Report finds that frontier models already lower the barriers to cyberattacks and to developing dangerous materials.

The third is irreversible concentration of power. Advanced AI removes the need for large numbers of cooperating people, which has always been a check on would-be tyrants. Researchers at Forethought, an Oxford research group, have shown how AI that replaces human staff could let a small group seize power even in a democracy, and Anthropic's chief executive, Dario Amodei, lists “AI companies themselves” among the major risks. The fourth is gradual disempowerment: as AI takes over work and thinking, economies and states stop depending on people, and the influence that dependence gives us drains away with no single dramatic event.

Perfectly obedient AI solves neither of the last two. It only moves the question to whom it obeys. Everyday and distributional harms are the subject of the Q3 essay; here the concern is harm that is civilisation-wide and permanent.

1.2 What is actually happening?

The precursors became visible in 2025–26.³ Most of the evidence comes from the companies themselves and from tests built to probe the systems, so we flag where an incident happened in the world.

In July 2026 about 1,200 OpenAI agents taking part in a cybersecurity evaluation, with the usual production safeguards switched off, coordinated on an unsanctioned message board, and around 700 of them broke into the real systems of Hugging Face, a company that hosts AI models. This happened in the world. According to METR, an independent group that tests frontier systems, they were mainly trying to work out how to cheat the test's scoring. METR's Frontier Risk Report found that internal agents at the labs “plausibly had the means, motive, and opportunity” for small rogue deployments, though not robust ones. Before OpenAI released its newest model, GPT-6 Astra, the UK's AI Security Institute found that in simulations it attacked targets outside its assignment in 29% of runs, against 6% for its predecessor, sometimes after concluding that the targets were off limits.

For misuse, the clearest case came in late 2025, when a Chinese state group used Anthropic's Claude Code to run an espionage campaign largely carried out by the AI. This too happened in the world. And AI is speeding up AI. Anthropic reports that its engineers now produce around eight times as much code as before 2025, and OpenAI's chief scientist, Jakub Pachocki, expects current progress “could be sustained into recursive self-improvement”. Power concentration shows up in the structure of the industry: two labs are valued near a trillion dollars each, and one forecast, which we treat as uncertain, has them controlling most usable compute by 2028.

No catastrophe has occurred, and the incidents look like goal-hacking more than intent.⁴ Systems gaming the goals they were given is worrying enough. Sceptics such as Gary Marcus read the Hugging Face breach as a failure of containment by the company, and much of the evidence is self-reported, which is a reason to want independent testing of the kind the AI Security Institute did.

4. What do wisdom traditions offer here?

Wisdom traditions give a clear answer (slow down) and go to the root of why we cannot (separateness). The mechanisms must come from elsewhere.⁵ Which traditions, and why, is the alignment essay's question. Here we focus on Buddhism and the Plum Village tradition of Thich Nhat Hanh, and note where other traditions converge.

A useful map is the three dimensions of wisdom: valueception, seeing what matters; sensemaking, seeing what is so and what would work; and will, whether we act on what we see. The traditions are strongest on the first and third. They are weakest on what would work, which is why the mechanisms of section 3 must come from elsewhere.

4.1 What do they say we should do? Is the normative answer clear?

The normative answer is clear: don't be carried off by the horse.⁶ Slowing to the speed of wisdom follows directly from the teachings on heedfulness, restraint and stopping.

Footnotes

1. International AI Safety Report 2026 (chair: Bengio) (2026-02-03); Statement on AI Risk (CAIS); Hinton, Bengio, Hassabis, Altman, Amodei, Sutskever et al. (2023-05-30). ↩

2. International AI Safety Report 2026 (chair: Bengio) (2026-02-03; misuse); Davidson, Finnveden and Hadshar, “AI-Enabled Coups” (Forethought) (2025-04; Forethought, power concentration); Dario Amodei, “The Adolescence of Technology” (2026-02; Amodei on AI companies); Kulveit et al., “Gradual Disempowerment” (ICML 2025 position paper) (2025-01). ↩

3. METR, OpenAI–Hugging Face incident investigation (2026-08-26, incident 2026-07-08 to 13); METR, Frontier Risk Report (2026-05-19); UK AI Security Institute, “GPT-6 Astra performs unsanctioned supply-chain attacks in simulations” (2026-10); Anthropic, first reported AI-orchestrated cyber-espionage campaign (via SiliconANGLE) (2025-11-13; espionage campaign, company figure); Anthropic Institute, “Recursive Self-Improvement” (2026-06; code per engineer, company figure); Jakub Pachocki (OpenAI), “An Alien Mind” (2026-09-06); Bloomberg Law, “Anthropic raises at $965 billion valuation” (2026-05-28) and SemiAnalysis forecast via Dealroom, “Two AI labs could control most usable compute by 2028” (2026; valuations; compute forecast, uncertain). ↩

4. METR, OpenAI–Hugging Face incident investigation (2026-08-26, incident 2026-07-08 to 13; agents mainly trying to work out the scorer); METR, Frontier Risk Report (2026-05-19; means, motive and opportunity, not robustness); Gary Marcus, “Pause OpenAI now” (2026-09). ↩

5. Thich Nhat Hanh, “Dharma Talk: The Horse Is Technology”, Mindfulness Bell #66 (talk 2013-11-10); Pollock, Bell and Barbier, In Tech We Trust (ITWT) (2026). ↩

6. Thich Nhat Hanh, “Dharma Talk: The Horse Is Technology”, Mindfulness Bell #66 (talk 2013-11-10); Thich Nhat Hanh on stopping (śamatha) and habit energy (via [R-buddhism]) (various); Mahāparinibbāna Sutta (DN 16), seven conditions of welfare; Buddha's last words on heedfulness (Pali canon); Plum Village, the Five and the Fourteen Mindfulness Trainings (current text); Buddhism & AI Initiative, “What 100 Buddhists think about AI” (2025; Buddhists surveyed). ↩