When Jacob Coxon quit Anthropic, his warning rocked the world. His colleague Evan’s “>10% chance” that AI would “kill all humans” might seem extreme, but it’s not unusual:
Most AI researchers think there’s a 10% chance of AI destroying or disempowering humanity, according to our survey, released today.
https://x.com/AIImpacts/status/2099581080223576403
Katja Grace
in reply to Katja Grace • •Katja Grace
in reply to Katja Grace • •We’ve been conducting these surveys since 2016, so we can see how expert opinion has moved over the years.
Two trends:
1) Researchers' 'timelines' to human-level AI have shrunk, fast.
2) Most researchers have seen major future risks of extinction-level catastrophes for a long time, and their concerns are lately trending upward
Katja Grace
in reply to Katja Grace • •1) AI is advancing much faster than researchers expected
2) Catastrophic risks have been looming for a while, and people are increasingly worried
Katja Grace
in reply to Katja Grace • •Katja Grace
in reply to Katja Grace • •Also interesting: many people worry that China will race ahead in AI if the US slows down to avoid human extinction. (It’s unclear to me why China would not also want to slow down in this account).
But Asian researchers in our survey were actually more worried!
https://x.com/AIImpacts/status/2099581081901211878
Katja Grace
in reply to Katja Grace • •https://x.com/AIImpacts/status/2099581078210236850
Katja Grace
in reply to Katja Grace • •