A plan to pause growth of new high-risk AI systems while expanding use of current models! This seems very promising. https://x.com/DavidSKrueger/status/2108628712816365623
We could pause by ending production of advanced compute, and that might be best. But we can also make compute that runs specific models, yet can't easily be used to train new ones. https://x.com/KatjaGrace/status/2108659391209890083
I think it's helpful to split up 'but how would AI kill us?' into 'how could AI kill us?' and 'why would AI want to kill us?' (and also importantly, 'if AI didn't want to kill us, how would it kill us anyway?')
While people are thinking about if AI could get access to nukes or bioweapons or invent new destructive technology, in the back of their minds is 'yeah but that would be very sci-fi for them to want to kill us'
And while people are thinking about what the new agents are trying to do, in the back of their minds is 'ok but even if they're a bit crazy, it's very hard to kill everyone!'
When Jacob Coxon quit Anthropic, his warning rocked the world. His colleague Evan’s “>10% chance” that AI would “kill all humans” might seem extreme, but it’s not unusual:
Most AI researchers think there’s a 10% chance of AI destroying or disempowering humanity, according to our survey, released today. https://x.com/AIImpacts/status/2099581080223576403
I’m an independent researcher who runs the largest and longest-running peer-reviewed survey of AI researchers. These researchers are experts who publish papers at the top machine learning and artificial intelligence venues.
We’ve been conducting these surveys since 2016, so we can see how expert opinion has moved over the years.
Two trends: 1) Researchers' 'timelines' to human-level AI have shrunk, fast. 2) Most researchers have seen major future risks of extinction-level catastrophes for a long time, and their concerns are lately trending upward
This loosely mirrors what I hear from talking to people in the field: 1) AI is advancing much faster than researchers expected 2) Catastrophic risks have been looming for a while, and people are increasingly worried
Also interesting: many people worry that China will race ahead in AI if the US slows down to avoid human extinction. (It’s unclear to me why China would not also want to slow down in this account).
But Asian researchers in our survey were actually more worried! https://x.com/AIImpacts/status/2099581081901211878
e.g. last time I recall playing One Night Ultimate Werewolf I just admitted I was a werewolf because I was lying so ineffectually and flusteredly. AI is already much better at lying than me, but this actually gives me an advantage in being trustworthy.
I’ve really wanted more good short accounts of why AI poses an existential risk. Working on one myself has been one of those incredibly high priorities I keep putting off. Meanwhile Ben Bradford of NPR, has just made a podcast version of my case for AI x-risk that I am thrilled with!
Does anyone have a really satisfying way to store clothes? An account of my present objections to approximately all of the ones I know of, which I'm not recommending you read unless you are curious or want a break from talking about AI: worldspiritsockpuppet.substack…
I hate the way ads add a bunch of fake commercially incentivized voices to the conversation. AI is taking it further, which I dislike for these reasons and more!
I got my first DEXA scan in January of 2020. Between then and March 2020, I lost 2.5 bodyfat percentage points, but then I moved to the middle of nowhere with MIRI for COVID, and gained 12 percentage points between then and February 2023, which was my next scan date. My DEXA scan from today finally again shows a net decrease in body fat percentage compared with 2020! I'm still 25lbs heavier than I was then, but 20 of those are lean mass!
Freaked out a little bit today during my workout: I've been doing progressive overload on various dumbbell lifts for a couple months, and today one of my lifts was way harder than I expected. I hit failure ~ 4 reps earlier than Thursday. Only after failing even earlier on the second set did I realize that I had set the weight 20% too high. I felt dumb but also it's cool to get a little confirmation that I'm not just hitting "failure" via confirmation bias!
Has Scott Alexander already done a kabbalah-style bit about how MAGA is a multilingual recursive acronym? I.e. it's obviously cognate with PIE *meg- (meaning "great"; root of magnificence, magnitude, maharishi, major) and also the G stands for "great"
TIL: the thrift store near my house sells paperbacks for $1 and hard covers for $2. And if you don't mind the fact that they've organized the books by spine color, you can find excellent books. In fact I found one paperback that I ordered on Amazon yesterday for 10x the price.
Katja Grace
in reply to Katja Grace • •https://x.com/KatjaGrace/status/2108659391209890083