Skip to main content

I think it's helpful to split up 'but how would AI kill us?' into 'how could AI kill us?' and 'why would AI want to kill us?' (and also importantly, 'if AI didn't want to kill us, how would it kill us anyway?')
in reply to Katja Grace

Katja Grace
Katja Grace
•
While people are thinking about if AI could get access to nukes or bioweapons or invent new destructive technology, in the back of their minds is 'yeah but that would be very sci-fi for them to want to kill us'
in reply to Katja Grace

Katja Grace
Katja Grace
•
And while people are thinking about what the new agents are trying to do, in the back of their minds is 'ok but even if they're a bit crazy, it's very hard to kill everyone!'

When Jacob Coxon quit Anthropic, his warning rocked the world. His colleague Evan’s “>10% chance” that AI would “kill all humans” might seem extreme, but it’s not unusual:

Most AI researchers think there’s a 10% chance of AI destroying or disempowering humanity, according to our survey, released today.
https://x.com/AIImpacts/status/2099581080223576403

in reply to Katja Grace

Katja Grace
Katja Grace
•
Our survey has many other interesting results, read the AI Impacts thread for more!
https://x.com/AIImpacts/status/2099581078210236850

Suggestion: end sicknesses with an official end-of-sickness ritual

There are lots of ways humans benefit from being limited, many connected to being trustworthy.
in reply to Katja Grace

Katja Grace
Katja Grace
•
What does the trajectory look like toward AI surpassing us at these 'capabilities' that involve a lack of capability?

I’ve really wanted more good short accounts of why AI poses an existential risk. Working on one myself has been one of those incredibly high priorities I keep putting off. Meanwhile Ben Bradford of NPR, has just made a podcast version of my case for AI x-risk that I am thrilled with!
in reply to Katja Grace

Katja Grace
Katja Grace
•
Bonus within the 39 minutes: what Hamza Chaudhry of FLI thinks we should do about it (who I was delighted to later meet as a consequence!)

Does anyone have a really satisfying way to store clothes? An account of my present objections to approximately all of the ones I know of, which I'm not recommending you read unless you are curious or want a break from talking about AI: worldspiritsockpuppet.substack…

I hate the way ads add a bunch of fake commercially incentivized voices to the conversation. AI is taking it further, which I dislike for these reasons and more!

I generally enjoy talking to journalists and am not very stressed or strategic about it. Am I doing it wrong? worldspiritsockpuppet.substack…

I've lately come to think of multi-person organizations as an overpowered negotiating tool

We should be talking seriously about coordination around not building dangerous AI.

Common wisdom says that it is incredibly hard to coordinate to not build more dangerous AI...but is that just naive?

AI as a Trojan Horse Race
in reply to Katja Grace

Katja Grace
Katja Grace
•
I think people struggle to differentiate between a situation that legitimately incentivizes racing and on the other hand people moving fast

"IdK, ClAuDe SeEmS ALiGneD to mE!"

(intended in a spirit of fun; my actual take is like, "I think we are missing the concepts we need in order to talk usefully about this")

I got my first DEXA scan in January of 2020. Between then and March 2020, I lost 2.5 bodyfat percentage points, but then I moved to the middle of nowhere with MIRI for COVID, and gained 12 percentage points between then and February 2023, which was my next scan date. My DEXA scan from today finally again shows a net decrease in body fat percentage compared with 2020! I'm still 25lbs heavier than I was then, but 20 of those are lean mass!

Freaked out a little bit today during my workout: I've been doing progressive overload on various dumbbell lifts for a couple months, and today one of my lifts was way harder than I expected. I hit failure ~ 4 reps earlier than Thursday. Only after failing even earlier on the second set did I realize that I had set the weight 20% too high. I felt dumb but also it's cool to get a little confirmation that I'm not just hitting "failure" via confirmation bias!

Has Scott Alexander already done a kabbalah-style bit about how MAGA is a multilingual recursive acronym? I.e. it's obviously cognate with PIE *meg- (meaning "great"; root of magnificence, magnitude, maharishi, major) and also the G stands for "great"

TIL: the thrift store near my house sells paperbacks for $1 and hard covers for $2. And if you don't mind the fact that they've organized the books by spine color, you can find excellent books. In fact I found one paperback that I ordered on Amazon yesterday for 10x the price.

is it just me or does the current generation of big models produce more typos than the previous generation? Here's one I noticed today from Opus 4.5 (non-drowsy -> non-drowning):

I've noticed at least two from GPT 5 and 5.1 as well, though I didn't think to screenshot them.

Today in "asking models about their preferences". I find this one a bit uncomfortable, tbh.

Today I am trying out wearing drawstring-based underwear. I really like it! Since I've been fat, elastic bands always fold or roll over themselves, unless I get a size that's so big the elastic doesn't even work to hold them up. With a drawstring I can choose the exactly right fit, which is typically much less pinchy/constricting than even a moderately well-fitting pair of underwear. The big downside is that using the bathroom takes an extra 30s or so for tying/untying the drawstring (though, if using a urinal, you can of course use the fly instead).

I originally bought them because they're ~the only literally 100% cellulose-based underwear you can buy (elastic bands are made of mostly spandex/elastane) and I was curious. I think I will buy at least a few more pairs of these, and maybe start wearing them by default.

I'm pretty excited about the prospect of making a wallet out of UHMWPE tape. It will be like the duct tape wallets of my youth except that its material properties will be superior to expensive wallets. Also it will probably be more expensive than expensive wallets if I factor in the cost of prototyping it (before even accounting for time spent).

"social health", as a concept kind of like "mental health", seems pretty interesting to me

They minted the last penny today! Thus ends the era of the US government handing out both required solid pieces of a voltaic cell in one (not very) convenient package.

Q: Is the AI Impacts survey methodology any good?

A: Yes—I explain in an FAQ:

aiimpacts.substack.com/p/faq-e…


FAQ: Expert Survey on Progress in AI methodology

Context
Katja Grace (AI Impacts blog)

(warning: posting this for archival purposes rather than because it's interesting at all)

I asked my mom what she remembers her grandparents eating; here's some of what she remembers, a lot of which they grew themselves:

  • Potatoes
  • Cabbage
  • Tomatoes
  • Beans and green beans
  • Liver and onions
  • Chicken
  • Eggs
  • Pork / ham as a treat
  • Blackberries and raspberries
  • Rhubarb
  • Lettuce
  • Homemade bread
  • Corn
  • Peas
in reply to Ben Weinstein-Raun

PAR
PAR
•
Adding on here: spinach, parsnips, carrots, celery, lots of onions with other things. cucumbers, and summer and winter squash, cottage cheese, and apples and pears, cherries in the late spring, grapefruit from Florida or California in the winter and oranges in the summer.
⇧