New AXRP with Jaime Sevilla!
Epoch AI is the premier organization that tracks the trajectory of AI - how much compute is used, the role of algorithmic improvements, the growth in data used, and when the above trends might hit an end. In this episode, I speak with the director of Epoch AI, Jaime Sevilla, about how compute, data, and algorithmic improvements are impacting AI, and whether continuing to scale can get us AGI.
William Morris (24 March 1834 – 3 October 1896)
Acanthus original design
(Taken from @ marysia_cc on twitter, https://x.com/marysia_cc/status/1841750032535765407)
https://x.com/Philippe4269/status/1841365949720474004
A thing I didn't realize would be a consequence of making a niche podcast is how much podcast spam email I get. Mostly of the form "bring my client as a guest on your podcast", sometimes of the form "use our social medium" etc. Examples that I've received in the last ~week at the end of this post.
IDK maybe this is unsurprising given that podcasts have an email attached to them. Interestingly it does seem to happen more now that my podcast is more prominent than it was ~2 years ago.
AXRP Episode 36 - Adam Shai and Paul Riechers on Computational Mechanics
I made an episode about computational mechanics and I think it's cool and you should watch (or listen or read as the case may be)!
Blurb I wrote:
Sometimes, people talk about transformers as having "world models" as a result of being trained to predict text data on the internet. But what does this even mean? In this episode, I talk with Adam Shai and Paul Riechers about their work applying computational mechanics, a sub-field of physics studying how to predict random processes, to neural networks.
ToC if that's interesting:
- 0:00:42 - What computational mechanics is
- 0:29:49 - Computational mechanics vs other approaches
- 0:36:16 - What world models are
- 0:48:41 - Fractals
- 0:57:43 - How the fractals are formed
- 1:09:55 - Scaling computational mechanics for transformers
- 1:21:52 - How Adam and Paul found computational mechanics
- 1:36:16 - Computational mechanics for AI safety
- 1:46:05 - Following Adam and Paul's research
Overrated:
- Marcus Aurelius
- "Saint" Augustine of Hippo (are the haters Eastern Orthodox or atheist?)
- Cleopatra
Underrated
- Justinian I (who dislikes that guy?)
- Cicero
Doctrine of the Mean trutherism
Translate things
Sometimes people are like "noooooooo I can't just translate Tian as Heaven or Dao as Way or junzi as gentleman, the terms have slightly different connotations and are used by this writer in a sort of distinct way". Nah. Writers in English use common terms idiosyncratically all the time. If you're worried we'll misunderstand, put an asterisk the first place the terms are used and refer us to a glossary. In the meantime, the reader will have a better understanding of what the terms mean.
In general I think we should translate more things so that you can understand what people are saying. E.g. "Hamas" is just short for "the Islamic Resistance Movement", call them that or the IRM if you want to save space. Most English speakers don't speak foreign languages, and it's important to be comprehensible!
