Skip to main content

AXRP Episode 36 - Adam Shai and Paul Riechers on Computational Mechanics

I made an episode about computational mechanics and I think it's cool and you should watch (or listen or read as the case may be)!

YouTube link
Transcript

Blurb I wrote:

Sometimes, people talk about transformers as having "world models" as a result of being trained to predict text data on the internet. But what does this even mean? In this episode, I talk with Adam Shai and Paul Riechers about their work applying computational mechanics, a sub-field of physics studying how to predict random processes, to neural networks.


ToC if that's interesting:
- 0:00:42 - What computational mechanics is
- 0:29:49 - Computational mechanics vs other approaches
- 0:36:16 - What world models are
- 0:48:41 - Fractals
- 0:57:43 - How the fractals are formed
- 1:09:55 - Scaling computational mechanics for transformers
- 1:21:52 - How Adam and Paul found computational mechanics
- 1:36:16 - Computational mechanics for AI safety
- 1:46:05 - Following Adam and Paul's research

in reply to Daniel Filan

Daniel Filan
Daniel Filan
•
@Ben Weinstein-Raun Using Markdown didn't seem to work when I made this post - I feel like it used to tho? And I have it set as a thing that should work.
@Ben Weinstein-Raun
in reply to Daniel Filan

Ben Weinstein-Raun
Ben Weinstein-Raun
•
The markdown processor is definitely less robust than the BBcode processing; the latter is the "preferred" format, and iiuc the markdown ultimately ends up converted to BBcode. I think it's especially likely to get confused when markdown is mixed with BBcode; not sure if there much I could do to fix short of submitting a PR against the plugin
in reply to Daniel Filan

Daniel Filan
Daniel Filan
•
Fun facts:
- I recorded this episode 3 months ago to the day - one of my longest publishing lags ever, in large part because of being busy with MATS.
- Some of the stuff we discuss got worked on during MATS.
- IDK I think this episode is pretty cool.
in reply to Daniel Filan

Ben Weinstein-Raun
Ben Weinstein-Raun
•
Watched the first 30 minutes before I saw this post - will return having seen this!
⇧