AXRP Episode 36 - Adam Shai and Paul Riechers on Computational Mechanics
I made an episode about computational mechanics and I think it's cool and you should watch (or listen or read as the case may be)!
Blurb I wrote:
Sometimes, people talk about transformers as having "world models" as a result of being trained to predict text data on the internet. But what does this even mean? In this episode, I talk with Adam Shai and Paul Riechers about their work applying computational mechanics, a sub-field of physics studying how to predict random processes, to neural networks.
ToC if that's interesting:
- 0:00:42 - What computational mechanics is
- 0:29:49 - Computational mechanics vs other approaches
- 0:36:16 - What world models are
- 0:48:41 - Fractals
- 0:57:43 - How the fractals are formed
- 1:09:55 - Scaling computational mechanics for transformers
- 1:21:52 - How Adam and Paul found computational mechanics
- 1:36:16 - Computational mechanics for AI safety
- 1:46:05 - Following Adam and Paul's research
Ben Weinstein-Raun likes this.
Daniel Filan
in reply to Daniel Filan • •Ben Millwood
in reply to Daniel Filan • •Daniel Filan
in reply to Ben Millwood • •Ben Weinstein-Raun
in reply to Daniel Filan • •Daniel Filan
in reply to Ben Weinstein-Raun • •[Links](google.com)
Ben Weinstein-Raun likes this.
Ben Millwood
in reply to Daniel Filan • •Ben Weinstein-Raun likes this.
Daniel Filan
in reply to Ben Millwood • •Ben Weinstein-Raun likes this.
Ben Weinstein-Raun
in reply to Daniel Filan • •Daniel Filan likes this.
Daniel Filan
in reply to Daniel Filan • •- I recorded this episode 3 months ago to the day - one of my longest publishing lags ever, in large part because of being busy with MATS.
- Some of the stuff we discuss got worked on during MATS.
- IDK I think this episode is pretty cool.
Ben Weinstein-Raun
in reply to Daniel Filan • •