Skip to main content



in reply to Ben Weinstein-Raun

[watching blade runner because that phrase kept running through my head when I was in Nagoya, Tokyo, and in Japanese department stores, which are surprisingly similar to the blade runner setting]


Practicing on the onewheel today; first proper wipeout since... Maybe since I learned to ride a bike? Glad I was wearing wrist guards


Owain on AXRP!!!


Earlier this year, the paper "Emergent Misalignment" made the rounds on AI x-risk social media for seemingly showing LLMs generalizing from 'misaligned' training data of insecure code to acting comically evil in response to innocuous questions. In this episode, I chat with one of the authors of that paper, Owain Evans, about that research as well as other work he's done to understand the psychology of large language models.

Video
Transcript



Training for the backpacking part of my vacation seems to have dropped my resting heart rate by nearly 10 bpm over the last two months.

Unfortunately I probably won't do as much training for a while now that the trip is over, but maybe I can keep it low-ish for a while by eating healthier.



Japanese towns all have public loudspeaker systems, that they test daily by playing cute little melodies at certain times of day. This is both very pleasant and (imo) a mostly-better way to test these systems than the one we use in the bay area (Berkeley and SF both have warning systems that are tested via weekly/monthly sirens), since the test sounds are easily distinguishable from actual alerts even without looking at your watch.


in reply to Ben Weinstein-Raun

One weird thing about Fennec is that somehow I can't zoom into pictures on Twitter.

Ben Weinstein-Raun doesn't like this.

in reply to Ben Weinstein-Raun

Update: There are several minor-ish annoyances with LibreWolf:

  • (as with probably most non-big-boy browsers, I think), it doesn't seem to support Widevine, which means you can't use some streaming services, and others don't support HD video.
  • Google Maps zooming, which is normally smooth in most browsers, is jerky and a little annoying in LibreWolf
  • Some other webapps use maps libraries that also don't seem to work well (e.g. I can't see the DoorDash delivery map)
  • You can't easily add Google as a search engine; it seems to have a special case where it will refuse to add a custom search engine named "Google"; you have to call it something else (!). This seems like a very weird / user-hostile choice, but you can still add the search engine as long as you call it something else (e.g. "G" or "Google Search")

I'm going to keep using it, because I find these issues less annoying than upstream Firefox.



New AXRP episode with Lee Sharkey!


What's the next step forward in interpretability? In this episode, I chat with Lee Sharkey about his proposal for detecting computational mechanisms within neural networks: Attribution-based Parameter Decomposition, or APD for short.

Video
Transcript





🐸 Gentlemen, I am pleased to report that fifteen years after first hearing the song "Osaka Loop Line" by Discovery, I have successfully taken the Osaka Loop Line


I think the thing I really like about LLM-assisted coding is that it makes context switching easier.

I can be in "words mode" or "code mode", and switching between these takes time and effort. (There are more categories, but they don't change the fundamental point.)

In my job, I have to spend a lot of time in words mode, due to things like hiring and managing. Historically, this has meant that I only really get engineering work done when I have 2+ hour chunks to focus on it. But now I can often get work done in much shorter chunks, while still in words mode.

I would not like to spend all my time in words mode – I enjoy digging into the details – but it's really nice to have the option.



I think laptops should play (uniformly random) typing sounds while you type in your passwords; it's getting to be too easy to analyze the sounds and extract the contents.


Ok my new beliefs about blister prevention, after three weekends of backpacking for eight hours a day, and watching a bunch of YouTube videos:

  • blisters are caused by layers of skin delaminating, not "friction" / heat directly, though typically the delamination is due to static friction on the outer layer of skin, combined with wet skin. Dynamic friction is more likely to cause raw spots / wear straight through the skin.
  • popping them as soon as you find them is basically always the right call unless you plan to be able to avoid the activity that caused them for a week; otherwise they just keep growing as you continue to do the activity
  • blister donuts and moleskin work okay as long as you can keep them in place somehow, but they don't stick well on their own
  • leukotape, very very widely recommended, is worse than useless because the adhesive seeps through the tape and makes your skin stick to your socks even more tightly than it was before.
  • toe socks are pretty good
  • KT tape is very good
  • Vaseline/similar is pretty good as long as you can get it to stay in the right spots


Anyone have luck getting LLMs to write tests without mocks? The tests I want are often just 1-2 lines of code, but anything I get from Claude or Gemini ends up being 20-30 lines long, despite requests for conciseness, saying no mocks are needed, and seeing using real resources is ok.

(I use LLMs a lot for other stuff, but tests seem to be particularly bad.)

in reply to Satvik

I think Sonnet 4 is not *that* much smarter than 3.7 but it should be significantly more steerable and less likely to insert silly mocks
in reply to Daniel Ziegler

Sonnet 4 is tremendously more effective for my use cases, probably because I use a niche programming language (Julia). Two weeks ago I would have said LLMs make me ~10% more productive, now it looks closer to +100%.

And I'm not even committing LLM-generated code – I just use it to iterate and test on designs, then delete the code and implement from scratch manually.



so, I'm going to japan in a few weeks, to do this pilgrimage backpacking trip with a friend.

I'm very out of shape compared to the difficulty of the route (alltrails.com/explore/map/map-… : 4 days; average of 10 miles and 3200ft elevation gain)

So my plan is to train as much as I can between now and then. I've figured out this practice loop, starting from my house, that I'm going to try to work up to doing on both the 17th and 18th: alltrails.com/explore/map/kuma…

It takes a pretty cool path over the hills and down to the reservoir.

Anyone want to join for any of this? As you might guess I expect to be very slow and take lots of breaks (today I did only about half of this loop; 6-ish miles; and it took me like 4 hours)

in reply to Ben Weinstein-Raun

Yo that's really cool. I wish I was in the bay to practice with you.

Consider wearing a backpack on the trek if you aren't.



I tentatively think that rain jackets would work better if they were more like coats of feathers.

Usually rain jackets are either (a) totally waterproof, in which case you sweat and it condenses on the inside of the jacket, or (b) "breathable", in which case they fairly quickly "wet out" and the sweat actually still condenses on the inside.

Feathers work partly like a "breathable" rain jacket, in that they're porous and hydrophobic on the outermost layer, but they're also anisotropic: rain jacket material is the same in all directions, while feathers work kinda like roof shingles: The water rolls off, but there's space for air to pass underneath the feathers. This is fine because rain mostly comes from above, and anyway I bet you can make fairly complicated labyrinths of air passageways such that even splashing water is very unlikely to make it through the jacket.

in reply to Ben Weinstein-Raun

Which side do you think Gore-Tex jackets are on? I find them pretty good at both staying dry and being breathable (though not perfect at either)
in reply to Daniel Ziegler

I think they seem pretty breathable until they get wet, at which point I seem to see condensation on the inside


I think the thing that makes (much) Latin poetry most unrewarding for me is that I'm not at the stage where I can appreciate the rhythm and the meaning at the same time; I have to focus on one or the other, and in isolation neither is so great.
in reply to Daniel Filan

Of course presumably this is a skill issue to be rectified in time.


Went on a photo walk today, mostly around Berkeley, and tested out some new camera settings. Most of the photos didn't turn out as well as I hoped, but I got a few that I like after a little postprocessing in Lightroom.

flickr.com/photos/spiritfox/54…

flickr.com/photos/spiritfox/54…

flickr.com/photos/spiritfox/54…



Okay, what the heck is up with people doing deceptive things to prevent "panic"? What are the actual dangers of "panic?" I was just watching this new Veritasium video about an engineering firm discovering that their already-built Manhattan skyscraper has a 1 to 5% chance of collapsing per year, and deciding that they're not going to tell anyone about it while they spent months fixing it. The head engineer explicitly says in a recorded lecture that this was justified because people "don't need to be terrorized". Is that even plausible?

in reply to Ben Weinstein-Raun

Oof, yeah. It does sound like they had an evacuation plan if a hurricane came, but it still seems pretty indefensible to lie about the situation.
in reply to Ben Weinstein-Raun

I had similar thoughts watching this. It's not right to allow people to make the uninformed choice to enter this building.


C'mon Fabulae Syrae, this is not a very good explanation of what a harundō is.


I tried out making an unboxing video:

I'm pleased with how it turned out, though the subject matter is objectively not very interesting.



in reply to Daniel Filan

Interestingly actual Ovid seems easier to understand than the verses the textbook author wrote himself, but maybe it's because he selected the bits of Ovid to be easy to understand vs wrote a whole story in his own verse.
in reply to Daniel Filan

OK the next part was hard again so I think I'm not just becoming stronger.


ok ok the ghibli thing is very cute

i am annoyed that my local community is such that sharing this adorable picture of me + child would get people mad at me for using AI

in reply to Kevin Gibbons

openAI finally added this capability to the API, which is why I am only now playing with it, because I am much happier paying 30¢ per image I actually generate than paying $20/mo regardless of how much I use

also if you want me to generate images for you I am very happy to do that









Quest complete: Eat the traditional hotdog from each of Denmark, Norway and Sweden




in reply to Ben Weinstein-Raun

I've heard this from my mother rather than from Knife Guy YouTube. My anecdotal experience of kitchen work is that sharpening my knives does make a huge difference in a) the amount of force I use for basic tasks and b) the likelihood the knife will slip while doing them so I do feel the advice holds at least for regular household tasks and moderate vs. no knife care. I hone my knives every few uses for this reason but don't sharpen them often so they're probably not "knife guy" sharp? They're currently easy to work with and so getting them sharper would probably have diminishing returns.
This entry was edited (2 months ago)
in reply to Jen Blight

Yeah I think I buy it for fixed-blade knives, especially kitchen knives that are used for chopping and repeated fast slicing, but for pocket knives they're just so fiddly and used for such non-repetitive tasks that it seems pretty different.
in reply to Ben Weinstein-Raun

It's also very not good to use a much sharper knife than you're used to. That's part of how I cut off 2mm of my thumbtip several years ago.


I've been getting really into pocket knives this week, and especially learning about knife steels. The biggest surprise has been that one of the several aspects of Atlas Shrugged that caused me to lose suspension of disbelief, has become much more believable in retrospect:

When I read it I felt like this whole part about the "guy invents a new metal that's just straightforwardly better than existing metals, and names it after himself" was just too farfetched.

But it turns out that actually this is just a thing that can happen. This guy Larrin Thomas basically straight-up did this with a knife steel alloy in 2021. The alloy is notably better than others for the purposes of pocket knives in almost every respect. Like, in any single dimension there are steels that do better, but this alloy is like the Hawaii of the knife steel Pareto frontier. He didn't name it after himself but I think he could have called it "Larrin Metal" if he had wanted to. He actually called it "Magnacut".

in reply to Ben Weinstein-Raun

I guess my intuition is that alloys would be "smooth" in their properties. You add more chromium and certain properties increase. Maybe they stop increasing or start reversing after a while, but it's not hard to find the optional points for each property over time.

With that intuition it seems surprising that it's hard to find new alloys that haven't already been found.

in reply to JP Addison



Have been wearing minimalist/"barefoot" sandals for the last couple days, and it feels somehow unhinged to say this, but I think they're making me feel noticeably happier?

Like, it reminds me of the thing where anosmia is linked to depression. It's like I regained a nontrivial part of my sense of "what's going on in the world".





AXRP Jason Gross


How do we figure out whether interpretability is doing its job? One way is to see if it helps us prove things about models that we care about knowing. In this episode, I speak with Jason Gross about his agenda to benchmark interpretability in this way, and his exploration of the intersection of proofs and modern machine learning.

Transcript
YouTube



Finally actually followed through on my long-standing intention to donate blood! Went pretty smoothly, and I'm ~500mL lighter. Next month hopefully I'll try to donate plasma.
in reply to Ben Weinstein-Raun

Proud of ya! Please make sure you understand the whole plasma donation process before you go forward with that. As I have said elsewhere, my experience with plasma donation was not good . . . so of course, I am concerned for you.


On the way home from the coffee shop I encountered a bright blue bird as I walked past a bush, about half a foot from me, which stared at me for a second, quietly squawked, and flew away.

Then soon afterward, in a different bush, I encountered an ooze (pictured, made of bubbly foam) trying to stealth.

I'm concerned that someone has changed the genre of my RPG, in a worrying direction. Will be avoiding taverns.

in reply to Ben Weinstein-Raun

Indigo Bunting? Western Bluebird? Scrub Jay? (Of course I don't understand the humor . . . but I bet if I did I would laugh!)




things it's the anniversary of this weekend:
- covid lockdowns
- the final alphago match
- the end of HPMOR
- the killing of Julius Caesar


don't like this

in reply to kip

Me too! Sounds like a really upsetting issue, and I hope it turns out to be easily/clearly resolvable in spite of initial indications.