I think I have dyscalculia. It was never a noticeable problem in school, and I've never gotten a diagnosis, but it does seem like maybe a reasonable way to model my problems. (Maybe?? I'm unsure how many of my problems count as "dyscalculia" considering I feel pretty not-central-example.)

1. When I got extensive cognitive testing, one test required me to count letters/numbers in an audio recording. I found it overwhelmingly difficult and scored 2nd percentile

2. By far, the hardest part of cooking is remembering more than 1/2 of a recipe-instruction at a time. (e.g. Recipe calls for 3 Tbsp sugar, so I grab the tablespoon, and then I need to re-check how many Tbsp I have to measure out, and then I dump 2 Tbsp into the bowl, and then sometimes I lose count at this point and forget how many Tbsp I've put in so far.)

3. Counting reps during exercise is absurdly difficult for me

4. Learning step patterns in dance also seems way more difficult to me than it is for anyone else -- even though I've been doing a form of partner dancing for years. (I chose one without precise step patterns due to my troubles, and it never got easier.) Relatedly: It feels near-impossible to grok rhythmic patterns in music

5. When someone spells something out loud, my brain reacts to this almost as if I am hearing meaningless gibberish. I have to listen really closely and carefully to actually get a spelling out of this. (I know this isn't about numbers but it feels related.) When I spell things out loud for other people, I feel like I can't keep track of what I'm saying, and I'm super-not-confident I actually said it right.

6. I embarrassingly forget a lot of numbers that I feel like I am expected to remember (e.g. approximate populations of countries, numbers relevant to CS when I was a software engineer, rough timelines of earth/humanity, birthdays)

7. CURRENT PROBLEM: Ugh I wanna knit but even the most basic patterns seem so hard for me!! Even when I avoid patterns that require counting! I'm doing a pattern where the odd rows are very slightly different from the even rows. Apparently I cannot keep track of whether I am on an odd row or an even row. Might sew a temporary sign onto the "right side" of the work.

Cross-posted from my blog, also a quick take on LessWrong

Epistemic status:
- I listened to the Dwarkesh episode with Gwern and started attempting to think about life, the universe, and everything
- less than an hour of thought has gone into this post
- that said, it comes from a background of me thinking for a while about how the field of AI alignment should relate to agent foundations research

Maybe obvious to everyone but me, or totally wrong (this doesn't really grapple with the challenges of working in a domain where an intelligent being might be working against you), but:

• we currently don't know how to make super-smart computers that do our will
  
  • this is not just a problem of having a design that is not feasible to implement: we do not even have a sense of what the design would be
  • I'm trying to somewhat abstract over intent alignment vs control approaches, but am mostly thinking about intent alignment
  • I have not thought that much about societal/systemic risks very much, and this post doesn't really address them.
• ideally we would figure out how to do this
• the closest traction that we have: deep learning seems to work well in practice, altho our theoretical knowledge of why it works so well or how capabilities are implemented is lagging
• how should we proceed? Well:
  • thinking about theory alone has not been practical
  • probably we need to look at things that exhibit alignment-related phenomena and understand them, and that will help us develop the requisite theory
    
    • said things are probably neural networks
  • there are two ways we can look at neural networks: their behaviour, and their implementation.
  • looking at behaviour is conceptually straightforward, and valuable, and being done
  • looking at their implementation is less obvious
  • what we need is tooling that lets us see relevant things about how neural networks are working
  • such tools (e.g. SAEs) are not impossible to create, but it is not obvious that their outputs tell us quantities that are actually of interest
  • in order to discipline the creation of such tools, we should demand that they help us understand models in ways that matter
    
    • see Stephen Casper's engineer's interpretability sequence, Jason Gross on compact proofs
  • once we get such tools, we should be trying to use them to understand alignment-relevant phenomena, to build up our theory of what we want out of alignment and how it might be implemented
    
    • this is also a thing that looking at the external behaviour of models in alignment-relevant contexts should be doing

    
    

• so should we be just doing totally empirical things? No.
  
  • firstly, we need to be disciplined along the way by making sure that we are looking at settings that are in fact relevant to the alignment problem, when we do our behavioural analysis and benchmark our interpretability tools. This involves having a model of what situations are in fact alignment-relevant, what problems we will face as models get smarter, etc
  • secondly, once we have the building blocks for theory, ideally we will put them together and make some actual theorems like "in such-and-such situations models will never become deceptive" (where 'deceptive' has been satisfactorily operationalized in a way that suffices to derive good outcomes from no deception and relatively benign humans)
• I'm imagining the above as being analogous to an imagined history of statistical mechanics (people who know this history or who have read "inventing temperature" should let me know if I'm totally wrong about it):
  
  • first we have steam engines etc
  • then we figure out that 'temperature' and 'entropy' are relevant things to track for making the engines run
  • then we relate temperature, entropy, and pressure
  • then we get a good theory of thermodynamics
  • then we develop statistical mechanics
• exceptions to "theory without empiricism doesn't work":
  
  • thinking about deceptive mesa-optimization
  • RLHF failures
  • CIRL analysis
• lesson of above: theory does seem to help us analyze some issues and raise possibilities