Skip to main content


One of my favorite tests for chatbots is asking for book recommendations. I give it a list of books I liked and books I didn't like (and some flavor for why) and ask them what to read.

They're... ok at this, mostly. It's funny because I always feel like this should be a very straightforward traditional ML problem to do with Goodreads data or whatever but none of the things which purport to be that (Storygraph, etc) are any good at all.

Anyway, o3-mini seems to be the best at this so far for whatever reason. With the same prompt as I've been using elsewhere, it gave me 7 books of which I'd already read and enjoyed 5. Best hit rate on that metric from other chatbots was ~1/4, and in several cases they included books in a series I'd explicitly said as part of the prompt that I didn't enjoy.

in reply to Kevin Gibbons

in reply to Kevin Gibbons

in reply to Kevin Gibbons

Claude Sonnet 3.7 and GPT 4.5 both do pretty well by this metric.

Oddly, they both recommended Senlin Ascends, which no previous model has mentioned. It's from 2013, so it's not like it wasn't in the training data for the other models. I guess I'll have to try it!

3.7 is also the first to recommend Terra Ignota (Too Like the Lightning), which are I think my favorite books I've ever read. (I didn't put them in the prompt because they're pretty weird and most books which are that weird are not to my taste.)

in reply to Kevin Gibbons

I also appreciated this from Claude:

> Ender's Game by Orson Scott Card - Strategic thinking protagonist if you haven't read it already

... Yes, Clause, you're absolutely correct to assume that I've probably already read Ender's Game based on the list of books I enjoyed, well done.

in reply to Kevin Gibbons