Notes on Claude 3.5 Sonnet (new)'s ability to find errors in Latin text
I took an excerpt from a short story written for beginners, and asked Claude to evaluate it, noting that such short stories often contain errors. In a separate chat, I asked the same question, but replaced "Rōmae" with "Rōmā", which I believe is an error (and Claude in the first chat also thinks is an error). In that chat, Claude also thought the text was correct (but had some unrelated complaints). In a third example, I changed the case of a direct object to the ablative/dative instead of the accusative, and it noticed that. So it looks like Claude is not currently consistent at finding errors in Latin grammar.
Ben Weinstein-Raun likes this.
Daniel Filan
in reply to Daniel Filan • •Daniel Filan
in reply to Daniel Filan • •As a follow-up, I tried the same thing with another short story extract, this time changing one verb from tenētur to tenentur (which should make the story not make sense - now it's saying that not just the kid who can't walk who's bed-ridden, but also the slave who can't hear well). Claude doesn't notice the problem.
Prompt 1:
... show moreAs a follow-up, I tried the same thing with another short story extract, this time changing one verb from tenētur to tenentur (which should make the story not make sense - now it's saying that not just the kid who can't walk who's bed-ridden, but also the slave who can't hear well). Claude doesn't notice the problem.
Prompt 1:
Claude response 1:
Prompt 2:
Claude response 2:
Daniel Filan
in reply to Daniel Filan • •Another iteration with more spelling mistakes added, that Claude also fails at:
Prompt 3:
... show moreAnother iteration with more spelling mistakes added, that Claude also fails at:
Prompt 3:
Claude response 3:
Daniel Filan
in reply to Daniel Filan • •As for why I care about this capability, see this post
Daniel Filan
2024-10-27 00:16:38
Ben Weinstein-Raun likes this.