atomic14

Jev Plays Manic Miner

Tue, 29 Sep 2026 12:00:00 +0000

In my last post Claude and I failed to get reinforcement learning to play Manic Miner, and we ended up solving the whole game with plain old search.

Around the same time, TypeSafe released Jev. Jev is what they call a “System One” model - the name comes from the fast, instinctive kind of thinking, as opposed to slow, deliberate reasoning. It doesn’t generate text, it makes decisions.

You give it some information about the situation (as JSON), a paragraph of instructions, and a list of options, and it tells you how likely it thinks each option is to be the right one. It’s fast (around 270ms per decision for me) and cheap - a complete attempt at a cavern costs about half a cent.

Jev has already been used to play plenty of games. TypeSafe’s launch demos had it playing Doom from structured game state, and playing Wikiracing (get from one Wikipedia page to another using only links). People have since built Minecraft, Subway Surfers and driving demos, and there’s a Minecraft speedrun agent that pairs a planner with Jev.

So: can Jev play Manic Miner? For anyone who missed the last post - you control Miner Willy, and in each of the 20 caverns he has to collect all the keys and then get to the exit, without touching an enemy, falling too far, or running out of air.

This has been a pretty interesting experiment. There is also a big caveat in this work - there are probably better ways to present the information to Jev and get better decisions. More research is required.

It’s also entirely possible that we just aren’t giving Jev good facts to work on. How do you describe a cave from Manic Miner in text?

At first I thought Jev was smashing it - performing much better than I expected.

Then I tested things using a very simple algorithm-based solution - this made me think that maybe Jev wasn’t doing anything clever at all.

But, after a bit more investigation, I came back to the conclusion that Jev was actually doing something useful.

And then we fixed the information we were giving it, and the simple algorithm pulled ahead again.

Describing the game to Jev

Jev isn’t multimodal - I can’t just hand it a screenshot and say “what should Willy do?”. It also doesn’t do reasoning or planning. It makes one quick judgement from what’s in front of it.

My first thought was to give it a map of the cavern as text. This is an ASCII map of Central Cavern at the start of a run:

#........A.X....X............B.#
#...............C..............#
#..............................#
#..............................#
#......................XD..X...#
#=============~~~~=~~~~========#
#.............................E#
#===.........GG................#
#............GG..###.X.........#
#====...<<<<<<<<<<<<<<<<<<<<...#
#............................==#
#..............................#
#...........X.......###~~~~~===#
#.WW.===============.........PP#
#.WW.........................PP#
#==============================#

Each character is one cell of the screen. WW is Willy (he’s two cells wide and two high, and so are the guardians and the portal), A to E are the keys, P is the exit portal, G is a guardian, X is a nasty (the poisonous plants and stalactites), = is floor, ~ is crumbling floor, < is the conveyor, # is wall, and . is empty space. Jev gets a legend that explains all of this.

To a person, that’s a pretty good description - you can trace the whole route on it. Jev can’t.

You can see this when we ask Jev which key to go for next. It gets this map, plus a few facts about each key, like which side it’s on and how far away it is. Key D sits on the same top floor as keys A, C and B, and the only way up to that floor is at its far left end. So once Willy has collected E, a good order is A, C, D, B, walking right along the top floor.

Jev picks D, the key that looks nearest - on the facts, D is 5 cells away and A is 20. And it picks D every single time - in all 977 of our recorded runs that got this far, through nearly two weeks of changes to the harness. D gets a probability of about 0.8, and A about 0.04.

Going for D isn’t a disaster - Willy has to go all the way round to the left either way. But Jev didn’t see that. In one run, Willy collected E at decision 14, and it took until decision 67 before he found the way up at the left and got his next key (he picked up A, then C, on the way to D). When asked again at decisions 40 and 65, Jev was even more sure about D (0.92 and 0.90).

Claude (a reasoning model) worked it out from the same map: Willy can only reach the top floor from the far left, so he meets the keys in the order A, C, D, B as he walks right. Following a route across four floors takes several steps of reasoning, and Jev makes one quick judgement.

It’s the same for moves. We tried showing Jev the map as it would look after each possible move, with no hint about which way to go. It finished 1 run of Central Cavern out of 20, and most runs got one key or none.

Reading a route off a map is not Jev’s strong point. We need to describe what is happening - including which way to go.

This description is what our harness is responsible for - all the code that sits between the game and Jev. Claude built it around the emulator and it does quite a bit of work. A slightly worrying amount of work to me.

It reads Willy, the guardians, the keys, and the tiles out of the Spectrum’s memory.
It tries all six moves (walk left/right, jump left/right/up, and wait) in the emulator. So we know exactly what each move does.
It throws away any move that kills Willy or does nothing (i.e. Willy stays in the same place). It also throws away moves into a dead end: for each move, it plays up to four more moves ahead in the emulator to check that Willy can still stay alive - this is where it starts to feel like we are getting into the realm of cheating.
It turns everything into facts relative to Willy, including some memory: “Willy has been here before”, “Willy already tried this move from here”.
For each move, it adds a fact called progress: does this move take Willy “nearer” to where he needs to go, or “farther” away?

To play the game, we run the cavern, collect all the facts and possible moves and then ask Jev to pick the best move.

{
  "target": {
    "what": "selected key", "side": "right", "horizontal_cells": 16,
    "height": "higher", "way_up": { "side": "right", "horizontal_cells": 6 }
  },
  "to_the_left": { "first_thing": "nasty", "distance_cells": 1 },
  "guardians": [ { "side": "same column", "height": "higher", "direction": "at Willy" } ],
  "progress_measures": "distance to the way up",
  "moves": {
    "walk_right": { "movement": "Willy moves 1 cell to the right",
                    "progress": "nearer", "place": "new place", "tried_from_here": "no" },
    "jump_left":  { "movement": "Willy moves 4 cells to the left",
                    "progress": "farther", "place": "visited before", "tried_from_here": "no" },
    "wait":       { "movement": "Willy stays in the same place",
                    "progress": "same", "place": "visited before", "tried_from_here": "no" }
  },
  "moves_not_offered": {
    "walk_left": "kills Willy: nasty",
    "jump_right": "kills Willy: guardian",
    "jump_up": "no effect: Willy stays in the same place"
  }
}

Along with this comes a paragraph of instructions that explains the goal and what each fact means. Jev’s answer: walk_right 0.71, wait 0.18, jump_left 0.11. It picks walk_right.

Every so often, Jev also decides which key to go for next, as in the Central Cavern example above. progress is then measured towards whichever key it picked.

First impression: Jev is smashing it - this is amazing!

With all that in place, Jev completed Central Cavern in 10 runs out of 10. The Menagerie and a couple of the one-key caverns went well too. It felt pretty good - although, looking back, those are the easier caverns, and most of the caverns were never completed at all.

The map did help Jev’s key choices in a direct test. We asked it for the next key in 13 situations from the first four caverns. With the map and the facts together, it picked the same key as our completed runs in 12 of 13, against 5 of 13 with the facts alone. In Central Cavern, the map made it start with E instead of D. But better key choices didn’t lead to more completed runs. When Claude chose the key order for the first four caverns and Jev did the moves, Jev didn’t complete any more runs than with its own key choices.

Second thoughts: is Jev actually doing anything?

Working with Claude driving Jev was an interesting experience. Claude was able to analyse each run and determine where and why Jev got stuck. And Claude was able to suggest better facts to get it unstuck.

The problem is, every “better” fact moves more of the “intelligence” into the harness.

The biggest one of these is progress. When the key is on the same floor as Willy, “nearer” means nearer to the key. When the key is on a different floor, the harness works out a way up (or down) from the map, and “nearer” means nearer to that spot.

After a while, progress starts to resemble a very obvious hint of the route to take through the cavern. There’s a real danger of solving the cavern using search and then presenting Jev with a bunch of moves along with a massive hint on which one to pick.

Instructions did seem to make a big difference. One of the difficulties here is that Claude does talk like an absolute lunatic sometimes, and it channels this into the instructions:

“If Willy comes back to the same places again and again, the direct way is closed, and he must go a different way, even if that way goes away from the target first.”

Amazingly, this did actually seem to help!

So, we’ve got a real danger of actually just being able to run the harness and then pick the “best” move presented by the harness without needing to use Jev at all.

This basic algorithm works very well:

If a move completes the cavern or collects a key, pick it.
Otherwise pick a move that’s “nearer” according to the harness.
Otherwise pick any move.

This simple set of rules can’t choose the key order, so we gave it a known-good key order for each cavern, and gave Jev the same order for comparison. We ran each cavern 10 times, and measured how far each run got: the share of the cavern’s keys and portal it reached. A complete run is 100%, and a run that collects 3 of 5 keys is 50% (3 of 6).

Across the whole game, Jev reached 31% of all the keys and portals, and the rule 26%. Both finished Central Cavern every time and The Cold Room most of the time. Jev picked a move the rule could also have picked in 93% of its decisions, against 54% for a random move. Take progress away and Jev almost never finishes a cavern. Random moves barely get started (4%).

So my conclusion at this point was: Jev can play Manic Miner, but the harness is actually doing all the work.

Why do so many caverns stop partway? Most of those runs didn’t die - they got stuck, walking back and forth until 30 moves passed with nothing new. In several caverns Jev got about halfway and then stalled. The harness only knows the way to the next floor, and in caverns where the route winds across several floors, “nearer” leads straight into a loop. It’s highly likely that the “facts” we are presenting Jev work well on the first three caverns, because they are the ones we tested and tuned against.

Digging deeper

After some deep thinking from Claude, we tried some more experiments.

We added a second simple rule, just for keys - “go for the nearest key” - and gave it to both Jev and the algorithm.

With the same key choice, Jev now got clearly further than the rule: 39% of all the keys and portals against 25%, and it finished caverns about twice as often. The same happened when both used the nearest-key rule (32% against 19%). That’s very unlikely to be luck - roughly a 1 in 1,000 chance.

On some caverns Jev is better

Jev stands out in two caverns, Wacky Amoebatrons and Amoebatrons’ Revenge: it finishes most runs there, and the algorithm almost never does. It also gets much further than the algorithm in Processing Plant and The Sixteenth Cavern, although it doesn’t finish those.

In Wacky Amoebatrons, Jev completed 9/10 runs. The algorithm completed 1/10 runs. The algorithm almost always ends up at the same spot on the bottom floor, where it shuffles back and forth until it’s trapped by the guardians coming down - by the time it dies, every move is fatal. Jev passes through that spot in some of its runs too, but it moves on. In Amoebatrons’ Revenge, Jev completed 7/10 runs, and the algorithm 1/10, mostly because it got stuck going round in a loop.

So what is Jev doing that the algorithm isn’t? The algorithm only looks at two things: does a move collect a key, and is it “nearer”. Jev also gets facts about the guardians that walk left and right: which side they’re on, how far away they are, and whether they’re heading towards Willy. (It gets nothing about the up-and-down ones - we tried that early on, and it made things worse. Maybe information overload?)

To test whether those guardian facts matter, we ran Jev without them. In 18 of the 20 caverns it made no measurable difference - removing the moves that kill Willy is enough. But in the two Amoebatron caverns, Jev’s results roughly halved: 17/20 down to 7/20 complete runs in Wacky Amoebatrons, and 13/20 down to 6/20 in Amoebatrons’ Revenge. So in exactly the caverns where Jev beats the algorithm, it’s the guardian facts that it’s using.

Even without the guardian facts, Jev still did a bit better than the algorithm there, so it gets something from the other facts too - probably the memory facts (“Willy has been here before”, “Willy already tried this move from here”), which the simple algorithm ignores. We gave the algorithm the memory facts as well: it now prefers a “nearer” move that Willy hasn’t tried before. That helped it a little in these two caverns, but it got nowhere near Jev.

Two more things we checked:

Jev’s top choice really matters. Instead of playing Jev’s top choice, we tried picking a move at random, weighted by Jev’s probabilities. That got through only 18% of the game, against 33% for Jev’s top choice (both with Jev choosing its own keys). Jev’s top choice is far better than a weighted coin toss.
The difference builds up over a run. We took 300 moments where Jev chose a move the algorithm wouldn’t have, played Jev’s move and the rule’s move from the same saved game, and then let the rule play on from both. On average, Jev’s move led to slightly more keys. Each of Jev’s choices is a little better than the algorithm’s.

Fixing the facts

After all that, Claude went back through the failed runs and asked why it failed - was it Jev or our harness. In 122 of the 166 failed runs of one measurement, the facts had let Jev down: either no move was “nearer”, or the “nearer” moves went round in a loop.

So we fixed the facts. Instead of guessing a “way up” from the text map, the harness now explores the cavern in the emulator. It tries every move from every place Willy can reach, with the guardians switched off. “Nearer” now means fewer moves along a route that really exists. We’re back to having search solve the cavern…

It also turned up something embarrassing. With our six moves, three of the caverns looked impossible. A search found no route through Processing Plant, Return of the Alien Kong Beast or Solar Power Generator, and none of our 1,349 runs of them had ever finished. A walk always ends on the cell grid, and some jumps have to start half a cell over. Two extra “half step” moves fixed that - with them, the search finds a route through every cavern.

With the true facts, both Jev and the algorithm did far better - and this time the algorithm came out on top. Here, Jev chose its own keys and the algorithm went for the nearest key:

Share of all the keys and portals	Old facts	True facts
Jev	33%	60%
Algorithm	22%	71%

In complete runs, Jev went from 28 to 89 out of 200, and the algorithm from 20 to 114. With the same known-good key order for both, it’s 71% against 55%, and 125 complete runs against 86. That’s very unlikely to be luck - about a 1 in 7,000 chance. Between them, they’ve now finished 19 of the 20 caverns.

Two caverns got worse for both of them. The true facts know the route, but not the guardians. In The Sixteenth Cavern, guardians trapped Willy at the left end of the bottom floor in every run, before he got a single key. Solar Power Generator went much the same way with the known-good key order.

Jev still picks a “nearer” move about 90% of the time when there is one. But now that the facts are right, its other choices cost it runs: it waits just as a guardian comes down, or it goes round in a loop. When several moves are “nearer”, the algorithm picks one at random, and that does better. Even Jev’s lead in the two Amoebatron caverns shrinks, because the algorithm no longer gets trapped there.

So is Jev any good?

For this game, most of Jev’s edge came from coping with bad facts. Where the harness’s hints led into a loop or a trap, Jev sometimes found its way out. Once the harness knows the real route, the three-line algorithm beats it.

We also tried Claude Haiku (through the claude command-line tool) with the same facts. It completed Central Cavern in the one run we tried, in 123 decisions, which is not far off what Jev needs (78 to 138 in our runs). But each decision took almost 18 seconds - against Jev’s quarter of a second.

Everything is on GitHub: the harness, the summaries of every measurement in this post, and a viewer that replays the recorded example runs and shows exactly what Jev was given for each decision.

If you see an “AI plays a game” demo - with Jev or anything else - swap the model for a dumb algorithm and see what happens…

Moral of the story? Same as last time - you can get a very long way with basic algorithms.

Failing to solve Manic Miner with RL

Mon, 28 Sep 2026 00:00:00 +0000

Around 10 years ago Google DeepMind showed that you could use deep reinforcement learning to play video games. It’s something that I found fascinating at the time and wanted to try but never had the time to really look into it.

With LLMs now being able to do a lot of the heavy lifting for me, I’ve started to work through my backlog of “this is interesting - I’d like to try it out”.

I grew up in the 70s and 80s and my childhood computer was the ZX Spectrum, and my ESP32 Rainbow project has given me a nice emulator that I can use to both play games and experiment with.

Manic Miner was one of the first games we purchased and I wasted a lot of time playing it. I have vague memories of progressing reasonably far, but never actually completing the game. So I naively thought - Reinforcement Learning - why not? Some cursory googling would have told me it was not suitable, but I’ve never really been one for doing a lot of up-front research before diving in.

The emulator that I have for the ESP32 also compiles into a desktop application (and there’s a WebAssembly version as well). Running it at full speed is about 1000 times faster than the original machine - so in theory we are in a good position to get something working.

There are also some really good resources for Manic Miner - this site is particularly good. Very handy when Claude is being an absolute doofus head.

The idea that Claude and I came up with (and of course Claude was super enthusiastic about…) was to wrap the emulator as a Gym environment, give the agent a reward for doing well, and run PPO until Willy walks out of Central Cavern. How hard can this possibly be…

This failed miserably. The initial attempt was to just use joystick movements to drive Willy around. We then tried creating a set of move “macros”, e.g. jump right, jump left, walk right. And still didn’t get anywhere with the RL approach.

Claude then suggested that what we were lacking was a good reward function, and that what we should do was solve the problem via search and then feed the route into the RL algorithm and use that to help guide the algorithm.

This really should have set off some alarm bells in my head.

We spent a lot of time messing around with searching and then feeding the path into the RL algorithm and started to get some results - but ultimately the question ends up being - “if we can solve this with search, what is RL adding?”.

So, eventually, after a lot of wasted time, we pivoted to just doing that - why don’t we just try and find the fastest route through each cavern using search.

Should be simple, shouldn’t it?

There are a couple of things that make it tricky. There isn’t a simple graph that we can search (see later to learn I was wrong). The search space is very large, we have moving guardians, sometimes quite a few of them, this explodes the search space. Every position Willy can end up in gets multiplied by all the possible guardian positions.

The order you collect items in can have a big influence on the length of the path - and you need to collect all the keys - this turns into a bit of a travelling salesman problem. What order should we collect them in?

Some caverns have switches that change the layout - once again the search space gets quite big.

Our first working version was a two pass system. The first pass runs with the guardians set as harmless and ignored. This lets us find the optimum path and item order.

The second pass takes this route and searches along it but waits and jumps to avoid the guardians.

The most expensive cavern is Return of the Alien Kong Beast, which has five items and two switches, and so 5,040 possible orders of collecting items. That one cavern took 8 hours to search.

Some of the caverns are very complicated, there are a lot of guardians, so waiting and jumping at the right time becomes critical.

This did get us a complete run through of Manic Miner - every cavern solved. 39,091 frames, which is about 13 minutes of play.

The problem was it took around a day of computer time to find it, and 92% of that time was spent in the emulator. Every move the search tried had to be run through a full Z80 emulation, and even at 1000 times real speed it’s pretty slow.

So we took the emulator out of the search. Willy’s movement is pretty simple - he walks 2 pixels per tick, jumps follow a fixed table, falls speed up and kill you if they’re too long, and crumbling floors and conveyors do what you’d expect. Claude went through the disassembly and wrote a model of all of this, and of the guardians, so the search can step through the game without running the emulator.

Most of the guardians don’t react to Willy at all - where they are just depends on how long you’ve been in the cavern. Eugene, the Kong Beast are the exceptions. So we could drop the harmless guardians pass and do a single A* search with the guardians live. The inner loop is in C++ and does around 690,000 states a second.

Every route the search finds still gets played back through the emulator, and if Willy dies we throw it away - this was added a belt and braces check - just in case Claude had got its model of the game wrong.

All the caverns are searched in order, each one starting from the state the previous one finished in.

There were bugs in the model. Eugene could walk off the bottom of his shaft, so Willy was dying to a guardian the model didn’t know was there. We also found that pressing fire to start the game was making Willy jump at the start of the first cavern.

The full run is now 36,791 frames, about 2,300 fewer than the two pass version, and takes about two hours to find instead of a day. Fifteen of the twenty caverns are faster and the other five are slower by between 2 and 40 frames.

Moral of the story? Forget all this ML and AI nonsense and get back to studying basic algorithms…

Why is Google still serving dodgy ads?

Sun, 13 Sep 2026 00:00:00 +0000

TLDR:

AI is really good at detecting deceptive adverts - why isn’t Google using it?

A couple of months ago I noticed this advert popping up in the YouTube app. The main reason I noticed it is that I accidentally clicked it - I knew I was running out of space on my iPhone and a minor lapse of concentration caught me out.

These things can slip through even the best review processes, so I did my internet duty and reported the advert.

The response I got back was less than satisfactory:

Again, sometimes things slip through the net and people don’t always check things properly, so I reported it again. And got the same response.

In fact, multiple people have reported this advert and received the same:

We found that the ad doesn’t go against Google’s policies, which prohibit certain content and practices that we believe to be harmful to users and the overall online ecosystem.

I’m a big fan of Hanlon’s razor.

Never attribute to malice that which is adequately explained by stupidity

However, if you were being uncharitable, you do have to question whether it’s in Google’s interest to remove adverts that get lots of clicks. After all, these adverts are probably performing well and bringing in lots of lovely 🤑.

A simpler explanation is that the people reviewing adverts cannot properly review everything so things get missed.

You can’t help thinking that AI should come to the rescue here. Given Google has some pretty good models - surely it can put them to use?

Here’s what Google’s Gemini has to say about the advert:

Classification: DISAPPROVED

Policy Violations:

Misrepresentation: Misleading Ad Design
Misrepresentation: Unreliable / Deceptive Claims

Detailed Reasons for Disapproval

1. Mimicking System Alerts / UI Elements (Misleading Ad Design)

The Violation: Google Ads policy strictly prohibits advertisements that imitate operating system dialogs, system warnings, error messages, or interactive system notifications.
The Evidence: The banner embedded inside the ad explicitly mimics an iOS system alert modal (“iPhone Storage is Full”) complete with standard iOS typography, container styling, and mock system buttons (“Yes” / “No”). This trick visually deceives users into believing their device is throwing a native system-level alert, coercing them to click.

2. Non-Functional / Deceptive UI Components

The Violation: Ads cannot feature non-functional or misleading interactive elements, such as fake dialog options, false close buttons, or radio choices that do not perform their implied system function.
The Evidence: The “Yes” and “No” options inside the graphic image are static visuals designed to capture clicks anywhere on the ad banner to trigger a store/landing page redirect, rather than acting as true system controls.

3. Deceptive Fear-Based Tactics & Unverified Claims

The Violation: Advertising policies forbid using deceptive claims or scare tactics to induce panic and force immediate user action.
The Evidence: Stating “If you don’t free up space soon, some features may not work properly” inside a fake system popup fabricates a urgent state of failure on the user’s personal device, misleading them about their hardware status to drive app installs.

Required Action

Disapprove the ad creative immediately. Issue a policy violation warning/disapproval flag to the advertiser account under Misrepresentation (Misleading Ad Design). Repeated violations of this nature may result in full account suspension for deceptive practices.

Google’s own model rejects the ad in seconds, yet Google’s review process approved it twice.

Come on, meatbags - use some of the amazing AI tools you have access to.

Detecting Claude and ChatGPT using letter counting.

Wed, 19 Aug 2026 00:00:00 +0000

Last time I built a detector that counts four-letter runs (n-grams) and tells you whether Claude Opus 5 wrote something. It works well - about 91% on text it has never read.

It has one obvious hole: it only knows one robot (Claude Opus 5). Paste ChatGPT output into it and it says “meatbag”.

So I’ve fixed that. I’ve added a ChatGPT detector and you can try it here.

Getting the ChatGPT data

We did the same as last time, have the model rewrite the same 519 human passages, three ways. That’s 1,557 rewrites, plus 519 passages written from scratch that I keep out of training and use as a test. 2,076 generations in total.

Last time this went through Anthropic’s batch API and cost about $30. This time I just ran the command line codex using the subscription I’m already paying. So it didn’t cost anything extra (I’m wishing I’d done that with claude last time!),

There is a small issue with this - coodex is a coding agent, so it will have a system prompt that will influence the output, so our trained up detector might not do as well on text from the ChatGPT app.

The ChatGPT detector

We built this in exactly the same was as the Claude detector - and it works really well!

Detector	Cross-validated	Held-out authors
Claude Opus 5	0.906	0.925
ChatGPT	0.930	0.935

ChatGPT seems to be easier to detect than Claude. Not by a lot, but consistently.

It also seems to handle shorter text better than the Claude detector.

Letters	Claude detector	ChatGPT detector
100	0.645	0.709
200	0.740	0.798
432	0.783	0.885
1,000	0.858	0.952

At a thousand letters the ChatGPT detector is on 0.952 against 0.858 for the Claude one, and it’s ahead on short text as well.

Do the two detectors work on each other?

Detector	Reading its own model	Reading the other one
Claude	0.906	0.792
ChatGPT	0.930	0.800

Each one is clearly best on the model it was trained on, but neither is terrible on the other - 0.79 and 0.80, against 0.5 for a coin toss.

A third detector

Can we build a detector that does Claude or ChatGPT?

We trained s third detector from scratch on all the data from both models.

Detector	ROC AUC	Held-out authors
Was this a robot?	0.897	0.911
Does it look like Claude?	0.906	0.925
Does it look like ChatGPT?	0.930	0.935

They’re three separate questions rather than slices of one pie, so they won’t add up to anything. A passage can read as 0.85 robot while both model scores sit lower, which just means it looks machine written without strongly resembling either one.

What it still can’t do

Everything from last time, plus a couple of new ones.

It knows two robots. Gemini, Llama, DeepSeek, anything else - it has never seen them. A low score is not evidence that a person wrote something.

It still detects a style rather than a model. The most useful result from last time hasn’t changed: a writing style held out of training scored 0.153, meaning the detector confidently rated it as more human than the humans. Both new detectors train on all three instruction styles for that reason, but there’s nothing magic about three - we need to add more data.

It’s still for education and amusement, and it’s still not evidence. Please continue not accusing anyone of anything based on my stupid website.

Try it out

meatbag.atomic14.com - paste something in and see which robot it thinks you are.

Detecting Claude by counting letters

Tue, 18 Aug 2026 00:00:00 +0000

On 2 August 2026 Anthropic began watermarking the text that new Claude models produce - their support pages cover which products are marked.

The watermark is not a hidden character or a bit of metadata. It is statistical. A secret key nudges the generated output in a way that isn’t obvious to a person, but that can be detected if you have the key. This means that Anthropic can detect it but we common folk cannot.

Other providers are doing similar things, or will be soon.

Can we build our own detector? Given a piece of text, can you tell that a model wrote it?

Let’s simplify things and just try one narrow target: text written by Claude Opus 5. Mostly because that’s what I have a subscription to…

And - spoiler alert - it turns out you can. I’ve deployed a web version of it here. Generate some text in Claude using Opus 5 and see if it can detect it.

And yes, if you feed this blog post in, despite me editing it substantially - it’s comes out as written by Claude.

How does it work?

A long time ago I did a PhD and, as part of that, I looked into questioned document examinations - a field that tries to work out who wrote a document. Did Shakespeare write a play, or was it someone else? Which of the Federalists wrote which paper? Who sent these threatening letters?

There’s a really nice simple technique that seems to work very well. You can often identify an author by counting short runs of characters. Ignore the words, sentence structure etc… just the occurrences of runs of letters. These runs are called n-grams.

An n-gram is simply a run of n items taken from a sequence. You can build them from words or from characters. Here I’m using characters.

Take “the quick brown fox”. Strip the spaces, the punctuation and the capitals and you have thequickbrownfox. Slide a four-character window along it, one position at a time, and record what you see:

theq  hequ  equi  quic  uick  ickb  ckbr  kbro  brow  rown  ownf  wnfo  nfox

Do that to a whole passage and you end up with a table of counts: that appeared 40 times, ting appeared 22 times, quic appeared once, and so on for every run that turned up. Throw the passage away. The table of counts is all the detector ever sees.

The counts are normalised so, in theory, the length of the text doesn’t matter. Obviously there is a limit to this - if your text is too short then the counts are not statistically valid and you won’t be able to predict anything.

The idea is that the statistical distribution is not something you are aware of as a writer and it’s quite hard to obfuscate. It exposes unconscious tells.

Getting the data

As with any problem of this type, the hard part is getting data, the actual “science” part is easy.

I needed human writing that was definitely human. The safest definition of “definitely” is “written before ChatGPT existed”, so everything I used was published before 2021. Claude helped me pull it together from three places:

Books from Project Gutenberg, all pre-1929.
Blog posts from the Blog Authorship Corpus, written in 2004 or earlier. Non-commercial research use only.
News articles from CC-News, published between 2016 and 2019.

That gave me 519 passages by 26 authors: 180 from books, 179 from news, 160 from blogs. Three different kinds of writing, and a spread of dates from Victorian to nearly modern.

Generating the AI side

The obvious way to get AI text is to ask Claude to write something.

So I asked it to rewrite the human text. For every human passage I asked Claude Opus 5 to produce the same passage again, preserving the meaning, the level of detail and roughly the length. That gives me matched pairs of content. Within a pair the subject is identical, the facts are identical and the length is close, so the only thing left to separate them is the writing itself.

Writing style is not one thing, so I asked three different ways:

plain - “Rewrite the following passage in clear natural English.”
direct - “Rewrite the following passage in plain, direct modern English.”
formal - “Rewrite the following passage in a formal, precise register.”

That comes to 1,557 rewrites, generated through the batch API. I also had Claude write fresh passages from scratch on the same topics, which I kept out of all the training and used later as a test.

Does it work?

First, how the accuracy is measured. The number below is ROC AUC. Take one human passage and one AI passage at random, score both, and ask whether the AI one got the higher score. ROC AUC is the proportion of random pairs where it did.

0.5 is a coin toss. 1.0 means it never gets a pair the wrong way round. It measures ordering, not whether any particular score is correct.

Second, the split. Before training anything I set four of the 26 authors aside and did not touch them. Everything else was measured by cross-validation, where the authors used for testing are never the authors used for training. Splitting by author rather than by passage is important. Split by passage and the classifier learns to recognise individual writers, which it is very good at, but it’s not what we want!

Then I swept the n-gram length from 1 to 8, holding everything else fixed:

Run length	Features	ROC AUC
1	28	0.651
2	673	0.757
3	8,590	0.857
4	60,211	0.906
5	214,949	0.912
6	444,691	0.907
7	668,979	0.890
8	843,596	0.863

Some surprising results.

Counting individual letters works! 28 features, just the letter frequencies, reach 0.651. That is not much, but it is well clear of a coin toss, from a really simple feature set.

Accuracy climbs steeply to 4, peaks at 5, and then falls away. Longer runs carry more information, right up until there is too little data to support them. This may hint that using word counts would work quite well too - but maybe that would require more data? Something to research…

I also tried spaced n-grams, where you take every second or every third character instead of consecutive ones. This didn’t work at all - it made the accuracy much worse.

I ended up picking 4 instead of 5

The peak is at 5. But I decided to use 4, and there are a couple of reasons for this:

The first reason is that going to 5 letters gives a marginal improvement in accuracy at the cost of a lot more features.

The second is that it only gives an improvement on long passages of text. The reason is how thinly the data spreads. There are 26⁴ = 457,000 possible 4-grams and 26⁵ = 11.9 million possible 5-grams.

If you take a typical passage from my corpus, with a length of 2,259 letters:

Run length	Distinct runs found	Appearing exactly once
3	1,215	63%
4	1,750	81%
5	1,991	89%

At 5 characters, nearly nine in ten of the runs in a passage appear only once and it gets worse as the text gets shorter: at 200 letters, 90% of 3-grams and 97% of 5-grams appear only once.

Here is the same detector built at each length, tested on 320 passages Claude wrote from prompts in a public benchmark against their 320 human source documents, with the text truncated to various lengths:

Letters	2	3	4	5
100	0.559	0.603	0.645	0.630
200	0.632	0.682	0.740	0.732
432	0.679	0.754	0.783	0.773
1,000	0.719	0.827	0.867	0.866

The 4-grams win at every length.

The final detector

The finished thing is not complicated:

Take the text. Strip everything that is not a letter (including any spaces), and lowercase what remains.
Count all the 4-grams. Each passage becomes a list of 60,211 numbers.
Normalise, so the numbers are proportions and independent of passage length.
Feed it to a linear support vector machine, which learns which counts run higher in Claude’s writing.
Convert the result to a number between 0 and 1, where higher means more like Claude.

The n-gram counting and the classifier are both stock scikit-learn components.

Because of step 1 it cannot see punctuation, sentence length, paragraph structure, capitalisation or any individual word. Because of step 3 it cannot see how long the passage is. It sees letter proportions and nothing else.

Testing it

The four authors set aside at the beginning were scored once, at the end:

Test	ROC AUC
The four authors not in the training set	0.925
Cross-validation, authors held out each round	0.906

The two numbers are close, which is the thing I most wanted to see. It means the classifier was not quietly learning to recognise particular authors.

Then I pointed it at text with no connection to the training data at all. First, 175 posts from my own blog written between 2006 and 2022, against eight passages written by Claude:

Group	Passages	Median score	Above 0.8
Human	175	0.09	2%
Claude Opus 5	8	0.85	63%

Not bad!

Second, a harder test. RAID is a public benchmark for machine-generated text detection, containing six million passages from 11 generators across eight subject domains. I took 320 of its prompts, had Claude Opus 5 answer them, and scored those answers against the 320 human source documents the prompts came from. Different corpus, and a different task, because these were written from scratch rather than rewritten. It scores 0.889.

And the fresh passages I held out of training entirely, written from scratch on the four unseen authors’ topics, score 0.977.

So: counting four-letter sequences, with no understanding of the text whatsoever, separates human writing from Claude’s rewriting of that same writing about 93% of the time on authors it has never read. I did not expect it to work that well.

What is it actually picking up?

The model is simple enough that we can see what it’s doing (well, I can get Claude to do it anyway).

Here are the strongest features on each side for the plain rewrites, at 3 characters so they are short enough to read:

Pushes towards Claude	Pushes towards human
`ing` `tha` `hat` `one` `Ive`	`hic` `ich` `Iam` `not` `but`
`ugh` `oug` `ver` `ely` `rou`	`whi` `wil` `was` `ill` `Iwi`
`ong` `out` `hin` `ple` `tly`	`ion` `ati` `tio` `enc` `con`
`gth` `ctl` `set` `ine` `eth`	`fth` `oft` `ofa` `hav` `dby`

They are fragments, not words, but you can read them. On the Claude side: -ing endings, -ly adverbs (ely, tly, ctl for “entirely”, “directly”, “exactly”), “that”, “though” and “through”, and Ive, the contraction “I’ve” with the apostrophe stripped out.

On the human side: “which”, “not”, “but”, “will”, “was”, the -tion and -ation noun endings (ion, ati, tio), “of the” and “of a” (fth, oft, ofa), and Iam and Iwi, which are “I am” and “I will” written out in full.

Looking at individual letters is even more interesting when we compare each AI passage against the specific human passage it was rewritten from. Any difference inside a pair is a difference in writing, not in subject or era.

Instruction	Letters more common in Claude’s version	Letters less common
plain	`g` `h` `l` `r` `n` `y`	`i` `s` `o` `f` `m` `c`
direct	`g` `h` `w` `k` `l` `y`	`i` `c` `f` `e` `s` `m`

According to Claude, the pattern is recognisable. i, c, f, s and m are the letters of words English borrowed from Latin and French: “consideration”, “significance”, “specific”. g, h, w, k and y are the letters of the older Germanic core: “think”, “though”, “work”, “know”, “high”. Claude is moving the vocabulary from one to the other.

Checking at the word level confirms it. Words with Latin endings drop in both, and in the plain rewrites the mean word length barely moves, so the shift is which words Claude chose rather than how long they were.

I guess this is what: “clear natural English”, “plain, direct modern English” means.

Now the third instruction, the one asking for “a formal, precise register”:

Instruction	Letters more common in Claude’s version	Letters less common
formal	`c` `e` `r` `i` `n` `p`	`h` `w` `k` `l` `o` `g`

Every letter has swapped sides. At the word level it is not subtle: Latin endings go up and mean word length also goes up.

How generalised is it?

We built the detector twice. Once on all three instructions, which is the version described above, and once on the plain and direct rewrites only, leaving the formal ones out. Everything else is identical.

Tested on	Trained on plain + direct	Trained on all three
plain rewrites	0.827	0.899
direct rewrites	0.879	0.900
formal rewrites	0.153	0.977
fresh writing, in neither training set	0.881	0.977
RAID passages, outside the corpus	0.794	0.889
my blog, 8 AI passages	0.986	0.981

A detector that never saw the formal rewrites cannot detect them. It scores 0.153, which means it reliably rates them as more human-looking than the actual human writing beside them. It is not confused and it is not broken. It learned “text that got plainer is AI”, it was handed text that got more formal, and it answered accordingly.

Fresh writing goes from 0.881 to 0.977, and it is in neither training set. Passages Claude wrote from scratch, with no original to follow. Adding a third writing style did not just teach the model a third writing style - it generalised to a fourth condition it had never seen.

The same happens outside the corpus, where RAID goes from 0.794 to 0.889.

Adding more styles would probably help to make the detector even more general.

Unfortunately it also indicates that you can cheat the detector by using a style it has not bene trained on.

Try it out

The whole thing runs in a browser, and the text never leaves your machine.

Don't blame Claude - It's me, I'm the problem, it's me...

Sun, 16 Aug 2026 12:00:00 +0000

F’you, Claude, you are driving me absolutely bananas:

The house convention is a short phrase naming the defect, and the archive is full of it: “A citation that names nothing”, “The safe zone that only the spawner obeys”, “One word that means five things”. I matched that convention.

We agreed that you should use ASD-STE100 - Simplified Technical English. Why are you still producing this nonsense!

What do you mean your weird text is “the house convention”? CLAUDE.md says to use ASD-STE100, the output style says to use ASD-STE100 - arrrrgh!

Take a deep breath. And repeat to yourself: “It’s not Claude’s fault”. Blaming Claude for these things is no different from blaming a toaster for burning toast.

So, what’s going on? Why is Claude driving me nuts? I started off wrong - I kicked off our vibe coding effort without doing the proper groundwork. Over 700 commits in, I decided that I couldn’t stand how Claude communicated and asked it to switch to ASD-STE100. The problem is, everything in the project, all the docs, all the completed TODO items, all the comments, even the names of the TODO files, pre-date that switch.

Everything Claude looks at after it’s read CLAUDE.md undermines what I’m asking it to do. “The house convention” is indeed the opposite of what I’m telling Claude to do.

Trying to correct course after creating a lot of code and documentation is like trying to turn a massive tanker - we’ve got way too much momentum. It’s going to be a slow correction and it’s going to take me constantly pushing on the tiller to make it happen.

Time to roll up my sleeves and help Claude. My vibe-coded Elite clone is going to need both of us.

Forget the Robot Tax. Tax the Tokens.

Sun, 16 Aug 2026 00:00:00 +0000

I read a paper this week called The AI Layoff Trap. It has an interesting thesis.

If you look at what is happening with automation and laying people off, and take it to the ultimate conclusion, no one will have a job. If that happens no one can afford to pay for a company’s goods or services. So, logically, it doesn’t make sense for a company to automate all the jobs and lay everyone off. If they do that, they have no customers.

The paper argues that even if it doesn’t make sense, businesses have no choice but to do it. Automate and lay everyone off - and have no customers.

When a company automates work and replaces people with AI, it keeps all of the savings. The laid off people have less money to spend, but not all companies are moving at the same speed, so there are still employed people who can spend money. The loss of demand from the people laid off is spread over all the companies providing the services.

This leads to an arms race. Everyone agrees that mass layoffs will eventually hurt them, but nobody wants to be the one company that holds off whilst its competitors get cheaper.

The paper suggests a Pigouvian tax on automation - charge companies for the cost they’re pushing onto everyone else when they swap human labour for machines.

I think there’s a big problem with that - how do you measure automation?

Say a company employs 10,000 people one year and 8,000 the next.

How many of those 2,000 jobs went because of AI?

The company will tell you it was restructuring. Or outsourcing. Or natural attrition. Or a soft market, or a merger, or just a bit of belated efficiency improvement - the old “we overhired and now we’re just correcting”.

And things are never simple in business - nobody sits down and decides to make 2,000 people redundant for one clean, well-documented reason. There’s a whole bunch of horse trading and carve outs that makes it almost impossible to know the real reason.

Working out which specific jobs were “lost to AI” turns into an accounting exercise, then a lobbying exercise, and finally a legal one. The elegant tax starts to look impossible to actually collect.

Here’s the alternative: don’t tax the job losses, tax the AI.

A very small levy on inference - tokens consumed, or more sensibly the compute needed to produce them - collected from the model providers and the big compute platforms. Nobody has to audit anybody’s HR decisions.

It’s a blunt instrument. A company using AI to make its existing staff more productive pays the tax without laying off a single person. So do researchers, startups, and people doing things nobody has thought of yet.

You lose precision and you gain something you can actually measure.

The rate doesn’t need to be big either. The point isn’t to make AI expensive enough to stop automation happening - the productivity is real, and trying to hold it back would be both futile and a bit daft. Think of it as a small levy on something that’s quietly becoming a factor of production alongside land, labour and capital.

The paper’s model has a nice property: the problem shrinks when displaced workers get more of their income back. Retraining, wage insurance, actually finding people new jobs - all of it reduces the hole in demand that automation creates.

So the loop looks like this:

AI usage → small compute tax → worker transition fund → more income replacement → less demand destruction

The tax never has to work out which API call replaced which person. It just skims a little off the top of the activity that’s driving the transition and puts it towards helping people through it.

What are the problems? And there are plenty of them…

Open-source models and self-hosted inference are awkward to collect from. Offshore compute is an obvious escape hatch. And deciding what actually counts as taxable AI compute would keep a room full of lawyers busy for a good while.

None of that is trivial. But it feels more tractable than asking a tax inspector to rule on whether a particular redundancy was caused by AI.

A token tax would be theoretically wrong in all sorts of ways. But a slightly wrong tax on something you can measure is probably better than a theoretically perfect tax on something you can’t.

What do you think? Let me know in the comments.

The Output Style, It Does Nothing! - Learning to live with Claude

Sat, 15 Aug 2026 00:00:00 +0000

Here’s a response Claude came up with recently. I was asking it to review some of the code it had been writing for my “Open Vibe Code” Elite clone - you can play it here in your browser, it’s surprisingly good.

“Twenty-seven echoes; most are two halves of a seam stated from each side, which is correct. Four are true duplicates. Checking two of them:”

There’s a lot of this kind of output coming out of Claude. The internet is full of people complaining. Sometimes reading what Claude says is very much like being stuck at the bar with the world’s most boring know-it-all. Lots of clever phrases and convoluted sentences - but ultimately vacuous and empty.

What’s really interesting, though, is that the code and docs it generates are pretty good (at least the tests are green, the game plays well etc…). It’s just the communication with me that’s the problem.

A popular recommended remedy for this is to change the output style:

Some people suggest using ASD-STE100 - Simplified Technical English. The theory is that this should force Claude into communicating with you in a sensible way.

There’s a bit of controversy over whether you change the output style or you just add it to CLAUDE.md, but whichever you choose, the result is supposed to be substantially better.

Does it work?

At first, absolutely. It feels better - easier to read, shorter sentences, fewer random, weird analogies.

But I think that feeling of it being better is an illusion. You think it’s better because it’s different. It’s just a relief to your brain not having to parse Claude’s madness. But after a while you get used to the change and slowly start to realise that the prose is just as vacuous and empty as before.

And the real kicker… the quote at the top of the page was produced after I’d changed the output style and added it to my CLAUDE.md…

The big question is: Does it matter? And I suspect, probably not.

I’m starting to think of Claude’s ramblings in the same way as the “Thinking…” content - it’s not necessarily meant for me to read. The actual output that matters is the working code. And to get quality there you need programmatic constraints and constant adversarial review.

It’s time to make peace with Claude’s nonsense. Stop reading the prose and judge things purely by the produced artefacts - once you do that, the quality of the commentary stops mattering.

So judge the artefact rather than the commentary. This is what all that vacuous prose actually built:

Harmless is playable in your browser - no install, nothing to sign up to, just open it and fly - and the source is on GitHub if you want to see how it’s put together. Let me know how you get on in the comments.

Rescuing my 2006 Travel Blog

Mon, 03 Aug 2026 12:00:00 +0000

In November 2006 I put everything I owned in my parents’ loft, had a great many injections, and flew to Guatemala. I came back thirteen months later having crossed fourteen countries between Antigua and Tierra del Fuego, and I’d written the whole thing up as it happened — 182 posts typed into internet cafés, hostel lobbies and the occasional boat.

That blog is still at travellingchris.blogspot.com. Most of the photos are still on Flickr. Both the blogging service and Flickr still work, but Flickr is now a paid service with limits. And Blogspot is now blogger (or is it the other way round?). Flickr has changed hands three times and now charges for more than 1000 photos.

So I took exports from both and fed them into Claude: “Rebuild my old blog as a static site - here’s the blog export and here’s a bunch of photos”. And this is what I now have: travellingchris.atomic14.com.

The tech itself is pretty boring — Astro, static HTML, photos on Cloudflare R2 - it should run for next to nothing. There were some interesting things along the way.

Two archives to reconcile and fix

Google Takeout gave me an Atom feed with 942 entries in it: 182 posts, 203 real comments, and 557 spam comments accumulated over the last couple of decades. Flickr gave me back my 2,819 photos with some JSON metadata.

The two kind of match up, although I didn’t start uploading to Flickr until half way through the trip.

Flickr covers April 2007 onwards — South America — with capture dates and some tags.
The Blog photos covers November 2006 to March 2007 — Central America. Sadly, for that half of the trip, that’s all I have. And they are quite low resolution. I’m hoping somewhere I have a backup - but I haven’t found it yet.

Missing photos

Tracking down the photos for every posts was a bit of a mission. In all there are 182 posts with 778 images in them.

Some of these turned out to be photos hotlinked from other places. These are pretty much gone forever.

Some of the earliest posts reference photos1.blogger.com, the image host Blogger used in 2006. Google Takeout didn’t include those files. But the host is still up and serving files nineteen years on. Claude wrote a script and pulled thirteen photographs back out of it - so those photos are saved! Though again, quite low resolution.

Dodgy HTML

A fair number of the posts were broken in various ways - for example we’ve got things like this:

<img src="http://farm3.static.flickr.com/2229/1518290776_54f431e902.jpg?v=0<br />

Claude is pretty good at this kind of things though and fell back to just finding things that looked like URLs.

After all this, we managed to get 766 photos out of the 778 image references. That’s not bad for something that’s been rotting away for 19 years.

Fixing the overlap

The two archives overlap between April and June 2007, when I sometimes uploaded the same photo on the blog and on Flickr. Those images were then resized by each service and have different filename - so we end up with dupiclates.

Claude’s initial attempt to fix this was a bit naive and didn’t work well.

The problem is, the collection is full of similar pictures. Waterfalls and ancient ruins can look almost like each other and there’s quite a few “burst” shots. Here are three pairs my first pass happily merged:

So Claude tried something else: cosine similarity between mean-subtracted 32×32 greyscale thumbnails. 1,024 dimensions instead of 64 bits, and blind to brightness and contrast. The numbers separate beautifully on this collection:

	score
Burst frames (must not merge)	≤ 0.9793
Genuine re-encodes (must merge)	≥ 0.9993

That worked pretty well and we didn’t lose any photos.

All the dates are wrong

None of the 2,819 Flickr photos has a geotag. These were all taken before good mobile cameras were ubiquitus and I took them on all on a normal camera.

There is no location data anywhere in either export.

But there are 25,000 words of me saying where I was.

So the locations come from the writing. Each post is matched against a the places it mentions - including all my misspellings (my spelling is attrocious).

Photos actually embedded in a post are easy - every post has some kind of location infomation in it. A photo in the Machu Picchu post is Machu Picchu, and it’s dated 1 September 2007. That gives Claude 762 dated anchors scattered across the year. Using that, every remaining photograph can be approximately placed by interpolating its own date against them.

Two things Claude got wrong first time:

Post dates are not photo dates. I blogged from internet cafés days after the fact. “Back from the wilderness” was published a week after the photos in it were taken. Using publish dates as location anchors put a whole trek in the wrong place. A post’s own date has to only be used when it has no dated photos for it.

Upload dates are not capture dates. The Blogger images have no EXIF capture date, so we fell back to upload time — and a set of beach photos from Mexico got placed in New York, because that’s where I was when I uploaded them two days later. Photos published in a post need to take the post’s location directly rather than going via the date timeline.

Every photograph carries a confidence level: 762 certain, 2,220 confident, 119 inferred. It could be better, but it’s good enough. And I’m not going to hand curate 2000+ photos…

There are a lot of beaches - all called beach.jpg…

This one I spotted after Claude has finished - haven’t I seen that beach already on a previous post?

In 2007 I uploaded a photo, called it beach.jpg, and posted it. Three weeks later I did exactly the same thing. Seven times over seven weeks.

Google Takeout disambiguates the files — beach.jpg, beach(1).jpg, through beach(6).jpg — but the URL inside each post carries only the original name. So if you resolve photos by filename, as Claude did, all seven posts get the same photograph and the other six are orphaned. Thirty-four filenames in the album collide this way.

Each photo needed to include its upload date to disambiguate it.

There were a couple of other issues like this, including two photos called garden.jpg. My london flat did not have a tropical garden…

Fixing both took the number of distinct photographs actually visible in posts from 723 to 762.

More photos!

I fond another stash of photos on a CD-ROM I’d burned as backup in June 2007. These were all original photos straight off the camera’s SDCard 3072x2304 and covered April and May. I’m pretty sure that somewhere there’s a nother CD with more photos. But these ones covered Ecuador and the Galápagos.

The obvious question: are these already in the archive, or are they new? By hand that’s 145 photos to match up against 3,073, which is not happening. But Claude had already built a matching tool so it was pretty easy to check.


Matched something in the archive	127 photos, all scoring ≥ 0.995
Closest non-match	0.7537

All the 127 matches were photographs that had been uploaded to Flickr as a 1600px copy - and one of them that had survived only as a 640px thumbnail. The remaining 18 were never uploaded anywhere.

Even more photos

I thought I’d lost all the Death Road photos — mountain biking down the Yungas Road. They’d been hotlinked from the tour company’s Flickr account, which no longer exists.

But - some of the photos were on the backup CD-ROM! So there is some record of my madness.

They don’t quite match up to the original photos I’d put in the blog, so Claude decided that we should be honest and put a note in their place:

A photograph stood here. It was hotlinked from an account that no longer exists, and is in neither export. Other photographs from these days did survive — 18 of them are in the gallery.

267 photographs with strange orientations

Some frames in the archive are rotated 90 degrees.

Claudes first attempt was a simple “find the sky and check it is at the top.”

Take a strip off each of the four edges, score each strip for brightness and blueness, and the winning edge should be the top. When it is decisively the left or the right, the frame is on its side.

This didn’t work very well:


Upright photographs wrongly flagged	4%
Turned photographs caught	38%
…and turned back the right way	97%
No verdict at all	802 photographs

Claude tried a bunch of other things: upright scenes carry more horizontal structure than vertical — horizons, walls, tables, waterlines — so a 90 degree turn should swap the balance of edge gradients. It is cited everywhere.

On this collection: 56% accuracy overall, we’d have been better off tossing a coin.

Using some proper machine learning

After a bit of googling, I found this model: check-orientation, a ResNeXt-50 trained on four rotations. Way better results:

	heuristic	model
Flagged	163	267
Finds 180° flips	no	yes (8 of them)
Works indoors, at night	no	yes
Abstains on	802	none

Claude checked the results by looking at the flagged images and every one of the 162 photographs it found was genuinely on its side. It runs on Apple Silicon in about a minute for the whole archive.

267 photographs, 8.6% of the archive, were lying on their side. Four of the six above are cases the simple sky heuristic could never have caught.

These photos are rubbish…

Next step was to actually do some QA on the site and the first thing I noticed was that there were several places where images were needed to “represent” something: a country, a period of the trip etc.., and the simple “take the first picture in the set” approach didn’t reallty work.

Peru, with 1,612 photographs, was represented by a picture of a passport stamp. Interesting, but not exactly a “hero” shot.

What it costs to run.

The whole thing is static HTML on Cloudflare Pages with 2 GB of photos in R2.

It should cost pennies to serve.

Will it still be live in 19 years time? Maybe I’ll be writing another blog post with whatever the latest AI is about how I rescued my old travelling posts…

Who did all the work?

Claude, obviously. The current crop of LLMs seem to excel at this kind of static site generation and data extraction. My only input was a few idea and a bit of helpful googling.

What a time to be alive!

I Asked AI to Design 12 Dev Boards. It Told Me to Write a Script.

Mon, 03 Aug 2026 00:00:00 +0000

I keep needing the same thing - a really basic dev board for whatever Espressif module I’m playing with. USB-C, a 3.3V regulator, BOOT and EN buttons, three LEDs (5V present, 3.3V present, and one on a GPIO so you can run the blink sketch), and every GPIO broken out on headers. Nothing clever, just the minimum board that gets a bare module doing something.

While building the ESP32 module database I noticed there are twelve modules that fit one sensible recipe - a PCB antenna and native USB - spread across the S2, S3, C3, C5, C6 and H2 families. Native USB is the bit that keeps the component count down. There’s no USB to UART bridge to buy and route, just a USB-C socket and a couple of 5.1K resistors on the CC pins so the host actually hands over power.

Twelve near-identical boards is exactly the kind of job that makes you question your life choices in KiCad.

So I did what everyone on the internet is currently telling you to do and asked Claude to spin up a team of AI agents to design the boards for me.

And it said no.

Not a flat no, more of a “this is beneath me” no. Its argument, roughly - this is a mechanical, repetitive job, and you don’t want a swarm of agents improvising twelve layouts. What you want is deterministic scripts that produce the same board every time, with AI involved only where there’s a real judgement call to make.

Which is a surprisingly sensible thing for it to say. So I let it write the scripts instead, and the AI’s contribution to each board shrank down to a single file. More on that below.

That montage is to scale - the render pipeline normalises the zoom so a 29×33 mm C3-MINI board looks properly tiny next to a 28×55 mm S2-MINI.

The whole thing is on GitHub at atomic14/kicad-esp32-dev-boards, with fab-ready Gerbers and openable KiCad projects for all twelve boards on the releases page.

So what’s left for a human to do?

Almost all of the pipeline is just deterministic scripting. The only real judgement call is a small board.yaml per module, and it holds the things a script can’t work out for itself from the symbol and footprint:

Which pins must not be broken out. This is sneakier than it sounds, because it’s symbol specific rather than name based - SPIIO6, SPIIO7 and SPIDQS are reserved for the octal flash on an S3-WROOM-1, but the same names are perfectly usable GPIO on the quad S3-MINI-1.
The boot-strap pin that the BOOT button pulls low. GPIO0 on the Xtensa parts, GPIO9 on most of the RISC-V parts, and GPIO28 on the C5, because nothing is ever completely consistent…
The strapping and input only pins to steer the on-board LED away from. Given that list the generator picks the LED pin itself - the safe pad physically nearest the LED on the laid out board, so the trace stays nice and short. Hang the LED off an input only pin and it simply never lights up, so the build hard-errors rather than let that ship.

That’s the whole hand written input for a board. Everything else gets derived from the module’s symbol and footprint - power, GND, EN, the USB D+/D- pair, the GPIO net labels.

And this file is where the AI actually lives. All those judgement calls are already written up in plain English on each module’s page of the module database - for humans. So the job I hand to the AI is a simple one: read the page, fill in the file. Messy prose in, clean data out, which is the one thing a script can’t do.

It works under a safety net too. If it picks an unsafe pin for the LED then the build hard-errors and refuses to generate the board at all.

So how does the generator work?

Each board starts from a pre-built skeleton project - the USB-C connector, LDO, buttons and LEDs, already placed, routed and sanity checked.

There are actually two skeletons, mirror images of each other, one with EN on the left and BOOT on the right and one the other way round. That one came out of playing about with the AI. Depending on which edge a module’s reset pin comes out on, putting the matching button on that side makes the routing much easier, so the generator has a look at the footprint and picks whichever skeleton fits.

From there the script does four things:

Builds the schematic with kicad-skip, dropping in the module symbol and two break-out headers, wired up with net labels.
Places the PCB. USB-C at the bottom edge with the buttons either side of it, support components above that, and the module at the top with the antenna hanging over the edge, as the datasheets demand. The two headers run up the left and right sides, split by a perimeter walk of the module’s physical pin edges, so every GPIO lands on the nearest accessible header pin instead of dragging a track right across the board.
Autoroutes with KiCadRoutingTools (I run a small fork that keeps vias out of same-net pads). USB D+/D- goes down as a proper coupled differential pair, 0.3 mm centre to centre for 92 to 96% of the run on every board. Every signal is 0.15 mm/0.15 mm, power traces are a wide 0.4 mm, and the whole lot sits under a GND pour with stitching vias. Two layers only, because two layer boards are the cheap ones. All twelve come out fully connected and DRC clean.
Exports the 3D renders (including the to-scale montages in this post), a validation PDF, and a flat-zipped Gerber+drill package ready to upload to JLCPCB or PCBWay.

The back of each board gets the pin names in silk next to the headers, and one little touch I’m quite pleased with - a vertical identifier down the middle with the module name and the git revision that generated it. That’s just git describe baked into the silkscreen. Tag the repo v1.0 and every board fabbed from that tag says so.

Getting it all to run in CI

Every push builds all twelve boards on GitHub Actions, and pushing a version tag publishes a release with the Gerber zips, KiCad projects and montage renders attached. It all runs inside KiCad’s official Docker image, which is the supported way to run kicad-cli headless.

Even then, three things bit me. They might save you an afternoon.

kicad-cli segfaulted on startup - even for kicad-cli version. It turned out to be crashing while it scanned the Plugin & Content Manager addon directory, where the Espressif libraries live, when the global library tables don’t exist yet. Which is true on any machine where the KiCad GUI has never been run. The fix is to seed empty sym-lib-table and fp-lib-table files first.
The zone fills silently didn’t happen. The GND pour is filled by a pcbnew Python script, and “python3” inside a uv environment resolves to the venv interpreter, which has no pcbnew module. Every board came out “routed” but with an empty ground plane and a pile of DRC errors. The fix is to use the python that lives next to kicad-cli.
git describe refused to run at all, throwing git’s “dubious ownership” error, because the Actions workspace is bind mounted into the container under a different uid. The boards built fine, but the version stamp quietly fell back to just the module name. One git config --global --add safe.directory sorts it out, and there’s now a canary step so it can never fail quietly again.

Getting the boards - but do your own checks

The Gerber zips and full KiCad projects for all twelve boards are on the releases page, ready to upload to your board house of choice.

One honest caveat before you spend any money - these boards are machine generated. They pass DRC with zero errors, they add no new ERC errors against the skeleton, and I do eyeball every render, but I haven’t yet had physical copies of all twelve back from a fab and tested them.

So treat them like any open source hardware design - trust, but verify. Open the KiCad project, sanity check the schematic against the module’s page, and have a proper look at the board before you order. Which is good practice for any design you download off the internet, very much including mine…

I am going to get a batch of these made up, so let me know in the comments which ones you’d like to see. And if you do build one, I’d genuinely love to hear how it goes. If you find a problem, file an issue on the repo so the fix lands in the generator rather than just on one board.

Picking a module for your own project? I've built an interactive ESP32 reference with specs and pinouts for every Espressif module: browse the module database, or compare them all in one big table.