From hand-written rules to a trained model

Up to here every decision came from rules I wrote by hand, and some of them read a map of the whole grid, which no real bot could see. So I replaced the bots' decision-making with a trained model.

The setup

Moves that aren't legal right now are blocked before the bot chooses, so the network decides which safe move to make, never whether to let go. This is the same idea as Kuan's Neural Particle Automata (one shared network in every particle, seeing only its neighbours), except our output is "which of 19 moves" because the bots have to stay attached.

How it learned

1. Copying: I recorded the hand-written rules building shapes and trained the network to predict their move from what each bot could see.
2. DAgger: the network drives the swarm itself, the rules label what they would have done at each decision, and it retrains on that. This teaches it to recover from its own mistakes.

A few minutes on a CPU. No GPU needed at this size.

What went wrong, and what it taught me

• First try: it built a flat wall every time but finished the arch only 2 times in 5. A diagnostic run, giving it the global map the rules use, made it build everything. So the network wasn't too small; it was missing information. Relaying the work signal between bots up to 2 cells apart (like Kilobot infrared) fixed it: 15 out of 15, including a tower it had never seen.
• In the browser it failed every build. Every training build had sat in the same spot, so it had quietly learned "drift this way". A compass (the direction to the nearest open part of the shape, from the map and the bot's own position) fixed it. Then it built every shape in the sandbox, up to ~900 bots, including eight it never trained on.

Lesson: when a learned policy fails, check whether it's missing information before making it bigger. And test it somewhere that doesn't look like training.