Teaching a stick figure to get back up with aelife

JeffTechnical Articles & Notes

aelife is the name I gave a personal project in artificial life. Its origins go back to a neural network training algorithm I invented in the early 1990s. My first application was to spider webs. I also used the algorithm to design a bridge that won a competition on my mechanical engineering degree course at Durham University.

How my mutation algorithm worked

I designed the training algorithm myself, drawing on genetic mutation and selection. I took one parent, made several copies and randomly changed some of the values in each copy. I tested these offspring against the unchanged parent on the same task.

Each new parent started with a wide mutation range. For every generation in which the parent retained the best score, I narrowed the range. That allowed progressively finer adjustments. As soon as an offspring beat the parent and won the generation, it became the new parent and I immediately widened the range around it. Each improvement was therefore a starting point for a broader search, with a chance of reaching a better solution beyond the nearest local maximum.

A tune on the spacebar

After the spider webs, I tried my algorithm on tune recognition. I tapped out the rhythm on the spacebar of my BBC B 32K home computer. Every note came from the same key, so the network had only the timing to go on. I knew which tune I’d tapped and could check its answer after every attempt.

I set a stopping rule of more than 95% correct identifications on the test input. Training with my algorithm kept running without reaching it. I then refactored some C++ examples from Stephen T. Welstead’s yellow and green paperback, Neural Network and Fuzzy Logic Applications in C/C++, to run in BBC BASIC on the same machine. That version took 24 hours to reach the threshold.

The tune recogniser learned by adjusting its weights and biases. A weight determines how strongly a signal affects the next neuron, and a bias shifts the point at which that neuron responds. With the book-based version I could feed in a rhythm, measure how far the output was from the answer I knew and use that error to adjust the numbers. I remember regression or simple calculus being involved.

Back to aelife, November 2026

In November 2026 I returned to the problem of training a network, this time with a simulated body. The film shows one of the later runners recovering from a fall. It swings its legs over, brings its feet under its hips and walks. The network supplies the joint targets throughout. The movement follows from those targets, the forces at the joints and contact with the ground.

One successful back-fall trial, played at real speed. The ruler shows how far the figure walks after it stands.

The spacebar recogniser had a considerable advantage. I knew which tune I had tapped. With the runner I could measure a fall, but I had no reference sequence telling me what each of twelve joints should have done beforehand. The useful angle of the left knee depended partly on when the right foot took the load. I could specify the outcome I wanted without knowing the movements that would produce it.

Energy and reproduction in the petri dish

I gave a cell a few things it could do. It could collect energy from light, spend some of it moving and, if it had enough left over, make a copy of itself. Small differences passed to its offspring. I wanted to know whether there was any advantage in growing a bigger body. An extra cell might help a creature paddle towards food. It also needed feeding, every second of its life.

I noticed that when an organism died, its remains sometimes appeared some distance away. I had capped the number of food particles the simulation could draw. At the limit it added the remains to an existing particle, which deprived nearby organisms of a food source. I changed the simulation so the remains stayed where the organism died and gradually became edible.

Trails show the paths the swimmers take through the dish. Some chains bend their whole bodies to move. Try the petri dish yourself.

Returning to my algorithm for the runners

I started the runners with my own mutation algorithm again. After waiting a day for the BBC to recognise a rhythm, I expected modern hardware to let me test generations millions of times faster. That was my expectation going in. In the first races I gave five runners identical bodies. One ran the parent network and four ran mutated copies. In practice, each generation cost me a physical simulation for every entrant. One early hybrid run took just under three minutes of computing time for 100 training rounds with 120-second races. I also tried extending a successful change in the same direction to measure how far the improvement continued.

The parent and one altered copy start from the same crouch. The distance beside each is how far its body has travelled forwards. Try the classic stick-figure races.

The choice of score changed the gait I selected. Measuring total forward distance rewarded a runner for falling onto its face and sliding. Counting only forward distance travelled upright gave me a measure closer to the behaviour I wanted.

The browser runner grew to 31,373 weights and biases. Its 102 inputs feed two hidden layers of 128 neurons apiece. Each neuron adds up its weighted inputs and a bias, then passes that sum through a smooth curve called tanh that keeps the result between −1 and +1. The last layer produces 13 numbers. Twelve set targets for the waist, neck and five joints on each side of the body. The thirteenth sets the pace of an internal rhythm. A knee output asks for an angle. The simulated joint still has to get there under load.

I had underestimated the increase in the number of dimensions. Each of those 31,373 parameters was another value the search could vary. Moving the trailing foot earlier might help only if the opposite leg also took the weight sooner. Changing either movement alone could reduce the score. A wider mutation could disturb a gait that already worked, and a narrower one could miss the combination entirely. The computer could run more trials, but the distance at the finish still gave me little information about which weights to change together.

Training with mutation and PPO

With the spacebar I could use the difference between the network’s answer and the known tune to guide training. For the runner I needed a way to learn from the consequences of a movement. I tried proximal policy optimisation (PPO). During practice the runner varies its actions, and a second network estimates the future reward from its current state. If an action is followed by a better return than predicted, PPO adjusts the policy to make that action more likely in similar states. The update rule discourages large changes in action probabilities in a single training step. I could train from a sequence of observations, actions and rewards without supplying a target angle for every joint.

I then put PPO and my mutation algorithm to work on the same controller. In one browser version I ran eight mutation races, followed by a PPO practice update and a race for the resulting network. Both methods could supply the next parent. When a mutated runner set a new record, PPO could start from its weights. When a PPO-trained runner won, its weights became the starting point for mutation. The two methods used different information to propose a change, and the race decided whether I kept it.

What surprised me was how much of the measurable improvement came from my mutation algorithm. In one 120-second distance test the combined trainer improved a saved runner from 26.65 m to 30.06 m. Both new records came from mutated offspring. I repeated those first 52 rounds from the same saved weights, then ran the two components separately. The 47 mutation races alone reached exactly the same distance. The five PPO updates alone left the record at 26.65 m. This comparison isolates the contribution of those particular updates to that runner’s distance record.

I also compared the methods on unfamiliar obstacle courses. From the same trained runner, I gave each method two minutes per run and used five paired random seeds. On 128 layouts reserved for testing, mutation improved the average score by 38.3% and PPO by 10.6%. Mutation finished ahead in all five comparisons. The score rewarded upright forward distance with a small bonus for covering it sooner. Neither method produced a complete block clearance on those layouts. The combination gave me a practical way to train the runners. How much each method contributed depended on the starting network and what I was asking it to do.

How natural populations explore alternatives

My original algorithm selected a single parent for each generation. In a natural population several lineages can persist at once. An inherited difference that increases reproductive success tends to spread. A nearly neutral difference can also persist by chance. If a subsequent mutation is beneficial only in individuals carrying the first change, that lineage can acquire an advantage over time. My selection rule discarded offspring whose score merely equalled the parent’s, closing off that route.

In sexually reproducing populations, recombination can bring together beneficial variants that arose in different lineages. Selection can act on those combinations across many generations without a programmed cycle of tightening and widening mutation sizes. A population can still settle on a local peak, and environmental change can alter which traits confer an advantage. An experiment in yeast measured how recombination accelerated adaptation.

Sensory feedback and memory

The runner’s movements feed straight back into its next decision. The network asks a foot to move, the foot hits the floor, and on the next pass the network can sense contact, load and the change in body angle. I also fed 32 smoothed values from the second hidden layer back round as inputs. That gives it a short memory. A raised foot could be on the way up or on the way down. Joint speed and a little recent history help distinguish those two situations.

Revisiting the spider webs

When I revisited the spider webs in the browser, I used the same idea of a parent competing with its mutated children. Here they inherit the instructions for building a web. A mutation can move an anchor or put a strand across a gap. I used a different rule for the mutation range in this version. A winning child keeps the mutation size that produced it. When the parent wins, the range shrinks until it reaches the minimum. After 20 rounds at that minimum, it opens wide again.

I tested competing designs against the same fly positions and counted the catches. Rewarding catches alone favoured filling the available area with silk, so the score also accounted for the amount of silk used and the time required to build the web.

The parent and its variant face the same 40 fly positions on each scatter. Green dots are catches. Grey dots landed in gaps. Try evolving a web yourself.

A single distribution of flies could favour a poor web by chance. I compared each challenger with its parent on five fresh distributions, then checked its performance on the same 24 reference patterns before promoting it. I later faced the same question with the runners. Clearing one familiar block course said little about performance on a new layout.

Moving the block

I took the training onto my Mac so I could run more trials and replay a fall from the exact position where it happened. Here’s one of the earlier split-network runners on a course it knew. Watch the trailing foot. I only counted the block once the whole runner was past it, stayed upright beyond the edge for a second and carried on for at least another half metre.

Allrounder 03 on a familiar 25 cm block. The command stays at Walk, 0.7 metres per second. On 768 unfamiliar layouts this brain earned one clearance.

Recognising a fall while the walk command stays active

In the Mac gyms I supplied five command inputs. One is active at a time for Walk, Run, Get up, Balance left or Balance right. A sixth input carries the requested speed for walking or running. A fallen runner can still be receiving a Walk command. Recognising that it must first get up depends on what its contact and posture sensors report.

I first wrote a detector that intervened. After a fifth of a second of sustained evidence of a fall, the program supplied Get up. After half a second stable on its feet, it gave the original command back. That answered a different question from the one I was interested in. I wanted to leave Walk on and have the brain recognise that getting up was now part of walking. I left the detector in place to score and replay attempts, and took away its ability to change the live command.

I tried giving two skills room inside one network. In Allrounder 03, each hidden layer has 64 walking and running neurons and 64 recovery neurons. Each group gets 16 of the memory signals. I kept the groups’ internal connections separate so I could practise getting up while preserving the walking weights. Both groups still see the body and the command, and both feed the same 13 outputs. At the left knee, their weighted contributions add to a shared bias before the final curve. There’s no switch that declares one half in charge.

The split brain has 110 inputs and stores 32,733 coefficients. That includes 336 slots for two earlier feedback experiments, all zero in this version. Try the left knee in the explorer below. With Walk held at 1 m/s and the runner on its back, the target is 148.9°. Untick Recovery and it drops to 20.8°. Put Recovery back and choose At the block. The target is 48.6° either way. I can ask for the same walk in both places and see the recovery neurons make a large difference in one pose and none at that knee in the other. The diagrams and joint targets come from the actual trained brain, evaluated with the body and its memory held still.

Open the network explorer on its own.

I ran the old and split brains on the same 512 back starts with Walk held on. The old one never got up. The split one rose 203 times, then walked at least a metre in 42 of those trials. With just the recovery group’s connections to the outputs removed, it got up zero times in a separate 64-start check. In a later version I trained a posture estimate using four measurements. Two located the head horizontally and vertically relative to the hip. The other two measured how high the head and hip were above the ground. That let it distinguish a seated runner with its head up from one that had actually got its hips off the floor.

For an obstacle, recognising posture is only part of the job. One runner could finish its practice course with a 25 cm block. I kept the height the same and gave it 128 unfamiliar courses. It finished precisely none. Another runner, trained on a wider mix, finished 70 of 128 new courses with a 5 cm block. I started mixing up the heights and positions and putting learners back on the original starting blocks. Improving the part after the fall was only so useful if the runner had forgotten how to set off.

Getting up the other way round

For the version I called aelife5, I started a fresh PPO network and gave it 32 simulated worlds to practise in. This time I gave the network two snapshots, the previous and current 89 readings, including the last 13 actions it asked for. That makes 178 inputs. Two layers of 128 neurons feed 13 movement outputs and one training output that estimates future reward. Counting the weights and biases gives 41,230 parameters. It chooses new joint targets 30 times a second. The earlier split brain is the one in the interactive diagram.

By 20 November one version could get up from a face-down start and walk three metres in all 100 trials. From its back the same runner completed none of 100. It could lift its head and sometimes reach a seated position, but it failed to raise its hips clear of the floor.

I tried starting it partway to standing. Once it found a movement that worked, I lowered the starting angle a little and tried again. Eventually it found movements it could repeat from flat on its back. In one search it got up and walked three metres in 166 of 2,600 attempts. That gave it examples of its own successful movements to practise. Then I took the trained runner and measured what it could do on fresh starts.

My runner at training update 15,650 on 20 November. Each column has 100 trials per task.
TaskStarts used for comparisonFresh starts
Stand up from its back5857
Get up from its back and walk three metres5754
Get up from its front and walk three metres99100
Walk three metres from the runner’s start9797
Sustain a run for three metres100100
Clear at least one block9593

The opening film shows one successful recovery. The more informative result is 54 completed three-metre walks from 100 fresh back falls. The other 46 starts show how much variation the policy still struggles with. I continue to test ordinary starts and obstacle clearance alongside recovery so that improvements in one task do not conceal losses in another.

Try the runner in your browser

I’ve put a copy of my aelife5 champion from training update 29,748 in the browser runner gym. Choose Endless rising blocks and watch the trailing foot. The first block is 5 cm high, each one after it is a centimetre taller, and the gaps vary. If the runner falls, give it a moment. It can put its hands down and try to get its feet back underneath it.

You can train your own copy with mutation, PPO or a mixture of the two. Try changing how quickly the mutation range narrows when the parent wins. Replay course gives you the same gaps again. Test learner also checks what happened to its ability to get up. The gym saves your progress in this browser, and you can export the brain and import it again later.

I compiled the physics from my Mac trainer to run in the browser and tried my first public champion, from update 24,588, on the same obstacle course in both. It cleared seven blocks in 18 seconds each time, though its footsteps gradually drifted apart. The choice of brain mattered too. I gave an earlier runner and that first public champion the same 20 back falls in the browser. The earlier one never got up and walked three metres. The first public champion did it 15 times. The new starter managed it 16 times.

The Small North Face

I made a five-minute mountain film out of one of my runner’s trials for the Aetherweb Film Festival 2026. The first block is five centimetres high. By the time we get to forty, it seems only fair to hear from two walkers who didn’t make it.

A film by Jeff Snoxell · 5 minutes · English captions. Festival programme and transcript.