RLDQN-lite / Q-learning
Reinforcement Learning
Learns by trial and error to maximize reward (food + survival).
Six paradigms · one arena
Same eyes. Same moves.Six ways to learn.
Every agent here sees an identical 13-number view of the world and picks from the same eight directions. The only thing that differs is how its policy was learned. They were trained separately in Python, then dropped into one arena to forage, hunt each other, and race to level 10 — so the match is a controlled comparison of learning methods, not of hand-tuned behaviour.
- Held constant13-feature observation
- Held constant8 discrete actions
- The variableHow the policy was learned
01
The arena
Press Start and watch. Eat pellets to grow, swallow anything smaller than you, avoid anything bigger. First to level 10 — or the last one alive — takes the match.
step 06 / 6 aliveseed
—Live standings
| Agent | Lvl | Score | Progress to 10 |
|---|
Tournament
Every finished match is counted here.
| Agent | Wins | Rate | Share |
|---|
No matches played yet.
02
The six contestants
One colour per paradigm, used consistently everywhere on this page — in the arena, the standings and the meters. Each was trained solo against food, prey and hazards before ever meeting the others.
SUPBehavioral cloning
Supervised (Imitation)
Copies an expert's state→action examples. Only as good as its teacher.
SELFWorld model + planning
Self-Supervised
Learns a world model (predicts the next state), then plans one step ahead.
SEMIPseudo-labeling
Semi-Supervised
Trained on a few expert labels plus lots of pseudo-labeled data. Between imitation and RL.
EVOGenetic algorithm
Evolutionary
A population of networks evolves over generations; the fittest reproduce with mutations.
UNSk-means clustering
Unsupervised
No reward. Clusters food with k-means and heads to the best nearby cluster; flees threats.
03
How it works
Training happens offline in Python; the browser only runs inference. Nothing is fetched at runtime — the weights ship with the page.
1
Train in Python
Each paradigm learns solo in a small numpy arena. Training the whole roster takes about a minute — and training separately sidesteps the instability of multi-agent competitive RL.
2
Export the policies
Every learned policy is written to policies.json — network weights, or a config for the two heuristic agents.
3
Ship the weights
The page imports those weights directly and runs the forward pass in a few dozen lines of plain JavaScript. No backend, no inference server, nothing fetched at runtime.
4
Prove the port is faithful
The browser's answers are checked against Python's on 400 random states, for all five networked agents, on every push.
0mismatches between browser and Python inference across 400 states
7e-16largest numerical difference — floating-point noise, nothing more
173 → 14worst “spinning in place” streak, before and after the anti-stuck rule
2.5–20×how far every exported paradigm beats a random baseline
04
Who actually wins
One match is an anecdote. These are cumulative win rates over 300 seeded matches, run headlessly with the same engine this page uses.
300 seeded matches
| Paradigm | Wins | Share |
|---|---|---|
| SUP | 80% | |
| SEMI | 6% | |
| RL | 6% | |
| EVO | 4% | |
| UNS | 2% | |
| SELF | 1% |
Reproduce it yourself: node run_many.js --games 300 --seed 1
A note on that result
Imitation learning wins most games, and that is an honest finding rather than a bug. Copying a near-optimal expert is very hard to beat at a task that rewards both efficient foraging and opportunistic hunting — the expert already solved both.
It is also the sharpest lesson the arena teaches: a paradigm's ranking here says as much about the problem as about the method. Change the task so no expert exists to copy, and the ordering changes with it.
Random food, random starting positions and a touch of exploration noise still produce regular upsets from every other paradigm — which is what the seed controls are for.
1 seedis a complete description of a match —
?seed= reproduces it anywhere, bit for bitheadless test suites drive the exact engine module this page runs
0runtime dependencies — nothing is fetched to play a match