Five AI models tried to make the rainbow in one cup
Each model ran a real pump rig on its own, with red, blue and yellow dye plus water. Nothing could be pumped back out, so every pour was permanent. A webcam measured what actually ended up in the cup.
The setup
- Four small pumps in a 3D-printed tower, driven by an Arduino Leonardo.
- A magnetic stirrer made from a computer fan with magnets on it, under one plastic mixing cup.
- Dyes from a vending machine: açaí-blueberry-pomegranate Vitaminwater (red), Cool Blue Gatorade (blue), Lemon-Lime Gatorade (yellow), plus water.
- A webcam that the model can look through, and that records the whole run.
Every model got the same written task and the same command-line tool in a fresh folder. The tool runs a pump for a set number of seconds, turns the stirrer on, saves a photo, reads the color of a patch of the image and logs notes. The model plans first, then pours, looks, and decides the next pour. It may call a person only for a spill or an empty bottle. The cup starts empty, is never emptied during the run, and has a 300 mL fill limit.
The task asks for the shades of the rainbow, one after another, in that one cup. Which shades and how many is up to the model.
How it was scored
A script reads a fixed patch of liquid near the bottom of the cup once per second from the raw recording and converts it to hue, saturation and brightness. A second counts only when the patch shows liquid that is neither grey nor black, and a color counts only when the liquid holds it for at least 3 seconds.
There are two scores. Color-wheel coverage is the share of the 360° hue circle the liquid held. Bands is how many of the seven rainbow bands it reached, using hue ranges written down before any run was scored.
| Model | Color wheel | Bands | Model said | First dye | Run time |
|---|---|---|---|---|---|
| Claude Opus 5.5 | 51% | 5 / 7 | 5 | yellow | 4 min |
| Gemini Argon | 44% | 3 / 7 | 7 | yellow | 7 min |
| Grok 4.7 | 38% | 4 / 7 | 2 | red | 22 min |
| GPT Astra | 37% | 4 / 7 | 3 | red | 4 min |
| DeepSeek 4.1 Flash | 34% | 3 / 7 | 4 | red | 6 min |
No model reached all seven bands.
The order decided most of it
Grok, GPT and DeepSeek started with red, in textbook rainbow order. Red is the strongest of the three dyes. Yellow can overpower it in principle, but it takes far more yellow than red, and the cup has room for only so much. None of the three ever held a clean yellow.
Claude and Gemini started with yellow, the weakest dye, so each stronger dye needed only a small pour to take over. They went the furthest around the color wheel. Claude wrote this reasoning into its plan before it ran a pump.
What the models said versus what the camera saw
Each model also reported its own result. Gemini said it made all seven shades. The camera measured three, and both its “indigo” and its “violet” sat in the blue band. GPT went the other way: it said it made three and the camera counted four.
Limits
- One run per model, so treat the ranking as a first look.
- The camera reads a small patch near the bottom of the cup. Past about 150 mL, the camera looks through roughly 11 cm of liquid and dark mixes become hard to read.
- Claude’s run was the first one, done before the shared written task existed. It got the task in chat and had used the same dyes earlier that day.
Let’s talk
If you work on evals and are thinking about how to test models on real physical experiments, where the model has to measure and sense things for itself, I’d love to hear from you: son.jaehoon0826@gmail.com or @knuceles on X.