100 computers at once: Workpaper on ComputeSDK's Sandbox Burst TTI

Burst TTI score

90.3

Started

100/100

Median

0.95s

On the node

0.36s

ComputeSDK's Sandbox Burst TTI is the closest thing sandbox providers have to a shared scoreboard. It asks for 100 sandboxes in the same instant and times each one from create() until its first command, node -v, comes back. Twenty-seven providers are on it, and it reruns every day.

It is a harder test than it looks. Starting one computer quickly is easy. Starting a hundred at once, on the same machine, without any of them failing, is an orchestration problem: everything that is cheap once becomes a queue when it happens a hundred times in the same second.

How is the Sandbox Burst TTI score calculated?

Each run produces 100 times. The fastest and slowest five percent are trimmed, then the median, the 95th percentile and the 99th percentile are scored against a ten-second ceiling and blended:

score = (0.6 · median + 0.25 · p95 + 0.15 · p99) × share started
         where each term is 100 × (1 − t / 10 s)

The last factor is the one that matters most. A provider that loses one sandbox in a hundred loses a full point, more than a hundred milliseconds of median speed is worth. Reliability is not a footnote here; it multiplies everything else.

Workpaper's Sandbox Burst TTI result

We ran ComputeSDK's harness unchanged apart from adding Workpaper as a provider, with the same flags as their daily job: 100 iterations at a concurrency of 100.

The spread is the interesting part. Every one of the hundred computers finished within about fifty milliseconds of each other. Nothing queued behind anything else; each computer paid the same fixed cost, and almost all of that cost was distance, which we come back to below.

How Workpaper starts 100 sandboxes at once

Computers are already running before you ask

Workpaper keeps a pool of computers booted and waiting. A create request is answered by claiming one, close to the caller, and a replacement starts behind it. A burst of a hundred never waits on a hundred cold boots, and the pool is back to full within seconds.

Memory that doesn't stall the first command

Each computer resumes from a snapshot. Resuming normally pulls memory back in one small page at a time, the first time a program touches it, which is why a computer's first command is usually its slowest. We back computers with large memory pages instead, so a resumed computer's memory arrives in big pieces. A fresh computer's first node -v went from about 300 ms to 8 ms.

Bursts wait their turn instead of failing

When a hundred requests land in the same instant, admission control holds them in a fair queue and starts them as fast as the node allows, instead of turning some away and making the caller retry. That is how every run in our testing started all 100.

How fast is Workpaper from India?

The benchmark measures one specific thing: a hundred computers at once, called from a runner in the United States. Most of our users are in India and start one computer at a time. From a laptop in India, a Workpaper computer is running its first command in about half a second. That is the service you sign up for today, at ₹1 an hour; the quickstart gets you there in one command.

What limited this result

• One node. All hundred computers ran on a single machine. The providers above us spread a burst across fleets.
• An ocean in the way. The runner was in Texas and the node in Europe, so every first command crossed the Atlantic and back. On the node itself the same burst took 0.36 s.
• A test deployment. The run used capacity set up for it. Workpaper's production footprint today is smaller and sits in India.
• Self-run. ComputeSDK's own daily runs use their own runners. Read our placing as an estimate, not a ranking. We are not on their official board yet.

Method and how to reproduce it