Clone, install, type-check: the DAX benchmark, run on Workpaper
ComputeSDK’s DAX benchmark clones OpenCode, installs its dependencies and type-checks it in a fresh sandbox. Our median of 46.7 s would sit 6th of 21 on the 9 October 2026 board. Our best run would sit 5th.
- Median total
- 46.7s
- Board position
- 6th/21
- Typecheck
- 23.3s
- Clean runs
- 3/3
Abstract
We ran ComputeSDK’s DAX script, unchanged, through ComputeSDK’s own benchmark harness on three fresh 8 vCPU / 16 GB Workpaper computers. All three runs completed every phase. The median total, 46.7 s, falls between Blaxel (45.9 s) and Beam (51.7 s) on the 9 October 2026 board; our runs ranged from 44.3 s to 49.4 s (5th to 6th). Type-checking is close to the leaders. Cloning and installing, which download from GitHub and npm, are where we lose time. This is a self-run, not a listing. We have submitted Workpaper to ComputeSDK (computesdk/computesdk#941) so their weekly runs include it.
1What DAX measures
DAX is a developer’s first minute in a new repository: clone OpenCode at a pinned commit, install it with Bun, and run its type-check across the monorepo. It mixes network (clone, install), disk (unpacking thousands of packages) and CPU (the type-check). ComputeSDK runs it weekly against every listed provider and ranks them by total time.
2Method
| Goal | Place Workpaper on the DAX board under the board’s own rules. |
|---|---|
| System | One fresh 8 vCPU / 16 GB Workpaper computer per run, created through the public SDK. |
| Workload | ComputeSDK’s dax-benchmark.sh at OpenCode commit 08fb47373509, unchanged, driven by ComputeSDK’s harness. |
| Metric | Total wall time of the script, as the harness measures it, plus each phase. |
| Design | Three runs, one at a time. The board ranks by the median of a provider’s runs; its weekly runs use one. |
| Comparison | ComputeSDK’s published results of 9 October 2026, counting only providers whose run finished every phase (20 providers). |
3Results
Each run
| Run | Clone | Install | Typecheck | Total | Position |
|---|---|---|---|---|---|
| Run 1 | 3.6 | 15.7 | 23.3 | 49.4 | 6th |
| Run 2 | 3.6 | 13.0 | 23.4 | 46.7 | 6th |
| Run 3 | 4.0 | 11.4 | 22.3 | 44.3 | 5th |
Where the time goes
| Phase (s) | Workpaper | Isorun | Namespace | Beam |
|---|---|---|---|---|
| Prepare | 2.7 | 1.2 | 4.4 | 2.8 |
| Bun download | 0.5 | 0.1 | 0.2 | 0.3 |
| Bun unpack | 0.4 | 0.5 | 0.4 | 0.6 |
| Clone | 3.6 | 1.1 | 1.0 | 2.2 |
| Install | 13.0 | 9.5 | 8.5 | 11.9 |
| Typecheck | 23.3 | 18.9 | 19.9 | 27.3 |
The type-check, the heaviest phase, takes 23.3 s against 18.9 s for Isorun. Clone and install together take 16.7 s against 10.7 s.
4Discussion
What it took. Our first attempt didn’t finish: the computer stalled during the type-check. Getting to three clean runs meant four changes, each of which every customer now gets:
- Memory limits sized for the server. A cap meant for a smaller machine throttled the 16 GB computer until it stopped responding.
- Huge pages for large computers. Backing guest memory with 2 MB pages cut the type-check from about 40 s to about 22 s.
- RAM-backed
/tmpwith swap behind it. Fast deletes of a 3 GBnode_modules, without the out-of-memory kills we saw with RAM alone. - Package lists kept in the image.
apt-get installno longer downloads the full index first, which halved the prepare phase.
What’s left. Clone and install fetch from GitHub and npm. Providers whose servers sit next to those services finish them faster; that gap is distance, not compute, and install time varied by more than 4 s between our runs.
5Threats to validity
Construct: is total time what users feel?
It is the board’s metric: wall time of the whole script, as the harness sees it, including the final cleanup of the 3 GB working tree. It excludes creating the computer.
Internal: could the setup explain the result?
We ran the harness from our own machine, not ComputeSDK’s runners, five days after the reference results. The script and the scoring are theirs and unchanged.
External: will the weekly run match?
Their scheduled run uses one attempt. With our spread of 44.3 to 49.4 s, a single week can land anywhere from 5th to 6th. Other providers also change week to week.
Conclusion: are the gaps real?
Three runs support a range, not a precise position. The gap to Isorun (34.5 s) is real; the gap to our neighbours is within our own run-to-run spread.
6Reproducibility
Every phase of every run, as the harness wrote it, is on GitHub.