{"about":"Sandbox bench: cloud sandbox providers for AI agents, timed through one SDK (Sandbox SDK, built by us; 0.2.0 is public on npm and the 1.0 used here will be published soon) from one server in Florida, US, on two tracks: each provider's default machine, and the same 4 vCPU, 8 GB machine. Medians, p95s and 95% confidence intervals per provider; every provider-side failure kept, failures caused by our own code set aside.","howToRead":"Pick a track first: `tracks.default` compares each provider's default sandbox, so sizes differ; `tracks[\"same-spec\"]` asks every provider for 4 vCPU and 8 GB. In same-spec, `allocation.equalSpec: false` means the machine measured inside was not 4 vCPU, 8 GB, so compare that provider with care. median and p95 are in `unit`; `better` says which direction wins; results are sorted best first. attempts counts every try, failed included. A null median means the provider does not support the test, every attempt failed, or the result is excluded (see `status`). Cold start is the provider's own time, create() until the first command; `sdkReady` adds our SDK's setup and is ours, not the provider's. `costPerTask.kind` is billed (provider charge), metered (provider usage at published rates) or estimate (list price, where billing gives no amount); a task is one sandbox that is created, runs the 100-call tool replay and is deleted, from a separate cost run: never compare them as if they were the same kind without saying so.","task":"A task is: create a sandbox, run the 100-call tool replay, delete it. Cost per task comes from a separate cost run, billed per sandbox.","commonSize":"4 vCPU, 8 GB","comingNextUpdate":["Boat","Snapshot and restore","Bursts of 10 and 25","The same-machine (4 vCPU, 8 GB) track"],"tracks":{"default":{"id":"default","label":"Default machines","description":"Each provider's default sandbox, so machine sizes differ.","published":true,"run":{"source":"m-default","start":"2026-10-22T08:22:01.273Z","end":"2026-10-22T15:59:29.329Z","runs":["sbx53-default-main-r3","sbx53-default-snapshot-r5","sbx53-default-cost-r5","sbx53-default-burst-r5","sbx53-default-burst-r6"],"observations":12311},"absent":[{"provider":"boat","reason":"coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that."}],"providers":[{"id":"e2b","name":"E2B","allocation":{"requested":{"size":null,"template":"base","compute":{"cpu":2,"memoryGiB":0.5},"provisioned":{"cpu":2,"memoryGiB":0.5},"text":"2 vCPU, 512 MB, template base"},"measured":[{"cpu":2,"memoryGiB":0.462,"diskGiB":21.535}],"measuredText":"2 vCPU, 473 MB","equalSpec":false,"verified":true,"mismatches":[],"pricedAt":{"cpu":2,"memoryGiB":0.5}},"allocationNote":"E2B's default base template is 2 vCPU and 512 MB. E2B's pricing page says its default sandbox has 4 GB, so E2B ran smaller here than that.","advertisedDefault":{"vcpu":2,"memoryGiB":0.5,"diskGiB":21.534767150878903,"source":"https://e2b.dev/pricing"},"ranIn":"The Dalles, US","cpu":"Intel Xeon Processor @ 2.60GHz","hardware":{"chip":"Intel Xeon Platinum 8373C","family":"Ice Lake, 2.6 GHz, 54 MB L3","chipBasis":"inferred","memory":{"type":"DDR4","basis":"inferred","detail":"DDR4-3200, the chip's memory type"},"cloud":"Google Cloud","region":"us-west1, The Dalles, Oregon","runsAs":"Firecracker microVM, nested inside a Google Cloud VM","hidden":["The CPU name, microcode and clock leaves are hidden. The chip is identified from its model number, cache size and 2.6 GHz clock.","The default 2 vCPUs are one physical core with two hyperthreads, so two-thread work scales well below twice one thread.","No DMI or PCI devices, so the memory type can't be read inside; DDR4 follows from the chip."],"probed":"Oct 22, 2026"},"sdkReady":{"median":1.33,"p95":4.592,"sdkSetupMedian":0.778,"coldStarts":100},"costPerTask":{"kind":"metered","perTaskUsd":0.0004933,"per1000TasksUsd":0.4933,"confidence":"medium","tasks":10,"source":"https://api.e2b.app/events/sandboxes/{id}","basis":"Metered over 10 tasks, medium confidence"},"price":{"currency":"USD","perHour":0.109,"perVcpuHour":0.054,"perHourUsd":0.109,"commonSizeUsd":0.331,"note":"10 GiB Hobby / 20 GiB Pro included disk. Subscription excluded. Per-second billing; no separate creation fee listed.","usdNote":null,"source":"https://e2b.dev/pricing"},"pause":{"supported":true,"split":{"pause":0.2633845460000448,"resume":0.45188361200038346,"n":100}},"reliability":{"attempts":1076,"succeeded":1076,"successRate":1},"failures":[],"setAside":[]},{"id":"daytona","name":"Daytona","allocation":{"requested":{"size":null,"template":null,"compute":{"cpu":1,"memoryGiB":1},"provisioned":{"cpu":1,"memoryGiB":1,"diskGiB":3},"text":"1 vCPU, 1 GB"},"measured":[{"cpu":1,"memoryGiB":1,"diskGiB":3}],"measuredText":"1 vCPU, 1 GB","equalSpec":false,"verified":true,"mismatches":[],"pricedAt":{"cpu":1,"memoryGiB":1}},"allocationNote":null,"advertisedDefault":{"vcpu":1,"memoryGiB":1,"diskGiB":3,"source":"https://www.daytona.io/docs/en/sandboxes/"},"ranIn":null,"cpu":"AMD EPYC 9354P 32-Core Processor","hardware":{"chip":"AMD EPYC 9354P, 9354, 9254 or 9275F","family":"Zen 4 (Genoa); the 9275F is Zen 5 (Turin). Hosts vary","chipBasis":"named by the CPU","memory":{"type":"DDR5","basis":"measured","detail":"Registered DDR5, read from the memory controller on a 9254 host"},"cloud":"Bare-metal servers","region":"Location not visible; control plane on AWS us-east-1","runsAs":"Docker container, shares the host's kernel; cgroups set its CPU and RAM","hidden":["Nothing at the CPU level is hidden: the real CPU name, microcode, server model (Dell PowerEdge R6615 in the probe, a Supermicro AS-2115GT-HNTR in the benchmark) and disks are visible.","nproc and lscpu show every host thread (48 or 64) and /proc/meminfo shows the host's RAM, while the default sandbox may use 1 CPU and 1 GB. Tools that size thread pools or caches from those numbers oversubscribe.","Hosts differ in clock speed, so single-thread results vary from sandbox to sandbox. Neighbours share the host kernel.","Outbound internet is blocked on our account tier, so the sandbox's own network location could not be read."],"probed":"Oct 22, 2026"},"sdkReady":{"median":5.822,"p95":12.701,"sdkSetupMedian":2.638,"coldStarts":100},"costPerTask":{"kind":"billed","perTaskUsd":0.0002428,"per1000TasksUsd":0.2428,"confidence":"medium","tasks":10,"source":"https://www.daytona.io/pricing","basis":"Billed over 10 tasks, medium confidence"},"price":{"currency":"USD","perHour":0.067,"perVcpuHour":0.067,"perHourUsd":0.067,"commonSizeUsd":0.331,"note":"Hourly published rates converted to seconds. Assumed one-second increment; minimum and creation fee not stated. Snapshot storage excluded.","usdNote":null,"source":"https://www.daytona.io/pricing"},"pause":{"supported":false,"note":"Daytona's container sandboxes have no pause: stop and start restarts the sandbox and ends every running process."},"reliability":{"attempts":973,"succeeded":973,"successRate":1},"failures":[],"setAside":[]},{"id":"upstash","name":"Upstash Box","allocation":{"requested":{"size":null,"template":null,"compute":{"cpu":2,"memoryGiB":4},"provisioned":{"cpu":2,"memoryGiB":4,"diskGiB":5},"text":"2 vCPU, 4 GB"},"measured":[{"cpu":2,"memoryGiB":4,"diskGiB":5}],"measuredText":"2 vCPU, 4 GB","equalSpec":false,"verified":true,"mismatches":[],"pricedAt":{"cpu":2,"memoryGiB":4}},"allocationNote":null,"advertisedDefault":{"vcpu":2,"memoryGiB":4,"diskGiB":5,"source":"https://upstash.com/pricing/box"},"ranIn":"Ashburn, US","cpu":null,"hardware":{"chip":"AWS Graviton4","family":"Arm Neoverse V2, 2.8 GHz; Graviton4 from the m8g instance type","chipBasis":"inferred","memory":{"type":"DDR5","basis":"inferred","detail":"DDR5-5600, from AWS's Graviton4 spec"},"cloud":"AWS EC2","region":"us-east-1, Ashburn, Virginia","runsAs":"Docker container, on a shared m8g.16xlarge VM; cgroups set its CPU and RAM","hidden":["The only Arm provider here: x86-only binaries don't run, and compile and run times differ for that reason alone.","nproc and /proc/meminfo show the host's 64 CPUs and 256 GB, while a box may use 2 CPUs and 4 GB (4 and 8 at the medium size).","No DMI tables or clock readings inside, so memory type and clock come from AWS's published spec.","Neighbours share the host kernel and the chip's L3 cache."],"probed":"Oct 22, 2026"},"sdkReady":{"median":11.946,"p95":30.909,"sdkSetupMedian":7.274,"coldStarts":100},"costPerTask":{"kind":"billed","perTaskUsd":0.0002318,"per1000TasksUsd":0.2318,"confidence":"high","tasks":10,"source":"https://us-east-1.box.upstash.com/v2/box/{id}","basis":"Billed over 10 tasks, high confidence"},"price":{"currency":"USD","perHour":0.1,"perVcpuHour":0.05,"perHourUsd":0.1,"commonSizeUsd":0.2,"note":"PAYG ACTIVE CPU billing: Small $0.10, Medium $0.20, Large $0.40 per active box-hour; tier price × wall time is an upper-bound estimate, not metered usage. Disk allocated capacity is an upper bound for written data. 30-day month. No RAM fee listed. Increment/minimum unspecified.","usdNote":null,"source":"https://upstash.com/pricing/box"},"pause":{"supported":true,"split":{"pause":0.7307477690000087,"resume":0.5890358970002271,"n":100}},"reliability":{"attempts":1070,"succeeded":1067,"successRate":0.9972},"failures":[{"kind":"provider-operation","reason":"Upstash Box exec failed (provider). (provider)","count":1,"runs":1,"tests":["Agent tool replay"]},{"kind":"provider-operation","reason":"Upstash Box files.write failed (unavailable, HTTP 500). (unavailable, HTTP 500, operation=files.write)","count":1,"runs":1,"tests":["npm install"]},{"kind":"provider-operation","reason":"Guest exit 143: Cloning into '/workspace/bench-clone'...","count":1,"runs":1,"tests":["git clone"]}],"setAside":[]},{"id":"boxd","name":"boxd","allocation":{"requested":{"size":null,"template":null,"compute":{"cpu":2,"memoryGiB":8},"provisioned":{"cpu":2,"memoryGiB":8,"diskGiB":100},"text":"2 vCPU, 8 GB"},"measured":[{"cpu":2,"memoryGiB":7.777,"diskGiB":97.872}],"measuredText":"2 vCPU, 7.8 GB","equalSpec":false,"verified":true,"mismatches":[],"pricedAt":{"cpu":2,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":{"vcpu":2,"memoryGiB":8,"diskGiB":100,"source":"https://boxd.sh/"},"ranIn":"Frankfurt am Main, DE","cpu":"AMD EPYC","hardware":{"chip":"AMD EPYC 9554P, likely","family":"Zen 4 (Genoa), 3.1 GHz","chipBasis":"inferred","memory":{"type":"DDR5","basis":"inferred","detail":"DDR5-4800, the Genoa platform's memory"},"cloud":"Latitude.sh bare metal","region":"Frankfurt, Germany","runsAs":"Firecracker microVM, with Docker installed inside the VM","hidden":["The CPU name is trimmed to \"AMD EPYC\" and the microcode and topology are hidden. The chip is inferred from its 3.1 GHz clock and the Genoa servers Latitude.sh sells; a dual-socket 9554 can't be ruled out.","No DMI, so the memory type is inferred.","AVX-512 is exposed, which can help vectorised work against hosts that hide it.","The disk reports itself as rotational although it is virtual, so tools that tune for hard drives may behave differently."],"probed":"Oct 22, 2026"},"sdkReady":{"median":3.197,"p95":6.998,"sdkSetupMedian":1.794,"coldStarts":100},"costPerTask":{"kind":"metered","perTaskUsd":0.001001,"per1000TasksUsd":1.001,"confidence":"medium","tasks":10,"source":"boxd orgs.usage / orgs.billingActivityPage","basis":"Metered over 10 tasks, medium confidence"},"price":{"currency":"EUR","perHour":0.174,"perVcpuHour":0.087,"perHourUsd":0.195,"commonSizeUsd":null,"note":"EUR; RAM in use and written disk billed, so allocated resources are an upper bound. Metered per second, settled every 15 minutes (not a rounding increment). FX budget uses conservative 2 USD/EUR, not a market quote.","usdNote":"Converted at the ECB rate of 1.1234 USD per EUR on October 21, 2026.","source":"https://boxd.sh/pricing"},"pause":{"supported":true,"split":{"pause":0.1264708259999752,"resume":0.412353187000379,"n":100}},"reliability":{"attempts":1073,"succeeded":1072,"successRate":0.99907},"failures":[{"kind":"provider-operation","reason":"boxd request failed. (provider)","count":1,"runs":1,"tests":["Write + read 10 MB"]}],"setAside":[{"reason":"Our ownership check failed during one burst of 25 (16 creates, 5 deletes), so that burst is not counted","count":21,"runs":1}]},{"id":"freestyle","name":"Freestyle","allocation":{"requested":{"size":null,"template":null,"compute":{"cpu":4,"memoryGiB":8},"provisioned":{"cpu":4,"memoryGiB":8,"diskGiB":32},"text":"4 vCPU, 8 GB"},"measured":[{"cpu":4,"memoryGiB":7.755,"diskGiB":31.201}],"measuredText":"4 vCPU, 7.8 GB","equalSpec":true,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":{"vcpu":4,"memoryGiB":8,"diskGiB":32,"source":"https://www.freestyle.sh/docs/guides/run-hexclave-in-a-sandbox"},"ranIn":"San Francisco, US","cpu":null,"hardware":{"chip":"AMD EPYC 7763, likely","family":"Zen 3 (Milan), 2.45 GHz","chipBasis":"inferred","memory":{"type":"DDR4","basis":"inferred","detail":"DDR4-3200, the Milan platform's memory"},"cloud":"Freestyle's own network (AS36320)","region":"San Francisco Bay Area","runsAs":"Firecracker microVM, 4 vCPU, 8 GB, no hyperthreads exposed","hidden":["The CPU name, microcode and topology are hidden. The chip is inferred from its 2.45 GHz clock, cache and Zen 3 features.","Memory is added after boot: about 3 GB is free at first, 7.9 GB later.","No DMI, so the memory type is inferred."],"probed":"Oct 22, 2026"},"sdkReady":{"median":1.617,"p95":4.71,"sdkSetupMedian":0.969,"coldStarts":100},"costPerTask":{"kind":"estimate","perTaskUsd":0.001707,"per1000TasksUsd":1.707,"confidence":"low","tasks":10,"source":"https://www.freestyle.sh/pricing","basis":"Estimate over 10 tasks, low confidence"},"price":{"currency":"USD","perHour":0.264,"perVcpuHour":0.066,"perHourUsd":0.264,"commonSizeUsd":0.264,"note":"Published overage rates, allowances/subscriptions excluded. $0.02/GB transfer excluded. Billing increment/minimum and creation fees unspecified; one-second estimate.","usdNote":null,"source":"https://www.freestyle.sh/pricing"},"pause":{"supported":true,"split":{"pause":0.2712142259999964,"resume":0.32223399200011044,"n":100}},"reliability":{"attempts":1068,"succeeded":1068,"successRate":1},"failures":[],"setAside":[{"reason":"Our SDK timed out or wrapped the error moving 10 MB, so we can't tell whose side failed","count":8,"runs":8}]},{"id":"tenki","name":"Tenki","allocation":{"requested":{"size":null,"template":null,"compute":null,"provisioned":null,"text":"provider default"},"measured":[{"cpu":2,"memoryGiB":3.797,"diskGiB":4.84}],"measuredText":"2 vCPU, 3.8 GB","equalSpec":false,"verified":null,"mismatches":[],"pricedAt":{"cpu":2,"memoryGiB":3.796802520751953}},"allocationNote":"Tenki documents a 5 GiB default disk but not a default vCPU or RAM, so those are what we measured.","advertisedDefault":{"vcpu":2,"memoryGiB":3.796802520751953,"diskGiB":5,"source":"https://tenki.cloud/docs/sandbox/cli.md"},"ranIn":"Seattle, US","cpu":"AMD EPYC 7B13 64-Core Processor","hardware":{"chip":"AMD EPYC 7B13, as reported","family":"Zen 3 (Milan), 2.25 GHz. The 4 vCPU VM also showed EPYC 9575F (Zen 5, Turin) cores","chipBasis":"conflicting","memory":{"type":null,"note":"No memory data inside the VM"},"cloud":"Luxor (AS19538)","region":"Seattle","runsAs":"Cloud Hypervisor microVM, with a container inside","hidden":["On the 4 vCPU VM, two vCPUs report an EPYC 7B13 (Milan) and two an EPYC 9575F (Turin). That fits a snapshot taken on a Milan host and restored on a Turin one, so the real host may be newer than the name says.","CPU features are capped at the Milan set on every vCPU, so no AVX-512 even on Turin.","The clock reads 2.25 GHz, the 7B13's base, which may be carried over from the snapshot."],"probed":"Oct 22, 2026"},"sdkReady":{"median":1.959,"p95":6.494,"sdkSetupMedian":1.043,"coldStarts":100},"costPerTask":{"kind":"metered","perTaskUsd":0.000897,"per1000TasksUsd":0.897,"confidence":"medium","tasks":10,"source":"https://tenki.cloud/pricing","basis":"Metered over 10 tasks, medium confidence"},"price":{"currency":"USD","perHour":0.162,"perVcpuHour":0.081,"perHourUsd":0.162,"commonSizeUsd":0.331,"note":"Per-second, no minimum. Separate persistent storage $0.000000076/GiB-second excluded. Credits excluded.","usdNote":null,"source":"https://tenki.cloud/pricing"},"pause":{"supported":true,"split":{"pause":55.56055400800053,"resume":2.2910779680004345,"n":100}},"reliability":{"attempts":1075,"succeeded":1074,"successRate":0.99907},"failures":[{"kind":"provider-operation","reason":"Tenki request failed (not_found). (not_found, 5, operation=snapshot)","count":1,"runs":1,"tests":["Take snapshot"]}],"setAside":[{"reason":"Our cleanup check could not confirm the sandbox after Tenki's snapshot failed, so we can't tell whose side the delete failure was on","count":2,"runs":1}]},{"id":"superserve","name":"Superserve","allocation":{"requested":{"size":null,"template":"sandbox-bench-node22-py311-1x1-pinned","compute":{"cpu":1,"memoryGiB":1},"provisioned":{"cpu":1,"memoryGiB":1,"diskGiB":4},"text":"1 vCPU, 1 GB, template sandbox-bench-node22-py311-1x1-pinned"},"measured":[{"cpu":1,"memoryGiB":0.971,"diskGiB":3.859}],"measuredText":"1 vCPU, 994 MB","equalSpec":false,"verified":true,"mismatches":[],"pricedAt":{"cpu":1,"memoryGiB":1}},"allocationNote":"Superserve ran a template we built with Node 22 and Python 3.11 preinstalled, at its default 1 vCPU, 1 GB RAM and 4 GB disk. Superserve's own templates have Node (node-22) or Python (python-3.11), not both, and our tests need both.","advertisedDefault":{"vcpu":1,"memoryGiB":1,"diskGiB":4,"source":"https://docs.superserve.ai"},"ranIn":"Washington, US","cpu":"Intel Xeon Processor","hardware":{"chip":"Intel Xeon, Sapphire Rapids","family":"Likely Xeon Platinum 8481C; 2.2 GHz, 105 MB L3","chipBasis":"inferred","memory":{"type":"DDR5","basis":"inferred","detail":"DDR5, the Sapphire Rapids platform's memory"},"cloud":"Google Cloud","region":"us-east4, Northern Virginia, from its IP range","runsAs":"Firecracker microVM, likely inside a Google Cloud VM","hidden":["The CPU name and microcode are hidden. The chip family is clear from its model number; the exact SKU is not.","The old 4.14 guest kernel can't turn on the chip's AMX units, so AMX workloads can't use them.","No DMI, so the memory type is inferred."],"probed":"Oct 22, 2026"},"sdkReady":{"median":1.335,"p95":6.115,"sdkSetupMedian":0.811,"coldStarts":100},"costPerTask":{"kind":"estimate","perTaskUsd":0.0002484,"per1000TasksUsd":0.2484,"confidence":"low","tasks":10,"source":"https://www.superserve.ai/pricing","basis":"Estimate over 10 tasks, low confidence"},"price":{"currency":"USD","perHour":0.067,"perVcpuHour":0.067,"perHourUsd":0.067,"commonSizeUsd":0.331,"note":"Published vCPU and GiB-hour rates converted to seconds. Credits excluded. One-second rounding assumed; snapshot storage, disk, transfer and any minimum charge excluded. Ran on our template sandbox-bench-node22-py311-1x1-pinned (Node 22 and Python 3.11 preinstalled), 1 vCPU, 1 GiB RAM, 4 GiB disk; resources are template-owned. Cold start ends at the provider's first command; our SDK's setup is reported separately as SDK-ready.","usdNote":null,"source":"https://www.superserve.ai/pricing"},"pause":{"supported":true,"split":{"pause":0.35829360100021584,"resume":0.5446929919999093,"n":100}},"reliability":{"attempts":1076,"succeeded":1076,"successRate":1},"failures":[],"setAside":[]}],"quickPicks":[{"need":"Fastest cold start","pick":"Superserve 0.51 s","runnerUp":"E2B 0.56 s"},{"need":"Fastest agent work (tool replay)","pick":"Daytona 11 s","runnerUp":"Superserve 12 s"},{"need":"Fastest 25-sandbox fan-out","pick":"Superserve 0.79 s","runnerUp":"E2B 1.0 s"},{"need":"Fastest reconnect","pick":"Daytona 0.33 s","runnerUp":"Superserve 0.34 s"},{"need":"Cheapest task","pick":"Upstash Box $0.00023 per task, $0.23 per 1,000 (billed)","runnerUp":"Daytona $0.00024 per task, $0.24 per 1,000 (billed)"},{"need":"Most reliable","pick":"E2B, Daytona, Freestyle, Superserve 100.00% (tied)","runnerUp":"Tenki 99.91%"}],"metrics":[{"id":"cold","group":"Start-up","label":"Cold start","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.505,"p95":3.444,"ci95":[0.5,0.526],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":0.559,"p95":3.193,"ci95":[0.553,0.571],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.627,"p95":1.423,"ci95":[0.613,0.644],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.874,"p95":4.13,"ci95":[0.847,0.892],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":1.373,"p95":4.204,"ci95":[1.355,1.383],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":1.558,"p95":5.785,"ci95":[1.364,1.81],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":2.962,"p95":22.503,"ci95":[2.655,4.244],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"tool-replay","group":"Agent work","label":"Agent tool replay","unit":"s","better":"lower","results":[{"provider":"daytona","median":10.955,"p95":31.325,"ci95":[10.387,13.967],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":11.99,"p95":14.614,"ci95":[11.82,12.653],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":14.753,"p95":16.989,"ci95":[14.506,15.054],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":17.483,"p95":21.31,"ci95":[16.714,17.999],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":19.591,"p95":23.17,"ci95":[19.353,20.241],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":24.053,"p95":30.038,"ci95":[23.579,24.663],"attempts":30,"failed":1,"status":"measured","equalSpec":false},{"provider":"boxd","median":31.132,"p95":33.655,"ci95":[30.728,31.522],"attempts":30,"failed":0,"status":"measured","equalSpec":false}]},{"id":"npm-install","group":"Agent work","label":"npm install","unit":"s","better":"lower","results":[{"provider":"upstash","median":0.971,"p95":1.061,"ci95":[0.967,0.981],"attempts":30,"failed":1,"status":"measured","equalSpec":false},{"provider":"daytona","median":1.135,"p95":1.78,"ci95":[1.044,1.223],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":1.23,"p95":1.854,"ci95":[1.199,1.665],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":1.414,"p95":2.068,"ci95":[1.36,1.619],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":1.445,"p95":2.235,"ci95":[1.08,1.597],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":1.617,"p95":7.147,"ci95":[1.543,1.682],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.939,"p95":2.612,"ci95":[1.851,2.13],"attempts":30,"failed":0,"status":"measured","equalSpec":false}]},{"id":"pip-install","group":"Agent work","label":"pip install","unit":"s","better":"lower","results":[{"provider":"tenki","median":3.234,"p95":5.843,"ci95":[2.945,4.62],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":3.582,"p95":4.163,"ci95":[3.371,3.971],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":3.652,"p95":4.154,"ci95":[3.61,3.718],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":4.273,"p95":4.683,"ci95":[4.221,4.321],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":4.375,"p95":4.87,"ci95":[4.272,4.533],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":4.623,"p95":5.988,"ci95":[4.572,4.793],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":7.342,"p95":10.432,"ci95":[7.337,7.348],"attempts":30,"failed":0,"status":"measured","equalSpec":true}]},{"id":"git-clone","group":"Agent work","label":"git clone","unit":"s","better":"lower","results":[{"provider":"daytona","median":0.452,"p95":0.998,"ci95":[0.438,0.719],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":0.545,"p95":0.893,"ci95":[0.53,0.552],"attempts":30,"failed":1,"status":"measured","equalSpec":false},{"provider":"e2b","median":1.029,"p95":1.989,"ci95":[1.004,1.118],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":1.102,"p95":1.469,"ci95":[1.056,1.155],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":1.284,"p95":1.784,"ci95":[1.258,1.332],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":1.341,"p95":2.511,"ci95":[1.065,1.498],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":1.465,"p95":1.721,"ci95":[1.46,1.517],"attempts":30,"failed":0,"status":"measured","equalSpec":false}]},{"id":"dev-server","group":"Agent work","label":"Dev server ready","unit":"s","better":"lower","results":[{"provider":"superserve","median":1.644,"p95":2.652,"ci95":[1.627,1.664],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":1.846,"p95":3.105,"ci95":[1.794,1.921],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":1.869,"p95":4.921,"ci95":[1.811,1.926],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":2.32,"p95":2.958,"ci95":[2.169,2.353],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":2.954,"p95":8.557,"ci95":[2.761,3.937],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":3.156,"p95":4.809,"ci95":[3.058,3.435],"attempts":29,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":3.708,"p95":4.31,"ci95":[3.661,3.724],"attempts":30,"failed":0,"status":"measured","equalSpec":false}]},{"id":"preview-first-byte","group":"Agent work","label":"Preview URL first byte","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.156,"p95":0.24,"ci95":[0.146,0.166],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":0.189,"p95":0.556,"ci95":[0.186,0.197],"attempts":29,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":0.192,"p95":0.355,"ci95":[0.184,0.209],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":0.226,"p95":0.437,"ci95":[0.22,0.239],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":0.35,"p95":0.442,"ci95":[0.331,0.367],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.369,"p95":0.541,"ci95":[0.365,0.394],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":0.812,"p95":0.992,"ci95":[0.805,0.826],"attempts":30,"failed":0,"status":"measured","equalSpec":false}]},{"id":"edit-loop","group":"Agent work","label":"Edit, rebuild, reload","unit":"s","better":"lower","results":[{"provider":"daytona","median":0.248,"p95":0.816,"ci95":[0.233,0.325],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.258,"p95":0.558,"ci95":[0.256,0.26],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.318,"p95":0.506,"ci95":[0.307,0.366],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":0.394,"p95":0.433,"ci95":[0.387,0.405],"attempts":29,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":0.499,"p95":1.395,"ci95":[0.466,0.508],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":0.505,"p95":0.77,"ci95":[0.495,0.517],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":0.964,"p95":0.978,"ci95":[0.96,0.97],"attempts":30,"failed":0,"status":"measured","equalSpec":false}]},{"id":"command","group":"Everyday calls","label":"Run a command","unit":"s","better":"lower","results":[{"provider":"daytona","median":0.116,"p95":0.294,"ci95":[0.11,0.125],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":0.118,"p95":0.284,"ci95":[0.117,0.119],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.147,"p95":0.203,"ci95":[0.145,0.15],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":0.155,"p95":0.232,"ci95":[0.153,0.157],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":0.166,"p95":0.19,"ci95":[0.161,0.169],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":0.22,"p95":0.315,"ci95":[0.218,0.224],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":0.286,"p95":0.292,"ci95":[0.285,0.286],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"files-1kb","group":"Everyday calls","label":"Write + read 1 KB","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.078,"p95":0.092,"ci95":[0.077,0.08],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":0.127,"p95":0.478,"ci95":[0.123,0.133],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.181,"p95":0.235,"ci95":[0.18,0.184],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.192,"p95":0.298,"ci95":[0.191,0.193],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":0.2,"p95":0.218,"ci95":[0.197,0.202],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":0.345,"p95":3.036,"ci95":[0.343,0.349],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":0.466,"p95":0.477,"ci95":[0.466,0.468],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"files-10mb","group":"Everyday calls","label":"Write + read 10 MB","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.589,"p95":3.595,"ci95":[0.557,0.613],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":0.742,"p95":3.372,"ci95":[0.725,0.772],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":1.315,"p95":4.837,"ci95":[1.233,1.362],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":2.376,"p95":7.529,"ci95":[1.767,4.922],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":3.073,"p95":6.096,"ci95":[2.972,3.261],"attempts":100,"failed":1,"status":"measured","equalSpec":false},{"provider":"freestyle","median":3.538,"p95":21.942,"ci95":[3.268,4.584],"attempts":92,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":3.734,"p95":9.73,"ci95":[2.716,4.597],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"reconnect","group":"Everyday calls","label":"Reconnect","unit":"s","better":"lower","results":[{"provider":"daytona","median":0.333,"p95":0.883,"ci95":[0.322,0.351],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":0.337,"p95":2.194,"ci95":[0.333,0.341],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":0.51,"p95":3.353,"ci95":[0.502,0.557],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":0.511,"p95":2.476,"ci95":[0.507,0.517],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.529,"p95":0.91,"ci95":[0.522,0.535],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.778,"p95":0.995,"ci95":[0.764,0.797],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":1.107,"p95":3.73,"ci95":[1.103,1.111],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"destroy","group":"Everyday calls","label":"Destroy","unit":"s","better":"lower","results":[{"provider":"e2b","median":0.113,"p95":0.124,"ci95":[0.113,0.114],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":0.128,"p95":0.157,"ci95":[0.124,0.131],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":0.146,"p95":0.16,"ci95":[0.145,0.146],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.186,"p95":0.28,"ci95":[0.169,0.194],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":0.2,"p95":0.425,"ci95":[0.188,0.209],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":0.754,"p95":9.412,"ci95":[0.585,0.863],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":0.954,"p95":4.123,"ci95":[0.814,1.038],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"pause-resume","group":"Pause and snapshots","label":"Pause + resume","unit":"s","better":"lower","results":[{"provider":"boxd","median":0.54,"p95":3.198,"ci95":[0.538,0.541],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.641,"p95":2.783,"ci95":[0.6,0.688],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.739,"p95":3.624,"ci95":[0.717,0.836],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":0.915,"p95":3.687,"ci95":[0.899,0.943],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":1.735,"p95":12.987,"ci95":[1.336,2.772],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":58.057,"p95":81.559,"ci95":[56.15,59.47],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"unsupported: Daytona's container sandboxes have no pause: stop and start restarts the sandbox and ends every running process.","equalSpec":false}]},{"id":"snapshot","group":"Pause and snapshots","label":"Take snapshot","unit":"s","better":"lower","results":[{"provider":"freestyle","median":0.262,"p95":0.685,"ci95":[0.247,0.299],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.327,"p95":0.595,"ci95":[0.313,0.341],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":0.364,"p95":0.456,"ci95":[0.356,0.375],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":3.209,"p95":4.709,"ci95":[2.973,3.215],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":4.137,"p95":34.83,"ci95":[2.101,6.186],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":6.485,"p95":389.727,"ci95":[6.159,7.115],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":6.844,"p95":8.877,"ci95":[6.81,6.901],"attempts":30,"failed":1,"status":"measured","equalSpec":false}]},{"id":"create-from-snapshot","group":"Pause and snapshots","label":"Start from snapshot","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.471,"p95":0.562,"ci95":[0.462,0.512],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.665,"p95":1.049,"ci95":[0.628,0.78],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.836,"p95":1.168,"ci95":[0.692,0.971],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":1.346,"p95":1.596,"ci95":[1.325,1.419],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":2.045,"p95":3.815,"ci95":[1.958,2.209],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":2.118,"p95":2.967,"ci95":[1.8,2.603],"attempts":29,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":4.652,"p95":23.816,"ci95":[2.695,4.728],"attempts":30,"failed":0,"status":"measured","equalSpec":false}]},{"id":"burst-10","group":"Bursts","label":"10 at once, all ready","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.716,"p95":0.722,"ci95":[0.637,0.722],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":1.005,"p95":1.205,"ci95":[0.982,1.205],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":1.068,"p95":1.122,"ci95":[0.855,1.122],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":1.215,"p95":1.797,"ci95":[1.122,1.797],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":2.188,"p95":2.902,"ci95":[2.037,2.902],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":2.849,"p95":4.389,"ci95":[2.155,4.389],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":3.812,"p95":13.018,"ci95":[2.803,13.018],"attempts":3,"failed":0,"status":"measured","equalSpec":false}]},{"id":"burst-25","group":"Bursts","label":"25 at once, all ready","unit":"s","better":"lower","results":[{"provider":"superserve","median":0.789,"p95":0.923,"ci95":[0.783,0.923],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":1.04,"p95":1.114,"ci95":[0.975,1.114],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":1.177,"p95":1.52,"ci95":[1.117,1.52],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":1.327,"p95":2.257,"ci95":[1.212,2.257],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"not-run: our Daytona account's 10 vCPU quota can't hold 25 sandboxes at once","equalSpec":false},{"provider":"upstash","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"stopped-early: two of its three bursts of 25 never started, because our harness still had two of its own sandboxes open and Upstash's 25-sandbox account cap left no room; not counted; coming in the next update","equalSpec":false},{"provider":"boxd","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"stopped-early: one of its three bursts of 25 was cut short by our own ownership check (16 creates and 5 deletes failed on our side), so it is not counted; coming in the next update","equalSpec":false}]},{"id":"cpu-single","group":"Machine speed","label":"CPU, one core","unit":"ops/s","better":"higher","results":[{"provider":"superserve","median":18683170.183,"p95":19049021.781,"ci95":[18650260.505,18731331.068],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":16667047.786,"p95":20887996.349,"ci95":[11331154.836,18836954.443],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":13703148.979,"p95":13765568.563,"ci95":[13689005.981,13715412.057],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":12178453.213,"p95":12984124.439,"ci95":[11887398.943,12446492.22],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":10842679.144,"p95":15494319.937,"ci95":[9806048.424,11614889.522],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":9826441.941,"p95":10443864.124,"ci95":[9684666.391,10022049.005],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":9024649.504,"p95":9908577.979,"ci95":[8899603.191,9095410.965],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"cpu-all","group":"Machine speed","label":"CPU, all cores","unit":"ops/s","better":"higher","results":[{"provider":"freestyle","median":27522653.673,"p95":32568740.361,"ci95":[26628687.231,28243249.48],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":24696352.134,"p95":25737412.11,"ci95":[24516605.609,24908037.584],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":20158814.568,"p95":30483901.916,"ci95":[17687768.419,21748030.107],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":15765569.139,"p95":20635682.616,"ci95":[15557839.967,16032367.166],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":15420286.065,"p95":20707542.347,"ci95":[14623425.694,16539904.973],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":12309661.096,"p95":16668221.812,"ci95":[12215251.196,12487255.819],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":6971368.11,"p95":7797060.359,"ci95":[6856028.247,7064641.246],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"memory","group":"Machine speed","label":"Memory bandwidth","unit":"MB/s","better":"higher","results":[{"provider":"tenki","median":37197.195,"p95":51676.63,"ci95":[23927.763,46487.818],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":36044.461,"p95":40999.529,"ci95":[35610.128,36803.201],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":30791.145,"p95":33920.215,"ci95":[30273.88,31314.264],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":22336.213,"p95":24768.746,"ci95":[21764.269,22803.886],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":16761.816,"p95":17139.846,"ci95":[16316.474,16838.136],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":14074.769,"p95":20027.101,"ci95":[13188.138,16277.627],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":12586.427,"p95":14277.134,"ci95":[12314.347,12884.637],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"disk-write","group":"Machine speed","label":"Disk write","unit":"MB/s","better":"higher","results":[{"provider":"upstash","median":1735.616,"p95":2127.764,"ci95":[1542.792,1821.616],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":1155.78,"p95":2857.299,"ci95":[875.807,1294.918],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":947.243,"p95":1213.611,"ci95":[917.262,988.714],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":743.377,"p95":817.266,"ci95":[732.061,748.514],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":312.593,"p95":579.916,"ci95":[266.492,452.821],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":228.703,"p95":257.887,"ci95":[219.936,232.058],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"superserve","median":155.586,"p95":837.648,"ci95":[153.599,160.104],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"disk-read","group":"Machine speed","label":"Disk read","unit":"MB/s","better":"higher","results":[{"provider":"upstash","median":20945.471,"p95":22884.697,"ci95":[20773.864,21277.318],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":16418.967,"p95":17579.083,"ci95":[16280.758,16562.25],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":11204.576,"p95":14352.12,"ci95":[10920.534,11473.946],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":10892.491,"p95":21677.313,"ci95":[8682.649,11732.75],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":7151.283,"p95":9112.178,"ci95":[6690.886,7517.516],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":6477.376,"p95":7975.741,"ci95":[6257.897,6615.528],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"superserve","median":4743.594,"p95":8009.962,"ci95":[3715.857,4818.737],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"iops-write","group":"Machine speed","label":"4K random write","unit":"IOPS","better":"higher","results":[{"provider":"superserve","median":20783.571,"p95":34469.475,"ci95":[19953.446,22009.417],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":19290.181,"p95":29880.054,"ci95":[18280.611,21119.056],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":8461.341,"p95":13673.507,"ci95":[8041.635,9167.37],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":7223.318,"p95":9125.654,"ci95":[6951.055,7539.336],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":6719.069,"p95":8636.718,"ci95":[6293.573,7049.066],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":382.682,"p95":562.102,"ci95":[327.382,439.986],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":363.623,"p95":423.063,"ci95":[323.862,380.512],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"iops-read","group":"Machine speed","label":"4K random read","unit":"IOPS","better":"higher","results":[{"provider":"superserve","median":713364.153,"p95":753129.459,"ci95":[698569.842,718320.253],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":509060.499,"p95":527306.276,"ci95":[506712.605,512680.79],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":479868.936,"p95":644417.482,"ci95":[461600.6,497741.692],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"boxd","median":425296.491,"p95":465729.754,"ci95":[412669.603,431572.217],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":344149.858,"p95":400797.149,"ci95":[328546.661,362871.292],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":288876.463,"p95":345778.653,"ci95":[264557.375,299294.787],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":221733.331,"p95":587815.233,"ci95":[217570.519,225753.541],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]}]},"same-spec":{"id":"same-spec","label":"Same machine: 4 vCPU, 8 GB","description":"Every provider asked for Boat's machine, 4 vCPU and 8 GB, or its nearest size.","published":true,"run":{"source":"m-same","start":"2026-10-22T12:41:17.842Z","end":"2026-10-22T16:13:46.713Z","runs":["sbx53-same-spec-main-r5","sbx53-same-spec-cost-r5","sbx53-same-spec-snapshot-r5","sbx53-same-spec-burst-r6","sbx53-same-spec-snapshot-r6"],"observations":10121},"absent":[{"provider":"boat","reason":"coming in the next update. Our Boat plan allows 200 creates a day, and this run used them up; a higher plan or quota may lift that."},{"provider":"superserve","reason":"not in this track: our Superserve plan caps templates at 2 vCPU, so there was no 4 vCPU machine to test. A paid plan may lift this; we will add it if it does."}],"providers":[{"id":"e2b","name":"E2B","allocation":{"requested":{"size":null,"template":"r1h7gud12rf1f6iu29r0","compute":{"cpu":4,"memoryGiB":8},"provisioned":{"cpu":4,"memoryGiB":8},"text":"4 vCPU, 8 GB, template r1h7gud12rf1f6iu29r0"},"measured":[{"cpu":4,"memoryGiB":7.77,"diskGiB":21.535}],"measuredText":"4 vCPU, 7.8 GB","equalSpec":true,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":null,"ranIn":"The Dalles, US","cpu":"Intel Xeon Processor @ 2.60GHz","hardware":{"chip":"Intel Xeon Platinum 8373C","family":"Ice Lake, 2.6 GHz, 54 MB L3","chipBasis":"inferred","memory":{"type":"DDR4","basis":"inferred","detail":"DDR4-3200, the chip's memory type"},"cloud":"Google Cloud","region":"us-west1, The Dalles, Oregon","runsAs":"Firecracker microVM, nested inside a Google Cloud VM","hidden":["The CPU name, microcode and clock leaves are hidden. The chip is identified from its model number, cache size and 2.6 GHz clock.","The default 2 vCPUs are one physical core with two hyperthreads, so two-thread work scales well below twice one thread.","No DMI or PCI devices, so the memory type can't be read inside; DDR4 follows from the chip."],"probed":"Oct 22, 2026"},"sdkReady":{"median":2.053,"p95":4.321,"sdkSetupMedian":0.897,"coldStarts":100},"costPerTask":{"kind":"metered","perTaskUsd":0.001552,"per1000TasksUsd":1.552,"confidence":"medium","tasks":10,"source":"https://api.e2b.app/events/sandboxes/{id}","basis":"Metered over 10 tasks, medium confidence"},"price":{"currency":"USD","perHour":0.331,"perVcpuHour":0.083,"perHourUsd":0.331,"commonSizeUsd":0.331,"note":"10 GiB Hobby / 20 GiB Pro included disk. Subscription excluded. Per-second billing; no separate creation fee listed.","usdNote":null,"source":"https://e2b.dev/pricing"},"pause":{"supported":true,"split":{"pause":1.2916321539999918,"resume":0.5057368029998616,"n":100}},"reliability":{"attempts":1076,"succeeded":1076,"successRate":1},"failures":[],"setAside":[]},{"id":"daytona","name":"Daytona","allocation":{"requested":{"size":null,"template":"daytona-large","compute":{"cpu":4,"memoryGiB":8},"provisioned":{"cpu":4,"memoryGiB":8,"diskGiB":10},"text":"4 vCPU, 8 GB, template daytona-large"},"measured":[{"cpu":4,"memoryGiB":8,"diskGiB":10}],"measuredText":"4 vCPU, 8 GB","equalSpec":true,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":null,"ranIn":null,"cpu":"AMD EPYC 9254 24-Core Processor","hardware":{"chip":"AMD EPYC 9354P, 9354, 9254 or 9275F","family":"Zen 4 (Genoa); the 9275F is Zen 5 (Turin). Hosts vary","chipBasis":"named by the CPU","memory":{"type":"DDR5","basis":"measured","detail":"Registered DDR5, read from the memory controller on a 9254 host"},"cloud":"Bare-metal servers","region":"Location not visible; control plane on AWS us-east-1","runsAs":"Docker container, shares the host's kernel; cgroups set its CPU and RAM","hidden":["Nothing at the CPU level is hidden: the real CPU name, microcode, server model (Dell PowerEdge R6615 in the probe, a Supermicro AS-2115GT-HNTR in the benchmark) and disks are visible.","nproc and lscpu show every host thread (48 or 64) and /proc/meminfo shows the host's RAM, while the default sandbox may use 1 CPU and 1 GB. Tools that size thread pools or caches from those numbers oversubscribe.","Hosts differ in clock speed, so single-thread results vary from sandbox to sandbox. Neighbours share the host kernel.","Outbound internet is blocked on our account tier, so the sandbox's own network location could not be read."],"probed":"Oct 22, 2026"},"sdkReady":{"median":3.4,"p95":8.036,"sdkSetupMedian":1.835,"coldStarts":100},"costPerTask":{"kind":"billed","perTaskUsd":0.001189,"per1000TasksUsd":1.189,"confidence":"medium","tasks":10,"source":"https://www.daytona.io/pricing","basis":"Billed over 10 tasks, medium confidence"},"price":{"currency":"USD","perHour":0.331,"perVcpuHour":0.083,"perHourUsd":0.331,"commonSizeUsd":0.331,"note":"Hourly published rates converted to seconds. Assumed one-second increment; minimum and creation fee not stated. Snapshot storage excluded.","usdNote":null,"source":"https://www.daytona.io/pricing"},"pause":{"supported":false,"note":"Daytona's container sandboxes have no pause: stop and start restarts the sandbox and ends every running process."},"reliability":{"attempts":910,"succeeded":910,"successRate":1},"failures":[],"setAside":[]},{"id":"upstash","name":"Upstash Box","allocation":{"requested":{"size":"medium","template":null,"compute":{"cpu":4,"memoryGiB":8},"provisioned":null,"text":"4 vCPU, 8 GB (\"medium\")"},"measured":[{"cpu":4,"memoryGiB":8,"diskGiB":10}],"measuredText":"4 vCPU, 8 GB","equalSpec":true,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":null,"ranIn":"Ashburn, US","cpu":null,"hardware":{"chip":"AWS Graviton4","family":"Arm Neoverse V2, 2.8 GHz; Graviton4 from the m8g instance type","chipBasis":"inferred","memory":{"type":"DDR5","basis":"inferred","detail":"DDR5-5600, from AWS's Graviton4 spec"},"cloud":"AWS EC2","region":"us-east-1, Ashburn, Virginia","runsAs":"Docker container, on a shared m8g.16xlarge VM; cgroups set its CPU and RAM","hidden":["The only Arm provider here: x86-only binaries don't run, and compile and run times differ for that reason alone.","nproc and /proc/meminfo show the host's 64 CPUs and 256 GB, while a box may use 2 CPUs and 4 GB (4 and 8 at the medium size).","No DMI tables or clock readings inside, so memory type and clock come from AWS's published spec.","Neighbours share the host kernel and the chip's L3 cache."],"probed":"Oct 22, 2026"},"sdkReady":{"median":8.857,"p95":17.155,"sdkSetupMedian":5.181,"coldStarts":99},"costPerTask":{"kind":"billed","perTaskUsd":0.0004738,"per1000TasksUsd":0.4738,"confidence":"high","tasks":8,"source":"https://us-east-1.box.upstash.com/v2/box/{id}","basis":"Billed over 8 tasks, high confidence"},"price":{"currency":"USD","perHour":0.2,"perVcpuHour":0.05,"perHourUsd":0.2,"commonSizeUsd":0.2,"note":"PAYG ACTIVE CPU billing: Small $0.10, Medium $0.20, Large $0.40 per active box-hour; tier price × wall time is an upper-bound estimate, not metered usage. Disk allocated capacity is an upper bound for written data. 30-day month. No RAM fee listed. Increment/minimum unspecified.","usdNote":null,"source":"https://upstash.com/pricing/box"},"pause":{"supported":true,"split":{"pause":0.8366179980000016,"resume":0.5705873199999333,"n":99}},"reliability":{"attempts":1062,"succeeded":1061,"successRate":0.99906},"failures":[{"kind":"provider-operation","reason":"Upstash Box exec failed (provider). (provider)","count":1,"runs":1,"tests":["Cold start"]}],"setAside":[{"reason":"Our ownership check could not confirm the sandbox before deleting it, so we can't tell whose side failed","count":5,"runs":3},{"reason":"Our ownership check failed on 4 snapshot cleanup steps; none is counted","count":4,"runs":3},{"reason":"Our ownership check failed on 3 burst deletes; none is counted","count":3,"runs":3}]},{"id":"boxd","name":"boxd","allocation":{"requested":{"size":{"cpu":4,"memoryMb":16384},"template":null,"compute":{"cpu":4,"memoryGiB":16},"provisioned":null,"text":"4 vCPU, 16 GB"},"measured":[{"cpu":4,"memoryGiB":15.64,"diskGiB":97.872}],"measuredText":"4 vCPU, 16 GB","equalSpec":false,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":16}},"allocationNote":"ran 4 vCPU, 16 GB, not 4 vCPU, 8 GB. boxd has no 8 GB size at 4 vCPU; this is its nearest.","advertisedDefault":null,"ranIn":"Frankfurt am Main, DE","cpu":"AMD EPYC","hardware":{"chip":"AMD EPYC 9554P, likely","family":"Zen 4 (Genoa), 3.1 GHz","chipBasis":"inferred","memory":{"type":"DDR5","basis":"inferred","detail":"DDR5-4800, the Genoa platform's memory"},"cloud":"Latitude.sh bare metal","region":"Frankfurt, Germany","runsAs":"Firecracker microVM, with Docker installed inside the VM","hidden":["The CPU name is trimmed to \"AMD EPYC\" and the microcode and topology are hidden. The chip is inferred from its 3.1 GHz clock and the Genoa servers Latitude.sh sells; a dual-socket 9554 can't be ruled out.","No DMI, so the memory type is inferred.","AVX-512 is exposed, which can help vectorised work against hosts that hide it.","The disk reports itself as rotational although it is virtual, so tools that tune for hard drives may behave differently."],"probed":"Oct 22, 2026"},"sdkReady":{"median":3.212,"p95":6.197,"sdkSetupMedian":1.782,"coldStarts":100},"costPerTask":{"kind":"metered","perTaskUsd":0.001999,"per1000TasksUsd":1.999,"confidence":"medium","tasks":10,"source":"boxd orgs.usage / orgs.billingActivityPage","basis":"Metered over 10 tasks, medium confidence"},"price":{"currency":"EUR","perHour":0.348,"perVcpuHour":0.087,"perHourUsd":0.391,"commonSizeUsd":null,"note":"EUR; RAM in use and written disk billed, so allocated resources are an upper bound. Metered per second, settled every 15 minutes (not a rounding increment). FX budget uses conservative 2 USD/EUR, not a market quote.","usdNote":"Converted at the ECB rate of 1.1234 USD per EUR on October 21, 2026.","source":"https://boxd.sh/pricing"},"pause":{"supported":true,"split":{"pause":0.12380898099998013,"resume":0.4039313779999502,"n":99}},"reliability":{"attempts":1064,"succeeded":1062,"successRate":0.99812},"failures":[{"kind":"provider-operation","reason":"boxd request failed. (invalid_argument)","count":1,"runs":1,"tests":["Pause + resume"]},{"kind":"provider-operation","reason":"boxd request failed. (provider)","count":1,"runs":1,"tests":["Write + read 10 MB"]}],"setAside":[{"reason":"Our ownership check failed on 4 snapshot attempts, and our 30-second cleanup deadline ran out 8 times; none is counted","count":12,"runs":12},{"reason":"Our ownership check failed on 1 burst create and 13 burst deletes; none is counted","count":14,"runs":4}]},{"id":"freestyle","name":"Freestyle","allocation":{"requested":{"size":null,"template":null,"compute":{"cpu":4,"memoryGiB":8},"provisioned":{"cpu":4,"memoryGiB":8,"diskGiB":32},"text":"4 vCPU, 8 GB"},"measured":[{"cpu":4,"memoryGiB":7.755,"diskGiB":31.201}],"measuredText":"4 vCPU, 7.8 GB","equalSpec":true,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":null,"ranIn":"San Francisco, US","cpu":null,"hardware":{"chip":"AMD EPYC 7763, likely","family":"Zen 3 (Milan), 2.45 GHz","chipBasis":"inferred","memory":{"type":"DDR4","basis":"inferred","detail":"DDR4-3200, the Milan platform's memory"},"cloud":"Freestyle's own network (AS36320)","region":"San Francisco Bay Area","runsAs":"Firecracker microVM, 4 vCPU, 8 GB, no hyperthreads exposed","hidden":["The CPU name, microcode and topology are hidden. The chip is inferred from its 2.45 GHz clock, cache and Zen 3 features.","Memory is added after boot: about 3 GB is free at first, 7.9 GB later.","No DMI, so the memory type is inferred."],"probed":"Oct 22, 2026"},"sdkReady":{"median":1.656,"p95":4.093,"sdkSetupMedian":0.959,"coldStarts":100},"costPerTask":{"kind":"estimate","perTaskUsd":0.001782,"per1000TasksUsd":1.782,"confidence":"low","tasks":10,"source":"https://www.freestyle.sh/pricing","basis":"Estimate over 10 tasks, low confidence"},"price":{"currency":"USD","perHour":0.264,"perVcpuHour":0.066,"perHourUsd":0.264,"commonSizeUsd":0.264,"note":"Published overage rates, allowances/subscriptions excluded. $0.02/GB transfer excluded. Billing increment/minimum and creation fees unspecified; one-second estimate.","usdNote":null,"source":"https://www.freestyle.sh/pricing"},"pause":{"supported":true,"split":{"pause":0.3201344599998556,"resume":0.34602604700019585,"n":100}},"reliability":{"attempts":1073,"succeeded":1072,"successRate":0.99907},"failures":[{"kind":"provider-operation","reason":"Preview HTTP 502, no first byte","count":1,"runs":1,"tests":["Preview URL first byte"]}],"setAside":[{"reason":"Our SDK timed out or wrapped the error moving 10 MB, so we can't tell whose side failed","count":3,"runs":3}]},{"id":"tenki","name":"Tenki","allocation":{"requested":{"size":{"cpu":4,"memoryMb":8192},"template":null,"compute":{"cpu":4,"memoryGiB":8},"provisioned":null,"text":"4 vCPU, 8 GB"},"measured":[{"cpu":4,"memoryGiB":7.797,"diskGiB":4.84}],"measuredText":"4 vCPU, 7.8 GB","equalSpec":true,"verified":true,"mismatches":[],"pricedAt":{"cpu":4,"memoryGiB":8}},"allocationNote":null,"advertisedDefault":null,"ranIn":"Seattle, US","cpu":"AMD EPYC 7B13 64-Core Processor","hardware":{"chip":"AMD EPYC 7B13, as reported","family":"Zen 3 (Milan), 2.25 GHz. The 4 vCPU VM also showed EPYC 9575F (Zen 5, Turin) cores","chipBasis":"conflicting","memory":{"type":null,"note":"No memory data inside the VM"},"cloud":"Luxor (AS19538)","region":"Seattle","runsAs":"Cloud Hypervisor microVM, with a container inside","hidden":["On the 4 vCPU VM, two vCPUs report an EPYC 7B13 (Milan) and two an EPYC 9575F (Turin). That fits a snapshot taken on a Milan host and restored on a Turin one, so the real host may be newer than the name says.","CPU features are capped at the Milan set on every vCPU, so no AVX-512 even on Turin.","The clock reads 2.25 GHz, the 7B13's base, which may be carried over from the snapshot."],"probed":"Oct 22, 2026"},"sdkReady":{"median":2.017,"p95":5.429,"sdkSetupMedian":1.047,"coldStarts":78},"costPerTask":{"kind":"metered","perTaskUsd":0.001886,"per1000TasksUsd":1.886,"confidence":"medium","tasks":10,"source":"https://tenki.cloud/pricing","basis":"Metered over 10 tasks, medium confidence"},"price":{"currency":"USD","perHour":0.331,"perVcpuHour":0.083,"perHourUsd":0.331,"commonSizeUsd":0.331,"note":"Per-second, no minimum. Separate persistent storage $0.000000076/GiB-second excluded. Credits excluded.","usdNote":null,"source":"https://tenki.cloud/pricing"},"pause":{"supported":true,"split":{"pause":60.53191615900025,"resume":2.449111862000078,"n":77}},"reliability":{"attempts":897,"succeeded":897,"successRate":1},"failures":[],"setAside":[{"reason":"The run was stopped during this attempt, so it is not counted","count":1,"runs":1}]}],"quickPicks":[{"need":"Fastest cold start","pick":"Freestyle 0.66 s","runnerUp":"E2B 0.95 s"},{"need":"Fastest agent work (tool replay)","pick":"Daytona 11 s","runnerUp":"E2B 15 s"},{"need":"Fastest 25-sandbox fan-out","pick":"Freestyle 1.4 s","runnerUp":"E2B 1.5 s"},{"need":"Fastest reconnect","pick":"Daytona 0.31 s","runnerUp":"Upstash Box 0.52 s"},{"need":"Cheapest task","pick":"Upstash Box $0.00047 per task, $0.47 per 1,000 (billed)","runnerUp":"Daytona $0.0012 per task, $1.19 per 1,000 (billed)"},{"need":"Most reliable","pick":"E2B, Daytona, Tenki 100.00% (tied)","runnerUp":"Freestyle 99.91%"}],"metrics":[{"id":"cold","group":"Start-up","label":"Cold start","unit":"s","better":"lower","results":[{"provider":"freestyle","median":0.657,"p95":2.81,"ci95":[0.643,0.681],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.954,"p95":3.097,"ci95":[0.921,1.042],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.963,"p95":2.96,"ci95":[0.913,1.07],"attempts":78,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":1.135,"p95":3.017,"ci95":[1.019,1.284],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":1.382,"p95":3.922,"ci95":[1.37,1.4],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"upstash","median":2.652,"p95":9.054,"ci95":[2.615,2.667],"attempts":100,"failed":1,"status":"measured","equalSpec":true}]},{"id":"tool-replay","group":"Agent work","label":"Agent tool replay","unit":"s","better":"lower","results":[{"provider":"daytona","median":10.837,"p95":28.921,"ci95":[9.93,11.893],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":15.083,"p95":16.3,"ci95":[14.521,15.474],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":20.078,"p95":28.165,"ci95":[19.719,21.694],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":21.384,"p95":33.779,"ci95":[20.482,25.434],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":27.529,"p95":54.402,"ci95":[24.251,30.214],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":33.202,"p95":49.173,"ci95":[32.258,40.688],"attempts":30,"failed":0,"status":"measured","equalSpec":false}]},{"id":"npm-install","group":"Agent work","label":"npm install","unit":"s","better":"lower","results":[{"provider":"daytona","median":0.738,"p95":1.22,"ci95":[0.704,0.764],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":0.986,"p95":2.581,"ci95":[0.965,1.018],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":1.479,"p95":3.576,"ci95":[1.135,1.693],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":1.504,"p95":2.633,"ci95":[1.359,1.604],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":1.55,"p95":2.71,"ci95":[1.483,1.665],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.86,"p95":2.843,"ci95":[1.689,2.132],"attempts":30,"failed":0,"status":"measured","equalSpec":true}]},{"id":"pip-install","group":"Agent work","label":"pip install","unit":"s","better":"lower","results":[{"provider":"tenki","median":3.318,"p95":6.131,"ci95":[3.088,4.719],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":3.639,"p95":6.324,"ci95":[3.602,3.679],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":4.144,"p95":6.143,"ci95":[3.965,4.254],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":4.62,"p95":5.948,"ci95":[4.476,4.63],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":4.759,"p95":5.733,"ci95":[4.452,5.05],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":7.351,"p95":8.072,"ci95":[7.342,7.381],"attempts":30,"failed":0,"status":"measured","equalSpec":true}]},{"id":"git-clone","group":"Agent work","label":"git clone","unit":"s","better":"lower","results":[{"provider":"daytona","median":0.438,"p95":2.116,"ci95":[0.402,0.53],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":0.563,"p95":2.807,"ci95":[0.542,0.588],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":1.094,"p95":3.198,"ci95":[1.036,1.137],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.11,"p95":1.945,"ci95":[1.087,1.18],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":1.558,"p95":2.561,"ci95":[1.504,1.566],"attempts":30,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":1.702,"p95":2.758,"ci95":[1.443,1.834],"attempts":30,"failed":0,"status":"measured","equalSpec":true}]},{"id":"dev-server","group":"Agent work","label":"Dev server ready","unit":"s","better":"lower","results":[{"provider":"tenki","median":1.92,"p95":4.354,"ci95":[1.894,2.006],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.928,"p95":2.897,"ci95":[1.84,1.987],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":1.964,"p95":3.192,"ci95":[1.918,2.101],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":2.946,"p95":9.468,"ci95":[2.697,3.525],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":3.199,"p95":8.545,"ci95":[3.087,3.537],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":3.708,"p95":7.095,"ci95":[3.683,4.272],"attempts":30,"failed":0,"status":"measured","equalSpec":false}]},{"id":"preview-first-byte","group":"Agent work","label":"Preview URL first byte","unit":"s","better":"lower","results":[{"provider":"daytona","median":0.18,"p95":0.356,"ci95":[0.169,0.186],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":0.187,"p95":0.23,"ci95":[0.182,0.189],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.217,"p95":0.448,"ci95":[0.213,0.223],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.342,"p95":1.829,"ci95":[0.333,0.367],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":0.392,"p95":0.486,"ci95":[0.371,0.414],"attempts":30,"failed":1,"status":"measured","equalSpec":true},{"provider":"boxd","median":0.829,"p95":1.929,"ci95":[0.819,0.879],"attempts":30,"failed":0,"status":"measured","equalSpec":false}]},{"id":"edit-loop","group":"Agent work","label":"Edit, rebuild, reload","unit":"s","better":"lower","results":[{"provider":"daytona","median":0.241,"p95":0.995,"ci95":[0.228,0.261],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":0.272,"p95":0.48,"ci95":[0.26,0.283],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.304,"p95":0.497,"ci95":[0.3,0.31],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":0.389,"p95":1.523,"ci95":[0.374,0.398],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.515,"p95":1.8,"ci95":[0.504,0.528],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":0.96,"p95":3.212,"ci95":[0.954,0.967],"attempts":30,"failed":0,"status":"measured","equalSpec":false}]},{"id":"command","group":"Everyday calls","label":"Run a command","unit":"s","better":"lower","results":[{"provider":"daytona","median":0.107,"p95":0.278,"ci95":[0.103,0.11],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.122,"p95":0.135,"ci95":[0.12,0.123],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":0.15,"p95":0.215,"ci95":[0.145,0.153],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.166,"p95":0.204,"ci95":[0.162,0.172],"attempts":78,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":0.226,"p95":0.502,"ci95":[0.221,0.232],"attempts":99,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":0.281,"p95":0.288,"ci95":[0.279,0.283],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"files-1kb","group":"Everyday calls","label":"Write + read 1 KB","unit":"s","better":"lower","results":[{"provider":"daytona","median":0.124,"p95":0.451,"ci95":[0.121,0.127],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":0.183,"p95":0.397,"ci95":[0.18,0.189],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.205,"p95":0.346,"ci95":[0.202,0.209],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.268,"p95":0.371,"ci95":[0.26,0.287],"attempts":77,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":0.339,"p95":1.098,"ci95":[0.336,0.345],"attempts":99,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":0.468,"p95":0.481,"ci95":[0.466,0.47],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"files-10mb","group":"Everyday calls","label":"Write + read 10 MB","unit":"s","better":"lower","results":[{"provider":"daytona","median":0.761,"p95":3.678,"ci95":[0.728,0.788],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.31,"p95":2.548,"ci95":[1.258,1.359],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":2.461,"p95":5.503,"ci95":[2.385,2.784],"attempts":97,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":2.976,"p95":4.759,"ci95":[2.917,3.019],"attempts":99,"failed":1,"status":"measured","equalSpec":false},{"provider":"upstash","median":3.047,"p95":9.597,"ci95":[2.644,3.737],"attempts":99,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":6.188,"p95":10.83,"ci95":[6.022,6.418],"attempts":78,"failed":0,"status":"measured","equalSpec":true}]},{"id":"reconnect","group":"Everyday calls","label":"Reconnect","unit":"s","better":"lower","results":[{"provider":"daytona","median":0.313,"p95":1.197,"ci95":[0.306,0.325],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":0.52,"p95":1.14,"ci95":[0.514,0.532],"attempts":99,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":0.526,"p95":2.295,"ci95":[0.523,0.529],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":0.543,"p95":1.764,"ci95":[0.536,0.552],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.781,"p95":1.581,"ci95":[0.774,0.795],"attempts":77,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":1.099,"p95":3.274,"ci95":[1.096,1.105],"attempts":100,"failed":0,"status":"measured","equalSpec":false}]},{"id":"destroy","group":"Everyday calls","label":"Destroy","unit":"s","better":"lower","results":[{"provider":"e2b","median":0.116,"p95":0.125,"ci95":[0.114,0.117],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":0.133,"p95":0.172,"ci95":[0.131,0.137],"attempts":78,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":0.142,"p95":0.159,"ci95":[0.142,0.143],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.205,"p95":0.429,"ci95":[0.185,0.22],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":0.877,"p95":2.395,"ci95":[0.771,0.997],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":1.02,"p95":18.426,"ci95":[0.602,1.557],"attempts":95,"failed":0,"status":"measured","equalSpec":true}]},{"id":"pause-resume","group":"Pause and snapshots","label":"Pause + resume","unit":"s","better":"lower","results":[{"provider":"boxd","median":0.528,"p95":1.078,"ci95":[0.526,0.53],"attempts":100,"failed":1,"status":"measured","equalSpec":false},{"provider":"freestyle","median":0.725,"p95":2.609,"ci95":[0.682,0.808],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":1.636,"p95":11.733,"ci95":[1.271,2.053],"attempts":99,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.809,"p95":3.493,"ci95":[1.634,1.962],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":63.779,"p95":84.195,"ci95":[60.58,67.704],"attempts":77,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"unsupported: Daytona's container sandboxes have no pause: stop and start restarts the sandbox and ends every running process.","equalSpec":true}]},{"id":"snapshot","group":"Pause and snapshots","label":"Take snapshot","unit":"s","better":"lower","results":[{"provider":"freestyle","median":0.258,"p95":0.766,"ci95":[0.242,0.341],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.485,"p95":2.229,"ci95":[0.973,1.767],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":2.106,"p95":10.275,"ci95":[2.094,4.14],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":4.009,"p95":4.767,"ci95":[3.652,4.022],"attempts":26,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":6.776,"p95":6.979,"ci95":[6.765,6.821],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"not-run: our Daytona account's 10 vCPU quota kept its 4 vCPU snapshot tests from running","equalSpec":true}]},{"id":"create-from-snapshot","group":"Pause and snapshots","label":"Start from snapshot","unit":"s","better":"lower","results":[{"provider":"freestyle","median":0.648,"p95":0.919,"ci95":[0.617,0.684],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.313,"p95":2.612,"ci95":[0.976,1.609],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":1.406,"p95":1.463,"ci95":[1.389,1.427],"attempts":26,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":1.862,"p95":3.611,"ci95":[1.632,1.984],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":4.643,"p95":27.134,"ci95":[2.746,4.66],"attempts":30,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"not-run: our Daytona account's 10 vCPU quota kept its 4 vCPU snapshot tests from running","equalSpec":true}]},{"id":"burst-10","group":"Bursts","label":"10 at once, all ready","unit":"s","better":"lower","results":[{"provider":"freestyle","median":1.009,"p95":1.028,"ci95":[0.889,1.028],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":1.234,"p95":1.267,"ci95":[1.223,1.267],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.445,"p95":1.562,"ci95":[1.168,1.562],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":21.231,"p95":25.277,"ci95":[4.78,25.277],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"not-run: our Daytona account's 10 vCPU quota can't hold ten 4 vCPU sandboxes at once","equalSpec":true},{"provider":"boxd","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"stopped-early: one of its three bursts of 10 was cut short by our own ownership check (1 create failed on our side), so it is not counted; coming in the next update","equalSpec":false}]},{"id":"burst-25","group":"Bursts","label":"25 at once, all ready","unit":"s","better":"lower","results":[{"provider":"freestyle","median":1.405,"p95":1.539,"ci95":[1.32,1.539],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":1.492,"p95":2.898,"ci95":[1.42,2.898],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":1.652,"p95":2.559,"ci95":[1.423,2.559],"attempts":3,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":6.235,"p95":6.924,"ci95":[6.199,6.924],"attempts":3,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"not-run: our Daytona account's 10 vCPU quota can't hold 25 4 vCPU sandboxes at once","equalSpec":true},{"provider":"upstash","median":null,"p95":null,"ci95":null,"attempts":0,"failed":0,"status":"stopped-early: its third burst of 25 never started, because our harness still had one of its own sandboxes open and Upstash's 25-sandbox account cap left no room; not counted; coming in the next update","equalSpec":true}]},{"id":"cpu-single","group":"Machine speed","label":"CPU, one core","unit":"ops/s","better":"higher","results":[{"provider":"tenki","median":19575166.044,"p95":20994581.781,"ci95":[18301917.695,20010902.34],"attempts":78,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":13706222.315,"p95":13769646.569,"ci95":[13694209.687,13720265.757],"attempts":99,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":12108815.189,"p95":12869277.976,"ci95":[11859872.71,12357053.603],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"e2b","median":10798481.397,"p95":16097045.093,"ci95":[9921833.218,11887665.885],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":10034541.499,"p95":10449583.731,"ci95":[9892075.087,10201474.639],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":9236875.307,"p95":14168601.933,"ci95":[9094525.649,9429433.471],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"cpu-all","group":"Machine speed","label":"CPU, all cores","unit":"ops/s","better":"higher","results":[{"provider":"upstash","median":49910790.273,"p95":50642143.09,"ci95":[49528825.592,50137506.175],"attempts":99,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":36705717.635,"p95":46630779.308,"ci95":[35042645.849,39024522.737],"attempts":78,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":28434905.279,"p95":31617783.239,"ci95":[27630343.228,29121367.923],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":28011834.549,"p95":32688469.372,"ci95":[27191139.334,28684855.459],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":27143907.224,"p95":36925050.539,"ci95":[25322362.859,28655816.266],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":26344521.418,"p95":31304438.22,"ci95":[25893375.561,26830494.577],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"memory","group":"Machine speed","label":"Memory bandwidth","unit":"MB/s","better":"higher","results":[{"provider":"tenki","median":43552.935,"p95":51854.744,"ci95":[36797.209,47528.696],"attempts":78,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":35997.701,"p95":41744.575,"ci95":[34447.097,37042.238],"attempts":99,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":31283.246,"p95":57714.149,"ci95":[30324.219,31818.032],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":22102.988,"p95":24686.364,"ci95":[21610.663,22630.887],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":17013.858,"p95":20821.552,"ci95":[16163.381,17802.601],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":12648.499,"p95":14404.397,"ci95":[12444.585,12854.286],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"disk-write","group":"Machine speed","label":"Disk write","unit":"MB/s","better":"higher","results":[{"provider":"upstash","median":1886.446,"p95":2164.796,"ci95":[1669.499,1981.815],"attempts":99,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":1392.829,"p95":3036.95,"ci95":[1270.576,1518.449],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":858.676,"p95":1062.934,"ci95":[806.27,890.8],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":742.193,"p95":809.021,"ci95":[736.636,747.394],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"tenki","median":464.804,"p95":594.563,"ci95":[418.359,484.765],"attempts":78,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":228.414,"p95":257.796,"ci95":[223.236,235.444],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"disk-read","group":"Machine speed","label":"Disk read","unit":"MB/s","better":"higher","results":[{"provider":"upstash","median":21352.081,"p95":23241.814,"ci95":[21021.111,21615.594],"attempts":99,"failed":0,"status":"measured","equalSpec":true},{"provider":"daytona","median":17425.782,"p95":24801.265,"ci95":[17168.978,17912.325],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":12171.206,"p95":21209.001,"ci95":[10421.508,14053.276],"attempts":78,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":11721.335,"p95":13671.429,"ci95":[11449.424,11884.316],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":7530.66,"p95":9303.108,"ci95":[6846.807,7757.376],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":6308.614,"p95":7555.718,"ci95":[6052.816,6531.484],"attempts":100,"failed":0,"status":"measured","equalSpec":true}]},{"id":"iops-write","group":"Machine speed","label":"4K random write","unit":"IOPS","better":"higher","results":[{"provider":"daytona","median":20959.572,"p95":32080.117,"ci95":[20064.62,23151.779],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":8476.479,"p95":14216.725,"ci95":[7799.402,9279.87],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"freestyle","median":6976.672,"p95":8848.803,"ci95":[6660.456,7307.927],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":6674.076,"p95":9095.697,"ci95":[6439.704,6963.153],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"upstash","median":402.354,"p95":561.117,"ci95":[335.507,458.791],"attempts":99,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":357.836,"p95":414.369,"ci95":[331.027,365.883],"attempts":78,"failed":0,"status":"measured","equalSpec":true}]},{"id":"iops-read","group":"Machine speed","label":"4K random read","unit":"IOPS","better":"higher","results":[{"provider":"upstash","median":511097.427,"p95":531525.551,"ci95":[504513.097,512798.431],"attempts":99,"failed":0,"status":"measured","equalSpec":true},{"provider":"e2b","median":474327.927,"p95":661592.733,"ci95":[455866.431,500302.918],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"boxd","median":427364.685,"p95":462886.778,"ci95":[420517.309,433651.334],"attempts":100,"failed":0,"status":"measured","equalSpec":false},{"provider":"daytona","median":357538.712,"p95":833519.034,"ci95":[338099.88,366110.734],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"freestyle","median":298657.563,"p95":347887.544,"ci95":[281081.768,303983.847],"attempts":100,"failed":0,"status":"measured","equalSpec":true},{"provider":"tenki","median":221002.106,"p95":603930.265,"ci95":[218934.405,230273.81],"attempts":78,"failed":0,"status":"measured","equalSpec":true}]}]}}}