Our Build Sandbox Failed

If your deployment failed with Our build sandbox failed, the isolated machine that runs your build stopped responding before it could report a result. That is a fault in our build infrastructure…

If your deployment failed with Our build sandbox failed, the isolated machine that runs your build stopped responding before it could report a result. That is a fault in our build infrastructure. Nothing in your repository caused it, there is nothing in your code to fix, and in most cases we have already queued the build again on a fresh sandbox without you asking.

This guide explains what the sandbox is, how we know the fault was ours rather than yours, why the build log you were given is unusually short, what we do about it automatically, and the one case where you should come and talk to us.

What the Sandbox Is

Every Orbit build runs in its own microVM: a real virtual machine with its own kernel, booted for your build and destroyed when it finishes. It is not a container sharing a kernel with other tenants, and it is never reused between builds. One build, one machine, then gone.

That machine does not talk to the internet to report progress. It talks to a runner agent on the build host over a direct control channel, and the agreement between them is simple. The runner boots the machine, hands it the job, and waits. The guest announces each phase as it starts (download, install, build, artifact), streams its output back, and at the end sends a single done message carrying the result, whether that result is success or a failure in your build.

Everything the deployment page shows you comes through that channel.

What Actually Happened

The guest never sent done.

The runner was waiting for that message and the channel closed instead. From the host's side the machine simply stopped being there: it was booted, it was running, and then it was not answering. The runner records what it observed in one of a small number of ways:

guest control stream closed without 'done'
guest never became ready: guest did not become ready in time
send job spec to guest: ...

These say slightly different things about how far the process got. The first means the machine was running your build and went away mid-flight. The second means it was booted but never reached the point of speaking at all. The third means we could not even hand it the work, which is the earliest of the three and happens before your repository has been read.

All three have the same character: the machine we started stopped answering, and it never got to tell us why, because the thing that would have told us is the thing that is gone.

Why We Can Say This One Was Ours

This is the part worth being precise about, because "it was our fault" is a claim a platform should have to earn.

The sentences above are not found in your build output. They are written by our runner agent on the build host, in code we own, about a machine we started. They are the platform reporting on the platform. Your build log is a different stream entirely: that is what your commands printed inside the guest.

We keep those two apart deliberately, and we only ever read these patterns against the runner's own message. If we matched them against the combined text, then any repository that happened to print the words "guest control stream closed" in a test fixture or a log line could hand itself a verdict of "this was the platform's fault". That would be the same error this system exists to prevent, just pointed the other way: instead of blaming you for our fault, blaming ourselves for yours. Both are confidently wrong, and a customer acts on both.

So the rule is narrow on purpose. We say this was ours only when our own infrastructure reported that our own sandbox stopped answering.

We should also say plainly that we did not always get this right. Until recently these failures were filed as Unknown error, which reads to a customer as "your build broke and we cannot say why", and the automated diagnosis attached to them used to suggest reviewing microVM resource allocation and system logs, which no customer can reach and which was never their job. This category exists because that was wrong, and it is currently the single most common way an Orbit build fails, which makes it the most important thing on this platform for us to be honest about.

Why Your Build Log Is So Short

A build log for one of these is often only a line or two, sometimes just:

[guest-init] phase=download ts=...

That is not a truncated log and nothing was hidden from you. It is the complete record of everything the guest managed to send before it stopped sending. The log ends where the sandbox ended.

This is the opposite of the usual situation, where a short log means your build printed nothing. Here it means we lost the machine that was doing the printing. The deployment page says so beside the log, because a two-line log presented without that sentence looks exactly like a build that did nothing, and it would send you to inspect a build command that was working fine.

For what a full log normally contains and how to read it, see Viewing Build Logs.

What We Do About It Automatically

We retry it. This is the only failure caused by us that is worth retrying, and the reasoning is worth stating because it is the same reasoning that makes us refuse to retry other categories.

We do not retry a failure that will reproduce itself. If our network policy refused a destination, the same policy refuses it on the next run. If a configured root directory does not exist, it does not exist next time either. Retrying those spends your build minutes to reach an identical result, so we do not do it.

A dead sandbox is the opposite shape. The next attempt gets a new machine, booted fresh, on a host chosen again. It is not a repeat of the same attempt, it is a genuinely different one, and it usually succeeds.

Two details about that retry matter to you:

It happens even if your project has automatic retries switched off. The project-level retry setting governs whether you want your build minutes spent again on a failure with your name on it. This is not that. You did not opt into it, you could not have prevented it, and you cannot see it, so we retry it on our own account regardless of that setting. If you have configured more retries than that, your setting still applies: we raise a floor, we never lower your ceiling.
It is bounded at one. We will not silently loop. One fresh sandbox is enough to tell a one-off host fault, which is the overwhelming majority, apart from something that fails every time. If the second attempt dies the same way, that is evidence, and evidence should reach a person rather than be eaten by a retry loop.

The deployment page tells you which of these you are in. If a retry is queued it says so and you need do nothing. If it is not, it invites you to retry when you are ready.

What You Should Do

In order:

Usually, nothing. If the page says a retry is queued, wait for it. The new deployment appears in the project's deployment list.
If no retry was queued, retry the deployment. The next attempt gets a fresh sandbox, which is generally all it takes.
Do not go looking for the cause in your code. There is nothing there to find. Changing your build command, your dependencies or your configuration in response to this failure will not make it less likely, and it risks breaking something that currently works.
If it fails the same way twice, open a ticket and quote the deployment ID. That is the case we want to see.

Why This Category Does Not Swallow Real Errors

A dead sandbox is compatible with almost any cause, including causes that are genuinely in your build. If we treated it as the first explanation for everything, we would start telling customers "this was ours" about builds that failed on their own merits, and that is a worse failure than saying nothing.

So this is the last thing we check, not the first. Anything your build actually said takes precedence:

A build whose output says JavaScript heap out of memory and then takes the machine down with it is filed as Out of memory, not as a sandbox failure.
A build that printed a TypeScript error before the machine went is still a Compile error.
A registry timeout is still a Network error.
A build that ran to completion and exited non-zero sent done normally. The guest was alive and reported a result, so that is your build failing, and we file it as whatever it was.

This category claims only the narrow population where our runner says the sandbox stopped answering and the log offers no other explanation at all. It converts the part of "Unknown error" that is provably ours and leaves the rest alone.

When It Happens Twice

Two identical failures on two different machines is a different fact from one, and it deserves a different response.

Most of the time this still turns out to be ours, and a ticket is how it reaches the people who can look at the host. But there is one honest exception worth naming: a build can take its own sandbox down. A process that exhausts the machine's memory violently enough can kill the guest before anything has a chance to write an out-of-memory message, and from the host's side that looks identical to a machine that failed on its own.

We do not guess between those two from a single occurrence, because guessing would mean either blaming you without evidence or absolving ourselves without evidence. A second identical failure is the signal that makes the question worth answering properly, with the deployment in front of us.

So if you see this twice on the same project, open a ticket from the panel or email support and quote the deployment ID. If it turns out your build is taking the machine down, Troubleshooting Failed Builds covers reducing build memory use, and we will tell you that plainly rather than leaving you to discover it.