Our Build Machine Could Not Run This

If your deployment says Our build machine could not run this, the machine we started for your build reported that something was wrong with itself, before or instead of anything going wrong with your code. That is a fault at our end. Nothing in your repository, your environment variables or your project settings caused it, and there is nothing there for you to change.

This guide explains what that machine is, the three ways it can tell us it is not fit to run your build, how we know the fault was ours rather than yours, why we do not automatically try again for this one, and what to do next.

What the Build Machine Is

Every Orbit build runs in its own microVM: a real virtual machine with its own kernel, started for your build and destroyed when it finishes. One build, one machine, then gone.

That machine boots from a prepared disk image. The image is where the toolchains live: Node, the package managers, and the compilers and runtimes for the other ecosystems we support. It is built from a recipe we keep under version control, and it is installed onto each build host separately from the software that schedules your builds. Those two things ship by different mechanisms, which is the root of the most common version of this failure.

The machine also prepares a workspace for your source, resolves the directory your project builds from, and reports back to the platform over a private channel. Each of those is a thing the machine does for itself, on your behalf, and each of them is a thing it can discover it cannot do.

What Actually Happened

One of three things, and all three are ours.

The image did not carry a program your project needs. We work out which runtime your project builds with, we choose the command, and then the machine checks that the program that command starts is actually installed before it does anything else. If it is not, it stops immediately and says which program, which framework and which command. This is what a build host running an older image looks like from the outside: the recipe that builds the image installs the toolchain, and this particular machine was built before that.

The failure arrives early on purpose. Without that check the build would run on, fetch your source, restore a cache, and eventually die deep inside your install or build step with nothing more useful than a missing program, which reads exactly like a mistake in your project. Refusing at the top, and saying which program and whose fault it is, is the difference between a wasted afternoon and a support conversation that takes a minute.

The machine could not resolve its own workspace. Before any of your commands run, the machine has to establish where the build will happen and prove that it lies inside the workspace it prepared. If it cannot resolve that workspace at all, it refuses rather than continue: not being able to tell whether something is safe is not the same as it being safe, and a build machine that guesses at this is a build machine that can run your build somewhere it should not. This is ours by construction. The workspace is a filesystem we mount inside a machine we booted.

It reported a result we could not use. For projects that run a server, the build finishes, your application starts, and the machine sends us the address to route traffic to. If that message arrives without an address in it, the build has genuinely succeeded and we still cannot put it in front of anyone. Nothing you wrote composes that message. It is written by our software, and a message our software cannot read is our defect.

To you these are one event, which is why they carry one category. Which of the three it was is in the message shown above the panel on your deployment page, and that message is what support will ask you to quote.

Why We Can Say This One Was Ours

"It was our fault" is a claim a platform should have to earn, so here is the ground for it.

Most build failures are classified by reading text somebody else wrote: your compiler, your package manager, a registry. That reading can be wrong, and a category built on it can be argued with. These three are different. Each is decided by a marker our own code writes about our own infrastructure, and nothing in your build output is consulted at all.

That cuts in both directions, which is the point. A repository that happens to print one of those words in a test fixture cannot hand itself a verdict of "the platform was at fault", because the markers are read only from the channel our own build machine reports on, never from your build log. And a future reword of one of our sentences cannot quietly make this category stop appearing, which is how a failure like this ends up filed as an unknown error again.

We should say plainly what this replaced, because it is the reason the category exists. Until recently the first of these three failures reached you as an unclassified error, under a heading that read "We cannot tell you why this build failed", while the message immediately above it said "Nothing is wrong with your repository. Contact support." We knew. The answer was already written and we threw it away at the last step, and a product that declines to name a cause it is holding is worse than one that never had it.

Why Your Build Log Is Short

For the first two of the three, the decision is taken before your install or build command has run. Your source may not even have been downloaded. The log holds our own start-up lines and nothing else.

Nothing was truncated and nothing was hidden. Usually a short build log means a build that printed little, which sends you to inspect a build command that is working fine. Here it means your commands never started.

For the third, the log is complete and it shows a build that worked, because it did.

Why We Do Not Retry This One Automatically

We do retry some failures on our own account, without spending a decision of yours: when our sandbox dies mid-build, or when it could not be given a network address, the next attempt gets a brand new machine on a host chosen again, so a retry is a genuinely different attempt rather than a repeat.

This category is not like that, and saying so is more useful than a retry that cannot work. An image that does not carry a program will not carry it on the next boot either, because the image is the same image. Software that composes a message without an address in it will compose the same message again. Only one of the three is plausibly a one-off, and re-running the other two would spend your build minutes to reach an identical result.

So we would rather tell you the truth and hand you a reference than look busy.

What To Do

Contact support and quote the deployment. The deployment identifier is in the address bar of the page you are reading the failure on, and the message above the panel carries everything we need. For the image case it also carries a phrase our team uses to find the exact recipe that needs refreshing.

Do not spend time on your repository. Nothing in it caused this. If the build worked before and your last commit is small, that instinct is a good one and this is the one case where it will not help you.

Retrying is not forbidden, it is just unlikely to help. If you would rather try once before writing to us, nothing stops you, and if your project happens to be dispatched to a host with a newer image the first case can resolve on its own. We do not do it for you because we cannot promise it, and a promise we cannot keep is worth less than a clear answer.

Related

• Our build sandbox failed covers the different case where the machine stopped answering partway through, which we do retry automatically.
• Our build sandbox got no address covers the case where no machine was ever usable because the build network had no address to give it.