werkator/docs/prs/2026-09-10-PR#48-build-timeout.md

WARNING: This document describes only the change applied in this PR. It may already be outdated once the next PR is merged. Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.

The Problem

A build that hangs blocks the machine until somebody notices.

On the production instance main@jvm of one repository sat at running for 68 minutes at a system load of 0.17 — it was not computing anything. It held the single executor slot the whole time, so two builds of another repository waited behind it, and the rollout that finally cleared it was a restart. Nothing in Werkator would have ended it: a build runs until its process exits, is cancelled by hand, or the server shuts down.

Non-Goals

• No detection of why a build hangs. A timeout is a deadline, not a diagnosis.
• No per-command limit. Five commands must not each be allowed the full time, so the limit is one deadline for the whole build.
• No retry. A build stopped by the clock stays stopped; repeating it unasked would burn the slot again.

The Scenarios

Feature: a build that runs too long

Background

• executor is an instance-level section and is stripped from a branch's own configuration, so a branch cannot write it.

Scenario#48.01: Stopped by the clock is its own outcome

So that it can be told apart from a failure and from a cancellation.

• Given a build that exceeds the time it is allowed
• Then its result is timed_out, not failed — nothing about the work failed, it did not finish — and not cancelled, because nobody decided to stop it
  • and it is terminal, and unlike interrupted it is not re-enqueued on its own
  • and it is published to Gitea as failure, because nobody will re-run it and a commit whose build never finished is not a green one
  • and the page shows it in the watch colour, between a failure and a nothing

Verified by

• BuildStatusTest

Scenario#48.02: The host names a ceiling, a build may ask for less

So that a repository can promise to be quick without being able to occupy the machine.

• Given executor.buildTimeout: 30m on the host
• When a build definition sets timeout: 10m
• Then the build is allowed ten minutes
• When it sets timeout: 4h
• Then it is allowed thirty minutes — a longer wish is capped, not refused
• Given a branch's own committed configuration
• Then it may set builds.<name>.timeout, which is a description of its own build
  • and it cannot set executor.buildTimeout, which is stripped with the rest of the section
• Given neither side setting one
• Then the build may run as long as it likes, which is what an empty ceiling means

Verified by

• BuildTimeoutsTest
• ConfigLoaderTest

Scenario#48.03: The slot is free again

• Given a build sleeping far longer than it is allowed
• When the deadline passes
• Then its process tree is terminated, the live log says why, and the result is timed_out
  • and the executor slot is released, so the next build starts
• Given a build that finishes in time
• Then nothing about it changes

Verified by

• BuildExecutorTest

The Solution

BuildTimeouts.effective is the whole rule, in one place: the smaller of the host's ceiling and the build's own wish, either side empty meaning "no opinion", both empty meaning no limit. The asymmetry is not enforced by stripping the key — it is enforced by taking the minimum, which is why builds.<name>.timeout is one of the few settings a branch may write. A branch is trusted to describe its own build; it is not trusted to occupy the machine because a value in a file said so.

The deadline is computed once, where the build's configuration is resolved, and each command then waits bounded by what is left of it. On expiry the process tree is destroyed exactly as a cancellation destroys it, the live log says so, and the command sequence ends.

BuildStatus.TIMED_OUT is new. It costs a branch in the Gitea mapping, a colour, and one entry in the status vocabulary the page knows — and it buys a build history in which a timeout does not look like a failing test.

Open Questions

• The default ceiling of 30 minutes is a guess at what "far too long" means here, taken from build times of a few minutes. A host with a genuinely long build has to raise it, and will notice the first time.
• Nothing warns when a build is close to its deadline. A build that regularly finishes at 29 minutes under a 30-minute ceiling is a problem waiting for a slow day.

Additional Changes

• Version 1.6.0 and its release-notes entry, since this is what gets deployed.
• The three configuration places are in sync: the data classes, the init templates, and docs/configuration.md.