werkator/docs/prs/2026-09-09-PR#40-system-page-charts-and-disk-usage.md

WARNING: This document describes only the change applied in this PR. It may already be outdated once the next PR is merged. Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.

Related Links

• RFC 0002 — the evaluation and the drafts behind the last two changes; its decisions A1 and B1 are what this PR implements.

The Problem

Five things the System page could not say, asked for in one breath and answered in one PR. The first three are small; the last two needed an evaluation first, which is RFC 0002 and part of this branch.

The gauge charts only ever showed one colour for the value line. What looked like a red line in the 24 h, 7 d and 30 d ranges was the maxima line, permanently drawn in the critical colour whatever it showed. So the very ranges a capacity question is asked in suggested a level that was not being stated, and the 1 h and 4 h ranges — the ones looked at while something is going on — said nothing about level at all.

A gauge card answered "what is the machine doing now" and, one line further down, "what was the single worst moment". The number in between, what the machine usually does over the shown range, was only readable by eyeballing the chart.

The sizing advice waited for seven days of history before saying anything. On a freshly deployed instance that is a week of a grey "collecting" chip, although a single day already covers a full daily build cycle and a rough answer beats none.

The disk gauge said how full a disk is and nothing else. When it turned yellow there was no lever: build workspaces, unpacked build environments, artifacts and the checkouts all live under directories Werkator created, and none of them was broken out. The one existing number, repoSizeGib, is the sum of the primary checkouts, so it neither adds up to the gauge nor separates what can be pruned from what cannot.

The last gap was the budget itself. Since PR#16 the gauge showed the tightest of the user quota, the group quota and the volume. That is right for the number and wrong for the explanation: on a webspace the quota is normally binding, so the gauge shows the quota — and if the volume then fills up through other users of it, the volume becomes the tighter candidate and the gauge switches. Three things happened at once, none of them announced: the total jumped from a few hundred GiB to a couple of TiB, the percentage dropped although the situation had got worse, and the disk's min/max/avg were reset on purpose so the old ceiling would not survive.

Non-Goals

• The sizing formulas themselves are unchanged; only when they are applied and how the basis is labelled.
• No average for the disk gauges: a disk's mean over an hour is its current fill, and would only add a number that never differs from the one above it.
• No new sampling for the breakdown. Nothing is measured every 15 seconds that was not measured before.
• No per-repository split of the breakdown; the dialog answers "which category", not "which repository". RFC 0002 keeps that as a later addition inside the same dialog.
• Docker images and volumes and caches inside containers stay unattributed: they belong to the daemon and may not even be on the measured filesystem. They are part of "other", never guessed at.
• No new rule in the machine verdict (RFC 0002 option B3 was offered and rejected): the statement stays where the numbers are.

The Scenarios

Feature: reading a gauge chart

Background

• A gauge chart draws one series over the selected range; the y scale is the resource's total, so 80 % and 90 % of the height are where the gauge's own colours change.
• The aggregated ranges (24 h, 7 d, 30 d) draw a second, thinner line: the maximum within each bucket.

Scenario#40.01: A line above a threshold is drawn in that threshold's colour

So that the range in which a resource was tight can be seen without reading the peak line.

• Given a series that rises from 20 % of the total to 95 % and falls back
• When the chart is drawn, in any of the five ranges
• Then the line is cut where it crosses 80 % and where it crosses 90 %
  • and the piece below 80 % is drawn in the accent colour, the piece between the thresholds in the watch colour, the piece above 90 % in the critical colour
  • and the cut sits exactly on the threshold, interpolated between the two samples, so the pieces join without a gap
  • and the maxima line of an aggregated range is cut and coloured by the same rule

Verified by

• the pure part of drawSpark in werkator.js — no JS test harness exists in this repository; verified in the browser against a seeded history, light and dark

Feature: what a resource usually needs

Scenario#40.02: The mean of the shown range stands next to the current value

So that a spike can be told apart from a level.

• Given a history range with values recorded for CPU and RAM
• When GET /api/system/history is asked for that range
• Then the response carries an averages entry per series, the mean of the values column
  • and the mean ignores gaps, and a series with no value at all has none
  • and on an aggregated range the bucket averages are averaged, never the maxima, because those answer a different question
• And when the system page renders a CPU or RAM card
• Then the mean appears right-aligned in the row of the current value, marked with the average sign and the range in its tooltip
  • and a disk card shows none

Verified by

• HistoryAveragesTest
• SystemApiControllerTest

Feature: sizing advice earlier

Scenario#40.03: The advice starts after one day and says how much it is worth

So that a new instance is not silent for a week.

• Given less than a day of quarter-hour history
• Then every resource is unknown and the reason reads collecting history: 0.5 of 1 day
• Given at least one day but less than seven
• Then the advice is computed as usual
  • and capacity.preliminary is true
  • and the basis line reads sizing from 3 d of history — preliminary, reliable from 7 d
• Given seven days or more
• Then capacity.preliminary is false and the basis line reads as before

Verified by

• CapacitySizingTest
• UiViewsTest
• UiControllerTest

Feature: what fills a disk

Background

• Werkator owns four kinds of directory: the build worktrees and the unpacked build environments under a repository's .git/werkator/, the artifact store under the state home, and the checkouts themselves.
• The usage walk is throttled to every 40th sample, as the repository-size walk always was.

Scenario#40.04: One walk, tallied by subtree

So that a breakdown costs no more than the size number it replaces.

• Given a repository with files in its worktrees, in its build environments, and elsewhere
• When the usage walk runs
• Then each regular file is counted into the first subtree it lies under, and everything else under none
  • and a directory that does not exist tallies nothing rather than failing the sample

Verified by

• DiskUsageTest

Scenario#40.05: Every disk gets what lies on it, and one honest rest

So that the dialog never claims an authority it does not have.

• Given repositories on one file store and the artifact store on another
• When the breakdown of a disk is built
• Then it holds only the categories measured on that disk's own file store
  • and other is the used space minus what was explained, never negative
  • and a disk with nothing of Werkator's on it is all other
• And when the disk card's button is used
• Then a dialog shows the slices as a donut and as a legend with sizes and shares, naming when the walk ran

Verified by

• DiskUsageTest
• SystemMetricsCollectorTest

Feature: quota free, volume full

Scenario#40.06: The quota keeps the scale, the volume marks its ceiling

So that a filling volume is stated instead of silently moving every number.

• Given a group quota that still allows 342 GiB
  • and a volume behind it with 1 GiB left
• When a sample is taken
• Then the shown total, used and free stay the quota's
  • and physicalFreeGib/physicalSource carry the volume's headroom
  • and the level is judged against used plus that headroom, so the card turns critical
  • and the bar marks where the headroom ends and hatches the rest
  • and the note reads "the volume nearly full: 1.00 GiB left, the group quota mih09 still allows 342.00 GiB"
• Given a volume with more room than the quota allows
• Then none of this appears and the card is exactly as before

Verified by

• SystemMetricsCollectorTest
• UiViewsTest

The Solution

drawSpark no longer maps a segment straight to screen coordinates. It keeps the points in value space, cuts every gap-free segment at the two thresholds with an interpolated crossing point, and draws each piece as its own polyline carrying its level in the class name. The maxima line goes through the same function, which is why it stops being red by decree. The level thresholds stay the single ones utilizationLevel already owned; the CSS only adds the two colour overrides per line kind.

HistoryAverages is the counterpart of HistoryPeaks — the same shape, the same place in GET /api/system/history, computed on the server so it can be tested in Kotlin rather than only in a browser. The card renders it in JavaScript alone, like the peak line: the server has no range to average over.

CapacitySizing.MIN_DAYS drops from seven to one and RELIABLE_DAYS takes over the seven, as a label rather than a gate. CapacityAdvice.preliminary carries the distinction into the API, and UiFormats.adviceBasis and its JavaScript twin render the same three cases.

DiskUsage is the new pure part: tallyBytes walks a directory once and sums each file into the first of the given subtrees it lies under, and breakdownOf turns those sums plus the disk's used space into the slices, other included. The collector's throttled probe now returns a UsageMeasurement — the repository size and the categories per file store — instead of one number, so the breakdown is a by-product of the walk that already happened; the artifact store is the one subtree that is new, and it is shallow.

readDisk stops treating the volume as one candidate among equals. Where a quota applies it is the budget, and the tightest quota wins among quotas; the volume only is the budget where no quota applies, as before. When the volume will run out before the shown budget does, it becomes the physical headroom instead: DiskMetrics.levelCeilingGib is what the level is judged against, while the total, the bar and every text stay on the budget being managed.

On the page the disk card gains three things: a marker and a hatched tail on the bar, a note in the critical colour, and a button opening one shared <dialog> that renders the donut and the legend from the polled snapshot. The dialog never asks the server for anything; it shows what the last walk found and says when that was. UiFormats.physicalHeadroomNote/headroomMarkPercent and their JavaScript twins render the same texts and positions, as the invariant requires.

Open Questions

• The average is a plain mean over the range, not weighted by how long each bucket actually covered; with a gap in the history the two differ slightly. Weighting would be more correct and less obvious to read.
• The maxima line is now grey while a range stays below 80 %, where it used to be red. It is more honest and a little less conspicuous.
• The breakdown is attributed by file store, which is one level coarser than the mount point. Two monitored directories on the same file store would each be shown the same categories; there is no such configuration today.
• bindingDiskSpace is still the tightest-wins helper and is still used to pick among quotas; its name now describes less than it did.

Additional Changes

• The field(root, name) helper moved from inside initSystemPage to module scope in werkator.js, so the new render functions can use it.
• The gauge-bar fill got a class of its own: the level colours had been applied by .gauge-bar div, which would have painted the new marker and tail as well.