werkator/docs/prs/2026-09-12-PR#58-sizing-by-waiting-in-bookable-steps.md

WARNING: This document describes only the change applied in this PR. It may already be outdated once the next PR is merged. Historic PR-documentation is not maintained along with new PRs — treat it as a snapshot, not as current documentation.

Related Links

• RFC 0004: Sizing by Waiting, in Bookable Steps — the evidence and the four decisions this PR implements (A1, B1, C3, D1).

The Problem

On 2026-09-12 the System page of the production host gave two recommendations that do not hold.

RAM: "upgrade to 8 GiB", red. The reason read "peak 5.51 of 5.79 GiB usable (95 %) in 4 d, 6 GiB booked". The measured value is MemTotal − MemAvailable, so page cache is not even in it; what filled the memory was a Gradle build, and it was allowed to. A fill level is not a shortage: JVMs and the kernel take what is there, and nothing is wrong as long as nobody waits.

The history already holds the number that says whether anybody waited — PSI some avg60 of /proc/pressure/memory, sampled as memoryPressure since PR#27 — and it said no. Over 30 days of quarter hours, 1 % averaged a stall of at least 5 % and none reached 10 %. On the same machine 24.7 % of the quarter hours averaged a CPU stall of at least 5 % and 16.2 % at least 10 % — and the CPU was judged fits, because it is sized from waiting and the load never exceeded the threads. The resource that was waited for eight times less was the one shown in red. That is not a wrong threshold; it is two different yardsticks for two resources.

HDD: "downsize to 11 GiB", amber. The reason read "peak 7.57 of 20.00 GiB (38 %) in 4 d" on a group quota. There is no 11 GiB at Hostsharing: a quota is booked in steps of 10 GB, the physical storage of a Managed Server in steps of 250 GB. The next bookable size below, 10 GiB, would put the same peak at 76 % — above DISK_TARGET_RATIO (70 %) and inside the watch band the advice itself would then flag. A downsize that immediately produces a tight is not advice.

Non-Goals

• No change to the CPU sizing of PR#42: it was already sized from waiting, and it is the yardstick RAM is being brought onto.
• No change to the hosting detection or to the order mail: what the page composes and whom it names stays as PR#42 left it.
• No configuration key for the bookable steps: the caller already knows where the instance runs, and a step maintained by hand per instance would produce quietly wrong advice for values the detection knows.
• oom_kill is sampled but not judged: the counter is machine-wide, so on a shared hive it counts every webspace's kills, and reading it into a verdict waits for experience with it.
• No change to SystemHealth: its live thresholds stay, and the sizing shares one of them rather than inventing a second scale.

The Scenarios

Feature: a machine is sized by what it waited for, in sizes that can be booked

Background

• A quarter hour is one bucket of the 30-day tier of the metrics history; the sizing reads its averages.
• A quarter hour is stalled when its average PSI memory stall reached MEMORY_STALL_PERCENT (10 %, shared with SystemHealth.MEMORY_OVERLOAD_PERCENT).
• A bookable step is the size a resource is actually ordered in: at Hostsharing 10 GiB per quota, 250 GB per volume, 1 GiB of RAM, one hardware thread.
• The fill peak is the highest ramUsedGib of the window — MemTotal − MemAvailable, page cache excluded.

Scenario#58.01: memory is judged by waiting, not by fill level

So that a build using the memory it was given is not read as a machine running out of it.

• Given a 30-day window whose fill peak reached 95 % of the usable RAM
  • and not one quarter hour was stalled
• When the sizing runs
• Then the verdict is fits
  • and the reason names the peak and says that no quarter hour stalled
• And when at least 1 % of the quarter hours were stalled
• Then the verdict is tight
• And when at least 5 % of them were stalled
• Then the verdict is upgrade
  • and the recommended size is the peak plus its headroom plus the host's reserve, rounded to the bookable step and at least one step above what is booked

Verified by

• CapacitySizingTest: a RAM peak at 90 % of the total that nothing waited for is not a shortage
• CapacitySizingTest: memory stalls in a twentieth of the quarter hours are an upgrade to the peak with headroom
• CapacitySizingTest: a hundredth of the quarter hours stalled is tight, not an alarm
• CapacitySizingTest: a quarter hour below the overload level is a build evicting page cache, not a shortage
• CapacitySizingTest: a history without the pressure column says so instead of judging the fill level
• CapacitySizingTest: the production host of 2026-09-12, both verdicts it got wrong

Scenario#58.02: a recommendation names a size that can be booked

So that the order dialog offers a line the service desk can act on.

• Given the instance runs on a recognised Hostsharing webspace
• When a raise is due for a disk whose budget is a quota
• Then the recommended size is rounded up to the next multiple of 10 GiB
• And when the disk's budget is the volume itself
• Then it is rounded up to the next multiple of 250 GB, named in the GiB the rest of the page is in
• And when no hosting is recognised
• Then every resource keeps the whole unit it was advised in before

Verified by

• CapacitySizingTest: a raise is rounded to the size the hosting books in
• CapacitySizingTest: a volume is advised in the steps its physical storage is booked in
• CapacitySizingTest: RAM is booked and raised in whole GiB
• UiViewsTest: advice chips read like werkator.js adviceLabel

Scenario#58.03: a downsize is only advice when the smaller size still fits

So that no recommendation produces the very warning it was meant to avoid.

• Given a 20 GiB group quota whose peak was 7.57 GiB
• When the sizing runs with a bookable step of 10 GiB
• Then the verdict is fits
  • and the reason says that the 10 GiB below would be tight
• And when the same peak sits on a 40 GiB quota, where 20 GiB would still keep the target headroom
• Then the verdict is downsize to 20 GiB
• And when the budget is a volume booked in 250 GB pieces and holds 2.2 GiB
• Then the verdict is fits with a reason saying there is no smaller size to book

Verified by

• CapacitySizingTest: a downsize the next smaller size would flag is no downsize
• CapacitySizingTest: a downsize that still leaves the headroom stays advice
• CapacitySizingTest: a volume whose next step is 250 GB away is simply fits
• CapacitySizingTest: the production host of 2026-09-12, both verdicts it got wrong

Scenario#58.04: OOM kills are recorded so that a later verdict can read them

So that the one memory shortage nobody waits for leaves a trace.

• Given the kernel counts oom_kill in /proc/vmstat
• When a sample is taken
• Then the history records the kills since the previous sample, not the counter since boot
  • and the first sample of a process records no point at all
  • and a counter that went backwards — a reboot — counts as no kill
• And when the kernel has no such counter, or the persisted history predates the column
• Then the column is simply absent or starts where the sampling did, and no sample fails

Verified by

• SystemMetricsCollectorTest: OOM kills are recorded as the count per sample, not as the counter since boot
• SystemMetricsCollectorTest: a rebooted counter counts as no kill, and a kernel without the counter as no column
• SystemMetricsCollectorTest: a persisted history from before the OOM column simply starts it

The Solution

RAM is judged by waiting (RFC 0004 A1, C3). CapacitySizing.ram reads the memoryPressure column of the quarter-hour tier — its averages, never its maxima, exactly as the CPU is sized — and counts the quarter hours whose average reached MEMORY_STALL_PERCENT. That constant is SystemHealth.MEMORY_OVERLOAD_PERCENT: sharing it is what keeps a machine the health verdict calls healthy from being red in the sizing. Health's lower pressure level (2 %) is deliberately not the mark — eighteen seconds of waiting per quarter hour during a build is a build evicting page cache, not a machine short of memory. tight follows from 1 % of stalled quarter hours in the window, upgrade from 5 %, which is the share CPU_WAITING_SHARE already uses.

The fill peak keeps two jobs and loses one. It is named in the reason, it sizes a raise when one is due — the PR#42 formula, rounded to the bookable step and at least one step above what is booked, because a stall proves the peak was the ceiling — but it no longer produces tight or upgrade. The one asymmetry is deliberate: a fill level cannot prove a shortage, but it can prove slack, so a downsize stays peak-based and additionally requires that not one quarter hour stalled. Swap occupancy plays no part at all; two gigabytes of cold pages held for days are not a shortage, which is why SWAP_PRESSURE_RATIO stays a live signal of SystemHealth and nothing more. A history without the column says so in its reason instead of quietly falling back to judging the fill level.

Sizes are the ones that can be booked (RFC 0004 B1). CapacitySizing.of takes a BookableSteps parameter and ram, byPeak and disk round to it. The caller derives it — HostingEnvironments.bookableSteps — so the sizing object stays pure and free of a dependency on the server package, no configuration key joins the three places that must stay in sync, and the steps and the order dialog come from one source that cannot drift apart. Which of the two disk steps applies follows from what a disk's budget is, a quota or the volume. Everything is carried in GiB, the unit the metrics and the gauges are in, so the 250 GB step is converted once on the way in and renders through the existing UiFormats/werkator.js twins unchanged. Unlike the hosting itself, the steps reach every viewer: they name neither webspace nor machine, and one instance advising two different sizes depending on who asks would be worse than either.

A downsize must survive its own advice (RFC 0004 D1). Beyond DOWNSIZE_RATIO, a downsize now requires that the recommendation fits into the next bookable size below the current one — the largest step multiple strictly below it, so a 12 GiB quota steps down to 10 rather than to 2. Otherwise the verdict is fits, and the reason says which of the two reasons applied: the size below would be tight, or there is no smaller size to book at all. The second is the volume case the RFC names, where the next step is 250 GB away from an 8 GiB peak.

oom_kill is sampled, not judged (RFC 0004 A1). PSI measures waiting, and an OOM kill is the one memory shortage nobody waits for: the process does not stall, it dies. SystemMetricsCollector records the difference to the previous sample — the count per bucket rather than a number that only grows — under the oomKills history column. The sizing does not read it: the counter is machine-wide, so on a shared hive it counts every webspace's kills, and a verdict on it waits until there is experience with the numbers.

Additional Changes

• MetricsHistory.BASE_KEYS gains the new column, so a history written before it simply starts it — the tiers carry the union of what was sampled and need no migration.
• The advice-chip twin test pins how a size booked in GB renders: Kotlin's %.2f and werkator.js's toFixed(2) both produce 232.83 GiB, the first recommendation that is not a whole number.

Follow-up PRs

• Reading oom_kill into the verdict, once there is experience with the counter on a shared hive.
• Whether an amber order line should be preselected in the order dialog.