RAM Sizing by waiting und proper HDD Quota steps #58

2026-09-12T20:52:48+02:00

mi commented

2026-09-12 20:52:48 +02:00

No description provided.

mi added 6 commits 2026-09-12 20:52:49 +02:00

metrics: sample oom_kill per bucket into the history

werkator build succeeded after 02:06

Details

d4b6f3dc4b

PSI measures waiting, and an OOM kill is the one memory shortage nobody
waits for: the process does not stall, it dies. RFC 0004 A1 therefore
judges RAM from memoryPressure alone but starts sampling oom_kill now,
so the data exists when a follow-up reads it. The sizing does not read
it -- the counter is machine-wide, and on a shared hive it counts every
webspace's kills, not Werkator's.

/proc/vmstat counts monotonically since boot, so what is recorded is the
difference to the previous sample: the count of this bucket rather than a
number that only grows. The first sample of a process has nothing to
compare against and records no point, a counter that went backwards is a
reboot and counts as no kill, and a kernel without the field leaves the
column absent like any other unreadable source. A persisted history from
before the column simply starts it -- the tiers carry the union of what
was sampled.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

metrics: size the machine in the steps it can be booked in f3ed2dae41

The production host was advised to shrink a 20 GiB quota to 11 GiB.
There is no 11 GiB: a quota is booked in steps of 10 GB, the physical
storage of a Managed Server in steps of 250 GB. Advice that names a size
nobody can order is not advice.

CapacitySizing.of therefore takes the bookable steps as a parameter
(RFC 0004 B1) and byPeak, ram and disk round to them. The caller derives
them from the hosting -- HostingEnvironments.bookableSteps -- so the
object stays pure and free of a dependency on the server package, no
configuration key is added, and the steps and the order dialog come from
one source that cannot drift apart. Which of the two disk steps applies
follows from what a disk's budget is: a quota is raised in 10 GiB, a
volume in 250 GB. Everything is carried in GiB, the unit the metrics and
the gauges are in, so the GB step is converted once on the way in.

Unlike the hosting itself the steps are not kept from an untrusted
viewer: they name nothing, and one machine giving two different
recommendations depending on who asks would be worse than either.
Without a recognised hosting every resource keeps the whole unit it was
advised in before, so nothing changes off Hostsharing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

metrics: judge RAM by whether anything waited for it bdf28c7391

The production host was shown in red -- "upgrade to 8 GiB, peak 5.51 of
5.79 GiB (95 %)" -- while its memory stalls stayed eight times below what
its CPU was doing under a fits. That is not a wrong threshold but two
yardsticks for two resources: CPU is sized from waiting, RAM was sized
from a fill level. A fill level is not a shortage. JVMs and the kernel
take what is there, page cache is not even in the measured
MemTotal - MemAvailable, and nothing is wrong as long as nobody waits.

RAM now reads the memoryPressure column the history has held since PR#27
(RFC 0004 C3): a quarter hour is stalled when its average stall reached
MEMORY_OVERLOAD_PERCENT -- shared with SystemHealth, so a machine the
health verdict calls healthy cannot be red in the sizing -- and RAM is
tight from 1 % and short from 5 % of stalled quarter hours, the latter
being the share CPU already uses. Health's pressure level (2 %) is
deliberately not the mark: eighteen seconds of waiting per quarter hour
during a build is a build evicting page cache.

The peak is only named in the reason now. It keeps two jobs: it sizes a
raise when one is due -- the PR#42 formula, rounded to the bookable step
and at least one step up, because a stall proves the peak was the
ceiling -- and it may still prove slack, which is why a downsize stays
peak-based and now also requires that not one quarter hour stalled. Swap
occupancy plays no part: cold pages held for days are not a shortage
either. A history without the column says so rather than falling back to
judging the fill level.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

metrics: advise a downsize only when the smaller size still fits 143e3df28d

"Downsize to 11 GiB", amber, on a 20 GiB group quota holding 7.57 GiB.
There is no 11 GiB, and the bookable 10 below it would have put the same
peak at 76 % -- above DISK_TARGET_RATIO, inside the watch band this very
advice would then flag. A downsize that immediately produces a tight is
not advice.

A downsize therefore requires, beyond the peak staying below
DOWNSIZE_RATIO, that the recommendation fits into the next bookable size
below the current one (RFC 0004 D1): the largest step multiple strictly
below it, so a quota of 12 GiB steps down to 10 rather than to 2.
Otherwise the verdict is fits -- and the reason says why, because silence
would read as "nothing to see here" on a gauge showing 38 %. Where the
step is absurd against the peak, a volume booked in 250 GB pieces holding
8 GiB, there is no smaller size to name at all and the reason says that
instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

metrics: the production host of 2026-09-12 as a regression case

werkator build succeeded after 01:40

Details

f2d24d24d3

The two verdicts that prompted RFC 0004, as one test: 6 GiB booked of
which 5.79 are visible, a 5.51 GiB peak, the measured stall distribution
of the 30-day window -- 23 % of the quarter hours above 0.5 %, 15 % above
1 %, 7 % above 2 %, 1 % above 5 %, none reaching 10 % -- and the 20 GiB
group quota holding 7.57 GiB. Both were advice before: "upgrade to
8 GiB", red, and "downsize to 11 GiB", amber. Both are fits now, and the
disk says in its reason why it is not being shrunk.

The series is built to the numbers rather than copied from the host, so
the test states the case it is making and a failure names it.

The chip twins need no change: a size booked in GB is shown in the GiB
everything else is in, and Kotlin's %.2f and werkator.js's toFixed(2)
render 232.83 GiB alike -- asserted now, since the volume step is the
first recommendation that is not a whole number.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

release 1.12.0, and the documentation of RFC 0004

werkator build succeeded after 01:40

Details

cd258538f9

The PR-doc with the four decisions and their scenarios, the release notes
entry, the version a running instance is identified by, and the RFC's
implemented line. AGENTS.md and the architecture skill now state the
yardstick as an invariant: a resource is sized by whether anything waited
for it, never by how full it is, and every advised size is one that can
be booked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

mi merged commit 9f81b191bd into main

2026-09-12 20:53:27 +02:00

mi deleted branch sizing-by-waiting

2026-09-12 20:53:27 +02:00

mi referenced this pull request

2026-10-05 18:46:31 +02:00

RAM and disk sizing advice split into their steps, every advice chip tested (v1.37.4) #123

mi referenced this issue from a commit

2026-10-05 18:58:31 +02:00

RAM and disk sizing advice split into their steps, every advice chip tested (v1.37.4) (#123)

Sign in to join this conversation.