Commit bb6d47a
Paul C
v24.7.10: upgrader — survive the console tab closing before the post-install restart fires
Sponsor report (PapaSchlumpf, 2026-05-25): upgrading from 24.7.7 to
24.7.9 via the in-app console finishes the install but the dashboard
still shows the OLD version on every node. Closing the upgrade
console tab leaves the node at 24.7.7.
## Root cause
setup.sh ends with:
nohup bash -c "sleep 3 && systemctl restart wolfstack" \
>/dev/null 2>&1 &
The 3-second sleep gives the operator time to read the "**** UPGRADE
COMPLETE ****" + "Please Refresh your browser…" lines before the
service blinks. But the upgrade console is a pty spawned by
WolfStack itself; when the operator closes the browser tab, the
websocket closes, the pty is torn down, and every child in the pty
session is signalled. `nohup` only ignores SIGHUP — SIGTERM/SIGKILL
from cgroup teardown or session cleanup still land. If the operator
closes the tab within the 3 seconds, the pending restart dies with
the pty. The new binary is on disk; the service keeps running the
old one. Dashboard stays at 24.7.7.
PapaSchlumpf's screenshot shows the script reached "UPGRADE COMPLETE"
and "Please Refresh your browser if upgrading…" — exactly the
moment a reasonable operator closes the tab, before the 3-second
restart fires.
## Fix
Schedule the restart via a transient systemd timer so it's owned
by PID 1, not our pty shell. `systemd-run --on-active=3s` creates
a one-shot transient timer that fires in 3 seconds independently
of whatever spawned it — closing the browser tab cannot kill it.
Fallback for hosts without systemd-run (containers, minimal images):
`setsid bash -c "sleep 3 && systemctl restart wolfstack"
</dev/null >/dev/null 2>&1 &` — `setsid` puts the child in a new
session detached from the pty's controlling terminal, the best
a plain shell can do.
Applied to both setup.sh (the canonical installer) and
ibmsetup.sh (the IBM Cloud variant). Both are fetched fresh from
GitHub `master` by the upgrade console on every run, so once
this commit is on master the very next upgrade attempt picks up
the fix — PapaSchlumpf doesn't need to do anything manual to
recover, just re-run the upgrade from the console.
## Safety
* `bash -n setup.sh` and `bash -n ibmsetup.sh` parse clean.
* `cargo build` clean (2 pre-existing dead-code warnings unrelated).
* If `systemd-run` itself fails for any reason (e.g. dbus down),
the `||` falls back to `setsid` — same outcome as today,
minus the SIGHUP immunity that nohup provided. setsid is
strictly stronger than nohup against pty teardown.
* The new binary is already on disk before the restart logic
runs; if neither path fires the service, the operator's
existing fallback ("sudo systemctl restart wolfstack" printed
in the Manage section above) still works.
## Operator recovery
PapaSchlumpf can re-run the upgrade from the console — the 24.7.7
binary fetches a fresh setup.sh from master on every run, so the
fixed restart path kicks in immediately.1 parent a31825e commit bb6d47a
3 files changed
Lines changed: 47 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
| 3 | + | |
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1273 | 1273 | | |
1274 | 1274 | | |
1275 | 1275 | | |
| 1276 | + | |
| 1277 | + | |
| 1278 | + | |
| 1279 | + | |
| 1280 | + | |
1276 | 1281 | | |
1277 | | - | |
| 1282 | + | |
| 1283 | + | |
| 1284 | + | |
| 1285 | + | |
| 1286 | + | |
| 1287 | + | |
| 1288 | + | |
| 1289 | + | |
| 1290 | + | |
| 1291 | + | |
| 1292 | + | |
1278 | 1293 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2176 | 2176 | | |
2177 | 2177 | | |
2178 | 2178 | | |
| 2179 | + | |
| 2180 | + | |
| 2181 | + | |
| 2182 | + | |
| 2183 | + | |
| 2184 | + | |
| 2185 | + | |
| 2186 | + | |
| 2187 | + | |
| 2188 | + | |
| 2189 | + | |
| 2190 | + | |
| 2191 | + | |
| 2192 | + | |
| 2193 | + | |
| 2194 | + | |
| 2195 | + | |
2179 | 2196 | | |
2180 | | - | |
| 2197 | + | |
| 2198 | + | |
| 2199 | + | |
| 2200 | + | |
| 2201 | + | |
| 2202 | + | |
| 2203 | + | |
| 2204 | + | |
| 2205 | + | |
| 2206 | + | |
| 2207 | + | |
| 2208 | + | |
| 2209 | + | |
2181 | 2210 | | |
0 commit comments