Skip to content

fix(runtime-host): bind managed retirement to one service generation #3583

Description

@M4n5ter
English

Problem

The managed Runtime Host retirement path added by #3557 can report the identity of the Host that prepared retirement even if systemd briefly restarted a successor before the service stop completed.

The narrow sequence is:

  1. Host A accepts host.upgrade.prepare, fences admission, and returns its Epoch and PID.
  2. Host A exits abnormally before the service manager submits systemctl stop.
  3. Restart=always starts Host B, which may acquire the State Root writer.
  4. systemctl stop stops Host B; the service reaches a stable stopped state and the writer is released.
  5. The operator reports { kind: "retired", hostEpoch: A, pid: A } even though the final stop acted on Host B.

Writer exclusivity remains intact and the timing window requires an abnormal exit, so this is not a release blocker. The result is nevertheless not truthful about the retired service generation, and Host B may have performed recovery work before it was stopped.

Expected direction

Bind retirement completion to the service generation across prepare and stop. A fix may use service-manager restart/generation evidence, or acquire the State Root fence after Host A releases it and before a successor can take ownership. Simply attempting to acquire the writer fence while Host A is still draining is insufficient.

The final result should either prove that the prepared Host generation retired, or return an explicit identity-change/conflict result. Tests should exercise the abnormal-exit/restart sequence rather than only another PID value in a mock status.

Context: #3231, #3557 and the non-blocking P3 in the approving review on #3557.

简体中文

问题

#3557 增加的 managed Runtime Host retirement 路径可能在 systemd 于 stop 完成前短暂重启 successor 的情况下,仍报告最初准备退场的 Host identity。

窄小的触发顺序是:

  1. Host A 接受 host.upgrade.prepare,fence admission,并返回自己的 Epoch 与 PID;
  2. service manager 提交 systemctl stop 前,Host A 异常退出;
  3. Restart=always 启动 Host B,Host B 可能取得 State Root writer;
  4. systemctl stop 停止 Host B,service 达到稳定 stopped 状态,writer 也已释放;
  5. operator 仍返回 { kind: "retired", hostEpoch: A, pid: A },但最终 stop 实际作用于 Host B。

Writer exclusivity 没有被破坏,且该窗口需要异常退出,因此不阻塞 release。但结果没有真实描述退场的 service generation,Host B 也可能在被停止前执行过 recovery work。

期望方向

把 retirement completion 与 preparestop 之间的 service generation 绑定起来。修复可以使用 service-manager restart/generation evidence,或者在 Host A 释放 writer 后、successor 取得 ownership 前获取 State Root fence。仅在 Host A 尚未完成 drain 时尝试取得 writer fence 并不足够。

最终结果必须能够证明 prepared Host generation 已退场,或者返回明确的 identity-change/conflict。测试应覆盖真实的异常退出/重启顺序,而不是只在 mock status 中换一个 PID。

相关背景:#3231#3557,以及 #3557 approving review 中的非阻塞 P3。

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions