LLM -> Qwen3-4B-Instruct-2507 (A/B-Sieger, Whole-Pass bis 32k)

- Selbst konvertiert (w8a8_g128, ctx 16384, RTX 3090 ~35 min, Kit wiederverwendet)
- A/B vs Gemma: 3/3 Prompts besser; 21.8k-Whole-Pass fehlerfrei gewichtet,
  wo Gemma englisch abdriftete bzw. nur das Ende zusammenfasste
- rkllm_server.cpp: ChatML-Template; llm.js: WHOLE_MAX_CHARS 32000,
  READY-Timeout 60s->180s (Qwen3 laedt ~96s)
- E2E validiert: #70 Whole-Pass 15.4 min, #68 Kurzclip sauber
- Rollback: .bak-gemmatmpl + Drop-in auf gemma3-Modell

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0113vDKdSNtD3Dwcw4QMu9yV

app/llm.js

@@ -32,7 +32,9 @@ function spawnServer() {
   proc.stderr.on('data', d => process.stderr.write('[rkllm] ' + d));
   ready = new Promise((resolve, reject) => {
     settleReject = reject;
-    const to = setTimeout(() => { if (!done) { done = true; reject(new Error('rkllm READY timeout')); } }, 60000);
+    // 180s: Qwen3-4B (5.3GB) needs ~96s to load on the capped clock — the old
+    // 60s killed it mid-load in an endless retry loop (Gemma was ~72s, barely fit).
+    const to = setTimeout(() => { if (!done) { done = true; reject(new Error('rkllm READY timeout')); } }, 180000);
     rl = readline.createInterface({ input: proc.stdout });
     rl.on('line', line => {
       if (line === 'READY') { if (!done) { done = true; clearTimeout(to); resolve(); } return; }
@@ -323,17 +325,15 @@ const estTokens = (s) => Math.ceil((s || '').length / 3.3);   // ~German chars p
 // and keeps proper nouns (Com4Tec/René/Tobi+Niklas verified on #70, 2026-07-16).
 const CHUNK_CHARS = 4200;
 const SINGLE_MAX_CHARS = 4800;   // below this: single combined call (full cleaning)
-// The 16K-converted Gemma model is RUN at a 12288 runtime context (rkllm_server
-// max_context_len) — but 20000 chars stays the whole-pass cutoff: it is a MODEL
-// limit, not a context limit. Verified on #70 (22.6k chars, fits in 12288 ctx):
-// above ~6k input tokens Gemma-3-4B ignores the instruction (English chat),
-// fixates on the END of the call (recency bias: summarized the closing car
-// smalltalk), and its JSON decays (curly quotes, repeated keys, repetition
-// loops). At ≤20k chars whole-pass quality is validated good (#69 era). Longer
-// recordings use the chunked map-reduce below, whose REDUCE prompt weights
-// topics by share across sections. Do NOT raise without re-validating content
-// quality — a bigger window does not make the model read evenly.
-const WHOLE_MAX_CHARS = 20000;
+// Whole-pass cutoff. Model-dependent, NOT just a context question:
+// - Gemma-3-4B fell apart above ~6k input tokens (English chat, recency bias,
+//   JSON decay) → its limit was 20000 chars even at 12288 runtime ctx.
+// - Qwen3-4B-Instruct-2507 (2026-07-16 A/B on #70) nailed the SAME 21.8k-char
+//   whole-pass in one call: correctly weighted German JSON, car smalltalk kept
+//   out of title/summary, concrete names — the failure mode is gone.
+// Budget at 12288 ctx: 12288 − 1536 gen ≈ 10752 tok ≈ 35k chars; 32000 leaves
+// margin. Beyond that the section-conglomerate path below takes over.
+const WHOLE_MAX_CHARS = 32000;
 
 // Split transcript into <=maxChars pieces on natural boundaries (labeled lines
 // first, else sentences, else hard char cut) so no chunk splits mid-word.

app/rkllm_server.cpp

@@ -74,7 +74,7 @@ int main(int argc, char** argv) {
     // without this the model gets the raw prompt with no turn markers and just
     // rambles generic filler. prefix/postfix wrap each user prompt; <bos> is
     // added by the runtime (add_bos_token=True at conversion).
-    rkllm_set_chat_template(h, "", "<start_of_turn>user\n", "<end_of_turn>\n<start_of_turn>model\n");
+    rkllm_set_chat_template(h, "", "<|im_start|>user\n", "<|im_end|>\n<|im_start|>assistant\n");
     printf("READY\n"); fflush(stdout);
     fprintf(stderr, "rkllm model loaded\n"); fflush(stderr);