LLM -> Qwen3-4B-Instruct-2507 (A/B-Sieger, Whole-Pass bis 32k)
- Selbst konvertiert (w8a8_g128, ctx 16384, RTX 3090 ~35 min, Kit wiederverwendet) - A/B vs Gemma: 3/3 Prompts besser; 21.8k-Whole-Pass fehlerfrei gewichtet, wo Gemma englisch abdriftete bzw. nur das Ende zusammenfasste - rkllm_server.cpp: ChatML-Template; llm.js: WHOLE_MAX_CHARS 32000, READY-Timeout 60s->180s (Qwen3 laedt ~96s) - E2E validiert: #70 Whole-Pass 15.4 min, #68 Kurzclip sauber - Rollback: .bak-gemmatmpl + Drop-in auf gemma3-Modell Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0113vDKdSNtD3Dwcw4QMu9yV
This commit is contained in:
co-authored by
Claude Fable 5
parent
2b63a6b02f
commit
a6e02aea44
@@ -74,7 +74,7 @@ int main(int argc, char** argv) {
|
||||
// without this the model gets the raw prompt with no turn markers and just
|
||||
// rambles generic filler. prefix/postfix wrap each user prompt; <bos> is
|
||||
// added by the runtime (add_bos_token=True at conversion).
|
||||
rkllm_set_chat_template(h, "", "<start_of_turn>user\n", "<end_of_turn>\n<start_of_turn>model\n");
|
||||
rkllm_set_chat_template(h, "", "<|im_start|>user\n", "<|im_end|>\n<|im_start|>assistant\n");
|
||||
printf("READY\n"); fflush(stdout);
|
||||
fprintf(stderr, "rkllm model loaded\n"); fflush(stderr);
|
||||
|
||||
|
||||
Reference in New Issue
Block a user