fix: Geisterzaehler nach Neustart + rkllm-Leck in /api/search/answer

Aufgefallen beim Nachsehen, warum die GUI bei 29:32 Audio 2h34min Analysezeit zeigte.

1. Ein systemctl restart SIGKILLt alle Kindprozesse, die DB-Zeile blieb aber auf
   analysis_status=pending mit gesetztem analysis_started_at. Die GUI zaehlte dadurch
   ewig hoch fuer Arbeit, die es nicht mehr gab. db.failStaleJobs() setzt beim Start
   pending/transcribing auf error - zum Startzeitpunkt kann kein Job legitim laufen.

2. /api/search/answer lud rkllm und gab es nie frei: 1h48min lang 82,6% RAM belegt,
   nur 237 MB verfuegbar. Genau der Zustand, den analyzeRecording und rediarize mit
   llm.stopServer() bewusst vermeiden, weil der TitaNet-Helfer daneben an OOM stirbt
   (auf #71 seinerzeit 48 Rohsprecher). stopServer() jetzt im finally der Route.
This commit is contained in:
2026-07-31 15:25:49 +00:00
parent 914ab2ddc8
commit 963ce427da
3 changed files with 57 additions and 1 deletions
+15
View File
@@ -191,6 +191,21 @@ export async function setTranscript(id, transcript, words) {
await replaceWords(id, words || []);
}
// Every in-flight job dies with the service: systemd SIGKILLs the whisper,
// spk_embed and rkllm children on stop, and nothing resumes them. The row
// however still reads 'pending' with a start stamp, so the GUI shows
// "KI-Analyse läuft…" and a timer counting up forever on work that no longer
// exists (observed: 2 h 34 min on a 29-minute recording after a deploy
// restart). Called once at boot — at that moment no job CAN legitimately be
// running — so the retry button appears instead of a phantom counter.
export async function failStaleJobs() {
const [a] = await pool.query(
"UPDATE recordings SET analysis_status='error' WHERE analysis_status='pending' AND analysis_started_at IS NOT NULL");
const [t] = await pool.query(
"UPDATE recordings SET status='error' WHERE status='transcribing'");
return { analyses: a.affectedRows, transcriptions: t.affectedRows };
}
// Stamp the wall-clock start of a processing run (transcribe+diar+LLM). The
// GUI shows a live counter from here; updateAnalysis('done') freezes the total
// into analysis_ms. Called at every trigger (record-stop, retranscribe, reanalyze).
+12 -1
View File
@@ -493,7 +493,15 @@ app.post('/api/search/answer', wrap(async (req, res) => {
const q = String((req.body && req.body.q) || '').trim();
if (!q) return res.status(400).json({ error: 'q fehlt' });
const { hits } = await search.search(q, { limit: 20 });
res.json(await withNpuLock('search-answer', () => search.answer(q, hits)));
// stopServer() afterwards is NOT optional: rkllm stays resident with ~6GB
// once loaded, and the next speaker-embedding pass then OOM-dies next to it
// (analyzeRecording/rediarize free it for exactly that reason). A search
// answer left the model squatting 82% of RAM for hours.
try {
res.json(await withNpuLock('search-answer', () => search.answer(q, hits)));
} finally {
llm.stopServer();
}
}));
app.get('/api/search/status', wrap(async (req, res) => {
const [[{ n }]] = await db.pool.query('SELECT COUNT(*) AS n FROM passages');
@@ -718,6 +726,9 @@ app.get(/^(?!\/api|\/media).*/, (req, res) => res.sendFile(path.join(__dirname,
const PORT = process.env.PORT || 3000;
db.ensureSchema()
.catch(e => console.error('ensureSchema:', e.message))
.then(() => db.failStaleJobs())
.then(r => { if (r && (r.analyses || r.transcriptions)) console.log(`boot: ${r.analyses} abgebrochene Analysen, ${r.transcriptions} abgebrochene Transkriptionen als Fehler markiert`); })
.catch(e => console.error('failStaleJobs:', e.message))
.finally(() => app.listen(PORT, '0.0.0.0', () => {
console.log(`callrec listening on :${PORT}`);
// Retention sweep at boot + daily. unref() so it never holds the process open.