fix: Geisterzaehler nach Neustart + rkllm-Leck in /api/search/answer
Aufgefallen beim Nachsehen, warum die GUI bei 29:32 Audio 2h34min Analysezeit zeigte. 1. Ein systemctl restart SIGKILLt alle Kindprozesse, die DB-Zeile blieb aber auf analysis_status=pending mit gesetztem analysis_started_at. Die GUI zaehlte dadurch ewig hoch fuer Arbeit, die es nicht mehr gab. db.failStaleJobs() setzt beim Start pending/transcribing auf error - zum Startzeitpunkt kann kein Job legitim laufen. 2. /api/search/answer lud rkllm und gab es nie frei: 1h48min lang 82,6% RAM belegt, nur 237 MB verfuegbar. Genau der Zustand, den analyzeRecording und rediarize mit llm.stopServer() bewusst vermeiden, weil der TitaNet-Helfer daneben an OOM stirbt (auf #71 seinerzeit 48 Rohsprecher). stopServer() jetzt im finally der Route.
This commit is contained in:
@@ -191,6 +191,21 @@ export async function setTranscript(id, transcript, words) {
|
||||
await replaceWords(id, words || []);
|
||||
}
|
||||
|
||||
// Every in-flight job dies with the service: systemd SIGKILLs the whisper,
|
||||
// spk_embed and rkllm children on stop, and nothing resumes them. The row
|
||||
// however still reads 'pending' with a start stamp, so the GUI shows
|
||||
// "KI-Analyse läuft…" and a timer counting up forever on work that no longer
|
||||
// exists (observed: 2 h 34 min on a 29-minute recording after a deploy
|
||||
// restart). Called once at boot — at that moment no job CAN legitimately be
|
||||
// running — so the retry button appears instead of a phantom counter.
|
||||
export async function failStaleJobs() {
|
||||
const [a] = await pool.query(
|
||||
"UPDATE recordings SET analysis_status='error' WHERE analysis_status='pending' AND analysis_started_at IS NOT NULL");
|
||||
const [t] = await pool.query(
|
||||
"UPDATE recordings SET status='error' WHERE status='transcribing'");
|
||||
return { analyses: a.affectedRows, transcriptions: t.affectedRows };
|
||||
}
|
||||
|
||||
// Stamp the wall-clock start of a processing run (transcribe+diar+LLM). The
|
||||
// GUI shows a live counter from here; updateAnalysis('done') freezes the total
|
||||
// into analysis_ms. Called at every trigger (record-stop, retranscribe, reanalyze).
|
||||
|
||||
+12
-1
@@ -493,7 +493,15 @@ app.post('/api/search/answer', wrap(async (req, res) => {
|
||||
const q = String((req.body && req.body.q) || '').trim();
|
||||
if (!q) return res.status(400).json({ error: 'q fehlt' });
|
||||
const { hits } = await search.search(q, { limit: 20 });
|
||||
res.json(await withNpuLock('search-answer', () => search.answer(q, hits)));
|
||||
// stopServer() afterwards is NOT optional: rkllm stays resident with ~6GB
|
||||
// once loaded, and the next speaker-embedding pass then OOM-dies next to it
|
||||
// (analyzeRecording/rediarize free it for exactly that reason). A search
|
||||
// answer left the model squatting 82% of RAM for hours.
|
||||
try {
|
||||
res.json(await withNpuLock('search-answer', () => search.answer(q, hits)));
|
||||
} finally {
|
||||
llm.stopServer();
|
||||
}
|
||||
}));
|
||||
app.get('/api/search/status', wrap(async (req, res) => {
|
||||
const [[{ n }]] = await db.pool.query('SELECT COUNT(*) AS n FROM passages');
|
||||
@@ -718,6 +726,9 @@ app.get(/^(?!\/api|\/media).*/, (req, res) => res.sendFile(path.join(__dirname,
|
||||
const PORT = process.env.PORT || 3000;
|
||||
db.ensureSchema()
|
||||
.catch(e => console.error('ensureSchema:', e.message))
|
||||
.then(() => db.failStaleJobs())
|
||||
.then(r => { if (r && (r.analyses || r.transcriptions)) console.log(`boot: ${r.analyses} abgebrochene Analysen, ${r.transcriptions} abgebrochene Transkriptionen als Fehler markiert`); })
|
||||
.catch(e => console.error('failStaleJobs:', e.message))
|
||||
.finally(() => app.listen(PORT, '0.0.0.0', () => {
|
||||
console.log(`callrec listening on :${PORT}`);
|
||||
// Retention sweep at boot + daily. unref() so it never holds the process open.
|
||||
|
||||
Reference in New Issue
Block a user