From d5b9733e78b0769acddf7379ca17ae191446c207 Mon Sep 17 00:00:00 2001 From: Aime Board Date: Fri, 31 Jul 2026 13:20:20 +0000 Subject: [PATCH] feat(gui+search): Inhaltssuche mit Zeitstempel-Verweis, Papierkorb (30 Tage), Titel umbenennen Drei vom Nutzer angeforderte Funktionen. Titel umbenennen: PATCH /api/recordings/:id/title. Neue Spalte recordings.title_locked, damit eine spaetere KI-Analyse einen von Hand gesetzten Titel nicht wieder ueberschreibt. Papierkorb: DELETE setzt nur noch deleted_at, nichts wird angefasst. Wiederherstellen, endgueltig loeschen, Aufraeumlauf beim Boot und taeglich (Boot-Lauf, weil das Board oft tagelang aus ist). Behebt nebenbei einen Altfehler: das bisherige Loeschen liess die words-Zeilen verwaist in der DB und die WAV/MP3-Dateien auf der Karte zurueck. Inhaltssuche: neue Tabelle passages (1689 Zeilen aus 22 Aufnahmen) mit Sprecher und Zeitfenster je Passage. Trefferliste kommt aus SQL in Millisekunden, die formulierte Antwort mit Belegen [1][2] wird nachgereicht (Qwen3-4B, ~140 s) - deshalb zweistufig. Hybrid: MariaDB FULLTEXT + Passage-Embedding + Dokument-Embedding aus Titel/Zusammenfassung (multilingual-e5-small, ONNX/CPU, taskset A55). Ohne Modell degradiert die Suche sauber auf Volltext statt auszufallen. Gemessene Grenzen sind dokumentiert statt geschoent: e5-small trennt auf verrauschtem ASR-Text schwach (Kosinus-Spanne aller Passagen nur 0,8085-0,8560; eine irrelevante Passage schlug die richtige Antwort). Dokument-Ebene und IDF-Wertung heben die richtige Passage auf Rang 11 von 296, aber nicht in die Top-8, die das LLM liest. Naechster Hebel waere multilingual-e5-base. Relative Datumsangaben werden deterministisch aufgeloest, nicht vom LLM - ein falsches Kalenderfenster waere schlimmer als gar keins. Neu: app/search.js, app/emb.js, app/inference/embed_text.py, docs/09-suche-und-verwaltung.md Board-Backups: .bak-search --- README.md | 1 + app/db.js | 586 ++++---- app/emb.js | 94 ++ app/inference/embed_text.py | 75 + app/search.js | 420 ++++++ app/server.js | 1367 ++++++++++--------- app/web/src/main.js | 2270 +++++++++++++++++-------------- app/web/src/style.css | 872 ++++++------ docs/08-changelog.md | 72 + docs/09-suche-und-verwaltung.md | 180 +++ 10 files changed, 3651 insertions(+), 2286 deletions(-) create mode 100644 app/emb.js create mode 100644 app/inference/embed_text.py create mode 100644 app/search.js create mode 100644 docs/09-suche-und-verwaltung.md diff --git a/README.md b/README.md index aeb2301..2c682ac 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,7 @@ KI-Zusammenfassung (Gemma-3-4B auf der NPU) — **komplett offline**, keine Clou | [06 – iPhone-Bluetooth](docs/06-iphone-bt.md) | HFP-Pfad, Routing-Watcher, Echo-Fix | | [07 – Betrieb](docs/07-betrieb.md) | Services, Zugänge, Backups, Troubleshooting | | [08 – Changelog](docs/08-changelog.md) | Chronik aller Entwicklungsschritte | +| [09 – Suche & Verwaltung](docs/09-suche-und-verwaltung.md) | Inhaltssuche mit Zeitstempel-Verweis, Papierkorb, Titel umbenennen | ## System-Überblick diff --git a/app/db.js b/app/db.js index 4c1ba49..ca038ee 100644 --- a/app/db.js +++ b/app/db.js @@ -1,254 +1,332 @@ -import mysql from 'mysql2/promise'; - -export const pool = mysql.createPool({ - host: 'localhost', user: 'callrec', password: 'callrec123', - database: 'callrec', connectionLimit: 5, waitForConnections: true -}); - -// Idempotent migration: the LLM analysis columns (auto title lives in `title`). -// MariaDB supports ADD COLUMN IF NOT EXISTS. -export async function ensureSchema() { - await pool.query(`ALTER TABLE recordings - ADD COLUMN IF NOT EXISTS summary TEXT NULL, - ADD COLUMN IF NOT EXISTS todos TEXT NULL, - ADD COLUMN IF NOT EXISTS participants TEXT NULL, - ADD COLUMN IF NOT EXISTS transcript_clean TEXT NULL, - ADD COLUMN IF NOT EXISTS wav_file VARCHAR(255) NULL, - ADD COLUMN IF NOT EXISTS speaker_names TEXT NULL, - ADD COLUMN IF NOT EXISTS speaker_labels TEXT NULL, - ADD COLUMN IF NOT EXISTS analysis_status VARCHAR(16) NOT NULL DEFAULT 'pending', - ADD COLUMN IF NOT EXISTS analysis_started_at BIGINT NULL, - ADD COLUMN IF NOT EXISTS analysis_ms INT NULL`); - await pool.query(`ALTER TABLE words ADD COLUMN IF NOT EXISTS speaker VARCHAR(32) NULL`); - await pool.query(`CREATE TABLE IF NOT EXISTS voiceprints ( - id INT AUTO_INCREMENT PRIMARY KEY, - name VARCHAR(128) NOT NULL, - dim INT NOT NULL, - vec MEDIUMTEXT NOT NULL, - created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP)`); - // n = number of samples averaged into this voiceprint (running centroid). - await pool.query(`ALTER TABLE voiceprints ADD COLUMN IF NOT EXISTS n INT NOT NULL DEFAULT 1`); - // Embedding model the vector came from: prints of different models are NOT - // comparable (cosine across models is noise), so identification only ever - // uses the rows of the model currently loaded. NULL = pre-2026-07-27 TitaNet. - await pool.query(`ALTER TABLE voiceprints ADD COLUMN IF NOT EXISTS model VARCHAR(64) NULL`); - await pool.query(`UPDATE voiceprints SET model='nemo_titanet_small' WHERE model IS NULL`); - // Domain glossary: known proper nouns / terms the ASR mis-hears (e.g. "Aime" - // heard as "Amy"). Fed into the LLM correction step so soundalikes get mapped - // to the correct spelling. `soundalikes` = optional comma-list of mis-hearings. - await pool.query(`CREATE TABLE IF NOT EXISTS glossary ( - id INT AUTO_INCREMENT PRIMARY KEY, - term VARCHAR(190) NOT NULL UNIQUE, - soundalikes TEXT NULL, - enabled TINYINT NOT NULL DEFAULT 1, - created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP)`); - // Live speaker markers: names typed DURING the recording ("who is talking - // right now"). t_ms = offset from recording start. After diarization each - // marker is resolved to the acoustic cluster that was speaking around t_ms - // (`speaker`), which names that cluster — ground truth instead of guessing. - await pool.query(`CREATE TABLE IF NOT EXISTS markers ( - id INT AUTO_INCREMENT PRIMARY KEY, - recording_id INT NOT NULL, - t_ms INT NOT NULL, - name VARCHAR(128) NOT NULL, - speaker VARCHAR(32) NULL, - conf VARCHAR(8) NULL, - created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, - INDEX rec_idx (recording_id))`); -} - -// --- live speaker markers --------------------------------------------------- -export async function addMarkers(recId, markers) { - const list = (markers || []).filter(m => m && m.name); - if (!list.length) return; - await pool.query('INSERT INTO markers (recording_id, t_ms, name) VALUES ?', - [list.map(m => [recId, Math.max(0, Math.round(m.t_ms || 0)), String(m.name).slice(0, 128)])]); -} -export async function listMarkers(recId) { - const [rows] = await pool.query( - 'SELECT id, t_ms, name, speaker, conf FROM markers WHERE recording_id=? ORDER BY t_ms', [recId]); - return rows; -} -export async function setMarkerSpeaker(id, speaker, conf) { - await pool.execute('UPDATE markers SET speaker=?, conf=? WHERE id=?', [speaker || null, conf || null, id]); -} -export async function deleteMarker(id) { await pool.execute('DELETE FROM markers WHERE id=?', [id]); } -export async function addMarker(recId, tMs, name) { - const [res] = await pool.execute('INSERT INTO markers (recording_id, t_ms, name) VALUES (?,?,?)', - [recId, Math.max(0, Math.round(tMs || 0)), String(name).slice(0, 128)]); - return res.insertId; -} - -export async function listGlossary() { - const [rows] = await pool.query('SELECT id, term, soundalikes, enabled, created_at FROM glossary ORDER BY term'); - return rows.map(r => ({ ...r, enabled: !!r.enabled })); -} -export async function addGlossary(term, soundalikes) { - await pool.execute( - 'INSERT INTO glossary (term, soundalikes) VALUES (?,?) ON DUPLICATE KEY UPDATE soundalikes=VALUES(soundalikes), enabled=1', - [term, soundalikes || null]); - const [rows] = await pool.execute('SELECT id, term, soundalikes, enabled FROM glossary WHERE term=?', [term]); - return rows[0] ? { ...rows[0], enabled: !!rows[0].enabled } : null; -} -export async function deleteGlossary(id) { - await pool.execute('DELETE FROM glossary WHERE id=?', [id]); -} -export async function updateGlossary(id, term, soundalikes) { - await pool.execute('UPDATE glossary SET term=?, soundalikes=? WHERE id=?', [term, soundalikes || null, id]); - const [rows] = await pool.execute('SELECT id, term, soundalikes, enabled FROM glossary WHERE id=?', [id]); - return rows[0] ? { ...rows[0], enabled: !!rows[0].enabled } : null; -} - -export async function addVoiceprint(name, vec, model) { - const [res] = await pool.execute( - 'INSERT INTO voiceprints (name, dim, vec, model) VALUES (?,?,?,?)', - [name, vec.length, JSON.stringify(vec), model || null]); - return res.insertId; -} - -// Add a sample to a named voiceprint, keeping ONE row per name as a running -// centroid (mean of all samples). This is how the app "learns who is who": every -// time a speaker is named in a recording, their voice is folded into the centroid, -// so future recordings match them more reliably. New name -> fresh row. -export async function upsertVoiceprint(name, vec, model) { - const [rows] = await pool.query( - 'SELECT id, vec, n FROM voiceprints WHERE name=? AND (model=? OR (model IS NULL AND ? IS NULL)) ORDER BY id LIMIT 1', - [name, model || null, model || null]); - if (!rows.length) { - const [res] = await pool.execute('INSERT INTO voiceprints (name, dim, vec, n, model) VALUES (?,?,?,1,?)', - [name, vec.length, JSON.stringify(vec), model || null]); - return res.insertId; - } - const old = rows[0]; - const on = old.n || 1; - const ov = JSON.parse(old.vec); - const mean = vec.map((x, i) => ((ov[i] || 0) * on + x) / (on + 1)); - await pool.execute('UPDATE voiceprints SET vec=?, n=?, dim=? WHERE id=?', - [JSON.stringify(mean), on + 1, mean.length, old.id]); - return old.id; -} -// model given → only the prints of that embedding model (what identification -// must use); omitted → all rows (GUI listing). -export async function listVoiceprints(model) { - const [rows] = model - ? await pool.query('SELECT id,name,dim,vec,model,created_at FROM voiceprints WHERE model=? ORDER BY name', [model]) - : await pool.query('SELECT id,name,dim,vec,model,created_at FROM voiceprints ORDER BY name'); - return rows.map(r => ({ ...r, vec: JSON.parse(r.vec) })); -} -export async function deleteVoiceprint(id) { await pool.query('DELETE FROM voiceprints WHERE id=?', [id]); } - -export async function addRecording(r) { - const [res] = await pool.execute( - 'INSERT INTO recordings (title, mp3_file, wav_file, device, duration_ms, transcript, language, status, analysis_status) VALUES (?,?,?,?,?,?,?,?,?)', - [r.title, r.mp3_file, r.wav_file || null, r.device, r.duration_ms, r.transcript, r.language, r.status || 'done', 'pending']); - return res.insertId; -} - -export async function addWords(recId, words) { - if (!words || !words.length) return; - const values = words.map((w, i) => [recId, w.word, w.t_start_ms, w.t_end_ms, i, w.speaker || null]); - await pool.query('INSERT INTO words (recording_id, word, t_start_ms, t_end_ms, idx, speaker) VALUES ?', [values]); -} - -// Replace a recording's words wholesale (used when re-diarizing: the text is -// unchanged, only the speaker labels differ). Words are re-indexed by array order. -export async function replaceWords(recId, words) { - await pool.query('DELETE FROM words WHERE recording_id=?', [recId]); - await addWords(recId, words); -} - -// Store the transcript + words after async (thermal-guarded) transcription and -// flip the row out of 'transcribing'. The wav + row were saved before whisper -// ran, so this just fills them in. -export async function setTranscript(id, transcript, words) { - await pool.execute('UPDATE recordings SET transcript=?, status=? WHERE id=?', [transcript || '', 'done', id]); - await replaceWords(id, words || []); -} - -// Stamp the wall-clock start of a processing run (transcribe+diar+LLM). The -// GUI shows a live counter from here; updateAnalysis('done') freezes the total -// into analysis_ms. Called at every trigger (record-stop, retranscribe, reanalyze). -export async function markAnalysisStart(id) { - await pool.execute('UPDATE recordings SET analysis_started_at=?, analysis_ms=NULL WHERE id=?', [Date.now(), id]); -} - -// Update just the coarse recording status ('transcribing' | 'done' | 'error'). -export async function setStatus(id, status) { - await pool.execute('UPDATE recordings SET status=? WHERE id=?', [status, id]); -} - -// Per-recording map { rawSpeaker -> displayName }. Manual edits (setSpeakerName) -// are authoritative; acoustic identification only fills blanks (mergeSpeakerNames). -async function getSpeakerNames(id) { - const [rows] = await pool.query('SELECT speaker_names FROM recordings WHERE id=?', [id]); - try { return JSON.parse((rows[0] && rows[0].speaker_names) || '{}') || {}; } catch { return {}; } -} -export async function setSpeakerName(id, speaker, name) { - const m = await getSpeakerNames(id); - if (name) m[speaker] = name; else delete m[speaker]; // empty name clears back to "Sprecher N" - await pool.execute('UPDATE recordings SET speaker_names=? WHERE id=?', [JSON.stringify(m), id]); -} -// The label EVERY speaker carried in the text the LLM produced ({speaker_07: -// "Sprecher 5"}). Stored with the analysis so a later rename can swap the name -// into the finished summary/todos instead of re-running the 20-minute analysis. -export async function setSpeakerLabels(id, labels) { - await pool.execute('UPDATE recordings SET speaker_labels=? WHERE id=?', [JSON.stringify(labels || {}), id]); -} -export async function getSpeakerLabels(id) { - const [rows] = await pool.query('SELECT speaker_labels FROM recordings WHERE id=?', [id]); - try { return JSON.parse((rows[0] && rows[0].speaker_labels) || '{}') || {}; } catch { return {}; } -} - -// Seed names from acoustic identification WITHOUT clobbering existing (manual) ones. -export async function mergeSpeakerNames(id, map) { - if (!map || !Object.keys(map).length) return; - const m = await getSpeakerNames(id); - let changed = false; - for (const k of Object.keys(map)) if (!m[k] && map[k]) { m[k] = map[k]; changed = true; } - if (changed) await pool.execute('UPDATE recordings SET speaker_names=? WHERE id=?', [JSON.stringify(m), id]); -} - -// Store the LLM result. `todos` is an array → persisted as JSON text. -export async function updateAnalysis(id, { title, summary, todos, participants, transcriptClean, status }) { - const st = status || 'done'; - await pool.execute( - `UPDATE recordings SET title=COALESCE(?,title), summary=?, todos=?, participants=?, transcript_clean=?, analysis_status=?, - analysis_ms=CASE WHEN ?='done' AND analysis_started_at IS NOT NULL THEN ?-analysis_started_at ELSE analysis_ms END - WHERE id=?`, - [title || null, summary || null, todos ? JSON.stringify(todos) : null, - participants ? JSON.stringify(participants) : null, transcriptClean || null, st, - st, Date.now(), id]); -} - -function parseJsonCols(row) { - for (const c of ['todos', 'participants']) { - if (row && typeof row[c] === 'string') { - try { row[c] = JSON.parse(row[c]); } catch { row[c] = []; } - } - } - return row; -} -const parseTodos = parseJsonCols; - -export async function listRecordings() { - const [rows] = await pool.query( - 'SELECT id,title,mp3_file,device,duration_ms,language,status,summary,todos,participants,analysis_status,analysis_started_at,analysis_ms,created_at FROM recordings ORDER BY id DESC'); - return rows.map(parseTodos); -} - -export async function getRecording(id) { - const [recRows] = await pool.query('SELECT * FROM recordings WHERE id=?', [id]); - const rec = recRows[0]; - if (!rec) return null; - const [words] = await pool.query( - 'SELECT word,t_start_ms,t_end_ms,idx,speaker FROM words WHERE recording_id=? ORDER BY idx', [id]); - let speaker_names = {}; - try { speaker_names = JSON.parse(rec.speaker_names || '{}') || {}; } catch {} - const markers = await listMarkers(id).catch(() => []); - return { ...parseTodos(rec), speaker_names, words, markers }; -} - -export async function deleteRecording(id) { - await pool.query('DELETE FROM markers WHERE recording_id=?', [id]).catch(() => {}); - await pool.query('DELETE FROM recordings WHERE id=?', [id]); -} +import mysql from 'mysql2/promise'; + +export const pool = mysql.createPool({ + host: 'localhost', user: 'callrec', password: 'callrec123', + database: 'callrec', connectionLimit: 5, waitForConnections: true +}); + +// Idempotent migration: the LLM analysis columns (auto title lives in `title`). +// MariaDB supports ADD COLUMN IF NOT EXISTS. +export async function ensureSchema() { + await pool.query(`ALTER TABLE recordings + ADD COLUMN IF NOT EXISTS summary TEXT NULL, + ADD COLUMN IF NOT EXISTS todos TEXT NULL, + ADD COLUMN IF NOT EXISTS participants TEXT NULL, + ADD COLUMN IF NOT EXISTS transcript_clean TEXT NULL, + ADD COLUMN IF NOT EXISTS wav_file VARCHAR(255) NULL, + ADD COLUMN IF NOT EXISTS speaker_names TEXT NULL, + ADD COLUMN IF NOT EXISTS speaker_labels TEXT NULL, + ADD COLUMN IF NOT EXISTS analysis_status VARCHAR(16) NOT NULL DEFAULT 'pending', + ADD COLUMN IF NOT EXISTS analysis_started_at BIGINT NULL, + ADD COLUMN IF NOT EXISTS analysis_ms INT NULL`); + await pool.query(`ALTER TABLE words ADD COLUMN IF NOT EXISTS speaker VARCHAR(32) NULL`); + await pool.query(`CREATE TABLE IF NOT EXISTS voiceprints ( + id INT AUTO_INCREMENT PRIMARY KEY, + name VARCHAR(128) NOT NULL, + dim INT NOT NULL, + vec MEDIUMTEXT NOT NULL, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP)`); + // n = number of samples averaged into this voiceprint (running centroid). + await pool.query(`ALTER TABLE voiceprints ADD COLUMN IF NOT EXISTS n INT NOT NULL DEFAULT 1`); + // Embedding model the vector came from: prints of different models are NOT + // comparable (cosine across models is noise), so identification only ever + // uses the rows of the model currently loaded. NULL = pre-2026-07-27 TitaNet. + await pool.query(`ALTER TABLE voiceprints ADD COLUMN IF NOT EXISTS model VARCHAR(64) NULL`); + await pool.query(`UPDATE voiceprints SET model='nemo_titanet_small' WHERE model IS NULL`); + // Domain glossary: known proper nouns / terms the ASR mis-hears (e.g. "Aime" + // heard as "Amy"). Fed into the LLM correction step so soundalikes get mapped + // to the correct spelling. `soundalikes` = optional comma-list of mis-hearings. + await pool.query(`CREATE TABLE IF NOT EXISTS glossary ( + id INT AUTO_INCREMENT PRIMARY KEY, + term VARCHAR(190) NOT NULL UNIQUE, + soundalikes TEXT NULL, + enabled TINYINT NOT NULL DEFAULT 1, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP)`); + // Live speaker markers: names typed DURING the recording ("who is talking + // right now"). t_ms = offset from recording start. After diarization each + // marker is resolved to the acoustic cluster that was speaking around t_ms + // (`speaker`), which names that cluster — ground truth instead of guessing. + await pool.query(`CREATE TABLE IF NOT EXISTS markers ( + id INT AUTO_INCREMENT PRIMARY KEY, + recording_id INT NOT NULL, + t_ms INT NOT NULL, + name VARCHAR(128) NOT NULL, + speaker VARCHAR(32) NULL, + conf VARCHAR(8) NULL, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + INDEX rec_idx (recording_id))`); + // Trash instead of hard delete: `deleted_at` set = in the bin, rows and audio + // files stay untouched until purgeExpired() passes TRASH_DAYS. Every listing + // filters on it, so a binned recording is invisible everywhere else. + // `title_locked` marks a title the USER typed — a later re-analysis must not + // overwrite it with a fresh LLM guess (see updateAnalysis). + await pool.query(`ALTER TABLE recordings + ADD COLUMN IF NOT EXISTS deleted_at TIMESTAMP NULL DEFAULT NULL, + ADD COLUMN IF NOT EXISTS title_locked TINYINT(1) NOT NULL DEFAULT 0, + ADD COLUMN IF NOT EXISTS doc_vec MEDIUMTEXT NULL`); + // Search index. One row per passage (a few sentences of ONE speaker, see + // search.js/buildPassages), carrying the time window so a hit can name the + // exact minute and the player can jump there. `vec` = optional text embedding + // for semantic ranking; NULL when the embedding model is not installed, in + // which case search degrades to full-text only. + await pool.query(`CREATE TABLE IF NOT EXISTS passages ( + id INT AUTO_INCREMENT PRIMARY KEY, + recording_id INT NOT NULL, + idx INT NOT NULL, + speaker VARCHAR(32) NULL, + speaker_name VARCHAR(128) NULL, + t_start_ms INT NOT NULL, + t_end_ms INT NOT NULL, + text TEXT NOT NULL, + vec MEDIUMTEXT NULL, + INDEX rec_idx (recording_id), + FULLTEXT KEY ft_text (text)) ENGINE=InnoDB`); +} + +// --- live speaker markers --------------------------------------------------- +export async function addMarkers(recId, markers) { + const list = (markers || []).filter(m => m && m.name); + if (!list.length) return; + await pool.query('INSERT INTO markers (recording_id, t_ms, name) VALUES ?', + [list.map(m => [recId, Math.max(0, Math.round(m.t_ms || 0)), String(m.name).slice(0, 128)])]); +} +export async function listMarkers(recId) { + const [rows] = await pool.query( + 'SELECT id, t_ms, name, speaker, conf FROM markers WHERE recording_id=? ORDER BY t_ms', [recId]); + return rows; +} +export async function setMarkerSpeaker(id, speaker, conf) { + await pool.execute('UPDATE markers SET speaker=?, conf=? WHERE id=?', [speaker || null, conf || null, id]); +} +export async function deleteMarker(id) { await pool.execute('DELETE FROM markers WHERE id=?', [id]); } +export async function addMarker(recId, tMs, name) { + const [res] = await pool.execute('INSERT INTO markers (recording_id, t_ms, name) VALUES (?,?,?)', + [recId, Math.max(0, Math.round(tMs || 0)), String(name).slice(0, 128)]); + return res.insertId; +} + +export async function listGlossary() { + const [rows] = await pool.query('SELECT id, term, soundalikes, enabled, created_at FROM glossary ORDER BY term'); + return rows.map(r => ({ ...r, enabled: !!r.enabled })); +} +export async function addGlossary(term, soundalikes) { + await pool.execute( + 'INSERT INTO glossary (term, soundalikes) VALUES (?,?) ON DUPLICATE KEY UPDATE soundalikes=VALUES(soundalikes), enabled=1', + [term, soundalikes || null]); + const [rows] = await pool.execute('SELECT id, term, soundalikes, enabled FROM glossary WHERE term=?', [term]); + return rows[0] ? { ...rows[0], enabled: !!rows[0].enabled } : null; +} +export async function deleteGlossary(id) { + await pool.execute('DELETE FROM glossary WHERE id=?', [id]); +} +export async function updateGlossary(id, term, soundalikes) { + await pool.execute('UPDATE glossary SET term=?, soundalikes=? WHERE id=?', [term, soundalikes || null, id]); + const [rows] = await pool.execute('SELECT id, term, soundalikes, enabled FROM glossary WHERE id=?', [id]); + return rows[0] ? { ...rows[0], enabled: !!rows[0].enabled } : null; +} + +export async function addVoiceprint(name, vec, model) { + const [res] = await pool.execute( + 'INSERT INTO voiceprints (name, dim, vec, model) VALUES (?,?,?,?)', + [name, vec.length, JSON.stringify(vec), model || null]); + return res.insertId; +} + +// Add a sample to a named voiceprint, keeping ONE row per name as a running +// centroid (mean of all samples). This is how the app "learns who is who": every +// time a speaker is named in a recording, their voice is folded into the centroid, +// so future recordings match them more reliably. New name -> fresh row. +export async function upsertVoiceprint(name, vec, model) { + const [rows] = await pool.query( + 'SELECT id, vec, n FROM voiceprints WHERE name=? AND (model=? OR (model IS NULL AND ? IS NULL)) ORDER BY id LIMIT 1', + [name, model || null, model || null]); + if (!rows.length) { + const [res] = await pool.execute('INSERT INTO voiceprints (name, dim, vec, n, model) VALUES (?,?,?,1,?)', + [name, vec.length, JSON.stringify(vec), model || null]); + return res.insertId; + } + const old = rows[0]; + const on = old.n || 1; + const ov = JSON.parse(old.vec); + const mean = vec.map((x, i) => ((ov[i] || 0) * on + x) / (on + 1)); + await pool.execute('UPDATE voiceprints SET vec=?, n=?, dim=? WHERE id=?', + [JSON.stringify(mean), on + 1, mean.length, old.id]); + return old.id; +} +// model given → only the prints of that embedding model (what identification +// must use); omitted → all rows (GUI listing). +export async function listVoiceprints(model) { + const [rows] = model + ? await pool.query('SELECT id,name,dim,vec,model,created_at FROM voiceprints WHERE model=? ORDER BY name', [model]) + : await pool.query('SELECT id,name,dim,vec,model,created_at FROM voiceprints ORDER BY name'); + return rows.map(r => ({ ...r, vec: JSON.parse(r.vec) })); +} +export async function deleteVoiceprint(id) { await pool.query('DELETE FROM voiceprints WHERE id=?', [id]); } + +export async function addRecording(r) { + const [res] = await pool.execute( + 'INSERT INTO recordings (title, mp3_file, wav_file, device, duration_ms, transcript, language, status, analysis_status) VALUES (?,?,?,?,?,?,?,?,?)', + [r.title, r.mp3_file, r.wav_file || null, r.device, r.duration_ms, r.transcript, r.language, r.status || 'done', 'pending']); + return res.insertId; +} + +export async function addWords(recId, words) { + if (!words || !words.length) return; + const values = words.map((w, i) => [recId, w.word, w.t_start_ms, w.t_end_ms, i, w.speaker || null]); + await pool.query('INSERT INTO words (recording_id, word, t_start_ms, t_end_ms, idx, speaker) VALUES ?', [values]); +} + +// Replace a recording's words wholesale (used when re-diarizing: the text is +// unchanged, only the speaker labels differ). Words are re-indexed by array order. +export async function replaceWords(recId, words) { + await pool.query('DELETE FROM words WHERE recording_id=?', [recId]); + await addWords(recId, words); +} + +// Store the transcript + words after async (thermal-guarded) transcription and +// flip the row out of 'transcribing'. The wav + row were saved before whisper +// ran, so this just fills them in. +export async function setTranscript(id, transcript, words) { + await pool.execute('UPDATE recordings SET transcript=?, status=? WHERE id=?', [transcript || '', 'done', id]); + await replaceWords(id, words || []); +} + +// Stamp the wall-clock start of a processing run (transcribe+diar+LLM). The +// GUI shows a live counter from here; updateAnalysis('done') freezes the total +// into analysis_ms. Called at every trigger (record-stop, retranscribe, reanalyze). +export async function markAnalysisStart(id) { + await pool.execute('UPDATE recordings SET analysis_started_at=?, analysis_ms=NULL WHERE id=?', [Date.now(), id]); +} + +// Update just the coarse recording status ('transcribing' | 'done' | 'error'). +export async function setStatus(id, status) { + await pool.execute('UPDATE recordings SET status=? WHERE id=?', [status, id]); +} + +// Per-recording map { rawSpeaker -> displayName }. Manual edits (setSpeakerName) +// are authoritative; acoustic identification only fills blanks (mergeSpeakerNames). +async function getSpeakerNames(id) { + const [rows] = await pool.query('SELECT speaker_names FROM recordings WHERE id=?', [id]); + try { return JSON.parse((rows[0] && rows[0].speaker_names) || '{}') || {}; } catch { return {}; } +} +export async function setSpeakerName(id, speaker, name) { + const m = await getSpeakerNames(id); + if (name) m[speaker] = name; else delete m[speaker]; // empty name clears back to "Sprecher N" + await pool.execute('UPDATE recordings SET speaker_names=? WHERE id=?', [JSON.stringify(m), id]); +} +// The label EVERY speaker carried in the text the LLM produced ({speaker_07: +// "Sprecher 5"}). Stored with the analysis so a later rename can swap the name +// into the finished summary/todos instead of re-running the 20-minute analysis. +export async function setSpeakerLabels(id, labels) { + await pool.execute('UPDATE recordings SET speaker_labels=? WHERE id=?', [JSON.stringify(labels || {}), id]); +} +export async function getSpeakerLabels(id) { + const [rows] = await pool.query('SELECT speaker_labels FROM recordings WHERE id=?', [id]); + try { return JSON.parse((rows[0] && rows[0].speaker_labels) || '{}') || {}; } catch { return {}; } +} + +// Seed names from acoustic identification WITHOUT clobbering existing (manual) ones. +export async function mergeSpeakerNames(id, map) { + if (!map || !Object.keys(map).length) return; + const m = await getSpeakerNames(id); + let changed = false; + for (const k of Object.keys(map)) if (!m[k] && map[k]) { m[k] = map[k]; changed = true; } + if (changed) await pool.execute('UPDATE recordings SET speaker_names=? WHERE id=?', [JSON.stringify(m), id]); +} + +// Store the LLM result. `todos` is an array → persisted as JSON text. +// A user-typed title (title_locked=1) survives every re-analysis: without the +// CASE the LLM would silently rename the recording back on the next run. +export async function updateAnalysis(id, { title, summary, todos, participants, transcriptClean, status }) { + const st = status || 'done'; + await pool.execute( + `UPDATE recordings SET title=CASE WHEN title_locked=1 THEN title ELSE COALESCE(?,title) END, + summary=?, todos=?, participants=?, transcript_clean=?, analysis_status=?, + analysis_ms=CASE WHEN ?='done' AND analysis_started_at IS NOT NULL THEN ?-analysis_started_at ELSE analysis_ms END + WHERE id=?`, + [title || null, summary || null, todos ? JSON.stringify(todos) : null, + participants ? JSON.stringify(participants) : null, transcriptClean || null, st, + st, Date.now(), id]); +} + +// --- title rename ------------------------------------------------------------ +// Empty/blank title unlocks again: the next analysis is then free to propose one. +export async function renameRecording(id, title) { + const t = (title || '').trim().slice(0, 255); + await pool.execute('UPDATE recordings SET title=?, title_locked=? WHERE id=?', [t || null, t ? 1 : 0, id]); + const [rows] = await pool.query('SELECT id,title,title_locked FROM recordings WHERE id=?', [id]); + return rows[0] || null; +} + +function parseJsonCols(row) { + for (const c of ['todos', 'participants']) { + if (row && typeof row[c] === 'string') { + try { row[c] = JSON.parse(row[c]); } catch { row[c] = []; } + } + } + return row; +} +const parseTodos = parseJsonCols; + +const LIST_COLS = 'id,title,mp3_file,device,duration_ms,language,status,summary,todos,participants,analysis_status,analysis_started_at,analysis_ms,created_at'; + +export async function listRecordings() { + const [rows] = await pool.query( + `SELECT ${LIST_COLS} FROM recordings WHERE deleted_at IS NULL ORDER BY id DESC`); + return rows.map(parseTodos); +} + +export async function getRecording(id) { + const [recRows] = await pool.query('SELECT * FROM recordings WHERE id=?', [id]); + const rec = recRows[0]; + if (!rec) return null; + const [words] = await pool.query( + 'SELECT word,t_start_ms,t_end_ms,idx,speaker FROM words WHERE recording_id=? ORDER BY idx', [id]); + let speaker_names = {}; + try { speaker_names = JSON.parse(rec.speaker_names || '{}') || {}; } catch {} + const markers = await listMarkers(id).catch(() => []); + return { ...parseTodos(rec), speaker_names, words, markers }; +} + +// --- trash ------------------------------------------------------------------- +// Deleting only stamps `deleted_at`; nothing is destroyed. Everything (words, +// markers, passages, wav/mp3) stays put so a restore is complete, not partial. +export const TRASH_DAYS = +(process.env.AIME_TRASH_DAYS || 30); + +export async function deleteRecording(id) { + await pool.execute('UPDATE recordings SET deleted_at=CURRENT_TIMESTAMP WHERE id=? AND deleted_at IS NULL', [id]); +} + +export async function restoreRecording(id) { + await pool.execute('UPDATE recordings SET deleted_at=NULL WHERE id=?', [id]); +} + +// Binned recordings, newest first, with the days left before auto-purge. +export async function listTrash() { + const [rows] = await pool.query( + `SELECT ${LIST_COLS}, deleted_at, + GREATEST(0, ? - TIMESTAMPDIFF(DAY, deleted_at, CURRENT_TIMESTAMP)) AS days_left + FROM recordings WHERE deleted_at IS NOT NULL ORDER BY deleted_at DESC`, [TRASH_DAYS]); + return rows.map(parseTodos); +} + +// Hard delete. Returns the audio file names so the caller can unlink them — +// db.js deliberately does no filesystem work. Note this also clears the `words` +// rows, which the pre-trash delete leaked (they outlived their recording). +export async function purgeRecording(id) { + const [rows] = await pool.query('SELECT mp3_file, wav_file FROM recordings WHERE id=?', [id]); + const files = rows[0] ? [rows[0].mp3_file, rows[0].wav_file].filter(Boolean) : []; + for (const t of ['markers', 'words', 'passages']) + await pool.query(`DELETE FROM ${t} WHERE recording_id=?`, [id]).catch(() => {}); + await pool.query('DELETE FROM recordings WHERE id=?', [id]); + return files; +} + +// Ids whose retention window has run out. The caller purges them one by one so +// a single bad row cannot abort the whole sweep. +export async function expiredTrashIds() { + const [rows] = await pool.query( + 'SELECT id FROM recordings WHERE deleted_at IS NOT NULL AND deleted_at < (CURRENT_TIMESTAMP - INTERVAL ? DAY)', + [TRASH_DAYS]); + return rows.map(r => r.id); +} diff --git a/app/emb.js b/app/emb.js new file mode 100644 index 0000000..f1626f9 --- /dev/null +++ b/app/emb.js @@ -0,0 +1,94 @@ +// Text embeddings for semantic search (multilingual-e5-small via onnxruntime). +// +// Entirely optional: if the model directory is missing, available() is false, +// every call resolves to null and search.js silently falls back to full-text. +// That keeps the search working on a fresh install before anything is +// downloaded, instead of 500-ing. +// +// Same daemon pattern as spk.js: one long-lived python process on the A55 cores +// (4-7 are reserved for whisper/sherpa), line-delimited JSON over stdio, so the +// ~2 s model load is paid once instead of per query. +import { spawn } from 'node:child_process'; +import fs from 'node:fs'; +import readline from 'node:readline'; +import path from 'node:path'; + +const PY = process.env.AIME_PY || '/opt/aime-venv/bin/python'; +const DIR = process.env.AIME_EMB_DIR || '/opt/aime-models/e5-small'; +const SCRIPT = '/opt/callrec/inference/embed_text.py'; +// e5 models are trained with these prefixes and lose a lot of accuracy without +// them — the asymmetric pair is what makes a short question match a long passage. +const Q_PREFIX = 'query: '; +const P_PREFIX = 'passage: '; +const BATCH = +(process.env.AIME_EMB_BATCH || 16); + +let proc = null, ready = null, rl = null, pending = null, chain = Promise.resolve(); + +export function available() { + return fs.existsSync(path.join(DIR, 'model.onnx')) && fs.existsSync(path.join(DIR, 'tokenizer.json')); +} + +export function status() { + return { available: available(), dir: DIR, running: !!proc }; +} + +function start() { + if (proc) return ready; + if (!available()) return Promise.reject(new Error('Embedding-Modell fehlt: ' + DIR)); + // Cores 0-3 (A55): the big cores stay free for whisper and sherpa, which are + // the actual bottleneck. Embedding a query is a few ms either way. + proc = spawn('taskset', ['-c', '0-3', PY, SCRIPT, DIR], { stdio: ['pipe', 'pipe', 'pipe'] }); + proc.stderr.on('data', d => process.stderr.write('[emb] ' + d)); + proc.on('exit', () => { proc = null; ready = null; rl = null; if (pending) { pending.reject(new Error('emb-Daemon beendet')); pending = null; } }); + ready = new Promise((res, rej) => { + const to = setTimeout(() => rej(new Error('emb READY timeout')), 120000); + rl = readline.createInterface({ input: proc.stdout }); + rl.on('line', line => { + if (line.startsWith('READY')) { clearTimeout(to); res(); return; } + if (!pending) return; + const p = pending; pending = null; + try { + const msg = JSON.parse(line); + if (msg.error) p.reject(new Error(msg.error)); else p.resolve(msg.vecs); + } catch (e) { p.reject(new Error('emb: unlesbare Antwort — ' + line.slice(0, 120))); } + }); + }); + return ready; +} + +// Serialised through `chain`: the daemon handles one request at a time, and a +// concurrent write would interleave two JSON lines on the same stdin. +function request(texts) { + const run = async () => { + await start(); + return new Promise((resolve, reject) => { + pending = { resolve, reject }; + const to = setTimeout(() => { if (pending) { pending = null; reject(new Error('emb timeout')); } }, 120000); + const done = fn => v => { clearTimeout(to); fn(v); }; + pending = { resolve: done(resolve), reject: done(reject) }; + proc.stdin.write(JSON.stringify({ texts }) + '\n'); + }); + }; + chain = chain.then(run, run); + return chain; +} + +export async function embedQuery(text) { + if (!available()) return null; + const v = await request([Q_PREFIX + String(text || '').slice(0, 2000)]); + return v && v[0]; +} + +// Returns null (not a throw) when the model is absent, so indexing just stores +// passages without vectors. +export async function embedMany(texts) { + if (!available() || !texts || !texts.length) return null; + const out = []; + for (let i = 0; i < texts.length; i += BATCH) { + const batch = texts.slice(i, i + BATCH).map(t => P_PREFIX + String(t || '').slice(0, 2000)); + out.push(...await request(batch)); + } + return out; +} + +export function stop() { if (proc) { try { proc.kill(); } catch {} proc = null; ready = null; } } diff --git a/app/inference/embed_text.py b/app/inference/embed_text.py new file mode 100644 index 0000000..d71ce28 --- /dev/null +++ b/app/inference/embed_text.py @@ -0,0 +1,75 @@ +"""Text-Embedding-Daemon für die Inhaltssuche (multilingual-e5-small, ONNX/CPU). + +Warum kein sentence-transformers: das zöge torch + transformers ins venv, ~2 GB +auf einer Karte, auf der schon whisper, sherpa und ein 5-GB-LLM liegen. Hier +reichen `tokenizers` (Rust, schon installiert) und `onnxruntime` — beides ist +bereits für whisper/NPU da, es kommt keine neue Abhängigkeit dazu. + +Protokoll wie spk_embed: eine JSON-Zeile rein, eine JSON-Zeile raus. + {"texts": ["query: ...", "passage: ..."]} -> {"vecs": [[...], [...]]} +Fehler: {"error": "..."} + +Aufruf: python embed_text.py /opt/aime-models/e5-small +""" +import json +import sys +import os + +import numpy as np +import onnxruntime as ort +from tokenizers import Tokenizer + +MAX_LEN = 512 + + +def log(msg): + print(msg, file=sys.stderr, flush=True) + + +def main(): + if len(sys.argv) < 2: + log("usage: embed_text.py ") + return 1 + d = sys.argv[1] + tok = Tokenizer.from_file(os.path.join(d, "tokenizer.json")) + tok.enable_truncation(max_length=MAX_LEN) + tok.enable_padding() + + so = ort.SessionOptions() + # Vier A55-Kerne, passend zum taskset in emb.js. Mehr Threads als Kerne + # kosten hier nur Kontextwechsel. + so.intra_op_num_threads = 4 + so.inter_op_num_threads = 1 + sess = ort.InferenceSession(os.path.join(d, "model.onnx"), so, providers=["CPUExecutionProvider"]) + want = {i.name for i in sess.get_inputs()} + + print("READY", flush=True) + + for line in sys.stdin: + line = line.strip() + if not line: + continue + try: + texts = json.loads(line)["texts"] + encs = tok.encode_batch(texts) + ids = np.array([e.ids for e in encs], dtype=np.int64) + mask = np.array([e.attention_mask for e in encs], dtype=np.int64) + feed = {"input_ids": ids, "attention_mask": mask} + # Manche ONNX-Exporte verlangen token_type_ids, andere kennen es nicht. + if "token_type_ids" in want: + feed["token_type_ids"] = np.zeros_like(ids) + feed = {k: v for k, v in feed.items() if k in want} + out = sess.run(None, feed)[0] # [B, T, H] last_hidden_state + # Mean-Pooling über die echten Token (Padding darf nicht mitzählen), + # danach L2 — so ist das Skalarprodukt direkt der Kosinus. + m = mask[..., None].astype(np.float32) + vec = (out * m).sum(axis=1) / np.clip(m.sum(axis=1), 1e-9, None) + vec = vec / np.clip(np.linalg.norm(vec, axis=1, keepdims=True), 1e-9, None) + print(json.dumps({"vecs": [[round(float(x), 6) for x in v] for v in vec]}), flush=True) + except Exception as e: # noqa: BLE001 - Daemon darf nie sterben + print(json.dumps({"error": f"{type(e).__name__}: {e}"}), flush=True) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/app/search.js b/app/search.js new file mode 100644 index 0000000..d5f673f --- /dev/null +++ b/app/search.js @@ -0,0 +1,420 @@ +// Content search across all meetings: "which project did X present last week?" +// +// Retrieval is deliberately split from the answer. The passage hits come from +// SQL and are on screen in milliseconds; the written answer is a separate, +// slow (~5 tok/s on this NPU) LLM call the GUI loads afterwards. Asking the LLM +// first would mean staring at a spinner for a minute before seeing anything. +// +// Ranking is hybrid: +// full-text — MariaDB FULLTEXT over the passage text. Nails proper nouns +// ("Nordlicht", "Jeckel") that an embedding blurs away. +// semantic — cosine over text embeddings, so a question worded differently +// than the meeting still hits. Optional: without the model +// (see emb.js) `vec` stays NULL and search runs full-text only. +// boosts — speaker match, exact phrase, recency. +// Neither alone is enough: full-text misses paraphrases, embeddings miss names. +import { pool } from './db.js'; +import * as emb from './emb.js'; +import * as llm from './llm.js'; + +// A passage is the retrieval unit. Too short and a hit has no context to show +// or to answer from; too long and the timestamp stops meaning anything and the +// embedding averages several topics into mush. ~2-4 sentences of one speaker. +const PASSAGE_CHARS = +(process.env.AIME_PASSAGE_CHARS || 420); +const PASSAGE_MAX_CHARS = +(process.env.AIME_PASSAGE_MAX_CHARS || 700); +// A speaker change always ends a passage, but a long monologue is still split. +const TOPK = +(process.env.AIME_SEARCH_TOPK || 20); +const ANSWER_PASSAGES = +(process.env.AIME_ANSWER_PASSAGES || 8); + +// German stop words. Natural-language FULLTEXT would rank on "welches"/"hat" +// and drown the two words that carry the question. MariaDB's built-in stopword +// list is English only, so the filtering has to happen here. +const STOP = new Set(`aber alle allem allen aller alles als also am an ander andere anderem anderen anderer anderes +auch auf aus bei beim bin bis bist da damit dann der den des dem die das dass daß derselbe derselben denn dazu dein +deine dem den der des dessen deshalb dies diese diesem diesen dieser dieses doch dort du durch ein eine einem einen +einer eines einig einige einigem einigen einiger einiges einmal er ihn ihm es etwas euer eure für gegen gewesen hab +habe haben hat hatte hatten hier hin hinter ich mich mir ihr ihre ihrem ihren ihrer ihres euch im in indem ins ist +jede jedem jeden jeder jedes jene jenem jenen jener jenes jetzt kann kannst können könnt machen man manche manchem +manchen mancher manches mein meine mit muss musst müssen nach nicht nichts noch nun nur ob oder ohne sehr sein seine +seinem seinen seiner seines selbst sich sie sind so solche solchem solchen solcher solches soll sollte sondern sonst +über um und uns unse unsem unsen unser unses unter viel vom von vor während war waren warst weg weil weiter +wenn werde werden wieder will wir wird wirst wollen wollte würde würden zu zum zur zwar zwischen mal schon ja nein +okay halt eben eigentlich quasi irgendwie +was wer wen wem wessen wann wo wohin woher wie warum wieso weshalb welche welchem welchen welcher welches +wurde wurden worden gibt gab geht ging steht stand macht gemacht gesagt sagt sagte bitte danke`.split(/\s+/)); + +// Query words worth searching for. +// +// Capitalisation is NOT a licence to keep a stop word: German capitalises every +// noun, and a question starts with a capitalised "Welches"/"Was"/"Wer". Keeping +// those was what made "Welches Projekt hat X letzte Woche vorgestellt" search +// for "Welches" and "Woche" and rank the wrong passages to the top. +function contentTerms(q) { + const raw = (q || '').split(/[^\p{L}\p{N}äöüßÄÖÜ-]+/u).filter(Boolean); + const out = []; + for (const w of raw) { + if (w.length < 3) continue; // below innodb_ft_min_token_size + if (STOP.has(w.toLowerCase())) continue; + out.push(w); + } + return [...new Set(out)]; +} + +// --- relative dates ---------------------------------------------------------- +// "letzte Woche" is answered deterministically, not by the LLM: a wrong week is +// worse than no filter at all, and date arithmetic is exactly what a 4B model +// gets subtly wrong. Weeks are Mon-Sun (ISO), matching how people say it here. +function startOfDay(d) { const x = new Date(d); x.setHours(0, 0, 0, 0); return x; } +function mondayOf(d) { + const x = startOfDay(d); + x.setDate(x.getDate() - ((x.getDay() + 6) % 7)); // getDay(): 0=Sun + return x; +} +const MONTHS = ['januar', 'februar', 'märz', 'april', 'mai', 'juni', 'juli', 'august', 'september', 'oktober', 'november', 'dezember']; + +// Returns {from, to, label, match} — `match` is the text that expressed the date +// so search() can strip it before extracting search terms. Leaving "letzte +// Woche" in the terms makes every passage containing "Woche" a candidate. +export function parseDateRange(q, now = new Date()) { + const s = (q || '').toLowerCase(); + const day = 86400000; + const rules = [ + [/\bvorgestern\b/, () => [new Date(+startOfDay(now) - 2 * day), new Date(+startOfDay(now) - day), 'vorgestern']], + [/\bgestern\b/, () => [new Date(+startOfDay(now) - day), startOfDay(now), 'gestern']], + [/\bheute\b/, () => [startOfDay(now), new Date(+startOfDay(now) + day), 'heute']], + [/\b(diese|dieser)\s+woche\b/, () => [mondayOf(now), new Date(+mondayOf(now) + 7 * day), 'diese Woche']], + [/\b(letzte|letzter|letzten|vergangene|vergangener|vergangenen|vorige|voriger|vorigen)\s+woche\b/, () => { + const from = new Date(+mondayOf(now) - 7 * day); + return [from, new Date(+from + 7 * day), 'letzte Woche']; + }], + [/\b(diesen|diesem)\s+monat\b/, () => + [new Date(now.getFullYear(), now.getMonth(), 1), new Date(now.getFullYear(), now.getMonth() + 1, 1), 'diesen Monat']], + [/\b(letzten|letztem|vergangenen|vorigen)\s+monat\b/, () => + [new Date(now.getFullYear(), now.getMonth() - 1, 1), new Date(now.getFullYear(), now.getMonth(), 1), 'letzten Monat']], + [new RegExp(`\\b(im\\s+)?(${MONTHS.join('|')})\\b`), (m) => { + const mi = MONTHS.indexOf(m[2]); + // A month still ahead of us this year must mean last year's. + const year = mi > now.getMonth() ? now.getFullYear() - 1 : now.getFullYear(); + return [new Date(year, mi, 1), new Date(year, mi + 1, 1), MONTHS[mi][0].toUpperCase() + MONTHS[mi].slice(1)]; + }], + ]; + for (const [re, fn] of rules) { + const m = re.exec(s); + if (!m) continue; + const [from, to, label] = fn(m); + return { from, to, label, match: m[0] }; + } + return null; +} + +// --- indexing ---------------------------------------------------------------- +// Passages are cut from the `words` rows, not from transcript_clean: only the +// words carry per-word timestamps, and the timestamp is the whole point of a +// hit. A passage never spans two speakers. +export function buildPassages(words, speakerNames = {}) { + const out = []; + let cur = null; + const flush = () => { + if (cur && cur.text.trim().length >= 25) { // drop "Ja." / "Mhm." fragments + cur.text = cur.text.trim(); + out.push(cur); + } + cur = null; + }; + for (const w of words) { + const sp = w.speaker || null; + if (cur && cur.speaker !== sp) flush(); + if (!cur) cur = { speaker: sp, speaker_name: speakerNames[sp] || null, t_start_ms: w.t_start_ms, t_end_ms: w.t_end_ms, text: '' }; + cur.text += (cur.text ? ' ' : '') + w.word; + cur.t_end_ms = w.t_end_ms; + // Prefer to break on a sentence end once the target length is reached; + // force a break at the hard cap so one monologue is not a single passage. + const endsSentence = /[.!?…]["»)]?$/.test(w.word); + if ((cur.text.length >= PASSAGE_CHARS && endsSentence) || cur.text.length >= PASSAGE_MAX_CHARS) flush(); + } + flush(); + return out.map((p, i) => ({ ...p, idx: i })); +} + +// Rebuild one recording's passages. Called after transcription/diarization and +// after a speaker rename (the speaker name is denormalised into the row so a +// search can filter by person without joining back). +export async function indexRecording(recId) { + const [recRows] = await pool.query('SELECT speaker_names, title, summary FROM recordings WHERE id=?', [recId]); + if (!recRows.length) return 0; + let names = {}; + try { names = JSON.parse(recRows[0].speaker_names || '{}') || {}; } catch {} + await indexDoc(recId, recRows[0]); + const [words] = await pool.query( + 'SELECT word,t_start_ms,t_end_ms,speaker FROM words WHERE recording_id=? ORDER BY idx', [recId]); + const passages = buildPassages(words, names); + await pool.query('DELETE FROM passages WHERE recording_id=?', [recId]); + if (!passages.length) return 0; + const vecs = await emb.embedMany(passages.map(p => p.text)).catch(() => null); + const rows = passages.map((p, i) => [ + recId, p.idx, p.speaker, p.speaker_name, p.t_start_ms, p.t_end_ms, p.text, + vecs && vecs[i] ? JSON.stringify(vecs[i]) : null]); + await pool.query( + 'INSERT INTO passages (recording_id, idx, speaker, speaker_name, t_start_ms, t_end_ms, text, vec) VALUES ?', [rows]); + console.log(`search: #${recId} indexiert — ${passages.length} Passagen${vecs ? ' (mit Embeddings)' : ' (nur Volltext)'}`); + return passages.length; +} + +// Document-level embedding over title + summary. +// +// Measured on this corpus: passage-level cosine over raw ASR text separates +// badly — for "wie werden unsere Daten gesichert" the whole candidate field sat +// between 0.809 and 0.856, and an off-topic passage outranked the two passages +// that literally describe the backup job. The transcript is conversational and +// the ASR breaks compounds ("Backup -Job"), which is exactly what an embedding +// handles worst. Title and summary are clean LLM prose about the MEETING'S +// topic, so a second similarity on that level says "this recording is about +// your question" even when no single passage embeds well. Cheap: one vector per +// recording, 22 of them. +async function indexDoc(recId, row) { + const doc = [row.title || '', row.summary || ''].filter(Boolean).join('. ').slice(0, 4000); + if (!doc.trim()) return; + const v = await emb.embedMany([doc]).catch(() => null); + await pool.execute('UPDATE recordings SET doc_vec=? WHERE id=?', + [v && v[0] ? JSON.stringify(v[0]) : null, recId]); +} + +// Refresh only the document vectors (seconds, vs ~9 min for a full passage +// re-embed). Used after installing the model or changing titles/summaries. +export async function reindexDocs() { + const [rows] = await pool.query( + 'SELECT id, title, summary FROM recordings WHERE deleted_at IS NULL'); + let n = 0; + for (const r of rows) { + try { await indexDoc(r.id, r); n++; } catch (e) { console.error(`search: doc #${r.id} — ${e.message}`); } + } + console.log(`search: ${n} Dokument-Vektoren aktualisiert`); + return n; +} + +// Index everything that has words but no passages (first run after the update), +// or everything when force=true. Sequential on purpose — this competes with +// transcription for the same cores. +export async function reindexAll({ force = false, onProgress = null } = {}) { + const [rows] = await pool.query( + force + ? 'SELECT id FROM recordings WHERE deleted_at IS NULL ORDER BY id' + : `SELECT r.id FROM recordings r + WHERE r.deleted_at IS NULL + AND EXISTS (SELECT 1 FROM words w WHERE w.recording_id=r.id) + AND NOT EXISTS (SELECT 1 FROM passages p WHERE p.recording_id=r.id) + ORDER BY r.id`); + let done = 0; + for (const r of rows) { + try { await indexRecording(r.id); } catch (e) { console.error(`search: #${r.id} fehlgeschlagen — ${e.message}`); } + if (onProgress) onProgress(++done, rows.length); + } + return { indexed: done, total: rows.length }; +} + +// --- retrieval --------------------------------------------------------------- +const norm = s => (s || '').toLowerCase().replace(/[^\p{L}\p{N}]+/gu, ' ').trim(); + +function cosine(a, b) { + let s = 0, na = 0, nb = 0; + for (let i = 0; i < a.length; i++) { s += a[i] * b[i]; na += a[i] * a[i]; nb += b[i] * b[i]; } + return s / (Math.sqrt(na) * Math.sqrt(nb) + 1e-9); +} + +export async function search(q, { limit = TOPK, recordingId = null } = {}) { + const range = parseDateRange(q); + // The date phrase is a FILTER, not a search term — strip it before tokenising. + // Case-insensitively but WITHOUT lowercasing the rest: the capitalisation of + // the remaining words is what identifies names for the speaker boost below. + const terms = contentTerms( + range ? q.replace(new RegExp(range.match.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'), 'i'), ' ') : q); + const where = ['r.deleted_at IS NULL']; + const args = []; + if (recordingId) { where.push('p.recording_id=?'); args.push(recordingId); } + if (range) { where.push('r.created_at >= ? AND r.created_at < ?'); args.push(range.from, range.to); } + + // "Was lief letzte Woche?" — a date with nothing else to match on. Returning + // "no hits" would be misleading (there ARE meetings), so show that week's + // recordings from their opening passages instead of an empty page. + if (!terms.length && range) { + const [rows] = await pool.query( + `SELECT p.id, p.recording_id, p.speaker, p.speaker_name, p.t_start_ms, p.t_end_ms, p.text, + r.title, r.created_at, r.duration_ms + FROM passages p JOIN recordings r ON r.id=p.recording_id + WHERE ${where.join(' AND ')} AND p.idx < 3 + ORDER BY r.created_at DESC, p.idx ASC LIMIT ?`, [...args, limit]); + return { + hits: rows.map(r => ({ ...r, score: 0, ft: 0, vec: 0 })), + terms, range: { label: range.label, from: range.from, to: range.to }, + semantic: false, total: rows.length, dateOnly: true, + }; + } + + // Full-text candidates. BOOLEAN MODE with OR'd terms: a passage matching only + // "Jeckel" must still be a candidate, the ranking sorts it out afterwards. + let ftRows = []; + if (terms.length) { + const clean = terms.map(t => t.replace(/[+\-><()~*"@]/g, '')); + const boolExpr = clean.map(t => `${t}*`).join(' '); // candidate selection (prefix, OR) + // Scored in NATURAL LANGUAGE mode, which is IDF-weighted; BOOLEAN MODE only + // counts matches, so a passage repeating a common word like "Daten" beat the + // passage that actually contained the rare, meaning-carrying term. + const nlExpr = clean.join(' '); + const [rows] = await pool.query( + `SELECT p.id, p.recording_id, p.speaker, p.speaker_name, p.t_start_ms, p.t_end_ms, p.text, p.vec, + r.title, r.created_at, r.duration_ms, r.doc_vec, + MATCH(p.text) AGAINST(?) AS ft + FROM passages p JOIN recordings r ON r.id=p.recording_id + WHERE ${where.join(' AND ')} AND MATCH(p.text) AGAINST(? IN BOOLEAN MODE) + ORDER BY ft DESC LIMIT 200`, + [nlExpr, ...args, boolExpr]); + ftRows = rows; + } + + // Semantic candidates. Only worth loading when the model is there AND some + // passage actually carries a vector. + let vecRows = []; + let qvec = null; + if (emb.available()) { + qvec = await emb.embedQuery(q).catch(() => null); + if (qvec) { + const [rows] = await pool.query( + `SELECT p.id, p.recording_id, p.speaker, p.speaker_name, p.t_start_ms, p.t_end_ms, p.text, p.vec, + r.title, r.created_at, r.duration_ms, r.doc_vec + FROM passages p JOIN recordings r ON r.id=p.recording_id + WHERE ${where.join(' AND ')} AND p.vec IS NOT NULL`, args); + vecRows = rows; + } + } + + // Merge both candidate sets, then score every candidate on both axes. + const byId = new Map(); + for (const r of [...ftRows, ...vecRows]) if (!byId.has(r.id)) byId.set(r.id, r); + const ftMax = Math.max(1e-9, ...ftRows.map(r => r.ft || 0)); + const ftById = new Map(ftRows.map(r => [r.id, (r.ft || 0) / ftMax])); + + const qn = norm(q); + const nameTerms = terms.filter(t => t[0] === t[0].toUpperCase()); + const now = Date.now(); + + // Raw e5 cosines sit in a narrow band (measured here: 0.82-0.86 across ALL + // passages, relevant or not) because the model puts everything in the same + // cone. Feeding that raw into the score makes the semantic axis worth ~0.01 + // while full-text is worth up to 0.55 — the ranking then ignores semantics + // entirely except when full-text finds nothing. So spread the band: rank by + // how far above the field a passage is, not by its absolute cosine. + const cand = [...byId.values()]; + const cosById = new Map(); + if (qvec) { + for (const r of cand) { + if (!r.vec) continue; + try { cosById.set(r.id, cosine(qvec, JSON.parse(r.vec))); } catch {} + } + } + const cosVals = [...cosById.values()]; + const cosMean = cosVals.length ? cosVals.reduce((a, b) => a + b, 0) / cosVals.length : 0; + const cosSd = cosVals.length + ? Math.sqrt(cosVals.reduce((a, b) => a + (b - cosMean) ** 2, 0) / cosVals.length) : 0; + const cosMin = cosVals.length ? Math.min(...cosVals) : 0; + const cosMax = cosVals.length ? Math.max(...cosVals) : 0; + // Min-max, not standard deviations: with a 0.047-wide band an sd-based squash + // saturates at 1.0 for dozens of passages and throws away the ordering + // exactly where it matters — at the top. + const vecNorm = id => { + if (!cosById.has(id) || cosMax - cosMin < 1e-9) return 0; + return (cosById.get(id) - cosMin) / (cosMax - cosMin); + }; + + // Document level: how close is the RECORDING (title + summary) to the + // question. One value per recording, normalised over the recordings actually + // in play. + const docCos = new Map(); + if (qvec) { + for (const r of cand) { + if (docCos.has(r.recording_id) || !r.doc_vec) continue; + try { docCos.set(r.recording_id, cosine(qvec, JSON.parse(r.doc_vec))); } catch {} + } + } + const dVals = [...docCos.values()]; + const dMin = dVals.length ? Math.min(...dVals) : 0; + const dMax = dVals.length ? Math.max(...dVals) : 0; + const docNorm = recId => { + if (!docCos.has(recId) || dMax - dMin < 1e-9) return 0; + return (docCos.get(recId) - dMin) / (dMax - dMin); + }; + + // Relevance floor for semantic-only candidates, still in sd terms (a scale + // that means "stands out from the field" rather than "is the best of a bad + // field", which is what min-max would say). + const semFloor = cosMean + cosSd; + + const hits = []; + for (const r of cand) { + const ft = ftById.get(r.id) || 0; + const vec = vecNorm(r.id); + const doc = docNorm(r.recording_id); + if (!ft && (cosById.get(r.id) || 0) < semFloor) continue; + // Speaker boost: the person asked about is usually the one TALKING, and + // their own words never contain their name — without this the passage that + // actually answers the question ranks below every passage mentioning them. + const spk = nameTerms.some(t => norm(r.speaker_name).includes(norm(t))) ? 0.25 : 0; + const phrase = qn.length > 8 && norm(r.text).includes(qn) ? 0.35 : 0; + // Mild recency tilt — "last week" questions are the common case. Half a + // point of decay per year; never enough to beat a clearly better match. + const ageDays = (now - new Date(r.created_at).getTime()) / 86400000; + const recency = 0.1 / (1 + ageDays / 180); + // Passage semantics get less weight than the document level because the + // measurement above showed them to be the noisier of the two signals. + const score = 0.45 * ft + 0.25 * vec + 0.30 * doc + spk + phrase + recency; + hits.push({ + id: r.id, recording_id: r.recording_id, title: r.title, created_at: r.created_at, + duration_ms: r.duration_ms, speaker: r.speaker, speaker_name: r.speaker_name, + t_start_ms: r.t_start_ms, t_end_ms: r.t_end_ms, text: r.text, + score, ft, vec: +vec.toFixed(4), doc: +doc.toFixed(4), + cos: +(cosById.get(r.id) || 0).toFixed(4), + }); + } + hits.sort((a, b) => b.score - a.score); + return { + hits: hits.slice(0, limit), + terms, + range: range ? { label: range.label, from: range.from, to: range.to } : null, + semantic: !!qvec, + total: hits.length, + }; +} + +// --- answer ------------------------------------------------------------------ +const ts = ms => `${String(Math.floor(ms / 60000)).padStart(2, '0')}:${String(Math.floor(ms / 1000) % 60).padStart(2, '0')}`; +const dateDe = d => new Date(d).toLocaleDateString('de-DE', { day: '2-digit', month: '2-digit', year: 'numeric' }); + +// Writes the prose answer over the top hits. Every passage is numbered and the +// model is told to cite those numbers, so each claim can be traced back to a +// recording and a minute — an uncited sentence is the tell for a hallucination. +export async function answer(q, hits) { + const use = hits.slice(0, ANSWER_PASSAGES); + if (!use.length) return { answer: 'Dazu finde ich in den Aufnahmen nichts.', sources: [] }; + const ctx = use.map((h, i) => + `[${i + 1}] Meeting "${h.title || 'Aufnahme #' + h.recording_id}" (#${h.recording_id}) vom ${dateDe(h.created_at)}, ` + + `${h.speaker_name || h.speaker || 'unbekannt'} bei ${ts(h.t_start_ms)}:\n${h.text}`).join('\n\n'); + const prompt = `Du beantwortest eine Frage AUSSCHLIESSLICH aus den folgenden Ausschnitten aus Meeting-Mitschriften. + +${ctx} + +Frage: ${q} + +Regeln: +- Antworte kurz und konkret auf Deutsch, in höchstens 4 Sätzen. +- Belege JEDE Aussage mit der Nummer des Ausschnitts, z.B. [2]. +- Steht die Antwort nicht in den Ausschnitten, schreibe genau: "Dazu finde ich in den Aufnahmen nichts." +- Rate nicht und ergänze nichts aus eigenem Wissen. + +Antwort:`; + const text = (await llm.askRaw(prompt) || '').trim(); + // Map the [n] the model used back to real recordings + timestamps. + const cited = new Set((text.match(/\[(\d+)\]/g) || []).map(m => +m.slice(1, -1))); + const sources = use + .map((h, i) => ({ n: i + 1, ...h })) + .filter(h => cited.size === 0 || cited.has(h.n)); + return { answer: text, sources }; +} diff --git a/app/server.js b/app/server.js index 25076c5..9b813cf 100644 --- a/app/server.js +++ b/app/server.js @@ -1,635 +1,732 @@ -import express from 'express'; -import path from 'node:path'; -import fs from 'node:fs'; -import { execFile } from 'node:child_process'; -import { promisify } from 'node:util'; -import { fileURLToPath } from 'node:url'; -import * as bt from './bt.js'; -import * as rec from './recorder.js'; -import * as db from './db.js'; -import * as llm from './llm.js'; -import * as diar from './diar.js'; -import * as spk from './spk.js'; -const pexec = promisify(execFile); - -// Glossary → { soundalike(lowercased): correctTerm }. Only entries WITH -// soundalikes get a deterministic swap; term-only entries are left to the LLM hint. -function glossaryMap(glossary) { - const m = {}; - for (const g of glossary || []) { - if (!g.term || !g.soundalikes) continue; - for (const s of String(g.soundalikes).split(',')) { - const k = s.trim().toLowerCase(); - if (k) m[k] = String(g.term).trim(); - } - } - return m; -} -// Whole-word replace (Unicode-aware, case-insensitive match): swaps any token -// equal to a known soundalike for its correct term, leaving punctuation intact. -function applyGlossaryText(text, m) { - if (!text || !m || !Object.keys(m).length) return text; - return String(text).replace(/[\p{L}\p{N}]+/gu, (tok) => m[tok.toLowerCase()] || tok); -} - -// Build the LLM input from stored words. If acoustic diarization tagged speakers, -// produce a labelled dialogue ("Sprecher 1: …") so the LLM keeps real speakers -// instead of guessing; otherwise the plain transcript. -// Also returns `labels` = the exact label every speaker carries in the text the -// LLM sees ({speaker_07: "Sprecher 5"}); it is stored with the analysis so a -// rename afterwards can be patched into the finished summary (relabelAnalysis). -function buildBasis(rec, nameMap = {}) { - const words = rec.words || []; - const hasSpk = words.some(w => w.speaker); - if (!hasSpk) return { text: rec.transcript || '', labeled: false, labels: {} }; - const turns = diar.toTurns(words); - const labels = {}; let n = 0; - // Labels MUST be unique. On #79 a mis-anchored marker named Sascha's cluster - // "Markus" while the voiceprint named Markus' own cluster "Markus" too — the - // LLM then read two different people under one name and blended them into one - // person, and relabelAnalysis refuses to patch an ambiguous label afterwards. - const used = new Set(); - const uniq = (name) => { - if (!used.has(name)) { used.add(name); return name; } - for (let k = 2; ; k++) if (!used.has(`${name} ${k}`)) { used.add(`${name} ${k}`); return `${name} ${k}`; } - }; - const label = sp => labels[sp] || (labels[sp] = uniq(nameMap[sp] || `Sprecher ${++n}`)); - const text = turns.map(t => `${label(t.speaker)}: ${t.text}`).join('\n'); - return { text, labeled: true, labels }; -} - -// Swap speaker labels for real names in the ALREADY FINISHED analysis text. -// Renaming a speaker after the analysis used to leave "Sprecher 5" all over the -// summary/todos; re-running the LLM costs ~20 min on this board, while the label -// each speaker had is known exactly (recordings.speaker_labels). Whole-word -// replacement in summary, todos, transcript_clean and participants; the stored -// label is then updated to the new name so a second rename works too. -async function relabelAnalysis(id) { - const rec = await db.getRecording(id); - if (!rec) return { changed: 0 }; - const names = rec.speaker_names || {}; - const labels = await db.getSpeakerLabels(id); - if (!Object.keys(names).length || !Object.keys(labels).length) return { changed: 0 }; - // A label that TWO speakers carried (the voice-ID once matched "Fabio" onto two - // clusters) is ambiguous in the text — leave those alone instead of renaming - // both occurrences to one person. - const labelCount = {}; - for (const l of Object.values(labels)) labelCount[l] = (labelCount[l] || 0) + 1; - const pairs = Object.entries(names) - .filter(([sp, name]) => name && labels[sp] && labels[sp] !== name && labelCount[labels[sp]] === 1) - .map(([sp, name]) => ({ sp, from: labels[sp], to: name })) - // longest label first: "Sprecher 1" must not eat "Sprecher 12" - .sort((a, b) => b.from.length - a.from.length); - const esc = s => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); - const swap = (text) => { - if (!text) return text; - let out = String(text); - for (const p of pairs) out = out.replace(new RegExp(`(? swap(String(t))) : rec.todos; - // Participants are REBUILT (not string-swapped) from the diarized speakers in - // order of first appearance — a swap would leave the entries of speakers that - // were never mentioned in the text untouched and can duplicate names. - const spkOrder = []; - for (const w of (rec.words || [])) if (w.speaker && !spkOrder.includes(w.speaker)) spkOrder.push(w.speaker); - const participants = spkOrder.length - ? spkOrder.map((sp, i) => names[sp] || labels[sp] || `Sprecher ${i + 1}`) - : (Array.isArray(rec.participants) ? rec.participants.map(p => swap(String(p))) : rec.participants); - const partChanged = JSON.stringify(participants || null) !== JSON.stringify(rec.participants || null); - if (!pairs.length && !partChanged) return { changed: 0 }; - await db.pool.execute( - 'UPDATE recordings SET summary=?, todos=?, transcript_clean=?, participants=? WHERE id=?', - [swap(rec.summary), todos ? JSON.stringify(todos) : null, swap(rec.transcript_clean), - participants ? JSON.stringify(participants) : null, id]); - const next = { ...labels }; - for (const p of pairs) next[p.sp] = p.to; - await db.setSpeakerLabels(id, next); - console.log(`relabel #${id}: ${pairs.length ? pairs.map(p => `${p.from}→${p.to}`).join(', ') : 'nur Teilnehmerliste'}`); - return { changed: pairs.length || (partChanged ? 1 : 0) }; -} - -const REC_DIR2 = path.join(path.dirname(fileURLToPath(import.meta.url)), 'recordings'); -// Match each diarization cluster against enrolled voiceprints → { speaker: name }. -// Cuts the speaker's longest contiguous run from the kept wav, embeds it, cosine- -// matches. Returns only confident matches (others stay "Sprecher N"). -async function identifySpeakers(recData) { - const words = recData.words || []; - const speakers = [...new Set(words.filter(w => w.speaker).map(w => w.speaker))]; - if (!speakers.length || !recData.wav_file) return {}; - const wavPath = path.join(REC_DIR2, recData.wav_file); - if (!fs.existsSync(wavPath)) return {}; - const vps = await db.listVoiceprints(spk.modelId()); - if (!vps.length) return {}; - const best = {}; let cur = null; - for (const w of words) { - if (!cur || cur.speaker !== w.speaker) cur = { speaker: w.speaker, start: w.t_start_ms, end: w.t_end_ms }; - else cur.end = w.t_end_ms; - const len = cur.end - cur.start; - if (!best[cur.speaker] || len > best[cur.speaker].len) best[cur.speaker] = { start: cur.start, end: cur.end, len }; - } - // Cut at most ID_CUT_SEC: onnxruntime ABORTS the embedding helper on very long - // inputs (reproduced: 180 s -> "Non-zero status code returned while running - // Where"; 20/60 s fine). On #74 the longest run of a moderator was minutes - // long, which killed the helper on the first speaker and — through the then - // poisoned promise chain in spk.js — every speaker after it ("spk beendet"), - // i.e. voice auto-naming was dead for the whole recording. - const ID_CUT_SEC = 25; - const hits = []; - for (const sp of speakers) { - const r = best[sp]; if (!r) continue; - const seg = `/tmp/idseg_${recData.id}_${sp}.wav`; - const from = r.start / 1000; - const to = Math.min(r.end / 1000, from + ID_CUT_SEC); - try { - await pexec('ffmpeg', ['-y', '-i', wavPath, '-ss', String(from), '-to', String(to), '-ar', '16000', '-ac', '1', seg]); - const emb = await spk.embed(seg); - const m = spk.bestMatch(emb, vps); - if (m) hits.push({ speaker: sp, name: m.name, score: m.score }); - } catch (e) { console.error('identify', sp, e.message); } - finally { try { fs.unlinkSync(seg); } catch {} } - } - // One enrolled person can only BE one cluster: keep the best-scoring match per - // name (#74 handed "Fabio" to two clusters at once). - const bestByName = {}; - for (const h of hits) if (!bestByName[h.name] || h.score > bestByName[h.name].score) bestByName[h.name] = h; - const map = {}; - for (const h of Object.values(bestByName)) map[h.speaker] = h.name; - return map; -} - -// Fold a speaker's longest clean run into the named voiceprint centroid, so the -// next recording can auto-identify them. Shared by the manual rename popup and -// the live-marker resolution. Returns true if a sample was learned. -async function enrollSpeakerVoice(recData, speaker, name) { - if (!name || !recData.wav_file) return false; - const seg = longestRun(recData.words, speaker); - if (!seg || (seg.end - seg.start) < 1500) return false; - const wavPath = path.join(REC_DIR2, recData.wav_file); - if (!fs.existsSync(wavPath)) return false; - const cut = `/tmp/enr_${recData.id}_${speaker}.wav`; - try { - await pexec('ffmpeg', ['-y', '-i', wavPath, '-ss', String(seg.start / 1000), - '-to', String(Math.min(seg.end, seg.start + 20000) / 1000), '-ar', '16000', '-ac', '1', cut]); - const emb = await spk.embed(cut); - await db.upsertVoiceprint(name, emb, spk.modelId()); - return true; - } catch (e) { console.error('enroll failed:', e.message); return false; } - finally { try { fs.unlinkSync(cut); } catch {} } -} - -// Apply the names the user typed DURING the recording to the diarized clusters. -// Runs after every (re-)diarization, before the LLM analysis, so the summary -// already talks about real people. Two effects: -// 1. each resolved cluster gets its typed name (authoritative, manual-level), -// 2. if ONE name confidently lands on SEVERAL clusters, those clusters are the -// same person split by the diarizer → they are merged (user-supervised fix -// for the residual over-segmentation). -async function applyMarkers(id) { - const rec = await db.getRecording(id); - if (!rec || !(rec.markers || []).length || !(rec.words || []).some(w => w.speaker)) return null; - const resolved = diar.resolveMarkers(rec.words, rec.markers); - for (const m of resolved) await db.setMarkerSpeaker(m.id, m.speaker, m.conf).catch(() => {}); - - // name -> clusters. NAMING may follow a 'mid' hit, but the cluster MERGE below - // is destructive and unguarded (no acoustic check at this level), so only a - // 'high' hit — marker inside the turn, that turn clearly dominant — is allowed - // to fold two clusters together. #79 lost a speaker exactly this way one layer - // deeper, in the diarizer's must-link. - const byName = {}; - for (const m of resolved) { - if (!m.speaker || m.conf !== 'high') continue; - (byName[m.name] = byName[m.name] || new Set()).add(m.speaker); - } - const nameOnly = {}; - for (const m of resolved) { - if (!m.speaker || m.conf === 'low' || m.conf === null) continue; - (nameOnly[m.name] = nameOnly[m.name] || new Set()).add(m.speaker); - } - const talk = {}; - for (const w of rec.words) if (w.speaker) talk[w.speaker] = (talk[w.speaker] || 0) + (w.t_end_ms - w.t_start_ms); - - const fold = {}; // extra cluster -> primary cluster - const nameOf = {}; // cluster -> typed name - for (const [name, set] of Object.entries(byName)) { - const cls = [...set].sort((a, b) => (talk[b] || 0) - (talk[a] || 0)); - const primary = cls[0]; - nameOf[primary] = name; - for (const c of cls.slice(1)) fold[c] = primary; - } - // Naming without merging: a 'mid' hit labels the cluster it landed on, as long - // as that cluster is not already claimed by somebody else. - for (const [name, set] of Object.entries(nameOnly)) { - if (byName[name]) continue; - const cls = [...set].sort((a, b) => (talk[b] || 0) - (talk[a] || 0)); - if (cls[0] && !nameOf[cls[0]]) nameOf[cls[0]] = name; - } - let merged = 0; - if (Object.keys(fold).length) { - for (const w of rec.words) if (fold[w.speaker]) w.speaker = fold[w.speaker]; - await db.replaceWords(id, rec.words); - merged = Object.keys(fold).length; - console.log(`markers: merged ${merged} cluster(s) into marker-named speakers on #${id}`); - } - for (const [sp, name] of Object.entries(nameOf)) await db.setSpeakerName(id, sp, name).catch(() => {}); - // Learn the voices so the NEXT recording identifies them automatically. - const rec2 = await db.getRecording(id); - let learned = 0; - for (const [sp, name] of Object.entries(nameOf)) if (await enrollSpeakerVoice(rec2, sp, name)) learned++; - const named = Object.keys(nameOf).length; - await relabelAnalysis(id).catch(() => {}); - console.log(`markers on #${id}: ${resolved.length} marker(s) → ${named} named speaker(s), ${merged} merged, ${learned} voice(s) learned`); - return { resolved, named, merged, learned }; -} - -// Run LLM analysis for a recording and persist title/summary/todos/participants/ -// clean dialogue. Background (NPU ~30-90s) so record/stop stays fast. -// A76 clock cap for the whole run: rkllm burns ~1.7 CPU cores alongside the NPU -// and a long Gemma generation hit 95-97°C uncapped on the fanless board (#70, -// 16-min run) — same shutdown risk transcribeWav/rediarize already guard against. -async function analyzeRecording(id) { - // Free the resident rkllm (~5.9GB) BEFORE identifySpeakers: the TitaNet - // spk-embed helper OOM-dies next to it ("spk beendet" per cluster, #71 got - // 48 raw speakers). llm.analyze() below respawns it (~96s load, noise - // against a 15-min analysis). - llm.stopServer(); - await pexec('sudo', ['-n', '/opt/callrec/aime-cpucap.sh', '1608000']).catch(() => {}); - try { - const rec = await db.getRecording(id); - if (!rec) return; - const nameMap = await identifySpeakers(rec).catch(() => ({})); - // Seed the per-recording speaker names from acoustic identification (fills - // blanks only; manual edits stay). Manual names win for the LLM labelling. - await db.mergeSpeakerNames(id, nameMap).catch(() => {}); - // Stored names normally win over acoustic identification (they are typed by - // hand). Exception: a name that came from a NON-'high' marker is only a - // guess about WHICH cluster was speaking — resolveMarkers looks back up to - // 25 s, so in a fast four-way discussion it lands on the wrong voice (#79 - // put "Markus" on Sascha's cluster). If an enrolled voiceprint disagrees - // with such a weak marker name, the voiceprint wins. - const markers = await db.listMarkers(id).catch(() => []); - const weakName = new Map(); - for (const m of markers) if (m.speaker && m.conf !== 'high') weakName.set(m.speaker, m.name); - const names = { ...nameMap, ...(rec.speaker_names || {}) }; - for (const [sp, vname] of Object.entries(nameMap)) { - if (names[sp] !== vname && weakName.get(sp) === names[sp]) { - console.log(`identify #${id}: Stimmabdruck "${vname}" schlägt unsicheren Marker "${names[sp]}" auf ${sp}`); - names[sp] = vname; - await db.setSpeakerName(id, sp, vname).catch(() => {}); - } - } - const glossary = (await db.listGlossary().catch(() => [])).filter(g => g.enabled); - // Deterministic soundalike replacement: the 3B model is unreliable at obeying - // a "map Amy->Aime" instruction (it treats "Amy" as a valid name), so entries - // WITH soundalikes are applied as a hard whole-word swap on the words FIRST — - // then the LLM sees the correct term and the turn view shows it too. Entries - // without soundalikes still rely on the (contextual) LLM glossary hint. - const gmap = glossaryMap(glossary); - if (Object.keys(gmap).length && Array.isArray(rec.words) && rec.words.length) { - let changed = false; - for (const w of rec.words) { const nt = applyGlossaryText(w.word, gmap); if (nt !== w.word) { w.word = nt; changed = true; } } - if (changed) { await db.replaceWords(id, rec.words); console.log(`glossary: applied soundalike swaps to #${id} words`); } - } - const { text, labeled, labels } = buildBasis(rec, names); - // Remember which label each speaker had in the LLM input, so renaming a - // speaker later can patch the finished text (relabelAnalysis). - await db.setSpeakerLabels(id, labels).catch(() => {}); - const a = await llm.analyze(text, { labeled, glossary }); - // Participants = the ACOUSTICALLY diarized speakers (named where known), - // not the LLM's list — the model returned every name MENTIONED in the - // conversation as a "Teilnehmer" (11 names on a 3-person call, #69). - // LLM participants stay only as fallback when there is no diarization. - const spkOrder = []; - for (const w of (rec.words || [])) if (w.speaker && !spkOrder.includes(w.speaker)) spkOrder.push(w.speaker); - // Same labels as in the dialogue the LLM read — otherwise the participant - // list numbers the speakers differently than the summary talks about them. - if (spkOrder.length) a.participants = spkOrder.map((sp, i) => names[sp] || labels[sp] || `Sprecher ${i + 1}`); - // Belt-and-suspenders: also swap in the LLM outputs (cheap, catches any the - // model reintroduced). - for (const k of ['title', 'summary', 'transcriptClean']) if (a[k]) a[k] = applyGlossaryText(a[k], gmap); - a.participants = (a.participants || []).map(p => applyGlossaryText(p, gmap)); - a.todos = (a.todos || []).map(t => applyGlossaryText(t, gmap)); - await db.updateAnalysis(id, { ...a, status: 'done' }); - // A speaker can be named WHILE the analysis runs (it takes ~15 min, and the - // marker/voiceprint passes themselves rename clusters). The finished text - // still carries the label from analysis start ("Sprecher 1"), so patch it - // once at the end — same mechanism as renaming a speaker by hand. - await relabelAnalysis(id).catch(() => {}); - console.log(`analysis done for #${id}: ${a.title}`); - } catch (e) { - console.error(`analysis failed for #${id}:`, e.message); - await db.updateAnalysis(id, { title: null, summary: null, todos: null, status: 'error' }).catch(() => {}); - } finally { - await pexec('sudo', ['-n', '/opt/callrec/aime-cpucap.sh', '2256000']).catch(() => {}); - } -} - -const __dirname = path.dirname(fileURLToPath(import.meta.url)); -const app = express(); -app.use(express.json()); - -const REC_DIR = path.join(__dirname, 'recordings'); -app.use('/media', express.static(REC_DIR)); -app.use(express.static(path.join(__dirname, 'web/dist'))); - -const wrap = fn => (req, res) => fn(req, res).catch(e => res.status(500).json({ error: String(e && e.message || e) })); - -app.get('/api/health', (req, res) => res.json({ ok: true, ...rec.status() })); - -// --- Bluetooth --- -app.post('/api/bt/scan', wrap(async (req, res) => { - const secs = Math.min(+(req.body && req.body.seconds) || 10, 30); - res.json(await bt.scan(secs)); -})); -app.get('/api/bt/devices', wrap(async (req, res) => res.json(await bt.knownDevices()))); -app.post('/api/bt/pair', wrap(async (req, res) => res.json(await bt.pair(req.body.mac)))); -app.post('/api/bt/connect', wrap(async (req, res) => res.json(await bt.connect(req.body.mac)))); -app.post('/api/bt/repair', wrap(async (req, res) => res.json(await bt.repair(req.body.mac)))); -app.post('/api/bt/disconnect', wrap(async (req, res) => res.json(await bt.disconnect(req.body.mac)))); - -// --- Audio --- -app.get('/api/audio/sources', wrap(async (req, res) => res.json(await rec.listSources()))); -app.post('/api/audio/headset-mic', wrap(async (req, res) => res.json(await rec.enableHeadsetMic((req.body && req.body.codec) || 'auto')))); - -// --- Global NPU pipeline serializer --------------------------------------- -// Transcription (whisper-turbo ENCODER on the NPU) and LLM analysis (rkllm on -// the NPU) cannot run simultaneously: while the whisper encoder holds the NPU, -// rkllm_init fails with "failed to allocate handle, ret:-1, errno:14, Bad -// address", which surfaces as "rkllm READY timeout" and a failed/empty -// analysis. This bit when two recordings finished within seconds of each other -// (both fired transcribe+analyze fire-and-forget with no coordination). Every -// heavy per-recording job (transcribe / analyze / re-diarize) now runs through -// this single FIFO queue, so exactly one touches the NPU at a time; a second -// recording simply waits its turn instead of colliding. -let npuChain = Promise.resolve(); -let npuDepth = 0; -function withNpuLock(label, fn) { - npuDepth++; - const waiting = npuDepth - 1; - const run = npuChain.then(async () => { - if (waiting > 0) console.log(`npu-queue: ${label} start (${waiting} ahead done)`); - try { return await fn(); } - finally { npuDepth--; } - }); - npuChain = run.catch(() => {}); // keep the chain alive across a job's failure - return run; -} - -// --- Recording --- -app.post('/api/record/start', wrap(async (req, res) => res.json(await rec.startRecording(req.body || {})))); -// Live speaker markers: type the name of whoever is talking WHILE recording. -app.post('/api/record/marker', wrap(async (req, res) => res.json(rec.addMarker((req.body && req.body.name) || '')))); -app.get('/api/record/markers', (req, res) => res.json(rec.listMarkers())); -app.delete('/api/record/marker/:index', wrap(async (req, res) => res.json({ markers: rec.removeMarker(req.params.index) }))); -// Transcribe (thermal-guarded whisper + diarize) then analyze, in the background. -// The wav + DB row already exist, so a crash here loses nothing — the recording -// stays with status 'transcribing' and can be recovered via /retranscribe. -async function transcribeAndAnalyze(id, wavFile, numSpeakers) { - await db.markAnalysisStart(id).catch(() => {}); - try { - // Live markers constrain the diarization itself (same name = one speaker, - // different names = never merged), so they are loaded before transcribing. - const markers = await db.listMarkers(id).catch(() => []); - // Enrolled voices do the same for people who were never typed — the box - // owner above all (see diar.clusterWindows). - const voiceprints = await db.listVoiceprints(spk.modelId()).catch(() => []); - const { transcript, words } = await rec.transcribeWav(wavFile, numSpeakers, { markers, voiceprints }); - await db.setTranscript(id, transcript, words); - console.log(`transcription done for #${id}: ${words.length} segments`); - // Names typed live → cluster names (+ user-supervised cluster merge) BEFORE - // the LLM sees the dialogue. - await applyMarkers(id).catch(e => console.error('applyMarkers:', e.message)); - } catch (e) { - console.error(`transcription failed for #${id}:`, e.message); - await db.setStatus(id, 'error').catch(() => {}); - // also mark analysis 'error' so the UI stops polling and shows a retry - await db.updateAnalysis(id, { title: null, summary: null, todos: null, status: 'error' }).catch(() => {}); - return; // no transcript -> skip LLM analysis - } - analyzeRecording(id); -} - -app.post('/api/record/stop', wrap(async (req, res) => { - // Persist wav + mp3 + DB row FIRST (crash-safe), respond immediately, then - // transcribe/analyze in the background. The frontend already polls while - // analysis_status === 'pending', so the transcript + analysis fill in live. - const r = await rec.stopAndSave(); - const id = await db.addRecording({ ...r, transcript: '', status: 'transcribing' }); - await db.addMarkers(id, r.markers).catch(e => console.error('addMarkers:', e.message)); - res.json({ id, title: r.title, mp3_file: r.mp3_file, duration_ms: r.duration_ms, - transcript: '', status: 'transcribing', analysis_status: 'pending', - markers: (r.markers || []).length }); - withNpuLock(`transcribe #${id}`, () => transcribeAndAnalyze(id, r.wav_file, r.numSpeakers)); -})); - -// --- Recordings --- -app.get('/api/recordings', wrap(async (req, res) => res.json(await db.listRecordings()))); -app.get('/api/recordings/:id', wrap(async (req, res) => { - const r = await db.getRecording(+req.params.id); - r ? res.json(r) : res.status(404).json({ error: 'not found' }); -})); -app.delete('/api/recordings/:id', wrap(async (req, res) => { await db.deleteRecording(+req.params.id); res.json({ ok: true }); })); - -// --- Voiceprints / enrollment --- -app.get('/api/voiceprints', wrap(async (req, res) => { - const vps = await db.listVoiceprints(); - const cur = spk.modelId(); - res.json(vps.map(v => ({ id: v.id, name: v.name, dim: v.dim, model: v.model, - stale: v.model !== cur, created_at: v.created_at }))); -})); -app.delete('/api/voiceprints/:id', wrap(async (req, res) => { await db.deleteVoiceprint(+req.params.id); res.json({ ok: true }); })); -app.post('/api/enroll/start', wrap(async (req, res) => res.json(await rec.startRecording({ source: req.body && req.body.source })))); -app.post('/api/enroll/stop', wrap(async (req, res) => { - const name = (req.body && req.body.name || '').trim(); - if (!name) return res.status(400).json({ error: 'Name fehlt' }); - const wav = await rec.stopRaw(); - try { - const emb = await spk.embed(wav); - const id = await db.addVoiceprint(name, emb, spk.modelId()); - res.json({ id, name, dim: emb.length }); - } finally { try { fs.unlinkSync(wav); } catch {} } -})); - -// --- Glossary (known proper nouns / terms; fed into LLM correction) --- -app.get('/api/glossary', wrap(async (req, res) => res.json(await db.listGlossary()))); -app.post('/api/glossary', wrap(async (req, res) => { - const term = (req.body && req.body.term || '').trim(); - const soundalikes = (req.body && req.body.soundalikes || '').trim(); - if (!term) return res.status(400).json({ error: 'Begriff fehlt' }); - res.json(await db.addGlossary(term, soundalikes)); -})); -app.delete('/api/glossary/:id', wrap(async (req, res) => { await db.deleteGlossary(+req.params.id); res.json({ ok: true }); })); -app.put('/api/glossary/:id', wrap(async (req, res) => { - const term = (req.body && req.body.term || '').trim(); - const soundalikes = (req.body && req.body.soundalikes || '').trim(); - if (!term) return res.status(400).json({ error: 'Begriff fehlt' }); - const g = await db.updateGlossary(+req.params.id, term, soundalikes); - if (!g) return res.status(404).json({ error: 'Eintrag nicht gefunden' }); - res.json(g); -})); - -// --- LLM --- -app.get('/api/llm/status', (req, res) => res.json(llm.status())); -app.get('/api/spk/status', (req, res) => res.json(spk.status())); -// Re-diarize a recording's words with an (optional) fixed speaker count, then -// persist the new speaker labels. Text/timestamps are untouched. Needs the wav. -async function rediarize(id, numSpeakers) { - const r = await db.getRecording(id); - if (!r || !r.wav_file) throw new Error('wav für Re-Diarisierung fehlt'); - const wav = path.join(REC_DIR, r.wav_file); - if (!fs.existsSync(wav)) throw new Error('wav-Datei nicht gefunden: ' + r.wav_file); - const words = r.words || []; - // Free the resident rkllm first — diar's consolidation embeds every cluster - // via TitaNet, which OOM-dies next to a resident 5.9GB LLM (see - // analyzeRecording; on #71 that silently yielded 48 raw speakers). - llm.stopServer(); - // Diarization runs on the A76 cores; the board is fanless, so cap the clock - // for the whole run (transcribeWav does this for the record path — rediarize - // is the other entry point and must do it too, or a long re-diar overheats). - await pexec('sudo', ['-n', '/opt/callrec/aime-cpucap.sh', '1608000']).catch(() => {}); - try { - // Enrolled voices go IN, not just out: they keep two known people (above all - // the box owner, who is enrolled and talks in every recording) from ending up - // in one cluster. See the cannot-link block in diar.clusterWindows. - const voiceprints = await db.listVoiceprints(spk.modelId()).catch(() => []); - const segs = await diar.diarize(wav, numSpeakers, { markers: r.markers || [], voiceprints }); - diar.assignSpeakers(words, segs); - await db.replaceWords(id, words); - // Re-diarization RENUMBERS the clusters, so every previously stored name - // ("speaker_02" = Fabio) would now point at a different voice. Drop them; the - // live markers and the voiceprint identification re-fill them right after. - await db.pool.query('UPDATE recordings SET speaker_names=NULL WHERE id=?', [id]).catch(() => {}); - console.log(`re-diarized #${id} (numSpeakers=${numSpeakers || 'auto'}) → ${new Set(segs.map(s => s.speaker)).size} speakers`); - } finally { - await pexec('sudo', ['-n', '/opt/callrec/aime-cpucap.sh', '2256000']).catch(() => {}); - } -} - -// Re-run analysis (e.g. the model wasn't downloaded yet at record time). Optional -// body { numSpeakers } re-diarizes first (fixes over-segmentation on a known count). -app.post('/api/recordings/:id/reanalyze', wrap(async (req, res) => { - const id = +req.params.id; - const r = await db.getRecording(id); - if (!r) return res.status(404).json({ error: 'not found' }); - const numSpeakers = +(req.body && req.body.numSpeakers) || 0; - await db.updateAnalysis(id, { title: null, summary: null, todos: null, status: 'pending' }); - await db.markAnalysisStart(id).catch(() => {}); - res.json({ ok: true, status: 'pending', reDiarize: true }); - // Always re-diarize: numSpeakers>0 pins the count, 0 = auto (now consolidated, - // so auto no longer over-segments). This is the "fix my speakers" path. - withNpuLock(`reanalyze #${id}`, async () => { - try { await rediarize(id, numSpeakers); } - catch (e) { console.error(`re-diarize failed for #${id}:`, e.message); } - await applyMarkers(id).catch(e => console.error('applyMarkers:', e.message)); - await analyzeRecording(id); - }); -})); - -// --- markers of a stored recording ----------------------------------------- -app.get('/api/recordings/:id/markers', wrap(async (req, res) => res.json(await db.listMarkers(+req.params.id)))); -// Add a marker after the fact (e.g. while listening back) — same resolution path. -app.post('/api/recordings/:id/markers', wrap(async (req, res) => { - const name = String((req.body && req.body.name) || '').trim(); - const tMs = +(req.body && req.body.t_ms); - if (!name || !Number.isFinite(tMs)) return res.status(400).json({ error: 'name/t_ms fehlt' }); - await db.addMarker(+req.params.id, tMs, name); - res.json({ ok: true, markers: await db.listMarkers(+req.params.id) }); -})); -app.delete('/api/recordings/:id/markers/:mid', wrap(async (req, res) => { - await db.deleteMarker(+req.params.mid); - res.json({ ok: true, markers: await db.listMarkers(+req.params.id) }); -})); -// Re-resolve the markers against the current diarization (no LLM, no sherpa). -app.post('/api/recordings/:id/apply-markers', wrap(async (req, res) => { - const out = await applyMarkers(+req.params.id); - const r = await db.getRecording(+req.params.id); - res.json({ ok: true, ...(out || { named: 0, merged: 0, learned: 0 }), - speaker_names: (r && r.speaker_names) || {}, markers: (r && r.markers) || [] }); -})); - -// LLM-only re-analysis: re-run title/summary/todos on the EXISTING words + -// speaker labels, WITHOUT re-diarizing (fast — skips the ~40min sherpa pass when -// the speakers are already correct and only the LLM/prompts changed). -app.post('/api/recordings/:id/reanalyze-llm', wrap(async (req, res) => { - const id = +req.params.id; - const r = await db.getRecording(id); - if (!r) return res.status(404).json({ error: 'not found' }); - await db.updateAnalysis(id, { title: null, summary: null, todos: null, status: 'pending' }); - await db.markAnalysisStart(id).catch(() => {}); - res.json({ ok: true, status: 'pending', reDiarize: false }); - withNpuLock(`reanalyze-llm #${id}`, async () => { await analyzeRecording(id); }); -})); - -// Re-run transcription from the saved wav (recovery: a crash/thermal-shutdown -// during whisper leaves the row at status 'transcribing' with an empty -// transcript, or 'error'; the wav is on disk). Optional { numSpeakers }. -app.post('/api/recordings/:id/retranscribe', wrap(async (req, res) => { - const id = +req.params.id; - const r = await db.getRecording(id); - if (!r) return res.status(404).json({ error: 'not found' }); - if (!r.wav_file) return res.status(400).json({ error: 'kein wav für Re-Transkription' }); - const numSpeakers = +(req.body && req.body.numSpeakers) || 0; - await db.setStatus(id, 'transcribing'); - await db.updateAnalysis(id, { title: null, summary: null, todos: null, status: 'pending' }); - res.json({ ok: true, status: 'transcribing' }); - withNpuLock(`retranscribe #${id}`, () => transcribeAndAnalyze(id, r.wav_file, numSpeakers)); -})); - -// Longest contiguous run of one diarization cluster (best clean sample of a voice). -function longestRun(words, speaker) { - let best = null, cur = null; - for (const w of words || []) { - if (w.speaker !== speaker) { cur = null; continue; } - if (!cur) cur = { start: w.t_start_ms, end: w.t_end_ms }; - else cur.end = w.t_end_ms; - if (!best || (cur.end - cur.start) > (best.end - best.start)) best = { start: cur.start, end: cur.end }; - } - return best; -} - -// Name a diarized speaker in a recording and (default) LEARN their voice: fold -// the speaker's longest clean segment into the named voiceprint centroid, so -// future recordings auto-identify them. Empty name clears the label. -app.post('/api/recordings/:id/speaker', wrap(async (req, res) => { - const id = +req.params.id; - const speaker = String((req.body && req.body.speaker) || '').trim(); - const name = String((req.body && req.body.name) || '').trim(); - const enroll = !(req.body && req.body.enroll === false); - if (!speaker) return res.status(400).json({ error: 'speaker fehlt' }); - const r = await db.getRecording(id); - if (!r) return res.status(404).json({ error: 'not found' }); - await db.setSpeakerName(id, speaker, name); - // Patch the name into the already generated summary/todos/participants. - const relabel = await relabelAnalysis(id).catch(e => { console.error('relabel:', e.message); return { changed: 0 }; }); - - let enrolled = false, learnMsg = null; - if (enroll && name && r.wav_file) { - const seg = longestRun(r.words, speaker); - const wavPath = path.join(REC_DIR, r.wav_file); - if (!seg || (seg.end - seg.start) < 1500) { - learnMsg = 'Segment zu kurz zum Lernen (<1,5s)'; - } else if (!fs.existsSync(wavPath)) { - learnMsg = 'wav fehlt – nicht gelernt'; - } else { - const cut = `/tmp/enr_${id}_${speaker}.wav`; - try { - await pexec('ffmpeg', ['-y', '-i', wavPath, '-ss', String(seg.start / 1000), - '-to', String(seg.end / 1000), '-ar', '16000', '-ac', '1', cut]); - const emb = await spk.embed(cut); - await db.upsertVoiceprint(name, emb, spk.modelId()); - enrolled = true; - } catch (e) { console.error('enroll-from-recording failed:', e.message); learnMsg = 'Lernen fehlgeschlagen'; } - finally { try { fs.unlinkSync(cut); } catch {} } - } - } - const r2 = await db.getRecording(id); - res.json({ ok: true, speaker_names: r2.speaker_names || {}, enrolled, learnMsg, - relabeled: relabel.changed }); -})); - -app.get(/^(?!\/api|\/media).*/, (req, res) => res.sendFile(path.join(__dirname, 'web/dist/index.html'))); - -const PORT = process.env.PORT || 3000; -db.ensureSchema() - .catch(e => console.error('ensureSchema:', e.message)) - .finally(() => app.listen(PORT, '0.0.0.0', () => console.log(`callrec listening on :${PORT}`))); +import express from 'express'; +import path from 'node:path'; +import fs from 'node:fs'; +import { execFile } from 'node:child_process'; +import { promisify } from 'node:util'; +import { fileURLToPath } from 'node:url'; +import * as bt from './bt.js'; +import * as rec from './recorder.js'; +import * as db from './db.js'; +import * as llm from './llm.js'; +import * as diar from './diar.js'; +import * as spk from './spk.js'; +import * as search from './search.js'; +import * as emb from './emb.js'; +const pexec = promisify(execFile); + +// Glossary → { soundalike(lowercased): correctTerm }. Only entries WITH +// soundalikes get a deterministic swap; term-only entries are left to the LLM hint. +function glossaryMap(glossary) { + const m = {}; + for (const g of glossary || []) { + if (!g.term || !g.soundalikes) continue; + for (const s of String(g.soundalikes).split(',')) { + const k = s.trim().toLowerCase(); + if (k) m[k] = String(g.term).trim(); + } + } + return m; +} +// Whole-word replace (Unicode-aware, case-insensitive match): swaps any token +// equal to a known soundalike for its correct term, leaving punctuation intact. +function applyGlossaryText(text, m) { + if (!text || !m || !Object.keys(m).length) return text; + return String(text).replace(/[\p{L}\p{N}]+/gu, (tok) => m[tok.toLowerCase()] || tok); +} + +// Build the LLM input from stored words. If acoustic diarization tagged speakers, +// produce a labelled dialogue ("Sprecher 1: …") so the LLM keeps real speakers +// instead of guessing; otherwise the plain transcript. +// Also returns `labels` = the exact label every speaker carries in the text the +// LLM sees ({speaker_07: "Sprecher 5"}); it is stored with the analysis so a +// rename afterwards can be patched into the finished summary (relabelAnalysis). +function buildBasis(rec, nameMap = {}) { + const words = rec.words || []; + const hasSpk = words.some(w => w.speaker); + if (!hasSpk) return { text: rec.transcript || '', labeled: false, labels: {} }; + const turns = diar.toTurns(words); + const labels = {}; let n = 0; + // Labels MUST be unique. On #79 a mis-anchored marker named Sascha's cluster + // "Markus" while the voiceprint named Markus' own cluster "Markus" too — the + // LLM then read two different people under one name and blended them into one + // person, and relabelAnalysis refuses to patch an ambiguous label afterwards. + const used = new Set(); + const uniq = (name) => { + if (!used.has(name)) { used.add(name); return name; } + for (let k = 2; ; k++) if (!used.has(`${name} ${k}`)) { used.add(`${name} ${k}`); return `${name} ${k}`; } + }; + const label = sp => labels[sp] || (labels[sp] = uniq(nameMap[sp] || `Sprecher ${++n}`)); + const text = turns.map(t => `${label(t.speaker)}: ${t.text}`).join('\n'); + return { text, labeled: true, labels }; +} + +// Swap speaker labels for real names in the ALREADY FINISHED analysis text. +// Renaming a speaker after the analysis used to leave "Sprecher 5" all over the +// summary/todos; re-running the LLM costs ~20 min on this board, while the label +// each speaker had is known exactly (recordings.speaker_labels). Whole-word +// replacement in summary, todos, transcript_clean and participants; the stored +// label is then updated to the new name so a second rename works too. +async function relabelAnalysis(id) { + const rec = await db.getRecording(id); + if (!rec) return { changed: 0 }; + const names = rec.speaker_names || {}; + const labels = await db.getSpeakerLabels(id); + if (!Object.keys(names).length || !Object.keys(labels).length) return { changed: 0 }; + // A label that TWO speakers carried (the voice-ID once matched "Fabio" onto two + // clusters) is ambiguous in the text — leave those alone instead of renaming + // both occurrences to one person. + const labelCount = {}; + for (const l of Object.values(labels)) labelCount[l] = (labelCount[l] || 0) + 1; + const pairs = Object.entries(names) + .filter(([sp, name]) => name && labels[sp] && labels[sp] !== name && labelCount[labels[sp]] === 1) + .map(([sp, name]) => ({ sp, from: labels[sp], to: name })) + // longest label first: "Sprecher 1" must not eat "Sprecher 12" + .sort((a, b) => b.from.length - a.from.length); + const esc = s => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); + const swap = (text) => { + if (!text) return text; + let out = String(text); + for (const p of pairs) out = out.replace(new RegExp(`(? swap(String(t))) : rec.todos; + // Participants are REBUILT (not string-swapped) from the diarized speakers in + // order of first appearance — a swap would leave the entries of speakers that + // were never mentioned in the text untouched and can duplicate names. + const spkOrder = []; + for (const w of (rec.words || [])) if (w.speaker && !spkOrder.includes(w.speaker)) spkOrder.push(w.speaker); + const participants = spkOrder.length + ? spkOrder.map((sp, i) => names[sp] || labels[sp] || `Sprecher ${i + 1}`) + : (Array.isArray(rec.participants) ? rec.participants.map(p => swap(String(p))) : rec.participants); + const partChanged = JSON.stringify(participants || null) !== JSON.stringify(rec.participants || null); + if (!pairs.length && !partChanged) return { changed: 0 }; + await db.pool.execute( + 'UPDATE recordings SET summary=?, todos=?, transcript_clean=?, participants=? WHERE id=?', + [swap(rec.summary), todos ? JSON.stringify(todos) : null, swap(rec.transcript_clean), + participants ? JSON.stringify(participants) : null, id]); + const next = { ...labels }; + for (const p of pairs) next[p.sp] = p.to; + await db.setSpeakerLabels(id, next); + console.log(`relabel #${id}: ${pairs.length ? pairs.map(p => `${p.from}→${p.to}`).join(', ') : 'nur Teilnehmerliste'}`); + return { changed: pairs.length || (partChanged ? 1 : 0) }; +} + +const REC_DIR2 = path.join(path.dirname(fileURLToPath(import.meta.url)), 'recordings'); +// Match each diarization cluster against enrolled voiceprints → { speaker: name }. +// Cuts the speaker's longest contiguous run from the kept wav, embeds it, cosine- +// matches. Returns only confident matches (others stay "Sprecher N"). +async function identifySpeakers(recData) { + const words = recData.words || []; + const speakers = [...new Set(words.filter(w => w.speaker).map(w => w.speaker))]; + if (!speakers.length || !recData.wav_file) return {}; + const wavPath = path.join(REC_DIR2, recData.wav_file); + if (!fs.existsSync(wavPath)) return {}; + const vps = await db.listVoiceprints(spk.modelId()); + if (!vps.length) return {}; + const best = {}; let cur = null; + for (const w of words) { + if (!cur || cur.speaker !== w.speaker) cur = { speaker: w.speaker, start: w.t_start_ms, end: w.t_end_ms }; + else cur.end = w.t_end_ms; + const len = cur.end - cur.start; + if (!best[cur.speaker] || len > best[cur.speaker].len) best[cur.speaker] = { start: cur.start, end: cur.end, len }; + } + // Cut at most ID_CUT_SEC: onnxruntime ABORTS the embedding helper on very long + // inputs (reproduced: 180 s -> "Non-zero status code returned while running + // Where"; 20/60 s fine). On #74 the longest run of a moderator was minutes + // long, which killed the helper on the first speaker and — through the then + // poisoned promise chain in spk.js — every speaker after it ("spk beendet"), + // i.e. voice auto-naming was dead for the whole recording. + const ID_CUT_SEC = 25; + const hits = []; + for (const sp of speakers) { + const r = best[sp]; if (!r) continue; + const seg = `/tmp/idseg_${recData.id}_${sp}.wav`; + const from = r.start / 1000; + const to = Math.min(r.end / 1000, from + ID_CUT_SEC); + try { + await pexec('ffmpeg', ['-y', '-i', wavPath, '-ss', String(from), '-to', String(to), '-ar', '16000', '-ac', '1', seg]); + const emb = await spk.embed(seg); + const m = spk.bestMatch(emb, vps); + if (m) hits.push({ speaker: sp, name: m.name, score: m.score }); + } catch (e) { console.error('identify', sp, e.message); } + finally { try { fs.unlinkSync(seg); } catch {} } + } + // One enrolled person can only BE one cluster: keep the best-scoring match per + // name (#74 handed "Fabio" to two clusters at once). + const bestByName = {}; + for (const h of hits) if (!bestByName[h.name] || h.score > bestByName[h.name].score) bestByName[h.name] = h; + const map = {}; + for (const h of Object.values(bestByName)) map[h.speaker] = h.name; + return map; +} + +// Fold a speaker's longest clean run into the named voiceprint centroid, so the +// next recording can auto-identify them. Shared by the manual rename popup and +// the live-marker resolution. Returns true if a sample was learned. +async function enrollSpeakerVoice(recData, speaker, name) { + if (!name || !recData.wav_file) return false; + const seg = longestRun(recData.words, speaker); + if (!seg || (seg.end - seg.start) < 1500) return false; + const wavPath = path.join(REC_DIR2, recData.wav_file); + if (!fs.existsSync(wavPath)) return false; + const cut = `/tmp/enr_${recData.id}_${speaker}.wav`; + try { + await pexec('ffmpeg', ['-y', '-i', wavPath, '-ss', String(seg.start / 1000), + '-to', String(Math.min(seg.end, seg.start + 20000) / 1000), '-ar', '16000', '-ac', '1', cut]); + const emb = await spk.embed(cut); + await db.upsertVoiceprint(name, emb, spk.modelId()); + return true; + } catch (e) { console.error('enroll failed:', e.message); return false; } + finally { try { fs.unlinkSync(cut); } catch {} } +} + +// Apply the names the user typed DURING the recording to the diarized clusters. +// Runs after every (re-)diarization, before the LLM analysis, so the summary +// already talks about real people. Two effects: +// 1. each resolved cluster gets its typed name (authoritative, manual-level), +// 2. if ONE name confidently lands on SEVERAL clusters, those clusters are the +// same person split by the diarizer → they are merged (user-supervised fix +// for the residual over-segmentation). +async function applyMarkers(id) { + const rec = await db.getRecording(id); + if (!rec || !(rec.markers || []).length || !(rec.words || []).some(w => w.speaker)) return null; + const resolved = diar.resolveMarkers(rec.words, rec.markers); + for (const m of resolved) await db.setMarkerSpeaker(m.id, m.speaker, m.conf).catch(() => {}); + + // name -> clusters. NAMING may follow a 'mid' hit, but the cluster MERGE below + // is destructive and unguarded (no acoustic check at this level), so only a + // 'high' hit — marker inside the turn, that turn clearly dominant — is allowed + // to fold two clusters together. #79 lost a speaker exactly this way one layer + // deeper, in the diarizer's must-link. + const byName = {}; + for (const m of resolved) { + if (!m.speaker || m.conf !== 'high') continue; + (byName[m.name] = byName[m.name] || new Set()).add(m.speaker); + } + const nameOnly = {}; + for (const m of resolved) { + if (!m.speaker || m.conf === 'low' || m.conf === null) continue; + (nameOnly[m.name] = nameOnly[m.name] || new Set()).add(m.speaker); + } + const talk = {}; + for (const w of rec.words) if (w.speaker) talk[w.speaker] = (talk[w.speaker] || 0) + (w.t_end_ms - w.t_start_ms); + + const fold = {}; // extra cluster -> primary cluster + const nameOf = {}; // cluster -> typed name + for (const [name, set] of Object.entries(byName)) { + const cls = [...set].sort((a, b) => (talk[b] || 0) - (talk[a] || 0)); + const primary = cls[0]; + nameOf[primary] = name; + for (const c of cls.slice(1)) fold[c] = primary; + } + // Naming without merging: a 'mid' hit labels the cluster it landed on, as long + // as that cluster is not already claimed by somebody else. + for (const [name, set] of Object.entries(nameOnly)) { + if (byName[name]) continue; + const cls = [...set].sort((a, b) => (talk[b] || 0) - (talk[a] || 0)); + if (cls[0] && !nameOf[cls[0]]) nameOf[cls[0]] = name; + } + let merged = 0; + if (Object.keys(fold).length) { + for (const w of rec.words) if (fold[w.speaker]) w.speaker = fold[w.speaker]; + await db.replaceWords(id, rec.words); + merged = Object.keys(fold).length; + console.log(`markers: merged ${merged} cluster(s) into marker-named speakers on #${id}`); + } + for (const [sp, name] of Object.entries(nameOf)) await db.setSpeakerName(id, sp, name).catch(() => {}); + // Learn the voices so the NEXT recording identifies them automatically. + const rec2 = await db.getRecording(id); + let learned = 0; + for (const [sp, name] of Object.entries(nameOf)) if (await enrollSpeakerVoice(rec2, sp, name)) learned++; + const named = Object.keys(nameOf).length; + await relabelAnalysis(id).catch(() => {}); + console.log(`markers on #${id}: ${resolved.length} marker(s) → ${named} named speaker(s), ${merged} merged, ${learned} voice(s) learned`); + return { resolved, named, merged, learned }; +} + +// Run LLM analysis for a recording and persist title/summary/todos/participants/ +// clean dialogue. Background (NPU ~30-90s) so record/stop stays fast. +// A76 clock cap for the whole run: rkllm burns ~1.7 CPU cores alongside the NPU +// and a long Gemma generation hit 95-97°C uncapped on the fanless board (#70, +// 16-min run) — same shutdown risk transcribeWav/rediarize already guard against. +async function analyzeRecording(id) { + // Free the resident rkllm (~5.9GB) BEFORE identifySpeakers: the TitaNet + // spk-embed helper OOM-dies next to it ("spk beendet" per cluster, #71 got + // 48 raw speakers). llm.analyze() below respawns it (~96s load, noise + // against a 15-min analysis). + llm.stopServer(); + await pexec('sudo', ['-n', '/opt/callrec/aime-cpucap.sh', '1608000']).catch(() => {}); + try { + const rec = await db.getRecording(id); + if (!rec) return; + const nameMap = await identifySpeakers(rec).catch(() => ({})); + // Seed the per-recording speaker names from acoustic identification (fills + // blanks only; manual edits stay). Manual names win for the LLM labelling. + await db.mergeSpeakerNames(id, nameMap).catch(() => {}); + // Stored names normally win over acoustic identification (they are typed by + // hand). Exception: a name that came from a NON-'high' marker is only a + // guess about WHICH cluster was speaking — resolveMarkers looks back up to + // 25 s, so in a fast four-way discussion it lands on the wrong voice (#79 + // put "Markus" on Sascha's cluster). If an enrolled voiceprint disagrees + // with such a weak marker name, the voiceprint wins. + const markers = await db.listMarkers(id).catch(() => []); + const weakName = new Map(); + for (const m of markers) if (m.speaker && m.conf !== 'high') weakName.set(m.speaker, m.name); + const names = { ...nameMap, ...(rec.speaker_names || {}) }; + for (const [sp, vname] of Object.entries(nameMap)) { + if (names[sp] !== vname && weakName.get(sp) === names[sp]) { + console.log(`identify #${id}: Stimmabdruck "${vname}" schlägt unsicheren Marker "${names[sp]}" auf ${sp}`); + names[sp] = vname; + await db.setSpeakerName(id, sp, vname).catch(() => {}); + } + } + const glossary = (await db.listGlossary().catch(() => [])).filter(g => g.enabled); + // Deterministic soundalike replacement: the 3B model is unreliable at obeying + // a "map Amy->Aime" instruction (it treats "Amy" as a valid name), so entries + // WITH soundalikes are applied as a hard whole-word swap on the words FIRST — + // then the LLM sees the correct term and the turn view shows it too. Entries + // without soundalikes still rely on the (contextual) LLM glossary hint. + const gmap = glossaryMap(glossary); + if (Object.keys(gmap).length && Array.isArray(rec.words) && rec.words.length) { + let changed = false; + for (const w of rec.words) { const nt = applyGlossaryText(w.word, gmap); if (nt !== w.word) { w.word = nt; changed = true; } } + if (changed) { await db.replaceWords(id, rec.words); console.log(`glossary: applied soundalike swaps to #${id} words`); } + } + const { text, labeled, labels } = buildBasis(rec, names); + // Remember which label each speaker had in the LLM input, so renaming a + // speaker later can patch the finished text (relabelAnalysis). + await db.setSpeakerLabels(id, labels).catch(() => {}); + const a = await llm.analyze(text, { labeled, glossary }); + // Participants = the ACOUSTICALLY diarized speakers (named where known), + // not the LLM's list — the model returned every name MENTIONED in the + // conversation as a "Teilnehmer" (11 names on a 3-person call, #69). + // LLM participants stay only as fallback when there is no diarization. + const spkOrder = []; + for (const w of (rec.words || [])) if (w.speaker && !spkOrder.includes(w.speaker)) spkOrder.push(w.speaker); + // Same labels as in the dialogue the LLM read — otherwise the participant + // list numbers the speakers differently than the summary talks about them. + if (spkOrder.length) a.participants = spkOrder.map((sp, i) => names[sp] || labels[sp] || `Sprecher ${i + 1}`); + // Belt-and-suspenders: also swap in the LLM outputs (cheap, catches any the + // model reintroduced). + for (const k of ['title', 'summary', 'transcriptClean']) if (a[k]) a[k] = applyGlossaryText(a[k], gmap); + a.participants = (a.participants || []).map(p => applyGlossaryText(p, gmap)); + a.todos = (a.todos || []).map(t => applyGlossaryText(t, gmap)); + await db.updateAnalysis(id, { ...a, status: 'done' }); + // A speaker can be named WHILE the analysis runs (it takes ~15 min, and the + // marker/voiceprint passes themselves rename clusters). The finished text + // still carries the label from analysis start ("Sprecher 1"), so patch it + // once at the end — same mechanism as renaming a speaker by hand. + await relabelAnalysis(id).catch(() => {}); + // Rebuild the search index last: it needs the final speaker names, which + // relabelAnalysis may only just have settled. + await search.indexRecording(id).catch(e => console.error('search index:', e.message)); + console.log(`analysis done for #${id}: ${a.title}`); + } catch (e) { + console.error(`analysis failed for #${id}:`, e.message); + await db.updateAnalysis(id, { title: null, summary: null, todos: null, status: 'error' }).catch(() => {}); + } finally { + await pexec('sudo', ['-n', '/opt/callrec/aime-cpucap.sh', '2256000']).catch(() => {}); + } +} + +const __dirname = path.dirname(fileURLToPath(import.meta.url)); +const app = express(); +app.use(express.json()); + +const REC_DIR = path.join(__dirname, 'recordings'); +app.use('/media', express.static(REC_DIR)); +app.use(express.static(path.join(__dirname, 'web/dist'))); + +const wrap = fn => (req, res) => fn(req, res).catch(e => res.status(500).json({ error: String(e && e.message || e) })); + +app.get('/api/health', (req, res) => res.json({ ok: true, ...rec.status() })); + +// --- Bluetooth --- +app.post('/api/bt/scan', wrap(async (req, res) => { + const secs = Math.min(+(req.body && req.body.seconds) || 10, 30); + res.json(await bt.scan(secs)); +})); +app.get('/api/bt/devices', wrap(async (req, res) => res.json(await bt.knownDevices()))); +app.post('/api/bt/pair', wrap(async (req, res) => res.json(await bt.pair(req.body.mac)))); +app.post('/api/bt/connect', wrap(async (req, res) => res.json(await bt.connect(req.body.mac)))); +app.post('/api/bt/repair', wrap(async (req, res) => res.json(await bt.repair(req.body.mac)))); +app.post('/api/bt/disconnect', wrap(async (req, res) => res.json(await bt.disconnect(req.body.mac)))); + +// --- Audio --- +app.get('/api/audio/sources', wrap(async (req, res) => res.json(await rec.listSources()))); +app.post('/api/audio/headset-mic', wrap(async (req, res) => res.json(await rec.enableHeadsetMic((req.body && req.body.codec) || 'auto')))); + +// --- Global NPU pipeline serializer --------------------------------------- +// Transcription (whisper-turbo ENCODER on the NPU) and LLM analysis (rkllm on +// the NPU) cannot run simultaneously: while the whisper encoder holds the NPU, +// rkllm_init fails with "failed to allocate handle, ret:-1, errno:14, Bad +// address", which surfaces as "rkllm READY timeout" and a failed/empty +// analysis. This bit when two recordings finished within seconds of each other +// (both fired transcribe+analyze fire-and-forget with no coordination). Every +// heavy per-recording job (transcribe / analyze / re-diarize) now runs through +// this single FIFO queue, so exactly one touches the NPU at a time; a second +// recording simply waits its turn instead of colliding. +let npuChain = Promise.resolve(); +let npuDepth = 0; +function withNpuLock(label, fn) { + npuDepth++; + const waiting = npuDepth - 1; + const run = npuChain.then(async () => { + if (waiting > 0) console.log(`npu-queue: ${label} start (${waiting} ahead done)`); + try { return await fn(); } + finally { npuDepth--; } + }); + npuChain = run.catch(() => {}); // keep the chain alive across a job's failure + return run; +} + +// --- Recording --- +app.post('/api/record/start', wrap(async (req, res) => res.json(await rec.startRecording(req.body || {})))); +// Live speaker markers: type the name of whoever is talking WHILE recording. +app.post('/api/record/marker', wrap(async (req, res) => res.json(rec.addMarker((req.body && req.body.name) || '')))); +app.get('/api/record/markers', (req, res) => res.json(rec.listMarkers())); +app.delete('/api/record/marker/:index', wrap(async (req, res) => res.json({ markers: rec.removeMarker(req.params.index) }))); +// Transcribe (thermal-guarded whisper + diarize) then analyze, in the background. +// The wav + DB row already exist, so a crash here loses nothing — the recording +// stays with status 'transcribing' and can be recovered via /retranscribe. +async function transcribeAndAnalyze(id, wavFile, numSpeakers) { + await db.markAnalysisStart(id).catch(() => {}); + try { + // Live markers constrain the diarization itself (same name = one speaker, + // different names = never merged), so they are loaded before transcribing. + const markers = await db.listMarkers(id).catch(() => []); + // Enrolled voices do the same for people who were never typed — the box + // owner above all (see diar.clusterWindows). + const voiceprints = await db.listVoiceprints(spk.modelId()).catch(() => []); + const { transcript, words } = await rec.transcribeWav(wavFile, numSpeakers, { markers, voiceprints }); + await db.setTranscript(id, transcript, words); + console.log(`transcription done for #${id}: ${words.length} segments`); + // Names typed live → cluster names (+ user-supervised cluster merge) BEFORE + // the LLM sees the dialogue. + await applyMarkers(id).catch(e => console.error('applyMarkers:', e.message)); + } catch (e) { + console.error(`transcription failed for #${id}:`, e.message); + await db.setStatus(id, 'error').catch(() => {}); + // also mark analysis 'error' so the UI stops polling and shows a retry + await db.updateAnalysis(id, { title: null, summary: null, todos: null, status: 'error' }).catch(() => {}); + return; // no transcript -> skip LLM analysis + } + analyzeRecording(id); +} + +app.post('/api/record/stop', wrap(async (req, res) => { + // Persist wav + mp3 + DB row FIRST (crash-safe), respond immediately, then + // transcribe/analyze in the background. The frontend already polls while + // analysis_status === 'pending', so the transcript + analysis fill in live. + const r = await rec.stopAndSave(); + const id = await db.addRecording({ ...r, transcript: '', status: 'transcribing' }); + await db.addMarkers(id, r.markers).catch(e => console.error('addMarkers:', e.message)); + res.json({ id, title: r.title, mp3_file: r.mp3_file, duration_ms: r.duration_ms, + transcript: '', status: 'transcribing', analysis_status: 'pending', + markers: (r.markers || []).length }); + withNpuLock(`transcribe #${id}`, () => transcribeAndAnalyze(id, r.wav_file, r.numSpeakers)); +})); + +// --- Recordings --- +app.get('/api/recordings', wrap(async (req, res) => res.json(await db.listRecordings()))); +app.get('/api/recordings/:id', wrap(async (req, res) => { + const r = await db.getRecording(+req.params.id); + r ? res.json(r) : res.status(404).json({ error: 'not found' }); +})); +// Rename. An empty title clears it AND unlocks, so the next analysis may +// propose one again (see db.renameRecording / updateAnalysis). +app.patch('/api/recordings/:id/title', wrap(async (req, res) => { + const id = +req.params.id; + const row = await db.renameRecording(id, String((req.body && req.body.title) || '')); + if (!row) return res.status(404).json({ error: 'not found' }); + res.json({ ok: true, id, title: row.title, title_locked: !!row.title_locked }); +})); + +// --- Trash ------------------------------------------------------------------- +// Delete = move to the bin. Nothing on disk is touched until the retention +// window expires, so a restore brings back audio, transcript and analysis. +app.delete('/api/recordings/:id', wrap(async (req, res) => { + await db.deleteRecording(+req.params.id); + res.json({ ok: true, trashed: true, days: db.TRASH_DAYS }); +})); +app.get('/api/trash', wrap(async (req, res) => res.json({ days: db.TRASH_DAYS, items: await db.listTrash() }))); +app.post('/api/trash/:id/restore', wrap(async (req, res) => { + await db.restoreRecording(+req.params.id); + res.json({ ok: true }); +})); +// Irreversible: drops the row, its words/markers/passages and the audio files. +app.delete('/api/trash/:id', wrap(async (req, res) => { + res.json({ ok: true, purged: await purgeRecording(+req.params.id) }); +})); + +// Shared by the manual purge route and the retention sweep. +async function purgeRecording(id) { + const files = await db.purgeRecording(id); + let removed = 0; + for (const f of files) { + try { fs.unlinkSync(path.join(REC_DIR, f)); removed++; } + catch (e) { if (e.code !== 'ENOENT') console.error(`purge #${id}: ${f} — ${e.message}`); } + } + return { id, files: removed }; +} + +// Retention sweep: once at boot (the box is often off for days, so a timer +// alone would keep missing the window) and then daily. +async function sweepTrash() { + const ids = await db.expiredTrashIds().catch(() => []); + for (const id of ids) { + try { await purgeRecording(id); console.log(`trash: #${id} nach ${db.TRASH_DAYS} Tagen endgültig gelöscht`); } + catch (e) { console.error(`trash sweep #${id}:`, e.message); } + } +} + +// --- Search ------------------------------------------------------------------ +// Hits are pure SQL and come back immediately; the written answer is a separate +// call because the LLM needs ~30-90 s on this hardware (see search.js). +app.get('/api/search', wrap(async (req, res) => { + const q = String(req.query.q || '').trim(); + if (q.length < 2) return res.json({ hits: [], terms: [], range: null, semantic: false, total: 0 }); + res.json(await search.search(q, { limit: Math.min(+req.query.limit || 20, 50) })); +})); +// Runs through the NPU queue: rkllm and the whisper encoder cannot share the NPU. +app.post('/api/search/answer', wrap(async (req, res) => { + const q = String((req.body && req.body.q) || '').trim(); + if (!q) return res.status(400).json({ error: 'q fehlt' }); + const { hits } = await search.search(q, { limit: 20 }); + res.json(await withNpuLock('search-answer', () => search.answer(q, hits))); +})); +app.get('/api/search/status', wrap(async (req, res) => { + const [[{ n }]] = await db.pool.query('SELECT COUNT(*) AS n FROM passages'); + const [[{ v }]] = await db.pool.query('SELECT COUNT(*) AS v FROM passages WHERE vec IS NOT NULL'); + res.json({ passages: n, embedded: v, ...emb.status(), indexing: !!indexJob }); +})); +// Background reindex; `force` rebuilds everything, otherwise only recordings +// that have no passages yet (the normal case after this update). +let indexJob = null; +app.post('/api/search/reindex', wrap(async (req, res) => { + if (indexJob) return res.json({ ok: true, already: true }); + const force = !!(req.body && req.body.force); + indexJob = search.reindexAll({ force, onProgress: (d, t) => console.log(`search: reindex ${d}/${t}`) }) + .then(r => { console.log(`search: reindex fertig — ${r.indexed}/${r.total}`); return r; }) + .catch(e => { console.error('search: reindex:', e.message); }) + .finally(() => { indexJob = null; }); + res.json({ ok: true, started: true }); +})); + +// --- Voiceprints / enrollment --- +app.get('/api/voiceprints', wrap(async (req, res) => { + const vps = await db.listVoiceprints(); + const cur = spk.modelId(); + res.json(vps.map(v => ({ id: v.id, name: v.name, dim: v.dim, model: v.model, + stale: v.model !== cur, created_at: v.created_at }))); +})); +app.delete('/api/voiceprints/:id', wrap(async (req, res) => { await db.deleteVoiceprint(+req.params.id); res.json({ ok: true }); })); +app.post('/api/enroll/start', wrap(async (req, res) => res.json(await rec.startRecording({ source: req.body && req.body.source })))); +app.post('/api/enroll/stop', wrap(async (req, res) => { + const name = (req.body && req.body.name || '').trim(); + if (!name) return res.status(400).json({ error: 'Name fehlt' }); + const wav = await rec.stopRaw(); + try { + const emb = await spk.embed(wav); + const id = await db.addVoiceprint(name, emb, spk.modelId()); + res.json({ id, name, dim: emb.length }); + } finally { try { fs.unlinkSync(wav); } catch {} } +})); + +// --- Glossary (known proper nouns / terms; fed into LLM correction) --- +app.get('/api/glossary', wrap(async (req, res) => res.json(await db.listGlossary()))); +app.post('/api/glossary', wrap(async (req, res) => { + const term = (req.body && req.body.term || '').trim(); + const soundalikes = (req.body && req.body.soundalikes || '').trim(); + if (!term) return res.status(400).json({ error: 'Begriff fehlt' }); + res.json(await db.addGlossary(term, soundalikes)); +})); +app.delete('/api/glossary/:id', wrap(async (req, res) => { await db.deleteGlossary(+req.params.id); res.json({ ok: true }); })); +app.put('/api/glossary/:id', wrap(async (req, res) => { + const term = (req.body && req.body.term || '').trim(); + const soundalikes = (req.body && req.body.soundalikes || '').trim(); + if (!term) return res.status(400).json({ error: 'Begriff fehlt' }); + const g = await db.updateGlossary(+req.params.id, term, soundalikes); + if (!g) return res.status(404).json({ error: 'Eintrag nicht gefunden' }); + res.json(g); +})); + +// --- LLM --- +app.get('/api/llm/status', (req, res) => res.json(llm.status())); +app.get('/api/spk/status', (req, res) => res.json(spk.status())); +// Re-diarize a recording's words with an (optional) fixed speaker count, then +// persist the new speaker labels. Text/timestamps are untouched. Needs the wav. +async function rediarize(id, numSpeakers) { + const r = await db.getRecording(id); + if (!r || !r.wav_file) throw new Error('wav für Re-Diarisierung fehlt'); + const wav = path.join(REC_DIR, r.wav_file); + if (!fs.existsSync(wav)) throw new Error('wav-Datei nicht gefunden: ' + r.wav_file); + const words = r.words || []; + // Free the resident rkllm first — diar's consolidation embeds every cluster + // via TitaNet, which OOM-dies next to a resident 5.9GB LLM (see + // analyzeRecording; on #71 that silently yielded 48 raw speakers). + llm.stopServer(); + // Diarization runs on the A76 cores; the board is fanless, so cap the clock + // for the whole run (transcribeWav does this for the record path — rediarize + // is the other entry point and must do it too, or a long re-diar overheats). + await pexec('sudo', ['-n', '/opt/callrec/aime-cpucap.sh', '1608000']).catch(() => {}); + try { + // Enrolled voices go IN, not just out: they keep two known people (above all + // the box owner, who is enrolled and talks in every recording) from ending up + // in one cluster. See the cannot-link block in diar.clusterWindows. + const voiceprints = await db.listVoiceprints(spk.modelId()).catch(() => []); + const segs = await diar.diarize(wav, numSpeakers, { markers: r.markers || [], voiceprints }); + diar.assignSpeakers(words, segs); + await db.replaceWords(id, words); + // Re-diarization RENUMBERS the clusters, so every previously stored name + // ("speaker_02" = Fabio) would now point at a different voice. Drop them; the + // live markers and the voiceprint identification re-fill them right after. + await db.pool.query('UPDATE recordings SET speaker_names=NULL WHERE id=?', [id]).catch(() => {}); + console.log(`re-diarized #${id} (numSpeakers=${numSpeakers || 'auto'}) → ${new Set(segs.map(s => s.speaker)).size} speakers`); + } finally { + await pexec('sudo', ['-n', '/opt/callrec/aime-cpucap.sh', '2256000']).catch(() => {}); + } +} + +// Re-run analysis (e.g. the model wasn't downloaded yet at record time). Optional +// body { numSpeakers } re-diarizes first (fixes over-segmentation on a known count). +app.post('/api/recordings/:id/reanalyze', wrap(async (req, res) => { + const id = +req.params.id; + const r = await db.getRecording(id); + if (!r) return res.status(404).json({ error: 'not found' }); + const numSpeakers = +(req.body && req.body.numSpeakers) || 0; + await db.updateAnalysis(id, { title: null, summary: null, todos: null, status: 'pending' }); + await db.markAnalysisStart(id).catch(() => {}); + res.json({ ok: true, status: 'pending', reDiarize: true }); + // Always re-diarize: numSpeakers>0 pins the count, 0 = auto (now consolidated, + // so auto no longer over-segments). This is the "fix my speakers" path. + withNpuLock(`reanalyze #${id}`, async () => { + try { await rediarize(id, numSpeakers); } + catch (e) { console.error(`re-diarize failed for #${id}:`, e.message); } + await applyMarkers(id).catch(e => console.error('applyMarkers:', e.message)); + await analyzeRecording(id); + }); +})); + +// --- markers of a stored recording ----------------------------------------- +app.get('/api/recordings/:id/markers', wrap(async (req, res) => res.json(await db.listMarkers(+req.params.id)))); +// Add a marker after the fact (e.g. while listening back) — same resolution path. +app.post('/api/recordings/:id/markers', wrap(async (req, res) => { + const name = String((req.body && req.body.name) || '').trim(); + const tMs = +(req.body && req.body.t_ms); + if (!name || !Number.isFinite(tMs)) return res.status(400).json({ error: 'name/t_ms fehlt' }); + await db.addMarker(+req.params.id, tMs, name); + res.json({ ok: true, markers: await db.listMarkers(+req.params.id) }); +})); +app.delete('/api/recordings/:id/markers/:mid', wrap(async (req, res) => { + await db.deleteMarker(+req.params.mid); + res.json({ ok: true, markers: await db.listMarkers(+req.params.id) }); +})); +// Re-resolve the markers against the current diarization (no LLM, no sherpa). +app.post('/api/recordings/:id/apply-markers', wrap(async (req, res) => { + const out = await applyMarkers(+req.params.id); + const r = await db.getRecording(+req.params.id); + res.json({ ok: true, ...(out || { named: 0, merged: 0, learned: 0 }), + speaker_names: (r && r.speaker_names) || {}, markers: (r && r.markers) || [] }); +})); + +// LLM-only re-analysis: re-run title/summary/todos on the EXISTING words + +// speaker labels, WITHOUT re-diarizing (fast — skips the ~40min sherpa pass when +// the speakers are already correct and only the LLM/prompts changed). +app.post('/api/recordings/:id/reanalyze-llm', wrap(async (req, res) => { + const id = +req.params.id; + const r = await db.getRecording(id); + if (!r) return res.status(404).json({ error: 'not found' }); + await db.updateAnalysis(id, { title: null, summary: null, todos: null, status: 'pending' }); + await db.markAnalysisStart(id).catch(() => {}); + res.json({ ok: true, status: 'pending', reDiarize: false }); + withNpuLock(`reanalyze-llm #${id}`, async () => { await analyzeRecording(id); }); +})); + +// Re-run transcription from the saved wav (recovery: a crash/thermal-shutdown +// during whisper leaves the row at status 'transcribing' with an empty +// transcript, or 'error'; the wav is on disk). Optional { numSpeakers }. +app.post('/api/recordings/:id/retranscribe', wrap(async (req, res) => { + const id = +req.params.id; + const r = await db.getRecording(id); + if (!r) return res.status(404).json({ error: 'not found' }); + if (!r.wav_file) return res.status(400).json({ error: 'kein wav für Re-Transkription' }); + const numSpeakers = +(req.body && req.body.numSpeakers) || 0; + await db.setStatus(id, 'transcribing'); + await db.updateAnalysis(id, { title: null, summary: null, todos: null, status: 'pending' }); + res.json({ ok: true, status: 'transcribing' }); + withNpuLock(`retranscribe #${id}`, () => transcribeAndAnalyze(id, r.wav_file, numSpeakers)); +})); + +// Longest contiguous run of one diarization cluster (best clean sample of a voice). +function longestRun(words, speaker) { + let best = null, cur = null; + for (const w of words || []) { + if (w.speaker !== speaker) { cur = null; continue; } + if (!cur) cur = { start: w.t_start_ms, end: w.t_end_ms }; + else cur.end = w.t_end_ms; + if (!best || (cur.end - cur.start) > (best.end - best.start)) best = { start: cur.start, end: cur.end }; + } + return best; +} + +// Name a diarized speaker in a recording and (default) LEARN their voice: fold +// the speaker's longest clean segment into the named voiceprint centroid, so +// future recordings auto-identify them. Empty name clears the label. +app.post('/api/recordings/:id/speaker', wrap(async (req, res) => { + const id = +req.params.id; + const speaker = String((req.body && req.body.speaker) || '').trim(); + const name = String((req.body && req.body.name) || '').trim(); + const enroll = !(req.body && req.body.enroll === false); + if (!speaker) return res.status(400).json({ error: 'speaker fehlt' }); + const r = await db.getRecording(id); + if (!r) return res.status(404).json({ error: 'not found' }); + await db.setSpeakerName(id, speaker, name); + // Patch the name into the already generated summary/todos/participants. + const relabel = await relabelAnalysis(id).catch(e => { console.error('relabel:', e.message); return { changed: 0 }; }); + + let enrolled = false, learnMsg = null; + if (enroll && name && r.wav_file) { + const seg = longestRun(r.words, speaker); + const wavPath = path.join(REC_DIR, r.wav_file); + if (!seg || (seg.end - seg.start) < 1500) { + learnMsg = 'Segment zu kurz zum Lernen (<1,5s)'; + } else if (!fs.existsSync(wavPath)) { + learnMsg = 'wav fehlt – nicht gelernt'; + } else { + const cut = `/tmp/enr_${id}_${speaker}.wav`; + try { + await pexec('ffmpeg', ['-y', '-i', wavPath, '-ss', String(seg.start / 1000), + '-to', String(seg.end / 1000), '-ar', '16000', '-ac', '1', cut]); + const emb = await spk.embed(cut); + await db.upsertVoiceprint(name, emb, spk.modelId()); + enrolled = true; + } catch (e) { console.error('enroll-from-recording failed:', e.message); learnMsg = 'Lernen fehlgeschlagen'; } + finally { try { fs.unlinkSync(cut); } catch {} } + } + } + const r2 = await db.getRecording(id); + // The passage rows carry the speaker NAME denormalised (so search can filter + // by person without a join) — a rename has to be written through. + search.indexRecording(id).catch(e => console.error('search index:', e.message)); + res.json({ ok: true, speaker_names: r2.speaker_names || {}, enrolled, learnMsg, + relabeled: relabel.changed }); +})); + +app.get(/^(?!\/api|\/media).*/, (req, res) => res.sendFile(path.join(__dirname, 'web/dist/index.html'))); + +const PORT = process.env.PORT || 3000; +db.ensureSchema() + .catch(e => console.error('ensureSchema:', e.message)) + .finally(() => app.listen(PORT, '0.0.0.0', () => { + console.log(`callrec listening on :${PORT}`); + // Retention sweep at boot + daily. unref() so it never holds the process open. + sweepTrash().catch(e => console.error('trash sweep:', e.message)); + setInterval(() => sweepTrash().catch(e => console.error('trash sweep:', e.message)), 24 * 3600 * 1000).unref(); + // Backfill the search index for everything recorded before this update. + // Deliberately not under the NPU lock: it is CPU/SQL only and must not + // delay a transcription that is already queued. + search.reindexAll({ onProgress: (d, t) => { if (d === 1 || d === t) console.log(`search: backfill ${d}/${t}`); } }) + .then(r => { if (r.total) console.log(`search: backfill fertig — ${r.indexed}/${r.total} Aufnahmen`); }) + .catch(e => console.error('search backfill:', e.message)); + })); diff --git a/app/web/src/main.js b/app/web/src/main.js index 90629a4..853114b 100644 --- a/app/web/src/main.js +++ b/app/web/src/main.js @@ -1,1010 +1,1260 @@ -import './style.css'; - -const api = { - async get(u) { const r = await fetch(u); if (!r.ok) throw new Error((await r.json()).error || r.status); return r.json(); }, - async post(u, b) { const r = await fetch(u, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify(b || {}) }); if (!r.ok) throw new Error((await r.json()).error || r.status); return r.json(); }, - async del(u) { const r = await fetch(u, { method: 'DELETE' }); return r.json(); }, - async put(u, b) { const r = await fetch(u, { method: 'PUT', headers: { 'content-type': 'application/json' }, body: JSON.stringify(b || {}) }); if (!r.ok) throw new Error((await r.json()).error || r.status); return r.json(); } -}; - -const app = document.querySelector('#app'); -const state = { tab: 'recordings', recording: false, recStart: 0, timer: null, selectedId: null, collapsedDates: new Set(), lastListSig: null, markers: [], mkDraft: '' }; - -function fmt(ms) { const s = Math.floor(ms / 1000); return `${String(Math.floor(s / 60)).padStart(2, '0')}:${String(s % 60).padStart(2, '0')}`; } -function toast(msg, err) { const t = document.createElement('div'); t.className = 'toast' + (err ? ' err' : ''); t.textContent = msg; document.body.appendChild(t); setTimeout(() => t.remove(), 4000); } - -// Theme: data-theme on , chosen in index.html before first paint (no -// flash); the toggle persists to localStorage. -function applyTheme(t) { - document.documentElement.dataset.theme = t; - localStorage.setItem('aime-theme', t); - const b = document.querySelector('#themetoggle'); - if (b) b.textContent = t === 'dark' ? '☀️' : '🌙'; -} - -function render() { - const theme = document.documentElement.dataset.theme || 'dark'; - app.innerHTML = ` -
-
- -

Aime Anruf-Assistent

-
-
- - ${state.recording ? fmt(Date.now() - state.recStart) : '00:00'} - ${state.recording ? ` - - 🏷 ${state.markers.length}` : ''} -
- - -
-
`; - app.querySelectorAll('.tab').forEach(b => b.onclick = () => { state.tab = b.dataset.tab; render(); }); - app.querySelector('#themetoggle').onclick = () => - applyTheme(document.documentElement.dataset.theme === 'dark' ? 'light' : 'dark'); - app.querySelector('#recbtn').onclick = toggleRecord; - bindMarkerInput(); - if (state.recording) startTimerUI(); - ({ bluetooth: viewBluetooth, recordings: viewRecordings, voices: viewVoices, glossary: viewGlossary })[state.tab](); -} - -// Header record button: start = defaults only (default source = CallRecMix, -// codec auto, speaker count auto — the old options tab is gone on purpose). -// A live placeholder card appears at the top of the recordings list instantly. -async function toggleRecord() { - const btn = document.querySelector('#recbtn'); - if (!state.recording) { - try { - await api.post('/api/record/start', {}); - state.recording = true; state.recStart = Date.now(); - state.markers = []; state.mkDraft = ''; - render(); - if (state.tab === 'recordings') { state.lastListSig = null; refreshRecList(); } - } catch (err) { toast(err.message, true); } - } else { - if (btn) { btn.disabled = true; } - clearInterval(state.timer); state.timer = null; - try { - const r = await api.post('/api/record/stop', {}); - state.recording = false; - toast('Aufnahme gespeichert – wird transkribiert…'); - state.tab = 'recordings'; render(); - state.lastListSig = null; refreshRecList(); - openRecording(r.id); - } catch (err) { state.recording = false; toast('Stop fehlgeschlagen: ' + err.message, true); render(); } - } -} -// ---------- live speaker markers ---------- -// While recording: type the name of whoever is talking, Enter. The server stamps -// the offset; after diarization the name lands on that acoustic speaker. Only -// reliable way to tell 20+ conference participants apart. -function bindMarkerInput() { - const inp = document.querySelector('#mkname'); - if (!inp) return; - inp.oninput = () => { state.mkDraft = inp.value; }; - inp.onkeydown = async (e) => { - if (e.key !== 'Enter') return; - const name = inp.value.trim(); - if (!name) return; - inp.value = ''; state.mkDraft = ''; - try { - const r = await api.post('/api/record/marker', { name }); - state.markers = r.markers || []; - updateMarkerUI(); - toast(`🏷 ${fmt(r.t_ms)} ${name}`); - } catch (err) { toast('Marker fehlgeschlagen: ' + err.message, true); } - }; -} -// Refresh the marker count + chip list in place (no re-render → focus stays in -// the input, so names can be typed back-to-back). -function updateMarkerUI() { - const c = document.querySelector('#mkcount'); - if (c) { c.textContent = '🏷 ' + state.markers.length; c.classList.toggle('ok', !!state.markers.length); } - const chips = document.querySelector('#mkchips'); - if (chips) chips.innerHTML = markerChipsHtml(); - bindMarkerChips(); -} -function markerChipsHtml() { - if (!state.markers.length) return 'noch keine Namen markiert'; - return state.markers.map((m, i) => - `${fmt(m.t_ms)} ${escapeHtml(m.name)}`).join(''); -} -function bindMarkerChips() { - document.querySelectorAll('.mkdel').forEach(b => b.onclick = async (e) => { - e.stopPropagation(); - try { - const r = await api.del('/api/record/marker/' + b.dataset.mk); - state.markers = r.markers || []; - updateMarkerUI(); - } catch (err) { toast(err.message, true); } - }); -} - -// Tick every .rectime element (header + list placeholder) from one interval. -function startTimerUI() { - clearInterval(state.timer); - state.timer = setInterval(() => { - const t = fmt(Date.now() - state.recStart); - document.querySelectorAll('[data-rectime]').forEach(el => el.textContent = t); - }, 500); -} - -// ---------- Bluetooth ---------- -async function viewBluetooth() { - const v = document.querySelector('#view'); - v.innerHTML = ` -
-
- - - -
-
Lade…
-
`; - const listEl = v.querySelector('#btlist'); - const msg = v.querySelector('#btmsg'); - - async function load(devices) { - const d = devices || await api.get('/api/bt/devices'); - if (!d.length) { listEl.innerHTML = '
Keine Geräte. „Suchen“ drücken (Headset in Pairing-Modus).
'; return; } - listEl.innerHTML = d.map(dev => ` -
-
- ${dev.name || '(unbenannt)'} ${dev.audio ? '🎧' : ''} -
${dev.mac}
-
- ${dev.connected ? 'verbunden' : dev.paired ? 'gepaart' : 'neu'} - ${dev.connected - ? `` - : dev.paired - ? `` - : ``} - ${dev.audio ? `` : ''} -
`).join(''); - listEl.querySelectorAll('button[data-act]').forEach(b => b.onclick = () => act(b.dataset.act, b.dataset.mac, b)); - } - async function act(a, mac, btn) { - btn.disabled = true; btn.textContent = a === 'repair' ? 'neu koppeln… (~15s)' : '…'; - try { - const r = await api.post('/api/bt/' + a, { mac }); - if (a === 'repair') toast(r.ok ? 'neu verbunden ✓' : 'fehlgeschlagen: ' + (r.msg || r.connect || 'siehe Headset im Pairing-Modus?'), !r.ok); - else toast(`${a}: ${r.connected ? 'verbunden' : 'ok'}`); - if ((a === 'pair' || a === 'connect' || a === 'repair') && (r.connected || r.ok)) await api.post('/api/audio/headset-mic', {}); - } - catch (e) { toast(a + ' fehlgeschlagen: ' + e.message, true); } - await load(); - } - v.querySelector('#scan').onclick = async (e) => { e.target.disabled = true; msg.textContent = 'suche…'; try { await load(await api.post('/api/bt/scan', { seconds: 10 })); } catch (err) { toast(err.message, true); } msg.textContent = ''; e.target.disabled = false; }; - v.querySelector('#refresh').onclick = () => load(); - load().catch(e => listEl.innerHTML = '
' + e.message + '
'); -} - -// ---------- Voices (enrollment) ---------- -async function viewVoices() { - const v = document.querySelector('#view'); - v.innerHTML = ` -
-

Stimme anlernen

-
Person spricht ~15-20s frei (Headset). Danach wird ihr Stimmprofil gespeichert und in Aufnahmen automatisch erkannt.
- - -
-
00:00
- -
-
-
-

Angelernte Stimmen

Lade…
`; - const listEl = v.querySelector('#vplist'); - async function load() { - try { - const vps = await api.get('/api/voiceprints'); - listEl.innerHTML = vps.length ? vps.map(p => ` -
👤 ${escapeHtml(p.name)} -
${new Date(p.created_at).toLocaleString('de-DE')} · ${p.dim}-dim
-
`).join('') - : '
Noch keine Stimmen. Oben anlernen.
'; - listEl.querySelectorAll('[data-del]').forEach(b => b.onclick = async () => { await api.del('/api/voiceprints/' + b.dataset.del); load(); }); - } catch (e) { listEl.innerHTML = '
' + e.message + '
'; } - } - await load(); - const btn = v.querySelector('#vbtn'), msg = v.querySelector('#vmsg'), timerEl = v.querySelector('#vtimer'); - if (state.enrolling) startTimerUI(timerEl); - btn.onclick = async () => { - if (!state.enrolling) { - const name = v.querySelector('#vname').value.trim(); - if (!name) { toast('Bitte Namen eingeben', true); return; } - state.enrollName = name; - try { await api.post('/api/enroll/start', {}); state.enrolling = true; state.recStart = Date.now(); render(); } - catch (e) { toast(e.message, true); } - } else { - btn.disabled = true; btn.textContent = 'Speichere…'; clearInterval(state.timer); - try { - const r = await api.post('/api/enroll/stop', { name: state.enrollName }); - state.enrolling = false; state.enrollName = ''; toast(`Stimme „${r.name}" gespeichert ✓`); render(); - } catch (e) { state.enrolling = false; toast('Fehlgeschlagen: ' + e.message, true); render(); } - } - }; -} - -// ---------- Glossary ---------- -async function viewGlossary() { - const v = document.querySelector('#view'); - v.innerHTML = ` -
-

Begriff hinzufügen

-
Eigennamen & Fachbegriffe, die die Spracherkennung verhört (z.B. „Aime" wird als „Amy" gehört). Die KI-Korrektur ersetzt lautähnliche Wörter dann durch die richtige Schreibweise.
- - - - - - -
-

Glossar

Lade…
`; - const listEl = v.querySelector('#glist'); - let editId = null; // id of the entry currently loaded into the form - const formTitle = v.querySelector('#gformtitle'), saveBtn = v.querySelector('#gadd'), cancelBtn = v.querySelector('#gcancel'); - function setEdit(g) { - editId = g ? g.id : null; - formTitle.textContent = g ? `„${g.term}" bearbeiten` : 'Begriff hinzufügen'; - saveBtn.textContent = g ? 'Speichern' : 'Hinzufügen'; - cancelBtn.style.display = g ? '' : 'none'; - v.querySelector('#gterm').value = g ? g.term : ''; - v.querySelector('#gsound').value = g ? (g.soundalikes || '') : ''; - if (g) v.querySelector('#gterm').focus(); - } - async function load() { - try { - const gs = await api.get('/api/glossary'); - listEl.innerHTML = gs.length ? gs.map(g => ` -
📖 ${escapeHtml(g.term)} - ${g.soundalikes ? `
↔ ${escapeHtml(g.soundalikes)}
` : ''}
- -
`).join('') - : '
Noch keine Begriffe. Oben hinzufügen.
'; - listEl.querySelectorAll('[data-del]').forEach(b => b.onclick = async () => { await api.del('/api/glossary/' + b.dataset.del); if (editId === +b.dataset.del) setEdit(null); load(); }); - listEl.querySelectorAll('[data-edit]').forEach(b => b.onclick = () => setEdit(gs.find(g => g.id === +b.dataset.edit))); - } catch (e) { listEl.innerHTML = '
' + e.message + '
'; } - } - await load(); - cancelBtn.onclick = () => setEdit(null); - saveBtn.onclick = async () => { - const term = v.querySelector('#gterm').value.trim(); - if (!term) { toast('Bitte Begriff eingeben', true); return; } - const soundalikes = v.querySelector('#gsound').value.trim(); - try { - if (editId != null) await api.put('/api/glossary/' + editId, { term, soundalikes }); - else await api.post('/api/glossary', { term, soundalikes }); - toast(`„${term}" gespeichert ✓`); - setEdit(null); - load(); - } catch (e) { toast(e.message, true); } - }; -} - -// ---------- Recordings ---------- -// Analysis-duration counter. Shows the total wall time of a processing run -// (transcribe+diar+LLM): a live 1-Hz counter while it runs (from analysis_started_at), -// frozen to analysis_ms once done. Returns the inner only; callers add the ⏱. -function anaTimeSpan(r) { - if (r.analysis_ms != null) return `${fmt(r.analysis_ms)}`; - if (r.analysis_started_at && (r.analysis_status === 'pending' || r.status === 'transcribing')) - return `${fmt(Date.now() - r.analysis_started_at)}`; - return ''; -} -function ensureAnaTicker() { - if (state.anaTicker) return; - state.anaTicker = setInterval(() => { - document.querySelectorAll('.ana-time.live').forEach(el => { - const t0 = +el.dataset.anaStart; - if (t0) el.textContent = fmt(Date.now() - t0); - }); - }, 1000); -} - -// Split layout: LEFT = date-grouped, collapsible recording list; RIGHT = the -// selected recording's detail (with its own audio player). Crucially, the -// while-analysing auto-refresh only rebuilds the LEFT list — it never touches -// the RIGHT detail pane, so you can keep listening to an old recording while a -// new one is being transcribed/analysed (the old bug: the whole view — detail + -// audio element — was re-rendered every 5s, stopping playback). -function viewRecordings() { - const v = document.querySelector('#view'); - // Entering the tab: show only the newest day expanded, the rest collapsed - // (applied in refreshRecList once the day groups are known). - state.resetCollapse = true; - v.innerHTML = ` -
- -
-
← Aufnahme wählen
-
`; - // Draggable splitter: list width between LIST_MIN (the old fixed width) and - // twice that; persisted for all sessions. - const LIST_MIN = 340, LIST_MAX = 680; - const listPane = v.querySelector('.rec-list-pane'); - const applyW = (w) => { listPane.style.flex = `0 0 ${Math.min(LIST_MAX, Math.max(LIST_MIN, w))}px`; }; - applyW(+localStorage.getItem('aime-list-width') || LIST_MIN); - v.querySelector('.rec-splitter').onmousedown = (e) => { - e.preventDefault(); - const startX = e.clientX, startW = listPane.getBoundingClientRect().width; - document.body.classList.add('resizing'); - const move = (ev) => applyW(startW + (ev.clientX - startX)); - const up = () => { - document.removeEventListener('mousemove', move); - document.removeEventListener('mouseup', up); - document.body.classList.remove('resizing'); - localStorage.setItem('aime-list-width', Math.round(listPane.getBoundingClientRect().width)); - }; - document.addEventListener('mousemove', move); - document.addEventListener('mouseup', up); - }; - refreshRecList(); - if (state.selectedId != null) openRecording(state.selectedId); -} - -// (Re)build ONLY the left list pane. Called on entry and by the auto-refresh -// timer; safe to run while the right pane holds a playing recording. -async function refreshRecList() { - const listEl = document.querySelector('#reclist'); - if (!listEl) return; // left the tab - let recs; - try { recs = await api.get('/api/recordings'); } - catch (e) { listEl.innerHTML = '
' + e.message + '
'; return; } - // Reschedule the poll ONLY while something is still processing. - const stillRunning = recs.some(r => r.analysis_status === 'pending' || r.status === 'transcribing'); - const scheduleNext = () => { if (stillRunning) setTimeout(() => { if (state.tab === 'recordings') refreshRecList(); }, 5000); }; - try { - if (!recs.length && !state.recording) { listEl.innerHTML = '
Noch keine Aufnahmen.
'; state.lastListSig = 'empty'; scheduleNext(); return; } - // Signature of everything the list RENDERS except the live ⏱ seconds (those - // tick via ensureAnaTicker without a rebuild). If nothing meaningful changed, - // DON'T touch the DOM at all — no flicker, no scroll jump, selection kept. - // This is the real fix for "I get pulled away from my recording every 5s": - // the 5s poll now no-ops unless a status/title/todo actually changed. - const sig = recs.map(r => `${r.id}|${r.status}|${r.analysis_status}|${r.title || ''}|${(r.todos || []).length}|${r.summary ? 1 : 0}|${r.duration_ms || 0}`).join('~'); - if (sig === state.lastListSig && listEl.querySelector('.day-group')) { ensureAnaTicker(); scheduleNext(); return; } - state.lastListSig = sig; - // Preserve the left pane's scroll position across the rebuild. - const scroller = listEl.closest('.rec-list-pane') || listEl; - const savedTop = scroller.scrollTop; - // Group by calendar day (recs come newest-first). Each day is a collapsible - // section; the day header shows the recording count. - const groups = []; - const byDay = {}; - for (const r of recs) { - const day = new Date(r.created_at).toLocaleDateString('de-DE', { weekday: 'short', day: '2-digit', month: '2-digit', year: 'numeric' }); - if (!byDay[day]) { byDay[day] = []; groups.push(day); } - byDay[day].push(r); - } - // Fresh tab entry: only the newest day (first group, recs are newest-first) - // stays open; manual toggles afterwards are respected until the next entry. - if (state.resetCollapse) { state.collapsedDates = new Set(groups.slice(1)); state.resetCollapse = false; } - // Live placeholder while a recording runs: its own card at the very top. - const liveHtml = state.recording ? ` -
- -
Aufnahme läuft… -
wird nach Stop transkribiert & analysiert
-
${markerChipsHtml()}
- ${fmt(Date.now() - state.recStart)} -
` : ''; - listEl.innerHTML = liveHtml + groups.map(day => { - const collapsed = state.collapsedDates.has(day); - const items = byDay[day].map(r => { const at = anaTimeSpan(r); return ` -
-
- ${escapeHtml(r.title) || 'Aufnahme #' + r.id} ${analysisBadge(r.analysis_status)} -
${new Date(r.created_at).toLocaleTimeString('de-DE', { hour: '2-digit', minute: '2-digit' })} · ${fmt(r.duration_ms || 0)} · ${r.language || ''}${(r.todos && r.todos.length) ? ' · ✅ ' + r.todos.length : ''}${at ? ' · ⏱ ' + at : ''}
- ${r.summary ? `
${escapeHtml(r.summary)}
` : ''} -
- -
`; }).join(''); - return ` -
-
▾ ${escapeHtml(day)} ${byDay[day].length}
-
${items}
-
`; - }).join(''); - // Click a row (not the trash button) → open in the right pane. - listEl.querySelectorAll('.item.rec').forEach(it => it.onclick = (e) => { - if (e.target.closest('[data-del]')) return; - openRecording(+it.dataset.open); - }); - listEl.querySelectorAll('[data-del]').forEach(b => b.onclick = async (e) => { - e.stopPropagation(); - const rec = recs.find(r => r.id == b.dataset.del); - const name = (rec && rec.title) ? `„${rec.title}"` : 'Aufnahme #' + b.dataset.del; - if (!confirm(`${name} wirklich löschen?\nDie Aufnahme, das Transkript und die Analyse werden unwiderruflich entfernt.`)) return; - await api.del('/api/recordings/' + b.dataset.del); - if (state.selectedId == b.dataset.del) { state.selectedId = null; const d = document.querySelector('#detail'); if (d) d.innerHTML = '
← Aufnahme wählen
'; } - state.lastListSig = null; refreshRecList(); - }); - bindMarkerChips(); - listEl.querySelectorAll('.day-head').forEach(h => h.onclick = () => { - const day = h.dataset.day; - if (state.collapsedDates.has(day)) state.collapsedDates.delete(day); else state.collapsedDates.add(day); - h.closest('.day-group').classList.toggle('collapsed'); - }); - scroller.scrollTop = savedTop; // keep the list where the user had it - ensureAnaTicker(); - scheduleNext(); - } catch (e) { listEl.innerHTML = '
' + e.message + '
'; } -} - -async function openRecording(id) { - state.selectedId = id; - const detail = document.querySelector('#detail'); - if (!detail) { state.tab = 'recordings'; render(); return setTimeout(() => openRecording(id), 100); } - // Highlight the selected row without rebuilding the list (keeps it stable). - document.querySelectorAll('#reclist .item.rec').forEach(it => it.classList.toggle('sel', it.dataset.open == id)); - detail.innerHTML = '
Lade…
'; - const r = await api.get('/api/recordings/' + id); - const pending = r.analysis_status === 'pending'; - const errored = r.analysis_status === 'error'; - const transcribing = r.status === 'transcribing'; - const transcribeFailed = r.status === 'error'; - const scls = speakerClasses(r.words || []); // rawSpeaker -> sp0..sp3 - const spkOrder = Object.keys(scls); // first-appearance order - const names = r.speaker_names || {}; // rawSpeaker -> name - const spkLabel = (sp, i) => names[sp] || ('Sprecher ' + (i + 1)); - // Speaker segments + turns for the lane visualizer and the live-text box. - const segsBySpk = speakerSegments(r.words || []); - const turns = wordTurns(r.words || []); - const durMs = r.duration_ms || (r.words && r.words.length ? r.words[r.words.length - 1].t_end_ms : 0) || 1; - - // Draggable module blocks; the saved order wins, unknown/new modules append. - // NOT modules: the transcript (own sticky right column) and the speaker-lane - // visualizer (fixed full-width strip under the player, spanning both columns). - const MOD_TITLES = { analyse: 'Zusammenfassung', todos: 'ToDos', transcript: 'Transkript' }; - const vizHtml = spkOrder.length ? ` -
-
-
- ${spkOrder.map((sp, i) => ` -
- ${escapeHtml(spkLabel(sp, i))} -
${(segsBySpk[sp] || []).map(s => - ``).join('')} -
-
`).join('')} -
-
-
- -
` : ''; - // Live-typed name markers: which cluster each one was resolved to, plus the - // option to add one while listening back (same resolution path) and to re-run - // the assignment after a re-diarization. - const CONF_LBL = { high: 'sicher', mid: 'wahrscheinlich', low: 'unsicher' }; - const markersHtml = ` -

Namensmarker (während der Aufnahme getippt)

- ${(r.markers || []).length ? `
    ${r.markers.map(m => { - const i = spkOrder.indexOf(m.speaker); - const to = m.speaker ? spkLabel(m.speaker, i >= 0 ? i : 0) : '– kein Sprecher gefunden'; - return `
  • - ${escapeHtml(m.name)} → ${escapeHtml(to)} - ${CONF_LBL[m.conf] || ''} -
    • `; - }).join('')}
` : '
Keine Marker. Tipp: während der Aufnahme Namen eintippen (Feld oben) – oder hier beim Zuhören nachtragen.
'} -
- - - ${(r.markers || []).length ? '' : ''} -
`; - const mods = { - analyse: ` - ${r.summary ? `

${escapeHtml(r.summary)}

` : '
Noch keine Zusammenfassung.
'} - ${(r.participants && r.participants.length) ? `

Teilnehmer

    ${r.participants.map(p => `
  • 👤 ${escapeHtml(p)}
    • `).join('')}
` : ''} - ${markersHtml}`, - todos: (r.todos && r.todos.length) ? `
    ${r.todos.map(t => `
  • ${escapeHtml(t)}
    • `).join('')}
` : '
Keine ToDos erkannt.
', - transcript: ` -
- ${r.transcript_clean ? `

Dialog (KI · Sprecher aus Kontext)

${renderDialog(r.transcript_clean)}
` : ''} -
Klick auf ein Wort → Audio springt dorthin. Leertaste = Play/Pause.
-
${renderTurns(r.words, names, scls, spkOrder)}
-
- `, - }; - // Two drop columns; every module (incl. the transcript/table panel) can be - // dragged within AND between them. Layout persists for all recordings. - const defLayout = { main: ['analyse', 'todos'], side: ['transcript'] }; - let layout = null; - try { layout = JSON.parse(localStorage.getItem('aime-detail-layout') || 'null'); } catch {} - if (!layout || !Array.isArray(layout.main) || !Array.isArray(layout.side)) { - // migrate the old single-column order if present - let old = []; - try { old = JSON.parse(localStorage.getItem('aime-detail-order') || '[]'); } catch {} - layout = { main: old.filter(m => m !== 'transcript'), side: [] }; - } - layout.main = layout.main.filter(m => mods[m] !== undefined); - layout.side = layout.side.filter(m => mods[m] !== undefined && !layout.main.includes(m)); - for (const m of [...defLayout.main, ...defLayout.side]) - if (!layout.main.includes(m) && !layout.side.includes(m)) - (defLayout.side.includes(m) ? layout.side : layout.main).push(m); - - const modHtml = (m) => ` -
-
⠿

${MOD_TITLES[m]}

-
${mods[m]}
-
`; - - detail.innerHTML = ` -
-

${escapeHtml(r.title) || 'Aufnahme #' + r.id} ${analysisBadge(r.analysis_status)}

- - ${vizHtml} -
- ${anaTimeSpan(r) ? `
⏱ Analysezeit: ${anaTimeSpan(r)}
` : ''} - ${transcribing ? '
📝 Transkription läuft… lange Aufnahmen dauern (A76-Takt gegen Überhitzung gedrosselt).
' : ''} - ${transcribeFailed ? `
Transkription fehlgeschlagen.
` : ''} - ${(pending && !transcribing && !transcribeFailed) ? '
🧠 KI-Analyse läuft… (NPU)
' : ''} - ${(errored && !transcribeFailed) ? `
KI-Analyse fehlgeschlagen.
` : ''} - ${(!pending && !errored && !transcribing && !transcribeFailed) ? `` : ''} - ${spkOrder.length ? `` : ''} -
-
-
${layout.main.map(modHtml).join('')}
-
${layout.side.map(modHtml).join('')}
-
-
`; - - // Drag & drop across BOTH columns: grab any module by its header, drop it in - // either column at any position; the layout is saved for all recordings. - const cols = [...detail.querySelectorAll('.mod-col')]; - // ":empty" never matches because of template whitespace -> class-driven. - const markEmpty = () => cols.forEach(c => c.classList.toggle('empty', !c.querySelector('.module'))); - markEmpty(); - let dragged = null; - detail.querySelectorAll('.mod-head').forEach(h => { - h.addEventListener('dragstart', (e) => { - dragged = h.closest('.module'); - dragged.classList.add('dragging'); - e.dataTransfer.effectAllowed = 'move'; - e.dataTransfer.setData('text/plain', dragged.dataset.mod); // Firefox needs data - }); - h.addEventListener('dragend', () => { - if (!dragged) return; - dragged.classList.remove('dragging'); - dragged = null; - const save = {}; - for (const c of cols) save[c.dataset.col] = [...c.querySelectorAll('.module')].map(m => m.dataset.mod); - localStorage.setItem('aime-detail-layout', JSON.stringify(save)); - markEmpty(); - }); - }); - cols.forEach(col => col.addEventListener('dragover', (e) => { - if (!dragged) return; - e.preventDefault(); - const others = [...col.querySelectorAll('.module:not(.dragging)')]; - const next = others.find(m => e.clientY < m.getBoundingClientRect().top + m.offsetHeight / 2); - if (next) col.insertBefore(dragged, next); else col.appendChild(dragged); - markEmpty(); - })); - const reBtn = detail.querySelector('#reanalyze'); - if (reBtn) reBtn.onclick = async () => { - // Re-runs speaker separation every time: empty = automatic detection (now - // consolidated, so no more "14 speakers" for a 2-person call), a number pins - // the count. Cancel aborts. - const ans = window.prompt('Sprecher neu erkennen?\n(leer = automatisch erkennen · Zahl = feste Anzahl vorgeben)', ''); - if (ans === null) return; - const numSpeakers = parseInt(ans, 10); - const body = numSpeakers > 0 ? { numSpeakers } : {}; - reBtn.disabled = true; reBtn.textContent = 'trennt neu…'; - try { await api.post('/api/recordings/' + id + '/reanalyze', body); setTimeout(() => openRecording(id), 1500); } - catch (e) { toast(e.message, true); } - }; - const rtBtn = detail.querySelector('#retranscribe'); - if (rtBtn) rtBtn.onclick = async () => { - rtBtn.disabled = true; rtBtn.textContent = 'startet…'; - try { await api.post('/api/recordings/' + id + '/retranscribe', {}); setTimeout(() => openRecording(id), 1500); } - catch (e) { toast(e.message, true); rtBtn.disabled = false; } - }; - ensureAnaTicker(); - // Poll while transcription OR the NPU analysis is still running. The old - // version re-ran openRecording(id) blindly every 5s — a stale closure: after - // switching to ANOTHER recording the timer still fired and yanked the user - // (and the playing audio) back to the analysing one. Now: stop when the - // selection changed, poll ONLY the status while busy (no DOM rebuild), and - // rebuild once when the results are in — restoring the player position. - if (pending || transcribing) { - const pollDetail = async () => { - if (state.selectedId !== id || state.tab !== 'recordings') return; // user moved on - if (!document.querySelector('#detail')) return; // view gone - let cur; - try { cur = await api.get('/api/recordings/' + id); } catch { return; } - if (state.selectedId !== id || state.tab !== 'recordings') return; // re-check after await - const busy = cur.analysis_status === 'pending' || cur.status === 'transcribing'; - if (busy) { setTimeout(pollDetail, 5000); return; } - // Results arrived → one rebuild; keep the audio where the user had it. - const a = document.querySelector('#detail #player'); - const keep = a && !a.paused ? { t: a.currentTime, play: true } : (a ? { t: a.currentTime, play: false } : null); - await openRecording(id); - if (keep && keep.t > 0) { - const na = document.querySelector('#detail #player'); - if (na) { na.currentTime = keep.t; if (keep.play) na.play().catch(() => {}); } - } - }; - setTimeout(pollDetail, 5000); - } - const audio = detail.querySelector('#player'); - // ---- marker actions (seek / add at playhead / delete / re-assign) ---- - detail.querySelectorAll('.mkseek').forEach(b => b.onclick = () => { - audio.currentTime = (+b.dataset.t) / 1000 + 0.001; audio.play(); - }); - const mkAt = detail.querySelector('#mkat'), mkAtBtn = detail.querySelector('#mkatbtn'); - const addAt = async () => { - const name = (mkAt.value || '').trim(); - if (!name) return; - try { - await api.post(`/api/recordings/${id}/markers`, { name, t_ms: Math.round(audio.currentTime * 1000) }); - mkAt.value = ''; - await api.post(`/api/recordings/${id}/apply-markers`, {}); - toast(`🏷 ${name} gesetzt & zugeordnet`); - openRecording(id); - } catch (e) { toast(e.message, true); } - }; - if (mkAtBtn) mkAtBtn.onclick = addAt; - if (mkAt) mkAt.onkeydown = (e) => { if (e.key === 'Enter') addAt(); }; - detail.querySelectorAll('[data-mkdel]').forEach(b => b.onclick = async () => { - try { await api.del(`/api/recordings/${id}/markers/${b.dataset.mkdel}`); openRecording(id); } - catch (e) { toast(e.message, true); } - }); - const amk = detail.querySelector('#applymk'); - if (amk) amk.onclick = async () => { - amk.disabled = true; amk.textContent = 'ordne zu…'; - try { - const r2 = await api.post(`/api/recordings/${id}/apply-markers`, {}); - toast(`${r2.named} Sprecher benannt, ${r2.merged} zusammengeführt, ${r2.learned} Stimme(n) gelernt`); - openRecording(id); - } catch (e) { toast(e.message, true); amk.disabled = false; } - }; - const spans = [...detail.querySelectorAll('.w')]; - spans.forEach(s => s.onclick = () => { audio.currentTime = (+s.dataset.start) / 1000 + 0.001; audio.play(); }); - // Whisper (batched) word timestamps run slightly AHEAD of the audio, so a - // pure [start,end] match lit the NEXT word ("eine Zeile weiter") before it was - // spoken. Compensate with a fixed lead offset and highlight the LAST word whose - // start has passed (stays lit through inter-word gaps → no flicker). - // Tune SYNC_MS if the marker still leads (raise) or lags (lower). - const SYNC_MS = 250; - - // ---- View switch: transcript <-> conference-table scene ---- - const switchBtn = detail.querySelector('#viewswitch'); - const tView = detail.querySelector('#table-view'); - const trView = detail.querySelector('#transcript-view'); - const sideTitle = detail.querySelector('#sidetitle'); - const bubble = detail.querySelector('#tbubble'); - const tIdle = detail.querySelector('#tidle'); - const persons = [...detail.querySelectorAll('.tperson')]; - let bubbleTurnIdx = -1; - const applyViewMode = () => { - const table = localStorage.getItem('aime-side-view') === 'table'; - if (tView) tView.classList.toggle('hidden', !table); - if (trView) trView.classList.toggle('hidden', table); - if (sideTitle) sideTitle.textContent = table ? 'Am Tisch' : 'Transkript'; - if (switchBtn) switchBtn.innerHTML = table ? ICON_SHEET : ICON_TABLE; - }; - if (switchBtn) { - switchBtn.onclick = () => { - const now = localStorage.getItem('aime-side-view') === 'table' ? 'text' : 'table'; - localStorage.setItem('aime-side-view', now); - applyViewMode(); - }; - applyViewMode(); - } - // All eyes on the speaker: shift every person's pupils toward the active seat. - const lookAt = (activeSp) => { - const target = persons.find(p => p.dataset.sp === activeSp); - for (const p of persons) { - const eyes = p.querySelector('.teyes'); - if (!eyes) continue; - if (!target || p === target) { eyes.style.transform = 'translate(0, 1px)'; continue; } - const dx = (+target.dataset.x) - (+p.dataset.x); - const dy = (+target.dataset.y) - (+p.dataset.y); - const len = Math.hypot(dx, dy) || 1; - eyes.style.transform = `translate(${(dx / len * 3.2).toFixed(1)}px, ${(dy / len * 2.4).toFixed(1)}px)`; - } - }; - const updateTable = (t) => { - if (!tView || tView.classList.contains('hidden')) return; - let idx = -1; - for (let i = 0; i < turns.length; i++) { if (turns[i].start <= t) idx = i; else break; } - if (idx < 0) return; - const tu = turns[idx]; - if (idx !== bubbleTurnIdx) { - bubbleTurnIdx = idx; - const i = spkOrder.indexOf(tu.sp); - const seats = tableSeats(Math.min(persons.length, spkOrder.length)); - const seat = seats[i >= 0 ? Math.min(i, persons.length - 1) : 0]; - if (bubble && seat) { - // The bubble sits FIXED on the table; only its tail (the "Öffnung") - // wanders to point at whoever is speaking. - bubble.className = `tbubble ${scls[tu.sp] || ''} arrow-${seat.side}`; - bubble.style.left = ''; - bubble.style.top = ''; - // Tail x-position along the top/bottom edge = the speaker's seat mapped - // into bubble-local coordinates (bubble is centred, BUBBLE_W wide). - const ax = Math.max(8, Math.min(92, (seat.x - (50 - BUBBLE_W / 2)) / BUBBLE_W * 100)); - bubble.style.setProperty('--ax', ax.toFixed(1) + '%'); - bubble.querySelector('.tb-name').textContent = spkLabel(tu.sp, i >= 0 ? i : 0); - // The full turn as clickable word spans — highlighted + scrolled below. - bubble.querySelector('.tb-text').innerHTML = - tu.words.map(w => `${escapeHtml(w.w)}`).join(' '); - } - if (tIdle) tIdle.classList.add('hidden'); - lookAt(tu.sp); - } - persons.forEach(p => p.classList.toggle('talking', p.dataset.sp === tu.sp && !audio.paused)); - // Word highlight + follow-scroll inside the bubble (like the transcript). - if (bubble && !bubble.classList.contains('hidden')) { - const scroller = bubble.querySelector('.tb-scroll'); - const bws = bubble.querySelectorAll('.bw'); - let act = null; - for (const s of bws) { if (+s.dataset.s <= t) act = s; else break; } - bws.forEach(s => s.classList.toggle('active', s === act)); - if (act && scroller) { - const target = act.offsetTop - scroller.clientHeight / 2 + 10; - if (Math.abs(scroller.scrollTop - target) > 24) scroller.scrollTo({ top: Math.max(0, target), behavior: 'smooth' }); - } - } - }; - if (bubble) bubble.addEventListener('click', (e) => { - const s = e.target.closest('.bw'); - if (s) { audio.currentTime = (+s.dataset.s) / 1000 + 0.001; audio.play().catch(() => {}); } - }); - - // Lane visualizer wiring: playhead + speaking lane + live-text box. - const lanesEl = detail.querySelector('#lanes'); - const playheadEl = detail.querySelector('#playhead'); - const liveboxEl = detail.querySelector('#livebox'); - let liveTurnIdx = -1; - if (lanesEl) { - // Click inside a LANE seeks; clicks on the label column open the editor. - const seek = (e) => { - if (!e.target.closest('.lane')) return; - const lane = lanesEl.querySelector('.lane'); - if (!lane) return; - const rc = lane.getBoundingClientRect(); - const frac = Math.min(1, Math.max(0, (e.clientX - rc.left) / rc.width)); - audio.currentTime = frac * durMs / 1000; - audio.play().catch(() => {}); - }; - lanesEl.addEventListener('click', seek); - } - // Speaker editor popover: click a lane label -> rename + optionally learn the - // voice (replaces the old separate "Sprecher" module). - const pop = detail.querySelector('#spkpop'); - if (pop && lanesEl) { - let popSp = null; - const popName = pop.querySelector('#spkpop-name'); - lanesEl.querySelectorAll('.lane-label').forEach(l => l.onclick = (e) => { - e.stopPropagation(); - const row = l.closest('.lane-row'); - popSp = row.dataset.sp; - pop.style.top = (row.offsetTop + 30) + 'px'; - pop.classList.remove('hidden'); - const i = spkOrder.indexOf(popSp); - popName.value = names[popSp] || ''; - popName.placeholder = spkLabel(popSp, i >= 0 ? i : 0); - popName.focus(); - }); - pop.querySelector('#spkpop-cancel').onclick = () => pop.classList.add('hidden'); - pop.querySelector('#spkpop-save').onclick = async () => { - const name = popName.value.trim(); - const enroll = pop.querySelector('#spkpop-learn').checked; - const btn = pop.querySelector('#spkpop-save'); - btn.disabled = true; btn.textContent = '…'; - try { - const res = await api.post('/api/recordings/' + id + '/speaker', { speaker: popSp, name, enroll }); - const base = res.enrolled ? `„${name}" gespeichert + Stimme gelernt` : (res.learnMsg ? 'Gespeichert (' + res.learnMsg + ')' : 'Gespeichert'); - toast(res.relabeled ? base + ' · Name in Zusammenfassung übernommen' : base); - openRecording(id); - } catch (e) { toast(e.message, true); btn.disabled = false; btn.textContent = 'Speichern'; } - }; - } - const updateViz = (t) => { - if (!lanesEl) return; - if (playheadEl) { - const lane = lanesEl.querySelector('.lane'); - if (lane) { - const left = lane.offsetLeft + Math.min(1, Math.max(0, t / durMs)) * lane.offsetWidth; - playheadEl.style.left = left + 'px'; - playheadEl.style.display = 'block'; - } - } - // Which speakers talk right now → light their lane rows. - lanesEl.querySelectorAll('.lane-row').forEach(row => { - const sp = row.dataset.sp; - const on = (segsBySpk[sp] || []).some(s => t >= s.start && t <= s.end); - row.classList.toggle('speaking', on && !audio.paused); - }); - // Live text: last turn whose start has passed (stays through gaps). - if (liveboxEl) { - let idx = -1; - for (let i = 0; i < turns.length; i++) { if (turns[i].start <= t) idx = i; else break; } - if (idx !== liveTurnIdx && idx >= 0) { - liveTurnIdx = idx; - const tu = turns[idx]; - const i = spkOrder.indexOf(tu.sp); - liveboxEl.innerHTML = ` -
- ${escapeHtml(spkLabel(tu.sp, i >= 0 ? i : 0))} - ${escapeHtml(tu.text)} -
`; - } - } - }; - audio.onpause = () => { - if (lanesEl) lanesEl.querySelectorAll('.lane-row.speaking').forEach(r2 => r2.classList.remove('speaking')); - persons.forEach(p => p.classList.remove('talking')); - }; - audio.ontimeupdate = () => { - const t = audio.currentTime * 1000 - SYNC_MS; - let active = null; - for (const s of spans) { if (+s.dataset.start <= t) active = s; else break; } - spans.forEach(s => s.classList.toggle('active', s === active)); - // Follow the active word INSIDE the transcript panel (it scrolls itself now — - // scrollIntoView would also yank the page). - if (active && (!trView || !trView.classList.contains('hidden'))) { - const panel = detail.querySelector('#transcript-panel'); - if (panel) { - const pr = panel.getBoundingClientRect(), ar = active.getBoundingClientRect(); - if (ar.top < pr.top + 40 || ar.bottom > pr.bottom - 40) - panel.scrollTo({ top: panel.scrollTop + (ar.top - pr.top) - panel.clientHeight / 2, behavior: 'smooth' }); - } - } - updateViz(t); - updateTable(t); - }; - // (Speaker renaming lives in the lane-label popover above.) - - // Spacebar toggles play/pause of the open recording (unless typing in a field). - if (state.spaceHandler) document.removeEventListener('keydown', state.spaceHandler); - state.spaceHandler = (e) => { - if (e.code !== 'Space' && e.key !== ' ') return; - const t = e.target, tag = (t.tagName || '').toLowerCase(); - if (tag === 'input' || tag === 'textarea' || t.isContentEditable) return; - const a = document.querySelector('#player'); - if (!a) return; - e.preventDefault(); - if (a.paused) a.play(); else a.pause(); - }; - document.addEventListener('keydown', state.spaceHandler); - - // Two-pane layout: scroll the detail pane to its top. On a narrow (stacked) - // screen also bring the detail into view since it sits below the list. - const pane = detail.closest('.rec-detail-pane'); - if (pane) pane.scrollTo({ top: 0, behavior: 'smooth' }); - if (window.matchMedia('(max-width: 800px)').matches) detail.scrollIntoView({ behavior: 'smooth' }); -} - -function escapeHtml(s) { return (s || '').replace(/[&<>"]/g, c => ({ '&': '&', '<': '<', '>': '>', '"': '"' }[c])); } - -// ---- Konferenztisch-Ansicht ---- -// Seat positions (person center, % of scene) per participant count, arranged -// around the rectangular table; each seat knows which bubble-arrow side points -// at it from the table-centre bubble. -const SEAT_LAYOUTS = { - 1: [{ x: 50, y: 84, side: 'bottom' }], - 2: [{ x: 50, y: 14, side: 'top' }, { x: 50, y: 84, side: 'bottom' }], - 3: [{ x: 50, y: 13, side: 'top' }, { x: 26, y: 84, side: 'bottom' }, { x: 74, y: 84, side: 'bottom' }], - 4: [{ x: 50, y: 12, side: 'top' }, { x: 90, y: 50, side: 'right' }, { x: 50, y: 86, side: 'bottom' }, { x: 10, y: 50, side: 'left' }], - 5: [{ x: 32, y: 12, side: 'top' }, { x: 68, y: 12, side: 'top' }, { x: 90, y: 60, side: 'right' }, { x: 50, y: 86, side: 'bottom' }, { x: 10, y: 60, side: 'left' }], - 6: [{ x: 32, y: 12, side: 'top' }, { x: 68, y: 12, side: 'top' }, { x: 90, y: 50, side: 'right' }, { x: 68, y: 86, side: 'bottom' }, { x: 32, y: 86, side: 'bottom' }, { x: 10, y: 50, side: 'left' }], -}; -// Bubble width in scene-% (kept in sync with .tbubble in style.css) — needed to -// map a seat's x onto the bubble edge so the tail points AT the speaker. -const BUBBLE_W = 54; -function tableSeats(n) { return SEAT_LAYOUTS[Math.min(6, Math.max(1, n))] || SEAT_LAYOUTS[6]; } - -// Toggle-button icons: three people around a table / a text sheet. -const ICON_TABLE = ``; -const ICON_SHEET = ``; - -// The scene: rounded table, one person per speaker (their lane colour), a -// speech bubble ON the table whose arrow + border follow whoever talks. -function buildTableScene(spkOrder, scls, names, spkLabel) { - if (!spkOrder.length) return '
Keine Sprecherdaten.
'; - const seats = tableSeats(spkOrder.length); - return ` -
-
- ${spkOrder.slice(0, seats.length).map((sp, i) => { - const s = seats[i]; - return ` -
-
-
-
${escapeHtml(spkLabel(sp, i))}
-
`; - }).join('')} - -
▶ Abspielen — die Runde beginnt zu sprechen.
-
`; -} - -// Per-speaker talk segments for the lane visualizer: consecutive words of the -// same speaker, gaps ≤ 1.2 s bridged (word-level gaps would render as dust). -function speakerSegments(words) { - const by = {}; - for (const w of words) { - if (!w.speaker) continue; - const list = by[w.speaker] || (by[w.speaker] = []); - const last = list[list.length - 1]; - if (last && w.t_start_ms - last.end <= 1200) last.end = Math.max(last.end, w.t_end_ms); - else list.push({ start: w.t_start_ms, end: w.t_end_ms }); - } - return by; -} -// Turn list for the table scene: speaker, joined text AND the single words -// (with start times) so the bubble can highlight word-by-word like the transcript. -function wordTurns(words) { - const turns = []; - for (const w of words) { - const last = turns[turns.length - 1]; - if (last && last.sp === w.speaker) { last.text += ' ' + w.word; last.end = w.t_end_ms; last.words.push({ w: w.word, s: w.t_start_ms }); } - else turns.push({ sp: w.speaker, text: w.word, start: w.t_start_ms, end: w.t_end_ms, words: [{ w: w.word, s: w.t_start_ms }] }); - } - return turns; -} -// Map distinct raw speaker labels (speaker_00…) to colour classes sp0..sp3. -function speakerClasses(words) { - const m = {}; let i = 0; const palette = ['sp0', 'sp1', 'sp2', 'sp3']; - for (const w of words) { if (w.speaker && !(w.speaker in m)) { m[w.speaker] = palette[i % palette.length]; i++; } } - return m; -} -// Render the raw transcript as speaker turns with a NAME header per turn. Each -// word stays its own .w span (click-to-seek + active highlight still work). -function renderTurns(words, names, scls, order) { - if (!words || !words.length) return 'Kein Transkript.'; - const idx = {}; order.forEach((s, i) => idx[s] = i); - const label = sp => (names && names[sp]) || (sp ? 'Sprecher ' + ((idx[sp] ?? 0) + 1) : ''); - let html = '', cur = '\0', buf = []; - const flush = () => { - if (!buf.length) return; - const nm = label(cur === '\0' ? null : cur); - html += `
${nm ? `${escapeHtml(nm)} ` : ''}${buf.join(' ')}
`; - buf = []; - }; - for (const w of words) { - const sp = w.speaker || null; - if (sp !== cur) { flush(); cur = sp; } - buf.push(`${escapeHtml(w.word)}`); - } - flush(); - return html; -} -// Render "Name: Text" lines as a colour-coded dialogue (speakers inferred by -// the LLM from context — not acoustic). -function renderDialog(text) { - const lines = (text || '').split(/\n+/).map(l => l.trim()).filter(Boolean); - const colors = {}; let ci = 0; const palette = ['sp0', 'sp1', 'sp2', 'sp3']; - return lines.map(l => { - const m = l.match(/^([^:]{1,30}):\s*(.*)$/); - if (!m) return `
${escapeHtml(l)}
`; - const spk = m[1].trim(); - if (!(spk in colors)) { colors[spk] = palette[ci % palette.length]; ci++; } - return `
${escapeHtml(spk)} ${escapeHtml(m[2])}
`; - }).join(''); -} -function analysisBadge(st) { - if (st === 'pending') return '🧠 Analyse…'; - if (st === 'error') return 'KI-Fehler'; - if (st === 'done') return 'KI ✓'; - return ''; -} - -render(); +import './style.css'; + +const api = { + async get(u) { const r = await fetch(u); if (!r.ok) throw new Error((await r.json()).error || r.status); return r.json(); }, + async post(u, b) { const r = await fetch(u, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify(b || {}) }); if (!r.ok) throw new Error((await r.json()).error || r.status); return r.json(); }, + async del(u) { const r = await fetch(u, { method: 'DELETE' }); return r.json(); }, + async put(u, b) { const r = await fetch(u, { method: 'PUT', headers: { 'content-type': 'application/json' }, body: JSON.stringify(b || {}) }); if (!r.ok) throw new Error((await r.json()).error || r.status); return r.json(); }, + async patch(u, b) { const r = await fetch(u, { method: 'PATCH', headers: { 'content-type': 'application/json' }, body: JSON.stringify(b || {}) }); if (!r.ok) throw new Error((await r.json()).error || r.status); return r.json(); } +}; + +const app = document.querySelector('#app'); +const state = { tab: 'recordings', recording: false, recStart: 0, timer: null, selectedId: null, collapsedDates: new Set(), lastListSig: null, markers: [], mkDraft: '', + // search: `q` is what was actually submitted (not what is being typed), so a + // tab switch and back restores the result page instead of an empty form. + searchQ: '', searchRes: null, searchAnswer: null, searchBusy: false, trashOpen: false }; + +function fmt(ms) { const s = Math.floor(ms / 1000); return `${String(Math.floor(s / 60)).padStart(2, '0')}:${String(s % 60).padStart(2, '0')}`; } +function toast(msg, err) { const t = document.createElement('div'); t.className = 'toast' + (err ? ' err' : ''); t.textContent = msg; document.body.appendChild(t); setTimeout(() => t.remove(), 4000); } + +// Theme: data-theme on , chosen in index.html before first paint (no +// flash); the toggle persists to localStorage. +function applyTheme(t) { + document.documentElement.dataset.theme = t; + localStorage.setItem('aime-theme', t); + const b = document.querySelector('#themetoggle'); + if (b) b.textContent = t === 'dark' ? '☀️' : '🌙'; +} + +function render() { + const theme = document.documentElement.dataset.theme || 'dark'; + app.innerHTML = ` +
+
+ +

Aime Anruf-Assistent

+
+
+ + ${state.recording ? fmt(Date.now() - state.recStart) : '00:00'} + ${state.recording ? ` + + 🏷 ${state.markers.length}` : ''} +
+ + +
+
`; + app.querySelectorAll('.tab').forEach(b => b.onclick = () => { state.tab = b.dataset.tab; render(); }); + app.querySelector('#themetoggle').onclick = () => + applyTheme(document.documentElement.dataset.theme === 'dark' ? 'light' : 'dark'); + app.querySelector('#recbtn').onclick = toggleRecord; + bindMarkerInput(); + if (state.recording) startTimerUI(); + ({ bluetooth: viewBluetooth, recordings: viewRecordings, search: viewSearch, voices: viewVoices, glossary: viewGlossary })[state.tab](); +} + +// Header record button: start = defaults only (default source = CallRecMix, +// codec auto, speaker count auto — the old options tab is gone on purpose). +// A live placeholder card appears at the top of the recordings list instantly. +async function toggleRecord() { + const btn = document.querySelector('#recbtn'); + if (!state.recording) { + try { + await api.post('/api/record/start', {}); + state.recording = true; state.recStart = Date.now(); + state.markers = []; state.mkDraft = ''; + render(); + if (state.tab === 'recordings') { state.lastListSig = null; refreshRecList(); } + } catch (err) { toast(err.message, true); } + } else { + if (btn) { btn.disabled = true; } + clearInterval(state.timer); state.timer = null; + try { + const r = await api.post('/api/record/stop', {}); + state.recording = false; + toast('Aufnahme gespeichert – wird transkribiert…'); + state.tab = 'recordings'; render(); + state.lastListSig = null; refreshRecList(); + openRecording(r.id); + } catch (err) { state.recording = false; toast('Stop fehlgeschlagen: ' + err.message, true); render(); } + } +} +// ---------- live speaker markers ---------- +// While recording: type the name of whoever is talking, Enter. The server stamps +// the offset; after diarization the name lands on that acoustic speaker. Only +// reliable way to tell 20+ conference participants apart. +function bindMarkerInput() { + const inp = document.querySelector('#mkname'); + if (!inp) return; + inp.oninput = () => { state.mkDraft = inp.value; }; + inp.onkeydown = async (e) => { + if (e.key !== 'Enter') return; + const name = inp.value.trim(); + if (!name) return; + inp.value = ''; state.mkDraft = ''; + try { + const r = await api.post('/api/record/marker', { name }); + state.markers = r.markers || []; + updateMarkerUI(); + toast(`🏷 ${fmt(r.t_ms)} ${name}`); + } catch (err) { toast('Marker fehlgeschlagen: ' + err.message, true); } + }; +} +// Refresh the marker count + chip list in place (no re-render → focus stays in +// the input, so names can be typed back-to-back). +function updateMarkerUI() { + const c = document.querySelector('#mkcount'); + if (c) { c.textContent = '🏷 ' + state.markers.length; c.classList.toggle('ok', !!state.markers.length); } + const chips = document.querySelector('#mkchips'); + if (chips) chips.innerHTML = markerChipsHtml(); + bindMarkerChips(); +} +function markerChipsHtml() { + if (!state.markers.length) return 'noch keine Namen markiert'; + return state.markers.map((m, i) => + `${fmt(m.t_ms)} ${escapeHtml(m.name)}`).join(''); +} +function bindMarkerChips() { + document.querySelectorAll('.mkdel').forEach(b => b.onclick = async (e) => { + e.stopPropagation(); + try { + const r = await api.del('/api/record/marker/' + b.dataset.mk); + state.markers = r.markers || []; + updateMarkerUI(); + } catch (err) { toast(err.message, true); } + }); +} + +// Tick every .rectime element (header + list placeholder) from one interval. +function startTimerUI() { + clearInterval(state.timer); + state.timer = setInterval(() => { + const t = fmt(Date.now() - state.recStart); + document.querySelectorAll('[data-rectime]').forEach(el => el.textContent = t); + }, 500); +} + +// ---------- Bluetooth ---------- +async function viewBluetooth() { + const v = document.querySelector('#view'); + v.innerHTML = ` +
+
+ + + +
+
Lade…
+
`; + const listEl = v.querySelector('#btlist'); + const msg = v.querySelector('#btmsg'); + + async function load(devices) { + const d = devices || await api.get('/api/bt/devices'); + if (!d.length) { listEl.innerHTML = '
Keine Geräte. „Suchen“ drücken (Headset in Pairing-Modus).
'; return; } + listEl.innerHTML = d.map(dev => ` +
+
+ ${dev.name || '(unbenannt)'} ${dev.audio ? '🎧' : ''} +
${dev.mac}
+
+ ${dev.connected ? 'verbunden' : dev.paired ? 'gepaart' : 'neu'} + ${dev.connected + ? `` + : dev.paired + ? `` + : ``} + ${dev.audio ? `` : ''} +
`).join(''); + listEl.querySelectorAll('button[data-act]').forEach(b => b.onclick = () => act(b.dataset.act, b.dataset.mac, b)); + } + async function act(a, mac, btn) { + btn.disabled = true; btn.textContent = a === 'repair' ? 'neu koppeln… (~15s)' : '…'; + try { + const r = await api.post('/api/bt/' + a, { mac }); + if (a === 'repair') toast(r.ok ? 'neu verbunden ✓' : 'fehlgeschlagen: ' + (r.msg || r.connect || 'siehe Headset im Pairing-Modus?'), !r.ok); + else toast(`${a}: ${r.connected ? 'verbunden' : 'ok'}`); + if ((a === 'pair' || a === 'connect' || a === 'repair') && (r.connected || r.ok)) await api.post('/api/audio/headset-mic', {}); + } + catch (e) { toast(a + ' fehlgeschlagen: ' + e.message, true); } + await load(); + } + v.querySelector('#scan').onclick = async (e) => { e.target.disabled = true; msg.textContent = 'suche…'; try { await load(await api.post('/api/bt/scan', { seconds: 10 })); } catch (err) { toast(err.message, true); } msg.textContent = ''; e.target.disabled = false; }; + v.querySelector('#refresh').onclick = () => load(); + load().catch(e => listEl.innerHTML = '
' + e.message + '
'); +} + +// ---------- Voices (enrollment) ---------- +async function viewVoices() { + const v = document.querySelector('#view'); + v.innerHTML = ` +
+

Stimme anlernen

+
Person spricht ~15-20s frei (Headset). Danach wird ihr Stimmprofil gespeichert und in Aufnahmen automatisch erkannt.
+ + +
+
00:00
+ +
+
+
+

Angelernte Stimmen

Lade…
`; + const listEl = v.querySelector('#vplist'); + async function load() { + try { + const vps = await api.get('/api/voiceprints'); + listEl.innerHTML = vps.length ? vps.map(p => ` +
👤 ${escapeHtml(p.name)} +
${new Date(p.created_at).toLocaleString('de-DE')} · ${p.dim}-dim
+
`).join('') + : '
Noch keine Stimmen. Oben anlernen.
'; + listEl.querySelectorAll('[data-del]').forEach(b => b.onclick = async () => { await api.del('/api/voiceprints/' + b.dataset.del); load(); }); + } catch (e) { listEl.innerHTML = '
' + e.message + '
'; } + } + await load(); + const btn = v.querySelector('#vbtn'), msg = v.querySelector('#vmsg'), timerEl = v.querySelector('#vtimer'); + if (state.enrolling) startTimerUI(timerEl); + btn.onclick = async () => { + if (!state.enrolling) { + const name = v.querySelector('#vname').value.trim(); + if (!name) { toast('Bitte Namen eingeben', true); return; } + state.enrollName = name; + try { await api.post('/api/enroll/start', {}); state.enrolling = true; state.recStart = Date.now(); render(); } + catch (e) { toast(e.message, true); } + } else { + btn.disabled = true; btn.textContent = 'Speichere…'; clearInterval(state.timer); + try { + const r = await api.post('/api/enroll/stop', { name: state.enrollName }); + state.enrolling = false; state.enrollName = ''; toast(`Stimme „${r.name}" gespeichert ✓`); render(); + } catch (e) { state.enrolling = false; toast('Fehlgeschlagen: ' + e.message, true); render(); } + } + }; +} + +// ---------- Glossary ---------- +async function viewGlossary() { + const v = document.querySelector('#view'); + v.innerHTML = ` +
+

Begriff hinzufügen

+
Eigennamen & Fachbegriffe, die die Spracherkennung verhört (z.B. „Aime" wird als „Amy" gehört). Die KI-Korrektur ersetzt lautähnliche Wörter dann durch die richtige Schreibweise.
+ + + + + + +
+

Glossar

Lade…
`; + const listEl = v.querySelector('#glist'); + let editId = null; // id of the entry currently loaded into the form + const formTitle = v.querySelector('#gformtitle'), saveBtn = v.querySelector('#gadd'), cancelBtn = v.querySelector('#gcancel'); + function setEdit(g) { + editId = g ? g.id : null; + formTitle.textContent = g ? `„${g.term}" bearbeiten` : 'Begriff hinzufügen'; + saveBtn.textContent = g ? 'Speichern' : 'Hinzufügen'; + cancelBtn.style.display = g ? '' : 'none'; + v.querySelector('#gterm').value = g ? g.term : ''; + v.querySelector('#gsound').value = g ? (g.soundalikes || '') : ''; + if (g) v.querySelector('#gterm').focus(); + } + async function load() { + try { + const gs = await api.get('/api/glossary'); + listEl.innerHTML = gs.length ? gs.map(g => ` +
📖 ${escapeHtml(g.term)} + ${g.soundalikes ? `
↔ ${escapeHtml(g.soundalikes)}
` : ''}
+ +
`).join('') + : '
Noch keine Begriffe. Oben hinzufügen.
'; + listEl.querySelectorAll('[data-del]').forEach(b => b.onclick = async () => { await api.del('/api/glossary/' + b.dataset.del); if (editId === +b.dataset.del) setEdit(null); load(); }); + listEl.querySelectorAll('[data-edit]').forEach(b => b.onclick = () => setEdit(gs.find(g => g.id === +b.dataset.edit))); + } catch (e) { listEl.innerHTML = '
' + e.message + '
'; } + } + await load(); + cancelBtn.onclick = () => setEdit(null); + saveBtn.onclick = async () => { + const term = v.querySelector('#gterm').value.trim(); + if (!term) { toast('Bitte Begriff eingeben', true); return; } + const soundalikes = v.querySelector('#gsound').value.trim(); + try { + if (editId != null) await api.put('/api/glossary/' + editId, { term, soundalikes }); + else await api.post('/api/glossary', { term, soundalikes }); + toast(`„${term}" gespeichert ✓`); + setEdit(null); + load(); + } catch (e) { toast(e.message, true); } + }; +} + +// ---------- Search ---------- +// Two-stage on purpose (see search.js): the hit list is SQL and lands in +// milliseconds, the written answer is a ~30-90 s LLM call that streams in +// afterwards. Never make the user wait on the slow half to see the fast half. +const SEARCH_EXAMPLES = [ + 'Welches Projekt hat Johannes Jeckel letzte Woche vorgestellt?', + 'Was wurde über die Preiserhöhung besprochen?', + 'Welche Aufgaben hat Markus im Juli übernommen?', +]; + +function viewSearch() { + const v = document.querySelector('#view'); + const hasResults = !!state.searchRes; + v.innerHTML = ` +
+ + ${hasResults ? '' : ` +
+
Durchsucht alle Mitschriften. Beispiele:
+ ${SEARCH_EXAMPLES.map(e => ``).join('')} +
`} +
+
`; + + const input = v.querySelector('#searchq'); + v.querySelector('#searchform').onsubmit = (e) => { e.preventDefault(); runSearch(input.value.trim()); }; + const clr = v.querySelector('#searchclear'); + if (clr) clr.onclick = () => { state.searchQ = ''; state.searchRes = null; state.searchAnswer = null; viewSearch(); }; + v.querySelectorAll('.example').forEach(b => b.onclick = () => { input.value = b.dataset.ex; runSearch(b.dataset.ex); }); + if (!hasResults) input.focus(); + if (hasResults) renderSearchResults(); +} + +async function runSearch(q) { + if (!q || q.length < 2) return; + state.searchQ = q; + state.searchAnswer = null; + state.searchBusy = true; + const out = document.querySelector('#searchout'); + if (out) out.innerHTML = '
Suche…
'; + document.querySelector('.search-page').classList.add('has-results'); + try { + state.searchRes = await api.get('/api/search?q=' + encodeURIComponent(q)); + } catch (e) { + state.searchRes = null; + if (out) out.innerHTML = '
' + escapeHtml(e.message) + '
'; + state.searchBusy = false; + return; + } + state.searchBusy = false; + renderSearchResults(); + // Only bother the NPU when there is something to summarise. + if (state.searchRes.hits.length) askSearchAnswer(q); +} + +// The written answer. Fired after the hits are already on screen; the box shows +// a live elapsed counter because on this hardware it genuinely takes a while. +async function askSearchAnswer(q) { + const box = document.querySelector('#answerbox'); + if (!box) return; + const t0 = Date.now(); + const tick = setInterval(() => { + const el = box.querySelector('.answer-elapsed'); + if (el) el.textContent = Math.round((Date.now() - t0) / 1000) + ' s'; + }, 1000); + try { + const a = await api.post('/api/search/answer', { q }); + // A late answer to a query the user has already replaced must not overwrite + // the newer result. + if (state.searchQ !== q) return; + state.searchAnswer = a; + renderAnswer(); + } catch (e) { + if (state.searchQ !== q) return; + box.innerHTML = `
Antwort
${escapeHtml(e.message)}
`; + } finally { clearInterval(tick); } +} + +// Highlight the query's content words inside a snippet, the way a search engine +// bolds the matched terms. Escaped first, so the highlight cannot inject markup. +function highlight(text, terms) { + let html = escapeHtml(text); + for (const t of (terms || [])) { + if (t.length < 3) continue; + const safe = t.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); + html = html.replace(new RegExp(`(${safe})`, 'gi'), '$1'); + } + return html; +} + +function renderSearchResults() { + const out = document.querySelector('#searchout'); + if (!out || !state.searchRes) return; + const { hits, terms, range, semantic, total } = state.searchRes; + if (!hits.length) { + out.innerHTML = `
+ Keine Treffer für „${escapeHtml(state.searchQ)}". +
${range ? `Eingeschränkt auf ${escapeHtml(range.label)} — vielleicht liegt es daran. ` : ''} + Andere Formulierung oder weniger Wörter versuchen.
`; + return; + } + out.innerHTML = ` +
+
Antwort …
+
+
+
+ ${total} Fundstelle${total === 1 ? '' : 'n'}${range ? ` · Zeitraum: ${escapeHtml(range.label)}` : ''} + · ${semantic ? 'Volltext + semantisch' : 'nur Volltext'} +
+
    + ${hits.map(h => ` +
  1. +
    ${escapeHtml(new Date(h.created_at).toLocaleDateString('de-DE', { weekday: 'short', day: '2-digit', month: '2-digit', year: 'numeric' }))} + · Aufnahme #${h.recording_id}${h.speaker_name ? ' · ' + escapeHtml(h.speaker_name) : ''}
    + ${escapeHtml(h.title) || 'Aufnahme #' + h.recording_id} + bei ${fmt(h.t_start_ms)} +
    ${highlight(h.text, terms)}
    +
    2. `).join('')} +
`; + out.querySelectorAll('.result').forEach(li => li.onclick = (e) => { + e.preventDefault(); + openFromSearch(+li.dataset.rec, +li.dataset.t); + }); +} + +function renderAnswer() { + const box = document.querySelector('#answerbox'); + if (!box || !state.searchAnswer) return; + const { answer, sources } = state.searchAnswer; + box.innerHTML = ` +
Antwort
+
${escapeHtml(answer)}
+ ${sources && sources.length ? `
+ ${sources.map(s => ``).join('')} +
` : ''}`; + box.querySelectorAll('.src-chip').forEach(b => b.onclick = () => openFromSearch(+b.dataset.rec, +b.dataset.t)); +} + +// Jump from a hit into the recording, positioned at the spoken minute. The +// detail pane lives in the recordings tab, so switch there first and wait for +// the player element to exist before seeking. +async function openFromSearch(recId, tMs) { + state.tab = 'recordings'; + state.selectedId = recId; + state.trashOpen = false; + render(); + for (let i = 0; i < 40; i++) { + const p = document.querySelector('#player'); + if (p) { + const seek = () => { try { p.currentTime = tMs / 1000; p.play().catch(() => {}); } catch {} }; + if (p.readyState >= 1) seek(); else p.addEventListener('loadedmetadata', seek, { once: true }); + return; + } + await new Promise(r => setTimeout(r, 100)); + } +} + +// ---------- Recordings ---------- +// Analysis-duration counter. Shows the total wall time of a processing run +// (transcribe+diar+LLM): a live 1-Hz counter while it runs (from analysis_started_at), +// frozen to analysis_ms once done. Returns the inner only; callers add the ⏱. +function anaTimeSpan(r) { + if (r.analysis_ms != null) return `${fmt(r.analysis_ms)}`; + if (r.analysis_started_at && (r.analysis_status === 'pending' || r.status === 'transcribing')) + return `${fmt(Date.now() - r.analysis_started_at)}`; + return ''; +} +function ensureAnaTicker() { + if (state.anaTicker) return; + state.anaTicker = setInterval(() => { + document.querySelectorAll('.ana-time.live').forEach(el => { + const t0 = +el.dataset.anaStart; + if (t0) el.textContent = fmt(Date.now() - t0); + }); + }, 1000); +} + +// Split layout: LEFT = date-grouped, collapsible recording list; RIGHT = the +// selected recording's detail (with its own audio player). Crucially, the +// while-analysing auto-refresh only rebuilds the LEFT list — it never touches +// the RIGHT detail pane, so you can keep listening to an old recording while a +// new one is being transcribed/analysed (the old bug: the whole view — detail + +// audio element — was re-rendered every 5s, stopping playback). +function viewRecordings() { + const v = document.querySelector('#view'); + // Entering the tab: show only the newest day expanded, the rest collapsed + // (applied in refreshRecList once the day groups are known). + state.resetCollapse = true; + v.innerHTML = ` +
+ +
+
← Aufnahme wählen
+
`; + v.querySelector('#trashtoggle').onclick = () => { + state.trashOpen = !state.trashOpen; + state.lastListSig = null; + v.querySelector('#trashtoggle').classList.toggle('active', state.trashOpen); + refreshRecList(); + }; + // Draggable splitter: list width between LIST_MIN (the old fixed width) and + // twice that; persisted for all sessions. + const LIST_MIN = 340, LIST_MAX = 680; + const listPane = v.querySelector('.rec-list-pane'); + const applyW = (w) => { listPane.style.flex = `0 0 ${Math.min(LIST_MAX, Math.max(LIST_MIN, w))}px`; }; + applyW(+localStorage.getItem('aime-list-width') || LIST_MIN); + v.querySelector('.rec-splitter').onmousedown = (e) => { + e.preventDefault(); + const startX = e.clientX, startW = listPane.getBoundingClientRect().width; + document.body.classList.add('resizing'); + const move = (ev) => applyW(startW + (ev.clientX - startX)); + const up = () => { + document.removeEventListener('mousemove', move); + document.removeEventListener('mouseup', up); + document.body.classList.remove('resizing'); + localStorage.setItem('aime-list-width', Math.round(listPane.getBoundingClientRect().width)); + }; + document.addEventListener('mousemove', move); + document.addEventListener('mouseup', up); + }; + refreshRecList(); + if (state.selectedId != null) openRecording(state.selectedId); +} + +// (Re)build ONLY the left list pane. Called on entry and by the auto-refresh +// timer; safe to run while the right pane holds a playing recording. +async function refreshRecList() { + const listEl = document.querySelector('#reclist'); + if (!listEl) return; // left the tab + if (state.trashOpen) return refreshTrashList(listEl); + let recs; + try { recs = await api.get('/api/recordings'); } + catch (e) { listEl.innerHTML = '
' + e.message + '
'; return; } + // Reschedule the poll ONLY while something is still processing. + const stillRunning = recs.some(r => r.analysis_status === 'pending' || r.status === 'transcribing'); + const scheduleNext = () => { if (stillRunning) setTimeout(() => { if (state.tab === 'recordings') refreshRecList(); }, 5000); }; + try { + if (!recs.length && !state.recording) { listEl.innerHTML = '
Noch keine Aufnahmen.
'; state.lastListSig = 'empty'; scheduleNext(); return; } + // Signature of everything the list RENDERS except the live ⏱ seconds (those + // tick via ensureAnaTicker without a rebuild). If nothing meaningful changed, + // DON'T touch the DOM at all — no flicker, no scroll jump, selection kept. + // This is the real fix for "I get pulled away from my recording every 5s": + // the 5s poll now no-ops unless a status/title/todo actually changed. + const sig = recs.map(r => `${r.id}|${r.status}|${r.analysis_status}|${r.title || ''}|${(r.todos || []).length}|${r.summary ? 1 : 0}|${r.duration_ms || 0}`).join('~'); + if (sig === state.lastListSig && listEl.querySelector('.day-group')) { ensureAnaTicker(); scheduleNext(); return; } + state.lastListSig = sig; + // Preserve the left pane's scroll position across the rebuild. + const scroller = listEl.closest('.rec-list-pane') || listEl; + const savedTop = scroller.scrollTop; + // Group by calendar day (recs come newest-first). Each day is a collapsible + // section; the day header shows the recording count. + const groups = []; + const byDay = {}; + for (const r of recs) { + const day = new Date(r.created_at).toLocaleDateString('de-DE', { weekday: 'short', day: '2-digit', month: '2-digit', year: 'numeric' }); + if (!byDay[day]) { byDay[day] = []; groups.push(day); } + byDay[day].push(r); + } + // Fresh tab entry: only the newest day (first group, recs are newest-first) + // stays open; manual toggles afterwards are respected until the next entry. + if (state.resetCollapse) { state.collapsedDates = new Set(groups.slice(1)); state.resetCollapse = false; } + // Live placeholder while a recording runs: its own card at the very top. + const liveHtml = state.recording ? ` +
+ +
Aufnahme läuft… +
wird nach Stop transkribiert & analysiert
+
${markerChipsHtml()}
+ ${fmt(Date.now() - state.recStart)} +
` : ''; + listEl.innerHTML = liveHtml + groups.map(day => { + const collapsed = state.collapsedDates.has(day); + const items = byDay[day].map(r => { const at = anaTimeSpan(r); return ` +
+
+ ${escapeHtml(r.title) || 'Aufnahme #' + r.id} ${analysisBadge(r.analysis_status)} +
${new Date(r.created_at).toLocaleTimeString('de-DE', { hour: '2-digit', minute: '2-digit' })} · ${fmt(r.duration_ms || 0)} · ${r.language || ''}${(r.todos && r.todos.length) ? ' · ✅ ' + r.todos.length : ''}${at ? ' · ⏱ ' + at : ''}
+ ${r.summary ? `
${escapeHtml(r.summary)}
` : ''} +
+ + +
`; }).join(''); + return ` +
+
▾ ${escapeHtml(day)} ${byDay[day].length}
+
${items}
+
`; + }).join(''); + // Click a row (not one of its buttons) → open in the right pane. + listEl.querySelectorAll('.item.rec').forEach(it => it.onclick = (e) => { + if (e.target.closest('[data-del]') || e.target.closest('[data-ren]')) return; + openRecording(+it.dataset.open); + }); + listEl.querySelectorAll('[data-ren]').forEach(b => b.onclick = async (e) => { + e.stopPropagation(); + const rec = recs.find(r => r.id == b.dataset.ren); + await renameRecording(+b.dataset.ren, (rec && rec.title) || ''); + }); + listEl.querySelectorAll('[data-del]').forEach(b => b.onclick = async (e) => { + e.stopPropagation(); + const rec = recs.find(r => r.id == b.dataset.del); + const name = (rec && rec.title) ? `„${rec.title}"` : 'Aufnahme #' + b.dataset.del; + if (!confirm(`${name} in den Papierkorb verschieben?\nWiederherstellbar für 30 Tage, danach wird sie automatisch endgültig gelöscht.`)) return; + await api.del('/api/recordings/' + b.dataset.del); + if (state.selectedId == b.dataset.del) { state.selectedId = null; const d = document.querySelector('#detail'); if (d) d.innerHTML = '
← Aufnahme wählen
'; } + toast('In den Papierkorb verschoben'); + state.lastListSig = null; refreshRecList(); + }); + bindMarkerChips(); + listEl.querySelectorAll('.day-head').forEach(h => h.onclick = () => { + const day = h.dataset.day; + if (state.collapsedDates.has(day)) state.collapsedDates.delete(day); else state.collapsedDates.add(day); + h.closest('.day-group').classList.toggle('collapsed'); + }); + scroller.scrollTop = savedTop; // keep the list where the user had it + ensureAnaTicker(); + scheduleNext(); + } catch (e) { listEl.innerHTML = '
' + e.message + '
'; } +} + +// Rename a recording's title. An empty answer clears the title and hands +// naming back to the LLM on the next analysis (see db.renameRecording). +async function renameRecording(id, current) { + const t = prompt('Titel der Aufnahme:', current || ''); + if (t === null) return false; // cancelled + try { + const r = await api.patch(`/api/recordings/${id}/title`, { title: t }); + state.lastListSig = null; + refreshRecList(); + // Keep the open detail pane in sync without rebuilding it (audio keeps playing). + const h = document.querySelector('#detail h2'); + if (h && state.selectedId == id) { + const badge = h.querySelector('.badge'); + h.textContent = r.title || 'Aufnahme #' + id; + if (badge) h.appendChild(badge); + } + toast(r.title ? 'Titel geändert' : 'Titel zurückgesetzt'); + return true; + } catch (e) { toast('Umbenennen fehlgeschlagen: ' + e.message, true); return false; } +} + +// Trash pane: same list slot, different contents. Deliberately flat (no day +// grouping) — the interesting order here is "when did I delete it", and the +// remaining days matter more than the recording date. +async function refreshTrashList(listEl) { + let data; + try { data = await api.get('/api/trash'); } + catch (e) { listEl.innerHTML = '
' + e.message + '
'; return; } + const items = data.items || []; + if (!items.length) { + listEl.innerHTML = `
Papierkorb ist leer.
Gelöschte Aufnahmen bleiben ${data.days} Tage hier.
`; + return; + } + listEl.innerHTML = `
Wird nach ${data.days} Tagen automatisch endgültig gelöscht.
` + items.map(r => ` +
+
+ ${escapeHtml(r.title) || 'Aufnahme #' + r.id} +
${new Date(r.created_at).toLocaleDateString('de-DE')} · ${fmt(r.duration_ms || 0)} + · noch ${r.days_left} Tag${r.days_left === 1 ? '' : 'e'}
+
+ + +
`).join(''); + listEl.querySelectorAll('[data-restore]').forEach(b => b.onclick = async () => { + await api.post(`/api/trash/${b.dataset.restore}/restore`, {}); + toast('Wiederhergestellt'); + refreshTrashList(listEl); + }); + listEl.querySelectorAll('[data-purge]').forEach(b => b.onclick = async () => { + const rec = items.find(r => r.id == b.dataset.purge); + const name = (rec && rec.title) ? `„${rec.title}"` : 'Aufnahme #' + b.dataset.purge; + if (!confirm(`${name} ENDGÜLTIG löschen?\nAudio, Transkript und Analyse werden unwiderruflich entfernt. Das lässt sich nicht rückgängig machen.`)) return; + await api.del('/api/trash/' + b.dataset.purge); + toast('Endgültig gelöscht'); + refreshTrashList(listEl); + }); +} + +async function openRecording(id) { + state.selectedId = id; + const detail = document.querySelector('#detail'); + if (!detail) { state.tab = 'recordings'; render(); return setTimeout(() => openRecording(id), 100); } + // Highlight the selected row without rebuilding the list (keeps it stable). + document.querySelectorAll('#reclist .item.rec').forEach(it => it.classList.toggle('sel', it.dataset.open == id)); + detail.innerHTML = '
Lade…
'; + const r = await api.get('/api/recordings/' + id); + const pending = r.analysis_status === 'pending'; + const errored = r.analysis_status === 'error'; + const transcribing = r.status === 'transcribing'; + const transcribeFailed = r.status === 'error'; + const scls = speakerClasses(r.words || []); // rawSpeaker -> sp0..sp3 + const spkOrder = Object.keys(scls); // first-appearance order + const names = r.speaker_names || {}; // rawSpeaker -> name + const spkLabel = (sp, i) => names[sp] || ('Sprecher ' + (i + 1)); + // Speaker segments + turns for the lane visualizer and the live-text box. + const segsBySpk = speakerSegments(r.words || []); + const turns = wordTurns(r.words || []); + const durMs = r.duration_ms || (r.words && r.words.length ? r.words[r.words.length - 1].t_end_ms : 0) || 1; + + // Draggable module blocks; the saved order wins, unknown/new modules append. + // NOT modules: the transcript (own sticky right column) and the speaker-lane + // visualizer (fixed full-width strip under the player, spanning both columns). + const MOD_TITLES = { analyse: 'Zusammenfassung', todos: 'ToDos', transcript: 'Transkript' }; + const vizHtml = spkOrder.length ? ` +
+
+
+ ${spkOrder.map((sp, i) => ` +
+ ${escapeHtml(spkLabel(sp, i))} +
${(segsBySpk[sp] || []).map(s => + ``).join('')} +
+
`).join('')} +
+
+
+ +
` : ''; + // Live-typed name markers: which cluster each one was resolved to, plus the + // option to add one while listening back (same resolution path) and to re-run + // the assignment after a re-diarization. + const CONF_LBL = { high: 'sicher', mid: 'wahrscheinlich', low: 'unsicher' }; + const markersHtml = ` +

Namensmarker (während der Aufnahme getippt)

+ ${(r.markers || []).length ? `
    ${r.markers.map(m => { + const i = spkOrder.indexOf(m.speaker); + const to = m.speaker ? spkLabel(m.speaker, i >= 0 ? i : 0) : '– kein Sprecher gefunden'; + return `
  • + ${escapeHtml(m.name)} → ${escapeHtml(to)} + ${CONF_LBL[m.conf] || ''} +
    • `; + }).join('')}
` : '
Keine Marker. Tipp: während der Aufnahme Namen eintippen (Feld oben) – oder hier beim Zuhören nachtragen.
'} +
+ + + ${(r.markers || []).length ? '' : ''} +
`; + const mods = { + analyse: ` + ${r.summary ? `

${escapeHtml(r.summary)}

` : '
Noch keine Zusammenfassung.
'} + ${(r.participants && r.participants.length) ? `

Teilnehmer

    ${r.participants.map(p => `
  • 👤 ${escapeHtml(p)}
    • `).join('')}
` : ''} + ${markersHtml}`, + todos: (r.todos && r.todos.length) ? `
    ${r.todos.map(t => `
  • ${escapeHtml(t)}
    • `).join('')}
` : '
Keine ToDos erkannt.
', + transcript: ` +
+ ${r.transcript_clean ? `

Dialog (KI · Sprecher aus Kontext)

${renderDialog(r.transcript_clean)}
` : ''} +
Klick auf ein Wort → Audio springt dorthin. Leertaste = Play/Pause.
+
${renderTurns(r.words, names, scls, spkOrder)}
+
+ `, + }; + // Two drop columns; every module (incl. the transcript/table panel) can be + // dragged within AND between them. Layout persists for all recordings. + const defLayout = { main: ['analyse', 'todos'], side: ['transcript'] }; + let layout = null; + try { layout = JSON.parse(localStorage.getItem('aime-detail-layout') || 'null'); } catch {} + if (!layout || !Array.isArray(layout.main) || !Array.isArray(layout.side)) { + // migrate the old single-column order if present + let old = []; + try { old = JSON.parse(localStorage.getItem('aime-detail-order') || '[]'); } catch {} + layout = { main: old.filter(m => m !== 'transcript'), side: [] }; + } + layout.main = layout.main.filter(m => mods[m] !== undefined); + layout.side = layout.side.filter(m => mods[m] !== undefined && !layout.main.includes(m)); + for (const m of [...defLayout.main, ...defLayout.side]) + if (!layout.main.includes(m) && !layout.side.includes(m)) + (defLayout.side.includes(m) ? layout.side : layout.main).push(m); + + const modHtml = (m) => ` +
+
⠿

${MOD_TITLES[m]}

+
${mods[m]}
+
`; + + detail.innerHTML = ` +
+

${escapeHtml(r.title) || 'Aufnahme #' + r.id} ${analysisBadge(r.analysis_status)}

+ + + ${vizHtml} +
+ ${anaTimeSpan(r) ? `
⏱ Analysezeit: ${anaTimeSpan(r)}
` : ''} + ${transcribing ? '
📝 Transkription läuft… lange Aufnahmen dauern (A76-Takt gegen Überhitzung gedrosselt).
' : ''} + ${transcribeFailed ? `
Transkription fehlgeschlagen.
` : ''} + ${(pending && !transcribing && !transcribeFailed) ? '
🧠 KI-Analyse läuft… (NPU)
' : ''} + ${(errored && !transcribeFailed) ? `
KI-Analyse fehlgeschlagen.
` : ''} + ${(!pending && !errored && !transcribing && !transcribeFailed) ? `` : ''} + ${spkOrder.length ? `` : ''} +
+
+
${layout.main.map(modHtml).join('')}
+
${layout.side.map(modHtml).join('')}
+
+
`; + + // Drag & drop across BOTH columns: grab any module by its header, drop it in + // either column at any position; the layout is saved for all recordings. + const cols = [...detail.querySelectorAll('.mod-col')]; + // ":empty" never matches because of template whitespace -> class-driven. + const markEmpty = () => cols.forEach(c => c.classList.toggle('empty', !c.querySelector('.module'))); + markEmpty(); + let dragged = null; + detail.querySelectorAll('.mod-head').forEach(h => { + h.addEventListener('dragstart', (e) => { + dragged = h.closest('.module'); + dragged.classList.add('dragging'); + e.dataTransfer.effectAllowed = 'move'; + e.dataTransfer.setData('text/plain', dragged.dataset.mod); // Firefox needs data + }); + h.addEventListener('dragend', () => { + if (!dragged) return; + dragged.classList.remove('dragging'); + dragged = null; + const save = {}; + for (const c of cols) save[c.dataset.col] = [...c.querySelectorAll('.module')].map(m => m.dataset.mod); + localStorage.setItem('aime-detail-layout', JSON.stringify(save)); + markEmpty(); + }); + }); + cols.forEach(col => col.addEventListener('dragover', (e) => { + if (!dragged) return; + e.preventDefault(); + const others = [...col.querySelectorAll('.module:not(.dragging)')]; + const next = others.find(m => e.clientY < m.getBoundingClientRect().top + m.offsetHeight / 2); + if (next) col.insertBefore(dragged, next); else col.appendChild(dragged); + markEmpty(); + })); + const renBtn = detail.querySelector('#renametitle'); + if (renBtn) renBtn.onclick = () => renameRecording(r.id, r.title || ''); + const reBtn = detail.querySelector('#reanalyze'); + if (reBtn) reBtn.onclick = async () => { + // Re-runs speaker separation every time: empty = automatic detection (now + // consolidated, so no more "14 speakers" for a 2-person call), a number pins + // the count. Cancel aborts. + const ans = window.prompt('Sprecher neu erkennen?\n(leer = automatisch erkennen · Zahl = feste Anzahl vorgeben)', ''); + if (ans === null) return; + const numSpeakers = parseInt(ans, 10); + const body = numSpeakers > 0 ? { numSpeakers } : {}; + reBtn.disabled = true; reBtn.textContent = 'trennt neu…'; + try { await api.post('/api/recordings/' + id + '/reanalyze', body); setTimeout(() => openRecording(id), 1500); } + catch (e) { toast(e.message, true); } + }; + const rtBtn = detail.querySelector('#retranscribe'); + if (rtBtn) rtBtn.onclick = async () => { + rtBtn.disabled = true; rtBtn.textContent = 'startet…'; + try { await api.post('/api/recordings/' + id + '/retranscribe', {}); setTimeout(() => openRecording(id), 1500); } + catch (e) { toast(e.message, true); rtBtn.disabled = false; } + }; + ensureAnaTicker(); + // Poll while transcription OR the NPU analysis is still running. The old + // version re-ran openRecording(id) blindly every 5s — a stale closure: after + // switching to ANOTHER recording the timer still fired and yanked the user + // (and the playing audio) back to the analysing one. Now: stop when the + // selection changed, poll ONLY the status while busy (no DOM rebuild), and + // rebuild once when the results are in — restoring the player position. + if (pending || transcribing) { + const pollDetail = async () => { + if (state.selectedId !== id || state.tab !== 'recordings') return; // user moved on + if (!document.querySelector('#detail')) return; // view gone + let cur; + try { cur = await api.get('/api/recordings/' + id); } catch { return; } + if (state.selectedId !== id || state.tab !== 'recordings') return; // re-check after await + const busy = cur.analysis_status === 'pending' || cur.status === 'transcribing'; + if (busy) { setTimeout(pollDetail, 5000); return; } + // Results arrived → one rebuild; keep the audio where the user had it. + const a = document.querySelector('#detail #player'); + const keep = a && !a.paused ? { t: a.currentTime, play: true } : (a ? { t: a.currentTime, play: false } : null); + await openRecording(id); + if (keep && keep.t > 0) { + const na = document.querySelector('#detail #player'); + if (na) { na.currentTime = keep.t; if (keep.play) na.play().catch(() => {}); } + } + }; + setTimeout(pollDetail, 5000); + } + const audio = detail.querySelector('#player'); + // ---- marker actions (seek / add at playhead / delete / re-assign) ---- + detail.querySelectorAll('.mkseek').forEach(b => b.onclick = () => { + audio.currentTime = (+b.dataset.t) / 1000 + 0.001; audio.play(); + }); + const mkAt = detail.querySelector('#mkat'), mkAtBtn = detail.querySelector('#mkatbtn'); + const addAt = async () => { + const name = (mkAt.value || '').trim(); + if (!name) return; + try { + await api.post(`/api/recordings/${id}/markers`, { name, t_ms: Math.round(audio.currentTime * 1000) }); + mkAt.value = ''; + await api.post(`/api/recordings/${id}/apply-markers`, {}); + toast(`🏷 ${name} gesetzt & zugeordnet`); + openRecording(id); + } catch (e) { toast(e.message, true); } + }; + if (mkAtBtn) mkAtBtn.onclick = addAt; + if (mkAt) mkAt.onkeydown = (e) => { if (e.key === 'Enter') addAt(); }; + detail.querySelectorAll('[data-mkdel]').forEach(b => b.onclick = async () => { + try { await api.del(`/api/recordings/${id}/markers/${b.dataset.mkdel}`); openRecording(id); } + catch (e) { toast(e.message, true); } + }); + const amk = detail.querySelector('#applymk'); + if (amk) amk.onclick = async () => { + amk.disabled = true; amk.textContent = 'ordne zu…'; + try { + const r2 = await api.post(`/api/recordings/${id}/apply-markers`, {}); + toast(`${r2.named} Sprecher benannt, ${r2.merged} zusammengeführt, ${r2.learned} Stimme(n) gelernt`); + openRecording(id); + } catch (e) { toast(e.message, true); amk.disabled = false; } + }; + const spans = [...detail.querySelectorAll('.w')]; + spans.forEach(s => s.onclick = () => { audio.currentTime = (+s.dataset.start) / 1000 + 0.001; audio.play(); }); + // Whisper (batched) word timestamps run slightly AHEAD of the audio, so a + // pure [start,end] match lit the NEXT word ("eine Zeile weiter") before it was + // spoken. Compensate with a fixed lead offset and highlight the LAST word whose + // start has passed (stays lit through inter-word gaps → no flicker). + // Tune SYNC_MS if the marker still leads (raise) or lags (lower). + const SYNC_MS = 250; + + // ---- View switch: transcript <-> conference-table scene ---- + const switchBtn = detail.querySelector('#viewswitch'); + const tView = detail.querySelector('#table-view'); + const trView = detail.querySelector('#transcript-view'); + const sideTitle = detail.querySelector('#sidetitle'); + const bubble = detail.querySelector('#tbubble'); + const tIdle = detail.querySelector('#tidle'); + const persons = [...detail.querySelectorAll('.tperson')]; + let bubbleTurnIdx = -1; + const applyViewMode = () => { + const table = localStorage.getItem('aime-side-view') === 'table'; + if (tView) tView.classList.toggle('hidden', !table); + if (trView) trView.classList.toggle('hidden', table); + if (sideTitle) sideTitle.textContent = table ? 'Am Tisch' : 'Transkript'; + if (switchBtn) switchBtn.innerHTML = table ? ICON_SHEET : ICON_TABLE; + }; + if (switchBtn) { + switchBtn.onclick = () => { + const now = localStorage.getItem('aime-side-view') === 'table' ? 'text' : 'table'; + localStorage.setItem('aime-side-view', now); + applyViewMode(); + }; + applyViewMode(); + } + // All eyes on the speaker: shift every person's pupils toward the active seat. + const lookAt = (activeSp) => { + const target = persons.find(p => p.dataset.sp === activeSp); + for (const p of persons) { + const eyes = p.querySelector('.teyes'); + if (!eyes) continue; + if (!target || p === target) { eyes.style.transform = 'translate(0, 1px)'; continue; } + const dx = (+target.dataset.x) - (+p.dataset.x); + const dy = (+target.dataset.y) - (+p.dataset.y); + const len = Math.hypot(dx, dy) || 1; + eyes.style.transform = `translate(${(dx / len * 3.2).toFixed(1)}px, ${(dy / len * 2.4).toFixed(1)}px)`; + } + }; + const updateTable = (t) => { + if (!tView || tView.classList.contains('hidden')) return; + let idx = -1; + for (let i = 0; i < turns.length; i++) { if (turns[i].start <= t) idx = i; else break; } + if (idx < 0) return; + const tu = turns[idx]; + if (idx !== bubbleTurnIdx) { + bubbleTurnIdx = idx; + const i = spkOrder.indexOf(tu.sp); + const seats = tableSeats(Math.min(persons.length, spkOrder.length)); + const seat = seats[i >= 0 ? Math.min(i, persons.length - 1) : 0]; + if (bubble && seat) { + // The bubble sits FIXED on the table; only its tail (the "Öffnung") + // wanders to point at whoever is speaking. + bubble.className = `tbubble ${scls[tu.sp] || ''} arrow-${seat.side}`; + bubble.style.left = ''; + bubble.style.top = ''; + // Tail x-position along the top/bottom edge = the speaker's seat mapped + // into bubble-local coordinates (bubble is centred, BUBBLE_W wide). + const ax = Math.max(8, Math.min(92, (seat.x - (50 - BUBBLE_W / 2)) / BUBBLE_W * 100)); + bubble.style.setProperty('--ax', ax.toFixed(1) + '%'); + bubble.querySelector('.tb-name').textContent = spkLabel(tu.sp, i >= 0 ? i : 0); + // The full turn as clickable word spans — highlighted + scrolled below. + bubble.querySelector('.tb-text').innerHTML = + tu.words.map(w => `${escapeHtml(w.w)}`).join(' '); + } + if (tIdle) tIdle.classList.add('hidden'); + lookAt(tu.sp); + } + persons.forEach(p => p.classList.toggle('talking', p.dataset.sp === tu.sp && !audio.paused)); + // Word highlight + follow-scroll inside the bubble (like the transcript). + if (bubble && !bubble.classList.contains('hidden')) { + const scroller = bubble.querySelector('.tb-scroll'); + const bws = bubble.querySelectorAll('.bw'); + let act = null; + for (const s of bws) { if (+s.dataset.s <= t) act = s; else break; } + bws.forEach(s => s.classList.toggle('active', s === act)); + if (act && scroller) { + const target = act.offsetTop - scroller.clientHeight / 2 + 10; + if (Math.abs(scroller.scrollTop - target) > 24) scroller.scrollTo({ top: Math.max(0, target), behavior: 'smooth' }); + } + } + }; + if (bubble) bubble.addEventListener('click', (e) => { + const s = e.target.closest('.bw'); + if (s) { audio.currentTime = (+s.dataset.s) / 1000 + 0.001; audio.play().catch(() => {}); } + }); + + // Lane visualizer wiring: playhead + speaking lane + live-text box. + const lanesEl = detail.querySelector('#lanes'); + const playheadEl = detail.querySelector('#playhead'); + const liveboxEl = detail.querySelector('#livebox'); + let liveTurnIdx = -1; + if (lanesEl) { + // Click inside a LANE seeks; clicks on the label column open the editor. + const seek = (e) => { + if (!e.target.closest('.lane')) return; + const lane = lanesEl.querySelector('.lane'); + if (!lane) return; + const rc = lane.getBoundingClientRect(); + const frac = Math.min(1, Math.max(0, (e.clientX - rc.left) / rc.width)); + audio.currentTime = frac * durMs / 1000; + audio.play().catch(() => {}); + }; + lanesEl.addEventListener('click', seek); + } + // Speaker editor popover: click a lane label -> rename + optionally learn the + // voice (replaces the old separate "Sprecher" module). + const pop = detail.querySelector('#spkpop'); + if (pop && lanesEl) { + let popSp = null; + const popName = pop.querySelector('#spkpop-name'); + lanesEl.querySelectorAll('.lane-label').forEach(l => l.onclick = (e) => { + e.stopPropagation(); + const row = l.closest('.lane-row'); + popSp = row.dataset.sp; + pop.style.top = (row.offsetTop + 30) + 'px'; + pop.classList.remove('hidden'); + const i = spkOrder.indexOf(popSp); + popName.value = names[popSp] || ''; + popName.placeholder = spkLabel(popSp, i >= 0 ? i : 0); + popName.focus(); + }); + pop.querySelector('#spkpop-cancel').onclick = () => pop.classList.add('hidden'); + pop.querySelector('#spkpop-save').onclick = async () => { + const name = popName.value.trim(); + const enroll = pop.querySelector('#spkpop-learn').checked; + const btn = pop.querySelector('#spkpop-save'); + btn.disabled = true; btn.textContent = '…'; + try { + const res = await api.post('/api/recordings/' + id + '/speaker', { speaker: popSp, name, enroll }); + const base = res.enrolled ? `„${name}" gespeichert + Stimme gelernt` : (res.learnMsg ? 'Gespeichert (' + res.learnMsg + ')' : 'Gespeichert'); + toast(res.relabeled ? base + ' · Name in Zusammenfassung übernommen' : base); + openRecording(id); + } catch (e) { toast(e.message, true); btn.disabled = false; btn.textContent = 'Speichern'; } + }; + } + const updateViz = (t) => { + if (!lanesEl) return; + if (playheadEl) { + const lane = lanesEl.querySelector('.lane'); + if (lane) { + const left = lane.offsetLeft + Math.min(1, Math.max(0, t / durMs)) * lane.offsetWidth; + playheadEl.style.left = left + 'px'; + playheadEl.style.display = 'block'; + } + } + // Which speakers talk right now → light their lane rows. + lanesEl.querySelectorAll('.lane-row').forEach(row => { + const sp = row.dataset.sp; + const on = (segsBySpk[sp] || []).some(s => t >= s.start && t <= s.end); + row.classList.toggle('speaking', on && !audio.paused); + }); + // Live text: last turn whose start has passed (stays through gaps). + if (liveboxEl) { + let idx = -1; + for (let i = 0; i < turns.length; i++) { if (turns[i].start <= t) idx = i; else break; } + if (idx !== liveTurnIdx && idx >= 0) { + liveTurnIdx = idx; + const tu = turns[idx]; + const i = spkOrder.indexOf(tu.sp); + liveboxEl.innerHTML = ` +
+ ${escapeHtml(spkLabel(tu.sp, i >= 0 ? i : 0))} + ${escapeHtml(tu.text)} +
`; + } + } + }; + audio.onpause = () => { + if (lanesEl) lanesEl.querySelectorAll('.lane-row.speaking').forEach(r2 => r2.classList.remove('speaking')); + persons.forEach(p => p.classList.remove('talking')); + }; + audio.ontimeupdate = () => { + const t = audio.currentTime * 1000 - SYNC_MS; + let active = null; + for (const s of spans) { if (+s.dataset.start <= t) active = s; else break; } + spans.forEach(s => s.classList.toggle('active', s === active)); + // Follow the active word INSIDE the transcript panel (it scrolls itself now — + // scrollIntoView would also yank the page). + if (active && (!trView || !trView.classList.contains('hidden'))) { + const panel = detail.querySelector('#transcript-panel'); + if (panel) { + const pr = panel.getBoundingClientRect(), ar = active.getBoundingClientRect(); + if (ar.top < pr.top + 40 || ar.bottom > pr.bottom - 40) + panel.scrollTo({ top: panel.scrollTop + (ar.top - pr.top) - panel.clientHeight / 2, behavior: 'smooth' }); + } + } + updateViz(t); + updateTable(t); + }; + // (Speaker renaming lives in the lane-label popover above.) + + // Spacebar toggles play/pause of the open recording (unless typing in a field). + if (state.spaceHandler) document.removeEventListener('keydown', state.spaceHandler); + state.spaceHandler = (e) => { + if (e.code !== 'Space' && e.key !== ' ') return; + const t = e.target, tag = (t.tagName || '').toLowerCase(); + if (tag === 'input' || tag === 'textarea' || t.isContentEditable) return; + const a = document.querySelector('#player'); + if (!a) return; + e.preventDefault(); + if (a.paused) a.play(); else a.pause(); + }; + document.addEventListener('keydown', state.spaceHandler); + + // Two-pane layout: scroll the detail pane to its top. On a narrow (stacked) + // screen also bring the detail into view since it sits below the list. + const pane = detail.closest('.rec-detail-pane'); + if (pane) pane.scrollTo({ top: 0, behavior: 'smooth' }); + if (window.matchMedia('(max-width: 800px)').matches) detail.scrollIntoView({ behavior: 'smooth' }); +} + +function escapeHtml(s) { return (s || '').replace(/[&<>"]/g, c => ({ '&': '&', '<': '<', '>': '>', '"': '"' }[c])); } + +// ---- Konferenztisch-Ansicht ---- +// Seat positions (person center, % of scene) per participant count, arranged +// around the rectangular table; each seat knows which bubble-arrow side points +// at it from the table-centre bubble. +const SEAT_LAYOUTS = { + 1: [{ x: 50, y: 84, side: 'bottom' }], + 2: [{ x: 50, y: 14, side: 'top' }, { x: 50, y: 84, side: 'bottom' }], + 3: [{ x: 50, y: 13, side: 'top' }, { x: 26, y: 84, side: 'bottom' }, { x: 74, y: 84, side: 'bottom' }], + 4: [{ x: 50, y: 12, side: 'top' }, { x: 90, y: 50, side: 'right' }, { x: 50, y: 86, side: 'bottom' }, { x: 10, y: 50, side: 'left' }], + 5: [{ x: 32, y: 12, side: 'top' }, { x: 68, y: 12, side: 'top' }, { x: 90, y: 60, side: 'right' }, { x: 50, y: 86, side: 'bottom' }, { x: 10, y: 60, side: 'left' }], + 6: [{ x: 32, y: 12, side: 'top' }, { x: 68, y: 12, side: 'top' }, { x: 90, y: 50, side: 'right' }, { x: 68, y: 86, side: 'bottom' }, { x: 32, y: 86, side: 'bottom' }, { x: 10, y: 50, side: 'left' }], +}; +// Bubble width in scene-% (kept in sync with .tbubble in style.css) — needed to +// map a seat's x onto the bubble edge so the tail points AT the speaker. +const BUBBLE_W = 54; +function tableSeats(n) { return SEAT_LAYOUTS[Math.min(6, Math.max(1, n))] || SEAT_LAYOUTS[6]; } + +// Toggle-button icons: three people around a table / a text sheet. +const ICON_TABLE = ``; +const ICON_SHEET = ``; + +// The scene: rounded table, one person per speaker (their lane colour), a +// speech bubble ON the table whose arrow + border follow whoever talks. +function buildTableScene(spkOrder, scls, names, spkLabel) { + if (!spkOrder.length) return '
Keine Sprecherdaten.
'; + const seats = tableSeats(spkOrder.length); + return ` +
+
+ ${spkOrder.slice(0, seats.length).map((sp, i) => { + const s = seats[i]; + return ` +
+
+
+
${escapeHtml(spkLabel(sp, i))}
+
`; + }).join('')} + +
▶ Abspielen — die Runde beginnt zu sprechen.
+
`; +} + +// Per-speaker talk segments for the lane visualizer: consecutive words of the +// same speaker, gaps ≤ 1.2 s bridged (word-level gaps would render as dust). +function speakerSegments(words) { + const by = {}; + for (const w of words) { + if (!w.speaker) continue; + const list = by[w.speaker] || (by[w.speaker] = []); + const last = list[list.length - 1]; + if (last && w.t_start_ms - last.end <= 1200) last.end = Math.max(last.end, w.t_end_ms); + else list.push({ start: w.t_start_ms, end: w.t_end_ms }); + } + return by; +} +// Turn list for the table scene: speaker, joined text AND the single words +// (with start times) so the bubble can highlight word-by-word like the transcript. +function wordTurns(words) { + const turns = []; + for (const w of words) { + const last = turns[turns.length - 1]; + if (last && last.sp === w.speaker) { last.text += ' ' + w.word; last.end = w.t_end_ms; last.words.push({ w: w.word, s: w.t_start_ms }); } + else turns.push({ sp: w.speaker, text: w.word, start: w.t_start_ms, end: w.t_end_ms, words: [{ w: w.word, s: w.t_start_ms }] }); + } + return turns; +} +// Map distinct raw speaker labels (speaker_00…) to colour classes sp0..sp3. +function speakerClasses(words) { + const m = {}; let i = 0; const palette = ['sp0', 'sp1', 'sp2', 'sp3']; + for (const w of words) { if (w.speaker && !(w.speaker in m)) { m[w.speaker] = palette[i % palette.length]; i++; } } + return m; +} +// Render the raw transcript as speaker turns with a NAME header per turn. Each +// word stays its own .w span (click-to-seek + active highlight still work). +function renderTurns(words, names, scls, order) { + if (!words || !words.length) return 'Kein Transkript.'; + const idx = {}; order.forEach((s, i) => idx[s] = i); + const label = sp => (names && names[sp]) || (sp ? 'Sprecher ' + ((idx[sp] ?? 0) + 1) : ''); + let html = '', cur = '\0', buf = []; + const flush = () => { + if (!buf.length) return; + const nm = label(cur === '\0' ? null : cur); + html += `
${nm ? `${escapeHtml(nm)} ` : ''}${buf.join(' ')}
`; + buf = []; + }; + for (const w of words) { + const sp = w.speaker || null; + if (sp !== cur) { flush(); cur = sp; } + buf.push(`${escapeHtml(w.word)}`); + } + flush(); + return html; +} +// Render "Name: Text" lines as a colour-coded dialogue (speakers inferred by +// the LLM from context — not acoustic). +function renderDialog(text) { + const lines = (text || '').split(/\n+/).map(l => l.trim()).filter(Boolean); + const colors = {}; let ci = 0; const palette = ['sp0', 'sp1', 'sp2', 'sp3']; + return lines.map(l => { + const m = l.match(/^([^:]{1,30}):\s*(.*)$/); + if (!m) return `
${escapeHtml(l)}
`; + const spk = m[1].trim(); + if (!(spk in colors)) { colors[spk] = palette[ci % palette.length]; ci++; } + return `
${escapeHtml(spk)} ${escapeHtml(m[2])}
`; + }).join(''); +} +function analysisBadge(st) { + if (st === 'pending') return '🧠 Analyse…'; + if (st === 'error') return 'KI-Fehler'; + if (st === 'done') return 'KI ✓'; + return ''; +} + +render(); diff --git a/app/web/src/style.css b/app/web/src/style.css index 427d7e1..4709051 100644 --- a/app/web/src/style.css +++ b/app/web/src/style.css @@ -1,397 +1,495 @@ -/* Aime — Anruf-Assistent. Zwei Themes über CSS-Variablen; data-theme setzt - index.html vor dem ersten Paint, der Header-Toggle schaltet um. - Akzent = warmes Kupfer (Tonband-Anmutung), Rot bleibt der Aufnahme reserviert. */ +/* Aime — Anruf-Assistent. Zwei Themes über CSS-Variablen; data-theme setzt + index.html vor dem ersten Paint, der Header-Toggle schaltet um. + Akzent = warmes Kupfer (Tonband-Anmutung), Rot bleibt der Aufnahme reserviert. */ + +:root[data-theme="dark"] { + --bg: #14161b; + --surface: #1d222a; + --surface-2: #262c36; + --inset: #10141a; + --line: #303845; + --fg: #e9ecf1; + --muted: #98a2b3; + --accent: #e8804c; + --accent-fg: #17130f; + --accent-soft: rgba(232, 128, 76, .14); + --ok: #55b381; + --ok-soft: rgba(85, 179, 129, .14); + --warn: #e2b93b; + --warn-soft: rgba(226, 185, 59, .14); + --danger: #e5595e; + --danger-soft: rgba(229, 89, 94, .12); + --shadow: none; + --sp0: #5bc0eb; --sp1: #ef8fc0; --sp2: #a8cc4e; --sp3: #e6b84c; + --sp0-w: #8fd6f5; --sp1-w: #f5b5d6; --sp2-w: #c5e07e; --sp3-w: #f0d08a; +} +:root[data-theme="light"] { + --bg: #f4f2ee; + --surface: #ffffff; + --surface-2: #eceae4; + --inset: #f7f6f2; + --line: #ddd8cf; + --fg: #23272e; + --muted: #6d7683; + --accent: #c05f2d; + --accent-fg: #ffffff; + --accent-soft: rgba(192, 95, 45, .1); + --ok: #2f8a5d; + --ok-soft: rgba(47, 138, 93, .12); + --warn: #9a7514; + --warn-soft: rgba(154, 117, 20, .12); + --danger: #c73e43; + --danger-soft: rgba(199, 62, 67, .09); + --shadow: 0 1px 3px rgba(35, 39, 46, .07); + --sp0: #1179a8; --sp1: #c2417f; --sp2: #5f7d16; --sp3: #a4700d; + --sp0-w: #0e6b95; --sp1-w: #ad3a72; --sp2-w: #556f14; --sp3-w: #92640b; +} + +* { box-sizing: border-box; } + +/* Scrollbalken im Logo-Kupfer (Firefox: scrollbar-color; Chromium/WebKit: ::-webkit-*) */ +* { scrollbar-width: thin; scrollbar-color: var(--accent) transparent; } +::-webkit-scrollbar { width: 10px; height: 10px; } +::-webkit-scrollbar-track { background: transparent; } +::-webkit-scrollbar-thumb { + background: var(--accent); border-radius: 5px; + border: 2px solid transparent; background-clip: padding-box; +} +::-webkit-scrollbar-thumb:hover { background: var(--accent); filter: brightness(1.15); } +::-webkit-scrollbar-corner { background: transparent; } +html { color-scheme: dark; } +:root[data-theme="light"] { color-scheme: light; } +body { + margin: 0; background: var(--bg); color: var(--fg); + font: 15px/1.5 system-ui, "Segoe UI", Roboto, sans-serif; + transition: background .2s, color .2s; +} +/* Volle Fensterbreite für ALLE Tabs. Früher war nur „Aufnahmen“ breit (.wide) + und der Rest auf 860px zentriert — dadurch sprang die Kopfzeile beim + Tabwechsel in die Mitte. */ +#app { max-width: none; margin: 0 auto; padding: 16px 28px; } + +/* ---- Topbar / Brand ---- */ +.topbar { display: flex; align-items: center; gap: 16px; flex-wrap: wrap; padding: 6px 0 14px; } +.brand { display: flex; align-items: center; gap: 10px; } +.brand-wave { width: 30px; height: 18px; color: var(--accent); flex: none; } +.topbar h1 { margin: 0; font-size: 21px; font-weight: 650; letter-spacing: -.01em; } +.topbar .sub { font-size: 12px; color: var(--muted); font-weight: 400; margin-left: 6px; } +nav { display: flex; gap: 6px; flex-wrap: wrap; } +.topbar nav { margin-left: 0; } + +/* Aufnahme-Steuerung in der Kopfzeile (zwischen Logo und Tabs) */ +.rec-ctl { display: flex; align-items: center; gap: 10px; margin: 0 auto; } +.recbtn { + width: 38px; height: 38px; border-radius: 50%; border: 2px solid var(--danger); + background: var(--danger-soft); color: var(--danger); font-size: 15px; line-height: 1; + cursor: pointer; transition: background .15s, transform .1s; +} +.recbtn:hover { background: var(--danger); color: #fff; } +.recbtn.rec { background: var(--danger); color: #fff; animation: recpulse 1.4s ease-in-out infinite; } +@keyframes recpulse { 0%, 100% { box-shadow: 0 0 0 0 rgba(229, 89, 94, .45); } 50% { box-shadow: 0 0 0 9px rgba(229, 89, 94, 0); } } +.recbtn:disabled { opacity: .55; cursor: default; animation: none; } +.rec-ctl .rectime { font-size: 14px; font-variant-numeric: tabular-nums; color: var(--muted); min-width: 44px; } + +/* Namensmarker während der Aufnahme (Kopfzeile) */ +.rec-ctl .mkin { width: 210px; padding: 6px 10px; font-size: 13px; } +@media (max-width: 1100px) { .rec-ctl .mkin { width: 130px; } } + +/* Live-Platzhalter in der Aufnahmen-Liste */ +.item.rec-live { border-color: var(--danger); background: var(--danger-soft); } +.rec-dot { width: 10px; height: 10px; border-radius: 50%; background: var(--danger); flex: none; animation: recpulse 1.4s ease-in-out infinite; } +.item.rec-live .rectime { color: var(--danger); font-weight: 650; } +.mkchips { display: flex; flex-wrap: wrap; gap: 5px; margin-top: 6px; } +.mkchip { + display: inline-flex; align-items: center; gap: 5px; font-size: 12px; + background: var(--surface-2); border: 1px solid var(--line); border-radius: 999px; padding: 2px 4px 2px 8px; +} +.mkchip .mono { color: var(--muted); } +.mkdel { background: none; border: none; color: var(--muted); cursor: pointer; font-size: 12px; line-height: 1; padding: 2px 4px; } +.mkdel:hover { color: var(--danger); } + +/* Namensmarker-Liste im Detail */ +.markers { list-style: none; margin: 6px 0; padding: 0; } +.markers li { display: flex; align-items: center; gap: 7px; padding: 3px 0; font-size: 13px; } +.mkseek { + background: var(--surface-2); border: 1px solid var(--line); border-radius: 7px; + color: var(--accent); cursor: pointer; font-size: 12px; padding: 2px 7px; +} +.mkseek:hover { border-color: var(--accent); } +.mkadd-row { margin-top: 8px; } +.mkadd-row .input { flex: 1; min-width: 120px; } +.tab { + background: transparent; color: var(--muted); border: 1px solid transparent; + padding: 7px 14px; border-radius: 999px; cursor: pointer; font-size: 14px; + transition: color .12s, background .12s; +} +.tab:hover { color: var(--fg); background: var(--surface-2); } +.tab.active { color: var(--accent); background: var(--accent-soft); font-weight: 600; } +.btn.icon.theme { border-radius: 999px; font-size: 15px; line-height: 1; } + +/* ---- Bausteine ---- */ +.card { + background: var(--surface); border: 1px solid var(--line); border-radius: 14px; + padding: 18px; margin-bottom: 16px; box-shadow: var(--shadow); +} +h2 { margin: 0 0 12px; font-size: 18px; letter-spacing: -.01em; } +.row { display: flex; gap: 8px; align-items: center; margin-bottom: 8px; } +.grow { flex: 1; } +.btn { + background: var(--surface-2); color: var(--fg); border: 1px solid var(--line); + border-radius: 9px; padding: 8px 14px; cursor: pointer; font-size: 14px; + transition: border-color .12s, background .12s; +} +.btn:hover { border-color: var(--accent); } +.btn.primary { background: var(--accent); border-color: var(--accent); color: var(--accent-fg); font-weight: 600; } +.btn.primary:hover { filter: brightness(1.07); } +.btn.danger { background: var(--danger-soft); border-color: transparent; color: var(--danger); } +.btn.danger:hover { border-color: var(--danger); } +.btn.big { font-size: 17px; padding: 14px 22px; border-radius: 12px; } +.btn.small { font-size: .8rem; padding: 3px 8px; } +.btn.icon { padding: 6px 9px; flex: none; } +.btn:disabled { opacity: .55; cursor: default; } +:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; } +.lbl { display: block; font-size: 12px; color: var(--muted); margin: 10px 0 4px; text-transform: uppercase; letter-spacing: .04em; } +.input, select { + width: 100%; background: var(--inset); color: var(--fg); + border: 1px solid var(--line); border-radius: 9px; padding: 9px 12px; font-size: 14px; +} +.input:focus, select:focus { border-color: var(--accent); outline: none; } +.list { display: flex; flex-direction: column; gap: 8px; } +.item { + display: flex; gap: 10px; align-items: center; + background: var(--inset); border: 1px solid var(--line); border-radius: 10px; padding: 10px 12px; +} +.badge { font-size: 11px; padding: 3px 8px; border-radius: 20px; background: var(--surface-2); color: var(--muted); } +.badge.ok { background: var(--ok-soft); color: var(--ok); } +.badge.warn { background: var(--warn-soft); color: var(--warn); } +.muted { color: var(--muted); } +.mono { font-family: ui-monospace, monospace; font-size: 12px; } +.err { color: var(--danger); } +.err-inline { color: var(--danger); margin-bottom: 6px; } +.hint { font-size: .8rem; color: var(--muted); margin-bottom: 10px; } + +/* ---- Aufnahme ---- */ +.rec-panel { display: flex; flex-direction: column; align-items: center; gap: 14px; margin: 22px 0 8px; } +.timer { font-size: 40px; font-variant-numeric: tabular-nums; font-weight: 600; } +audio { width: 100%; margin-bottom: 10px; } + +/* ---- Transkript ---- */ +.transcript { line-height: 2.1; font-size: 16px; } +.transcript .w { cursor: pointer; padding: 1px 2px; border-radius: 4px; transition: background .1s; } +.transcript .w:hover { background: var(--surface-2); } +.transcript .w.active, .w.active { background: var(--accent); color: var(--accent-fg); } +.w.sp0 { color: var(--sp0-w); } .w.sp1 { color: var(--sp1-w); } +.w.sp2 { color: var(--sp2-w); } .w.sp3 { color: var(--sp3-w); } + +.toast { + position: fixed; bottom: 20px; left: 50%; transform: translateX(-50%); + background: var(--surface); border: 1px solid var(--accent); color: var(--fg); + padding: 10px 18px; border-radius: 10px; z-index: 99; max-width: 90%; box-shadow: var(--shadow); +} +.toast.err { border-color: var(--danger); } + +/* ---- Analyse ---- */ +.summary-snip { + font-size: .85rem; color: var(--muted); margin-top: 4px; overflow: hidden; + text-overflow: ellipsis; display: -webkit-box; -webkit-line-clamp: 2; -webkit-box-orient: vertical; +} +.analysis { margin: 10px 0; padding: 12px; background: var(--inset); border: 1px solid var(--line); border-radius: 10px; } +.analysis h3 { margin: 8px 0 4px; font-size: .9rem; color: var(--muted); } +.summary { margin: 0 0 6px; line-height: 1.55; white-space: pre-line; } +.todos { margin: 0; padding-left: 20px; } +.todos li { margin: 3px 0; } +.parts { margin: 0; padding-left: 20px; } .parts li { margin: 3px 0; } +.clean { line-height: 1.55; background: var(--inset); padding: 8px 10px; border-radius: 8px; margin: 0 0 4px; } + +/* ---- Dialog-Turns / Sprecher ---- */ +.dialog { display: flex; flex-direction: column; gap: 6px; margin-bottom: 4px; } +.turn { padding: 6px 10px; border-radius: 8px; background: var(--inset); line-height: 1.5; } +.turn .spk { margin-right: 6px; font-weight: 600; } +.turn.sp0 { border-left: 3px solid var(--sp0); } .turn.sp0 .spk { color: var(--sp0); } +.turn.sp1 { border-left: 3px solid var(--sp1); } .turn.sp1 .spk { color: var(--sp1); } +.turn.sp2 { border-left: 3px solid var(--sp2); } .turn.sp2 .spk { color: var(--sp2); } +.turn.sp3 { border-left: 3px solid var(--sp3); } .turn.sp3 .spk { color: var(--sp3); } + +.spk-edit { display: flex; flex-direction: column; gap: 6px; margin: 4px 0 8px; } +.spk-row { display: flex; align-items: center; gap: 8px; padding: 4px 8px; border-radius: 8px; background: var(--inset); border-left: 3px solid var(--line); } +.spk-row.sp0 { border-left-color: var(--sp0); } .spk-row.sp1 { border-left-color: var(--sp1); } +.spk-row.sp2 { border-left-color: var(--sp2); } .spk-row.sp3 { border-left-color: var(--sp3); } +.spk-dot { width: 10px; height: 10px; border-radius: 50%; background: var(--muted); flex: none; } +.spk-row.sp0 .spk-dot { background: var(--sp0); } .spk-row.sp1 .spk-dot { background: var(--sp1); } +.spk-row.sp2 .spk-dot { background: var(--sp2); } .spk-row.sp3 .spk-dot { background: var(--sp3); } +.spk-name { flex: 1; min-width: 80px; padding: 5px 8px; border-radius: 6px; border: 1px solid var(--line); background: var(--surface); color: var(--fg); } +.spk-learn-l { display: flex; align-items: center; gap: 4px; font-size: 12px; color: var(--muted); white-space: nowrap; } + +.ana-time { font-variant-numeric: tabular-nums; font-weight: 600; } +.ana-time.live { color: var(--accent); } +.ana-line { margin: 2px 0 8px; } + +/* ---- Aufnahmen: Liste links, Detail rechts ---- */ +.rec-layout { display: flex; gap: 8px; align-items: flex-start; } +.rec-splitter { + flex: 0 0 6px; align-self: stretch; cursor: col-resize; border-radius: 3px; + background: transparent; transition: background .12s; +} +.rec-splitter:hover, body.resizing .rec-splitter { background: var(--accent-soft); } +body.resizing { cursor: col-resize; user-select: none; } +.rec-list-pane { + flex: 0 0 340px; position: sticky; top: 12px; max-height: calc(100vh - 90px); + overflow-y: auto; background: var(--surface); border: 1px solid var(--line); + border-radius: 14px; padding: 10px; box-shadow: var(--shadow); +} +.rec-detail-pane { flex: 1 1 auto; min-width: 0; } +.rec-detail-pane .card { margin-bottom: 0; } +.detail-empty { padding: 40px 16px; text-align: center; border: 1px dashed var(--line); border-radius: 14px; } + +.day-group { margin-bottom: 6px; } +.day-head { + display: flex; align-items: center; gap: 6px; cursor: pointer; font-size: 12px; + color: var(--muted); text-transform: uppercase; letter-spacing: .03em; padding: 6px 4px; user-select: none; +} +.day-head:hover { color: var(--fg); } +.day-caret { display: inline-block; transition: transform .15s; font-size: 10px; } +.day-group.collapsed .day-caret { transform: rotate(-90deg); } +.day-group.collapsed .day-items { display: none; } +.day-count { margin-left: auto; background: var(--accent-soft); color: var(--accent); border-radius: 20px; padding: 1px 8px; font-size: 11px; font-weight: 650; } +.day-items { display: flex; flex-direction: column; gap: 6px; } + +.item.rec { cursor: pointer; transition: border-color .1s, background .1s; } +.item.rec:hover { border-color: var(--accent); } +.item.rec.sel { border-color: var(--accent); background: var(--accent-soft); } + +/* ---- Detail: Module links, Transkript als eigene Scroll-Spalte rechts ---- */ +.status-line { margin: 4px 0 10px; } +.detail-cols { display: flex; gap: 12px; align-items: flex-start; } +.mod-col { min-width: 0; display: flex; flex-direction: column; gap: 10px; min-height: 60px; } +.mod-col[data-col="main"] { flex: 1 1 54%; } +.mod-col[data-col="side"] { flex: 1 1 46%; } +.mod-col.empty { flex: 0 0 26px; border: 1px dashed var(--line); border-radius: 12px; align-self: stretch; } +.transcript-panel { max-height: calc(100vh - 90px); overflow-y: auto; position: sticky; top: 12px; } +.transcript-panel .mod-head { position: sticky; top: 0; z-index: 1; } +@media (max-width: 1100px) { + .detail-cols { flex-direction: column; } + .mod-col { width: 100%; } + .mod-col.empty { display: none; } + .transcript-panel { max-height: 60vh; position: static; } +} +.module { background: var(--inset); border: 1px solid var(--line); border-radius: 12px; overflow: hidden; } +.module.dragging { opacity: .45; border-style: dashed; } +.mod-head { + display: flex; align-items: center; gap: 8px; padding: 8px 12px; cursor: grab; + background: var(--surface-2); user-select: none; +} +.mod-head:active { cursor: grabbing; } +.mod-head h3 { margin: 0; font-size: .85rem; color: var(--muted); text-transform: uppercase; letter-spacing: .04em; } +.mod-grip { color: var(--muted); font-size: 14px; letter-spacing: -1px; } +.mod-body { padding: 12px; } +.mod-body h3 { margin: 10px 0 4px; font-size: .9rem; color: var(--muted); } + +/* ---- Sprecher-Verlauf: fester Streifen unter dem Player, volle Breite ---- */ +.viz-top { + position: relative; margin: 4px 0 10px; padding: 10px 12px; background: var(--inset); + border: 1px solid var(--line); border-radius: 12px; +} +.lane-label { cursor: pointer; } +.lane-label:hover { color: var(--fg); text-decoration: underline dotted; } +.spkpop { + position: absolute; left: 12px; z-index: 5; width: 270px; + background: var(--surface); border: 1px solid var(--accent); border-radius: 10px; + padding: 10px 12px; box-shadow: 0 6px 22px rgba(0, 0, 0, .35); +} +.spkpop .input { margin-bottom: 6px; } +.viz-wrap { display: flex; gap: 14px; align-items: stretch; } +.lanes { flex: 1 1 58%; min-width: 0; position: relative; display: flex; flex-direction: column; gap: 6px; cursor: pointer; } +.lane-row { display: flex; align-items: center; gap: 8px; } +.lane-label { + flex: 0 0 92px; display: flex; align-items: center; gap: 6px; font-size: 12px; + color: var(--muted); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; +} +.lane { position: relative; flex: 1; height: 22px; background: var(--surface-2); border-radius: 6px; overflow: hidden; } +.seg { position: absolute; top: 3px; bottom: 3px; border-radius: 3px; opacity: .75; transition: opacity .15s; } +.lane-row.sp0 .seg { background: var(--sp0); } .lane-row.sp1 .seg { background: var(--sp1); } +.lane-row.sp2 .seg { background: var(--sp2); } .lane-row.sp3 .seg { background: var(--sp3); } +.lane-row.sp0 .spk-dot { background: var(--sp0); } .lane-row.sp1 .spk-dot { background: var(--sp1); } +.lane-row.sp2 .spk-dot { background: var(--sp2); } .lane-row.sp3 .spk-dot { background: var(--sp3); } +.lane-row.speaking .seg { opacity: 1; } +.lane-row.speaking .lane-label { color: var(--fg); font-weight: 600; } +.lane-row.speaking .spk-dot { animation: pulse 1s ease-in-out infinite; } +@keyframes pulse { 0%, 100% { transform: scale(1); } 50% { transform: scale(1.55); } } +.playhead { + display: none; position: absolute; top: 0; bottom: 0; width: 2px; + background: var(--accent); border-radius: 1px; pointer-events: none; z-index: 2; + transition: left .25s linear; +} +.lanes { flex: 1 1 100%; } + +@media (max-width: 800px) { + #app { padding: 16px; } + .rec-layout { flex-direction: column; } + .rec-list-pane { flex: none !important; width: 100%; position: static; max-height: 340px; } + .rec-splitter { display: none; } + .topbar nav { margin-left: 0; width: 100%; order: 3; } +} + +/* ---- Konferenztisch-Ansicht ---- */ +.hidden { display: none !important; } +.tablescene { position: relative; width: 100%; aspect-ratio: 4 / 3; max-height: 420px; } +.ttable { + position: absolute; left: 50%; top: 50%; transform: translate(-50%, -50%); + width: 58%; height: 44%; border-radius: 18px; + background: var(--surface-2); border: 2px solid var(--line); + box-shadow: inset 0 0 0 6px var(--inset); +} +.tperson { position: absolute; transform: translate(-50%, -50%); display: flex; flex-direction: column; align-items: center; gap: 2px; width: 76px; } +.thead { + position: relative; width: 30px; height: 30px; border-radius: 50%; + background: var(--muted); z-index: 2; +} +.tperson.sp0 .thead, .tperson.sp0 .tbody { background: var(--sp0); } +.tperson.sp1 .thead, .tperson.sp1 .tbody { background: var(--sp1); } +.tperson.sp2 .thead, .tperson.sp2 .tbody { background: var(--sp2); } +.tperson.sp3 .thead, .tperson.sp3 .tbody { background: var(--sp3); } +.teyes { position: absolute; left: 0; right: 0; top: 9px; display: flex; justify-content: center; gap: 7px; transition: transform .25s; } +.teye { width: 5px; height: 5px; border-radius: 50%; background: #17130f; } +:root[data-theme="light"] .teye { background: #fff; } +.tmouth { + position: absolute; left: 50%; bottom: 6px; transform: translateX(-50%); + width: 9px; height: 3px; border-radius: 2px; background: rgba(0, 0, 0, .55); opacity: 0; +} +:root[data-theme="light"] .tmouth { background: rgba(255, 255, 255, .8); } +.tperson.talking .tmouth { opacity: 1; animation: talkmouth .35s ease-in-out infinite; } +@keyframes talkmouth { 0%, 100% { height: 2px; } 50% { height: 7px; } } +.tperson.talking .thead { box-shadow: 0 0 0 3px var(--accent-soft), 0 0 10px var(--accent-soft); } +.tbody { width: 46px; height: 20px; border-radius: 12px 12px 5px 5px; background: var(--muted); margin-top: -6px; opacity: .85; } +.tname { font-size: 11px; color: var(--muted); max-width: 76px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } +.tperson.talking .tname { color: var(--fg); font-weight: 650; } +.tbubble { + position: absolute; left: 50%; top: 50%; transform: translate(-50%, -50%); + width: 54%; z-index: 3; + background: var(--surface); border: 2px solid var(--line); border-radius: 14px; + padding: 8px 10px; font-size: 13px; line-height: 1.55; + transition: border-color .2s; +} +.tb-scroll { max-height: 150px; overflow-y: auto; } +.tbubble .tb-name { display: block; font-weight: 650; font-size: 11px; margin-bottom: 2px; position: sticky; top: 0; background: var(--surface); padding: 2px 0; z-index: 1; } +.tbubble .bw { cursor: pointer; padding: 1px 2px; border-radius: 4px; } +.tbubble .bw:hover { background: var(--surface-2); } +.tbubble .bw.active { background: var(--accent); color: var(--accent-fg); } +.tbubble.sp0 { border-color: var(--sp0); } .tbubble.sp0 .tb-name { color: var(--sp0); } +.tbubble.sp1 { border-color: var(--sp1); } .tbubble.sp1 .tb-name { color: var(--sp1); } +.tbubble.sp2 { border-color: var(--sp2); } .tbubble.sp2 .tb-name { color: var(--sp2); } +.tbubble.sp3 { border-color: var(--sp3); } .tbubble.sp3 .tb-name { color: var(--sp3); } +.tbubble::after { + content: ''; position: absolute; width: 12px; height: 12px; + background: inherit; border: inherit; border-width: 0 2px 2px 0; + display: none; +} +.tbubble[class*="arrow-"]::after { display: block; } +.tbubble.arrow-top::after { top: -7px; left: var(--ax, 50%); transform: translateX(-50%) rotate(-135deg); } +.tbubble.arrow-bottom::after { bottom: -7px; left: var(--ax, 50%); transform: translateX(-50%) rotate(45deg); } +.tbubble.arrow-left::after { left: -7px; top: 50%; transform: translateY(-50%) rotate(135deg); } +.tbubble.arrow-right::after { right: -7px; top: 50%; transform: translateY(-50%) rotate(-45deg); } +.tidle { position: absolute; left: 50%; top: 50%; transform: translate(-50%, -50%); font-size: 13px; text-align: center; } +#viewswitch svg { display: block; } + +@media (prefers-reduced-motion: reduce) { + * { transition: none !important; animation: none !important; } +} -:root[data-theme="dark"] { - --bg: #14161b; - --surface: #1d222a; - --surface-2: #262c36; - --inset: #10141a; - --line: #303845; - --fg: #e9ecf1; - --muted: #98a2b3; - --accent: #e8804c; - --accent-fg: #17130f; - --accent-soft: rgba(232, 128, 76, .14); - --ok: #55b381; - --ok-soft: rgba(85, 179, 129, .14); - --warn: #e2b93b; - --warn-soft: rgba(226, 185, 59, .14); - --danger: #e5595e; - --danger-soft: rgba(229, 89, 94, .12); - --shadow: none; - --sp0: #5bc0eb; --sp1: #ef8fc0; --sp2: #a8cc4e; --sp3: #e6b84c; - --sp0-w: #8fd6f5; --sp1-w: #f5b5d6; --sp2-w: #c5e07e; --sp3-w: #f0d08a; -} -:root[data-theme="light"] { - --bg: #f4f2ee; - --surface: #ffffff; - --surface-2: #eceae4; - --inset: #f7f6f2; - --line: #ddd8cf; - --fg: #23272e; - --muted: #6d7683; - --accent: #c05f2d; - --accent-fg: #ffffff; - --accent-soft: rgba(192, 95, 45, .1); - --ok: #2f8a5d; - --ok-soft: rgba(47, 138, 93, .12); - --warn: #9a7514; - --warn-soft: rgba(154, 117, 20, .12); - --danger: #c73e43; - --danger-soft: rgba(199, 62, 67, .09); - --shadow: 0 1px 3px rgba(35, 39, 46, .07); - --sp0: #1179a8; --sp1: #c2417f; --sp2: #5f7d16; --sp3: #a4700d; - --sp0-w: #0e6b95; --sp1-w: #ad3a72; --sp2-w: #556f14; --sp3-w: #92640b; -} +/* ========================================================================== + Suche — Google-Anmutung, aber in der Aime-Palette: viel Weißraum, eine + dominante Eingabezeile, Treffer als Textblöcke ohne Karten-Rahmen. + ========================================================================== */ +.search-page { max-width: 46rem; margin: 0 auto; padding: 1rem 1rem 4rem; } +/* Startzustand: Suchfeld optisch mittig, wie eine leere Suchseite. */ +.search-page.landing { padding-top: 12vh; } +.search-page.has-results { padding-top: 1.25rem; } -* { box-sizing: border-box; } +.search-box { + display: flex; align-items: center; gap: .5rem; + border: 1px solid var(--line); border-radius: 2rem; + background: var(--surface); box-shadow: var(--shadow); + padding: .35rem .35rem .35rem 1rem; + transition: box-shadow .15s, border-color .15s; +} +.search-box:focus-within { border-color: var(--accent); box-shadow: 0 0 0 3px var(--accent-soft); } +.search-ico { opacity: .55; font-size: 1rem; } +.search-input { + flex: 1; border: 0; outline: 0; background: transparent; color: var(--fg); + font: inherit; font-size: 1.05rem; padding: .55rem 0; +} +.search-clear { border: 0; background: transparent; color: var(--muted); cursor: pointer; font-size: .95rem; padding: .25rem .4rem; } +.search-clear:hover { color: var(--fg); } +.search-go { + border: 0; border-radius: 1.6rem; cursor: pointer; + background: var(--accent); color: var(--accent-fg); + font: inherit; font-weight: 600; padding: .55rem 1.1rem; +} +.search-go:hover { filter: brightness(1.07); } -/* Scrollbalken im Logo-Kupfer (Firefox: scrollbar-color; Chromium/WebKit: ::-webkit-*) */ -* { scrollbar-width: thin; scrollbar-color: var(--accent) transparent; } -::-webkit-scrollbar { width: 10px; height: 10px; } -::-webkit-scrollbar-track { background: transparent; } -::-webkit-scrollbar-thumb { - background: var(--accent); border-radius: 5px; - border: 2px solid transparent; background-clip: padding-box; +.search-hints { margin-top: 1.5rem; display: flex; flex-direction: column; gap: .5rem; align-items: flex-start; } +.search-hints .example { + border: 1px solid var(--line); border-radius: 1rem; background: var(--surface-2); + color: var(--fg); font: inherit; font-size: .88rem; padding: .35rem .8rem; cursor: pointer; text-align: left; } -::-webkit-scrollbar-thumb:hover { background: var(--accent); filter: brightness(1.15); } -::-webkit-scrollbar-corner { background: transparent; } -html { color-scheme: dark; } -:root[data-theme="light"] { color-scheme: light; } -body { - margin: 0; background: var(--bg); color: var(--fg); - font: 15px/1.5 system-ui, "Segoe UI", Roboto, sans-serif; - transition: background .2s, color .2s; -} -/* Volle Fensterbreite für ALLE Tabs. Früher war nur „Aufnahmen“ breit (.wide) - und der Rest auf 860px zentriert — dadurch sprang die Kopfzeile beim - Tabwechsel in die Mitte. */ -#app { max-width: none; margin: 0 auto; padding: 16px 28px; } +.search-hints .example:hover { border-color: var(--accent); color: var(--accent); } -/* ---- Topbar / Brand ---- */ -.topbar { display: flex; align-items: center; gap: 16px; flex-wrap: wrap; padding: 6px 0 14px; } -.brand { display: flex; align-items: center; gap: 10px; } -.brand-wave { width: 30px; height: 18px; color: var(--accent); flex: none; } -.topbar h1 { margin: 0; font-size: 21px; font-weight: 650; letter-spacing: -.01em; } -.topbar .sub { font-size: 12px; color: var(--muted); font-weight: 400; margin-left: 6px; } -nav { display: flex; gap: 6px; flex-wrap: wrap; } -.topbar nav { margin-left: 0; } +.searching { padding: 1.5rem 0; } +.search-empty { padding: 1.5rem 0; } -/* Aufnahme-Steuerung in der Kopfzeile (zwischen Logo und Tabs) */ -.rec-ctl { display: flex; align-items: center; gap: 10px; margin: 0 auto; } -.recbtn { - width: 38px; height: 38px; border-radius: 50%; border: 2px solid var(--danger); - background: var(--danger-soft); color: var(--danger); font-size: 15px; line-height: 1; - cursor: pointer; transition: background .15s, transform .1s; +/* Antwortkasten: abgesetzt wie eine KI-Übersicht, nicht wie ein Treffer. */ +.answer-box { + margin: 1.25rem 0 .5rem; padding: .9rem 1rem; + background: var(--accent-soft); border: 1px solid var(--line); border-radius: .75rem; } -.recbtn:hover { background: var(--danger); color: #fff; } -.recbtn.rec { background: var(--danger); color: #fff; animation: recpulse 1.4s ease-in-out infinite; } -@keyframes recpulse { 0%, 100% { box-shadow: 0 0 0 0 rgba(229, 89, 94, .45); } 50% { box-shadow: 0 0 0 9px rgba(229, 89, 94, 0); } } -.recbtn:disabled { opacity: .55; cursor: default; animation: none; } -.rec-ctl .rectime { font-size: 14px; font-variant-numeric: tabular-nums; color: var(--muted); min-width: 44px; } +.answer-head { font-weight: 600; font-size: .82rem; letter-spacing: .04em; text-transform: uppercase; color: var(--muted); margin-bottom: .5rem; } +.answer-body { line-height: 1.55; white-space: pre-wrap; } +/* Platzhalterbalken, solange das LLM rechnet (30-90 s sind hier normal). */ +.answer-body.skeleton { display: flex; flex-direction: column; gap: .45rem; } +.answer-body.skeleton span { + height: .7rem; border-radius: .35rem; background: var(--line); opacity: .55; + animation: pulse 1.4s ease-in-out infinite; +} +.answer-body.skeleton span:nth-child(1) { width: 92%; } +.answer-body.skeleton span:nth-child(2) { width: 78%; animation-delay: .18s; } +.answer-body.skeleton span:nth-child(3) { width: 45%; animation-delay: .36s; } +@keyframes pulse { 0%, 100% { opacity: .3; } 50% { opacity: .7; } } +.answer-src { margin-top: .7rem; display: flex; flex-wrap: wrap; gap: .4rem; } +.src-chip { + border: 1px solid var(--line); background: var(--surface); color: var(--fg); + border-radius: 1rem; font: inherit; font-size: .8rem; padding: .2rem .6rem; cursor: pointer; +} +.src-chip:hover { border-color: var(--accent); color: var(--accent); } -/* Namensmarker während der Aufnahme (Kopfzeile) */ -.rec-ctl .mkin { width: 210px; padding: 6px 10px; font-size: 13px; } -@media (max-width: 1100px) { .rec-ctl .mkin { width: 130px; } } +.result-meta { margin: .9rem 0 .3rem; } +.results { list-style: none; margin: 0; padding: 0; } +.result { padding: .85rem 0; cursor: pointer; border-bottom: 1px solid var(--line); } +.result:hover .result-title { text-decoration: underline; } +.result-src { color: var(--muted); font-size: .78rem; margin-bottom: .15rem; } +.result-title { display: block; color: var(--accent); font-size: 1.05rem; text-decoration: none; } +.result-at { color: var(--muted); font-size: .8rem; font-weight: 400; margin-left: .4rem; } +.result-snip { color: var(--fg); font-size: .9rem; line-height: 1.5; margin-top: .2rem; } +.result-snip mark { background: var(--warn-soft); color: inherit; font-weight: 600; padding: 0 .1em; border-radius: .15em; } -/* Live-Platzhalter in der Aufnahmen-Liste */ -.item.rec-live { border-color: var(--danger); background: var(--danger-soft); } -.rec-dot { width: 10px; height: 10px; border-radius: 50%; background: var(--danger); flex: none; animation: recpulse 1.4s ease-in-out infinite; } -.item.rec-live .rectime { color: var(--danger); font-weight: 650; } -.mkchips { display: flex; flex-wrap: wrap; gap: 5px; margin-top: 6px; } -.mkchip { - display: inline-flex; align-items: center; gap: 5px; font-size: 12px; - background: var(--surface-2); border: 1px solid var(--line); border-radius: 999px; padding: 2px 4px 2px 8px; -} -.mkchip .mono { color: var(--muted); } -.mkdel { background: none; border: none; color: var(--muted); cursor: pointer; font-size: 12px; line-height: 1; padding: 2px 4px; } -.mkdel:hover { color: var(--danger); } +/* ========================================================================== + Papierkorb + Umbenennen + ========================================================================== */ +.list-bar { display: flex; justify-content: flex-end; padding: .35rem .5rem; border-bottom: 1px solid var(--line); } +.btn.ghost { background: transparent; border-color: transparent; color: var(--muted); } +.btn.ghost:hover { color: var(--fg); border-color: var(--line); } +.btn.ghost.active { color: var(--accent); border-color: var(--accent); background: var(--accent-soft); } +.trash-hint, .trash-empty { padding: .6rem .75rem; } +.item.rec.trashed { opacity: .8; } +.item.rec.trashed b { text-decoration: line-through; text-decoration-color: var(--muted); } +.days-left { color: var(--muted); } +.days-left.urgent { color: var(--danger); font-weight: 600; } +/* Der Stift sitzt neben der H2 des Detailkopfs, ohne die Zeile umzubrechen. */ +.rename-btn { vertical-align: middle; margin-left: -.25rem; opacity: .55; } +.rename-btn:hover { opacity: 1; } +.item.rec [data-ren] { opacity: .45; } +.item.rec:hover [data-ren] { opacity: 1; } -/* Namensmarker-Liste im Detail */ -.markers { list-style: none; margin: 6px 0; padding: 0; } -.markers li { display: flex; align-items: center; gap: 7px; padding: 3px 0; font-size: 13px; } -.mkseek { - background: var(--surface-2); border: 1px solid var(--line); border-radius: 7px; - color: var(--accent); cursor: pointer; font-size: 12px; padding: 2px 7px; -} -.mkseek:hover { border-color: var(--accent); } -.mkadd-row { margin-top: 8px; } -.mkadd-row .input { flex: 1; min-width: 120px; } -.tab { - background: transparent; color: var(--muted); border: 1px solid transparent; - padding: 7px 14px; border-radius: 999px; cursor: pointer; font-size: 14px; - transition: color .12s, background .12s; -} -.tab:hover { color: var(--fg); background: var(--surface-2); } -.tab.active { color: var(--accent); background: var(--accent-soft); font-weight: 600; } -.btn.icon.theme { border-radius: 999px; font-size: 15px; line-height: 1; } - -/* ---- Bausteine ---- */ -.card { - background: var(--surface); border: 1px solid var(--line); border-radius: 14px; - padding: 18px; margin-bottom: 16px; box-shadow: var(--shadow); -} -h2 { margin: 0 0 12px; font-size: 18px; letter-spacing: -.01em; } -.row { display: flex; gap: 8px; align-items: center; margin-bottom: 8px; } -.grow { flex: 1; } -.btn { - background: var(--surface-2); color: var(--fg); border: 1px solid var(--line); - border-radius: 9px; padding: 8px 14px; cursor: pointer; font-size: 14px; - transition: border-color .12s, background .12s; -} -.btn:hover { border-color: var(--accent); } -.btn.primary { background: var(--accent); border-color: var(--accent); color: var(--accent-fg); font-weight: 600; } -.btn.primary:hover { filter: brightness(1.07); } -.btn.danger { background: var(--danger-soft); border-color: transparent; color: var(--danger); } -.btn.danger:hover { border-color: var(--danger); } -.btn.big { font-size: 17px; padding: 14px 22px; border-radius: 12px; } -.btn.small { font-size: .8rem; padding: 3px 8px; } -.btn.icon { padding: 6px 9px; flex: none; } -.btn:disabled { opacity: .55; cursor: default; } -:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; } -.lbl { display: block; font-size: 12px; color: var(--muted); margin: 10px 0 4px; text-transform: uppercase; letter-spacing: .04em; } -.input, select { - width: 100%; background: var(--inset); color: var(--fg); - border: 1px solid var(--line); border-radius: 9px; padding: 9px 12px; font-size: 14px; -} -.input:focus, select:focus { border-color: var(--accent); outline: none; } -.list { display: flex; flex-direction: column; gap: 8px; } -.item { - display: flex; gap: 10px; align-items: center; - background: var(--inset); border: 1px solid var(--line); border-radius: 10px; padding: 10px 12px; -} -.badge { font-size: 11px; padding: 3px 8px; border-radius: 20px; background: var(--surface-2); color: var(--muted); } -.badge.ok { background: var(--ok-soft); color: var(--ok); } -.badge.warn { background: var(--warn-soft); color: var(--warn); } -.muted { color: var(--muted); } -.mono { font-family: ui-monospace, monospace; font-size: 12px; } -.err { color: var(--danger); } -.err-inline { color: var(--danger); margin-bottom: 6px; } -.hint { font-size: .8rem; color: var(--muted); margin-bottom: 10px; } - -/* ---- Aufnahme ---- */ -.rec-panel { display: flex; flex-direction: column; align-items: center; gap: 14px; margin: 22px 0 8px; } -.timer { font-size: 40px; font-variant-numeric: tabular-nums; font-weight: 600; } -audio { width: 100%; margin-bottom: 10px; } - -/* ---- Transkript ---- */ -.transcript { line-height: 2.1; font-size: 16px; } -.transcript .w { cursor: pointer; padding: 1px 2px; border-radius: 4px; transition: background .1s; } -.transcript .w:hover { background: var(--surface-2); } -.transcript .w.active, .w.active { background: var(--accent); color: var(--accent-fg); } -.w.sp0 { color: var(--sp0-w); } .w.sp1 { color: var(--sp1-w); } -.w.sp2 { color: var(--sp2-w); } .w.sp3 { color: var(--sp3-w); } - -.toast { - position: fixed; bottom: 20px; left: 50%; transform: translateX(-50%); - background: var(--surface); border: 1px solid var(--accent); color: var(--fg); - padding: 10px 18px; border-radius: 10px; z-index: 99; max-width: 90%; box-shadow: var(--shadow); -} -.toast.err { border-color: var(--danger); } - -/* ---- Analyse ---- */ -.summary-snip { - font-size: .85rem; color: var(--muted); margin-top: 4px; overflow: hidden; - text-overflow: ellipsis; display: -webkit-box; -webkit-line-clamp: 2; -webkit-box-orient: vertical; -} -.analysis { margin: 10px 0; padding: 12px; background: var(--inset); border: 1px solid var(--line); border-radius: 10px; } -.analysis h3 { margin: 8px 0 4px; font-size: .9rem; color: var(--muted); } -.summary { margin: 0 0 6px; line-height: 1.55; white-space: pre-line; } -.todos { margin: 0; padding-left: 20px; } -.todos li { margin: 3px 0; } -.parts { margin: 0; padding-left: 20px; } .parts li { margin: 3px 0; } -.clean { line-height: 1.55; background: var(--inset); padding: 8px 10px; border-radius: 8px; margin: 0 0 4px; } - -/* ---- Dialog-Turns / Sprecher ---- */ -.dialog { display: flex; flex-direction: column; gap: 6px; margin-bottom: 4px; } -.turn { padding: 6px 10px; border-radius: 8px; background: var(--inset); line-height: 1.5; } -.turn .spk { margin-right: 6px; font-weight: 600; } -.turn.sp0 { border-left: 3px solid var(--sp0); } .turn.sp0 .spk { color: var(--sp0); } -.turn.sp1 { border-left: 3px solid var(--sp1); } .turn.sp1 .spk { color: var(--sp1); } -.turn.sp2 { border-left: 3px solid var(--sp2); } .turn.sp2 .spk { color: var(--sp2); } -.turn.sp3 { border-left: 3px solid var(--sp3); } .turn.sp3 .spk { color: var(--sp3); } - -.spk-edit { display: flex; flex-direction: column; gap: 6px; margin: 4px 0 8px; } -.spk-row { display: flex; align-items: center; gap: 8px; padding: 4px 8px; border-radius: 8px; background: var(--inset); border-left: 3px solid var(--line); } -.spk-row.sp0 { border-left-color: var(--sp0); } .spk-row.sp1 { border-left-color: var(--sp1); } -.spk-row.sp2 { border-left-color: var(--sp2); } .spk-row.sp3 { border-left-color: var(--sp3); } -.spk-dot { width: 10px; height: 10px; border-radius: 50%; background: var(--muted); flex: none; } -.spk-row.sp0 .spk-dot { background: var(--sp0); } .spk-row.sp1 .spk-dot { background: var(--sp1); } -.spk-row.sp2 .spk-dot { background: var(--sp2); } .spk-row.sp3 .spk-dot { background: var(--sp3); } -.spk-name { flex: 1; min-width: 80px; padding: 5px 8px; border-radius: 6px; border: 1px solid var(--line); background: var(--surface); color: var(--fg); } -.spk-learn-l { display: flex; align-items: center; gap: 4px; font-size: 12px; color: var(--muted); white-space: nowrap; } - -.ana-time { font-variant-numeric: tabular-nums; font-weight: 600; } -.ana-time.live { color: var(--accent); } -.ana-line { margin: 2px 0 8px; } - -/* ---- Aufnahmen: Liste links, Detail rechts ---- */ -.rec-layout { display: flex; gap: 8px; align-items: flex-start; } -.rec-splitter { - flex: 0 0 6px; align-self: stretch; cursor: col-resize; border-radius: 3px; - background: transparent; transition: background .12s; -} -.rec-splitter:hover, body.resizing .rec-splitter { background: var(--accent-soft); } -body.resizing { cursor: col-resize; user-select: none; } -.rec-list-pane { - flex: 0 0 340px; position: sticky; top: 12px; max-height: calc(100vh - 90px); - overflow-y: auto; background: var(--surface); border: 1px solid var(--line); - border-radius: 14px; padding: 10px; box-shadow: var(--shadow); -} -.rec-detail-pane { flex: 1 1 auto; min-width: 0; } -.rec-detail-pane .card { margin-bottom: 0; } -.detail-empty { padding: 40px 16px; text-align: center; border: 1px dashed var(--line); border-radius: 14px; } - -.day-group { margin-bottom: 6px; } -.day-head { - display: flex; align-items: center; gap: 6px; cursor: pointer; font-size: 12px; - color: var(--muted); text-transform: uppercase; letter-spacing: .03em; padding: 6px 4px; user-select: none; -} -.day-head:hover { color: var(--fg); } -.day-caret { display: inline-block; transition: transform .15s; font-size: 10px; } -.day-group.collapsed .day-caret { transform: rotate(-90deg); } -.day-group.collapsed .day-items { display: none; } -.day-count { margin-left: auto; background: var(--accent-soft); color: var(--accent); border-radius: 20px; padding: 1px 8px; font-size: 11px; font-weight: 650; } -.day-items { display: flex; flex-direction: column; gap: 6px; } - -.item.rec { cursor: pointer; transition: border-color .1s, background .1s; } -.item.rec:hover { border-color: var(--accent); } -.item.rec.sel { border-color: var(--accent); background: var(--accent-soft); } - -/* ---- Detail: Module links, Transkript als eigene Scroll-Spalte rechts ---- */ -.status-line { margin: 4px 0 10px; } -.detail-cols { display: flex; gap: 12px; align-items: flex-start; } -.mod-col { min-width: 0; display: flex; flex-direction: column; gap: 10px; min-height: 60px; } -.mod-col[data-col="main"] { flex: 1 1 54%; } -.mod-col[data-col="side"] { flex: 1 1 46%; } -.mod-col.empty { flex: 0 0 26px; border: 1px dashed var(--line); border-radius: 12px; align-self: stretch; } -.transcript-panel { max-height: calc(100vh - 90px); overflow-y: auto; position: sticky; top: 12px; } -.transcript-panel .mod-head { position: sticky; top: 0; z-index: 1; } -@media (max-width: 1100px) { - .detail-cols { flex-direction: column; } - .mod-col { width: 100%; } - .mod-col.empty { display: none; } - .transcript-panel { max-height: 60vh; position: static; } -} -.module { background: var(--inset); border: 1px solid var(--line); border-radius: 12px; overflow: hidden; } -.module.dragging { opacity: .45; border-style: dashed; } -.mod-head { - display: flex; align-items: center; gap: 8px; padding: 8px 12px; cursor: grab; - background: var(--surface-2); user-select: none; -} -.mod-head:active { cursor: grabbing; } -.mod-head h3 { margin: 0; font-size: .85rem; color: var(--muted); text-transform: uppercase; letter-spacing: .04em; } -.mod-grip { color: var(--muted); font-size: 14px; letter-spacing: -1px; } -.mod-body { padding: 12px; } -.mod-body h3 { margin: 10px 0 4px; font-size: .9rem; color: var(--muted); } - -/* ---- Sprecher-Verlauf: fester Streifen unter dem Player, volle Breite ---- */ -.viz-top { - position: relative; margin: 4px 0 10px; padding: 10px 12px; background: var(--inset); - border: 1px solid var(--line); border-radius: 12px; -} -.lane-label { cursor: pointer; } -.lane-label:hover { color: var(--fg); text-decoration: underline dotted; } -.spkpop { - position: absolute; left: 12px; z-index: 5; width: 270px; - background: var(--surface); border: 1px solid var(--accent); border-radius: 10px; - padding: 10px 12px; box-shadow: 0 6px 22px rgba(0, 0, 0, .35); -} -.spkpop .input { margin-bottom: 6px; } -.viz-wrap { display: flex; gap: 14px; align-items: stretch; } -.lanes { flex: 1 1 58%; min-width: 0; position: relative; display: flex; flex-direction: column; gap: 6px; cursor: pointer; } -.lane-row { display: flex; align-items: center; gap: 8px; } -.lane-label { - flex: 0 0 92px; display: flex; align-items: center; gap: 6px; font-size: 12px; - color: var(--muted); white-space: nowrap; overflow: hidden; text-overflow: ellipsis; -} -.lane { position: relative; flex: 1; height: 22px; background: var(--surface-2); border-radius: 6px; overflow: hidden; } -.seg { position: absolute; top: 3px; bottom: 3px; border-radius: 3px; opacity: .75; transition: opacity .15s; } -.lane-row.sp0 .seg { background: var(--sp0); } .lane-row.sp1 .seg { background: var(--sp1); } -.lane-row.sp2 .seg { background: var(--sp2); } .lane-row.sp3 .seg { background: var(--sp3); } -.lane-row.sp0 .spk-dot { background: var(--sp0); } .lane-row.sp1 .spk-dot { background: var(--sp1); } -.lane-row.sp2 .spk-dot { background: var(--sp2); } .lane-row.sp3 .spk-dot { background: var(--sp3); } -.lane-row.speaking .seg { opacity: 1; } -.lane-row.speaking .lane-label { color: var(--fg); font-weight: 600; } -.lane-row.speaking .spk-dot { animation: pulse 1s ease-in-out infinite; } -@keyframes pulse { 0%, 100% { transform: scale(1); } 50% { transform: scale(1.55); } } -.playhead { - display: none; position: absolute; top: 0; bottom: 0; width: 2px; - background: var(--accent); border-radius: 1px; pointer-events: none; z-index: 2; - transition: left .25s linear; -} -.lanes { flex: 1 1 100%; } - -@media (max-width: 800px) { - #app { padding: 16px; } - .rec-layout { flex-direction: column; } - .rec-list-pane { flex: none !important; width: 100%; position: static; max-height: 340px; } - .rec-splitter { display: none; } - .topbar nav { margin-left: 0; width: 100%; order: 3; } -} - -/* ---- Konferenztisch-Ansicht ---- */ -.hidden { display: none !important; } -.tablescene { position: relative; width: 100%; aspect-ratio: 4 / 3; max-height: 420px; } -.ttable { - position: absolute; left: 50%; top: 50%; transform: translate(-50%, -50%); - width: 58%; height: 44%; border-radius: 18px; - background: var(--surface-2); border: 2px solid var(--line); - box-shadow: inset 0 0 0 6px var(--inset); -} -.tperson { position: absolute; transform: translate(-50%, -50%); display: flex; flex-direction: column; align-items: center; gap: 2px; width: 76px; } -.thead { - position: relative; width: 30px; height: 30px; border-radius: 50%; - background: var(--muted); z-index: 2; -} -.tperson.sp0 .thead, .tperson.sp0 .tbody { background: var(--sp0); } -.tperson.sp1 .thead, .tperson.sp1 .tbody { background: var(--sp1); } -.tperson.sp2 .thead, .tperson.sp2 .tbody { background: var(--sp2); } -.tperson.sp3 .thead, .tperson.sp3 .tbody { background: var(--sp3); } -.teyes { position: absolute; left: 0; right: 0; top: 9px; display: flex; justify-content: center; gap: 7px; transition: transform .25s; } -.teye { width: 5px; height: 5px; border-radius: 50%; background: #17130f; } -:root[data-theme="light"] .teye { background: #fff; } -.tmouth { - position: absolute; left: 50%; bottom: 6px; transform: translateX(-50%); - width: 9px; height: 3px; border-radius: 2px; background: rgba(0, 0, 0, .55); opacity: 0; -} -:root[data-theme="light"] .tmouth { background: rgba(255, 255, 255, .8); } -.tperson.talking .tmouth { opacity: 1; animation: talkmouth .35s ease-in-out infinite; } -@keyframes talkmouth { 0%, 100% { height: 2px; } 50% { height: 7px; } } -.tperson.talking .thead { box-shadow: 0 0 0 3px var(--accent-soft), 0 0 10px var(--accent-soft); } -.tbody { width: 46px; height: 20px; border-radius: 12px 12px 5px 5px; background: var(--muted); margin-top: -6px; opacity: .85; } -.tname { font-size: 11px; color: var(--muted); max-width: 76px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } -.tperson.talking .tname { color: var(--fg); font-weight: 650; } -.tbubble { - position: absolute; left: 50%; top: 50%; transform: translate(-50%, -50%); - width: 54%; z-index: 3; - background: var(--surface); border: 2px solid var(--line); border-radius: 14px; - padding: 8px 10px; font-size: 13px; line-height: 1.55; - transition: border-color .2s; -} -.tb-scroll { max-height: 150px; overflow-y: auto; } -.tbubble .tb-name { display: block; font-weight: 650; font-size: 11px; margin-bottom: 2px; position: sticky; top: 0; background: var(--surface); padding: 2px 0; z-index: 1; } -.tbubble .bw { cursor: pointer; padding: 1px 2px; border-radius: 4px; } -.tbubble .bw:hover { background: var(--surface-2); } -.tbubble .bw.active { background: var(--accent); color: var(--accent-fg); } -.tbubble.sp0 { border-color: var(--sp0); } .tbubble.sp0 .tb-name { color: var(--sp0); } -.tbubble.sp1 { border-color: var(--sp1); } .tbubble.sp1 .tb-name { color: var(--sp1); } -.tbubble.sp2 { border-color: var(--sp2); } .tbubble.sp2 .tb-name { color: var(--sp2); } -.tbubble.sp3 { border-color: var(--sp3); } .tbubble.sp3 .tb-name { color: var(--sp3); } -.tbubble::after { - content: ''; position: absolute; width: 12px; height: 12px; - background: inherit; border: inherit; border-width: 0 2px 2px 0; - display: none; -} -.tbubble[class*="arrow-"]::after { display: block; } -.tbubble.arrow-top::after { top: -7px; left: var(--ax, 50%); transform: translateX(-50%) rotate(-135deg); } -.tbubble.arrow-bottom::after { bottom: -7px; left: var(--ax, 50%); transform: translateX(-50%) rotate(45deg); } -.tbubble.arrow-left::after { left: -7px; top: 50%; transform: translateY(-50%) rotate(135deg); } -.tbubble.arrow-right::after { right: -7px; top: 50%; transform: translateY(-50%) rotate(-45deg); } -.tidle { position: absolute; left: 50%; top: 50%; transform: translate(-50%, -50%); font-size: 13px; text-align: center; } -#viewswitch svg { display: block; } - -@media (prefers-reduced-motion: reduce) { - * { transition: none !important; animation: none !important; } -} +@media (max-width: 640px) { + .search-page.landing { padding-top: 6vh; } + .search-go { padding: .55rem .8rem; } +} \ No newline at end of file diff --git a/docs/08-changelog.md b/docs/08-changelog.md index 8387ae7..d845812 100644 --- a/docs/08-changelog.md +++ b/docs/08-changelog.md @@ -3,6 +3,78 @@ Chronik der Entwicklungsschritte. **Regel: jede zukünftige Änderung bekommt hier einen Eintrag** (Datum, Was, Warum, betroffene Dateien/Backups). +## 2026-07-31 — Inhaltssuche, Papierkorb, Titel umbenennen (+ WLAN-Ausfall behoben) + +Drei Funktionen auf Wunsch des Nutzers, dazu die Störung vom Vormittag. Details: +[09 – Suche & Verwaltung](09-suche-und-verwaltung.md). + +### Vorab: Board war nicht erreichbar — SSID-Rename killt den PSK-Hash + +Das Board lief weiter als USB-Soundkarte, war aber im Netz unsichtbar. `wpa_supplicant` hing auf +`ssid="RupuWolf5"`; das WLAN heißt seit dem Zusammenlegen der Bänder nur noch **`RupuWolf`** (4 BSSIDs, +2437/2462/5300/5500 MHz). Nach dem SSID-Fix scheiterte die Anmeldung **trotz unverändertem Passwort**: + +``` +wlP4p65s0: WPA: 4-Way Handshake failed - pre-shared key may be incorrect +CTRL-EVENT-SSID-TEMP-DISABLED ssid="RupuWolf" auth_failures=1 reason=WRONG_KEY +``` + +Ursache: die Conf speichert `psk=<64 Hex>`, und **WPA-PSK leitet diesen Hash per PBKDF2 mit der SSID als +Salt ab** — ein SSID-Rename entwertet jeden vorberechneten Hash. Fix: Hash aus dem Klartext der +`#psk="…"`-Kommentarzeile neu rechnen (`wpa_passphrase "RupuWolf" "$PLAIN"`). +Backups: `.bak-rupuwolf5` (alte SSID), `.bak-oldhash`. + +Außerdem: Hostname auf **`AIme`** gesetzt, `avahi-daemon` installiert → **`AIme.local`** löst per mDNS auf +und ersetzt das IP-Suchen. Die FritzBox zeigt weiter `armbian-sd`, weil sie den Gerätenamen beim ersten +DHCP-Kontakt einfriert; ein Mitschnitt belegt, dass das Board korrekt `Hostname (12): "AIme"` sendet. + +### Titel umbenennen + +`PATCH /api/recordings/:id/title` + Stift in Liste und Detailkopf. Neue Spalte +**`recordings.title_locked`**: ein von Hand gesetzter Titel überlebt jede spätere KI-Analyse. Leeres Feld +löscht Titel und Sperre. + +### Papierkorb, 30 Tage + +`DELETE /api/recordings/:id` setzt nur noch `deleted_at`; Audio, Transkript und Analyse bleiben vollständig +liegen. Neue Ansicht über der Liste mit Resttagen (rot ab 3 Tagen), Wiederherstellen, endgültig Löschen. +Aufräumlauf beim Boot **und** täglich — der Boot-Lauf, weil das Board oft tagelang aus ist. + +**Altfehler mitbehoben:** das bisherige Löschen ließ die `words`-Zeilen verwaist in der Datenbank und die +WAV/MP3-Dateien für immer auf der Karte zurück. `purgeRecording` räumt beides ab. + +### Inhaltssuche mit Zeitstempel-Verweis + +Neue Tabelle `passages` (1689 Zeilen aus 22 Aufnahmen), neuer Tab „Suche" in Google-Optik: Trefferliste +sofort aus SQL, formulierte Antwort mit Belegen `[1] [2]` wird nachgereicht (Qwen3-4B, **~140 s**). Jeder +Treffer nennt Meeting, Sprecher und Gesprächsminute und springt beim Klick genau dorthin. + +Hybrid aus Volltext (MariaDB FULLTEXT), Passage-Embedding und Dokument-Embedding +(`multilingual-e5-small`, ONNX/CPU auf den A55-Kernen). **Ohne das Modell degradiert die Suche sauber auf +Volltext statt auszufallen.** + +**Gemessene Grenzen — bewusst dokumentiert statt geschönt:** + +- Volltext allein findet „Steueroptimierung" **nicht** (im Transkript steht eine andere Wortform) → genau + dafür ist die semantische Ebene da, die findet #71 mit `vec=0,92`. +- `e5-small` trennt auf verrauschtem ASR-Text schwach: für „Wie werden unsere Daten gesichert" lag die + Kosinus-Spanne **aller** 1689 Passagen bei nur 0,8085–0,8560, und eine irrelevante Passage („Agenda + zusammenpacken", 0,8560) schlug die richtige Antwort („Backup-Job startet um 11 Uhr", 0,8419). +- Gegenmaßnahmen: Dokument-Ebene aus Titel + Zusammenfassung (sauberer LLM-Text statt ASR-Rauschen) und + IDF-Wertung (Auswahl per `BOOLEAN MODE`, Wertung per `NATURAL LANGUAGE MODE` — Boolean gewichtet nicht + nach Seltenheit, weshalb „Daten"-Wiederholung „gesichert" schlug). +- Ergebnis: richtige Passage auf **Rang 11 von 296** statt gar nicht. Auffindbar, aber nicht in den Top-8, + die das LLM liest. **Nächster Hebel: `multilingual-e5-base`.** + +Relative Datumsangaben („letzte Woche") werden **deterministisch** aufgelöst, nicht vom LLM — ein falsches +Kalenderfenster wäre schlimmer als keins. Der Datumsausdruck wird aus den Suchbegriffen entfernt. + +### Dateien + +Neu: `app/search.js`, `app/emb.js`, `app/inference/embed_text.py`, `docs/09-suche-und-verwaltung.md`. +Geändert: `app/db.js`, `app/server.js`, `app/web/src/main.js`, `app/web/src/style.css`, `README.md`. +Board-Backups: `.bak-search`. Nicht im Repo: `/opt/aime-models/e5-small/` (470 MB Modell). + ## 2026-07-30 (nachmittags) — v2 machte den Uplink stumm, v3 behebt es **Symptom (Nutzer, Testanruf 13:51):** „Die Gegenseite hat mich nicht mehr gehört." Regression aus v2 vom diff --git a/docs/09-suche-und-verwaltung.md b/docs/09-suche-und-verwaltung.md new file mode 100644 index 0000000..e1099c0 --- /dev/null +++ b/docs/09-suche-und-verwaltung.md @@ -0,0 +1,180 @@ +# 09 — Inhaltssuche, Papierkorb, Titel umbenennen + +Drei Funktionen, die am 2026-07-31 dazukamen. Die Suche ist der große Teil, die anderen beiden sind +kleine Ergänzungen an derselben Datenschicht. + +## Titel nachträglich umbenennen + +`PATCH /api/recordings/:id/title`. Der Stift steht in der Aufnahmeliste und im Detailkopf. + +Der eigentliche Punkt ist die Spalte **`recordings.title_locked`**: sobald ein Titel von Hand gesetzt +wurde, darf die nächste KI-Analyse ihn nicht mehr überschreiben. Ohne diese Sperre hätte ein +„↻ Analyse neu" den selbst vergebenen Namen still wieder gegen einen LLM-Vorschlag getauscht. + +```sql +-- db.js/updateAnalysis +UPDATE recordings SET title = CASE WHEN title_locked=1 THEN title ELSE COALESCE(?,title) END, ... +``` + +Ein leeres Feld löscht den Titel **und** die Sperre — danach darf die KI wieder benennen. + +## Papierkorb (30 Tage) + +Löschen setzt nur noch `recordings.deleted_at`. Nichts wird angefasst: Wörter, Marker, Passagen und die +WAV/MP3-Dateien bleiben liegen, damit eine Wiederherstellung vollständig ist und nicht halb. + +```mermaid +flowchart LR + A[Aufnahme sichtbar
deleted_at IS NULL] -->|"🗑 Löschen"| B[Papierkorb
deleted_at = jetzt] + B -->|"↩︎ Wiederherstellen"| A + B -->|"✕ Endgültig (manuell)"| C[purgeRecording] + B -->|"Aufräumlauf: älter als 30 Tage"| C + C --> D["DELETE markers + words + passages
DELETE recordings
unlink wav + mp3"] +``` + +Der Aufräumlauf (`sweepTrash` in `server.js`) startet **beim Boot und danach täglich**. Der Boot-Lauf ist +nötig, weil das Board oft tagelang aus ist — ein reiner 24-Stunden-Timer würde das Fenster immer wieder +verpassen. + +**Altfehler mitbehoben:** das frühere `deleteRecording` löschte nur `markers` und die `recordings`-Zeile. +Die `words`-Zeilen blieben verwaist in der Datenbank zurück und die Audiodateien für immer auf der Karte. +`purgeRecording` räumt jetzt beides mit ab. + +Retention über `AIME_TRASH_DAYS` einstellbar (Vorgabe 30). + +## Inhaltssuche + +### Aufbau + +```mermaid +flowchart TD + Q["Frage:
„Welches Projekt hat X letzte Woche vorgestellt?“"] --> DR[parseDateRange
deterministisch, kein LLM] + Q --> CT[contentTerms
Stoppwörter raus] + DR -->|"Filter: 20.–27.07."| SEL + CT -->|"Projekt, X, vorgestellt"| SEL + + subgraph SEL["Kandidaten + Bewertung (SQL, Millisekunden)"] + FT["Volltext
BOOLEAN = Auswahl
NATURAL LANGUAGE = Wertung (IDF)"] + VP["Passage-Embedding
Kosinus, min-max"] + VD["Dokument-Embedding
Titel + Zusammenfassung"] + end + + SEL --> SC["score = 0,45·ft + 0,25·vec + 0,30·doc
+ Sprecher + Wortgruppe + Aktualität"] + SC --> HITS["Trefferliste sofort
Meeting · Sprecher · mm:ss"] + HITS --> LLM["Qwen3-4B über die Top-8
~140 s, wird nachgereicht"] + LLM --> ANS["Antwort mit Belegen [1] [2]"] +``` + +Die Zweiteilung ist Absicht: die Trefferliste kommt aus SQL und steht sofort, die formulierte Antwort +braucht auf dieser NPU **~140 s**. Erst antworten und dann anzeigen hieße eine Minute Leerlauf. + +### Passagen + +Eine Passage ist die Sucheinheit: 2–4 Sätze **eines** Sprechers, ~420 Zeichen, harte Grenze 700. Sie wird +aus den `words`-Zeilen geschnitten, nicht aus `transcript_clean` — nur die Wörter tragen Zeitstempel, und +der Zeitstempel ist der ganze Zweck eines Treffers. Ein Sprecherwechsel beendet eine Passage immer. + +Stand 2026-07-31: 22 Aufnahmen → **1689 Passagen** (Neuaufbau in ~1 s ohne, ~9 min mit Embeddings). + +### Warum hybrid — mit Messwerten + +Keine der beiden Ebenen genügt allein, und beide Grenzen sind an diesem Korpus **gemessen**: + +| Frage | Volltext | Semantisch | +|---|---|---| +| `Steueroptimierung` | **0 Treffer** — im Transkript steht eine andere Wortform | findet #71, `vec=0,92` | +| `Wie können wir Steuern sparen` | trifft über „Steuern" | bestätigt #71 | +| `OneDrive` | 21 Treffer, exakt | Eigennamen verwischen im Embedding | + +### Bekannte Grenze der semantischen Ebene + +`multilingual-e5-small` trennt auf **verrauschtem ASR-Text** schwach. Gemessen für „Wie werden unsere Daten +gesichert" über alle 1689 Passagen: + +``` +Kosinus-Spanne der Kandidaten: 0,8085 .. 0,8560 (Breite nur 0,047) + +cos=0,8560 #77 „Vielleicht packen wir nochmal eine Agenda zusammen…“ ← irrelevant, Rang 1 +cos=0,8419 #85 „…jeden Abend um 11 Uhr der Backup-Job startet.“ ← die Antwort, Rang 5 +``` + +Das Embedding selbst rankt also falsch; keine Normalisierung repariert eine falsche Reihenfolge. Grund ist +vermutlich, dass die ASR Komposita zerbricht („Backup -Job", „KI -Binnenstrich") — genau das, womit ein +Embedding am schlechtesten umgeht. + +Zwei Gegenmaßnahmen sind drin und messbar wirksam, lösen den Fall aber nicht ganz: + +1. **Dokument-Ebene** (`recordings.doc_vec`): Titel + Zusammenfassung sind saubere LLM-Prosa statt + ASR-Rauschen. Sagt „diese *Aufnahme* passt zur Frage", auch wenn keine einzelne Passage gut einbettet. +2. **IDF-Wertung**: Kandidaten per `BOOLEAN MODE` auswählen, aber mit `NATURAL LANGUAGE MODE` bewerten. + Boolean gewichtet nicht nach Seltenheit, weshalb eine Passage, die „Daten" mehrfach wiederholt, die + Passage schlug, die „gesichert" tatsächlich beschreibt. + +Ergebnis für den schwierigen Fall: die richtige Passage steht auf **Rang 11 von 296** statt gar nicht — sie +ist auffindbar, erreicht aber die Top-8 nicht, die das LLM zu sehen bekommt. **Nächster Hebel wäre +`multilingual-e5-base`** (~1,1 GB statt 470 MB, Neuindexierung ~25–30 min). + +### Relative Datumsangaben + +„letzte Woche" wird **deterministisch** aufgelöst, nicht vom LLM: ein falsches Kalenderfenster ist +schlimmer als gar kein Filter, und Datumsarithmetik ist genau das, was ein 4B-Modell subtil falsch macht. +Wochen laufen Montag–Sonntag. Der erkannte Datumsausdruck wird aus den Suchbegriffen **entfernt** — sonst +sucht die Anfrage nach dem Wort „Woche". + +Wenn nur ein Zeitraum und kein Suchbegriff übrig bleibt („Was lief letzte Woche?"), zeigt die Suche die +Aufnahmen dieses Zeitraums ab ihren Eröffnungspassagen statt einer leeren Seite. + +### Embedding-Modell + +`/opt/aime-models/e5-small/` — `model.onnx` (470 MB, fp32), `tokenizer.json`, `config.json`, geladen von +`intfloat/multilingual-e5-small`. Läuft über `inference/embed_text.py` als Daemon auf onnxruntime/CPU, +**`taskset -c 0-3`** (A55), damit die A76-Kerne für whisper und sherpa frei bleiben. + +Bewusst **ohne** sentence-transformers: das zöge torch + transformers (~2 GB) auf eine Karte, auf der +bereits whisper, sherpa und ein 5-GB-LLM liegen. `tokenizers` und `onnxruntime` waren für den NPU-Pfad +ohnehin schon im venv. + +Die e5-Präfixe `query: ` und `passage: ` sind Pflicht — ohne sie verliert das Modell deutlich an +Genauigkeit, weil genau dieses asymmetrische Paar kurze Fragen auf lange Passagen abbildet. + +**Ohne das Modell degradiert die Suche sauber auf reinen Volltext** (`emb.available()` ist dann `false`), +sie fällt nicht aus. Das war die Voraussetzung dafür, die Suche vor dem Download in Betrieb zu nehmen. + +### Wann neu indexiert wird + +| Auslöser | Umfang | +|---|---| +| Analyse einer Aufnahme fertig | diese Aufnahme (nach `relabelAnalysis`, damit die Sprechernamen stehen) | +| Sprecher umbenannt | diese Aufnahme (`speaker_name` liegt denormalisiert in `passages`) | +| Dienststart | alle Aufnahmen **ohne** Passagen (Nachzügler) | +| `POST /api/search/reindex {"force":true}` | alles | + +## API + +| Route | Zweck | +|---|---| +| `PATCH /api/recordings/:id/title` | Titel setzen/löschen | +| `DELETE /api/recordings/:id` | in den Papierkorb | +| `GET /api/trash` | Inhalt + Resttage | +| `POST /api/trash/:id/restore` | wiederherstellen | +| `DELETE /api/trash/:id` | endgültig löschen (inkl. Dateien) | +| `GET /api/search?q=` | Trefferliste, sofort | +| `POST /api/search/answer` | formulierte Antwort mit Belegen (NPU-Warteschlange) | +| `GET /api/search/status` | Passagen, davon eingebettet, Modell da? | +| `POST /api/search/reindex` | Index neu bauen (`{"force":true}` = alles) | + +## Stellschrauben + +| Variable | Vorgabe | Wirkung | +|---|---|---| +| `AIME_TRASH_DAYS` | 30 | Aufbewahrung im Papierkorb | +| `AIME_PASSAGE_CHARS` | 420 | Ziellänge einer Passage | +| `AIME_SEARCH_TOPK` | 20 | Treffer je Suche | +| `AIME_ANSWER_PASSAGES` | 8 | Passagen für die LLM-Antwort (5 ≈ 90 s statt 140 s) | +| `AIME_EMB_DIR` | `/opt/aime-models/e5-small` | Embedding-Modell | + +## Dateien + +`app/search.js` (neu), `app/emb.js` (neu), `app/inference/embed_text.py` (neu), `app/db.js`, +`app/server.js`, `app/web/src/main.js`, `app/web/src/style.css`. +Backups auf dem Board: `.bak-search`.