Commit Graph

104 Commits

Author SHA1 Message Date
cjones 17b1639a9b Get ready to add diarization to openvino backend. 2026-07-23 00:05:02 -04:00
makaveli10 ecb052c873 Enable single-model mode for stock models when batch_inference is set
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2026-06-23 10:52:02 -04:00
Aaron Boxer 44940e2834 Support known speaker hints in REST API 2026-06-14 12:17:44 -04:00
nightcityblade 32c1b18c9f fix: support uint8 websocket audio format 2026-06-04 16:04:38 -04:00
Aaron Boxer 5334ea0f7a Add WebSocket authentication via api_key
- When --api_key is set, WebSocket connections require auth too
- Supports Authorization: Bearer <key> header or ?token=<key> query param
- Unauthenticated connections receive HTTP 401 before upgrade
- Uses websockets process_request callback (no resource allocation before auth)
- Added 5 unit tests for WebSocket auth handler
2026-05-26 23:07:50 -04:00
Aaron Boxer b648bcb2a4 Add optional API key auth and rate limiting for REST API
- api_key param: requires 'Authorization: Bearer <key>' header
- rate_limit_rpm param: per-IP sliding-window rate limit (requests/min)
- Both are off by default (backward compatible)
- CLI flags: --api_key, --rate_limit_rpm
- Added 5 unit tests for auth and rate limiting
2026-05-26 23:02:43 -04:00
Aaron Boxer 8396763444 fix: set proper mock_info attributes in SSE streaming tests
MagicMock auto-attributes are not JSON serializable. Set language,
language_probability, and duration explicitly. Also exclude metadata
events from segment count assertion.
2026-05-25 09:44:21 -04:00
Aaron Boxer 8ac98dceec feat: add SSE streaming for REST transcription endpoint
- stream=true now returns text/event-stream with per-segment SSE events
- Each segment yields 'data: {json}' followed by 'data: [DONE]'
- Error events streamed as 'data: {"error": ...}'
- Temp files cleaned up in finally block
- 5 new tests in test_server_extended.py (183 total passing)
2026-05-25 09:44:21 -04:00
Aaron Boxer c028c4b584 feat: add segment_post_processor hook for external plugins
Add a minimal, non-breaking hook to WhisperLive that allows external
projects to post-process transcription segments before they are sent
to the client.

Changes:
- ServeClientBase: add segment_post_processor attribute (default None)
- ServeClientBase.send_transcription_to_client: apply post_processor
  per-segment with error handling (falls back to original segment)
- TranscriptionServer: add segment_post_processor parameter to run()
  and wire it to each client on creation

This enables downstream projects to plug in custom processing
(e.g. formatting, PII redaction, diarization tagging) without
modifying WhisperLive core code.
2026-05-15 10:45:31 -04:00
Aaron Boxer 4e31f8c61b Add word-level timestamps and confidence scores
- New word_timestamps option (default False) in client handshake
- When enabled, each segment includes 'words' array with per-word
  start/end times and probability scores
- Wired through entire pipeline: client → server → backend → transcribe()
- Words include timestamp_offset for accurate absolute times
- REST API already supported word timestamps; now WebSocket does too
- Added 9 unit tests for word timestamp extraction and formatting
2026-05-15 10:35:27 -04:00
Aaron Boxer 18b897277f Add real-time speaker diarization support
- New whisper_live/diarization.py: SpeakerDiarizer with online clustering
- Uses pyannote.audio speaker embeddings (optional dependency)
- Cosine similarity threshold for speaker matching (default 0.55)
- Running average embedding update for speaker stability
- Configurable max_speakers limit (default 10)
- Client options: enable_diarization, max_speakers
- Segments include 'speaker' field when diarization is active
- Graceful fallback: logs warning if pyannote not installed
- Added 12 unit tests (mock-based, no GPU required)
2026-05-13 10:50:59 -04:00
Aaron Boxer ced4bdb737 feat: add Prometheus metrics instrumentation
- New whisper_live/metrics.py with Counter, Gauge, Histogram metrics
- Track connections (opened/closed/rejected), transcription latency,
  audio processed, segments emitted, REST requests, and errors
- All metric helpers are no-ops when prometheus_client not installed
- --metrics_port CLI flag to expose /metrics endpoint (0 = disabled)
- Metrics integrated into server.py, base.py at key instrumentation points
- 17 new tests in tests/test_metrics.py (178 total passing)
2026-05-13 10:45:40 -04:00
Aaron Boxer 4210697ca6 Add custom vocabulary / hotwords support 2026-05-13 10:32:28 -04:00
Aaron Boxer 18bce1864a Validate server parameters on startup
- max_clients must be >= 1
- max_connection_time must be > 0
- batch_max_size must be >= 1 (when batch enabled)
- batch_window_ms must be >= 0 (when batch enabled)
- Added 5 new tests for parameter validation
2026-04-17 09:30:19 -04:00
Aaron Boxer 9e5e4a9970 Bound transcript memory and translation queue size
- Add MAX_TRANSCRIPT_LENGTH (500) and MAX_TRANSLATION_QUEUE_SIZE (100)
  class constants to ServeClientBase
- Trim transcript and text lists after each update_segments() call
- Create translation queue with maxsize to prevent unbounded growth
- Added tests for _trim_transcript()
2026-04-17 09:29:38 -04:00
Aaron Boxer 81cdbbca95 Add thread safety to ClientManager with threading.Lock
- All ClientManager methods (add_client, get_client, remove_client,
  get_wait_time, is_server_full, is_client_timeout) now protected by
  a threading.Lock
- cleanup() called outside the lock to avoid holding it during I/O
- is_server_full() computes wait time inline under lock instead of
  calling get_wait_time() to avoid nested lock acquisition
- Added concurrent thread safety tests for add/remove and get operations
2026-04-17 09:27:37 -04:00
Aaron Boxer f5340ddf1e audio: add support for raw pcm input via server flag
fixes #
2026-04-17 09:21:06 -04:00
ianwh02 3508b39584 Fix missing batch_config init causing CI test hang 2026-03-09 11:42:15 +00:00
ianwh02 e8bd4fd532 Add cross-client GPU batch inference for faster_whisper backend 2026-02-26 00:00:51 +00:00
Aaron Boxer 29ee640409 api: add support OpenAI REST transcription api 2026-02-05 22:31:59 -05:00
makaveli10 f5256fc62f feat: support HuggingFace model IDs for faster_whisper_custom_model_path
Previously, the server only accepted local file paths for custom Faster Whisper
models. This change allows passing HuggingFace repo IDs which are automatically
downloaded and converted to CTranslate2 format by the backend if not already in
CTranslate2 format.

Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2026-01-13 16:34:04 +05:30
makaveli10 04db67170b ServeClientTranslation import only when enable_tranlsation is True
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2025-07-22 15:47:06 +00:00
makaveli10 2b8b245fa8 Add translation backend
Translate from any language to any language with alirezamsh/small100
running in a thread and reading from a queue shared with transcription thread.

Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2025-07-22 08:54:10 +00:00
makaveli10 8d6ddd4f7b Change max_clients max_connection_time from server only
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2025-07-21 22:52:24 +05:30
Kiran Lonikar 40edd25468 remove commented code 2025-07-12 22:29:30 +05:30
Kiran Lonikar 9954548075 issue 371
model name is of form namespace/repo_name and not os path.
2025-07-06 16:10:49 +05:30
makaveli10 74abf66d48 Make cache path configurable to save auto converted ct2 models
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2025-06-01 09:04:22 +00:00
giubots 275ed4e45b Merge branch 'main' into configure-more-params 2025-05-02 12:23:45 +02:00
giubots a2271806c3 feat: client sends new parameters to server 2025-04-28 17:21:33 +02:00
makaveli10 47ee035f65 Upgrade tensorrt_llm to v0.18.2
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2025-04-22 12:33:03 +00:00
makaveli a62495b090 Integrate OpenVINO backend
Signed-off-by: makaveli <vineet.suryan@collabora.com>
2025-03-31 12:57:19 +05:30
makaveli10 c1ac71ada0 Refactor 🔨
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2025-03-24 16:48:33 +05:30
makaveli10 b6b73730fb Fix: typo
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2025-01-13 11:35:22 +00:00
makaveli10 182b5cbd6d Fix skipped audio chunk by recording the time of the first repition of a segment
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2025-01-08 13:58:34 +00:00
makaveli10 c936e5f727 Add lock to thread shared variables updates/reads
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2025-01-06 06:34:02 +00:00
makaveli10 2eff360b9e Support loading hf models
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2024-11-26 11:14:18 +05:30
makaveli10 a6523b6b71 Minor fixes for better punctuations
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2024-11-19 02:13:11 -05:00
makaveli10 778a9c5903 Upgrade faster-whisper 1.1.0rc0
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2024-11-14 10:07:37 -05:00
makaveli10 81c57ae40c Send completed bool with each segment
Completed bool represents if the segment is completely processed by the server

Signed-off-by: makaveli10 <suryanvineet47@gmail.com>
2024-11-05 18:11:32 +05:30
makaveli10 0d74790c67 Expose ClientManager arguments to be passed from client
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2024-10-10 08:37:45 -04:00
makaveli10 380f07394b Set compute_type based on device capability
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2024-09-03 00:47:38 -04:00
makaveli10 01c6bc1ecd Initialize last_segment to None
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2024-08-30 07:47:03 -04:00
makaveli10 fac62aaccc Fix hallucinations with no_speech_thres
Signed-off-by: makaveli10 <vineet.suryan@collabora.com>
2024-08-08 06:05:12 -04:00
berkaybilik 2f1c934ea2 always use the BackendType enum to reference the backend inside the TranscriptionServer 2024-06-27 00:20:17 +01:00
berkaybilik b220ccb330 fixed reference before assignment error/warning 2024-06-27 00:11:06 +01:00
berkaybilik 5e3906fc7b use enum to validate backend validity in server.run 2024-06-26 23:59:07 +01:00
Andreas Peldszus 3a96f60006 Raise error for invalid model paths 2024-05-31 15:06:54 +02:00
Andreas Peldszus 3c09289dea Add single model mode for custom models
- Use a threadlock around the model in single model mode
2024-05-31 15:06:47 +02:00
makaveli10 18b6168807 fix: cpu usage issue 2024-02-28 13:55:37 +05:30
makaveli10 a41e714801 fix: lanuguage, task prefix in decoder start ids 2024-02-26 23:31:19 -05:00