feat: support manual audio streaming from any source

This commit is contained in:
Quang Tran
2026-04-21 11:41:05 +07:00
parent d9459ebf2d
commit f4f1b1d8be
3 changed files with 349 additions and 0 deletions
+58
View File
@@ -23,6 +23,7 @@ input from microphone and pre-recorded audio files.
- [Speaker Diarization](#speaker-diarization)
- [Batch Inference](#batch-inference)
- [Raw PCM Input](#raw-pcm-input)
- [Streaming Client (Manual Audio Chunking)](#streaming-client-manual-audio-chunking)
- [Browser Extensions](#browser-extensions)
- [Whisper Live Server in Docker](#whisper-live-server-in-docker)
- [Troubleshooting](#troubleshooting)
@@ -260,6 +261,63 @@ python3 run_server.py --port 9090 --backend faster_whisper --raw_pcm_input
```
Audio is automatically normalized to float32 range [-1.0, 1.0]. Clients can also set `audio_format` in the initial websocket options to `float32` (default), `int16`, or `uint8`.
## Streaming Client (manual audio streaming from any source)
`StreamingTranscriptionClient` lets you push raw PCM audio bytes from any source — a live microphone capture loop, a network stream, an audio pipeline — and receive transcripts via callbacks as speech is detected. Unlike `TranscriptionClient`, it does not manage audio capture internally; you control when and how audio is fed.
A runnable example that reads from an audio file and stream the chunks is at [`examples/manual_audio_chunking.py`](examples/manual_audio_chunking.py):
```bash
python examples/manual_audio_chunking.py --file assets/jfk.flac
```
Example usage:
```python
from whisper_live.client import StreamingTranscriptionClient
client = StreamingTranscriptionClient(
"localhost", 9090,
lang="en",
model="small",
on_session_started=lambda: print("Server ready"),
on_partial_transcript=lambda text, segs: print(f"{text}", end="\r"),
on_committed_transcript=lambda text, segs: print(f"{text}"),
on_error=lambda e: print(f"Error: {e}"),
on_close=lambda: print("Closed"),
)
with client:
for chunk in my_audio_source: # any cadence, any chunk size
client.send(chunk, pcm_format="int16")
```
Audio must be **mono, 16 kHz PCM**. Two formats are accepted:
| `pcm_format` | Description |
|---|---|
| `"int16"` (default for raw microphone data) | 16-bit signed integers, normalized internally |
| `"float32"` | 32-bit floats in `[-1, 1]`, passed through directly |
NumPy arrays can be sent with `send_array()`:
```python
import numpy as np
samples = np.frombuffer(raw_bytes, dtype=np.int16)
client.send_array(samples)
```
**Callbacks**
| Callback | Signature | When fired |
|---|---|---|
| `on_session_started` | `() -> None` | Server handshake complete, ready to receive audio |
| `on_partial_transcript` | `(text, segments) -> None` | In-progress segment updated |
| `on_committed_transcript` | `(text, segments) -> None` | Segment finalized |
| `on_translation` | `(text, segments) -> None` | Translated segment ready (requires `enable_translation=True`) |
| `on_error` | `(error) -> None` | WebSocket error |
| `on_close` | `() -> None` | Connection closed |
## Browser Extensions
- Run the server with your desired backend as shown [here](https://github.com/collabora/WhisperLive?tab=readme-ov-file#running-the-server).
- Transcribe audio directly from your browser using our Chrome or Firefox extensions. Refer to [Audio-Transcription-Chrome](https://github.com/collabora/whisper-live/tree/main/Audio-Transcription-Chrome#readme) and https://github.com/collabora/WhisperLive/blob/main/TensorRT_whisper.md