cjones/WhisperLive
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

docs: fix setup instructions errors reported in #405

Browse Source 
- Clarify that setup.sh installs portaudio system dependency and list
  per-distro package names
- Add missing --gpus all flag to TensorRT Docker run command
- Fix Docker TensorRT example showing multiple --trt_model_path on one
  command (should be separate alternatives)
- Document --trt_py_session flag as workaround for TensorRT C++ session
  crashes (CrossAttentionMask warnings)

Closes #405

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
nightcityblade
2026-04-13 11:09:40 +08:00
parent f8869906b0
commit 68a8b57e66
1 changed files with 8 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+8 -5
View File
@@ -25,10 +25,11 @@ input from microphone and pre-recorded audio files.
- [Citations](#citations) - [Citations](#citations)
## Installation ## Installation
- Install PortAudio - Install PortAudio (required system dependency for microphone input via PyAudio)
```bash ```bash
bash scripts/setup.sh bash scripts/setup.sh
``` ```
On Debian/Ubuntu this installs `portaudio19-dev`, on Fedora `portaudio-devel`, on macOS it uses Homebrew (`portaudio`).
- Install whisper-live from pip - Install whisper-live from pip
```bash ```bash
@@ -101,6 +102,7 @@ python3 run_server.py -p 9090 \
--max_clients 4 \ --max_clients 4 \
--max_connection_time 600 --max_connection_time 600
``` ```
> **Note:** The TensorRT backend uses a C++ session by default. If you experience issues (e.g. repeated `CrossAttentionMask` warnings or crashes), add the `--trt_py_session` flag to use the Python session instead.
- Use `--max_clients` option to restrict the number of clients the server should allow. Defaults to 4. - Use `--max_clients` option to restrict the number of clients the server should allow. Defaults to 4.
- Use `--max_connection_time` options to limit connection time for a client in seconds. Defaults to 600. - Use `--max_connection_time` options to limit connection time for a client in seconds. Defaults to 600.
- WhisperLive now supports the [OpenVINO](https://github.com/openvinotoolkit/openvino) backend for efficient inference on Intel CPUs, iGPU and dGPUs. Currently, we tested the models uploaded to [huggingface by OpenVINO](https://huggingface.co/OpenVINO?search_models=whisper). - WhisperLive now supports the [OpenVINO](https://github.com/openvinotoolkit/openvino) backend for efficient inference on Intel CPUs, iGPU and dGPUs. Currently, we tested the models uploaded to [huggingface by OpenVINO](https://huggingface.co/OpenVINO?search_models=whisper).
@@ -206,19 +208,20 @@ Refer to [`ios-client`](https://github.com/collabora/WhisperLive/tree/main/Audio
- TensorRT. Refer to [TensorRT_whisper readme](https://github.com/collabora/WhisperLive/blob/main/TensorRT_whisper.md) for setup and more tensorrt backend configurations. - TensorRT. Refer to [TensorRT_whisper readme](https://github.com/collabora/WhisperLive/blob/main/TensorRT_whisper.md) for setup and more tensorrt backend configurations.
```bash ```bash
docker build . -f docker/Dockerfile.tensorrt -t whisperlive-tensorrt docker build . -f docker/Dockerfile.tensorrt -t whisperlive-tensorrt
docker run -p 9090:9090 --runtime=nvidia --entrypoint /bin/bash -it whisperlive-tensorrt docker run -p 9090:9090 --runtime=nvidia --gpus all --entrypoint /bin/bash -it whisperlive-tensorrt
# Build small.en engine # Build small.en engine
bash build_whisper_tensorrt.sh /app/TensorRT-LLM-examples small.en # float16 bash build_whisper_tensorrt.sh /app/TensorRT-LLM-examples small.en # float16
bash build_whisper_tensorrt.sh /app/TensorRT-LLM-examples small.en int8 # int8 weight only quantization bash build_whisper_tensorrt.sh /app/TensorRT-LLM-examples small.en int8 # int8 weight only quantization
bash build_whisper_tensorrt.sh /app/TensorRT-LLM-examples small.en int4 # int4 weight only quantization bash build_whisper_tensorrt.sh /app/TensorRT-LLM-examples small.en int4 # int4 weight only quantization
# Run server with small.en # Run server with small.en (pick one engine)
python3 run_server.py --port 9090 \ python3 run_server.py --port 9090 \
--backend tensorrt \ --backend tensorrt \
--trt_model_path "/app/TensorRT-LLM-examples/whisper/whisper_small_en_float16" --trt_model_path "/app/TensorRT-LLM-examples/whisper/whisper_small_en_float16"
--trt_model_path "/app/TensorRT-LLM-examples/whisper/whisper_small_en_int8" # or int8 / int4:
--trt_model_path "/app/TensorRT-LLM-examples/whisper/whisper_small_en_int4" # --trt_model_path "/app/TensorRT-LLM-examples/whisper/whisper_small_en_int8"
# --trt_model_path "/app/TensorRT-LLM-examples/whisper/whisper_small_en_int4"
``` ```
- OpenVINO - OpenVINO