cjones/WhisperLive
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

update READM

Browse Source
This commit is contained in:
makaveli10
2024-01-19 12:02:18 +00:00
parent f25ff1785a
commit b955e63dc1
1 changed files with 13 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+13 -8
View File
@@ -17,7 +17,7 @@ Unlike traditional speech recognition systems that rely on continuous audio stre
``` ```
### Setting up NVIDIA/TensorRT-LLM for TensorRT backend ### Setting up NVIDIA/TensorRT-LLM for TensorRT backend
- Please follow [TensorRT_whisper readme]() for installation of [NVIDIA/TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM) and for building Whisper-TensorRT engine. - Please follow [TensorRT_whisper readme](https://github.com/collabora/WhisperLive/blob/main/TensorRT_whisper.md) for installation of [NVIDIA/TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM) and for building Whisper-TensorRT engine.
## Getting Started ## Getting Started
The server supports two backends `faster_whisper` and `tensorrt`. If running `tensorrt` backend follow [TensorRT_whisper readme](https://github.com/collabora/WhisperLive/blob/main/TensorRT_whisper.md) The server supports two backends `faster_whisper` and `tensorrt`. If running `tensorrt` backend follow [TensorRT_whisper readme](https://github.com/collabora/WhisperLive/blob/main/TensorRT_whisper.md)
@@ -27,20 +27,25 @@ The server supports two backends `faster_whisper` and `tensorrt`. If running `te
```bash ```bash
python3 run_server.py --port 9090 \ python3 run_server.py --port 9090 \
--backend faster_whisper --backend faster_whisper
# running with custom model
python3 run_server.py --port 9090 \
--backend faster_whisper
-fw "/path/to/custom/faster/whisper/model"
``` ```
- TensorRT backend. Currently, we only recommend docker setup for TensorRT as shown in the [TensorRT_whisper readme](https://github.com/collabora/WhisperLive/blob/main/TensorRT_whisper.md) which works as expected. Make sure you follow the readme and build your TensorRT Engines before running the server with TensorRT backend. - TensorRT backend. Currently, we only recommend docker setup for TensorRT as shown in the [TensorRT_whisper readme](https://github.com/collabora/WhisperLive/blob/main/TensorRT_whisper.md) which works as expected. Make sure you follow the readme and build your TensorRT Engines before running the server with TensorRT backend.
```bash ```bash
# Run English only model # Run English only model
python3 run_server.py --port 9090 \ python3 run_server.py -p 9090 \
--backend tensorrt \ -b tensorrt \
--whisper_tensorrt_path /home/TensorRT-LLM/examples/whisper/whisper_small_en -trt /home/TensorRT-LLM/examples/whisper/whisper_small_en
# Run Multilingual model # Run Multilingual model
python3 run_server.py --port 9090 \ python3 run_server.py -p 9090 \
--backend tensorrt \ -b tensorrt \
--whisper_tensorrt_path /home/TensorRT-LLM/examples/whisper/whisper_small \ -trt /home/TensorRT-LLM/examples/whisper/whisper_small \
--trt_multilingual -m
``` ```