Add AMD ROCm GPU support for faster_whisper backend
Add ROCm_whisper.md (Docker + native install guide) and docker/Dockerfile.rocm based on rocm/pytorch:rocm7.2.4 (PyTorch 2.10.0) that installs the official CTranslate2 v4.8.0 ROCm wheel. The default faster_whisper backend runs on AMD GPUs out of the box with no code changes. Tested on Radeon AI PRO R9700 (gfx1201) and Ryzen AI Max+ 395 / Radeon 8060S (gfx1151) with ROCm 7.2.4. Addresses #520.
This commit is contained in:
@@ -117,6 +117,9 @@ python3 run_server.py -p 9090 \
|
||||
python3 run_server.py -p 9090 -b openvino
|
||||
```
|
||||
|
||||
### Setting up AMD ROCm for faster_whisper backend
|
||||
- Please follow [ROCm_whisper readme](https://github.com/collabora/WhisperLive/blob/main/ROCm_whisper.md) for setup of AMD ROCm GPU support with the CTranslate2 ROCm wheel.
|
||||
|
||||
|
||||
#### Controlling OpenMP Threads
|
||||
To control the number of threads used by OpenMP, you can set the `OMP_NUM_THREADS` environment variable. This is useful for managing CPU resources and ensuring consistent performance. If not specified, `OMP_NUM_THREADS` is set to `1` by default. You can change this by using the `--omp_num_threads` argument:
|
||||
@@ -301,6 +304,15 @@ Refer to [`ios-client`](https://github.com/collabora/WhisperLive/tree/main/Audio
|
||||
docker run -it --device=/dev/dri -p 9090:9090 ghcr.io/collabora/whisperlive-openvino
|
||||
```
|
||||
|
||||
- AMD ROCm (faster-whisper on AMD GPU via CTranslate2 ROCm wheel)
|
||||
```bash
|
||||
docker build -f docker/Dockerfile.rocm -t whisperlive-rocm .
|
||||
docker run --rm -it --device=/dev/kfd --device=/dev/dri \
|
||||
--group-add "$(getent group video | cut -d: -f3)" \
|
||||
--group-add "$(getent group render | cut -d: -f3)" \
|
||||
-p 9090:9090 whisperlive-rocm
|
||||
```
|
||||
|
||||
- CPU
|
||||
- Faster-whisper
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user