2e466b765a
Add AMD ROCm GPU support for faster_whisper backend
...
Add ROCm_whisper.md (Docker + native install guide) and docker/Dockerfile.rocm
based on rocm/pytorch:rocm7.2.4 (PyTorch 2.10.0) that installs the official
CTranslate2 v4.8.0 ROCm wheel. The default faster_whisper backend runs on AMD
GPUs out of the box with no code changes.
Tested on Radeon AI PRO R9700 (gfx1201) and Ryzen AI Max+ 395 / Radeon 8060S
(gfx1151) with ROCm 7.2.4.
Addresses #520 .
2026-07-15 16:13:37 +08:00
47ee035f65
Upgrade tensorrt_llm to v0.18.2
...
Signed-off-by: makaveli10 <vineet.suryan@collabora.com >
2025-04-22 12:33:03 +00:00
a62495b090
Integrate OpenVINO backend
...
Signed-off-by: makaveli <vineet.suryan@collabora.com >
2025-03-31 12:57:19 +05:30
779baff9c3
Add pynvml missing dep for tensorrt
...
Signed-off-by: makaveli10 <vineet.suryan@collabora.com >
2025-01-22 05:11:26 -05:00
5120afbc25
Revert to 12.4.1 base image
...
Signed-off-by: makaveli10 <vineet.suryan@collabora.com >
2025-01-17 10:19:38 +00:00
30617dfd44
Checkout git tensorrt_llm==v0.15.0 in dockerfile
...
Signed-off-by: makaveli10 <vineet.suryan@collabora.com >
2024-12-03 18:27:11 +05:30
c25a036c02
Upgrade tensorrt_llm to 0.15.0
...
Signed-off-by: makaveli10 <vineet.suryan@collabora.com >
2024-11-21 06:13:00 +00:00
aaa47cfab5
Fix requirements & upgrade tensorrt-llm
...
Signed-off-by: makaveli10 <vineet.suryan@collabora.com >
2024-09-16 00:25:23 -04:00
e4579ef291
Dockerfile tensorrt use cuda-runtimee as base image to reduce size
...
Signed-off-by: makaveli10 <vineet.suryan@collabora.com >
2024-06-03 06:44:09 -04:00
f73a146eb9
Update TensorRT backend tensorrt_llm==0.9.0
...
Signed-off-by: makaveli10 <vineet.suryan@collabora.com >
2024-06-03 05:24:41 -04:00
dccfce2a3c
Improve cpu and gpu Dockerfiles, resulting in much smaller images
2024-04-26 15:51:11 +02:00
8a06ba802b
update cuda version 12.2.2 gpu dockerfile
2024-03-04 03:24:49 -05:00
ea64d47899
run server with python3
2024-02-26 18:50:18 +05:30
d1de4948ee
update base cuda version to 11.8; some dockerfile-gpu fixes
2024-02-19 04:38:41 -05:00
6cabbe441b
update cpu dockerfile with python-slim-buster base image
2024-02-16 18:58:38 +05:30
44a2e20c68
remove trt-llm dockerfile
2024-01-22 11:44:21 +00:00
1e2faa3f2b
fix: tensorrt llm idocker setup & docs
2024-01-19 10:40:59 -05:00
e3084b34cb
update tensorrt docker & readme
2024-01-19 07:30:50 -05:00
75001ae6b7
updatetensorrt-llm dockerfile
2024-01-19 11:58:40 +00:00
389bb5ae37
add docker setup for tensorrt-llm; update readme
2024-01-12 08:15:53 +00:00
71a062b726
update dockerfiles
2024-01-10 14:30:57 +00:00
e602ba1282
add cpu&gpu specifice Dockerfiles
2023-08-07 23:12:53 +08:00
Kaihui-AMD