diff --git a/TensorRT_whisper.md b/TensorRT_whisper.md index 1e3e3cf..aeed8db 100644 --- a/TensorRT_whisper.md +++ b/TensorRT_whisper.md @@ -12,7 +12,7 @@ cd WhisperLive ``` - Build the TensorRT-LLM docker image ```bash -docker build --file docker/Dockerfile.tensorrt --tag tensorrt_llm/devel:latest . +docker build --file docker/Dockerfile.tensorrt --tag tensorrt_llm:latest . ``` **NOTE**: This could take some time. - Next, we run the docker image and mount WhisperLive repo to the containers `/home` directory. @@ -20,7 +20,7 @@ docker build --file docker/Dockerfile.tensorrt --tag tensorrt_llm/devel:latest . docker run -it --gpus all --shm-size=8g \ --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 \ -v /path/to/WhisperLive:/home/WhisperLive \ - tensorrt_llm/devel:latest + tensorrt_llm:latest ``` - Once inside the docker container, make sure to test the installation. ```bash @@ -31,7 +31,7 @@ python -c "import torch; import tensorrt; import tensorrt_llm" **NOTE**: Uncomment and update library paths if imports fail. ## Whisper TensorRT Engine -- For this demo we build `small.en` and `small` multilingual TensorRT engine. The script logs the path of the directory with Whisper TensorRT engine. We need the model_path to run the server. +- We build `small.en` and `small` multilingual TensorRT engine. The script logs the path of the directory with Whisper TensorRT engine. We need the model_path to run the server. ```bash # convert small.en bash build_whisper_tensorrt /path/to/TensorRT-LLM/examples small.en diff --git a/docker/Dockerfile.tensorrt b/docker/Dockerfile.tensorrt index c91dc8e..25fd666 100644 --- a/docker/Dockerfile.tensorrt +++ b/docker/Dockerfile.tensorrt @@ -5,8 +5,8 @@ ARG CUDA_ARCHITECTURES="89-real" FROM ${BASE_IMAGE}:${BASE_TAG} as base WORKDIR /home -COPY script/install_trt_llm.sh install_trt_llm.sh -RUN bash install_trt_llm.sh ${CUDA_ARCHITECTURES} && rm install_trt_llm.sh +COPY scripts/install_tensorrt_llm.sh install_tensorrt_llm.sh +RUN bash install_tensorrt_llm.sh ${CUDA_ARCHITECTURES} && rm install_tensorrt_llm.sh COPY scripts/setup.sh /home RUN apt update && bash setup.sh && rm setup.sh