From 1ac7a278bb7cd0ba3c0a7576449cb2ecae832048 Mon Sep 17 00:00:00 2001
From: Andreas Peldszus <andreas.peldszus@retresco.de>
Date: Fri, 31 May 2024 15:01:32 +0200
Subject: [PATCH] Update README

---
 README.md | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/README.md b/README.md
index 53de8d6..2a363c3 100644
--- a/README.md
+++ b/README.md
@@ -61,6 +61,18 @@ python3 run_server.py --port 9090 \
                       --omp_num_threads 4
 ```
 
+#### Single model mode
+By default, the server will instantiate a new whisper model for every client connection. I.e. for 4 concurrent connections, you need to have (V)RAM for 4x the model. Also, the connection will have to wait until the model is loaded, which might take a few seconds.
+
+If you use a TensorRT or a custom faster_whisper model, you also set the server to instantiate the model only once and use the single on for all client connections, using the `--single-model` or `-sm` option:
+```bash
+python3 run_server.py -p 9090 \
+                      -b tensorrt \
+                      -trt /home/TensorRT-LLM/examples/whisper/whisper_large_v3 \
+                      -m \
+                      --single_model
+```
+
 ### Running the Client
 - Initializing the client with below parameters:
   - `lang`: Language of the input audio, applicable only if using a multilingual model.