Fix argparser option

This commit is contained in:
Andreas Peldszus
2024-06-05 10:12:54 +02:00
parent ab17c4dbc6
commit 14077315ae
2 changed files with 6 additions and 7 deletions
+2 -2
View File
@@ -64,9 +64,9 @@ python3 run_server.py --port 9090 \
#### Single model mode
By default, when running the server without specifying a model, the server will instantiate a new whisper model for every client connection. This has the advantage, that the server can use different model sizes, based on the client's requested model size. On the other hand, it also means you have to wait for the model to be loaded upon client connection and you will have increased (V)RAM usage.
When serving a custom TensorRT model using the `-trt` or a custom faster_whisper model using the `-fw` option, the server will instead make use of the `--single_model true` option: It will only instantiate the custom model once and reuse that for all client connections.
When serving a custom TensorRT model using the `-trt` or a custom faster_whisper model using the `-fw` option, the server will instead only instantiate the custom model once and then reuse it for all client connections.
If you don't want this, set `--single_model false`.
If you don't want this, set `--no_single_model`.
### Running the Client
+4 -5
View File
@@ -25,10 +25,9 @@ if __name__ == "__main__":
type=int,
default=1,
help="Number of threads to use for OpenMP")
parser.add_argument('--single_model', '-sm',
type=bool,
default=True,
help='Set to false if every connection should instantiate its own model. Only relevant for custom model, passed using -trt or -fw.')
parser.add_argument('--no_single_model', '-nsm',
action='store_true',
help='Set this if every connection should instantiate its own model. Only relevant for custom model, passed using -trt or -fw.')
args = parser.parse_args()
if args.backend == "tensorrt":
@@ -47,5 +46,5 @@ if __name__ == "__main__":
faster_whisper_custom_model_path=args.faster_whisper_custom_model_path,
whisper_tensorrt_path=args.trt_model_path,
trt_multilingual=args.trt_multilingual,
single_model=args.single_model,
single_model=not args.no_single_model,
)