cjones/immich
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(ml): round-robin device assignment (#13237)

Browse Source 
* round-robin device assignment

* docs and tests

clarify doc
This commit is contained in:
Mert
2024-10-07 17:37:45 -04:00
committed by GitHub
parent 063969ca05
commit bd826b0b9b
8 changed files with 62 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
docs/docs/install/environment-variables.md
View File

@@ -164,6 +164,7 @@ Redis (Sentinel) URL example JSON before encoding:
| `MACHINE_LEARNING_ANN` | Enable ARM-NN hardware acceleration if supported | `True` | machine learning |
| `MACHINE_LEARNING_ANN_FP16_TURBO` | Execute operations in FP16 precision: increasing speed, reducing precision (applies only to ARM-NN) | `False` | machine learning |
| `MACHINE_LEARNING_ANN_TUNING_LEVEL` | ARM-NN GPU tuning level (1: rapid, 2: normal, 3: exhaustive) | `2` | machine learning |
| `MACHINE_LEARNING_DEVICE_IDS`<sup>\*4</sup> | Device IDs to use in multi-GPU environments | `0` | machine learning |
\*1: It is recommended to begin with this parameter when changing the concurrency levels of the machine learning service and then tune the other ones.
@@ -171,6 +172,8 @@ Redis (Sentinel) URL example JSON before encoding:
\*3: For scenarios like HPA in K8S. https://github.com/immich-app/immich/discussions/12064
\*4: Using multiple GPUs requires `MACHINE_LEARNING_WORKERS` to be set greater than 1. A single device is assigned to each worker in round-robin priority.
:::info
Other machine learning parameters can be tuned from the admin UI.