feat(server): separate face clustering job (#5598)

* separate facial clustering job * update api * fixed some tests * invert clustering * hdbscan * update api * remove commented code * wip dbscan * cleanup removed cluster endpoint remove commented code * fixes updated tests minor fixes and formatting fixed queuing refinements * scale search range based on library size * defer non-core faces * optimizations removed unused query option * assign faces individually for correctness fixed unit tests remove unused method * don't select face embedding update sql linting fixed ml typing * updated job mock * paginate people query * select face embeddings because typeorm * fix setting face detection concurrency * update sql formatting linting * simplify logic remove unused imports * more specific delete signature * more accurate typing for face stubs * add migration formatting * chore: better typing * don't select embedding by default remove unused import * updated sql * use normal try/catch * stricter concurrency typing and enforcement * update api * update job concurrency panel to show disabled queues formatting * check jobId in queueAll fix tests * remove outdated comment * better facial recognition icon * wording wording formatting * fixed tests * fix * formatting & sql * try to fix sql check * more detailed description * update sql * formatting * wording * update `minFaces` description --------- Co-authored-by: Jason Rasmussen <jrasm91@gmail.com> Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-01-18 00:08:48 -05:00
parent 44873b4224
commit 68f52818ae
57 changed files with 1081 additions and 631 deletions
@@ -30,14 +30,14 @@ export class RecognitionConfig extends ModelConfig {
  @Min(0)
  @Max(1)
  @Type(() => Number)
-  @ApiProperty({ type: 'integer' })
+  @ApiProperty({ type: 'number', format: 'float' })
  minScore!: number;

  @IsNumber()
  @Min(0)
  @Max(2)
  @Type(() => Number)
-  @ApiProperty({ type: 'integer' })
+  @ApiProperty({ type: 'number', format: 'float' })
  maxDistance!: number;

  @IsNumber()
@@ -1,6 +1,5 @@
 import { ImmichLogger } from '@app/infra/logger';
 import { Inject, Injectable } from '@nestjs/common';
-import { setTimeout } from 'timers/promises';
 import { usePagination } from '../domain.util';
 import { IBaseJob, IEntityJob, JOBS_ASSET_PAGINATION_SIZE, JobName, QueueName } from '../job';
 import {
@@ -34,13 +33,7 @@ export class SmartInfoService {
  async init() {
    await this.jobRepository.pause(QueueName.SMART_SEARCH);

-    let { isActive } = await this.jobRepository.getQueueStatus(QueueName.SMART_SEARCH);
-    while (isActive) {
-      this.logger.verbose('Waiting for CLIP encoding queue to stop...');
-      await setTimeout(1000).then(async () => {
-        ({ isActive } = await this.jobRepository.getQueueStatus(QueueName.SMART_SEARCH));
-      });
-    }
+    await this.jobRepository.waitForQueueCompletion(QueueName.SMART_SEARCH);

    const { machineLearning } = await this.configCore.getConfig();