Commit Graph

15 Commits

Author SHA1 Message Date
Mert ae7f174948 refactor: rename clip -> smart search (#6713) 2024-01-29 09:51:22 -05:00
Mert 68f52818ae feat(server): separate face clustering job (#5598)
* separate facial clustering job

* update api

* fixed some tests

* invert clustering

* hdbscan

* update api

* remove commented code

* wip dbscan

* cleanup

removed cluster endpoint

remove commented code

* fixes

updated tests

minor fixes and formatting

fixed queuing

refinements

* scale search range based on library size

* defer non-core faces

* optimizations

removed unused query option

* assign faces individually for correctness

fixed unit tests

remove unused method

* don't select face embedding

update sql

linting

fixed ml typing

* updated job mock

* paginate people query

* select face embeddings because typeorm

* fix setting face detection concurrency

* update sql

formatting

linting

* simplify logic

remove unused imports

* more specific delete signature

* more accurate typing for face stubs

* add migration

formatting

* chore: better typing

* don't select embedding by default

remove unused import

* updated sql

* use normal try/catch

* stricter concurrency typing and enforcement

* update api

* update job concurrency panel to show disabled queues

formatting

* check jobId in queueAll

fix tests

* remove outdated comment

* better facial recognition icon

* wording

wording

formatting

* fixed tests

* fix

* formatting & sql

* try to fix sql check

* more detailed description

* update sql

* formatting

* wording

* update `minFaces` description

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-01-18 00:08:48 -05:00
Michael Manganiello 4a5b8c3770 feat(server): Enqueue jobs in bulk (#5974)
* feat(server): Enqueue jobs in bulk

The Job Repository now has a `queueAll` method, that enqueues messages
in bulk (using BullMQ's
[`addBulk`](https://docs.bullmq.io/guide/queues/adding-bulks)),
improving performance when many jobs must be enqueued within the same
operation.

Primary change is in `src/domain/job/job.service.ts`, and other services
have been refactored to use `queueAll` when useful.

As a simple local benchmark, triggering a full thumbnail generation
process over a library of ~1,200 assets and ~350 faces went from
**~600ms** to **~250ms**.

* fix: Review feedback
2024-01-01 15:45:42 -05:00
Mert 8119d4bb26 chore(server): refactor locks (#5953)
* lock refactor

* add mocks

* add await

* move database repo injection to service

* update tests

* add mock implementation

* remove unused imports

* this
2023-12-27 18:36:51 -05:00
Mert 092a23fd7f feat(server,ml): remove image tagging (#5903)
* remove image tagging

* updated lock

* fixed tests, improved logging

* be nice

* fixed tests
2023-12-20 20:47:56 -05:00
Mert cb1201e690 chore(web): update job dashboard (#5745)
* rename clip encoding to smart search

* update job subtitles

* update api

* update smart search job title and subtitle

* fix `getJobName`

* change smart search icon

* formatting

* wording

* update reference to clip

* formatting

* update reference to Encode CLIP
2023-12-16 10:50:46 -06:00
Jason Rasmussen 9768931275 feat(web,server)!: runtime log level (#5672)
* feat: change log level at runtime

* chore: open api

* chore: prefer env over runtime

* chore: remove default env value
2023-12-14 16:55:40 +00:00
Jason Rasmussen 1e99ba8167 feat: use pgvecto.rs (#3605) 2023-12-08 11:15:46 -05:00
Jason Rasmussen 2ea080cacd refactor: domain repositories (#4403) 2023-10-09 14:25:03 +00:00
Daniel Dietzler 0243570c0b fix(server): make system config core singleton (#4392)
* make system config core singleton

* refactor

* fix tests

* chore: fix tests

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-09 00:51:03 +00:00
Jason Rasmussen 816d040d81 fix(server): lint import order (#3974)
* fix: use prettier extension

* chore: format fix
2023-09-04 21:45:59 +02:00
Mert bcc36d14a1 feat(ml)!: customizable ML settings (#3891)
* consolidated endpoints, added live configuration

* added ml settings to server

* added settings dashboard

* updated deps, fixed typos

* simplified modelconfig

updated tests

* Added ml setting accordion for admin page

updated tests

* merge `clipText` and `clipVision`

* added face distance setting

clarified setting

* add clip mode in request, dropdown for face models

* polished ml settings

updated descriptions

* update clip field on error

* removed unused import

* add description for image classification threshold

* pin safetensors for arm wheel

updated poetry lock

* moved dto

* set model type only in ml repository

* revert form-data package install

use fetch instead of axios

* added slotted description with link

updated facial recognition description

clarified effect of disabling tasks

* validation before model load

* removed unnecessary getconfig call

* added migration

* updated api

updated api

updated api

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-08-29 08:58:00 -05:00
Jason Rasmussen 8211afb726 feat(web,server)!: configure machine learning via the UI (#3768) 2023-08-25 06:15:03 +02:00
Mert 017214fd56 fix(server): empty tag responses should be considered valid (#2993)
* accept empty tag array

* renamed test
2023-06-28 09:40:30 -05:00
Jason Rasmussen 8ebac41318 refactor(server)*: tsconfigs (#2689)
* refactor(server): tsconfigs

* chore: dummy commit

* fix: start.sh

* chore: restore original entry scripts
2023-06-08 10:01:07 -05:00