Commit Graph

51 Commits

Author SHA1 Message Date
Jason Rasmussen 5cd186d3d4 refactor: duplicate queries (#19136) 2025-06-12 14:23:02 -04:00
xCJPECKOVERx 14d785cec9 feat(server): Add album filter to search (#18985)
* - updated dtos
- added inAlbums to search builder
- only check isNotInAlbum if albumIds is blank/empty

* - consider inAlbums as OR

* - make open-api-dart

* - lint & format

* - remove inAlbums groupBy clause

* - merge main open-api

* - make open-api

* - inAlbums filter AND instead of OR
2025-06-09 11:11:43 -04:00
Jonathan Gilbert fb4be6e231 feat(server): add /search/statistics resource (#18885) 2025-06-06 21:12:53 -04:00
Mert fa22e865a4 fix(server): tighten asset visibility (#18699)
* tighten visibility

* update sql

* elevated access util function

* fix potential sync issue

* include in user stats

* include hidden assets in size usage

* filter visibility in search duplicates query

* stack visibility
2025-06-02 09:33:08 -05:00
Mert 0d773af6c3 feat: vectorchord (#18042)
* wip

auto-detect available extensions

auto-recovery, fix reindexing check

use original image for ml

* set probes

* update image for sql checker

update images for gha

* cascade

* fix new instance

* accurate dummy vector

* simplify dummy

* preexisiting pg docs

* handle different db name

* maybe fix sql generation

* revert refreshfaces sql change

* redundant switch

* outdated message

* update docker compose files

* Update docs/docs/administration/postgres-standalone.md

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>

* tighten range

* avoid always printing "vector reindexing complete"

* remove nesting

* use new images

* add vchord to unit tests

* debug e2e image

* mention 1.107.2 in startup error

* support new vchord versions

---------

Co-authored-by: Daniel Dietzler <36593685+danieldietzler@users.noreply.github.com>
2025-05-20 08:36:43 -05:00
Alex d33ce13561 feat(server): visibility column (#17939)
* feat: private view

* pr feedback

* sql generation

* feat: visibility column

* fix: set visibility value as the same as the still part after unlinked live photos

* fix: test

* pr feedback
2025-05-06 12:12:48 -05:00
Jason Rasmussen 1fde02ee1e chore: remove unused types and code (#17999) 2025-04-30 13:41:23 -04:00
Thomas 3ce353393a chore(server): don't insert embeddings if the model has changed (#17885)
* chore(server): don't insert embeddings if the model has changed

We're moving away from the heuristic of waiting for queues to complete. The job
which inserts embeddings can simply check if the model has changed before
inserting, rather than attempting to lock the queue.

* more robust dim size update

* use check constraint

* index command cleanup

* add create statement

* update medium test, create appropriate extension

* new line

* set dimension size when running on all assets

* why does it want braces smh

* take 2

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
2025-04-29 14:23:01 -04:00
Daniel Dietzler dd1fcd5be5 chore: remove asset entity (#17703) 2025-04-18 21:39:56 +00:00
Daniel Dietzler 52ae06c119 refactor: remove album entity, update types (#17450) 2025-04-18 23:10:34 +02:00
Jason Rasmussen 8da5f21fcf refactor: medium tests (#17634) 2025-04-15 15:54:23 -04:00
Jason Rasmussen b710ad36f3 feat: upgrade kysely (#17630)
* feat: upgrade kysely

* chore: pr feedback
2025-04-15 13:26:56 -04:00
Mert 25f2b9602f refactor(server): remove face, person and face search entities (#17535)
* remove face, person and face search entities

update tests and mappers

check if face relation exists

update sql

unused imports

* pr feedback

generate sql, remove unused imports
2025-04-11 14:44:45 -04:00
Jason Rasmussen 8203b6c450 refactor: stop using geodata entity type (#17444) 2025-04-08 00:15:43 -04:00
Abhinav Valecha b621281351 feat(server): Avoid face match with people born after file creation #4743 (#16918)
* feat(server): Avoid face matching with people born after file creation date (#4743)

* lint

* add medium tests for facial recognition

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2025-04-02 10:37:26 -05:00
Jason Rasmussen 2cdbb0a37c refactor: database repository (#16593)
* refactor: database repository

* fix error reindex check

* chore: remove WIP code

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
2025-03-06 13:33:24 -05:00
Mert a708649504 fix(server): skip stacked assets in duplicate detection (#16380)
* skip stacked assets in duplicate detection

* update sql

* handle stacking after duplicate detection runs
2025-02-27 10:16:13 -06:00
Jason Antwi-Appah 34b88bb47a feat(web): support searching by EXIF rating (#16208)
* Add rating to search DTO

* Add search by EXIF rating in search query builder

* Generate OpenAPI spec

* Add rating filter on web

* Add rating filter to search docs

* Format / lint

* Hide rating filter if ratings are disabled

* chore: component order in form

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2025-02-20 16:17:06 +00:00
Jason Rasmussen 9d85272c2b refactor: repositories (#16036) 2025-02-11 14:08:13 -05:00
sudbrack 206412267a fix(server): /search/random API returns same assets every call (#15682)
* Fix for server searchRandom function not returning random results

* Fix lint
2025-01-26 14:06:18 +00:00
Mert 72fa31f9e9 fix(server): changing vector dim size (#15630) 2025-01-24 20:01:24 -05:00
Mert ba105d9f19 fix(server): searchRandom response (#15580)
* fix searchRandom

* add e2e

* set outer limit
2025-01-24 00:41:54 -05:00
Jason Rasmussen d3446f3092 refactor: logging repository (#15540) 2025-01-23 08:31:30 -05:00
Daniel Dietzler 332a865ce6 refactor: migrate person repository to kysely (#15242)
* refactor: migrate person repository to kysely

* `asVector` begone

* linting

* fix metadata faces

* update test

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
2025-01-21 13:12:28 -05:00
Mert 2e12c46980 refactor(server): use kysely (#12857) 2025-01-09 11:15:41 -05:00
Mert 25ca3b1124 refactor(server): use includeNull in query for search suggestions (#14626)
* use `includeNull`

* push down `includeNull` into query, inner joins

* remove filter

* update sql

* fix tests

* maybe fix e2e

* more e2e tests

* handle no exif row

* whoops

* update sql
2024-12-10 16:22:37 -05:00
Mert 34fae31fd4 fix(server): remove unnecessary guc settings for vector search (#14237)
remove unnecessary guc settings
2024-11-19 15:37:39 -06:00
Jason Rasmussen 64831e2328 refactor: remove smart info table (#13985) 2024-11-07 11:25:10 -05:00
Jason Rasmussen e6a666f1d3 refactor(server): telemetry (#13588)
refactor: telemetry
2024-10-21 23:52:30 +00:00
Mert 063969ca05 fix(server): searching with both personIds and withPeople (#13254)
* use cte

* linting
2024-10-07 15:44:04 -04:00
Jason Rasmussen 3ac00b0ffa refactor(server): db env (#13167) 2024-10-03 17:48:40 -04:00
Mert 7adb35e59e fix(server): /search/random failing with certain options (#13040)
* fix relation handling, remove pagination

* update api, sql

* update mock
2024-09-30 00:29:35 -04:00
Jason Rasmussen 12da250028 refactor: enums (#12988) 2024-09-27 10:28:42 -04:00
jschwalbe 9f8a7e0bea feat(server): sort assets randomly from the API 'api/search/metadata' endpoint by including 'order': 'rand' in the API call. (#12741)
feat(server): search metadata random sort order

Co-authored-by: Jason Rasmussen <jason@rasm.me>
2024-09-23 12:09:26 -04:00
Jason Rasmussen 4f25cec6df refactor(server): metadata repository (#12759) 2024-09-18 08:44:22 -04:00
Jason Rasmussen b288241a5c refactor(server): enums (#11809) 2024-08-15 06:57:01 -04:00
Mert 4ed75f2ac9 refactor(server): add config events for clip (#11575)
use config events for clip, add tests

formatting
2024-08-04 21:00:36 +00:00
Mert 42f3b50422 fix(server): /places entries sometimes not ordered alphabetically (#10514) 2024-06-20 23:48:19 -04:00
Mert 79a8ab71ef fix(server): reindex after changing to a model with a different dimension size (#10496)
reindex after truncating
2024-06-19 17:25:02 -04:00
Mert 6b1b5054f8 feat(server): separate face search relation (#10371)
* wip

* various fixes

* new migration

* fix test

* add face search entity, update sql

* update e2e

* set storage to external
2024-06-16 19:25:27 +00:00
Mert e7c8501930 fix(server): search duplicates of the same asset type (#9747)
* search by type

* make sql

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2024-05-26 22:04:23 +00:00
Jason Rasmussen 1df7be8436 refactor(server): version logic (#9615)
* refactor(server): version

* test: better version and log checks
2024-05-20 20:31:36 -04:00
Mert 64636c0618 feat(server): near-duplicate detection (#8228)
* duplicate detection job, entity, config

* queueing

* job panel, update api

* use embedding in db instead of fetching

* disable concurrency

* only queue visible assets

* handle multiple duplicateIds

* update concurrent queue check

* add provider

* add web placeholder, server endpoint, migration, various fixes

* update sql

* select embedding by default

* rename variable

* simplify

* remove separate entity, handle re-running with different threshold, set default back to 0.02

* fix tests

* add tests

* add index to entity

* formatting

* update asset mock

* fix `upsertJobStatus` signature

* update sql

* formatting

* default to 0.03

* optimize clustering

* use asset's `duplicateId` if present

* update sql

* update tests

* expose admin setting

* refactor

* formatting

* skip if ml is disabled

* debug trash e2e

* remove from web

* remove from sidebar

* test if ml is disabled

* update sql

* separate duplicate detection from clip in config, disable by default for now

* fix doc

* lower minimum `maxDistance`

* update api

* Add and Use Duplicate Detection Feature Flag (#9364)

* Add Duplicate Detection Flag

* Use Duplicate Detection Flag

* Attempt Fixes for Failing Checks

* lower minimum `maxDistance`

* fix tests

---------

Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>

* chore: fixes and additions after rebase

* chore: update api (remove new Role enum)

* fix: left join smart search so getAll works without machine learning

* test: trash e2e go back to checking length of assets is zero

* chore: regen api after rebase

* test: fix tests after rebase

* redundant join

---------

Co-authored-by: Nicholas Flamy <30300649+NicholasFlamy@users.noreply.github.com>
Co-authored-by: Zack Pollard <zackpollard@ymail.com>
Co-authored-by: Zack Pollard <zack@futo.org>
2024-05-16 18:08:37 +01:00
AmAn Sharma 6e6deec40c feat: use ILoggerRepository (#8855)
* Migrate ImmichLogger over to injected ILoggerRepository

* chore: cleanup and tests

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-04-16 17:30:31 -04:00
Mert 5bc9158724 fix(server): penalize null geodata fields when searching places (#8408) 2024-03-31 10:59:11 -04:00
Jason Rasmussen 4b39d37cae fix: sql generation issues (#8361)
chore: fix sql gen issues
2024-03-30 00:16:06 -04:00
Mert 4a6a0aa142 fix(server): places page not working with partner sharing (#8257) 2024-03-25 01:59:11 -04:00
Mert b07a565e34 chore(server): change upsert signature for search repo (#8210)
* upsert embedding

* remove unused imports
2024-03-23 14:37:06 -04:00
Daniel Dietzler b588a87d4a chore(server): rename domain repositories -> interfaces (#8147)
rename domain repositories
2024-03-21 06:59:49 -05:00
Jason Rasmussen 16d0df796c refactor: infra folder (#8138) 2024-03-20 23:15:09 -04:00