* fix(server): Split database queries based on PostgreSQL bound params limit
PostgreSQL uses a 16-bit integer to indicate the number of bound
parameters.
This means that the maximum number of parameters for any query is 65535.
Any query that tries to bind more than that (e.g. searching by a list of
IDs) requires splitting the query into multiple chunks.
This change includes refactoring every Repository that runs queries
using a list of ids, and either flattening or merging results.
Fixes#5788, #5997.
Also, potentially a fix for #4648 (at least based on
[this comment](https://github.com/immich-app/immich/issues/4648#issuecomment-1826134027)).
References:
* https://github.com/typeorm/typeorm/issues/7565
* [PostgreSQL message format - Bind](https://www.postgresql.org/docs/15/protocol-message-formats.html#PROTOCOL-MESSAGE-FORMATS-BIND)
* misc: Create Chunked decorator to simplify implementation
* feat: Add ChunkedArray/ChunkedSet decorators
* feat(server): Throw error when PostgreSQL version is not within the supported versions
The pgvecto.rs extension, though not distributed, can be built for PostgreSQL 12 and 13.
An installation of PostgreSQL 12 with the pgvecto.rs extensions installed will not be caught by immich.
This causes immich to attempt to run the database migrations without having a proper environment.
With assertPostgresql the server will throw an error if the PostgreSQL version is not within the supported range.
* Replaced assertion with lesser than comparison
As requested by @zackpollard
* Changed the comparison to use the minPostgresVersion variable.
If we define one we might as well use it. makes changing the versioning later easier
* Added two new tests, modified two existing tests
`should return if minimum supported PostgreSQL and vectors version are installed`:
Check if init returns properly and that getPostgresVersion is called twice
`should thrown an error if PostgreSQL version is below minimum supported version`:
Checks if the init function correctly returns an error
`should suggest image with postgres ${major} if database is ${major}`:
Modified to set MockResolvedValue instead of MockResolvedValueOnce. With the new check we get the PostgreSQL version twice. So it needs to be set during the entire test.
`should not suggest image if postgres version is not in 14, 15 or 16`:
Modified the bounds to [14, 18]. Because values below 14 now will not get called.
Also Modified to call `getPostgresVersion.MockResolvedValueOnce` for twice, because it gets called twice.
* Fixed two mistakes in the jest functions from previous commit #2abcb60
`should thrown an error if PostgreSQL version is below minimum supported version`:
The regex function I wrote mistakingly used the negate function which check that the error *did not* contain the phrase "PostgreSQL". Which is the opposite
`should not suggest image if postgres version is not in 14, 15 or 16`:
confused bounds for a normal javascript array. Changed the test to only check for values above 16. As values below 14 will get thrown out by test `should return if minimum supported PostgreSQL and vectors version are installed`
I apologise for the mistakes in my previous commit.
* Format fix
---------
Co-authored-by: max <wak@vanling.net>
The current `removeAsset` implementation just builds the query but does
not execute it. That also seems to be the reason the `@GenerateSql`
decorator was commented out.
* feat(web): onboarding
* feat: openapi
* feat: modulization
* feat: page advancing
* Animation
* Add storage templaete settings
* sql
* more style
* Theme
* information and styling
* hide/show table
* Styling
* Update user property
* fix test
* fix test:
* fix e2e
* test
* Update web/src/lib/components/onboarding-page/onboarding-hello.svelte
Co-authored-by: bo0tzz <git@bo0tzz.me>
* naming
* use System Metadata
* better return type
* onboarding using server metadata
* revert previous changes in user entity
* sql
* test web
* fix test server
* server/web test
* more test
* consolidate color theme change logic
* consolidate save button to storage template
* merge main
* fix web
---------
Co-authored-by: bo0tzz <git@bo0tzz.me>
* fix(deps): update dependency @nestjs/schedule to v4
* fix: change CronJob in addCronJob to match new type required by nestjs schedule module
---------
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Zack Pollard <zackpollard@ymail.com>
* Update libraries.md - External Library step by step set up guide added
When setting up immmich external libraries, I found some missing details around the relationship between external paths and import paths. By providing an example, I am hoping it helps new users.
* chore: fix formatting
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* Update hardware-transcoding.md
I accidentally closed#5924 for some odd reason. Copied the same code changes to this one.
* updated formatting
* Update hardware-transcoding.md
Added line 66 saying that `NVIDIA_DRIVER_CAMPABILITITES` is no longer required.
* feat(server): Enqueue jobs in bulk
The Job Repository now has a `queueAll` method, that enqueues messages
in bulk (using BullMQ's
[`addBulk`](https://docs.bullmq.io/guide/queues/adding-bulks)),
improving performance when many jobs must be enqueued within the same
operation.
Primary change is in `src/domain/job/job.service.ts`, and other services
have been refactored to use `queueAll` when useful.
As a simple local benchmark, triggering a full thumbnail generation
process over a library of ~1,200 assets and ~350 faces went from
**~600ms** to **~250ms**.
* fix: Review feedback
Alex The Bot