cjones/immich
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(ml)!: cuda and openvino acceleration (#5619)

Browse Source 
* cuda and openvino ep, refactor, update dockerfile

* updated workflow

* typing fixes

* added tests

* updated ml test gh action

* updated README

* updated docker-compose

* added compute to hwaccel.yml

* updated gh matrix

updated gh matrix

updated gh matrix

updated gh matrix

updated gh matrix

give up

* remove cuda/arm64 build

* add hwaccel image tags to docker-compose

* remove unnecessary quotes

* add suffix to git tag

* fixed kwargs in base model

* armnn ld_library_path

* update pyproject.toml

* add armnn workflow

* formatting

* consolidate hwaccel files, update docker compose

* update hw transcoding docs

* add ml hwaccel docs

* update dev and prod docker-compose

* added armnn prerequisite docs

* support 3.10

* updated docker-compose comments

* formatting

* test coverage

* don't set arena extend strategy for openvino

* working openvino

* formatting

* fix dockerfile

* added type annotation

* add wsl configuration for openvino

* updated lock file

* copy python3

* comment out extends section

* fix platforms

* simplify workflow suffix tagging

* simplify aio transcoding doc

* update docs and workflow for `hwaccel.yml` change

* revert docs
This commit is contained in:
Mert
2024-01-21 18:22:39 -05:00
committed by GitHub
parent 6b419a984c
commit 95cfe22866
23 changed files with 962 additions and 460 deletions
Download Patch File Download Diff File Expand all files Collapse all files

86
machine-learning/Dockerfile
View File

@@ -1,43 +1,74 @@
FROM python:3.11-bookworm@sha256:497c00ec2cff14316a6859c4e30fc88e7ab1f11dd254fb43b8f4b201ca657596 as builder
ARG DEVICE=cpu
FROM python:3.11-bookworm@sha256:ba7a7ac30c38e119c4304f98ef0e188f90f4f67a958bb6899da9defb99bfb471 as builder-cpu
FROM openvino/ubuntu22_runtime:2023.1.0 as builder-openvino
USER root
RUN apt-get update && apt-get install -y --no-install-recommends python3-dev
FROM builder-cpu as builder-cuda
FROM builder-cpu as builder-armnn
ENV ARMNN_PATH=/opt/armnn
COPY ann /opt/ann
RUN mkdir /opt/armnn && \
curl -SL "https://github.com/ARM-software/armnn/releases/download/v23.11/ArmNN-linux-aarch64.tar.gz" | tar -zx -C /opt/armnn && \
cd /opt/ann && \
sh build.sh
FROM builder-${DEVICE} as builder
ARG DEVICE
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=true
PIP_NO_CACHE_DIR=true \
VIRTUAL_ENV="/opt/venv" \
PATH="/opt/venv/bin:${PATH}"
RUN pip install --upgrade pip && pip install poetry
RUN poetry config installer.max-workers 10 && \
poetry config virtualenvs.create false
RUN python -m venv /opt/venv
ENV VIRTUAL_ENV="/opt/venv" PATH="/opt/venv/bin:${PATH}"
poetry config virtualenvs.create false
RUN python3 -m venv /opt/venv
COPY poetry.lock pyproject.toml ./
RUN poetry install --sync --no-interaction --no-ansi --no-root --only main
RUN poetry install --sync --no-interaction --no-ansi --no-root --with ${DEVICE} --without dev
FROM python:3.11-slim-bookworm@sha256:cfd7ed5c11a88ce533d69a1da2fd932d647f9eb6791c5b4ddce081aedf7f7876 as prod-cpu
ARG TARGETPLATFORM
ENV ARMNN_PATH=/opt/armnn
COPY ann /opt/ann
RUN if [ "$TARGETPLATFORM" = "linux/arm64" ]; then \
mkdir /opt/armnn && \
curl -SL "https://github.com/ARM-software/armnn/releases/download/v23.11/ArmNN-linux-aarch64.tar.gz" | tar -zx -C /opt/armnn && \
cd /opt/ann && \
sh build.sh; \
else \
mkdir /opt/armnn; \
fi
FROM openvino/ubuntu22_runtime:2023.1.0 as prod-openvino
USER root
FROM nvidia/cuda:11.8.0-cudnn8-runtime-ubuntu22.04 as prod-cuda
FROM python:3.11-slim-bookworm@sha256:637774748f62b832dc11e7b286e48cd716727ed04b45a0322776c01bc526afc3
ARG TARGETPLATFORM
RUN apt-get update && apt-get install -y --no-install-recommends tini libmimalloc2.0 && rm -rf /var/lib/apt/lists/*
COPY --from=builder-cuda /usr/local/bin/python3 /usr/local/bin/python3
COPY --from=builder-cuda /usr/local/lib/python3.11 /usr/local/lib/python3.11
COPY --from=builder-cuda /usr/local/lib/libpython3.11.so /usr/local/lib/libpython3.11.so
RUN if [ "$TARGETPLATFORM" = "linux/arm64" ]; then \
apt-get update && apt-get install -y --no-install-recommends ocl-icd-libopencl1 mesa-opencl-icd && \
FROM prod-cpu as prod-armnn
ENV LD_LIBRARY_PATH=/opt/armnn
RUN apt-get update && apt-get install -y --no-install-recommends ocl-icd-libopencl1 mesa-opencl-icd && \
rm -rf /var/lib/apt/lists/* && \
mkdir --parents /etc/OpenCL/vendors && \
echo "/usr/lib/libmali.so" > /etc/OpenCL/vendors/mali.icd && \
mkdir /opt/armnn; \
fi
echo "/usr/lib/libmali.so" > /etc/OpenCL/vendors/mali.icd
COPY --from=builder-armnn \
/opt/armnn/libarmnn.so.?? \
/opt/armnn/libarmnnOnnxParser.so.?? \
/opt/armnn/libarmnnDeserializer.so.?? \
/opt/armnn/libarmnnTfLiteParser.so.?? \
/opt/armnn/libprotobuf.so.?.??.?.? \
/opt/ann/libann.s[o] \
/opt/ann/build.sh \
/opt/armnn/
FROM prod-${DEVICE} as prod
RUN apt-get update && \
apt-get install -y --no-install-recommends tini libmimalloc2.0 && \
rm -rf /var/lib/apt/lists/*
WORKDIR /usr/src/app
ENV NODE_ENV=production \
@@ -45,8 +76,7 @@ ENV NODE_ENV=production \
PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PATH="/opt/venv/bin:$PATH" \
PYTHONPATH=/usr/src \
LD_LIBRARY_PATH=/opt/armnn
PYTHONPATH=/usr/src
# prevent core dumps
RUN echo "hard core 0" >> /etc/security/limits.conf && \
@@ -54,10 +84,8 @@ RUN echo "hard core 0" >> /etc/security/limits.conf && \
echo 'ulimit -S -c 0 > /dev/null 2>&1' >> /etc/profile
COPY --from=builder /opt/venv /opt/venv
COPY --from=builder /opt/armnn/libarmnn.so.?? /opt/armnn/libarmnnOnnxParser.so.?? /opt/armnn/libarmnnDeserializer.so.?? /opt/armnn/libarmnnTfLiteParser.so.?? /opt/armnn/libprotobuf.so.?.??.?.? /opt/ann/libann.s[o] /opt/ann/build.sh /opt/armnn
COPY ann/ann.py /usr/src/ann/ann.py
COPY start.sh log_conf.json ./
COPY app .
ENTRYPOINT ["tini", "--"]
CMD ["./start.sh"]