Perf: Optimiser builds backend avec image de base (90-95% plus rapide)

Architecture en 2 images: - Image base (audio-classifier-base): deps système + Python (~15min, 1x/semaine) - Image app (audio-classifier-backend): code uniquement (~30s-2min, chaque commit) Fichiers ajoutés: - backend/Dockerfile.base: Image de base avec toutes les dépendances - .gitea/workflows/docker-base.yml: CI pour build de l'image de base - backend/DOCKER_BUILD.md: Documentation complète Fichiers modifiés: - backend/Dockerfile: Utilise l'image de base (FROM audio-classifier-base) - .gitea/workflows/docker.yml: Passe BASE_IMAGE en build-arg Gains de performance: - Build normal: 15-25min → 30s-2min (90-95% plus rapide) - Trigger auto du build base: quand requirements.txt change - Trigger manuel: via interface Gitea Actions 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Feature: Sélection multiple d'instruments dans les filtres
2025-12-26 22:04:13 +01:00 · 2025-12-26 20:10:39 +01:00
9 changed files with 319 additions and 67 deletions
--- a/.gitea/workflows/docker-base.yml
+++ b/.gitea/workflows/docker-base.yml
@@ -0,0 +1,61 @@
+name: Build Base Docker Image
+
+# Build base image only when requirements.txt changes or manually triggered
+on:
+  push:
+    branches:
+      - main
+    paths:
+      - 'backend/requirements.txt'
+      - 'backend/Dockerfile.base'
+  workflow_dispatch:  # Allow manual trigger
+
+env:
+  REGISTRY: git.benoitsz.com
+  IMAGE_BASE: audio-classifier-base
+
+jobs:
+  build-base:
+    name: Build Base Image
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Login to Gitea Container Registry
+        uses: docker/login-action@v3
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ gitea.actor }}
+          password: ${{ secrets.REGISTRY_TOKEN }}
+
+      - name: Extract metadata
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ${{ env.REGISTRY }}/${{ gitea.repository_owner }}/${{ env.IMAGE_BASE }}
+          tags: |
+            type=raw,value=latest
+            type=sha,prefix=sha-,format=short
+
+      - name: Build and push base image
+        uses: docker/build-push-action@v5
+        with:
+          context: ./backend
+          file: ./backend/Dockerfile.base
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ gitea.repository_owner }}/${{ env.IMAGE_BASE }}:buildcache
+          cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ gitea.repository_owner }}/${{ env.IMAGE_BASE }}:buildcache,mode=max
+          platforms: linux/amd64
+
+      - name: Image built successfully
+        run: |
+          echo "✅ Base image built and pushed successfully"
+          echo "📦 Image: ${{ env.REGISTRY }}/${{ gitea.repository_owner }}/${{ env.IMAGE_BASE }}:latest"
+          echo "⏱️  This image will be used by the main backend builds to speed up CI/CD"
--- a/.gitea/workflows/docker.yml
+++ b/.gitea/workflows/docker.yml
@@ -62,10 +62,12 @@ jobs:
          push: true
          build-args: |
            VERSION=${{ steps.version.outputs.VERSION }}
+            BASE_IMAGE=${{ env.REGISTRY }}/${{ gitea.repository_owner }}/audio-classifier-base:latest
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ gitea.repository_owner }}/${{ env.IMAGE_BACKEND }}:buildcache
          cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ gitea.repository_owner }}/${{ env.IMAGE_BACKEND }}:buildcache,mode=max
+          platforms: linux/amd64

  build-frontend:
    name: Build Frontend Image
--- a/backend/DOCKER_BUILD.md
+++ b/backend/DOCKER_BUILD.md
@@ -0,0 +1,136 @@
+# Docker Build Optimization
+
+Cette configuration utilise une approche en 2 images pour accélérer les builds backend de **15-25 minutes** à **30 secondes - 2 minutes**.
+
+## Architecture
+
+### Image 1 : Base (`audio-classifier-base`)
+Contient toutes les dépendances système et Python qui changent rarement :
+- Python 3.9 + apt packages (ffmpeg, libsndfile, etc.)
+- numpy, scipy, essentia-tensorflow
+- Toutes les dépendances de `requirements.txt`
+
+**Build** : ~15 minutes (1 fois par semaine ou quand `requirements.txt` change)
+
+### Image 2 : App (`audio-classifier-backend`)
+Hérite de l'image de base et ajoute uniquement le code applicatif :
+- Code source (`src/`)
+- Fichiers de configuration (`alembic.ini`)
+- Modèles Essentia (`models/`)
+
+**Build** : ~30 secondes - 2 minutes (à chaque commit)
+
+## Workflows CI/CD
+
+### 1. Build de l'image de base (`.gitea/workflows/docker-base.yml`)
+Se déclenche automatiquement quand :
+- `backend/requirements.txt` est modifié
+- `backend/Dockerfile.base` est modifié
+- Déclenchement manuel via l'interface Gitea
+
+```bash
+# Image produite :
+git.benoitsz.com/benoit/audio-classifier-base:latest
+git.benoitsz.com/benoit/audio-classifier-base:sha-<commit>
+```
+
+### 2. Build de l'image app (`.gitea/workflows/docker.yml`)
+Se déclenche à chaque push sur `main` :
+- Utilise l'image de base comme FROM
+- Copie uniquement le code source
+- Build rapide (~30s-2min)
+
+```bash
+# Image produite :
+git.benoitsz.com/benoit/audio-classifier-backend:dev
+git.benoitsz.com/benoit/audio-classifier-backend:dev-<commit>
+```
+
+## Utilisation en local
+
+### Build de l'image de base
+```bash
+cd backend
+docker build -f Dockerfile.base -t audio-classifier-base:local .
+```
+
+### Build de l'image app (utilise l'image de base)
+```bash
+# Depuis la racine du projet
+docker build \
+  --build-arg BASE_IMAGE=audio-classifier-base:local \
+  -f backend/Dockerfile \
+  -t audio-classifier-backend:local \
+  .
+```
+
+### Build direct (sans image de base) - pour tests
+Si tu veux tester un build complet sans dépendre de l'image de base :
+```bash
+# Revenir temporairement au Dockerfile original
+git show HEAD~1:backend/Dockerfile > backend/Dockerfile.monolithic
+docker build -f backend/Dockerfile.monolithic -t audio-classifier-backend:monolithic .
+```
+
+## Mise à jour des dépendances
+
+Quand tu modifies `requirements.txt` :
+
+1. **Push les changements sur `main`**
+   ```bash
+   git add backend/requirements.txt
+   git commit -m "Update dependencies"
+   git push
+   ```
+
+2. **Le workflow `docker-base.yml` se déclenche automatiquement**
+   - Build de la nouvelle image de base (~15 min)
+   - Push vers `git.benoitsz.com/benoit/audio-classifier-base:latest`
+
+3. **Les prochains builds backend utiliseront la nouvelle base**
+   - Builds futurs rapides (~30s-2min)
+
+## Déclenchement manuel
+
+Pour rebuild l'image de base manuellement (sans modifier `requirements.txt`) :
+
+1. Va sur Gitea : `https://git.benoitsz.com/benoit/audio-classifier/actions`
+2. Sélectionne le workflow "Build Base Docker Image"
+3. Clique sur "Run workflow"
+
+## Monitoring
+
+Vérifie les builds dans Gitea Actions :
+- **Base image** : `.gitea/workflows/docker-base.yml`
+- **App image** : `.gitea/workflows/docker.yml`
+
+Les logs montrent la durée de build pour chaque étape.
+
+## Gains de performance attendus
+
+| Scénario | Avant | Après | Gain |
+|----------|-------|-------|------|
+| Build normal (code change) | 15-25 min | 30s-2min | **90-95%** |
+| Build après update deps | 15-25 min | 15-25 min (base) + 30s-2min (app) | 0% (1ère fois) |
+| Builds suivants | 15-25 min | 30s-2min | **90-95%** |
+
+## Troubleshooting
+
+### Erreur "base image not found"
+L'image de base n'existe pas encore dans le registry. Solutions :
+1. Trigger le workflow `docker-base.yml` manuellement
+2. Ou build localement et push :
+   ```bash
+   docker build -f backend/Dockerfile.base -t git.benoitsz.com/benoit/audio-classifier-base:latest backend/
+   docker push git.benoitsz.com/benoit/audio-classifier-base:latest
+   ```
+
+### Build app lent malgré l'image de base
+Vérifie que le build-arg `BASE_IMAGE` est bien passé :
+```yaml
+build-args: |
+  BASE_IMAGE=${{ env.REGISTRY }}/${{ gitea.repository_owner }}/audio-classifier-base:latest
+```
+
+### Dépendances Python pas à jour dans l'app
+L'image de base doit être rebuildée. Trigger `docker-base.yml`.
--- a/backend/Dockerfile
+++ b/backend/Dockerfile
@@ -1,49 +1,12 @@
-# Use amd64 platform for better Essentia compatibility, works with emulation on ARM
-FROM --platform=linux/amd64 python:3.9-slim
+# Use pre-built base image with all dependencies
+# Base image includes: Python 3.9, system deps, numpy, scipy, essentia-tensorflow, all pip deps
+# Only rebuild base when requirements.txt changes
+ARG BASE_IMAGE=git.benoitsz.com/benoit/audio-classifier-base:latest
+FROM ${BASE_IMAGE}

-# Install system dependencies
-RUN apt-get update && apt-get install -y \
-    ffmpeg \
-    libsndfile1 \
-    libsndfile1-dev \
-    gcc \
-    g++ \
-    gfortran \
-    libopenblas-dev \
-    liblapack-dev \
-    pkg-config \
-    curl \
-    build-essential \
-    libyaml-dev \
-    libfftw3-dev \
-    libavcodec-dev \
-    libavformat-dev \
-    libavutil-dev \
-    libswresample-dev \
-    libsamplerate0-dev \
-    libtag1-dev \
-    libchromaprint-dev \
-    && rm -rf /var/lib/apt/lists/*
-
-# Set working directory
+# Working directory already set in base image
 WORKDIR /app

-# Upgrade pip, setuptools, wheel
-RUN pip install --no-cache-dir --upgrade pip setuptools wheel
-
-# Copy requirements
-COPY backend/requirements.txt .
-
-# Install Python dependencies in stages for better caching
-# Using versions compatible with Python 3.9
-RUN pip install --no-cache-dir numpy==1.24.3
-RUN pip install --no-cache-dir scipy==1.11.4
-
-# Install Essentia-TensorFlow - Python 3.9 AMD64 support
-RUN pip install --no-cache-dir essentia-tensorflow
-
-RUN pip install --no-cache-dir -r requirements.txt
-
 # Copy application code
 COPY backend/src/ ./src/
 COPY backend/alembic.ini .
--- a/backend/Dockerfile.base
+++ b/backend/Dockerfile.base
@@ -0,0 +1,59 @@
+# Base image for Audio Classifier Backend
+# This image contains all system dependencies and Python packages
+# Build this image only when dependencies change (requirements.txt updates)
+
+# Use amd64 platform for better Essentia compatibility
+FROM --platform=linux/amd64 python:3.9-slim
+
+LABEL maintainer="benoit.schw@gmail.com"
+LABEL description="Base image with all dependencies for Audio Classifier Backend"
+
+# Install system dependencies
+RUN apt-get update && apt-get install -y \
+    ffmpeg \
+    libsndfile1 \
+    libsndfile1-dev \
+    gcc \
+    g++ \
+    gfortran \
+    libopenblas-dev \
+    liblapack-dev \
+    pkg-config \
+    curl \
+    build-essential \
+    libyaml-dev \
+    libfftw3-dev \
+    libavcodec-dev \
+    libavformat-dev \
+    libavutil-dev \
+    libswresample-dev \
+    libsamplerate0-dev \
+    libtag1-dev \
+    libchromaprint-dev \
+    && rm -rf /var/lib/apt/lists/*
+
+# Set working directory
+WORKDIR /app
+
+# Upgrade pip, setuptools, wheel
+RUN pip install --no-cache-dir --upgrade pip setuptools wheel
+
+# Copy requirements
+COPY requirements.txt .
+
+# Install Python dependencies in stages for better caching
+# Using versions compatible with Python 3.9
+RUN pip install --no-cache-dir numpy==1.24.3
+RUN pip install --no-cache-dir scipy==1.11.4
+
+# Install Essentia-TensorFlow - Python 3.9 AMD64 support
+RUN pip install --no-cache-dir essentia-tensorflow
+
+# Install remaining dependencies
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Verify installations
+RUN python -c "import essentia.standard; import numpy; import scipy; import fastapi; print('All dependencies installed successfully')"
+
+# This image is meant to be used as a base
+# The application code will be copied in the derived Dockerfile
--- a/backend/src/api/routes/tracks.py
+++ b/backend/src/api/routes/tracks.py
@@ -24,6 +24,7 @@ async def get_tracks(
    has_vocals: Optional[bool] = None,
    key: Optional[str] = None,
    instrument: Optional[str] = None,
+    instruments: Optional[List[str]] = Query(None),
    tempo_range: Optional[str] = Query(None, regex="^(slow|medium|fast)$"),
    sort_by: str = Query("analyzed_at", regex="^(analyzed_at|tempo_bpm|duration_seconds|filename|energy)$"),
    sort_desc: bool = True,
@@ -42,7 +43,8 @@ async def get_tracks(
        energy_max: Maximum energy
        has_vocals: Filter by vocal presence
        key: Filter by musical key
-        instrument: Filter by instrument
+        instrument: Filter by instrument (deprecated, use instruments)
+        instruments: Filter by multiple instruments (must have ALL)
        tempo_range: Filter by tempo range (slow: <100, medium: 100-140, fast: >140)
        sort_by: Field to sort by
        sort_desc: Sort descending
@@ -61,6 +63,9 @@ async def get_tracks(
        elif tempo_range == "fast":
            bpm_min = 140.0 if bpm_min is None else max(bpm_min, 140.0)

+    # Use instruments if provided, otherwise fall back to instrument
+    final_instruments = instruments if instruments else ([instrument] if instrument else None)
+
    tracks, total = crud.get_tracks(
        db=db,
        skip=skip,
@@ -73,7 +78,7 @@ async def get_tracks(
        energy_max=energy_max,
        has_vocals=has_vocals,
        key=key,
-        instrument=instrument,
+        instruments=final_instruments,
        sort_by=sort_by,
        sort_desc=sort_desc,
    )
--- a/backend/src/models/crud.py
+++ b/backend/src/models/crud.py
@@ -104,7 +104,7 @@ def get_tracks(
    energy_max: Optional[float] = None,
    has_vocals: Optional[bool] = None,
    key: Optional[str] = None,
-    instrument: Optional[str] = None,
+    instruments: Optional[List[str]] = None,
    sort_by: str = "analyzed_at",
    sort_desc: bool = True,
 ) -> Tuple[List[AudioTrack], int]:
@@ -122,7 +122,7 @@ def get_tracks(
        energy_max: Maximum energy (0-1)
        has_vocals: Filter by vocal presence
        key: Filter by musical key
-        instrument: Filter by instrument
+        instruments: Filter by instruments (track must have ALL instruments in the list)
        sort_by: Field to sort by
        sort_desc: Sort descending if True

@@ -168,7 +168,9 @@ def get_tracks(
    if key:
        query = query.filter(AudioTrack.key == key)

-    if instrument:
+    if instruments:
+        # Track must have ALL specified instruments
+        for instrument in instruments:
            query = query.filter(AudioTrack.instruments.any(instrument))

    # Get total count before pagination
--- a/frontend/components/FilterPanel.tsx
+++ b/frontend/components/FilterPanel.tsx
@@ -93,23 +93,40 @@ export default function FilterPanel({
        </select>
      </div>

-      {/* Instrument Filter */}
+      {/* Instrument Filter - Multiple Selection */}
      <div>
        <label className="block text-sm font-semibold text-slate-700 mb-2">
-          Instrument
+          Instruments
        </label>
-        <select
-          value={localFilters.instrument || ""}
-          onChange={(e) => handleFilterChange("instrument", e.target.value || undefined)}
-          className="w-full px-3 py-2 bg-slate-50 border border-slate-300 rounded-lg text-sm focus:outline-none focus:ring-2 focus:ring-orange-500 focus:border-transparent"
+        <div className="space-y-2 max-h-48 overflow-y-auto bg-slate-50 border border-slate-300 rounded-lg p-2">
+          {availableInstruments.length === 0 ? (
+            <p className="text-xs text-slate-500 p-2">Aucun instrument disponible</p>
+          ) : (
+            availableInstruments.map((instrument) => {
+              const isSelected = localFilters.instruments?.includes(instrument) || false
+              return (
+                <label
+                  key={instrument}
+                  className="flex items-center gap-2 p-2 hover:bg-slate-100 rounded cursor-pointer transition-colors"
                >
-          <option value="">Tous les instruments</option>
-          {availableInstruments.map((instrument) => (
-            <option key={instrument} value={instrument}>
-              {instrument}
-            </option>
-          ))}
-        </select>
+                  <input
+                    type="checkbox"
+                    checked={isSelected}
+                    onChange={(e) => {
+                      const currentInstruments = localFilters.instruments || []
+                      const newInstruments = e.target.checked
+                        ? [...currentInstruments, instrument]
+                        : currentInstruments.filter(i => i !== instrument)
+                      handleFilterChange("instruments", newInstruments.length > 0 ? newInstruments : undefined)
+                    }}
+                    className="w-4 h-4 text-orange-500 border-slate-300 rounded focus:ring-2 focus:ring-orange-500 focus:ring-offset-0"
+                  />
+                  <span className="text-sm text-slate-700">{instrument}</span>
+                </label>
+              )
+            })
+          )}
+        </div>
      </div>

      {/* Key Filter */}
@@ -165,10 +182,16 @@ export default function FilterPanel({
                <span className="font-medium text-slate-800">{localFilters.mood}</span>
              </div>
            )}
-            {localFilters.instrument && (
-              <div className="flex items-center justify-between text-xs">
-                <span className="text-slate-600">Instrument:</span>
-                <span className="font-medium text-slate-800">{localFilters.instrument}</span>
+            {localFilters.instruments && localFilters.instruments.length > 0 && (
+              <div className="text-xs">
+                <span className="text-slate-600">Instruments:</span>
+                <div className="flex flex-wrap gap-1 mt-1">
+                  {localFilters.instruments.map((instrument) => (
+                    <span key={instrument} className="inline-flex items-center px-2 py-0.5 rounded text-xs bg-emerald-50 text-emerald-700">
+                      {instrument}
+                    </span>
+                  ))}
+                </div>
              </div>
            )}
            {localFilters.key && (
--- a/frontend/lib/types.ts
+++ b/frontend/lib/types.ts
@@ -60,6 +60,7 @@ export interface FilterParams {
  has_vocals?: boolean
  key?: string
  instrument?: string
+  instruments?: string[] // Multiple instruments filter
  tempo_range?: 'slow' | 'medium' | 'fast' // Lent (<100), Moyen (100-140), Rapide (>140)
  sort_by?: 'analyzed_at' | 'tempo_bpm' | 'duration_seconds' | 'filename' | 'energy'
  sort_desc?: boolean