fix(ai): BUG-044 capacités des modèles lues chez le fournisseur (Mistral vision)
CI / lint (push) Successful in 1m20s
CI / security (push) Successful in 53s
CI / test (push) Successful in 2m27s
CI / build (push) Successful in 50s
CI / e2e (push) Successful in 10m48s

Le panneau de modèle par défaut et la bulle ⓘ n'affichaient aucun modèle Mistral
« Vision capable » alors que GET api.mistral.ai/v1/models en déclare 28 : la table
de capacités était entièrement statique et aucun de ses motifs ne correspondait aux
familles Mistral actuelles (seul `pixtral`, retiré de l'API, les matchait).

- backend/provider_capabilities.py (nouveau) : capacités déclarées par le
  fournisseur (Mistral `capabilities`, OpenRouter `architecture`), détectées par
  la forme du payload, snapshot en cache process-wide (TTL 30 min, surchargeable
  par AI_CAPABILITIES_TTL_SECONDS) rempli par GET /api/config/ai-models.
- backend/model_capabilities.py : une déclaration prime sur la table statique
  pour chaque drapeau mentionné ; la table ne comble que le reste (Mistral ne
  déclare jamais `embedding`). Table corrigée pour le repli hors ligne : familles
  vision Mistral (ministral, magistral, mistral-small, mistral-medium,
  mistral-vibe-cli, labs-leanstral), mistral-ocr = vision sans chat, et défaut du
  fournisseur Mistral sans `embeddings` (mistral-large / codestral n'étaient plus
  des « embedders »).
- backend/ai_routes.py : GET /api/ai/model-capabilities reste sans appel réseau
  (cache froid → table statique).
- Tests : tests/test_provider_capabilities.py (nouveau), TestMistralFamilies et
  TestDeclaredCapabilities (bout en bout via l'API).
- Docs : CHANGELOG [Unreleased], registre + journal ISSUES_TODOLIST,
  fiche docs/features/ai-provider-picker.md (§L).

Vérifié : 28/28 modèles vision déclarés par Mistral détectés (0 avant), 0 écart
dans les deux sens ; pytest 1007 passed / 6 skipped ; ruff 0 (backend) ; mypy 0 ;
tests frontend unit 9/9 + IA 66/66 + validate-imports 37 modules ; instance de test
reconstruite et vérifiée sur http://localhost:2020.
This commit is contained in:
2026-09-15 09:26:31 -04:00
parent 140a8f6efe
commit b69cb9b0f8
10 changed files with 723 additions and 23 deletions
+7 -3
View File
@@ -253,10 +253,14 @@ async def api_model_capabilities(
model: str = Query("", description="Model identifier (optional)"),
current_user=Depends(require_auth),
):
"""Return the curated capability flags for a provider/model pair.
"""Return the capability flags for a provider/model pair.
The table is static (see ``backend.model_capabilities``) so the answer is
available offline and never depends on a provider API call.
Two layers (BUG-044): flags the provider itself declares in its models
endpoint (Mistral ``capabilities``, OpenRouter ``architecture``) win, the
curated table in ``backend.model_capabilities`` fills the rest. The
declaration snapshot is populated by ``GET /api/config/ai-models``; when it
is cold (or the provider declares nothing) the curated table answers alone,
so this endpoint never performs a blocking provider call.
"""
return {
"provider": provider,
+7
View File
@@ -3738,6 +3738,7 @@ async def api_list_ai_models(provider: str = Query(...), current_user=Depends(re
provider = provider.lower()
from backend.model_capabilities import get_capabilities_for_models
from backend.provider_capabilities import remember_declared_capabilities
all_providers = ("deepseek", "openrouter", "gemini", "nvidia", "qwencloud", "xiaomi", "mistral")
if provider not in all_providers:
@@ -3791,6 +3792,12 @@ async def api_list_ai_models(provider: str = Query(...), current_user=Depends(re
else:
models = [m.get("id", "") for m in data.get("data", []) if m.get("id")]
# Cache the capabilities the provider declares for these models
# (BUG-044) — get_capabilities_for_models() below then returns the
# provider's own truth for the flags it declares, the curated table
# for the rest. Providers that declare nothing are left untouched.
remember_declared_capabilities(provider, data)
if models:
# Prepend the configured default if not already present
default = PROVIDERS.get(provider, {}).get("model")
+55 -16
View File
@@ -1,21 +1,34 @@
"""Curated model-capability metadata for the AI assistant.
"""Model-capability metadata for the AI assistant.
ObsiGate does not query every provider for the modalities a model supports
(not all of them expose that information, and the network call is not always
reliable). Instead a static, curated table maps known model-name patterns to
capability flags, with per-provider defaults. The UI uses this to show, when a
model is selected, which features it supports (Chat, Embeddings, Rerank,
Images, Video, Audio Speech, Audio Transcriptions, Vision).
Two layers, in order of trust:
The table is intentionally conservative: an unknown model falls back to the
provider default (usually ``chat`` only), so we never claim a capability the
model may not have.
1. **Provider-declared** (:mod:`backend.provider_capabilities`) — when the
provider publishes per-model capabilities in its models endpoint (Mistral
``capabilities``, OpenRouter ``architecture``), that declaration *wins* for
every flag it mentions. The snapshot is cached by the model-list endpoint.
2. **Curated table** (this module) — a static, hand-maintained map of known
model-name patterns to capability flags with per-provider defaults. It fills
the flags the provider stays silent about, and is the only source for
providers and models that declare nothing (offline, no API key, DeepSeek,
NVIDIA, QwenCloud, Xiaomi…).
The UI uses the result to show, when a model is selected, which features it
supports (Chat, Embeddings, Rerank, Images, Video, Audio Speech, Audio
Transcriptions, Vision).
The curated layer is intentionally conservative: an unknown model falls back to
the provider default (usually ``chat`` only), so we never claim a capability the
model may not have. A provider declaration, on the other hand, is authoritative
in both directions — it can also *revoke* a flag the curated table guessed
wrongly (e.g. ``mistral-embed`` declares no chat).
"""
from __future__ import annotations
from typing import Any
from backend.provider_capabilities import get_declared_capabilities
# Ordered list of capability keys exposed to the UI. Keep in sync with the
# frontend ``AI_CAPABILITY_KEYS`` and the i18n ``ai.cap_*`` labels.
CAPABILITY_KEYS: tuple[str, ...] = (
@@ -46,7 +59,10 @@ _PROVIDER_DEFAULTS: dict[str, dict[str, bool]] = {
"nvidia": _caps(chat=True),
"qwencloud": _caps(chat=True),
"xiaomi": _caps(chat=True),
"mistral": _caps(chat=True, embeddings=True),
# Mistral: chat only. ``embeddings`` used to be assumed for every Mistral
# model, which wrongly labelled mistral-large / codestral as embedders
# (BUG-044); the ``embed`` rule below covers the real embedding models.
"mistral": _caps(chat=True),
}
# Ordered (substrings, capabilities) rules — the first matching rule wins.
@@ -76,6 +92,8 @@ _MODEL_RULES: list[tuple[tuple[str, ...], dict[str, bool]]] = [
),
# Video generation.
(("veo-", "sora", "video-gen", "-video"), _caps(video=True)),
# Document OCR models — image input, but not a chat endpoint.
(("mistral-ocr",), _caps(vision=True)),
# Vision-capable chat models (multimodal input).
(
(
@@ -97,15 +115,36 @@ _MODEL_RULES: list[tuple[tuple[str, ...], dict[str, bool]]] = [
"minicpm-v",
"llama-3.2-vision",
"mimo-vl",
# Mistral vision families (BUG-044): text-only mistral-large,
# codestral and voxtral are deliberately absent.
"ministral",
"magistral",
"mistral-small",
"mistral-medium",
"mistral-vibe-cli",
"labs-leanstral",
),
_caps(chat=True, vision=True),
),
]
def _curated_capabilities(provider: str, model: str) -> dict[str, bool]:
"""Curated table lookup (model rules first, then the provider default)."""
if model:
for needles, caps in _MODEL_RULES:
if any(needle in model for needle in needles):
return dict(caps)
return dict(_PROVIDER_DEFAULTS.get(provider, _caps(chat=True)))
def get_model_capabilities(provider: str, model: str) -> dict[str, bool]:
"""Return the capability flags for a ``provider``/``model`` pair.
A provider declaration (see :mod:`backend.provider_capabilities`) overrides
the curated table for every flag it mentions; the curated table supplies the
rest.
Args:
provider: Provider identifier (e.g. ``"deepseek"``). Case-insensitive.
model: Model identifier (e.g. ``"deepseek-chat"``). May be empty, in
@@ -116,11 +155,11 @@ def get_model_capabilities(provider: str, model: str) -> dict[str, bool]:
"""
provider = (provider or "").strip().lower()
name = (model or "").strip().lower()
if name:
for needles, caps in _MODEL_RULES:
if any(needle in name for needle in needles):
return dict(caps)
return dict(_PROVIDER_DEFAULTS.get(provider, _caps(chat=True)))
caps = _curated_capabilities(provider, name)
declared = get_declared_capabilities(provider, name)
if declared:
caps.update({key: value for key, value in declared.items() if key in CAPABILITY_KEYS})
return caps
def get_capabilities_for_models(
+174
View File
@@ -0,0 +1,174 @@
"""Provider-declared model capabilities (live) with a short-lived cache.
The curated table in :mod:`backend.model_capabilities` has to be edited by hand
every time a provider ships or renames a model, and it ages badly: Mistral alone
declares ``vision`` on 28 of its models while the curated table knew none of
them (BUG-044). Providers that *do* publish per-model capabilities in their
public models endpoint are therefore asked first; the curated table is only
used to fill the flags the provider stays silent about (e.g. Mistral never
declares ``embedding``, only the absence of ``completion_chat``).
Supported declarations, detected by *payload shape* so a provider that starts
exposing them is picked up without a code change:
* ``capabilities`` dict (Mistral): ``completion_chat`` → ``chat``, ``vision``,
``audio_transcription`` (+ ``audio_transcription_realtime``),
``audio_speech``.
* ``architecture`` dict (OpenRouter): ``input_modalities`` /
``output_modalities`` → ``vision`` (image input), ``images`` (image output),
``audio_transcription`` (audio input), ``audio_speech`` (audio output),
``video`` (video output), ``chat`` (text output).
The cache is in-process and shared by every request. Its TTL only bounds how
long a *stale* declaration can survive: a fresh provider call (the model list
endpoint) overwrites the provider entry immediately.
"""
from __future__ import annotations
import logging
import os
import time
from typing import Any
logger = logging.getLogger("obsigate.ai.capabilities")
#: How long a provider-declared snapshot stays usable, in seconds.
#: ``0`` disables expiry (the snapshot lives until the next provider call).
TTL_SECONDS = float(os.getenv("AI_CAPABILITIES_TTL_SECONDS", "1800") or 0)
#: Guard against a provider returning a runaway model list.
MAX_MODELS_PER_PROVIDER = 2000
# provider → (timestamp, model count, {normalized model id: declared flags})
_CACHE: dict[str, tuple[float, int, dict[str, dict[str, bool]]]] = {}
def _norm(value: str) -> str:
"""Lower-case and strip the ``models/`` prefix Gemini uses."""
return (value or "").strip().lower().removeprefix("models/")
def _from_capability_flags(flags: dict[str, Any]) -> dict[str, bool]:
"""Map a Mistral-style ``capabilities`` dict onto ObsiGate flags."""
declared: dict[str, bool] = {}
if isinstance(flags.get("completion_chat"), bool):
declared["chat"] = flags["completion_chat"]
if isinstance(flags.get("vision"), bool):
declared["vision"] = flags["vision"]
transcription = flags.get("audio_transcription")
realtime = flags.get("audio_transcription_realtime")
if isinstance(transcription, bool) or isinstance(realtime, bool):
declared["audio_transcription"] = bool(transcription or realtime)
if isinstance(flags.get("audio_speech"), bool):
declared["audio_speech"] = flags["audio_speech"]
return declared
def _from_architecture(architecture: dict[str, Any]) -> dict[str, bool]:
"""Map an OpenRouter-style ``architecture`` dict onto ObsiGate flags."""
inputs = architecture.get("input_modalities")
outputs = architecture.get("output_modalities")
if not isinstance(inputs, list) and not isinstance(outputs, list):
return {}
in_modalities = [str(m).lower() for m in inputs] if isinstance(inputs, list) else []
out_modalities = [str(m).lower() for m in outputs] if isinstance(outputs, list) else []
return {
"chat": "text" in out_modalities,
"vision": "image" in in_modalities,
"images": "image" in out_modalities,
"audio_transcription": "audio" in in_modalities,
"audio_speech": "audio" in out_modalities,
"video": "video" in out_modalities,
}
def parse_declared_capabilities(entry: Any) -> dict[str, bool] | None:
"""Extract the capability flags a single provider model entry declares.
Args:
entry: One item of a provider models payload (``/v1/models``).
Returns:
A partial ``{flag: bool}`` mapping (only the flags the provider
actually declares), or ``None`` when the entry declares nothing.
"""
if not isinstance(entry, dict):
return None
flags = entry.get("capabilities")
declared = _from_capability_flags(flags) if isinstance(flags, dict) else {}
if not declared:
architecture = entry.get("architecture")
declared = _from_architecture(architecture) if isinstance(architecture, dict) else {}
return declared or None
def remember_declared_capabilities(provider: str, payload: Any) -> int:
"""Cache the capabilities declared by a provider models payload.
Args:
provider: Provider identifier (e.g. ``"mistral"``).
payload: Raw JSON body of the provider models endpoint, or the model
list itself.
Returns:
The number of models with declared capabilities that were cached.
"""
provider = (provider or "").strip().lower()
entries: Any = payload.get("data") if isinstance(payload, dict) else payload
if not isinstance(entries, list):
return 0
parsed: dict[str, dict[str, bool]] = {}
for entry in entries[:MAX_MODELS_PER_PROVIDER]:
if not isinstance(entry, dict):
continue
model_id = entry.get("id") or entry.get("name") or ""
if not isinstance(model_id, str) or not model_id.strip():
continue
declared = parse_declared_capabilities(entry)
if declared:
parsed[_norm(model_id)] = declared
if not parsed:
return 0
_CACHE[provider] = (time.time(), len(parsed), parsed)
logger.info(f"Capabilities declared by {provider}: {len(parsed)} models cached")
return len(parsed)
def get_declared_capabilities(provider: str, model: str) -> dict[str, bool] | None:
"""Return the cached declared capabilities for one provider/model pair.
Returns ``None`` when nothing was declared for that pair (cache cold,
expired, or the provider is silent about this model).
"""
snapshot = _CACHE.get((provider or "").strip().lower())
if not snapshot:
return None
timestamp, _count, table = snapshot
if TTL_SECONDS and (time.time() - timestamp) > TTL_SECONDS:
return None
declared = table.get(_norm(model))
return dict(declared) if declared else None
def clear_declared_capabilities(provider: str | None = None) -> None:
"""Drop the cached snapshot for one provider, or all of them (tests/ops)."""
if provider is None:
_CACHE.clear()
return
_CACHE.pop(provider.strip().lower(), None)
def cache_info() -> dict[str, dict[str, Any]]:
"""Diagnostics: per-provider cache age and model count."""
now = time.time()
return {
provider: {
"models": count,
"age_seconds": round(now - timestamp, 1),
"expired": bool(TTL_SECONDS and (now - timestamp) > TTL_SECONDS),
}
for provider, (timestamp, count, _table) in _CACHE.items()
}