Compare commits

...
94 Commits
Author SHA1 Message Date
bruno 0d4f43a8bf test(ai): collecter toutes les requetes pour eviter la course avec l'hydratation d'historique
CI / lint (push) Successful in 1m31s
CI / security (push) Successful in 1m0s
CI / test (push) Successful in 2m42s
CI / build (push) Successful in 55s
CI / e2e (push) Successful in 11m10s
2026-09-16 20:22:15 -04:00
bruno 4231f2e929 feat(sidebar+assistant): filtre Recents/Sauvegardes (#99), pastille Deep Research (#100) et icone du bouton + (BUG-049)
CI / lint (push) Failing after 1m20s
CI / test (push) Skipped
CI / build (push) Skipped
CI / e2e (push) Skipped
CI / security (push) Successful in 59s
2026-09-16 20:16:41 -04:00
bruno 8f26a418a9 test(ai): rendre le test du menu mention non flaky (nettoyage d'etat + attente du fetch)
CI / lint (push) Successful in 1m29s
CI / security (push) Successful in 1m3s
CI / test (push) Successful in 2m55s
CI / build (push) Successful in 55s
CI / e2e (push) Successful in 10m44s
2026-09-16 19:30:15 -04:00
bruno f50a9f5bf3 docs(roadmap): clore la livraison #98 + BUG-048 avec la confirmation CI (v2.5.0, run #1511)
CI / lint (push) Failing after 1m19s
CI / test (push) Skipped
CI / build (push) Skipped
CI / e2e (push) Skipped
CI / security (push) Successful in 58s
2026-09-16 16:29:36 -04:00
bruno 8e2b6203a1 feat: filtre de recherche dans la sidebar Historique IA (#98) + menus '@' et '/' depuis le menu '+' (BUG-048)
CI / lint (push) Successful in 1m28s
CI / security (push) Successful in 58s
CI / test (push) Successful in 2m19s
CI / build (push) Successful in 54s
CI / e2e (push) Successful in 10m44s
2026-09-16 16:11:54 -04:00
bruno c9e3240ae2 feat(assistant): #94-#97 historique permanent, sidebar IA, panneau + et bouton rond
CI / lint (push) Successful in 1m28s
CI / security (push) Successful in 58s
CI / test (push) Successful in 2m9s
CI / build (push) Successful in 1m3s
CI / e2e (push) Successful in 11m12s
2026-09-16 14:56:45 -04:00
bruno 0841778b99 docs(agents): versionnage automatique documente dans AGENTS.md + menage des fichiers temporaires
CI / lint (push) Successful in 1m26s
CI / security (push) Successful in 57s
CI / test (push) Successful in 2m16s
CI / build (push) Successful in 53s
CI / e2e (push) Successful in 10m40s
AGENTS.md : la checklist de fin de tache rappelle que VERSION est la source unique de verite, incrementee a chaque commit par le hook, avec resynchronisation des derives et publication du tag au push ; ligne ajoutee a la cartographie documentaire. Suppression des captures de diagnostic (diag-*.png, ogdiag.png, state.png) et du script jetable tmp-verify-inline.cjs.
2026-09-16 12:12:06 -04:00
bruno 788d84a2bf fix(editeur): #93 garde-fous de session d'edition inline (onglets, ecriture externe) + hooks de versionnage documentes
CI / lint (push) Successful in 1m27s
CI / security (push) Successful in 57s
CI / test (push) Successful in 2m52s
CI / build (push) Successful in 53s
CI / e2e (push) Successful in 10m45s
Une session d'edition inline (Editer/Forge) qui possede la zone de contenu etait detruite par deux chemins qui vident cette zone : activation d'onglet (TabManager.activate, PaneTabManager.activate) -> detachInlineEditor() avant le placeholder ; evenement SSE index_updated -> reloadExternalWrite() recharge le tampon de l'editeur au lieu de re-rendre la vue lecture. Nouveau helper queryEditor() ; closeEditor() remet le conteneur dans la modale avant de restaurer en-tete/pied/marque. docs/CONTRIBUTING.md documente scripts/install-hooks.sh.
2026-09-16 11:43:41 -04:00
bruno 168261964e docs(version): preciser le SHA reecrit par l'amend du hook post-commit
CI / lint (push) Successful in 1m27s
CI / security (push) Successful in 57s
CI / test (push) Successful in 3m27s
CI / build (push) Successful in 53s
CI / e2e (push) Successful in 10m59s
2026-09-16 11:20:29 -04:00
bruno 2f129c3f23 chore(scripts): outil de controle du CI Gitea (runs et jobs par head_sha)
scripts/check_ci.py : interroge l'API Gitea (jeton via git credential fill, User-Agent navigateur) et rapporte l'etat des jobs du run correspondant au HEAD local. Bump automatique de version = demonstration du hook prepare-commit-msg.
2026-09-16 11:20:13 -04:00
bruno d98c0c2f33 fix(version): BUG-047 la version suit chaque livraison (source unique VERSION + bump SemVer automatique)
CI / lint (push) Successful in 1m27s
CI / security (push) Successful in 57s
CI / test (push) Successful in 3m11s
CI / build (push) Successful in 54s
CI / e2e (push) Successful in 10m59s
VERSION (racine) devient la source unique de verite MAJEUR.MINEUR.CORRECTIF, incrementee a chaque commit par le hook prepare-commit-msg (BREAKING -> majeur, feat -> mineur, sinon correctif) ; post-commit rattache les fichiers derives au commit et cree le tag vX.Y.Z, publie au push (push.followTags). Tous les derives sont resynchronises : package.json, desktop Tauri, README.md/README.fr.md, docs/ROADMAP.md, CHANGELOG.md. Backend, image Docker et desktop lisent le meme fichier (plus de numero code en dur). Outils : scripts/bump_version.py, scripts/install-hooks.sh. Tests : tests/test_version.py (46, garde-fou de coherence globale).
2026-09-16 11:02:47 -04:00
bruno 3261917d62 fix(ai+editeur): BUG-046 erreur lisible et vault réel pour l'ajout IA au document; BUG-045 fin de la double scrollbar
CI / lint (push) Successful in 1m26s
CI / security (push) Successful in 57s
CI / test (push) Successful in 2m51s
CI / build (push) Successful in 54s
CI / e2e (push) Successful in 11m2s
BUG-046: 'Ajouter du texte au document courant' échouait avec 'Échec de
l'action: [object Object]'. Trois causes corrigées:
- continuation de confirmation sans payload -> second Appliquer POSTait
  sans 'message' (422); le payload d'origine est désormais transmis
- detail 422 FastAPI (tableau d'objets) stringifié tel quel -> nouvelle
  _responseError() qui aplatit en message lisible
- prompt documents/dossier muet sur le vault -> le modèle inventait
  'vault: test' et append_to_file échouait en silence; build_system_prompt
  nomme le vault et enjoint les outils d'écriture de l'utiliser exactement

BUG-045: double barre de défilement dans l'éditeur Editer (fichiers
longs): #editor-body overflow:auto + cm-editor height:100% sous la barre
IA -> corps et scroller CodeMirror défilant tous les deux. Flex column,
seul le scroller défile; override global .cm-scroller retiré. SW v20.

Vérifié: pytest 1038, ruff backend 0, tests frontend AI 84/84,
editor-inline 25/25, validate-imports 38 modules; Playwright sur
l'instance de test (un seul conteneur scrollable) et flux agent complet
append_to_file ok=true sur fichier réel
2026-09-16 09:10:58 -04:00
bruno e3df4bbf00 feat(editeur): #93 edition inline — Editer/Forge remplacent la vue lecture
CI / lint (push) Successful in 1m26s
CI / security (push) Successful in 1m0s
CI / test (push) Successful in 3m17s
CI / build (push) Successful in 56s
CI / e2e (push) Successful in 10m47s
Le conteneur d'edition (#editor-container, CodeMirror ou iframe Forge) est deplace dans la zone de contenu du document (#content-area ou pane active) au lieu de l'overlay plein ecran : le mode edition remplace la vue lecture. L'overlay reste monte (transparent, pointer-events: none) car le ruban d'edition mobile y est ancre ; il ne sert plus que de repli quand la cible n'est pas le document affiche. Garde-fou dans renderFile() pour liberer proprement la session (destroy CodeMirror/Yjs, retrait de l'iframe Forge) ; forge-close passe par closeEditor() ; cache d'onglet invalide au retour en lecture.

Assistant IA : app_context.editing annonce le document en cours d'edition (frontend bookslm.js + backend bookslm.py), et chaque ecriture d'outil (edit_file, append_to_file, create_file, restore_backup) recharge le document affiche (obsigate:file-written) — tampon CodeMirror remplace avec auto-save neutralisee, parent-reload pour l'iframe Forge, re-rendu de la vue lecture sinon.

Tests : tests/frontend/editor-inline.test.mjs (19), tests/test_bookslm.py::TestGeneralPrompt (3 nouveaux). Fiche : docs/features/editeur-inline.md.
2026-09-15 23:40:10 -04:00
bruno f630771150 feat(assistant): #91 etapes enrichies (sous-sections Reflechon/Sources, chevrons, indicateur anime, suppression barre de chargement)
CI / lint (push) Successful in 1m23s
CI / security (push) Successful in 58s
CI / test (push) Successful in 2m43s
CI / build (push) Successful in 55s
CI / e2e (push) Successful in 11m8s
2026-09-15 22:53:15 -04:00
bruno 431fb5626d docs(ai): inventaire reel des 28 outils du registre (risques, libelles UI, outils web) + sections prevues
CI / lint (push) Successful in 1m28s
CI / security (push) Successful in 1m3s
CI / test (push) Successful in 3m16s
CI / build (push) Successful in 50s
CI / e2e (push) Successful in 10m53s
2026-09-15 20:59:06 -04:00
bruno 19e12fa22e fix(assistant): #91 le SSE renvoie le modele reellement utilise (tag fournisseur - modele complet)
CI / lint (push) Successful in 1m24s
CI / security (push) Successful in 58s
CI / test (push) Successful in 2m21s
CI / build (push) Successful in 52s
CI / e2e (push) Successful in 10m38s
2026-09-15 20:46:58 -04:00
bruno daf9ff3b2d fix(assistant): #91 question positionnee au bord haut de la fenetre (padding dynamique, overflow-anchor, passe de correction)
CI / lint (push) Successful in 1m27s
CI / security (push) Successful in 1m0s
CI / test (push) Successful in 3m23s
CI / build (push) Successful in 51s
CI / e2e (push) Successful in 10m56s
2026-09-15 20:29:01 -04:00
bruno 1ed0d52619 fix(assistant): #91 bulle utilisateur elargie (81% du fil) + ancrage de la question des l'envoi ; docs: feuille de route outils #92
CI / lint (push) Successful in 1m23s
CI / security (push) Successful in 58s
CI / test (push) Successful in 2m37s
CI / build (push) Successful in 51s
CI / e2e (push) Successful in 10m47s
2026-09-15 19:34:05 -04:00
bruno 0e9fcb78e0 chore(pwa): bump SW_VERSION v10 pour le cache bust #91 (labels + outils web)
CI / lint (push) Successful in 1m26s
CI / security (push) Successful in 56s
CI / test (push) Successful in 3m1s
CI / build (push) Successful in 51s
CI / e2e (push) Successful in 11m6s
2026-09-15 17:47:12 -04:00
bruno 19d5b931c4 feat(assistant): #91 steps humains en direct + outils web_search/fetch_url (labels backend, SSE step, garde SSRF)
CI / lint (push) Successful in 1m22s
CI / security (push) Successful in 55s
CI / test (push) Successful in 3m0s
CI / build (push) Successful in 51s
CI / e2e (push) Successful in 10m41s
2026-09-15 17:46:51 -04:00
bruno 18b7dee2f6 fix(assistant): #91 maintien de l'ancre en haut pendant le streaming, libération au scroll manuel
CI / lint (push) Successful in 1m22s
CI / security (push) Successful in 1m0s
CI / test (push) Successful in 3m12s
CI / build (push) Successful in 50s
CI / e2e (push) Successful in 10m34s
2026-09-15 16:24:57 -04:00
bruno 731b3e4b47 chore(i18n): normalisation des fins de ligne locales JSON (CRLF -> LF, cohérence dépôt)
CI / lint (push) Successful in 1m22s
CI / security (push) Successful in 58s
CI / test (push) Successful in 2m3s
CI / build (push) Successful in 1m5s
CI / e2e (push) Successful in 10m48s
2026-09-15 15:53:39 -04:00
bruno cdb4d29676 feat(assistant): #91 refonte Notion de la zone de discussion — question ancrée en haut, fil chronologique, étapes repliables, barre Copier/Ajouter
CI / lint (push) Successful in 1m23s
CI / security (push) Successful in 55s
CI / test (push) Successful in 2m55s
CI / build (push) Successful in 50s
CI / e2e (push) Successful in 10m58s
2026-09-15 15:51:19 -04:00
bruno 256f5a4a03 feat(assistant): #91 fenêtre de résultats — post épinglé en haut, fournisseur/modèle discret & bouton copier
CI / lint (push) Successful in 1m20s
CI / security (push) Successful in 57s
CI / test (push) Successful in 2m41s
CI / build (push) Successful in 55s
CI / e2e (push) Successful in 10m40s
2026-09-15 12:04:38 -04:00
bruno b69cb9b0f8 fix(ai): BUG-044 capacités des modèles lues chez le fournisseur (Mistral vision)
CI / lint (push) Successful in 1m20s
CI / security (push) Successful in 53s
CI / test (push) Successful in 2m27s
CI / build (push) Successful in 50s
CI / e2e (push) Successful in 10m48s
Le panneau de modèle par défaut et la bulle ⓘ n'affichaient aucun modèle Mistral
« Vision capable » alors que GET api.mistral.ai/v1/models en déclare 28 : la table
de capacités était entièrement statique et aucun de ses motifs ne correspondait aux
familles Mistral actuelles (seul `pixtral`, retiré de l'API, les matchait).

- backend/provider_capabilities.py (nouveau) : capacités déclarées par le
  fournisseur (Mistral `capabilities`, OpenRouter `architecture`), détectées par
  la forme du payload, snapshot en cache process-wide (TTL 30 min, surchargeable
  par AI_CAPABILITIES_TTL_SECONDS) rempli par GET /api/config/ai-models.
- backend/model_capabilities.py : une déclaration prime sur la table statique
  pour chaque drapeau mentionné ; la table ne comble que le reste (Mistral ne
  déclare jamais `embedding`). Table corrigée pour le repli hors ligne : familles
  vision Mistral (ministral, magistral, mistral-small, mistral-medium,
  mistral-vibe-cli, labs-leanstral), mistral-ocr = vision sans chat, et défaut du
  fournisseur Mistral sans `embeddings` (mistral-large / codestral n'étaient plus
  des « embedders »).
- backend/ai_routes.py : GET /api/ai/model-capabilities reste sans appel réseau
  (cache froid → table statique).
- Tests : tests/test_provider_capabilities.py (nouveau), TestMistralFamilies et
  TestDeclaredCapabilities (bout en bout via l'API).
- Docs : CHANGELOG [Unreleased], registre + journal ISSUES_TODOLIST,
  fiche docs/features/ai-provider-picker.md (§L).

Vérifié : 28/28 modèles vision déclarés par Mistral détectés (0 avant), 0 écart
dans les deux sens ; pytest 1007 passed / 6 skipped ; ruff 0 (backend) ; mypy 0 ;
tests frontend unit 9/9 + IA 66/66 + validate-imports 37 modules ; instance de test
reconstruite et vérifiée sur http://localhost:2020.
2026-09-15 09:26:31 -04:00
bruno 140a8f6efe feat(viewer): #90 toolbar groupes aussi pour les fichiers texte non markdown
CI / lint (push) Successful in 1m18s
CI / security (push) Successful in 53s
CI / test (push) Successful in 2m30s
CI / build (push) Successful in 50s
CI / e2e (push) Successful in 10m27s
[pop-out][Bookmark] | [Editer][Source][.ext][Pretty] | [Copier] | [Partager]
- TOC ancre uniquement au markdown (plus de bouton TOC sur .sh/.py/...)
- PDF/Export deja ancres au markdown: groupe export = [Copier] seul sur texte
- tests toolbar-order mis a jour (conditions is_markdown sur nav/export)
2026-09-15 08:24:22 -04:00
bruno 14b99826eb feat(viewer): #90 regroupement fonctionnel des boutons de la barre d'actions + spacers
CI / lint (push) Successful in 1m18s
CI / security (push) Successful in 55s
CI / test (push) Successful in 2m10s
CI / build (push) Successful in 48s
CI / e2e (push) Successful in 11m0s
Ordre: [TOC][pop-out][Bookmark] | [Editer][Source][.md][Forge] | [Copier][PDF][Export] | [Partager]
- viewer.js: construction par groupes nav/edit/export/share, spacer .action-sep
  entre groupes non vides; Pretty ancre au groupe edition
- popout.html: meme assemblage groupe, ordre aligne
- style.css: regle .file-actions .action-sep (1px, var(--border-md))
- tests: tests/frontend/toolbar-order.test.mjs (9) branche au CI
2026-09-14 22:47:46 -04:00
bruno 0201d963b7 fix(ui): panneau TOc 'Sur cette page' toujours ferme au demarrage, ouvre uniquement via bouton TOC
CI / lint (push) Successful in 1m19s
CI / security (push) Successful in 55s
CI / test (push) Successful in 2m24s
CI / build (push) Successful in 48s
CI / e2e (push) Successful in 10m57s
2026-09-14 22:06:33 -04:00
bruno 9be0fe6626 fix(viewer): gouttiere de numeros de ligne synchronisee au scroll en mode pretty print
CI / lint (push) Successful in 1m18s
CI / security (push) Successful in 56s
CI / test (push) Successful in 2m40s
CI / build (push) Successful in 48s
CI / e2e (push) Successful in 11m20s
2026-09-14 20:46:35 -04:00
bruno 24b5dd033a feat(viewer): couverture pretty print tous langages code (aliases hljs, langues hors bundle common, Dockerfile/Makefile)
CI / lint (push) Successful in 1m18s
CI / security (push) Successful in 56s
CI / test (push) Successful in 2m31s
CI / build (push) Successful in 48s
CI / e2e (push) Successful in 10m41s
2026-09-14 20:33:54 -04:00
bruno 686a6da019 fix(viewer): pretty print utilisait un cache raw partagé entre fichiers (contenu/colouration du fichier precedent)
CI / lint (push) Successful in 1m18s
CI / security (push) Successful in 55s
CI / test (push) Successful in 1m58s
CI / build (push) Successful in 48s
CI / e2e (push) Successful in 10m29s
2026-09-14 20:10:11 -04:00
bruno bb0c8e7731 fix(viewer): pretty print fallback tokenizer quand highlight.js ne produit aucun token (txt/log/plaintext) (#89)
CI / lint (push) Successful in 1m18s
CI / security (push) Successful in 56s
CI / test (push) Successful in 2m57s
CI / build (push) Successful in 48s
CI / e2e (push) Successful in 10m47s
2026-09-14 19:43:50 -04:00
bruno d3f7a04411 feat(viewer): mode pretty print avec coloration syntaxique et numeros de ligne pour fichiers texte (#89)
CI / lint (push) Successful in 1m17s
CI / security (push) Successful in 56s
CI / test (push) Successful in 2m25s
CI / build (push) Successful in 48s
CI / e2e (push) Successful in 10m35s
2026-09-14 19:12:43 -04:00
bruno 96bb1cbcae fix(viewer): rendu unifié type markdown pour fichiers texte/JSON/CSV + autofocus arborescence apres import (#89)
CI / lint (push) Successful in 1m18s
CI / security (push) Successful in 51s
CI / test (push) Successful in 2m44s
CI / build (push) Successful in 48s
CI / e2e (push) Successful in 10m32s
2026-09-14 18:15:21 -04:00
bruno 93fc562c44 fix(dragdrop): arborescence récursive, recherche/filtre de dossiers et saisie de sous-dossier (#89)
CI / lint (push) Successful in 1m17s
CI / security (push) Successful in 51s
CI / test (push) Successful in 2m46s
CI / build (push) Successful in 47s
CI / e2e (push) Successful in 10m45s
2026-09-14 16:39:37 -04:00
bruno c59f6b5fca fix(dragdrop): modale de sélection de voûte et sous-répertoire alignée avec le thème ObsiGate (#89)
CI / lint (push) Successful in 1m17s
CI / security (push) Successful in 51s
CI / test (push) Successful in 2m30s
CI / build (push) Successful in 47s
CI / e2e (push) Successful in 10m57s
2026-09-14 16:29:14 -04:00
bruno ba7b49c35b fix(dragdrop): amélioration sélection répertoire cible, déplacement arborescence et ouverture auto IA (#89)
CI / lint (push) Successful in 1m17s
CI / security (push) Successful in 51s
CI / test (push) Successful in 3m11s
CI / build (push) Successful in 48s
CI / e2e (push) Successful in 11m2s
2026-09-14 15:55:41 -04:00
bruno 1db3e0ad2f feat(dragdrop): drag & drop complet de fichiers/dossiers & intégration IA (#89)
CI / lint (push) Successful in 1m20s
CI / security (push) Successful in 50s
CI / test (push) Successful in 2m42s
CI / build (push) Successful in 51s
CI / e2e (push) Successful in 10m29s
2026-09-14 15:31:12 -04:00
bruno 150b57d538 docs(ai): CI Gitea verte pour BUG-043 (lint, test, security, build, e2e)
CI / lint (push) Successful in 1m20s
CI / security (push) Successful in 51s
CI / test (push) Successful in 2m44s
CI / build (push) Successful in 46s
CI / e2e (push) Successful in 10m47s
2026-09-14 14:13:35 -04:00
bruno ce2f0b6a6c fix(ai): la liste des fournisseurs de l'assistant suit la configuration (BUG-043)
CI / lint (push) Successful in 1m17s
CI / security (push) Successful in 56s
CI / test (push) Successful in 2m58s
CI / build (push) Successful in 47s
CI / e2e (push) Successful in 10m38s
Le picker lit /api/ai/status une seule fois, à sa construction, et le panneau
de l'assistant est un singleton monté pour toute la session : ajouter ou
supprimer une clé API dans la configuration du projet laissait la liste des
fournisseurs figée jusqu'à un rechargement de page.

- ai.js : nouveau refreshAIPickers() qui reconstruit chaque picker monté dans
  son emplacement .ai-picker-slot (constante PICKER_SLOT_CLASS) ; le slot est
  conservé même sans fournisseur configuré, donc le premier fournisseur ajouté
  s'y monte aussi ; une sélection persistée dont le fournisseur n'est plus
  configuré est purgée de obsigate_ai_picker (retour au défaut, plus de modèle
  fantôme dans le déclencheur).
- bookslm.js : l'emplacement .bookslm-picker-host porte la classe
  ai-picker-slot et reste dans la barre (replaceChildren au lieu de replaceWith).
- config.js : refreshAIPickers() après saveAIKeys() et deleteAIKey().
- tests : +4 tests JSDOM (ajout/retrait dans la barre, premier montage dans un
  slot vide, purge de la sélection orpheline, câblage save/delete).

Vérifié : tests frontend 66/66 (IA) + 9 suites JSDOM, validate-imports 36
modules, pytest 963 passed / 6 skipped, ruff 0, et contrôle navigateur
(Playwright) sur l'instance de test — ajout de nvidia visible sans rechargement,
retrait effectif + sélection réinitialisée.
2026-09-14 13:59:08 -04:00
bruno c3e6841293 fix(ai): accents dans les chemins et resolution des chemins prefixes par le vault (BUG-042)
CI / lint (push) Successful in 1m16s
CI / security (push) Successful in 50s
CI / test (push) Successful in 2m24s
CI / build (push) Successful in 47s
CI / e2e (push) Successful in 10m35s
- accents : classes de caracteres Unicode (\p{L}\p{N}\p{M}) pour les motifs
  de liens et _looksLikePath (PATH_NAME_RE) -> les chemins/fichiers accentues
  produisent enfin un lien cliquable ; comparaison de chemins normalisee NFC
  (_normKey), donc une mention decomposee (e + accent combinant, style macOS)
  correspond a une entree d'index precomposee, et inversement.
- prefixe de vault : _splitVaultPrefix() retire un premier segment egal au nom
  d'un vault connu (TestVault/Recettes/Pizza Maison.md) ; _fetchPathsForVault()
  interroge l'index de CE vault sans ecraser le cache du vault actif ;
  _openFileLink/_revealPath recoivent le vault cible ; repli 'retirer le
  premier segment' si le prefixe ne correspond a aucun vault connu.

Tests : frontend IA 62/62 (+5), 9 suites JSDOM, validate-imports 36 modules,
pytest 963 passed / 6 skipped. Verifie aussi contre l'instance live sur donnees
reelles accentuees/espaces (Recettes/Preparation.md, Recettes/Pat... ) : 8/8.
2026-09-14 13:37:23 -04:00
bruno 12a53d647e fix(ai): espaces dans les noms de fichiers et chemins des liens (BUG-042)
CI / lint (push) Successful in 1m17s
CI / security (push) Successful in 51s
CI / test (push) Successful in 2m30s
CI / build (push) Successful in 51s
CI / e2e (push) Successful in 10m29s
Les noms/paths de l'assistant peuvent contenir des espaces ; la detection et
la resolution les prennent desormais en charge de bout en bout :

- liens markdown : cible avec espaces, encadree par <...> ou avec un "titre",
  et URL percent-encodees (%20) decodees ;
- code inline : `Ma note.md` / `Mon dossier/Ma note.md` reconnus
  (_looksLikePath accepte les espaces, jeu de caracteres strict pour rejeter
  extraits de code et commandes shell) ;
- mentions brutes : liees uniquement si presentes dans l'index du vault
  (_linkifySpacePaths + _confirmPathInCache), avec acceptation du plus long
  suffixe aligne sur un mot -> le mot de prose precedent n'est pas avale ;
- _normalizeLinkPath : trim ; _renderMdLink : decodage %20.

Tests : frontend IA 57/57 (+5), 9 suites JSDOM, validate-imports 36 modules,
pytest 963 passed / 6 skipped.
2026-09-14 13:20:09 -04:00
bruno 314e37c433 fix(ai): contexte applicatif de l'assistant, dossier vide et liens fiables (#88, BUG-041, BUG-042)
CI / lint (push) Successful in 1m16s
CI / security (push) Successful in 50s
CI / test (push) Successful in 3m0s
CI / build (push) Successful in 47s
CI / e2e (push) Successful in 10m58s
- BUG-041 : le contexte d'un dossier vide ne renvoie plus 404 ; il degrade
  vers le prompt General enrichi d'un bloc « Dossier vide ».
- BUG-042 : liens de fichiers deterministes (nom -> presse-papiers,
  dossier -> arborescence, chemin -> viewer) et resolution du chemin contre
  l'index du vault (exact -> suffixe -> basename) avant ouverture, fin des
  'File not found'.
- #88 : app_context (documents ouverts, repertoire/vault courants, recherche
  + resultats affiches) et fichiers recemment modifies injectes dans le
  prompt General.

Tests : pytest 963 passed / 6 skipped, ruff 0, mypy 0, frontend 52/52 +
validate-imports 36 modules.
2026-09-14 11:28:30 -04:00
bruno 05ed36240b style(mobile): harmonise la couleur des boutons de la barre du bas
CI / lint (push) Successful in 1m14s
CI / security (push) Successful in 52s
CI / test (push) Successful in 2m43s
CI / build (push) Successful in 45s
CI / e2e (push) Successful in 10m26s
2026-09-13 22:11:17 -04:00
bruno 520059d866 feat(mobile): ruban d'edition style Obsidian + correctifs mobile (#83, BUG-017 a BUG-020)
CI / lint (push) Successful in 1m13s
CI / security (push) Successful in 47s
CI / test (push) Successful in 2m41s
CI / build (push) Successful in 43s
CI / e2e (push) Successful in 11m14s
- #83: ruban horizontal defilable ancre au-dessus du clavier, commandes
  etendues (titres, barre, surligne, code bloc, citation, listes, cases,
  wikilinks, tags, pieces jointes, undo/redo, colle, police) et
  personnalisation persistee via l'icone engrenage
- BUG-017: cartes du tableau de bord compressibles (min-width:0)
- BUG-018: en-tete editeur mobile compact, boutons Annuler/Sauvegarder
  accessibles, barre IA plus tactile
- BUG-019: ancrage du ruban via visualViewport (--kb-offset)
- BUG-020: suppression de la double barre de defilement

Tests: mobile-editor 35/35, ai 50/50, 9 suites JSDOM, validate-imports,
pytest 961 passed / 6 skipped, ruff + mypy 0.
2026-09-13 11:49:45 -04:00
bruno 162a5b4acc fix(security): consolidation & securite phase 1 (#84, BUG-021 a BUG-034)
CI / lint (push) Successful in 1m20s
CI / security (push) Successful in 47s
CI / test (push) Successful in 2m21s
CI / build (push) Successful in 43s
CI / e2e (push) Successful in 10m48s
- sanitizer XSS serveur (markdown + page de partage) [BUG-021/022]
- rate-limit/lockout MFA [BUG-023]
- isolation vaults par segments [BUG-024]
- caps regex ReDoS [BUG-025]
- SSRF webhooks + secrets externalises [BUG-026]
- rotation/revocation des jetons [BUG-027]
- politique de mot de passe + invalidation sessions [BUG-028]
- verrous users.json [BUG-029]
- IP reelle dans les audits [BUG-030]
- rate-limit par compte [BUG-031]
- symlinks hors vault ignores [BUG-032]
- recherche simple via inverted index [BUG-033]
- token en memoire + cookie HttpOnly, CSP durcie [BUG-034]

Tests: pytest 961 passed / 6 skipped, ruff 0, mypy 0, frontend vert.
2026-09-13 10:51:42 -04:00
bruno d47fcbf2be fix(mobile): bouton mode lecture visible uniquement fichier ouvert (BUG-016)
CI / lint (push) Successful in 1m10s
CI / security (push) Successful in 46s
CI / test (push) Successful in 2m23s
CI / build (push) Successful in 41s
CI / e2e (push) Successful in 10m28s
2026-09-12 22:44:15 -04:00
bruno cb07b92543 style(ai): selecteurs Fournisseur/Modele alignes a droite et reordonnes (#82)
CI / lint (push) Successful in 1m14s
CI / security (push) Successful in 53s
CI / test (push) Successful in 2m16s
CI / build (push) Successful in 41s
CI / e2e (push) Successful in 10m59s
2026-09-12 21:42:01 -04:00
bruno 9274dbd49f feat(ai): menu @ instantane, selecteurs agrandis et BooksLM racine (#82)
CI / lint (push) Successful in 1m11s
CI / security (push) Successful in 47s
CI / test (push) Successful in 2m12s
CI / build (push) Successful in 45s
CI / e2e (push) Successful in 10m21s
Nouvel endpoint GET /api/vault/{vault}/paths + prechargement et filtrage client du menu @ (affichage instantane des fichiers/repertoires). Selecteurs Fournisseur/Modele agrandis (0,8rem / 34px) pour s'aligner sur le reste du site. Ajout de BooksLM au menu contextuel de la racine des vaults.
2026-09-12 20:09:08 -04:00
bruno d6cdd670b1 fix(ai): selection visible des menus et retrait de l'indice clavier (BUG-015)
CI / lint (push) Successful in 1m14s
CI / security (push) Successful in 44s
CI / test (push) Successful in 3m0s
CI / build (push) Successful in 1m3s
CI / e2e (push) Successful in 10m38s
BUG-015: l'etat actif des menus / et @ (et de la liste de modeles) utilise --bg-hover + barre d'accent a gauche, car --surface2 est identique a --bg-primary en theme sombre. Retrait de l'indice 'Entree pour envoyer...' sous la zone de saisie.
2026-09-12 19:00:05 -04:00
bruno a5201a62e0 fix(ai): liens sans vault, navigation clavier des menus et purge caches (BUG-013, BUG-014)
CI / lint (push) Successful in 1m9s
CI / security (push) Successful in 45s
CI / test (push) Successful in 2m27s
CI / build (push) Successful in 40s
CI / e2e (push) Successful in 10m47s
BUG-013: _openFileLink/_revealPath utilisent _activeVault() (contexte -> selection -> premier vault). BUG-014: navigation up/down geree au niveau du panneau en phase de capture, independante du focus. Caches: SW_VERSION v6 + migration qui purge tous les caches obsigate-*.
2026-09-12 16:44:12 -04:00
bruno ee4c273d73 feat(ai): sidebar epuree, suivi visuel des requetes et correctif menu @ (BUG-012, #82)
CI / lint (push) Successful in 1m9s
CI / security (push) Successful in 44s
CI / test (push) Successful in 2m29s
CI / build (push) Successful in 39s
CI / e2e (push) Successful in 10m39s
BUG-012: le menu @ est toujours rendu et _mentionVault() retombe sur le vault de contexte puis le premier vault disponible. #82: suppression des intitules Fournisseur & modele / Fournisseur:, description du contexte General en info-bulle, placeholder retire, bouton Envoyer en emoji, indicateur d'activite (envoi/reception/outils/confirmation/succes/echec), SW_VERSION v5.
2026-09-12 14:21:33 -04:00
bruno eea2108ac2 fix(ai): lisibilite des modeles + contexte @ en mode General (BUG-010, BUG-011)
CI / lint (push) Successful in 1m8s
CI / security (push) Successful in 44s
CI / test (push) Successful in 2m5s
CI / build (push) Successful in 39s
CI / e2e (push) Successful in 10m37s
BUG-011: popover des modeles aligne a droite (right:0), largeur 340px bornee, noms sur plusieurs lignes + title. BUG-010: la selection @ capture le vault renvoye par tree-search et _contextVault() le propage aux requetes /context et /chat, corrigeant le contexte ad-hoc en mode General.
2026-09-12 13:50:14 -04:00
bruno 7f0e34d318 fix(ai): liste de modeles corrompue et cles i18n brutes (BUG-009)
CI / lint (push) Successful in 1m9s
CI / security (push) Successful in 43s
CI / test (push) Successful in 1m58s
CI / build (push) Successful in 41s
CI / e2e (push) Successful in 10m31s
Locales chargees en cache no-store + SW_VERSION v4 pour eviter les cles brutes (fr.json obsolete). Picker : styles critiques en ligne (popover/liste/options), plafond de 200 modeles avec indicateur, et non-chevauchement entre la liste de modeles et la bulle de capacites.
2026-09-12 13:15:27 -04:00
bruno d6b7c0e9cb feat(ai): section Fournisseur & modele compacte + correctifs menus @/ (#82, BUG-007, BUG-008)
CI / lint (push) Successful in 1m13s
CI / security (push) Successful in 43s
CI / test (push) Successful in 2m33s
CI / build (push) Successful in 39s
CI / e2e (push) Successful in 10m32s
Refonte de la section Fournisseur & modele de l'assistant : capacites regroupees dans une bulle d'information (survol/clic/appui long) au lieu d'une liste permanente, et liste de modeles avec recherche. Correctifs : detection Unicode + filtrage insensible aux accents des commandes / et mentions @, navigation clavier fiable, et prise en compte du contexte ad-hoc en mode General cote backend.
2026-09-12 12:40:03 -04:00
bruno f049e208b6 feat(ai): commandes @/ & skills, analyse d'images et capacites des modeles (#81)
CI / lint (push) Successful in 1m10s
CI / security (push) Successful in 43s
CI / test (push) Successful in 2m34s
CI / build (push) Successful in 43s
CI / e2e (push) Successful in 10m59s
2026-09-12 11:38:46 -04:00
bruno 88bb817a62 chore(release): sync version desktop + workflow desktop en manuel (#77)
CI / lint (push) Successful in 1m18s
CI / security (push) Successful in 43s
CI / test (push) Successful in 3m8s
CI / build (push) Successful in 37s
CI / e2e (push) Successful in 10m25s
- bump_version.sh : met a jour tauri.conf.json, Cargo.toml et Cargo.lock (obsigate-desktop), commit de release chore(release): vX.Y.Z puis tag. --push pousse branche+tag, --no-files = tag seul, --dry-run previsualise.

- desktop-build.yml : declenchement workflow_dispatch uniquement (aucun runner self-hosted).

- Docs : DEVELOPMENT_AND_RELEASES (bump automatise) + CHANGELOG.
2026-09-12 10:25:29 -04:00
bruno 8ddee212e5 feat(desktop): manifeste de mise a jour latest.json + builds locaux signes (#77)
CI / lint (push) Successful in 1m7s
CI / security (push) Successful in 43s
CI / test (push) Successful in 2m12s
CI / build (push) Successful in 42s
CI / e2e (push) Successful in 10m43s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
- scripts/updater_manifest.py : construit latest.json (version, pub_date, platforms Windows/Linux avec signature .sig et URLs des assets).

- publish_release.py : genere latest.json, upload des .sig + latest.json, rappel de commit.

- tauri.conf.json : endpoint updater -> raw/branch/main/desktop/latest.json.

- build-windows.bat / build-linux.sh : detection automatique de obsigate-updater.key (signature) ou repli non signe.

- .gitignore : ignore desktop/key/ ; tests/test_updater_manifest.py (8 tests).
2026-09-12 10:01:56 -04:00
bruno d3299166bb feat(desktop): signature updater Tauri + wizard persistant + protocole E2E (#77)
CI / lint (push) Successful in 1m11s
CI / security (push) Successful in 43s
CI / test (push) Successful in 2m22s
CI / build (push) Successful in 1m41s
CI / e2e (push) Successful in 10m50s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
- Signature des mises a jour Tauri : paire de cles minisign generee, cle publique dans tauri.conf.json, createUpdaterArtifacts actif, secrets CI exposes (repli build non signe si secret absent).

- Wizard 1er lancement : etat persistant wizard_done cote Rust (get_wizard_state/complete_wizard) pour ne plus reafficher la banniere, retro-compatible.

- Protocole des 6 tests E2E manuels : docs/DESKTOP_E2E_CHECKLIST.md.

- CI : desktop.test.mjs ajoute au job lint.

- Docs : ROADMAP, fiche desktop-tauri, CHANGELOG, README desktop, guide releases.

- fix(build): retirer les libs WeasyPrint inutiles du stage builder Docker.
2026-09-12 09:29:47 -04:00
bruno 31d4be8015 feat(search): recherche semantique - embeddings vectoriels + hybride RRF (#70) 2026-09-12 00:12:20 -04:00
bruno 45be3125de feat(mobile): editeur mobile natif - interface tactile optimisee (#69)
CI / lint (push) Successful in 1m3s
CI / security (push) Successful in 46s
CI / test (push) Successful in 2m43s
CI / build (push) Successful in 38s
CI / e2e (push) Successful in 11m12s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
Barre d'outils Markdown flottante (gras/italique/code/liste/lien) sur la selection CodeMirror, bouton Coller persistant (contournement iOS), zoom par pincement + hauteur ajustable persistes, raccourcis swipe (gauche -> liens entrants, droite -> table des matieres), mode lecture plein ecran avec navigation entre fichiers du dossier. 22 tests JSDOM + spec E2E mobile. Docs: fiche feature, CHANGELOG, ROADMAP, guide i18n FR/EN, README.
2026-09-11 23:51:16 -04:00
bruno 063b02e996 feat: collaboration temps reel - edition simultanee (#62)
CI / lint (push) Successful in 1m1s
CI / security (push) Successful in 41s
CI / test (push) Successful in 1m47s
CI / build (push) Successful in 1m20s
CI / e2e (push) Successful in 10m36s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
WebSocket /ws/collab/{vault}/{path} (rooms par fichier), relais Yjs/CRDT, awareness (curseurs colores + presence), persistance serveur debounce 2s, auth WS + check_vault_access, reconnexion automatique. Frontend frontend/js/collab.js, backend backend/collab.py. Tests: 17 backend (5 clients simultanes) + 10 frontend. Docs: CHANGELOG, ROADMAP, fiche features/collaboration.md, README FR/EN.
2026-09-11 23:27:22 -04:00
bruno 3f7b7847a5 docs: cloturer #80 (assistant IA UX) - roadmap et fiche validees
CI / lint (push) Successful in 1m0s
CI / security (push) Successful in 41s
CI / test (push) Successful in 1m21s
CI / build (push) Successful in 36s
CI / e2e (push) Successful in 10m41s
2026-09-11 22:31:02 -04:00
bruno 7d87ad387b fix(security): retirer une cle API exposee dans .env.example (BUG-006)
CI / lint (push) Successful in 59s
CI / security (push) Successful in 46s
CI / test (push) Successful in 1m21s
CI / build (push) Successful in 37s
CI / e2e (push) Successful in 10m36s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
2026-09-11 21:57:17 -04:00
bruno 9dce341bc8 feat(ai): durcissement phase F (#79) - rate limit, redaction, OpenAPI/MCP, E2E 2026-09-11 21:56:50 -04:00
bruno 88eecd7671 feat(ai): serveur MCP Streamable HTTP + confirmations two-step (#79 phase E)
CI / lint (push) Successful in 1m3s
CI / security (push) Successful in 41s
CI / test (push) Successful in 1m20s
CI / build (push) Successful in 1m19s
CI / e2e (push) Successful in 10m56s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
2026-09-11 21:28:05 -04:00
bruno 83c412b33d feat(ai): catalogue d'outils mutations + confirmations two-step (#79 phase D)
CI / lint (push) Successful in 59s
CI / security (push) Successful in 45s
CI / test (push) Successful in 1m22s
CI / build (push) Successful in 37s
CI / e2e (push) Successful in 10m37s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
2026-09-11 20:52:02 -04:00
bruno 31b65ade5b feat(ai): catalogue d'outils lecture & recherche (#79 phase C)
CI / lint (push) Successful in 59s
CI / security (push) Successful in 49s
CI / test (push) Successful in 1m24s
CI / build (push) Successful in 37s
CI / e2e (push) Successful in 10m24s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
2026-09-11 20:33:15 -04:00
bruno 7cf7d33b6d fix(pwa): BUG-005 chargement mobile via Cloudflare (SW network-first + no-cache)
CI / lint (push) Successful in 58s
CI / security (push) Successful in 40s
CI / test (push) Successful in 1m19s
CI / build (push) Successful in 37s
CI / e2e (push) Successful in 10m53s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
2026-09-11 17:13:16 -04:00
bruno 4c4e415975 feat(ai): services partages A2 + SSE streaming B4 + confirmations UI B5 (#79)
CI / lint (push) Successful in 58s
CI / security (push) Successful in 40s
CI / test (push) Successful in 1m15s
CI / build (push) Successful in 37s
CI / e2e (push) Successful in 10m15s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
2026-09-11 17:06:40 -04:00
bruno c55e3e0cbc feat(ai): rendu Markdown, liens fichiers/paths et sessions (#80)
CI / lint (push) Successful in 57s
CI / security (push) Successful in 43s
CI / test (push) Successful in 1m14s
CI / build (push) Successful in 37s
CI / e2e (push) Successful in 10m36s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
Assistant IA (BooksLM) :

- renderer Markdown complet (titres, listes, tableaux, citations, code) avec echappement HTML

- mentions fichiers/paths cliquables : ouverture du fichier / revelation dans l'arborescence

- gestion multi-sessions par contexte (historique, rechargement, suppression, migration)

- i18n FR/EN, aide in-app, styles, tests frontend (24)
2026-09-11 16:14:21 -04:00
bruno 240fd8586e fix: corriger 33 erreurs mypy (CI bloquant) + lien README (BUG-003, BUG-004)
CI / lint (push) Successful in 1m1s
CI / security (push) Successful in 40s
CI / test (push) Successful in 1m17s
CI / build (push) Successful in 38s
CI / e2e (push) Successful in 10m55s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
BUG-003: annotations de types, gardes None sur get_user(), PdfReader: Any et import PROVIDERS manquant (bug latent main.py:4523). Etape mypy du CI rendue bloquante (etait advisory).

BUG-004: lien README.md -> docs/CONTRIBUTING.md corrige (+ DELIVERY_WORKFLOW.md), arbre projet mis a jour, parite README.fr.md.

Verifie: mypy 0 erreur, ruff OK, pytest 728 passed / 5 skipped, frontend OK, liens md OK.
2026-09-11 14:57:55 -04:00
bruno ce23ab38f7 docs: restructurer le suivi et unifier la methode de livraison
CI / lint (push) Successful in 57s
CI / security (push) Successful in 40s
CI / test (push) Successful in 1m13s
CI / build (push) Successful in 34s
CI / e2e (push) Successful in 10m33s
- ROADMAP: ne garde que le travail a venir + index compact du complete (995 -> ~155 lignes); detail deplace vers docs/features/ et docs/archive/

- docs/features/: fiches detaillees #74, #75, #76, #77, #78, #79

- docs/archive/COMPLETED_v1-v2.md: detail des items courts livres

- CHANGELOG: alignement sur les tags (2.0.0 date, 2.2.0/2.2.1 ajoutes, Unreleased = travail #79 post-2.2.1)

- AGENTS.md + docs/DELIVERY_WORKFLOW.md: methode de livraison unique (Definition of Done) referencee par ROADMAP, CONTRIBUTING, ISSUES_TODOLIST
2026-09-11 14:07:56 -04:00
bruno 55696bfb31 feat(ai): phase B function calling in-app (agent loop + endpoint /agent)
CI / lint (push) Successful in 57s
CI / security (push) Successful in 39s
CI / test (push) Successful in 1m13s
CI / build (push) Successful in 36s
CI / e2e (push) Successful in 10m13s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
- backend/ai_chat.py: chat_completion provider-agnostique (OpenAI-compat tools/tool_calls + Gemini functionDeclarations/functionCall), retry sans tools si rejete
- backend/agent/loop.py: run_agent multi-etapes (limite 10, truncation, confirmation two-step), LLM injectable
- endpoint opt-in POST /api/ai/bookslm/agent (events SSE tool/message/confirmation), extraction _resolve_system_prompt
- tests: test_ai_chat.py, test_agent_loop.py + 3 tests endpoint (728 passed au total)
- ROADMAP B1/B2/B3/B7 livres ; B4/B5/B6 restants
2026-09-11 12:45:16 -04:00
bruno 400224a089 fix(ai): facade backend/tools/api.py (namespace packages, __init__ ignore)
CI / lint (push) Successful in 57s
CI / security (push) Successful in 39s
CI / test (push) Successful in 1m12s
CI / build (push) Successful in 36s
CI / e2e (push) Successful in 10m22s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
Le .gitignore exclut _*.py, donc backend/tools/__init__.py n'etait pas versionne et
la CI echouait a l'import (unknown location). Remplacement par une facade api.py
qui enregistre les outils et reexporte l'API publique. Tests adaptes.
2026-09-11 12:18:35 -04:00
bruno 5c1823d6d2 feat(ai): phase 0 couche d'outils partagee (backend/tools) + tests
CI / lint (push) Successful in 57s
CI / security (push) Successful in 39s
CI / test (push) Failing after 41s
CI / build (push) Skipped
CI / e2e (push) Skipped
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
- backend/tools/: ToolContext, registry @tool + schemas JSON, audit ai_tool_call
- Services lecture/recherche: list_vaults, list_directory, read_file, search_fulltext, list_tags
- Permissions check_vault_access + resolve_safe_path, confirmation gating (two-step)
- Redaction des secrets, limites de taille, audit JSONL (args sensibles resumes)
- tests/test_tools.py: 30 tests (registry, contexte, execution, confirmation, audit)
- ROADMAP: item #79 phase A livree (A2 partiel)
2026-09-11 12:03:21 -04:00
bruno a3642caa3d feat(ai): selection fournisseur/modele par defaut + guide architecture
- Persistance ai_default_provider / ai_default_models dans data/config.json
- Lecture + rechargement a chaud dans backend/ai.py (get_default_provider, reload_ai_config)
- UI: selecteurs Fournisseur/Modele par defaut dans la section Cles API IA (i18n FR/EN)
- docs/AI_ARCHITECTURE_GUIDE.md: architecture cible, catalogue d'outils, decisions MCP
2026-09-11 12:03:16 -04:00
bruno e2153435dc chore(version): passer la version a 2.2.1
CI / lint (push) Successful in 56s
CI / security (push) Successful in 38s
CI / test (push) Successful in 1m11s
CI / build (push) Successful in 35s
CI / e2e (push) Successful in 11m20s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
Defaut de build Docker et docs alignes sur 2.2.1.
2026-09-11 10:15:40 -04:00
bruno f307ecb380 fix(mobile): relever la boite d'edition de l'assistant au-dessus de la barre
Sur mobile, le textarea et le bouton envoyer de l'assistant AI etaient masques par la barre de navigation fixe en bas. Ajout d'un padding-bottom au panneau bookslm pour degager la barre.
2026-09-11 10:15:36 -04:00
bruno 50b823e76d feat(mobile): bouton assistant AI central dans la barre du bas
CI / lint (push) Successful in 57s
CI / security (push) Successful in 39s
CI / test (push) Successful in 1m16s
CI / build (push) Successful in 43s
CI / e2e (push) Successful in 19m16s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
Sur mobile, la pastille flottante etait masquee par la barre d'outils.

- Masque .ai-fab sur mobile
- Ajoute un bouton rond sureleve #mt-ai au centre de la barre (5 items)
- Extrait openAssistant() dans ai-fab.js (reutilise par la FAB et le bouton mobile)
- mobile-toolbar.js : branche le bouton sur openAssistant()
2026-09-11 09:58:04 -04:00
bruno 84bda90065 fix(version): afficher 2.2.0 au lieu de 2.1.0 dans l'UI
Les 3 affichages (badge header, A propos, pied de l'aide) lisaient la version bakee dans l'image Docker, qui retombait sur l'ARG VERSION=2.1.0 car build.sh/build.ps1 ne transmettaient pas le tag git.

- Dockerfile / docker-compose.yml : defaut porte a 2.2.0
- build.sh : export VERSION pour substitution par docker compose
- build.ps1 : version derivee de git describe puis exportee
- CHANGELOG / ROADMAP alignes sur 2.2.0
2026-09-11 09:57:59 -04:00
bruno d6fdcef4d4 feat(ai): assistant AI multi-contexte + ergonomie sidebar et clavier
CI / lint (push) Successful in 1m12s
CI / security (push) Successful in 43s
CI / test (push) Successful in 1m10s
CI / build (push) Successful in 35s
CI / e2e (push) Successful in 11m47s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
Contexte:
- Repertoire (mode historique BooksLM, via menu contextuel dossier)
- Documents (bouton flottant quand des documents sont ouverts)
- General (bouton flottant sans document: aide app + creation de fichiers)
- Le contexte est affiche dans le header; suggestions adaptees au mode
- Blocs obsigate-action rendus en carte avec bouton Appliquer (pas d'execution auto)

Backend:
- /api/ai/bookslm/context et /chat acceptent mode + context_files
- collect_files_context(), empty_context(), build_general_system_prompt()
- mode documents degrade vers general si aucun fichier lisible

Corrige:
- bouton flottant passait le chemin d'un fichier comme repertoire -> erreur
  'Aucun fichier markdown trouve dans ce dossier'
- panneau precedent masque (localStorage) ne se rouvrait plus
- selecteur fournisseur/modele deplace sur une ligne dediee (header actions visibles)
- Entree envoie, Ctrl+Entree insere un saut de ligne

Tests: test_bookslm.py (+13) et tests/frontend/ai.test.mjs (+8)
2026-09-11 09:11:00 -04:00
bruno 62023acd4d feat(api,ai): #72 OpenAPI 3.1 enrichie + fiabilisation de l'outil AI UI
CI / lint (push) Successful in 55s
CI / security (push) Successful in 37s
CI / test (push) Successful in 1m19s
CI / build (push) Successful in 34s
CI / e2e (push) Successful in 10m27s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
API (#72):
- backend/openapi_docs.py: 18 tags documentes, assignation auto par prefixe,
  securite bearerAuth/cookieAuth, erreurs 401/403/404/422/500, exemples,
  page /api autonome (liens /docs, /redoc, /openapi.json)
- backend/schemas.py: response_model Pydantic pour ~35 endpoints sans modele
- main.py: FastAPI enrichi + override app.openapi; routes /api et /api/
- ai_routes/bookslm_routes: response_model + doc SSE
- frontend: entree 'API' du menu (i18n FR/EN)
- tests/test_openapi.py (58 tests)

AI UI:
- BooksLM: requetes authentifiees (AuthManager), parsing SSE {token}/error,
  message utilisateur correct (plus le placeholder vide), barre de contexte
- AI Editor: Ctrl/Cmd+J lie une seule fois, libelles i18n, modale de
  reecriture accessible a la place de window.prompt()
- tests/frontend/ai.test.mjs (7 tests) + CI
2026-09-11 01:40:21 -04:00
bruno 75ef597ece ci(e2e): rendre le vault de test inscriptible (docker cp root -> UID 1000)
CI / lint (push) Successful in 54s
CI / security (push) Successful in 39s
CI / test (push) Successful in 1m10s
CI / build (push) Successful in 32s
CI / e2e (push) Successful in 10m32s
Les fixtures copiees via docker cp appartiennent a root ; l'app tourne en UID 1000 et os.access(W_OK) renvoyait False -> POST /api/file 403 'Vault is read-only' -> le test E2E de creation .excalidraw (C8) echouait. Ajout d'un chmod -R a+rwX apres le health check.
2026-09-10 23:07:33 -04:00
bruno 0cbbbf260a fix(search,excalidraw,e2e): recherche cassee + corruption .excalidraw.md + tests E2E #78
CI / lint (push) Successful in 54s
CI / security (push) Successful in 38s
CI / test (push) Successful in 1m7s
CI / build (push) Successful in 33s
CI / e2e (push) Failing after 10m52s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
- frontend/js/search.js : onSearchFilter appelee avec (vault, query, results) — renvoyait undefined, cassait le rendu et basculait sur la recherche offline (0 resultat). Regression plugins depuis 30f2df3.
- frontend/excalidraw-editor.html + js/excalidraw-viewer.js : ignorer onChange au montage (plus d'auto-save spurieux qui reecrivait .excalidraw.md en JSON brut) + preservation du format Obsidian frontmatter + compressed-json a la sauvegarde.
- frontend/js/app.js : __OBSIGATE_BOOTED pose apres init() (splash + E2E attendent les handlers lies).
- tests/e2e : goHome/login attendent le boot ; excalidraw.spec.js reecrit avec les selecteurs reels (modal, context menu, canvas.first()).
- ROADMAP : C8 (creation via menu contextuel) marque fait.
- Verifie : 86/86 E2E chromium-desktop, 599 backend, ruff, validate-imports, JSDOM.
2026-09-10 22:51:08 -04:00
bruno 4f99d5ce61 docs(excalidraw,roadmap): #78 H1-H3 doc utilisateur + #61/#74 optionnels non retenus
CI / lint (push) Successful in 55s
CI / security (push) Successful in 39s
CI / test (push) Successful in 1m6s
CI / build (push) Successful in 34s
CI / e2e (push) Failing after 22m30s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
- README FR/EN : .excalidraw dans les formats supportes + section Diagrammes Excalidraw
- Guide integre : nouvelle section Excalidraw (i18n FR/EN) + note compatibilite plugin Obsidian
- ROADMAP : #78 termine (B5/F3/H1-H3), C8/F2 non retenus ; #61 et #74 optionnels marques non retenus
- CHANGELOG : entree Documentation
2026-09-10 20:51:25 -04:00
bruno f96c4ccd64 fix(ui,help): #61 version display in About modal + help guide footer (footer_tagline i18n, about-version id)
CI / lint (push) Successful in 52s
CI / security (push) Successful in 35s
CI / test (push) Successful in 1m5s
CI / build (push) Successful in 32s
CI / e2e (push) Failing after 20m14s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
2026-09-10 10:10:32 -04:00
bruno ffa07b7df4 fix(ui,help): #61 unified dynamic version display + full guide sections for AI editor & plugins
CI / lint (push) Successful in 54s
CI / security (push) Successful in 35s
CI / test (push) Successful in 1m13s
CI / build (push) Successful in 32s
CI / e2e (push) Failing after 22m0s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
2026-09-10 09:44:41 -04:00
bruno e81dd9a7ec fix(ruff): #61 I001 import sorting + RUF012 ClassVar annotation
CI / lint (push) Successful in 51s
CI / security (push) Successful in 34s
CI / test (push) Successful in 1m9s
CI / build (push) Successful in 32s
CI / e2e (push) Failing after 21m30s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
2026-09-10 09:05:30 -04:00
bruno 30f2df3004 feat: #61 Plugin system — backend API, sandboxed Web Worker, hooks wired to real app flow, user guide, 47+21 tests
CI / lint (push) Failing after 30s
CI / test (push) Skipped
CI / build (push) Skipped
CI / e2e (push) Skipped
CI / security (push) Successful in 35s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
2026-09-10 09:01:38 -04:00
bruno ac16fc1f0e feat: #78 Excalidraw finitions + #67 push + #68 health-detailed + #77 desktop jumplist
CI / lint (push) Successful in 52s
CI / security (push) Successful in 36s
CI / test (push) Successful in 1m6s
CI / build (push) Successful in 1m15s
CI / e2e (push) Failing after 12m40s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
#78 Excalidraw — finitions B5/C8/F2/F3
- backend/indexer.py: port Python pur de lz-string decompressFromBase64 (bitsPerChar=6, resetValue=32) validé contre 4 fixtures JS truth (texte accentué, edge cases) — remplace le stub base64 non-fonctionnel
- B5 indexation: .excalidraw.md (format plugin Obsidian) maintenant décompressé côté Python → texte indexé pour recherche TF-IDF
- 7 nouveaux tests B5 dans tests/test_excalidraw.py (13/13 total)
- F2: excalidraw-viewer.test.mjs enregistré dans CI (lint job)
- F3: tests/e2e/excalidraw.spec.js (9 specs — ouverture .excalidraw/.excalidraw.md, création via modale/menu contextuel, toolbar, thème, pop-out, recherche)
- Fixtures test_vault/diagram.excalidraw + diagram.excalidraw.md

#67 Push notifications (Web Push API + VAPID)
- backend/push.py: router + VAPID keys + send_push_notification (pywebpush>=2.3.0)
- frontend/js/push.js: subscription UI + service worker integration
- tests/test_push.py: 10 tests (subscribe/list/unsubscribe/vapid key)
- requirements.txt + sw.js + locales push strings

#68 Health check enrichi
- backend/main.py: GET /api/health/detailed (admin) — memory/cpu/disk/backups/index/SSE connections
- backend/indexer.py: _last_full_index_ts tracking
- tests/conftest.py: admin_client fixture
- tests/test_api_main.py: 3 nouveaux tests health detailed

#77 Desktop jumplist
- desktop/src/jumplist.rs: Windows jumplist integration
- Cargo.toml/lock + capabilities + main.rs
- frontend/js/desktop.js: Tauri bridge (isTauriEnv, invoke, getSystemTheme, syncSystemTheme, shouldShowWizard, crash banner)
- tests/frontend/desktop.test.mjs: 21 tests JSDOM (détection, invoke degradation, theme gating, wizard, crash banner)
- tests/frontend/unit.test.mjs: desktop.js whitelisted (standalone global reader)

# Frontend & CI
- frontend/sw.js: réécriture complète (precache + push event handlers + offline)
- frontend/index.html: section push dans settings + about repositionné
- frontend/js/app.js: initDesktopIntegration + initPush
- CI .gitea/workflows/ci.yml: excalidraw-viewer.test.mjs ajouté au lint job

All checks: ruff clean, 552 backend tests pass, 9 frontend unit, 5 excalidraw-viewer, 21 desktop, 9 pane-manager, validate-imports 33 modules.
2026-09-09 23:18:29 -04:00
bruno e4aca3b31e fix(pdf,excalidraw): BUG-001 + BUG-002
CI / lint (push) Successful in 47s
CI / security (push) Successful in 32s
CI / test (push) Successful in 1m0s
CI / build (push) Successful in 30s
CI / e2e (push) Successful in 9m22s
Desktop Build / build-windows (push) Canceled after 0s
Desktop Build / build-linux (push) Canceled after 0s
BUG-001 (PDF stream 500) : le nom Unicode du fichier etait injecte brut
dans Content-Disposition -> en-tete HTTP invalide -> 500. Nouveau helper
_content_disposition (RFC 5987 : filename ASCII + filename*=UTF-8'').
Applique aux 2 branches /pdf/stream. Test: tests/test_pdf_stream.py.

BUG-002 (Excalidraw 'Loading...' infini) : deux causes.
1) L'import esm.sh avec ?alias=react:... etait propage a chaque dep
   transitive; esm.sh renvoie 408 sur les builds a gamme semver (jotai@>=2.9.2)
   -> import entier echoue. On retire l'alias et on utilise React 19 coherent
   (spec a gamme + target identique a celle d'Excalidraw) -> plus de 408.
2) L'editeur passait la prop legacy 'excalidrawRef', inoperante en 0.18
   (la prop reelle est 'excalidrawAPI') -> l'API n'etait jamais recue,
   le 'Loading' ne se masquait pas, save/export morts.

Verifie : 534 tests backend verts (dont 3 nouveaux PDF) + E2E navigateur
(loading masque + cycle requestSave->save OK).
2026-09-09 13:04:40 -04:00
bruno 212753f311 test(e2e): #75 I3 — matrice ouverture/fermeture split view (21 tests) + fix chemins Windows run-e2e-local.sh
CI / lint (push) Successful in 42s
CI / security (push) Successful in 27s
CI / test (push) Successful in 55s
CI / build (push) Successful in 23s
CI / e2e (push) Successful in 9m33s
2026-09-08 09:38:10 -04:00
bruno ccfcb58ed1 docs(roadmap): corrections cases #74/#75/#76/#78 — optionnels non retenus de-coches, reels coches, dedup I2/I3
CI / lint (push) Successful in 44s
CI / security (push) Successful in 26s
CI / test (push) Successful in 53s
CI / build (push) Successful in 22s
CI / e2e (push) Successful in 6m0s
2026-09-08 06:25:46 -04:00
209 changed files with 41722 additions and 3621 deletions
+1
View File
@@ -15,6 +15,7 @@ docker-compose.yml
.env.*
*.log
.obsigate-backup
backend/VERSION
.pytest_cache
htmlcov
.coverage
+11 -2
View File
@@ -16,8 +16,17 @@ OBSIGATE_ADMIN_PASSWORD=chab30
# Rate limiting
# OBSIGATE_LOGIN_MAX_ATTEMPTS=10
# OBSIGATE_ACCOUNT_MAX_ATTEMPTS=10
# OBSIGATE_LOGIN_WINDOW_SECONDS=900
# IP client derrière un reverse proxy (fait confiance à X-Forwarded-For)
# OBSIGATE_TRUST_PROXY=false
# Webhooks : sécurité SSRF
# OBSIGATE_WEBHOOK_ALLOW_HTTP=false # autoriser http:// (défaut : HTTPS requis)
# OBSIGATE_WEBHOOK_ALLOW_PRIVATE=false # autoriser les IP privées/boucle
# Secret d'un webhook : OBSIGATE_WEBHOOK_SECRET_<ID_WEBHOOK_EN_MAJUSCULES>
# Watcher
# OBSIGATE_WATCHER_ENABLED=true
# OBSIGATE_WATCHER_USE_POLLING=false
@@ -48,8 +57,8 @@ OBSIGATE_ADMIN_PASSWORD=chab30
# AI_DEFAULT_PROVIDER=deepseek # deepseek | openrouter | gemini
# DeepSeek (recommandé, bon marché)
DEEPSEEK_API_KEY=sk-87d93019602e4c279679dfe80d504bc3
DEEPSEEK_MODEL=deepseek-v4-pro
DEEPSEEK_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
DEEPSEEK_MODEL=deepseek-chat
# OpenRouter (accès à plusieurs modèles)
# OPENROUTER_API_KEY=sk-or-v1-...
+36 -8
View File
@@ -30,7 +30,7 @@ jobs:
run: ruff check backend/
- name: Mypy (type checker)
run: mypy backend/ --ignore-missing-imports || echo "mypy found type errors (advisory — 28 pre-existing issues)"
run: mypy backend/ --ignore-missing-imports
- name: Frontend validation
run: node tests/frontend/validate-imports.mjs
@@ -38,15 +38,39 @@ jobs:
- name: Frontend unit tests
run: node tests/frontend/unit.test.mjs
- name: Frontend JSDOM tests (PaneManager)
- name: Frontend JSDOM tests (PaneManager + Excalidraw + Plugins + AI + SW + Collab + Mobile + Semantic + Desktop + Inline edition)
run: |
cd tests/frontend
if [ -d node_modules ]; then
node pane-manager.test.mjs
node excalidraw-viewer.test.mjs
node plugins.test.mjs
node ai.test.mjs
node ai-sidebar.test.mjs
node sidebar-filters.test.mjs
node sw.test.mjs
node collab.test.mjs
node mobile-editor.test.mjs
node semantic-search.test.mjs
node desktop.test.mjs
node toolbar-order.test.mjs
node editor-inline.test.mjs
else
echo "tests/frontend/node_modules missing — installing jsdom"
echo "tests/frontend/node_modules missing - installing jsdom"
npm install --no-audit --no-fund --silent
node pane-manager.test.mjs
node excalidraw-viewer.test.mjs
node plugins.test.mjs
node ai.test.mjs
node ai-sidebar.test.mjs
node sidebar-filters.test.mjs
node sw.test.mjs
node collab.test.mjs
node mobile-editor.test.mjs
node semantic-search.test.mjs
node desktop.test.mjs
node toolbar-order.test.mjs
node editor-inline.test.mjs
fi
# ── Tests ─────────────────────────────────────────────────────────
@@ -106,10 +130,10 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Generate VERSION file
run: |
VERSION=$(git describe --tags --dirty 2>/dev/null | sed 's/^v//' || echo "0.0.0-dev")
echo "$VERSION" > backend/VERSION
- name: Version livrée
# VERSION (racine du dépôt) est copié dans l'image par le Dockerfile :
# plus aucun numéro généré ni codé en dur dans le pipeline.
run: echo "Version livree = $(cat VERSION)"
- name: Configure DNS (workaround flaky 127.0.0.11 resolver)
# GitHub Actions runners occasionally fail to resolve auth.docker.io via
@@ -188,6 +212,10 @@ jobs:
sleep 1
done
curl -sf "http://$GW:2029/api/health" >/dev/null || { echo "App not reachable at $GW:2029"; exit 1; }
# Les fixtures sont copiées via `docker cp` en root → rendre le vault
# inscriptible par l'utilisateur non-root de l'app (UID 1000), sinon
# toute création/édition de fichier renvoie 403 « Vault is read-only ».
docker exec -u 0 obsigate-e2e chmod -R a+rwX /vaults/TestVault /vaults/TestDir || true
- name: Run E2E tests
run: |
@@ -205,4 +233,4 @@ jobs:
- name: Cleanup
if: always()
run: docker rm -f obsigate-e2e
run: docker rm -f obsigate-e2e
+44 -11
View File
@@ -1,13 +1,10 @@
name: Desktop Build
on:
push:
branches: [main]
paths:
- 'desktop/**'
- 'frontend/**'
- 'backend/**'
workflow_dispatch: # permet de lancer manuellement
# Lancement manuel uniquement : aucun runner self-hosted [windows/linux, desktop]
# n'est enregistré dans Gitea. Le build Windows se fait en local
# (desktop/build-windows.bat). Ce workflow reste disponible pour un futur runner.
workflow_dispatch:
jobs:
build-windows:
@@ -37,13 +34,27 @@ jobs:
- name: Build MSI
working-directory: desktop
run: cargo tauri build --bundles msi
shell: powershell
env:
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
run: |
if ([string]::IsNullOrEmpty($env:TAURI_SIGNING_PRIVATE_KEY)) {
Write-Host "TAURI_SIGNING_PRIVATE_KEY absent - build sans artefacts de mise a jour."
cargo tauri build --bundles msi --config '{"bundle":{"createUpdaterArtifacts":false}}'
} else {
Write-Host "Signature des artefacts de mise a jour activee."
cargo tauri build --bundles msi
}
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
- name: Upload MSI artifact
uses: actions/upload-artifact@v4
with:
name: obsigate-windows-msi
path: desktop/target/release/bundle/msi/*.msi
path: |
desktop/target/release/bundle/msi/*.msi
desktop/target/release/bundle/msi/*.msi.sig
retention-days: 30
- name: Publish to Gitea Release
@@ -82,11 +93,31 @@ jobs:
- name: Build AppImage
working-directory: desktop
run: cargo tauri build --bundles appimage
env:
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
run: |
if [ -z "$TAURI_SIGNING_PRIVATE_KEY" ]; then
echo "TAURI_SIGNING_PRIVATE_KEY absent - build sans artefacts de mise a jour."
cargo tauri build --bundles appimage --config '{"bundle":{"createUpdaterArtifacts":false}}'
else
echo "Signature des artefacts de mise a jour activee."
cargo tauri build --bundles appimage
fi
- name: Build deb
working-directory: desktop
run: cargo tauri build --bundles deb
env:
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
run: |
if [ -z "$TAURI_SIGNING_PRIVATE_KEY" ]; then
echo "TAURI_SIGNING_PRIVATE_KEY absent - build sans artefacts de mise a jour."
cargo tauri build --bundles deb --config '{"bundle":{"createUpdaterArtifacts":false}}'
else
echo "Signature des artefacts de mise a jour activee."
cargo tauri build --bundles deb
fi
- name: Upload Linux artifacts
uses: actions/upload-artifact@v4
@@ -94,5 +125,7 @@ jobs:
name: obsigate-linux
path: |
desktop/target/release/bundle/appimage/*.AppImage
desktop/target/release/bundle/appimage/*.AppImage.sig
desktop/target/release/bundle/deb/*.deb
desktop/target/release/bundle/deb/*.deb.sig
retention-days: 30
+68
View File
@@ -0,0 +1,68 @@
#!/bin/sh
# -----------------------------------------------------------------------------
# ObsiGate — rattache l'incrément de version au commit qui vient d'être créé,
# puis crée le tag `vX.Y.Z` de la version livrée.
#
# Deux situations :
# 1. `prepare-commit-msg` a incrémenté ./VERSION et mis à jour les fichiers
# dérivés (package.json, desktop, READMEs, ROADMAP, CHANGELOG) : on les
# rattache ici au commit via `--amend` (le commit n'est pas encore poussé) ;
# 2. le message venait d'un éditeur : l'incrément a été différé, il est
# calculé ici à partir du message final (`COMMIT_EDITMSG`).
#
# Résultat : la version livrée, le contenu du commit et le tag correspondent
# toujours. Le tag est publié avec la branche à chaque push (`push.followTags`,
# posé par scripts/install-hooks.sh).
# -----------------------------------------------------------------------------
set -eu
[ "${SKIP_VERSION_BUMP:-}" = "" ] || exit 0
[ "${OBSIGATE_VERSION_AMENDING:-}" = "" ] || exit 0 # garde anti-récursion
root="$(git rev-parse --show-toplevel)"
version_file="$root/VERSION"
[ -f "$version_file" ] || exit 0
# Opérations en cours (rebase, merge, cherry-pick…) : ne jamais amender ici.
for state in rebase-merge rebase-apply CHERRY_PICK_HEAD REVERT_HEAD MERGE_HEAD BISECT_LOG; do
if [ -e "$(git rev-parse --git-path "$state")" ]; then
exit 0
fi
done
list="$(git rev-parse --git-path obsigate-version-files)"
defer="$(git rev-parse --git-path obsigate-version-defer)"
# Cas 2 : l'incrément avait été différé (commit rédigé dans un éditeur).
if [ -f "$defer" ]; then
rm -f "$defer"
rm -f "$list"
tool="$root/scripts/bump_version.py"
py=""
for cand in python3 python; do
if command -v "$cand" >/dev/null 2>&1; then py="$cand"; break; fi
done
if [ -n "$py" ] && [ -f "$tool" ]; then
"$py" "$tool" --from-message-file "$(git rev-parse --git-path COMMIT_EDITMSG)" \
--files-out "$list" || true
fi
fi
# Rattachement des fichiers synchronisés au commit qui vient d'être créé.
if [ -s "$list" ]; then
# shellcheck disable=SC2046
git add -- $(tr -d '\r' < "$list")
OBSIGATE_VERSION_AMENDING=1 \
git commit --amend --no-edit --no-verify --quiet
rm -f "$list"
fi
ver="$(tr -d ' \t\r\n' < "$version_file")"
[ -n "$ver" ] || exit 0
if git rev-parse -q --verify "refs/tags/v$ver" >/dev/null 2>&1; then
exit 0
fi
git tag -a "v$ver" -m "ObsiGate v$ver"
echo "ObsiGate version : tag v$ver créé (poussé avec la branche)."
+80
View File
@@ -0,0 +1,80 @@
#!/bin/sh
# -----------------------------------------------------------------------------
# ObsiGate — incrément automatique de la version livrée (SemVer) à chaque commit.
#
# Source unique de vérité : ./VERSION (MAJEUR.MINEUR.CORRECTIF)
#
# `!:` / `BREAKING CHANGE:` -> MAJEUR (x.0.0)
# `feat:` -> MINEUR (x.y.0)
# tout le reste (fix, perf…) -> CORRECTIF (x.y.z)
#
# Aucun incrément pour : merge, squash, revert, `chore(release)`, amend.
# Commit rédigé dans un éditeur (message pas encore saisi) : l'incrément est
# différé à `post-commit`, qui dispose du message final.
#
# `scripts/bump_version.py` incrémente VERSION et resynchronise les fichiers
# dérivés (package.json, desktop, READMEs, ROADMAP, CHANGELOG) ; le `git add` est
# fait ici, par le hook lui-même. La liste des fichiers est laissée à
# `.githooks/post-commit`, qui rattache l'incrément au commit (git fige l'arbre
# avant `prepare-commit-msg` : un `git add` à cet instant ne serait pris en
# compte qu'au commit suivant).
#
# Neutraliser ponctuellement : SKIP_VERSION_BUMP=1 git commit ...
# -----------------------------------------------------------------------------
set -eu
[ "${SKIP_VERSION_BUMP:-}" = "" ] || exit 0
msg_file="${1:-}"
source_kind="${2:-message}"
case "$source_kind" in
merge|squash|commit) exit 0 ;; # merge / squash / amend (-c, -C, --amend)
esac
[ -n "$msg_file" ] && [ -f "$msg_file" ] || exit 0
root="$(git rev-parse --show-toplevel)"
tool="$root/scripts/bump_version.py"
[ -f "$tool" ] || exit 0
py=""
for cand in python3 python; do
if command -v "$cand" >/dev/null 2>&1; then py="$cand"; break; fi
done
if [ -z "$py" ]; then
echo "⚠ version : aucun interpréteur Python trouvé — version non incrémentée." >&2
exit 0
fi
list="$(git rev-parse --git-path obsigate-version-files)"
defer="$(git rev-parse --git-path obsigate-version-defer)"
# Message pas encore rédigé (ouverture d'un éditeur) : on diffère l'incrément.
if ! grep -qvE '^[[:space:]]*(#|$)' "$msg_file"; then
: > "$defer"
exit 0
fi
rm -f "$defer" # marqueur d'un commit précédent avorté
out=""
if out="$("$py" "$tool" --from-message-file "$msg_file" --files-out "$list" 2>&1)"; then
printf '%s\n' "$out"
else
status=$?
if [ "$status" = "3" ]; then
rm -f "$list"
exit 0 # 3 = version inchangée (revert, release…) : pas une erreur
fi
echo "✗ version : échec de l'incrément automatique de ./VERSION" >&2
printf '%s\n' "$out" >&2
echo " → corriger, ou passer outre avec : SKIP_VERSION_BUMP=1 git commit ..." >&2
exit 1
fi
# Ajout des fichiers synchronisés (chemins explicites).
if [ -s "$list" ]; then
# shellcheck disable=SC2046
git add -- $(tr -d '\r' < "$list")
fi
+6
View File
@@ -31,3 +31,9 @@ desktop/backend/
desktop/frontend/
backend/VERSION
# Tauri updater signing keys (private key — never commit)
desktop/*.key
desktop/*.key.pub
desktop/key/
+54
View File
@@ -0,0 +1,54 @@
# AGENTS.md — Instructions obligatoires du dépôt ObsiGate
> Ces instructions s'appliquent à **toute** intervention (humaine ou IA) sur ce dépôt.
## Règle n°1 — Méthode de livraison unique
Avant toute tâche (fonctionnalité, bug, refactor), **lire et appliquer**
[`docs/DELIVERY_WORKFLOW.md`](./docs/DELIVERY_WORKFLOW.md) (Definition of Done).
Aucune tâche n'est terminée avant que sa checklist soit complète **et le CI vert**.
## Avant de commencer
1. Lire [`docs/ROADMAP.md`](./docs/ROADMAP.md) (travail à venir + index) et
[`docs/ISSUES_TODOLIST.md`](./docs/ISSUES_TODOLIST.md) (bugs).
2. Identifier ou créer l'**ID stable** (`#NN` pour une feature, `BUG-NNN` pour un bug)
et passer son statut à « en cours » **avant** de coder.
## À la fin de chaque tâche (obligatoire)
- Ajouter/mettre à jour les **tests unitaires**.
- Vérifications locales vertes : `pytest`, `ruff`, `mypy`, tests frontend (`E2E` si UI).
- Mettre à jour la documentation requise : `CHANGELOG.md` (`[Unreleased]`), `docs/ROADMAP.md`
(statut + index), fiche `docs/features/` **ou** `docs/archive/`, `docs/ISSUES_TODOLIST.md`,
guide utilisateur i18n FR/EN + README si impact utilisateur.
- **Commit** conventionnel référençant l'ID, puis **push**.
- Version : le fichier VERSION (racine du dépôt) est la **source unique de
vérité (MAJEUR.MINEUR.CORRECTIF), incrémenté automatiquement à chaque commit** par le hook
.githooks/prepare-commit-msg — feat → mineur, !: / BREAKING CHANGE → majeur, sinon
correctif. Le même commit resynchronise package.json, le desktop Tauri, README.md/
README.fr.md, docs/ROADMAP.md et publie la section [Unreleased] du CHANGELOG.md en
[X.Y.Z] — date ; le tag vX.Y.Z est créé au commit et publié au push (push.followTags).
Hooks à installer une fois par clone : scripts/install-hooks.sh. Garde-fou :
tests/test_version.py (détail : docs/DELIVERY_WORKFLOW.md §7).
- Vérifier le **CI Gitea vert** (jobs `lint`, `test`, `security`, `build`, `e2e`).
## Cartographie documentaire
| Sujet | Fichier |
|---|---|
| Méthode de livraison / DoD | `docs/DELIVERY_WORKFLOW.md` |
| Version livrée (source unique) | VERSION + scripts/bump_version.py |
| Travail à venir + index | `docs/ROADMAP.md` |
| Historique des versions | `CHANGELOG.md` |
| Conception par feature | `docs/features/<slug>.md` |
| Archive du complété | `docs/archive/COMPLETED_v1-v2.md` |
| Bugs / TODO | `docs/ISSUES_TODOLIST.md` |
| Build & releases | `docs/DEVELOPMENT_AND_RELEASES.md` |
| Standards de code | `docs/CONTRIBUTING.md` |
## Conventions
- Commits : `type: description` — `feat`, `fix`, `perf`, `refactor`, `docs`, `style`, `chore`, `test`.
- **Ne jamais** committer de secrets, clés ou tokens.
- Réponses et documentation en **français** ; respecter le style du code existant.
+965 -4
View File
@@ -5,12 +5,964 @@ Toutes les modifications notables d'ObsiGate sont documentées dans ce fichier.
Format basé sur [Keep a Changelog](https://keepachangelog.com/fr/1.1.0/),
et [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
> **En cours de développement** : les changements non publiés sont dans la section
> [2.1.0](#210--2026-09-07). La dernière version publiée est **2.0.0**.
> **En cours de développement** : les changements à venir sont listés dans la section
> [Unreleased](#unreleased). La dernière version livrée est **2.6.1**.
---
## [2.1.0] — 2026-09-07
## [Unreleased]
---
## [2.6.1] — 2026-09-16
---
## [2.6.0] — 2026-09-16
### Ajouté
- **Barre de filtrage de la sidebar sur « Récents » et « Sauvegardes » (#99)** : la barre
de recherche de la sidebar agit désormais sur les onglets **Récents**
(`filterRecentFiles`, filtrage titre/chemin/vault/aperçu/tags) et **Sauvegardes**
(`filterSavedSearches`, cumulable avec les pills Tous/Recherches/Répertoires) —
insensible à la casse et aux accents, avec message d'absence de résultat et
placeholders dédiés (`sidebar.filter_recent`, `sidebar.filter_saved`). Le routage de
`initSidebarFilter` est unifié (`routeFilter`/`routeClear`) pour couvrir les cinq
onglets. Fiche : [docs/features/sidebar-filters.md](./docs/features/sidebar-filters.md).
- **Assistant IA — Deep Research en pastille (#100)** : « Deep Research » ajoute
désormais une **pastille** (comme les skills) au lieu d'écrire la directive dans la
zone de saisie ; le mode Agent est activé et la directive est injectée au moment de
l'envoi, sans polluer le message affiché.
### Corrigé
- **BUG-049 — icône du bouton « + » de l'assistant invisible** : la règle générique
`.bookslm-input-area button` écrasait `.bookslm-btn-plus` (`padding: 8px 16px` sur une
largeur de 32 px ⇒ largeur de contenu nulle ⇒ SVG à 0 px). Sélecteur porté à
`.bookslm-input-area button.bookslm-btn-plus`, l'icône `plus` est de nouveau visible
(vérifié en navigateur : SVG 0 px → 18 px). Tests : `tests/frontend/ai.test.mjs` (+1),
`tests/frontend/sidebar-filters.test.mjs` (nouveau, 8 tests).
---
## [2.5.2] — 2026-09-16
---
## [2.5.1] — 2026-09-16
---
## [2.5.0] — 2026-09-16
### Ajouté
- **Assistant IA — filtre de recherche dans la sidebar « Historique IA » (#98)** : la barre
de filtrage de la sidebar agit désormais sur l'onglet Historique IA. `filterAIHistory()`
(`frontend/js/config.js`) filtre les sessions par titre, aperçu, répertoire, contexte ou
libellé de mode — insensible à la casse et aux accents (`_aiNorm`) — avec cache sessions
(`_aiSessionsCache`), message « aucune conversation ne correspond à la recherche »
(`bookslm.history_no_match`) et placeholder dédié `sidebar.filter_ai`. Le routage
saisie / touche casse / bouton « × » vers ce filtre est géré dans `initSidebarFilter`
(`frontend/js/sidebar.js`) quand l'onglet IA est actif.
### Corrigé
- **BUG-048 — menu d'ajout « + » : « Contextes » et « Skills » ouvrent enfin leurs menus
« @ » / « / »** : le clic sur une entrée du panneau `.bookslm-ext-menu` remontait au
gestionnaire du panneau qui annulait le rendu asynchrone du menu (jamais affiché) ;
`e.stopPropagation()` sur les entrées corrige le comportement. Le bouton « + » porte
l'icône Lucide `plus`. Tests : `tests/frontend/ai.test.mjs` (+3), nouvelle suite
`tests/frontend/ai-sidebar.test.mjs` (6 tests).
---
## [2.4.0] — 2026-09-16
### Ajouté
- **Assistant IA — historique permanent des conversations (#95)** : les échanges sont
désormais **persistés côté backend** (`backend/ai_history.py`, `data/ai_history/{user}.json`,
cap 200, écriture atomique) et plus seulement en localStorage. Nouveaux endpoints
`GET/PUT/DELETE /api/ai/bookslm/history[…]` (liste résumée sans messages → full,
upsert qui force l'id, suppression, isolation par utilisateur). Le panneau synchronise
ses sessions au chargement (debounce 600 ms, repli hors-ligne sur le cache local),
migre les anciennes clés `bookslm-sessions-*` / `bookslm-history-*` et notifie la
sidebar via l'événement `bookslm:history-updated`.
- **Assistant IA — accès rapide à l'historique dans la sidebar (#96)** : cinquième onglet de
navigation `#sidebar-tab-ai` (icône `messages-square`) listant les conversations par
ordre chronologique ; un clic ouvre la conversation dans le panneau Assistant IA
(`openWithSession`).
- **Assistant IA — panneau « + » extensible (#97)** : le bouton « Attach an image » est
remplacé par un bouton **« + »** ouvrant un panneau modulaire (registre `_extensions`
simple à étendre) : Fichiers, Image, Contextes, Skills, Deep Research (mode agent +
prompt pré-rempli), Recherche web et Canva (les deux derniers en « Bientôt »).
- **Assistant IA — bouton de soumission arrondi (#94)** : `.bookslm-btn-send` circulaire
(40 px), icône Lucide `arrow-up` remplaçant l'avion ✈️.
- **Fiche feature** : [docs/features/ai-assistant-history.md](./docs/features/ai-assistant-history.md).
Tests : `tests/test_bookslm.py` (+11), `tests/frontend/ai.test.mjs` (+3).
---
## [2.3.4] — 2026-09-16
### Documentation
- **AGENTS.md — versionnage automatique** : la méthode obligatoire de fin de tâche rappelle
désormais que `VERSION` (racine) est la source unique de vérité (`MAJEUR.MINEUR.CORRECTIF`),
incrémentée à chaque commit par le hook `.githooks/prepare-commit-msg`, que les dérivés
(`package.json`, desktop Tauri, READMEs, `docs/ROADMAP.md`, `CHANGELOG.md`) sont
resynchronisés dans le même commit, et que le tag `vX.Y.Z` est publié au push. Ligne
ajoutée à la cartographie documentaire. Ménage : suppression des captures de diagnostic
(`diag-*.png`, `ogdiag.png`, `state.png`) et du script jetable `tmp-verify-inline.cjs`
restés à la racine du dépôt.
---
## [2.3.3] — 2026-09-16
### Corrigé
- **#93 (complément) — garde-fous de session d'édition inline** : une session d'édition
(« Editer » ou Forge) qui possède la zone de contenu était détruite par deux chemins
qui vident cette zone sans la libérer d'abord — l'**activation d'un onglet**
(`TabManager.activate`, `PaneTabManager.activate`) et l'événement SSE **`index_updated`**
sur le fichier affiché (un fichier modifié hors de l'app relançait `openFile()` et écrasait
le DOM de l'éditeur). L'activation d'onglet appelle désormais `detachInlineEditor()` avant
le placeholder de chargement, et l'écriture externe passe par `reloadExternalWrite()` (le
**tampon de l'éditeur** est rechargé depuis le disque au lieu d'un re-rendu de la vue
lecture). Nouveau helper `queryEditor()` (`editor-inline.js`) : l'en-tête, le pied et la
marque voyagent avec le conteneur en mode inline, un `modal.querySelector()` les manquait ;
`closeEditor()` remet le conteneur dans la modale **avant** de restaurer en-tête/pied/marque.
Fiche : [docs/features/editeur-inline.md](./docs/features/editeur-inline.md).
Tests : `tests/frontend/editor-inline.test.mjs` (25).
- **Documentation — hooks git obligatoires à l'installation** : `docs/CONTRIBUTING.md`
documente l'appel à `scripts/install-hooks.sh` (versionnage automatique) dans les étapes
de mise en place d'un clone.
---
## [2.3.2] — 2026-09-16
---
## [2.3.1] — 2026-09-16
---
## [2.3.0] — 2026-09-16
### Ajouté
- **#89 Drag & Drop complet de fichiers/dossiers & intégration Assistant IA** — solution
complète de glisser-déposer de fichiers individuels, multiples et **dossiers récursifs**
depuis l'OS vers le vault (`POST /api/vault/{vault}/batch-upload`, FileSystem API),
overlay plein écran à double cible (dépôt vault vs analyse IA), surbrillance des cibles dans
l'arborescence, et dropzone BooksLM pour injection de contexte ou pièces jointes.
Fiche : [docs/features/drag-and-drop-ai.md](./docs/features/drag-and-drop-ai.md).
- **#89 Pretty print fichiers texte** — bouton « Pretty » dans le mode lecture des fichiers
non-markdown (code, JSON, configs, logs...) : formatage JSON, numéros de ligne avec gouttière
et coloration syntaxique highlight.js alignée sur le thème clair/sombre, avec colorateur
générique de secours (chaînes, commentaires, nombres, niveaux de log) quand highlight.js
ne reconnaît pas le langage (txt, log, plaintext).
- **#88 Assistant IA : contexte applicatif** — l'assistant en mode **Général**
reçoit l'état vivant de l'interface (`app_context`) et connaît désormais les
documents ouverts, le répertoire et le vault courants, la recherche en cours
(requête + premiers résultats affichés) et les fichiers récemment modifiés du
utilisateur. Le prompt Général est enrichi d'un bloc « Contexte applicatif
actuel » (`build_general_system_prompt(vaults, app_context, recent_files)`).
Fiche : [docs/features/ai-app-context.md](./docs/features/ai-app-context.md).
- **#90 Barre d'actions du document — regroupement fonctionnel** : les boutons
en haut du document sont réordonnés en quatre groupes séparés par des
barres verticales (`span.action-sep`) :
`[TOC] [pop-out] [Bookmark] | [Editer] [Source] [.md] [Forge] | [Copier] [PDF] [Export] | [Partager]`.
Le bouton « Pretty » (fichiers texte) rejoint le groupe édition, les
boutons non applicables à un type de fichier sont omis et les spacers d'un
groupe vide ne sont pas rendus. Fichiers texte non markdown :
`[pop-out] [Bookmark] | [Editer] [Source] [.ext] [Pretty] | [Copier] | [Partager]`
(pas de TOC, pas de PDF/Export). La fenêtre pop-out (`popout.html`) est
alignée sur le même ordre markdown (groupes nav / édition / export / partage).
Tests : `tests/frontend/toolbar-order.test.mjs` (9).
- **#91 Assistant IA — zone de discussion façon Notion** : le fil de discussion est
reformaté (messages chronologiques, post utilisateur en bulle alignée à droite,
réponse assistant pleine largeur sans fond, appels d'outils agent repliés en
bloc « N étapes ») ; après l'envoi, la **question est ancrée en haut** de la zone
visible (`scrollIntoView` `block:start` + `scroll-margin-top`) et la réponse se
diffuse en dessous — le streaming ne provoque plus de saut de scroll ; un libellé
**discret** au-dessus de chaque réponse rapporte le fournisseur et le modèle
réellement utilisés (SSE `provider`/`model`) ; une **barre d'actions** au survol
sous chaque bloc : « Copier » (texte brut + toast) et, pour les réponses,
« Ajouter » (insertion dans le document ouvert dans l'éditeur Forge).
Complément : la section « N étapes » affiche désormais des libellés humains
produits par le backend (`backend/tools/labels.py`, événements SSE en direct
pendant l'exécution + ligne « Réflexion »), et deux nouveaux outils web
principaux sont disponibles en mode agent : `web_search` (SearXNG
auto-hébergé) et `fetch_url` (page publique → texte, garde SSRF). Le reste
des catégories d'outils Notion est documenté pour le futur dans la fiche.
Fiche :
[docs/features/ai-assistant-conversation-ux.md](./docs/features/ai-assistant-conversation-ux.md)
Tests : `tests/frontend/ai.test.mjs` (76), `tests/test_tool_labels.py` (8), `tests/test_web_tools.py` (8).
- **#91 Assistant IA — section d'étapes enrichie** : l'en-tête affiche « N étapes »
suivi d'un **chevron ▶ / ▼** (fin du préfixe « > ») et, pendant l'exécution, un
**indicateur animé** de trois points devant le libellé ; la **barre de chargement**
au-dessus de la zone de saisie est supprimée (en chat simple, l'indicateur s'affiche
dans la bulle de réponse en attente). Chaque note de raisonnement devient une
**sous-section « Réflexion ▶ / ▼ »** contenant le texte du modèle, et les recherches
web ajoutent une sous-section **« Sources (N) »** avec les **liens cliquables**
(le backend émet `sources [{title, url}]` dans l'événement SSE `tool`, résultats
`web_search` plafonnés à 8 + page `fetch_url`). Les états d'ouverture (étapes,
réflexion, sources) sont mémorisés sur le message : un token re-rendu ne referme
jamais ce que l'utilisateur a déplié. `web_search` signale désormais explicitement
au modèle une instance SearXNG sans résultat (moteurs suspendus/CAPTCHA) au lieu de
le laisser relancer la même recherche jusqu'au quota.
Tests : `tests/frontend/ai.test.mjs` (82), `tests/test_bookslm.py::TestToolEventSources` (4),
`tests/test_web_tools.py` (10).
### Modifié
- **#93 Édition inline — « Editer » et « Forge » remplacent la vue lecture** : le conteneur
d'édition (`#editor-container`, CodeMirror ou iframe Forge) est déplacé **dans la zone de
contenu du document** (`#content-area`, ou le panneau actif en vue fractionnée) au lieu
d'être affiché dans l'overlay plein écran : le mode édition **remplace** le document en mode
lecture (bandeau, métadonnées, tags et contenu). L'overlay reste monté mais neutralisé
(transparent, sans capture des clics, `z-index` conservé) car le ruban d'édition mobile y est
ancré. Retour à la lecture par ✓ (sauvegarde), ✕, Échap ou l'ouverture d'un autre fichier —
trois garde-fous libèrent proprement la session (destruction CodeMirror/Yjs, retrait de
l'iframe Forge) avant qu'un rendu concurrent n'efface son DOM : `renderFile()`, l'activation
d'onglet (`TabManager.activate`, `PaneTabManager.activate`, qui vident la zone avant de
rendre) et l'événement SSE `index_updated` (un fichier modifié hors de l'app recharge le
**tampon de l'éditeur** au lieu de re-rendre la vue lecture). L'overlay n'est conservé qu'en
repli, quand la cible n'est pas le document affiché (ex. fichier créé depuis la palette).
Conséquence directe : l'éditeur ne
recouvre plus le panneau de l'assistant IA, qui peut donc mettre le document à jour **sous
les yeux de l'utilisateur** — `app_context.editing` annonce au modèle le document en cours
d'édition (et sa surface), et toute écriture d'outil (`edit_file`, `append_to_file`,
`create_file`, `restore_backup`) recharge le document affiché depuis le disque
(`obsigate:file-written`) : le tampon de l'éditeur est rafraîchi au lieu d'écraser la
modification de l'assistant par son auto-sauvegarde, et l'iframe Forge reçoit
`parent-reload`. Aucun identifiant DOM n'a changé (autosave, collab Yjs, barre IA, aperçu
Mermaid, Forge intacts). Fiche : [docs/features/editeur-inline.md](./docs/features/editeur-inline.md).
Tests : `tests/frontend/editor-inline.test.mjs` (19), `tests/test_bookslm.py::TestGeneralPrompt` (3 nouveaux).
### Corrigé
- **BUG-047 — La version affichée ne suivait pas les livraisons** : la version
provenait du **dernier tag git**, or aucun tag n'était créé lors des livraisons
→ l'application restait bloquée sur `2.2.1` alors que 66 commits avaient été
livrés (et les numéros codés en dur divergeaient : `package.json` 1.0.0,
desktop Tauri 2.0.0, `Dockerfile` 2.2.1, README 1.7.0). Le fichier **`VERSION`
(racine du dépôt) devient la source unique de vérité** (`MAJEUR.MINEUR.CORRECTIF`),
incrémenté **automatiquement à chaque commit** par le hook versionné
`.githooks/prepare-commit-msg` (`!:`/`BREAKING CHANGE` → MAJEUR, `feat` → MINEUR, sinon
CORRECTIF) ; `.githooks/post-commit` crée le tag `vX.Y.Z` et `push.followTags`
le publie à chaque push. Tous les dérivés sont resynchronisés dans le même
commit : `package.json`, desktop Tauri (`tauri.conf.json`, `Cargo.toml`,
`Cargo.lock`), `docs/ROADMAP.md`, `README.md`/`README.fr.md` et rotation de
`CHANGELOG.md` (`[Unreleased]` → `[X.Y.Z] — date`). Le backend
(`backend/version.py`), l'image Docker (`COPY VERSION`) et le desktop Tauri
lisent ce même fichier : plus aucun numéro codé en dur dans le pipeline.
Outils : `scripts/bump_version.py` (+ `scripts/bump_version.sh`),
`scripts/install-hooks.sh`. Tests : `tests/test_version.py`, dont un garde-fou
de cohérence globale (VERSION ↔ package.json ↔ desktop ↔ CHANGELOG ↔ ROADMAP ↔
README). Vérifié : `curl /api/health` sur l'instance de test → `2.3.0`.
- **BUG-045 — Double barre de défilement dans l'éditeur « Editer »** : sur les
documents longs, `#editor-body` (qui gardait `overflow:auto`) et le scroller
CodeMirror débordaient simultanément — la barre d'outils IA empilée sous un
`.cm-editor` en `height:100%` faisait déborder le corps, et un ancien override
global `.cm-scroller` forçait une seconde zone défilante. La surface
d'édition est passée en flex column : le corps ne défile plus (`overflow:
hidden`), l'éditeur remplit l'espace restant et **seul le scroller CodeMirror
défile**. Repli textarea (fallback) cohérent ; la règle mobile BUG-020 est
préservée.
- **BUG-046 — « Échec de l'action : [object Object] » (assistant IA, ajout de
texte au document courant)** : trois causes corrigées ensemble. (1) La carte
de confirmation « Appliquer » reprenait un payload dont le message était
absent après une première application échouée (continuation `payload:null`)
→ réponse 422 du backend. (2) Cette réponse 422 porte un `detail` **tableau
d'objets** (format validation FastAPI) que `new Error(detail)` réduisait en
`[object Object]` dans le toast — nouvelle méthode `_responseError()` qui
aplatit tableaux et objets en message lisible. (3) Le prompt des modes
documents/dossier ne nommait pas le vault : le modèle inventait parfois
`"vault":"test"` et l'outil d'écriture échouait silencieusement — le prompt
indique désormais le vault exact et enjoint les outils `edit_file` /
`append_to_file` / `create_file` de l'utiliser. Cache service worker
`SW_VERSION` v20.
- **#91 — Bulle utilisateur trop étroite & ancre d'envoi retardée** : le plafond de
largeur était appliqué **deux fois** (wrapper 80 % × bulle 85 % ≈ 68 % du fil), d'où
des lignes très courtes ; le plafond est désormais porté par la seule bulle (90 %),
le wrapper s'étirant → ~81 % de la largeur du fil (≈48 caractères/ligne au lieu de
~28). Par ailleurs, le rendu du message assistant « placeholder » écrasait la
position de défilement juste après l'envoi (l'ancre était annulée : la question
restait en bas jusqu'au premier token). `_isLoading` est maintenant posé avant ce
rendu et l'ancrage d'envoi est **instantané** — la question passe en haut du fil dès
la soumission et y reste pendant tout le streaming.
- **#91 — Question complètement en haut** : la marge `scroll-margin-top` (14 px)
laissait visible la fin de la réponse précédente au-dessus du post, et une réponse
courte ne remplissait pas la fenêtre — le navigateur bloquait alors le défilement et
la question restait à mi-hauteur. La marge est supprimée et le `padding-bottom` du fil
est augmenté de la place manquante tant que l'ancre est active (retiré au
dé-épinglage, aucun remplissage si le fil tient dans la fenêtre). Le rendu final
post-streaming conserve aussi l'ancre, et la libération se fait sur molette, toucher
**ou** clic dans le fil. Enfin, `overflow-anchor: none` sur le fil neutralise le
ré-ancrage automatique de Chrome (le fil est reconstruit à chaque token, ce qui
décalait la vue de ~33 px) et une passe de correction recale le défilement d'après la
géométrie mesurée.
- **#91 — Tag fournisseur/modèle incomplet** : le backend renvoyait le modèle *demandé
par le client* (`req.model or ""`), donc une valeur vide dès que le client s'en remet
au défaut du fournisseur — le tag se réduisait à « openrouter ». `_effective_model()`
résout désormais le modèle réellement utilisé dans les deux flux SSE (`/chat` et
`/agent`) : le tag affiche « openrouter · openai/gpt-4o-mini ». Couvert par
`tests/test_bookslm.py::TestEffectiveModel` et le test SSE de l'endpoint agent.
- **BUG-041 — Assistant IA bloqué sur un répertoire vide** : l'assistant ne
renvoie plus `⚠ Error: Aucun fichier markdown trouvé dans ce dossier` (HTTP 404).
Le contexte vide dégrade vers le prompt Général augmenté d'un bloc « Dossier
vide », et la requête aboutit sans contexte documentaire.
- **BUG-042 — Liens de fichiers de l'assistant non fiables** : les liens des
réponses suivent désormais une règle déterministe — un simple nom de fichier
copie le nom dans le presse-papiers, un chemin de dossier est révélé dans
l'arborescence, un chemin de fichier l'ouvre. Le chemin est **résolu contre
l'index du vault** (exact → suffixe → basename unique) avant d'agir ; les
chemins non résolus copient le nom au lieu d'afficher `File not found`.
Les **noms/chemins contenant des espaces** sont pris en charge (`Mon dossier/Ma
note.md`) : liens markdown (y compris cibles `<…>` et `%20`), code inline, et
mentions brutes liées uniquement si le chemin existe dans l'index du vault.
Les **caractères accentués** sont reconnus (classes de caractères Unicode
`\p{L}\p{N}\p{M}`, comparaison normalisée NFC : une mention décomposée
`e`+accent correspond à une entrée d'index précomposée) et un chemin
**préfixé par le nom du vault** (`TestVault/Recettes/Pizza Maison.md`) est
résolu puis ouvert dans ce vault, le préfixe étant retiré.
- **BUG-043 — Liste des fournisseurs de l'assistant non synchronisée avec la
configuration** : ajouter ou supprimer une clé API dans la configuration du
projet met désormais à jour **immédiatement** le menu Fournisseur de la barre
latérale de l'assistant, sans recharger la page. Le picker ne lisait
`/api/ai/status` qu'une fois, à sa construction, et le panneau de l'assistant
est un singleton monté pour toute la session : la liste restait figée. Le
nouveau `refreshAIPickers()` (`frontend/js/ai.js`) reconstruit chaque picker
monté dans son emplacement `.ai-picker-slot` et est appelé après
l'enregistrement (`saveAIKeys`) et la suppression (`deleteAIKey`) d'une clé
(`frontend/js/config.js`). Un emplacement est conservé même sans fournisseur
configuré, donc le **premier** fournisseur ajouté s'y monte aussi. Une
sélection dont le fournisseur n'est plus configuré est purgée de
`obsigate_ai_picker` (retour au défaut au lieu d'un modèle fantôme).
- **BUG-044 — Capacités des modèles erronées (aucun modèle Mistral vision)** : la
bulle ⓘ et le sélecteur de modèle par défaut n'affichaient **aucun** modèle Mistral
« Vision capable », alors que `GET https://api.mistral.ai/v1/models` en déclare 28
(`mistral-medium`, `mistral-small`, `ministral-*`, `magistral-*`, `mistral-ocr-*`,
`mistral-vibe-cli-*`). Cause : la table de capacités était **entièrement statique** et
aucun de ses motifs ne correspondait aux familles Mistral actuelles (seul `pixtral`,
retiré de l'API, les matchait). ObsiGate lit désormais les capacités **déclarées par le
fournisseur** (nouveau `backend/provider_capabilities.py`, snapshot mis en cache par
`GET /api/config/ai-models`) : Mistral (`capabilities.completion_chat` / `vision` /
`audio_transcription` / `audio_speech`) et OpenRouter (`architecture.input_modalities` /
`output_modalities`) sont pris en charge ; la table statique ne sert plus qu'à combler
les drapeaux non déclarés et de repli hors ligne. La table est également corrigée
(familles vision Mistral, `mistral-ocr` = vision sans chat) et le défaut du fournisseur
Mistral ne prétend plus qu'un modèle non reconnu sait produire des embeddings
(BUG-044bis : `mistral-large-latest` / `codestral-latest` étaient annoncés
« Embeddings »). Conséquence : la porte vision de l'assistant
(`frontend/js/bookslm.js`, `backend/bookslm_routes.py`) accepte enfin les images avec un
modèle Mistral vision.
### Sécurité
- **#84 Consolidation & sécurité — phase 1 (BUG-021 → BUG-034)** — traitement des
vulnérabilités de la revue statique du 2026-09-13 :
- **BUG-021/022 — XSS stocké** : nouveau sanitizer serveur en liste blanche
(`backend/services/sanitizer.py`, stdlib) appliqué au rendu markdown ; échappement
systématique du `title`, du frontmatter et du JSON de la page publique `/s/{token}`
(`</script>` neutralisé). Tests : `tests/test_security_hardening.py` (9).
- **BUG-023 — brute-force MFA** : rate-limit IP + compte et verrouillage de compte sur
`mfa/totp/verify`, `mfa/recovery` et `mfa/webauthn/verify` (`_enforce_mfa_rate_limit`).
- **BUG-024 — traversal inter-vaults** : `resolve_safe_path` compare désormais les chemins
par **segment** (`Path.relative_to` + repli casse-insensible), plus par préfixe de chaîne :
`vault` ne peut plus lire `vault-evil`.
- **BUG-025 — ReDoS** : validation des regex utilisateur (longueur max, rejet des
quantificateurs imbriqués/backrefs), contenu tronqué et nombre de matchs plafonné
(`backend/services/regex_safety.py`), appliqué à la recherche avancée et au find/replace.
- **BUG-026 — SSRF webhooks** : validation d'URL (HTTPS par défaut, IP privées/boucle
interdites, résolution DNS vérifiée au dispatch, redirections non suivies) et
**externalisation du secret** dans `data/webhook_secrets.json` (0600) ou variable
`OBSIGATE_WEBHOOK_SECRET_<ID>` (plus de secret en clair dans `webhooks.json`).
- **BUG-027 — sessions** : rotation du refresh token à chaque usage, révocation du JTI de
l'access token au logout, et vérification de la révocation dans le middleware.
- **BUG-028 — politique de mot de passe** : validation centralisée (8–128 caractères) à la
création, à la modification admin et au changement ; `password_changed_at` invalide tous
les jetons émis avant un changement de mot de passe.
- **BUG-029 — race `users.json`** : verrou `threading.RLock` autour des cycles
lecture-modification-écriture.
- **BUG-030 — audits** : l'adresse IP réelle du client (`X-Forwarded-For` si
`OBSIGATE_TRUST_PROXY=true`) est injectée dans `current_user` et consignée dans les audits.
- **BUG-031 — rate-limiter** : budget **par compte** en plus du budget par IP (rotation d'IP
neutralisée) ; limite mono-process documentée.
- **BUG-032 — indexation** : `_scan_vault` utilise `os.walk(followlinks=False)` et refuse
tout symlink sortant de la racine du vault.
- **BUG-033 — recherche O(N)** : la recherche simple et l'outil IA `search_fulltext`
utilisent l'inverted index (repli sur le scan pendant la construction).
- **BUG-034 — CSP & jetons** : le token d'accès n'est plus persisté dans `sessionStorage`
(mémoire + cookie `HttpOnly`) ; directives CSP durcies (`object-src`, `base-uri`,
`form-action`, `frame-ancestors`). *Reste : migration CSP par nonce (exige la conversion
des gestionnaires d'événements inline).*
### Ajouté
- **#83 Barre d'outils d'édition mobile — ruban style Obsidian Android** — la barre de mise en
forme flottante est remplacée par un **ruban horizontal défilable** ancré juste au-dessus du
clavier virtuel : fond anthracite aux coins arrondis, insertion/enrobage au curseur ou sur la
sélection. Commandes : annuler/refaire, `[[ ]]` (lien interne), fichiers/modèles, tag `#`,
pièce jointe, H1–H6, gras, italique, barré (`~~`), surligné (`==`), code en ligne/bloc, citation,
lien externe, listes à puces/numérotées, case à cocher, indenter/désindenter, coller, A−/A+.
Une icône **⚙** ouvre un panneau de personnalisation (ajouter / retirer / réordonner, persistance
`localStorage`). i18n FR/EN. Helpers purs exportés (`formatChange`, `normalizeRibbonCommands`,
`moveRibbonCommand`). Tests JSDOM `tests/frontend/mobile-editor.test.mjs` (+13) et E2E mobile.
- **#82 Assistant IA — menu `@` instantané** — nouvel endpoint `GET /api/vault/{vault}/paths`
(liste plate et plafonnée de l'index des chemins). Le frontend la récupère **une fois par vault**
(préchargée à l'ouverture du panneau) puis filtre **côté client** pendant la frappe : l'affichage
des fichiers/répertoires est immédiat, au lieu d'une requête réseau par caractère. Tests :
`tests/test_api_main.py` (+2), `tests/frontend/ai.test.mjs` (+1).
- **#82 Assistant IA — Section « Fournisseur & modèle » compacte** — la barre de sélection du
modèle devient **discrète** : la liste complète des types d'endpoints (Chat, Embeddings, Vision…)
est remplacée par un **bouton d'information ⓘ** qui affiche les capacités dans une **bulle** au
survol, au clic ou par **appui long** (mobile). Le choix du modèle passe par une **liste
déroulante avec recherche** (filtrage insensible aux accents) au lieu d'un `<select>` natif, tout
en conservant un select masqué pour les commandes admin (`/model`). Le picker partagé
(`frontend/js/ai.js`) est utilisé par l'assistant et la barre d'édition IA. Tests JSDOM
(`tests/frontend/ai.test.mjs` +1).
- **#81 Assistant IA — Commandes `@`/`/`, skills, images & capacités des modèles** — l'assistant
devient plus flexible et multimodal. **Panneau redimensionnable** à la souris (largeur 320–1000 px
persistée). **Commande `@`** pour attacher des contextes ad-hoc (fichiers, répertoires) au contexte
courant. **Commande `/`** pour lancer des **skills** réutilisables (`/research`, `/resume`,
`/actions`, `/reformuler`, `/correction`, `/brainstorm`, `/plan`, `/ask`, `/meeting-note`,
`/livrable`) ou des commandes admin (`/help`, `/providers`, `/provider`, `/model`, `/keys`) ;
`/create-new-skill` permet de créer des skills personnalisés persistés par utilisateur
(`data/skills.json`). **Analyse d'images** : coller une image dans la zone de saisie ou référencer
une image d'un répertoire (`@image.png`), avec garde-fou de compatibilité vision. **Capacités des
modèles** (Chat, Embeddings, Rerank, Images, Video, Audio Speech, Audio Transcriptions, Vision)
affichées à la sélection dans l'assistant et le panneau de configuration. Backend : table curée
`backend/model_capabilities.py`, endpoints `GET /api/ai/model-capabilities` et `GET /api/ai/skills`,
support multimodal dans `backend/ai_chat.py` et `backend/bookslm_routes.py`. Tests :
`tests/test_model_capabilities.py` (16), `tests/test_skills.py` (16), `tests/test_ai_vision.py` (12)
+ extension `tests/test_ai_models.py` et `tests/frontend/ai.test.mjs` (+9). Détail :
[docs/features/ai-assistant-commands.md](./docs/features/ai-assistant-commands.md).
- **#77 Desktop — Signature des mises à jour Tauri** — la paire de clés `minisign`
de l'updater est générée et sa clé publique est embarquée dans
`desktop/tauri.conf.json` (`plugins.updater.pubkey`, remplace le placeholder) ;
`bundle.createUpdaterArtifacts: true` produit les fichiers `.sig` par artefact.
Le workflow `.gitea/workflows/desktop-build.yml` expose les secrets
`TAURI_SIGNING_PRIVATE_KEY` / `TAURI_SIGNING_PRIVATE_KEY_PASSWORD` aux builds
Windows et Linux, avec repli automatique en build **non signé** si le secret est
absent (CI toujours verte). Procédure complète :
[docs/DEVELOPMENT_AND_RELEASES.md](./docs/DEVELOPMENT_AND_RELEASES.md#2bis-signature-des-mises-à-jour-updater-tauri).
- **#77 Desktop — Manifeste de mise à jour `latest.json`** — nouveau
`scripts/updater_manifest.py` qui construit le document consommé par l'updater
Tauri (version, notes, `pub_date`, `platforms` Windows/Linux avec signature
`.sig` et URLs des assets). Il est généré automatiquement par
`scripts/publish_release.py` (écrit `desktop/latest.json`, l'ajoute aux assets
de la release, upload des `.sig`) et peut être lancé seul. L'endpoint de
l'updater (`desktop/tauri.conf.json`) pointe désormais sur le manifeste
versionné `.../raw/branch/main/desktop/latest.json`. Les builds locaux
(`build-windows.bat`, `build-linux.sh`) détectent automatiquement
`desktop/obsigate-updater.key` pour signer, ou désactivent les artefacts de
mise à jour s'il est absent. Tests : `tests/test_updater_manifest.py` (8).
- **#77 Desktop — Protocole de tests E2E manuels** — nouveau
[`docs/DESKTOP_E2E_CHECKLIST.md`](./docs/DESKTOP_E2E_CHECKLIST.md) : prérequis,
étapes et résultat attendu pour les 6 scénarios OS non automatisables
(installation, tray, notifications natives, association `.md`, auto-update,
désinstallation), avec emplacements des logs/config et procédure de signature
de l'updater Tauri.
- **#70 Recherche sémantique — Embeddings vectoriels** — la recherche comprend désormais le
**sens** de la requête en plus des mots-clés. **Embeddings** : chaque document est découpé en
chunks de 512 mots (recouvrement 64) puis vectorisé (384 dim) via `all-MiniLM-L6-v2`
(`sentence-transformers`), un endpoint `/embeddings` compatible OpenAI, ou un provider de repli
**sans dépendance** (hachage déterministe). **Stockage vectoriel** : `numpy`/`faiss`
(`IndexFlatIP`) si disponibles, sinon cosinus pur Python. **Recherche hybride** : fusion du
classement TF-IDF et du classement sémantique par **RRF** (Reciprocal Rank Fusion), avec
`semantic_score` par résultat. **Indexation incrémentale** branchée sur le watcher (un fichier
modifié régénère son embedding). **UI** : toggle « Recherche sémantique » (`~`, raccourci
`Alt+S`) dans la barre de résultats + affichage du score de similarité, clés i18n FR/EN.
Nouveau module `backend/semantic_search.py`, paramètre `semantic` sur
`/api/search/advanced`, dépendances **optionnelles** dans
`backend/requirements-semantic.txt`. Tests : `tests/test_semantic_search.py` (27) +
`tests/frontend/semantic-search.test.mjs` (4). Détail :
[docs/features/semantic-search.md](./docs/features/semantic-search.md).
- **#69 Éditeur mobile natif — Interface tactile optimisée** — refonte de l'expérience
d'édition sur téléphone/tablette. **Barre d'outils flottante** dans l'éditeur (gras, italique,
code, liste à puces, lien) opérant directement sur la sélection CodeMirror (ou le textarea de
secours), avec **toggle** (re-appui pour dé-formater) ; **bouton « Coller » persistant** qui
contourne la restriction presse-papiers d'iOS (repli sur un message d'aide si l'accès est
refusé). **Adaptation CodeMirror** : zoom par **pincement à deux doigts**, boutons A−/A+ et
**hauteur ajustable** via une poignée (valeurs persistées en `localStorage`). Côté lecture :
**raccourcis swipe** (gauche → liens entrants, droite → table des matières) et **mode lecture**
plein écran (masquage de l'en-tête, des barres latérales et de la barre mobile, API Fullscreen,
**swipe horizontal** pour passer au fichier suivant/précédent du même dossier). Nouveau module
`frontend/js/mobile-editor.js`, styles `frontend/style.css`, clés i18n FR/EN. Tests :
`tests/frontend/mobile-editor.test.mjs` (22 tests). Détail :
[docs/features/mobile-editor.md](./docs/features/mobile-editor.md).
- **#62 Collaboration temps réel — Édition simultanée** — plusieurs utilisateurs peuvent éditer le
même document markdown en même temps, façon Google Docs. **WebSocket** : nouvel endpoint
`/ws/collab/{vault}/{path}` (une *room* par fichier) qui relaie les changements instantanément ;
authentification manuelle (cookie `access_token` ou paramètre `token`) + `check_vault_access` et
`resolve_safe_path` par connexion. **Yjs/CRDT** côté client : `Y.Doc`/`Y.Text` fusionnent les
modifications concurrentes sans conflit ni perte. **Awareness** : curseurs distants colorés et
sélections dans CodeMirror, indicateur de présence (avatars + statut) dans l'en-tête de l'éditeur.
**Persistance serveur** : le dernier texte reçu est écrit sur disque avec un debounce de 2 s
(`backend/collab.py`). **Reconnexion** automatique avec backoff exponentiel et merge de l'état au
retour. Nouveau module frontend `frontend/js/collab.js` (provider + liaison Y.Text↔CodeMirror +
curseurs distants) et `backend/collab.py` (`CollabManager`). Tests : `tests/test_collab.py`
(17 tests, dont 5 clients simultanés) et `tests/frontend/collab.test.mjs`. Détail :
[docs/features/collaboration.md](./docs/features/collaboration.md).
- **#79 Phase F — Durcissement & documentation (rate limiting, redaction, OpenAPI/MCP, E2E)** —
clôture de la feature #79. **Rate limiting** par jeton et par outil
(`backend/tools/ratelimit.py`, fenêtre glissante ; identité = JTI du jeton sinon id/username ;
`OBSIGATE_TOOL_RATE_LIMIT`, `OBSIGATE_TOOL_RATE_LIMIT_PER_TOOL`, `OBSIGATE_TOOL_RATE_WINDOW` ;
erreur `rate_limited`). **Quotas** `BOOKSLM_MAX_*` : `BOOKSLM_MAX_TOOL_CALLS` plafonne les appels
d'outils par run d'agent (`stopped="quota_exceeded"`) et `BOOKSLM_MAX_TOOL_READ_BYTES` plafonne
`read_file`. **Redaction** systématique de **tout** résultat d'outil avant retour au LLM
(`backend/tools/redaction.py`, récursif : diffs, extraits de recherche, lectures). **OpenAPI** :
tag `MCP` + injection du path `/mcp` (Streamable HTTP) dans le schéma ; nouveau
[`docs/MCP_GUIDE.md`](./docs/MCP_GUIDE.md) (config Claude Desktop/Cursor, tools/resources/prompts,
sécurité, variables, dépannage). Tests E2E : `tests/test_ai_e2e.py` (agent read→confirm→write,
quota, rate limit, redaction, flux MCP `read`→`propose`→`apply`→`resources/read`).
- **#79 Phase E — Serveur MCP (Streamable HTTP)** — nouvel endpoint `/mcp` exposant ObsiGate à des
clients MCP externes (Claude Desktop, Cursor…) via le SDK Python `mcp==1.9.4`. Les deux fronts
(assistant in-app et MCP) consomment la même couche d'outils. **Tools** : outils de lecture/recherche
directs ; outils d'écriture/destructifs en **two-step `propose_<tool>` / `apply_<tool>`** avec
**jeton JWT signé, usage unique et TTL** (`OBSIGATE_MCP_CONFIRMATION_TTL`, défaut 300 s) et
blacklist de JTI persistée (`data/mcp_used_tokens.json`, anti-rejeu). **Resources** :
`vault://<name>` et `vault://<name>/<path>` (lecture seule, secrets redactés). **Prompts** :
`summarize-directory`, `generate-note`, `find-related`. Auth `Authorization: Bearer <JWT>` →
`get_current_user` ; permissions et toggle destructif par vault (`aiDestructiveTools`) appliqués.
Nouveaux modules `backend/mcp/server.py` et `backend/mcp/confirmations.py` ; dépendances
`mcp==1.9.4` + `sse-starlette==2.1.3` (compatibles FastAPI 0.110 / starlette 0.37). Tests :
`tests/test_mcp.py` (13 tests).
- **#79 Phase D — Catalogue d'outils mutations + confirmations two-step** — 11 nouveaux outils IA
destructifs ou d'écriture exposés par la couche partagée (`backend/tools/service.py`) :
`create_file`, `create_directory`, `edit_file`, `append_to_file`, `rename_file`,
`rename_directory`, `move_path`, `replace_in_files`, `delete_file`, `delete_directory`,
`restore_backup`. Nouveau service réutilisable `backend/services/mutations.py` (anti
path-traversal, garde lecture seule, **backup automatique** avant écriture/suppression/restauration) ;
les routes `/api/file/{vault}` (POST/PATCH/DELETE), `/api/file/{vault}/save`, `/api/directory/{vault}`
(POST/PATCH/DELETE), `/api/move/{vault}`, `/api/file/{vault}/restore` et `/api/search/replace`
délèguent désormais à cette couche. `backend/services/backups.py` expose `create_backup`
(source unique, réutilisée par `main._backup_file`). Confirmations **two-step** : tout outil
`WRITE`/`DANGEROUS` déclenche `ToolConfirmationRequired` (carte Apply in-app, `propose`/`apply`
côté MCP en phase E). **Toggle par vault** `aiDestructiveTools` (défaut : activés) pour désactiver
delete/rename/move/replace. Tests : `tests/test_tools_mutations.py` (40 tests).
- **#79 Phase C — Catalogue d'outils lecture & recherche** — 10 nouveaux outils IA exposés par la
couche partagée (`backend/tools/service.py`) : `list_all_files`, `read_file_raw`, `get_backlinks`,
`list_backups`, `diff_backup`, `get_graph`, `search_advanced`, `search_paths`, `suggest_tags`,
`list_recent`. Nouveaux services réutilisables `backend/services/backups.py`, `graph.py`,
`recent.py` et extensions de `vaults.py` (`list_all_files`) / `search.py`
(`advanced_search_vaults`, `search_paths`). Les routes `/api/recent`, `/api/vault/{vault}/files`,
`/api/file/{vault}/backups`, `/api/file/{vault}/diff`, `/api/graph/{vault}`, `/api/tree-search`
et `/api/search/advanced` délèguent désormais à ces services (source unique de vérité).
Filtrage systématique par permissions vault ; secrets redactés sur `read_file`/`read_file_raw`.
Tests : `tests/test_tools.py` (+17 tests).
- **#79 Assistant IA — Outils (function calling) & serveur MCP — Phase 0 (couche d'outils partagée)** —
nouveau paquet `backend/tools/` : `context.py` (`ToolContext`), `registry.py` (décorateur `@tool`
+ schéma JSON), `schemas.py`, `service.py`, `audit.py` (journalisation JSONL des appels), façade
`backend/tools/api.py`. Services lecture/recherche livrés (`list_vaults`, `list_directory`,
`read_file`, `search_fulltext`, `list_tags`) ; permissions `check_vault_access` + `resolve_safe_path`
appliquées par outil. Tests : `tests/test_tools.py` (30 tests).
- **#79 Phase B (partielle) — function calling in-app** — abstraction tool-calling
provider-agnostique `backend/ai_chat.py` (`chat_completion`, `ToolCall`, `LLMResponse`, OpenAI-compat
+ Gemini) ; agent loop `backend/agent/loop.py` (boucle tool→résultat→tool, limite 10 itérations,
truncation) ; endpoint opt-in `POST /api/ai/bookslm/agent` (events SSE `tool`/`message`/`confirmation`) ;
fallback automatique en chat simple si le provider rejette les tools. Tests : `tests/test_agent_loop.py`,
`tests/test_ai_chat.py`, `tests/test_bookslm.py`.
- **#79 A2/B4/B5 — services partagés, SSE streaming & confirmations UI** — extraction de la logique
lecture/recherche dans `backend/services/` (`errors.py`, `paths.py`, `vaults.py`, `files.py`,
`search.py`) : routes REST et outils IA consomment la même source ; `ServiceError` mappée vers
`HTTPException` (routes) et `ToolError` (outils). `/api/ai/bookslm/chat` **streame réellement** les
tokens (`stream_completion`, OpenAI-compat + Gemini ; GZip ignoré pour les SSE BooksLM). Mode agent
côté UI (toggle, événements `tool`/`confirmation`, carte Apply + aperçu diff LCS) et reprise
`confirm`/`confirm_messages` de l'agent loop (confirmation one-shot). Tests : `tests/test_services.py`,
`tests/test_ai_chat.py`, `tests/test_agent_loop.py`, `tests/test_bookslm.py`.
- **#79 G — Sélection fournisseur/modèle par défaut** — `ai_default_provider` + `ai_default_models`
persistés dans `data/config.json`, rechargement à chaud dans `backend/ai.py`, sélecteurs
« Fournisseur par défaut » / « Modèle par défaut » dans `#cfg-ai`, i18n FR/EN.
- **#80 Assistant IA — Rendu Markdown, liens fichiers/paths & sessions** — réponses de l'assistant
rendues en **Markdown formaté** (titres, listes, tableaux, citations, code, emphase) via un
renderer auto-contenu ; les **fichiers et chemins** mentionnés deviennent des liens cliquables
(clic fichier → ouverture dans le viewer, clic répertoire → révélation/surlignage dans
l'arborescence, outils `open_file`/`reveal_in_tree` côté in-app) ; **gestion des sessions** par
contexte (historique consultable, rechargeable, supprimable) avec migration de l'ancienne
conversation unique. Tests : `tests/frontend/ai.test.mjs`.
### Modifié
- **Barre inférieure mobile — couleurs harmonisées** — les boutons « settings.search », « Recherche »
et « Onglets » adoptent la couleur d'accent (`--accent`) de l'icône et du libellé du bouton
« Commandes » (`frontend/style.css`), pour un rendu uniforme de la barre du bas.
- **#82 Assistant IA — sélecteurs Fournisseur/Modèle alignés à droite et réordonnés** — dans la
barre de l'assistant, le groupe de sélection est aligné sur le bord droit et l'ordre devient
**capacité du modèle (ⓘ) → fournisseur → modèle**. La bulle de capacités s'ouvre désormais vers
la droite (`left: 0`) pour ne pas déborder de la barre latérale. Tests JSDOM
(`tests/frontend/ai.test.mjs`).
- **`scripts/bump_version.sh` — synchronisation de la version desktop** — le script met
désormais à jour `desktop/tauri.conf.json`, `desktop/Cargo.toml` et `desktop/Cargo.lock`
(`obsigate-desktop`) avant de créer un commit de release `chore(release): vX.Y.Z` puis le
tag. `--push` pousse la branche **et** le tag ; `--no-files` conserve l'ancien comportement
(tag seul) et `--dry-run` prévisualise. Indispensable au bon fonctionnement de l'updater
Tauri (dont le manifeste reprend la version de `tauri.conf.json`).
- **`.gitea/workflows/desktop-build.yml` — déclenchement manuel uniquement** — le workflow
`Desktop Build` ne s'exécute plus sur `push` (aucun runner self-hosted `[windows/linux, desktop]`
enregistré) mais uniquement via `workflow_dispatch`. Le build Windows se fait en local
(`desktop/build-windows.bat`) ; le workflow reste disponible pour un futur runner.
- **Assistant IA (`frontend/js/bookslm.js`)** — l'ancien rendu Markdown minimal est remplacé par un
renderer bloc/inline complet ; l'en-tête expose un bouton « Historique des sessions » ; les badges
de sources ouvrent désormais réellement le fichier (événement `obsigate:open-file`).
- **Assistant IA (`frontend/js/bookslm.js`)** — bouton « mode agent » dans l'en-tête (persisté en
`localStorage`) : l'envoi bascule vers `/api/ai/bookslm/agent`, affiche la trace des appels d'outils
et les cartes de confirmation (Apply + aperçu diff) ; `/chat` reste le défaut. i18n FR/EN.
### Modifié
- **#82 Assistant IA — barre latérale épurée & suivi visuel des requêtes** — la section
« Fournisseur & modèle » perd ses intitulés (titre de section et libellé « Fournisseur : »)
pour rester discrète ; la description du contexte Général n'est plus affichée (bandeau de
statut masqué, description accessible au **survol** de l'en-tête) ; le placeholder
« Posez une question sur ces documents… » est retiré ; l'indice clavier
« Entrée pour envoyer · Ctrl+Entrée… » est retiré ; le bouton « Envoyer » devient un **emoji
compact** (✈️). Les sélecteurs Fournisseur/Modèle adoptent la **taille des autres contrôles du
site** (police 0,8 rem, hauteur 34 px, rayon 6 px) au lieu d'être plus fins. Un **indicateur
d'activité** apparaît pendant le traitement et détaille le workflow : envoi, réception de la
réponse, appels d'outils (`Outil : X…`), attente de confirmation, succès ou échec. Le menu
contextuel de la **racine d'une vault** expose aussi « 🧠 BooksLM », comme les répertoires.
i18n FR/EN (`ai.activity_*`, `ai.composer_label`). Tests JSDOM
(`tests/frontend/ai.test.mjs` +2).
### Corrigé
- **BUG-017 — Accueil mobile : tuiles « Favoris » / « Récents » mal dimensionnées** — les cartes
de tableau de bord débordaient horizontalement à cause d'un titre/chemin `nowrap` : ajout de
`min-width: 0` et `overflow: hidden` sur `.dashboard-card` (et ses en-têtes/pieds) pour que la
grille mobile (1 colonne) respecte la largeur du viewport. Les tuiles « Partagés » étaient déjà
conformes.
- **BUG-018 — Éditeur mobile : boutons « Annuler » / « Sauvegarder » inaccessibles** — l'en-tête
cumulait marque, titre, nom de fichier et actions, poussant les boutons hors écran. Sur mobile,
les éléments secondaires (marque, séparateur, espaceur, présence) sont masqués, les champs
titre/nom de fichier deviennent compressibles (`flex`, `min-width: 0`) et le libellé « Saved »
est masqué (le point d'état reste). La barre d'outils IA reçoit des cibles tactiles plus grandes.
- **BUG-019 — Éditeur mobile : barre d'outils masquée par le clavier** — le ruban est désormais
ancré via `window.visualViewport` (`--kb-offset`) pour rester juste au-dessus du clavier, quel
que soit le défilement, et le scroller CodeMirror reçoit un `padding-bottom` pour ne pas cacher
le curseur.
- **BUG-020 — Éditeur mobile : double barre de défilement** — `.editor-body` passe en
`overflow: hidden` + colonne flex ; seul le scroller CodeMirror défile.
- **BUG-016 — Mobile : le bouton « mode lecture » recouvre le bouton d'envoi de l'assistant** —
le bouton flottant `📖` (`#me-reading-btn`, `z-index: 890`) passait au-dessus du panneau
assistant plein écran sur mobile (`z-index: 100`) et masquait le bouton d'envoi `✈️`. Il
n'apparaît désormais que lorsqu'un **fichier est ouvert** (`state.currentPath`) **et** que le
panneau assistant est fermé ; sa visibilité est resynchronisée à chaque mutation de
`#content-area` et sur les événements `bookslm:opened`/`bookslm:closed`. Tests JSDOM
(`tests/frontend/mobile-editor.test.mjs` +2) et E2E mis à jour.
- **BUG-015 — Assistant IA : sélection des menus `/` et `@` invisible au clavier** — l'élément
actif utilisait `background: var(--surface2)`, or `--surface2` est **identique** à
`--bg-primary` (fond du menu) dans le thème sombre par défaut : la ligne sélectionnée par les
flèches ↑/↓ ne se distinguait pas. L'état actif (et le survol) utilise désormais
`--bg-hover` avec une **barre d'accent** à gauche (`box-shadow: inset 3px 0 0 var(--accent)`),
pour les menus de commandes/mentions **et** la liste de modèles. Test JSDOM mis à jour.
- **BUG-014 — Assistant IA : navigation clavier ↑/↓ des menus inopérante** — la navigation était
liée au `keydown` du champ de saisie : dès que le focus quittait la zone de texte (clic sur le
menu, bouton Envoyer…), les flèches ne faisaient plus défiler les items. La gestion est déplacée
au niveau du **panneau en phase de capture**, ce qui fonctionne quel que soit l'élément focalisé
dans la barre latérale. Tests JSDOM (`tests/frontend/ai.test.mjs` +1).
- **BUG-013 — Assistant IA : « Aucun vault actif pour ouvrir ce lien »** — les liens
fichiers/répertoires des réponses utilisaient `_resolveVault()` seul, nul en mode Général sans
document ouvert. `_openFileLink()` et `_revealPath()` utilisent désormais `_activeVault()` qui
retombe sur le vault de contexte, le vault sélectionné dans la barre latérale, puis le premier
vault disponible. Tests JSDOM (`tests/frontend/ai.test.mjs` +1).
- **Assistant IA — caches obsolètes** — `SW_VERSION` porté à **v6** et migration de purge élargie à
tous les caches `obsigate-*` (clé `obsigate-sw-migration` → `v3`), afin que les clients bloqués sur
une ancienne version récupèrent bien le menu `@` et les correctifs.
- **BUG-012 — Assistant IA : la commande `@` n'affichait aucun menu en mode Général** — sans vault
résolu, la détection `@` ne listait rien (menu masqué) et le backend ignorait ensuite le
contexte. Le menu est désormais **toujours rendu** et `_mentionVault()` retombe sur le vault de
contexte de la barre latérale, puis sur le premier vault disponible, avant de basculer sur
`vault=all` pour les recherches. Tests JSDOM (`tests/frontend/ai.test.mjs` +1).
- **BUG-011 — Assistant IA : noms de modèles illisibles dans la liste déroulante** — la liste
déroulante des modèles s'ouvrait alignée à gauche (`left: 0`) et pouvait dépasser le bord droit
de la barre latérale (ancrée à droite), rendant les noms tronqués/illisibles ; elle est
désormais **alignée à droite** du déclencheur, large de 340 px (bornée à `100vw - 24px`), et les
noms **reviennent à la ligne** (`overflow-wrap: anywhere`, police 0,78 rem) avec l'info-bulle
`title` complète. Tests JSDOM (`tests/frontend/ai.test.mjs`).
- **BUG-010 — Assistant IA : la commande `@` n'ajoutait pas le contexte en mode Général** — les
fichiers/répertoires choisis via `@` étaient bien ajoutés sous forme de puces, mais le **vault
n'était pas transmis** : en mode Général (`_vault` nul), la requête partait sans `vault` et le
backend ignorait le contexte ad-hoc. La sélection capture maintenant le `vault` renvoyé par
`/api/tree-search`, `_contextVault()` le propage aux requêtes `/context` et `/chat`, et les
recherches suivantes restent dans ce vault. Tests JSDOM (`tests/frontend/ai.test.mjs`).
- **BUG-009 — Assistant IA : liste de modèles corrompue et clés i18n brutes** — deux causes :
(1) les fichiers de locale n'étant pas *content-hashed*, un `fr.json` en cache HTTP affichait
les clés brutes (`ai.model_search`, `mobile_editor.*`, …) ; le chargement i18n utilise désormais
`cache: 'no-store'` et `SW_VERSION` est incrémenté pour purger les anciens caches du service
worker. (2) la liste de modèles (OpenRouter = plusieurs centaines d'entrées) pouvait se retrouver
non stylée si la feuille de style était en cache ; le picker applique maintenant les styles
critiques **en ligne** (popover, liste en colonne, options `display:block`), plafonne le rendu à
200 entrées avec un indicateur « … N autres — affinez la recherche », et la recherche filtre le
reste. Tests JSDOM (`tests/frontend/ai.test.mjs` +1).
- **BUG-007 — Assistant IA : menus `/` et `@` (navigation ↑/↓ et filtrage accentué)** — la
détection des commandes/mentions et le filtrage des skills utilisaient des motifs ASCII
(`[a-z0-9-]`, `\w`) : dès qu'un caractère accentué était saisi (`/résumé`, `@café`), le menu se
fermait. Les motifs sont désormais **Unicode** (`\p{L}`) et la recherche est **insensible aux
accents** (normalisation NFD). La navigation clavier est fiabilisée par un **jeton de séquence**
qui ignore les rendus asynchrones obsolètes (frappe rapide) et fait défiler l'élément actif dans
la vue (`scrollIntoView`). Tests JSDOM (`tests/frontend/ai.test.mjs` +3).
- **BUG-008 — Assistant IA : commande `@` et contexte ad-hoc en mode Général** — deux causes :
(1) le backend ignorait les fichiers/répertoires ad-hoc en mode Général car `_resolve_system_prompt`
ne résolvait pas de vault (`backend/bookslm_routes.py`) ; le vault optionnel est maintenant résolu
dès que du contexte `@` est présent. (2) sans vault courant, la recherche de mention échouait ;
elle bascule désormais sur `vault=all`. Tests backend (`tests/test_bookslm.py` +2) et frontend.
- **#77 Desktop — bannière de premier lancement réaffichée à chaque démarrage** —
l'état du wizard « Choisissez votre vault » n'était mémorisé que dans le
`localStorage` de la webview et le paramètre `hasVaultPath` n'était jamais
renseigné. Ajout d'un booléen persistant `wizard_done` dans la config desktop
(`desktop/src/main.rs`, commandes `get_wizard_state` / `complete_wizard`,
rétro-compatible via `#[serde(default)]`) : la bannière ne réapparaît plus après
un choix de dossier ou un clic « Plus tard », même si le stockage webview est
vidé. Tests Rust (`desktop/src/main.rs`) et JSDOM
(`tests/frontend/desktop.test.mjs`) ajoutés.
- **Build Docker — « No space left » lors de `apt-get install`** — le stage *builder*
installait inutilement `libpango-1.0-0`, `libpangocairo-1.0-0` et `shared-mime-info`
(libs runtime de WeasyPrint, sans usage à la compilation), dupliquant les téléchargements
apt. Ces paquets sont retirés du builder et `apt-get clean` est ajouté dans les deux
stages pour purger les `.deb` conservés par défaut sous Debian trixie, réduisant
l'empreinte disque et le temps de build. En complément, purge du cache de build
(`docker builder prune`) ayant libéré l'espace nécessaire.
- **BUG-005 — Chargement mobile incomplet derrière Cloudflare (`og.dracodev.net`)** —
le service worker utilisait une stratégie **cache-first avec un nom de cache fixe**
(`obsigate-v1`) et un précache de chemins erronés (`/frontend/js/…`) : le Cache Storage
du SW (distinct du cache navigateur/Cloudflare) servait un ancien build indéfiniment,
même après purge. Le middleware renvoyait en plus `Cache-Control: public, max-age=31536000,
immutable` sur des assets **non fingerprintés**. Correctifs : `frontend/sw.js` réécrit
(**network-first** pour HTML/JS/CSS, caches versionnés `SW_VERSION`, précache corrigé
`/static/js/…`, `skipWaiting`/`clients.claim`) ; migration ponctuelle `localStorage`
(suppression des anciens caches) en remplacement du kill-switch de session ; en-têtes
`no-cache` sur `/static`, `index.html`, `manifest.json` et pages HTML ; rechargement unique
sur `controllerchange`. Tests : `tests/frontend/sw.test.mjs` + `TestStaticCaching`.
Guide Cloudflare : [docs/PWA_GUIDE.md](./docs/PWA_GUIDE.md).
- **Typage backend (mypy) — 33 erreurs corrigées** (`backend/main.py`, `indexer.py`,
`auth/router.py`, `pdf_reader.py`, `export.py`, `bookslm_routes.py`) : annotations de types,
gardes `None` sur `get_user()`, `PdfReader: Any` et import `PROVIDERS` manquant dans `main.py`
(bug latent : le modèle par défaut n'était jamais prépendé à la liste des modèles live).
L'étape `mypy` du CI devient **bloquante** (elle était en mode advisory).
- **README** — lien « Contributing » corrigé vers `docs/CONTRIBUTING.md` (était cassé vers la
racine) ; arbre du projet mis à jour.
- **#79 façade `backend/tools/api.py`** — gestion des namespace packages et du `__init__` ignoré
lors du chargement des modules d'outils.
### Documentation
- **Guide d'architecture IA** — `docs/AI_ARCHITECTURE_GUIDE.md` (architecture, catalogue d'outils,
sécurité, phases).
- **Refonte de la documentation** — la [Roadmap](./docs/ROADMAP.md) ne contient plus que le travail
à venir + un index compact ; les fonctionnalités livrées sont archivées dans
`docs/archive/COMPLETED_v1-v2.md` et documentées par feature dans `docs/features/`.
- **Méthode de livraison unifiée** — nouveau [`docs/DELIVERY_WORKFLOW.md`](./docs/DELIVERY_WORKFLOW.md)
(Definition of Done : tests, documentation, commit, push, CI) et [`AGENTS.md`](./AGENTS.md)
(instructions obligatoires lues à chaque session). Référencés depuis la Roadmap, CONTRIBUTING
et ISSUES_TODOLIST.
---
## [2.2.1] — 2026-09-11
### Ajouté
- **Mobile — assistant AI central** — bouton assistant dans la barre du bas (commit 50b823e).
### Corrigé
- **Version affichée dans l'UI** — affichait `2.1.0` au lieu de `2.2.0` (commit 84bda90).
- **Mobile** — boîte d'édition de l'assistant relevée au-dessus de la barre du bas (commit f307ecb).
---
## [2.2.0] — 2026-09-11
### Ajouté
- **Assistant AI contextuel (sidebar)** — l'assistant s'adapte désormais au contexte d'ouverture :
- **Répertoire** (`directory`) : ouvert depuis le menu contextuel d'un dossier — le contexte est
le dossier et son contenu (mode historique BooksLM).
- **Documents** (`documents`) : ouvert via le bouton flottant quand un ou plusieurs documents
sont ouverts — le contexte est la liste des fichiers ouverts (tous onglets/panneaux).
- **Général** (`general`) : ouvert via le bouton flottant quand aucun document n'est ouvert —
l'assistant répond sur l'application et peut proposer des actions (créer un fichier/dossier)
via un bloc `obsigate-action` rendu en carte avec bouton « Appliquer » (aucune exécution
automatique).
- Le contexte est affiché dans le header de la sidebar (icône, titre, sous-titre) et les
suggestions de questions s'adaptent au mode.
- Backend : `/api/ai/bookslm/context` et `/chat` acceptent `mode` (`directory`/`documents`/
`general`) et `context_files` ; `collect_files_context()`, `empty_context()` et
`build_general_system_prompt()` dans `backend/bookslm.py` ; le mode documents dégrade
proprement vers le mode général si aucun fichier n'est lisible.
- Tests : `tests/test_bookslm.py` (+13 tests modes/documents/général) et
`tests/frontend/ai.test.mjs` (+8 tests détection de contexte, clavier, actions, header).
- **#72 API publique documentée — OpenAPI 3.1 enrichie** — documentation interactive complète
de l'API REST, générée automatiquement et enrichie.
- Backend `backend/openapi_docs.py` : métadonnées de tags (18 catégories : System, Auth, Files,
PDF, Vaults, Search, Bookmarks, Backups, Export, AI, BooksLM, Sharing, Webhooks, Conflicts,
Admin, Plugins, Push, Frontend), assignation automatique des tags par préfixe de route
(normalisation des tags des routeurs), schémas de sécurité (`bearerAuth`, `cookieAuth`),
exemples de requête/réponse sur les endpoints clés, réponses d'erreur documentées
(401/403/404/422/500), serveur et `externalDocs`.
- `backend/schemas.py` : nouveaux `response_model` Pydantic pour les endpoints qui n'en avaient
pas (recent, bookmarks, saved searches, backups, diff/restore/backlinks, pdf/info, replace,
vaults, attachments, settings, config, AI keys/models, diagnostics, dashboard, webhooks,
shares, conflicts, BooksLM context, AI status).
- `backend/main.py` : `FastAPI(...)` enrichi (description Markdown, contact, licence, tags) et
override `app.openapi` pour post-traiter le schéma ; page de documentation `/api` (HTML
autonome listant les endpoints groupés par catégorie, liens vers `/docs`, `/redoc`,
`/openapi.json`).
- Frontend : entrée « API » dans le menu d'options (i18n FR/EN) ouvrant Swagger UI.
- Tests : `tests/test_openapi.py` (58 tests — version 3.1, tags, sécurité, erreurs, exemples,
schémas de réponse, pages `/api`, `/docs`, `/redoc`, `/openapi.json`).
- **#61 Plugins système au complet** — système de plugins permettant d'étendre ObsiGate
(renderers personnalisés, filtres de recherche, actions d'éditeur), sandboxé pour la sécurité.
- Backend `backend/plugins.py` : validation du manifest `plugin.json` (name regex lowercase,
semver, hooks/permissions autorisés), stockage par vault `<vault>/.obsigate-plugins/`, lifecycle
complet (install/uninstall/enable/disable via marker `.disabled`), validation ZIP à l'upload
(anti path-traversal, max 100 fichiers, 500KB/fichier, 5MB upload), 9 endpoints `/api/plugins/*`
(list/get/hooks/code/template + install/uninstall/enable/disable admin-gated), template API,
scan au démarrage par vault (`get_plugin_registry().scan_vault`).
- Frontend `frontend/js/plugins.js` : PluginManager (install/uninstall/enable/disable/view-code),
sandbox d'exécution via Web Worker (code chargé par blob URL, protocole `postMessage` structuré,
isolation DOM/localStorage/network selon permissions), hooks dispatch
(`executeHook`, `onFileRender`, `onSearchFilter`, `onEditorAction`, `onSidebarItem`,
`onFileCreate`, `onFileDelete`, `onVaultMount`), UI Settings > Plugins.
- Sécurité : manifest de permissions (`read_files`, `write_files`, `network_request`, …
restreint), CSP stricte sans `importScripts`, limites de taille.
- Tests : `tests/test_plugins.py` (44 tests — validation manifest, lifecycle manager,
validation ZIP/directory, démarrage scan) + `tests/frontend/plugins.test.mjs`
(21 tests JSDOM — protocole sandbox Worker, isolation DOM, lifecycle mirror).
- CI : job lint ajoute `node plugins.test.mjs` aux tests JSDOM.
- Docs : `docs/PLUGINS.md` (manifest, hooks, permissions, modèle de sécurité, API, guide).
- **#78 Excalidraw — finitions** : décompression lz-string du format plugin Obsidian,
création depuis la modale et le menu contextuel, extraction du texte pour la recherche
(B5), support `.excalidraw.md` (commit ac16fc1 + correctifs associés).
- **#67 Notifications web (Push API)** : `backend/push.py`, `POST /api/push/subscribe`,
clés VAPID dans `config.json`, toggle par vault, payload (fichier/vault/action) et
ouverture du fichier au clic (commit ac16fc1).
- **#68 Health check enrichi** : `GET /api/health/detailed` (admin-gated) exposant index,
mémoire (RSS/heap), uptime, connexions SSE, backups et espace disque (commit ac16fc1).
- **#77 Desktop — jumplist vaults récents** dans le menu Démarrer (commit ac16fc1).
- **#75 Split View — matrice E2E complète (37 tests)** : 16 tests `split-view.spec.js`
(raccourcis, drag & drop, persistance, navigation clavier) + 21 tests
`split-view-matrix.spec.js` couvrant 100% des cas d'ouverture/fermeture.
### Documentation
- **#78 Excalidraw — doc utilisateur finalisée** : `.excalidraw` / `.excalidraw.md` ajoutés aux
formats supportés (README FR/EN), nouvelle section « Diagrammes Excalidraw » dans le guide
intégré (i18n FR/EN) et note de compatibilité avec le plugin Obsidian Excalidraw.
- **Roadmap** : #78 marqué terminé (B5 extraction texte, C8 menu contextuel, F3 E2E
`tests/e2e/excalidraw.spec.js`, H1-H3 docs) ; F2 non retenu. Points optionnels #61
(dépôt communautaire de plugins) et #74 (D2/E4/H2/I2) explicitement marqués non retenus.
### Corrigé
- **Assistant AI — sidebar qui ne s'ouvrait plus & erreur de contexte** :
- Le bouton flottant passait le **chemin d'un fichier** comme répertoire de contexte, d'où
l'erreur « Aucun fichier markdown trouvé dans ce dossier » à l'ouverture sur un document.
Le contexte est maintenant détecté automatiquement (documents ouverts → mode documents,
sinon mode général).
- Un panneau précédemment masqué (`obsigate-bookslm-hidden=true` persisté en localStorage)
restait caché lors d'une ouverture explicite → l'ouverture force désormais l'affichage.
- **Ergonomie du header** : le sélecteur fournisseur/modèle était sur la même ligne que les
actions et masquait ces dernières ; il est déplacé sur une seconde ligne dédiée (toolbar)
pour que « nouvelle conversation / export / plein écran / fermer » restent toujours visibles.
- **Clavier de la boîte de chat** : `Entrée` envoie la requête, `Ctrl+Entrée` insère un saut
de ligne (au lieu de l'inverse), avec un texte d'aide sous le champ.
- **Outil AI de l'UI — fiabilisation** :
- **BooksLM (chat IA par répertoire)** : les requêtes `context`/`chat` n'envoyaient pas le
jeton d'authentification (échec 401 quand l'auth est activée) et la réponse SSE était
mal parsée (le backend émet `{token: ...}`, le front lisait `data.content`) → les réponses
n'apparaissaient jamais. Le message utilisateur envoyé était le placeholder assistant vide
au lieu du texte saisi. Corrigé : requêtes authentifiées via `AuthManager` (retry 401),
parsing SSE `event:`/`data:` avec support `token`/`error`, historique construit avant
l'ajout du message courant, barre de progression basée sur `max_total_chars`.
- **AI Editor** : le raccourci `Ctrl/Cmd+J` (complétion inline) était re-lié à chaque
ouverture de l'éditeur (N listeners → N requêtes) ; un seul listener global est désormais
posé. Les libellés de la barre d'outils AI étaient codés en dur (FR/EN mélangés) → i18n
complet. La réécriture personnalisée utilisait `window.prompt()` → remplacée par une modale
accessible (Échap annule, Ctrl/Cmd+Entrée valide).
- **Recherche cassée (régression plugins)** : `onSearchFilter()` était appelée avec 2 arguments
au lieu de 3 (`frontend/js/search.js`), renvoyait `undefined`, faisait échouer le rendu des
résultats et basculait silencieusement sur la recherche hors-ligne (0 résultat). Corrigé →
la recherche full-text et l'autocomplétion refonctionnent (20 tests E2E).
- **`.excalidraw.md` corrompu à l'ouverture** : Excalidraw émet `onChange` au montage, ce qui
déclenchait un auto-save 2 s plus tard réécrivant le fichier au format JSON brut et détruisant
le format plugin Obsidian (`frontend/excalidraw-editor.html` + `frontend/js/excalidraw-viewer.js`).
Les changements initiaux sont désormais ignorés et le format `frontmatter + compressed-json`
est préservé à la sauvegarde.
- **Splash / tests E2E** : `window.__OBSIGATE_BOOTED` était posé avant la fin de `init()` — le
splash disparaissait trop tôt et les tests E2E interagissaient avant que les handlers soient
liés. Le flag est maintenant posé après résolution de `init()` (`frontend/js/app.js`), et
`goHome()`/`login()` attendent ce signal.
- **Tests E2E Excalidraw** : réécriture de `tests/e2e/excalidraw.spec.js` avec les sélecteurs
réels (modale `.obsigate-modal` + `#file-ext-select`/`#file-name-input`, menu contextuel,
`canvas.first()` pour les deux couches canvas).
---
## [2.1.0] — 2026-09-08
### Ajouté
@@ -27,6 +979,15 @@ et [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
- **#77 Desktop — port auto-increment** : `pick_free_port()` scanne 17890..17899 si le port
est occupé (2 instances côte à côte possibles) + 3 tests Rust (19 au total côté desktop).
- Docs : sections PDF README FR/EN (Range, /pdf/info), variables env, guide WebAuthn.
- **#75 Split View — matrice E2E complète (37 tests)** : 16 tests `split-view.spec.js` (raccourcis,
drag & drop, persistance, navigation clavier) + 21 tests `split-view-matrix.spec.js` couvrant
100% des cas d'ouverture/fermeture : boutons ⊞→/⊞↓, menu contextuel (diviser bas, fermer, fermer
les autres/à droite/tout, fermer le panneau, flyout « Déplacer vers… »), palette (diviser, focus
suivant/précédent, reset), fermeture par X/double-clic/molette/Ctrl+W, auto-collapse d'un panneau
vidé, plafond 4 panneaux, drag-to-split bas, pas de doublon à la réouverture.
- Fix `scripts/run-e2e-local.sh` : chemins Windows via `cygpath` (les env vars MSYS n'étaient pas
converties → vaults vides, les 16 tests split-view échouaient en local alors qu'ils passaient
en CI).
### Corrigé
@@ -38,7 +999,7 @@ et [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
---
## [2.0.0] — Unreleased
## [2.0.0] — 2026-08-25
> ObsiGate passe à **2.0.0** avec sa **version bureau native** et de nombreuses fonctionnalités
> de productivité (branch main, 341 commits après le tag v1.8.0).
+7 -7
View File
@@ -3,7 +3,8 @@
FROM python:3.11-slim AS builder
RUN apt-get update \
&& apt-get install -y --no-install-recommends gcc libffi-dev libc6-dev libpango-1.0-0 libpangocairo-1.0-0 shared-mime-info \
&& apt-get install -y --no-install-recommends gcc libffi-dev libc6-dev \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /build
@@ -14,7 +15,6 @@ RUN pip install --no-cache-dir --prefix=/install -r requirements.txt
FROM python:3.11-slim
LABEL maintainer="Bruno Beloeil" \
version="1.4.0" \
description="ObsiGate — lightweight web interface for Obsidian vaults"
WORKDIR /app
@@ -25,17 +25,17 @@ COPY --from=builder /install /usr/local
# WeasyPrint runtime dependencies
RUN apt-get update \
&& apt-get install -y --no-install-recommends libpango-1.0-0 libpangocairo-1.0-0 shared-mime-info \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Copy application code
COPY backend/ ./backend/
COPY frontend/ ./frontend/
# Bake version: build.sh/CI pre-generate backend/VERSION (copied above);
# plain `docker compose build` has no such file (gitignored) — fall back
# to the VERSION build arg so the image always carries a version string.
ARG VERSION=0.0.0-dev
RUN test -f backend/VERSION || echo "$VERSION" > backend/VERSION
# Version livrée : `VERSION` (racine du dépôt) est la source unique de vérité —
# copié dans l'image, jamais un numéro codé en dur. `backend/version.py` le lit
# et /api/health l'expose (header + boîte À propos).
COPY VERSION ./VERSION
# Create non-root user for security + data directory for auth persistence
# Using explicit UID/GID 1000 to match common host user and docker-compose settings
+46 -6
View File
@@ -4,7 +4,7 @@
**Porte d'entrée web ultra-léger pour vos vaults Obsidian** — Accédez, naviguez et recherchez dans toutes vos notes Obsidian depuis n'importe quel appareil via une interface web moderne et responsive.
[![Version](https://img.shields.io/badge/Version-2.0.0--dev-blue.svg)]()
[![Version](https://img.shields.io/badge/Version-2.6.1-blue.svg)]()
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Docker](https://img.shields.io/badge/Docker-Ready-blue.svg)](https://www.docker.com/)
[![Python](https://img.shields.io/badge/Python-3.11+-green.svg)](https://www.python.org/)
@@ -55,16 +55,20 @@
## ✨ Fonctionnalités
- **🤖 AI Editor intégré** — Éditeur CodeMirror 6 avec toolbar IA : amélioration, correction, traduction, génération, réécriture personnalisée, toolbox (liste, tableau, frontmatter, canvas) — multi-provider DeepSeek/OpenRouter/Gemini
- **🧩 Serveur MCP & agent IA** — Serveur Model Context Protocol intégré (`/mcp`) et assistant avec function calling : lisez, cherchez et modifiez vos vaults depuis Claude Desktop, Cursor… avec confirmations two-step, permissions par vault, rate limiting et redaction des secrets ([guide](docs/MCP_GUIDE.md))
- **👥 Collaboration temps réel** — Édition simultanée d'un même document (Yjs/CRDT) : curseurs distants colorés, indicateur de présence, fusion sans conflit, reconnexion automatique et persistance serveur ([détail](docs/features/collaboration.md))
- **📱 Éditeur mobile natif** — Édition optimisée pour le tactile : barre d'outils Markdown flottante (gras/italique/code/liste/lien), bouton « Coller » persistant (contournement iOS), zoom par pincement et hauteur ajustable, raccourcis swipe (liens entrants / table des matières) et mode lecture plein écran avec navigation entre fichiers ([détail](docs/features/mobile-editor.md))
- **🗺️ Vue graphe interactive** — Canvas force-directed avec Barnes-Hut O(n log n), filtres (tag, type), profondeur, mode focus, historique de navigation ←→↑, export PNG, aperçu au survol (Ctrl+click)
- **🗂️ Multi-vault** : Visualisez plusieurs vaults Obsidian simultanément
- **🌳 Navigation arborescente** : Parcourez vos dossiers et fichiers dans la sidebar
- **🔍 Recherche avancée** : Moteur TF-IDF avec stemming français, normalisation des accents, snippets surlignés, facettes, pagination et tri
- **🔍 Recherche avancée** : Moteur TF-IDF avec stemming français, normalisation des accents, snippets surlignés, facettes, pagination et tri — plus une **recherche sémantique** optionnelle (embeddings `all-MiniLM-L6-v2`, fusion hybride TF-IDF + RRF) activable via le toggle `~` ([détail](docs/features/semantic-search.md))
- **💡 Autocomplétion intelligente** : Suggestions de fichiers, tags et historique avec navigation clavier
- **🧩 Syntaxe de requête** : Opérateurs `tag:`, `#`, `vault:`, `title:`, `path:`, `ext:` avec chips visuels
- **📜 Historique de recherche** : Persisté en localStorage (max 50 entrées, LIFO, dédupliqué)
- **🏷️ Tag cloud** : Filtrage par tags extraits des frontmatters YAML
- **🔗 Wikilinks** : Les `[[liens internes]]` Obsidian sont cliquables
- **🖼️ Images Obsidian** : Support complet des syntaxes d'images Obsidian avec résolution intelligente
- **🎨 Diagrammes Excalidraw** : Visualiseur/éditeur natif des fichiers `.excalidraw` et `.excalidraw.md` (iframe sandboxée, auto-save, thème clair/sombre, texte des diagrammes indexé pour la recherche)
- **🎨 Syntax highlight** : Coloration syntaxique des blocs de code
- **🌓 Thème clair/sombre** : Toggle persisté en localStorage
- **📡 Synchronisation temps réel** : Surveillance automatique des fichiers via watchdog avec mise à jour incrémentale de l'index
@@ -271,7 +275,11 @@ Un compte **admin** connecté voit une icône 🛡️ dans le header : liste, cr
| `OBSIGATE_ACCESS_TOKEN_TTL` | Durée de vie token JWT (secondes) | `3600` |
| `OBSIGATE_REFRESH_TOKEN_TTL` | Durée de vie refresh token (secondes) | `2592000` |
| `OBSIGATE_LOGIN_MAX_ATTEMPTS` | Tentatives de login max par IP | `10` |
| `OBSIGATE_ACCOUNT_MAX_ATTEMPTS` | Tentatives de login max par compte | `10` |
| `OBSIGATE_LOGIN_WINDOW_SECONDS` | Fenêtre de rate limiting (secondes) | `900` |
| `OBSIGATE_TRUST_PROXY` | Faire confiance à `X-Forwarded-For` pour l'IP client (reverse proxy) | `false` |
| `OBSIGATE_WEBHOOK_ALLOW_HTTP` | Autoriser les webhooks non HTTPS | `false` |
| `OBSIGATE_WEBHOOK_ALLOW_PRIVATE` | Autoriser les webhooks vers des adresses privées/boucle | `false` |
| `OBSIGATE_PDF_MAX_SIZE_MB` | Taille max des PDF extraits (text indexation) | `50` |
| `OBSIGATE_PDF_EXTRACT_TIMEOUT` | Timeout extraction PDF (secondes) | `30` |
@@ -550,6 +558,25 @@ Cycle de vie : Tauri spawn le backend Python → health check → splash de dém
---
## 👥 Collaboration temps réel
Plusieurs utilisateurs peuvent éditer le même document markdown simultanément (façon Google Docs) :
- **Fusion sans conflit** grâce à Yjs (CRDT) : deux personnes peuvent taper au même endroit, aucune
modification n'est perdue.
- **Curseurs distants colorés** et sélections visibles dans CodeMirror, avec le nom de chaque
utilisateur.
- **Indicateur de présence** dans l'en-tête de l'éditeur (avatars + statut de connexion).
- **Reconnexion automatique** (backoff exponentiel) : l'état est fusionné au retour.
- **Persistance serveur** : le document est écrit sur disque 2 s après la dernière modification.
- **Transport** : WebSocket `ws(s)://<hôte>/ws/collab/{vault}/{chemin}`, authentifié par cookie
`access_token` (ou `?token=`) et soumis au contrôle d'accès par vault.
Aucune configuration n'est nécessaire : ouvrez le même fichier dans deux navigateurs (ou deux
fenêtres) pour voir la collaboration en action.
---
## 🔌 API
ObsiGate expose une API REST complète :
@@ -572,7 +599,7 @@ ObsiGate expose une API REST complète :
| `/api/file/{vault}/download?path=` | Téléchargement d'un fichier | GET | Oui |
| `/api/file/{vault}/save?path=` | Sauvegarder un fichier | PUT | Oui |
| `/api/file/{vault}?path=` | Supprimer un fichier | DELETE | Oui |
| `/api/search/advanced` | Recherche avancée TF-IDF | GET | Oui |
| `/api/search/advanced` | Recherche avancée TF-IDF (+ `semantic=true` pour l'hybride) | GET | Oui |
| `/api/suggest` / `/api/tags/suggest` | Autocomplétion | GET | Oui |
| `/api/tags?vault=` | Tags uniques avec compteurs | GET | Oui |
| `/api/index/reload` | Force un re-scan des vaults | GET | Admin |
@@ -623,6 +650,13 @@ Les métadonnées (pages, titre, auteur) sont disponibles via `GET /api/file/{va
**Limitations :** pas d'OCR (les PDF scannés ne sont pas recherchables), pas d'annotation, pas d'édition du PDF lui-même.
### Diagrammes Excalidraw
Les fichiers `.excalidraw` et `.excalidraw.md` s'ouvrent dans un éditeur visuel Excalidraw complet, intégré dans une iframe sandboxée — dessinez, modifiez et sauvegardez sans quitter ObsiGate.
Les modifications sont sauvegardées automatiquement (2 s) ou avec `Ctrl+S` ; l'éditeur suit le thème clair/sombre.
Le texte des éléments du diagramme est extrait à l'indexation : il est donc recherchable via la recherche full-text (`ext:excalidraw`).
Les fichiers créés avec le **plugin Obsidian Excalidraw** (y compris le format `.excalidraw.md` compressé) sont compatibles.
### Raccourcis clavier
| Raccourci | Action |
@@ -639,6 +673,11 @@ Les métadonnées (pages, titre, auteur) sont disponibles via `GET /api/file/{va
- **Boost titre** : correspondances dans le titre ×3
- **Normalisation des accents** : `resume` trouve `résumé`
- **Snippets surlignés** (`<mark>`), **facettes** (compteurs par vault/tag), **pagination** (50/page), **tri** pertinence/date, **chips** de filtres, **historique** (50 recherches)
- **Recherche sémantique** (optionnelle) : le toggle `~` (ou `Alt+S`) fusionne le classement
TF-IDF avec un classement par embeddings (RRF). Fonctionne sans dépendance avec un provider de
hachage ; installez `backend/requirements-semantic.txt` et/ou renseignez `OBSIGATE_EMBEDDING_*`
pour de vrais embeddings `all-MiniLM-L6-v2`. Voir
[docs/features/semantic-search.md](docs/features/semantic-search.md).
---
@@ -857,7 +896,8 @@ ObsiGate/
### Contribuer
Voir [docs/CONTRIBUTING.md](./docs/CONTRIBUTING.md) pour les détails.
Voir [docs/CONTRIBUTING.md](./docs/CONTRIBUTING.md) pour les standards de code et
[docs/DELIVERY_WORKFLOW.md](./docs/DELIVERY_WORKFLOW.md) pour la méthode de livraison obligatoire.
---
@@ -876,8 +916,8 @@ Ce projet est sous licence **MIT** — voir le fichier [LICENSE](LICENSE) pour l
## 📝 Changelog
Consultez le [CHANGELOG.md](./CHANGELOG.md) pour l'historique complet de toutes les versions (v1.0.0 → v2.0.0-dev).
Consultez le [CHANGELOG.md](./CHANGELOG.md) pour l'historique complet de toutes les versions (v1.0.0 → v2.6.1).
---
*Projet : ObsiGate | Version : 2.0.0-dev | Dernière mise à jour : Juin 2026*
*Projet : ObsiGate | Version : 2.6.1 | Dernière mise à jour : Juin 2026*
+44 -8
View File
@@ -2,7 +2,7 @@
**Ultra-light web gateway for your Obsidian vaults** — Access, browse, and search all your Obsidian notes from any device via a modern, responsive web interface.
[![Version](https://img.shields.io/badge/Version-1.7.0-blue.svg)]()
[![Version](https://img.shields.io/badge/Version-2.6.1-blue.svg)]()
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Docker](https://img.shields.io/badge/Docker-Ready-blue.svg)](https://www.docker.com/)
[![Python](https://img.shields.io/badge/Python-3.11+-green.svg)](https://www.python.org/)
@@ -48,16 +48,20 @@
## ✨ Features
- **🤖 Integrated AI Editor** — CodeMirror 6 editor with AI toolbar: improve, correct, translate, generate, custom rewrite, toolbox (list, table, frontmatter, canvas) — multi-provider DeepSeek/OpenRouter/Gemini
- **🧩 MCP Server & AI Agent** — Built-in Model Context Protocol server (`/mcp`) and tool-calling assistant: read, search and edit your vaults from Claude Desktop, Cursor… with two-step confirmations, per-vault permissions, rate limiting and secret redaction ([guide](docs/MCP_GUIDE.md))
- **👥 Real-time Collaboration** — Simultaneous editing of the same document (Yjs/CRDT): colored remote cursors, presence indicator, conflict-free merge, automatic reconnection and server-side persistence ([details](docs/features/collaboration.md))
- **📱 Native Mobile Editor** — Touch-optimised editing: floating Markdown toolbar (bold/italic/code/list/link), persistent Paste button (iOS workaround), pinch-zoom font & adjustable height, swipe shortcuts (backlinks / table of contents) and a full-screen reading mode with page navigation ([details](docs/features/mobile-editor.md))
- **🗺️ Interactive Graph View** — Canvas force-directed with Barnes-Hut O(n log n), filters (tag, type), depth, focus mode, navigation history ←→↑, export PNG, preview on hover (Ctrl+click)
- **🗂️ Multi-vault** : View multiple Obsidian vaults simultaneously
- **🌳 Tree Navigation** : Browse your folders and files in the sidebar
- **🔍 Advanced Search** : TF-IDF search engine with French stemming, accent normalization, highlighted snippets, facets, pagination, and sorting
- **🔍 Advanced Search** : TF-IDF search engine with French stemming, accent normalization, highlighted snippets, facets, pagination, and sorting — plus an optional **semantic search** (embeddings via `all-MiniLM-L6-v2`, hybrid TF-IDF + RRF fusion) toggled with `~` ([details](docs/features/semantic-search.md))
- **💡 Smart Autocomplete** : Suggestions for files, tags, and history with keyboard navigation
- **🧩 Query Syntax** : Operators `tag:`, `#`, `vault:`, `title:`, `path:`, `ext:` with visual chips
- **📜 Search History** : Persisted in localStorage (max 50 entries, LIFO, deduplicated)
- **🏷️ Tag Cloud** : Filtering by tags extracted from YAML frontmatters
- **🔗 Wikilinks** : `[[internal links]]` from Obsidian are clickable
- **🖼️ Obsidian Images** : Full support for all Obsidian image syntaxes with intelligent resolution
- **🎨 Excalidraw Diagrams** : Native viewer/editor for `.excalidraw` and `.excalidraw.md` files (sandboxed iframe, autosave, dark/light theme, diagram text indexed for search)
- **🎨 Syntax Highlight** : Syntax highlighting for code blocks
- **🌓 Light/Dark Theme** : Toggle persisted in localStorage
- **📡 Real-time Sync** : Automatic file monitoring via watchdog with incremental index updates
@@ -309,7 +313,11 @@ When an **admin** account is logged in, a 🛡️ icon appears in the header. Cl
| `OBSIGATE_ACCESS_TOKEN_TTL` | JWT token lifetime (seconds) | `3600` |
| `OBSIGATE_REFRESH_TOKEN_TTL` | Refresh token lifetime (seconds) | `2592000` |
| `OBSIGATE_LOGIN_MAX_ATTEMPTS` | Max login attempts per IP | `10` |
| `OBSIGATE_ACCOUNT_MAX_ATTEMPTS` | Max login attempts per account | `10` |
| `OBSIGATE_LOGIN_WINDOW_SECONDS` | Rate limiting window (seconds) | `900` |
| `OBSIGATE_TRUST_PROXY` | Trust `X-Forwarded-For` for the client IP (reverse proxy) | `false` |
| `OBSIGATE_WEBHOOK_ALLOW_HTTP` | Allow non-HTTPS webhook targets | `false` |
| `OBSIGATE_WEBHOOK_ALLOW_PRIVATE` | Allow webhooks to private/loopback addresses | `false` |
| `OBSIGATE_PDF_MAX_SIZE_MB` | Max PDF size for text extraction | `50` |
| `OBSIGATE_PDF_EXTRACT_TIMEOUT` | PDF extraction timeout (seconds) | `30` |
@@ -666,6 +674,22 @@ Lifecycle: Tauri spawns the Python backend → health check → opens the webvie
---
## 👥 Real-time Collaboration
Multiple users can edit the same markdown document simultaneously (Google Docs style):
- **Conflict-free merge** via Yjs (CRDT): two people can type in the same place, no change is lost.
- **Colored remote cursors** and visible selections in CodeMirror, labelled with each user's name.
- **Presence indicator** in the editor header (avatars + connection status).
- **Automatic reconnection** (exponential backoff): state is merged on return.
- **Server-side persistence**: the document is written to disk 2 s after the last change.
- **Transport**: WebSocket `ws(s)://<host>/ws/collab/{vault}/{path}`, authenticated via the
`access_token` cookie (or `?token=`) and subject to per-vault access control.
No configuration is required: open the same file in two browsers (or two windows) to see it live.
---
## 🔌 API
ObsiGate exposes a complete REST API :
@@ -688,7 +712,7 @@ ObsiGate exposes a complete REST API :
| `/api/file/{vault}/download?path=` | Download a file | GET | Yes |
| `/api/file/{vault}/save?path=` | Save a file | PUT | Yes |
| `/api/file/{vault}?path=` | Delete a file | DELETE | Yes |
| `/api/search/advanced` | Advanced TF-IDF search | GET | Yes |
| `/api/search/advanced` | Advanced TF-IDF search (+ `semantic=true` for hybrid) | GET | Yes |
| `/api/suggest` / `/api/tags/suggest` | Autocomplete | GET | Yes |
| `/api/tags?vault=` | Unique tags with counters | GET | Yes |
| `/api/index/reload` | Force a rescan of vaults | GET | Admin |
@@ -740,7 +764,7 @@ curl "http://localhost:2020/api/file/Recipes?path=pizza.md"
| `ext:<type>` | Filter by file type | `ext:md kubernetes` |
| `"exact phrase"` | Phrase search | `tag:"multiple words"` |
Extension filter examples: `ext:sh` for bash scripts, `ext:py` for Python scripts, `ext:md` for Markdown files, `ext:pdf` for PDF documents (text-extracted content is indexed).
Extension filter examples: `ext:sh` for bash scripts, `ext:py` for Python scripts, `ext:md` for Markdown files, `ext:pdf` for PDF documents, `ext:excalidraw` for Excalidraw diagrams (text-extracted content is indexed).
### PDF support
@@ -752,6 +776,13 @@ PDF metadata (pages, title, author) is available via `GET /api/file/{vault}/pdf/
**Limitations:** no OCR (scanned PDFs aren't searchable), no annotation, no editing of the PDF itself.
### Excalidraw diagrams
`.excalidraw` and `.excalidraw.md` files open in a full visual Excalidraw editor embedded in a sandboxed iframe — draw, edit and save without leaving ObsiGate.
Changes are saved automatically (2s debounce) or with `Ctrl+S`; the editor follows the light/dark theme.
Text labels inside the diagram elements are extracted on indexing, so diagram content is searchable via the full-text search (`ext:excalidraw`).
Files created with the **Obsidian Excalidraw plugin** (including the compressed `.excalidraw.md` format) are compatible.
Operators are combinable: `tag:linux vault:IT ext:md server web` searches for "server web" in Markdown files of the IT vault with the linux tag.
### Keyboard Shortcuts
@@ -775,6 +806,10 @@ Operators are combinable: `tag:linux vault:IT ext:md server web` searches for "s
- **Sorting** : By relevance (TF-IDF) or modification date
- **Visual chips** : Active filters are shown as removable colored chips
- **History** : Last 50 searches are stored in localStorage
- **Semantic search** (optional) : Toggle `~` (or `Alt+S`) fuses the TF-IDF ranking with an
embedding-based ranking (RRF). Works out of the box with a dependency-free hashing embedder;
install `backend/requirements-semantic.txt` and/or set `OBSIGATE_EMBEDDING_*` for real
`all-MiniLM-L6-v2` embeddings. See [docs/features/semantic-search.md](docs/features/semantic-search.md).
---
@@ -1024,12 +1059,13 @@ ObsiGate/
├── Dockerfile # Multi-stage, healthcheck, non-root
├── docker-compose.yml # Deployment with healthcheck and auth env vars
├── build.sh # Automated build & deployment (docker compose build + up)
└── CONTRIBUTING.md # Contribution guide
└── docs/CONTRIBUTING.md # Contribution guide
```
### Contributing
See [CONTRIBUTING.md](CONTRIBUTING.md) for details.
See [CONTRIBUTING.md](docs/CONTRIBUTING.md) for code standards and
[docs/DELIVERY_WORKFLOW.md](docs/DELIVERY_WORKFLOW.md) for the mandatory delivery process.
---
@@ -1049,8 +1085,8 @@ This project is licensed under the **MIT License** - see the [LICENSE](LICENSE)
## 📝 Changelog
See [CHANGELOG.md](./CHANGELOG.md) for the complete version history (v1.0.0 → v1.7.0).
See [CHANGELOG.md](./CHANGELOG.md) for the complete version history (v1.0.0 → v2.6.1).
---
*Project: ObsiGate | Version: 1.7.0 | Last updated: May 2026*
*Project: ObsiGate | Version: 2.6.1 | Last updated: May 2026*
+1
View File
@@ -0,0 +1 @@
2.6.1
+308
View File
@@ -0,0 +1,308 @@
"""In-app agent loop — multi-step tool calling.
The loop drives an LLM that may request tool calls, executes them through the
shared tool layer (``backend.tools``), feeds the results back, and repeats
until the model produces a final answer or the iteration budget is exhausted.
The LLM is injected as an async callable so the loop is fully testable without
network access::
async def fake_llm(messages, tools):
return LLMResponse(content="done")
result = await run_agent(messages, ctx=ctx, llm=fake_llm)
"""
from __future__ import annotations
import json
import logging
import os
from collections.abc import Callable
from dataclasses import dataclass, field
from typing import Any
from backend.tools.api import (
ToolConfirmationRequired,
ToolContext,
ToolError,
ToolScope,
call_tool,
get_tool_schemas,
)
from backend.tools.labels import thought_step_label, tool_step_label
logger = logging.getLogger("obsigate.agent.loop")
DEFAULT_MAX_ITERATIONS = 10
# Cap the size of a tool result fed back to the model (chars).
MAX_TOOL_RESULT_CHARS = 100_000
# Quota: maximum tool calls executed per agent run (``BOOKSLM_MAX_TOOL_CALLS``).
DEFAULT_MAX_TOOL_CALLS = int(os.environ.get("BOOKSLM_MAX_TOOL_CALLS", "25"))
# Stopping reasons
STOP_DONE = "done"
STOP_MAX_ITERATIONS = "max_iterations"
STOP_CONFIRMATION_REQUIRED = "confirmation_required"
STOP_QUOTA_EXCEEDED = "quota_exceeded"
@dataclass
class ToolCallRecord:
"""Audit-friendly record of one executed tool call."""
name: str
arguments: dict[str, Any]
ok: bool
result: Any
# Human-readable « step » label for the Notion-style UI
# ({key, params} — see backend.tools.labels).
step: dict[str, Any] = field(default_factory=dict)
@dataclass
class AgentResult:
"""Outcome of an agent run."""
content: str = ""
messages: list[dict[str, Any]] = field(default_factory=list)
tool_calls: list[ToolCallRecord] = field(default_factory=list)
# Ordered Notion-style step descriptors ({key, params}); tool steps and
# intermediate reasoning notes interleaved by execution order.
steps: list[dict[str, Any]] = field(default_factory=list)
iterations: int = 0
stopped: str = STOP_DONE
pending: dict[str, Any] | None = None
async def provider_llm(messages: list[dict[str, Any]], tools: list[dict[str, Any]] | None):
"""Default LLM adapter backed by ``backend.ai_chat.chat_completion``."""
from backend.ai_chat import chat_completion
return await chat_completion(messages, tools=tools)
def _truncate(payload: Any) -> Any:
"""Truncate an oversized tool result before feeding it back to the model."""
serialized = json.dumps(payload, ensure_ascii=False, default=str)
if len(serialized) <= MAX_TOOL_RESULT_CHARS:
return payload
return {"truncated": True, "content": serialized[:MAX_TOOL_RESULT_CHARS]}
def _assistant_tool_message(content: str | None, tool_calls: list[Any]) -> dict[str, Any]:
"""Build the OpenAI-style assistant message carrying tool calls."""
return {
"role": "assistant",
"content": content or "",
"tool_calls": [
{
"id": call.id,
"type": "function",
"function": {
"name": call.name,
"arguments": json.dumps(call.arguments, ensure_ascii=False, default=str),
},
}
for call in tool_calls
],
}
def _execute_confirmed(
ctx: ToolContext,
confirm_pending: dict[str, Any],
convo: list[dict[str, Any]],
executed: list[ToolCallRecord],
on_tool_call: Callable[[ToolCallRecord], None] | None,
) -> None:
"""Apply a previously-paused mutating tool call and feed its result back.
The pending payload is the ``error`` object emitted by a ``confirmation``
event. The assistant tool-call message is expected to already be in
``convo`` (it is part of the snapshot returned with the confirmation).
"""
from backend.ai_chat import ToolCall
error = confirm_pending.get("error", confirm_pending)
name = error.get("tool")
arguments = error.get("arguments") or {}
call_id = error.get("id") or "call_pending"
if not name:
raise ToolError("Malformed confirmation payload", code="invalid_confirmation")
# Make sure the assistant tool-call message is present in the snapshot.
if not any(
m.get("role") == "assistant" and any(
tc.get("id") == call_id for tc in (m.get("tool_calls") or [])
)
for m in convo
):
convo.append(_assistant_tool_message(None, [ToolCall(id=call_id, name=name, arguments=arguments)]))
try:
result = call_tool(name, ctx, arguments, confirm=True)
payload = result.data
ok = True
except ToolError as e:
payload = e.to_dict()
ok = False
record = ToolCallRecord(
name=name, arguments=arguments, ok=ok, result=payload,
step=tool_step_label(name, arguments),
)
executed.append(record)
if on_tool_call is not None:
on_tool_call(record)
convo.append({
"role": "tool",
"tool_call_id": call_id,
"name": name,
"content": json.dumps(_truncate(payload), ensure_ascii=False, default=str),
})
async def run_agent(
messages: list[dict[str, Any]],
*,
ctx: ToolContext,
llm: Callable[..., Any] | None = None,
tools: list[dict[str, Any]] | None = None,
max_iterations: int = DEFAULT_MAX_ITERATIONS,
max_tool_calls: int | None = None,
on_tool_call: Callable[[ToolCallRecord], None] | None = None,
on_thought: Callable[[dict[str, Any]], None] | None = None,
resume_messages: list[dict[str, Any]] | None = None,
confirm_pending: dict[str, Any] | None = None,
) -> AgentResult:
"""Run the tool-calling loop until completion.
Args:
messages: Initial conversation (OpenAI-style), typically a system
message followed by the conversation history and the user message.
ctx: Tool execution context (identity, mode, confirmation state).
llm: Async callable ``(messages, tools) -> LLMResponse``. Defaults to
the real provider adapter.
tools: Tool schemas to expose. ``None`` exposes all in-app tools;
pass ``[]`` to disable tool calling (plain chat).
max_iterations: Hard cap on LLM round-trips.
max_tool_calls: Hard cap on the total number of executed tool calls
(quota, defaults to ``BOOKSLM_MAX_TOOL_CALLS``).
on_tool_call: Optional callback invoked after each executed tool call.
resume_messages: Conversation snapshot from a paused run (returned with
a ``confirmation`` event). When set, the loop resumes from it.
confirm_pending: Pending mutating tool call to apply before resuming
(two-step propose/apply).
Returns:
An :class:`AgentResult`. ``stopped`` is ``done``, ``max_iterations`` or
``confirmation_required`` (in which case ``pending`` holds the payload
to confirm, for the two-step propose/apply flow).
"""
llm = llm or provider_llm
if tools is None:
tools = get_tool_schemas(scope=ToolScope.IN_APP)
quota = DEFAULT_MAX_TOOL_CALLS if max_tool_calls is None else max_tool_calls
steps: list[dict[str, Any]] = []
def _emit_note(text: str) -> None:
"""Record an intermediate reasoning note as a visible step."""
note = thought_step_label(text)
if note["params"]["value"]:
steps.append(note)
if on_thought is not None:
on_thought(note)
convo = [dict(m) for m in (resume_messages if resume_messages is not None else messages)]
executed: list[ToolCallRecord] = []
if confirm_pending:
if quota is not None and len(executed) >= quota:
return AgentResult(
content="",
messages=convo,
tool_calls=executed,
steps=steps,
iterations=0,
stopped=STOP_QUOTA_EXCEEDED,
)
_execute_confirmed(ctx, confirm_pending, convo, executed, on_tool_call)
for iteration in range(1, max_iterations + 1):
response = await llm(convo, tools)
if not response.has_tool_calls:
return AgentResult(
content=response.content or "",
messages=convo,
tool_calls=executed,
steps=steps,
iterations=iteration,
stopped=STOP_DONE,
)
# Intermediate reasoning shown alongside tool calls → a "thought" step.
_emit_note(response.content or "")
convo.append(_assistant_tool_message(response.content, response.tool_calls))
for call in response.tool_calls:
if quota is not None and len(executed) >= quota:
logger.warning(f"Agent reached the tool-call quota ({quota})")
return AgentResult(
content=response.content or "",
messages=convo,
tool_calls=executed,
steps=steps,
iterations=iteration,
stopped=STOP_QUOTA_EXCEEDED,
)
try:
result = call_tool(call.name, ctx, call.arguments)
payload = result.data
ok = True
except ToolConfirmationRequired as e:
logger.info(f"Agent paused: confirmation required for '{call.name}'")
pending = e.to_dict()
# Include the tool-call id so the client can echo it back.
pending["error"]["id"] = call.id
return AgentResult(
content=response.content or "",
messages=convo,
tool_calls=executed,
steps=steps,
iterations=iteration,
stopped=STOP_CONFIRMATION_REQUIRED,
pending=pending,
)
except ToolError as e:
payload = e.to_dict()
ok = False
record = ToolCallRecord(
name=call.name, arguments=call.arguments, ok=ok, result=payload,
step=tool_step_label(call.name, call.arguments),
)
executed.append(record)
steps.append(record.step)
if on_tool_call is not None:
on_tool_call(record)
convo.append({
"role": "tool",
"tool_call_id": call.id,
"name": call.name,
"content": json.dumps(_truncate(payload), ensure_ascii=False, default=str),
})
logger.warning(f"Agent reached max iterations ({max_iterations})")
return AgentResult(
content="",
messages=convo,
tool_calls=executed,
steps=steps,
iterations=max_iterations,
stopped=STOP_MAX_ITERATIONS,
)
+61 -17
View File
@@ -18,6 +18,7 @@ ProviderName = Literal["deepseek", "openrouter", "gemini", "ollama", "nvidia", "
# Provider configurations — keys loaded from file or .env
AI_KEYS_FILE = Path("data/api_keys.json")
APP_CONFIG_FILE = Path(__file__).resolve().parent.parent / "data" / "config.json"
def _read_ai_keys() -> dict:
if not AI_KEYS_FILE.exists():
@@ -27,6 +28,16 @@ def _read_ai_keys() -> dict:
except Exception:
return {}
def _read_app_config() -> dict:
"""Read the persisted application config (``data/config.json``)."""
if not APP_CONFIG_FILE.exists():
return {}
try:
return json.loads(APP_CONFIG_FILE.read_text(encoding="utf-8"))
except Exception:
return {}
def get_ai_key(env_name: str) -> str:
"""Get AI key: stored file first, then .env fallback."""
keys = _read_ai_keys()
@@ -36,7 +47,7 @@ def get_ai_key(env_name: str) -> str:
def _load_provider_keys():
"""Load AI keys from stored file, falling back to .env."""
return {
providers = {
"deepseek": {
"api_key": get_ai_key("DEEPSEEK_API_KEY"),
"base_url": "https://api.deepseek.com/v1",
@@ -90,17 +101,46 @@ def _load_provider_keys():
"auth_header": "Bearer {api_key}",
},
}
# Apply persisted per-provider model overrides (data/config.json).
overrides = _read_app_config().get("ai_default_models") or {}
if isinstance(overrides, dict):
for name, model in overrides.items():
if name in providers and isinstance(model, str) and model:
providers[name]["model"] = model
return providers
PROVIDERS = _load_provider_keys()
DEFAULT_PROVIDER: ProviderName = os.getenv("AI_DEFAULT_PROVIDER", "deepseek") # type: ignore
def get_default_provider() -> str:
"""Resolve the default provider: ``data/config.json`` > env > ``deepseek``."""
provider: str = str(_read_app_config().get("ai_default_provider") or os.getenv("AI_DEFAULT_PROVIDER", "deepseek"))
return provider if provider in PROVIDERS else "deepseek"
def reload_ai_config() -> str:
"""Reload provider keys and model overrides from disk into ``PROVIDERS`` in place.
Mutating ``PROVIDERS`` (rather than rebinding it) keeps references held by
other modules valid. Returns the resolved default provider.
"""
global DEFAULT_PROVIDER
PROVIDERS.clear()
PROVIDERS.update(_load_provider_keys())
DEFAULT_PROVIDER = get_default_provider() # type: ignore[assignment]
logger.info(f"AI config reloaded (default provider: {DEFAULT_PROVIDER})")
return DEFAULT_PROVIDER
def _get_provider_config(provider: ProviderName | None = None) -> dict:
"""Get provider config, falling back to default if requested provider unavailable."""
p = provider or DEFAULT_PROVIDER
default = get_default_provider()
p = provider or default
if p not in PROVIDERS:
p = DEFAULT_PROVIDER
p = default
cfg = PROVIDERS[p]
if not cfg["api_key"]:
# Try next available provider
@@ -112,6 +152,23 @@ def _get_provider_config(provider: ProviderName | None = None) -> dict:
return {"name": p, **cfg}
def _build_headers(cfg: dict) -> dict:
"""Build HTTP headers for an OpenAI-compatible provider config.
Most providers use ``Authorization: Bearer KEY``. Some (Xiaomi MiMo) use a
dedicated header like ``api-key: KEY`` — supported via the
``auth_header_name`` key in PROVIDERS (defaults to ``Authorization``).
"""
header_name = cfg.get("auth_header_name") or "Authorization"
header_value = cfg["auth_header"].format(api_key=cfg["api_key"])
if header_name == "Authorization" and not header_value.lower().startswith("bearer "):
header_value = "Bearer " + header_value
return {
header_name: header_value,
"Content-Type": "application/json",
}
async def _call_deepseek_openrouter(prompt: str, system: str, provider: ProviderName | None = None,
temperature: float = 0.7, max_tokens: int = 2048) -> str:
"""Call OpenAI-compatible API (DeepSeek, OpenRouter, Xiaomi MiMo, etc.)."""
@@ -119,20 +176,7 @@ async def _call_deepseek_openrouter(prompt: str, system: str, provider: Provider
# Debug: log masked key to diagnose 401
key_preview = cfg["api_key"][:8] + "..." + cfg["api_key"][-4:] if len(cfg["api_key"]) > 12 else "***"
logger.info(f"AI call: provider={cfg['name']} model={cfg['model']} key={key_preview}")
# Most providers use "Authorization: Bearer KEY". Some (Xiaomi MiMo) use a
# dedicated header like "api-key: KEY". We support both via the
# `auth_header_name` key in PROVIDERS — defaults to "Authorization".
header_name = cfg.get("auth_header_name") or "Authorization"
header_value = cfg["auth_header"].format(api_key=cfg["api_key"])
# If the auth_header template doesn't include "Bearer " but the default
# header is Authorization, prepend it. This preserves backward compatibility
# for providers that store just the raw key.
if header_name == "Authorization" and not header_value.lower().startswith("bearer "):
header_value = "Bearer " + header_value
headers = {
header_name: header_value,
"Content-Type": "application/json",
}
headers = _build_headers(cfg)
payload = {
"model": cfg["model"],
"messages": [
+382
View File
@@ -0,0 +1,382 @@
"""Provider-agnostic chat completion with native tool (function) calling.
This module complements ``backend.ai`` (which handles the 16 stateless editor
actions). It exposes a single ``chat_completion`` entry point used by the
in-app agent loop:
- OpenAI-compatible providers (DeepSeek, OpenRouter, NVIDIA, QwenCloud,
Xiaomi, Mistral) use the ``tools`` / ``tool_calls`` protocol.
- Google Gemini uses ``functionDeclarations`` / ``functionCall``.
When a provider rejects the ``tools`` parameter (model without function
calling support), the call is transparently retried without tools so callers
degrade to plain chat (fallback protocol, see ``docs/AI_ARCHITECTURE_GUIDE.md``).
"""
from __future__ import annotations
import json
import logging
import re
from collections.abc import AsyncIterator
from dataclasses import dataclass, field
from typing import Any
import httpx
from backend.ai import PROVIDERS, _build_headers, _get_provider_config
logger = logging.getLogger("obsigate.ai_chat")
# Status codes that usually mean "tools not supported by this model".
_TOOLS_UNSUPPORTED_STATUS = {400, 404, 422}
@dataclass
class ToolCall:
"""A single tool invocation requested by the model."""
id: str
name: str
arguments: dict[str, Any] = field(default_factory=dict)
@dataclass
class LLMResponse:
"""Normalized provider response (text and/or tool calls)."""
content: str | None = None
tool_calls: list[ToolCall] = field(default_factory=list)
provider: str = ""
model: str = ""
@property
def has_tool_calls(self) -> bool:
return bool(self.tool_calls)
def _parse_arguments(raw: Any) -> dict[str, Any]:
"""Parse tool-call arguments that may arrive as a JSON string or dict."""
if isinstance(raw, dict):
return raw
if isinstance(raw, str) and raw.strip():
try:
parsed = json.loads(raw)
return parsed if isinstance(parsed, dict) else {"value": parsed}
except json.JSONDecodeError:
return {"_raw": raw}
return {}
async def chat_completion(
messages: list[dict[str, Any]],
*,
tools: list[dict[str, Any]] | None = None,
provider: str | None = None,
model: str | None = None,
temperature: float = 0.3,
max_tokens: int = 4096,
) -> LLMResponse:
"""Run a chat completion, optionally with tool calling.
Args:
messages: OpenAI-style messages (``system`` / ``user`` / ``assistant`` /
``tool``). Assistant messages may carry ``tool_calls``.
tools: OpenAI-style tool schemas (see ``get_tool_schemas``).
provider: Provider override; falls back to the configured default.
model: Model override; falls back to the provider's default model.
temperature: Sampling temperature.
max_tokens: Maximum output tokens.
Returns:
:class:`LLMResponse` with the text content and/or requested tool calls.
"""
cfg = _get_provider_config(provider) # type: ignore[arg-type]
name = cfg["name"]
resolved_model = model or cfg["model"]
if name == "gemini":
return await _gemini_chat(messages, tools, resolved_model, temperature, max_tokens)
return await _openai_chat(messages, tools, cfg, resolved_model, temperature, max_tokens)
async def _openai_chat(
messages: list[dict[str, Any]],
tools: list[dict[str, Any]] | None,
cfg: dict[str, Any],
model: str,
temperature: float,
max_tokens: int,
) -> LLMResponse:
"""Call an OpenAI-compatible ``/chat/completions`` endpoint."""
headers = _build_headers(cfg)
url = f"{cfg['base_url']}/chat/completions"
def _build_payload(use_tools: list[dict[str, Any]] | None) -> dict[str, Any]:
payload: dict[str, Any] = {
"model": model,
"messages": messages,
"temperature": temperature,
"max_tokens": max_tokens,
}
if use_tools:
payload["tools"] = use_tools
payload["tool_choice"] = "auto"
return payload
try:
data = await _post_json(url, headers, _build_payload(tools))
except httpx.HTTPStatusError as e:
if tools and e.response.status_code in _TOOLS_UNSUPPORTED_STATUS:
logger.warning(f"Provider '{cfg['name']}' rejected tools — retrying without tools")
data = await _post_json(url, headers, _build_payload(None))
else:
raise
message = data["choices"][0]["message"]
content = (message.get("content") or "").strip() or None
tool_calls: list[ToolCall] = []
for idx, tc in enumerate(message.get("tool_calls") or []):
fn = tc.get("function", {}) or {}
tool_calls.append(ToolCall(
id=tc.get("id") or f"call_{idx}",
name=fn.get("name", ""),
arguments=_parse_arguments(fn.get("arguments")),
))
return LLMResponse(content=content, tool_calls=tool_calls, provider=cfg["name"], model=model)
def _gemini_tools(tools: list[dict[str, Any]] | None) -> list[dict[str, Any]] | None:
"""Convert OpenAI-style tool schemas to Gemini ``functionDeclarations``."""
if not tools:
return None
declarations = []
for spec in tools:
fn = spec.get("function", spec)
declarations.append({
"name": fn.get("name", ""),
"description": fn.get("description", ""),
"parameters": fn.get("parameters", {"type": "object", "properties": {}}),
})
return [{"functionDeclarations": declarations}]
_DATA_URL_RE = re.compile(r"^data:([^;,]+);base64,(.*)$", re.DOTALL)
def _content_to_gemini_parts(content: Any) -> list[dict[str, Any]]:
"""Convert OpenAI-style message content to Gemini ``parts``.
Accepts either a plain string or a multimodal content array
(``[{"type": "text", ...}, {"type": "image_url", ...}]``). Data URLs are
turned into ``inlineData`` parts so images can be sent to vision models.
"""
if content is None:
return [{"text": ""}]
if isinstance(content, str):
return [{"text": content}]
parts: list[dict[str, Any]] = []
if isinstance(content, list):
for item in content:
if isinstance(item, str):
parts.append({"text": item})
continue
if not isinstance(item, dict):
continue
item_type = item.get("type")
if item_type == "text":
parts.append({"text": item.get("text", "")})
elif item_type == "image_url":
url = (item.get("image_url") or {}).get("url", "")
match = _DATA_URL_RE.match(url or "")
if match:
parts.append({
"inlineData": {"mimeType": match.group(1), "data": match.group(2)},
})
elif url:
parts.append({"fileData": {"fileUri": url}})
if not parts:
parts.append({"text": ""})
return parts
def _gemini_contents(messages: list[dict[str, Any]]) -> tuple[str, list[dict[str, Any]]]:
"""Split OpenAI-style messages into Gemini ``system`` text + ``contents``."""
system_parts: list[str] = []
contents: list[dict[str, Any]] = []
for msg in messages:
role = msg.get("role")
if role == "system":
content = msg.get("content") or ""
system_parts.append(content if isinstance(content, str) else "")
elif role == "tool":
contents.append({
"role": "user",
"parts": [{
"functionResponse": {
"name": msg.get("name", ""),
"response": {"content": msg.get("content", "")},
},
}],
})
else:
contents.append({
"role": "user" if role == "user" else "model",
"parts": _content_to_gemini_parts(msg.get("content")),
})
return "\n".join(p for p in system_parts if p).strip(), contents
async def _gemini_chat(
messages: list[dict[str, Any]],
tools: list[dict[str, Any]] | None,
model: str,
temperature: float,
max_tokens: int,
) -> LLMResponse:
"""Call Gemini's ``generateContent`` endpoint with optional tools."""
cfg = PROVIDERS["gemini"]
system, contents = _gemini_contents(messages)
payload: dict[str, Any] = {
"contents": contents,
"generationConfig": {"temperature": temperature, "maxOutputTokens": max_tokens},
}
if system:
payload["system_instruction"] = {"parts": [{"text": system}]}
gemini_tools = _gemini_tools(tools)
if gemini_tools:
payload["tools"] = gemini_tools
url = f"{cfg['base_url']}/models/{model}:generateContent?key={cfg['api_key']}"
data = await _post_json(url, None, payload)
parts = data["candidates"][0]["content"].get("parts", [])
content = "".join(p.get("text", "") for p in parts if "text" in p).strip() or None
tool_calls: list[ToolCall] = []
for idx, part in enumerate(parts):
fc = part.get("functionCall")
if fc:
tool_calls.append(ToolCall(
id=f"call_{idx}",
name=fc.get("name", ""),
arguments=_parse_arguments(fc.get("args")),
))
return LLMResponse(content=content, tool_calls=tool_calls, provider="gemini", model=model)
async def _post_json(url: str, headers: dict[str, str] | None, payload: dict[str, Any]) -> dict[str, Any]:
"""POST JSON and return the parsed body, raising on HTTP errors."""
async with httpx.AsyncClient(timeout=120.0) as client:
resp = await client.post(url, headers=headers or {}, json=payload)
resp.raise_for_status()
return resp.json()
# ── Streaming (SSE token stream) ────────────────────────────────────────
async def stream_completion(
messages: list[dict[str, Any]],
*,
provider: str | None = None,
model: str | None = None,
temperature: float = 0.3,
max_tokens: int = 4096,
) -> AsyncIterator[str]:
"""Yield content deltas from a chat completion as they arrive.
Only text content is streamed (no tool calling): this backs the plain
``/api/ai/bookslm/chat`` endpoint. The tool-calling ``/agent`` endpoint
keeps using :func:`chat_completion` because tool calls need the complete
response before they can be executed.
"""
cfg = _get_provider_config(provider) # type: ignore[arg-type]
resolved_model = model or cfg["model"]
if cfg["name"] == "gemini":
stream = _gemini_stream(messages, resolved_model, temperature, max_tokens)
else:
stream = _openai_stream(messages, cfg, resolved_model, temperature, max_tokens)
async for chunk in stream:
yield chunk
async def _openai_stream(
messages: list[dict[str, Any]],
cfg: dict[str, Any],
model: str,
temperature: float,
max_tokens: int,
) -> AsyncIterator[str]:
"""Stream an OpenAI-compatible ``/chat/completions`` response."""
headers = _build_headers(cfg)
url = f"{cfg['base_url']}/chat/completions"
payload: dict[str, Any] = {
"model": model,
"messages": messages,
"temperature": temperature,
"max_tokens": max_tokens,
"stream": True,
}
async with httpx.AsyncClient(timeout=120.0) as client, client.stream("POST", url, headers=headers, json=payload) as resp:
resp.raise_for_status()
async for line in resp.aiter_lines():
if not line or not line.startswith("data:"):
continue
raw = line[5:].strip()
if raw == "[DONE]":
break
try:
chunk = json.loads(raw)
except json.JSONDecodeError:
continue
choices = chunk.get("choices") or []
if not choices:
continue
delta = choices[0].get("delta") or {}
content = delta.get("content")
if content:
yield content
async def _gemini_stream(
messages: list[dict[str, Any]],
model: str,
temperature: float,
max_tokens: int,
) -> AsyncIterator[str]:
"""Stream Gemini's ``streamGenerateContent`` response (SSE)."""
cfg = PROVIDERS["gemini"]
system, contents = _gemini_contents(messages)
payload: dict[str, Any] = {
"contents": contents,
"generationConfig": {"temperature": temperature, "maxOutputTokens": max_tokens},
}
if system:
payload["system_instruction"] = {"parts": [{"text": system}]}
url = f"{cfg['base_url']}/models/{model}:streamGenerateContent?alt=sse&key={cfg['api_key']}"
async with httpx.AsyncClient(timeout=120.0) as client, client.stream("POST", url, json=payload) as resp:
resp.raise_for_status()
async for line in resp.aiter_lines():
if not line or not line.startswith("data:"):
continue
raw = line[5:].strip()
if not raw:
continue
try:
chunk = json.loads(raw)
except json.JSONDecodeError:
continue
for candidate in chunk.get("candidates") or []:
for part in candidate.get("content", {}).get("parts", []):
text = part.get("text")
if text:
yield text
+175
View File
@@ -0,0 +1,175 @@
# backend/ai_history.py
"""Persistent assistant conversation history (#95).
Each authenticated user owns a flat list of conversation sessions tagged with
their context (mode / vault / directory / documents). Sessions survive page
reloads and are the source of truth for the panel history menu (#96 sidebar
"Historique IA" reads the same store).
Format of a session (JS/JSON shape kept identical to the client, minus
transient fields):
{
"id": "s-…",
"title": "…",
"mode": "directory" | "documents" | "general",
"vault": "…" | None,
"directory": "…" | "",
"documents": [{"vault": "…", "path": "…"}],
"context": "directory-…", # _contextKey() of the assistant
"createdAt": 1234567890,
"updatedAt": 1234567890,
"messages": [{"role": "user|assistant", "content": "…"}]
}
Sessions are capped per user (see MAX_SESSIONS); the oldest ones are dropped
when the cap is reached.
"""
import json
import logging
import shutil
from pathlib import Path
from typing import Any
logger = logging.getLogger("obsigate.ai_history")
AI_HISTORY_DIR = Path("data/ai_history")
MAX_SESSIONS = 200
def _get_user_file(username: str) -> Path:
AI_HISTORY_DIR.mkdir(parents=True, exist_ok=True)
return AI_HISTORY_DIR / f"{username}.json"
def _read_sessions(username: str) -> list[dict[str, Any]]:
path = _get_user_file(username)
if not path.exists():
return []
try:
data = json.loads(path.read_text(encoding="utf-8"))
except Exception as e: # pragma: no cover - defensive I/O guard
logger.error(f"Failed to read AI history for {username}: {e}")
return []
if not isinstance(data, list):
return []
return [s for s in data if isinstance(s, dict)]
def _write_sessions(username: str, sessions: list[dict[str, Any]]) -> None:
path = _get_user_file(username)
try:
tmp = path.with_suffix(".tmp")
tmp.write_text(
json.dumps(sessions, indent=2, ensure_ascii=False),
encoding="utf-8",
)
shutil.move(str(tmp), str(path))
except Exception as e: # pragma: no cover - defensive I/O guard
logger.error(f"Failed to write AI history for {username}: {e}")
def _summary(session: dict[str, Any]) -> dict[str, Any]:
"""Compact representation (no messages) used by the list endpoint."""
messages = session.get("messages") or []
preview = ""
for msg in reversed(messages):
content = (msg.get("content") or "").strip() if isinstance(msg, dict) else ""
if content:
preview = content[:120]
break
return {
"id": session.get("id", ""),
"title": session.get("title", "") or "",
"mode": session.get("mode", "general"),
"vault": session.get("vault"),
"directory": session.get("directory", ""),
"context": session.get("context", ""),
"createdAt": session.get("createdAt", 0),
"updatedAt": session.get("updatedAt", session.get("createdAt", 0)),
"message_count": len(messages),
"preview": preview,
}
def list_sessions(username: str, *, include_messages: bool = False) -> list[dict[str, Any]]:
"""Return the user's sessions, most recently updated first.
With ``include_messages=False`` (default) a compact summary is returned;
the full conversation is fetched per id via :func:`get_session`.
"""
if not username:
return []
sessions = sorted(
_read_sessions(username),
key=lambda s: s.get("updatedAt") or s.get("createdAt") or 0,
reverse=True,
)
if include_messages:
return sessions
return [_summary(s) for s in sessions]
def get_session(username: str, session_id: str) -> dict[str, Any] | None:
if not username or not session_id:
return None
for session in _read_sessions(username):
if session.get("id") == session_id:
return session
return None
def upsert_session(username: str, session: dict[str, Any]) -> dict[str, Any] | None:
"""Create or update a conversation for the user.
Returns the stored session, or None when there is no valid id.
"""
if not username:
return None
session_id = (session.get("id") or "").strip()
if not session_id:
return None
now = session.get("updatedAt") or session.get("createdAt") or 0
stored = {
"id": session_id,
"title": session.get("title", "") or "",
"mode": session.get("mode") or "general",
"vault": session.get("vault"),
"directory": session.get("directory", ""),
"documents": session.get("documents") or [],
"context": session.get("context", ""),
"createdAt": session.get("createdAt") or now,
"updatedAt": now,
"messages": session.get("messages") or [],
}
sessions = _read_sessions(username)
replaced = False
for i, existing in enumerate(sessions):
if existing.get("id") == session_id:
sessions[i] = stored
replaced = True
break
if not replaced:
sessions.append(stored)
sessions.sort(key=lambda s: s.get("updatedAt") or s.get("createdAt") or 0, reverse=True)
if len(sessions) > MAX_SESSIONS:
logger.info(f"AI history cap reached for {username}: trimming to {MAX_SESSIONS}")
sessions = sessions[:MAX_SESSIONS]
_write_sessions(username, sessions)
return stored
def delete_session(username: str, session_id: str) -> bool:
if not username or not session_id:
return False
sessions = _read_sessions(username)
remaining = [s for s in sessions if s.get("id") != session_id]
if len(remaining) == len(sessions):
return False
_write_sessions(username, remaining)
return True
+40 -4
View File
@@ -2,11 +2,10 @@
import logging
from fastapi import APIRouter, HTTPException
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel, Field
from backend.ai import (
DEFAULT_PROVIDER,
PROVIDERS,
ai_change_tone,
ai_continue_writing,
@@ -24,13 +23,17 @@ from backend.ai import (
ai_simplify,
ai_summarize,
ai_translate,
get_default_provider,
)
from backend.auth.middleware import require_auth
from backend.model_capabilities import get_model_capabilities
from backend.schemas import AIStatusResponse
logger = logging.getLogger("obsigate.ai_routes")
router = APIRouter(prefix="/api/ai", tags=["AI"])
@router.get("/status")
@router.get("/status", response_model=AIStatusResponse)
async def api_status():
"""Check if AI is configured and which providers are available.
@@ -90,7 +93,7 @@ async def api_status():
return {
"configured": any(p["available"] for p in providers.values()),
"default_provider": DEFAULT_PROVIDER,
"default_provider": get_default_provider(),
"providers": providers,
"autocomplete": autocomplete,
}
@@ -231,3 +234,36 @@ async def api_inline_complete(req: AIRequest):
async def api_to_canvas(req: AIRequest):
"""Convert to Mermaid diagram or outline."""
return await _handle(ai_convert_to_canvas, req)
class ModelCapabilitiesResponse(BaseModel):
"""Capabilities of a single provider/model pair."""
provider: str
model: str
capabilities: dict[str, bool] = Field(
description="Flags: chat, embeddings, rerank, images, video, "
"audio_speech, audio_transcription, vision",
)
@router.get("/model-capabilities", response_model=ModelCapabilitiesResponse)
async def api_model_capabilities(
provider: str = Query(..., description="Provider identifier"),
model: str = Query("", description="Model identifier (optional)"),
current_user=Depends(require_auth),
):
"""Return the capability flags for a provider/model pair.
Two layers (BUG-044): flags the provider itself declares in its models
endpoint (Mistral ``capabilities``, OpenRouter ``architecture``) win, the
curated table in ``backend.model_capabilities`` fills the rest. The
declaration snapshot is populated by ``GET /api/config/ai-models``; when it
is cold (or the provider declares nothing) the curated table answers alone,
so this endpoint never performs a blocking provider call.
"""
return {
"provider": provider,
"model": model,
"capabilities": get_model_capabilities(provider, model),
}
+8 -3
View File
@@ -62,16 +62,21 @@ def create_access_token(user: dict) -> str:
return jwt.encode(payload, get_secret_key(), algorithm=ALGORITHM)
def create_refresh_token(username: str) -> tuple:
"""Create a JWT refresh token. Returns (token_string, jti)."""
def create_refresh_token(username: str, remember: bool = False) -> tuple:
"""Create a JWT refresh token. Returns (token_string, jti).
``remember`` is carried as a claim so token rotation can preserve the
30-day vs 7-day lifetime chosen at login.
"""
now = int(time.time())
jti = str(uuid.uuid4())
payload = {
"sub": username,
"jti": jti,
"iat": now,
"exp": now + REFRESH_TOKEN_EXPIRE_SECONDS,
"exp": now + (2592000 if remember else REFRESH_TOKEN_EXPIRE_SECONDS),
"type": "refresh",
"remember": remember,
}
return jwt.encode(payload, get_secret_key(), algorithm=ALGORITHM), jti
+23 -1
View File
@@ -8,7 +8,9 @@ import os
from fastapi import Depends, HTTPException, Request
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from .jwt_handler import decode_token
from backend.services.net import get_client_ip
from .jwt_handler import decode_token, is_token_revoked
from .user_store import get_user
logger = logging.getLogger("obsigate.auth.middleware")
@@ -42,6 +44,7 @@ def get_current_user(
"vaults": ["*"],
"active": True,
"_token_vaults": ["*"],
"_request_ip": get_client_ip(request),
}
token = None
@@ -57,12 +60,31 @@ def get_current_user(
if not payload or payload.get("type") != "access":
return None
# BUG-027: access tokens revoked at logout must be rejected immediately.
jti = payload.get("jti")
if jti and is_token_revoked(jti):
return None
user = get_user(payload["sub"])
if not user or not user.get("active"):
return None
# BUG-028: a password change invalidates every token issued before it.
pca = user.get("password_changed_at")
iat = payload.get("iat")
if pca is not None and iat is not None:
try:
if int(iat) < int(float(pca)):
return None
except (TypeError, ValueError):
return None
# Attach vault permissions from the token (snapshot at login time)
user["_token_vaults"] = payload.get("vaults", [])
# Attach the token id for per-token rate limiting (AI tool layer).
user["_token_jti"] = payload.get("jti")
# BUG-030: expose the real client IP to the audit log.
user["_request_ip"] = get_client_ip(request)
return user
+26
View File
@@ -13,6 +13,32 @@ ph = PasswordHasher(
salt_len=16,
)
# Password policy (BUG-028). Applied by the API validators at account creation
# and password change so the rules stay consistent across both paths.
MIN_PASSWORD_LENGTH = 8
MAX_PASSWORD_LENGTH = 128
def validate_password_strength(password: str) -> str:
"""Validate a plaintext password against the project policy.
Args:
password: Candidate password.
Returns:
The password unchanged when valid.
Raises:
ValueError: When the password is too short, too long or blank.
"""
if password is None or len(password) < MIN_PASSWORD_LENGTH:
raise ValueError(f"Minimum {MIN_PASSWORD_LENGTH} caractères")
if len(password) > MAX_PASSWORD_LENGTH:
raise ValueError(f"Maximum {MAX_PASSWORD_LENGTH} caractères")
if not password.strip():
raise ValueError("Le mot de passe ne peut pas être vide")
return password
def hash_password(password: str) -> str:
"""Hash a password with Argon2id."""
+126 -16
View File
@@ -8,9 +8,12 @@ import re
from fastapi import APIRouter, Body, Depends, HTTPException, Request, Response
from pydantic import BaseModel, validator
from backend.ratelimit import is_rate_limited
from backend.ratelimit import is_account_rate_limited, is_rate_limited
from backend.ratelimit import record_account_failure as rl_record_account_failure
from backend.ratelimit import record_account_success as rl_record_account_success
from backend.ratelimit import record_failure as rl_record_failure
from backend.ratelimit import record_success as rl_record_success
from backend.services.net import get_client_ip
from .jwt_handler import (
ACCESS_TOKEN_EXPIRE_SECONDS,
@@ -29,7 +32,7 @@ from .mfa import (
verify_totp,
)
from .middleware import is_auth_enabled, require_admin, require_auth
from .password import hash_password, verify_password
from .password import hash_password, validate_password_strength, verify_password
from .user_store import (
create_user,
delete_user,
@@ -61,9 +64,7 @@ class ChangePasswordRequest(BaseModel):
@validator("new_password")
def password_strength(cls, v):
if len(v) < 8:
raise ValueError("Minimum 8 caractères")
return v
return validate_password_strength(v)
class CreateUserRequest(BaseModel):
@@ -73,6 +74,10 @@ class CreateUserRequest(BaseModel):
role: str = "user"
vaults: list[str] = []
@validator("password")
def password_valid(cls, v):
return validate_password_strength(v)
@validator("username")
def username_valid(cls, v):
if not re.match(r"^[a-zA-Z0-9_-]{2,32}$", v):
@@ -93,6 +98,12 @@ class UpdateUserRequest(BaseModel):
password: str | None = None
role: str | None = None
@validator("password")
def password_valid(cls, v):
if v is None:
return v
return validate_password_strength(v)
# ── Public endpoints ──────────────────────────────────────────────────
@@ -128,16 +139,21 @@ async def login(body: LoginRequest, response: Response, request: Request):
raise HTTPException(403, "Compte désactivé")
# IP-based rate limiting (10 failures / 15 min per IP)
client_ip = request.client.host if request.client else "unknown"
client_ip = get_client_ip(request)
if is_rate_limited(client_ip):
raise HTTPException(429, "Trop de tentatives depuis cette adresse IP (15min)")
# BUG-031: per-account budget still applies when the attacker rotates IPs.
if is_account_rate_limited(body.username):
raise HTTPException(429, "Trop de tentatives sur ce compte (15min)")
if is_locked(body.username):
raise HTTPException(429, "Compte temporairement verrouillé (15min)")
if not verify_password(body.password, user["password_hash"]):
attempts = record_login_failure(body.username)
rl_attempts, rl_remaining = rl_record_failure(client_ip)
rl_record_account_failure(body.username)
remaining = max(0, 5 - attempts)
detail = "Identifiants invalides"
if 0 < remaining <= 2:
@@ -164,9 +180,10 @@ async def login(body: LoginRequest, response: Response, request: Request):
def _issue_tokens(user: dict, username: str, remember_me: bool, response: Response) -> dict:
"""Issue JWT tokens after successful authentication (password or MFA verified)."""
record_login_success(username)
rl_record_account_success(username)
access_token = create_access_token(user)
refresh_token, refresh_jti = create_refresh_token(username)
refresh_token, refresh_jti = create_refresh_token(username, remember=remember_me)
import os
max_age = 2592000 if remember_me else 604800 # 30d or 7d
@@ -208,6 +225,8 @@ async def refresh_token_endpoint(request: Request, response: Response):
"""Renew access token via refresh token cookie.
Called automatically by the frontend when the access token expires.
The refresh token is rotated on every use (BUG-027) and rejected if it
predates the user's last password change (BUG-028).
"""
refresh_tok = request.cookies.get("refresh_token")
if not refresh_tok:
@@ -224,11 +243,38 @@ async def refresh_token_endpoint(request: Request, response: Response):
if not user or not user.get("active"):
raise HTTPException(401, "Utilisateur introuvable ou inactif")
# BUG-028: reject refresh tokens issued before the last password change.
pca = user.get("password_changed_at")
iat = payload.get("iat")
if pca is not None and iat is not None:
try:
stale = int(iat) < int(float(pca))
except (TypeError, ValueError):
stale = True
if stale:
raise HTTPException(401, "Session expirée, veuillez vous reconnecter")
import os
secure = os.environ.get("OBSIGATE_SECURE_COOKIES", "false").lower() == "true"
remember_me = bool(payload.get("remember", False))
# BUG-027: rotate the refresh token — the old one is now single-use.
revoke_token(payload["jti"])
new_refresh_token, _new_jti = create_refresh_token(user["username"], remember=remember_me)
max_age = 2592000 if remember_me else 604800
response.set_cookie(
key="refresh_token",
value=new_refresh_token,
max_age=max_age,
httponly=True,
samesite="strict",
secure=secure,
path="/api/auth/refresh",
)
new_access_token = create_access_token(user)
# Update cookies
import os
secure = os.environ.get("OBSIGATE_SECURE_COOKIES", "false").lower() == "true"
response.set_cookie(
key="access_token",
value=new_access_token,
@@ -251,7 +297,7 @@ async def logout(
request: Request,
response: Response,
):
"""Logout: revoke refresh token and delete cookies."""
"""Logout: revoke refresh and access tokens, then delete cookies."""
refresh_tok = request.cookies.get("refresh_token")
if refresh_tok:
payload = decode_token(refresh_tok)
@@ -261,6 +307,21 @@ async def logout(
except Exception:
pass # token already revoked
# BUG-027: revoke the access token too, otherwise it stays valid until expiry.
access_tok = None
auth_header = request.headers.get("authorization", "")
if auth_header.lower().startswith("bearer "):
access_tok = auth_header[7:].strip()
if not access_tok:
access_tok = request.cookies.get("access_token")
if access_tok:
access_payload = decode_token(access_tok)
if access_payload and access_payload.get("type") == "access":
try:
revoke_token(access_payload["jti"])
except Exception:
pass
response.delete_cookie("refresh_token", path="/api/auth/refresh")
response.delete_cookie("access_token", path="/")
response.delete_cookie("access_token", path="/api") # just in case
@@ -313,19 +374,51 @@ async def patch_me(req: UpdateMeRequest, current_user=Depends(require_auth)):
@router.post("/change-password")
async def change_password(
req: ChangePasswordRequest,
response: Response,
current_user=Depends(require_auth),
):
"""Change own password."""
"""Change own password.
BUG-028: changing the password invalidates all previously issued tokens;
a fresh pair is issued to keep the current session alive.
"""
user = get_user(current_user["username"])
assert user is not None, f"User {current_user['username']} not found"
if not verify_password(req.current_password, user["password_hash"]):
raise HTTPException(400, "Mot de passe actuel incorrect")
update_user(current_user["username"], {"password": req.new_password})
return {"message": "Mot de passe mis à jour"}
updated = get_user(current_user["username"])
result: dict = {"message": "Mot de passe mis à jour"}
if updated is not None:
result.update(_issue_tokens(updated, updated["username"], False, response))
return result
# ── MFA endpoints ────────────────────────────────────────────────────
def _enforce_mfa_rate_limit(request: Request, username: str) -> str:
"""Reject MFA attempts from a rate-limited IP or on a locked account.
BUG-023: the second-factor endpoints were previously unprotected, making
the 6-digit TOTP brute-forceable. Returns the resolved client IP.
"""
client_ip = get_client_ip(request)
if is_rate_limited(client_ip):
raise HTTPException(429, "Trop de tentatives depuis cette adresse IP (15min)")
if is_account_rate_limited(username):
raise HTTPException(429, "Trop de tentatives sur ce compte (15min)")
if is_locked(username):
raise HTTPException(429, "Compte temporairement verrouillé (15min)")
return client_ip
def _record_mfa_failure(client_ip: str, username: str) -> None:
"""Record a failed MFA attempt for the IP, the account and the lockout."""
record_login_failure(username)
rl_record_failure(client_ip)
rl_record_account_failure(username)
class MfaVerifyRequest(BaseModel):
username: str
code: str
@@ -379,6 +472,8 @@ async def mfa_totp_enable(
from .user_store import get_user, update_user
user = get_user(current_user["username"])
if user is None:
raise HTTPException(404, "Utilisateur introuvable")
secret = user.get("mfa_secret_pending")
if not secret:
raise HTTPException(400, "Aucune configuration MFA en cours. Commencez par /mfa/totp/setup")
@@ -415,6 +510,8 @@ async def mfa_totp_disable(
from .user_store import get_user, update_user
user = get_user(current_user["username"])
if user is None:
raise HTTPException(404, "Utilisateur introuvable")
if not user.get("mfa_enabled"):
raise HTTPException(400, "MFA non activé")
@@ -485,6 +582,8 @@ async def mfa_webauthn_register(
from .webauthn_mfa import complete_registration
user = get_user(current_user["username"])
if user is None:
raise HTTPException(404, "Utilisateur introuvable")
try:
record = complete_registration(current_user["username"], req.credential,
label=req.label)
@@ -529,6 +628,8 @@ def user_credentials_response(creds: list[dict]) -> list[dict]:
async def mfa_webauthn_list(current_user=Depends(require_auth)):
from .user_store import get_user
user = get_user(current_user["username"])
if user is None:
raise HTTPException(404, "Utilisateur introuvable")
return {"credentials": user_credentials_response(user.get("webauthn_credentials", []))}
@@ -542,6 +643,8 @@ async def mfa_webauthn_remove(
from .webauthn_mfa import clear_pending
user = get_user(current_user["username"])
if user is None:
raise HTTPException(404, "Utilisateur introuvable")
if not verify_password(req.password, user["password_hash"]):
raise HTTPException(400, "Mot de passe incorrect")
@@ -591,6 +694,8 @@ async def mfa_webauthn_verify(
from .user_store import get_user, update_user
from .webauthn_mfa import complete_authentication
client_ip = _enforce_mfa_rate_limit(request, body.username)
user = get_user(body.username)
if not user:
hash_password("dummy_timing_protection")
@@ -606,8 +711,10 @@ async def mfa_webauthn_verify(
raise ValueError("Credential non enregistré")
new_count = complete_authentication(body.username, body.credential, stored)
except ValueError as e:
_record_mfa_failure(client_ip, body.username)
raise HTTPException(401, str(e))
except Exception as e:
_record_mfa_failure(client_ip, body.username)
logger.warning(f"WebAuthn verification failed for {body.username}: {e}")
raise HTTPException(401, "Vérification WebAuthn échouée")
@@ -617,7 +724,6 @@ async def mfa_webauthn_verify(
c["sign_count"] = new_count
update_user(body.username, {"webauthn_credentials": updated})
client_ip = request.client.host if request.client else "unknown"
rl_record_success(client_ip)
logger.info(f"User '{body.username}' logged in via WebAuthn")
return _issue_tokens(user, body.username, body.remember_me, response)
@@ -645,6 +751,8 @@ async def mfa_totp_verify(body: MfaVerifyRequest, response: Response, request: R
"""
from .user_store import get_user
client_ip = _enforce_mfa_rate_limit(request, body.username)
user = get_user(body.username)
if not user:
# Timing-safe: simulate work
@@ -655,10 +763,10 @@ async def mfa_totp_verify(body: MfaVerifyRequest, response: Response, request: R
raise HTTPException(400, "MFA non activé pour cet utilisateur")
if not verify_totp(user["mfa_secret"], body.code):
_record_mfa_failure(client_ip, body.username)
raise HTTPException(401, "Code TOTP invalide")
# Clear IP rate limit on success
client_ip = request.client.host if request.client else "unknown"
rl_record_success(client_ip)
return _issue_tokens(user, body.username, body.remember_me, response)
@@ -672,6 +780,8 @@ async def mfa_recovery_login(body: MfaRecoveryRequest, response: Response, reque
"""
from .user_store import get_user, update_user
client_ip = _enforce_mfa_rate_limit(request, body.username)
user = get_user(body.username)
if not user:
hash_password("dummy_timing_protection")
@@ -686,6 +796,7 @@ async def mfa_recovery_login(body: MfaRecoveryRequest, response: Response, reque
idx = verify_recovery_code(body.recovery_code, hashed_codes)
if idx is None:
_record_mfa_failure(client_ip, body.username)
raise HTTPException(401, "Code de récupération invalide")
# Remove used recovery code (single-use)
@@ -693,7 +804,6 @@ async def mfa_recovery_login(body: MfaRecoveryRequest, response: Response, reque
update_user(body.username, {"mfa_recovery_codes": hashed_codes})
# Clear IP rate limit
client_ip = request.client.host if request.client else "unknown"
rl_record_success(client_ip)
logger.info(f"User '{body.username}' logged in via recovery code")
+64 -49
View File
@@ -6,6 +6,7 @@
import json
import logging
import shutil
import threading
import uuid
from datetime import datetime, timedelta, timezone
from pathlib import Path
@@ -16,6 +17,12 @@ logger = logging.getLogger("obsigate.auth.users")
USERS_FILE = Path("data/users.json")
# Serialises read-modify-write cycles on users.json. ``RLock`` because a few
# helpers (e.g. ``record_login_failure``) call other mutators while holding it.
# BUG-029: without this, concurrent MFA enable + password change could lose one
# of the two updates (last writer wins).
_users_lock = threading.RLock()
def _read() -> dict:
"""Read users.json. Returns empty structure if file doesn't exist."""
@@ -75,26 +82,28 @@ def create_user(
display_name: str | None = None,
) -> dict:
"""Create a new user. Raises ValueError if username already taken."""
data = _read()
if username in data["users"]:
raise ValueError(f"User '{username}' already exists")
with _users_lock:
data = _read()
if username in data["users"]:
raise ValueError(f"User '{username}' already exists")
user = {
"id": str(uuid.uuid4()),
"username": username,
"display_name": display_name or username,
"password_hash": hash_password(password),
"role": role,
"vaults": vaults or [],
"active": True,
"language": "fr", # default UI language
"created_at": datetime.now(timezone.utc).isoformat(),
"last_login": None,
"failed_attempts": 0,
"locked_until": None,
}
data["users"][username] = user
_write(data)
user = {
"id": str(uuid.uuid4()),
"username": username,
"display_name": display_name or username,
"password_hash": hash_password(password),
"role": role,
"vaults": vaults or [],
"active": True,
"language": "fr", # default UI language
"created_at": datetime.now(timezone.utc).isoformat(),
"password_changed_at": datetime.now(timezone.utc).timestamp(),
"last_login": None,
"failed_attempts": 0,
"locked_until": None,
}
data["users"][username] = user
_write(data)
logger.info(f"Created user '{username}' (role={role})")
return {k: v for k, v in user.items() if k != "password_hash"}
@@ -105,28 +114,33 @@ def update_user(username: str, updates: dict) -> dict:
Forbidden fields (id, username, created_at) are silently ignored.
If 'password' is in updates, it's hashed and stored as password_hash.
"""
data = _read()
if username not in data["users"]:
raise ValueError(f"User '{username}' not found")
with _users_lock:
data = _read()
if username not in data["users"]:
raise ValueError(f"User '{username}' not found")
forbidden = {"id", "username", "created_at"}
safe_updates = {k: v for k, v in updates.items() if k not in forbidden}
forbidden = {"id", "username", "created_at"}
safe_updates = {k: v for k, v in updates.items() if k not in forbidden}
if "password" in safe_updates:
safe_updates["password_hash"] = hash_password(safe_updates.pop("password"))
if "password" in safe_updates:
safe_updates["password_hash"] = hash_password(safe_updates.pop("password"))
# BUG-028: invalidate every token issued before this change.
safe_updates["password_changed_at"] = datetime.now(timezone.utc).timestamp()
data["users"][username].update(safe_updates)
_write(data)
return {k: v for k, v in data["users"][username].items() if k != "password_hash"}
data["users"][username].update(safe_updates)
_write(data)
result = {k: v for k, v in data["users"][username].items() if k != "password_hash"}
return result
def delete_user(username: str):
"""Delete a user. Raises ValueError if not found."""
data = _read()
if username not in data["users"]:
raise ValueError(f"User '{username}' not found")
del data["users"][username]
_write(data)
with _users_lock:
data = _read()
if username not in data["users"]:
raise ValueError(f"User '{username}' not found")
del data["users"][username]
_write(data)
logger.info(f"Deleted user '{username}'")
@@ -144,24 +158,25 @@ def record_login_failure(username: str) -> int:
After 5 failures, locks the account for 15 minutes.
"""
data = _read()
user = data["users"].get(username)
if not user:
return 0
with _users_lock:
data = _read()
user = data["users"].get(username)
if not user:
return 0
attempts = user.get("failed_attempts", 0) + 1
updates = {"failed_attempts": attempts}
attempts = user.get("failed_attempts", 0) + 1
updates = {"failed_attempts": attempts}
# Lock after 5 failed attempts (15 minutes)
if attempts >= 5:
locked_until = (
datetime.now(timezone.utc) + timedelta(minutes=15)
).isoformat()
updates["locked_until"] = locked_until
logger.warning(f"Account '{username}' locked after {attempts} failed attempts")
# Lock after 5 failed attempts (15 minutes)
if attempts >= 5:
locked_until = (
datetime.now(timezone.utc) + timedelta(minutes=15)
).isoformat()
updates["locked_until"] = locked_until
logger.warning(f"Account '{username}' locked after {attempts} failed attempts")
update_user(username, updates)
return attempts
update_user(username, updates)
return attempts
def is_locked(username: str) -> bool:
+376 -8
View File
@@ -5,9 +5,11 @@ redaction, builds a system prompt with file contents, and caches results
for repeated queries.
"""
import base64
import hashlib
import json
import logging
import mimetypes
import os
import time
from pathlib import Path
@@ -21,6 +23,10 @@ logger = logging.getLogger("obsigate.bookslm")
BOOKSLM_MAX_FILES = int(os.getenv("BOOKSLM_MAX_FILES", "200"))
BOOKSLM_MAX_TOTAL_CHARS = int(os.getenv("BOOKSLM_MAX_TOTAL_CHARS", "200000"))
BOOKSLM_MAX_FILE_CHARS = int(os.getenv("BOOKSLM_MAX_FILE_CHARS", "30000"))
# Maximum size of an image sent to a vision model (bytes, before base64).
BOOKSLM_MAX_IMAGE_BYTES = int(os.getenv("BOOKSLM_MAX_IMAGE_BYTES", "10000000"))
IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".gif", ".webp", ".bmp", ".avif"}
# ── Cache ──
_cache: dict[str, dict[str, Any]] = {}
@@ -165,6 +171,9 @@ def collect_directory_context(vault_path: Path, directory: str) -> dict[str, Any
"total_chars": total_chars,
"file_count": len(collected),
"directory_tree": dir_tree,
"max_total_chars": BOOKSLM_MAX_TOTAL_CHARS,
"max_files": BOOKSLM_MAX_FILES,
"scope": "directory",
}
# Store in cache
@@ -173,6 +182,219 @@ def collect_directory_context(vault_path: Path, directory: str) -> dict[str, Any
return result
def _file_entry(target: Path, rel_path: str, remaining: int) -> dict[str, Any] | None:
"""Read, redact and truncate a single file into a context entry."""
suffix = target.suffix.lower()
try:
if suffix == ".pdf":
from backend.pdf_reader import extract_pdf_text
content = extract_pdf_text(target)
file_type = "pdf"
else:
content = target.read_text(encoding="utf-8", errors="replace")
file_type = "markdown"
except Exception as e:
logger.warning(f"Cannot read {rel_path}: {e}")
return None
content = redact_file_content(content, rel_path)
if len(content) > BOOKSLM_MAX_FILE_CHARS:
content = content[:BOOKSLM_MAX_FILE_CHARS] + "\n\n[... tronqué]"
if len(content) > remaining:
content = content[:remaining] + "\n\n[... tronqué]"
return {
"path": rel_path,
"title": target.stem.replace("-", " ").replace("_", " ").title(),
"content": content,
"type": file_type,
}
def collect_files_context(
vault_path: Path,
rel_paths: list[str],
scope: str = "documents",
) -> dict[str, Any]:
"""Collect an explicit list of files (open documents) as AI context.
Unlike :func:`collect_directory_context`, this reads only the requested
relative paths (markdown or PDF), never the whole directory. Paths outside
the vault are silently ignored.
Args:
vault_path: Absolute path to the vault root.
rel_paths: Relative file paths within the vault (in display order).
scope: Context label exposed to the UI ("documents").
Returns:
Same shape as :func:`collect_directory_context`.
"""
vault_resolved = vault_path.resolve()
collected: list[dict[str, Any]] = []
total_chars = 0
seen: set[str] = set()
for rel in rel_paths:
if len(collected) >= BOOKSLM_MAX_FILES or total_chars >= BOOKSLM_MAX_TOTAL_CHARS:
break
if not rel or rel in seen:
continue
seen.add(rel)
try:
target = (vault_resolved / rel).resolve()
target.relative_to(vault_resolved)
except (ValueError, OSError):
continue
if not target.is_file():
continue
entry = _file_entry(target, rel, BOOKSLM_MAX_TOTAL_CHARS - total_chars)
if entry is None:
continue
collected.append(entry)
total_chars += len(entry["content"])
return {
"files": collected,
"total_chars": total_chars,
"file_count": len(collected),
"directory_tree": "",
"max_total_chars": BOOKSLM_MAX_TOTAL_CHARS,
"max_files": BOOKSLM_MAX_FILES,
"scope": scope,
}
def collect_adhoc_context(
vault_path: Path,
files: list[str] | None = None,
directories: list[str] | None = None,
scope: str = "general",
) -> dict[str, Any]:
"""Collect an ad-hoc mix of explicit files and directories.
Backs the assistant ``@`` command: the user can attach extra files and
directories to the current context without changing the base mode. Paths
outside the vault are ignored by the underlying collectors.
Args:
vault_path: Absolute path to the vault root.
files: Relative file paths to include.
directories: Relative directory paths to include (recursive).
scope: Context label exposed to the UI.
Returns:
Same shape as :func:`collect_directory_context`.
"""
vault_resolved = vault_path.resolve()
collected: list[dict[str, Any]] = []
seen: set[str] = set()
total_chars = 0
def _add(entries: list[dict[str, Any]]) -> None:
nonlocal total_chars
for entry in entries:
if len(collected) >= BOOKSLM_MAX_FILES or total_chars >= BOOKSLM_MAX_TOTAL_CHARS:
return
path = entry.get("path")
if not path or path in seen:
continue
seen.add(path)
collected.append(entry)
total_chars += len(entry.get("content", ""))
if files:
_add(collect_files_context(vault_resolved, files, scope=scope)["files"])
for directory in directories or []:
_add(collect_directory_context(vault_resolved, directory)["files"])
return {
"files": collected,
"total_chars": total_chars,
"file_count": len(collected),
"directory_tree": "",
"max_total_chars": BOOKSLM_MAX_TOTAL_CHARS,
"max_files": BOOKSLM_MAX_FILES,
"scope": scope,
}
def merge_contexts(base: dict[str, Any], extra: dict[str, Any]) -> dict[str, Any]:
"""Merge two context payloads, de-duplicating files by path."""
files: list[dict[str, Any]] = []
seen: set[str] = set()
total_chars = 0
for entry in list(base.get("files", [])) + list(extra.get("files", [])):
path = entry.get("path")
if not path or path in seen:
continue
if len(files) >= BOOKSLM_MAX_FILES or total_chars >= BOOKSLM_MAX_TOTAL_CHARS:
break
seen.add(path)
files.append(entry)
total_chars += len(entry.get("content", ""))
tree = base.get("directory_tree", "") or extra.get("directory_tree", "")
scope = extra.get("scope") or base.get("scope", "general")
return {
"files": files,
"total_chars": total_chars,
"file_count": len(files),
"directory_tree": tree,
"max_total_chars": BOOKSLM_MAX_TOTAL_CHARS,
"max_files": BOOKSLM_MAX_FILES,
"scope": scope,
}
def is_image_path(path: str) -> bool:
"""True when the path has a supported image extension."""
return Path(path or "").suffix.lower() in IMAGE_EXTENSIONS
def load_vault_image_data_url(vault_path: Path, rel_path: str) -> str | None:
"""Read a vault image and return it as a ``data:`` URL for vision models.
Returns ``None`` when the path is outside the vault, missing, too large,
or not a supported image.
"""
if not rel_path or not is_image_path(rel_path):
return None
vault_resolved = vault_path.resolve()
try:
target = (vault_resolved / rel_path).resolve()
target.relative_to(vault_resolved)
except (ValueError, OSError):
return None
if not target.is_file():
return None
try:
if target.stat().st_size > BOOKSLM_MAX_IMAGE_BYTES:
logger.warning("Image too large to send: %s", rel_path)
return None
raw = target.read_bytes()
except OSError as exc:
logger.warning("Cannot read image %s: %s", rel_path, exc)
return None
mime = mimetypes.guess_type(str(target))[0] or "image/png"
encoded = base64.b64encode(raw).decode("ascii")
return f"data:{mime};base64,{encoded}"
def empty_context(scope: str = "general") -> dict[str, Any]:
"""Return an empty context payload (used by the General assistant)."""
return {
"files": [],
"total_chars": 0,
"file_count": 0,
"directory_tree": "",
"max_total_chars": BOOKSLM_MAX_TOTAL_CHARS,
"max_files": BOOKSLM_MAX_FILES,
"scope": scope,
}
def _build_directory_tree(target_dir: Path, vault_root: Path) -> str:
"""Build a text representation of the directory tree (dirs + .md files)."""
lines: list[str] = []
@@ -192,11 +414,16 @@ def _build_directory_tree(target_dir: Path, vault_root: Path) -> str:
return "\n".join(lines)
def build_system_prompt(context: dict[str, Any]) -> str:
"""Build a system prompt for directory-scoped AI chat.
def build_system_prompt(context: dict[str, Any], scope: str = "directory", vault_name: str | None = None) -> str:
"""Build a system prompt for document-scoped AI chat.
Args:
context: Output of collect_directory_context().
context: Output of collect_directory_context()/collect_files_context().
scope: "directory" (whole folder) or "documents" (open files).
vault_name: Vault the context files belong to. When set, the prompt
states it explicitly with write-tool guidance (BUG-046: without
it the model invented vault names — e.g. "test" — and every
confirmed ``append_to_file``/``edit_file`` call failed).
Returns:
System prompt string with file contents.
@@ -211,12 +438,26 @@ def build_system_prompt(context: dict[str, Any]) -> str:
if est_tokens > 100_000:
token_warning = f"\n⚠️ Attention : le contexte est très volumineux (~{est_tokens:,} tokens estimés). Les réponses peuvent être moins précises.\n"
if scope == "documents":
role = (
"Tu es un assistant de recherche documentaire intégré à ObsiGate. "
"Tu réponds UNIQUEMENT en te basant sur les documents ouverts par l'utilisateur ci-dessous. "
"Cite tes sources avec le nom du fichier quand tu utilises une information. "
"Si l'information ne se trouve pas dans les documents, dis-le clairement."
)
context_label = "📄 Documents ouverts"
else:
role = (
"Tu es un assistant de recherche documentaire. "
"Tu réponds UNIQUEMENT en te basant sur les documents fournis ci-dessous. "
"Cite tes sources avec le nom du fichier quand tu utilises une information. "
"Si l'information ne se trouve pas dans les documents, dis-le clairement."
)
context_label = "📚 Contexte"
prompt = (
"Tu es un assistant de recherche documentaire. "
"Tu réponds UNIQUEMENT en te basant sur les documents fournis ci-dessous. "
"Cite tes sources avec le nom du fichier quand tu utilises une information. "
"Si l'information ne se trouve pas dans les documents, dis-le clairement."
f"\n\n📚 Contexte : {file_count} fichier(s) ({total_chars:,} caractères)"
f"{role}"
f"\n\n{context_label} : {file_count} fichier(s) ({total_chars:,} caractères)"
f"{token_warning}\n"
)
@@ -232,6 +473,133 @@ def build_system_prompt(context: dict[str, Any]) -> str:
prompt += "\nFin du contexte. Réponds à la question de l'utilisateur en te basant uniquement sur ces documents."
if vault_name:
prompt += (
f"\n\nCes documents appartiennent au vault « {vault_name} ». Quand tu utilises un outil "
"d'écriture (`append_to_file`, `edit_file`, `create_file`), passe TOUJOURS "
f"exactement `\"vault\": \"{vault_name}\"` (jamais un nom inventé) et un `path` "
"relatif au vault, identique à celui affiché ci-dessus."
)
return prompt
GENERAL_SYSTEM_PROMPT = """Tu es l'assistant intégré d'ObsiGate, une application web auto-hébergée pour consulter, rechercher et éditer des vaults Obsidian (Markdown).
Tes deux rôles :
1. **Aider sur l'application** : expliquer la navigation, la recherche (full-text, filtres `tag:`, `created:`, `path:`), l'éditeur (CodeMirror, autosave, raccourcis), les onglets et le split view, les sauvegardes et la restauration, le partage public, l'export (HTML/Markdown/ePub/PDF), Mermaid, Excalidraw, les plugins, les thèmes, le mode hors-ligne, le MFA, etc.
2. **Proposer des actions concrètes** : créer un fichier ou un dossier dans un vault.
Quand l'utilisateur demande explicitement de créer un fichier, inclus EXACTEMENT un bloc de ce type dans ta réponse (et rien d'autre à l'intérieur du bloc) :
```obsigate-action
{"action": "create_file", "vault": "<nom du vault>", "path": "<chemin/relatif.md>", "content": "<contenu markdown>"}
```
Pour créer un dossier :
```obsigate-action
{"action": "create_directory", "vault": "<nom du vault>", "path": "<chemin/relatif>"}
```
Règles :
- Ne propose une action que si l'utilisateur la demande explicitement.
- Explique en une phrase ce que fait l'action avant le bloc.
- Utilise un chemin relatif se terminant par `.md` pour un fichier.
- N'invente jamais un nom de vault : utilise l'un des vaults disponibles listés ci-dessous.
- Réponds dans la langue de l'utilisateur, de façon concise et structurée (Markdown).
"""
def _format_app_context(app_context: dict[str, Any] | None, recent_files: list[dict[str, Any]] | None) -> str:
"""Render the live application state for the General assistant prompt.
The General assistant has no document context; without this block it only
knows the app exists. Passing what the user currently sees (open documents,
current directory, active search, recently modified files) lets it answer
"résume ce que je fais / où j'en suis" style questions.
"""
app_context = app_context or {}
lines: list[str] = []
vault = app_context.get("vault") or app_context.get("current_vault")
if vault:
lines.append(f"- Vault sélectionné : {vault}")
directory = app_context.get("directory")
if directory:
lines.append(f"- Répertoire courant : {directory}")
current_path = app_context.get("current_path")
if current_path:
lines.append(f"- Document affiché dans le viewer : {current_path}")
docs = app_context.get("open_documents") or []
rendered_docs = []
for doc in docs:
if not isinstance(doc, dict) or not doc.get("path"):
continue
rendered_docs.append(f"{doc['path']} (vault {doc['vault']})" if doc.get("vault") else str(doc["path"]))
if rendered_docs:
lines.append("- Documents ouverts dans les onglets/panneaux : " + ", ".join(rendered_docs))
editing = app_context.get("editing")
if isinstance(editing, dict) and editing.get("path"):
surface = "Forge" if editing.get("surface") == "forge" else "l'éditeur"
location = f"{editing['path']} (vault {editing['vault']})" if editing.get("vault") else str(editing["path"])
lines.append(
f"- Document en cours d'édition dans {surface} : {location} — c'est le document affiché à la "
"place de la vue lecture. Pour le mettre à jour, utilise les outils d'écriture "
"(`edit_file`, `append_to_file`) : la modification est rechargée automatiquement dans "
"l'éditeur et la vue lecture dès l'exécution de l'outil."
)
query = app_context.get("search_query")
if query:
total = app_context.get("search_total")
suffix = f" ({total} résultat(s))" if isinstance(total, int) else ""
lines.append(f"- Recherche en cours : « {query} »{suffix}")
results = app_context.get("search_results") or []
rendered_results = [str(r.get("path")) for r in results[:10] if isinstance(r, dict) and r.get("path")]
if rendered_results:
lines.append("- Premiers résultats affichés : " + ", ".join(rendered_results))
if recent_files:
rendered_recent = [
f"{f.get('vault')}/{f.get('path')}" for f in recent_files[:10] if f.get("path")
]
if rendered_recent:
lines.append("- Fichiers récemment modifiés : " + ", ".join(rendered_recent))
if not lines:
return ""
return (
"## Contexte applicatif actuel\n"
"Voici ce que l'utilisateur voit ou fait en ce moment dans ObsiGate. "
"Sers-t'en pour comprendre sa demande ; ne le répète pas inutilement.\n"
+ "\n".join(lines)
+ "\n"
)
def build_general_system_prompt(
vaults: list[str] | None = None,
app_context: dict[str, Any] | None = None,
recent_files: list[dict[str, Any]] | None = None,
) -> str:
"""System prompt for the General assistant (app help + actions).
``app_context`` carries the live UI state (open documents, current
directory, active search) and ``recent_files`` the last modified files, so
the assistant knows what the user is doing rather than answering blind.
"""
prompt = GENERAL_SYSTEM_PROMPT
if vaults:
prompt += "\nVaults disponibles : " + ", ".join(sorted(vaults)) + "\n"
else:
prompt += "\nAucun vault n'est actuellement configuré.\n"
block = _format_app_context(app_context, recent_files)
if block:
prompt += "\n" + block
return prompt
+613 -94
View File
@@ -3,31 +3,90 @@
import json
import logging
from pathlib import Path
from typing import Any
from fastapi import APIRouter, Depends, HTTPException
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, Field
from backend.agent.loop import run_agent
from backend.ai_chat import chat_completion, stream_completion
from backend.ai_history import delete_session, get_session, list_sessions, upsert_session
from backend.auth.middleware import check_vault_access, require_auth
from backend.bookslm import build_system_prompt, collect_directory_context
from backend.indexer import get_vault_data
from backend.bookslm import (
build_general_system_prompt,
build_system_prompt,
collect_adhoc_context,
collect_directory_context,
collect_files_context,
empty_context,
load_vault_image_data_url,
merge_contexts,
)
from backend.indexer import get_vault_data, index
from backend.model_capabilities import model_supports_vision
from backend.schemas import BooksLMContextResponse
from backend.skills import get_skill_prompt
from backend.tools.api import ToolContext, ToolMode
logger = logging.getLogger("obsigate.bookslm_routes")
router = APIRouter(prefix="/api/ai/bookslm", tags=["BooksLM"])
VALID_MODES = {"directory", "documents", "general"}
# ── Request models ──
class BooksLMContextRequest(BaseModel):
vault: str = Field(description="Vault name")
vault: str | None = Field(default=None, description="Vault name (required for directory/documents modes)")
directory: str = Field(default="", description="Relative directory path within the vault")
mode: str = Field(default="directory", description="Context mode: 'directory', 'documents' or 'general'")
context_files: list[str] = Field(
default_factory=list,
description="Relative file paths to use as context (documents mode)",
)
extra_files: list[str] = Field(
default_factory=list,
description="Ad-hoc files added with the '@' command (any mode)",
)
extra_directories: list[str] = Field(
default_factory=list,
description="Ad-hoc directories added with the '@' command (any mode)",
)
app_context: dict[str, Any] | None = Field(
default=None,
description="Live client UI state for the General assistant: open_documents, "
"current_path, directory, vault, search_query, search_total, search_results.",
)
class BooksLMChatRequest(BaseModel):
vault: str = Field(description="Vault name")
vault: str | None = Field(default=None, description="Vault name (required for directory/documents modes)")
directory: str = Field(default="", description="Relative directory path within the vault")
mode: str = Field(default="directory", description="Context mode: 'directory', 'documents' or 'general'")
context_files: list[str] = Field(
default_factory=list,
description="Relative file paths to use as context (documents mode)",
)
extra_files: list[str] = Field(
default_factory=list,
description="Ad-hoc files added with the '@' command (any mode)",
)
extra_directories: list[str] = Field(
default_factory=list,
description="Ad-hoc directories added with the '@' command (any mode)",
)
message: str = Field(description="User message")
images: list[dict[str, Any]] = Field(
default_factory=list,
description="Images for vision models. Each item: {data, mime_type} (pasted) "
"or {path} (vault-relative file).",
)
skill: str | None = Field(
default=None,
description="Skill id selected with the '/' command; its prompt is added to the system prompt.",
)
conversation_history: list[dict[str, str]] = Field(
default_factory=list,
description="Previous conversation turns [{role, content}]",
@@ -41,120 +100,369 @@ class BooksLMChatRequest(BaseModel):
default=None,
description="Model name to use for this request. If not set, uses the provider's default model.",
)
confirm: dict[str, Any] | None = Field(
default=None,
description="Pending tool confirmation to apply (two-step propose/apply). "
"Shape: the ``error`` object of a previous ``confirmation`` event.",
)
confirm_messages: list[dict[str, Any]] | None = Field(
default=None,
description="Conversation snapshot returned alongside a ``confirmation`` event, "
"echoed back to resume the agent run.",
)
app_context: dict[str, Any] | None = Field(
default=None,
description="Live client UI state for the General assistant: open_documents, "
"current_path, directory, vault, search_query, search_total, search_results.",
)
def _normalize_mode(mode: str | None) -> str:
mode = (mode or "directory").lower()
return mode if mode in VALID_MODES else "directory"
def _resolve_vault_path(vault: str | None, current_user):
"""Validate vault access and return (vault_name, vault_path)."""
if not vault:
raise HTTPException(status_code=400, detail="Champ 'vault' requis pour ce contexte")
if not check_vault_access(vault, current_user):
raise HTTPException(status_code=403, detail=f"Accès refusé à la vault '{vault}'")
vault_data = get_vault_data(vault)
if not vault_data:
raise HTTPException(status_code=404, detail=f"Vault '{vault}' not found")
return vault, Path(vault_data["path"])
def _build_context(
mode: str,
vault_path: Path | None,
directory: str,
context_files: list[str],
extra_files: list[str] | None = None,
extra_directories: list[str] | None = None,
):
"""Collect the context payload for the requested mode.
Ad-hoc files/directories (``@`` command) are merged on top of the base
context. In General mode, attaching ad-hoc files promotes the effective
scope to ``documents`` so the prompt actually includes their content.
"""
if mode == "general":
base = empty_context("general")
elif mode == "documents":
ctx = collect_files_context(vault_path, context_files, scope="documents") # type: ignore[arg-type]
# No open document could be read → degrade gracefully to General.
base = ctx if ctx["file_count"] else empty_context("general")
else:
base = collect_directory_context(vault_path, directory) # type: ignore[arg-type]
if (extra_files or extra_directories) and vault_path is not None:
adhoc_scope = "documents" if base.get("scope") == "general" else base.get("scope", "directory")
adhoc = collect_adhoc_context(vault_path, extra_files, extra_directories, scope=adhoc_scope)
if adhoc["file_count"]:
base = merge_contexts(base, adhoc)
return base
def _submitted_app_context(req) -> dict[str, Any] | None:
"""Return the client-submitted live app state (may be None)."""
ctx = getattr(req, "app_context", None)
return ctx if isinstance(ctx, dict) else None
def _recent_files_for_prompt(current_user, limit: int = 10) -> list[dict[str, Any]]:
"""Best-effort list of the user's most recently modified files.
Used to give the General assistant a sense of what the user has been
working on. Never raises: any failure yields an empty list.
"""
try:
from backend.services.recent import list_recent
username = current_user.get("username") if isinstance(current_user, dict) else None
user_vaults = (
current_user.get("_token_vaults") or current_user.get("vaults", [])
if isinstance(current_user, dict)
else []
)
data = list_recent(username, user_vaults, limit=limit, mode="modified")
return list(data.get("files", []))
except Exception: # pragma: no cover - defensive, prompt enrichment only
logger.debug("Could not gather recent files for assistant prompt", exc_info=True)
return []
def _resolve_system_prompt(req, current_user) -> str:
"""Resolve the vault access and build the assistant system prompt.
Shared by the classic chat endpoint and the tool-calling agent endpoint.
"""
mode = _normalize_mode(req.mode)
vault_path: Path | None = None
if mode != "general":
_, vault_path = _resolve_vault_path(req.vault, current_user)
elif getattr(req, "extra_files", None) or getattr(req, "extra_directories", None):
# General mode has no base context, but ad-hoc files/directories added
# with `@` still need a vault to be read from.
vault_path = _resolve_optional_vault_path(req, current_user)
context = _build_context(
mode,
vault_path,
req.directory,
req.context_files,
getattr(req, "extra_files", None),
getattr(req, "extra_directories", None),
)
effective_mode = context.get("scope", mode)
if effective_mode == "general":
prompt = build_general_system_prompt(
list(index.keys()),
app_context=_submitted_app_context(req),
recent_files=_recent_files_for_prompt(current_user),
)
elif effective_mode == "documents":
prompt = build_system_prompt(context, scope="documents", vault_name=req.vault)
elif context["file_count"] == 0:
# Empty (or unreadable) directory: don't block the request. Answer as
# the General assistant would, telling the model the folder is empty so
# it can still help (create a file, explain the app, etc.).
prompt = build_general_system_prompt(
list(index.keys()),
app_context=_submitted_app_context(req),
recent_files=_recent_files_for_prompt(current_user),
)
prompt += (
f"\n## Dossier vide\nLe dossier « {req.directory or '/'} » "
f"(vault {req.vault}) ne contient aucun fichier markdown exploitable. "
"Réponds quand même à la demande de l'utilisateur sans contexte "
"documentaire, et propose une action de création si c'est pertinent.\n"
)
else:
prompt = build_system_prompt(context, scope="directory", vault_name=req.vault)
skill_id = getattr(req, "skill", None)
if skill_id:
skill_prompt = get_skill_prompt(skill_id, current_user)
if skill_prompt:
prompt += "\n\n## Skill actif\n" + skill_prompt
return prompt
def _resolve_optional_vault_path(req, current_user) -> Path | None:
"""Best-effort vault path resolution (never raises)."""
if not getattr(req, "vault", None):
return None
try:
_, vault_path = _resolve_vault_path(req.vault, current_user)
return vault_path
except HTTPException:
return None
def _build_user_content(req, vault_path: Path | None):
"""Build the user message content, attaching images when present.
Returns a plain string when there is no image, otherwise an OpenAI-style
multimodal content array (which ``ai_chat`` adapts for Gemini).
"""
images = getattr(req, "images", None) or []
if not images:
return req.message
parts: list[dict[str, Any]] = [{"type": "text", "text": req.message}]
for image in images:
if not isinstance(image, dict):
continue
data_url: str | None = None
if image.get("data"):
mime = image.get("mime_type") or "image/png"
data_url = f"data:{mime};base64,{image['data']}"
elif image.get("path") and vault_path is not None:
data_url = load_vault_image_data_url(vault_path, str(image["path"]))
if data_url:
parts.append({"type": "image_url", "image_url": {"url": data_url}})
return parts
def _validate_vision_support(req) -> None:
"""Reject image requests when the selected model cannot analyse images."""
if not (getattr(req, "images", None)):
return
provider = _resolve_provider_name(req.provider)
if not provider:
return
from backend.ai import PROVIDERS
model = req.model or PROVIDERS.get(provider, {}).get("model", "")
if not model_supports_vision(provider, model):
raise HTTPException(
status_code=400,
detail=f"Le modèle '{model or provider}' ne supporte pas l'analyse d'images. "
"Choisissez un modèle compatible vision.",
)
def _resolve_provider_name(requested: str | None) -> str | None:
"""Pick the provider to use: explicit override, else first available."""
from backend.ai import DEFAULT_PROVIDER, PROVIDERS
cfg_name = (requested or DEFAULT_PROVIDER).lower()
if cfg_name in PROVIDERS and PROVIDERS[cfg_name].get("api_key"):
return cfg_name
for pname, pcfg in PROVIDERS.items():
if pcfg.get("api_key") and pname != "gemini":
return pname
return None
def _effective_model(provider: str | None, requested: str | None) -> str:
"""Model actually used for a request.
The client may leave `model` empty (provider default) — reporting the raw
request would show nothing in the "provider · model" tag, so the provider's
configured default is returned instead.
"""
if requested:
return requested
if not provider:
return ""
from backend.ai import PROVIDERS
return PROVIDERS.get(provider, {}).get("model", "") or ""
def _tool_sources(rec) -> list[dict[str, str]]:
"""Compact web sources of a tool result (rendered as links in the UI).
Only the web tools produce sources: ``web_search`` returns ranked results,
``fetch_url`` a single page. Everything else yields an empty list so the
SSE payload stays small.
"""
data = rec.result if isinstance(rec.result, dict) else {}
name = rec.name or ""
sources: list[dict[str, str]] = []
if name == "web_search":
for item in (data.get("results") or [])[:8]:
if not isinstance(item, dict):
continue
url = item.get("url") or ""
if not url:
continue
sources.append({"title": item.get("title") or url, "url": url})
elif name == "fetch_url":
url = data.get("url") or ""
if url:
sources.append({"title": data.get("title") or url, "url": url})
return sources
def _tool_event_sse(rec) -> str:
"""Serialize one executed tool call as an SSE ``tool`` event."""
payload = json.dumps(
{
"name": rec.name,
"ok": rec.ok,
"arguments": rec.arguments,
"step": rec.step,
"sources": _tool_sources(rec),
},
ensure_ascii=False,
)
return f"event: tool\ndata: {payload}\n\n"
def _thought_event_sse(note: dict) -> str:
"""Serialize one intermediate reasoning note as an SSE ``step`` event."""
payload = json.dumps({"step": note}, ensure_ascii=False)
return f"event: step\ndata: {payload}\n\n"
# ── Endpoints ──
@router.post("/context")
@router.post("/context", response_model=BooksLMContextResponse)
async def api_bookslm_context(
req: BooksLMContextRequest,
current_user=Depends(require_auth),
):
"""Collect directory context for BooksLM.
"""Collect context for the AI assistant.
Returns file list, content, and metadata for the specified directory.
Three modes are supported:
* ``directory`` — every supported file under a vault directory;
* ``documents`` — only the explicitly listed open documents;
* ``general`` — no document context (assistant for the app itself).
"""
if not check_vault_access(req.vault, current_user):
raise HTTPException(status_code=403, detail=f"Accès refusé à la vault '{req.vault}'")
vault_data = get_vault_data(req.vault)
if not vault_data:
raise HTTPException(status_code=404, detail=f"Vault '{req.vault}' not found")
vault_path = Path(vault_data["path"])
context = collect_directory_context(vault_path, req.directory)
return context
mode = _normalize_mode(req.mode)
if mode == "general":
if not (req.extra_files or req.extra_directories):
return empty_context("general")
vault_path = _resolve_optional_vault_path(req, current_user)
else:
_, vault_path = _resolve_vault_path(req.vault, current_user)
return _build_context(
mode, vault_path, req.directory, req.context_files,
req.extra_files, req.extra_directories,
)
@router.post("/chat")
@router.post(
"/chat",
response_class=StreamingResponse,
responses={200: {"content": {"text/event-stream": {}}, "description": "SSE token stream"}},
)
async def api_bookslm_chat(
req: BooksLMChatRequest,
current_user=Depends(require_auth),
):
"""Chat with AI about directory contents (BooksLM).
"""Chat with the AI assistant.
Builds context from the directory, then sends the user message
with a system prompt containing all file contents to the AI provider.
Returns an SSE stream with the response.
Builds a system prompt from the selected context (directory, open
documents or general app knowledge), then streams the provider's answer
as Server-Sent Events.
"""
if not check_vault_access(req.vault, current_user):
raise HTTPException(status_code=403, detail=f"Accès refusé à la vault '{req.vault}'")
_validate_vision_support(req)
system_prompt = _resolve_system_prompt(req, current_user)
vault_path = _resolve_optional_vault_path(req, current_user)
vault_data = get_vault_data(req.vault)
if not vault_data:
raise HTTPException(status_code=404, detail=f"Vault '{req.vault}' not found")
vault_path = Path(vault_data["path"])
# Collect context
context = collect_directory_context(vault_path, req.directory)
if context["file_count"] == 0:
raise HTTPException(status_code=404, detail="Aucun fichier markdown trouvé dans ce dossier")
# Build system prompt
system_prompt = build_system_prompt(context)
# Call AI provider
from backend.ai import DEFAULT_PROVIDER, PROVIDERS, _call_deepseek_openrouter, _call_gemini
# Build messages with conversation history
messages_text = ""
if req.conversation_history:
for turn in req.conversation_history:
role = turn.get("role", "user")
content = turn.get("content", "")
if role == "user":
messages_text += f"\n\nUtilisateur : {content}"
elif role == "assistant":
messages_text += f"\n\nAssistant : {content}"
# Current message
user_prompt = req.message
if messages_text:
user_prompt = f"Historique de la conversation :{messages_text}\n\nQuestion actuelle : {req.message}"
messages: list[dict[str, Any]] = [{"role": "system", "content": system_prompt}]
for turn in req.conversation_history:
role = turn.get("role")
content = turn.get("content", "")
if role in ("user", "assistant") and content:
messages.append({"role": role, "content": content})
messages.append({"role": "user", "content": _build_user_content(req, vault_path)})
async def generate_sse():
try:
# Resolve provider: explicit override wins, else default.
# Fall back to first available if the requested one isn't configured.
cfg_name = (req.provider or DEFAULT_PROVIDER).lower()
if cfg_name not in PROVIDERS or not PROVIDERS[cfg_name].get("api_key"):
# Try next available provider
for pname, pcfg in PROVIDERS.items():
if pcfg.get("api_key") and pname != "gemini":
cfg_name = pname
break
else:
# No provider available at all
err = "Aucun fournisseur AI configuré (clés API manquantes)"
error_data = json.dumps({"error": err}, ensure_ascii=False)
yield f"event: error\ndata: {error_data}\n\n"
return
# Resolve provider: explicit override wins, else first available.
cfg_name = _resolve_provider_name(req.provider)
if cfg_name is None:
err = "Aucun fournisseur AI configuré (clés API manquantes)"
error_data = json.dumps({"error": err}, ensure_ascii=False)
yield f"event: error\ndata: {error_data}\n\n"
return
# Optional per-request model override
original_model = None
if req.model and cfg_name in PROVIDERS:
original_model = PROVIDERS[cfg_name].get("model")
PROVIDERS[cfg_name]["model"] = req.model
try:
if cfg_name == "gemini":
response = await _call_gemini(user_prompt, system_prompt, temperature=0.3, max_tokens=4096)
else:
response = await _call_deepseek_openrouter(
user_prompt, system_prompt,
provider=cfg_name,
temperature=0.3,
max_tokens=4096,
)
finally:
# Restore the original model so other calls aren't affected
if original_model is not None and cfg_name in PROVIDERS:
PROVIDERS[cfg_name]["model"] = original_model
# Send the full response as a single SSE event
data = json.dumps({"token": response, "provider": cfg_name, "model": req.model or PROVIDERS.get(cfg_name, {}).get("model", "")}, ensure_ascii=False)
yield f"event: message\ndata: {data}\n\n"
# Stream token deltas as they arrive from the provider.
async for token in stream_completion(
messages,
provider=cfg_name,
model=req.model,
temperature=0.3,
max_tokens=4096,
):
data = json.dumps(
{
"token": token,
"provider": cfg_name,
"model": _effective_model(cfg_name, req.model),
},
ensure_ascii=False,
)
yield f"event: message\ndata: {data}\n\n"
yield "event: done\ndata: {}\n\n"
except Exception as e:
logger.error(f"BooksLM chat error: {e}")
@@ -170,3 +478,214 @@ async def api_bookslm_chat(
"X-Accel-Buffering": "no",
},
)
@router.post(
"/agent",
response_class=StreamingResponse,
responses={200: {"content": {"text/event-stream": {}}, "description": "SSE tool/agent stream"}},
)
async def api_bookslm_agent(
req: BooksLMChatRequest,
current_user=Depends(require_auth),
):
"""Chat with the tool-calling agent.
Same context as ``/chat`` but the model may call tools (read/search the
vault) through the shared tool layer. Emits one ``tool`` event per executed
tool call, then a final ``message`` event. Mutating tools pause the run with
a ``confirmation`` event (two-step propose/apply) carrying the pending call
and the conversation snapshot; the client resumes by echoing them back in
``confirm`` / ``confirm_messages``.
"""
_validate_vision_support(req)
system_prompt = _resolve_system_prompt(req, current_user)
vault_path = _resolve_optional_vault_path(req, current_user)
messages: list[dict] = [{"role": "system", "content": system_prompt}]
for turn in req.conversation_history:
role = turn.get("role")
content = turn.get("content", "")
if role in ("user", "assistant") and content:
messages.append({"role": role, "content": content})
messages.append({"role": "user", "content": _build_user_content(req, vault_path)})
ctx = ToolContext(user=current_user, mode=ToolMode.IN_APP)
async def _llm(msgs, tool_schemas):
return await chat_completion(
msgs,
tools=tool_schemas,
provider=req.provider,
model=req.model,
temperature=0.3,
max_tokens=4096,
)
async def generate_sse():
import asyncio
try:
cfg_name = _resolve_provider_name(req.provider)
if cfg_name is None:
error_data = json.dumps(
{"error": "Aucun fournisseur AI configuré (clés API manquantes)"},
ensure_ascii=False,
)
yield f"event: error\ndata: {error_data}\n\n"
return
# Stream tool events live: each executed step is pushed on the
# queue by the loop callback and emitted as soon as it happens,
# so the UI can grow its « N steps » block while thinking.
queue: asyncio.Queue = asyncio.Queue()
def _on_tool(rec) -> None:
queue.put_nowait(("tool", rec))
run_task = asyncio.create_task(run_agent(
messages,
ctx=ctx,
llm=_llm,
resume_messages=req.confirm_messages,
confirm_pending=req.confirm,
on_tool_call=_on_tool,
on_thought=lambda note: queue.put_nowait(("thought", note)),
))
# Drain every completed step as soon as it lands, while the agent
# keeps running in the background. If the client disconnects, the
# generator is cancelled: release the run so it cannot orphan.
try:
while True:
try:
kind, item = await asyncio.wait_for(queue.get(), timeout=0.25)
except asyncio.TimeoutError:
if run_task.done():
break
continue
yield _tool_event_sse(item) if kind == "tool" else _thought_event_sse(item)
while not queue.empty():
kind, item = queue.get_nowait()
yield _tool_event_sse(item) if kind == "tool" else _thought_event_sse(item)
result = run_task.result()
finally:
if not run_task.done():
run_task.cancel()
if result.stopped == "confirmation_required":
pending = json.dumps(
{"pending": result.pending or {}, "messages": result.messages},
ensure_ascii=False,
)
yield f"event: confirmation\ndata: {pending}\n\n"
else:
data = json.dumps(
{
"token": result.content,
"provider": cfg_name,
"model": _effective_model(cfg_name, req.model),
"iterations": result.iterations,
"stopped": result.stopped,
},
ensure_ascii=False,
)
yield f"event: message\ndata: {data}\n\n"
yield "event: done\ndata: {}\n\n"
except Exception as e:
logger.error(f"BooksLM agent error: {e}")
error_data = json.dumps({"error": str(e)}, ensure_ascii=False)
yield f"event: error\ndata: {error_data}\n\n"
return StreamingResponse(
generate_sse(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
},
)
# ── Persistent conversation history (#95) ─────────────────────────────
class BookslmSession(BaseModel):
"""Full assistant conversation persisted server-side (#95).
The shape mirrors the client session object so round-tripping is lossless
(transient fields such as ``confirmation``/``payload`` are stripped
client-side before upload).
"""
id: str = Field(description="Stable session id (s-<ts36>-<rand>)")
title: str = Field(default="", description="Derived human title")
mode: str = Field(default="general", description="'directory', 'documents' or 'general'")
vault: str | None = Field(default=None, description="Vault name for the context")
directory: str = Field(default="", description="Relative directory path")
documents: list[dict[str, str]] = Field(
default_factory=list,
description="Open documents for the 'documents' mode",
)
context: str = Field(default="", description="Client context key of the assistant")
createdAt: int | None = Field(default=None, description="Creation ISO ms timestamp")
updatedAt: int | None = Field(default=None, description="Last update ISO ms timestamp")
messages: list[dict[str, Any]] = Field(
default_factory=list,
description="Conversation turns [{role, content}]",
)
def _session_user(current_user) -> str:
return current_user.get("username") if isinstance(current_user, dict) else "" # type: ignore[return-value]
@router.get("/history", response_model=dict[str, list[dict[str, Any]]])
async def api_bookslm_history_list(current_user=Depends(require_auth)):
"""List the user's assistant conversations (summaries, most recent first).
Messages are not included to keep the list light; fetch the full
conversation with ``GET /history/{id}`` when a session is opened.
"""
return {"sessions": list_sessions(_session_user(current_user))}
@router.get("/history/{session_id}", response_model=dict[str, Any] | None)
async def api_bookslm_history_get(
session_id: str,
current_user=Depends(require_auth),
):
"""Return a single full conversation, or 404 when unknown."""
session = get_session(_session_user(current_user), session_id)
if session is None:
raise HTTPException(status_code=404, detail="Conversation not found")
return session
@router.put("/history/{session_id}", response_model=dict[str, Any])
async def api_bookslm_history_upsert(
session_id: str,
session: BookslmSession,
current_user=Depends(require_auth),
):
"""Create or update a conversation for the current user.
The body id is forced to the path id so a client never writes under a
different key by mistake.
"""
payload = session.model_dump()
payload["id"] = session_id
stored = upsert_session(_session_user(current_user), payload)
if stored is None:
raise HTTPException(status_code=400, detail="Invalid session id")
return stored
@router.delete("/history/{session_id}", response_model=dict[str, bool])
async def api_bookslm_history_delete(
session_id: str,
current_user=Depends(require_auth),
):
"""Delete a conversation for the current user."""
return {"ok": delete_session(_session_user(current_user), session_id)}
+378
View File
@@ -0,0 +1,378 @@
"""Collaboration temps réel — édition simultanée (ROADMAP #62).
Ce module implémente le cœur serveur de l'édition collaborative :
* un **relais WebSocket** : une *room* est créée par fichier ouvert
(clé ``vault::chemin``) et tous les clients qui éditent le même fichier
rejoignent la même room ;
* un **relais de mises à jour Yjs** (CRDT) : le serveur ne décode pas le
format binaire Yjs, il stocke le journal des mises à jour reçues et le
rejoue aux nouveaux arrivants. La fusion sans conflit est assurée côté
client par Yjs ;
* un **awareness** (curseurs colorés + sélections) relayé entre clients ;
* une **persistance différée** : le texte markdown reçu des clients est écrit
sur disque après un debounce (2 s par défaut).
Sécurité : chaque connexion est authentifiée manuellement (les dépendances
FastAPI ``Depends`` ne s'exécutent pas pour ``@app.websocket``), puis le
chemin est validé via :func:`backend.services.paths.resolve_safe_path` et
l'accès à la vault via ``check_vault_access``.
"""
from __future__ import annotations
import asyncio
import base64
import json
import logging
import time
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
from fastapi import WebSocket
from starlette.websockets import WebSocketDisconnect
logger = logging.getLogger("obsigate.collab")
#: Délai (secondes) sans modification avant écriture sur disque.
SAVE_DEBOUNCE_SECONDS = 2.0
#: Taille maximale d'une mise à jour Yjs encodée (protection anti-abus).
MAX_UPDATE_BYTES = 8 * 1024 * 1024
#: Taille maximale d'un snapshot texte (protection anti-abus).
MAX_TEXT_CHARS = 8 * 1024 * 1024
#: Palette de couleurs attribuées aux utilisateurs (curseurs + avatars).
PEER_COLORS = [
"#e6194b", "#3cb44b", "#4363d8", "#f58231", "#911eb4",
"#008080", "#9a6324", "#800000", "#808000", "#000075",
]
def color_for_index(index: int) -> str:
"""Return a deterministic cursor color for a peer index."""
return PEER_COLORS[index % len(PEER_COLORS)]
def _b64encode(data: bytes) -> str:
return base64.b64encode(data).decode("ascii")
def _b64decode(data: str) -> bytes:
return base64.b64decode(data.encode("ascii"))
def authenticate_websocket(websocket: WebSocket) -> dict[str, Any] | None:
"""Authenticate a WebSocket connection.
Mirrors :func:`backend.auth.middleware.get_current_user` but works on the
WebSocket scope: the JWT is read from the ``access_token`` cookie (sent
automatically by same-origin browsers during the handshake) or, as a
fallback, from the ``token`` query parameter.
Returns the user dict, or ``None`` if authentication fails.
"""
from backend.auth.jwt_handler import decode_token
from backend.auth.middleware import is_auth_enabled
from backend.auth.user_store import get_user
if not is_auth_enabled():
return {
"username": "anonymous",
"display_name": "Anonymous",
"role": "admin",
"vaults": ["*"],
"active": True,
"_token_vaults": ["*"],
}
token = websocket.query_params.get("token") or websocket.cookies.get("access_token")
if not token:
return None
payload = decode_token(token)
if not payload or payload.get("type") != "access":
return None
user = get_user(payload["sub"])
if not user or not user.get("active"):
return None
user["_token_vaults"] = payload.get("vaults", [])
user["_token_jti"] = payload.get("jti")
return user
@dataclass
class CollabClient:
"""A single WebSocket connection inside a collaboration room."""
conn_id: int
websocket: WebSocket
username: str
display_name: str
color: str
y_client_id: int | None = None
awareness: dict[str, Any] | None = None
def peer(self) -> dict[str, Any]:
return {
"connId": self.conn_id,
"clientId": self.y_client_id,
"username": self.username,
"displayName": self.display_name,
"color": self.color,
}
@dataclass
class CollabRoom:
"""State shared by every client editing the same file."""
vault: str
path: str
file_path: Path
initial_text: str = ""
clients: dict[int, CollabClient] = field(default_factory=dict)
#: Journal des mises à jour Yjs (binaires) depuis la création de la room.
updates: list[bytes] = field(default_factory=list)
has_updates: bool = False
seed_sent: bool = False
pending_text: str | None = None
save_task: asyncio.Task | None = None
lock: asyncio.Lock = field(default_factory=asyncio.Lock)
@property
def key(self) -> str:
return f"{self.vault}::{self.path}"
def peers(self) -> list[dict[str, Any]]:
return [client.peer() for client in self.clients.values()]
def awareness_snapshot(self) -> list[dict[str, Any]]:
return [
{"clientId": c.y_client_id, "state": c.awareness}
for c in self.clients.values()
if c.y_client_id is not None and c.awareness is not None
]
class CollabManager:
"""Manages collaboration rooms, broadcasting and disk persistence."""
def __init__(self, save_debounce: float = SAVE_DEBOUNCE_SECONDS) -> None:
self._rooms: dict[str, CollabRoom] = {}
self._save_debounce = save_debounce
self._next_conn_id = 1
self._lock = asyncio.Lock()
# -- introspection (used by tests / diagnostics) ------------------------
@property
def room_count(self) -> int:
return len(self._rooms)
def room_peer_count(self, vault: str, path: str) -> int:
room = self._rooms.get(f"{vault}::{path}")
return len(room.clients) if room else 0
def get_room(self, vault: str, path: str) -> CollabRoom | None:
return self._rooms.get(f"{vault}::{path}")
# -- lifecycle ----------------------------------------------------------
async def connect(
self,
websocket: WebSocket,
vault: str,
path: str,
file_path: Path,
user: dict[str, Any],
) -> None:
"""Register *websocket* in the room and relay messages until it closes."""
async with self._lock:
key = f"{vault}::{path}"
room = self._rooms.get(key)
if room is None:
try:
initial_text = file_path.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
initial_text = ""
room = CollabRoom(vault=vault, path=path, file_path=file_path, initial_text=initial_text)
self._rooms[key] = room
conn_id = self._next_conn_id
self._next_conn_id += 1
client = CollabClient(
conn_id=conn_id,
websocket=websocket,
username=user.get("username", "anonymous"),
display_name=user.get("display_name") or user.get("username", "anonymous"),
color=color_for_index(conn_id - 1),
)
room.clients[conn_id] = client
seed: str | None = None
if not room.has_updates and not room.seed_sent:
seed = room.initial_text
room.seed_sent = True
await websocket.send_json({
"type": "init",
"connId": conn_id,
"color": client.color,
"seed": seed,
"updates": [_b64encode(u) for u in room.updates],
"peers": room.peers(),
"awareness": room.awareness_snapshot(),
})
await self._broadcast(room, {"type": "peer_joined", "peer": client.peer()}, exclude=conn_id)
try:
while True:
raw = await websocket.receive_text()
await self._on_message(room, client, raw)
except WebSocketDisconnect:
pass
except Exception as exc: # pragma: no cover - defensive
logger.debug("Collab connection error (%s): %s", room.key, exc)
finally:
await self.disconnect(room, client)
async def disconnect(self, room: CollabRoom, client: CollabClient) -> None:
"""Remove *client* from *room*, flushing and cleaning up if empty."""
async with self._lock:
room.clients.pop(client.conn_id, None)
empty = not room.clients
if empty:
await self._flush(room)
async with self._lock:
# Only delete if nobody rejoined while we were flushing.
if not room.clients and self._rooms.get(room.key) is room:
if room.save_task:
room.save_task.cancel()
self._rooms.pop(room.key, None)
else:
await self._broadcast(
room,
{
"type": "peer_left",
"peer": client.peer(),
"clientId": client.y_client_id,
},
)
async def stop(self) -> None:
"""Flush and cancel every room (called on application shutdown)."""
async with self._lock:
rooms = list(self._rooms.values())
self._rooms.clear()
for room in rooms:
if room.save_task:
room.save_task.cancel()
await self._flush(room)
# -- message handling ---------------------------------------------------
async def _on_message(self, room: CollabRoom, client: CollabClient, raw: str) -> None:
try:
message = json.loads(raw)
except (ValueError, TypeError):
return
if not isinstance(message, dict):
return
msg_type = message.get("type")
if msg_type in ("sync", "update"):
encoded = message.get("update")
if not isinstance(encoded, str):
return
try:
update = _b64decode(encoded)
except (ValueError, TypeError):
return
if not update or len(update) > MAX_UPDATE_BYTES:
return
async with self._lock:
room.updates.append(update)
room.has_updates = True
await self._broadcast(
room,
{"type": "update", "update": encoded, "from": client.conn_id},
exclude=client.conn_id,
)
elif msg_type == "awareness":
y_client_id = message.get("clientId")
state = message.get("state")
if not isinstance(y_client_id, int):
return
client.y_client_id = y_client_id
client.awareness = state if isinstance(state, dict) else None
await self._broadcast(
room,
{
"type": "awareness",
"clientId": y_client_id,
"state": client.awareness,
"from": client.conn_id,
},
exclude=client.conn_id,
)
elif msg_type == "text":
text = message.get("text")
if not isinstance(text, str) or len(text) > MAX_TEXT_CHARS:
return
room.pending_text = text
self._schedule_save(room)
elif msg_type == "ping":
await client.websocket.send_json({"type": "pong", "t": int(time.time() * 1000)})
# -- broadcasting -------------------------------------------------------
async def _broadcast(self, room: CollabRoom, message: dict[str, Any], exclude: int | None = None) -> None:
dead: list[CollabClient] = []
for client in list(room.clients.values()):
if exclude is not None and client.conn_id == exclude:
continue
try:
await client.websocket.send_json(message)
except Exception:
dead.append(client)
for client in dead:
room.clients.pop(client.conn_id, None)
# -- persistence --------------------------------------------------------
def _schedule_save(self, room: CollabRoom) -> None:
if room.save_task and not room.save_task.done():
room.save_task.cancel()
try:
loop = asyncio.get_running_loop()
except RuntimeError: # pragma: no cover - no running loop (tests)
return
room.save_task = loop.create_task(self._debounced_save(room))
async def _debounced_save(self, room: CollabRoom) -> None:
try:
await asyncio.sleep(self._save_debounce)
except asyncio.CancelledError:
return
await self._flush(room)
async def _flush(self, room: CollabRoom) -> None:
"""Write the last received text snapshot to disk (if any)."""
async with room.lock:
text = room.pending_text
room.pending_text = None
if text is None:
return
try:
await asyncio.to_thread(room.file_path.write_text, text, encoding="utf-8")
logger.debug("Collab persisted %s", room.key)
except OSError as exc:
logger.warning("Collab persist failed for %s: %s", room.key, exc)
#: Process-wide singleton used by the WebSocket endpoint.
collab_manager = CollabManager()
+1 -1
View File
@@ -159,7 +159,7 @@ def _safe_name(name: str) -> str:
def _collect_markdown_files(vault_path: Path) -> list[Path]:
"""List all markdown files in the vault, sorted by relative path."""
vault_path = Path(vault_path)
results = []
results: list[Path] = []
if not vault_path.is_dir():
return results
for p in sorted(vault_path.rglob("*")):
+280 -72
View File
@@ -1,4 +1,5 @@
import asyncio
import json
import logging
import os
import re
@@ -28,6 +29,9 @@ _async_index_lock: asyncio.Lock | None = None # initialized lazily
# (e.g. the inverted index in search.py) can detect staleness.
_index_generation: int = 0
# Timestamp of last full index rebuild (ISO format, empty if never built)
_last_full_index_ts: str = ""
# Hook for incremental inverted index updates: called as (action, vault, path, file_info)
_on_index_change: Callable[..., None] | None = None
@@ -197,6 +201,178 @@ def _extract_title(post: frontmatter.Post, filepath: Path) -> str:
return str(title)
_EXCALIDRAW_TEXT_FIELDS = ("text", "originalText", "label", "title")
def extract_excalidraw_text_from_elements(elements: list[dict[str, Any]]) -> str:
"""Concatenate all user-visible text from an Excalidraw elements list.
Iterates diagram elements and collects the text-bearing fields
(``text`` for text elements, ``label``/``title`` for bound shapes,
``originalText`` as the stable source of a text element). Non-text
elements and geometry-only shapes contribute nothing. This gives the
TF-IDF search engine human-readable content instead of raw JSON.
"""
chunks: list[str] = []
for el in elements:
if not isinstance(el, dict):
continue
collected = set()
for field in _EXCALIDRAW_TEXT_FIELDS:
val = el.get(field)
if isinstance(val, str) and val.strip():
collected.add(val.strip())
if collected:
chunks.append(" ".join(sorted(collected)))
return "\n".join(chunks)
_EXCALIDRAW_B64_ALPHABET = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/="
def _decompress_excalidraw(compressed: str) -> dict[str, Any] | None:
"""Decompress the Obsidian ``compressed-json`` block of a .excalidraw.md file.
The Obsidian Excalidraw plugin stores scene data as an ``lz-string``
``compressToBase64`` payload (LZ-based compression, alphabet = standard
base64). We port the reference ``lz-string`` ``_decompress`` algorithm
(bitsPerChar=6/base64 key, resetValue=32) in pure Python so indexation
can recover the diagram text without the JS client. The port is validated
against multiple JS-truth fixtures in ``test_excalidraw.py`` including
accented text and edge cases. Returns the parsed scene dict, or None if
the payload is not valid base64-lz or does not parse as JSON.
Only ``decompress`` is needed for indexation (read side); compression
stays on the JS client.
"""
if not compressed:
return None
length = len(compressed)
def _get_char_value(index: int) -> int:
# Mirror JS: input.charAt(index), 0-indexed.
ch = compressed[index] if 0 <= index < length else "="
pos = _EXCALIDRAW_B64_ALPHABET.find(ch)
return 0 if pos == -1 else pos
# ── Build a bit reader over the base64 characters ────────────────────
# JS decompressFromBase64 calls _decompress(length, 32, getNextValue).
# state = current 6-bit value + position mask + next char index.
value = _get_char_value(0)
position = 32 # resetValue
index = 1
def _read_bit() -> int:
nonlocal value, position, index
resb = value & position
position >>= 1
if position == 0:
position = 32
value = _get_char_value(index)
index += 1
return 1 if resb > 0 else 0
def _read_bits(n: int) -> int:
out = 0
for i in range(n):
out |= _read_bit() << i
return out
# ── Decompressor state ────────────────────────────────────────────────
dictionary: list[Any] = [0, 1, 2] # values 0,1,2 as in JS
enlarge_in = 4
dict_size = 4
num_bits = 3
first = _read_bits(2)
if first == 0:
c = chr(_read_bits(8))
elif first == 1:
c = chr(_read_bits(16))
elif first == 2:
return None # empty stream
else:
return None
dictionary.append(c)
w: str = c
result = [c]
while True:
if index > length:
return None
code = _read_bits(num_bits)
if code == 0:
dictionary.append(chr(_read_bits(8)))
dict_size += 1
code = dict_size - 1
enlarge_in -= 1
elif code == 1:
dictionary.append(chr(_read_bits(16)))
dict_size += 1
code = dict_size - 1
enlarge_in -= 1
elif code == 2:
break # end of stream
if enlarge_in == 0:
enlarge_in = 1 << num_bits
num_bits += 1
if code < len(dictionary) and dictionary[code]:
entry = dictionary[code]
elif code == dict_size:
entry = w + w[0]
else:
return None
result.append(entry)
dictionary.append(w + entry[0])
dict_size += 1
enlarge_in -= 1
w = entry
if enlarge_in == 0:
enlarge_in = 1 << num_bits
num_bits += 1
text = "".join(result)
try:
data = json.loads(text)
except Exception:
return None
if not isinstance(data, dict):
return None
return data
def extract_excalidraw_indexable(raw: str) -> str:
"""Return indexable text content for a raw .excalidraw / .excalidraw.md file.
Supports both the pure JSON format (``type: "excalidraw"``) and the
Obsidian ``compressed-json`` block. Falls back to ``""`` when neither
format can be parsed so the file is still indexed (title only).
"""
# .excalidraw.md — Obsidian plugin embeds a compressed-json block
if "excalidraw-plugin:" in raw:
m = re.search(r"```compressed-json\n(.*?)\n```", raw, re.DOTALL)
if m:
parsed = _decompress_excalidraw(m.group(1).strip())
if parsed:
return extract_excalidraw_text_from_elements(parsed.get("elements") or [])
return ""
# Pure .excalidraw JSON
try:
data = json.loads(raw)
except Exception:
return ""
if not isinstance(data, dict) or data.get("type") != "excalidraw":
return ""
return extract_excalidraw_text_from_elements(data.get("elements") or [])
def parse_markdown_file(raw: str) -> frontmatter.Post:
"""Parse markdown frontmatter, falling back to plain content if YAML is invalid.
@@ -247,90 +423,113 @@ def _scan_vault(vault_name: str, vault_path: str, vault_cfg: dict[str, Any] | No
logger.warning(f"Vault path does not exist: {vault_path}")
return {"files": [], "tags": {}, "path": vault_path, "paths": []}
for fpath in vault_root.rglob("*"):
# Skip ignored directories
if any(part in IGNORED_DIRS for part in fpath.relative_to(vault_root).parts):
continue
root_resolved = vault_root.resolve(strict=False)
rel_path_str = str(fpath.relative_to(vault_root)).replace("\\", "/")
# Add all paths (files and directories) to path index
if fpath.is_dir():
# BUG-032: walk without following symlinks and refuse any symlink that
# escapes the vault root, so external data can never be indexed/exposed.
for dirpath, dirnames, filenames in os.walk(vault_root, followlinks=False):
current_dir = Path(dirpath)
# Prune ignored and symlinked directories in place (no recursion).
dirnames[:] = [
d for d in dirnames
if d not in IGNORED_DIRS and not (current_dir / d).is_symlink()
]
for d in dirnames:
dpath = current_dir / d
rel_path_str = str(dpath.relative_to(vault_root)).replace("\\", "/")
paths.append({
"path": rel_path_str,
"name": fpath.name,
"name": d,
"type": "directory"
})
continue
# Files only from here
if not fpath.is_file():
continue
ext = fpath.suffix.lower()
# Also match extensionless files named like Dockerfile, Makefile
basename_lower = fpath.name.lower()
if ext not in SUPPORTED_EXTENSIONS and basename_lower not in ("dockerfile", "makefile", "cmakelists.txt"):
continue
# Add file to path index
paths.append({
"path": rel_path_str,
"name": fpath.name,
"type": "file"
})
try:
relative = fpath.relative_to(vault_root)
stat = fpath.stat()
modified = datetime.fromtimestamp(stat.st_mtime, tz=timezone.utc).isoformat()
# PDF handling — special path (binary, uses pdf_reader)
if ext == ".pdf":
from backend.pdf_reader import extract_pdf_metadata, extract_pdf_text
raw = extract_pdf_text(fpath, max_chars=100000)
pdf_meta = extract_pdf_metadata(fpath)
title = pdf_meta.get("title") or fpath.stem.replace("-", " ").replace("_", " ")
content_preview = raw[:200].strip()
tags: list[str] = []
else:
raw = fpath.read_text(encoding="utf-8", errors="replace")
tags: list[str] = []
title = fpath.stem.replace("-", " ").replace("_", " ")
content_preview = raw[:200].strip()
for fname in filenames:
fpath = current_dir / fname
if ext == ".md":
post = parse_markdown_file(raw)
tags = _extract_tags(post)
inline_tags = _extract_inline_tags(post.content)
tags = list(set(tags) | set(inline_tags))
title = _extract_title(post, fpath)
content_preview = post.content[:200].strip()
if fpath.is_symlink():
try:
target = fpath.resolve(strict=True)
except OSError:
continue
try:
target.relative_to(root_resolved)
except ValueError:
logger.warning(f"Skipping symlink outside vault: {fpath}")
continue
_extract_wikilinks_for_backlinks(
vault_name, str(relative).replace("\\", "/"),
title, post.content
)
rel_path_str = str(fpath.relative_to(vault_root)).replace("\\", "/")
files.append({
"path": str(relative).replace("\\", "/"),
"title": title,
"tags": tags,
"content_preview": content_preview,
"content": raw[:SEARCH_CONTENT_LIMIT],
"size": stat.st_size,
"modified": modified,
"extension": ext,
ext = fpath.suffix.lower()
# Also match extensionless files named like Dockerfile, Makefile
basename_lower = fpath.name.lower()
if ext not in SUPPORTED_EXTENSIONS and basename_lower not in ("dockerfile", "makefile", "cmakelists.txt"):
continue
# Add file to path index
paths.append({
"path": rel_path_str,
"name": fname,
"type": "file"
})
for tag in tags:
tag_counts[tag] = tag_counts.get(tag, 0) + 1
try:
relative = fpath.relative_to(vault_root)
stat = fpath.stat()
modified = datetime.fromtimestamp(stat.st_mtime, tz=timezone.utc).isoformat()
except PermissionError:
logger.debug(f"Permission denied, skipping {fpath}")
continue
except Exception as e:
logger.error(f"Error indexing {fpath}: {e}")
continue
# PDF handling — special path (binary, uses pdf_reader)
tags: list[str] = []
if ext == ".pdf":
from backend.pdf_reader import extract_pdf_metadata, extract_pdf_text
raw = extract_pdf_text(fpath, max_chars=100000)
pdf_meta = extract_pdf_metadata(fpath)
title = pdf_meta.get("title") or fpath.stem.replace("-", " ").replace("_", " ")
content_preview = raw[:200].strip()
elif ext == ".excalidraw" or fpath.name.lower().endswith(".excalidraw.md"):
raw = fpath.read_text(encoding="utf-8", errors="replace")
raw = extract_excalidraw_indexable(raw)
title = fpath.stem.replace(".excalidraw", "").replace("-", " ").replace("_", " ")
content_preview = raw[:200].strip()
else:
raw = fpath.read_text(encoding="utf-8", errors="replace")
title = fpath.stem.replace("-", " ").replace("_", " ")
content_preview = raw[:200].strip()
if ext == ".md":
post = parse_markdown_file(raw)
tags = _extract_tags(post)
inline_tags = _extract_inline_tags(post.content)
tags = list(set(tags) | set(inline_tags))
title = _extract_title(post, fpath)
content_preview = post.content[:200].strip()
_extract_wikilinks_for_backlinks(
vault_name, str(relative).replace("\\", "/"),
title, post.content
)
files.append({
"path": str(relative).replace("\\", "/"),
"title": title,
"tags": tags,
"content_preview": content_preview,
"content": raw[:SEARCH_CONTENT_LIMIT],
"size": stat.st_size,
"modified": modified,
"extension": ext,
})
for tag in tags:
tag_counts[tag] = tag_counts.get(tag, 0) + 1
except PermissionError:
logger.debug(f"Permission denied, skipping {fpath}")
continue
except Exception as e:
logger.error(f"Error indexing {fpath}: {e}")
continue
logger.info(f"Vault '{vault_name}': indexed {len(files)} files, {len(paths)} paths, {len(tag_counts)} unique tags")
return {"files": files, "tags": tag_counts, "path": vault_path, "paths": paths, "config": {}}
@@ -411,6 +610,10 @@ async def build_index(progress_callback=None) -> None:
if tasks:
await asyncio.gather(*tasks)
# Record timestamp of full index rebuild
global _last_full_index_ts
_last_full_index_ts = datetime.now(timezone.utc).isoformat()
# Build attachment index
from backend.attachment_indexer import build_attachment_index
await build_attachment_index(vault_config)
@@ -556,6 +759,11 @@ def _index_single_file_sync(vault_name: str, vault_path: str, file_path: str, va
pdf_meta = extract_pdf_metadata(fpath)
title = pdf_meta.get("title") or title
content_preview = raw[:200].strip()
elif ext == ".excalidraw" or fpath.name.lower().endswith(".excalidraw.md"):
raw = fpath.read_text(encoding="utf-8", errors="replace")
raw = extract_excalidraw_indexable(raw)
title = fpath.stem.replace(".excalidraw", "").replace("-", " ").replace("_", " ")
content_preview = raw[:200].strip()
else:
raw = fpath.read_text(encoding="utf-8", errors="replace")
content_preview = raw[:200].strip()
+901 -1290
View File
File diff suppressed because it is too large Load Diff
+142
View File
@@ -0,0 +1,142 @@
"""Signed, single-use confirmation tokens for MCP mutations (Phase E4).
MCP has no "Apply" button, so mutating tools are exposed in two steps:
``propose_<tool>`` returns a preview plus a **signed token**, and
``apply_<tool>`` consumes that token to execute the mutation.
The token is a short-lived JWT (same secret as the app) carrying the tool name
and its arguments. Single use is enforced by a persisted JTI blacklist, which
also protects against replay and TOCTOU (a stale proposal cannot be re-applied).
"""
from __future__ import annotations
import json
import logging
import os
import threading
import time
import uuid
from pathlib import Path
from typing import Any
from jose import JWTError, jwt
from backend.auth.jwt_handler import get_secret_key
logger = logging.getLogger("obsigate.mcp.confirmations")
ALGORITHM = "HS256"
TOKEN_TYPE = "mcp_confirmation"
# Default token lifetime (seconds); override with OBSIGATE_MCP_CONFIRMATION_TTL.
DEFAULT_TTL = int(os.environ.get("OBSIGATE_MCP_CONFIRMATION_TTL", "300"))
_USED_TOKENS_FILE = Path("data/mcp_used_tokens.json")
_used_lock = threading.RLock()
_used_loaded = False
_used_jtis: dict[str, int] = {}
class ConfirmationError(Exception):
"""Raised when a confirmation token is invalid, expired, or already used."""
def __init__(self, message: str, *, code: str = "invalid_confirmation"):
super().__init__(message)
self.message = message
self.code = code
def _load_used() -> None:
"""Load used JTIs from disk once, dropping expired entries."""
global _used_loaded, _used_jtis
if _used_loaded:
return
with _used_lock:
if _used_loaded:
return
if _USED_TOKENS_FILE.exists():
try:
data = json.loads(_USED_TOKENS_FILE.read_text(encoding="utf-8"))
now = int(time.time())
_used_jtis = {jti: exp for jti, exp in data.items() if int(exp) > now}
except Exception as e: # pragma: no cover - corrupt store
logger.warning(f"Failed to load used MCP tokens: {e}")
_used_jtis = {}
_used_loaded = True
def _save_used() -> None:
"""Persist used JTIs with their expiry (best-effort)."""
try:
_USED_TOKENS_FILE.parent.mkdir(parents=True, exist_ok=True)
tmp = _USED_TOKENS_FILE.with_suffix(".tmp")
tmp.write_text(json.dumps(_used_jtis), encoding="utf-8")
tmp.replace(_USED_TOKENS_FILE)
except Exception as e: # pragma: no cover - disk error
logger.warning(f"Failed to persist used MCP tokens: {e}")
def create_confirmation_token(
tool: str,
arguments: dict[str, Any],
username: str,
*,
ttl: int | None = None,
) -> str:
"""Create a signed confirmation token for a pending mutation."""
now = int(time.time())
payload = {
"type": TOKEN_TYPE,
"tool": tool,
"arguments": arguments,
"sub": username,
"jti": str(uuid.uuid4()),
"iat": now,
"exp": now + (ttl if ttl is not None else DEFAULT_TTL),
}
return jwt.encode(payload, get_secret_key(), algorithm=ALGORITHM)
def peek_confirmation_token(token: str) -> dict[str, Any]:
"""Decode and validate a token without consuming it (signature + expiry)."""
try:
payload = jwt.decode(token, get_secret_key(), algorithms=[ALGORITHM])
except JWTError as e:
raise ConfirmationError("Invalid or expired confirmation token", code="invalid_confirmation") from e
if payload.get("type") != TOKEN_TYPE:
raise ConfirmationError("Wrong token type", code="invalid_confirmation")
if not payload.get("tool") or not payload.get("sub"):
raise ConfirmationError("Malformed confirmation token", code="invalid_confirmation")
return payload
def consume_confirmation_token(token: str, username: str) -> dict[str, Any]:
"""Validate a token, enforce single use, and return its payload.
Raises:
ConfirmationError: invalid/expired token, wrong user, or replay.
"""
payload = peek_confirmation_token(token)
if payload.get("sub") != username:
raise ConfirmationError("Confirmation token does not belong to this user", code="forbidden")
jti = payload.get("jti")
if not jti:
raise ConfirmationError("Malformed confirmation token", code="invalid_confirmation")
_load_used()
now = int(time.time())
with _used_lock:
if jti in _used_jtis:
raise ConfirmationError("Confirmation token already used", code="token_reused")
# Record as used *before* returning so a concurrent replay is rejected.
_used_jtis[jti] = int(payload.get("exp", now))
# Opportunistic cleanup of expired JTIs.
for old_jti in [k for k, exp in _used_jtis.items() if exp <= now]:
_used_jtis.pop(old_jti, None)
_save_used()
return payload
+485
View File
@@ -0,0 +1,485 @@
"""ObsiGate MCP server — Streamable HTTP transport (Phase E).
Exposes the shared AI tool layer (``backend.tools``) to external MCP clients
(Claude Desktop, Cursor…). The same registry that powers the in-app assistant
is registered here, so the two fronts never diverge.
Primitives:
- **Tools** — read/search tools directly; mutating tools as a two-step
``propose_<tool>`` / ``apply_<tool>`` pair (signed, single-use token).
- **Resources** — accessible vaults (``vault://<name>``) and files
(``vault://<name>/<path>``), read-only and secret-redacted.
- **Prompts** — reusable note/summary templates.
Transport: Streamable HTTP mounted at ``/mcp`` (auth ``Authorization: Bearer
<JWT>``). ``stdio`` is left for later.
The transport manager is started lazily on the first request so the endpoint
also works in tests (where the ASGI lifespan is not run).
"""
from __future__ import annotations
import asyncio
import difflib
import json
import logging
from pathlib import Path
from typing import Any
from urllib.parse import urlparse
from fastapi.responses import JSONResponse
from mcp import types
from mcp.server.lowlevel import Server
from mcp.server.lowlevel.helper_types import ReadResourceContents
from mcp.server.streamable_http_manager import StreamableHTTPSessionManager
from pydantic import AnyUrl
from starlette._utils import get_route_path
from starlette.requests import Request
from starlette.routing import BaseRoute, Match
from starlette.types import ASGIApp, Receive, Scope, Send
from backend.auth.middleware import get_current_user, is_auth_enabled
from backend.services.files import read_file_text
from backend.services.vaults import list_accessible_vaults
from backend.tools.api import (
ToolContext,
ToolError,
ToolMode,
ToolRisk,
ToolScope,
call_tool,
get_tool,
list_tools,
)
logger = logging.getLogger("obsigate.mcp")
SERVER_NAME = "obsigate"
SERVER_INSTRUCTIONS = (
"ObsiGate exposes your Obsidian vaults: read, search and (with confirmation) "
"create, edit, rename, move or delete notes. Mutating tools require the "
"two-step propose_/apply_ flow."
)
# Cap on the file size returned by resources (bytes).
MAX_RESOURCE_BYTES = 200_000
def _anonymous_user() -> dict[str, Any]:
return {
"username": "anonymous",
"display_name": "Anonymous",
"role": "admin",
"vaults": ["*"],
"active": True,
"_token_vaults": ["*"],
}
def _authenticate(request: Request) -> dict[str, Any] | None:
"""Resolve the caller from the ``Authorization: Bearer`` header."""
if not is_auth_enabled():
return _anonymous_user()
from fastapi.security import HTTPAuthorizationCredentials
header = request.headers.get("authorization", "")
if not header.lower().startswith("bearer "):
return None
credentials = HTTPAuthorizationCredentials(scheme="Bearer", credentials=header[7:].strip())
return get_current_user(request, credentials)
def _user_from_context(server: Server) -> dict[str, Any]:
"""Return the authenticated user attached to the current request scope."""
request = server.request_context.request
if request is None: # pragma: no cover - stdio not supported yet
raise ToolError("No request context available", code="no_request_context")
user = getattr(request.state, "user", None)
if not user:
raise ToolError("Unauthenticated MCP request", code="unauthenticated")
return user
def _text(payload: Any) -> list[types.Content]:
"""Serialize a payload as a single text content block."""
return [types.TextContent(type="text", text=json.dumps(payload, ensure_ascii=False, default=str))]
def _unified_diff(old: str, new: str) -> str:
diff = difflib.unified_diff(
old.splitlines(keepends=True),
new.splitlines(keepends=True),
fromfile="current",
tofile="proposed",
)
return "".join(diff)
def _build_preview(spec: Any, params: Any) -> dict[str, Any]:
"""Build a human-readable preview for a proposed mutation."""
arguments = params.model_dump()
preview: dict[str, Any] = {"tool": spec.name, "arguments": arguments}
if spec.name in ("create_file", "edit_file", "append_to_file"):
vault = arguments.get("vault")
path = arguments.get("path", "")
content = arguments.get("content", "")
try:
current = read_file_text(vault, path, redact=True, max_bytes=MAX_RESOURCE_BYTES)["content"]
except Exception:
current = ""
if spec.name == "append_to_file":
separator = "" if (not current or current.endswith("\n")) else "\n"
proposed = current + separator + content
else:
proposed = content
preview["diff"] = _unified_diff(current, proposed)
return preview
def _tool_definitions() -> list[types.Tool]:
"""Build the MCP tool list from the shared registry."""
definitions: list[types.Tool] = []
for spec in list_tools(scope=ToolScope.MCP):
if spec.risk == ToolRisk.READ:
definitions.append(
types.Tool(
name=spec.name,
description=spec.description,
inputSchema=spec.parameters_schema(),
annotations=types.ToolAnnotations(readOnlyHint=True, openWorldHint=False),
)
)
continue
destructive = spec.risk == ToolRisk.DANGEROUS
definitions.append(
types.Tool(
name=f"propose_{spec.name}",
description=(
f"Propose to run '{spec.name}' and return a confirmation token. "
"No change is made until apply_ is called."
),
inputSchema=spec.parameters_schema(),
annotations=types.ToolAnnotations(
readOnlyHint=True, destructiveHint=False, idempotentHint=True, openWorldHint=False
),
)
)
definitions.append(
types.Tool(
name=f"apply_{spec.name}",
description=f"Apply a previously proposed '{spec.name}' using its confirmation token.",
inputSchema={
"type": "object",
"properties": {
"confirmation_token": {
"type": "string",
"description": f"Token returned by propose_{spec.name}",
}
},
"required": ["confirmation_token"],
},
annotations=types.ToolAnnotations(
readOnlyHint=False, destructiveHint=destructive, idempotentHint=False, openWorldHint=False
),
)
)
return definitions
def _parse_vault_uri(uri: str) -> tuple[str, str]:
"""Split ``vault://name/path`` into ``(vault, path)``."""
parsed = urlparse(uri)
if parsed.scheme != "vault" or not parsed.netloc:
raise ValueError(f"Unsupported resource URI: {uri}")
return parsed.netloc, parsed.path.lstrip("/")
def build_server() -> Server:
"""Create and configure the low-level MCP server."""
from backend.version import get_version
server: Server = Server(SERVER_NAME, version=get_version(), instructions=SERVER_INSTRUCTIONS)
# ── Tools ──────────────────────────────────────────────────────────
@server.list_tools()
async def _list_tools() -> list[types.Tool]:
return _tool_definitions()
@server.call_tool()
async def _call_tool(name: str, arguments: dict[str, Any]) -> list[types.Content]:
user = _user_from_context(server)
if name.startswith("propose_"):
return _propose(user, name[len("propose_"):], arguments)
if name.startswith("apply_"):
return _apply(user, name[len("apply_"):], arguments)
spec = get_tool(name)
if spec is None:
raise ValueError(f"Unknown tool: {name}")
if spec.risk != ToolRisk.READ:
raise ValueError(f"Tool '{name}' is mutating; use propose_{name}/apply_{name}")
ctx = ToolContext(user=user, mode=ToolMode.MCP)
try:
result = call_tool(name, ctx, arguments)
except ToolError as e:
return _text(e.to_dict())
return _text({"ok": True, "data": result.data})
# ── Resources ──────────────────────────────────────────────────────
@server.list_resources()
async def _list_resources() -> list[types.Resource]:
user = _user_from_context(server)
return [
types.Resource(
uri=AnyUrl(f"vault://{v['name']}"),
name=v["name"],
description=f"ObsiGate vault '{v['name']}' ({v['file_count']} files)",
mimeType="application/x-obsigate-vault",
)
for v in list_accessible_vaults(user)
]
@server.list_resource_templates()
async def _list_resource_templates() -> list[types.ResourceTemplate]:
return [
types.ResourceTemplate(
uriTemplate="vault://{vault}/{path}",
name="Vault file",
description="Read a text file from a vault (secrets redacted)",
mimeType="text/markdown",
)
]
@server.read_resource()
async def _read_resource(uri: Any) -> list[ReadResourceContents]:
user = _user_from_context(server)
vault, path = _parse_vault_uri(str(uri))
ctx = ToolContext(user=user, mode=ToolMode.MCP)
ctx.require_vault_access(vault)
data = read_file_text(vault, path, redact=True, max_bytes=MAX_RESOURCE_BYTES)
mime = "text/markdown" if Path(path).suffix.lower() == ".md" else "text/plain"
return [ReadResourceContents(content=data["content"], mime_type=mime)]
# ── Prompts ────────────────────────────────────────────────────────
@server.list_prompts()
async def _list_prompts() -> list[types.Prompt]:
return [
types.Prompt(
name="summarize-directory",
description="Summarize every note in a vault directory.",
arguments=[
types.PromptArgument(name="vault", description="Vault name", required=True),
types.PromptArgument(name="path", description="Directory path (empty = root)", required=False),
],
),
types.Prompt(
name="generate-note",
description="Draft a new note on a topic, using the vault for context.",
arguments=[
types.PromptArgument(name="topic", description="Note topic", required=True),
types.PromptArgument(name="vault", description="Target vault name", required=True),
],
),
types.Prompt(
name="find-related",
description="Find notes related to a given note.",
arguments=[
types.PromptArgument(name="vault", description="Vault name", required=True),
types.PromptArgument(name="path", description="Reference note path", required=True),
],
),
]
@server.get_prompt()
async def _get_prompt(name: str, arguments: dict[str, str] | None) -> types.GetPromptResult:
args = arguments or {}
def _require(key: str) -> str:
value = (args.get(key) or "").strip()
if not value:
raise ValueError(f"Missing required prompt argument: {key}")
return value
if name == "summarize-directory":
vault = _require("vault")
path = (args.get("path") or "").strip()
text = (
f"List the directory '{path or '/'}' of vault '{vault}' (use list_directory), "
"read the notes it contains, then write a concise summary with the key ideas."
)
elif name == "generate-note":
topic = _require("topic")
vault = _require("vault")
text = (
f"Draft a well-structured Markdown note about '{topic}' in vault '{vault}'. "
"Search the vault first for existing material, then propose the note content."
)
elif name == "find-related":
vault = _require("vault")
path = _require("path")
text = (
f"Read '{path}' in vault '{vault}', then find related notes via backlinks and "
"full-text search. Return a short list with why each note is related."
)
else:
raise ValueError(f"Unknown prompt: {name}")
return types.GetPromptResult(
description=f"ObsiGate prompt: {name}",
messages=[types.PromptMessage(role="user", content=types.TextContent(type="text", text=text))],
)
return server
def _propose(user: dict[str, Any], tool: str, arguments: dict[str, Any]) -> list[types.Content]:
"""Validate a mutation, return a preview and a confirmation token."""
from backend.mcp.confirmations import DEFAULT_TTL, create_confirmation_token
spec = get_tool(tool)
if spec is None:
raise ValueError(f"Unknown tool: {tool}")
if spec.risk == ToolRisk.READ:
raise ValueError(f"Tool '{tool}' is read-only; call it directly")
params = spec.input_model.model_validate(arguments)
vault = getattr(params, "vault", None)
ctx = ToolContext(user=user, mode=ToolMode.MCP)
if vault and vault != "all":
ctx.require_vault_access(vault)
if spec.risk == ToolRisk.DANGEROUS:
ctx.require_destructive_allowed(vault)
token = create_confirmation_token(tool, arguments, ctx.username)
payload = _build_preview(spec, params)
payload["confirmation_token"] = token
payload["expires_in"] = DEFAULT_TTL
return _text(payload)
def _apply(user: dict[str, Any], tool: str, arguments: dict[str, Any]) -> list[types.Content]:
"""Consume a confirmation token and execute the mutation."""
from backend.mcp.confirmations import ConfirmationError, consume_confirmation_token
token = (arguments or {}).get("confirmation_token")
if not token:
raise ValueError("Missing 'confirmation_token'")
try:
payload = consume_confirmation_token(token, user.get("username", ""))
except ConfirmationError as e:
return _text({"ok": False, "error": {"code": e.code, "message": e.message}})
if payload.get("tool") != tool:
return _text(
{
"ok": False,
"error": {
"code": "token_mismatch",
"message": f"Token was issued for '{payload.get('tool')}', not '{tool}'",
},
}
)
ctx = ToolContext(user=user, mode=ToolMode.MCP, confirmed=True)
try:
result = call_tool(tool, ctx, payload.get("arguments") or {}, confirm=True)
except ToolError as e:
return _text(e.to_dict())
return _text({"ok": True, "data": result.data})
class McpASGIApp:
"""ASGI wrapper: authenticates the request, then runs the MCP transport.
The session manager is started lazily on first use and kept alive for the
process lifetime, so the endpoint works both under uvicorn (with lifespan)
and under the test client (without entering the ASGI lifespan).
"""
def __init__(self) -> None:
self._manager: StreamableHTTPSessionManager | None = None
self._start_lock: asyncio.Lock | None = None
self._run_task: asyncio.Task | None = None
def _get_manager(self) -> StreamableHTTPSessionManager:
if self._manager is None:
self._manager = StreamableHTTPSessionManager(
app=build_server(),
json_response=True,
stateless=False,
)
return self._manager
async def _ensure_started(self) -> StreamableHTTPSessionManager:
manager = self._get_manager()
if getattr(manager, "_task_group", None) is not None:
return manager
if self._start_lock is None:
self._start_lock = asyncio.Lock()
async with self._start_lock:
if getattr(manager, "_task_group", None) is None:
self._run_task = asyncio.create_task(self._run_manager(manager))
for _ in range(500):
if getattr(manager, "_task_group", None) is not None:
break
await asyncio.sleep(0.005)
return manager
async def _run_manager(self, manager: StreamableHTTPSessionManager) -> None:
async with manager.run():
await asyncio.Event().wait()
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
if scope["type"] != "http":
return
request = Request(scope, receive)
user = _authenticate(request)
if user is None:
response = JSONResponse(
{"detail": "Authentification requise"},
status_code=401,
headers={"WWW-Authenticate": "Bearer"},
)
await response(scope, receive, send)
return
scope.setdefault("state", {})["user"] = user
manager = await self._ensure_started()
await manager.handle_request(scope, receive, send)
# Module-level singleton mounted by ``backend.main`` at ``/mcp``.
mcp_app = McpASGIApp()
class McpMount(BaseRoute):
"""ASGI route matching ``/mcp`` and ``/mcp/...`` (unlike Starlette's Mount).
Starlette's :class:`~starlette.routing.Mount` compiles ``/mcp/{path:path}``
and therefore does **not** match the bare ``/mcp`` path used by MCP clients.
This route matches both forms and delegates to :data:`mcp_app` unchanged.
"""
def __init__(self, app: ASGIApp, path: str = "/mcp") -> None:
self.app = app
self.path = path.rstrip("/")
def matches(self, scope: Scope) -> tuple[Match, Scope]:
if scope["type"] == "http":
route_path = get_route_path(scope)
if route_path == self.path or route_path.startswith(self.path + "/"):
return Match.FULL, {"endpoint": self.app}
return Match.NONE, {}
async def handle(self, scope: Scope, receive: Receive, send: Send) -> None:
await self.app(scope, receive, send)
+183
View File
@@ -0,0 +1,183 @@
"""Model-capability metadata for the AI assistant.
Two layers, in order of trust:
1. **Provider-declared** (:mod:`backend.provider_capabilities`) — when the
provider publishes per-model capabilities in its models endpoint (Mistral
``capabilities``, OpenRouter ``architecture``), that declaration *wins* for
every flag it mentions. The snapshot is cached by the model-list endpoint.
2. **Curated table** (this module) — a static, hand-maintained map of known
model-name patterns to capability flags with per-provider defaults. It fills
the flags the provider stays silent about, and is the only source for
providers and models that declare nothing (offline, no API key, DeepSeek,
NVIDIA, QwenCloud, Xiaomi…).
The UI uses the result to show, when a model is selected, which features it
supports (Chat, Embeddings, Rerank, Images, Video, Audio Speech, Audio
Transcriptions, Vision).
The curated layer is intentionally conservative: an unknown model falls back to
the provider default (usually ``chat`` only), so we never claim a capability the
model may not have. A provider declaration, on the other hand, is authoritative
in both directions — it can also *revoke* a flag the curated table guessed
wrongly (e.g. ``mistral-embed`` declares no chat).
"""
from __future__ import annotations
from typing import Any
from backend.provider_capabilities import get_declared_capabilities
# Ordered list of capability keys exposed to the UI. Keep in sync with the
# frontend ``AI_CAPABILITY_KEYS`` and the i18n ``ai.cap_*`` labels.
CAPABILITY_KEYS: tuple[str, ...] = (
"chat",
"embeddings",
"rerank",
"images",
"video",
"audio_speech",
"audio_transcription",
"vision",
)
def _caps(**kwargs: bool) -> dict[str, bool]:
"""Build a full capability dict (missing flags default to False)."""
base = {key: False for key in CAPABILITY_KEYS}
base.update(kwargs)
return base
# Provider defaults, used when no model-name pattern matches.
_PROVIDER_DEFAULTS: dict[str, dict[str, bool]] = {
"deepseek": _caps(chat=True),
"openrouter": _caps(chat=True),
"gemini": _caps(chat=True, vision=True, embeddings=True, audio_speech=True),
"ollama": _caps(chat=True, embeddings=True),
"nvidia": _caps(chat=True),
"qwencloud": _caps(chat=True),
"xiaomi": _caps(chat=True),
# Mistral: chat only. ``embeddings`` used to be assumed for every Mistral
# model, which wrongly labelled mistral-large / codestral as embedders
# (BUG-044); the ``embed`` rule below covers the real embedding models.
"mistral": _caps(chat=True),
}
# Ordered (substrings, capabilities) rules — the first matching rule wins.
# More specific modalities are listed before the broad vision/chat rule.
_MODEL_RULES: list[tuple[tuple[str, ...], dict[str, bool]]] = [
# Rerankers.
(("rerank", "cross-encoder"), _caps(rerank=True)),
# Embedding models (usually not chat-capable).
(
("text-embedding", "embed", "bge-", "e5-", "nomic-embed", "gte-"),
_caps(embeddings=True),
),
# Speech-to-text / transcription.
(
("whisper", "transcrib", "-asr", "asr-", "speech-to-text"),
_caps(audio_transcription=True),
),
# Text-to-speech.
(
("-tts", "text-to-speech", "voiceclone", "voicedesign"),
_caps(audio_speech=True),
),
# Image generation.
(
("dall-e", "stable-diffusion", "flux", "imagen", "image-gen"),
_caps(images=True),
),
# Video generation.
(("veo-", "sora", "video-gen", "-video"), _caps(video=True)),
# Document OCR models — image input, but not a chat endpoint.
(("mistral-ocr",), _caps(vision=True)),
# Vision-capable chat models (multimodal input).
(
(
"gpt-4o",
"gpt-4.1",
"gpt-5",
"o3",
"o4",
"claude-3",
"claude-4",
"gemini",
"qwen-vl",
"-vl-",
"vl-",
"vision",
"llava",
"pixtral",
"internvl",
"minicpm-v",
"llama-3.2-vision",
"mimo-vl",
# Mistral vision families (BUG-044): text-only mistral-large,
# codestral and voxtral are deliberately absent.
"ministral",
"magistral",
"mistral-small",
"mistral-medium",
"mistral-vibe-cli",
"labs-leanstral",
),
_caps(chat=True, vision=True),
),
]
def _curated_capabilities(provider: str, model: str) -> dict[str, bool]:
"""Curated table lookup (model rules first, then the provider default)."""
if model:
for needles, caps in _MODEL_RULES:
if any(needle in model for needle in needles):
return dict(caps)
return dict(_PROVIDER_DEFAULTS.get(provider, _caps(chat=True)))
def get_model_capabilities(provider: str, model: str) -> dict[str, bool]:
"""Return the capability flags for a ``provider``/``model`` pair.
A provider declaration (see :mod:`backend.provider_capabilities`) overrides
the curated table for every flag it mentions; the curated table supplies the
rest.
Args:
provider: Provider identifier (e.g. ``"deepseek"``). Case-insensitive.
model: Model identifier (e.g. ``"deepseek-chat"``). May be empty, in
which case the provider default is returned.
Returns:
A dict with every key of :data:`CAPABILITY_KEYS` and boolean values.
"""
provider = (provider or "").strip().lower()
name = (model or "").strip().lower()
caps = _curated_capabilities(provider, name)
declared = get_declared_capabilities(provider, name)
if declared:
caps.update({key: value for key, value in declared.items() if key in CAPABILITY_KEYS})
return caps
def get_capabilities_for_models(
provider: str, models: list[str]
) -> dict[str, dict[str, bool]]:
"""Return a ``{model: capabilities}`` map for a list of models."""
return {model: get_model_capabilities(provider, model) for model in models}
def model_supports_vision(provider: str, model: str) -> bool:
"""True when the given model can accept image input."""
return bool(get_model_capabilities(provider, model).get("vision"))
def capabilities_payload(provider: str, model: str) -> dict[str, Any]:
"""Serialize capabilities for API responses."""
return {
"provider": provider,
"model": model,
"capabilities": get_model_capabilities(provider, model),
}
+526
View File
@@ -0,0 +1,526 @@
"""OpenAPI 3.1 documentation helpers for ObsiGate (#72).
This module keeps the API-documentation concerns out of ``backend/main.py``:
* :data:`TAGS_METADATA` — human descriptions and ordering for every tag.
* :func:`enrich_openapi_schema` — post-processes the schema generated by
FastAPI to assign tags, add servers/security/examples and document common
error responses.
* :func:`render_api_landing` — a self-contained ``/api`` landing page that
reads ``/openapi.json`` at runtime and groups endpoints by tag, with direct
links to the Swagger UI (``/docs``) and ReDoc (``/redoc``).
"""
from __future__ import annotations
import re
from typing import Any
# ---------------------------------------------------------------------------
# Tag metadata
# ---------------------------------------------------------------------------
TAGS_METADATA: list[dict[str, str]] = [
{"name": "System", "description": "Health checks, runtime configuration, diagnostics, dashboard stats and the SSE event stream."},
{"name": "Auth", "description": "Authentication, sessions, user profile, MFA (TOTP/WebAuthn) and API keys."},
{"name": "Files", "description": "Browse, read, create, rename, move, delete and download vault files."},
{"name": "PDF", "description": "PDF metadata and byte-range streaming for inline viewing."},
{"name": "Vaults", "description": "List, add and remove vaults; per-vault display settings; attachment indexing."},
{"name": "Search", "description": "Full-text and advanced search, tags, title suggestions and the link graph."},
{"name": "Bookmarks", "description": "Recently opened files, bookmarks and saved searches."},
{"name": "Backups", "description": "Automatic file backups, diffs, restore, compression and purge."},
{"name": "Export", "description": "Export notes or whole vaults to HTML, Markdown bundle or ePub."},
{"name": "AI", "description": "AI-powered editor actions, provider status and model discovery."},
{"name": "BooksLM", "description": "Directory-scoped AI chat (NotebookLM-style) over a vault folder."},
{"name": "MCP", "description": "Model Context Protocol server (Streamable HTTP) exposing the shared AI tool layer to external clients (Claude Desktop, Cursor…)."},
{"name": "Collaboration", "description": "Real-time collaborative editing over WebSocket (`/ws/collab/{vault}/{path}`): Yjs/CRDT updates, awareness (cursors) and debounced server-side persistence."},
{"name": "Sharing", "description": "Create and manage public read-only share links for documents."},
{"name": "Webhooks", "description": "HTTP callbacks signed with HMAC-SHA256 for file events."},
{"name": "Conflicts", "description": "Detect and resolve Syncthing sync-conflict files."},
{"name": "Admin", "description": "Admin-only system monitoring: stats, audit log, backup stats and live stream."},
{"name": "Plugins", "description": "Install, enable and manage user plugins."},
{"name": "Push", "description": "Web Push (VAPID) subscription management and test notifications."},
{"name": "Frontend", "description": "Static assets and SPA fallback routes."},
]
API_DESCRIPTION = """
**ObsiGate** exposes a REST API covering the entire application: vaults, files,
full-text search, backups, exports, AI actions, sharing and administration.
### Authentication
When authentication is enabled, send a bearer token obtained from
`POST /api/auth/login`:
```
Authorization: Bearer <access_token>
```
The same token is also accepted as an HTTP-only cookie, so browser clients can
simply use `credentials: "include"`.
### Real-time collaboration
Besides the REST API, a WebSocket endpoint `GET /ws/collab/{vault}/{path}` (upgrade) powers
simultaneous editing of the same file: clients exchange Yjs/CRDT updates and awareness (remote
cursors), and the server persists the document 2 s after the last change. Authentication uses the
`access_token` cookie (or a `token` query parameter) and vault access is enforced per connection.
See the collaboration feature documentation (`docs/features/collaboration.md`) for the protocol.
### Interactive documentation
* **Swagger UI** — [/docs](/docs): try requests directly from the browser.
* **ReDoc** — [/redoc](/redoc): clean, reading-oriented reference.
* **OpenAPI JSON** — [/openapi.json](/openapi.json): machine-readable schema.
### Errors
Errors use the standard FastAPI envelope `{"detail": "..."}` with the
appropriate HTTP status (`400`, `401`, `403`, `404`, `409`, `422`, `500`).
"""
# ---------------------------------------------------------------------------
# Automatic tag assignment
# ---------------------------------------------------------------------------
# Rules are evaluated in order; the first match wins. Keep the most specific
# prefixes first (e.g. BooksLM before AI, config/ai-* before config).
_TAG_RULES: list[tuple[re.Pattern[str], str]] = [
(re.compile(r"^/api/auth"), "Auth"),
(re.compile(r"^/api/admin"), "Admin"),
(re.compile(r"^/api/plugins"), "Plugins"),
(re.compile(r"^/api/push"), "Push"),
(re.compile(r"^/api/ai/bookslm"), "BooksLM"),
(re.compile(r"^/api/ai"), "AI"),
(re.compile(r"^/mcp"), "MCP"),
(re.compile(r"^/api/config/ai-"), "AI"),
(re.compile(r"^/api/share"), "Sharing"),
(re.compile(r"^/api/shares"), "Sharing"),
(re.compile(r"^/s/"), "Sharing"),
(re.compile(r"^/api/webhooks"), "Webhooks"),
(re.compile(r"^/api/conflicts"), "Conflicts"),
(re.compile(r"^/api/backups"), "Backups"),
(re.compile(r"^/api/file/[^/]+/(backups|diff|restore)"), "Backups"),
(re.compile(r"^/api/export"), "Export"),
(re.compile(r"^/api/file/[^/]+/pdf"), "PDF"),
(re.compile(r"^/api/search"), "Search"),
(re.compile(r"^/api/tags"), "Search"),
(re.compile(r"^/api/tree-search"), "Search"),
(re.compile(r"^/api/suggest"), "Search"),
(re.compile(r"^/api/graph"), "Search"),
(re.compile(r"^/api/recent"), "Bookmarks"),
(re.compile(r"^/api/bookmarks"), "Bookmarks"),
(re.compile(r"^/api/saved-searches"), "Bookmarks"),
(re.compile(r"^/api/vaults"), "Vaults"),
(re.compile(r"^/api/vault/"), "Vaults"),
(re.compile(r"^/api/index/reload"), "Vaults"),
(re.compile(r"^/api/attachments"), "Vaults"),
(re.compile(r"^/api/browse"), "Files"),
(re.compile(r"^/api/file"), "Files"),
(re.compile(r"^/api/directory"), "Files"),
(re.compile(r"^/api/move"), "Files"),
(re.compile(r"^/api/image"), "Files"),
(re.compile(r"^/api/health"), "System"),
(re.compile(r"^/api/config"), "System"),
(re.compile(r"^/api/diagnostics"), "System"),
(re.compile(r"^/api/dashboard"), "System"),
(re.compile(r"^/api/events"), "System"),
]
def tag_for_path(path: str) -> str:
"""Return the documentation tag for an API path (``Frontend`` as fallback)."""
for pattern, tag in _TAG_RULES:
if pattern.match(path):
return tag
return "Frontend"
# Normalise tags declared by individual routers (``tags=["auth"]``) to the
# canonical, documented tag names.
_TAG_ALIASES: dict[str, str] = {
"auth": "Auth",
"admin": "Admin",
"plugins": "Plugins",
"push": "Push",
"files": "Files",
"vaults": "Vaults",
"search": "Search",
"backups": "Backups",
"export": "Export",
"sharing": "Sharing",
"webhooks": "Webhooks",
"conflicts": "Conflicts",
"system": "System",
"frontend": "Frontend",
"ai": "AI",
"bookslm": "BooksLM",
"mcp": "MCP",
"pdf": "PDF",
"bookmarks": "Bookmarks",
}
_CANONICAL_TAGS = {tag["name"] for tag in TAGS_METADATA}
def canonical_tag(name: str) -> str:
"""Map a router-declared tag to its documented name (unchanged if unknown)."""
mapped = _TAG_ALIASES.get(name.lower(), name)
return mapped if mapped in _CANONICAL_TAGS else name
# ---------------------------------------------------------------------------
# Request/response examples injected into the schema
# ---------------------------------------------------------------------------
_ENDPOINT_EXAMPLES: dict[tuple[str, str], dict[str, Any]] = {
("post", "/api/file/{vault_name}"): {
"request": {"path": "notes/Nouvelle note.md", "content": "# Titre\n\nContenu"},
"response": {"success": True, "path": "notes/Nouvelle note.md"},
},
("put", "/api/file/{vault_name}/save"): {
"request": {"path": "notes/Accueil.md", "content": "# Accueil\n\nMis à jour."},
"response": {"status": "ok", "vault": "TestVault", "path": "notes/Accueil.md", "size": 26},
},
("post", "/api/search/replace"): {
"request": {"query": "Python", "replacement": "Python 3", "vault": "all", "dry_run": True},
"response": {"matches": [{"vault": "TestVault", "path": "note1.md", "title": "Python", "match_count": 3}], "total_matches": 3, "dry_run": True},
},
("post", "/api/ai/improve"): {
"request": {"text": "ce texte est bof", "provider": "deepseek"},
"response": {"result": "Ce texte mérite d'être amélioré.", "provider": "deepseek"},
},
("post", "/api/ai/bookslm/chat"): {
"request": {"vault": "TestVault", "directory": "projets", "message": "Résume ce dossier", "conversation_history": []},
"response": {"token": "Voici un résumé…", "provider": "deepseek", "model": "deepseek-chat"},
},
("post", "/api/share/{vault_name}"): {
"request": {"path": "notes/Accueil.md", "expires_in_hours": 168},
"response": {"id": "abc…", "token": "abc…", "vault": "TestVault", "path": "notes/Accueil.md", "url": "/s/abc…"},
},
("post", "/api/webhooks"): {
"request": {"name": "CI", "url": "https://example.com/hook", "events": ["file_modified"], "secret": "s3cr3t"},
"response": {"id": "3f2c…", "name": "CI", "url": "https://example.com/hook", "events": ["file_modified"], "enabled": True},
},
}
# Common error responses documented on every operation.
_COMMON_ERRORS: dict[str, dict[str, Any]] = {
"401": {
"description": "Authentication required or token expired",
"content": {"application/json": {"example": {"detail": "Not authenticated"}}},
},
"403": {
"description": "Access to the vault is denied",
"content": {"application/json": {"example": {"detail": "Accès refusé à la vault 'Notes'"}}},
},
"404": {
"description": "Resource not found",
"content": {"application/json": {"example": {"detail": "File not found: notes/x.md"}}},
},
"422": {
"description": "Validation error",
"content": {"application/json": {"example": {"detail": [{"loc": ["query", "path"], "msg": "field required", "type": "value_error.missing"}]}}},
},
"500": {
"description": "Internal server error",
"content": {"application/json": {"example": {"detail": "Internal server error"}}},
},
}
# Paths that return a binary/streaming body — never add a JSON error example
# to their 200 response, and skip the 500 JSON example for streams.
_BINARY_MEDIA = ("application/pdf", "application/octet-stream", "image/", "text/event-stream")
def _is_binary_operation(operation: dict[str, Any]) -> bool:
content = operation.get("responses", {}).get("200", {}).get("content", {})
return any(media.startswith(_BINARY_MEDIA) for media in content)
# ---------------------------------------------------------------------------
# MCP endpoint (not a FastAPI route: custom ASGI mount) — documented manually
# ---------------------------------------------------------------------------
_MCP_DESCRIPTION = (
"**Model Context Protocol** server over Streamable HTTP (JSON-RPC 2.0). "
"Exposes the shared AI tool layer to external MCP clients (Claude Desktop, "
"Cursor…). Authentication uses `Authorization: Bearer <JWT>` (the same "
"token as the REST API).\n\n"
"Primitives: read/search **tools** directly; write/destructive tools as a "
"two-step `propose_<tool>` / `apply_<tool>` pair (signed, single-use "
"confirmation token); **resources** `vault://<name>` and "
"`vault://<name>/<path>` (read-only, secrets redacted); **prompts** "
"`summarize-directory`, `generate-note`, `find-related`.\n\n"
"See `docs/MCP_GUIDE.md` for client setup."
)
def _inject_mcp_path(schema: dict[str, Any]) -> None:
"""Add the MCP Streamable HTTP endpoint to the schema (idempotent)."""
paths = schema.setdefault("paths", {})
if "/mcp" in paths:
return
paths["/mcp"] = {
"post": {
"tags": ["MCP"],
"summary": "MCP Streamable HTTP endpoint (JSON-RPC 2.0)",
"operationId": "mcp_streamable_http",
"description": _MCP_DESCRIPTION,
"requestBody": {
"required": True,
"content": {
"application/json": {
"example": {
"jsonrpc": "2.0",
"id": 1,
"method": "tools/list",
"params": {},
}
}
},
},
"responses": {
"200": {
"description": "JSON-RPC response (or 202 for notifications)",
"content": {
"application/json": {
"example": {
"jsonrpc": "2.0",
"id": 1,
"result": {"tools": []},
}
}
},
}
},
"security": [{"bearerAuth": []}],
}
}
def enrich_openapi_schema(schema: dict[str, Any]) -> dict[str, Any]:
"""Enrich a FastAPI-generated OpenAPI schema in place and return it.
The transformation is idempotent: calling it twice yields the same result.
"""
schema.setdefault("openapi", "3.1.0")
info = schema.setdefault("info", {})
info.setdefault("description", API_DESCRIPTION.strip())
info.setdefault("contact", {"name": "ObsiGate", "url": "https://git.dracodev.net/Projets/ObsiGate"})
info.setdefault("license", {"name": "MIT"})
schema.setdefault("servers", [{"url": "/", "description": "Current ObsiGate instance"}])
schema["externalDocs"] = {
"description": "ObsiGate source repository",
"url": "https://git.dracodev.net/Projets/ObsiGate",
}
schema["tags"] = TAGS_METADATA
_inject_mcp_path(schema)
components = schema.setdefault("components", {})
security_schemes = components.setdefault("securitySchemes", {})
security_schemes.setdefault("bearerAuth", {
"type": "http",
"scheme": "bearer",
"bearerFormat": "JWT",
"description": "JWT access token issued by POST /api/auth/login.",
})
security_schemes.setdefault("cookieAuth", {
"type": "apiKey",
"in": "cookie",
"name": "obsigate_token",
"description": "HTTP-only session cookie (browser clients).",
})
components.setdefault("schemas", {}).setdefault("ErrorResponse", {
"type": "object",
"properties": {"detail": {"title": "Detail", "description": "Error message"}},
})
for path, operations in schema.get("paths", {}).items():
default_tag = tag_for_path(path)
for method, operation in operations.items():
if not isinstance(operation, dict):
continue
if operation.get("tags"):
operation["tags"] = [canonical_tag(t) for t in operation["tags"]]
else:
operation["tags"] = [default_tag]
# Document common error responses without overriding explicit ones.
responses = operation.setdefault("responses", {})
binary = _is_binary_operation(operation)
for code, spec in _COMMON_ERRORS.items():
if code == "500" and binary:
continue
responses.setdefault(code, spec)
# Inject request/response examples for key endpoints.
example = _ENDPOINT_EXAMPLES.get((method, path))
if example:
if "request" in example:
body = operation.setdefault("requestBody", {})
content = body.setdefault("content", {}).setdefault("application/json", {})
content.setdefault("example", example["request"])
if "response" in example:
ok = responses.setdefault("200", {})
content = ok.setdefault("content", {}).setdefault("application/json", {})
content.setdefault("example", example["response"])
return schema
# ---------------------------------------------------------------------------
# /api landing page
# ---------------------------------------------------------------------------
_METHOD_COLORS = {
"get": "#2563eb",
"post": "#16a34a",
"put": "#d97706",
"patch": "#7c3aed",
"delete": "#dc2626",
"head": "#64748b",
"options": "#64748b",
}
def render_api_landing(version: str = "") -> str:
"""Return the self-contained HTML for the ``/api`` documentation page."""
return f"""<!DOCTYPE html>
<html lang="en" data-theme="dark">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>ObsiGate API{(' — v' + version) if version else ''}</title>
<style>
:root {{
--bg: #0f172a; --surface: #1e293b; --border: #334155;
--text: #e2e8f0; --muted: #94a3b8; --accent: #60a5fa; --accent-2: #34d399;
}}
html[data-theme="light"] {{
--bg: #f8fafc; --surface: #ffffff; --border: #e2e8f0;
--text: #0f172a; --muted: #64748b; --accent: #2563eb; --accent-2: #059669;
}}
* {{ box-sizing: border-box; }}
body {{
margin: 0; background: var(--bg); color: var(--text);
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
line-height: 1.5;
}}
.wrap {{ max-width: 1080px; margin: 0 auto; padding: 40px 24px 80px; }}
header h1 {{ margin: 0 0 4px; font-size: 1.9rem; }}
header p {{ margin: 0; color: var(--muted); }}
.actions {{ display: flex; flex-wrap: wrap; gap: 10px; margin: 24px 0 32px; }}
.actions a {{
text-decoration: none; color: var(--text); background: var(--surface);
border: 1px solid var(--border); border-radius: 8px; padding: 10px 16px;
font-weight: 600; font-size: .9rem; transition: border-color .15s, transform .15s;
}}
.actions a:hover {{ border-color: var(--accent); transform: translateY(-1px); }}
.actions a.primary {{ background: var(--accent); border-color: var(--accent); color: #fff; }}
.meta {{ color: var(--muted); font-size: .85rem; margin-bottom: 24px; }}
.tag {{ margin: 28px 0; }}
.tag > h2 {{
font-size: 1.1rem; margin: 0 0 2px; display: flex; align-items: center; gap: 8px;
}}
.tag > p {{ margin: 0 0 12px; color: var(--muted); font-size: .85rem; }}
.ops {{ display: flex; flex-direction: column; gap: 6px; }}
.op {{
display: flex; align-items: center; gap: 12px; padding: 8px 12px;
background: var(--surface); border: 1px solid var(--border); border-radius: 8px;
text-decoration: none; color: var(--text); font-size: .88rem;
}}
.op:hover {{ border-color: var(--accent); }}
.method {{
flex: 0 0 62px; text-align: center; font-weight: 700; font-size: .68rem;
text-transform: uppercase; letter-spacing: .04em; color: #fff; border-radius: 5px; padding: 3px 0;
}}
.op code {{ color: var(--accent-2); font-size: .85rem; }}
.op .summary {{ color: var(--muted); margin-left: auto; text-align: right; }}
.loading {{ color: var(--muted); }}
footer {{ margin-top: 48px; color: var(--muted); font-size: .8rem; }}
@media (max-width: 640px) {{ .op .summary {{ display: none; }} }}
</style>
</head>
<body>
<div class="wrap">
<header>
<h1>ObsiGate API</h1>
<p>Interactive reference for the ObsiGate REST API (OpenAPI 3.1).</p>
</header>
<div class="actions">
<a class="primary" href="/docs">Swagger UI — try it</a>
<a href="/redoc">ReDoc — reference</a>
<a href="/openapi.json">OpenAPI JSON</a>
<a href="/">← Back to ObsiGate</a>
</div>
<div class="meta" id="meta">Loading specification…</div>
<div id="tags"><p class="loading">Loading endpoints…</p></div>
<footer>Generated from the live OpenAPI schema. Authentication: <code>Authorization: Bearer &lt;token&gt;</code>.</footer>
</div>
<script>
(function () {{
var COLORS = {_js_colors()};
function el(tag, cls, text) {{
var e = document.createElement(tag);
if (cls) e.className = cls;
if (text != null) e.textContent = text;
return e;
}}
fetch('/openapi.json', {{ credentials: 'include' }})
.then(function (r) {{ if (!r.ok) throw new Error('HTTP ' + r.status); return r.json(); }})
.then(function (spec) {{
var info = spec.info || {{}};
var tagMeta = {{}};
(spec.tags || []).forEach(function (t) {{ tagMeta[t.name] = t.description || ''; }});
var groups = {{}};
var paths = spec.paths || {{}};
var opCount = 0;
Object.keys(paths).forEach(function (path) {{
var ops = paths[path];
Object.keys(ops).forEach(function (method) {{
if (['get','post','put','patch','delete','head','options'].indexOf(method) === -1) return;
var op = ops[method];
if (op.deprecated && false) return;
opCount++;
var tag = (op.tags && op.tags[0]) || 'Other';
(groups[tag] = groups[tag] || []).push({{ path: path, method: method, summary: op.summary || '' }});
}});
}});
document.getElementById('meta').textContent =
(info.title || 'ObsiGate') + (info.version ? ' v' + info.version : '') +
' — OpenAPI ' + (spec.openapi || '3.1') + ' — ' + opCount + ' operations';
var host = document.getElementById('tags');
host.innerHTML = '';
Object.keys(groups).forEach(function (tag) {{
var section = el('section', 'tag');
section.appendChild(el('h2', null, tag));
if (tagMeta[tag]) section.appendChild(el('p', null, tagMeta[tag]));
var ops = el('div', 'ops');
groups[tag].forEach(function (o) {{
var a = el('a', 'op');
a.href = '/docs#/' + encodeURIComponent(tag) + '/' + encodeURIComponent(o.path);
var m = el('span', 'method', o.method);
m.style.background = COLORS[o.method] || '#64748b';
a.appendChild(m);
a.appendChild(el('code', null, o.path));
if (o.summary) a.appendChild(el('span', 'summary', o.summary));
ops.appendChild(a);
}});
section.appendChild(ops);
host.appendChild(section);
}});
}})
.catch(function (e) {{
document.getElementById('meta').textContent = 'Failed to load OpenAPI schema: ' + e.message;
}});
}})();
</script>
</body>
</html>"""
def _js_colors() -> str:
"""Serialise the method→colour map for the landing page script."""
import json
return json.dumps(_METHOD_COLORS)
+5 -2
View File
@@ -6,6 +6,7 @@ import os
from concurrent.futures import ThreadPoolExecutor
from concurrent.futures import TimeoutError as FuturesTimeout
from pathlib import Path
from typing import Any
logger = logging.getLogger(__name__)
@@ -16,7 +17,7 @@ PDF_MAX_SIZE_MB: int = int(os.environ.get("OBSIGATE_PDF_MAX_SIZE_MB", "50"))
PDF_EXTRACT_TIMEOUT: float = float(os.environ.get("OBSIGATE_PDF_EXTRACT_TIMEOUT", "30"))
PDF_READER: str = "pypdf"
PdfReader = None # type: ignore
PdfReader: Any = None
try:
import fitz # pymupdf
PDF_READER = "pymupdf"
@@ -90,7 +91,7 @@ def extract_pdf_metadata(file_path: Path) -> dict:
info["title"] = meta.get("title", "")
info["author"] = meta.get("author", "")
doc.close()
else:
elif PdfReader is not None:
reader = PdfReader(str(file_path))
info["pages"] = len(reader.pages)
meta = reader.metadata or {}
@@ -159,6 +160,8 @@ def _extract_pymupdf(file_path: Path, max_chars: int) -> str:
def _extract_pypdf(file_path: Path, max_chars: int) -> str:
if PdfReader is None:
return ""
reader = PdfReader(str(file_path))
parts = []
total = 0
+624
View File
@@ -0,0 +1,624 @@
"""Plugin system for ObsiGate — backend API and manifest validation.
Plugins extend ObsiGate with custom renderers, search filters, and editor
actions. They run sandboxed in a Web Worker on the frontend; the backend
handles manifest validation, storage, and lifecycle (install / uninstall /
enable / disable), all scoped to a single vault.
"""
import json
import logging
import re
import time as _time
import zipfile
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, ClassVar
from fastapi import APIRouter, Depends, File, HTTPException, UploadFile
from fastapi.responses import JSONResponse
from backend.auth.middleware import check_vault_access, require_admin, require_auth
from backend.indexer import get_vault_data
logger = logging.getLogger("obsigate.plugins")
router = APIRouter(prefix="/api/plugins", tags=["plugins"])
# ── Constants ──────────────────────────────────────────────────────────────
PLUGINS_DIR_NAME = ".obsigate-plugins"
MANIFEST_FILENAME = "plugin.json"
MAX_PLUGIN_SIZE = 500_000 # 500 KB per plugin file
MAX_PLUGINS_PER_VAULT = 50
MAX_PLUGIN_FILES = 100 # max files in an installed plugin zip
ALLOWED_HOOKS = frozenset([
"onFileRender",
"onSearchFilter",
"onEditorAction",
"onSidebarItem",
"onFileCreate",
"onFileDelete",
"onVaultMount",
])
ALLOWED_PERMISSIONS = frozenset([
"read_files",
"write_files",
"read_vault_metadata",
"network_request",
"ui_notify",
"access_clipboard",
])
# ── Manifest ───────────────────────────────────────────────────────────────
@dataclass
class PluginManifest:
"""Validated plugin manifest from plugin.json."""
name: str
version: str
description: str
author: str
main: str
hooks: dict[str, str] = field(default_factory=dict)
permissions: list[str] = field(default_factory=list)
min_obsigate_version: str = "2.1.0"
homepage: str | None = None
repository: str | None = None
license: str = "MIT"
@classmethod
def from_dict(cls, data: dict) -> "PluginManifest":
"""Validate and build a manifest from a raw dict (JSON object)."""
required = ["name", "version", "description", "author", "main"]
for field_name in required:
if field_name not in data or not data[field_name]:
raise ValueError(f"Missing required field: {field_name}")
name = data["name"]
if not re.fullmatch(r"[a-z0-9]([a-z0-9-]*[a-z0-9])?", name):
raise ValueError(
"Plugin name must be lowercase alphanumeric with hyphens (e.g., 'my-plugin')"
)
version = data["version"]
if not re.fullmatch(r"\d+\.\d+\.\d+(-[a-zA-Z0-9.-]+)?", version):
raise ValueError("Version must be semantic version (e.g., '1.0.0')")
hooks = data.get("hooks", {})
if not isinstance(hooks, dict):
raise TypeError("'hooks' must be an object mapping hook name to handler name")
for hook_name, handler in hooks.items():
if hook_name not in ALLOWED_HOOKS:
raise ValueError(
f"Unknown hook: {hook_name}. Allowed: {sorted(ALLOWED_HOOKS)}"
)
if not isinstance(handler, str) or not handler:
raise ValueError(f"Hook handler for '{hook_name}' must be a non-empty string")
permissions = data.get("permissions", [])
if not isinstance(permissions, list):
raise TypeError("'permissions' must be a list of permission names")
for perm in permissions:
if perm not in ALLOWED_PERMISSIONS:
raise ValueError(
f"Unknown permission: {perm}. Allowed: {sorted(ALLOWED_PERMISSIONS)}"
)
return cls(
name=name,
version=version,
description=data["description"],
author=data["author"],
main=data["main"],
hooks=hooks,
permissions=permissions,
min_obsigate_version=data.get("min_obsigate_version", "2.1.0"),
homepage=data.get("homepage"),
repository=data.get("repository"),
license=data.get("license", "MIT"),
)
def to_dict(self) -> dict[str, Any]:
return {
"name": self.name,
"version": self.version,
"description": self.description,
"author": self.author,
"main": self.main,
"hooks": self.hooks,
"permissions": self.permissions,
"min_obsigate_version": self.min_obsigate_version,
"homepage": self.homepage,
"repository": self.repository,
"license": self.license,
}
# ── Storage path helpers ───────────────────────────────────────────────────
def _vault_plugins_dir(vault_name: str) -> Path:
"""Return the plugins directory for a vault (creating it if absent)."""
vault_data = get_vault_data(vault_name)
if not vault_data:
raise HTTPException(status_code=404, detail=f"Vault '{vault_name}' not found")
base = Path(vault_data["path"])
pd = base / PLUGINS_DIR_NAME
pd.mkdir(parents=True, exist_ok=True)
return pd
# ── Manager ────────────────────────────────────────────────────────────────
class PluginManager:
"""Manages plugin lifecycle within a single plugins directory."""
def __init__(self, plugins_dir):
self.plugins_dir = Path(plugins_dir)
self.plugins_dir.mkdir(parents=True, exist_ok=True)
def _plugin_dir(self, name: str) -> Path:
return self.plugins_dir / name
# ---- install / uninstall ----
def install(self, manifest: dict, code: str) -> dict[str, Any]:
"""Install a plugin from a manifest dict + entry point source string."""
m = PluginManifest.from_dict(manifest)
plugin_dir = self._plugin_dir(m.name)
if plugin_dir.exists():
raise ValueError(f"Plugin '{m.name}' is already installed")
installed = list(self.plugins_dir.iterdir()) if self.plugins_dir.exists() else []
if installed and len(installed) >= MAX_PLUGINS_PER_VAULT:
raise ValueError(f"Maximum of {MAX_PLUGINS_PER_VAULT} plugins per vault reached")
if len(code.encode("utf-8")) > MAX_PLUGIN_SIZE:
raise ValueError(f"Plugin file exceeds {MAX_PLUGIN_SIZE} bytes")
plugin_dir.mkdir(parents=True, exist_ok=True)
(plugin_dir / MANIFEST_FILENAME).write_text(
json.dumps(m.to_dict(), indent=2), encoding="utf-8"
)
(plugin_dir / m.main).write_text(code, encoding="utf-8")
logger.info("Installed plugin '%s' v%s", m.name, m.version)
return {
"name": m.name,
"version": m.version,
"description": m.description,
"author": m.author,
"main": m.main,
"enabled": True,
}
def uninstall(self, name: str) -> None:
plugin_dir = self._plugin_dir(name)
if not plugin_dir.exists():
raise ValueError(f"Plugin '{name}' not found")
import shutil
shutil.rmtree(plugin_dir)
logger.info("Uninstalled plugin '%s'", name)
# ---- enable / disable ----
def enable(self, name: str) -> None:
plugin_dir = self._plugin_dir(name)
if not plugin_dir.exists():
raise ValueError(f"Plugin '{name}' not found")
marker = plugin_dir / ".disabled"
if marker.exists():
marker.unlink()
def disable(self, name: str) -> None:
plugin_dir = self._plugin_dir(name)
if not plugin_dir.exists():
raise ValueError(f"Plugin '{name}' not found")
(plugin_dir / ".disabled").write_text("", encoding="utf-8")
def is_disabled(self, name: str) -> bool:
return (self._plugin_dir(name) / ".disabled").exists()
# ---- listing / metadata ----
def list_plugins(self) -> list[dict[str, Any]]:
if not self.plugins_dir.exists():
return []
out = []
for d in sorted(self.plugins_dir.iterdir()):
if not d.is_dir():
continue
manifest_file = d / MANIFEST_FILENAME
if not manifest_file.exists():
continue
try:
m = PluginManifest.from_dict(json.loads(manifest_file.read_text(encoding="utf-8")))
except (json.JSONDecodeError, ValueError) as exc:
logger.warning("Invalid plugin dir '%s': %s", d.name, exc)
continue
out.append({
"name": m.name,
"version": m.version,
"description": m.description,
"author": m.author,
"main": m.main,
"enabled": not self.is_disabled(m.name),
"hooks": m.hooks,
"permissions": m.permissions,
})
return out
def get_plugin(self, name: str) -> dict[str, Any] | None:
plugin_dir = self._plugin_dir(name)
manifest_file = plugin_dir / MANIFEST_FILENAME
if not manifest_file.exists():
return None
try:
m = PluginManifest.from_dict(json.loads(manifest_file.read_text(encoding="utf-8")))
except (json.JSONDecodeError, ValueError):
return None
return {
"manifest": m.to_dict(),
"name": m.name,
"version": m.version,
"enabled": not self.is_disabled(m.name),
}
def get_plugin_code(self, name: str, file_name: str) -> str:
plugin_dir = self._plugin_dir(name)
manifest_file = plugin_dir / MANIFEST_FILENAME
if not manifest_file.exists():
raise ValueError(f"Plugin '{name}' not found")
target = plugin_dir / file_name
# Prevent path traversal
try:
target.resolve().relative_to(plugin_dir.resolve())
except (ValueError, OSError):
raise ValueError(f"Invalid file path: {file_name}")
if not target.is_file():
raise ValueError(f"File '{file_name}' not found for plugin '{name}'")
return target.read_text(encoding="utf-8")
def get_hooks(self, name: str) -> dict[str, str]:
plugin = self.get_plugin(name)
if not plugin:
raise ValueError(f"Plugin '{name}' not found")
return plugin["manifest"].get("hooks", {})
class PluginRegistry:
"""Aggregates enabled plugins for a plugins directory."""
def get_enabled_plugins(self, plugins_dir) -> list[dict[str, Any]]:
manager = PluginManager(plugins_dir)
return [p for p in manager.list_plugins() if p["enabled"]]
def get_plugins_by_hook(self, plugins_dir, hook: str) -> list[dict[str, Any]]:
manager = PluginManager(plugins_dir)
return [
p for p in manager.list_plugins()
if p["enabled"] and hook in p.get("hooks", {})
]
def scan_vault(self, vault_name: str, vault_path) -> list[dict[str, Any]]:
"""Scan a vault directory for installed plugins (startup discovery).
Returns the list of installed-but-valid plugins so the app can log
how many are available per vault. Invalid/zombie plugin dirs are
skipped gracefully.
"""
pd = Path(vault_path) / PLUGINS_DIR_NAME
if not pd.is_dir():
return []
return PluginManager(pd).list_plugins()
# Module-level registry singleton (startup scan + runtime lookups)
_registry_singleton: PluginRegistry | None = None
def get_plugin_registry() -> PluginRegistry:
"""Return the shared PluginRegistry instance."""
global _registry_singleton
if _registry_singleton is None:
_registry_singleton = PluginRegistry()
return _registry_singleton
# ── Validation of uploads (ZIP / directory) ────────────────────────────────
def _plugin_manifest_from_member(zf: zipfile.ZipFile) -> dict[str, Any]:
try:
raw = zf.read(MANIFEST_FILENAME).decode("utf-8")
except KeyError:
raise ValueError("Plugin zip is missing plugin.json")
try:
data = json.loads(raw)
except json.JSONDecodeError:
raise ValueError("Invalid JSON in plugin.json")
m = PluginManifest.from_dict(data)
return m.to_dict()
def _validate_plugin_zip(zip_path) -> dict[str, Any]:
"""Validate an uploaded plugin zip and return its manifest dict."""
with zipfile.ZipFile(zip_path, "r") as zf:
members = zf.namelist()
if len(members) > MAX_PLUGIN_FILES:
raise ValueError(f"Plugin contains too many files (max {MAX_PLUGIN_FILES})")
for member in members:
if member.endswith("/"):
continue
norm = member.replace("\\", "/")
clean = norm.lstrip("/")
if ".." in clean.split("/"):
raise ValueError("Path traversal detected in plugin zip")
# Reject absolute paths and any ../../ escapes
if norm.startswith("/") or "/../" in f"/{norm}" or norm.endswith("/.."):
raise ValueError("Path traversal detected in plugin zip")
manifest = _plugin_manifest_from_member(zf)
# Entry point must exist
main = manifest["main"]
if main not in members and f"{main}/" not in members:
# accept with any leading ./ or subdir normalization
found = any(m.rstrip("/") == main or m.rstrip("/") == f"./{main}" for m in members)
if not found:
raise ValueError(f"Missing entry point: {main}")
return manifest
def _validate_plugin_directory(plugin_dir) -> dict[str, Any]:
"""Validate an installed plugin directory and return its manifest dict."""
d = Path(plugin_dir)
manifest_file = d / MANIFEST_FILENAME
if not manifest_file.is_file():
raise ValueError(f"Missing {MANIFEST_FILENAME}")
try:
data = json.loads(manifest_file.read_text(encoding="utf-8"))
except json.JSONDecodeError:
raise ValueError(f"Invalid JSON in {MANIFEST_FILENAME}")
m = PluginManifest.from_dict(data)
if not (d / m.main).is_file():
raise ValueError(f"Missing entry point: {m.main}")
return m.to_dict()
# ── API Endpoints ──────────────────────────────────────────────────────────
@router.get("", response_model=list[dict])
async def api_list_plugins(vault: str, current_user=Depends(require_auth)):
"""List installed plugins for a vault."""
if not check_vault_access(vault, current_user):
raise HTTPException(status_code=403, detail=f"Accès refusé à la vault '{vault}'")
plugins_dir = _vault_plugins_dir(vault)
return PluginManager(plugins_dir).list_plugins()
@router.get("/template")
async def api_plugin_template():
"""Return a starter plugin template (manifest + sample code)."""
template_manifest = {
"name": "my-plugin",
"version": "1.0.0",
"description": "Describe what your plugin does",
"author": "your-name",
"main": "index.js",
"hooks": {"onFileRender": "render"},
"permissions": ["read_files", "ui_notify"],
"license": "MIT",
}
template_code = (
"// ObsiGate plugin template\n"
"export function render(ctx) {\n"
" // ctx: { path, content, extension, vault }\n"
" if (ctx.content && ctx.path.endsWith('.md')) {\n"
" ctx.content = `> Plugin: ${self.name}\\n\\n` + ctx.content;\n"
" }\n"
" return ctx;\n"
"}\n"
)
return {
"manifest": template_manifest,
"code": template_code,
}
@router.get("/{plugin_name}", response_model=dict)
async def api_get_plugin(vault: str, plugin_name: str, current_user=Depends(require_auth)):
"""Get a single plugin's metadata."""
if not check_vault_access(vault, current_user):
raise HTTPException(status_code=403, detail=f"Accès refusé à la vault '{vault}'")
plugins_dir = _vault_plugins_dir(vault)
plugin = PluginManager(plugins_dir).get_plugin(plugin_name)
if not plugin:
raise HTTPException(status_code=404, detail=f"Plugin '{plugin_name}' not found")
return plugin
@router.get("/{plugin_name}/hooks", response_model=dict)
async def api_plugin_hooks(vault: str, plugin_name: str, current_user=Depends(require_auth)):
"""Get the hooks a plugin registers."""
if not check_vault_access(vault, current_user):
raise HTTPException(status_code=403, detail=f"Accès refusé à la vault '{vault}'")
plugins_dir = _vault_plugins_dir(vault)
try:
return PluginManager(plugins_dir).get_hooks(plugin_name)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc))
@router.get("/{plugin_name}/code/{file_name:path}", response_model=dict)
async def api_plugin_code(
vault: str, plugin_name: str, file_name: str, current_user=Depends(require_auth)
):
"""Get a plugin source file for sandboxed execution."""
if not check_vault_access(vault, current_user):
raise HTTPException(status_code=403, detail=f"Accès refusé à la vault '{vault}'")
plugins_dir = _vault_plugins_dir(vault)
try:
code = PluginManager(plugins_dir).get_plugin_code(plugin_name, file_name)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc))
return {"name": plugin_name, "file": file_name, "code": code}
@router.post("/install")
async def api_install_plugin(
vault: str,
file: UploadFile = File(...),
current_user=Depends(require_admin),
):
"""Install a plugin from an uploaded zip (admin only)."""
if not check_vault_access(vault, current_user):
raise HTTPException(status_code=403, detail=f"Accès refusé à la vault '{vault}'")
content = await file.read()
if len(content) > 5_000_000: # 5 MB upload cap
raise HTTPException(status_code=413, detail="Plugin zip too large")
import io
import tempfile
with tempfile.TemporaryDirectory() as tmp_dir:
zip_path = Path(tmp_dir) / "plugin.zip"
try:
zip_path.write_bytes(content)
manifest = _validate_plugin_zip(zip_path)
except ValueError as exc:
raise HTTPException(status_code=400, detail=f"Invalid plugin: {exc}")
# Extract to a temp dir then validate the entry point exists
plugins_dir = _vault_plugins_dir(vault)
manager = PluginManager(plugins_dir)
name = manifest["name"]
with tempfile.TemporaryDirectory() as td:
try:
with zipfile.ZipFile(io.BytesIO(content)) as zf:
zf.extractall(td)
except (zipfile.BadZipFile, RuntimeError):
raise HTTPException(status_code=400, detail="Invalid plugin zip")
extracted = _validate_plugin_directory(Path(td) / name if (Path(td) / name).is_dir() else Path(td))
code_file = manifest["main"]
with zipfile.ZipFile(io.BytesIO(content)) as zf:
code = zf.read(code_file).decode("utf-8")
try:
result = manager.install(extracted, code)
except ValueError as exc:
raise HTTPException(status_code=409, detail=str(exc))
return result
@router.delete("/{plugin_name}")
async def api_uninstall_plugin(vault: str, plugin_name: str, current_user=Depends(require_admin)):
if not check_vault_access(vault, current_user):
raise HTTPException(status_code=403, detail=f"Accès refusé à la vault '{vault}'")
plugins_dir = _vault_plugins_dir(vault)
try:
PluginManager(plugins_dir).uninstall(plugin_name)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc))
return JSONResponse({"ok": True})
@router.post("/{plugin_name}/enable")
async def api_enable_plugin(vault: str, plugin_name: str, current_user=Depends(require_admin)):
if not check_vault_access(vault, current_user):
raise HTTPException(status_code=403, detail=f"Accès refusé à la vault '{vault}'")
plugins_dir = _vault_plugins_dir(vault)
try:
PluginManager(plugins_dir).enable(plugin_name)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc))
return JSONResponse({"ok": True})
@router.post("/{plugin_name}/disable")
async def api_disable_plugin(vault: str, plugin_name: str, current_user=Depends(require_admin)):
if not check_vault_access(vault, current_user):
raise HTTPException(status_code=403, detail=f"Accès refusé à la vault '{vault}'")
plugins_dir = _vault_plugins_dir(vault)
try:
PluginManager(plugins_dir).disable(plugin_name)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc))
return JSONResponse({"ok": True})
# ── Plugin Event Bus ────────────────────────────────────────────────────────
# Lightweight pub/sub for backend events that plugins can subscribe to.
# Used to dispatch onFileCreate, onFileDelete, onVaultMount to enabled plugins.
class _PluginEventBus:
"""In-memory event dispatcher for plugin lifecycle events."""
_listeners: ClassVar[dict[str, list]] = {}
@classmethod
def subscribe(cls, event_type: str, callback) -> None:
cls._listeners.setdefault(event_type, []).append(callback)
@classmethod
async def emit(cls, event_type: str, data: dict) -> None:
for cb in cls._listeners.get(event_type, []):
try:
result = cb(data)
if hasattr(result, "__await__"):
await result
except Exception as e:
logger.warning(f"Plugin event '{event_type}' handler error: {e}")
plugin_events = _PluginEventBus
def emit_file_created(vault: str, path: str, title: str = "") -> None:
"""Called when a file is created via the API."""
import asyncio
try:
loop = asyncio.get_running_loop()
loop.create_task(plugin_events.emit("onFileCreate", {
"vault": vault, "path": path, "title": title,
"timestamp": _time.time(),
}))
except RuntimeError:
pass # No event loop running
def emit_file_deleted(vault: str, path: str) -> None:
"""Called when a file is deleted via the API."""
import asyncio
try:
loop = asyncio.get_running_loop()
loop.create_task(plugin_events.emit("onFileDelete", {
"vault": vault, "path": path,
"timestamp": _time.time(),
}))
except RuntimeError:
pass
def emit_vault_mounted(vault: str, path: str) -> None:
"""Called when a vault is loaded into the index at startup."""
import asyncio
try:
loop = asyncio.get_running_loop()
loop.create_task(plugin_events.emit("onVaultMount", {
"vault": vault, "path": path,
"timestamp": _time.time(),
}))
except RuntimeError:
pass
+174
View File
@@ -0,0 +1,174 @@
"""Provider-declared model capabilities (live) with a short-lived cache.
The curated table in :mod:`backend.model_capabilities` has to be edited by hand
every time a provider ships or renames a model, and it ages badly: Mistral alone
declares ``vision`` on 28 of its models while the curated table knew none of
them (BUG-044). Providers that *do* publish per-model capabilities in their
public models endpoint are therefore asked first; the curated table is only
used to fill the flags the provider stays silent about (e.g. Mistral never
declares ``embedding``, only the absence of ``completion_chat``).
Supported declarations, detected by *payload shape* so a provider that starts
exposing them is picked up without a code change:
* ``capabilities`` dict (Mistral): ``completion_chat`` → ``chat``, ``vision``,
``audio_transcription`` (+ ``audio_transcription_realtime``),
``audio_speech``.
* ``architecture`` dict (OpenRouter): ``input_modalities`` /
``output_modalities`` → ``vision`` (image input), ``images`` (image output),
``audio_transcription`` (audio input), ``audio_speech`` (audio output),
``video`` (video output), ``chat`` (text output).
The cache is in-process and shared by every request. Its TTL only bounds how
long a *stale* declaration can survive: a fresh provider call (the model list
endpoint) overwrites the provider entry immediately.
"""
from __future__ import annotations
import logging
import os
import time
from typing import Any
logger = logging.getLogger("obsigate.ai.capabilities")
#: How long a provider-declared snapshot stays usable, in seconds.
#: ``0`` disables expiry (the snapshot lives until the next provider call).
TTL_SECONDS = float(os.getenv("AI_CAPABILITIES_TTL_SECONDS", "1800") or 0)
#: Guard against a provider returning a runaway model list.
MAX_MODELS_PER_PROVIDER = 2000
# provider → (timestamp, model count, {normalized model id: declared flags})
_CACHE: dict[str, tuple[float, int, dict[str, dict[str, bool]]]] = {}
def _norm(value: str) -> str:
"""Lower-case and strip the ``models/`` prefix Gemini uses."""
return (value or "").strip().lower().removeprefix("models/")
def _from_capability_flags(flags: dict[str, Any]) -> dict[str, bool]:
"""Map a Mistral-style ``capabilities`` dict onto ObsiGate flags."""
declared: dict[str, bool] = {}
if isinstance(flags.get("completion_chat"), bool):
declared["chat"] = flags["completion_chat"]
if isinstance(flags.get("vision"), bool):
declared["vision"] = flags["vision"]
transcription = flags.get("audio_transcription")
realtime = flags.get("audio_transcription_realtime")
if isinstance(transcription, bool) or isinstance(realtime, bool):
declared["audio_transcription"] = bool(transcription or realtime)
if isinstance(flags.get("audio_speech"), bool):
declared["audio_speech"] = flags["audio_speech"]
return declared
def _from_architecture(architecture: dict[str, Any]) -> dict[str, bool]:
"""Map an OpenRouter-style ``architecture`` dict onto ObsiGate flags."""
inputs = architecture.get("input_modalities")
outputs = architecture.get("output_modalities")
if not isinstance(inputs, list) and not isinstance(outputs, list):
return {}
in_modalities = [str(m).lower() for m in inputs] if isinstance(inputs, list) else []
out_modalities = [str(m).lower() for m in outputs] if isinstance(outputs, list) else []
return {
"chat": "text" in out_modalities,
"vision": "image" in in_modalities,
"images": "image" in out_modalities,
"audio_transcription": "audio" in in_modalities,
"audio_speech": "audio" in out_modalities,
"video": "video" in out_modalities,
}
def parse_declared_capabilities(entry: Any) -> dict[str, bool] | None:
"""Extract the capability flags a single provider model entry declares.
Args:
entry: One item of a provider models payload (``/v1/models``).
Returns:
A partial ``{flag: bool}`` mapping (only the flags the provider
actually declares), or ``None`` when the entry declares nothing.
"""
if not isinstance(entry, dict):
return None
flags = entry.get("capabilities")
declared = _from_capability_flags(flags) if isinstance(flags, dict) else {}
if not declared:
architecture = entry.get("architecture")
declared = _from_architecture(architecture) if isinstance(architecture, dict) else {}
return declared or None
def remember_declared_capabilities(provider: str, payload: Any) -> int:
"""Cache the capabilities declared by a provider models payload.
Args:
provider: Provider identifier (e.g. ``"mistral"``).
payload: Raw JSON body of the provider models endpoint, or the model
list itself.
Returns:
The number of models with declared capabilities that were cached.
"""
provider = (provider or "").strip().lower()
entries: Any = payload.get("data") if isinstance(payload, dict) else payload
if not isinstance(entries, list):
return 0
parsed: dict[str, dict[str, bool]] = {}
for entry in entries[:MAX_MODELS_PER_PROVIDER]:
if not isinstance(entry, dict):
continue
model_id = entry.get("id") or entry.get("name") or ""
if not isinstance(model_id, str) or not model_id.strip():
continue
declared = parse_declared_capabilities(entry)
if declared:
parsed[_norm(model_id)] = declared
if not parsed:
return 0
_CACHE[provider] = (time.time(), len(parsed), parsed)
logger.info(f"Capabilities declared by {provider}: {len(parsed)} models cached")
return len(parsed)
def get_declared_capabilities(provider: str, model: str) -> dict[str, bool] | None:
"""Return the cached declared capabilities for one provider/model pair.
Returns ``None`` when nothing was declared for that pair (cache cold,
expired, or the provider is silent about this model).
"""
snapshot = _CACHE.get((provider or "").strip().lower())
if not snapshot:
return None
timestamp, _count, table = snapshot
if TTL_SECONDS and (time.time() - timestamp) > TTL_SECONDS:
return None
declared = table.get(_norm(model))
return dict(declared) if declared else None
def clear_declared_capabilities(provider: str | None = None) -> None:
"""Drop the cached snapshot for one provider, or all of them (tests/ops)."""
if provider is None:
_CACHE.clear()
return
_CACHE.pop(provider.strip().lower(), None)
def cache_info() -> dict[str, dict[str, Any]]:
"""Diagnostics: per-provider cache age and model count."""
now = time.time()
return {
provider: {
"models": count,
"age_seconds": round(now - timestamp, 1),
"expired": bool(TTL_SECONDS and (now - timestamp) > TTL_SECONDS),
}
for provider, (timestamp, count, _table) in _CACHE.items()
}
+285
View File
@@ -0,0 +1,285 @@
# backend/push.py
# Push Notifications — Web Push API with VAPID authentication
# ROADMAP #67
import base64
import json
import logging
import os
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel, Field
from backend.auth.middleware import require_auth
logger = logging.getLogger("obsigate.push")
router = APIRouter(prefix="/api/push", tags=["push"])
# VAPID keys storage
VAPID_KEYS_FILE = Path(os.environ.get("OBSIGATE_DATA_DIR", "data")) / "vapid_keys.json"
PUSH_SUBSCRIPTIONS_FILE = Path(os.environ.get("OBSIGATE_DATA_DIR", "data")) / "push_subscriptions.json"
# In-memory cache
_vapid_keys: dict[str, str] = {}
_push_subscriptions: list[dict[str, Any]] = []
def load_vapid_keys() -> dict[str, str]:
"""Load VAPID keys from file or generate new ones."""
global _vapid_keys
if _vapid_keys:
return _vapid_keys
try:
if VAPID_KEYS_FILE.exists():
with open(VAPID_KEYS_FILE, "r") as f:
_vapid_keys = json.load(f)
else:
# Generate new VAPID keys
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import ec
private_key = ec.generate_private_key(ec.SECP256R1())
public_key = private_key.public_key()
private_pem = private_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.PKCS8,
encryption_algorithm=serialization.NoEncryption()
).decode('utf-8')
public_pem = public_key.public_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PublicFormat.SubjectPublicKeyInfo
).decode('utf-8')
# Convert to base64url for Web Push
public_numbers = public_key.public_numbers()
def int_to_base64url(n: int) -> str:
byte_len = (n.bit_length() + 7) // 8
return base64.urlsafe_b64encode(n.to_bytes(byte_len, 'big')).decode('utf-8').rstrip('=')
_vapid_keys = {
"private_key": private_pem,
"public_key": public_pem,
"public_key_base64url": int_to_base64url(public_numbers.x) + "." + int_to_base64url(public_numbers.y)
}
VAPID_KEYS_FILE.parent.mkdir(parents=True, exist_ok=True)
with open(VAPID_KEYS_FILE, "w") as f:
json.dump(_vapid_keys, f)
except Exception as e:
logger.error(f"Failed to load/generate VAPID keys: {e}")
# Fallback: return empty (will use demo keys if needed)
_vapid_keys = {}
return _vapid_keys
def get_vapid_public_key() -> str:
"""Get VAPID public key in base64url format for client."""
keys = load_vapid_keys()
return keys.get("public_key_base64url", "")
def load_push_subscriptions() -> list[dict[str, Any]]:
"""Load push subscriptions from file."""
global _push_subscriptions
if _push_subscriptions:
return _push_subscriptions
try:
if PUSH_SUBSCRIPTIONS_FILE.exists():
with open(PUSH_SUBSCRIPTIONS_FILE, "r") as f:
_push_subscriptions = json.load(f)
except Exception as e:
logger.error(f"Failed to load push subscriptions: {e}")
_push_subscriptions = []
return _push_subscriptions
def save_push_subscriptions() -> None:
"""Save push subscriptions to file."""
try:
PUSH_SUBSCRIPTIONS_FILE.parent.mkdir(parents=True, exist_ok=True)
with open(PUSH_SUBSCRIPTIONS_FILE, "w") as f:
json.dump(_push_subscriptions, f, indent=2)
except Exception as e:
logger.error(f"Failed to save push subscriptions: {e}")
# ── Models ──────────────────────────────────────────────────────────────────
class PushSubscription(BaseModel):
"""Push subscription from client (matches Push API)."""
endpoint: str
keys: dict[str, str] # { p256dh, auth }
class SubscribeRequest(BaseModel):
"""Request to subscribe to push notifications."""
subscription: PushSubscription
vault: str = Field(description="Vault name this subscription is for")
class SubscribeResponse(BaseModel):
"""Response to subscription request."""
success: bool
subscription_id: str | None = None
class VapidPublicKeyResponse(BaseModel):
"""VAPID public key for client."""
public_key: str
class PushPayload(BaseModel):
"""Payload for sending a push notification."""
vault: str
title: str
body: str
data: dict[str, Any] = Field(default_factory=dict)
tag: str = "obsigate-notification"
# ── Endpoints ───────────────────────────────────────────────────────────────
@router.get("/vapid-public-key", response_model=VapidPublicKeyResponse)
async def get_vapid_public_key_endpoint():
"""Get VAPID public key for client subscription."""
public_key = get_vapid_public_key()
if not public_key:
# Return a demo key if generation failed (for testing)
return {"public_key": "demo-key-for-testing"}
return {"public_key": public_key}
@router.post("/subscribe", response_model=SubscribeResponse)
async def subscribe_push(
request: SubscribeRequest,
current_user=Depends(require_auth)
):
"""Subscribe to push notifications for a vault."""
username = current_user.get("username", "unknown")
# Check if user has access to this vault
user_vaults = current_user.get("_token_vaults") or current_user.get("vaults", [])
if "*" not in user_vaults and request.vault not in user_vaults:
raise HTTPException(status_code=403, detail="No access to this vault")
# Check if subscription already exists
for sub in _push_subscriptions:
if sub["endpoint"] == request.subscription.endpoint and sub["username"] == username:
return SubscribeResponse(success=True, subscription_id=sub.get("id"))
# Add new subscription
sub_id = base64.urlsafe_b64encode(os.urandom(16)).decode('utf-8').rstrip('=')
subscription = {
"id": sub_id,
"endpoint": request.subscription.endpoint,
"keys": request.subscription.keys,
"vault": request.vault,
"username": username,
"created_at": datetime.now(timezone.utc).isoformat()
}
_push_subscriptions.append(subscription)
save_push_subscriptions()
logger.info(f"Push subscription added for user={username}, vault={request.vault}")
return SubscribeResponse(success=True, subscription_id=sub_id)
@router.delete("/subscribe")
async def unsubscribe_push(
endpoint: str,
current_user=Depends(require_auth)
):
"""Unsubscribe from push notifications."""
username = current_user.get("username", "unknown")
global _push_subscriptions
original_len = len(_push_subscriptions)
_push_subscriptions = [
s for s in _push_subscriptions
if not (s["endpoint"] == endpoint and s["username"] == username)
]
if len(_push_subscriptions) < original_len:
save_push_subscriptions()
return {"success": True, "message": "Unsubscribed"}
return {"success": False, "message": "Subscription not found"}
@router.get("/subscriptions")
async def list_subscriptions(current_user=Depends(require_auth)):
"""List current user's push subscriptions."""
username = current_user.get("username", "unknown")
user_subs = [
{
"id": s["id"],
"vault": s["vault"],
"created_at": s["created_at"],
"endpoint": s["endpoint"][:50] + "..." # Truncate for privacy
}
for s in _push_subscriptions if s["username"] == username
]
return {"subscriptions": user_subs}
# ── Internal: Send push notification ────────────────────────────────────────
async def send_push_notification(vault: str, title: str, body: str, data: dict | None = None, tag: str = "obsigate-notification") -> int:
"""Send push notification to all subscribers of a vault. Returns count of sent notifications."""
subscriptions = [s for s in _push_subscriptions if s["vault"] == vault]
if not subscriptions:
return 0
payload = {
"title": title,
"body": body,
"data": data or {},
"tag": tag
}
# Import here to avoid circular dependency
from pywebpush import WebPushException, webpush
keys = load_vapid_keys()
vapid_private_key = keys.get("private_key")
vapid_claims = {
"sub": "mailto:[email protected]"
}
sent = 0
for sub in subscriptions:
try:
webpush(
subscription_info={
"endpoint": sub["endpoint"],
"keys": sub["keys"]
},
data=json.dumps(payload),
vapid_private_key=vapid_private_key,
vapid_claims=vapid_claims
)
sent += 1
except WebPushException as e:
logger.warning(f"Push failed for subscription {sub['id']}: {e}")
# If subscription expired/invalid, remove it
if e.response and e.response.status_code in (404, 410):
_push_subscriptions.remove(sub)
except Exception as e:
logger.error(f"Push error for {sub['id']}: {e}")
if sent < len(subscriptions):
save_push_subscriptions()
return sent
+64 -15
View File
@@ -1,13 +1,21 @@
"""
IP-based rate limiter for authentication endpoints.
In-memory rate limiter for authentication endpoints.
Tracks failed login attempts per IP address with automatic
cleanup of expired entries. Complements the per-account lockout
in user_store.py.
Tracks failed attempts per IP **and** per account with automatic cleanup of
expired entries. The per-IP budget stops a single source; the per-account
budget (BUG-031) still throttles an attacker who rotates IPs. It complements
the per-account lockout in ``user_store.py``.
.. note::
The counters live in process memory only. They are **not** shared between
multiple workers/containers and are lost on restart. For a multi-node
deployment, front this service with a shared store (Redis) or a single
worker. This limitation is intentional and documented (BUG-031).
Configuration via environment variables:
OBSIGATE_LOGIN_MAX_ATTEMPTS Max failures per IP (default: 10)
OBSIGATE_LOGIN_WINDOW_SECONDS Lockout window in seconds (default: 900 = 15min)
OBSIGATE_LOGIN_MAX_ATTEMPTS Max failures per IP (default: 10)
OBSIGATE_ACCOUNT_MAX_ATTEMPTS Max failures per account (default: 10)
OBSIGATE_LOGIN_WINDOW_SECONDS Lockout window in seconds (default: 900)
"""
import logging
@@ -19,29 +27,37 @@ logger = logging.getLogger("obsigate.ratelimit")
# --- Configuration ---
MAX_ATTEMPTS = int(os.environ.get("OBSIGATE_LOGIN_MAX_ATTEMPTS", "10"))
ACCOUNT_MAX_ATTEMPTS = int(os.environ.get("OBSIGATE_ACCOUNT_MAX_ATTEMPTS", "10"))
WINDOW_SECONDS = int(os.environ.get("OBSIGATE_LOGIN_WINDOW_SECONDS", "900")) # 15 min
# --- In-memory store: {ip: [(timestamp, success_bool), ...]} ---
# --- In-memory stores: {key: [(timestamp, success_bool), ...]} ---
_ip_attempts: dict[str, list] = defaultdict(list)
_account_attempts: dict[str, list] = defaultdict(list)
_last_cleanup = time.time()
CLEANUP_INTERVAL = 60 # seconds
def _prune(store: dict[str, list], cutoff: float) -> None:
"""Drop expired entries from one store in place."""
expired = []
for key, attempts in store.items():
store[key] = [a for a in attempts if a[0] > cutoff]
if not store[key]:
expired.append(key)
for key in expired:
del store[key]
def _cleanup_expired():
"""Remove entries older than the window."""
"""Remove entries older than the window from both stores."""
global _last_cleanup
now = time.time()
if now - _last_cleanup < CLEANUP_INTERVAL:
return
_last_cleanup = now
cutoff = now - WINDOW_SECONDS
expired_ips = []
for ip, attempts in _ip_attempts.items():
_ip_attempts[ip] = [a for a in attempts if a[0] > cutoff]
if not _ip_attempts[ip]:
expired_ips.append(ip)
for ip in expired_ips:
del _ip_attempts[ip]
_prune(_ip_attempts, cutoff)
_prune(_account_attempts, cutoff)
def record_failure(ip: str) -> tuple[int, int]:
@@ -72,6 +88,37 @@ def is_rate_limited(ip: str) -> bool:
return failures >= MAX_ATTEMPTS
def record_account_failure(account: str) -> tuple[int, int]:
"""Record a failed attempt for an account, regardless of source IP.
Returns:
(current_failure_count, remaining_attempts)
"""
_cleanup_expired()
key = account.lower()
_account_attempts[key].append((time.time(), False))
failures = sum(1 for _, success in _account_attempts[key] if not success)
remaining = max(0, ACCOUNT_MAX_ATTEMPTS - failures)
if failures >= ACCOUNT_MAX_ATTEMPTS:
logger.warning(f"Account {account} rate-limited after {failures} failed attempts")
return failures, remaining
def record_account_success(account: str):
"""Clear the per-account rate limit state after a successful login."""
_cleanup_expired()
_account_attempts[account.lower()] = [(time.time(), True)]
def is_account_rate_limited(account: str) -> bool:
"""Check if an account has exceeded the per-account rate limit."""
_cleanup_expired()
failures = sum(
1 for _, success in _account_attempts.get(account.lower(), []) if not success
)
return failures >= ACCOUNT_MAX_ATTEMPTS
def get_status(ip: str | None = None) -> dict:
"""Get rate limit status for an IP (for diagnostics)."""
_cleanup_expired()
@@ -87,7 +134,9 @@ def get_status(ip: str | None = None) -> dict:
}
return {
"tracked_ips": len(_ip_attempts),
"tracked_accounts": len(_account_attempts),
"max_attempts": MAX_ATTEMPTS,
"account_max_attempts": ACCOUNT_MAX_ATTEMPTS,
"window_seconds": WINDOW_SECONDS,
"limited_ips": sum(
1 for ip_addr in _ip_attempts
+18
View File
@@ -0,0 +1,18 @@
# ObsiGate — Optional dependencies for semantic search (#70)
#
# These are NOT required: the semantic search engine degrades gracefully to a
# dependency-free hashing embedder and a pure-Python cosine store when they are
# absent. Install this file to enable the full local model + fast vector index:
#
# pip install -r backend/requirements-semantic.txt
#
# NOTE: sentence-transformers pulls in PyTorch (large download). If you only
# want the vector acceleration, install numpy + faiss-cpu and configure an
# external embedding endpoint instead (OBSIGATE_EMBEDDING_*).
# Local embedding model (all-MiniLM-L6-v2, ~80 MB, CPU)
sentence-transformers>=2.2.0
# Vector storage / similarity search
numpy>=1.24.0
faiss-cpu>=1.7.4
+4
View File
@@ -1,5 +1,6 @@
fastapi==0.110.3
uvicorn==0.30.0
websockets>=12.0
python-frontmatter==1.1.0
mistune==3.0.2
python-multipart==0.0.9
@@ -16,3 +17,6 @@ pypdf>=4.0
pyotp>=2.10.0
webauthn==2.6.0
psutil>=5.9
pywebpush>=2.3.0
mcp==1.9.4
sse-starlette==2.1.3
+517
View File
@@ -0,0 +1,517 @@
"""Additional Pydantic response models for the ObsiGate REST API.
These models enrich the auto-generated OpenAPI 3.1 schema (#72). They are
intentionally permissive (``extra="allow"``) so that adding a new field to an
existing endpoint never breaks response validation — the model documents the
stable, public shape while still accepting internal additions.
Models whose endpoints are already typed in :mod:`backend.main` (e.g.
``FileContentResponse``) live there; this module only holds the ones used to
fill the gaps.
"""
from typing import Any
from pydantic import BaseModel, ConfigDict, Field
# ---------------------------------------------------------------------------
# Generic helpers
# ---------------------------------------------------------------------------
class StatusResponse(BaseModel):
"""Generic ``{"status": "..."}`` acknowledgement."""
model_config = ConfigDict(extra="allow")
status: str = Field(default="ok", description="Operation status")
class ErrorResponse(BaseModel):
"""Standard FastAPI error envelope."""
detail: str | list[dict[str, Any]] | None = Field(
default=None,
description="Human-readable error message, or a list of validation errors",
examples=["Vault 'Notes' not found"],
)
# ---------------------------------------------------------------------------
# Recent files & bookmarks
# ---------------------------------------------------------------------------
class RecentFileItem(BaseModel):
"""A recently opened or recently modified file."""
path: str = Field(description="Relative path within the vault")
title: str = Field(description="File title")
vault: str = Field(description="Vault name")
mtime: float | int | str | None = Field(default=None, description="Modification timestamp")
mtime_human: str | None = Field(default=None, description="Human-readable modification time")
mtime_iso: str | None = Field(default=None, description="ISO-8601 modification time")
size_bytes: int | None = Field(default=None, description="File size in bytes")
tags: list[str] = Field(default_factory=list, description="Up to 5 leading tags")
preview: str | None = Field(default=None, description="Short content preview")
bookmarked: bool | None = Field(default=None, description="Whether the file is bookmarked")
class RecentResponse(BaseModel):
"""Response for ``GET /api/recent``."""
files: list[RecentFileItem]
total: int = Field(description="Total number of files in the selected mode")
limit: int = Field(description="Applied limit")
mode: str = Field(description="'opened' or 'modified'")
model_config = ConfigDict(
json_schema_extra={
"example": {
"files": [
{
"path": "notes/Accueil.md",
"title": "Accueil",
"vault": "TestVault",
"mtime": 1750000000.0,
"mtime_human": "il y a 2 h",
"size_bytes": 1024,
"tags": ["#accueil"],
"preview": "# Bienvenue…",
"bookmarked": False,
}
],
"total": 1,
"limit": 20,
"mode": "opened",
}
}
)
class BookmarkFileItem(BaseModel):
"""A bookmarked file."""
path: str = Field(description="Relative path within the vault")
title: str = Field(description="File title")
vault: str = Field(description="Vault name")
mtime: float | int | str | None = Field(default=None, description="Bookmark timestamp")
mtime_human: str | None = Field(default=None, description="Human-readable bookmark time")
size_bytes: int | None = Field(default=None, description="File size in bytes")
tags: list[str] = Field(default_factory=list, description="Up to 5 leading tags")
bookmarked: bool | None = Field(default=True, description="Always true for this endpoint")
class BookmarksResponse(BaseModel):
"""Response for ``GET /api/bookmarks``."""
files: list[BookmarkFileItem]
total: int = Field(description="Number of bookmarked files")
class BookmarkToggleResponse(BaseModel):
"""Response for ``POST /api/bookmarks/toggle``."""
bookmarked: bool = Field(description="New bookmark state")
class SavedSearch(BaseModel):
"""A persisted search definition."""
model_config = ConfigDict(extra="allow")
id: str = Field(description="Unique identifier (millisecond timestamp)")
query: str = Field(description="Search query text")
vault: str = Field(default="all", description="Vault filter")
case_sensitive: bool = Field(default=False)
whole_word: bool = Field(default=False)
regex: bool = Field(default=False)
include_paths: str = Field(default="")
exclude_paths: str = Field(default="")
created_at: float = Field(description="Creation timestamp (epoch seconds)")
# ---------------------------------------------------------------------------
# Backups & diffs
# ---------------------------------------------------------------------------
class BackupsResponse(BaseModel):
"""Response for ``GET /api/file/{vault}/backups``."""
vault: str
path: str
backups: list[dict[str, Any]] = Field(description="Backups, newest first")
class BacklinksResponse(BaseModel):
"""Response for ``GET /api/file/{vault}/backlinks``."""
vault: str
path: str
backlinks: list[dict[str, Any]] = Field(description="Files linking to the target")
total: int
class BackupsListResponse(BaseModel):
"""Response for ``GET /api/backups``."""
backups: list[dict[str, Any]] = Field(description="All backups across vaults")
total: int
total_size_bytes: int
class BackupsDeletedResponse(BaseModel):
"""Response for backup deletion / purge endpoints."""
deleted: int = Field(description="Number of backup files deleted")
class BackupContentResponse(BaseModel):
"""Response for ``GET /api/backups/content``."""
content: str = Field(description="Backup file content (truncated to 100 KB)")
name: str = Field(description="Backup file name")
size: int = Field(description="Returned content size in characters")
class BackupsCompressResponse(BaseModel):
"""Response for ``POST /api/backups/compress``."""
compressed: int = Field(description="Number of backups processed")
saved_bytes: int = Field(description="Bytes saved by gzip compression")
dry_run: bool
class BackupsAutoResponse(BaseModel):
"""Response for ``POST /api/backups/auto``."""
backed_up: int = Field(description="Number of files backed up")
since_hours: int | float = Field(description="Look-back window in hours")
# ---------------------------------------------------------------------------
# PDF
# ---------------------------------------------------------------------------
class PdfInfoResponse(BaseModel):
"""Response for ``GET /api/file/{vault}/pdf/info``."""
vault: str
path: str
pages: int = Field(description="Page count")
title: str = Field(description="PDF title (metadata or filename)")
author: str = Field(default="", description="PDF author")
size_bytes: int = Field(description="File size in bytes")
model_config = ConfigDict(
json_schema_extra={
"example": {
"vault": "TestVault",
"path": "docs/rapport.pdf",
"pages": 12,
"title": "Rapport annuel",
"author": "ObsiGate",
"size_bytes": 524288,
}
}
)
# ---------------------------------------------------------------------------
# Search & replace
# ---------------------------------------------------------------------------
class ReplaceMatch(BaseModel):
"""A single file affected by a find/replace operation."""
model_config = ConfigDict(extra="allow")
vault: str
path: str
title: str | None = None
match_count: int | None = Field(default=None, description="Occurrences found (dry run)")
replacements: int | None = Field(default=None, description="Occurrences replaced")
preview: list[str] = Field(default_factory=list, description="Context snippets (dry run)")
class ReplaceResponse(BaseModel):
"""Response for ``POST /api/search/replace``."""
model_config = ConfigDict(extra="allow")
matches: list[ReplaceMatch] = Field(default_factory=list, description="Dry-run matches")
replaced: list[ReplaceMatch] = Field(default_factory=list, description="Applied replacements")
total_matches: int | None = Field(default=None, description="Total matches (dry run)")
total_replacements: int = Field(description="Total replacements performed or previewed")
dry_run: bool | None = Field(default=None, description="True when no file was written")
# ---------------------------------------------------------------------------
# Vaults, attachments & settings
# ---------------------------------------------------------------------------
class VaultStatsResponse(BaseModel):
"""Response for ``POST /api/vaults/add`` and ``GET /api/index/reload/{vault}``."""
model_config = ConfigDict(extra="allow")
status: str = Field(default="ok")
vault: str
stats: dict[str, Any] = Field(default_factory=dict, description="Index statistics for the vault")
class VaultActionResponse(BaseModel):
"""Response for ``DELETE /api/vaults/{vault}``."""
status: str = Field(default="ok")
vault: str
class VaultStatusEntry(BaseModel):
"""Per-vault status entry."""
file_count: int
tag_count: int
path: str = ""
watching: bool = False
class VaultsStatusResponse(BaseModel):
"""Response for ``GET /api/vaults/status``."""
vaults: dict[str, VaultStatusEntry]
watcher_active: bool
sse_clients: int
class AttachmentRescanResponse(BaseModel):
"""Response for ``POST /api/attachments/rescan/{vault}``."""
status: str = Field(default="ok")
vault: str
attachment_count: int
class AttachmentStatsResponse(BaseModel):
"""Response for ``GET /api/attachments/stats``."""
vaults: dict[str, Any] = Field(description="Vault name → attachment statistics")
class VaultSettingsResponse(BaseModel):
"""Response for the per-vault display settings endpoints."""
model_config = ConfigDict(extra="allow")
hideHiddenFiles: bool = Field(default=False, description="Hide dotfiles in the tree")
class AllVaultSettingsResponse(BaseModel):
"""Response for ``GET /api/vaults/settings/all``."""
model_config = ConfigDict(extra="allow")
class VaultFileEntry(BaseModel):
"""A file entry returned by the vault home listing."""
model_config = ConfigDict(extra="allow")
name: str
path: str
vault: str
size: int = 0
modified: float = 0
modified_iso: str | None = None
extension: str = ""
rel_dir: str | None = None
class VaultFilesResponse(BaseModel):
"""Response for ``GET /api/vault/{vault}/files``."""
vault: str
directory: str = ""
recursive: bool = True
count: int
files: list[VaultFileEntry]
# ---------------------------------------------------------------------------
# Configuration, AI keys & diagnostics
# ---------------------------------------------------------------------------
class AppConfigResponse(BaseModel):
"""Application configuration (``GET/POST /api/config``)."""
model_config = ConfigDict(extra="allow")
class AIKeysResponse(BaseModel):
"""Masked AI provider keys (``GET /api/config/ai-keys``)."""
model_config = ConfigDict(extra="allow")
class AIKeyDeleteResponse(BaseModel):
"""Response for ``DELETE /api/config/ai-keys/{provider}``."""
status: str = Field(default="deleted")
key: str = Field(description="Deleted environment variable name")
class AITestResponse(BaseModel):
"""Response for ``POST /api/config/ai-keys/test`` (provider → status)."""
model_config = ConfigDict(extra="allow")
class AIModelsResponse(BaseModel):
"""Response for ``GET /api/config/ai-models``."""
model_config = ConfigDict(extra="allow")
models: list[str] = Field(default_factory=list, description="Available model identifiers")
source: str = Field(default="fallback", description="'live', 'fallback' or 'validation'")
count: int | None = Field(default=None, description="Number of live models")
error: str | None = Field(default=None, description="Provider/network error, if any")
note: str | None = Field(default=None, description="Explanatory note when using fallback")
capabilities: dict[str, dict[str, bool]] = Field(
default_factory=dict,
description="Per-model capability flags (chat, embeddings, vision, …)",
)
class DiagnosticsResponse(BaseModel):
"""Response for ``GET /api/diagnostics``."""
model_config = ConfigDict(extra="allow")
index: dict[str, Any] = Field(default_factory=dict)
inverted_index: dict[str, Any] = Field(default_factory=dict)
config: dict[str, Any] = Field(default_factory=dict)
class DashboardVaultStat(BaseModel):
"""Per-vault dashboard statistics."""
name: str
file_count: int
tag_count: int
total_size_bytes: int
class DashboardResponse(BaseModel):
"""Response for ``GET /api/dashboard``."""
vaults: list[DashboardVaultStat]
total_files: int
total_tags: int
total_size_bytes: int
# ---------------------------------------------------------------------------
# Webhooks, sharing & conflicts
# ---------------------------------------------------------------------------
class WebhookModel(BaseModel):
"""A configured webhook."""
model_config = ConfigDict(extra="allow")
id: str = Field(description="Webhook UUID")
name: str
url: str = Field(description="Target HTTP(S) URL")
events: list[str] = Field(default_factory=list, description="Subscribed event types")
secret: str | None = Field(default=None, description="HMAC-SHA256 signing secret")
enabled: bool = True
created_at: str | None = None
last_fired_at: str | None = None
class ShareModel(BaseModel):
"""A public document share."""
model_config = ConfigDict(extra="allow")
id: str
token: str = Field(description="Opaque share token")
vault: str
path: str
url: str | None = Field(default=None, description="Relative public URL (/s/{token})")
created_by: str | None = None
created_at: str | None = None
expires_at: str | None = None
access_count: int = 0
last_accessed: str | None = None
class ConflictEntry(BaseModel):
"""A Syncthing sync-conflict file."""
model_config = ConfigDict(extra="allow")
vault: str
path: str
original_path: str | None = None
class ConflictsResponse(BaseModel):
"""Response for ``GET /api/conflicts``."""
conflicts: list[ConflictEntry]
total: int
class ConflictResolveResponse(BaseModel):
"""Response for ``POST /api/conflicts/resolve``."""
status: str = Field(default="resolved")
action: str | None = Field(default=None, description="'keep_local' or 'keep_conflict'")
# ---------------------------------------------------------------------------
# AI status & BooksLM context
# ---------------------------------------------------------------------------
class AIProviderStatus(BaseModel):
"""Availability of a single AI provider."""
available: bool
model: str | None = Field(default=None, description="Default model when the provider is configured")
class AIAutocompleteStatus(BaseModel):
"""Status of the local Ollama autocomplete backend."""
available: bool = False
server_ok: bool = False
model_loaded: bool = False
model: str = ""
error: str | None = None
class AIStatusResponse(BaseModel):
"""Response for ``GET /api/ai/status``."""
configured: bool = Field(description="True when at least one provider has an API key")
default_provider: str
providers: dict[str, AIProviderStatus]
autocomplete: AIAutocompleteStatus
class BooksLMContextFile(BaseModel):
"""A single document included in the BooksLM context."""
path: str
title: str
content: str
type: str = Field(default="markdown", description="'markdown' (and future types)")
class BooksLMContextResponse(BaseModel):
"""Response for ``POST /api/ai/bookslm/context``."""
files: list[BooksLMContextFile]
total_chars: int
file_count: int
directory_tree: str = ""
max_total_chars: int = Field(default=200000, description="Configured context character limit")
max_files: int = Field(default=200, description="Configured file-count limit")
scope: str = Field(default="directory", description="Context scope: 'directory', 'documents' or 'general'")
+274 -104
View File
@@ -4,13 +4,20 @@ import re
import time
import unicodedata
from collections import defaultdict
from collections.abc import Callable
from typing import Any
from snowballstemmer import stemmer as _snowball_stemmer
from sortedcontainers import SortedList
from backend import indexer as _indexer
from backend import semantic_search as _semantic
from backend.indexer import index
from backend.services.regex_safety import (
MAX_REGEX_MATCHES,
truncate_for_regex,
validate_regex,
)
logger = logging.getLogger("obsigate.search")
@@ -224,12 +231,16 @@ def _extract_regex_snippet(
if not content or not pattern_text:
return content[:200].strip() if content else ""
# BUG-025: bound the text scanned and the number of matches collected.
content = truncate_for_regex(content)
try:
validate_regex(pattern_text)
pattern = re.compile(pattern_text, re.IGNORECASE)
except re.error:
except (re.error, ValueError):
return _escape_html(content[:200].strip())
matches = list(pattern.finditer(content))
matches = list(pattern.finditer(content))[:MAX_REGEX_MATCHES]
if not matches:
return _escape_html(content[:200].strip())
@@ -655,6 +666,10 @@ def _on_index_change_hook(action: str, vault_name: str, path: str, file_info: di
inv.remove_document(vault_name, path)
except Exception as e:
logger.warning(f"Inverted index incremental update failed ({action} {vault_name}/{path}): {e}")
try:
_semantic.on_index_change(action, vault_name, path, file_info)
except Exception as e:
logger.warning(f"Semantic index incremental update failed ({action} {vault_name}/{path}): {e}")
# Register the hook with indexer (indexer is already imported at top of file)
@@ -723,60 +738,97 @@ def search(
query_lower = query.lower()
results: list[dict[str, Any]] = []
for vault_name, vault_data in index.items():
if vault_filter != "all" and vault_name != vault_filter:
inv = get_inverted_index()
use_index = (not inv.is_stale()) and inv.doc_count > 0
if use_index:
# BUG-033: retrieve candidates from the inverted index instead of
# scanning every document. Multi-term queries require all terms
# (a superset of exact-phrase matches), single terms use prefix
# expansion. Falls back to a full scan while the index is building.
if has_query:
terms = [t for t in tokenize(query) if t]
if not terms:
return []
doc_sets: list[set] = []
for term in terms:
term_docs: set = set(inv.word_index.get(term, {}).keys())
if len(term) >= MIN_PREFIX_LENGTH:
for expanded in inv.get_prefix_tokens(term):
term_docs.update(inv.word_index.get(expanded, {}).keys())
doc_sets.append(term_docs)
doc_keys = set.intersection(*doc_sets) if doc_sets else set()
else:
doc_keys = set(inv.doc_info.keys())
if vault_filter != "all":
doc_keys &= inv.vault_docs.get(vault_filter, set())
for tag in selected_tags:
doc_keys &= inv.tag_docs.get(tag.lower(), set())
candidates = [
(inv.doc_vault[dk], inv.doc_info[dk])
for dk in doc_keys
if dk in inv.doc_info
]
else:
candidates = [
(vault_name, file_info)
for vault_name, vault_data in index.items()
if vault_filter == "all" or vault_name == vault_filter
for file_info in vault_data["files"]
]
for vault_name, file_info in candidates:
# Tag filter: all selected tags must be present
if selected_tags and not all(tag in file_info["tags"] for tag in selected_tags):
continue
for file_info in vault_data["files"]:
# Tag filter: all selected tags must be present
if selected_tags and not all(tag in file_info["tags"] for tag in selected_tags):
continue
score = 0
snippet = file_info.get("content_preview", "")
score = 0
snippet = file_info.get("content_preview", "")
if has_query:
title_lower = file_info["title"].lower()
if has_query:
title_lower = file_info["title"].lower()
# Exact title match (highest weight)
if query_lower == title_lower:
score += 20
# Partial title match
elif query_lower in title_lower:
score += 10
# Exact title match (highest weight)
if query_lower == title_lower:
score += 20
# Partial title match
elif query_lower in title_lower:
score += 10
# Path match (folder/filename relevance)
if query_lower in file_info["path"].lower():
score += 5
# Path match (folder/filename relevance)
if query_lower in file_info["path"].lower():
score += 5
# Tag name match
for tag in file_info.get("tags", []):
if query_lower in tag.lower():
score += 3
break # count once per file
# Tag name match
for tag in file_info.get("tags", []):
if query_lower in tag.lower():
score += 3
break # count once per file
# Content match — use cached content (no disk I/O)
content = file_info.get("content", "")
content_lower = content.lower()
if query_lower in content_lower:
# Frequency-based scoring, capped to avoid over-weighting
occurrences = content_lower.count(query_lower)
score += min(occurrences, 10)
snippet = _extract_snippet(content, query)
else:
# Tag-only filter: all matching files get score 1
score = 1
# Content match — use cached content (no disk I/O)
content = file_info.get("content", "")
content_lower = content.lower()
if query_lower in content_lower:
# Frequency-based scoring, capped to avoid over-weighting
occurrences = content_lower.count(query_lower)
score += min(occurrences, 10)
snippet = _extract_snippet(content, query)
else:
# Tag-only filter: all matching files get score 1
score = 1
if score > 0:
results.append({
"vault": vault_name,
"path": file_info["path"],
"title": file_info["title"],
"tags": file_info["tags"],
"score": score,
"snippet": snippet,
"modified": file_info["modified"],
})
if score > 0:
results.append({
"vault": vault_name,
"path": file_info["path"],
"title": file_info["title"],
"tags": file_info["tags"],
"score": score,
"snippet": snippet,
"modified": file_info["modified"],
})
results.sort(key=lambda x: -x["score"])
return results[:limit]
@@ -902,12 +954,14 @@ def _passes_search_filters(
title = file_info.get("title", "")
content = file_info.get("content", "")
path = file_info.get("path", "")
search_text = f"{title} {content}"
# BUG-025: cap the text scanned by a user-supplied regex.
search_text = truncate_for_regex(f"{title} {content}")
search_text_norm = normalize_text(search_text)
# --- Regex mode ---
if regex and raw_query:
try:
validate_regex(raw_query)
flags = 0 if case_sensitive else re.IGNORECASE
if whole_word:
pattern = re.compile(rf"\b{raw_query}\b", flags)
@@ -915,7 +969,7 @@ def _passes_search_filters(
pattern = re.compile(raw_query, flags)
if not pattern.search(search_text):
return False
except re.error:
except (re.error, ValueError):
return False
return _passes_path_filters(path, include_paths, exclude_paths)
@@ -1102,6 +1156,7 @@ def advanced_search(
created: str | None = None,
modified: str | None = None,
size: str | None = None,
semantic: bool = False,
) -> dict[str, Any]:
"""Advanced full-text search with TF-IDF scoring, facets, and pagination.
@@ -1121,13 +1176,21 @@ def advanced_search(
limit: Max results per page.
offset: Pagination offset.
sort_by: ``"relevance"`` or ``"modified"``.
semantic: When True, fuse the TF-IDF ranking with the semantic
(embedding) ranking via Reciprocal Rank Fusion and expose a
``semantic_score`` per result.
Returns:
Dict with ``results``, ``total``, ``offset``, ``limit``, ``facets``,
``query_time_ms``.
``query_time_ms`` and ``semantic_available``.
"""
t0 = time.monotonic()
query = query.strip() if query else ""
# BUG-025: reject oversized / catastrophic regex patterns up front.
if regex and query:
validate_regex(query)
parsed = _parse_advanced_query(query)
# Merge explicit tag_filter with parsed tag: operators
@@ -1182,58 +1245,63 @@ def advanced_search(
# ------------------------------------------------------------------
# Step 2: Apply filters on candidate set
# ------------------------------------------------------------------
if effective_vault != "all":
candidates &= inv.vault_docs.get(effective_vault, set())
if all_tags and has_terms:
for t in all_tags:
candidates &= inv.tag_docs.get(t.lower(), set())
if parsed["title"]:
norm_title_filter = normalize_text(parsed["title"])
candidates = {
dk for dk in candidates
if norm_title_filter in normalize_text(inv.doc_info[dk].get("title", ""))
}
if parsed["path"]:
norm_path_filter = normalize_text(parsed["path"])
candidates = {
dk for dk in candidates
if norm_path_filter in normalize_text(inv.doc_info[dk].get("path", ""))
}
if parsed["ext"]:
ext_filter = parsed["ext"]
candidates = {
dk for dk in candidates
if (
inv.doc_info[dk].get("path", "").rsplit("/", 1)[-1].lower() == ext_filter
or inv.doc_info[dk].get("path", "").rsplit("/", 1)[-1].lower().endswith(f".{ext_filter}")
)
}
# Date and size filters (from query operators or API params)
date_range_created = _parse_date_range(created or parsed.get("created"))
if date_range_created:
candidates = {
dk for dk in candidates
if _matches_date_range(inv.doc_info[dk].get("created"), date_range_created)
}
date_range_modified = _parse_date_range(modified or parsed.get("modified"))
if date_range_modified:
candidates = {
dk for dk in candidates
if _matches_date_range(inv.doc_info[dk].get("modified"), date_range_modified)
}
size_range = _parse_size_range(size or parsed.get("size"))
if size_range:
candidates = {
dk for dk in candidates
if _matches_size_range(inv.doc_info[dk].get("size", 0), size_range)
}
def _apply_metadata_filters(docs: set) -> set:
"""Restrict a document-key set to the query's metadata filters."""
if effective_vault != "all":
docs &= inv.vault_docs.get(effective_vault, set())
if all_tags:
for t in all_tags:
docs &= inv.tag_docs.get(t.lower(), set())
if parsed["title"]:
norm_title_filter = normalize_text(parsed["title"])
docs = {
dk for dk in docs
if norm_title_filter in normalize_text(inv.doc_info[dk].get("title", ""))
}
if parsed["path"]:
norm_path_filter = normalize_text(parsed["path"])
docs = {
dk for dk in docs
if norm_path_filter in normalize_text(inv.doc_info[dk].get("path", ""))
}
if parsed["ext"]:
ext_filter = parsed["ext"]
docs = {
dk for dk in docs
if (
inv.doc_info[dk].get("path", "").rsplit("/", 1)[-1].lower() == ext_filter
or inv.doc_info[dk].get("path", "").rsplit("/", 1)[-1].lower().endswith(f".{ext_filter}")
)
}
if date_range_created:
docs = {
dk for dk in docs
if _matches_date_range(inv.doc_info[dk].get("created"), date_range_created)
}
if date_range_modified:
docs = {
dk for dk in docs
if _matches_date_range(inv.doc_info[dk].get("modified"), date_range_modified)
}
if size_range:
docs = {
dk for dk in docs
if _matches_size_range(inv.doc_info[dk].get("size", 0), size_range)
}
return docs
candidates = _apply_metadata_filters(candidates)
# ------------------------------------------------------------------
# Step 3: Score only the candidates (not all N documents)
@@ -1312,16 +1380,22 @@ def advanced_search(
"title": file_info["title"],
"tags": file_info.get("tags", []),
"score": round(score, 4),
"semantic_score": 0.0,
"snippet": snippet,
"modified": file_info.get("modified", ""),
"extension": file_info.get("extension", file_info.get("path", "").rsplit(".", 1)[-1] if "." in file_info.get("path", "") else ""),
}
scored_results.append((score, result))
# Facets
facet_vaults[vault_name] = facet_vaults.get(vault_name, 0) + 1
for tag in file_info.get("tags", []):
facet_tags[tag] = facet_tags.get(tag, 0) + 1
# ------------------------------------------------------------------
# Step 4: Optional semantic fusion (RRF with the TF-IDF ranking)
# ------------------------------------------------------------------
semantic_available = _semantic.get_semantic_index().is_ready()
if semantic and has_terms and not regex:
scored_results = _fuse_semantic_results(
scored_results, query, effective_vault, inv, _apply_metadata_filters,
include_paths, exclude_paths, limit,
)
# Sort
if sort_by == "modified":
@@ -1329,6 +1403,12 @@ def advanced_search(
else:
scored_results.sort(key=lambda x: -x[0])
# Facets are recomputed from the final result set (covers semantic-only docs)
for _, result in scored_results:
facet_vaults[result["vault"]] = facet_vaults.get(result["vault"], 0) + 1
for tag in result.get("tags", []):
facet_tags[tag] = facet_tags.get(tag, 0) + 1
total = len(scored_results)
page = scored_results[offset: offset + limit]
elapsed_ms = round((time.monotonic() - t0) * 1000, 1)
@@ -1343,9 +1423,99 @@ def advanced_search(
"vaults": dict(sorted(facet_vaults.items(), key=lambda x: -x[1])),
},
"query_time_ms": elapsed_ms,
"semantic_available": semantic_available,
}
def _fuse_semantic_results(
scored_results: list[tuple[float, dict[str, Any]]],
query: str,
vault_filter: str,
inv: InvertedIndex,
apply_metadata_filters: Callable[[set], set],
include_paths: str | None,
exclude_paths: str | None,
limit: int,
) -> list[tuple[float, dict[str, Any]]]:
"""Fuse the TF-IDF ranking with the semantic ranking using RRF.
Documents found only by the semantic engine are materialized from the
inverted index metadata (with a plain, non-highlighted snippet). The
returned tuples carry the fused score, and every result dict gets a
``semantic_score`` (cosine similarity, 0.0 when absent).
Args:
scored_results: Existing ``(tfidf_score, result_dict)`` tuples.
query: Raw free-text query.
vault_filter: Effective vault filter.
inv: Inverted index (document metadata source).
apply_metadata_filters: Callable restricting a doc-key set to the
query's tag/title/path/ext/date/size filters.
include_paths: Include glob patterns (or None).
exclude_paths: Exclude glob patterns (or None).
limit: Requested page size (drives how many semantic hits to fetch).
Returns:
New ``(fused_score, result_dict)`` list (unsorted).
"""
sem_hits = _semantic.semantic_search_docs(query, vault_filter=vault_filter, top_k=max(limit * 5, 200))
if not sem_hits:
return scored_results
# Semantic candidates must satisfy the same metadata + path filters.
semantic_universe = apply_metadata_filters(set(inv.doc_info.keys()))
semantic_universe = {
dk for dk in semantic_universe
if _passes_path_filters(inv.doc_info[dk].get("path", ""), include_paths, exclude_paths)
}
sem_scores: dict[str, float] = {}
sem_ranked: list[str] = []
for doc_key, similarity in sem_hits:
if doc_key not in semantic_universe:
continue
sem_scores[doc_key] = similarity
sem_ranked.append(doc_key)
if not sem_ranked:
return scored_results
existing: dict[str, dict[str, Any]] = {}
lexical_ranked: list[str] = []
for _, lex_result in sorted(scored_results, key=lambda item: -item[0]):
key = f"{lex_result['vault']}::{lex_result['path']}"
existing[key] = lex_result
lexical_ranked.append(key)
fused = _semantic.rrf_fuse([lexical_ranked, sem_ranked])
merged: list[tuple[float, dict[str, Any]]] = []
for doc_key, fused_score in fused.items():
result = existing.get(doc_key)
if result is None:
file_info = inv.doc_info.get(doc_key)
if file_info is None:
continue
content = file_info.get("content", "")
result = {
"vault": inv.doc_vault[doc_key],
"path": file_info["path"],
"title": file_info["title"],
"tags": file_info.get("tags", []),
"score": 0.0,
"semantic_score": 0.0,
"snippet": _escape_html(content[:200].strip()) if content else "",
"modified": file_info.get("modified", ""),
"extension": file_info.get(
"extension",
file_info.get("path", "").rsplit(".", 1)[-1] if "." in file_info.get("path", "") else "",
),
}
result["semantic_score"] = round(sem_scores.get(doc_key, 0.0), 4)
merged.append((fused_score, result))
return merged
# ---------------------------------------------------------------------------
# Suggestion helpers
# ---------------------------------------------------------------------------
+612
View File
@@ -0,0 +1,612 @@
"""ObsiGate — Semantic search: embeddings, vector store and hybrid retrieval.
This module adds a *semantic* layer on top of the existing TF-IDF search. Each
document is split into overlapping chunks, each chunk is converted into a dense
vector, and queries are matched by cosine similarity. Results are combined with
the lexical ranking through Reciprocal Rank Fusion (RRF).
Design goals
------------
* **Zero mandatory dependency.** ``sentence-transformers`` (local model),
``numpy`` and ``faiss`` are *optional*. They are imported lazily and, when
missing, the module falls back to a deterministic pure-Python hashing embedder
and a pure-Python cosine store. The feature therefore degrades gracefully and
the default CI (which only installs ``backend/requirements.txt``) keeps working.
* **Plug-in providers.** Embeddings can come from the local
``all-MiniLM-L6-v2`` model, from an OpenAI-compatible ``/embeddings`` endpoint
(configured via env vars), or from the deterministic fallback.
* **Incremental.** The index is updated document-by-document from the indexer
change hook (file watcher + API mutations), never rebuilt on each search.
Optional extras are listed in ``backend/requirements-semantic.txt``.
"""
from __future__ import annotations
import hashlib
import logging
import math
import os
import re
import threading
from abc import ABC, abstractmethod
from collections import Counter
from itertools import pairwise
from typing import Any
logger = logging.getLogger("obsigate.semantic")
# ---------------------------------------------------------------------------
# Constants
# ---------------------------------------------------------------------------
EMBEDDING_DIM = 384 # all-MiniLM-L6-v2 output dimension
CHUNK_TOKENS = 512 # target chunk size (whitespace tokens)
CHUNK_OVERLAP_TOKENS = 64 # overlap between consecutive chunks
DEFAULT_TOP_K = 200 # max documents returned by a semantic query
RRF_K = 60 # Reciprocal Rank Fusion smoothing constant
MAX_QUERY_CHARS = 2000 # guard against pathological queries
_WORD_RE = re.compile(r"[\w]+", re.UNICODE)
# ---------------------------------------------------------------------------
# Tokenization / chunking
# ---------------------------------------------------------------------------
def _simple_tokens(text: str) -> list[str]:
"""Split *text* into lowercase word tokens (keeps accents)."""
return _WORD_RE.findall(text.lower())
def chunk_text(
text: str,
chunk_tokens: int = CHUNK_TOKENS,
overlap: int = CHUNK_OVERLAP_TOKENS,
) -> list[str]:
"""Split *text* into overlapping windows of roughly *chunk_tokens* words.
Args:
text: Raw document text.
chunk_tokens: Target number of whitespace tokens per chunk.
overlap: Number of tokens shared by two consecutive chunks.
Returns:
A list of chunk strings. Empty input yields an empty list.
"""
if not text or not text.strip():
return []
if chunk_tokens <= 0:
chunk_tokens = CHUNK_TOKENS
overlap = max(0, min(overlap, chunk_tokens - 1))
words = text.split()
if len(words) <= chunk_tokens:
return [" ".join(words)]
step = max(1, chunk_tokens - overlap)
chunks: list[str] = []
for start in range(0, len(words), step):
window = words[start:start + chunk_tokens]
if not window:
break
chunks.append(" ".join(window))
if start + chunk_tokens >= len(words):
break
return chunks
# ---------------------------------------------------------------------------
# Embedding providers
# ---------------------------------------------------------------------------
class EmbeddingProvider(ABC):
"""Base class for embedding backends."""
name: str = "base"
def __init__(self, dimension: int = EMBEDDING_DIM) -> None:
self.dimension = dimension
@abstractmethod
def encode(self, texts: list[str]) -> list[list[float]]:
"""Return one L2-normalized vector per input text."""
def encode_one(self, text: str) -> list[float]:
"""Convenience wrapper returning the vector for a single text."""
vectors = self.encode([text])
return vectors[0] if vectors else [0.0] * self.dimension
class HashEmbeddingProvider(EmbeddingProvider):
"""Deterministic, dependency-free hashing embedder.
This is a *lexical* fallback: it hashes word unigrams, word bigrams and
character trigrams into fixed-size signed buckets (the "hashing trick"),
then L2-normalizes the result. It captures shared vocabulary and
morphological variants (``backup``/``backups``), so it already improves
recall over exact TF-IDF matching, but it does not understand synonyms the
way a real transformer model does.
"""
name = "hash"
def _add_feature(self, vec: list[float], key: str, weight: float) -> None:
digest = hashlib.blake2b(key.encode("utf-8"), digest_size=8).digest()
h = int.from_bytes(digest, "big")
idx = h % self.dimension
sign = 1.0 if (h >> 63) & 1 else -1.0
vec[idx] += sign * weight
def _encode_one(self, text: str) -> list[float]:
vec = [0.0] * self.dimension
tokens = _simple_tokens(text)
if not tokens:
return vec
tf = Counter(tokens)
for token, count in tf.items():
weight = 1.0 + math.log(count)
self._add_feature(vec, "w:" + token, weight)
for gram in _char_ngrams(token, 3):
self._add_feature(vec, "g:" + gram, weight * 0.5)
for first, second in pairwise(tokens):
self._add_feature(vec, "b:" + first + "_" + second, 0.5)
norm = math.sqrt(sum(v * v for v in vec))
if norm > 0.0:
vec = [v / norm for v in vec]
return vec
def encode(self, texts: list[str]) -> list[list[float]]:
return [self._encode_one(t or "") for t in texts]
def _char_ngrams(token: str, n: int) -> list[str]:
"""Return padded character n-grams for *token* (bounded to avoid blow-up)."""
if len(token) < n:
return [token]
if len(token) > 24:
token = token[:24]
return [token[i:i + n] for i in range(len(token) - n + 1)]
class SentenceTransformerProvider(EmbeddingProvider):
"""Local ``all-MiniLM-L6-v2`` embeddings via ``sentence-transformers``."""
name = "sentence-transformers"
def __init__(self, model_name: str = "all-MiniLM-L6-v2") -> None:
super().__init__(EMBEDDING_DIM)
self.model_name = model_name
self._model: Any | None = None
@staticmethod
def is_available() -> bool:
try:
import sentence_transformers # noqa: F401
except Exception:
return False
return True
def _get_model(self) -> Any:
if self._model is None:
from sentence_transformers import SentenceTransformer
self._model = SentenceTransformer(self.model_name)
return self._model
def encode(self, texts: list[str]) -> list[list[float]]:
if not texts:
return []
model = self._get_model()
vectors = model.encode(texts, normalize_embeddings=True)
return [[float(x) for x in vec] for vec in vectors]
class RemoteEmbeddingProvider(EmbeddingProvider):
"""OpenAI-compatible ``/embeddings`` endpoint (API key based)."""
name = "remote"
def __init__(
self,
base_url: str,
api_key: str,
model: str = "text-embedding-3-small",
dimension: int = EMBEDDING_DIM,
) -> None:
super().__init__(dimension)
self.base_url = base_url.rstrip("/")
self.api_key = api_key
self.model = model
def encode(self, texts: list[str]) -> list[list[float]]:
if not texts:
return []
import httpx
response = httpx.post(
f"{self.base_url}/embeddings",
headers={"Authorization": f"Bearer {self.api_key}"},
json={"model": self.model, "input": texts},
timeout=30.0,
)
response.raise_for_status()
payload = response.json()
data = sorted(payload.get("data", []), key=lambda item: item.get("index", 0))
return [self._normalize([float(x) for x in item["embedding"]]) for item in data]
@staticmethod
def _normalize(vec: list[float]) -> list[float]:
norm = math.sqrt(sum(v * v for v in vec))
if norm > 0.0:
return [v / norm for v in vec]
return vec
_provider: EmbeddingProvider | None = None
_provider_lock = threading.Lock()
def _build_provider() -> EmbeddingProvider:
"""Select the best available provider (respecting ``OBSIGATE_EMBEDDING_PROVIDER``)."""
requested = os.getenv("OBSIGATE_EMBEDDING_PROVIDER", "auto").strip().lower()
if requested in ("auto", "local", "sentence-transformers") and SentenceTransformerProvider.is_available():
model = os.getenv("OBSIGATE_EMBEDDING_MODEL", "all-MiniLM-L6-v2")
return SentenceTransformerProvider(model)
remote_key = os.getenv("OBSIGATE_EMBEDDING_API_KEY", "")
remote_url = os.getenv("OBSIGATE_EMBEDDING_BASE_URL", "")
if requested in ("auto", "remote") and remote_key and remote_url:
model = os.getenv("OBSIGATE_EMBEDDING_MODEL", "text-embedding-3-small")
dim = int(os.getenv("OBSIGATE_EMBEDDING_DIM", str(EMBEDDING_DIM)))
return RemoteEmbeddingProvider(remote_url, remote_key, model, dim)
if requested == "remote":
logger.warning(
"OBSIGATE_EMBEDDING_PROVIDER=remote but OBSIGATE_EMBEDDING_API_KEY/BASE_URL missing; using hash fallback"
)
return HashEmbeddingProvider()
def get_embedding_provider() -> EmbeddingProvider:
"""Return the cached embedding provider (built on first use)."""
global _provider
with _provider_lock:
if _provider is None:
_provider = _build_provider()
logger.info("Semantic embedding provider: %s (dim=%d)", _provider.name, _provider.dimension)
return _provider
def reset_embedding_provider() -> None:
"""Forget the cached provider (used by tests and config reloads)."""
global _provider
with _provider_lock:
_provider = None
# ---------------------------------------------------------------------------
# Vector store
# ---------------------------------------------------------------------------
class VectorStore:
"""In-memory vector store with optional numpy / faiss acceleration.
Vectors are always kept as Python lists (source of truth). A numpy matrix
and/or a faiss ``IndexFlatIP`` are built lazily and invalidated on mutation.
All vectors are expected to be L2-normalized, so the inner product equals
the cosine similarity.
"""
def __init__(self, dimension: int = EMBEDDING_DIM) -> None:
self.dimension = dimension
self._keys: list[str] = []
self._chunks: list[str] = []
self._vectors: list[list[float]] = []
self._dirty = True
self._numpy: Any | None = None
self._numpy_checked = False
self._matrix: Any | None = None
self._faiss: Any | None = None
self._faiss_checked = False
self._faiss_index: Any | None = None
def __len__(self) -> int:
return len(self._vectors)
def clear(self) -> None:
self._keys = []
self._chunks = []
self._vectors = []
self._dirty = True
def add(self, key: str, chunk: str, vector: list[float]) -> None:
self._keys.append(key)
self._chunks.append(chunk)
self._vectors.append(vector)
self._dirty = True
def remove_document(self, key: str) -> None:
"""Remove every chunk belonging to *key*."""
kept = [(k, c, v) for k, c, v in zip(self._keys, self._chunks, self._vectors) if k != key]
if len(kept) == len(self._vectors):
return
self._keys = [k for k, _, _ in kept]
self._chunks = [c for _, c, _ in kept]
self._vectors = [v for _, _, v in kept]
self._dirty = True
# -- optional accelerators -------------------------------------------------
def _get_numpy(self) -> Any | None:
if not self._numpy_checked:
self._numpy_checked = True
try:
import numpy as np
self._numpy = np
except Exception:
self._numpy = None
return self._numpy
def _get_faiss(self) -> Any | None:
if not self._faiss_checked:
self._faiss_checked = True
try:
import faiss
self._faiss = faiss
except Exception:
self._faiss = None
return self._faiss
def _rebuild_accelerators(self) -> None:
self._dirty = False
np = self._get_numpy()
if np is None or not self._vectors:
self._matrix = None
self._faiss_index = None
return
self._matrix = np.asarray(self._vectors, dtype="float32")
faiss = self._get_faiss()
if faiss is not None:
index = faiss.IndexFlatIP(self.dimension)
index.add(self._matrix)
self._faiss_index = index
else:
self._faiss_index = None
def search(self, query_vector: list[float], top_k: int = DEFAULT_TOP_K) -> list[tuple[str, float]]:
"""Return ``(doc_key, cosine_similarity)`` pairs sorted by similarity."""
if not self._vectors:
return []
top_k = max(1, min(top_k, len(self._vectors)))
if self._dirty:
self._rebuild_accelerators()
np = self._get_numpy()
if np is not None and self._faiss_index is not None and self._matrix is not None:
query = np.asarray([query_vector], dtype="float32")
scores, indices = self._faiss_index.search(query, top_k)
return [
(self._keys[int(idx)], float(score))
for score, idx in zip(scores[0], indices[0])
if idx >= 0
]
if np is not None and self._matrix is not None:
query = np.asarray(query_vector, dtype="float32")
scores = self._matrix @ query
order = np.argsort(scores)[::-1][:top_k]
return [(self._keys[int(i)], float(scores[int(i)])) for i in order]
scored = [(self._keys[i], _dot(self._vectors[i], query_vector)) for i in range(len(self._vectors))]
scored.sort(key=lambda item: item[1], reverse=True)
return scored[:top_k]
def chunk_of(self, index: int) -> str:
"""Return the stored chunk text at *index* (used by diagnostics/tests)."""
return self._chunks[index]
def _dot(a: list[float], b: list[float]) -> float:
"""Dot product for two equal-length vectors."""
return sum(x * y for x, y in zip(a, b))
# ---------------------------------------------------------------------------
# Reciprocal Rank Fusion
# ---------------------------------------------------------------------------
def rrf_fuse(rankings: list[list[str]], k: int = RRF_K) -> dict[str, float]:
"""Fuse several ranked key lists into a single score map.
``score(key) = Σ_rankings 1 / (k + rank(key))`` where ``rank`` is
1-based. Documents ranked highly by several methods rise to the top.
Args:
rankings: Ordered lists of document keys (best first).
k: RRF smoothing constant.
Returns:
Mapping ``doc_key -> fused score`` (insertion order is unspecified).
"""
scores: dict[str, float] = {}
for ranking in rankings:
seen: set[str] = set()
for rank, key in enumerate(ranking):
if key in seen:
continue
seen.add(key)
scores[key] = scores.get(key, 0.0) + 1.0 / (k + rank + 1)
return scores
# ---------------------------------------------------------------------------
# Semantic index
# ---------------------------------------------------------------------------
class SemanticIndex:
"""Holds document chunk embeddings and answers similarity queries."""
def __init__(self, provider: EmbeddingProvider | None = None) -> None:
self.provider = provider
self.store = VectorStore(provider.dimension if provider else EMBEDDING_DIM)
self.doc_keys: set[str] = set()
self._ready = False
self._lock = threading.Lock()
def is_ready(self) -> bool:
"""Return True once a full rebuild has completed."""
return self._ready
def is_stale(self) -> bool:
"""Alias used by callers that check index freshness."""
return not self._ready
def _ensure_provider(self) -> EmbeddingProvider:
if self.provider is None:
self.provider = get_embedding_provider()
self.store = VectorStore(self.provider.dimension)
return self.provider
@staticmethod
def _document_text(file_info: dict[str, Any]) -> str:
title = file_info.get("title", "") or ""
content = file_info.get("content", "") or ""
return (title + "\n\n" + content).strip()
def _embed_document(self, doc_key: str, file_info: dict[str, Any]) -> None:
text = self._document_text(file_info)
if not text:
return
provider = self._ensure_provider()
chunks = chunk_text(text)
if not chunks:
return
vectors = provider.encode(chunks)
for chunk, vector in zip(chunks, vectors):
self.store.add(doc_key, chunk, vector)
self.doc_keys.add(doc_key)
def rebuild(self) -> None:
"""Rebuild the whole index from the global in-memory index."""
from backend.indexer import index
provider = self._ensure_provider()
with self._lock:
self.store = VectorStore(provider.dimension)
self.doc_keys = set()
for vault_name, vault_data in index.items():
for file_info in vault_data.get("files", []):
doc_key = f"{vault_name}::{file_info.get('path', '')}"
try:
self._embed_document(doc_key, file_info)
except Exception as exc:
logger.warning("Semantic embedding failed for %s: %s", doc_key, exc)
self._ready = True
logger.info(
"Semantic index built: %d documents, %d chunks (provider=%s)",
len(self.doc_keys),
len(self.store),
provider.name,
)
def add_document(self, vault_name: str, path: str, file_info: dict[str, Any]) -> None:
"""Add or refresh a single document (no-op until the index is ready)."""
if not self._ready or not file_info:
return
doc_key = f"{vault_name}::{path}"
with self._lock:
self.store.remove_document(doc_key)
self.doc_keys.discard(doc_key)
try:
self._embed_document(doc_key, file_info)
except Exception as exc:
logger.warning("Semantic embedding failed for %s: %s", doc_key, exc)
def remove_document(self, vault_name: str, path: str) -> None:
"""Remove a single document (no-op until the index is ready)."""
if not self._ready:
return
doc_key = f"{vault_name}::{path}"
with self._lock:
self.store.remove_document(doc_key)
self.doc_keys.discard(doc_key)
def search(
self,
query: str,
vault_filter: str = "all",
top_k: int = DEFAULT_TOP_K,
) -> list[tuple[str, float]]:
"""Return ``(doc_key, best_chunk_similarity)`` pairs, best first."""
if not self._ready or not query or not query.strip():
return []
provider = self._ensure_provider()
query_vector = provider.encode_one(query[:MAX_QUERY_CHARS])
hits = self.store.search(query_vector, top_k=max(top_k * 4, top_k))
best: dict[str, float] = {}
for doc_key, score in hits:
if vault_filter != "all" and not doc_key.startswith(vault_filter + "::"):
continue
if doc_key not in best or score > best[doc_key]:
best[doc_key] = score
ordered = sorted(best.items(), key=lambda item: item[1], reverse=True)
return ordered[:top_k]
_semantic_index: SemanticIndex | None = None
_index_lock = threading.Lock()
def get_semantic_index() -> SemanticIndex:
"""Return the process-wide semantic index (created on first access)."""
global _semantic_index
with _index_lock:
if _semantic_index is None:
_semantic_index = SemanticIndex()
return _semantic_index
def reset_semantic_index() -> None:
"""Drop the singleton index (tests)."""
global _semantic_index
with _index_lock:
_semantic_index = None
def init_semantic_index() -> None:
"""Force a full semantic index build. Called after ``build_index`` on startup."""
from backend.indexer import index
if any(vdata.get("files") for vdata in index.values()):
get_semantic_index().rebuild()
def on_index_change(action: str, vault_name: str, path: str, file_info: dict[str, Any]) -> None:
"""Incremental hook registered with the indexer change notifier."""
index_obj = get_semantic_index()
if action == "add" and file_info:
index_obj.add_document(vault_name, path, file_info)
elif action == "remove":
index_obj.remove_document(vault_name, path)
def semantic_search_docs(
query: str,
vault_filter: str = "all",
top_k: int = DEFAULT_TOP_K,
) -> list[tuple[str, float]]:
"""Convenience wrapper around :meth:`SemanticIndex.search`."""
return get_semantic_index().search(query, vault_filter=vault_filter, top_k=top_k)
def semantic_status() -> dict[str, Any]:
"""Return provider/index diagnostics for the API and the UI."""
index_obj = get_semantic_index()
provider = index_obj.provider or get_embedding_provider()
return {
"available": index_obj.is_ready(),
"provider": provider.name,
"dimension": provider.dimension,
"documents": len(index_obj.doc_keys),
"chunks": len(index_obj.store),
}
+263
View File
@@ -0,0 +1,263 @@
"""Backup inspection services shared by REST routes and the AI tool layer.
Single source of truth for locating, listing and diffing the timestamped
backups created before each file mutation. The write-side (creating a backup)
stays in the route layer; only the read/inspection logic lives here so both
``/api/file/{vault}/backups``, ``/api/file/{vault}/diff`` and the tools
``list_backups`` / ``diff_backup`` behave identically.
"""
from __future__ import annotations
import difflib
import logging
import os
import shutil
import time
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
from backend.services.errors import ServiceError
from backend.services.paths import resolve_safe_path
from backend.services.vaults import get_vault_root
logger = logging.getLogger("obsigate.services.backups")
# Default number of backups kept per file (matches ``max_backups_per_file``).
DEFAULT_MAX_BACKUPS = 10
def _default_max_backups() -> int:
"""Read ``max_backups_per_file`` from app config (lazy, best-effort)."""
try:
from backend.main import _load_config
return int(_load_config().get("max_backups_per_file", DEFAULT_MAX_BACKUPS))
except Exception: # pragma: no cover - config unavailable
return DEFAULT_MAX_BACKUPS
def get_backup_dir(vault_name: str, relative_path: str) -> Path:
"""Return the directory where backups for a specific file are stored.
Resolves relative backup paths against the SPECIFIC vault's directory,
matching the backup-creation logic exactly.
"""
from backend.indexer import get_vault_data
backup_root = Path(os.environ.get("OBSIGATE_BACKUP_DIR", ".obsigate-backup"))
if not backup_root.is_absolute():
vault_data = get_vault_data(vault_name)
if vault_data:
backup_root = Path(vault_data["path"]) / backup_root
return backup_root / vault_name / Path(relative_path).parent
def create_backup(
file_path: Path,
vault_name: str,
relative_path: str,
*,
max_backups: int | None = None,
) -> Path | None:
"""Create a timestamped backup of a file before it is modified.
Backups are stored as ``{backup_root}/{vault}/{relative_path}.{ts}.bak``.
Missing or unreadable files are skipped silently (returns ``None``) so a
backup failure never blocks the caller's mutation.
Args:
file_path: Absolute path of the file to back up.
vault_name: Name of the vault the file belongs to.
relative_path: Vault-relative path (used to mirror the tree).
max_backups: Number of backups to keep per file. ``None`` reads
``max_backups_per_file`` from the app config.
Returns:
The created backup path, or ``None`` when skipped.
"""
try:
if not file_path.exists() or not file_path.is_file():
logger.debug(f"Backup skipped: file not found {file_path}")
return None
backup_dir = get_backup_dir(vault_name, relative_path)
backup_dir.mkdir(parents=True, exist_ok=True)
backup_path = backup_dir / f"{file_path.name}.{int(time.time())}.bak"
shutil.copy2(file_path, backup_path)
logger.info(f"Backup created: {relative_path} -> {backup_path}")
keep = max_backups if max_backups is not None else _default_max_backups()
all_backups = sorted(
[
f
for f in backup_dir.iterdir()
if f.is_file() and f.name.startswith(file_path.name + ".") and f.name.endswith(".bak")
],
key=lambda f: f.stat().st_mtime,
reverse=True,
)
for old in all_backups[keep:]:
old.unlink()
logger.debug(f"Auto-cleanup: removed old backup {old.name}")
return backup_path
except Exception as e:
logger.warning(f"Failed to backup {relative_path} (vault={vault_name}): {e}", exc_info=True)
return None
def list_backup_files(vault_name: str, relative_path: str) -> list[dict[str, Any]]:
"""List all backup files for a vault file, sorted newest first.
Backup filename format: ``{original_filename}.{timestamp}.bak``.
Returns a list of dicts with ``timestamp``, ``datetime``, ``size`` and
``filename``. Missing or unreadable backup directories yield an empty list.
"""
backup_dir = get_backup_dir(vault_name, relative_path)
if not backup_dir.exists():
return []
original_name = Path(relative_path).name
prefix = original_name + "."
backups: list[dict[str, Any]] = []
try:
dir_entries = list(backup_dir.iterdir())
except PermissionError:
logger.warning(f"Permission denied reading backup dir: {backup_dir}")
return []
except OSError as e:
logger.error(f"Error reading backup dir {backup_dir}: {e}")
return []
for f in dir_entries:
try:
if not f.is_file():
continue
name = f.name
if not name.startswith(prefix) or not name.endswith(".bak"):
continue
ts_part = name[len(prefix):-len(".bak")]
try:
ts = int(ts_part)
except (ValueError, TypeError):
continue
try:
dt = datetime.fromtimestamp(ts, tz=timezone.utc).isoformat()
except (OSError, OverflowError, ValueError) as ts_err:
logger.warning(f"Skipping backup with invalid timestamp {ts}: {ts_err}")
continue
st = f.stat()
backups.append({
"timestamp": ts,
"datetime": dt,
"size": st.st_size,
"filename": name,
})
except Exception as entry_err:
logger.warning(f"Skipping unreadable backup entry {f}: {entry_err}")
continue
backups.sort(key=lambda b: b["timestamp"], reverse=True)
return backups
def diff_backup(
vault_name: str,
path: str,
version: int,
compare_with: int | None = None,
) -> dict[str, Any]:
"""Generate a unified diff between a backup version and another version or the current file.
Args:
vault_name: Name of the vault.
path: Relative path of the file within the vault.
version: Timestamp of the backup used as the old/left side.
compare_with: Optional timestamp of another backup as the new/right
side. If omitted, the current file on disk is used.
Returns:
Dict with ``vault``, ``path``, ``version``, ``compare_with``, ``diff``,
``left_content`` and ``right_content``.
Raises:
ServiceError: ``not_found`` (404) when the file or a backup is missing,
``read_error`` (500) when a file cannot be read.
"""
root = get_vault_root(vault_name)
file_path = resolve_safe_path(root, path)
if not file_path.exists() or not file_path.is_file():
raise ServiceError(
f"File not found: {path}",
code="not_found",
status=404,
details={"vault": vault_name, "path": path},
)
original_name = Path(path).name
def _read_backup(ts: int) -> tuple[str, str]:
backup_dir = get_backup_dir(vault_name, path)
backup_path = backup_dir / f"{original_name}.{ts}.bak"
if not backup_path.exists():
raise ServiceError(
f"Backup version {ts} not found for {path}",
code="not_found",
status=404,
details={"vault": vault_name, "path": path, "version": ts},
)
try:
content = backup_path.read_text(encoding="utf-8", errors="replace")
except Exception as e:
raise ServiceError(
f"Failed to read backup {ts}: {e}",
code="read_error",
status=500,
details={"vault": vault_name, "path": path, "version": ts},
) from e
dt = datetime.fromtimestamp(ts, tz=timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC")
return content, f"{path}@{dt}"
try:
left_content, left_label = _read_backup(version)
if compare_with is not None:
right_content, right_label = _read_backup(compare_with)
else:
try:
right_content = file_path.read_text(encoding="utf-8", errors="replace")
except Exception as e:
raise ServiceError(
f"Failed to read current file: {e}",
code="read_error",
status=500,
details={"vault": vault_name, "path": path},
) from e
right_label = f"{path} (current)"
left_lines = left_content.splitlines(keepends=True)
right_lines = right_content.splitlines(keepends=True)
diff_lines = list(difflib.unified_diff(
left_lines, right_lines,
fromfile=left_label, tofile=right_label,
))
return {
"vault": vault_name,
"path": path,
"version": version,
"compare_with": compare_with,
"diff": "".join(diff_lines),
"left_content": left_content,
"right_content": right_content,
}
except ServiceError:
raise
except Exception as e:
logger.error(f"Error generating diff for {vault_name}/{path}: {type(e).__name__}: {e}", exc_info=True)
raise ServiceError(f"Erreur lors de la génération du diff: {e!s}", code="read_error", status=500) from e
+29
View File
@@ -0,0 +1,29 @@
"""Domain errors shared by the reusable service layer.
Services are transport-agnostic: they raise :class:`ServiceError` carrying a
stable ``code`` and an HTTP ``status`` hint. The REST layer maps it to an
``HTTPException`` (via the global handler in ``backend.main``) while the tool
layer maps it to a :class:`backend.tools.context.ToolError`.
"""
from __future__ import annotations
from typing import Any
class ServiceError(Exception):
"""Base error for the shared business-logic services."""
def __init__(
self,
message: str,
*,
code: str = "service_error",
status: int = 400,
details: dict[str, Any] | None = None,
):
super().__init__(message)
self.message = message
self.code = code
self.status = status
self.details = details or {}
+86
View File
@@ -0,0 +1,86 @@
"""File reading services shared by REST routes and the AI tool layer."""
from __future__ import annotations
import logging
from typing import Any
from backend.services.errors import ServiceError
from backend.services.paths import resolve_safe_path
from backend.services.vaults import get_vault_root
logger = logging.getLogger("obsigate.services.files")
def read_raw_file(vault_name: str, path: str) -> dict[str, Any]:
"""Return the raw text content of a vault file (no redaction)."""
root = get_vault_root(vault_name)
file_path = resolve_safe_path(root, path)
if not file_path.exists() or not file_path.is_file():
raise ServiceError(
f"File not found: {path}",
code="not_found",
status=404,
details={"vault": vault_name, "path": path},
)
try:
raw = file_path.read_text(encoding="utf-8", errors="replace")
except PermissionError as e:
logger.error(f"Permission denied reading raw file {path}: {e}")
raise ServiceError(f"Permission denied: cannot read file {path}", code="permission_denied", status=403) from e
except UnicodeDecodeError:
try:
raw = file_path.read_bytes().decode("utf-8", errors="replace")
except Exception as e:
logger.error(f"Error reading binary raw file {path}: {e}")
raise ServiceError(f"Cannot read file: {e!s}", code="read_error", status=500) from e
except Exception as e:
logger.error(f"Unexpected error reading raw file {path}: {e}")
raise ServiceError(f"Error reading file: {e!s}", code="read_error", status=500) from e
return {"vault": vault_name, "path": path, "raw": raw}
def read_file_text(
vault_name: str,
path: str,
*,
redact: bool = True,
max_bytes: int | None = None,
) -> dict[str, Any]:
"""Return a vault file's text content, optionally redacted and size-capped.
Raises:
ServiceError: ``not_found`` (404), ``file_too_large`` (413) or a read
error (500).
"""
root = get_vault_root(vault_name)
target = resolve_safe_path(root, path)
if not target.exists() or not target.is_file():
raise ServiceError(
f"File not found: {path}",
code="not_found",
status=404,
details={"vault": vault_name, "path": path},
)
size = target.stat().st_size
if max_bytes is not None and size > max_bytes:
raise ServiceError(
f"File too large ({size} bytes > {max_bytes})",
code="file_too_large",
status=413,
details={"vault": vault_name, "path": path, "size": size},
)
content = target.read_text(encoding="utf-8", errors="replace")
if redact:
from backend.secret_redactor import redact_file_content
content = redact_file_content(content, path)
return {"vault": vault_name, "path": path, "size": size, "content": content}
+199
View File
@@ -0,0 +1,199 @@
"""Graph data services shared by the REST route and the AI tool layer.
Builds the nodes/edges structure consumed by the graph view. The route
``/api/graph/{vault}`` and the tool ``get_graph`` both delegate here so the
graph semantics (parent/child edges, wikilinks, tag filter) stay identical.
Permission checks are the caller's responsibility.
"""
from __future__ import annotations
import re
from pathlib import Path
from typing import Any
from backend.indexer import SUPPORTED_EXTENSIONS, index
from backend.services.errors import ServiceError
from backend.services.paths import resolve_safe_path
from backend.services.vaults import get_vault_root
_WIKILINK_PATTERN = re.compile(r"\[\[([^\]|#]+)(?:[|#][^\]]+)?\]\]")
def get_graph(
vault_name: str,
path: str = "",
depth: int = 1,
scope: str = "directory",
tag: str = "",
) -> dict[str, Any]:
"""Return graph data (nodes and edges) for a vault or directory.
Args:
vault_name: Name of the vault.
path: Relative directory path to focus on (empty = root).
depth: Expansion depth (0 = only direct children, 1-3 = deeper).
scope: ``"directory"`` for a subtree, ``"full"`` for the whole vault.
tag: Optional tag filter (only files with this tag appear).
Returns:
Dict with ``vault``, ``path``, ``scope``, ``nodes`` and ``edges``.
Raises:
ServiceError: ``not_found`` (404) when the vault or path is missing.
"""
from backend.vault_settings import get_vault_setting
vault_root = get_vault_root(vault_name)
target = resolve_safe_path(vault_root, path) if path else vault_root.resolve()
if not target.exists():
raise ServiceError(
f"Path not found: {path}",
code="not_found",
status=404,
details={"vault": vault_name, "path": path},
)
nodes: list[dict[str, Any]] = []
edges: list[dict[str, Any]] = []
node_ids: set[str] = set()
def _add_node(name: str, ntype: str, npath: str, size: int = 0,
tags: list[str] | None = None, incoming: int = 0, outgoing: int = 0) -> str:
nid = f"{vault_name}:{npath}"
if nid not in node_ids:
node_ids.add(nid)
nodes.append({
"id": nid, "name": name, "type": ntype, "path": npath,
"size": size, "tags": tags or [],
"incoming_count": incoming, "outgoing_count": outgoing,
})
return nid
def _add_edge(source: str, target_id: str, relation: str) -> None:
edges.append({"source": source, "target": target_id, "relation": relation})
settings = get_vault_setting(vault_name) or {}
hide_hidden = settings.get("hideHiddenFiles", False)
# Build tag index from the in-memory index for fast lookups.
_tag_index: dict[str, list[str]] = {}
for doc_key, info in index.items():
vn, fp = doc_key.split("::", 1) if "::" in doc_key else ("", "")
if vn == vault_name:
for t in info.get("tags", []):
_tag_index.setdefault(t.lower(), []).append(fp)
if scope == "full":
target = vault_root.resolve()
effective_depth = depth if depth > 0 else 2
else:
target = resolve_safe_path(vault_root, path) if path else vault_root.resolve()
effective_depth = depth
if not target.exists():
raise ServiceError(
f"Path not found: {path}",
code="not_found",
status=404,
details={"vault": vault_name, "path": path},
)
focus_name = path.split("/")[-1] if path else vault_name
focus_type = "directory" if path else "vault"
focus_id = _add_node(focus_name, focus_type, path)
def _walk_dir(dir_path: Path, parent_id: str, current_depth: int) -> None:
if current_depth > effective_depth:
return
try:
for entry in sorted(dir_path.iterdir(), key=lambda e: (not e.is_dir(), e.name.lower())):
if hide_hidden and entry.name.startswith("."):
continue
rel = str(entry.relative_to(vault_root)).replace("\\", "/")
if tag and entry.is_file():
file_tags = [t.lower() for t in _tag_index.get(rel, [])]
if tag.lower() not in file_tags:
continue
if entry.is_dir():
did = _add_node(entry.name, "directory", rel)
_add_edge(parent_id, did, "parent")
if current_depth < effective_depth:
_walk_dir(entry, did, current_depth + 1)
elif entry.suffix.lower() in SUPPORTED_EXTENSIONS or entry.name.lower() in ("dockerfile", "makefile"):
file_tags = _tag_index.get(rel, [])
fid = _add_node(entry.name, "file", rel, entry.stat().st_size, tags=file_tags)
_add_edge(parent_id, fid, "parent")
except PermissionError:
pass
if target.is_dir():
_walk_dir(target, focus_id, 0)
elif target.is_file():
_walk_dir(target.parent, focus_id, 0)
_add_wikilink_edges(nodes, edges, vault_name)
edge_counts: dict[str, dict[str, int]] = {}
for node in nodes:
edge_counts[node["id"]] = {"incoming": 0, "outgoing": 0}
for edge in edges:
if edge["relation"] in ("wikilink", "backlink"):
src = edge["source"]
tgt = edge["target"]
if src in edge_counts:
edge_counts[src]["outgoing"] += 1
if tgt in edge_counts:
edge_counts[tgt]["incoming"] += 1
for node in nodes:
counts = edge_counts.get(node["id"], {"incoming": 0, "outgoing": 0})
node["incoming_count"] = counts["incoming"]
node["outgoing_count"] = counts["outgoing"]
return {"vault": vault_name, "path": path, "scope": scope, "nodes": nodes, "edges": edges}
def _add_wikilink_edges(nodes: list[dict[str, Any]], edges: list[dict[str, Any]], vault_name: str) -> None:
"""Add edges for wikilinks between markdown files in the current graph scope."""
file_nodes = [n for n in nodes if n["type"] == "file" and n["path"].endswith(".md")]
if len(file_nodes) < 2:
return
path_to_id = {n["path"]: n["id"] for n in file_nodes}
existing = {(e["source"], e["target"]) for e in edges}
for node in file_nodes:
vault_data = index.get(vault_name)
if not vault_data:
continue
file_entry = None
for f in vault_data.get("files", []):
if f["path"] == node["path"]:
file_entry = f
break
if not file_entry:
continue
content = file_entry.get("content", "")
if not content:
continue
for match in _WIKILINK_PATTERN.finditer(content):
target = match.group(1).strip()
target_lower = target.lower()
if not target_lower.endswith(".md"):
target_lower += ".md"
for target_path, target_id in path_to_id.items():
if target_id == node["id"]:
continue
target_name = target_path.rsplit("/", 1)[-1].lower()
if target_name == target_lower or target_path.lower() == target_lower:
edge_key = tuple(sorted([node["id"], target_id]))
if edge_key not in existing and (edge_key[1], edge_key[0]) not in existing:
edges.append({"source": node["id"], "target": target_id, "relation": "wikilink"})
existing.add((node["id"], target_id))
break
+811
View File
@@ -0,0 +1,811 @@
"""File and directory mutation services shared by REST routes and the AI tool layer.
Single source of truth for the write-side operations (create, edit, append,
rename, move, delete, restore, find/replace). Every function performs the
anti path-traversal check, the read-only guard and an automatic backup before
any destructive change, then returns a JSON-friendly result dict.
Index refresh, SSE broadcasts, webhooks and audit logging stay in the route /
agent layers: these services are synchronous and side-effect free apart from
the filesystem mutation (plus the backup).
"""
from __future__ import annotations
import logging
import os
import shutil
from collections.abc import Callable
from pathlib import Path
from typing import Any
from backend.services.backups import create_backup, get_backup_dir
from backend.services.errors import ServiceError
from backend.services.paths import resolve_safe_path
from backend.services.vaults import get_vault_root
logger = logging.getLogger("obsigate.services.mutations")
# Skeleton injected into empty ``.excalidraw`` files (mirrors the route logic).
_EXCALIDRAW_SKELETON = (
'{"type":"excalidraw","version":2,"elements":[],'
'"appState":{"viewBackgroundColor":"#ffffff"},"files":{}}'
)
def _rel(root: Path, path: Path) -> str:
"""Return *path* relative to *root* as a POSIX-style string."""
return str(path.relative_to(root)).replace("\\", "/")
def _ensure_writable(root: Path) -> None:
"""Raise ``read_only`` (403) when the vault root is not writable."""
if not os.access(root, os.W_OK):
raise ServiceError("Vault is read-only", code="read_only", status=403)
def _validate_extension(file_path: Path, *, allow_images: bool = False) -> None:
"""Reject unsupported file extensions (400)."""
from backend.indexer import SUPPORTED_EXTENSIONS
ext = file_path.suffix.lower()
allowed = SUPPORTED_EXTENSIONS
if allow_images:
from backend.attachment_indexer import IMAGE_EXTENSIONS
allowed = allowed | IMAGE_EXTENSIONS
if ext not in allowed and file_path.name.lower() not in ("dockerfile", "makefile"):
raise ServiceError(
f"Unsupported file extension: {ext}",
code="unsupported_extension",
status=400,
details={"extension": ext},
)
def _validate_new_name(name: str) -> str:
"""Validate a rename target (a plain name, not a path)."""
candidate = (name or "").strip()
if not candidate or candidate in (".", "..") or "/" in candidate or "\\" in candidate:
raise ServiceError(
f"Invalid name: {name!r}",
code="invalid_arguments",
status=400,
details={"new_name": name},
)
return candidate
# ── D1. Creation ───────────────────────────────────────────────────────────
def create_file(
vault_name: str,
path: str,
content: str = "",
*,
overwrite: bool = False,
) -> dict[str, Any]:
"""Create a text file in a vault.
Args:
vault_name: Name of the vault.
path: Vault-relative path of the new file.
content: Initial content (an Excalidraw skeleton is injected for empty
``.excalidraw`` files).
overwrite: When True, replace an existing file (with a backup) instead
of raising ``already_exists``.
Returns:
``{"success", "vault", "path", "size"}``.
Raises:
ServiceError: ``not_found`` (404) unknown vault, ``read_only`` (403),
``unsupported_extension`` (400) or ``already_exists`` (409).
"""
root = get_vault_root(vault_name)
_ensure_writable(root)
file_path = resolve_safe_path(root, path)
_validate_extension(file_path)
if file_path.exists():
if not overwrite:
raise ServiceError(
f"File already exists: {path}",
code="already_exists",
status=409,
details={"vault": vault_name, "path": path},
)
create_backup(file_path, vault_name, _rel(root, file_path))
try:
file_path.parent.mkdir(parents=True, exist_ok=True)
if file_path.suffix.lower() == ".excalidraw" and not content.strip():
content = _EXCALIDRAW_SKELETON
file_path.write_text(content, encoding="utf-8")
except PermissionError as e:
raise ServiceError("Permission denied: cannot create file", code="permission_denied", status=403) from e
rel_path = _rel(root, file_path)
logger.info(f"File created: {vault_name}/{rel_path}")
return {"success": True, "vault": vault_name, "path": rel_path, "size": len(content)}
def create_directory(vault_name: str, path: str) -> dict[str, Any]:
"""Create a directory (and its parents) in a vault.
Raises:
ServiceError: ``not_found`` (404), ``read_only`` (403) or
``already_exists`` (409).
"""
root = get_vault_root(vault_name)
_ensure_writable(root)
dir_path = resolve_safe_path(root, path)
if dir_path.exists():
raise ServiceError(
f"Directory already exists: {path}",
code="already_exists",
status=409,
details={"vault": vault_name, "path": path},
)
try:
dir_path.mkdir(parents=True, exist_ok=False)
except PermissionError as e:
raise ServiceError("Permission denied: cannot create directory", code="permission_denied", status=403) from e
rel_path = _rel(root, dir_path)
logger.info(f"Directory created: {vault_name}/{rel_path}")
return {"success": True, "vault": vault_name, "path": rel_path}
# ── D2. Edition / rename / move ────────────────────────────────────────────
def edit_file(
vault_name: str,
path: str,
content: str,
*,
backup: bool = True,
) -> dict[str, Any]:
"""Overwrite an existing file's content (with a backup by default).
Raises:
ServiceError: ``not_found`` (404) or ``read_only`` (403).
"""
root = get_vault_root(vault_name)
_ensure_writable(root)
file_path = resolve_safe_path(root, path)
if not file_path.exists() or not file_path.is_file():
raise ServiceError(
f"File not found: {path}",
code="not_found",
status=404,
details={"vault": vault_name, "path": path},
)
rel_path = _rel(root, file_path)
if backup:
create_backup(file_path, vault_name, rel_path)
try:
file_path.write_text(content, encoding="utf-8")
except PermissionError as e:
raise ServiceError("Permission denied: cannot save file", code="permission_denied", status=403) from e
logger.info(f"File saved: {vault_name}/{rel_path}")
return {"success": True, "vault": vault_name, "path": rel_path, "size": len(content)}
def append_to_file(
vault_name: str,
path: str,
content: str,
*,
backup: bool = True,
) -> dict[str, Any]:
"""Append text to an existing file (a newline is inserted if needed).
Raises:
ServiceError: ``not_found`` (404) or ``read_only`` (403).
"""
root = get_vault_root(vault_name)
_ensure_writable(root)
file_path = resolve_safe_path(root, path)
if not file_path.exists() or not file_path.is_file():
raise ServiceError(
f"File not found: {path}",
code="not_found",
status=404,
details={"vault": vault_name, "path": path},
)
rel_path = _rel(root, file_path)
if backup:
create_backup(file_path, vault_name, rel_path)
try:
existing = file_path.read_text(encoding="utf-8", errors="replace")
separator = "" if (not existing or existing.endswith("\n")) else "\n"
new_content = existing + separator + content
file_path.write_text(new_content, encoding="utf-8")
except PermissionError as e:
raise ServiceError("Permission denied: cannot append to file", code="permission_denied", status=403) from e
logger.info(f"File appended: {vault_name}/{rel_path} (+{len(content)} chars)")
return {
"success": True,
"vault": vault_name,
"path": rel_path,
"appended": len(content),
"size": len(new_content),
}
def rename_file(vault_name: str, path: str, new_name: str) -> dict[str, Any]:
"""Rename a file in place (same parent directory).
Raises:
ServiceError: ``not_found`` (404), ``read_only`` (403),
``unsupported_extension`` (400) or ``already_exists`` (409).
"""
root = get_vault_root(vault_name)
_ensure_writable(root)
old_path = resolve_safe_path(root, path)
if not old_path.exists() or not old_path.is_file():
raise ServiceError(
f"File not found: {path}",
code="not_found",
status=404,
details={"vault": vault_name, "path": path},
)
new_path = old_path.parent / _validate_new_name(new_name)
new_path = resolve_safe_path(root, _rel(root, new_path))
_validate_extension(new_path)
if new_path.exists():
raise ServiceError(
f"Destination already exists: {new_name}",
code="already_exists",
status=409,
details={"vault": vault_name, "new_name": new_name},
)
old_rel = _rel(root, old_path)
try:
old_path.rename(new_path)
except PermissionError as e:
raise ServiceError("Permission denied: cannot rename file", code="permission_denied", status=403) from e
new_rel = _rel(root, new_path)
logger.info(f"File renamed: {vault_name}/{old_rel} -> {new_rel}")
return {"success": True, "vault": vault_name, "old_path": old_rel, "new_path": new_rel}
def rename_directory(vault_name: str, path: str, new_name: str) -> dict[str, Any]:
"""Rename a directory in place (same parent directory).
Raises:
ServiceError: ``not_found`` (404), ``read_only`` (403) or
``already_exists`` (409).
"""
root = get_vault_root(vault_name)
_ensure_writable(root)
old_path = resolve_safe_path(root, path)
if not old_path.exists() or not old_path.is_dir():
raise ServiceError(
f"Directory not found: {path}",
code="not_found",
status=404,
details={"vault": vault_name, "path": path},
)
new_path = old_path.parent / _validate_new_name(new_name)
new_path = resolve_safe_path(root, _rel(root, new_path))
if new_path.exists():
raise ServiceError(
f"Destination already exists: {new_name}",
code="already_exists",
status=409,
details={"vault": vault_name, "new_name": new_name},
)
old_rel = _rel(root, old_path)
try:
old_path.rename(new_path)
except PermissionError as e:
raise ServiceError("Permission denied: cannot rename directory", code="permission_denied", status=403) from e
new_rel = _rel(root, new_path)
logger.info(f"Directory renamed: {vault_name}/{old_rel} -> {new_rel}")
return {"success": True, "vault": vault_name, "old_path": old_rel, "new_path": new_rel}
def move_path(vault_name: str, source_path: str, destination_dir: str = "") -> dict[str, Any]:
"""Move a file or directory to another directory within the same vault.
The item keeps its name; only its parent directory changes.
Raises:
ServiceError: ``not_found`` (404), ``read_only`` (403),
``unsupported_extension`` (400) or ``already_exists`` (409).
"""
root = get_vault_root(vault_name)
_ensure_writable(root)
source = resolve_safe_path(root, source_path)
if not source.exists():
raise ServiceError(
f"Source not found: {source_path}",
code="not_found",
status=404,
details={"vault": vault_name, "path": source_path},
)
is_directory = source.is_dir()
item_name = source.name
dest_clean = (destination_dir or "").strip("/")
if dest_clean:
dest_parent = resolve_safe_path(root, dest_clean)
if not dest_parent.exists() or not dest_parent.is_dir():
raise ServiceError(
f"Destination directory not found: {destination_dir}",
code="not_found",
status=404,
details={"vault": vault_name, "path": destination_dir},
)
else:
dest_parent = root
destination = resolve_safe_path(root, _rel(root, dest_parent / item_name))
if source.resolve() == destination.resolve():
rel = _rel(root, source)
return {
"success": True,
"vault": vault_name,
"old_path": rel,
"new_path": rel,
"item_type": "directory" if is_directory else "file",
}
if destination.exists():
raise ServiceError(
f"A file or directory already exists at the destination: {destination.name}",
code="already_exists",
status=409,
details={"vault": vault_name, "new_path": _rel(root, destination)},
)
if not is_directory:
_validate_extension(destination)
old_rel = _rel(root, source)
try:
source.rename(destination)
except PermissionError as e:
raise ServiceError("Permission denied: cannot move item", code="permission_denied", status=403) from e
new_rel = _rel(root, destination)
item_type = "directory" if is_directory else "file"
logger.info(f"Item moved: {vault_name}/{old_rel} -> {new_rel}")
return {
"success": True,
"vault": vault_name,
"old_path": old_rel,
"new_path": new_rel,
"item_type": item_type,
}
# ── D3. Find & replace ─────────────────────────────────────────────────────
def replace_in_files(
find: str,
replacement: str,
*,
vault: str = "all",
case_sensitive: bool = False,
whole_word: bool = False,
regex: bool = False,
include_paths: str | None = None,
exclude_paths: str | None = None,
replace_all: bool = False,
dry_run: bool = True,
is_vault_allowed: Callable[[str], bool] | None = None,
) -> dict[str, Any]:
"""Find and replace text across vault files (dry-run by default).
A backup is created before every file is rewritten. When
``is_vault_allowed`` is provided, files from vaults it rejects are skipped
(used by the tool layer to enforce per-vault permissions and the
destructive-tools toggle).
Returns:
``{"matches", "total_matches"}`` in dry-run mode (with
``"dry_run": True``) or ``{"replaced", "total_replacements"}`` when
applied.
"""
import re as re_mod
from backend.services.regex_safety import MAX_REGEX_MATCHES, validate_regex
from backend.services.search import advanced_search_vaults
if not find:
raise ServiceError("Query is required", code="invalid_arguments", status=400)
# BUG-025: validate the pattern before it is compiled / applied in bulk.
if regex:
try:
validate_regex(find)
except ValueError as e:
raise ServiceError(str(e), code="invalid_arguments", status=400) from e
try:
search_results = advanced_search_vaults(
find,
vault=vault,
case_sensitive=case_sensitive,
whole_word=whole_word,
regex=regex,
include_paths=include_paths,
exclude_paths=exclude_paths,
limit=500,
sort="relevance",
)
except ValueError as e:
raise ServiceError(str(e), code="invalid_arguments", status=400) from e
if not search_results["results"]:
return {"matches": [], "total_matches": 0, "dry_run": dry_run}
flags = 0 if case_sensitive else re_mod.IGNORECASE
if regex:
pattern = re_mod.compile(find, flags)
elif whole_word:
pattern = re_mod.compile(rf"\b{re_mod.escape(find)}\b", flags)
else:
pattern = re_mod.compile(re_mod.escape(find), flags)
matches: list[dict[str, Any]] = []
total = 0
for result in search_results["results"]:
result_vault = result["vault"]
if is_vault_allowed is not None and not is_vault_allowed(result_vault):
continue
try:
root = get_vault_root(result_vault)
file_path = resolve_safe_path(root, result["path"])
except ServiceError:
continue
if not file_path.exists() or not file_path.is_file():
continue
try:
original = file_path.read_text(encoding="utf-8", errors="replace")
except OSError:
continue
occurrences = list(pattern.finditer(original))[:MAX_REGEX_MATCHES]
if not occurrences:
continue
if dry_run:
previews = []
for m in occurrences[:3]:
start = max(0, m.start() - 40)
end = min(len(original), m.end() + 40)
previews.append(f"...{original[start:end]}...")
matches.append({
"vault": result_vault,
"path": result["path"],
"title": result.get("title", result["path"]),
"match_count": len(occurrences),
"preview": previews,
})
total += len(occurrences)
continue
new_content, count = pattern.subn(replacement, original)
if count == 0:
continue
create_backup(file_path, result_vault, result["path"])
try:
file_path.write_text(new_content, encoding="utf-8")
except PermissionError as e:
raise ServiceError(
f"Permission denied writing {result['path']}",
code="permission_denied",
status=403,
) from e
matches.append({
"vault": result_vault,
"path": result["path"],
"title": result.get("title", result["path"]),
"replacements": count,
"size": len(new_content),
})
total += count
if dry_run:
return {"matches": matches, "total_matches": total, "dry_run": True}
return {"replaced": matches, "total_replacements": total, "dry_run": False}
# ── D4. Deletion / restore ─────────────────────────────────────────────────
def delete_file(vault_name: str, path: str, *, backup: bool = True) -> dict[str, Any]:
"""Delete a file (with a backup by default).
Raises:
ServiceError: ``not_found`` (404) or ``read_only`` (403).
"""
root = get_vault_root(vault_name)
_ensure_writable(root)
file_path = resolve_safe_path(root, path)
if not file_path.exists() or not file_path.is_file():
raise ServiceError(
f"File not found: {path}",
code="not_found",
status=404,
details={"vault": vault_name, "path": path},
)
rel_path = _rel(root, file_path)
if backup:
create_backup(file_path, vault_name, rel_path)
try:
file_path.unlink()
except PermissionError as e:
raise ServiceError("Permission denied: cannot delete file", code="permission_denied", status=403) from e
logger.info(f"File deleted: {vault_name}/{rel_path}")
return {"success": True, "vault": vault_name, "path": rel_path}
def delete_directory(vault_name: str, path: str, *, recursive: bool = True) -> dict[str, Any]:
"""Delete a directory (recursively by default).
Raises:
ServiceError: ``not_found`` (404), ``read_only`` (403),
``not_empty`` (409) when non-recursive and not empty, or
``invalid_arguments`` (400) when targeting the vault root.
"""
root = get_vault_root(vault_name)
_ensure_writable(root)
dir_path = resolve_safe_path(root, path)
if not dir_path.exists() or not dir_path.is_dir():
raise ServiceError(
f"Directory not found: {path}",
code="not_found",
status=404,
details={"vault": vault_name, "path": path},
)
if dir_path.resolve() == root.resolve():
raise ServiceError(
"Refusing to delete the vault root",
code="invalid_arguments",
status=400,
details={"vault": vault_name},
)
file_count = sum(1 for p in dir_path.rglob("*") if p.is_file())
try:
if recursive:
shutil.rmtree(dir_path)
else:
if any(dir_path.iterdir()):
raise ServiceError(
f"Directory not empty: {path}",
code="not_empty",
status=409,
details={"vault": vault_name, "path": path},
)
dir_path.rmdir()
except PermissionError as e:
raise ServiceError("Permission denied: cannot delete directory", code="permission_denied", status=403) from e
rel_path = _rel(root, dir_path)
logger.info(f"Directory deleted: {vault_name}/{rel_path} ({file_count} files)")
return {"success": True, "vault": vault_name, "path": rel_path, "deleted_count": file_count}
def restore_backup(
vault_name: str,
path: str,
version: int,
*,
backup: bool = True,
) -> dict[str, Any]:
"""Restore a file from a backup version.
The current file is backed up first (when ``backup`` is True) so the
operation is reversible.
Raises:
ServiceError: ``not_found`` (404) when the file or backup is missing,
or ``read_only`` (403).
"""
root = get_vault_root(vault_name)
_ensure_writable(root)
file_path = resolve_safe_path(root, path)
backup_dir = get_backup_dir(vault_name, path)
backup_path = backup_dir / f"{Path(path).name}.{version}.bak"
if not backup_path.exists():
raise ServiceError(
f"Backup version {version} not found for {path}",
code="not_found",
status=404,
details={"vault": vault_name, "path": path, "version": version},
)
# Read the target version *before* backing up the current file: both use a
# second-resolution timestamp, so a same-second backup could otherwise
# overwrite the version we are about to restore.
try:
content = backup_path.read_text(encoding="utf-8")
except OSError as e:
raise ServiceError(
f"Failed to read backup {version}: {e}",
code="read_error",
status=500,
details={"vault": vault_name, "path": path, "version": version},
) from e
current_backed_up: int | None = None
if backup and file_path.exists() and file_path.is_file():
import time as _time
create_backup(file_path, vault_name, path)
current_backed_up = int(_time.time())
try:
file_path.write_text(content, encoding="utf-8")
except PermissionError as e:
raise ServiceError("Permission denied: cannot restore file", code="permission_denied", status=403) from e
logger.info(f"File restored from backup: {vault_name}/{path} <- version {version}")
return {
"success": True,
"vault": vault_name,
"path": path,
"restored_from": version,
"current_backed_up": current_backed_up,
}
# ── D3. Batch upload & raw file save ───────────────────────────────────────
def save_raw_file(
vault_name: str,
path: str,
content: bytes,
*,
overwrite: bool = True,
) -> dict[str, Any]:
"""Save a binary or text file to a vault (e.g. from upload / drag-and-drop).
Creates parent directories automatically and safely validates the path.
Supports supported text extensions, images and Excalidraw files.
Args:
vault_name: Name of the vault.
path: Vault-relative path.
content: Raw bytes to write.
overwrite: When True, replace existing files (with backup).
Returns:
Dict with ``success``, ``vault``, ``path``, and ``size``.
"""
root = get_vault_root(vault_name)
_ensure_writable(root)
file_path = resolve_safe_path(root, path)
_validate_extension(file_path, allow_images=True)
rel_path = _rel(root, file_path)
if file_path.exists():
if not overwrite:
raise ServiceError(
f"File already exists: {rel_path}",
code="already_exists",
status=409,
details={"vault": vault_name, "path": rel_path},
)
create_backup(file_path, vault_name, rel_path)
try:
file_path.parent.mkdir(parents=True, exist_ok=True)
file_path.write_bytes(content)
except PermissionError as e:
raise ServiceError("Permission denied: cannot save file", code="permission_denied", status=403) from e
logger.info(f"Raw file saved: {vault_name}/{rel_path} ({len(content)} bytes)")
return {"success": True, "vault": vault_name, "path": rel_path, "size": len(content)}
def batch_upload_files(
vault_name: str,
target_dir: str,
files: list[dict[str, Any]],
*,
overwrite: bool = True,
) -> dict[str, Any]:
"""Process a batch of uploaded files and directories into a vault.
Args:
vault_name: Name of the target vault.
target_dir: Base directory inside the vault (empty string for root).
files: List of dicts, each with:
- ``path``: relative path within the batch (e.g. ``"sub/doc.md"`` or ``"note.md"``).
- ``content``: bytes content (or base64 decoded).
- ``is_dir``: optional boolean for empty directories.
overwrite: Whether to overwrite existing files.
Returns:
Dict with ``uploaded`` (list of paths), ``created_dirs`` (list of paths),
and ``errors`` (list of error dicts).
"""
root = get_vault_root(vault_name)
_ensure_writable(root)
clean_target = (target_dir or "").strip().strip("/\\")
uploaded: list[str] = []
created_dirs: list[str] = []
errors: list[dict[str, Any]] = []
for item in files:
rel_subpath = (item.get("path") or "").strip().replace("\\", "/").lstrip("/")
if not rel_subpath:
continue
full_rel_path = f"{clean_target}/{rel_subpath}" if clean_target else rel_subpath
is_dir = item.get("is_dir", False)
if is_dir:
try:
dir_path = resolve_safe_path(root, full_rel_path)
dir_path.mkdir(parents=True, exist_ok=True)
created_dirs.append(_rel(root, dir_path))
except Exception as e:
errors.append({"path": full_rel_path, "error": str(e)})
continue
raw_bytes = item.get("content", b"")
if isinstance(raw_bytes, str):
raw_bytes = raw_bytes.encode("utf-8")
try:
res = save_raw_file(vault_name, full_rel_path, raw_bytes, overwrite=overwrite)
uploaded.append(res["path"])
except Exception as e:
errors.append({"path": full_rel_path, "error": str(e)})
return {
"success": len(errors) == 0,
"vault": vault_name,
"target_dir": clean_target,
"uploaded": uploaded,
"created_dirs": created_dirs,
"errors": errors,
"total_files": len(uploaded),
}
+34
View File
@@ -0,0 +1,34 @@
"""Network helpers shared by the auth middleware and rate limiter."""
from __future__ import annotations
import os
from fastapi import Request
__all__ = ["get_client_ip", "is_trusted_proxy"]
def is_trusted_proxy() -> bool:
"""Whether ``X-Forwarded-For`` should be trusted (reverse proxy in front)."""
return os.environ.get("OBSIGATE_TRUST_PROXY", "false").lower() == "true"
def get_client_ip(request: Request) -> str:
"""Return the best-known client IP for *request*.
When ``OBSIGATE_TRUST_PROXY=true`` the left-most ``X-Forwarded-For`` entry
is used (the original client behind the proxy). Otherwise the socket peer
address is returned. BUG-030: this value feeds the audit log so attacks
remain traceable.
"""
if is_trusted_proxy():
forwarded = request.headers.get("x-forwarded-for")
if forwarded:
first = forwarded.split(",")[0].strip()
if first:
return first
real_ip = request.headers.get("x-real-ip")
if real_ip:
return real_ip.strip()
return request.client.host if request.client else "unknown"
+62
View File
@@ -0,0 +1,62 @@
"""Vault path resolution shared by the REST routes and the AI tool layer.
This is the single implementation of the anti path-traversal check. Routes map
:class:`ServiceError` to ``HTTPException`` and tools map it to
:class:`backend.tools.context.ToolError`.
"""
from __future__ import annotations
import logging
from pathlib import Path
from backend.services.errors import ServiceError
logger = logging.getLogger("obsigate.services.paths")
def _is_within(resolved: Path, root: Path) -> bool:
"""Return True when *resolved* is *root* or lives below it.
The comparison is segment-aware so that a sibling directory whose name
merely shares a prefix (``vault`` vs ``vault-evil``) is rejected. A
case-insensitive fallback preserves the Windows / Docker behaviour where
the resolved casing can differ from the configured root.
"""
try:
resolved.relative_to(root)
return True
except ValueError:
pass
try:
resolved_parts = tuple(part.lower() for part in resolved.parts)
root_parts = tuple(part.lower() for part in root.parts)
except Exception:
return False
return resolved_parts[: len(root_parts)] == root_parts
def resolve_safe_path(vault_root: Path, relative_path: str | None) -> Path:
"""Resolve a vault-relative path, rejecting traversal outside the vault.
Raises:
ServiceError: ``path_error`` (500) when the path cannot be resolved,
``path_outside_vault`` (403) when it escapes the vault root.
"""
full_path = vault_root / (relative_path or "")
try:
resolved = full_path.resolve(strict=False)
root = vault_root.resolve(strict=False)
except Exception as e:
logger.error(f"Path resolution error - vault_root: {vault_root}, relative_path: {relative_path}, error: {e}")
raise ServiceError(f"Path resolution error: {e!s}", code="path_error", status=500) from e
if not _is_within(resolved, root):
logger.warning(f"Path outside vault - vault: {root}, requested: {relative_path}, resolved: {resolved}")
raise ServiceError(
"Access denied: path outside vault",
code="path_outside_vault",
status=403,
details={"path": relative_path},
)
return resolved
+138
View File
@@ -0,0 +1,138 @@
"""Recent-files services shared by the REST route and the AI tool layer.
Provides ``list_recent`` (last opened files, falling back to last modified)
and ``humanize_mtime``. The route ``/api/recent`` and the tool ``list_recent``
both delegate here. Permission filtering is applied against the caller's
vault list.
"""
from __future__ import annotations
import time
from datetime import datetime
from typing import Any
from backend.history import get_recent_opened, is_bookmarked
from backend.indexer import find_file_in_index, index
def humanize_mtime(mtime: float) -> str:
"""Return a short, human-friendly French rendering of a timestamp."""
delta = time.time() - mtime
if delta < 60:
return "à l'instant"
if delta < 3600:
return f"il y a {int(delta / 60)} min"
if delta < 86400:
return f"il y a {int(delta / 3600)} h"
if delta < 604800:
return f"il y a {int(delta / 86400)} j"
return datetime.fromtimestamp(mtime).strftime("%d %b %Y")
def _can_access(vault: str, user_vaults: list[str]) -> bool:
return "*" in user_vaults or vault in user_vaults
def list_recent(
username: str | None,
user_vaults: list[str] | None = None,
*,
vault: str | None = None,
limit: int = 20,
mode: str = "opened",
) -> dict[str, Any]:
"""Return the caller's recent files.
Args:
username: Caller username (needed for the "opened" history mode).
user_vaults: Vaults the caller may access (``["*"]`` = all).
vault: Optional single-vault filter.
limit: Maximum number of files to return.
mode: ``"opened"`` to use the open history, anything else for the
last-modified fallback.
Returns:
Dict with ``files``, ``total``, ``limit`` and ``mode``.
"""
user_vaults = user_vaults or []
if mode == "opened" and username:
history = get_recent_opened(username, vault_filter=vault, limit=limit)
files_resp: list[dict[str, Any]] = []
for item in history:
v_name = item["vault"]
if not _can_access(v_name, user_vaults):
continue
f_idx = find_file_in_index(item["path"], v_name)
if f_idx:
files_resp.append({
"path": f_idx["path"],
"title": f_idx.get("title") or item["path"].split("/")[-1],
"vault": v_name,
"mtime": item["opened_at"],
"mtime_human": humanize_mtime(item["opened_at"]),
"size_bytes": f_idx.get("size", 0),
"tags": [f"#{t}" for t in f_idx.get("tags", [])][:5],
"preview": f_idx.get("content_preview", "")[:120],
"bookmarked": is_bookmarked(username, v_name, f_idx["path"]),
})
else:
files_resp.append({
"path": item["path"],
"title": item.get("title") or item["path"].split("/")[-1],
"vault": v_name,
"mtime": item["opened_at"],
"mtime_human": humanize_mtime(item["opened_at"]),
"tags": [],
"preview": "",
"bookmarked": is_bookmarked(username, v_name, item["path"]),
})
return {
"files": files_resp,
"total": len(files_resp),
"limit": limit,
"mode": "opened",
}
all_files: list[tuple[str, dict[str, Any]]] = []
for v_name, v_data in index.items():
if vault and v_name != vault:
continue
if not _can_access(v_name, user_vaults):
continue
for f in v_data.get("files", []):
all_files.append((v_name, f))
all_files.sort(key=lambda x: x[1].get("modified", ""), reverse=True)
recent = all_files[:limit]
files_resp = []
for v_name, f in recent:
iso_modified = f.get("modified", "")
try:
mtime_dt = datetime.fromisoformat(iso_modified.replace("Z", "+00:00"))
mtime_val = mtime_dt.timestamp()
except Exception:
mtime_val = time.time()
files_resp.append({
"path": f["path"],
"title": f["title"],
"vault": v_name,
"mtime": mtime_val,
"mtime_human": humanize_mtime(mtime_val),
"mtime_iso": iso_modified,
"size_bytes": f.get("size", 0),
"tags": [f"#{t}" for t in f.get("tags", [])][:5],
"preview": f.get("content_preview", "")[:120],
"bookmarked": is_bookmarked(username, v_name, f["path"]) if username else False,
})
return {
"files": files_resp,
"total": len(all_files),
"limit": limit,
"mode": "modified",
}
+67
View File
@@ -0,0 +1,67 @@
"""Regex safety helpers (BUG-025).
User-supplied regular expressions are applied to large amounts of indexed
content. Python's :mod:`re` has no timeout, so a malicious pattern such as
``(a+)+$`` can pin a CPU for a long time (ReDoS). Without adding a native
dependency we mitigate by:
* bounding the pattern length,
* rejecting nested-quantifier constructs (the classic catastrophic form),
* capping the amount of text a single regex pass may scan,
* capping the number of matches collected.
"""
from __future__ import annotations
import re
__all__ = [
"MAX_PATTERN_LENGTH",
"MAX_REGEX_CONTENT",
"MAX_REGEX_MATCHES",
"truncate_for_regex",
"validate_regex",
]
MAX_PATTERN_LENGTH = 500
MAX_REGEX_CONTENT = 200_000
MAX_REGEX_MATCHES = 1000
# A quantified group whose body already contains a quantifier, immediately
# followed by another quantifier: ``(a+)+``, ``(.*)*``, ``(a+){2,}``, ...
_NESTED_QUANTIFIER_RE = re.compile(r"\([^()]*[+*][^()]*\)\s*(?:[+*?]|\{)")
# Backreferences combined with quantifiers are a common ReDoS vector too.
_BACKREF_QUANTIFIER_RE = re.compile(r"\\[1-9][0-9]*\s*(?:[+*]|\{)")
def validate_regex(pattern: str) -> str:
"""Validate a user-supplied regex against the safety policy.
Args:
pattern: Raw regex pattern.
Returns:
The pattern unchanged when acceptable.
Raises:
ValueError: When the pattern is empty, too long, or uses a construct
known to cause catastrophic backtracking.
"""
if not pattern:
raise ValueError("Expression régulière vide")
if len(pattern) > MAX_PATTERN_LENGTH:
raise ValueError(f"Expression régulière trop longue (max {MAX_PATTERN_LENGTH})")
if _NESTED_QUANTIFIER_RE.search(pattern) or _BACKREF_QUANTIFIER_RE.search(pattern):
raise ValueError("Expression régulière refusée (quantificateurs imbriqués)")
try:
re.compile(pattern)
except re.error as e:
raise ValueError(f"Expression régulière invalide : {e}") from e
return pattern
def truncate_for_regex(content: str, limit: int = MAX_REGEX_CONTENT) -> str:
"""Return at most *limit* characters to bound a single regex pass."""
if content and len(content) > limit:
return content[:limit]
return content
+286
View File
@@ -0,0 +1,286 @@
"""Whitelist HTML sanitizer used for untrusted markdown / AI output.
The markdown renderer runs with ``escape=False`` so raw HTML authored inside a
vault (or returned by a model) reaches the browser. This module scrubs the
rendered HTML against a strict whitelist of tags and attributes, drops
dangerous URL schemes and strips every event handler / ``style`` attribute.
Implemented with the standard library only (no third-party dependency) so the
runtime footprint stays unchanged. It is *not* a full HTML5 parser: it is a
conservative, allow-list based filter intended for already well-formed output
produced by mistune and the image/wikilink pre-processors.
"""
from __future__ import annotations
import html as _html
from html.parser import HTMLParser
__all__ = ["is_safe_url", "sanitize_html"]
# Tags whose *content* is discarded entirely (never rendered as text).
_DROP_CONTENT_TAGS = frozenset({
"script", "style", "iframe", "object", "embed", "template", "noscript",
"svg", "math", "applet", "form", "button", "select", "textarea", "option",
"frame", "frameset", "base", "link", "meta", "title", "head",
})
# Tags kept in the output (text content preserved for unknown tags).
_ALLOWED_TAGS = frozenset({
"a", "abbr", "b", "blockquote", "br", "caption", "code", "col", "colgroup",
"dd", "del", "details", "div", "dl", "dt", "em", "figcaption", "figure",
"h1", "h2", "h3", "h4", "h5", "h6", "hr", "i", "img", "input", "kbd", "li",
"mark", "ol", "p", "pre", "q", "s", "section", "small", "span", "strong",
"sub", "summary", "sup", "table", "tbody", "td", "tfoot", "th", "thead",
"time", "tr", "u", "ul", "video", "audio", "source", "track",
})
# Attributes allowed on any element.
_GLOBAL_ATTRS = frozenset({"class", "id", "title", "dir", "lang", "role"})
# Per-tag attribute whitelist (in addition to globals and ``data-*``).
_TAG_ATTRS: dict[str, frozenset[str]] = {
"a": frozenset({"href", "target", "rel", "name", "download"}),
"img": frozenset({"src", "alt", "width", "height", "loading"}),
"input": frozenset({"type", "checked", "disabled", "value"}),
"ol": frozenset({"start", "type", "reversed"}),
"ul": frozenset({"type"}),
"li": frozenset({"value"}),
"td": frozenset({"colspan", "rowspan", "align", "valign"}),
"th": frozenset({"colspan", "rowspan", "align", "valign", "scope"}),
"col": frozenset({"span", "width"}),
"colgroup": frozenset({"span"}),
"video": frozenset({"src", "controls", "width", "height", "loop", "muted",
"poster", "preload", "playsinline"}),
"audio": frozenset({"src", "controls", "loop", "muted", "preload"}),
"source": frozenset({"src", "type", "srcset", "media"}),
"track": frozenset({"src", "kind", "srclang", "label", "default"}),
"details": frozenset({"open"}),
"time": frozenset({"datetime"}),
"blockquote": frozenset({"cite"}),
"q": frozenset({"cite"}),
}
# URL-bearing attributes per tag, and whether ``data:`` URIs are acceptable.
_URL_ATTRS: dict[str, frozenset[str]] = {
"a": frozenset({"href"}),
"img": frozenset({"src"}),
"video": frozenset({"src", "poster"}),
"audio": frozenset({"src"}),
"source": frozenset({"src", "srcset"}),
"track": frozenset({"src"}),
"blockquote": frozenset({"cite"}),
"q": frozenset({"cite"}),
}
_SAFE_SCHEMES = frozenset({"http", "https", "mailto", "tel", "ftp"})
# Characters that browsers ignore inside a scheme (tab/newline/CR) and that
# could otherwise smuggle ``java\tscript:`` past a naive check.
_URL_STRIP_CHARS = "\t\n\r\x00"
def is_safe_url(value: str, *, allow_data: bool = False, tag: str = "") -> bool:
"""Return True when *value* is a URL with an allowed scheme.
Relative URLs (``/foo``, ``./foo``, ``#anchor``) are allowed. Dangerous
schemes such as ``javascript:`` and ``vbscript:`` are always rejected.
``data:`` URIs are only allowed for image/video/audio sources.
"""
if value is None:
return False
# Decode entities and strip whitespace/control chars before inspecting.
candidate = _html.unescape(str(value)).strip()
for ch in _URL_STRIP_CHARS:
candidate = candidate.replace(ch, "")
if not candidate:
return False
# Detect a scheme: ``scheme:`` where scheme is [a-zA-Z][a-zA-Z0-9+.-]*
lowered = candidate.lower()
if lowered.startswith("data:"):
if not allow_data:
return False
# Only media data URIs are permitted.
if tag in ("img",):
return lowered.startswith("data:image/")
if tag in ("video", "audio", "source", "track"):
return (
lowered.startswith("data:image/")
or lowered.startswith("data:video/")
or lowered.startswith("data:audio/")
)
return False
if lowered.startswith("blob:"):
return tag in ("img", "video", "audio", "source")
# No scheme at all (relative / fragment / protocol-relative) → safe.
colon = candidate.find(":")
slash = candidate.find("/")
if colon == -1 or (slash != -1 and slash < colon):
return True
# ``//host`` protocol-relative has no scheme.
if candidate.startswith("//"):
return True
scheme = lowered[:colon]
if not scheme or not scheme[0].isalpha():
return True # not a real scheme, treat as relative
return scheme in _SAFE_SCHEMES
class _Sanitizer(HTMLParser):
"""Rebuild HTML while dropping anything not explicitly allowed."""
def __init__(self) -> None:
super().__init__(convert_charrefs=True)
self._out: list[str] = []
# Stack of tag names currently open (only allowed tags).
self._open: list[str] = []
# Stack tracking dropped-content depth: each entry is the tag name.
self._suppress: list[str] = []
# -- helpers ----------------------------------------------------------
def _filter_attrs(self, tag: str, attrs: list[tuple[str, str | None]]) -> str:
allowed_extra = _TAG_ATTRS.get(tag, frozenset())
url_attrs = _URL_ATTRS.get(tag, frozenset())
parts: list[str] = []
seen: set[str] = set()
for name, value in attrs:
if value is None:
value = ""
lname = name.lower()
if lname in seen:
continue
seen.add(lname)
# Event handlers and style are never allowed.
if lname.startswith("on") or lname in ("style", "srcdoc", "formaction", "xlink:href"):
continue
if lname.startswith("data-") or lname.startswith("aria-"):
pass
elif lname not in _GLOBAL_ATTRS and lname not in allowed_extra:
continue
if lname in url_attrs:
allow_data = tag in ("img", "video", "audio", "source", "track")
# ``srcset`` may contain multiple comma-separated candidates.
if lname == "srcset":
if not _safe_srcset(value, tag):
continue
elif not is_safe_url(value, allow_data=allow_data, tag=tag):
continue
parts.append(f' {lname}="{_html.escape(value, quote=True)}"')
return "".join(parts)
def _emit_start(self, tag: str, attrs, self_closing: bool) -> None:
attrs_html = self._filter_attrs(tag, attrs)
if self_closing or tag in ("br", "hr", "img", "input", "col", "source", "track"):
self._out.append(f"<{tag}{attrs_html} />")
else:
self._out.append(f"<{tag}{attrs_html}>")
self._open.append(tag)
# -- HTMLParser callbacks --------------------------------------------
def handle_starttag(self, tag: str, attrs) -> None:
tag = tag.lower()
if tag in _DROP_CONTENT_TAGS:
self._suppress.append(tag)
return
if self._suppress:
return
if tag not in _ALLOWED_TAGS:
return # drop the tag, keep its text content
self._emit_start(tag, attrs, self_closing=False)
def handle_startendtag(self, tag: str, attrs) -> None:
tag = tag.lower()
if tag in _DROP_CONTENT_TAGS or self._suppress:
return
if tag not in _ALLOWED_TAGS:
return
self._emit_start(tag, attrs, self_closing=True)
def handle_endtag(self, tag: str) -> None:
tag = tag.lower()
if tag in _DROP_CONTENT_TAGS:
# Close the innermost matching suppress marker.
for i in range(len(self._suppress) - 1, -1, -1):
if self._suppress[i] == tag:
del self._suppress[i:]
break
return
if self._suppress:
return
if tag not in _ALLOWED_TAGS:
return
# Only close if currently open (tolerate malformed nesting).
if tag in self._open:
while self._open:
top = self._open.pop()
self._out.append(f"</{top}>")
if top == tag:
break
def handle_data(self, data: str) -> None:
if self._suppress:
return
self._out.append(_html.escape(data, quote=False))
def handle_comment(self, data: str) -> None:
return # comments are dropped
def handle_decl(self, decl: str) -> None:
return
def handle_pi(self, data: str) -> None:
return
def handle_entityref(self, name: str) -> None:
if self._suppress:
return
self._out.append(f"&{name};")
def handle_charref(self, name: str) -> None:
if self._suppress:
return
self._out.append(f"&#{name};")
def get_html(self) -> str:
# Close any tags left open by malformed input.
while self._open:
self._out.append(f"</{self._open.pop()}>")
return "".join(self._out)
def _safe_srcset(value: str, tag: str) -> bool:
"""Validate every candidate in a ``srcset`` attribute."""
for candidate in value.split(","):
candidate = candidate.strip()
if not candidate:
continue
url = candidate.split()[0] if candidate.split() else candidate
if not is_safe_url(url, allow_data=(tag == "img"), tag=tag):
return False
return True
def sanitize_html(html: str) -> str:
"""Return *html* with only whitelisted tags/attributes/schemes preserved.
Args:
html: Untrusted HTML (typically mistune output with ``escape=False``).
Returns:
Sanitized HTML string.
"""
if not html:
return ""
parser = _Sanitizer()
try:
parser.feed(html)
parser.close()
except Exception:
# Never let sanitization crash a request; fail closed to plain text.
return _html.escape(html)
return parser.get_html()
+144
View File
@@ -0,0 +1,144 @@
"""Search services shared by REST routes and the AI tool layer."""
from __future__ import annotations
from typing import Any
def search_vaults(
q: str,
vault: str = "all",
tag: str | None = None,
limit: int = 50,
offset: int = 0,
) -> dict[str, Any]:
"""Full-text search with pagination, returned as the API response payload.
No permission filtering is applied here: callers that need it (the tool
layer) filter the ``results`` list themselves.
"""
from backend.search import search
all_results = search(q, vault_filter=vault, tag_filter=tag)
total = len(all_results)
page = all_results[offset: offset + limit]
return {
"query": q,
"vault_filter": vault,
"tag_filter": tag,
"count": len(page),
"total": total,
"offset": offset,
"limit": limit,
"results": page,
}
def list_tags(vault: str | None = None) -> dict[str, int]:
"""Return tag → count, optionally restricted to a single vault."""
from backend.search import get_all_tags
return get_all_tags(vault_filter=vault)
def advanced_search_vaults(
query: str = "",
vault: str = "all",
tag: str | None = None,
limit: int = 50,
offset: int = 0,
sort: str = "relevance",
case_sensitive: bool = False,
whole_word: bool = False,
regex: bool = False,
include_paths: str | None = None,
exclude_paths: str | None = None,
created: str | None = None,
modified: str | None = None,
size: str | None = None,
semantic: bool = False,
) -> dict[str, Any]:
"""Advanced full-text search (TF-IDF, facets, operators).
When ``semantic`` is True, the TF-IDF ranking is fused with the embedding
(semantic) ranking via RRF. No permission filtering is applied: callers
that need it (the tool layer) filter the ``results`` list themselves.
"""
from backend.search import advanced_search
return advanced_search(
query,
vault_filter=vault,
tag_filter=tag,
limit=limit,
offset=offset,
sort_by=sort,
case_sensitive=case_sensitive,
whole_word=whole_word,
regex=regex,
include_paths=include_paths,
exclude_paths=exclude_paths,
created=created,
modified=modified,
size=size,
semantic=semantic,
)
def search_paths(q: str, vault: str = "all") -> dict[str, Any]:
"""Search files and directories by path substring using the path index.
No permission filtering is applied: callers that need it (the tool layer)
filter the ``results`` list themselves.
"""
from backend.indexer import path_index
if not q:
return {"query": q, "vault_filter": vault, "results": []}
query_lower = q.lower()
results: list[dict[str, Any]] = []
vaults_to_search = [vault] if vault != "all" else list(path_index.keys())
for vault_name in vaults_to_search:
for entry in path_index.get(vault_name, []):
if query_lower in entry["name"].lower() or query_lower in entry["path"].lower():
results.append({
"vault": vault_name,
"path": entry["path"],
"name": entry["name"],
"type": entry["type"],
"matched_path": entry["path"],
})
return {"query": q, "vault_filter": vault, "results": results}
def list_paths(vault: str, limit: int = 5000) -> dict[str, Any]:
"""Return a flat, capped list of every indexed path in a vault.
Backs the AI assistant ``@`` mention menu: fetching the whole path index
once lets the client filter files/directories instantly instead of issuing
a request per keystroke.
Args:
vault: Vault name.
limit: Maximum number of entries returned.
Returns:
``{"vault", "count", "results": [{vault, path, name, type}]}``.
"""
from backend.indexer import path_index
entries = path_index.get(vault, [])
results = [
{
"vault": vault,
"path": entry["path"],
"name": entry["name"],
"type": entry["type"],
}
for entry in entries[: max(0, limit)]
]
return {"vault": vault, "count": len(results), "results": results}
+214
View File
@@ -0,0 +1,214 @@
"""Vault listing and directory browsing services.
Single source of truth consumed by both the REST routes (``/api/vaults``,
``/api/browse/{vault}``) and the AI tool layer (``list_vaults``,
``list_directory``).
"""
from __future__ import annotations
from pathlib import Path
from typing import Any
from backend.services.errors import ServiceError
from backend.services.paths import resolve_safe_path
def list_accessible_vaults(user: dict[str, Any]) -> list[dict[str, Any]]:
"""Return the vaults *user* may access, with summary metadata."""
from backend.auth.middleware import check_vault_access
from backend.indexer import index
result: list[dict[str, Any]] = []
for name, data in index.items():
if not check_vault_access(name, user):
continue
result.append({
"name": name,
"file_count": len(data.get("files", [])),
"tag_count": len(data.get("tags", {})),
"type": data.get("config", {}).get("type", "VAULT"),
})
return result
def get_vault_root(vault_name: str) -> Path:
"""Return the filesystem root of *vault_name* or raise ``not_found``."""
from backend.indexer import get_vault_data
data = get_vault_data(vault_name)
if not data:
raise ServiceError(
f"Vault '{vault_name}' not found",
code="not_found",
status=404,
details={"vault": vault_name},
)
return Path(data["path"])
def browse_directory(vault_name: str, path: str = "") -> dict[str, Any]:
"""Return the direct children of a vault directory (directories first)."""
from backend.indexer import SUPPORTED_EXTENSIONS
from backend.vault_settings import get_vault_setting
root = get_vault_root(vault_name)
target = resolve_safe_path(root, path) if path else root.resolve()
if not target.exists():
raise ServiceError(
f"Path not found: {path}",
code="not_found",
status=404,
details={"vault": vault_name, "path": path},
)
hide_hidden = (get_vault_setting(vault_name) or {}).get("hideHiddenFiles", False)
items: list[dict[str, Any]] = []
try:
for entry in sorted(target.iterdir(), key=lambda e: (not e.is_dir(), e.name.lower())):
if hide_hidden and entry.name.startswith("."):
continue
rel = str(entry.relative_to(root)).replace("\\", "/")
if entry.is_dir():
# Count only direct children (files and subdirs) for performance.
try:
file_count = sum(
1 for child in entry.iterdir()
if (not hide_hidden or not child.name.startswith("."))
and (child.is_file() and (child.suffix.lower() in SUPPORTED_EXTENSIONS or child.name.lower() in ("dockerfile", "makefile"))
or child.is_dir())
)
except PermissionError:
file_count = 0
items.append({
"name": entry.name,
"path": rel,
"type": "directory",
"children_count": file_count,
})
elif entry.suffix.lower() in SUPPORTED_EXTENSIONS or entry.name.lower() in ("dockerfile", "makefile"):
items.append({
"name": entry.name,
"path": rel,
"type": "file",
"size": entry.stat().st_size,
"extension": entry.suffix.lower(),
})
except PermissionError:
raise ServiceError("Permission denied", code="permission_denied", status=403) from None
return {"vault": vault_name, "path": path, "items": items}
def list_all_files(
vault_name: str,
dir: str = "",
limit: int = 200,
recursive: bool = True,
) -> dict[str, Any]:
"""List files in a vault directory sorted by modification time (newest first).
Unlike :func:`browse_directory`, this returns files only (with metadata:
size, mtime, extension) and can recurse into subdirectories. Hidden files
and ignored directories are skipped.
Args:
vault_name: Name of the vault.
dir: Relative directory path within the vault (empty = root).
limit: Maximum number of files to return.
recursive: Recurse into subdirectories when True.
Returns:
Dict with ``vault``, ``directory``, ``recursive``, ``count`` and
``files``.
Raises:
ServiceError: ``not_found`` (404) if the directory does not exist,
``permission_denied`` (403) if it cannot be read.
"""
from datetime import datetime, timezone
from backend.indexer import IGNORED_DIRS
root = get_vault_root(vault_name)
dir_path = resolve_safe_path(root, dir) if dir else root
if not dir_path.exists() or not dir_path.is_dir():
raise ServiceError(
f"Directory not found: {dir}",
code="not_found",
status=404,
details={"vault": vault_name, "path": dir},
)
files: list[dict[str, Any]] = []
dir_prefix = (dir or "").strip("/")
try:
iterator = dir_path.rglob("*") if recursive else dir_path.iterdir()
for entry in iterator:
if not entry.is_file():
continue
if entry.name.startswith("."):
continue
if entry.name in IGNORED_DIRS:
continue
if recursive and dir_prefix:
skip = False
try:
for part in entry.relative_to(dir_path).parts[:-1]:
if part.startswith(".") or part in IGNORED_DIRS:
skip = True
break
except ValueError:
pass
if skip:
continue
stat = entry.stat()
rel_path = str(entry.relative_to(root)).replace("\\", "/")
if recursive and dir_prefix:
try:
rel_to_dir = str(entry.parent.relative_to(dir_path)).replace("\\", "/")
except ValueError:
rel_to_dir = ""
else:
rel_to_dir = ""
ext = entry.suffix.lower() if entry.suffix else ""
file_entry = {
"name": entry.name,
"path": rel_path,
"vault": vault_name,
"size": stat.st_size,
"modified": stat.st_mtime,
"modified_iso": datetime.fromtimestamp(stat.st_mtime, tz=timezone.utc).isoformat(),
"extension": ext.lstrip(".") if ext else "",
}
if rel_to_dir and rel_to_dir != ".":
file_entry["rel_dir"] = rel_to_dir
files.append(file_entry)
except PermissionError:
raise ServiceError(
"Permission denied reading directory",
code="permission_denied",
status=403,
details={"vault": vault_name, "path": dir},
) from None
files.sort(key=lambda f: f["modified"], reverse=True)
files = files[:limit]
return {
"vault": vault_name,
"directory": dir,
"recursive": recursive,
"count": len(files),
"files": files,
}
+317
View File
@@ -0,0 +1,317 @@
"""AI assistant skills & slash-commands.
A *skill* is a reusable prompt/workflow the user can trigger from the assistant
composer with ``/``. ObsiGate ships a set of built-in skills; users can create
their own with ``/create-new-skill``. Built-in skills are code constants, while
user skills are persisted per-user in ``data/skills.json``.
The same module also exposes the metadata for the *admin* commands
(``/help``, ``/providers``, ``/provider``, ``/model``, ``/keys``). Those are
executed client-side (they only touch the picker / display info), but listing
them here keeps the ``/`` menu single-sourced.
"""
from __future__ import annotations
import json
import logging
import re
from pathlib import Path
from typing import Any
logger = logging.getLogger("obsigate.skills")
SKILLS_FILE = Path("data/skills.json")
MAX_USER_SKILLS = 100
MAX_PROMPT_CHARS = 8000
_SKILL_ID_RE = re.compile(r"^[a-z0-9][a-z0-9_-]{0,47}$")
# ── Built-in skills ─────────────────────────────────────────────────────────
# ``prompt`` is appended to the assistant system prompt when the skill is
# selected. Keep prompts concise and language-agnostic: the model answers in
# the user's language.
BUILTIN_SKILLS: list[dict[str, Any]] = [
{
"id": "research",
"label": "Recherche structurée",
"icon": "🔎",
"type": "skill",
"description": "Recherche structurée + recommandation",
"prompt": (
"Applique un mode RECHERCHE STRUCTURÉE. Structure ta réponse en : "
"1) Contexte et question reformulée, 2) Constats appuyés sur le contenu fourni, "
"3) Options/approches avec avantages et limites, 4) Recommandation argumentée. "
"Cite les sources (fichiers) utilisées."
),
},
{
"id": "create-new-skill",
"label": "Créer un skill",
"icon": "🛠️",
"type": "skill",
"special": "create_skill",
"description": "Crée un workflow réutilisable (skill)",
"prompt": "",
},
{
"id": "resume",
"label": "Résumé",
"icon": "📄",
"type": "skill",
"description": "Résumé / synthèse structurée",
"prompt": (
"Produis un RÉSUMÉ structuré du contenu : idées clés, points importants, "
"conclusions. Utilise des titres et des puces concises."
),
},
{
"id": "actions",
"label": "Actions & to-dos",
"icon": "✅",
"type": "skill",
"description": "Extraire les actions & to-dos",
"prompt": (
"Extrais les ACTIONS et TO-DOS du contenu. Rends une liste de tâches markdown "
"`- [ ] ...`, avec responsable et échéance si mentionnés, sinon `(à préciser)`."
),
},
{
"id": "reformuler",
"label": "Reformuler",
"icon": "✍️",
"type": "skill",
"description": "Réécriture clarté / ton",
"prompt": (
"RÉÉCRIS le contenu pour améliorer la clarté et le ton, en préservant le sens. "
"Retourne uniquement le texte reformulé."
),
},
{
"id": "correction",
"label": "Correction",
"icon": "🔤",
"type": "skill",
"description": "Correction grammaire / orthographe / style",
"prompt": (
"CORRIGE la grammaire, l'orthographe et le style. Retourne le texte corrigé, "
"puis une courte liste des corrections notables."
),
},
{
"id": "brainstorm",
"label": "Brainstorm",
"icon": "💡",
"type": "skill",
"description": "Générer des idées, angles, variantes",
"prompt": (
"Mode BRAINSTORM : génère un maximum d'idées, angles et variantes pertinents. "
"Regroupe-les par thème, sans juger, puis signale les plus prometteuses."
),
},
{
"id": "plan",
"label": "Planifier",
"icon": "🧭",
"type": "skill",
"description": "Planifier / structurer un document",
"prompt": (
"PLANIFIE et structure un document : propose un plan détaillé (sections, "
"sous-sections, objectif de chaque partie) et une progression logique."
),
},
{
"id": "ask",
"label": "Q&R",
"icon": "💬",
"type": "skill",
"description": "Q&A sur un contenu référencé",
"prompt": (
"Mode QUESTION/RÉPONSE : réponds précisément à la question en te basant "
"strictement sur le contenu référencé. Cite les passages/fichiers utilisés et "
"dis clairement si l'information est absente."
),
},
{
"id": "meeting-note",
"label": "Note de réunion",
"icon": "📝",
"type": "skill",
"description": "Compte-rendu / note de réunion",
"prompt": (
"Rédige une NOTE DE RÉUNION : participants, ordre du jour, décisions, "
"points d'action (`- [ ] ...`), questions ouvertes et prochaines étapes."
),
},
{
"id": "livrable",
"label": "Livrable",
"icon": "📨",
"type": "skill",
"description": "Email / compte-rendu / message Slack",
"prompt": (
"Rédige un LIVRABLE de communication (email, compte-rendu ou message Slack) "
"clair et prêt à envoyer, adapté au canal et au destinataire indiqués."
),
},
]
# ── Admin commands (handled client-side) ────────────────────────────────────
ADMIN_COMMANDS: list[dict[str, Any]] = [
{
"id": "help",
"label": "Aide",
"icon": "❓",
"type": "admin",
"description": "Liste des commandes",
},
{
"id": "providers",
"label": "Fournisseurs",
"icon": "🔌",
"type": "admin",
"description": "Liste les fournisseurs actifs",
},
{
"id": "provider",
"label": "Changer de fournisseur",
"icon": "🔀",
"type": "admin",
"usage": "/provider <nom>",
"description": "Changer de fournisseur LLM",
},
{
"id": "model",
"label": "Changer de modèle",
"icon": "🧠",
"type": "admin",
"usage": "/model <nom>",
"description": "Changer de modèle LLM",
},
{
"id": "keys",
"label": "Clés API",
"icon": "🔑",
"type": "admin",
"description": "Fournisseurs avec clé API enregistrée",
},
]
def list_builtin_skills() -> list[dict[str, Any]]:
"""Return a copy of the built-in skills."""
return [dict(skill) for skill in BUILTIN_SKILLS]
def list_admin_commands() -> list[dict[str, Any]]:
"""Return a copy of the admin command metadata."""
return [dict(cmd) for cmd in ADMIN_COMMANDS]
def _read_store() -> dict[str, list[dict[str, Any]]]:
if not SKILLS_FILE.exists():
return {}
try:
data = json.loads(SKILLS_FILE.read_text(encoding="utf-8"))
return data if isinstance(data, dict) else {}
except Exception as exc: # pragma: no cover - corrupted file
logger.warning("Cannot read skills store: %s", exc)
return {}
def _write_store(store: dict[str, list[dict[str, Any]]]) -> None:
SKILLS_FILE.parent.mkdir(parents=True, exist_ok=True)
tmp = SKILLS_FILE.with_suffix(".tmp")
tmp.write_text(json.dumps(store, indent=2, ensure_ascii=False), encoding="utf-8")
tmp.replace(SKILLS_FILE)
def _username(user: dict | None) -> str:
if not user:
return "anonymous"
return str(user.get("username") or "anonymous")
def list_user_skills(user: dict | None) -> list[dict[str, Any]]:
"""Return the persisted custom skills for a user."""
store = _read_store()
skills = store.get(_username(user), [])
return [dict(s) for s in skills if isinstance(s, dict)]
def list_skills(user: dict | None) -> dict[str, list[dict[str, Any]]]:
"""Return built-in skills, admin commands and the user's custom skills."""
return {
"skills": list_builtin_skills() + list_user_skills(user),
"commands": list_admin_commands(),
}
def get_skill_prompt(skill_id: str | None, user: dict | None) -> str | None:
"""Resolve a skill id to its prompt, searching built-ins then user skills."""
if not skill_id:
return None
for skill in BUILTIN_SKILLS:
if skill["id"] == skill_id:
return skill.get("prompt") or None
for skill in list_user_skills(user):
if skill.get("id") == skill_id:
return skill.get("prompt") or None
return None
def create_user_skill(user: dict | None, payload: dict[str, Any]) -> dict[str, Any]:
"""Create and persist a custom skill for a user.
Raises:
ValueError: when the payload is invalid (bad id, duplicate, too many).
"""
skill_id = str(payload.get("id") or "").strip().lower()
label = str(payload.get("label") or "").strip()
prompt = str(payload.get("prompt") or "").strip()
description = str(payload.get("description") or "").strip()
if not _SKILL_ID_RE.match(skill_id):
raise ValueError("Identifiant invalide (a-z, 0-9, '-', '_', max 48)")
if any(s["id"] == skill_id for s in BUILTIN_SKILLS):
raise ValueError(f"L'identifiant '{skill_id}' est réservé")
if not label:
raise ValueError("Le nom du skill est requis")
if not prompt:
raise ValueError("Le prompt du skill est requis")
if len(prompt) > MAX_PROMPT_CHARS:
raise ValueError("Le prompt est trop long")
username = _username(user)
store = _read_store()
user_skills = store.get(username, [])
if any(s.get("id") == skill_id for s in user_skills):
raise ValueError(f"Le skill '{skill_id}' existe déjà")
if len(user_skills) >= MAX_USER_SKILLS:
raise ValueError("Trop de skills personnalisés")
skill = {
"id": skill_id,
"label": label,
"icon": str(payload.get("icon") or "🧩").strip() or "🧩",
"type": "skill",
"custom": True,
"description": description or label,
"prompt": prompt,
}
user_skills.append(skill)
store[username] = user_skills
_write_store(store)
return skill
def delete_user_skill(user: dict | None, skill_id: str) -> bool:
"""Delete a custom skill. Returns True when a skill was removed."""
username = _username(user)
store = _read_store()
user_skills = store.get(username, [])
remaining = [s for s in user_skills if s.get("id") != skill_id]
if len(remaining) == len(user_skills):
return False
store[username] = remaining
_write_store(store)
return True
+98
View File
@@ -0,0 +1,98 @@
"""API routes for AI assistant skills & slash-commands (``/api/ai/skills``)."""
from __future__ import annotations
import logging
from typing import Any
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel, Field
from backend.auth.middleware import require_auth
from backend.skills import (
create_user_skill,
delete_user_skill,
list_skills,
)
logger = logging.getLogger("obsigate.skills_routes")
router = APIRouter(prefix="/api/ai/skills", tags=["AI"])
class SkillModel(BaseModel):
"""A single skill or admin command."""
model_config = {"extra": "allow"}
id: str
label: str
icon: str = "🧩"
type: str = Field(default="skill", description="'skill' or 'admin'")
description: str = ""
prompt: str | None = None
usage: str | None = None
special: str | None = None
custom: bool = False
class SkillsResponse(BaseModel):
"""Response for ``GET /api/ai/skills``."""
skills: list[SkillModel]
commands: list[SkillModel]
class CreateSkillRequest(BaseModel):
"""Body for ``POST /api/ai/skills``."""
id: str = Field(description="Stable identifier (a-z, 0-9, '-', '_')")
label: str = Field(description="Display name")
prompt: str = Field(description="Instruction injected into the system prompt")
description: str = ""
icon: str = "🧩"
class DeleteSkillResponse(BaseModel):
"""Response for ``DELETE /api/ai/skills/{skill_id}``."""
status: str = "deleted"
id: str
@router.get("", response_model=SkillsResponse)
async def api_list_skills(current_user=Depends(require_auth)):
"""List built-in skills, admin commands and the user's custom skills."""
data = list_skills(current_user)
return data
@router.post("", response_model=SkillModel)
async def api_create_skill(body: CreateSkillRequest, current_user=Depends(require_auth)):
"""Create a custom skill for the current user."""
try:
skill = create_user_skill(current_user, body.model_dump())
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
return skill
@router.delete("/{skill_id}", response_model=DeleteSkillResponse)
async def api_delete_skill(skill_id: str, current_user=Depends(require_auth)):
"""Delete a custom skill owned by the current user."""
removed = delete_user_skill(current_user, skill_id)
if not removed:
raise HTTPException(status_code=404, detail="Skill introuvable")
return {"status": "deleted", "id": skill_id}
# Expose the resolved prompt of a skill (used by tests / clients that only
# need the instruction text without fetching the whole list).
@router.get("/{skill_id}/prompt")
async def api_skill_prompt(skill_id: str, current_user=Depends(require_auth)) -> dict[str, Any]:
"""Return the prompt text for a given skill id."""
from backend.skills import get_skill_prompt
prompt = get_skill_prompt(skill_id, current_user)
if prompt is None:
raise HTTPException(status_code=404, detail="Skill introuvable")
return {"id": skill_id, "prompt": prompt}
+70
View File
@@ -0,0 +1,70 @@
"""Public facade for the AI tool layer.
Importing this module registers all built-in tools (via
``backend.tools.service``) and re-exports the public API. Consumers (agent
loop, MCP server, tests) should import from here rather than from individual
submodules.
Note: ObsiGate uses implicit namespace packages (no tracked ``__init__.py``,
which ``.gitignore`` excludes via ``_*.py``), hence this explicit facade.
"""
from backend.tools import service as _service # noqa: F401 (registers tools)
from backend.tools import web as _web # noqa: F401 (registers web tools)
from backend.tools.context import (
ToolConfirmationRequired,
ToolContext,
ToolError,
ToolMode,
ToolNotFoundError,
ToolPermissionError,
ToolRateLimitError,
ToolRisk,
ToolScope,
ToolValidationError,
resolve_safe_path,
)
from backend.tools.ratelimit import (
check_and_record as check_tool_rate_limit,
)
from backend.tools.ratelimit import (
get_status as get_tool_rate_limit_status,
)
from backend.tools.ratelimit import (
reset as reset_tool_rate_limit,
)
from backend.tools.redaction import redact_payload
from backend.tools.registry import (
ToolSpec,
call_tool,
get_tool,
get_tool_schemas,
list_tools,
tool,
)
from backend.tools.schemas import ToolResult
__all__ = [
"ToolConfirmationRequired",
"ToolContext",
"ToolError",
"ToolMode",
"ToolNotFoundError",
"ToolPermissionError",
"ToolRateLimitError",
"ToolResult",
"ToolRisk",
"ToolScope",
"ToolSpec",
"ToolValidationError",
"call_tool",
"check_tool_rate_limit",
"get_tool",
"get_tool_rate_limit_status",
"get_tool_schemas",
"list_tools",
"redact_payload",
"reset_tool_rate_limit",
"resolve_safe_path",
"tool",
]
+54
View File
@@ -0,0 +1,54 @@
"""Audit logging for AI tool calls.
Reuses the application audit log (``data/audit.log``, JSON lines) and adds an
``ai_tool_call`` action. Argument values that may contain sensitive payloads
(file content, prompts) are summarized rather than stored verbatim.
"""
from __future__ import annotations
from datetime import datetime, timezone
from typing import Any
from backend.audit import _write_entry
# Argument keys whose values may contain secrets or large payloads.
_SENSITIVE_ARG_KEYS = {"content", "text", "body"}
_MAX_ARG_CHARS = 200
def _sanitize_arguments(arguments: dict[str, Any] | None) -> dict[str, Any]:
"""Return a log-safe view of tool arguments."""
safe: dict[str, Any] = {}
for key, value in (arguments or {}).items():
if key in _SENSITIVE_ARG_KEYS:
safe[key] = f"<{len(str(value))} chars>"
else:
safe[key] = str(value)[:_MAX_ARG_CHARS]
return safe
def log_tool_call(
*,
username: str,
mode: str,
tool: str,
arguments: dict[str, Any] | None = None,
ok: bool = True,
vault: str | None = None,
ip: str | None = None,
error: str | None = None,
) -> None:
"""Append an ``ai_tool_call`` entry to the audit log."""
_write_entry({
"timestamp": datetime.now(timezone.utc).isoformat(),
"action": "ai_tool_call",
"username": username,
"ip": ip or "unknown",
"mode": mode,
"tool": tool,
"vault": vault,
"ok": ok,
"error": error,
"arguments": _sanitize_arguments(arguments),
})
+210
View File
@@ -0,0 +1,210 @@
"""Execution context and shared primitives for the AI tool layer.
The tool layer is deliberately transport-agnostic: the same tool functions are
invoked by the in-app assistant (function calling) and by the MCP server. This
module holds the context object that carries the caller identity, the
permission helpers, and the domain error types shared across the layer.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass, field
from enum import Enum
from pathlib import Path
from typing import Any
from backend.auth.middleware import check_vault_access
from backend.services.errors import ServiceError
from backend.services.paths import resolve_safe_path as _resolve_service_path
logger = logging.getLogger("obsigate.tools")
class ToolMode(str, Enum):
"""How a tool was invoked."""
IN_APP = "in_app"
MCP = "mcp"
class ToolScope(str, Enum):
"""Which front a tool is exposed to."""
IN_APP = "in_app"
MCP = "mcp"
class ToolRisk(str, Enum):
"""Risk level of a tool.
``READ`` tools run without confirmation. ``WRITE`` and ``DANGEROUS`` tools
require an explicit confirmation (two-step ``propose``/``apply`` for MCP,
an "Apply" card in the in-app UI).
"""
READ = "read"
WRITE = "write"
DANGEROUS = "dangerous"
class ToolError(Exception):
"""Base class for tool execution errors.
Carries a stable, machine-readable ``code`` so callers (agent loop, MCP
server) can react without parsing the message.
"""
code = "tool_error"
def __init__(self, message: str, *, code: str | None = None, details: dict[str, Any] | None = None):
super().__init__(message)
self.message = message
if code:
self.code = code
self.details = details or {}
def to_dict(self) -> dict[str, Any]:
return {"ok": False, "error": {"code": self.code, "message": self.message, "details": self.details}}
class ToolNotFoundError(ToolError):
code = "not_found"
class ToolPermissionError(ToolError):
code = "permission_denied"
class ToolValidationError(ToolError):
code = "invalid_arguments"
class ToolConfirmationRequired(ToolError):
"""Raised when a mutating tool is invoked without confirmation.
This is the hook for the two-step ``propose``/``apply`` mechanism: the
``propose`` phase surfaces this payload to the user, and the ``apply``
phase re-invokes the tool with ``confirm=True``.
"""
code = "confirmation_required"
def __init__(self, tool: str, arguments: dict[str, Any], message: str | None = None):
super().__init__(message or f"Confirmation required for '{tool}'", code="confirmation_required")
self.tool = tool
self.arguments = arguments
def to_dict(self) -> dict[str, Any]:
payload = super().to_dict()
payload["error"]["tool"] = self.tool
payload["error"]["arguments"] = self.arguments
return payload
class ToolRateLimitError(ToolError):
"""Raised when an identity exceeds its tool-call rate limit (Phase F)."""
code = "rate_limited"
def __init__(self, tool: str, retry_after: int = 0, message: str | None = None):
super().__init__(
message or f"Rate limit exceeded for tool '{tool}'",
code="rate_limited",
details={"tool": tool, "retry_after": retry_after},
)
self.tool = tool
self.retry_after = retry_after
@dataclass
class ToolContext:
"""Carries the caller identity and execution options for a tool call.
Attributes:
user: Authenticated user dict (as returned by ``get_current_user``).
mode: Whether the call originates from the in-app assistant or MCP.
confirmed: True once a mutating action has been approved.
ip: Optional client IP for auditing.
audit_enabled: Set to False to skip audit logging (tests, dry runs).
metadata: Free-form caller metadata (conversation id, client name…).
"""
user: dict[str, Any]
mode: ToolMode = ToolMode.IN_APP
confirmed: bool = False
ip: str | None = None
audit_enabled: bool = True
metadata: dict[str, Any] = field(default_factory=dict)
@property
def username(self) -> str:
return self.user.get("username", "unknown")
@property
def rate_limit_identity(self) -> str:
"""Identity used by the per-token/per-tool rate limiter.
Prefers the JWT id (``_token_jti``, attached by the auth middleware),
then an explicit ``token_id`` in :attr:`metadata`, then the user id and
finally the username. This yields per-token limiting when a token id is
available and per-account limiting otherwise.
"""
token_id = self.metadata.get("token_id") or self.user.get("_token_jti")
if token_id:
return f"token:{token_id}"
user_id = self.user.get("id") or self.user.get("username")
return f"user:{user_id or 'anonymous'}"
def has_vault_access(self, vault: str) -> bool:
"""Return True if the caller may access *vault*."""
return bool(vault) and check_vault_access(vault, self.user)
def require_vault_access(self, vault: str) -> None:
"""Raise :class:`ToolPermissionError` unless the caller can access *vault*."""
if not self.has_vault_access(vault):
raise ToolPermissionError(
f"Access denied to vault '{vault}'",
code="vault_access_denied",
details={"vault": vault},
)
def destructive_tools_enabled(self, vault: str) -> bool:
"""Return whether destructive tools are allowed for *vault*.
Controlled by the per-vault ``aiDestructiveTools`` setting (default:
enabled). Disabling it blocks delete/rename/move/find-replace tools
while keeping create/edit/append available.
"""
from backend.vault_settings import get_vault_setting
settings = get_vault_setting(vault) or {}
return bool(settings.get("aiDestructiveTools", True))
def require_destructive_allowed(self, vault: str) -> None:
"""Raise :class:`ToolPermissionError` if destructive tools are disabled."""
if not self.destructive_tools_enabled(vault):
raise ToolPermissionError(
f"Destructive tools are disabled for vault '{vault}'",
code="destructive_tools_disabled",
details={"vault": vault},
)
def resolve_safe_path(vault_root: Path, relative_path: str | None) -> Path:
"""Resolve a vault-relative path, rejecting traversal outside the vault.
Delegates to the shared :func:`backend.services.paths.resolve_safe_path`
and maps its :class:`ServiceError` to tool domain errors so the tool layer
stays transport-agnostic.
Raises:
ToolPermissionError: When the resolved path escapes the vault root.
ToolError: When the path cannot be resolved.
"""
try:
return _resolve_service_path(vault_root, relative_path)
except ServiceError as e:
if e.code in ("path_outside_vault", "permission_denied"):
raise ToolPermissionError(e.message, code=e.code, details=e.details) from e
raise ToolError(e.message, code=e.code, details=e.details) from e
+93
View File
@@ -0,0 +1,93 @@
"""Human-readable labels for tool calls (Notion-style “steps” section).
The assistant's conversation window shows a collapsible « N steps » block
above each answer. Raw tool names (``search_fulltext``) are developer-centric;
this module maps every registered tool to an i18n message key plus the
argument worth surfacing (path, query, …), so the UI can render sentences like
« Recherche dans le vault : pizza » or « Fichier lu : notes/x.md ».
The label KEY is emitted with each ``tool`` SSE event; the client resolves it
against its locale dictionaries (``ai.step.<key>``). Unknown tools fall back to
a generic key with the raw name prettified, so a new tool never breaks the UI.
"""
from __future__ import annotations
from typing import Any
# tool name -> (message-key, primary argument name or None)
# NOTE: argument names MUST match the pydantic input models in
# backend/tools/schemas.py (verified by tests/test_tool_labels.py).
_STEP_LABELS: dict[str, tuple[str, str | None]] = {
"list_vaults": ("vaults", None),
"list_directory": ("directory", "path"),
"list_all_files": ("files", "dir"),
"read_file": ("file_read", "path"),
"read_file_raw": ("file_read", "path"),
"get_backlinks": ("backlinks", "path"),
"list_backups": ("backups", "path"),
"diff_backup": ("backup_diff", "path"),
"get_graph": ("graph", None),
"search_fulltext": ("search", "q"),
"search_advanced": ("search", "q"),
"search_paths": ("search_paths", "q"),
"list_tags": ("tags", None),
"suggest_tags": ("tags_suggest", "q"),
"list_recent": ("recent", None),
"create_file": ("file_create", "path"),
"create_directory": ("dir_create", "path"),
"edit_file": ("file_edit", "path"),
"append_to_file": ("file_append", "path"),
"rename_file": ("file_rename", "path"),
"rename_directory": ("dir_rename", "path"),
"move_path": ("move", "source_path"),
"replace_in_files": ("replace", "find"),
"delete_file": ("file_delete", "path"),
"delete_directory": ("dir_delete", "path"),
"restore_backup": ("backup_restore", "path"),
"web_search": ("web_search", "query"),
"fetch_url": ("fetch_url", "url"),
}
GENERIC_KEY = "generic"
def _prettify(name: str) -> str:
return name.replace("_", " ").strip()
def _primary_value(arguments: dict[str, Any], arg: str | None) -> str | None:
if not arg:
return None
value = arguments.get(arg)
if isinstance(value, str) and value.strip():
return value.strip()
return None
def tool_step_label(name: str, arguments: dict[str, Any] | None = None) -> dict[str, Any]:
"""Return ``{key, params}`` describing one tool call in human terms.
``key`` resolves client-side to ``ai.step.<key>``; ``params`` carries the
optional ``{value}`` placeholder (path, query, …). For an unknown tool the
generic key is used with the prettified name as the value.
"""
arguments = arguments or {}
key, arg = _STEP_LABELS.get(name, (GENERIC_KEY, None))
if key == GENERIC_KEY:
return {"key": GENERIC_KEY, "params": {"tool": _prettify(name)}}
value = _primary_value(arguments, arg)
params: dict[str, str] = {}
if value is not None:
params["value"] = value
return {"key": key, "params": params}
def thought_step_label(text: str) -> dict[str, Any]:
"""Step descriptor for one intermediate reasoning note of the model.
The note is shown expanded under a “Thought” sub-section in the UI, so it
is kept long enough to be readable (truncation is the safety net).
"""
cleaned = " ".join((text or "").split())
return {"key": "thought", "params": {"value": cleaned[:1200]}}
+153
View File
@@ -0,0 +1,153 @@
"""Rate limiting for the AI tool layer (Phase F).
Complements the IP-based login limiter (``backend.ratelimit``) with a
per-identity, per-tool sliding-window limiter applied to every tool call,
whether it originates from the in-app assistant or the MCP server.
Two counters are maintained per identity:
* a **global** counter (all tools combined), capped by
``OBSIGATE_TOOL_RATE_LIMIT``;
* a **per-tool** counter, capped by ``OBSIGATE_TOOL_RATE_LIMIT_PER_TOOL``
(defaults to the global cap) so a single expensive tool cannot consume the
whole budget.
The window length is ``OBSIGATE_TOOL_RATE_WINDOW`` seconds (default 60).
The identity is the token JTI when available (``_token_jti`` attached by the
auth middleware), otherwise the user id/username. This gives "per token / per
tool" limiting while remaining meaningful for anonymous/disabled-auth mode.
"""
from __future__ import annotations
import logging
import os
import time
from collections import deque
from typing import Any
logger = logging.getLogger("obsigate.tools.ratelimit")
# --- Configuration (read at call time so tests can monkeypatch env) ---
DEFAULT_RATE_LIMIT = 60
DEFAULT_RATE_WINDOW = 60
def _env_int(name: str, default: int) -> int:
try:
return int(os.environ.get(name, str(default)))
except (TypeError, ValueError):
return default
def _global_limit() -> int:
return _env_int("OBSIGATE_TOOL_RATE_LIMIT", DEFAULT_RATE_LIMIT)
def _per_tool_limit() -> int:
return _env_int("OBSIGATE_TOOL_RATE_LIMIT_PER_TOOL", _global_limit())
def _window() -> int:
return max(1, _env_int("OBSIGATE_TOOL_RATE_WINDOW", DEFAULT_RATE_WINDOW))
# --- In-memory store: {counter_key: deque[timestamp]} ---
_calls: dict[str, deque] = {}
def _identity_key(identity: str | None) -> str:
return identity or "anonymous"
def _counter_key(identity: str | None, tool: str | None) -> str:
base = _identity_key(identity)
return f"{base}\x00{tool}" if tool else base
def _prune(entries: deque, now: float, window: int) -> None:
cutoff = now - window
while entries and entries[0] <= cutoff:
entries.popleft()
def check_and_record(identity: str | None, tool: str, *, now: float | None = None) -> tuple[bool, int]:
"""Record one call and report whether it is allowed.
Args:
identity: Rate-limit identity (token JTI or username).
tool: Tool name (used for the per-tool counter).
now: Optional timestamp override (tests).
Returns:
``(allowed, retry_after)``. When ``allowed`` is False, ``retry_after``
is the number of seconds until the oldest call leaves the window.
"""
now = time.time() if now is None else now
window = _window()
global_limit = _global_limit()
tool_limit = _per_tool_limit()
global_entries = _calls.setdefault(_counter_key(identity, None), deque())
tool_entries = _calls.setdefault(_counter_key(identity, tool), deque())
_prune(global_entries, now, window)
_prune(tool_entries, now, window)
if len(global_entries) >= global_limit or len(tool_entries) >= tool_limit:
oldest = min(
global_entries[0] if len(global_entries) >= global_limit else now,
tool_entries[0] if len(tool_entries) >= tool_limit else now,
)
retry_after = max(1, int(oldest + window - now) + 1)
logger.warning(f"Tool rate limit exceeded for '{_identity_key(identity)}' on '{tool}'")
return False, retry_after
global_entries.append(now)
tool_entries.append(now)
return True, 0
def remaining(identity: str | None, tool: str | None = None) -> int:
"""Return the number of calls still allowed in the current window."""
now = time.time()
window = _window()
if tool:
entries = _calls.get(_counter_key(identity, tool))
if entries is None:
return _per_tool_limit()
_prune(entries, now, window)
return max(0, _per_tool_limit() - len(entries))
entries = _calls.get(_counter_key(identity, None))
if entries is None:
return _global_limit()
_prune(entries, now, window)
return max(0, _global_limit() - len(entries))
def reset(identity: str | None = None) -> None:
"""Clear rate-limit state (all identities, or a single one)."""
if identity is None:
_calls.clear()
return
prefix = f"{_identity_key(identity)}\x00"
for key in [k for k in _calls if k == _identity_key(identity) or k.startswith(prefix)]:
del _calls[key]
def get_status(identity: str | None = None) -> dict[str, Any]:
"""Diagnostic snapshot of the limiter (for tests / admin tooling)."""
if identity is None:
return {
"tracked_identities": len({k.split("\x00", 1)[0] for k in _calls}),
"global_limit": _global_limit(),
"per_tool_limit": _per_tool_limit(),
"window_seconds": _window(),
}
return {
"identity": _identity_key(identity),
"remaining_global": remaining(identity),
"global_limit": _global_limit(),
"per_tool_limit": _per_tool_limit(),
"window_seconds": _window(),
}
+39
View File
@@ -0,0 +1,39 @@
"""Secret redaction for tool results (Phase F).
Tool results are fed back to the LLM (in-app agent loop) or returned to an
external MCP client, so they must never leak credentials. ``read_file`` and
``read_file_raw`` already redact through the shared file service, but other
tools (``diff_backup``, ``search_*``) return content that has not been through
the redactor. This module applies :func:`backend.secret_redactor.redact` to
every string in a tool payload, recursively.
"""
from __future__ import annotations
from typing import Any
from backend.secret_redactor import redact
_MAX_DEPTH = 12
def redact_payload(payload: Any, _depth: int = 0) -> Any:
"""Return a copy of *payload* with every string secret-redacted.
Walks dicts, lists and tuples; scalars are returned unchanged. Strings are
run through :func:`backend.secret_redactor.redact`. A recursion cap avoids
pathological/cyclic structures (tool results are JSON-serialisable, so
cycles should not occur, but the guard keeps this safe).
"""
if _depth > _MAX_DEPTH:
return payload
if isinstance(payload, str):
redacted, count = redact(payload)
return redacted if count else payload
if isinstance(payload, dict):
return {key: redact_payload(value, _depth + 1) for key, value in payload.items()}
if isinstance(payload, list):
return [redact_payload(item, _depth + 1) for item in payload]
if isinstance(payload, tuple):
return tuple(redact_payload(item, _depth + 1) for item in payload)
return payload
+237
View File
@@ -0,0 +1,237 @@
"""Tool registry — declarative registration and uniform execution.
Tools are registered with the :func:`tool` decorator. Each tool declares its
input model (used both for JSON Schema exposure and argument validation), its
risk level, and the fronts (in-app / MCP) it is exposed to.
Execution goes through :func:`call_tool`, which centralizes argument
validation, vault permission checks, confirmation gating, and audit logging.
"""
from __future__ import annotations
import logging
import time
from collections.abc import Callable
from dataclasses import dataclass
from typing import Any
from pydantic import BaseModel, ValidationError
from backend.services.errors import ServiceError
from backend.tools.audit import log_tool_call
from backend.tools.context import (
ToolConfirmationRequired,
ToolContext,
ToolError,
ToolNotFoundError,
ToolPermissionError,
ToolRateLimitError,
ToolRisk,
ToolScope,
ToolValidationError,
)
from backend.tools.ratelimit import check_and_record
from backend.tools.redaction import redact_payload
from backend.tools.schemas import ToolResult
logger = logging.getLogger("obsigate.tools.registry")
@dataclass(frozen=True)
class ToolSpec:
"""Static description of a registered tool."""
name: str
description: str
input_model: type[BaseModel]
handler: Callable[[ToolContext, Any], Any]
risk: ToolRisk = ToolRisk.READ
scopes: tuple[ToolScope, ...] = (ToolScope.IN_APP, ToolScope.MCP)
requires_vault: bool = False
@property
def requires_confirmation(self) -> bool:
"""Mutating tools always require an explicit confirmation."""
return self.risk != ToolRisk.READ
def parameters_schema(self) -> dict[str, Any]:
"""JSON Schema of the tool arguments (for LLM function calling)."""
schema = self.input_model.model_json_schema()
schema.pop("title", None)
return schema
def openai_schema(self) -> dict[str, Any]:
"""OpenAI-compatible function-calling schema."""
return {
"type": "function",
"function": {
"name": self.name,
"description": self.description,
"parameters": self.parameters_schema(),
},
}
_REGISTRY: dict[str, ToolSpec] = {}
def tool(
*,
name: str,
description: str,
input_model: type[BaseModel],
risk: ToolRisk = ToolRisk.READ,
scopes: tuple[ToolScope, ...] = (ToolScope.IN_APP, ToolScope.MCP),
requires_vault: bool = False,
) -> Callable[[Callable[[ToolContext, Any], Any]], Callable[[ToolContext, Any], Any]]:
"""Register a tool. Returns the original handler unchanged."""
def decorator(func: Callable[[ToolContext, Any], Any]) -> Callable[[ToolContext, Any], Any]:
if name in _REGISTRY:
raise ValueError(f"Duplicate tool name: {name}")
_REGISTRY[name] = ToolSpec(
name=name,
description=description,
input_model=input_model,
handler=func,
risk=risk,
scopes=scopes,
requires_vault=requires_vault,
)
return func
return decorator
def get_tool(name: str) -> ToolSpec | None:
"""Return the spec for *name*, or ``None``."""
return _REGISTRY.get(name)
def list_tools(*, scope: ToolScope | None = None) -> list[ToolSpec]:
"""Return registered tools, optionally filtered by scope."""
specs = list(_REGISTRY.values())
if scope is not None:
specs = [s for s in specs if scope in s.scopes]
return specs
def get_tool_schemas(*, scope: ToolScope | None = None) -> list[dict[str, Any]]:
"""Return OpenAI-compatible schemas for registered tools."""
return [spec.openai_schema() for spec in list_tools(scope=scope)]
def _audit(ctx: ToolContext, spec: ToolSpec, arguments: dict[str, Any], *, ok: bool, error: str | None = None) -> None:
if not ctx.audit_enabled:
return
try:
log_tool_call(
username=ctx.username,
mode=ctx.mode.value,
tool=spec.name,
arguments=arguments,
ok=ok,
vault=arguments.get("vault"),
ip=ctx.ip,
error=error,
)
except Exception as e:
logger.debug(f"Tool audit failed for '{spec.name}': {e}")
def _map_service_error(e: ServiceError) -> ToolError:
"""Map a shared-layer :class:`ServiceError` to a tool domain error."""
if e.code == "not_found":
return ToolNotFoundError(e.message, details=e.details)
if e.code in ("permission_denied", "path_outside_vault", "vault_access_denied"):
return ToolPermissionError(e.message, code=e.code, details=e.details)
if e.code == "invalid_arguments":
return ToolValidationError(e.message, details=e.details)
return ToolError(e.message, code=e.code, details=e.details)
def call_tool(
name: str,
ctx: ToolContext,
arguments: dict[str, Any] | None = None,
*,
confirm: bool = False,
) -> ToolResult:
"""Validate, authorize, execute and audit a tool call.
Args:
name: Registered tool name.
ctx: Caller context (identity, mode, confirmation state).
arguments: Raw arguments to validate against the tool input model.
confirm: Explicit one-shot confirmation (two-step ``apply`` phase).
Raises:
ToolNotFoundError: Unknown tool.
ToolValidationError: Arguments fail validation.
ToolPermissionError: Vault access denied or path escapes the vault.
ToolConfirmationRequired: Mutating tool invoked without confirmation.
ToolError: Any other execution error.
"""
spec = _REGISTRY.get(name)
if spec is None:
raise ToolNotFoundError(f"Unknown tool: {name}", details={"tool": name})
arguments = arguments or {}
try:
params = spec.input_model.model_validate(arguments)
except ValidationError as e:
error = ToolValidationError(
f"Invalid arguments for '{name}'",
details={"errors": e.errors(include_url=False)},
)
_audit(ctx, spec, arguments, ok=False, error=error.code)
raise error from e
allowed, retry_after = check_and_record(ctx.rate_limit_identity, spec.name)
if not allowed:
rate_error = ToolRateLimitError(spec.name, retry_after)
_audit(ctx, spec, arguments, ok=False, error=rate_error.code)
raise rate_error
vault = getattr(params, "vault", None)
if spec.requires_vault and vault:
try:
ctx.require_vault_access(vault)
except ToolPermissionError as e:
_audit(ctx, spec, arguments, ok=False, error=e.code)
raise
if spec.risk == ToolRisk.DANGEROUS and vault and vault != "all":
try:
ctx.require_destructive_allowed(vault)
except ToolPermissionError as e:
_audit(ctx, spec, arguments, ok=False, error=e.code)
raise
if spec.requires_confirmation and not (confirm or ctx.confirmed):
raise ToolConfirmationRequired(spec.name, arguments)
started = time.perf_counter()
try:
data = spec.handler(ctx, params)
except ToolError as e:
_audit(ctx, spec, arguments, ok=False, error=e.code)
raise
except ServiceError as e:
mapped = _map_service_error(e)
_audit(ctx, spec, arguments, ok=False, error=mapped.code)
raise mapped from e
except Exception as e:
logger.error(f"Tool '{name}' failed: {e}")
exec_error = ToolError(f"Tool '{name}' failed: {e}", code="tool_execution_error")
_audit(ctx, spec, arguments, ok=False, error=exec_error.code)
raise exec_error from e
duration_ms = round((time.perf_counter() - started) * 1000, 2)
logger.debug(f"Tool '{name}' ok in {duration_ms}ms")
_audit(ctx, spec, arguments, ok=True)
# Never forward raw secrets to the model (defense in depth: some tools —
# diffs, search snippets — return content that was not pre-redacted).
return ToolResult(ok=True, data=redact_payload(data))
+261
View File
@@ -0,0 +1,261 @@
"""Pydantic input/output models for the AI tool layer.
Input models double as JSON Schemas advertised to LLMs (via
``model_json_schema``) and as validation for arguments received over MCP.
"""
from __future__ import annotations
from typing import Any
from pydantic import BaseModel, Field
class ListVaultsInput(BaseModel):
"""No parameters — lists the vaults the caller can access."""
class ListDirectoryInput(BaseModel):
"""Browse a directory inside a vault."""
vault: str = Field(..., description="Vault name")
path: str = Field("", description="Vault-relative directory path (empty = vault root)")
class ReadFileInput(BaseModel):
"""Read a text file's content from a vault."""
vault: str = Field(..., description="Vault name")
path: str = Field(..., description="Vault-relative file path")
class SearchFulltextInput(BaseModel):
"""Full-text search across one or all accessible vaults."""
q: str = Field(..., min_length=1, description="Search query")
vault: str = Field("all", description="Vault name or 'all'")
tag: str | None = Field(None, description="Optional comma-separated tag filter")
limit: int = Field(50, ge=1, le=200, description="Maximum number of results")
class ListTagsInput(BaseModel):
"""List tags and their occurrence counts."""
vault: str | None = Field(None, description="Vault name or 'all' (default: all accessible)")
class ListAllFilesInput(BaseModel):
"""List every file of a vault (optionally under a subdirectory)."""
vault: str = Field(..., description="Vault name")
dir: str = Field("", description="Vault-relative directory path (empty = vault root)")
limit: int = Field(200, ge=1, le=2000, description="Maximum number of files")
recursive: bool = Field(True, description="Recurse into subdirectories")
class ReadFileRawInput(BaseModel):
"""Read a text file's raw content (no HTML rendering)."""
vault: str = Field(..., description="Vault name")
path: str = Field(..., description="Vault-relative file path")
class GetBacklinksInput(BaseModel):
"""List files linking to a target file via wikilinks."""
vault: str = Field(..., description="Vault name")
path: str = Field(..., description="Vault-relative path of the target file")
class ListBackupsInput(BaseModel):
"""List available backup versions of a file."""
vault: str = Field(..., description="Vault name")
path: str = Field(..., description="Vault-relative file path")
class DiffBackupInput(BaseModel):
"""Unified diff between a backup version and another version or the current file."""
vault: str = Field(..., description="Vault name")
path: str = Field(..., description="Vault-relative file path")
version: int = Field(..., description="Backup timestamp used as the old/left side")
compare_with: int | None = Field(
None,
description="Backup timestamp used as the new/right side (omit to compare with the current file)",
)
class GetGraphInput(BaseModel):
"""Graph data (nodes and edges) for a vault or directory."""
vault: str = Field(..., description="Vault name")
path: str = Field("", description="Vault-relative directory path to focus on (empty = root)")
depth: int = Field(1, ge=0, le=3, description="Expansion depth")
scope: str = Field("directory", description="'directory' for a subtree, 'full' for the whole vault")
tag: str = Field("", description="Only include files carrying this tag")
class SearchAdvancedInput(BaseModel):
"""Advanced full-text search (TF-IDF, facets, operators)."""
q: str = Field("", description="Query (supports tag:, vault:, title:, path:, ext: operators)")
vault: str = Field("all", description="Vault name or 'all'")
tag: str | None = Field(None, description="Optional comma-separated tag filter")
limit: int = Field(50, ge=1, le=200, description="Maximum number of results")
offset: int = Field(0, ge=0, description="Pagination offset")
sort: str = Field("relevance", description="Sort by 'relevance' or 'modified'")
case_sensitive: bool = Field(False, description="Match case")
whole_word: bool = Field(False, description="Match whole words only")
regex: bool = Field(False, description="Treat query as a regular expression")
include_paths: str | None = Field(None, description="Comma-separated glob patterns to include")
exclude_paths: str | None = Field(None, description="Comma-separated glob patterns to exclude")
created: str | None = Field(None, description="Created date filter (>date, <date, date..date)")
modified: str | None = Field(None, description="Modified date filter (>date, <date, date..date, <Nd)")
size: str | None = Field(None, description="Size filter (>size, <size, size..size, e.g. >1MB)")
class SearchPathsInput(BaseModel):
"""Search files and directories by path substring."""
q: str = Field(..., min_length=1, description="Path substring to search for")
vault: str = Field("all", description="Vault name or 'all'")
class SuggestTagsInput(BaseModel):
"""Suggest tags matching a prefix."""
q: str = Field(..., min_length=1, description="Tag prefix (with or without leading '#')")
vault: str = Field("all", description="Vault name or 'all'")
limit: int = Field(10, ge=1, le=50, description="Maximum number of suggestions")
class ListRecentInput(BaseModel):
"""List recently opened (or modified) files."""
vault: str | None = Field(None, description="Optional single-vault filter")
limit: int = Field(20, ge=1, le=200, description="Maximum number of files")
mode: str = Field("opened", description="'opened' (history) or 'modified'")
# ── D. Mutations ───────────────────────────────────────────────────────────
class CreateFileInput(BaseModel):
"""Create a new text file in a vault."""
vault: str = Field(..., description="Vault name")
path: str = Field(..., description="Vault-relative path of the new file")
content: str = Field("", description="Initial file content")
class CreateDirectoryInput(BaseModel):
"""Create a new directory in a vault."""
vault: str = Field(..., description="Vault name")
path: str = Field(..., description="Vault-relative path of the new directory")
class EditFileInput(BaseModel):
"""Overwrite the full content of an existing file."""
vault: str = Field(..., description="Vault name")
path: str = Field(..., description="Vault-relative file path")
content: str = Field(..., description="New full content of the file")
class AppendToFileInput(BaseModel):
"""Append text to the end of an existing file."""
vault: str = Field(..., description="Vault name")
path: str = Field(..., description="Vault-relative file path")
content: str = Field(..., description="Text to append")
class RenameFileInput(BaseModel):
"""Rename a file in place (same parent directory)."""
vault: str = Field(..., description="Vault name")
path: str = Field(..., description="Current vault-relative file path")
new_name: str = Field(..., description="New file name (no directory separator)")
class RenameDirectoryInput(BaseModel):
"""Rename a directory in place (same parent directory)."""
vault: str = Field(..., description="Vault name")
path: str = Field(..., description="Current vault-relative directory path")
new_name: str = Field(..., description="New directory name (no directory separator)")
class MovePathInput(BaseModel):
"""Move a file or directory to another directory within the same vault."""
vault: str = Field(..., description="Vault name")
source_path: str = Field(..., description="Current vault-relative path of the file/directory")
destination_dir: str = Field("", description="Target directory path (empty = vault root)")
class ReplaceInFilesInput(BaseModel):
"""Find and replace text across vault files (destructive, dry-run by default)."""
find: str = Field(..., min_length=1, description="Text or pattern to search for")
replace: str = Field("", description="Replacement text")
vault: str = Field("all", description="Vault name or 'all'")
case_sensitive: bool = Field(False, description="Match case")
whole_word: bool = Field(False, description="Match whole words only")
regex: bool = Field(False, description="Treat 'find' as a regular expression")
include_paths: str | None = Field(None, description="Comma-separated glob patterns to include")
exclude_paths: str | None = Field(None, description="Comma-separated glob patterns to exclude")
replace_all: bool = Field(False, description="Apply the replacement (otherwise only preview)")
dry_run: bool | None = Field(
None,
description="Preview matches without writing. Defaults to the opposite of replace_all.",
)
class DeleteFileInput(BaseModel):
"""Delete a file from a vault (destructive)."""
vault: str = Field(..., description="Vault name")
path: str = Field(..., description="Vault-relative file path")
class DeleteDirectoryInput(BaseModel):
"""Delete a directory from a vault (destructive)."""
vault: str = Field(..., description="Vault name")
path: str = Field(..., description="Vault-relative directory path")
recursive: bool = Field(True, description="Delete non-empty directories recursively")
class RestoreBackupInput(BaseModel):
"""Restore a file from one of its backup versions."""
vault: str = Field(..., description="Vault name")
path: str = Field(..., description="Vault-relative file path")
version: int = Field(..., description="Backup timestamp to restore")
class WebSearchInput(BaseModel):
"""Search the public web through the configured meta-search engine."""
query: str = Field(..., description="Search terms")
max_results: int = Field(5, ge=1, le=10, description="Number of results to return")
category: str = Field("", description="Optional engine category: general, news, it, science")
language: str = Field("", description="Optional language code, e.g. 'fr'")
page: int = Field(1, ge=1, le=10, description="Result page number")
class FetchUrlInput(BaseModel):
"""Fetch one public web page and return its readable text."""
url: str = Field(..., description="Absolute http(s) URL of a public page")
class ToolResult(BaseModel):
"""Uniform result returned by :func:`backend.tools.registry.call_tool`."""
ok: bool = True
data: Any = None
error: str | None = None
+508
View File
@@ -0,0 +1,508 @@
"""Built-in tool services (Phase 0 + Phase C read/search + Phase D mutations).
These functions are the single source of truth consumed by both the in-app
assistant and the MCP server. They delegate to the shared business-logic
services (``backend.services``) so routes and tools never diverge.
Mutating tools (create/edit/rename/move/delete/restore) are registered with
``ToolRisk.WRITE`` or ``ToolRisk.DANGEROUS`` and gated by the registry's
confirmation mechanism (two-step propose/apply).
"""
from __future__ import annotations
import logging
import os
from typing import Any
from backend.indexer import get_backlinks as _get_backlinks
from backend.indexer import get_vault_names
from backend.services.backups import diff_backup as _diff_backup
from backend.services.backups import list_backup_files as _list_backup_files
from backend.services.files import read_file_text
from backend.services.graph import get_graph as _get_graph
from backend.services.mutations import (
append_to_file as _append_to_file,
)
from backend.services.mutations import (
create_directory as _create_directory,
)
from backend.services.mutations import (
create_file as _create_file,
)
from backend.services.mutations import (
delete_directory as _delete_directory,
)
from backend.services.mutations import (
delete_file as _delete_file,
)
from backend.services.mutations import (
edit_file as _edit_file,
)
from backend.services.mutations import (
move_path as _move_path,
)
from backend.services.mutations import (
rename_directory as _rename_directory,
)
from backend.services.mutations import (
rename_file as _rename_file,
)
from backend.services.mutations import (
replace_in_files as _replace_in_files,
)
from backend.services.mutations import (
restore_backup as _restore_backup,
)
from backend.services.recent import list_recent as _list_recent
from backend.services.search import advanced_search_vaults, search_vaults
from backend.services.search import list_tags as _list_tags
from backend.services.search import search_paths as _search_paths
from backend.services.vaults import browse_directory, list_accessible_vaults, list_all_files
from backend.tools.context import ToolContext, ToolRisk
from backend.tools.registry import tool
from backend.tools.schemas import (
AppendToFileInput,
CreateDirectoryInput,
CreateFileInput,
DeleteDirectoryInput,
DeleteFileInput,
DiffBackupInput,
EditFileInput,
GetBacklinksInput,
GetGraphInput,
ListAllFilesInput,
ListBackupsInput,
ListDirectoryInput,
ListRecentInput,
ListTagsInput,
ListVaultsInput,
MovePathInput,
ReadFileInput,
ReadFileRawInput,
RenameDirectoryInput,
RenameFileInput,
ReplaceInFilesInput,
RestoreBackupInput,
SearchAdvancedInput,
SearchFulltextInput,
SearchPathsInput,
SuggestTagsInput,
)
logger = logging.getLogger("obsigate.tools.service")
# Maximum file size returned by ``read_file`` (bytes). Quota configurable via
# ``BOOKSLM_MAX_TOOL_READ_BYTES``.
TOOL_MAX_READ_BYTES = int(os.environ.get("BOOKSLM_MAX_TOOL_READ_BYTES", "200000"))
# ── C1. Vaults / navigation ────────────────────────────────────────────────
@tool(
name="list_vaults",
description="List the vaults the current user is allowed to access.",
input_model=ListVaultsInput,
risk=ToolRisk.READ,
)
def list_vaults(ctx: ToolContext, _params: ListVaultsInput) -> list[dict[str, Any]]:
"""Return accessible vaults with a file count."""
return [
{"name": v["name"], "file_count": v["file_count"]}
for v in list_accessible_vaults(ctx.user)
]
@tool(
name="list_directory",
description="List files and subdirectories of a directory inside a vault.",
input_model=ListDirectoryInput,
risk=ToolRisk.READ,
requires_vault=True,
)
def list_directory(ctx: ToolContext, params: ListDirectoryInput) -> list[dict[str, Any]]:
"""Return the entries of a vault directory (direct children only)."""
data = browse_directory(params.vault, params.path)
return [
{"name": item["name"], "path": item["path"], "type": item["type"]}
for item in data["items"]
]
@tool(
name="list_all_files",
description="List every file of a vault (optionally under a subdirectory), newest first.",
input_model=ListAllFilesInput,
risk=ToolRisk.READ,
requires_vault=True,
)
def list_all_files_tool(ctx: ToolContext, params: ListAllFilesInput) -> dict[str, Any]:
"""Return a flat list of files with metadata."""
return list_all_files(params.vault, dir=params.dir, limit=params.limit, recursive=params.recursive)
# ── C2. Content reading ────────────────────────────────────────────────────
@tool(
name="read_file",
description="Read the text content of a file inside a vault (secrets redacted).",
input_model=ReadFileInput,
risk=ToolRisk.READ,
requires_vault=True,
)
def read_file(ctx: ToolContext, params: ReadFileInput) -> dict[str, Any]:
"""Return the (redacted) content of a vault file."""
return read_file_text(
params.vault,
params.path,
redact=True,
max_bytes=TOOL_MAX_READ_BYTES,
)
@tool(
name="read_file_raw",
description="Read a file's raw text content, without the size cap (secrets redacted).",
input_model=ReadFileRawInput,
risk=ToolRisk.READ,
requires_vault=True,
)
def read_file_raw(ctx: ToolContext, params: ReadFileRawInput) -> dict[str, Any]:
"""Return the full (redacted) raw text of a vault file."""
data = read_file_text(params.vault, params.path, redact=True, max_bytes=None)
return {"vault": data["vault"], "path": data["path"], "raw": data["content"]}
@tool(
name="get_backlinks",
description="List files that link to a target file via [[wikilinks]].",
input_model=GetBacklinksInput,
risk=ToolRisk.READ,
requires_vault=True,
)
def get_backlinks(ctx: ToolContext, params: GetBacklinksInput) -> list[dict[str, Any]]:
"""Return backlinks, filtered to accessible vaults."""
backlinks = _get_backlinks(params.vault, params.path)
return [b for b in backlinks if ctx.has_vault_access(b["vault"])]
@tool(
name="list_backups",
description="List the available backup versions of a file (newest first).",
input_model=ListBackupsInput,
risk=ToolRisk.READ,
requires_vault=True,
)
def list_backups(ctx: ToolContext, params: ListBackupsInput) -> dict[str, Any]:
"""Return the backup versions of a vault file."""
return {
"vault": params.vault,
"path": params.path,
"backups": _list_backup_files(params.vault, params.path),
}
@tool(
name="diff_backup",
description="Show a unified diff between a backup version and another version or the current file.",
input_model=DiffBackupInput,
risk=ToolRisk.READ,
requires_vault=True,
)
def diff_backup(ctx: ToolContext, params: DiffBackupInput) -> dict[str, Any]:
"""Return the unified diff for a backup version."""
return _diff_backup(params.vault, params.path, params.version, params.compare_with)
@tool(
name="get_graph",
description="Return the graph (nodes and edges: parent/child + wikilinks) of a vault or directory.",
input_model=GetGraphInput,
risk=ToolRisk.READ,
requires_vault=True,
)
def get_graph(ctx: ToolContext, params: GetGraphInput) -> dict[str, Any]:
"""Return graph data for a vault subtree or the whole vault."""
return _get_graph(
params.vault,
path=params.path,
depth=params.depth,
scope=params.scope,
tag=params.tag,
)
# ── C3. Search ─────────────────────────────────────────────────────────────
@tool(
name="search_fulltext",
description="Full-text search across one vault or all accessible vaults.",
input_model=SearchFulltextInput,
risk=ToolRisk.READ,
)
def search_fulltext(ctx: ToolContext, params: SearchFulltextInput) -> list[dict[str, Any]]:
"""Return ranked search results, filtered to accessible vaults."""
payload = search_vaults(params.q, vault=params.vault, tag=params.tag, limit=params.limit)
return [r for r in payload["results"] if ctx.has_vault_access(r["vault"])]
@tool(
name="search_advanced",
description="Advanced full-text search with operators, filters and facets.",
input_model=SearchAdvancedInput,
risk=ToolRisk.READ,
)
def search_advanced(ctx: ToolContext, params: SearchAdvancedInput) -> list[dict[str, Any]]:
"""Return advanced search results, filtered to accessible vaults."""
payload = advanced_search_vaults(
params.q,
vault=params.vault,
tag=params.tag,
limit=params.limit,
offset=params.offset,
sort=params.sort,
case_sensitive=params.case_sensitive,
whole_word=params.whole_word,
regex=params.regex,
include_paths=params.include_paths,
exclude_paths=params.exclude_paths,
created=params.created,
modified=params.modified,
size=params.size,
)
return [r for r in payload["results"] if ctx.has_vault_access(r["vault"])]
@tool(
name="search_paths",
description="Search files and directories by path substring.",
input_model=SearchPathsInput,
risk=ToolRisk.READ,
)
def search_paths(ctx: ToolContext, params: SearchPathsInput) -> list[dict[str, Any]]:
"""Return path matches, filtered to accessible vaults."""
payload = _search_paths(params.q, vault=params.vault)
return [r for r in payload["results"] if ctx.has_vault_access(r["vault"])]
@tool(
name="list_tags",
description="List tags with their occurrence counts for a vault or all accessible vaults.",
input_model=ListTagsInput,
risk=ToolRisk.READ,
)
def list_tags(ctx: ToolContext, params: ListTagsInput) -> list[dict[str, Any]]:
"""Return tags sorted by descending count."""
if params.vault and params.vault != "all":
ctx.require_vault_access(params.vault)
return [{"tag": tag, "count": count} for tag, count in _list_tags(params.vault).items()]
merged: dict[str, int] = {}
for name in get_vault_names():
if not ctx.has_vault_access(name):
continue
for tag, count in _list_tags(name).items():
merged[tag] = merged.get(tag, 0) + count
return [{"tag": tag, "count": count} for tag, count in sorted(merged.items(), key=lambda x: -x[1])]
@tool(
name="suggest_tags",
description="Suggest tags matching a prefix, across accessible vaults.",
input_model=SuggestTagsInput,
risk=ToolRisk.READ,
)
def suggest_tags(ctx: ToolContext, params: SuggestTagsInput) -> list[dict[str, Any]]:
"""Return tag suggestions, restricted to accessible vaults."""
from backend.search import suggest_tags as _suggest
if params.vault and params.vault != "all":
ctx.require_vault_access(params.vault)
return _suggest(params.q, vault_filter=params.vault, limit=params.limit)
merged: dict[str, int] = {}
for name in get_vault_names():
if not ctx.has_vault_access(name):
continue
for item in _suggest(params.q, vault_filter=name, limit=params.limit):
merged[item["tag"]] = merged.get(item["tag"], 0) + item["count"]
ordered = sorted(merged.items(), key=lambda x: -x[1])
return [{"tag": tag, "count": count} for tag, count in ordered[: params.limit]]
@tool(
name="list_recent",
description="List the current user's recently opened (or modified) files.",
input_model=ListRecentInput,
risk=ToolRisk.READ,
)
def list_recent(ctx: ToolContext, params: ListRecentInput) -> dict[str, Any]:
"""Return recent files for the calling user."""
user_vaults = ctx.user.get("_token_vaults") or ctx.user.get("vaults", [])
return _list_recent(
ctx.username,
user_vaults,
vault=params.vault,
limit=params.limit,
mode=params.mode,
)
# ── D. Mutations ───────────────────────────────────────────────────────────
@tool(
name="create_file",
description="Create a new text file in a vault with optional initial content.",
input_model=CreateFileInput,
risk=ToolRisk.WRITE,
requires_vault=True,
)
def create_file(ctx: ToolContext, params: CreateFileInput) -> dict[str, Any]:
"""Create a vault file (fails if it already exists)."""
return _create_file(params.vault, params.path, params.content)
@tool(
name="create_directory",
description="Create a new directory (and parents) in a vault.",
input_model=CreateDirectoryInput,
risk=ToolRisk.WRITE,
requires_vault=True,
)
def create_directory(ctx: ToolContext, params: CreateDirectoryInput) -> dict[str, Any]:
"""Create a vault directory."""
return _create_directory(params.vault, params.path)
@tool(
name="edit_file",
description="Overwrite the full content of an existing file (automatic backup).",
input_model=EditFileInput,
risk=ToolRisk.WRITE,
requires_vault=True,
)
def edit_file(ctx: ToolContext, params: EditFileInput) -> dict[str, Any]:
"""Replace a vault file's content."""
return _edit_file(params.vault, params.path, params.content)
@tool(
name="append_to_file",
description="Append text to the end of an existing file (automatic backup).",
input_model=AppendToFileInput,
risk=ToolRisk.WRITE,
requires_vault=True,
)
def append_to_file(ctx: ToolContext, params: AppendToFileInput) -> dict[str, Any]:
"""Append content to a vault file."""
return _append_to_file(params.vault, params.path, params.content)
@tool(
name="rename_file",
description="Rename a file in place (same parent directory). Destructive: requires confirmation.",
input_model=RenameFileInput,
risk=ToolRisk.DANGEROUS,
requires_vault=True,
)
def rename_file(ctx: ToolContext, params: RenameFileInput) -> dict[str, Any]:
"""Rename a vault file."""
return _rename_file(params.vault, params.path, params.new_name)
@tool(
name="rename_directory",
description="Rename a directory in place (same parent directory). Destructive: requires confirmation.",
input_model=RenameDirectoryInput,
risk=ToolRisk.DANGEROUS,
requires_vault=True,
)
def rename_directory(ctx: ToolContext, params: RenameDirectoryInput) -> dict[str, Any]:
"""Rename a vault directory."""
return _rename_directory(params.vault, params.path, params.new_name)
@tool(
name="move_path",
description="Move a file or directory to another directory in the same vault. Destructive: requires confirmation.",
input_model=MovePathInput,
risk=ToolRisk.DANGEROUS,
requires_vault=True,
)
def move_path(ctx: ToolContext, params: MovePathInput) -> dict[str, Any]:
"""Move a vault file or directory."""
return _move_path(params.vault, params.source_path, params.destination_dir)
@tool(
name="replace_in_files",
description=(
"Find and replace text across vault files. Previews by default "
"(dry_run); set replace_all=true to apply. Destructive: requires confirmation."
),
input_model=ReplaceInFilesInput,
risk=ToolRisk.DANGEROUS,
)
def replace_in_files(ctx: ToolContext, params: ReplaceInFilesInput) -> dict[str, Any]:
"""Preview or apply a find/replace, filtered to permitted vaults."""
if params.vault and params.vault != "all":
ctx.require_vault_access(params.vault)
ctx.require_destructive_allowed(params.vault)
dry_run = params.dry_run if params.dry_run is not None else not params.replace_all
def _allowed(vault: str) -> bool:
return ctx.has_vault_access(vault) and ctx.destructive_tools_enabled(vault)
return _replace_in_files(
params.find,
params.replace,
vault=params.vault,
case_sensitive=params.case_sensitive,
whole_word=params.whole_word,
regex=params.regex,
include_paths=params.include_paths,
exclude_paths=params.exclude_paths,
replace_all=params.replace_all,
dry_run=dry_run,
is_vault_allowed=_allowed,
)
@tool(
name="delete_file",
description="Delete a file from a vault (automatic backup). Destructive: requires confirmation.",
input_model=DeleteFileInput,
risk=ToolRisk.DANGEROUS,
requires_vault=True,
)
def delete_file(ctx: ToolContext, params: DeleteFileInput) -> dict[str, Any]:
"""Delete a vault file."""
return _delete_file(params.vault, params.path)
@tool(
name="delete_directory",
description="Delete a directory (recursive by default) from a vault. Destructive: requires confirmation.",
input_model=DeleteDirectoryInput,
risk=ToolRisk.DANGEROUS,
requires_vault=True,
)
def delete_directory(ctx: ToolContext, params: DeleteDirectoryInput) -> dict[str, Any]:
"""Delete a vault directory."""
return _delete_directory(params.vault, params.path, recursive=params.recursive)
@tool(
name="restore_backup",
description="Restore a file from one of its backup versions (current version backed up first).",
input_model=RestoreBackupInput,
risk=ToolRisk.WRITE,
requires_vault=True,
)
def restore_backup(ctx: ToolContext, params: RestoreBackupInput) -> dict[str, Any]:
"""Restore a vault file from a backup."""
return _restore_backup(params.vault, params.path, params.version)
+234
View File
@@ -0,0 +1,234 @@
"""Web tools for the assistant (Notion-style "research" capabilities).
Phase 1 of the documented web-toolset roadmap:
* ``web_search`` — query the self-hosted SearXNG instance (no API key).
* ``fetch_url`` — retrieve a public web page and return readable text.
Both are READ-risk tools (no confirmation), rate-limited through the shared
registry, SSRF-guarded (scheme + private-address rejection), and size-capped.
Configuration (environment):
* ``OBSIGATE_SEARXNG_URL`` — defaults to https://search.dracodev.net
* ``OBSIGATE_WEB_TIMEOUT`` — seconds, default 10
"""
from __future__ import annotations
import html as html_lib
import ipaddress
import logging
import os
import re
import socket
from typing import Any
from urllib.parse import urlparse
import httpx
from backend.tools.context import ToolError, ToolRisk, ToolScope
from backend.tools.registry import tool
from backend.tools.schemas import FetchUrlInput, WebSearchInput
logger = logging.getLogger("obsigate.tools.web")
SEARXNG_URL = os.environ.get("OBSIGATE_SEARXNG_URL", "https://search.dracodev.net")
WEB_TIMEOUT = float(os.environ.get("OBSIGATE_WEB_TIMEOUT", "10"))
USER_AGENT = "ObsiGateAssistant/1.0 (+self-hosted vault AI)"
MAX_FETCH_BYTES = 1_500_000
MAX_TEXT_CHARS = 20_000
_BLOCKED_TAGS_RE = re.compile(
r"<(script|style|noscript|template|svg)\b.*?</\1>", re.IGNORECASE | re.DOTALL
)
_TAG_RE = re.compile(r"<[^>]+>")
_BLOCK_SPLIT_RE = re.compile(
r"</?(?:p|div|br|li|h[1-6]|tr|table|ul|ol|section|article|header|footer)\b[^>]*>",
re.IGNORECASE,
)
class SSRFError(ToolError):
"""Raised for a URL whose host is private/loopback or scheme unsupported."""
def _assert_public_http_url(url: str) -> str:
"""Reject non-http(s) schemes and private/loopback/link-local targets."""
try:
parsed = urlparse(url)
except ValueError as e:
raise SSRFError("URL invalide", code="invalid_url") from e
if parsed.scheme not in ("http", "https"):
raise SSRFError("Seuls les schémas http/https sont autorisés", code="invalid_scheme")
host = parsed.hostname
if not host:
raise SSRFError("URL sans hôte", code="invalid_url")
# Resolve the host so DNS-rebinding to internal IPs is also caught.
try:
infos = socket.getaddrinfo(host, None)
except socket.gaierror as e:
raise SSRFError(f"Hôte introuvable: {host}", code="dns_error") from e
for info in infos:
ip = ipaddress.ip_address(info[4][0])
if (
ip.is_private
or ip.is_loopback
or ip.is_link_local
or ip.is_reserved
or ip.is_multicast
or ip.is_unspecified
):
raise SSRFError("Accès aux adresses internes interdit", code="ssrf_blocked")
return url
def _html_to_text(raw: str) -> str:
"""Cheap HTML → readable text: strip scripts/styles, tags, then compress.
Comments (which may carry script-like payloads) are removed first.
"""
text = re.sub(r"<!--.*?-->", " ", raw, flags=re.DOTALL)
text = _BLOCKED_TAGS_RE.sub(" ", text)
# Keep block boundaries as newlines before dropping the remaining tags.
text = _BLOCK_SPLIT_RE.sub("\n", text)
text = _TAG_RE.sub("", text)
text = html_lib.unescape(text)
text = re.sub(r"[ \t]+", " ", text)
text = re.sub(r" ?\n ?", "\n", text)
text = re.sub(r"\n{3,}", "\n\n", text)
return text.strip()
@tool(
name="web_search",
description=(
"Search the public web for current information and return ranked results "
"(title, url, snippet). Use for facts outside the vault: weather, news, "
"documentation, versions, prices, anything that needs live sources."
),
input_model=WebSearchInput,
risk=ToolRisk.READ,
scopes=(ToolScope.IN_APP,),
)
def web_search(ctx, params: WebSearchInput) -> dict[str, Any]:
"""Query the self-hosted SearXNG instance and return trimmed results."""
query = params.query.strip()
if not query:
raise ToolError("Requête vide", code="invalid_arguments")
url = SEARXNG_URL.rstrip("/") + "/search"
try:
resp = httpx.get(
url,
params={
"q": query,
"format": "json",
"categories": params.category or "general",
"pageno": max(1, params.page),
**({"language": params.language} if params.language else {}),
"safesearch": "1",
},
headers={"User-Agent": USER_AGENT},
timeout=WEB_TIMEOUT,
follow_redirects=False,
)
resp.raise_for_status()
data = resp.json()
except httpx.HTTPError as e:
logger.warning("web_search failed: %s", e)
raise ToolError(
"Le moteur de recherche web est momentanément indisponible.",
code="web_search_unavailable",
) from e
results: list[dict[str, Any]] = []
for item in (data.get("results") or [])[: params.max_results]:
results.append(
{
"title": (item.get("title") or "")[:300],
"url": item.get("url") or "",
"snippet": (item.get("content") or "")[:600],
"published": item.get("publishedDate"),
"score": item.get("score"),
}
)
unresponsive = [
name for entry in (data.get("unresponsive_engines") or [])
for name in ([entry[0]] if isinstance(entry, (list, tuple)) and entry else [entry])
if isinstance(name, str)
]
payload: dict[str, Any] = {
"query": query,
"engine": "searxng",
"results": results,
"count": len(results),
}
if unresponsive:
payload["unresponsive_engines"] = unresponsive[:8]
if not results:
# An instance whose upstream engines are all blocked (CAPTCHA / rate
# limit) answers 200 with an empty list. Without an explicit hint the
# model retries the same search until it burns its tool quota.
payload["warning"] = (
"Aucun résultat : les moteurs de recherche de l'instance SearXNG sont "
f"indisponibles ({', '.join(unresponsive[:5]) or 'inconnus'}). "
"Ne relance pas la même recherche — dis-le à l'utilisateur."
)
return payload
@tool(
name="fetch_url",
description=(
"Fetch a public web page (http/https) and return its readable text. "
"Use after web_search to read a promising result in detail. HTML is "
"converted to plain text; binary pages are rejected."
),
input_model=FetchUrlInput,
risk=ToolRisk.READ,
scopes=(ToolScope.IN_APP,),
)
def fetch_url(ctx, params: FetchUrlInput) -> dict[str, Any]:
"""Retrieve one page, guard against SSRF, and extract its text."""
url = _assert_public_http_url(params.url.strip())
try:
# Follow redirects manually so every hop is re-checked against the
# private-address SSRF guard (a public page can redirect to 127.0.0.1).
resp = None
for _hop in range(5):
resp = httpx.get(
url,
headers={"User-Agent": USER_AGENT, "Accept": "text/html,application/xhtml+xml,*/*"},
timeout=WEB_TIMEOUT,
follow_redirects=False,
)
if resp.status_code in (301, 302, 303, 307, 308):
location = resp.headers.get("location") or ""
if not location:
break
url = str(httpx.URL(url).join(location))
url = _assert_public_http_url(url)
continue
break
assert resp is not None
resp.raise_for_status()
except SSRFError:
raise
except httpx.HTTPError as e:
logger.warning("fetch_url failed for %s: %s", url, e)
raise ToolError("Impossible de récupérer la page.", code="fetch_unavailable") from e
ctype = (resp.headers.get("content-type") or "").lower()
if not any(t in ctype for t in ("html", "xml", "text", "json", "markdown")):
raise ToolError(
f"Type de contenu non pris en charge: {ctype.split(';')[0] or 'inconnu'}",
code="unsupported_content_type",
)
raw = (resp.content[:MAX_FETCH_BYTES]).decode(resp.encoding or "utf-8", errors="replace")
title_match = re.search(r"<title[^>]*>(.*?)</title>", raw, re.IGNORECASE | re.DOTALL)
title = html_lib.unescape(title_match.group(1)).strip()[:300] if title_match else ""
text = _html_to_text(raw)[:MAX_TEXT_CHARS]
return {
"url": str(resp.url),
"status": resp.status_code,
"title": title,
"text": text,
"truncated": len(raw) > MAX_TEXT_CHARS,
}
+51 -21
View File
@@ -1,23 +1,34 @@
"""
Version management for ObsiGate.
The canonical version is the numeric SemVer of the LATEST release tag
(MAJOR.MINOR.PATCH). get_version() always returns a clean "x.y.z" string so
the UI never shows "-dev", "0.0.0-dev" or a "-N-gHASH" suffix — the same
number appears in the header badge, the About modal and the API health.
Source unique de vérité : le fichier `VERSION` à la racine du dépôt, au format
`MAJEUR.MINEUR.CORRECTIF` (incrémenté à chaque livraison par
`scripts/bump_version.py`, hook git `commit-msg`). get_version() retourne
toujours une chaîne propre "x.y.z" — jamais "-dev", "0.0.0-dev" ou "-N-gHASH" :
le même numéro s'affiche dans le badge d'en-tête, la boîte À propos et /api/health.
Examples:
tag v2.0.0 -> "2.0.0"
HEAD 31 commits after -> "2.0.0" (release version, no dev clutter)
no git / no VERSION -> "0.0.0"
Les sources sont consultées dans cet ordre :
1. variable d'environnement OBSIGATE_VERSION (surcharge explicite / tests) ;
2. `VERSION` à la racine du dépôt (source unique, copiée dans l'image Docker) ;
3. `backend/VERSION` (ancien emplacement, encore produit par certains builds) ;
4. dernier tag git (`git describe --tags --abbrev=0`) ;
5. "0.0.0".
Exemples :
VERSION = 2.3.0 -> "2.3.0"
tag v2.3.0, 4 commits -> "2.3.0"
aucun VERSION / git -> "0.0.0"
"""
from __future__ import annotations
import os
import subprocess
from pathlib import Path
_ROOT = Path(__file__).resolve().parent.parent # ObsiGate repo root
_VERSION_FILE = Path(__file__).resolve().parent / "VERSION"
_ROOT = Path(__file__).resolve().parent.parent # racine du dépôt ObsiGate
VERSION_FILE = _ROOT / "VERSION" # source unique de vérité
LEGACY_VERSION_FILE = Path(__file__).resolve().parent / "VERSION" # backend/VERSION
_ENV_VAR = "OBSIGATE_VERSION"
def _run_git(args: list[str]) -> str:
@@ -57,6 +68,16 @@ def _clean_base(raw: str) -> str:
return ".".join(nums)
def _read_version_file(path: Path) -> str:
"""Clean base of a VERSION file, or '' when absent/unreadable/invalid."""
try:
if not path.exists():
return ""
return _clean_base(path.read_text(encoding="utf-8", errors="replace"))
except OSError:
return ""
def get_git_describe() -> str:
"""Full `git describe` string (e.g. "2.0.0-31-gabc1234") or '' if no git."""
return _run_git(["describe", "--tags", "--dirty=-dirty"]).lstrip("v")
@@ -68,23 +89,32 @@ def get_git_commit() -> str:
def get_version() -> str:
"""Return the clean release version x.y.z (latest tag) — never a -suffix.
"""Return the clean release version x.y.z — never a -suffix.
Priority: latest git tag -> backend/VERSION file -> "0.0.0".
Priority: OBSIGATE_VERSION -> ./VERSION -> backend/VERSION -> latest git tag
-> "0.0.0".
"""
# 1) Latest tag from git (works even with commits beyond the tag)
tag = _run_git(["describe", "--tags", "--abbrev=0"])
base = _clean_base(tag)
# 1) Explicit override (docker-compose, tests, builds hors dépôt)
override = _clean_base(os.environ.get(_ENV_VAR, ""))
if override:
return override
# 2) Source unique de vérité : VERSION à la racine du dépôt
base = _read_version_file(VERSION_FILE)
if base:
return base
# 2) backend/VERSION file (baked at build time by build.sh / CI / Docker)
if _VERSION_FILE.exists():
base = _clean_base(_VERSION_FILE.read_text(encoding="utf-8"))
if base:
return base
# 3) Ancien emplacement (backend/VERSION, baké par certains builds)
base = _read_version_file(LEGACY_VERSION_FILE)
if base:
return base
# 3) Nothing available
# 4) Dernier tag git
base = _clean_base(_run_git(["describe", "--tags", "--abbrev=0"]))
if base:
return base
# 5) Rien d'exploitable
return "0.0.0"
+172 -13
View File
@@ -4,6 +4,16 @@ Webhook management and dispatch for ObsiGate.
Webhooks are HTTP POST callbacks triggered on file/directory events.
Configuration is persisted in data/webhooks.json.
Security (BUG-026):
* target URLs are validated against SSRF (scheme + resolved IP must be public);
* HTTP is refused unless ``OBSIGATE_WEBHOOK_ALLOW_HTTP=true``;
* private/loopback/link-local targets are refused unless
``OBSIGATE_WEBHOOK_ALLOW_PRIVATE=true``;
* redirects are never followed;
* signing secrets are **not** stored in the public config file — they live in
``data/webhook_secrets.json`` (0600) or in an environment variable named
``OBSIGATE_WEBHOOK_SECRET_<ID>``.
Events: file_created, file_deleted, file_modified, file_renamed,
directory_created, directory_deleted, directory_renamed
"""
@@ -11,17 +21,22 @@ Events: file_created, file_deleted, file_modified, file_renamed,
import asyncio
import hashlib
import hmac
import ipaddress
import json
import logging
import os
import socket
import uuid
from datetime import datetime, timezone
from pathlib import Path
from urllib.parse import urlparse
import aiohttp
logger = logging.getLogger("obsigate.webhooks")
WEBHOOKS_FILE = Path("data/webhooks.json")
WEBHOOK_SECRETS_FILE = Path("data/webhook_secrets.json")
VALID_EVENTS = {
"file_created", "file_deleted", "file_modified", "file_renamed",
@@ -29,6 +44,81 @@ VALID_EVENTS = {
}
def _allow_http() -> bool:
return os.environ.get("OBSIGATE_WEBHOOK_ALLOW_HTTP", "false").lower() == "true"
def _allow_private() -> bool:
return os.environ.get("OBSIGATE_WEBHOOK_ALLOW_PRIVATE", "false").lower() == "true"
def _is_public_ip(ip_str: str) -> bool:
"""True when *ip_str* is a globally routable unicast address."""
try:
ip = ipaddress.ip_address(ip_str)
except ValueError:
return False
return not (
ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved
or ip.is_multicast or ip.is_unspecified
)
def validate_webhook_url(url: str) -> str:
"""Validate the URL syntax/scheme and literal-IP safety at config time.
Raises:
ValueError: When the URL is malformed or points at an obviously
forbidden scheme/host.
"""
if not url or not isinstance(url, str):
raise ValueError("URL requise")
parsed = urlparse(url)
if parsed.scheme not in ("http", "https"):
raise ValueError("L'URL doit utiliser http ou https")
if parsed.scheme == "http" and not _allow_http():
raise ValueError("HTTPS requis (définir OBSIGATE_WEBHOOK_ALLOW_HTTP=true pour autoriser http)")
host = parsed.hostname
if not host:
raise ValueError("Hôte manquant dans l'URL")
# Reject literal private/loopback IPs immediately (no DNS needed).
try:
ip = ipaddress.ip_address(host)
except ValueError:
return url # hostname — resolved and checked at dispatch time
if not _allow_private() and not _is_public_ip(str(ip)):
raise ValueError("Adresse privée/interne refusée")
return url
def is_safe_target(url: str) -> bool:
"""Full SSRF check performed right before dispatch (resolves the host).
Returns False when the URL is malformed, the scheme is forbidden, or any
resolved address is private/loopback/reserved.
"""
try:
validate_webhook_url(url)
except ValueError:
return False
if _allow_private():
return True
parsed = urlparse(url)
host = parsed.hostname or ""
port = parsed.port or (443 if parsed.scheme == "https" else 80)
try:
infos = socket.getaddrinfo(host, port, proto=socket.IPPROTO_TCP)
except socket.gaierror:
logger.warning(f"Webhook target host could not be resolved: {host}")
return False
for info in infos:
addr = str(info[4][0])
if not _is_public_ip(addr):
logger.warning(f"Webhook target resolves to a non-public address ({addr}); blocked")
return False
return True
def _read() -> list:
if not WEBHOOKS_FILE.exists():
return []
@@ -45,35 +135,94 @@ def _write(webhooks: list):
tmp.replace(WEBHOOKS_FILE)
def _read_secrets() -> dict:
if not WEBHOOK_SECRETS_FILE.exists():
return {}
try:
return json.loads(WEBHOOK_SECRETS_FILE.read_text(encoding="utf-8"))
except (json.JSONDecodeError, OSError):
return {}
def _write_secrets(secrets: dict):
WEBHOOK_SECRETS_FILE.parent.mkdir(parents=True, exist_ok=True)
tmp = WEBHOOK_SECRETS_FILE.with_suffix(".tmp")
tmp.write_text(json.dumps(secrets, indent=2), encoding="utf-8")
tmp.replace(WEBHOOK_SECRETS_FILE)
try:
WEBHOOK_SECRETS_FILE.chmod(0o600)
except OSError:
pass # Windows doesn't support Unix permissions
def _store_secret(wh_id: str, secret: str | None) -> None:
secrets = _read_secrets()
if secret:
secrets[wh_id] = secret
else:
secrets.pop(wh_id, None)
_write_secrets(secrets)
def _get_secret(wh: dict) -> str | None:
"""Resolve a webhook secret from env, dedicated store, or legacy record."""
env_key = "OBSIGATE_WEBHOOK_SECRET_" + wh["id"].replace("-", "_").upper()
env_val = os.environ.get(env_key)
if env_val:
return env_val
stored = _read_secrets().get(wh["id"])
if stored:
return stored
return wh.get("secret") # legacy inline secret
def _public_view(wh: dict) -> dict:
"""Return a webhook record safe to expose through the API."""
clean = {k: v for k, v in wh.items() if k != "secret"}
clean["has_secret"] = bool(_get_secret(wh))
return clean
def get_webhooks() -> list:
return _read()
return [_public_view(wh) for wh in _read()]
def create_webhook(name: str, url: str, events: list[str], secret: str | None = None) -> dict:
validate_webhook_url(url)
webhooks = _read()
wh_id = str(uuid.uuid4())
wh = {
"id": str(uuid.uuid4()),
"id": wh_id,
"name": name,
"url": url,
"events": [e for e in events if e in VALID_EVENTS],
"secret": secret,
"enabled": True,
"created_at": datetime.now(timezone.utc).isoformat(),
"last_fired_at": None,
}
webhooks.append(wh)
_write(webhooks)
if secret:
_store_secret(wh_id, secret)
logger.info(f"Created webhook '{name}' → {url}")
return wh
return _public_view(wh)
def update_webhook(wh_id: str, updates: dict) -> dict | None:
webhooks = _read()
for wh in webhooks:
if wh["id"] == wh_id:
wh.update({k: v for k, v in updates.items() if k != "id"})
if updates.get("url"):
validate_webhook_url(updates["url"])
if "secret" in updates:
_store_secret(wh_id, updates["secret"])
safe_updates = {
k: v for k, v in updates.items()
if k not in ("id", "secret")
}
wh.update(safe_updates)
_write(webhooks)
return wh
return _public_view(wh)
return None
@@ -83,6 +232,9 @@ def delete_webhook(wh_id: str) -> bool:
if len(new_list) == len(webhooks):
return False
_write(new_list)
secrets = _read_secrets()
if secrets.pop(wh_id, None) is not None:
_write_secrets(secrets)
return True
@@ -102,17 +254,24 @@ async def dispatch_webhooks(event_type: str, data: dict):
async def _post(wh):
try:
# BUG-026: re-check the target right before connecting (DNS rebinding).
if not is_safe_target(wh["url"]):
logger.warning(f"Webhook '{wh['name']}' blocked by SSRF policy")
return
headers = {"Content-Type": "application/json", "X-ObsiGate-Event": event_type}
if wh.get("secret"):
sig = hmac.new(wh["secret"].encode(), body.encode(), hashlib.sha256).hexdigest()
secret = _get_secret(wh)
if secret:
sig = hmac.new(secret.encode(), body.encode(), hashlib.sha256).hexdigest()
headers["X-ObsiGate-Signature"] = f"sha256={sig}"
timeout = aiohttp.ClientTimeout(total=5)
async with aiohttp.ClientSession(timeout=timeout) as session, session.post(wh["url"], data=body, headers=headers) as resp:
if resp.status < 400:
logger.debug(f"Webhook '{wh['name']}' OK ({resp.status})")
else:
logger.warning(f"Webhook '{wh['name']}' failed ({resp.status})")
async with aiohttp.ClientSession(timeout=timeout) as session, session.post(
wh["url"], data=body, headers=headers, allow_redirects=False
) as resp:
if resp.status < 400:
logger.debug(f"Webhook '{wh['name']}' OK ({resp.status})")
else:
logger.warning(f"Webhook '{wh['name']}' failed ({resp.status})")
update_webhook(wh["id"], {"last_fired_at": datetime.now(timezone.utc).isoformat()})
except Exception as e:
logger.warning(f"Webhook '{wh['name']}' error: {e}")
+8
View File
@@ -106,6 +106,14 @@ Ce vault est utilisé pour les tests de développement d'ObsiGate.
}
}
# ----- Resolve version from git before Docker build -----
$Version = (git describe --tags --dirty 2>$null) -replace '^v',''
if (-not $Version) { $Version = "0.0.0" }
$Version | Out-File -Encoding ascii -NoNewline -FilePath "backend\VERSION"
# Export so docker compose substitutes ${VERSION} in the build arg
$env:VERSION = $Version
Write-Info "Version : $Version"
# ----- Build the image -----
$BuildArgs = @("-f", $ComposeFile)
if (-not $UseCache) {
+5 -3
View File
@@ -193,9 +193,11 @@ else
info "Construction de l'image Docker (avec cache)..."
fi
# Generate VERSION file from git before Docker build
VERSION=$(git describe --tags --dirty 2>/dev/null | sed 's/^v//' || echo "0.0.0-dev")
echo "$VERSION" > backend/VERSION
# Version livrée : le fichier VERSION (racine du dépôt) est la source unique de
# vérité ; le Dockerfile le copie dans l'image. Aucun numéro codé en dur ici.
VERSION=$(cat VERSION 2>/dev/null | tr -d '[:space:]' | sed 's/^v//' || true)
[ -n "$VERSION" ] || VERSION="0.0.0"
export VERSION
info "Version: $VERSION"
$COMPOSE_CMD -f "$COMPOSE_FILE" build "${BUILD_FLAGS[@]}" "${BUILD_ARGS[@]}"
+5
View File
@@ -1,5 +1,10 @@
# Code Context — ObsiGate Bug Investigation
> **⚠️ Note de workflow (obsolète pour cette investigation)** : la méthode de livraison
> obligatoire du dépôt est définie dans [`AGENTS.md`](./AGENTS.md) et
> [`docs/DELIVERY_WORKFLOW.md`](./docs/DELIVERY_WORKFLOW.md). À appliquer pour toute nouvelle
> tâche ; le contenu ci-dessous est un artefact d'investigation conservé pour historique.
## Files Retrieved
1. `frontend/app.js` (lines 5585–5665) — `showWelcome()` rebuilds dashboard HTML with only bookmarks + recent sections
2. `frontend/index.html` (lines 360–406) — Initial dashboard DOM has all 4 sections: stats, bookmarks, conflicts, recent
+2 -1
View File
@@ -2626,7 +2626,7 @@ dependencies = [
[[package]]
name = "obsigate-desktop"
version = "2.0.0"
version = "2.6.1"
dependencies = [
"chrono",
"env_logger",
@@ -2646,6 +2646,7 @@ dependencies = [
"tauri-plugin-updater",
"tempfile",
"tokio",
"windows",
]
[[package]]
+12 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "obsigate-desktop"
version = "2.0.0"
version = "2.6.1"
description = "ObsiGate Desktop — Porte d'entrée native pour vos vaults Obsidian"
authors = ["Bruno Charest"]
edition = "2021"
@@ -29,6 +29,17 @@ chrono = "0.4"
[dev-dependencies]
tempfile = "3"
[target.'cfg(windows)'.dependencies]
windows = { version = "0.61", features = [
"Win32_Foundation",
"Win32_System_Com",
"Win32_System_Com_StructuredStorage",
"Win32_System_Variant",
"Win32_UI_Shell",
"Win32_UI_Shell_Common",
"Win32_UI_Shell_PropertiesSystem",
] }
[features]
default = ["custom-protocol"]
custom-protocol = ["tauri/custom-protocol"]
+66
View File
@@ -12,6 +12,7 @@ Application desktop native pour [ObsiGate](https://git.dracodev.net/Projets/Obsi
- [Configuration des vaults](#configuration-des-vaults)
- [Architecture](#architecture)
- [Fonctionnalités natives](#fonctionnalites-natives)
- [Signature de code Windows](#signature-de-code-windows)
- [Dépannage](#depannage)
---
@@ -49,6 +50,11 @@ sudo apt install libwebkit2gtk-4.1-0 libgtk-3-0 libayatana-appindicator3-1
:: Mêmes options, inclut le raccourci
```
> **Binaires non signés** : les releases ne sont pas signées avec un certificat
> Windows. Au premier lancement, SmartScreen affiche « Windows a protégé votre
> PC » → cliquer **Informations complémentaires → Exécuter quand même**.
> Voir [Signature de code](#signature-de-code-windows) pour les alternatives.
### macOS
> Non supporté pour le moment (priorité Linux/Windows).
@@ -260,6 +266,66 @@ Arrêt (tray → Quitter ou Ctrl+C) :
---
## Signature de code Windows
La signature de code est **optionnelle** : sans elle, l'application fonctionne,
mais SmartScreen affiche un avertissement au premier lancement. Les binaires
ObsiGate sont actuellement distribués **non signés**.
### Signer localement
Le script `scripts/sign-windows.ps1` signe le binaire et les installeurs après un
`cargo tauri build`. Il lit les identifiants depuis l'environnement (jamais
commités) et est un **no-op explicite** si aucun certificat n'est fourni :
```powershell
$env:OBSIGATE_SIGN_CERT_PFX = "C:\certs\obsigate.pfx"
$env:OBSIGATE_SIGN_CERT_PASSWORD = "..."
$env:OBSIGATE_SIGN_TIMESTAMP_URL = "http://timestamp.digicert.com"
.\scripts\sign-windows.ps1
```
### Alternatives au certificat
| Option | Coût indicatif | Effet SmartScreen |
|---|---|---|
| **Livrer non signé** (statu quo) | 0 € | Avertissement → « Exécuter quand même » |
| **SignPath.io** (projet open source) | Gratuit si éligible OSS | Réputation gérée par le service |
| **Certum Open Source Code Signing** | ~70-100 €/an | Réputation progressive |
| **Certificat OV** | ~150-400 €/an | Avertit tant que la réputation n'est pas établie |
| **Certificat EV** | ~300-700 €/an + token USB/HSM | Réputation **immédiate** |
| **Azure Trusted Signing** | ~10 $/mois | Bonne réputation, signature cloud |
| **Certificat auto-signé** | 0 € | Inutile en distribution publique |
### Signature de l'auto-update (gratuite)
Indépendante de la signature Windows, la signature des mises à jour Tauri repose
sur une paire de clés que vous générez vous-même :
```bash
cargo tauri signer generate -w obsigate-updater.key
```
- La **clé publique** est déjà renseignée dans `plugins.updater.pubkey`
(`tauri.conf.json`).
- La **clé privée** (`desktop/obsigate-updater.key`, gitignorée) est lue
automatiquement par `build-windows.bat` / `build-linux.sh` ; en CI, via les
secrets Gitea `TAURI_SIGNING_PRIVATE_KEY` / `TAURI_SIGNING_PRIVATE_KEY_PASSWORD`.
- Le CLI produit des `.sig` par artefact (`*.exe.sig`, `*.AppImage.sig`, …).
**Manifeste `latest.json`** (détection des mises à jour) :
```powershell
python scripts\updater_manifest.py --tag vX.Y.Z # ou via publish_release.py
```
`publish_release.py` le génère et l'ajoute aux assets ; il reste à **committer
`desktop/latest.json` sur `main`**. L'updater lit ce fichier versionné :
`https://git.dracodev.net/Projets/ObsiGate/raw/branch/main/desktop/latest.json`.
Détail : [DEVELOPMENT_AND_RELEASES §2bis](../docs/DEVELOPMENT_AND_RELEASES.md#2bis-signature-des-mises-à-jour-updater-tauri).
---
## Dépannage
### Le backend ne démarre pas
+11 -1
View File
@@ -38,9 +38,19 @@ cp -r ../backend backend
cp -r ../frontend frontend
echo "✅ Staged"
# ── 2c. Clé de signature des mises à jour (updater Tauri) ──────
if [ -f "obsigate-updater.key" ]; then
export TAURI_SIGNING_PRIVATE_KEY="$(cat obsigate-updater.key)"
SIGN_CONFIG=""
echo "[2c/5] Clé updater trouvée — artefacts .sig activés"
else
SIGN_CONFIG='--config {"bundle":{"createUpdaterArtifacts":false}}'
echo "[2c/5] Clé updater absente — build sans .sig"
fi
# ── 3. Build Tauri ────────────────────────────────────────────
echo "[3/5] Building Tauri application..."
cargo tauri build --target x86_64-unknown-linux-gnu --bundles deb,appimage
cargo tauri build --target x86_64-unknown-linux-gnu --bundles deb,appimage $SIGN_CONFIG
echo "✅ Build successful"
# ── 4. Copier le runtime à côté de l'exécutable ───────────────
+14 -2
View File
@@ -51,12 +51,23 @@ xcopy /E /I /Q /Y "..\backend" "backend"
xcopy /E /I /Q /Y "..\frontend" "frontend"
echo ✅ Staged
REM ── 2c. Clé de signature des mises à jour (updater Tauri) ──────
set "SIGN_CONFIG="
if exist "obsigate-updater.key" (
set /p TAURI_SIGNING_PRIVATE_KEY=<obsigate-updater.key
echo [2c/6] Cle updater trouvee - artefacts .sig actives
) else (
> updater-off.json echo {"bundle":{"createUpdaterArtifacts":false}}
set "SIGN_CONFIG=--config updater-off.json"
echo [2c/6] Cle updater absente - build sans .sig
)
REM ── 3. Build Tauri ────────────────────────────────────────────
echo [3/6] Building Tauri application (NSIS + MSI installers)...
cargo tauri build --target x86_64-pc-windows-msvc --bundles nsis,msi
cargo tauri build --target x86_64-pc-windows-msvc --bundles nsis,msi %SIGN_CONFIG%
if errorlevel 1 (
echo ⚠️ Standard bundle build failed, trying fallback to NSIS only...
cargo tauri build --target x86_64-pc-windows-msvc --bundles nsis
cargo tauri build --target x86_64-pc-windows-msvc --bundles nsis %SIGN_CONFIG%
if errorlevel 1 (
echo ❌ Build failed!
goto :cleanup
@@ -77,6 +88,7 @@ REM ── 5. Nettoyer les dossiers stagés ────────────
echo [5/6] Cleaning up staged dirs...
if exist "backend" rmdir /s /q "backend"
if exist "frontend" rmdir /s /q "frontend"
if exist "updater-off.json" del /q "updater-off.json"
echo ✅ Cleaned
REM ── 6. Résultats ───────────────────────────────────────────────
+19 -8
View File
@@ -1,12 +1,22 @@
fn main() {
// Get version from git describe (same as backend/version.py)
let version = std::process::Command::new("git")
.args(["describe", "--tags", "--dirty=-dirty"])
.output()
.ok()
.and_then(|o| String::from_utf8(o.stdout).ok())
.map(|s| s.trim().trim_start_matches('v').to_string())
.unwrap_or_else(|| "0.0.0-dev".to_string());
// Version livrée : `VERSION` (racine du dépôt) est la source unique de
// vérité (même fichier que backend/version.py et l'image Docker).
// Repli : `git describe` quand le fichier n'est pas dans le contexte de build.
let version = std::fs::read_to_string(
std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../VERSION"),
)
.ok()
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty())
.unwrap_or_else(|| {
std::process::Command::new("git")
.args(["describe", "--tags", "--dirty=-dirty"])
.output()
.ok()
.and_then(|o| String::from_utf8(o.stdout).ok())
.map(|s| s.trim().trim_start_matches('v').to_string())
.unwrap_or_else(|| "0.0.0".to_string())
});
// Extract short hash for display
let short_hash = std::process::Command::new("git")
@@ -23,6 +33,7 @@ fn main() {
println!("cargo:rustc-env=GIT_VERSION={}", version);
println!("cargo:rustc-env=GIT_HASH={}", short_hash);
println!("cargo:rustc-env=SEMVER={}", semver_base);
println!("cargo:rerun-if-changed=../VERSION");
println!("cargo:rerun-if-changed=.git/HEAD");
println!("cargo:rerun-if-changed=.git/refs/heads/main");
println!("cargo:rerun-if-changed=.git/refs/tags");
+3
View File
@@ -2,6 +2,9 @@
"identifier": "default",
"description": "Default capabilities for ObsiGate Desktop",
"windows": ["main"],
"remote": {
"urls": ["http://127.0.0.1:*", "http://localhost:*"]
},
"permissions": [
"core:default",
"shell:allow-open",
+1 -1
View File
@@ -1 +1 @@
{"default":{"identifier":"default","description":"Default capabilities for ObsiGate Desktop","local":true,"windows":["main"],"permissions":["core:default","shell:allow-open","shell:allow-execute","dialog:default","notification:default","fs:default","process:default","store:default"]}}
{"default":{"identifier":"default","description":"Default capabilities for ObsiGate Desktop","remote":{"urls":["http://127.0.0.1:*","http://localhost:*"]},"local":true,"windows":["main"],"permissions":["core:default","shell:allow-open","shell:allow-execute","dialog:default","notification:default","fs:default","process:default","store:default"]}}
+60
View File
@@ -0,0 +1,60 @@
#!/usr/bin/env pwsh
# ObsiGate Desktop — Windows code signing (scaffolding).
#
# Signe le binaire et l'installateur MSI/NSIS après un `cargo tauri build`.
# Aucun certificat n'est embarqué : les identifiants sont lus depuis
# l'environnement (jamais commités). Le script est un no-op explicite si
# l'env n'est pas configuré — le build reste donc fonctionnel sans signature.
#
# Variables d'environnement :
# OBSIGATE_SIGN_CERT_PFX chemin du .pfx (requis)
# OBSIGATE_SIGN_CERT_PASSWORD mot de passe du .pfx (optionnel)
# OBSIGATE_SIGN_TIMESTAMP_URL URL du serveur d'horodatage RFC3161
# (défaut : http://timestamp.digicert.com)
# OBSIGATE_SIGN_MODE "signtool" (Windows) ou "osslsigncode" (Linux)
#
# Usage :
# ./scripts/sign-windows.ps1 # après le build, signe les artefacts
# OBSIGATE_SIGN_CERT_PFX=cert.pfx ./scripts/sign-windows.ps1
$ErrorActionPreference = 'Stop'
$pfx = $env:OBSIGATE_SIGN_CERT_PFX
if (-not $pfx) {
Write-Host "⚠ Aucun certificat fourni (OBSIGATE_SIGN_CERT_PFX). Signature ignorée — build non signé."
exit 0
}
$password = $env:OBSIGATE_SIGN_CERT_PASSWORD
$tsUrl = if ($env:OBSIGATE_SIGN_TIMESTAMP_URL) { $env:OBSIGATE_SIGN_TIMESTAMP_URL } else { 'http://timestamp.digicert.com' }
$mode = if ($env:OBSIGATE_SIGN_MODE) { $env:OBSIGATE_SIGN_MODE } else { 'signtool' }
$targets = @(
'target/release/obsigate-desktop.exe',
'target/release/bundle/msi/*.msi',
'target/release/bundle/nsis/*-setup.exe'
) | ForEach-Object { Get-Item $_ -ErrorAction SilentlyContinue } | Where-Object { $_ }
if ($targets.Count -eq 0) {
Write-Host "⚠ Aucun artefact à signer trouvé (lancer d'abord `cargo tauri build`)."
exit 0
}
foreach ($t in $targets) {
Write-Host "✍ Signature de $($t.FullName) (mode: $mode)"
if ($mode -eq 'osslsigncode') {
$args = @('sign', '-pkcs12', $pfx, '-h', 'sha256')
if ($password) { $args += @('-pass', $password) }
$args += @('-ts', $tsUrl, '-in', $t.FullName, '-out', $t.FullName)
& osslsigncode @args
if ($LASTEXITCODE -ne 0) { throw "osslsigncode a échoué pour $($t.Name)" }
} else {
$args = @('sign', '/fd', 'SHA256', '/f', $pfx, '/tr', $tsUrl, '/td', 'SHA256')
if ($password) { $args += @('/p', $password) }
$args += $t.FullName
& signtool @args
if ($LASTEXITCODE -ne 0) { throw "signtool a échoué pour $($t.Name)" }
}
}
Write-Host "✅ $($targets.Count) artefact(s) signé(s)."
+178
View File
@@ -0,0 +1,178 @@
//! Windows Taskbar Jump List — recent vaults.
//!
//! Populates the taskbar jump list with the configured vaults so the user can
//! jump straight into a vault from the Windows taskbar / Start menu.
//!
//! Best-effort: every step is fallible and logged, never panics, and a failure
//! simply leaves the jump list at its OS-managed default state.
use log::{info, warn};
#[cfg(target_os = "windows")]
mod imp {
use super::*;
use std::os::windows::ffi::OsStrExt;
use windows::core::{Interface, HSTRING, PCWSTR};
use windows::Win32::Foundation::PROPERTYKEY;
use windows::Win32::System::Com::{
CoCreateInstance, CoInitializeEx, CLSCTX_INPROC_SERVER, COINIT_APARTMENTTHREADED,
};
use windows::Win32::System::Com::StructuredStorage::PROPVARIANT;
use windows::Win32::System::Variant::VT_LPWSTR;
use windows::Win32::UI::Shell::{
ICustomDestinationList, IShellLinkW, SetCurrentProcessExplicitAppUserModelID,
DestinationList, EnumerableObjectCollection, ShellLink,
};
use windows::Win32::UI::Shell::Common::IObjectCollection;
use windows::Win32::UI::Shell::PropertiesSystem::IPropertyStore;
const APP_ID: &str = "com.obsigate.desktop";
// PKEY_Title (System.Title): {F29F85E0-4FF9-1068-AB91-08002B27B3D9}, pid 2
const PKEY_TITLE: PROPERTYKEY = PROPERTYKEY {
fmtid: windows::core::GUID::from_u128(0xF29F85E0_4FF9_1068_AB91_08002B27B3D9),
pid: 2,
};
fn wide_null_terminated(s: &str) -> Vec<u16> {
std::ffi::OsStr::new(s).encode_wide().chain(std::iter::once(0)).collect()
}
/// Set the jump-list display title via the shell link's property store.
fn set_link_title(link: &IShellLinkW, title: &str) -> windows::core::Result<()> {
let store: IPropertyStore = link.cast()?;
let mut wide = wide_null_terminated(title);
let mut pv: PROPVARIANT = unsafe { std::mem::zeroed() };
unsafe {
let inner = &mut *pv.Anonymous.Anonymous;
inner.vt = VT_LPWSTR;
inner.Anonymous.pwszVal = windows::core::PWSTR(wide.as_mut_ptr());
store.SetValue(&PKEY_TITLE, &pv)?;
store.Commit()?;
}
Ok(())
}
/// Build one shell link for a vault. `arg` is the command-line argument the
/// app expects to open that vault (see file associations / single-instance).
fn build_shell_link(name: &str, arg: &str) -> windows::core::Result<IShellLinkW> {
let link: IShellLinkW =
unsafe { CoCreateInstance(&ShellLink, None, CLSCTX_INPROC_SERVER)? };
let exe = std::env::current_exe()
.map_err(|e| windows::core::Error::from(std::io::Error::other(e)))?;
let exe_wide = exe.as_os_str().encode_wide().chain(std::iter::once(0)).collect::<Vec<u16>>();
unsafe {
link.SetPath(PCWSTR(exe_wide.as_ptr()))?;
if !arg.is_empty() {
link.SetArguments(&HSTRING::from(arg))?;
}
link.SetDescription(&HSTRING::from(name))?;
}
// Title (display name) is best-effort — a missing title only means the
// jump-list entry falls back to the executable name.
let _ = set_link_title(&link, name);
Ok(link)
}
/// Populate the jump list with the given (name, arg) vault entries.
pub fn update_jumplist(vaults: &[(String, String)]) {
// COM must be initialised on this thread.
let _ = unsafe { CoInitializeEx(None, COINIT_APARTMENTTHREADED) };
let result = (|| -> windows::core::Result<()> {
unsafe {
SetCurrentProcessExplicitAppUserModelID(PCWSTR(wide_null_terminated(APP_ID).as_ptr()))?;
}
let list: ICustomDestinationList =
unsafe { CoCreateInstance(&DestinationList, None, CLSCTX_INPROC_SERVER)? };
let mut min_slots = 0u32;
unsafe {
list.BeginList::<windows::core::IUnknown>(&mut min_slots)?;
}
// "Vaults" category
let collection: IObjectCollection = unsafe {
CoCreateInstance(&EnumerableObjectCollection, None, CLSCTX_INPROC_SERVER)?
};
for (name, arg) in vaults {
match build_shell_link(name, arg) {
Ok(link) => unsafe {
let _ = collection.AddObject(&link);
},
Err(e) => warn!("jumplist: failed to build link for {}: {}", name, e),
}
}
unsafe {
let array: windows::Win32::UI::Shell::Common::IObjectArray = collection.cast()?;
list.AppendCategory(&HSTRING::from("Vaults"), &array)?;
list.CommitList()?;
}
Ok(())
})();
match result {
Ok(()) => info!("jumplist updated with {} vaults", vaults.len()),
Err(e) => warn!("jumplist update failed (non-fatal): {}", e),
}
}
}
#[cfg(not(target_os = "windows"))]
mod imp {
use super::*;
/// No-op on non-Windows platforms.
pub fn update_jumplist(_vaults: &[(String, String)]) {
info!("jumplist is Windows-only — skipped");
}
}
/// Public entry point: update the Windows jump list from the configured vaults.
/// Each entry maps a vault name to the CLI argument the app accepts to open it.
pub fn update_jumplist(vaults: &[(String, String)]) {
imp::update_jumplist(vaults);
}
/// Build the (name, arg) pairs for the jump list from vault paths.
/// Pure helper so the mapping is unit-testable on every platform.
pub fn build_vault_args(vaults: &[crate::VaultConfig]) -> Vec<(String, String)> {
vaults
.iter()
.map(|v| (v.name.clone(), format!("--vault={}", v.path)))
.collect()
}
#[cfg(test)]
mod tests {
use super::build_vault_args;
use crate::VaultConfig;
fn v(name: &str, path: &str) -> VaultConfig {
VaultConfig { name: name.to_string(), path: path.to_string() }
}
#[test]
fn test_build_vault_args_empty() {
assert!(build_vault_args(&[]).is_empty());
}
#[test]
fn test_build_vault_args_maps_name_and_path() {
let vaults = vec![v("Perso", "C:\\vaults\\perso"), v("IT", "/home/bruno/IT")];
let args = build_vault_args(&vaults);
assert_eq!(args.len(), 2);
assert_eq!(args[0], ("Perso".to_string(), "--vault=C:\\vaults\\perso".to_string()));
assert_eq!(args[1], ("IT".to_string(), "--vault=/home/bruno/IT".to_string()));
}
#[test]
fn test_build_vault_args_preserves_order() {
let vaults = vec![v("a", "/a"), v("b", "/b"), v("c", "/c")];
let args = build_vault_args(&vaults);
assert_eq!(args.iter().map(|(n, _)| n.as_str()).collect::<Vec<_>>(), vec!["a", "b", "c"]);
}
}
+67
View File
@@ -5,6 +5,8 @@
#![windows_subsystem = "windows"]
mod jumplist;
use log::{error, info, warn};
use serde::{Deserialize, Serialize};
use std::fs::{self, OpenOptions};
@@ -76,6 +78,11 @@ struct AppConfig {
window_y: Option<f64>,
window_width: Option<f64>,
window_height: Option<f64>,
/// True once the user has picked a vault or dismissed the first-run wizard.
/// `#[serde(default)]` keeps existing config files (written before this
/// field existed) parseable instead of falling back to a full reset.
#[serde(default)]
wizard_done: bool,
}
impl Default for AppConfig {
@@ -88,6 +95,7 @@ impl Default for AppConfig {
window_y: None,
window_width: Some(1200.0),
window_height: Some(800.0),
wizard_done: false,
}
}
}
@@ -284,6 +292,7 @@ fn get_config() -> AppConfig {
fn save_vault_path(path: String) -> Result<(), String> {
let mut config = load_config();
config.vault_path = Some(path);
config.wizard_done = true;
save_config(&config);
info!("Vault path saved: {}", config.vault_path.as_deref().unwrap_or("none"));
Ok(())
@@ -303,6 +312,7 @@ async fn pick_vault_folder(app: tauri::AppHandle) -> Result<String, String> {
let p_str = p.to_string();
let mut config = load_config();
config.vault_path = Some(p_str.clone());
config.wizard_done = true;
save_config(&config);
Ok(p_str)
}
@@ -310,6 +320,20 @@ async fn pick_vault_folder(app: tauri::AppHandle) -> Result<String, String> {
}
}
#[tauri::command]
fn get_wizard_state() -> bool {
load_config().wizard_done
}
#[tauri::command]
fn complete_wizard() -> Result<(), String> {
let mut config = load_config();
config.wizard_done = true;
save_config(&config);
info!("First-run wizard marked as completed");
Ok(())
}
#[tauri::command]
fn get_system_theme(app: tauri::AppHandle) -> String {
use tauri::Theme;
@@ -562,6 +586,8 @@ fn main() {
get_config,
save_vault_path,
get_vault_path,
get_wizard_state,
complete_wizard,
pick_vault_folder,
get_system_theme,
restart_backend,
@@ -579,6 +605,10 @@ fn main() {
// Build native menu bar (File / Edit / Help)
let _ = build_native_menu(app);
// Windows taskbar jump list — recent vaults (best-effort).
let jumplist_vaults = jumplist::build_vault_args(&config.vaults);
jumplist::update_jumplist(&jumplist_vaults);
// Restore window position/size
if let Some(window) = app.get_webview_window("main") {
if let (Some(x), Some(y)) = (config.window_x, config.window_y) {
@@ -872,8 +902,14 @@ fn build_tray_menu(app: &tauri::App) -> Result<(), Box<dyn std::error::Error>> {
mod tests {
use super::*;
use std::fs;
use std::sync::Mutex;
use tempfile::TempDir;
/// Serializes the `pick_free_port` tests: they bind real TCP ports, so
/// running them in parallel makes them clobber each other's port.
static PORT_TEST_LOCK: Mutex<()> = Mutex::new(());
/// Helper: write config to a temp dir and read it back
fn roundtrip_config(config: &AppConfig) -> AppConfig {
let tmp = TempDir::new().unwrap();
@@ -892,6 +928,7 @@ mod tests {
assert!(c.dirs.is_empty());
assert_eq!(c.window_width, Some(1200.0));
assert_eq!(c.window_height, Some(800.0));
assert!(!c.wizard_done);
}
#[test]
@@ -907,6 +944,7 @@ mod tests {
window_y: Some(200.0),
window_width: Some(1400.0),
window_height: Some(900.0),
wizard_done: true,
};
let loaded = roundtrip_config(&c);
assert_eq!(loaded.vault_path, Some("/test/vault".into()));
@@ -915,6 +953,33 @@ mod tests {
assert_eq!(loaded.vaults[1].path, "/v/work");
assert_eq!(loaded.dirs.len(), 1);
assert_eq!(loaded.window_x, Some(100.0));
assert!(loaded.wizard_done);
}
#[test]
fn test_config_without_wizard_done_parses() {
// Backward compatibility: config files written before `wizard_done`
// existed must still load (instead of resetting to defaults).
let json = r#"{
"vault_path": "/old/vault",
"vaults": [{"name": "Old", "path": "/old"}],
"dirs": [],
"window_x": null,
"window_y": null,
"window_width": 1000.0,
"window_height": 700.0
}"#;
let c: AppConfig = serde_json::from_str(json).unwrap();
assert_eq!(c.vaults.len(), 1);
assert_eq!(c.vaults[0].name, "Old");
assert!(!c.wizard_done);
}
#[test]
fn test_wizard_done_roundtrip() {
let mut c = AppConfig::default();
c.wizard_done = true;
assert!(roundtrip_config(&c).wizard_done);
}
#[test]
@@ -973,6 +1038,7 @@ mod tests {
#[test]
fn test_pick_free_port_is_free() {
let _guard = PORT_TEST_LOCK.lock().unwrap();
let port = pick_free_port();
assert!((DEFAULT_BACKEND_PORT..DEFAULT_BACKEND_PORT + PORT_SCAN_ATTEMPTS).contains(&port));
assert!(TcpListener::bind(("127.0.0.1", port)).is_ok());
@@ -980,6 +1046,7 @@ mod tests {
#[test]
fn test_pick_free_port_skips_busy_port() {
let _guard = PORT_TEST_LOCK.lock().unwrap();
let blocker = match TcpListener::bind(("127.0.0.1", DEFAULT_BACKEND_PORT)) {
Ok(l) => l,
Err(_) => return, // default port already busy on this machine — skip
+4 -3
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://raw.githubusercontent.com/nicedoc/obsigate/main/desktop/tauri.conf.schema.json",
"productName": "ObsiGate",
"version": "2.0.0",
"version": "2.6.1",
"identifier": "com.obsigate.desktop",
"build": {
"frontendDist": "../frontend",
@@ -33,6 +33,7 @@
"bundle": {
"active": true,
"targets": "all",
"createUpdaterArtifacts": true,
"icon": [
"icons/32x32.png",
"icons/128x128.png",
@@ -81,9 +82,9 @@
"store": null,
"updater": {
"endpoints": [
"https://git.dracodev.net/api/v1/repos/Projets/ObsiGate/releases/latest"
"https://git.dracodev.net/Projets/ObsiGate/raw/branch/main/desktop/latest.json"
],
"pubkey": "OBSIGATE_UPDATE_PUBKEY_PLACEHOLDER",
"pubkey": "dW50cnVzdGVkIGNvbW1lbnQ6IG1pbmlzaWduIHB1YmxpYyBrZXk6IDcwQjU2MDM4QUVEREY3NApSV1IwMysyS0ExWUxCK1RQMUgrYnplQWlpYWU2SHVYajVIUHhzNEFKLzZ2Z2puZW9pSUQ2RE8rUAo=",
"windows": {
"installMode": "passive"
}
-4
View File
@@ -11,10 +11,6 @@ services:
obsigate:
build:
context: .
args:
# VERSION est injecte par build.sh/CI via `git describe` ;
# sans variable d'env, l'image tombe sur 0.0.0-dev (fallback Dockerfile).
VERSION: ${VERSION:-0.0.0-dev}
image: obsigate:latest
container_name: obsigate
user: "1000:1000"
+388
View File
@@ -0,0 +1,388 @@
# ObsiGate — Guide d'architecture IA
> **Statut :** document de conception (référence pour l'implémentation)
> **Dernière mise à jour :** 2026-09-11
> **Portée :** sous-systèmes IA d'ObsiGate, couche d'outils (function calling), serveur MCP, sélection du fournisseur/modèle par défaut.
---
## 1. Vue d'ensemble
ObsiGate possède aujourd'hui **deux sous-systèmes IA** qui partagent la même couche de fournisseurs (`backend/ai.py`) :
| Sous-système | Rôle | Backend | Frontend |
|---|---|---|---|
| **Barre d'outils IA de l'éditeur** | 16 transformations de texte statiques (améliorer, traduire, résumer…) | `backend/ai.py`, `backend/ai_routes.py` | `frontend/js/ai.js` |
| **Assistant BooksLM** | Chat contextuel (dossier / documents / général) | `backend/bookslm.py`, `backend/bookslm_routes.py` | `frontend/js/bookslm.js` |
**Constat clé :** il n'existe **aucun tool calling / function calling natif ni MCP**. La seule « action » de l'assistant repose sur un protocole texte maison (`obsigate-action`) limité à `create_file` et `create_directory`.
### Objectif cible
1. Un **assistant in-app** capable de lire, chercher, lister, ouvrir et modifier, via **function calling natif**.
2. Un **serveur MCP** exposant ObsiGate aux clients externes (Claude Desktop, Cursor…).
3. Les deux fronts consomment **la même couche d'outils** — source unique de vérité.
```
┌──────────────────────────────┐
│ Couche d'outils partagée │
│ registry + services + perms │
└───────┬───────────────┬───────┘
│ │
┌─────────────▼─────┐ ┌─────▼──────────────────┐
│ Assistant in-app │ │ Serveur MCP (externe) │
│ function calling │ │ Claude Desktop, Cursor │
│ + confirmations UI│ │ tools + resources │
└───────────────────┘ └────────────────────────┘
```
---
## 2. État actuel (références de code)
### 2.1 Barre d'outils IA de l'éditeur
- `ai_complete()` — `backend/ai.py:175`
- 16 endpoints REST `POST /api/ai/*` — `backend/ai_routes.py:135+`
- Requête `AIRequest` (`text`, `instruction`, `target_lang`, `tone`, `provider`, `model`) — `backend/ai_routes.py:100`
- Statut `GET /api/ai/status` — `backend/ai_routes.py:34`
### 2.2 Assistant BooksLM
- Collecte du contexte : `collect_directory_context()` `backend/bookslm.py:69`, `collect_files_context()` `backend/bookslm.py:209`
- Arborescence : `_build_directory_tree()` `backend/bookslm.py:276`
- Limites : `BOOKSLM_MAX_FILES=200`, `BOOKSLM_MAX_TOTAL_CHARS=200000`, `BOOKSLM_MAX_FILE_CHARS=30000` — `backend/bookslm.py:21`
- Cache 5 min indexé par mtime — `backend/bookslm.py:26`, `:111`
- Prompts système : `build_system_prompt()` `backend/bookslm.py:295`, `build_general_system_prompt()` `backend/bookslm.py:380`
- Protocole d'action : `GENERAL_SYSTEM_PROMPT` `backend/bookslm.py:353-377`
- Routes : `POST /api/ai/bookslm/context`, `POST /api/ai/bookslm/chat` — `backend/bookslm_routes.py:96,118`
- Extraction/application d'action côté client : `_extractActions()` `frontend/js/bookslm.js:502`, `_applyAction()` `frontend/js/bookslm.js:550`
### 2.3 Couche fournisseurs
- `PROVIDERS` (dict module, chargé une fois à l'import) — `backend/ai.py:37-94`
- `DEFAULT_PROVIDER` — `backend/ai.py:96`
- Appel OpenAI-compatible : `_call_deepseek_openrouter()` — `backend/ai.py:115`
- Appel Gemini : `_call_gemini()` — `backend/ai.py:156`
- Clés stockées dans `data/api_keys.json`, fallback `.env` — `get_ai_key()` `backend/ai.py:30`
- Surcharge de modèle par requête : `_handle()` `backend/ai_routes.py:114`, `bookslm_routes.py:194`
### 2.4 Sécurité existante à réutiliser
| Mécanisme | Référence |
|---|---|
| Auth JWT | `require_auth` `backend/auth/middleware.py:69` |
| Admin | `require_admin` `backend/auth/middleware.py:80` |
| Accès vault | `check_vault_access` `backend/auth/middleware.py:87`, `require_vault_access` `:101` |
| Anti path-traversal | `_resolve_safe_path` `backend/main.py:867` |
| Redaction secrets | `backend/secret_redactor.py` (`redact_file_content`) |
| Audit | `data/audit.log` |
### 2.5 Limites de l'existant
1. **Pas de tool calling** : parsing regex fragile, pas de résultats structurés, pas de multi-étapes.
→ **résolu** (phases B/C/D) : tool calling natif, catalogue lecture/recherche + mutations.
2. **2 actions seulement** (`create_file`, `create_directory`) ; ni lecture active, ni recherche, ni édition, ni suppression exposées au modèle.
→ **résolu** (phases C/D) : catalogue complet (lecture, recherche, mutations avec confirmation).
3. **SSE** : ~~non réellement streaming~~ → **corrigé** (B4) : `ai_chat.stream_completion` alimente `/api/ai/bookslm/chat` token par token ; le tool-calling (`/agent`) reste non-streaming (les appels d'outils exigent la réponse complète).
4. **`PROVIDERS` chargé une seule fois** : les modèles par défaut ne sont modifiables que par variables d'environnement (pas de persistance UI).
---
## 3. Architecture cible
### 3.1 Couche 1 — Capacités (services backend)
Extraire la logique métier des routes de `backend/main.py` vers des fonctions réutilisables (services). Les routes REST, l'agent in-app et le serveur MCP appellent ces mêmes services.
**Nouveau module `backend/services/` (A2 lecture/recherche + C + D mutations) :**
```
backend/services/
├── errors.py # ServiceError (code + status HTTP)
├── paths.py # resolve_safe_path (anti path-traversal, source unique)
├── vaults.py # list_accessible_vaults, browse_directory, get_vault_root, list_all_files
├── files.py # read_raw_file, read_file_text (redaction + quota)
├── mutations.py # create/edit/append/rename/move/delete/restore/replace (D)
├── search.py # search_vaults (pagination), list_tags, advanced_search_vaults, search_paths
├── backups.py # get_backup_dir, create_backup, list_backup_files, diff_backup
├── graph.py # get_graph (nodes/edges, wikilinks)
└── recent.py # list_recent, humanize_mtime
```
Les routes `/api/vaults`, `/api/browse/{vault}`, `/api/file/{vault}/raw`, `/api/search`, `/api/tags`,
`/api/recent`, `/api/vault/{vault}/files`, `/api/file/{vault}/backups`, `/api/file/{vault}/diff`,
`/api/graph/{vault}`, `/api/tree-search`, `/api/search/advanced` (phase C) ainsi que
`/api/file/{vault}` (POST/PATCH/DELETE), `/api/file/{vault}/save`, `/api/file/{vault}/restore`,
`/api/directory/{vault}` (POST/PATCH/DELETE), `/api/move/{vault}` et `/api/search/replace` (phase D)
en sont de simples wrappers, tout comme les outils correspondants du catalogue. Un `ServiceError`
est traduit en `HTTPException` (handler global de `backend/main.py`) ou en `ToolError`
(`backend/tools/registry.py`).
**Nouveau module `backend/tools/` :**
```
backend/tools/
├── api.py # façade publique (enregistre les outils, réexporte l'API)
├── context.py # ToolContext (user, allowed_vaults, mode, confirmed)
├── registry.py # décorateur @tool(...) + schémas JSON
├── schemas.py # modèles Pydantic entrée/sortie
├── service.py # implémentations (wrappers des services métier)
└── audit.py # journalisation des appels d'outils
```
> ObsiGate utilise des **namespace packages** implicites (aucun `__init__.py` suivi — `.gitignore` exclut `_*.py`). La façade `api.py` joue le rôle de point d'entrée et déclenche l'enregistrement des outils.
### 3.2 Couche 2 — Registry d'outils
Chaque outil déclare : `name`, `description`, `parameters` (JSON Schema), `risk` (`read` | `write` | `dangerous`), `requires_confirmation` (bool), `scopes` (`in_app`, `mcp`).
### 3.3 Couche 3 — Agent loop (in-app)
Nouveau `backend/agent/loop.py` :
```
boucle (max N itérations):
réponse = LLM(messages, tools)
si tool_calls:
pour chaque appel:
vérifier permissions + confirmation
exécuter via la couche d'outils (ToolContext)
réinjecter le résultat comme message "tool"
continuer
sinon:
renvoyer le texte final
```
- Fallback protocole texte (`obsigate-action`) si le modèle ne supporte pas les tools.
- SSE réellement streaming sur `/chat` (`ai_chat.stream_completion`) ; `/agent` reste buffered
(les appels d'outils exigent la réponse complète avant exécution). Confirmations UI en deux temps
(`confirmation` → `confirm`/`confirm_messages`).
### 3.4 Couche 4 — Serveur MCP (livré, phase E)
`backend/mcp/server.py` s'appuie sur le SDK MCP Python (`mcp==1.9.4`) :
- **Tools** : enregistrés depuis le registry (scope `mcp`). Les outils `read` sont exposés
directement ; les outils `write`/`dangerous` le sont via la paire `propose_<tool>` /
`apply_<tool>` (jeton signé, usage unique, TTL).
- **Resources** : `vault://<name>` (vaults accessibles) et `vault://<name>/<path>` (fichiers,
lecture seule, secrets redactés).
- **Prompts** : `summarize-directory`, `generate-note`, `find-related`.
- Transport : **Streamable HTTP** (`/mcp`, SDK `StreamableHTTPSessionManager` en
`json_response=True`), auth `Authorization: Bearer <JWT>` → `get_current_user`. `stdio`
optionnel plus tard.
- Confirmations (E4) : `backend/mcp/confirmations.py` — jeton JWT (`type=mcp_confirmation`)
contenant outil + arguments + utilisateur ; blacklist de JTI persistée pour l'anti-rejeu.
- Le manager de session est démarré **paresseusement** à la première requête, pour fonctionner
aussi bien sous uvicorn que sous le client de test (sans lifespan ASGI).
---
## 4. Catalogue des outils
> **Inventaire au registre** — 28 outils enregistrés (`backend/tools/service.py`,
> `backend/tools/web.py`), vérifiable par
> `docker exec obsigate-test python -c "from backend.tools import api; from backend.tools.registry import _REGISTRY; print(len(_REGISTRY))"`.
> Les tables ci-dessous reflètent le code, pas l'intention : ce qui n'est pas
> listé n'est pas exposé au modèle.
Légende : **R** = lecture (appel automatique), **M** = mutation (confirmation
two-step obligatoire), **D** = dangereux (confirmation **+** réglage par vault
`aiDestructiveTools`). « Étape (UI) » = libellé affiché dans la section
« N étapes » de l'assistant (`backend/tools/labels.py` → clés `ai.step.*` FR/EN).
### Vaults & navigation
| Outil | Type | Paramètres | Étape (UI) |
|---|---|---|---|
| `list_vaults` | R | — | Liste des vaults consultée |
| `list_directory` | R | `vault`, `path` | Répertoire exploré : {chemin} |
| `list_all_files` | R | `vault`, `dir`, `limit`, `recursive` | Fichiers listés |
### Lecture de contenu
| Outil | Type | Paramètres | Étape (UI) |
|---|---|---|---|
| `read_file` | R | `vault`, `path` | Fichier lu : {chemin} |
| `read_file_raw` | R | `vault`, `path` | Fichier lu : {chemin} |
| `get_backlinks` | R | `vault`, `path` | Backlinks analysés |
| `list_backups` | R | `vault`, `path` | Sauvegardes consultées |
| `diff_backup` | R | `vault`, `path`, `version`, `compare_with` | Comparaison de sauvegarde : {chemin} |
| `get_graph` | R | `vault`, `path`, `depth`, `scope`, `tag` | Graphe du vault consulté |
### Recherche
| Outil | Type | Paramètres | Étape (UI) |
|---|---|---|---|
| `search_fulltext` | R | `q`, `vault`, `tag`, `limit` | Recherche dans le vault : {q} |
| `search_advanced` | R | `q`, `vault`, `tag`, `limit`, `offset`, `sort`, `case_sensitive`, `whole_word`, `regex`, `include_paths`, `exclude_paths`, `created`, `modified`, `size` | Recherche dans le vault : {q} |
| `search_paths` | R | `q`, `vault` | Chemins recherchés : {q} |
| `list_tags` | R | `vault` | Tags consultés |
| `suggest_tags` | R | `q`, `vault`, `limit` | Tags suggérés pour {q} |
| `list_recent` | R | `vault`, `limit`, `mode` | Fichiers récents consultés |
### Web (in-app uniquement, jamais exposé en MCP)
| Outil | Type | Paramètres | Étape (UI) |
|---|---|---|---|
| `web_search` | R | `query`, `max_results`, `category`, `language`, `page` | Recherche sur le web : {query} |
| `fetch_url` | R | `url` | Page web consultée : {url} |
`web_search` interroge l'instance SearXNG auto-hébergée
(`OBSIGATE_SEARXNG_URL`, `search.dracodev.net` par défaut — aucune clé API) ;
`fetch_url` extrait le texte d'une page publique après garde SSRF (URL et
redirections reverrouillées hop par hop, plafond de taille).
### Création / modification (confirmation)
| Outil | Type | Paramètres | Étape (UI) |
|---|---|---|---|
| `create_file` | M | `vault`, `path`, `content` | Fichier proposé : {chemin} |
| `create_directory` | M | `vault`, `path` | Dossier proposé : {chemin} |
| `edit_file` | M | `vault`, `path`, `content` | Fichier modifié : {chemin} |
| `append_to_file` | M | `vault`, `path`, `content` | Fichier complété : {chemin} |
| `restore_backup` | M | `vault`, `path`, `version` | Sauvegarde restaurée : {chemin} |
### Suppression & opérations destructives (confirmation + toggle par vault)
| Outil | Type | Paramètres | Étape (UI) |
|---|---|---|---|
| `rename_file` | D | `vault`, `path`, `new_name` | Fichier renommé : {chemin} |
| `rename_directory` | D | `vault`, `path`, `new_name` | Dossier renommé : {chemin} |
| `move_path` | D | `vault`, `source_path`, `destination_dir` | Élément déplacé : {chemin} |
| `replace_in_files` | D | `find`, `replace`, `vault`, `case_sensitive`, `whole_word`, `regex`, `include_paths`, `exclude_paths`, `replace_all`, `dry_run` | Remplacements : {motif} |
| `delete_file` | D | `vault`, `path` | Fichier supprimé : {chemin} |
| `delete_directory` | D | `vault`, `path`, `recursive` | Dossier supprimé : {chemin} |
### Prévu, non implémenté
Navigation front (`open_file`, `reveal_in_tree` — réalisés côté UI par les liens
cliquables de l'assistant, pas comme outils du registry), partage
(`create_share`, `list_shares`) et sources connectées (Gitea, Drive, courriel… :
voir [#92](./features/ai-tools-roadmap.md)).
---
## 5. Sécurité & permissions
Tout outil reçoit un `ToolContext` et applique **systématiquement** :
1. `check_vault_access(vault, user)` — `backend/auth/middleware.py:87`
2. `_resolve_safe_path(vault_root, path)` — `backend/main.py:867`
3. Confirmation utilisateur pour `risk in (write, dangerous)` (UI in-app : carte Apply ; MCP : two-step `propose`/`apply`).
4. Redaction des secrets avant tout envoi au LLM — `redact_file_content()` puis
`backend/tools/redaction.py` sur **tout** résultat d'outil (diffs, extraits de recherche).
5. Journalisation dans l'audit (`data/audit.log`).
6. **Rate limiting par jeton/outil** (`backend/tools/ratelimit.py` :
`OBSIGATE_TOOL_RATE_LIMIT`, `OBSIGATE_TOOL_RATE_LIMIT_PER_TOOL`,
`OBSIGATE_TOOL_RATE_WINDOW`) et **quotas** réutilisant `BOOKSLM_MAX_*`
(`BOOKSLM_MAX_TOOL_CALLS` par run d'agent, `BOOKSLM_MAX_TOOL_READ_BYTES`
pour `read_file`).
### Confirmations MCP — décision : two-step `propose`/`apply`
MCP n'a pas de bouton « Apply ». Mécanisme retenu :
- **Two-step (canonique)** : `propose_*` (non destructif) renvoie un **aperçu/diff** + un **`confirmation_token`** signé (usage unique, TTL) ; `apply_*` exige ce token pour exécuter. Universel (tout client), anti-rejeu/anti-TOCTOU, et **unifie in-app et MCP** (la carte Apply de l'UI est un `propose`→`apply`).
- **Élicitation (optionnelle, plus tard)** : quand le client l'annonce dans ses `capabilities`, afficher la confirmation inline. Fallback two-step sinon.
Les outils destructifs (`delete_*`, `rename_*`, `move_path`, `replace_in_files`) sont **autorisés** (ObsiGate + MCP = outil de gestion des vaults pour des agents), mais encadrés :
- **Confirmation two-step obligatoire** (`propose_*` → `apply_*`, token signé).
- **Backup automatique** avant toute opération destructive (mécanisme existant, `POST /api/file/{vault}/restore`).
- **Audit** systématique (`data/audit.log`).
- **Toggle par vault** pour désactiver les outils destructifs (défaut : activés).
---
## 6. Sélection du fournisseur et des modèles par défaut
Pour répondre au besoin de cibler un fournisseur/modèle sans dépendre uniquement des variables d'environnement, la configuration expose deux paramètres persistés dans `data/config.json` :
| Clé | Type | Défaut | Rôle |
|---|---|---|---|
| `ai_default_provider` | str | `deepseek` | Fournisseur utilisé quand aucun override n'est fourni |
| `ai_default_models` | dict | `{}` | Modèle par défaut par fournisseur (ex. `{"deepseek": "deepseek-chat"}`) |
- Lecture : `backend/ai.py` (`_read_app_config`, `get_default_provider`, `_load_provider_keys`).
- Écriture : `POST /api/config` (admin) — clés ajoutées à `_DEFAULT_CONFIG` (`backend/main.py:4270`).
- Rechargement à chaud : `reload_ai_config()` met à jour `PROVIDERS` **en place** (les imports existants restent valides).
- UI : section « Clés API Intelligence Artificielle » (`frontend/index.html` `#cfg-ai`), sélecteurs « Fournisseur par défaut » + « Modèle par défaut », sauvegardés par `saveAIKeys()` (`frontend/js/config.js`).
**Précédence de résolution du modèle** : override par requête > `ai_default_models[provider]` > variable d'environnement `*_MODEL` > défaut codé en dur.
**Précédence du fournisseur** : override par requête > `ai_default_provider` > env `AI_DEFAULT_PROVIDER` > `deepseek`.
---
## 6bis. Skills, commandes, vision & capacités (#81)
- **Skills & commandes `/`** — `backend/skills.py` définit les skills intégrés (id, icône, description,
prompt) et les métadonnées des commandes admin. Les skills utilisateur sont persistés par
utilisateur dans `data/skills.json`. Endpoints : `GET/POST /api/ai/skills`,
`DELETE /api/ai/skills/{id}`. Le champ `skill` d'une requête chat/agent injecte le prompt du skill
dans le system prompt (`_resolve_system_prompt`). Les commandes admin (`/help`, `/providers`,
`/provider`, `/model`, `/keys`) sont exécutées côté client.
- **Contexte ad-hoc `@`** — champs `extra_files` / `extra_directories` sur les requêtes
context/chat/agent ; collecte et fusion par `collect_adhoc_context()` + `merge_contexts()`
(`backend/bookslm.py`).
- **Vision** — les messages peuvent porter un contenu multimodal (tableau OpenAI
`text` + `image_url`). `backend/ai_chat.py` convertit les data URLs en `inlineData` Gemini
(`_content_to_gemini_parts`) ; l'OpenAI-compatible passe le tableau tel quel. Les images viennent
d'un copier-coller (base64) ou d'un fichier de vault (data URL chargée par
`load_vault_image_data_url`). Garde-fou : `_validate_vision_support` rejette (400) une requête
d'image si le modèle n'est pas vision.
- **Capacités** — table statique curée `backend/model_capabilities.py` (8 flags : chat, embeddings,
rerank, images, video, audio_speech, audio_transcription, vision). Exposée par
`GET /api/ai/model-capabilities` et par le champ `capabilities` de `GET /api/config/ai-models` ;
affichée dans le picker de l'assistant et le panneau de configuration.
---
## 7. Plan par phases
| Phase | Contenu | Livrable |
|---|---|---|
| **0 — Fondations** | `backend/tools/` (registry, context, service, audit) + extraction des services métier + tests unitaires | Couche d'outils testable sans IA |
| **1 — Function calling in-app** | Abstraction tool-calling multi-provider, agent loop, confirmations UI, SSE réel, outils de navigation | Assistant qui lit/cherche/lit/ouvre/modifie avec confirmation |
| **2 — Serveur MCP** | `backend/mcp/server.py` (tools + resources + prompts), **Streamable HTTP** (`/mcp`, auth JWT), confirmation two-step | ObsiGate accessible comme serveur MCP (local + distant, multi-utilisateur) |
| **3 — Durcissement** ✅ | Rate limiting (`backend/tools/ratelimit.py`), quotas `BOOKSLM_MAX_*`, redaction systématique des résultats (`backend/tools/redaction.py`), doc OpenAPI (tag/path MCP) + [guide MCP](./MCP_GUIDE.md), tests E2E | Observabilité et sécurité complètes |
Voir `docs/ROADMAP.md` (item dédié) pour le détail des activités.
---
## 8. Décisions
| # | Décision | Statut |
|---|---|---|
| 1 | **Transport MCP** : **Streamable HTTP** (`/mcp` dans FastAPI existant). `stdio` optionnel plus tard pour clients locaux sans MCP distant. | ✅ Décidé (2026-09-11) |
| 2 | **Confirmation MCP** : **two-step `propose`/`apply`** (token signé). Élicitation en bonus quand le client l'annonce. | ✅ Décidé (2026-09-11) |
| 3 | **Périmètre des mutations externes** : **toutes autorisées** via MCP — `create`, `edit`, `rename`, `move`, `delete` (ObsiGate + MCP = outil de gestion vault↔agents). Encadrées par confirmation two-step + backup auto + audit + toggle par vault. | ✅ Décidé (2026-09-11) |
| 4 | **Provider tool-calling** : DeepSeek par défaut → adressé par la sélection du fournisseur/modèle par défaut (§6). | ✅ Résolu |
### Justification
- **HTTP** : ObsiGate est déjà un serveur web avec JWT et permissions par vault ; exposer `/mcp` réutilise l'auth, le multi-utilisateur et l'accès distant. `stdio` imposerait un process séparé dupliquant l'infra pour un usage local mono-utilisateur.
- **Two-step** : universel (aucune dépendance aux capacités du client), permet un diff avant application, token signé à usage unique (anti-rejeu), et cohérent avec le flux in-app existant.
- **Mutations complètes** : l'objectif est de faire d'ObsiGate + MCP la couche de gestion entre les vaults et des agents ; restreindre delete/rename/move limiterait fortement les cas d'usage. La sécurité repose sur la confirmation, le backup automatique, l'audit et le toggle par vault plutôt que sur une interdiction par défaut.
---
## 9. Références
- `backend/ai.py`, `backend/ai_routes.py` — couche fournisseurs + actions éditeur
- `backend/services/` — logique métier partagée (vaults, files, search, backups, graph, recent) consommée par les routes et les outils
- `backend/ai_chat.py` — chat completion provider-agnostique avec tool calling et streaming (OpenAI-compat + Gemini)
- `backend/agent/loop.py` — agent loop in-app (multi-étapes, LLM injectable)
- `backend/mcp/server.py` — serveur MCP (Streamable HTTP `/mcp`, tools/resources/prompts)
- `backend/mcp/confirmations.py` — jetons de confirmation signés (two-step, anti-rejeu)
- `backend/tools/ratelimit.py` — rate limiting par jeton/outil (phase F)
- `backend/tools/redaction.py` — redaction récursive des résultats d'outils (phase F)
- `docs/MCP_GUIDE.md` — guide d'installation et d'utilisation des clients MCP
- `backend/bookslm.py`, `backend/bookslm_routes.py` — assistant contextuel (+ endpoint `/agent`)
- `frontend/js/ai.js`, `frontend/js/bookslm.js` — UI IA
- `backend/auth/middleware.py` — permissions
- `backend/secret_redactor.py` — redaction
- `docs/ROADMAP.md` — suivi des phases
+13
View File
@@ -2,6 +2,10 @@
Merci de votre intérêt pour ObsiGate ! Ce guide décrit les standards de code et le workflow de développement.
> **⚠️ À lire avant toute contribution** : la méthode de livraison complète (Definition of Done :
> tests, docs, commit, push, CI) est définie dans [`DELIVERY_WORKFLOW.md`](./DELIVERY_WORKFLOW.md).
> Ce document-ci détaille uniquement les standards de code.
---
## Prérequis
@@ -26,8 +30,17 @@ source .venv/bin/activate # Linux/macOS
# .venv\Scripts\activate # Windows
pip install -r backend/requirements.txt
# Activer les hooks git versionnés (.githooks) : incrément automatique de la
# version livrée (VERSION) à chaque commit + publication du tag vX.Y.Z au push.
scripts/install-hooks.sh
```
> **Hooks obligatoires** : `scripts/install-hooks.sh` pose `core.hooksPath=.githooks` et
> `push.followTags=true`. Sans eux, la version (`VERSION`) ne suit plus les livraisons et le
> CI reste sur un numéro obsolète. Détail : [DEVELOPMENT_AND_RELEASES.md](./DEVELOPMENT_AND_RELEASES.md) §3.
> Contournement ponctuel d'un commit : `SKIP_VERSION_BUMP=1 git commit …`.
### 2. Configurer les vaults de test
Créez un dossier `test_vault/` (ignoré par `.gitignore`) avec quelques fichiers `.md` :
+150
View File
@@ -0,0 +1,150 @@
# Méthode de livraison ObsiGate — Definition of Done
> **Document de référence obligatoire.** À consulter au début de **chaque** tâche (fonctionnalité,
> correction de bug, refactor) et à respecter avant de considérer le travail terminé.
> Référencé par [`AGENTS.md`](../AGENTS.md), la [Roadmap](./ROADMAP.md) et [CONTRIBUTING.md](./CONTRIBUTING.md).
---
## 1. Principe : une seule méthode, toujours la même
Quelle que soit la demande, on suit le même cycle. Une tâche n'est **jamais** « terminée » tant que
la checklist du §5 n'est pas entièrement verte, **CI compris**.
---
## 2. Où vit l'information (source unique de vérité)
| Fichier | Rôle | Quand le mettre à jour |
|---|---|---|
| [`VERSION`](../VERSION) (racine) | **Version livrée** — source unique de vérité (SemVer `MAJEUR.MINEUR.CORRECTIF`) | Automatiquement à chaque commit (hook `.githooks/prepare-commit-msg`) |
| [`docs/ROADMAP.md`](./ROADMAP.md) | Travail **à venir** (🔵 En cours + ⚪ Backlog) + index du complété | Au début (statut) et à la fin (index) |
| [`CHANGELOG.md`](../CHANGELOG.md) | Historique officiel par version (Keep a Changelog) | À chaque livraison : écrire dans `[Unreleased]` (publié en `[X.Y.Z] — date` par le bump) |
| [`docs/features/<slug>.md`](./features/) | **Conception / spec détaillée** d'une grosse feature | Quand l'item est livré |
| [`docs/archive/COMPLETED_v1-v2.md`](./archive/COMPLETED_v1-v2.md) | Détail des items courts livrés | Quand l'item est livré |
| [`docs/ISSUES_TODOLIST.md`](./ISSUES_TODOLIST.md) | Registre des **bugs / TODO** | À chaque bug (statut + correctif/commit) |
| [`docs/DEVELOPMENT_AND_RELEASES.md`](./DEVELOPMENT_AND_RELEASES.md) | Build local & publication des releases | Quand le process change |
| Guides `docs/*_GUIDE.md`, `docs/SPEC_*.md`, `docs/*_ARCHITECTURE*.md` | Conception technique détaillée | Si le domaine concerné change |
| Guide intégré (i18n `frontend/locales/fr.json` + `en.json`) | Guide **utilisateur** in-app | Si impact utilisateur |
| `README.md` / `README.fr.md` | Documentation grand public | Si impact utilisateur |
| Docstrings + `response_model` (`backend/`) + `backend/openapi_docs.py` | Documentation **API** | Si endpoint ajouté/modifié |
**Règle d'or : un fait = un seul fichier.** On ne duplique jamais le détail entre roadmap et changelog.
---
## 3. Choisir le bon registre
- **Nouvelle fonctionnalité** → item `#NN` dans la Roadmap (`⚪ Backlog` → `🔵 En cours`).
- **Bug** → ligne dans `ISSUES_TODOLIST.md` (statut `🔴 ouvert` → `🟠 en cours` → `🟢 corrigé` → `✅ vérifié`).
- **ID stable** : un `#NN` ou `BUG-NNN` ne change jamais et n'est jamais réutilisé. C'est la clé de
jointure entre roadmap, changelog, issues et commits.
---
## 4. Workflow standard (dans l'ordre)
1. **Cadrer** — identifier l'ID (`#NN` / `BUG-NNN`), lire la Roadmap et `ISSUES_TODOLIST.md`,
passer le statut à `🔵 En cours` / `🟠 en cours` **avant** de coder.
2. **Implémenter** — respecter les standards de [CONTRIBUTING.md](./CONTRIBUTING.md) : typage,
docstrings, `response_model`, CSS variables, i18n FR/EN, sécurité `_resolve_safe_path()`.
3. **Tester** — écrire/étendre les **tests unitaires**. Un correctif sans test de non-régression
n'est pas terminé.
4. **Vérifier en local** — exécuter les commandes du §6.
5. **Documenter** — CHANGELOG `[Unreleased]`, Roadmap / ISSUES, fiche feature ou archive, guide
utilisateur + i18n FR/EN, README si besoin, OpenAPI si API.
6. **Commit** — message conventionnel (`feat:`, `fix:`…) référençant `#NN` / `BUG-NNN` : le
préfixe détermine l'incrément SemVer de `VERSION` (MAJEUR / MINEUR / CORRECTIF), appliqué
automatiquement par le hook `.githooks/prepare-commit-msg`.
7. **Push** puis **vérifier le CI vert** (jobs `lint`, `test`, `security`, `build`, `e2e`) ; le
tag `vX.Y.Z` de la version livrée est publié avec la branche (`push.followTags`).
8. **Clôturer** — statut `🟢 corrigé` / index `✅` posé par l'IA ; l'utilisateur valide (`✅ vérifié`).
---
## 5. Checklist « Definition of Done »
### Code
- [ ] Comportement conforme à la demande
- [ ] Standards CONTRIBUTING respectés
- [ ] Aucun secret / clé committé
- [ ] i18n FR **et** EN si texte d'interface
### Tests
- [ ] Tests unitaires ajoutés ou mis à jour (backend pytest / frontend Node)
- [ ] `pytest` vert en local
- [ ] `ruff` + `mypy` : 0 erreur
- [ ] Tests frontend verts (`validate-imports` + `unit` + JSDOM ciblés)
- [ ] E2E Playwright si flow UI critique touché
### Documentation
- [ ] `VERSION` incrémenté et dérivés synchronisés (automatique via le hook ; `tests/test_version.py` vert)
- [ ] `CHANGELOG.md` → `[Unreleased]` (section Ajouté / Modifié / Corrigé)
- [ ] `docs/ROADMAP.md` → statut mis à jour + ligne dans l'index « Complété »
- [ ] Fiche `docs/features/<slug>.md` **ou** `docs/archive/` si l'item est livré
- [ ] `docs/ISSUES_TODOLIST.md` → statut + colonne « Correctif / Commit » (si bug)
- [ ] Guide d'utilisation (i18n) + README FR/EN si impact utilisateur
- [ ] OpenAPI / docstrings + `response_model` si API
### Livraison
- [ ] Commit conventionnel référençant l'ID
- [ ] Push effectué
- [ ] CI vert : `lint` → `test` → `security` → `build` → `e2e`
---
## 6. Commandes de vérification locale
```powershell
# Backend
.\.venv\Scripts\python.exe -m pytest tests/
.\.venv\Scripts\python.exe -m ruff check backend/
.\.venv\Scripts\python.exe -m mypy backend/ --ignore-missing-imports
# Frontend
node tests/frontend/validate-imports.mjs
node tests/frontend/unit.test.mjs
# tests JSDOM ciblés, ex :
node tests/frontend/pane-manager.test.mjs
# E2E (si UI)
npx playwright test
```
> Les mêmes vérifications tournent dans le CI Gitea (`.gitea/workflows/ci.yml`) : jobs
> `lint`, `test`, `security`, `build`, `e2e`.
---
## 7. Versionnement & release
- **Source unique de vérité : le fichier [`VERSION`](../VERSION)** (racine du dépôt), au format
`MAJEUR.MINEUR.CORRECTIF` (SemVer). Il est incrémenté **automatiquement à chaque commit** par
le hook versionné `.githooks/prepare-commit-msg` — `!:` / `BREAKING CHANGE` → **MAJEUR**, `feat` →
**MINEUR**, tout le reste → **CORRECTIF** — puis tagué `vX.Y.Z` par `.githooks/post-commit`
(tag publié au push grâce à `push.followTags`). Installation une fois par clone :
`scripts/install-hooks.sh`.
- Le même commit resynchronise les dérivés : `package.json`, desktop Tauri (`tauri.conf.json`,
`Cargo.toml`, `Cargo.lock`), `README.md`/`README.fr.md`, `docs/ROADMAP.md`, et publie la
section `[Unreleased]` du `CHANGELOG.md` en `[X.Y.Z] — date`.
- Le backend (`backend/version.py`), l'image Docker (`COPY VERSION`) et le desktop Tauri lisent
ce fichier : la version affichée par l'UI (`/api/health` → header, boîte À propos) suit donc
chaque livraison.
- **Garde-fou** : `tests/test_version.py::TestRepoVersionAlignment` échoue dès qu'un dérivé
diverge de `VERSION` (numéro codé en dur, CHANGELOG sans section, README non resynchronisé).
- Bump manuel si besoin : `scripts/bump_version.py --dry-run|--minor|--set X.Y.Z|--push`.
Commit sans incrément (cas exceptionnel) : `SKIP_VERSION_BUMP=1 git commit …`.
Le rattachement des fichiers au commit se fait par un `--amend` immédiat (commit non encore
poussé) : le SHA affiché par `git commit` est donc remplacé par celui de l'amend — `git log`,
`HEAD` et le tag `vX.Y.Z` restent, eux, alignés sur la version livrée.
- **Ne jamais** réécrire une version déjà publiée dans le CHANGELOG.
---
## 8. À ne jamais faire
- Marquer une tâche terminée sans tests verts ni CI vert.
- Committer sans mettre à jour `CHANGELOG.md` **et** le registre concerné (Roadmap / Issues).
- Dupliquer le détail entre Roadmap et CHANGELOG.
- Réutiliser un ID `#NN` / `BUG-NNN`.
- Pousser des secrets, clés ou tokens.
+184
View File
@@ -0,0 +1,184 @@
# ObsiGate Desktop — Protocole de tests E2E manuels
> **Rôle :** valider les 6 scénarios de bout en bout du desktop Tauri (#77) qui ne
> peuvent pas être automatisés (interactions OS : installeur, tray, notifications,
> association de fichiers, auto-update, désinstallation).
> **Statut :** protocole documenté — à exécuter manuellement par un humain.
> **Références :** [feature desktop-tauri](./features/desktop-tauri.md) ·
> [Roadmap](./ROADMAP.md) · [Build & releases](./DEVELOPMENT_AND_RELEASES.md)
---
## 1. Prérequis
- Un **build release** de l'application :
- Windows : `desktop\build-windows.bat` → `ObsiGate_x.y.z_x64.msi` / `-setup.exe`
- Linux : `desktop/build-linux.sh` → `obsigate_x.y.z_amd64.deb` / `.AppImage`
- Une **machine vierge ou un compte utilisateur propre** (pas d'installation
précédente) pour les tests d'installation/désinstallation.
- Le **mot de passe admin** affiché au premier lancement (ou `OBSIGATE_ADMIN_PASSWORD`
défini avant le lancement).
- Accès en écriture aux logs : `%APPDATA%\ObsiGate\logs\backend.log` (Windows) ou
`~/.config/obsigate/logs/backend.log` (Linux).
## 2. Comment remplir ce protocole
Pour chaque test : noter la **version testée** (`menu Aide → À propos`), le
**commit** du build, la **date**, puis cocher `✅ OK` ou `❌ Échec` et joindre les
logs/ captures en cas d'échec. Ne cocher un test qu'après avoir observé le
**résultat attendu**.
| Test | Version | Commit | Date | Résultat |
|---|---|---|---|---|
| T1 — Installation | | | | ⬜ |
| T2 — Tray icon | | | | ⬜ |
| T3 — Notifications natives | | | | ⬜ |
| T4 — Association `.md` | | | | ⬜ |
| T5 — Auto-update | | | | ⬜ |
| T6 — Désinstallation | | | | ⬜ |
---
## T1 — Installation → premier lancement → ouverture d'un fichier
**Objectif :** l'installeur installe l'app et le premier lancement démarre le
backend + affiche l'interface.
1. Lancer l'installeur (`.msi` ou `.deb`/`.AppImage`).
2. Installer dans le chemin par défaut → vérifier la création du raccourci
(bureau / menu Démarrer / menu applications).
3. Lancer ObsiGate depuis le raccourci.
4. Observer l'écran de démarrage (« ObsiGate démarre… ») puis l'interface.
5. Au premier lancement, cliquer **« Choisir mon dossier »** dans la bannière du
wizard et sélectionner un dossier vault.
6. Ouvrir un fichier `.md` depuis l'arborescence.
**Résultat attendu :**
- Aucun terminal / console visible.
- Le backend répond (`http://127.0.0.1:17890/api/health` → 200) en ~2 s.
- Le fichier s'ouvre dans le viewer.
- La bannière du wizard **ne réapparaît pas** au lancement suivant.
**Échec si :** écran blanc, backend non démarré, wizard qui revient à chaque
lancement.
---
## T2 — Tray icon → réduire → restaurer
**Objectif :** le tray fonctionne et contrôle la fenêtre.
1. Vérifier la présence de l'icône ObsiGate dans la zone de notification.
2. Clic **gauche** sur l'icône → la fenêtre se cache.
3. Clic **gauche** à nouveau → la fenêtre réapparaît et prend le focus.
4. Clic **droit** → menu (Ouvrir ObsiGate / À propos / Quitter).
5. Fermer la fenêtre avec le **X** → elle se réduit dans le tray, le backend
continue de tourner (l'icône reste).
**Résultat attendu :** toggle visible/caché immédiat, menu contextuel complet,
fermeture par X = réduction (pas d'arrêt du backend).
---
## T3 — Notifications natives (fichier modifié → popup OS)
**Objectif :** les notifications natives remplacent le push web.
1. Activer les notifications dans les préférences (par vault si proposé).
2. Modifier un fichier surveillé (édition externe dans le vault, ou édition
in-app avec le watcher actif).
3. Observer la notification du système d'exploitation.
**Résultat attendu :** popup OS affichant le nom du fichier/vault et l'action ;
un clic ouvre le fichier concerné. La notification apparaît même si la fenêtre
ObsiGate est réduite.
> Note : sur Windows, vérifier que les notifications ne sont pas bloquées dans
> *Paramètres → Système → Notifications*.
---
## T4 — Association `.md` → double-clic → ouvre dans ObsiGate
**Objectif :** l'association de fichiers ouvre l'app.
1. Vérifier que `.md` est associé à « ObsiGate Markdown » (Windows :
*Paramètres → Applications par défaut* ; Linux : `xdg-mime query default text/markdown`).
2. **Fermer** complètement ObsiGate (tray → Quitter).
3. Double-cliquer sur un fichier `.md` dans l'explorateur / gestionnaire de fichiers.
4. Observer le lancement d'ObsiGate et l'ouverture du fichier.
**Résultat attendu :** ObsiGate démarre et ouvre le fichier (ou le vault
contenant le fichier). Un second double-clic alors que l'app tourne **focus la
fenêtre existante** (single-instance) au lieu de lancer un doublon.
---
## T5 — Auto-update → nouvelle version → installation
**Objectif :** l'updater détecte et installe une nouvelle version.
1. S'assurer qu'une **release plus récente** existe sur Gitea (avec les artefacts
et le manifeste de mise à jour signé).
2. Lancer la version N.
3. Déclencher la vérification de mise à jour (menu ou au démarrage selon l'UI).
4. Accepter la mise à jour → l'app télécharge, vérifie la signature et installe.
5. Relancer → vérifier la version affichée (`À propos`).
**Résultat attendu :** détection de la version N+1, téléchargement, installation
sans intervention manuelle, version mise à jour après redémarrage.
**Prérequis bloquant :** la release N+1 doit être publiée (binaires **signés**
`.sig` + `latest.json`) et le fichier `desktop/latest.json` **commité sur `main`**
(le manifeste est généré par `scripts/publish_release.py`, cf. §4).
---
## T6 — Désinstallation propre
**Objectif :** la désinstallation ne laisse aucun processus ni résidu gênant.
1. Fermer ObsiGate (tray → Quitter) pour éviter un processus orphelin.
2. Désinstaller via le panneau de configuration (Windows) ou `dpkg -r obsigate`
/ supprimer l'AppImage (Linux).
3. Vérifier qu'aucun processus `obsigate-desktop` / Python backend ne tourne
encore.
4. Vérifier les résidus : raccourcis supprimés, entrée « Applications par
défaut » retirée.
5. (Optionnel) Vérifier le comportement des données utilisateur
(`%APPDATA%\ObsiGate` / `~/.config/obsigate`) : conservées ou supprimées selon
le choix documenté.
**Résultat attendu :** désinstallation sans erreur, aucun processus résiduel,
aucun raccourci cassé.
---
## 3. Emplacements utiles
| Élément | Windows | Linux |
|---|---|---|
| Config | `%APPDATA%\ObsiGate\config.json` | `~/.config/obsigate/config.json` |
| Logs backend | `%APPDATA%\ObsiGate\logs\backend.log` | `~/.config/obsigate/logs/backend.log` |
| Données (index, comptes) | `%APPDATA%\ObsiGate\data\` | `~/.config/obsigate/data\` |
## 4. Signature de code & auto-update
- **Signature Windows (optionnelle, hors périmètre de ce protocole) :** sans
certificat, SmartScreen affiche un avertissement au premier lancement
(*Informations complémentaires → Exécuter quand même*). Le script
`desktop/scripts/sign-windows.ps1` signe automatiquement si
`OBSIGATE_SIGN_CERT_PFX` est défini ; sinon il est un no-op explicite.
Alternatives détaillées dans le [README desktop](../desktop/README.md).
- **Signature de l'updater Tauri (gratuite, distincte de la signature Windows) :**
déjà configurée — `pubkey` dans `desktop/tauri.conf.json`, clé privée lue depuis
`desktop/obsigate-updater.key` (gitignorée) ou `TAURI_SIGNING_PRIVATE_KEY`.
Le manifeste `latest.json` est généré par `scripts/publish_release.py` puis
commité sur `main`. Procédure :
[DEVELOPMENT_AND_RELEASES §2bis](./DEVELOPMENT_AND_RELEASES.md#2bis-signature-des-mises-à-jour-updater-tauri).
## 5. Clôture
Une fois les 6 tests exécutés et OK, reporter le résultat dans
[`docs/features/desktop-tauri.md`](./features/desktop-tauri.md) (section F) et
mettre à jour le statut du #77 dans la [Roadmap](./ROADMAP.md).
+170 -14
View File
@@ -57,23 +57,179 @@ cargo tauri dev
---
## 3. Commit, Push & Tagging Git
## 2bis. Signature des mises à jour (updater Tauri)
Une fois les modifications testées et validées :
La signature de l'auto-update Tauri est **indépendante** de la signature de code
Windows et **gratuite**. Elle garantit qu'une mise à jour téléchargée provient bien
de vous. Elle repose sur une paire de clés `minisign` :
1. **Commit et Push du code source** :
```bash
git add .
git commit -m "feat: préparation release v2.0.0"
git push origin main
```
*Ceci déclenchera la vérification CI standard sur Gitea (lint, tests unitaires, build Docker).*
- La **clé publique** est embarquée dans `desktop/tauri.conf.json`
(`plugins.updater.pubkey`).
- La **clé privée** signe les artefacts au build. Elle ne doit **jamais** être
commitée (ignorée par `.gitignore`).
### A. Générer la paire de clés (une seule fois)
```bash
cd desktop
cargo tauri signer generate -w obsigate-updater.key
# La clé publique s'affiche et est écrite dans obsigate-updater.key.pub
```
> ⚠️ Conservez la clé privée en lieu sûr (gestionnaire de secrets). Si vous la
> perdez, les mises à jour ne pourront plus être signées. Pour la protéger par mot
> de passe : ajoutez `-p "<mot de passe>"`.
Copiez le contenu de `obsigate-updater.key.pub` dans
`desktop/tauri.conf.json` → `plugins.updater.pubkey`.
### B. Build local signé
```powershell
# Windows PowerShell
$env:TAURI_SIGNING_PRIVATE_KEY = Get-Content -Raw .\obsigate-updater.key
# $env:TAURI_SIGNING_PRIVATE_KEY_PASSWORD = "<mot de passe>" # si la clé en a un
cargo tauri build --bundles nsis,msi
```
```bash
# Linux / Bash
export TAURI_SIGNING_PRIVATE_KEY="$(cat obsigate-updater.key)"
# export TAURI_SIGNING_PRIVATE_KEY_PASSWORD="<mot de passe>"
cargo tauri build --bundles appimage,deb
```
Le CLI produit des fichiers `.sig` à côté de chaque artefact
(`*.exe.sig`, `*.msi.sig`, `*.AppImage.sig`, `*.deb.sig`).
> Sans clé définie, `createUpdaterArtifacts` est actif et le build échoue : dans
> le CI, l'étape désactive automatiquement les artefacts de mise à jour si le
> secret est absent.
### C. Secrets CI (Gitea)
Dans **Dépôt → Paramètres → Actions → Secrets**, créez :
| Secret | Valeur |
|---|---|
| `TAURI_SIGNING_PRIVATE_KEY` | contenu **intégral** du fichier `.key` |
| `TAURI_SIGNING_PRIVATE_KEY_PASSWORD` | mot de passe de la clé (vide si aucun) |
Le workflow `.gitea/workflows/desktop-build.yml` les expose aux étapes de build ;
les fichiers `.sig` sont uploadés comme artefacts.
### D. Manifeste de mise à jour (`latest.json`)
Le CLI Tauri génère les `.sig` mais **pas** le manifeste JSON consommé par
l'updater. Celui-ci est produit par `scripts/updater_manifest.py` :
```powershell
# Windows — après build-windows.bat
.\.venv\Scripts\python.exe scripts\updater_manifest.py --tag v2.3.0
```
`publish_release.py` l'appelle automatiquement : il écrit `desktop/latest.json`,
l'ajoute aux assets de la release, et rappelle la dernière étape manuelle.
**Flux complet de release :**
1. `desktop\build-windows.bat` (build + signature `.sig`) ;
2. `python scripts\publish_release.py --tag vX.Y.Z` (checksums + `latest.json` + upload) ;
3. **commit** de `desktop/latest.json` sur `main` puis `git push`.
L'endpoint de l'updater (`desktop/tauri.conf.json`) pointe vers ce fichier
versionné :
```
https://git.dracodev.net/Projets/ObsiGate/raw/branch/main/desktop/latest.json
```
Document produit (URLs construites vers les assets de la release) :
```json
{
"version": "2.3.0",
"notes": "…",
"pub_date": "2026-09-12T00:00:00Z",
"platforms": {
"windows-x86_64": { "signature": "<contenu .exe.sig>", "url": "<URL du .exe>" },
"linux-x86_64": { "signature": "<contenu .AppImage.sig>", "url": "<URL .AppImage>" }
}
}
```
> Seules les plateformes dont l'artefact **signé** est présent sont incluses.
> Sans `desktop/latest.json` à jour sur `main`, l'updater ne détecte aucune
> mise à jour (la signature, elle, reste opérationnelle).
---
## 3. Version, Commit & Push
### Source unique de vérité : `VERSION`
Le fichier **`VERSION`** à la racine du dépôt contient la version livrée au format
`MAJEUR.MINEUR.CORRECTIF`. C'est la **seule** source : elle est incrémentée à chaque
commit et tout le reste en découle.
| Dérivé | Contenu |
|---|---|
| `backend/version.py` | lit `VERSION` (priorité : `OBSIGATE_VERSION` > `./VERSION` > `backend/VERSION` > tag git) et l'expose via `/api/health` |
| `Dockerfile` | `COPY VERSION ./VERSION` — plus aucun numéro codé en dur dans l'image |
| `build.sh` / CI | affichent la version lue dans `VERSION` |
| `desktop/build.rs` | injecte `VERSION` dans `GIT_VERSION` (repli `git describe`) |
| `package.json`, `desktop/tauri.conf.json`, `desktop/Cargo.{toml,lock}`, `README.md`, `README.fr.md`, `docs/ROADMAP.md`, `CHANGELOG.md` | resynchronisés automatiquement à chaque incrément |
### Incrément automatique (hooks git versionnés)
`scripts/install-hooks.sh` (une seule fois par clone) pose `core.hooksPath=.githooks`
et `push.followTags=true`. Ensuite, chaque commit incrémente la version selon son
message (Conventional Commits) :
| Message de commit | Incrément |
|---|---|
| `!:` ou `BREAKING CHANGE:` | MAJEUR — `x.0.0` |
| `feat:` / `feat(scope):` | MINEUR — `x.y.0` |
| `fix:`, `perf:`, `docs:`, … | CORRECTIF — `x.y.z` |
`.githooks/prepare-commit-msg` incrémente `VERSION`, resynchronise les fichiers dérivés
et publie la section `[Unreleased]` du CHANGELOG en `[X.Y.Z] — date`. `.githooks/post-commit`
rattache ces fichiers au commit qui vient d'être créé (git fige l'arbre avant
`prepare-commit-msg` : un `git add` à cet instant ne serait repris qu'au commit suivant —
d'où un `--amend` immédiat, le commit n'étant pas encore poussé) puis crée le tag `vX.Y.Z`,
publié automatiquement au push (`push.followTags`). Aucun incrément pour un merge, un
revert, un `chore(release)` ou un `--amend`.
Livraison type :
```bash
git add <chemins explicites> # pas de `git add -A` (test_vault = brouillon utilisateur)
git commit -m "fix(ai): BUG-047 …" # → VERSION incrémentée + tag vX.Y.Z créé
git push origin main # → branche + tag publiés
```
Contournement ponctuel (commit sans incrément) : `SKIP_VERSION_BUMP=1 git commit …`.
> **SHA affiché vs SHA réel** : l'incrément et les fichiers dérivés sont rattachés au
> commit par un `--amend` immédiat (le commit n'est pas encore poussé), donc le SHA
> affiché par `git commit` est remplacé par celui de l'amend — `git log`, `HEAD` et le
> tag `vX.Y.Z` sont, eux, alignés sur ce dernier. `git push` envoie bien la version finale.
### Bump manuel / outillage
```bash
scripts/bump_version.py --print-version # version courante
scripts/bump_version.py --dry-run # prochaine version, sans rien écrire
scripts/bump_version.py --minor # incrément forcé + resynchronisation
scripts/bump_version.py --set 3.0.0 # version imposée
scripts/bump_version.py --major --commit --tag --push
scripts/bump_version.sh … # même outil (wrapper historique)
```
> La version desktop doit être synchronisée avec le tag pour que l'updater Tauri
> détecte les mises à jour (`latest.json` reprend la version de `tauri.conf.json`) :
> l'incrément automatique s'en charge.
2. **Création du Tag de Version** :
```bash
git tag -a v2.0.0 -m "Release v2.0.0"
git push origin v2.0.0
```
---
+80 -4
View File
@@ -2,6 +2,8 @@
> Ce document utilise un **format tabulaire simple et rigoureux** conçu pour être
> maintenu à la fois par un humain (éditeur texte) et par un agent IA. Les règles
> exactes sont définies dans la section **« Comment fonctionne ce document »**.
> La procédure de livraison obligatoire (tests, docs, commit, push, CI) est définie
> dans [`DELIVERY_WORKFLOW.md`](./DELIVERY_WORKFLOW.md).
# 🐛 ObsiGate — Suivi des Bugs / TODO de Correction
@@ -12,7 +14,7 @@
- **Projet** : ObsiGate — Porte d'entrée web pour vaults Obsidian
- **Stack** : Python 3.11+ (backend FastAPI) · JavaScript/Vanilla (frontend) · Tauri/Rust (desktop)
- **Dernière mise à jour** : *(à tenir à jour à chaque modification)*
- **Dernière mise à jour** : 2026-09-16
---
@@ -107,10 +109,55 @@ Avant de corriger quoi que ce soit, un agent IA doit :
| # | Titre | Statut | Priorité | Scope | Assigné | Zone (fichier) | Cmd de repro | Correctif / Commit | Notes |
|---|---|---|---|---|---|---|---|---|---|
| *BUG-001* | L'ouverture des fichier PDF ne fonctionne pas et donne l'erreur Internal Server Error | 🔴 ouvert | P1 | fichier PDF | IA | |
https://og.dracodev.net/api/file/Recettes/pdf/stream?path=98_Boite_Outils%2F98.2_Attachments%2FBi%C3%A8re%20blonde%20envoy%C3%A9%20par%20Desja.pdf| — | l'erreur se retrouve dans la section network de la console web et dans l'interface web lors du chargement d'un pdf |
| *BUG-002* | l'ouverture d'un fichier .excalidraw ne fonctionne pas et affiche toujours Loading *Excalidraw…* | 🔴 ouvert | P1 | fichier .excalidaw | IA | | | — | test d'accès réaliser via cloudflare et réseau local démontre que ce problème est au 2 endroits |
| *BUG-001* | L'ouverture des fichier PDF ne fonctionne pas et donne l'erreur Internal Server Error | 🟢 corrigé | P1 | fichier PDF | IA | `backend/main.py` | `GET /api/file/{vault}/pdf/stream?path=…(pdf à nom accentué)` | `backend/main.py` : Content-Disposition encodé RFC 5987 (helper `_content_disposition`) | 500 car nom Unicode brut dans l'en-tête → header invalide. Vérifié: stream 200 / Range 206 + test `tests/test_pdf_stream.py` |
| *BUG-002* | l'ouverture d'un fichier .excalidraw ne fonctionne pas et affiche toujours Loading *Excalidraw…* | 🟢 corrigé | P1 | fichier .excalidraw | IA | `frontend/excalidraw-editor.html` | Ouvrir un fichier `.excalidraw` | `frontend/excalidraw-editor.html` : alias esm.sh supprimé (408 jotai) + React 19 cohérent + prop `excalidrawAPI` | 2 causes: 408 esm.sh sur `?alias` + prop legacy `excalidrawRef` inopérante en 0.18. Vérifié navigateur: Loading masqué + cycle save OK |
| | | | | | | | | | |
| *BUG-003* | `mypy` : 33 erreurs de typage (étape CI en mode advisory → bloquante) | 🟢 corrigé | P2 | ⚙️ backend | IA | `backend/{main,indexer,export,pdf_reader,bookslm_routes}.py`, `backend/auth/router.py`, `.gitea/workflows/ci.yml` | `mypy backend/ --ignore-missing-imports` | Annotations de types, gardes `None` (auth/router), import `PROVIDERS` manquant (bug latent `main.py:4523`), `PdfReader: Any` ; CI mypy rendu bloquant | 0 erreur après correction. `PROVIDERS` non importé → `NameError` avalé par le `except` (le modèle par défaut n'était jamais prépendé). Tests : 728 passed |
| *BUG-004* | Lien cassé vers `CONTRIBUTING.md` dans `README.md` | 🟢 corrigé | P3 | 📄 docs | IA | `README.md` | Cliquer le lien « Contributing » | `README.md` : lien → `docs/CONTRIBUTING.md` (+ `DELIVERY_WORKFLOW.md`) ; arbre du projet corrigé | `README.fr.md` pointait déjà correctement vers `./docs/CONTRIBUTING.md` |
| *BUG-005* | Sur mobile via Cloudflare (`og.dracodev.net`), le site ne charge pas complètement même après purge des caches (téléphone + Cloudflare) | 🟢 corrigé | P1 | 📱 frontend + ⚙️ backend | IA | `frontend/sw.js`, `frontend/index.html`, `frontend/js/sync.js`, `backend/main.py` | Ouvrir `og.dracodev.net` sur mobile (PWA installée), purger les caches, recharger | SW **network-first** pour le code + caches versionnés (`SW_VERSION`) ; précache corrigé (`/static/js/…`) ; migration ponctuelle `localStorage` ; `Cache-Control: no-cache` (le middleware renvoyait `immutable` 1 an sur `/static`) ; tests `tests/frontend/sw.test.mjs` + `TestStaticCaching` | Cause : SW cache-first à nom fixe + en-têtes `immutable` non fingerprinted → ancien build servi indéfiniment (Cache Storage ≠ cache navigateur/CF). Vérifié : 754 tests backend, frontend OK |
| *BUG-006* | Clé API DeepSeek réelle commitée en clair dans `.env.example` | 🟢 corrigé | P0 | 📄 docs + 🔐 sécurité | IA | `.env.example` | `git grep "sk-" .env.example` | `.env.example` : clé remplacée par un placeholder (`sk-xxx…`) + modèle `deepseek-chat` | ⚠️ **Rotation de la clé requise** : elle reste dans l'historique Git → révoquer/régénérer côté DeepSeek. Le fichier `.env` (réel) est bien gitignoré |
| *BUG-007* | Assistant IA : menus `/` et `@` — navigation clavier ↑/↓ inopérante et filtrage des skills cassé par les accents | 🟢 corrigé | P1 | 📱 frontend | IA | `frontend/js/bookslm.js` | Taper `/résumé` ou `@café`, puis ↑/↓ | `bookslm.js` : motifs Unicode `\p{L}` (détection + sélection), recherche normalisée NFD (insensible aux accents), jeton `_menuSeq` (rendus async obsolètes), `scrollIntoView` de l'élément actif | Le menu se fermait dès la saisie d'un accent ; la frappe rapide pouvait écraser le menu avec un résultat obsolète. Tests : `tests/frontend/ai.test.mjs` (+3) |
| *BUG-008* | Assistant IA : commande `@` — chemins accentués + contexte ad-hoc ignoré en mode Général | 🟢 corrigé | P1 | 📱 frontend + ⚙️ backend | IA | `frontend/js/bookslm.js`, `backend/bookslm_routes.py` | `@fichier-accentué` puis envoyer en mode Général | Frontend : détection Unicode + repli `vault=all` sans vault courant. Backend : `_resolve_system_prompt` résout le vault optionnel dès qu'un contexte `@` est présent (mode Général) | Le backend ignorait `extra_files`/`extra_directories` en mode Général (`vault_path is None`). Tests : `tests/test_bookslm.py` (+2) et `tests/frontend/ai.test.mjs` |
| *BUG-009* | Assistant IA : liste de modèles corrompue (art ASCII) et clés i18n brutes (`ai.model_search`) sous OpenRouter | 🟢 corrigé | P1 | 📱 frontend | IA | `frontend/js/ai.js`, `frontend/js/i18n.js`, `frontend/sw.js`, `frontend/style.css` | Ouvrir le menu modèle sous « openrouter » | Locale non content-hashée en cache HTTP → `cache:'no-store'` + bump `SW_VERSION` v4 ; styles critiques du picker appliqués **en ligne** (popover absolu, liste en colonne, options `display:block`) ; rendu plafonné à 200 modèles + indicateur « … N autres » | OpenRouter expose plusieurs centaines de modèles ; si `style.css` est en cache, la liste s'affichait en bloc inline (art ASCII) et les clés i18n brutes provenaient d'un `fr.json` obsolète. Tests : `tests/frontend/ai.test.mjs` (+1) |
| *BUG-010* | Assistant IA : la commande `@` n'ajoute pas le contexte à la requête en mode Général | 🟢 corrigé | P1 | 📱 frontend | IA | `frontend/js/bookslm.js` | Assistant en mode Général, taper `@fichier`, sélectionner, envoyer | Capture du `vault` renvoyé par `/api/tree-search` dans la sélection `@` ; `_contextVault()` propage ce vault aux requêtes `/context` et `/chat` ; recherches `@` suivantes limitées à ce vault | En mode Général, `_vault` est nul : la requête partait sans vault et le backend ignorait `extra_files`. Tests : `tests/frontend/ai.test.mjs` (+1) |
| *BUG-011* | Assistant IA : noms de modèles illisibles dans la liste déroulante | 🟢 corrigé | P2 | 📱 frontend | IA | `frontend/js/ai.js`, `frontend/style.css` | Ouvrir le menu modèle dans la barre latérale de l'assistant | Popover aligné à droite du déclencheur (`right:0`), largeur 340 px bornée à `100vw - 24px`, noms sur plusieurs lignes (`overflow-wrap:anywhere`, police 0,78 rem) + `title` | La liste s'ouvrait vers la droite et dépassait le bord de la sidebar ancrée à droite ; noms tronqués. Tests : `tests/frontend/ai.test.mjs` |
| *BUG-012* | Assistant IA : la commande `@` n'affiche aucun menu de sélection en mode Général | 🟢 corrigé | P1 | 📱 frontend | IA | `frontend/js/bookslm.js` | Assistant en mode Général (aucun vault), taper `@` | `_showMentionMenu` rend **toujours** le menu (même vide) et `_activeVault()` retombe sur `state.selectedContextVault` puis le premier `state.allVaults`, avant `vault=all` pour les requêtes | Sans vault résolu, l'ancien code masquait le menu et le backend ignorait le contexte. Tests : `tests/frontend/ai.test.mjs` (+1) |
| *BUG-013* | Assistant IA : les liens fichiers/répertoires des réponses affichent « Aucun vault actif pour ouvrir ce lien » | 🟢 corrigé | P1 | 📱 frontend | IA | `frontend/js/bookslm.js` | En mode Général, cliquer un lien de fichier/dossier dans une réponse | `_openFileLink()` et `_revealPath()` utilisent `_activeVault()` (vault de contexte → vault sélectionné → premier vault) au lieu de `_resolveVault()` seul | `_resolveVault()` est nul sans document ouvert ; les liens échouaient en mode Général. Tests : `tests/frontend/ai.test.mjs` (+1) |
| *BUG-014* | Assistant IA : les flèches ↑/↓ ne naviguent pas dans les menus `/` et `@` | 🟢 corrigé | P1 | 📱 frontend | IA | `frontend/js/bookslm.js` | Ouvrir `/`, puis ↑/↓ (y compris après un clic hors zone de texte) | Navigation gérée au niveau du **panneau en phase de capture** (`panel.addEventListener('keydown', ..., true)`), donc indépendante du focus | Le `keydown` n'était écouté que sur la zone de texte ; dès que le focus changeait, les flèches étaient ignorées. Tests : `tests/frontend/ai.test.mjs` (+1) |
| *BUG-015* | Assistant IA : la ligne sélectionnée des menus `/` et `@` est invisible au clavier | 🟢 corrigé | P2 | 📱 frontend | IA | `frontend/style.css` | Ouvrir `/` ou `@`, naviguer avec ↑/↓ | État `.active` en `--bg-hover` + barre d'accent à gauche (`inset 3px 0 0 var(--accent)`) au lieu de `--surface2` | `--surface2` est identique à `--bg-primary` (fond du menu) en thème sombre : la sélection ne se voyait pas. Idem pour la liste de modèles |
| *BUG-016* | Mobile : le bouton « mode lecture » flottant recouvre le bouton d'envoi de l'assistant IA | 🟢 corrigé | P2 | 📱 frontend | IA | `frontend/js/mobile-editor.js`, `frontend/style.css` | Mobile, sans fichier ouvert : ouvrir l'assistant puis constater que `📖` masque `✈️` | `updateReadingButtonVisibility()` n'affiche `#me-reading-btn` que si `state.currentPath` est défini **et** que le panneau assistant est fermé ; resynchronisé à chaque mutation de `#content-area` et sur `bookslm:opened`/`bookslm:closed` ; règle `.me-reading-btn[hidden]{display:none}` | Le bouton flottant (z-index 890) passait au-dessus du panneau plein écran mobile (z-index 100). Tests : `tests/frontend/mobile-editor.test.mjs` (+2) |
| *BUG-017* | Accueil mobile : les tuiles de « Favoris » et « Récents » ne s'affichent pas correctement en largeur (l'onglet « Partagés » s'affiche correctement) | 🟢 corrigé | P2 | 📱 frontend | IA | `frontend/style.css` | Mobile : page d'accueil → onglets Favoris / Récents | `frontend/style.css` : `min-width:0` + `overflow:hidden` sur `.dashboard-card` (et header/footer) → la grille 1 colonne ne déborde plus | Cause : titre/chemin `nowrap` forçaient la largeur mini de la carte |
| *BUG-018* | Éditeur mobile : mise en page inadaptée — boutons « Annuler » / « Sauvegarder » inaccessibles et barre d'outils IA non optimisée | 🟢 corrigé | P1 | 📱 frontend | IA | `frontend/style.css` | Mobile : ouvrir un fichier → barre d'outils de l'éditeur puis panneau assistant | `frontend/style.css` : en-tête mobile compact (marque/séparateur/espaceur/présence masqués, champs compressibles, libellé « Saved » masqué) + cibles tactiles `.ai-toolbar-btn` agrandies | Les 3 boutons d'action restent accessibles ; voir aussi #83 pour le ruban |
| *BUG-019* | Éditeur mobile : la barre d'outils du bas n'est pas toujours visible au-dessus du clavier quand le curseur n'est pas en bas de l'éditeur | 🟢 corrigé | P1 | 📱 frontend | IA | `frontend/js/mobile-editor.js`, `frontend/style.css` | Mobile : ouvrir un fichier, placer le curseur au milieu du document, afficher le clavier | `initKeyboardAnchor()` utilise `window.visualViewport` (`--kb-offset`) pour ancrer le ruban au-dessus du clavier ; `padding-bottom` sur `.cm-scroller` | Fonctionne quel que soit le défilement / la position du curseur |
| *BUG-020* | Éditeur mobile : deux barres de défilement verticales à droite de la page en mode édition | 🟢 corrigé | P3 | 📱 frontend | IA | `frontend/style.css` | Mobile : ouvrir l'éditeur et observer le bord droit | `frontend/style.css` : `.editor-body` en `overflow:hidden` + colonne flex ; seul `.cm-scroller` défile | Une seule barre de défilement |
| *BUG-021* | [🔴 CRITIQUE] XSS stocké via le rendu markdown (`escape=False`) | 🟢 corrigé | P0 | 🔐 sécurité | IA | `backend/services/sanitizer.py`, `backend/main.py` | Ouvrir une note contenant `<img src=x onerror=alert(1)>` | `backend/services/sanitizer.py` (whitelist stdlib) appliqué après `_add_heading_ids` dans `_render_markdown` ; tests `tests/test_security_hardening.py` | HTML brut d'un vault injecté dans le DOM → JS dans le navigateur de chaque utilisateur. DOMPurify client non ajouté (défense serveur suffisante) |
| *BUG-022* | [🔴 CRITIQUE] XSS stocké sur la page publique `/s/{token}` (title + frontmatter non échappés) | 🟢 corrigé | P0 | 🔐 sécurité | IA | `backend/main.py` | Partager une note dont le frontmatter contient un `title` avec `<script>` | `backend/main.py` : `html.escape()` sur `title`/frontmatter + JSON échappé (`\u003c`) pour le bloc `<script>` et `a.download` | `a.download="{title}.md"` se trouvait dans un bloc script → breakout JS. Test d'intégration dans `tests/test_security_hardening.py` |
| *BUG-023* | [🔴 CRITIQUE] Brute-force MFA sans rate-limit ni verrouillage | 🟢 corrigé | P0 | 🔐 sécurité | IA | `backend/auth/router.py` | `POST /api/auth/mfa/totp/verify` en boucle avec des codes aléatoires | `_enforce_mfa_rate_limit` + `_record_mfa_failure` sur `totp/verify`, `recovery`, `webauthn/verify` (IP + compte + lockout) | TOTP 6 chiffres brute-forceable ; le login était protégé, pas le second facteur |
| *BUG-024* | [🔴 CRITIQUE] Traversal inter-vaults : `startswith()` sans séparateur dans `resolve_safe_path` | 🟢 corrigé | P0 | 🔐 sécurité | IA | `backend/services/paths.py` | Avec les vaults `vault` et `vault-evil`, lire un fichier de `vault-evil` via `vault` | `_is_within()` par segments (`relative_to` + repli casse-insensible) ; tests `tests/test_security_hardening.py::TestPathIsolation` | Rupture d'isolation entre vaults (lecture / écriture / suppression) |
| *BUG-025* | [🟡 IMPORTANT] ReDoS : regex utilisateur appliquée au contenu en masse | 🟢 corrigé | P1 | 🔐 sécurité | IA | `backend/services/regex_safety.py`, `backend/search.py`, `backend/services/mutations.py` | Recherche avancée avec `^(a+)+$` sur un gros document | Validation de pattern (longueur ≤500, rejet quantificateurs imbriqués/backrefs), contenu tronqué (200k) et matchs plafonnés ; 400 sur pattern refusé | Le dry-run `replace_in_files` déclenchait la charge ; la validation est faite avant compilation |
| *BUG-026* | [🟡 IMPORTANT] Webhooks : SSRF (URL non validée) + secret en clair | 🟢 corrigé | P1 | 🔐 sécurité | IA | `backend/webhooks.py`, `data/webhook_secrets.json` | Configurer une URL vers `http://169.254.169.254` puis déclencher le webhook | Validation d'URL (HTTPS par défaut, IP privées/boucle bloquées), résolution DNS au dispatch, redirections non suivies ; secret dans `webhook_secrets.json` (0600) ou env `OBSIGATE_WEBHOOK_SECRET_<ID>` | `get_webhooks()` n'expose plus le secret (`has_secret`) ; options `OBSIGATE_WEBHOOK_ALLOW_HTTP`/`_ALLOW_PRIVATE` |
| *BUG-027* | [🟡 IMPORTANT] Sessions : refresh non rotatif, access token non révoqué au logout | 🟢 corrigé | P1 | 🔐 sécurité | IA | `backend/auth/jwt_handler.py`, `backend/auth/router.py`, `backend/auth/middleware.py` | Logout puis réutilisation de l'ancien access token | Rotation du refresh token (`create_refresh_token(remember=…)`) + révocation de l'ancien JTI ; access token révoqué au logout et vérifié dans le middleware | JTI persistés dans `data/revoked_tokens.json` |
| *BUG-028* | [🟡 IMPORTANT] Politique de mot de passe incohérente + sessions non invalidées au changement | 🟢 corrigé | P1 | 🔐 sécurité | IA | `backend/auth/password.py`, `backend/auth/router.py`, `backend/auth/user_store.py`, `backend/auth/middleware.py` | Créer un utilisateur avec `password:""` (admin) ; changer son mot de passe puis réutiliser l'ancien token | `validate_password_strength` (8–128) sur création/modif admin/changement ; `password_changed_at` invalide les jetons émis avant le changement ; `change-password` réémet une paire | Tests `TestPasswordPolicy` + `TestTokenInvalidation` |
| *BUG-029* | [🟡 IMPORTANT] Race read-modify-write sur `users.json` (perte de mises à jour) | 🟢 corrigé | P1 | ⚙️ backend | IA | `backend/auth/user_store.py` | Activer MFA et changer son mot de passe en parallèle | `threading.RLock` (`_users_lock`) autour de `create_user`, `update_user`, `delete_user`, `record_login_failure` | RLock réentrant car `record_login_failure` appelle `update_user` |
| *BUG-030* | [🟡 IMPORTANT] Adresse IP jamais consignée dans les audits (`_request_ip` toujours « unknown ») | 🟢 corrigé | P1 | ⚙️ backend | IA | `backend/services/net.py`, `backend/auth/middleware.py` | Lire le journal d'audit après un save/delete | `get_client_ip()` (`request.client.host`, `X-Forwarded-For` si `OBSIGATE_TRUST_PROXY=true`) injecté dans `current_user["_request_ip"]` | Test `TestClientIp` |
| *BUG-031* | [🟡 IMPORTANT] Rate-limiter en mémoire, mono-process, budget exclusivement IP | 🟢 corrigé | P1 | ⚙️ backend | IA | `backend/ratelimit.py`, `backend/auth/router.py` | Lancer plusieurs workers et observer le partage du compteur | Budget **par compte** (`is_account_rate_limited`/`record_account_*`) en plus de l'IP, intégré login + MFA ; limite mono-process documentée | Rotation d'IP neutralisée ; stockage partagé (Redis) hors périmètre |
| *BUG-032* | [🟡 IMPORTANT] Indexation : symlinks suivis (contenu hors vault indexé) + scan initial coûteux | 🟢 corrigé | P1 | ⚙️ backend | IA | `backend/indexer.py` | Placer un symlink dans le vault vers un dossier externe puis relancer l'index | `_scan_vault` réécrit avec `os.walk(followlinks=False)` + refus des symlinks sortant de la racine ; test `TestSymlinkIndexing` | Scan incrémental/index persistant : voir #86 (phase 3) |
| *BUG-033* | [🟡 IMPORTANT] Recherche classique et tool IA `search_fulltext` en O(N) sans inverted index | 🟢 corrigé | P1 | ⚙️ backend | IA | `backend/search.py`, `backend/tools/service.py` | `GET /api/search` sur un vault de 50 000 fichiers | `search()` récupère les candidats via l'inverted index (intersection des termes + expansion de préfixes), repli sur le scan pendant la construction | `search_fulltext` en bénéficie automatiquement |
| *BUG-034* | [🟡 IMPORTANT] CSP affaiblie (`'unsafe-inline'` + CDN distants) et token d'accès en sessionStorage | 🟢 corrigé | P1 | 🔐 sécurité | IA | `backend/main.py`, `frontend/js/auth.js`, `frontend/js/admin.js`, `frontend/js/sync.js` | Inspecter les en-têtes CSP ; lire sessionStorage en console | Token en mémoire + cookie HttpOnly (plus de `sessionStorage`) ; CSP durcie (`object-src 'none'`, `base-uri`, `form-action`, `frame-ancestors`). *Reste : migration nonce* | `'unsafe-inline'` conservé tant que les gestionnaires inline n'ont pas été convertis (résidu documenté) |
| *BUG-035* | [🔵 MINEUR] `secret_redactor` : faux positifs sur les hashs hex (git, SHA) | 🔴 ouvert | P2 | ⚙️ backend | IA | `backend/secret_redactor.py` | Lire une note contenant un commit git (40 caractères hexadécimaux) | Restreindre le périmètre de détection (contexte clé/token) + whitelist | Contenus mutilés dans les lectures et réponses IA |
| *BUG-036* | [🔵 MINEUR] Collab WebSocket : token en query string | 🔴 ouvert | P2 | ⚙️ backend | IA | `backend/collab.py` | Observer l'URL du websocket dans le trafic réseau | Passer le token en header / étape d'authentification initiale ; borner la taille des messages | Jeton visible dans les logs/proxys |
| *BUG-037* | [🔵 MINEUR] Compte « anonymous » administrateur si auth désactivée | 🔴 ouvert | P2 | 🔐 sécurité | IA | `backend/auth/middleware.py` | Démarrer avec l'authentification désactivée | Avertissement explicite au démarrage + refus de déploiement public sans auth | Comportement par conception mais risqué si mal configuré |
| *BUG-038* | [🔵 MINEUR] Argon2 à 64 MB par vérification : risque d'épuisement mémoire | 🔴 ouvert | P2 | 🔐 sécurité | IA | `backend/auth/password.py:8` | Lancer de nombreux `POST /api/auth/login` simultanés | Recalibrer (~19 MB, t=2, p=1, norme OWASP actuelle) + maintien du rate-limit | DoS mémoire possible sur les petites instances |
| *BUG-039* | [🔵 MINEUR] Enumération de comptes : 429 (verrouillé) vs 401 (inconnu) | 🔴 ouvert | P3 | 🔐 sécurité | IA | `backend/auth/router.py:120` | Tenter un login sur un compte verrouillé puis un nom inconnu | Répondre 401 uniforme avec un timing équivalent | Le statut HTTP distingue l'existence d'un compte |
| *BUG-040* | [🔵 MINEUR] Extraction PDF intégrale (100 ko) au scan de démarrage | 🔴 ouvert | P2 | ⚙️ backend | IA | `backend/indexer.py:465` | Démarrer sur un vault contenant de nombreux PDF | Analyser les PDF en tâche de fond / à la demande (lazy) | Ralentit fortement le démarrage et le rebuild d'index |
| *BUG-041* | [🟡 IMPORTANT] Assistant IA : échec sur un répertoire vide (« Aucun fichier markdown trouvé dans ce dossier ») au lieu de répondre | 🟢 corrigé | P1 | 📱 frontend + ⚙️ backend | IA | `backend/bookslm_routes.py`, `backend/bookslm.py`, `frontend/js/bookslm.js` | Ouvrir l'assistant sur un dossier vide puis envoyer une question | `_resolve_system_prompt` dégrade vers le prompt Général + bloc « Dossier vide » (plus de 404) ; contexte applicatif `app_context` enrichi (documents ouverts, répertoire, recherche, fichiers récents) | Le 404 bloquait toute la requête. Feature #88, fiche `docs/features/ai-app-context.md`. Tests : `tests/test_bookslm.py` (+3), `tests/frontend/ai.test.mjs` |
| *BUG-042* | [🟡 IMPORTANT] Assistant IA : liens de fichiers non fiables (« File not found: ») — pas de règle déterministe nom / dossier / chemin | 🟢 corrigé | P1 | 📱 frontend | IA | `frontend/js/bookslm.js` | Cliquer les liens de fichiers/dossiers dans une réponse de l'assistant (noms avec espaces et/ou accents, chemin préfixé par le nom du vault) | `_classifyPath` distingue `name` (copie presse-papiers) / `dir` (révélation arborescence) / `file` (ouverture) ; `_activatePath()` résout le chemin contre l'index du vault (exact → suffixe → basename unique) avant d'agir ; espaces + accents pris en charge (classes Unicode `\p{L}\p{N}\p{M}`, comparaison normalisée NFC, markdown `<…>`/`%20`, code inline, mentions brutes confirmées par l'index) ; `_splitVaultPrefix` retire un préfixe `Vault/…` et ouvre dans ce vault (`_fetchPathsForVault`) | Les liens morts ouvraient un fichier inexistant. Feature #88. Tests : `tests/frontend/ai.test.mjs` (+11) |
| *BUG-043* | [🟡 IMPORTANT] Assistant IA : la liste des fournisseurs de la barre latérale ne suit pas les ajouts/retraits de clés API dans la configuration du projet | 🟢 corrigé | P1 | 📱 frontend | IA | `frontend/js/ai.js`, `frontend/js/bookslm.js`, `frontend/js/config.js` | Ajouter (ou supprimer) une clé de fournisseur AI dans la configuration puis observer le menu Fournisseur de l'assistant sans recharger la page | Le picker lit `/api/ai/status` **une seule fois**, à sa construction, et le panneau de l'assistant est un singleton monté pour toute la session → liste figée. Nouveau `refreshAIPickers()` (exporté par `ai.js`) qui reconstruit chaque picker monté dans son emplacement `.ai-picker-slot` (conservé même sans fournisseur configuré, donc un premier fournisseur s'y monte aussi) ; appelé après `saveAIKeys()` et `deleteAIKey()` (`config.js`) ; une sélection dont le fournisseur n'est plus configuré est purgée de `obsigate_ai_picker` (retour au défaut + modèle effacé au lieu d'un nom fantôme) | Il fallait recharger la page pour voir un nouveau fournisseur (ou en voir disparaître un). Feature #82. Tests : `tests/frontend/ai.test.mjs` (+4) |
| *BUG-044* | [🟡 IMPORTANT] Capacités des modèles IA erronées : aucun modèle Mistral n'est détecté « Vision capable » (et des modèles texte sont annoncés « Embeddings ») | 🟢 corrigé | P1 | ⚙️ backend + 🔌 api | IA | `backend/model_capabilities.py`, `backend/provider_capabilities.py`, `backend/main.py`, `backend/ai_routes.py` | `curl -H "Authorization: Bearer $MISTRAL_API_KEY" https://api.mistral.ai/v1/models` puis `GET /api/ai/model-capabilities?provider=mistral&model=mistral-small-latest` | Nouveau calque **déclaratif** (`backend/provider_capabilities.py`) : les capacités publiées par le fournisseur (Mistral `capabilities`, OpenRouter `architecture`) sont lues et mises en cache par `GET /api/config/ai-models` ; elles **priment** sur la table statique, qui ne comble plus que les drapeaux non déclarés (et sert de repli hors ligne / sans clé). Table statique corrigée : familles vision Mistral (`ministral`, `magistral`, `mistral-small`, `mistral-medium`, `mistral-vibe-cli`, `labs-leanstral`), `mistral-ocr` = vision sans chat, défaut fournisseur Mistral sans `embeddings`. Tests : `tests/test_provider_capabilities.py`, `tests/test_model_capabilities.py::TestMistralFamilies`, `tests/test_ai_models.py::TestDeclaredCapabilities` | Le panneau de modèle par défaut et la bulle ⓘ n'affichaient **aucun** Mistral vision alors que l'API en déclare 28 ; seul le motif `pixtral` (retiré de l'API) matchait. Effet secondaire corrigé : `mistral-large-latest` / `codestral-latest` étaient annoncés « Embeddings » (défaut fournisseur). La porte vision de l'assistant (`bookslm_routes.py`) accepte désormais les images avec un modèle Mistral vision |
| *BUG-045* | [🟡 IMPORTANT] Éditeur « Editer » : deux barres de défilement superposées sur les documents longs | 🟢 corrigé | P1 | 📱 frontend | Éditeur | `frontend/style.css` | Ouvrir un fichier long (ex. IT/Docker Guide.md), cliquer Editer, mesurer `#editor-body` et `.cm-scroller` | `.editor-body-cm` gardait `overflow:auto` et un `.cm-editor{height:100%}` sous la rangée barre d'outils IA → le corps (toolbar+éditeur) ET le scroller CodeMirror débordaient simultanément. L'override global legacy `.cm-scroller{min-height:100%;overflow-y:auto!important}` aggravait. Passé en flex column : corps `overflow:hidden`, toolbar `flex:0 0 auto`, `.cm-editor` `flex:1 1 auto; height:auto`, seul le scroller défile ; override legacy retiré. Test : `tests/frontend/editor-inline.test.mjs` (+1) ; vérifié Playwright sur l'instance de test (un seul conteneur scrollable). |
| *BUG-046* | [🔴 BLOQUANT] Assistant IA : « Échec de l'action : [object Object] » à l'application d'un ajout de texte au document courant | 🟢 corrigé | P0 | 📱 frontend + ⚙️ backend | IA | `frontend/js/bookslm.js`, `backend/bookslm.py`, `backend/bookslm_routes.py` | Mode agent : demander d'ajouter du texte au document ouvert puis cliquer « Appliquer » | Trois causes : (1) continuation de confirmation avec `payload:null` → second « Appliquer » sans `message` → 422 ; (2) `new Error(detail)` sur un `detail` tableau d'objets FastAPI → « [object Object] » ; (3) prompt documents/directory sans nom de vault → le modèle inventait `"vault":"test"` → échec silencieux de l'outil. Nouveau `_responseError()` (aplatit tableau/objet), payload porté à la continuation, bloc « Ces documents appartiennent au vault « X » » + consigne outils d'écriture (`build_system_prompt(vault_name=...)`). `SW_VERSION` v20. Tests : `tests/test_bookslm.py::test_vault_name_guidance`, `tests/frontend/ai.test.mjs` (+2). |
| *BUG-047* | [🔴 BLOQUANT] La version affichée par l'application ne suit pas les livraisons : 66 commits livrés depuis v2.2.1 et l'UI/API restent bloquées sur `2.2.1` (et les numéros codés en dur divergent : `package.json` 1.0.0, desktop Tauri 2.0.0, `Dockerfile` 2.2.1, README 1.7.0) | 🟢 corrigé | P0 | ⚙️ build + 📄 docs | IA | `VERSION` (nouveau), `scripts/bump_version.py` (nouveau), `.githooks/prepare-commit-msg` + `.githooks/post-commit` (nouveaux), `scripts/install-hooks.sh` (nouveau), `backend/version.py`, `Dockerfile`, `docker-compose.yml`, `build.sh`, `.gitea/workflows/ci.yml`, `desktop/build.rs`, `tests/test_version.py` (nouveau) | `git tag -l \| tail -1` puis `python scripts/bump_version.py --print-version` ; `curl -s http://localhost:2020/api/health \| jq .version` | Le numéro provenait du **dernier tag git** et aucun tag n'était créé aux livraisons (`bump_version.sh` jamais appelé) → version figée, plus quatre numéros codés en dur ailleurs. Corrigé : **`VERSION` (racine) = source unique de vérité**, incrémentée automatiquement à chaque commit par le hook versionné `prepare-commit-msg` (SemVer : `!:`/`BREAKING CHANGE` → MAJEUR, `feat` → MINEUR, sinon CORRECTIF), tag `vX.Y.Z` créé par `post-commit` et publié au push (`push.followTags`) ; `bump_version.py` resynchronise `package.json`, desktop Tauri, ROADMAP, READMEs et fait la rotation du CHANGELOG dans le même commit ; backend, image Docker (`COPY VERSION`) et desktop lisent ce fichier. Contournement ponctuel : `SKIP_VERSION_BUMP=1`. | Garde-fou : `tests/test_version.py::TestRepoVersionAlignment` échoue dès qu'un dérivé diverge de `VERSION`. Vérifié : pytest complet vert, ruff/mypy 0, `/api/health` → `2.3.0` sur l'instance de test. |
| | | | | | | | | | | |
### TODOs techniques (améliorations / nouvelles tâches)
@@ -130,6 +177,35 @@ https://og.dracodev.net/api/file/Recettes/pdf/stream?path=98_Boite_Outils%2F98.2
| Date | ID(s) traité(s) | Action | Fichiers modifiés | Résumé | Statut après |
|---|---|---|---|---|---|
| *(exemple)* 2026-06-15 | BUG-001 | Correction | `frontend/app.js` | Réécriture de `renderFile()` pour préserver le DOM dashboard | 🟢 corrigé (en attente vérif) |
| 2026-09-09 | BUG-001, BUG-002 | Correction | `backend/main.py`, `frontend/excalidraw-editor.html`, `tests/test_pdf_stream.py` | BUG-001: Content-Disposition RFC 5987 (nom PDF accentué ne casse plus l'en-tête → plus de 500). BUG-002: suppression alias esm.sh (408 jotai) + React 19 cohérent + prop `excalidrawAPI` → Loading masqué, save OK. Vérifié: 534 tests backend verts + E2E navigateur. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-11 | BUG-003, BUG-004 | Correction | `backend/{main,indexer,export,pdf_reader,bookslm_routes}.py`, `backend/auth/router.py`, `.gitea/workflows/ci.yml`, `README.md`, `README.fr.md` | BUG-003: 33 erreurs mypy corrigées (annotations, gardes `None`, import `PROVIDERS` manquant → bug latent) + étape CI mypy rendue bloquante. BUG-004: lien `README.md` → `docs/CONTRIBUTING.md`. Vérifié: mypy 0 erreur, ruff OK, pytest 728 passed, frontend OK. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-11 | BUG-005 | Correction | `frontend/sw.js`, `frontend/index.html`, `frontend/js/sync.js`, `backend/main.py`, `.gitea/workflows/ci.yml`, `tests/frontend/sw.test.mjs`, `tests/test_api_main.py` | BUG-005: chargement mobile incomplet via Cloudflare. SW réécrit en **network-first** pour HTML/JS/CSS + caches versionnés ; précache corrigé (`/static/js/…`) ; kill-switch de session remplacé par une migration `localStorage` ponctuelle ; en-têtes `/static` passés de `immutable 1 an` à `no-cache` ; `index.html`/`manifest`/SPA en `no-cache` ; reload unique sur `controllerchange`. Vérifié: 754 tests backend, frontend 9+28+8 OK, ruff/mypy OK. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-11 | BUG-006 | Correction sécurité | `.env.example` | Clé API DeepSeek réelle exposée dans `.env.example` → remplacée par un placeholder. **Rotation de la clé à faire côté DeepSeek** (présente dans l'historique Git). | 🟢 corrigé (rotation à confirmer par l'utilisateur) |
| 2026-09-12 | BUG-007, BUG-008, #82 | Correction + refonte UI | `frontend/js/bookslm.js`, `frontend/js/ai.js`, `frontend/style.css`, `frontend/locales/{fr,en}.json`, `backend/bookslm_routes.py`, `tests/frontend/ai.test.mjs`, `tests/test_bookslm.py` | BUG-007 : motifs Unicode + recherche NFD insensible aux accents + navigation ↑/↓ fiabilisée (jeton de séquence, scrollIntoView). BUG-008 : contexte ad-hoc `@` pris en compte en mode Général (backend) + repli `vault=all`. #82 : section « Fournisseur & modèle » compacte (recherche modèle + bulle capacités ℹ️ au survol/clic/appui long). Vérifié : tests frontend 41/41, `tests/test_bookslm.py` 47 passed. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-12 | BUG-009 | Correction | `frontend/js/ai.js`, `frontend/js/i18n.js`, `frontend/sw.js`, `frontend/style.css`, `frontend/locales/{fr,en}.json`, `tests/frontend/ai.test.mjs` | BUG-009 : locale non content-hashée servie depuis le cache HTTP → clés i18n brutes ; passage en `cache:'no-store'` + `SW_VERSION` v4. Liste de modèles corrompue (OpenRouter, styles en cache) → styles critiques en ligne, options `display:block`, plafond 200 + indicateur. Vérifié : tests frontend 42/42. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-12 | BUG-010, BUG-011 | Correction | `frontend/js/bookslm.js`, `frontend/js/ai.js`, `frontend/style.css`, `tests/frontend/ai.test.mjs` | BUG-010 : capture du `vault` des résultats `@` + `_contextVault()` propagé aux requêtes `/context`/`/chat` (mode Général). BUG-011 : popover modèle aligné à droite, largeur 340 px, noms multi-lignes + `title`. Vérifié en navigateur (Playwright) et tests frontend 43/43. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-12 | BUG-012, #82 | Correction + UX | `frontend/js/bookslm.js`, `frontend/js/ai.js`, `frontend/style.css`, `frontend/sw.js`, `frontend/locales/{fr,en}.json`, `tests/frontend/ai.test.mjs` | BUG-012 : le menu `@` est toujours rendu et `_mentionVault()` retombe sur le vault de contexte puis le premier vault disponible. #82 : suppression des intitulés « Fournisseur & modèle » / « Fournisseur : », description du contexte Général déplacée en info-bulle, placeholder retiré, bouton Envoyer en emoji ✈️, indicateur d'activité (envoi/réception/outils/confirmation/succès/échec), `SW_VERSION` v5. Vérifié Playwright + tests frontend 45/45. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-12 | BUG-013, BUG-014 | Correction | `frontend/js/bookslm.js`, `frontend/index.html`, `frontend/sw.js`, `tests/frontend/ai.test.mjs` | BUG-013 : liens fichiers/dossiers via `_activeVault()` (vault de contexte → sélection → premier vault). BUG-014 : navigation ↑/↓ gérée au niveau du panneau en phase de capture. Purge des caches élargie (`SW_VERSION` v6, migration `obsigate-*`). Vérifié : tests frontend 47/47, pytest 930. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-12 | BUG-015, #82 | Correction + UX | `frontend/js/bookslm.js`, `frontend/style.css`, `tests/frontend/ai.test.mjs` | BUG-015 : sélection des menus `/` et `@` rendue visible (`--bg-hover` + barre d'accent à gauche) car `--surface2` = `--bg-primary`. #82 : retrait de l'indice clavier sous la zone de saisie. Vérifié Playwright (contraste `#1f2430` vs `#0f1117` + barre accent) et tests frontend 47/47. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-12 | #82 | Amélioration perf + UX | `backend/services/search.py`, `backend/main.py`, `frontend/js/bookslm.js`, `frontend/js/ui.js`, `frontend/style.css`, `tests/test_api_main.py`, `tests/frontend/ai.test.mjs` | #82 : endpoint `GET /api/vault/{vault}/paths` + préchargement et filtrage client du menu `@` (instantané) ; sélecteurs Fournisseur/Modèle agrandis (0,8 rem / 34 px) ; « 🧠 BooksLM » ajouté au menu contextuel de la racine des vaults. Vérifié : pytest 151 (ciblés) + ruff/mypy, tests frontend 49/49. | ✅ livré (en attente vérif utilisateur) |
| 2026-09-12 | #82 | UX | `frontend/js/ai.js`, `frontend/style.css`, `tests/frontend/ai.test.mjs` | Sélecteurs Fournisseur/Modèle alignés à droite dans la barre de l'assistant et réordonnés : capacité du modèle (ⓘ) → fournisseur → modèle. Bulle de capacités ouverte vers la droite (`left: 0`). Test JSDOM de l'ordre. Vérifié : tests frontend 49/49. | ✅ livré (en attente vérif utilisateur) |
| 2026-09-12 | BUG-016 | Correction | `frontend/js/mobile-editor.js`, `frontend/style.css`, `tests/frontend/mobile-editor.test.mjs`, `tests/e2e/mobile-editor.spec.js` | BUG-016 : le bouton « mode lecture » n'apparaît que si un fichier est ouvert et que l'assistant est fermé, sinon il recouvrait le bouton d'envoi sur mobile. Vérifié : tests frontend 24/24 (mobile) + suite JSDOM verte. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-13 | BUG-017, BUG-018, BUG-019, BUG-020, #83 | Enregistrement | `docs/ISSUES_TODOLIST.md`, `docs/ROADMAP.md` | Signalements mobile consignés : accueil (tuiles Favoris/Récents), éditeur (boutons Annuler/Sauvegarder inaccessibles + barre IA), visibilité de la barre d'outils au-dessus du clavier, double barre de défilement. Feature #83 ajoutée : barre d'outils d'édition mobile style Obsidian Android. | 🔴 ouvert (à traiter) |
| 2026-09-13 | Revue sécurité statique (BUG-021 → BUG-040, roadmap #84 → #87) | Enregistrement | `docs/ISSUES_TODOLIST.md`, `docs/ROADMAP.md` | Revue statique 2026-09-13 consignée : XSS markdown (`escape=False`) + page de partage publique, brute-force MFA, traversal par préfixe `paths.py`, ReDoS, SSRF webhooks, cycle de vie des sessions, politique de mot de passe, races `users.json`, audits IP, rate-limit, indexation symlinks, recherche O(N), CSP. BUG-021→BUG-040 ouverts au registre ; roadmap #84→#87 (consolidation/sécurité, refonte architecturale, performance, CI/CD). | 🔴 ouvert (à traiter) |
| 2026-09-13 | BUG-021 → BUG-034 (#84 phase 1) | Correction | `backend/services/sanitizer.py`, `backend/services/paths.py`, `backend/services/net.py`, `backend/services/regex_safety.py`, `backend/main.py`, `backend/search.py`, `backend/webhooks.py`, `backend/indexer.py`, `backend/ratelimit.py`, `backend/auth/{router,user_store,password,jwt_handler,middleware}.py`, `frontend/js/{auth,admin,sync}.js`, `tests/test_security_hardening.py`, `tests/test_auth_api.py` | Sanitizer XSS serveur (markdown + page de partage), rate-limit/lockout MFA, isolation vaults par segments, caps regex (ReDoS), SSRF webhooks + secrets externalisés, rotation/révocation des jetons, politique de mot de passe + invalidation des sessions, verrous `users.json`, IP réelle dans les audits, rate-limit par compte, symlinks d'index ignorés, recherche simple via inverted index, token en mémoire + cookie HttpOnly + CSP durcie. Vérifié : pytest 961 passed / 6 skipped, ruff 0, mypy 0, tests frontend 36 modules + 9 JSDOM suites verts. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-13 | BUG-017 → BUG-020, #83 | Correction + feature | `frontend/js/mobile-editor.js`, `frontend/style.css`, `frontend/index.html`, `frontend/locales/{fr,en}.json`, `tests/frontend/mobile-editor.test.mjs`, `tests/e2e/mobile-editor.spec.js` | BUG-017 : cartes de tableau de bord compressibles (`min-width:0`) ; BUG-018 : en-tête éditeur mobile compact + barre IA tactile ; BUG-019 : ancrage du ruban via `visualViewport` ; BUG-020 : suppression de la double scrollbar. #83 : ruban d'édition horizontal défilable style Obsidian (commandes étendues + personnalisation ⚙ persistée). Vérifié : tests frontend 35/35 (mobile) + 50/50 (IA) + 9 JSDOM suites. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-14 | BUG-042 (complément) | Correction | `frontend/js/bookslm.js`, `tests/frontend/ai.test.mjs`, `docs/features/ai-app-context.md` | Prise en charge des **espaces** dans les noms de fichiers et chemins des liens de l'assistant : cibles markdown avec espaces / `<…>` / `%20` décodés, code inline reconnu (`_looksLikePath` élargi, rejette toujours les extraits de code), mentions brutes liées uniquement si présentes dans l'index du vault (`_linkifySpacePaths` + `_confirmPathInCache`, plus long suffixe aligné sur un mot pour ne pas avaler le mot de prose précédent). Vérifié : tests frontend 57/57 (IA) + 9 suites JSDOM, validate-imports 36 modules, pytest 963 passed / 6 skipped. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-14 | BUG-041, BUG-042, #88 | Correction + feature | `frontend/js/bookslm.js`, `backend/bookslm.py`, `backend/bookslm_routes.py`, `frontend/locales/{fr,en}.json`, `tests/test_bookslm.py`, `tests/frontend/ai.test.mjs`, `docs/features/ai-app-context.md` | BUG-041 : contexte de dossier vide → dégradation gracieuse vers le prompt Général + bloc « Dossier vide » (fin du 404). BUG-042 : liens de fichiers déterministes (nom → presse-papiers, dossier → arborescence, chemin → viewer) + résolution du chemin contre l'index du vault avant ouverture. #88 : `app_context` (documents ouverts, répertoire/vault courants, recherche + résultats affichés) et fichiers récemment modifiés injectés dans le prompt Général. Vérifié : pytest 963 passed / 6 skipped, ruff 0, mypy 0, tests frontend 52/52 + validate-imports 36 modules. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-14 | BUG-042 (accents + préfixe vault) | Correction | `frontend/js/bookslm.js`, `tests/frontend/ai.test.mjs`, `docs/features/ai-app-context.md` | **Caractères accentués** : classes de caractères Unicode (`\p{L}\p{N}\p{M}`) pour `PATH_WITH_DIR_RE` et `PATH_NAME_RE` (`_looksLikePath`), comparaison normalisée `NFC` via `_normKey()` → une mention décomposée (`e` + accent combinant, style macOS) correspond à une entrée d'index précomposée, et les liens sans espace mais accentués sont enfin produits. **Préfixe de vault** : `_splitVaultPrefix()` retire un premier segment égal à un vault connu (`TestVault/Recettes/Pizza Maison.md`) ; `_fetchPathsForVault()` interroge l'index de ce vault sans écraser le cache du vault actif ; `_openFileLink`/`_revealPath` reçoivent le vault cible ; repli « retirer le premier segment » si le préfixe est inconnu. Vérifié : tests frontend 62/62 (IA) + 9 suites JSDOM, validate-imports 36 modules, pytest 963 passed / 6 skipped, et vérification contre l'instance live (données réelles accentuées/espacées : `Recettes/Préparation.md`, `Recettes/Pâtes carbonara.md`, `Recettes/Pizza Maison.md`) — 8/8 contrôles. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-14 | BUG-043, #82 | Correction | `frontend/js/ai.js`, `frontend/js/bookslm.js`, `frontend/js/config.js`, `tests/frontend/ai.test.mjs`, `docs/features/ai-provider-picker.md` | La liste des fournisseurs de la barre latérale de l'assistant suit désormais la configuration du projet : nouveau `refreshAIPickers()` (`ai.js`) qui reconstruit chaque picker monté dans son emplacement `.ai-picker-slot` (host conservé dans la barre de l'assistant, montage par `replaceChildren`), appelé après `saveAIKeys()` et `deleteAIKey()` (`config.js`) ; slot préservé même sans fournisseur configuré (le premier fournisseur ajouté s'y monte) ; sélection persistée d'un fournisseur non configuré purgée de `obsigate_ai_picker` (retour au défaut, modèle effacé). Vérifié : tests frontend 66/66 (IA) + 9 suites JSDOM vertes, validate-imports 36 modules, pytest 963 passed / 6 skipped, ruff OK, et vérification en navigateur (Playwright) sur l'instance de test : ajout de `nvidia` → fournisseur visible sans rechargement, retrait → disparition + sélection réinitialisée (`{"provider":null,"model":null}`), un seul montage du picker dans son host. CI Gitea verte (lint, test, security, build, e2e). |
| 2026-09-15 | BUG-044 | Correction | `backend/provider_capabilities.py` (nouveau), `backend/model_capabilities.py`, `backend/main.py`, `backend/ai_routes.py`, `tests/test_provider_capabilities.py` (nouveau), `tests/test_model_capabilities.py`, `tests/test_ai_models.py`, `docs/features/ai-provider-picker.md`, `CHANGELOG.md` | BUG-044 : les capacités des modèles IA sont désormais lues chez le fournisseur quand il les publie (Mistral `capabilities`, OpenRouter `architecture`, détection par forme du payload), mises en cache par `GET /api/config/ai-models` (aucune requête supplémentaire) et prioritaires sur la table statique qui ne comble plus que les drapeaux non déclarés (repli hors ligne / sans clé). Table statique corrigée : familles vision Mistral (`ministral`, `magistral`, `mistral-small`, `mistral-medium`, `mistral-vibe-cli`, `labs-leanstral`), `mistral-ocr` = vision sans chat, défaut fournisseur Mistral sans `embeddings` (mistral-large / codestral n'étaient plus des « embedders »). Vérifié : diagnostic live avant/après sur l'API Mistral (28 modèles vision déclarés, **0** détectés avant → **28** après, 0 écart dans les deux sens), pytest 1007 passed / 6 skipped, ruff 0 (backend), mypy 0 (68 fichiers), tests frontend (unit 9/9, IA 66/66, validate-imports 37 modules), et vérification sur l'instance de test reconstruite (`obsigate-test`, http://localhost:2020) via les deux endpoints du panneau : `mistral-small/medium-latest`, `ministral-8b-latest`, `magistral-small-latest` → `chat,vision` ; `mistral-ocr-latest` → `vision` seul ; `mistral-large-latest`/`codestral-latest` → `chat` ; `mistral-embed` → `embeddings` seul ; `deepseek-chat` (fournisseur muet) inchangé. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-16 | BUG-045, BUG-046 | Correction | `frontend/style.css`, `frontend/js/bookslm.js`, `backend/bookslm.py`, `backend/bookslm_routes.py`, `frontend/sw.js`, `tests/test_bookslm.py`, `tests/frontend/ai.test.mjs`, `tests/frontend/editor-inline.test.mjs`, `CHANGELOG.md` | BUG-045 : fin de la double barre de défilement de l'éditeur « Editer » — `#editor-body` en flex column `overflow:hidden`, `.cm-editor` `flex:1 1 auto`, seul le scroller CodeMirror défile ; suppression de l'override global `.cm-scroller` (min-height/overflow-y !important). BUG-046 : « Échec de l'action : [object Object] » à l'application d'un ajout de texte au document courant — (1) `_responseError()` aplatit le `detail` 422 (tableau d'objets FastAPI) en message lisible, (2) la continuation de confirmation hérite du payload d'origine (le second « Appliquer » ne POSTe plus sans `message`), (3) le prompt documents/dossier nomme le vault et enjoint les outils d'écriture d'utiliser ce nom exact (le modèle inventait `"vault":"test"` → échec silencieux). `SW_VERSION` v20. Vérifié : pytest 1038 passed / 6 skipped, ruff 0 (backend), tests frontend IA 84/84, editor-inline 25/25, validate-imports 38 modules ; reproduction et validation Playwright sur l'instance de test (un seul conteneur scrollable après correctif). | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-16 | #93 (complément) | Correction | `frontend/js/editor-inline.js`, `frontend/js/utils.js`, `frontend/js/viewer.js`, `frontend/js/ui.js`, `frontend/js/pane-manager.js`, `frontend/js/sync.js`, `tests/frontend/editor-inline.test.mjs`, `docs/features/editeur-inline.md`, `docs/CONTRIBUTING.md`, `CHANGELOG.md` | Garde-fous de **session d'édition inline** (#93) : l'activation d'un onglet (`TabManager.activate`, `PaneTabManager.activate`) appelle `detachInlineEditor()` avant de vider la zone de contenu, et l'événement SSE `index_updated` sur le fichier affiché recharge le **tampon de l'éditeur** (`reloadExternalWrite()`) au lieu de re-rendre la vue lecture — ces deux chemins détruisaient la session CodeMirror/Forge en cours. Nouveau helper `queryEditor()` (l'en-tête/pied/marque voyagent avec le conteneur en mode inline, `modal.querySelector()` les manquait) ; `closeEditor()` remet le conteneur dans la modale avant de restaurer en-tête/pied/marque. `docs/CONTRIBUTING.md` documente `scripts/install-hooks.sh` (hooks de versionnage) dans la mise en place d'un clone. Vérifié : validate-imports 38 modules, unit 9/9, suites JSDOM (editor-inline 25/25, pane-manager, mobile-editor 35/35, toolbar-order, ai 84/84, sw 8/8, collab 10/10, semantic-search 4/4, desktop 23, plugins 21, excalidraw-viewer) toutes vertes, pytest 1082 passed / 6 skipped. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-16 | BUG-047 | Correction + build | `VERSION` (nouveau), `scripts/bump_version.py` (nouveau), `.githooks/prepare-commit-msg` + `.githooks/post-commit` (nouveaux), `scripts/install-hooks.sh` (nouveau), `scripts/bump_version.sh`, `backend/version.py`, `Dockerfile`, `docker-compose.yml`, `build.sh`, `.gitea/workflows/ci.yml`, `desktop/build.rs`, `package.json`, `README.md`, `README.fr.md`, `docs/ROADMAP.md`, `docs/DELIVERY_WORKFLOW.md`, `docs/DEVELOPMENT_AND_RELEASES.md`, `tests/test_version.py` (nouveau) | **Version alignée de bout en bout.** Cause : la version affichée venait du dernier **tag git** et aucun tag n'était créé aux livraisons → 66 commits livrés mais UI/API figées sur `2.2.1` ; quatre autres numéros codés en dur dérivaient (`package.json` 1.0.0, desktop 2.0.0, `Dockerfile` 2.2.1, README 1.7.0). Nouveau modèle : **`VERSION` (racine) = source unique de vérité** (`MAJEUR.MINEUR.CORRECTIF`), incrémentée **automatiquement au commit** par le hook versionné `prepare-commit-msg` (SemVer depuis le message : `!:`/`BREAKING CHANGE` → MAJEUR, `feat` → MINEUR, sinon CORRECTIF ; aucun bump pour merge/revert/`chore(release)`/amend) qui resynchronise `package.json`, desktop Tauri, ROADMAP, READMEs et publie la section `[Unreleased]` du CHANGELOG en `[X.Y.Z] — date` ; `post-commit` rattache ces fichiers au commit qui vient d'être créé (amend immédiat, le commit n'étant pas encore poussé) puis crée le tag `vX.Y.Z`, publié au push (`push.followTags`). `bump_version.py` (avec `--dry-run`, `--set`, `--major\|--minor\|--patch`, `--print-version`) reste utilisable à la main ; `scripts/install-hooks.sh` installe les hooks. Côté build : Docker `COPY VERSION` (plus d'`ARG VERSION` ni d'env compose codés en dur), `build.sh` et CI lisent `VERSION`, `desktop/build.rs` aussi. Vérifié : `tests/test_version.py` (46 tests, dont le garde-fou `TestRepoVersionAlignment` : VERSION ↔ package.json ↔ desktop ↔ CHANGELOG ↔ ROADMAP ↔ READMEs ↔ pipeline sans numéro codé en dur), pytest complet, ruff/mypy 0, `/api/health` → `2.3.0` sur l'instance de test reconstruite. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-16 | #94, #95, #96, #97 | Feature | `backend/ai_history.py` (nouveau), `backend/bookslm_routes.py`, `frontend/js/bookslm.js`, `frontend/js/config.js`, `frontend/index.html`, `frontend/style.css`, `frontend/locales/{fr,en}.json`, `tests/test_bookslm.py`, `tests/frontend/ai.test.mjs`, `docs/features/ai-assistant-history.md` (nouvelle), `docs/ROADMAP.md`, `CHANGELOG.md` | **Assistant IA** : #95 historique **permanent** (backend `data/ai_history/{user}.json`, cap 200, endpoints CRUD `/api/ai/bookslm/history[…]` résumés/full, sync frontend debounced 600 ms + repli localStorage + migration des clés legacy `bookslm-sessions-*`/`bookslm-history-*`, événement `bookslm:history-updated`) ; #96 onglet sidebar `#sidebar-tab-ai` (`messages-square`) + panneau `#sidebar-panel-ai` (liste chronologique, ouverture via `openWithSession`) ; #97 bouton **« + »** remplaçant « Attach an image » + panneau modulaire `.bookslm-ext-menu` (registre `_extensions` : Fichiers, Image, Contextes, Skills, Deep Research = mode agent + prompt, Recherche web & Canva en « Bientôt ») ; #94 bouton d'envoi circulaire + icône Lucide `arrow-up`. Vérifié : pytest 1088 passed / 6 skipped, ruff 0, mypy 0 (71 fichiers), tests frontend IA 84/84, unit 9/9, validate-imports 38 modules. | 🟢 corrigé (en attente vérif utilisateur) |
| 2026-09-16 | BUG-048, #98 | Correction + feature | `frontend/js/bookslm.js`, `frontend/js/config.js`, `frontend/js/sidebar.js`, `frontend/style.css`, `frontend/locales/{fr,en}.json`, `.gitea/workflows/ci.yml`, `tests/frontend/ai.test.mjs`, `tests/frontend/ai-sidebar.test.mjs` (nouveau), `docs/features/ai-assistant-history.md`, `docs/ROADMAP.md`, `CHANGELOG.md` | **BUG-048** : les entrées « Contextes » et « Skills » du menu « + » ouvraient bien leur menu (`@` / `/`), mais le clic remontait au gestionnaire du panneau qui annulait le rendu asynchrone → menu jamais affiché ; correction par `e.stopPropagation()` sur les entrées du panneau `.bookslm-ext-menu`. **#98** : la barre de filtrage de la sidebar agit désormais sur l'onglet « Historique IA » — `filterAIHistory()` (config.js) filtre par titre, aperçu, répertoire, contexte ou libellé de mode, insensible casse/accents (`_aiNorm`), cache sessions `_aiSessionsCache`, message « aucune correspondance » (`bookslm.history_no_match`) dans la liste et placeholder dédié (`sidebar.filter_ai`) ; `initSidebarFilter` (sidebar.js) route saisie/touche casse/bouton « × » vers `filterAIHistory` quand l'onglet IA est actif ; chaque entrée du panneau « + » porte l'icône Lucide `plus`. Vérifié : tests frontend IA 87/87 (+3), nouvelle suite `ai-sidebar` 6/6, unit 9/9, 9 suites JSDOM vertes, validate-imports 38 modules, pytest / ruff / mypy inchangés (aucune modification backend). | 🟢 corrigé (en attente vérif utilisateur) — CI Gitea verte (lint, test, security, build, e2e) pour v2.5.0 (run #1511) |
| 2026-09-16 | BUG-049, #99, #100 | Correction + feature | `frontend/style.css`, `frontend/js/config.js`, `frontend/js/viewer.js`, `frontend/js/sidebar.js`, `frontend/js/bookslm.js`, `frontend/locales/{fr,en}.json`, `.gitea/workflows/ci.yml`, `tests/frontend/ai.test.mjs`, `tests/frontend/sidebar-filters.test.mjs` (nouveau), `docs/features/sidebar-filters.md` (nouvelle), `docs/features/ai-assistant-history.md`, `docs/ROADMAP.md`, `CHANGELOG.md` | **BUG-049** : icône du bouton « + » de l'assistant invisible — la règle générique `.bookslm-input-area button` (spécificité supérieure) imposait `padding: 8px 16px` sur un bouton `width: 32px` ⇒ largeur de contenu nulle ⇒ SVG `width: 0px` ; sélecteur porté à `.bookslm-input-area button.bookslm-btn-plus` (+ `:hover`), vérifié en navigateur (Playwright : SVG 0 px → 18 px). **#99** : la barre de filtrage de la sidebar agit désormais sur les vues **Récents** (`filterRecentFiles`, titre/chemin/vault/aperçu/tags) et **Sauvegardes** (`filterSavedSearches`, cumulable avec les pills type), insensible casse/accents (`_sidebarNorm`/`_savedNorm`), message d'absence de résultat (`sidebar.no_results`) et placeholders dédiés (`sidebar.filter_recent`, `sidebar.filter_saved`) ; `initSidebarFilter` refactoré en `routeFilter`/`routeClear` couvrant les 5 onglets. **#100** : « Deep Research » ajoute une **pastille** `.bookslm-chip-deep-research` (au lieu d'injecter la directive dans le composeur), active le mode Agent et injecte la directive au moment de l'envoi. Vérifié : tests frontend IA 88/88 (+1), `sidebar-filters` 8/8 (nouveau), `ai-sidebar` 6/6, unit 9/9, 9 suites JSDOM vertes, validate-imports 38 modules, vérification navigateur du bouton « + » ; backend inchangé (pytest / ruff / mypy valides). | 🟢 corrigé (en attente vérif utilisateur) |
---
+182
View File
@@ -0,0 +1,182 @@
# ObsiGate — Guide MCP (Model Context Protocol)
> **Statut :** livré (#79 phase E + F) · **Dernière mise à jour :** 2026-09-11
> **Voir aussi :** [AI_ARCHITECTURE_GUIDE.md](./AI_ARCHITECTURE_GUIDE.md) ·
> [features/ai-tools-mcp.md](./features/ai-tools-mcp.md) · [ROADMAP.md](./ROADMAP.md)
ObsiGate expose ses vaults à des **clients MCP externes** (Claude Desktop, Cursor,
tout client compatible MCP) via un serveur **Streamable HTTP** monté sur `/mcp`.
Les outils sont les **mêmes** que ceux de l'assistant in-app : la couche
`backend/tools/` est la source unique de vérité.
---
## 1. Prérequis
1. Une instance ObsiGate accessible (locale ou distante).
2. Un **jeton JWT** valide (`Authorization: Bearer <token>`), obtenu via
`POST /api/auth/login` (ou une clé API). Le jeton porte les permissions par
vault de l'utilisateur — l'autorisation MCP réutilise `get_current_user`.
3. Si l'authentification est désactivée (`OBSIGATE_AUTH_ENABLED=false`), le
serveur MCP accepte un utilisateur anonyme disposant de tous les vaults.
> Le transport `stdio` n'est **pas** encore supporté ; utilisez le transport
> HTTP (un pont local type `mcp-remote` si votre client ne gère pas nativement
> le Streamable HTTP distant).
---
## 2. Endpoint & protocole
| Élément | Valeur |
|---|---|
| URL | `https://<obsigate>/mcp` |
| Transport | Streamable HTTP (`POST` JSON-RPC 2.0, `Accept: application/json, text/event-stream`) |
| Auth | `Authorization: Bearer <JWT>` |
| Protocole MCP | `2025-03-26` (négocié à l'`initialize`) |
| Réponses | JSON (`json_response=True`) |
Handshake minimal :
```bash
curl -sS https://obsigate.example/mcp \
-H "Authorization: Bearer $TOKEN" \
-H "Accept: application/json, text/event-stream" \
-H "Content-Type: application/json" \
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{
"protocolVersion":"2025-03-26","capabilities":{},
"clientInfo":{"name":"curl","version":"1.0"}}}'
```
La réponse contient l'en-tête `Mcp-Session-Id` à réutiliser pour les appels
suivants (`tools/list`, `tools/call`, `resources/read`, …).
---
## 3. Configuration des clients
### Claude Desktop (via pont `mcp-remote`)
```json
{
"mcpServers": {
"obsigate": {
"command": "npx",
"args": [
"-y", "mcp-remote",
"https://obsigate.example/mcp",
"--header", "Authorization: Bearer ${OBSIGATE_TOKEN}"
],
"env": { "OBSIGATE_TOKEN": "eyJ..." }
}
}
}
```
### Cursor
`.cursor/mcp.json` :
```json
{
"mcpServers": {
"obsigate": {
"url": "https://obsigate.example/mcp",
"headers": { "Authorization": "Bearer eyJ..." }
}
}
}
```
---
## 4. Primitives exposées
### 4.1 Tools
Les outils de **lecture/recherche** sont exposés directement. Les outils
**d'écriture/destructifs** sont exposés via une paire **two-step** :
`propose_<tool>` (aperçu + jeton de confirmation, aucune modification) puis
`apply_<tool>` (consomme le jeton et exécute).
| Catégorie | Outils |
|---|---|
| Vaults / navigation | `list_vaults`, `list_directory`, `list_all_files` |
| Lecture | `read_file`, `read_file_raw`, `get_backlinks`, `list_backups`, `diff_backup`, `get_graph` |
| Recherche | `search_fulltext`, `search_advanced`, `search_paths`, `list_tags`, `suggest_tags`, `list_recent` |
| Écriture (propose/apply) | `create_file`, `create_directory`, `edit_file`, `append_to_file`, `restore_backup` |
| Destructif (propose/apply) | `rename_file`, `rename_directory`, `move_path`, `replace_in_files`, `delete_file`, `delete_directory` |
Flux d'une mutation :
```text
1. tools/call { name: "propose_edit_file",
arguments: { vault, path, content } }
→ { tool, arguments, diff, confirmation_token, expires_in }
2. (l'utilisateur / l'agent valide)
3. tools/call { name: "apply_edit_file",
arguments: { confirmation_token } }
→ { ok: true, data: { ... } }
```
Le jeton est **signé (JWT), à usage unique et à durée de vie limitée**
(`OBSIGATE_MCP_CONFIRMATION_TTL`, défaut 300 s). Un rejeu renvoie
`token_reused`.
### 4.2 Resources
| URI | Contenu |
|---|---|
| `vault://<name>` | Vault accessible (métadonnées, nombre de fichiers) |
| `vault://<name>/<path>` | Contenu d'un fichier (lecture seule, **secrets redactés**) |
### 4.3 Prompts
`summarize-directory`, `generate-note`, `find-related`.
---
## 5. Sécurité
- **Permissions par vault** : `check_vault_access` est appliqué à chaque outil
et chaque resource ; un utilisateur ne voit que ses vaults.
- **Anti path-traversal** : `resolve_safe_path` rejette tout chemin hors du vault.
- **Confirmation two-step** pour toute mutation (jeton signé, usage unique).
- **Toggle par vault** `aiDestructiveTools` (défaut : activé) : le désactiver
bloque rename/move/replace/delete tout en laissant create/edit/append.
- **Backup automatique** avant chaque opération destructive.
- **Rate limiting** : par jeton et par outil
(`OBSIGATE_TOOL_RATE_LIMIT`, `OBSIGATE_TOOL_RATE_LIMIT_PER_TOOL`,
`OBSIGATE_TOOL_RATE_WINDOW`). Une limite dépassée renvoie le code
`rate_limited`.
- **Redaction des secrets** : les résultats d'outils (lectures, diffs,
extraits de recherche) sont nettoyés avant tout retour au client.
- **Audit** : chaque appel est journalisé (`data/audit.log`, action
`ai_tool_call`) avec arguments sensibles résumés.
### Variables d'environnement
| Variable | Défaut | Rôle |
|---|---|---|
| `OBSIGATE_MCP_CONFIRMATION_TTL` | `300` | Durée de vie (s) des jetons de confirmation |
| `OBSIGATE_TOOL_RATE_LIMIT` | `60` | Appels d'outils max par identité et par fenêtre |
| `OBSIGATE_TOOL_RATE_LIMIT_PER_TOOL` | = global | Appels max par outil et par fenêtre |
| `OBSIGATE_TOOL_RATE_WINDOW` | `60` | Longueur de la fenêtre (s) |
| `BOOKSLM_MAX_TOOL_CALLS` | `25` | Quota d'appels d'outils par run d'agent |
| `BOOKSLM_MAX_TOOL_READ_BYTES` | `200000` | Taille max renvoyée par `read_file` |
---
## 6. Dépannage
| Symptôme | Cause probable / remède |
|---|---|
| `401 Authentification requise` | En-tête `Authorization: Bearer` absent ou jeton expiré |
| `vault_access_denied` | Le jeton n'a pas accès à ce vault (`vaults` / `_token_vaults`) |
| `destructive_tools_disabled` | `aiDestructiveTools=false` pour ce vault |
| `confirmation_required` | Appeler d'abord `propose_<tool>` puis `apply_<tool>` |
| `token_reused` / `invalid_confirmation` | Jeton déjà consommé ou expiré → refaire un `propose_` |
| `rate_limited` | Quota dépassé ; respecter `retry_after` |
| Le client ne se connecte pas | Vérifier le transport Streamable HTTP / le pont `mcp-remote` |
+215
View File
@@ -0,0 +1,215 @@
# Plugin System — ObsiGate #61
## Overview
ObsiGate's plugin system enables extending the application with custom
renderers, search filters, and editor actions — all sandboxed for security.
Plugins run in a Web Worker sandbox and communicate with the main app via
structured `postMessage`. No plugin can access the DOM, localStorage, or
make network requests without explicit permission.
## Quick Start
### Installing a Plugin
1. Go to **Settings > Plugins** in the sidebar
2. Click **Install Plugin**
3. Upload a `.zip` file or a directory containing:
- `plugin.json` — the manifest
- Your main JS file (entry point specified in `main`)
### Creating a Plugin
Use the **View Template** button in Settings > Plugins to get started.
## Manifest (plugin.json)
```json
{
"name": "my-plugin",
"version": "1.0.0",
"description": "A brief description of what this plugin does",
"author": "your-name",
"main": "index.js",
"hooks": {
"onFileRender": "renderFile",
"onSearchFilter": "filterResults",
"onEditorAction": "handleAction"
},
"permissions": ["read_files"],
"min_obsigate_version": "2.1.0",
"license": "MIT",
"homepage": "https://example.com",
"repository": "https://github.com/user/plugin"
}
```
### Required Fields
| Field | Type | Description |
|---------------|--------|-------------------------------------------------|
| `name` | string | Lowercase alphanumeric + hyphens (e.g. `my-plugin`) |
| `version` | string | Semantic version (e.g. `1.0.0`, `1.0.0-beta.1`) |
| `description` | string | Brief description |
| `author` | string | Author name |
| `main` | string | Entry point file (relative to plugin root) |
### Optional Fields
| Field | Type | Default | Description |
|------------------------|--------|-------------|------------------------------------------|
| `hooks` | object | `{}` | Map of hook name → handler function name |
| `permissions` | array | `[]` | Required permissions |
| `min_obsigate_version` | string | `"2.1.0"` | Minimum ObsiGate version |
| `license` | string | `"MIT"` | License identifier |
| `homepage` | string | `null` | Plugin homepage URL |
| `repository` | string | `null` | Source repository URL |
## Available Hooks
| Hook | When it fires | Handler signature |
|-------------------|-----------------------------------|--------------------------------|
| `onFileRender` | Before a file is rendered | `(ctx) → transformed ctx` |
| `onSearchFilter` | During search result filtering | `(results) → filtered results` |
| `onEditorAction` | When editor action is triggered | `(action, state) → result` |
| `onSidebarItem` | Sidebar item is rendered | `(item) → enhanced item` |
| `onFileCreate` | After a file is created | `(file) → void` |
| `onFileDelete` | After a file is deleted | `(file) → void` |
| `onVaultMount` | When a vault is mounted | `(vault) → void` |
## Permissions
Plugins must declare the permissions they need. The sandbox enforces these
restrictions at runtime.
| Permission | Description |
|-----------------------|--------------------------------------------------|
| `read_files` | Read file contents from the vault |
| `write_files` | Write/create files in the vault |
| `read_vault_metadata` | Access vault configuration and metadata |
| `network_request` | Make HTTP requests to external services |
| `ui_notify` | Show toast notifications in the UI |
| `access_clipboard` | Read/write to the system clipboard |
## Security Model
### Sandbox Architecture
```
┌─────────────────────────────────┐
│ Main App (browser context) │
│ │
│ PluginManager │
│ ├── install/uninstall │
│ ├── enable/disable │
│ └── hook dispatch │
│ │ postMessage (C3) │
│ ▼ │
│ ┌─────────────────────────┐ │
│ │ Web Worker Sandbox │ │
│ │ (no DOM, no localStorage│ │
│ │ no network by default) │ │
│ │ │ │
│ │ Plugin code executes │ │
│ │ here with structured │ │
│ │ message passing only │ │
│ └─────────────────────────┘ │
└─────────────────────────────────┘
```
### Security Guarantees
- **C1**: Manifest validation (name format, semver, allowed hooks/permissions)
- **C2**: Vault isolation — plugins are scoped to a single vault
- **C3**: Web Worker sandbox — no DOM, no `localStorage`, no `importScripts`
- **C4**: ZIP validation — path traversal, file count, size limits
- **C5**: Directory validation — manifest + entry point presence
- **C6**: Permission enforcement at sandbox boundary
### Limits
- Max 50 plugins per vault
- Max 500 KB per plugin file
- Max 100 files per plugin ZIP
- No dynamic `import()` or `eval()`
## Plugin Storage
Plugins are installed under `<vault>/.obsigate-plugins/<plugin-name>/`:
```
.obsigate-plugins/
my-plugin/
plugin.json # manifest
index.js # entry point
.disabled # marker file (created when disabled)
```
## API Reference
### Backend Endpoints
| Method | Endpoint | Auth | Description |
|--------|---------------------------------|---------|--------------------------|
| GET | `/api/plugins` | require_auth | List installed plugins |
| POST | `/api/plugins/install` | require_admin | Install from ZIP |
| DELETE | `/api/plugins/{name}` | require_admin | Uninstall plugin |
| POST | `/api/plugins/{name}/enable` | require_admin | Enable plugin |
| POST | `/api/plugins/{name}/disable` | require_admin | Disable plugin |
| GET | `/api/plugins/{name}/code/{file}` | require_auth | Get plugin code |
| GET | `/api/plugins/{name}/hooks` | require_auth | Get plugin hooks |
| GET | `/api/plugins/template` | require_auth | Get plugin template |
### Frontend Module (`plugins.js`)
```javascript
import { PluginManager } from './plugins.js';
// Install
await PluginManager.installPlugin(manifest, code);
// Enable/Disable
await PluginManager.enablePlugin('my-plugin');
await PluginManager.disablePlugin('my-plugin');
// Uninstall
await PluginManager.uninstallPlugin('my-plugin');
// Get code for sandbox
const code = await PluginManager.getPluginCode('my-plugin', 'index.js');
// List
const plugins = await PluginManager.getCachedPlugins();
```
## Creating Your First Plugin
1. Create a directory with `plugin.json`:
```json
{
"name": "hello-world",
"version": "1.0.0",
"description": "My first ObsiGate plugin",
"author": "you",
"main": "index.js",
"hooks": {
"onFileRender": "render"
},
"permissions": ["read_files", "ui_notify"]
}
```
2. Create `index.js`:
```javascript
export function render(ctx) {
// Add a custom header to rendered markdown
if (ctx.content && ctx.path.endsWith('.md')) {
ctx.content = `> 📝 Plugin: hello-world\n\n${ctx.content}`;
}
return ctx;
}
```
3. Zip the directory and install via Settings > Plugins > Install Plugin.
+47 -6
View File
@@ -94,11 +94,14 @@ Définit les métadonnées de l'application :
### Service Worker (sw.js)
Gère le cache et le mode hors ligne :
- **Cache statique** : Interface et assets
- **Cache dynamique** : Données API (max 50 entrées)
- **Stratégies** : Cache-first et Network-first
- **Nettoyage** : Suppression automatique des anciens caches
Gère le cache et le mode hors ligne (version `SW_VERSION`) :
- **Code (HTML/JS/CSS/manifest)** : **network-first** (cache en secours hors-ligne)
- **API** : network-first (+ cache hors-ligne)
- **Autres assets (images, polices)** : stale-while-revalidate
- **Nettoyage** : purge automatique des caches d'une version antérieure à l'activation
> Le choix network-first est délibéré : les assets ne sont pas fingerprintés, un
> cache-first servait donc un ancien build indéfiniment sur mobile (BUG-005).
### Icônes PWA
@@ -271,7 +274,13 @@ location /sw.js {
}
location /manifest.json {
add_header Cache-Control "public, max-age=3600";
add_header Cache-Control "no-cache";
proxy_pass http://obsigate:8080;
}
# Les assets /static ne sont PAS fingerprintés : forcer la revalidation.
location /static/ {
add_header Cache-Control "no-cache";
proxy_pass http://obsigate:8080;
}
```
@@ -287,6 +296,38 @@ http:
Service-Worker-Allowed: "/"
```
### Cloudflare (BUG-005)
Derrière Cloudflare, un **service worker cache-first** combiné à un en-tête
`Cache-Control: immutable` sur des assets non fingerprintés peut servir un
ancien build indéfiniment — même après « Purge Everything » : le **Cache Storage
du navigateur (Service Worker) est un stockage séparé** que la purge Cloudflare
et le « vider le cache » du téléphone ne touchent pas.
Correctifs appliqués côté code (ObsiGate ≥ 2.2.1) :
- SW **network-first** pour HTML/JS/CSS (cache seulement en secours hors-ligne),
caches versionnés (`SW_VERSION`) purgés à l'activation.
- `Cache-Control: no-cache` sur `/static`, `index.html`, `manifest.json` et les
pages HTML (revalidation ETag, pas de cache long).
- Migration ponctuelle `localStorage` qui supprime les anciens caches sur les
appareils bloqués.
Réglages Cloudflare recommandés (dashboard → zone `og.dracodev.net`) :
| Réglage | Valeur | Pourquoi |
|---|---|---|
| **Rocket Loader** | **Désactivé** | Réécrit/defer les scripts et casse les `type="module"`. |
| **Auto Minify (JS/CSS)** | **Désactivé** | Peut casser la syntaxe JS moderne. |
| **Cache Rules / Page Rules** | Pas de « Cache Everything » sur HTML | Sinon Cloudflare ignore l'en-tête `no-cache`. |
| **Browser Cache TTL** | « Respect Existing Headers » | Laisse ObsiGate décider (`no-cache`). |
| **WebSockets** | Activé | Nécessaire aux connexions temps réel (SSE/WS). |
| **Purge Cache** | Après chaque déploiement | Évite un mix ancien/nouveau au premier accès. |
> Si un appareil reste bloqué malgré tout : sur mobile, désinstaller la PWA
> (ou « Site settings → Clear & reset ») puis rouvrir. La migration
> `obsigate-sw-migration` se charge normalement du nettoyage automatiquement.
## 🔒 Sécurité
### Content Security Policy
+157 -828
View File
File diff suppressed because it is too large Load Diff
+396
View File
@@ -0,0 +1,396 @@
# ObsiGate — Historique des fonctionnalités complétées (v1.0.0 → v2.2)
> **Rôle :** archive détaillée des fonctionnalités **livrées**. Le suivi des versions est la
> responsabilité de [CHANGELOG.md](../../CHANGELOG.md) ; la [Roadmap](../ROADMAP.md) ne contient
> plus que le travail **à venir** et un index compact vers ce fichier.
>
> Les grosses fonctionnalités disposent d'une fiche dédiée dans [`docs/features/`](../features/) :
> [#74 PDF](../features/pdf.md) · [#75 Split View](../features/split-view.md) ·
> [#76 BooksLM](../features/bookslm.md) · [#77 Desktop Tauri](../features/desktop-tauri.md) ·
> [#78 Excalidraw](../features/excalidraw.md).
---
## Fondations (v1.0.0 → v1.4.0)
| # | Feature | Effort | Impact |
|---|---|---|---|
| 1 | FastAPI backend — CRUD fichiers, vaults, recherche full-text | 5j | 🔴 |
| 2 | Moteur TF-IDF + stemming français (`snowballstemmer`) | 2j | 🔴 |
| 3 | Watchdog — indexation temps réel avec debounce | 1j | 🟡 |
| 4 | Interface SPA vanilla JS — sidebar, viewer, éditeur CodeMirror | 8j | 🔴 |
| 5 | Sécurité : JWT + Argon2id, rate limiting, audit log, CSP headers | 3j | 🔴 |
| 6 | Protection path traversal, utilisateur non-root Docker | 0.5j | 🔴 |
| 7 | Compression GZip SSE-safe, Cache-Control immutable | 0.5j | 🟢 |
| 8 | PWA : manifest, service worker, mode standalone | 1j | 🟡 |
## UX & Productivité (v1.5.0 → v1.6.0)
| # | Feature | Effort | Impact |
|---|---|---|---|
| 9 | Publication publique de documents (lien partageable, token) | 1j | 🟡 |
| 10 | Webhooks HTTP avec signature HMAC-SHA256 | 1j | 🟡 |
| 11 | Dashboard statistiques (fichiers, tags, taille, vaults) | 0.5j | 🟡 |
| 12 | Gestion des conflits Syncthing | 0.5j | 🟢 |
| 13 | Index inversé incrémental (hook pattern) | 1j | 🟡 |
| 14 | Backlinks panel dans le viewer | 0.5j | 🟡 |
| 15 | Fichiers non-supportés → UI download | 0.3j | 🟢 |
| 16 | Redaction de secrets (`secret_redactor.py`) | 0.5j | 🟡 |
| 17 | Backup automatique avant écriture, restauration | 1j | 🟡 |
| 18 | Vue graphe — Barnes-Hut, focus, plein écran, export PNG | 3j | 🟡 |
| 19 | Header flat design, sticky panels, navigation historique ← → ↑ | 1j | 🟢 |
| 20 | Ctrl+survol → aperçu contenu formaté | 0.5j | 🟢 |
## Architecture (v1.5.1)
| # | Feature | Effort | Impact |
|---|---|---|---|
| 21 | Split `app.js` (8 875 lignes) → 16 modules ES | 3j | 🔴 |
| 22 | Validateur imports/exports CI + tests unitaires frontend Node.js | 0.5j | 🟡 |
## CI/CD & Qualité (v1.6.0)
| # | Feature | Effort | Impact |
|---|---|---|---|
| 23 | Pipeline Gitea Actions : lint → test → security → build | 1j | 🔴 |
| 24 | Ruff (0 erreur) + Mypy (0 erreur) + Bandit SAST + Pip-audit | 0.5j | 🟡 |
| 25 | Pytest : 285 tests, 63% coverage | 3j | 🔴 |
## AI Editor (v1.7.0)
| # | Feature | Effort | Impact |
|---|---|---|---|
| 26 | Toolbar : Edit, Tone, Translate, Generate, Rewrite, Toolbox | 2j | 🟡 |
| 27 | Multi-provider : DeepSeek, OpenRouter, Gemini | 1j | 🟡 |
| 28 | 16 endpoints REST `/api/ai/{action}` + backend `ai.py` / `ai_routes.py` | 2j | 🟡 |
| 29 | Auto-save silencieux (2s debounce), loading toasts | 0.5j | 🟢 |
## Fonctionnalités avancées (v1.8.0 → v1.9.0)
| # | Feature | Effort | Impact |
|---|---|---|---|
| 30 | Export PDF via WeasyPrint (endpoint API + lien share public) | 1j | 🟡 |
| 31 | Palette de commandes `Ctrl+Alt+Space` | 1j | 🟡 |
| 32 | Barre d'outils mobile + palette fichiers/commandes 📱 | 1j | 🟡 |
| 33 | Drag & drop de fichiers | 1j | 🟡 |
| 34 | Filtres recherche avancés : `created:`, `modified:`, `size:` | 0.5j | 🟡 |
| 35 | Fichiers récents par vault | 0.5j | 🟡 |
| 36 | Indicateur AI Actif (header + toast) | 0.3j | 🟢 |
| 37 | Page d'accueil de vault (liste récursive par date, style recherche) | 0.5j | 🟡 |
| 38 | Indexation non-bloquante (background thread) | 0.5j | 🟡 |
| 39 | Git tags semver (v1.8.0, v1.9.0) | 0.2j | 🟢 |
## Gestion des Backups (v1.9.0)
| # | Feature | Effort | Impact |
|---|---|---|---|
| 40 | Diff viewer : unifié + côte à côte, restauration depuis backup | 1j | 🟡 |
| 41 | Gestionnaire de backups : page complète, filtre, suppression, purge | 1.5j | 🟡 |
| 42 | Purge par vault avec confirmation, preview contenu (100 Ko) | 0.5j | 🟡 |
| 43 | Compression gzip des vieux backups (niveau 6) | 0.5j | 🟢 |
| 44 | Backup automatique périodique (`POST /api/backups/auto`) | 1j | 🟡 |
| 45 | Restauration depuis le gestionnaire (extraction timestamp, confirmation) | 0.5j | 🟡 |
| 46 | Auto-nettoyage : `max_backups_per_file` (défaut 10) | 0.5j | 🟡 |
## Mermaid.js (v2.0.0-dev)
| # | Feature | Effort | Impact |
|---|---|---|---|
| 47 | CDN mermaid@11, `securityLevel: strict`, `startOnLoad: false` | 0.5j | 🟡 |
| 48 | `renderMermaidBlocks()` — parse, rendu SVG, bloc d'erreur stylisé | 1j | 🟡 |
| 49 | 22 templates (flowchart → sankey-beta) | 0.5j | 🟡 |
| 50 | Live preview dans l'éditeur (debounce 500ms, panneau `#mermaid-live-preview`) | 1j | 🟡 |
| 51 | Thème dark/light synchronisé, export SVG + PNG | 1j | 🟡 |
| 52 | Rendu inline dans le viewer Markdown | 0.5j | 🟡 |
| 53 | Zoom molette + drag + boutons +/- | 0.5j | 🟢 |
| 54 | Mode plein écran avec header bar (icônes zoom, copy, download, close) | 0.5j | 🟢 |
| 55 | Focus mode — clic diagramme → panneau latéral 480px | 0.3j | 🟡 |
| 56 | Pré-processeur Obsidian (`[[liens]]`, `![[img]]`, `==highlight==`) | 0.3j | 🟡 |
| 57 | Header bar : type de diagramme + toggle Code/Preview + copy SVG + download PNG | 1j | 🟡 |
---
## #58 — Tests E2E Playwright ✅
- **Effort :** 2-3 jours | **Impact :** 🔴
- **Description :** Tests navigateur automatisés pour les flows critiques : login, navigation vault, recherche full-text, ouverture/édition/sauvegarde de fichier, rendu Mermaid, export PDF.
- **Implémentation :** 44 tests Playwright (chromium-desktop) + 12 tests mobile dans `tests/e2e/obsigate.spec.ts`. Intégré au CI Gitea (job `e2e` après `build`).
- **Sous-tâches :**
- [x] Installation Playwright + config (`playwright.config.ts`)
- [x] Fixtures : vault de test (utilise les vaults Docker existants)
- [x] Test : dashboard → stats, tabs, Quick Help, sidebar
- [x] Test : recherche full-text → résultats, snippets, tri pertinence/date
- [x] Test : ouverture fichier → viewer Markdown, métadonnées
- [x] Test : éditeur → Forge, basic modal, Ctrl+S
- [x] Test : rendu Mermaid dans le viewer + preview Forge
- [x] Test : export PDF → bouton présent
- [x] Test : mode sombre → toggle, persistence localStorage
- [x] Test : responsive mobile → layout, recherche, barre flottante
- [x] Test : barre flottante résultats → compteur, nav, toggles Aa/wd
- [x] Test : sauvegardes → filtres Tous/Recherches/Répertoires
- [x] Test : raccourcis clavier → Ctrl+K, /, Escape
- [x] Test : menu contextuel répertoire
- [x] Intégration CI : job `e2e` dans `.gitea/workflows/ci.yml`
## #59 — Mode hors-ligne PWA complet ✅ TERMINÉ
- **Effort :** 3-4 jours | **Impact :** 🟡
- **Description :** Service worker avancé avec IndexedDB pour permettre la navigation et la recherche en mode hors-ligne, avec file de synchronisation au retour réseau.
- **Implémentation :** `offline-db.js` (377 lignes) + `offline.js` (209 lignes). IndexedDB 3 stores (files, content, pending). Badge hors-ligne dans le header. Modale résolution de conflits.
- **Sous-tâches :**
- [x] IndexedDB : stockage local de l'index des fichiers (paths, titles, tags)
- [x] Moteur de recherche offline via IndexedDB (cursor + filtre)
- [x] Cache des fichiers markdown récemment ouverts (derniers 50, prune)
- [x] Stratégie de cache : Network First avec fallback IndexedDB
- [x] File de synchronisation : modifications offline → appliquées au retour réseau
- [x] UI indicateur : badge « Hors-ligne » + compteur de modifications en attente
- [x] Gestion des conflits : détection et résolution manuelle (choix version locale vs serveur)
## #61 — Plugins système — Extensions utilisateur ✅
- **Effort :** 4-5 jours | **Impact :** 🟢 | **Statut :** ✅ Livré (backend + frontend + tests + docs)
- **Description :** Système de plugins permettant aux utilisateurs d'étendre ObsiGate avec des renderers personnalisés, des opérateurs de recherche, et des hooks d'UI. Inspiré du modèle de plugins Obsidian.
- **Sous-tâches :**
- [x] Spécification du format de plugin : `plugin.json` (name, version, hooks, permissions)
- [x] API de hooks : `onFileRender`, `onSearchFilter`, `onEditorAction`, `onSidebarItem`, `onFileCreate`, `onFileDelete`, `onVaultMount`
- [x] Sandbox d'exécution : Web Worker isolé pour le code plugin (blob URL, postMessage structuré, CSP sans importScripts)
- [x] UI : page « Plugins » dans les paramètres (installer, activer/désactiver, désinstaller, template, viewer)
- [ ] Distribution : dépôt de plugins communautaire (fichier JSON index) — ⚪ NON RETENU (backlog)
- [x] Hot-reload : activation/désactivation sans rechargement de page (marker `.disabled`)
- [x] Sécurité : manifest de permissions, validation path-traversal, CSP restrictif
- **Livré :**
- Backend `backend/plugins.py` — validation manifest (name regex, semver, hooks/permissions autorisés), stockage par vault `<vault>/.obsigate-plugins/`, lifecycle complet, validation ZIP (path traversal, limite 100 fichiers, 500KB/fichier), 9 endpoints `/api/plugins/*` (admin-gated pour install/uninstall/enable/disable), template API.
- Frontend `frontend/js/plugins.js` — PluginManager, sandbox Web Worker (code via blob URL, protocole postMessage structuré), UI Settings > Plugins, hooks dispatch (`executeHook`/`onFileRender`/`onSearchFilter`/…).
- Tests : `tests/test_plugins.py` (44) + `tests/frontend/plugins.test.mjs` (21) — validation, lifecycle, ZIP/dir sécurité, protocole sandbox, isolation DOM/CSP.
- Docs : `docs/PLUGINS.md`.
- **En backlog (non retenu) :** dépôt communautaire (index JSON), signature de code des plugins.
## #63 — Internationalisation (i18n) — Multilingue ✅ TERMINÉ
- **Effort :** 2-3 jours | **Impact :** 🟡 | **Statut :** ✅ Terminé
- **Description :** Support de l'anglais et du français via un système de clés de traduction.
- **Sous-tâches :**
- [x] Extraction des chaînes : ~1200 clés UI extraites
- [x] Format : JSON `fr.json` + `en.json` dans `frontend/locales/` → 1206 clés parfaitement synchronisées
- [x] Fonction `t(key)` → `frontend/js/i18n.js` avec `_applyDOM()`, `data-i18n`, `data-i18n-attr`, `data-i18n-placeholder`, `data-i18n-html`, support des templates `{var}`
- [x] Sélecteur de langue dans les paramètres (persisté localStorage `obsigate-lang`)
- [x] Traduction des messages backend → les toast/showToast sont maintenant i18n dans tous les fichiers JS
- [x] Documentation multilingue → `README.md` + `README.fr.md`
- [x] Nettoyage des clés inutilisées → locales nettoyées
- [x] Tous les fichiers JS utilisent `t()` → plus de texte FR en dur (ai.js, sync.js, graph.js, autocomplete.js)
- [x] Interface principale : dashboard, sidebar, editor, search, settings → EN/FR complet
- [x] Guide d'utilisation : 18 sections (Intro → Astuces) → tous les paragraphes traduits
- [x] Thèmes, palette de commandes, raccourcis, webhooks → EN/FR complet
- [x] Messages système : toasts, statuts, événements → EN/FR complet
## #64 — MFA — Authentification multi-facteurs ✅ TERMINÉ (TOTP + WebAuthn + recovery codes)
- **Effort :** 2 jours (réalisé) | **Impact :** 🟡
- **Description :** Ajout d'un second facteur d'authentification obligatoire pour les comptes administrateur. Deux méthodes sont proposées :
- **TOTP** (Time-based One-Time Password) : l'utilisateur scanne un QR code avec son app d'authentification (Google Authenticator, Authy, Bitwarden) qui génère un code à 6 chiffres renouvelé toutes les 30 secondes. Au login, après avoir saisi son mot de passe, l'utilisateur doit entrer le code affiché sur son téléphone. Même si le mot de passe est volé, le compte reste protégé car l'attaquant n'a pas le téléphone.
- **WebAuthn** (clés de sécurité physiques) : l'utilisateur enregistre une clé USB (YubiKey, SoloKey) ou utilise la biométrie de son appareil (empreinte digitale, Face ID, Windows Hello). Au login, le navigateur demande de toucher la clé physique ou de scanner le doigt. C'est le niveau de sécurité le plus élevé — résistant au phishing car la clé vérifie le domaine du site avant de répondre.
- **Codes de secours** : 8 codes à usage unique imprimables, à conserver en lieu sûr, qui permettent de se connecter même si on perd son téléphone ou sa clé. Chaque code ne fonctionne qu'une seule fois.
- **Pourquoi c'est important :** Le vol de mot de passe est la cause #1 de brèches de sécurité. Avec un vault Obsidian contenant des notes personnelles, projets sensibles, secrets et tokens API, l'authentification par simple mot de passe n'est plus suffisante. Le MFA empêche 99.9% des attaques de prise de compte automatisées (source : Microsoft Security).
- **Sous-tâches :**
- [x] TOTP : génération de secret, QR code, vérification code 6 chiffres
- [x] WebAuthn : enregistrement de clé, assertion, attestation (`backend/auth/webauthn_mfa.py`, lib `webauthn==2.6.0`, challenges in-memory TTL 180s à usage unique) — FAIT en 2026-09 (commit ab795ec)
- [x] UI : page « Sécurité du compte » avec activation/désactivation MFA + gestion des clés WebAuthn (liste, ajout, retrait)
- [x] Flow login : mot de passe → challenge TOTP OU WebAuthn selon `mfa_method` retourné par /login
- [x] Recovery codes : 8 codes de backup à usage unique (générés à l'activation, hachés SHA-256)
- [x] Stockage : `mfa_secret` + `webauthn_credentials[]` dans `users.json`
- [x] Tests : `tests/test_mfa.py` (29) + `tests/test_webauthn.py` (10, authentificateur virtuel CBOR/EC P-256)
## #65 — Thèmes personnalisés — CSS variables ✅ TERMINÉ
- **Effort :** 1-2 jours (réalisé) | **Impact :** 🟢
- **Description :** Exposition de variables CSS pour permettre aux utilisateurs de créer des thèmes personnalisés. Presets inclus : light, dark, high-contrast, sepia.
- **Sous-tâches :**
- [x] Audit des variables CSS existantes → 40+ variables
- [x] Presets: light, dark, high-contrast, sepia (générés dynamiquement)
- [x] UI : sélecteur de thème dans les paramètres (swatches grid)
- [x] Import/export de thème personnalisé (JSON)
- [x] Application dynamique via document.documentElement.style.setProperty
## #66 — Export multi-formats ✅ TERMINÉ
- **Effort :** 1-2 jours (réalisé) | **Impact :** 🟢
- **Description :** Export de notes individuelles ou de vaults entiers en HTML standalone, bundle Markdown (.zip), et ePub pour liseuses.
- **Sous-tâches :**
- [x] Export HTML standalone : CSS inliné, images en base64, navigation inter-fichiers
- [x] Export MD bundle : ZIP du vault avec structure préservée
- [x] Export ePub : conversion markdown → ePub (zipfile + mistune, 0 nouvelle dep)
- [x] UI : dropdown Export dans toolbar viewer (HTML / MD bundle / ePub)
- [x] Endpoints : `GET /api/export/html`, `GET /api/export/md-bundle`, `GET /api/export/epub`
## #67 — Notifications web — Push API ✅ TERMINÉ
- **Effort :** 2 jours | **Impact :** 🟢
- **Description :** Recevoir des notifications sur le bureau ou le téléphone quand un fichier est modifié, ajouté ou supprimé dans un de vos vaults, même si ObsiGate n'est pas ouvert dans le navigateur.
- **Fonctionnement** : Le navigateur s'abonne auprès du serveur via la Push API (standard W3C). Le serveur stocke l'abonnement (endpoint + clés de chiffrement). Quand un fichier change (détecté par le watcher existant), le serveur envoie une notification chiffrée au push service du navigateur (Firebase pour Chrome, APNs pour Safari, etc.), qui la relaye au navigateur même s'il est fermé. Le service worker ObsiGate affiche alors la notification système.
- **Contenu** : titre du fichier modifié, nom du vault, type d'action (créé/modifié/supprimé). Un clic sur la notification ouvre directement le fichier dans ObsiGate.
- **Configuration** : activation/désactivation par vault. L'utilisateur choisit pour quels vaults il reçoit des notifications.
- **VAPID** : protocole d'authentification volontaire qui permet au serveur de s'identifier auprès du push service sans avoir à s'enregistrer comme application. Une paire de clés publique/privée est générée — la clé publique est partagée avec le navigateur, la clé privée reste sur le serveur.
- **Pourquoi c'est important :** Collaborer sans avoir à constamment rafraîchir l'interface pour voir si quelqu'un a modifié quelque chose. Particulièrement utile en équipe ou pour les vaults partagés via Syncthing — on sait immédiatement quand une note est mise à jour.
- **Implémentation :** `backend/push.py`, endpoints `POST /api/push/subscribe`, config VAPID dans `config.json`, UI toggle par vault. Livré avec #77 (commit ac16fc1).
- **Sous-tâches :**
- [x] Souscription Push : endpoint `POST /api/push/subscribe` (stockage `subscription` + `vault`)
- [x] Envoi : webhook interne `on_file_change` → dispatch notification via Web Push
- [x] Configuration VAPID : clés publique/privée dans `config.json`
- [x] UI : permission navigateur + toggle activer/désactiver par vault
- [x] Payload : titre du fichier, vault, action (created/modified/deleted)
- [x] Clic sur notification → ouvre le fichier dans ObsiGate
## #68 — Health check enrichi ✅ TERMINÉ
- **Effort :** 1 jour | **Impact :** 🟢
- **Description :** Un endpoint `/api/health` qui ne se contente pas de dire « je suis vivant », mais donne un diagnostic complet de l'état du serveur. Essentiel pour le monitoring et le debugging.
- **Métriques exposées :**
- **Index** : nombre de fichiers indexés, nombre de tokens, date de la dernière indexation complète. Permet de détecter si l'indexeur est bloqué ou ne tourne plus.
- **Mémoire** : consommation RAM du processus (RSS), heap Python utilisé. Permet de détecter les fuites mémoire avant qu'elles ne crashent le serveur.
- **Uptime** : depuis quand le serveur tourne. Simple, mais indispensable pour corréler un problème avec un redémarrage.
- **Connexions** : nombre de connexions SSE actives (recherche en cours, streaming AI, etc.). Permet de savoir combien d'utilisateurs sont connectés.
- **Backups** : nombre total, âge du backup le plus ancien, espace disque consommé. Permet de détecter si les backups s'accumulent anormalement.
- **Disque** : espace libre sur la partition `/data`. Évite le crash silencieux quand le disque est plein.
- **Format** : JSON structuré, facile à intégrer dans des outils de monitoring (Prometheus, Grafana, Uptime Kuma, Healthchecks.io).
- **Sécurité** : l'endpoint public `/api/health` retourne `ok` ou `degraded`, l'endpoint détaillé est protégé par authentification admin.
- **Implémentation :** `GET /api/health/detailed` (admin-gated) dans `backend/main.py:1176`. Livré avec #77 (commit ac16fc1).
- **Sous-tâches :**
- [x] Métriques index : nombre de fichiers, nombre de tokens, génération courante
- [x] Métriques mémoire : RSS, heap used (via `psutil` ou `/proc/self/status`)
- [x] Métriques uptime : `time.time() - server_start_time`
- [x] Métriques backups : nombre total, âge du plus vieux, espace disque
- [x] Format réponse JSON structuré : `{ status, uptime, index, memory, backups, connections }`
- [x] Endpoint séparé `GET /api/health/detailed` (protégé admin)
## #71 — Tableau de bord administrateur ✅ Terminé
- **Effort :** 2 jours | **Impact :** 🟢 | **Statut :** ✅ Terminé (2026-08, commits 46be24f / 88ab8db / 01453bc)
- **Description :** Une page web dédiée accessible uniquement aux administrateurs qui centralise tout le monitoring et la gestion du serveur ObsiGate en un seul endroit. Un cockpit de pilotage pour le sysadmin.
- **Implémentation réelle (vérifiée) :**
- **Backend `backend/admin.py`** (nouveau, 261 lignes) — 4 endpoints admin-gated (`require_admin`) :
- `GET /api/admin/stats` — CPU/RAM/Disk/Uptime via psutil
- `GET /api/admin/audit` — 500 dernières entrées d'audit avec filtres `user`/`action`/`limit`/`offset`
- `GET /api/admin/backup-stats` — compte + taille + age par vault
- `GET /api/admin/stream` — Server-Sent Events qui push les stats toutes les 5s
- `backend/main.py` — routeur monté + middleware gzip bypass pour `/api/admin/stream`
- `backend/requirements.txt` — ajout `psutil>=5.9`
- **Tests :** `tests/test_admin.py` (13 tests, 100% verts) — couvrent auth + filtres + format SSE
- **Complété (2026-08) :**
- `frontend/admin.html` (472 l.) + `frontend/js/admin.js` (544 l.) — dashboard standalone avec navigation par sections sticky + thèmes
- Lien « Admin » dans le user menu (`#admin-menu-row`, gating `role === "admin"`)
- Widgets temps réel via EventSource `/api/admin/stream` + snapshot `/api/admin/stats`
- CRUD users + fix routing `/admin.html` + fix scroll
## #72 — API publique documentée — OpenAPI 3.1 ✅ TERMINÉ
- **Effort :** 1-2 jours (réalisé) | **Impact :** 🟢 | **Statut :** ✅ Livré (2026-09-11)
- **Implémentation réelle :**
- `backend/openapi_docs.py` — 18 tags documentés + assignation automatique par préfixe de route
(`tag_for_path`, `canonical_tag`), enrichissement du schéma (`enrich_openapi_schema` :
sécurité `bearerAuth`/`cookieAuth`, erreurs 401/403/404/422/500, exemples requête/réponse,
serveur, `externalDocs`), page `/api` autonome (`render_api_landing`).
- `backend/schemas.py` — `response_model` Pydantic pour ~35 endpoints qui n'en avaient pas.
- `backend/main.py` — `FastAPI(...)` enrichi (description Markdown, contact, licence, tags) +
override `app.openapi` ; routes `/api` et `/api/`.
- `backend/ai_routes.py` / `backend/bookslm_routes.py` — `response_model` (AI status, BooksLM
context) + documentation SSE.
- Frontend — entrée « API » du menu d'options (i18n FR/EN) ouvrant `/docs`.
- Tests — `tests/test_openapi.py` (58 tests) + `tests/frontend/ai.test.mjs` (7 tests).
- **Description :** Une page de documentation interactive et auto-générée de toutes les API REST d'ObsiGate, accessible via un bouton dans l'interface. L'équivalent d'un manuel technique mais qui se teste en direct.
- **OpenAPI 3.1** : c'est le format standard mondial pour décrire une API REST. Un seul fichier JSON/YAML contient la description de tous les endpoints, leurs paramètres, les formats de réponse, les codes d'erreur, et les modèles de données. Ce standard est supporté par des centaines d'outils.
- **Swagger UI** : une interface web qui lit le fichier OpenAPI et génère automatiquement une documentation interactive. L'utilisateur voit chaque endpoint, peut remplir les paramètres dans un formulaire, cliquer « Execute » et voir la réponse réelle de l'API en direct. Parfait pour les développeurs qui veulent intégrer ObsiGate à leurs scripts ou comprendre comment fonctionne l'API.
- **Redoc** : une alternative à Swagger UI, plus propre et orientée lecture, idéale pour la documentation publique.
- **Contenu documenté :**
- Les 40+ endpoints existants, regroupés par catégorie (Fichiers, Vaults, Recherche, Auth, AI, Backups).
- Chaque endpoint avec description, paramètres obligatoires/optionnels, exemples de requête et réponse.
- Les modèles de données Pydantic exposés comme schémas JSON (ex: structure d'un `FileInfo`, d'un `SearchResult`).
- Les codes d'erreur possibles avec leur signification.
- **Auto-génération** : FastAPI génère déjà partiellement le schéma OpenAPI. Le travail consiste à compléter les docstrings manquantes, ajouter `response_model` sur les endpoints qui n'en ont pas, et enrichir avec des exemples.
- **Pourquoi c'est important :** Une API sans documentation est comme un logiciel sans interface — techniquement fonctionnel mais inutilisable. Avec une doc OpenAPI, ObsiGate devient intégrable dans n'importe quel écosystème. Un développeur peut en 5 minutes comprendre comment uploader un fichier, chercher dans un vault, ou récupérer le contenu d'une note — et écrire un script qui automatise ses workflows.
- **Sous-tâches :**
- [x] Audit des endpoints existants → compléter les docstrings manquants
- [x] Ajout de `response_model` sur tous les endpoints (40+ actuellement, ~15 sans modèle)
- [x] Exemples dans les schémas : `examples=[...]` pour les endpoints clés
- [x] Tagging des endpoints par catégorie (Files, Vaults, Search, Auth, AI, Backups)
- [ ] Serveur mock : `prism` ou `openapi-generator` pour tests sans backend — ⚪ NON RETENU (le schéma 3.1 est validé par `tests/test_openapi.py`)
- [x] Page de documentation intégrée : lien dans le menu header (« API »)
---
## #84 — Consolidation & sécurité (phase 1) ✅ TERMINÉ
Traitement des vulnérabilités de la revue statique du 2026-09-13 (BUG-021 → BUG-034).
- **Sanitizer XSS serveur** (`backend/services/sanitizer.py`, stdlib, liste blanche
de balises/attributs/schémas) appliqué au rendu markdown et à la page publique
`/s/{token}` (titre, frontmatter et JSON échappés, `</script>` neutralisé).
- **Auth durcie** : rate-limit IP + compte et verrouillage sur les endpoints MFA ;
rotation du refresh token et révocation de l'access token au logout ; politique de
mot de passe centralisée (8–128) ; `password_changed_at` invalide les jetons
antérieurs ; verrou `RLock` sur `users.json`.
- **Isolation & réseau** : `resolve_safe_path` compare par segments (`vault` ≠
`vault-evil`) ; webhooks protégés contre le SSRF (HTTPS, IP privées bloquées,
résolution vérifiée, redirections interdites, secret externalisé) ; IP client
réelle dans les audits.
- **Robustesse** : validation des regex (ReDoS), symlinks hors vault ignorés à
l'indexation, recherche simple et `search_fulltext` branchés sur l'inverted index.
- **Frontend** : token d'accès en mémoire + cookie `HttpOnly` (plus de
`sessionStorage`), CSP durcie (`object-src`, `base-uri`, `form-action`,
`frame-ancestors`).
- **Tests** : `tests/test_security_hardening.py` (30) ; suite backend 961 passed /
6 skipped ; ruff + mypy 0 ; suites frontend vertes.
---
## #83 — Barre d'outils d'édition mobile (ruban style Obsidian) ✅ TERMINÉ
La barre de mise en forme flottante du mode édition mobile est remplacée par un
**ruban horizontal défilable** ancré juste au-dessus du clavier virtuel.
- **Ruban** (`frontend/js/mobile-editor.js`, `frontend/style.css`) : fond anthracite
aux coins arrondis, insertion/enrobage au curseur ou sur la sélection.
- **Commandes** : annuler/refaire, `[[ ]]` (lien interne), fichiers/modèles, tag `#`,
pièce jointe, H1–H6, gras, italique, barré (`~~`), surligné (`==`), code en ligne/bloc,
citation, lien externe, listes à puces/numérotées, case à cocher, indenter/désindenter,
coller, A−/A+.
- **Personnalisation** : icône **⚙** → ajouter / retirer / réordonner les commandes,
persisté dans `localStorage` (`obsigate-mobile-ribbon`).
- **Ancrage clavier** (`BUG-019`) : `window.visualViewport` → variable CSS `--kb-offset`.
- **Correctifs associés** : BUG-017 (tuiles du tableau de bord), BUG-018 (en-tête de
l'éditeur mobile), BUG-020 (double barre de défilement).
- **Tests** : `tests/frontend/mobile-editor.test.mjs` (35) + E2E `mobile-editor.spec.js`.
---
## #90 — Barre d'actions du document : regroupement fonctionnel ✅ TERMINÉ
Réordonnancement des boutons du haut du document en quatre groupes
séparés par des barres verticales (`span.action-sep`) :
`[TOC] [pop-out] [Bookmark] | [Editer] [Source] [.md] [Forge] | [Copier] [PDF] [Export] | [Partager]`
Fichiers texte non markdown :
`[pop-out] [Bookmark] | [Editer] [Source] [.ext] [Pretty] | [Copier] | [Partager]`
- **Viewer principal** (`frontend/js/viewer.js`, `renderFile`) : construction par
groupes `navBtns` / `editBtns` / `exportBtns` / `shareBtns` ; TOC ancré au
markdown seul (`.sh`/`.py`/etc. sans TOC) ; « Pretty » (fichiers texte) ancré
au groupe édition après le bouton d'extension ; les spacers d'un groupe vide
(ex. fichier image ou binaire) ne sont pas rendus.
- **Pop-out** (`frontend/popout.html`) : même assemblage groupé, ordre aligné.
- **CSS** (`frontend/style.css`) : règle `.file-actions .action-sep`
(trait 1 px, `var(--border-md)`, `align-self: stretch`).
- **Tests** : `tests/frontend/toolbar-order.test.mjs` (9, statiques).
---
## Grosses fonctionnalités — fiches dédiées
| # | Feature | Version | Fiche |
|---|---|---|---|
| 74 | Support complet des documents PDF | 2.1.0 | [features/pdf.md](../features/pdf.md) |
| 75 | Éditeur multi-panneaux (Split View) | 2.1.0 | [features/split-view.md](../features/split-view.md) |
| 76 | BooksLM — Console AI contextuelle par répertoire | 2.2.0 | [features/bookslm.md](../features/bookslm.md) |
| 77 | Application Desktop native — Tauri | 2.2.0+ (en cours) | [features/desktop-tauri.md](../features/desktop-tauri.md) |
| 78 | Éditeur Excalidraw | 2.2.0 | [features/excalidraw.md](../features/excalidraw.md) |
+104
View File
@@ -0,0 +1,104 @@
# #88 — Assistant IA : contexte applicatif & fiabilité des liens
> **Statut :** 🟢 corrigé / livré (en attente de vérification utilisateur)
> **Version :** 2.3.0
> **Composants :** `frontend/js/bookslm.js`, `backend/bookslm.py`, `backend/bookslm_routes.py`
> **Bugs liés :** BUG-041, BUG-042
## Contexte
Trois troubles d'usage signalés sur l'Assistant IA (BooksLM) :
1. **Répertoire vide bloquant** — ouvrir l'assistant sur un dossier ne contenant
aucun fichier markdown renvoyait `⚠ Error: Aucun fichier markdown trouvé dans ce dossier`
(HTTP 404) au lieu de répondre à la question de l'utilisateur.
2. **Assistant Général aveugle** — en mode Général, l'assistant ne savait que
lister les vaults : il ne connaissait ni les documents ouverts, ni le répertoire
courant, ni la page de résultats de recherche, ni les fichiers récemment modifiés.
3. **Liens de fichiers peu fiables** — l'assistant produisait des liens vers des
noms de fichiers qui échouaient avec `File not found: …`, sans règle claire
(nom simple, dossier, chemin complet).
## Conception
### 1. Dégradation gracieuse du dossier vide (BUG-041)
`_resolve_system_prompt` (`backend/bookslm_routes.py`) ne lève plus de 404 quand
le contexte d'un dossier est vide. Il construit le prompt Général (aide
applicative + actions), enrichi d'un bloc **« Dossier vide »**, et prévient le
modèle qu'aucun contexte documentaire n'est disponible — il répond quand même et
propose une création de fichier si pertinent.
Le prompt Général reçoit en outre les **fichiers récemment modifiés** de
l'utilisateur (`backend/services/recent.list_recent`, best-effort).
### 2. Contexte applicatif de l'assistant Général (#88)
Le frontend envoie un nouveau champ `app_context` dans les requêtes `/chat` et
`/agent` (`_buildAppContext()`), contenant l'état vivant de l'interface :
| Champ | Contenu |
|---|---|
| `open_documents` | Documents ouverts dans les onglets/panneaux (via `collectOpenDocuments()`) |
| `current_path` | Document affiché dans le viewer |
| `directory` | Répertoire courant de l'assistant |
| `vault` | Vault sélectionné |
| `search_query` / `search_total` | Dernière recherche et nombre de résultats |
| `search_results` | Fichiers affichés sur la page de résultats (max 20) |
Côté backend, `build_general_system_prompt(vaults, app_context, recent_files)`
rend un bloc **« Contexte applicatif actuel »** ajouté au prompt Général.
### 3. Liens de fichiers déterministes (BUG-042)
Classification des liens (`_classifyPath`) :
| Forme | `data-kind` | Action au clic |
|---|---|---|
| Nom de fichier seul (`readme.md`) | `name` | Copie le nom dans le presse-papiers |
| Chemin de dossier (`projets/2026`) | `dir` | Révèle le dossier dans l'arborescence |
| Chemin de fichier (`notes/a.md`) | `file` | Ouvre le fichier dans le viewer |
Avant d'agir, `_activatePath()` **résout** le chemin contre l'index du vault
(`_resolveExistingPath`) : correspondance exacte, puis suffixe
(segment de tête omis par le modèle), puis basename unique. Les chemins non
résolus copient le nom au lieu d'ouvrir un lien mort — plus de `File not found`.
### 4. Espaces, accents et préfixe de vault dans les chemins (BUG-042)
Les noms de fichiers/dossiers pouvant contenir des espaces et des accents, la
détection et la résolution sont adaptées :
- **Liens markdown** : la cible peut contenir des espaces, être encadrée par
`<…>` ou porter un `"titre"` ; les URL percent-encodées (`%20`) sont décodées.
- **Code inline** : `` `Ma note.md` `` / `` `Recettes/Préparation.md` `` sont
reconnus (`_looksLikePath` utilise `PATH_NAME_RE`, une classe Unicode
`\p{L}\p{N}\p{M}` — les accents sont acceptés, le jeu de caractères reste
strict pour rejeter extraits de code et commandes shell).
- **Mentions brutes** : une mention avec espaces n'est liée **que si elle existe
réellement dans l'index du vault** (`_linkifySpacePaths` +
`_confirmPathInCache`), et l'acceptation se fait sur le **plus long suffixe
aligné sur un mot** — ainsi « Ouvre Mon dossier/Ma note.md » ne crée pas de
lien englobant « Ouvre » et la prose ordinaire n'est jamais liée à tort. Les
chemins sans espace mais accentués sont liés par le motif `PATH_WITH_DIR_RE`,
lui aussi Unicode-aware.
- **Normalisation Unicode** : `_normKey()` compare en NFC + minuscules, donc une
mention décomposée (`e` + accent combinant, comme sur macOS) correspond à une
entrée d'index précomposée, et inversement.
- **Préfixe de vault** : `_splitVaultPrefix()` retire un premier segment égal au
nom d'un vault connu (`TestVault/Recettes/Pizza Maison.md` →
`Recettes/Pizza Maison.md`). La résolution interroge l'index **de ce vault**
(`_fetchPathsForVault`, fetch ponctuel sans écraser le cache du vault actif),
puis `_openFileLink`/`_revealPath` reçoivent le vault cible. Si le préfixe ne
correspond à aucun vault connu, `_resolveExistingPath` retente en retirant le
premier segment.
## Tests
- `tests/frontend/ai.test.mjs` : classification (`name`/`file`/`dir`), routage
`_activatePath` (ouvre/révèle/copie), clic d'un lien « nom » → presse-papiers,
et prise en charge des espaces (code inline, liens markdown `%20`, mentions
brutes liées uniquement si présentes dans l'index, résolution au clic).
- `tests/test_bookslm.py` : `build_general_system_prompt` avec `app_context` +
`recent_files` ; prompt inchangé sans contexte ; chat sur dossier vide ne
renvoie plus 404.
+71
View File
@@ -0,0 +1,71 @@
# #81 — Assistant IA — Commandes (`@`, `/`), skills, analyse d'images & capacités des modèles
> **Statut :** ✅ Livré
> **Effort :** 4-6 jours | **Impact :** 🟡
> **Références :** [Roadmap](../ROADMAP.md) · [Assistant IA #80](./ai-assistant-ux.md) · [Outils IA #79](./ai-tools-mcp.md) · [Changelog](../../CHANGELOG.md)
- **Description :** cinq améliorations de l'assistant IA (`frontend/js/bookslm.js`, `frontend/js/ai.js`) :
1. **Panneau redimensionnable** à la souris, largeur persistée.
2. **Commande `@`** pour ajouter des contextes ad-hoc (fichiers et répertoires) au contexte courant.
3. **Commande `/`** pour choisir un **skill** (workflow réutilisable) ou une **commande admin**.
4. **Analyse d'images** : coller une image dans la zone de saisie ou référencer une image d'un
répertoire (`@image.png`), avec garde-fou de compatibilité du modèle.
5. **Capacités des modèles** affichées à la sélection (Chat, Embeddings, Rerank, Images, Video,
Audio Speech, Audio Transcriptions, Vision capable).
## A. Panneau redimensionnable — ✅ livré
- [x] Poignée `.bookslm-resize-handle` sur le bord gauche, glisser pour ajuster (320–1000 px).
- [x] Largeur persistée dans `localStorage['obsigate-bookslm-width']` ; désactivée en plein écran/mobile.
## B. Contexte ad-hoc `@` — ✅ livré
- [x] Détection de `@query` au curseur → menu `.bookslm-mention-menu` alimenté par
`/api/tree-search` (avec repli sur `/api/vault/{vault}/files` quand la requête est vide).
- [x] Sélection : fichier → chip de contexte ; répertoire → chip de contexte ; image → pièce jointe.
- [x] Chips retirables (`.bookslm-attachments`), rechargement du contexte (`/context`).
- [x] Backend : `extra_files` / `extra_directories` dans les requêtes context/chat/agent, fusion via
`collect_adhoc_context()` + `merge_contexts()` (`backend/bookslm.py`). En mode General, l'ajout
de fichiers promeut le scope en `documents`.
## C. Commandes `/` & skills — ✅ livré
- [x] Menu `.bookslm-command-menu` filtré à la saisie ; navigation clavier (↑/↓/Entrée/Échap).
- [x] **Skills intégrés** (`backend/skills.py`) : `/research`, `/create-new-skill`, `/resume`,
`/actions`, `/reformuler`, `/correction`, `/brainstorm`, `/plan`, `/ask`, `/meeting-note`,
`/livrable`. Le prompt du skill est ajouté au system prompt (`skill` dans la requête chat).
- [x] **Skills utilisateur persistés** (`data/skills.json`, par utilisateur) créés via
`/create-new-skill` (modale) → `POST /api/ai/skills`, listés par `GET /api/ai/skills`,
supprimables par `DELETE /api/ai/skills/{id}`.
- [x] **Commandes admin** exécutées localement (sans LLM) : `/help`, `/providers`, `/provider <nom>`,
`/model <nom>`, `/keys`.
## D. Analyse d'images — ✅ livré
- [x] Coller une image dans la zone de saisie (`paste`) ou bouton « joindre une image ».
- [x] Référencer une image d'un répertoire via `@image.png` (chargée côté serveur en data URL).
- [x] Backend multimodal : `ai_chat._content_to_gemini_parts` (data URL → `inlineData`), messages
OpenAI-compatibles passés tels quels ; `_build_user_content` dans `bookslm_routes.py`.
- [x] Garde-fou : requête d'image rejetée (400) si le modèle ne supporte pas la vision ; côté
frontend, toast + blocage de l'envoi. Les images forcent l'endpoint `/chat` (pas l'agent).
## E. Capacités des modèles — ✅ livré
- [x] Table statique curée `backend/model_capabilities.py` (règles par motif de nom + défauts par
fournisseur) exposant 8 flags.
- [x] Endpoint `GET /api/ai/model-capabilities?provider=&model=` (auth) + `capabilities` par modèle
dans `GET /api/config/ai-models`.
- [x] Affichage ☑/□ dans le picker de l'assistant (`frontend/js/ai.js`) et dans le panneau de
configuration (`#cfg-ai-default-model-caps`, `frontend/js/config.js`).
- [x] **#82** — dans le picker de l'assistant, la liste permanente est remplacée par un bouton
d'information ⓘ et une bulle au survol/clic/appui long, avec recherche de modèle. Détail :
[ai-provider-picker.md](./ai-provider-picker.md).
## F. Tests & documentation — ✅ livré
- [x] Backend : `tests/test_model_capabilities.py`, `tests/test_skills.py`, `tests/test_ai_vision.py`,
extension `tests/test_ai_models.py` (capabilities).
- [x] Frontend : `tests/frontend/ai.test.mjs` (redimensionnement, `@`, `/`, payload, endpoint,
vision, capacités, commandes admin).
- [x] i18n FR/EN (`ai.cap_*`, `ai.commands`, `ai.attach_image`, `config.ai_capabilities`, …).
- [x] CHANGELOG + Roadmap + guide utilisateur (aide in-app).
## G. Points d'attention
- La table de capacités est **statique** : un modèle inconnu retombe sur le défaut du fournisseur
(souvent `chat` uniquement), afin de ne jamais annoncer une capacité non vérifiée.
- Les skills utilisateur sont stockés par utilisateur ; `/create-new-skill` ne peut pas écraser un
skill intégré ni réutiliser un identifiant existant.
@@ -0,0 +1,154 @@
# #91 — Assistant IA — Zone de discussion façon Notion : post ancré en haut, fournisseur/modèle discret & barre d'actions
> **Statut :** ✅ Livré
> **Effort :** 0,5-1 jour | **Impact :** 🟡
> **Références :** [Roadmap](../ROADMAP.md) · [BooksLM #76](./bookslm.md) · [Assistant UX #80](./ai-assistant-ux.md) · [Changelog](../../CHANGELOG.md)
- **Description :** Refonte de la présentation de la fenêtre de résultats de
l'assistant IA (`frontend/js/bookslm.js`, `frontend/style.css`) sur le modèle de
l'assistant Notion :
1. **Ancrage en haut** : après l'envoi, le post de l'utilisateur est amené tout en
haut de la zone visible (`scrollIntoView({ block: 'start' })` +
`scroll-margin-top`) ; la réponse se diffuse **en dessous**, lisible sans
défilement. Plus de scroll-into-bottom ni d'astuce `order: -1`.
2. **Fil Notion-style** : messages en ordre chronologique ; post utilisateur =
bulle arrondie alignée à droite (max 80 %) ; réponse assistant = texte pleine
largeur **sans bulle de fond** ; les appels d'outils du mode agent se replient
dans un bloc discret « N étapes » (`<details>`).
3. **Fournisseur & modèle discrets** : libellé au-dessus de chaque réponse,
issu du SSE réel du backend (`provider`/`model`).
4. **Barre d'actions au survol** : sous chaque bloc non vide, « Copier »
(utilisateur et assistant) ; sous une réponse, « Ajouter » qui insère le texte
dans le document ouvert dans l'éditeur Forge (aucun bouton factice).
## A. Positionnement automatique (ancre en haut) — ✅ livré
- [x] **A1.** `_renderMessages({ anchor: true })` appelé à l'envoi du message, à
l'ouverture d'un contexte, au rechargement d'une session et à la reprise après
confirmation agent : cible = dernier `.bookslm-msg.user`, `scrollIntoView`
`block: 'start'` (instantané à l'envoi, fluide à l'ouverture d'une session),
repli calcul `scrollTop += delta` si indisponible.
- [x] **A2.** Re-rendus de streaming : tant que l'ancre est active (flag
`_pinnedTurn` posé à l'envoi, libéré sur `wheel`/`touchmove`/`mousedown`), chaque
re-rendu — frames de streaming **et** rendu final — ré-ancre instantanément la
question en haut ; la réponse s'allonge sous la question sans jamais déplacer la
vue. Hors épinglage, la position de scroll est préservée (plus de
`scrollTop = scrollHeight`).
- [x] **A3.** Question **au bord** : plus de `scroll-margin-top` (le moindre écart
laissait visible la fin de la réponse précédente au-dessus du post) ; le post
s'aligne exactement sur le haut de la zone de défilement.
- [x] **A3bis.** **Remplissage dynamique** : une réponse courte ne remplit pas la
fenêtre — le navigateur bloquait alors le défilement et la question restait à
mi-hauteur. Le `padding-bottom` du fil est augmenté de la place manquante
(`target - max`) tant que l'ancre est active, puis retiré au dé-épinglage ; si
le fil tient entièrement dans la fenêtre, aucun remplissage n'est ajouté.
- [x] **A3ter.** **Anti-dérive** : `overflow-anchor: none` sur `.bookslm-messages`
(Chrome ré-ancrait le défilement à chaque reconstruction du fil — un token =
un re-rendu — laissant un décalage constant de ~33 px) et passe de correction
après `scrollIntoView` : la géométrie mesurée (`getBoundingClientRect`) fait foi,
pas la demande de défilement. Vérifié en live : 5 questions, 0 décalage.
- [x] **A4.** Seule la zone du fil défile : `.bookslm-messages { flex: 1; overflow-y: auto }`
(en-tête, toolbar, statut et barre de saisie fixes — structure déjà en place).
## B. Fil chronologique Notion-style — ✅ livré
- [x] **B1.** Ordre naturel (user puis assistant, empilés, `gap: 24px`).
- [x] **B2.** Utilisateur : bulle `--surface2` alignée à droite, `max-width: 80%`,
coins arrondis 18px.
- [x] **B3.** Assistant : `width: 100%`, pas de fond ni de padding de bulle —
le markdown occupe toute la largeur (titres, listes, code, sources inchangés).
- [x] **B4.** Mode agent : `_renderToolActivity()` → `<details class="bookslm-tool-trace">`
avec `<summary>` « N étapes » (i18n `ai.steps_count`) ; les lignes d'outils
restent accessibles en déroulant le bloc.
## C. Fournisseur & modèle discrets — ✅ livré
- [x] **C1.** `_streamResponse()` capture `data.provider` / `data.model` du flux SSE
(déjà émis par `/chat` et `/agent`) sur le message assistant.
- [x] **C2.** `.bookslm-msg-meta` au-dessus du bloc assistant :
« fournisseur · modèle » (persisté avec la session, visible au rechargement).
- [x] **C3.** Le backend émet le modèle **réellement utilisé** :
`_effective_model(provider, req.model)` résout le défaut du fournisseur quand le
client ne précise pas de modèle (avant : `req.model or ""` → tag réduit au seul
fournisseur). Idem dans les deux flux SSE (`/chat` et `/agent`).
## D. Barre d'actions — ✅ livré
- [x] **D1.** `_appendActionBar()` sous chaque bloc non vide : bouton Copier pour
les deux rôles ; bouton Ajouter pour l'assistant uniquement.
- [x] **D2.** Copier : texte brut du message (`navigator.clipboard`, repli
`execCommand`) + toast `bookslm.copied`.
- [x] **D3.** Ajouter (`_insertIntoEditor`) : insertion du texte à la position de
fin de sélection dans `state.editorView` (Forge) ; toast `bookslm.inserted` ou
`bookslm.insert_no_editor` si aucun éditeur ouvert.
- [x] **D4.** Révélation au survol / focus clavier (`.bookslm-msg:hover`,
`:focus-within`) — icônes Lucide `copy` / `corner-down-left`.
## E. Tests & documentation — ✅ livré
- [x] **E1.** `tests/frontend/ai.test.mjs` : ordre chronologique, ancrage
`scrollIntoView block:start`, tag provider/modèle, barre d'actions par rôle,
copie du texte brut, insertion éditeur (mock `state.editorView`), bloc
repliable des étapes, absence de barre pour contenu vide.
- [x] **E2.** i18n FR/EN : `bookslm.copied`, `bookslm.insert`, `bookslm.insert_hint`,
`bookslm.inserted`, `bookslm.insert_no_editor`, `ai.steps_count`.
- [x] **E3.** CHANGELOG + Roadmap ; `SW_VERSION` bump (cache bust).
## F. Points d'attention
- Le tag fournisseur provient du **backend réel** (SSE), donc correct même avec la
sélection « par défaut » du fournisseur.
- Le streaming appelle `_renderMessages()` à chaque chunk : l'ancre est posée une
seule fois à l'envoi, les re-rendus préservent ensuite le scroll — pas de sauts.
- L'ancien épinglage flex `order: -1` (première itération de #91) est **remplacé** :
l'ordre est redevenu chronologique, l'ancrage est fait par défilement explicite.
- Les pouces haut/bas Notion ne sont **pas** repris : aucun signal n'existe
côté backend ; « Ajouter » (insertion éditeur réelle) les remplace utilement.
## G. Section « steps » enrichie + outils web (complément #91) — ✅ livré
- [x] **G1.** Chaque étape est décrite par le **backend** (`backend/tools/labels.py`) :
événement SSE `tool` avec `step {key, params}` ; le frontend résout
`ai.step.<key>` (FR/EN) → phrases humaines : « Recherche dans le vault : pizza »,
« Fichier lu : notes/a.md », etc. Outil inconnu → libellé générique (jamais cassé).
- [x] **G2.** Événements en **direct** : les steps sont poussés sur une file
`asyncio.Queue` par la boucle agent et émis dès leur exécution — le bloc
« N étapes » grandit pendant que l'assistant travaille (plus de liste fin de run).
- [x] **G3.** Sous-section **« Réflexion ▶ / ▼ »** : quand le modèle émet un texte
intermédiaire avec ses appels d'outils, il est publié comme event SSE `step` et
rendu comme une **sous-section repliable** à l'intérieur du bloc d'étapes —
le chevron passe de ▶ à ▼ à l'ouverture et le texte (jusqu'à 1 200 caractères)
s'affiche en dessous, en retrait sur un filet vertical.
- [x] **G4.** Nouveaux outils principaux côté **web** (`backend/tools/web.py`,
risques READ, scope IN_APP, SSRF-guard + limite de taille) :
`web_search` (SearXNG auto-hébergé, configurable via `OBSIGATE_SEARXNG_URL`)
et `fetch_url` (lecture d'une page publique, HTML → texte).
Steps affichés : « Recherche sur le web : … » / « Page web consultée : … ».
- [x] **G5.** **Sources web** : le backend enrichit l'événement SSE `tool` de
`sources [{title, url}]` (`_tool_sources` : résultats `web_search` plafonnés à 8,
page `fetch_url`) et le frontend affiche une sous-section « Sources (N) » **ouverte
par défaut** avec les liens cliquables (`target="_blank"`, `rel="noopener
noreferrer"`). Aucun résultat → pas de section.
- [x] **G6.** **En-tête d'étapes** : libellé « N étapes » suivi du chevron
▶ / ▼ (fin du préfixe « > »), et **indicateur animé** (trois points en
pulsation décalée, pur CSS, respecte `prefers-reduced-motion`) devant le libellé
pendant l'exécution. La **barre de chargement** au-dessus de la zone de saisie est
supprimée ; en chat simple, l'indicateur s'affiche dans la bulle de réponse en
attente. Les états d'ouverture (étapes, réflexion, sources) sont mémorisés sur le
message : le re-rendu d'un token ne referme jamais ce que l'utilisateur a ouvert.
- [x] **G7.** `web_search` avertit explicitement le modèle quand l'instance SearXNG
ne remonte aucun résultat (moteurs amont suspendus/CAPTCHA) : `warning` +
`unresponsive_engines` dans le résultat — sans ce signal, l'assistant relançait la
même recherche jusqu'au quota d'outils.
### Outils restants — documentés pour le futur (hors #91)
La catégorie Notion « étapes » peut s'étendre ; chaque futur outil devra être un
tool du registre (`@tool`) + un libellé dans `labels.py` + deux clés i18n.
Priorités proposées (à transformer en items `#NN` quand implémentés) :
- **Créer/supprimer un fichier depuis une réponse** : déjà couvert par les cartes
d'action `create_file` / l'outil `create_file` — exposer une action « Créer la
note » dans la barre d'actions (post-traitement du texte copié).
- **Insérer dans le document courant** : l'équivalent « Ajouter » côté assistant
est livré (G) ; l'export vers une sélection précise de l'éditeur reste possible.
- **Convertir un format / tableur / document** : outils `convert_format`,
`create_spreadsheet` (csv/xlsx via backend) — risque WRITE (confirmation).
- **Calendrier / tâches** : intégrer l'API existante `n8n-automation` ou un
serveur MCP externe (la porte MCP est déjà ouverte côté ObsiGate).
- **Courriel / messagerie** : via le serveur MCP externe (Gmail/IMPT SMTP) —
jamais de clé en dur : passer par Infisical.
- **Sources connectées (Slack, GitHub, Drive)** : uniquement par **MCP externe**
(`docs/features/ai-tools-mcp.md`), pas d'outils natifs — l'attaque surface
reste dans le registre + rate-limit + audit.
+177
View File
@@ -0,0 +1,177 @@
# #94–#97 — Assistant IA : historique permanent, accès sidebar, bouton rond et panneau « + »
> **Statut :** ✅ Livré (en attente de validation utilisateur)
> **Effort :** ~7 jours | **Impact :** 🟡
> **Références :** [Roadmap](../ROADMAP.md) · [Changelog](../../CHANGELOG.md)
- **Description :** quatre améliorations de l'Assistant IA livrées ensemble :
1. **#94** — bouton de soumission **circulaire** avec icône Lucide `arrow-up`
(remplace l'avion ✈️).
2. **#95** — **historique permanent** des conversations **persisté côté backend**
(les échanges survivent aux rechargements de page et aux navigateurs).
3. **#96** — accès rapide à l'**historique depuis la sidebar de navigation** (onglet dédié).
4. **#97** — panneau **« + » extensible** remplaçant « Attach an image » (modules :
fichiers, image, contextes, skills, Deep Research, web, Canva).
## A. Backend — persistance des conversations (#95) — ✅ livré
- [x] **`backend/ai_history.py`** (nouveau) : stockage JSON par utilisateur
(`data/ai_history/{username}.json`), écriture atomique (tmp + `shutil.move`),
plafond `MAX_SESSIONS = 200` (purge des plus anciennes), tri par `updatedAt` desc.
- [x] `list_sessions`, `get_session`, `upsert_session`, `delete_session` ;
la liste renvoie des **résumés sans messages** (`_summary` : titre, mode, vault,
contexte, `message_count`, `preview`) pour rester légère.
- [x] **`backend/bookslm_routes.py`** : modèle Pydantic `BookslmSession` et 4 endpoints :
- `GET /api/ai/bookslm/history` — liste des conversations (résumés).
- `GET /api/ai/bookslm/history/{session_id}` — conversation complète (404 si absente).
- `PUT /api/ai/bookslm/history/{session_id}` — création/mise à jour (l'`id` du corps
est forcé à l'`id` du path → jamais d'écriture sous une autre clé).
- `DELETE /api/ai/bookslm/history/{session_id}` — suppression (booléen `ok`).
- [x] Isolation par utilisateur (`require_auth`) ; saisies défensives (username/id vides) →
`[]` ou `None`.
## B. Frontend — sync serveur + cache local (#95) — ✅ livré
- [x] `frontend/js/bookslm.js` : clé local globale `bookslm-sessions-all-{username}` ;
**migration** des anciennes clés périmées `bookslm-sessions-<ctx>` et
`bookslm-history-<ctx>` à la première ouverture.
- [x] `_loadHistory(preferredSessionId)` **async** au chargement du panneau :
lecture serveur (`GET /history`), hydratation du localStorage, repli hors-ligne
silencieux si le serveur ne répond pas.
- [x] Synchronisation **debounced 600 ms** (`_syncHistoryToServer` → `_flushHistoryToServer`)
via `_dirtySessionIds` / `_deletedIds` : `PUT`/`DELETE` seulement pour les sessions
modifiées ; pas d'appel réseau à la simple ouverture.
- [x] `_positionSession` : nouvelle session insérée en tête, session rechargée remise à jour.
- [x] `openContext(opts, preferredSessionId)` **async** ; `openWithSession(sessionOrId)`
public pour ouvrir une conversation connue (utilisé par la sidebar #96).
- [x] `_notifyHistoryChanged()` : événement `bookslm:history-updated` pour rafraîchir la
sidebar #96 sans rechargement.
## C. Frontend — sidebar & historique (#96) — ✅ livré
- [x] `frontend/index.html` : cinquième onglet `#sidebar-tab-ai` (icône Lucide
`messages-square`) et panneau `#sidebar-panel-ai` (`#ai-history-list`,
`#ai-history-empty`).
- [x] `frontend/js/config.js` : `loadAISessionList()` / `renderAIHistoryList()` (réutilise
les classes `.recent-*` de l'UI), placeholder « Aucune conversation », listener
`bookslm:history-updated` monté par `initSidebarTabs()` et actif quand l'onglet `ai`
est affiché.
- [x] Le clic sur une conversation l'ouvre dans le panneau Assistant IA
(via `openWithSession`) et bascule la sidebar.
## D. Frontend — panneau « + » extensible (#97) — ✅ livré
- [x] Bouton « Attach an image » remplacé par un bouton **« + »** circulaire
(`frontend/js/bookslm.js`, `.bookslm-btn-plus`).
- [x] Panneau overlay `.bookslm-ext-menu` (dropdown au-dessus de la zone de saisie,
fermeture au clic extérieur / Échap) listant les modules.
- [x] Architecture **modulaire** : registre `_extensions` (objet d'enregistrement
simple) — chaque entrée : `id`, icône, libellé i18n, action ;
`renderExtMenu()` construit la liste ; un nouveau module s'ajoute en une entrée.
- [x] Modules livrés : **Fichiers** (sélecteur général `.bookslm-files-input`),
**Image** (joindre une image), **Contextes/fichiers**, **Skills**,
**Deep Research** (mode agent + prompt pré-rempli), **Recherche sur Internet**,
**Canva**.
- [x] `web`/`canva` affichés **désactivés** avec badge « Bientôt » (`.bookslm-ext-soon`)
— le catalogue d'outils #92 les alimentera.
- [x] `_startDeepResearch()` : active le mode agent, injecte le prompt Deep Research et
déclenche l'envoi.
## E. UI — bouton rond & icône (#94) — ✅ livré
- [x] `.bookslm-btn-send` circulaire (40 px, `border-radius: 50%`), icône Lucide
`arrow-up` à la place de l'emoji ✈️, aligné en bas à droite de la zone de saisie.
- [x] i18n FR/EN : `bookslm.add_options`, `bookslm.ext_files`, `bookslm.ext_image_hint`,
`bookslm.ext_contexts`, `bookslm.ext_skills`, `bookslm.ext_deep_research`,
`bookslm.ext_web`, `bookslm.ext_canva`, `bookslm.coming_soon`, `bookslm.soon`,
`bookslm.ext_more_coming`, `bookslm.deep_research_prompt`,
`bookslm.deep_research_started`, `bookslm.mode_*`.
## F. Tests — ✅ livré
- [x] `tests/test_bookslm.py` : `TestBooksLMSessionHistoryEndpoints` (7) + `TestAIGatewayHistoryStore` (4) —
liste résumée sans messages, full 404, upsert qui force l'id et isole par utilisateur,
delete, cap à 200, purge.
- [x] `tests/frontend/ai.test.mjs` : persistance/reouverture/suppression de sessions,
migration de l'historique legacy, menu d'historique.
- [x] Vérifs : pytest **1088 passed / 6 skipped**, ruff 0, mypy 0 (71 fichiers),
frontend `ai.test.mjs` **84/84**, `unit.test.mjs` 9/9, `validate-imports` 38 modules.
## H. Complément — filtre de la sidebar et menu « + » (BUG-048, #98) — ✅ livré
En retour utilisateur sur la version 2.4.0 :
*(1) **Filtre fonctionnel sur l'onglet « Historique IA » (sidebar) — #98** — la barre de
filtrage globale de la sidebar agissait uniquement sur les onglets Fichiers/Tags. Elle
s'applique désormais aussi à l'historique IA :*
- Utile `filterAIHistory(query)` (`frontend/js/config.js`) : filtre le cache de sessions
`_aiSessionsCache` (peuplé par `loadAISessionList()`) sur **titre, aperçu, répertoire,
contexte ou libellé de mode** traduit, insensible à la casse et aux accents
(`_aiNorm` : `NFD` + suppression des diacritiques + `toLowerCase`). Une requête sans
résultat affiche `bookslm.history_no_match` (nouvelle clé i18n FR/EN) **dans la liste**,
en plus de l'état vide d'origine ; un champ vide restaure tout.
- Routage dans `initSidebarFilter` (`frontend/js/sidebar.js`) : la saisie (debounce 220 ms),
le bouton casse `Aa` et le bouton « × » dirigent vers `filterAIHistory` quand l'onglet IA
est actif (`state.activeSidebarTab`), au lieu de `filterTagCloud`. Le placeholder devient
`sidebar.filter_ai` (« Filtrer l'historique IA… » / « Filter AI history… ») — résolu dans
`switchSidebarTab` (config.js), qui re-charge aussi la liste à chaque entrée dans l'onglet
en ré-appliquant la requête courante.
*(2) **Menu « + » du panneau Assistant — BUG-048** — « ajouter des contextes » ouvrait un
menu « @ » jamais affiché (et idem pour « ajouter des skills » → « / ») :* le clic sur une
entrée du panneau `.bookslm-ext-menu` remontait au gestionnaire `click` du panneau
(`_hideMenus()` + `_menuSeq++`), qui annulait le rendu asynchrone du menu ouvert juste après.
`e.stopPropagation()` est ajouté sur chaque entrée de `_renderExtMenu()` (bookslm.js) :
les menus « @ » (contextes) et « / » (skills) restent affichés. Le bouton d'ouverture porte
bien l'icône Lucide `plus` (vérifié par test JSDOM).
**Points d'attention**
- Le filtrage reste **client-side** : la charge est triviale vu le cap de 200 sessions
(`MAX_SESSIONS`). Un futur filtrage serveur n'est justifié que si la rétention croît.
- `filterAIHistory` est exportée en plus de `loadAISessionList` : `validate-imports` couvre le
nouveau contrat d'import de `sidebar.js`.
### Tests du complément
- `tests/frontend/ai-sidebar.test.mjs` (nouveau, 6 tests) : rendu complet, filtre par titre
(casse), accents (« cafe » → « café au lait »), aperçu/répertoire, restauration complète,
message « aucune correspondance ».
- `tests/frontend/ai.test.mjs` (+3) : clic « Contextes » → menu « @ » visible listé, clic
« Skills » → menu « / » visible listé, icône `plus` présente sur le bouton « + ».
- Vérifs : frontend IA **87/87**, `ai-sidebar` **6/6**, `unit` 9/9, 9 suites JSDOM vertes,
`validate-imports` 38 modules ; backend inchangé (pytest / ruff / mypy valides).
## G. Points d'attention
- Le localStorage reste un **cache** : le serveur est la source de vérité (`data/ai_history/`).
En cas de données locales corrompues, un `localStorage.clear()` réinitialise proprement.
- `MAX_SESSIONS = 200` : le comportement de purge est testé (`TestAIGatewayHistoryStore`)
; la rétention configurable (item #95 du backlog) pourra s'appuyer sur ce plafond.
- Les modules web/Canva du panneau « + » sont volontairement désactivés tant que
l'écosystème d'outils phase 2 (#92) n'est pas livré.
## I. Complément — icône « + » et pastille Deep Research (BUG-049, #100) — ✅ livré
*(1) **BUG-049 — l'icône du bouton « + » n'était pas visible.*** Le SVG Lucide était
bien rendu, mais la règle générique `.bookslm-input-area button { padding: 8px 16px;
background: var(--accent); color:#fff }` l'emportait en **spécificité** sur
`.bookslm-btn-plus` (une classe seule). Le bouton conservait `width:32px` avec
`padding: 8px 16px` → **largeur de contenu = 0 px**, donc SVG à `width: 0px`
(invisible). Correctif CSS : sélecteur porté à
`.bookslm-input-area button.bookslm-btn-plus` (et `:hover`), qui reprend la main
(`padding:0`, fond transparent, couleur `--text-secondary`). Vérifié en navigateur
(Playwright) : `svgWidth` passe de `0px` à `18px`.
*(2) **#100 — Deep Research devient une pastille (comme les skills).*** Auparavant,
cliquer sur « Deep Research » injectait la directive de recherche dans la zone de
saisie. Désormais :
- `_startDeepResearch()` active le **mode Agent** (si nécessaire), positionne le drapeau
`_activeDeepResearch` et rend une **pastille** `.bookslm-chip-deep-research`
(`_renderAttachments()`), sans rien écrire dans le composeur.
- La pastille se retire via son « × » (comme les chips skills/fichiers) et remet le
drapeau à `false`.
- La directive (`bookslm.deep_research_prompt`) est injectée **au moment de l'envoi**
dans le `message` du payload (`_sendMessage()`), sans polluer le message affiché à
l'utilisateur.
- Message d'information mis à jour (`bookslm.deep_research_started`) : « Deep Research
activé — ajoutez votre question puis envoyez. »
### Tests du complément
- `tests/frontend/ai.test.mjs` (+1) : le clic sur « Deep Research » ajoute une pastille,
laisse le composeur vide, positionne le drapeau, et le retrait de la pastille remet
le drapeau à `false`.
- Vérification navigateur du bouton « + » (Playwright, instance de test).

Some files were not shown because too many files have changed in this diff Show More