Compare commits

...
7 Commits
42 changed files with 2436 additions and 1312 deletions
+5
View File
@@ -13,6 +13,9 @@ OBSIGATE_ADMIN_PASSWORD=chab30
# OBSIGATE_ALLOW_INSECURE=false
# Sécurité des cookies (activer si derrière HTTPS)
# false par défaut : les navigateurs ignorent les cookies `Secure` en HTTP,
# ce qui casserait les logins en local. En production (TLS + bind réseau),
# posez true — un avertissement est loggé au démarrage sinon (#87).
# OBSIGATE_SECURE_COOKIES=false
# Tokens TTL en secondes
@@ -23,6 +26,8 @@ OBSIGATE_ADMIN_PASSWORD=chab30
# OBSIGATE_LOGIN_MAX_ATTEMPTS=10
# OBSIGATE_ACCOUNT_MAX_ATTEMPTS=10
# OBSIGATE_LOGIN_WINDOW_SECONDS=900
# Compteurs partagés/persistants (SQLite WAL, multi-workers) — défaut : mémoire.
# OBSIGATE_RATELIMIT_DB=data/ratelimit.db
# IP client derrière un reverse proxy (fait confiance à X-Forwarded-For)
# OBSIGATE_TRUST_PROXY=false
+18 -4
View File
@@ -44,6 +44,11 @@ jobs:
node tests/frontend/config-mobile.test.mjs
node tests/frontend/settings-order-avatar.test.mjs
node tests/frontend/mobile-toolbar.test.mjs
node tests/frontend/upload.test.mjs
node tests/frontend/pretty.test.mjs
node tests/frontend/media-viewer.test.mjs
node tests/frontend/mfa-settings.test.mjs
node tests/frontend/config-ai-keys.test.mjs
- name: Frontend JSDOM tests (PaneManager + Excalidraw + Plugins + AI + SW + Collab + Mobile + Semantic + Desktop + Inline edition)
run: |
@@ -126,11 +131,17 @@ jobs:
pip install bandit pip-audit
pip install -r backend/requirements.txt
- name: Bandit (SAST)
run: bandit -r backend/ --skip B101,B110,B310 || echo "bandit found issues (non-blocking)"
- name: Bandit (SAST, bloquant — #87)
# B105 est exclu (aligné avec [tool.bandit] de pyproject.toml :
# faux positifs systématiques sur les noms de variables) ; les rares
# vrais positifs restants portent un `# nosec` justifié inline.
run: bandit -r backend/ --skip B101,B105,B110,B310
- name: Pip-audit (dependency vulnerabilities)
run: pip-audit || echo "pip-audit found vulnerabilities (non-blocking)"
- name: Pip-audit (consultatif — #87)
# Reste non bloquant tant que les montées de version requises
# (starlette via fastapi, weasyprint) ne sont pas qualifiées :
# upgrade FastAPI = chantier de régression dédié, hors périmètre.
run: pip-audit || echo "pip-audit found vulnerabilities (non-blocking, see #87)"
# ── Docker build ──────────────────────────────────────────────────
build:
@@ -192,6 +203,9 @@ jobs:
npm ci
npx playwright install --with-deps chromium
- name: Npm audit (bloquant — #87, 0 dépendance prod hors Playwright)
run: npm audit --omit=dev
- name: Start ObsiGate
run: |
docker rm -f obsigate-e2e 2>/dev/null || true
+84 -1
View File
@@ -6,7 +6,7 @@ Format basé sur [Keep a Changelog](https://keepachangelog.com/fr/1.1.0/),
et [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
> **En cours de développement** : les changements à venir sont listés dans la section
> [Unreleased](#unreleased). La dernière version livrée est **2.27.9**.
> [Unreleased](#unreleased). La dernière version livrée est **2.28.3**.
---
@@ -14,6 +14,60 @@ et [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
---
## [2.28.3] — 2026-09-26
---
## [2.28.2] — 2026-09-26
### Ajouté
- **#87 (T3) — cookies `Secure` et CORS explicites.**
Helper `is_secure_cookies()` centralisé (défaut `false` conservé pour ne
pas casser les logins HTTP locaux) + avertissement au démarrage sur bind
non-loopback sans `Secure` ; `tests/test_security_headers.py` atteste
l'absence de CORS permissif (same-origin par défaut du navigateur) et les
en-têtes de durcissement.
- **#87 (T2) — tests de durcissement : concurrence et regex.**
`tests/test_hardening_concurrency.py` : créations/mises à jour/`login
failures` concurrents sur `users.json` (zéro mise à jour perdue, JSON
toujours récupérable) et budget temps de la politique ReDoS (motifs
catastrophiques rejetés en < 1 s, motifs acceptés < 5 s sur 200 Ko).
---
## [2.28.1] — 2026-09-26
### Modifié
- **#87 (T1) — CI sécurité durcie.**
`bandit` passe en bloquant (`# nosec` justifiés : SHA1 non-crypto,
subprocess git à argv fixe, `saxutils.escape` sans parsing ; B105 exclu
comme `pyproject.toml`) ; `npm audit --omit=dev` bloquant (0 faille) ;
les 5 suites frontend hors CI (`upload`, `pretty`, `media-viewer`,
`mfa-settings`, `config-ai-keys`, vertes en local) rejoignent le job
`lint`. `pip-audit` reste consultatif (upgrades starlette/weasyprint à
qualifier, chantier dédié).
---
## [2.28.0] — 2026-09-26
---
## [2.27.12] — 2026-09-26
---
## [2.27.11] — 2026-09-26
---
## [2.27.10] — 2026-09-26
---
## [2.27.9] — 2026-09-26
---
@@ -46,6 +100,35 @@ et [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
### Modifié
- **#85 (T10) — persistance d'état et clôture de la refonte architecturale.**
Verrous `RLock` sur les stores JSON sans protection (`revoked_tokens`,
`shares`, `webhooks` + secrets, clés d'outils) avec tests de concurrence
(`tests/test_store_locks.py` — pertes prouvées sans verrou) ; rate-limit
auth persisté en option (`OBSIGATE_RATELIMIT_DB`, SQLite WAL, sémantique
identique, défaut mémoire inchangé, `tests/test_ratelimit_store.py`).
Contrat `tools/registry.py` audité (permissions/quotas/redaction déjà
câblés, rien à coder). Index non persisté : rebuild différentiel #86
suffisant (décision documentée). Fiche `docs/features/archi-refonte-85.md`,
#85 sorti du backlog (index roadmap).
- **#85 (T9) — extraction realtime + render hors du monolithe `backend/main.py`.**
Le stream SSE `/api/events` et le WebSocket `/ws/collab/*` sont servis par
`backend/routers/realtime.py`, le pipeline markdown (mistune, wikilinks,
slugs, sanitizer) par `backend/render.py` (imports directs, plus de
couplage différé). `main.py` (4 827 → ~760 lignes) ne contient plus que
l'assemblage : lifespan, middlewares, montage des 16 routers, racine
`/api`, statique/SPA et cales de compatibilité testées.
- **#85 (T8) — extraction vaults/history/conflicts hors du monolithe `backend/main.py`.**
13 routes servies par `backend/routers/vaults.py`, `history.py` et
`conflicts.py` ; `VaultInfo`/`BookmarkToggleRequest` dans `schemas.py`,
handle watcher partagé dans `backend/watcher_state.py`.
`tests/test_api_main.py` importe `humanize_mtime` depuis son module
canonique (`services.recent`).
- **#85 (T7) — extraction du domaine `config` hors du monolithe `backend/main.py`.**
`/api/config`, ai-keys (get/post/delete/test), tool-keys (×3), ai-models,
diagnostics et dashboard sont servis par `backend/routers/config.py`
(`_FALLBACK_MODELS`, store clés et config déplacés ; `main` réimporte
`_load_config` pour son lifespan, les fixtures de tests inchangées).
`tests/test_ai_models.py` patch désormais la référence du router.
- **#85 (T6c) — extraction media/pdf/export/guide hors du monolithe `backend/main.py`.**
file/pdf, exports (html/md-bundle/epub), guide/download, pdf/stream|info,
image, media+thumb, attachments (rescan/stats), vault settings (get/post/all)
+3 -3
View File
@@ -4,7 +4,7 @@
**Porte d'entrée web ultra-léger pour vos vaults Obsidian** — Accédez, naviguez et recherchez dans toutes vos notes Obsidian depuis n'importe quel appareil via une interface web moderne et responsive.
[![Version](https://img.shields.io/badge/Version-2.27.9-blue.svg)]()
[![Version](https://img.shields.io/badge/Version-2.28.3-blue.svg)]()
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Docker](https://img.shields.io/badge/Docker-Ready-blue.svg)](https://www.docker.com/)
[![Python](https://img.shields.io/badge/Python-3.11+-green.svg)](https://www.python.org/)
@@ -976,8 +976,8 @@ Ce projet est sous licence **MIT** — voir le fichier [LICENSE](LICENSE) pour l
## 📝 Changelog
Consultez le [CHANGELOG.md](./CHANGELOG.md) pour l'historique complet de toutes les versions (v1.0.0 → v2.27.9).
Consultez le [CHANGELOG.md](./CHANGELOG.md) pour l'historique complet de toutes les versions (v1.0.0 → v2.28.3).
---
*Projet : ObsiGate | Version : 2.27.9 | Dernière mise à jour : Septembre 2026*
*Projet : ObsiGate | Version : 2.28.3 | Dernière mise à jour : Septembre 2026*
+3 -3
View File
@@ -2,7 +2,7 @@
**Ultra-light web gateway for your Obsidian vaults** — Access, browse, and search all your Obsidian notes from any device via a modern, responsive web interface.
[![Version](https://img.shields.io/badge/Version-2.27.9-blue.svg)]()
[![Version](https://img.shields.io/badge/Version-2.28.3-blue.svg)]()
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Docker](https://img.shields.io/badge/Docker-Ready-blue.svg)](https://www.docker.com/)
[![Python](https://img.shields.io/badge/Python-3.11+-green.svg)](https://www.python.org/)
@@ -1151,8 +1151,8 @@ This project is licensed under the **MIT License** - see the [LICENSE](LICENSE)
## 📝 Changelog
See [CHANGELOG.md](./CHANGELOG.md) for the complete version history (v1.0.0 → v2.27.9).
See [CHANGELOG.md](./CHANGELOG.md) for the complete version history (v1.0.0 → v2.28.3).
---
*Project: ObsiGate | Version: 2.27.9 | Last updated: September 2026*
*Project: ObsiGate | Version: 2.28.3 | Last updated: September 2026*
+1 -1
View File
@@ -1 +1 @@
2.27.9
2.28.3
+33 -26
View File
@@ -119,25 +119,30 @@ def decode_token(token: str) -> dict | None:
_revoked_map: dict[str, int] = {}
_revoked_loaded = False
# ROADMAP #85 T10a — verrou autour du read-modify-write du store de
# révocation (perte de révocations en cas de logouts concurrents).
_revoked_lock = threading.RLock()
def _load_revoked():
"""Load revoked token JTIs from disk into memory (once)."""
global _revoked_loaded, _revoked_map
if _revoked_loaded:
return
if REVOKED_TOKENS_FILE.exists():
try:
data = json.loads(REVOKED_TOKENS_FILE.read_text())
# Drop entries whose underlying token has itself expired.
now = int(time.time())
_revoked_map = {
jti: int(exp) for jti, exp in data.items()
if int(exp) > now
}
except Exception as e:
logger.warning(f"Failed to load revoked tokens: {e}")
_revoked_map = {}
_revoked_loaded = True
with _revoked_lock:
if _revoked_loaded:
return
if REVOKED_TOKENS_FILE.exists():
try:
data = json.loads(REVOKED_TOKENS_FILE.read_text())
# Drop entries whose underlying token has itself expired.
now = int(time.time())
_revoked_map = {
jti: int(exp) for jti, exp in data.items()
if int(exp) > now
}
except Exception as e:
logger.warning(f"Failed to load revoked tokens: {e}")
_revoked_map = {}
_revoked_loaded = True
def _save_revoked():
@@ -154,24 +159,26 @@ def revoke_token(jti: str, expires_at: int | None = None):
``expires_at`` is the revoked token's own ``exp`` (unix seconds) — the
record is kept at least that long so a long-lived API token cannot
outlive its revocation. ``None`` means the token never expires (API/MCP
"sans fin") → the record is kept forever (capped at ~100 years, the JWT
"sans fin") → the record is kept forever (capped at ~100 years, the JWT
store's practical infinity). Default keeps 7 days (session tokens).
"""
_load_revoked()
now = int(time.time())
if expires_at is None:
until = now + 100 * 365 * 24 * 3600
else:
until = max(int(expires_at), now + REFRESH_TOKEN_EXPIRE_SECONDS)
_revoked_map[jti] = until
_save_revoked()
with _revoked_lock:
_load_revoked()
now = int(time.time())
if expires_at is None:
until = now + 100 * 365 * 24 * 3600
else:
until = max(int(expires_at), now + REFRESH_TOKEN_EXPIRE_SECONDS)
_revoked_map[jti] = until
_save_revoked()
logger.debug(f"Revoked token JTI: {jti[:8]}...")
def is_token_revoked(jti: str) -> bool:
"""Check if a token JTI has been revoked."""
_load_revoked()
return jti in _revoked_map
with _revoked_lock:
_load_revoked()
return jti in _revoked_map
# ---------------------------------------------------------------------------
+18 -7
View File
@@ -5,6 +5,7 @@
import base64
import binascii
import logging
import os
import re
from fastapi import APIRouter, Body, Depends, HTTPException, Request, Response
@@ -56,6 +57,17 @@ logger = logging.getLogger("obsigate.auth.router")
router = APIRouter(prefix="/api/auth", tags=["auth"])
def is_secure_cookies() -> bool:
"""True when auth cookies must carry the ``Secure`` flag (#87 T3).
Opt-in via ``OBSIGATE_SECURE_COOKIES=true`` (required behind TLS).
Default stays ``false`` so logins keep working over plain HTTP on
trusted loopback deployments — browsers drop ``Secure`` cookies sent
over HTTP, which would silently break localhost logins.
"""
return os.environ.get("OBSIGATE_SECURE_COOKIES", "false").lower() == "true"
# ── Pydantic request models ──────────────────────────────────────────
class LoginRequest(BaseModel):
@@ -229,9 +241,8 @@ def _issue_tokens(user: dict, username: str, remember_me: bool, response: Respon
access_token = create_access_token(user)
refresh_token, refresh_jti = create_refresh_token(username, remember=remember_me)
import os
max_age = 2592000 if remember_me else 604800 # 30d or 7d
secure = os.environ.get("OBSIGATE_SECURE_COOKIES", "false").lower() == "true"
secure = is_secure_cookies()
response.set_cookie(
key="refresh_token",
value=refresh_token,
@@ -253,7 +264,8 @@ def _issue_tokens(user: dict, username: str, remember_me: bool, response: Respon
)
return {
"access_token": access_token,
"token_type": "bearer", # nosec B105 — OAuth2 token_type, pas un mot de passe
# OAuth2 token_type, pas un mot de passe (B105) :
"token_type": "bearer", # nosec B105
"expires_in": ACCESS_TOKEN_EXPIRE_SECONDS,
"user": {
"username": user["username"],
@@ -299,9 +311,7 @@ async def refresh_token_endpoint(request: Request, response: Response):
if stale:
raise HTTPException(401, "Session expirée, veuillez vous reconnecter")
import os
secure = os.environ.get("OBSIGATE_SECURE_COOKIES", "false").lower() == "true"
secure = is_secure_cookies()
remember_me = bool(payload.get("remember", False))
# BUG-027: rotate the refresh token — the old one is now single-use.
@@ -332,7 +342,8 @@ async def refresh_token_endpoint(request: Request, response: Response):
return {
"access_token": new_access_token,
"token_type": "bearer", # nosec B105 — OAuth2 token_type, pas un mot de passe
# OAuth2 token_type, pas un mot de passe (B105) :
"token_type": "bearer", # nosec B105
"expires_in": ACCESS_TOKEN_EXPIRE_SECONDS,
}
+2 -1
View File
@@ -35,7 +35,8 @@ def diagram_png_for(code: str) -> Path | None:
Mermaid, ou None. Le hash doit rester synchrone avec le script de build :
sha1(unescape(code).strip())[:16]."""
normalized = html.unescape(code).strip()
sha = hashlib.sha1(normalized.encode("utf-8")).hexdigest()[:16]
# Identifiant de cache déterministe (pas un usage sécurité).
sha = hashlib.sha1(normalized.encode("utf-8")).hexdigest()[:16] # nosec B324
png = DIAGRAMS_DIR / (sha + ".png")
return png if png.exists() else None
+62 -1107
View File
File diff suppressed because it is too large Load Diff
+2 -1
View File
@@ -32,7 +32,8 @@ def thumb_cache_path(file_path: Path, size: int) -> Path:
stamp = f"{st.st_mtime_ns}:{st.st_size}"
except OSError:
stamp = "0:0"
key = hashlib.sha1(f"{file_path}:{stamp}:{size}".encode()).hexdigest()
# Clé de cache miniature (pas un usage sécurité).
key = hashlib.sha1(f"{file_path}:{stamp}:{size}".encode()).hexdigest() # nosec B324
return thumbs_cache_dir() / f"{key}.webp"
+172 -1
View File
@@ -12,14 +12,24 @@ the per-account lockout in ``user_store.py``.
deployment, front this service with a shared store (Redis) or a single
worker. This limitation is intentional and documented (BUG-031).
Opt-in persistence (ROADMAP #85 T10b) : if ``OBSIGATE_RATELIMIT_DB`` points
to a SQLite file, counters are stored there instead (WAL mode, one short
connection per call — safe across threads, processes and restarts sharing
the same file). Semantics (windows, budgets, success reset) are identical
to the in-memory store, which remains the default when the variable is
unset.
Configuration via environment variables:
OBSIGATE_LOGIN_MAX_ATTEMPTS Max failures per IP (default: 10)
OBSIGATE_ACCOUNT_MAX_ATTEMPTS Max failures per account (default: 10)
OBSIGATE_LOGIN_WINDOW_SECONDS Lockout window in seconds (default: 900)
OBSIGATE_RATELIMIT_DB SQLite file for shared/persistent counters (default: unset = memory)
"""
import logging
import os
import sqlite3
import threading
import time
from collections import defaultdict
@@ -37,6 +47,127 @@ _last_cleanup = time.time()
CLEANUP_INTERVAL = 60 # seconds
def _db_path() -> str | None:
"""SQLite file for shared counters, or ``None`` for the in-memory store."""
path = os.environ.get("OBSIGATE_RATELIMIT_DB", "").strip()
return path or None
def _db_connect(path: str) -> sqlite3.Connection:
"""Open a short-lived connection (WAL + busy timeout for concurrent workers)."""
_db_ensure_schema(path)
conn = sqlite3.connect(path, timeout=10.0)
conn.execute("PRAGMA busy_timeout=10000")
return conn
_schema_ready: set[str] = set()
_schema_lock = threading.Lock()
def _db_ensure_schema(path: str) -> None:
"""Create the store schema once per file (DDL under a process-wide lock)."""
with _schema_lock:
if path in _schema_ready:
return
conn = sqlite3.connect(path, timeout=10.0)
try:
conn.execute("PRAGMA journal_mode=WAL")
conn.execute(
"CREATE TABLE IF NOT EXISTS attempts"
" (kind TEXT NOT NULL, key TEXT NOT NULL, ts REAL NOT NULL, success INTEGER NOT NULL)"
)
conn.execute(
"CREATE INDEX IF NOT EXISTS idx_attempts_kind_key_ts"
" ON attempts (kind, key, ts)"
)
conn.commit()
finally:
conn.close()
_schema_ready.add(path)
def _db_write(fn, *args):
"""Run a write op, retrying once on lock contention (concurrent workers)."""
try:
return fn(*args)
except sqlite3.OperationalError as e:
if "locked" not in str(e).lower():
raise
time.sleep(0.05)
return fn(*args)
def _db_prune(conn: sqlite3.Connection, cutoff: float) -> None:
"""Drop expired entries (best-effort cap on disk growth)."""
conn.execute("DELETE FROM attempts WHERE ts <= ?", (cutoff,))
def _db_record(kind: str, key: str, success: bool) -> int:
"""Record one attempt in SQLite; return the live failure count."""
path = _db_path()
assert path is not None
now = time.time()
cutoff = now - WINDOW_SECONDS
def _write() -> int:
with _db_connect(path) as conn:
_db_prune(conn, cutoff)
if success:
# Mirror the in-memory reset: replace history with one success.
conn.execute("DELETE FROM attempts WHERE kind = ? AND key = ?", (kind, key))
conn.execute(
"INSERT INTO attempts (kind, key, ts, success) VALUES (?, ?, ?, ?)",
(kind, key, now, int(success)),
)
conn.commit()
(failures,) = conn.execute(
"SELECT COUNT(*) FROM attempts WHERE kind = ? AND key = ? AND ts > ? AND success = 0",
(kind, key, cutoff),
).fetchone()
return failures
return _db_write(_write)
def _db_failures(kind: str, key: str) -> int:
"""Live failure count in SQLite (expired entries never count)."""
path = _db_path()
assert path is not None
cutoff = time.time() - WINDOW_SECONDS
with _db_connect(path) as conn:
(failures,) = conn.execute(
"SELECT COUNT(*) FROM attempts WHERE kind = ? AND key = ? AND ts > ? AND success = 0",
(kind, key, cutoff),
).fetchone()
return failures
def _db_tracked(kind: str) -> int:
"""Number of distinct keys ever seen for one budget (SQLite)."""
path = _db_path()
assert path is not None
with _db_connect(path) as conn:
(n,) = conn.execute(
"SELECT COUNT(DISTINCT key) FROM attempts WHERE kind = ?", (kind,)
).fetchone()
return n
def _db_limited_count(kind: str, max_attempts: int) -> int:
"""Number of keys currently over budget (SQLite)."""
path = _db_path()
assert path is not None
cutoff = time.time() - WINDOW_SECONDS
with _db_connect(path) as conn:
rows = conn.execute(
"SELECT key, COUNT(*) FROM attempts"
" WHERE kind = ? AND ts > ? AND success = 0 GROUP BY key",
(kind, cutoff),
).fetchall()
return sum(1 for _, n in rows if n >= max_attempts)
def _prune(store: dict[str, list], cutoff: float) -> None:
"""Drop expired entries from one store in place."""
expired = []
@@ -66,6 +197,12 @@ def record_failure(ip: str) -> tuple[int, int]:
Returns:
(current_failure_count, remaining_attempts)
"""
if _db_path() is not None:
failures = _db_record("ip", ip, False)
remaining = max(0, MAX_ATTEMPTS - failures)
if failures >= MAX_ATTEMPTS:
logger.warning(f"IP {ip} rate-limited after {failures} failed logins")
return failures, remaining
_cleanup_expired()
_ip_attempts[ip].append((time.time(), False))
failures = sum(1 for _, success in _ip_attempts[ip] if not success)
@@ -77,12 +214,17 @@ def record_failure(ip: str) -> tuple[int, int]:
def record_success(ip: str):
"""Clear rate limit state for an IP after successful login."""
if _db_path() is not None:
_db_record("ip", ip, True)
return
_cleanup_expired()
_ip_attempts[ip] = [(time.time(), True)]
def is_rate_limited(ip: str) -> bool:
"""Check if an IP has exceeded the rate limit."""
if _db_path() is not None:
return _db_failures("ip", ip) >= MAX_ATTEMPTS
_cleanup_expired()
failures = sum(1 for _, success in _ip_attempts.get(ip, []) if not success)
return failures >= MAX_ATTEMPTS
@@ -94,8 +236,14 @@ def record_account_failure(account: str) -> tuple[int, int]:
Returns:
(current_failure_count, remaining_attempts)
"""
_cleanup_expired()
key = account.lower()
if _db_path() is not None:
failures = _db_record("account", key, False)
remaining = max(0, ACCOUNT_MAX_ATTEMPTS - failures)
if failures >= ACCOUNT_MAX_ATTEMPTS:
logger.warning(f"Account {account} rate-limited after {failures} failed attempts")
return failures, remaining
_cleanup_expired()
_account_attempts[key].append((time.time(), False))
failures = sum(1 for _, success in _account_attempts[key] if not success)
remaining = max(0, ACCOUNT_MAX_ATTEMPTS - failures)
@@ -106,12 +254,17 @@ def record_account_failure(account: str) -> tuple[int, int]:
def record_account_success(account: str):
"""Clear the per-account rate limit state after a successful login."""
if _db_path() is not None:
_db_record("account", account.lower(), True)
return
_cleanup_expired()
_account_attempts[account.lower()] = [(time.time(), True)]
def is_account_rate_limited(account: str) -> bool:
"""Check if an account has exceeded the per-account rate limit."""
if _db_path() is not None:
return _db_failures("account", account.lower()) >= ACCOUNT_MAX_ATTEMPTS
_cleanup_expired()
failures = sum(
1 for _, success in _account_attempts.get(account.lower(), []) if not success
@@ -121,6 +274,24 @@ def is_account_rate_limited(account: str) -> bool:
def get_status(ip: str | None = None) -> dict:
"""Get rate limit status for an IP (for diagnostics)."""
if _db_path() is not None:
if ip:
failures = _db_failures("ip", ip)
return {
"ip": ip,
"failures": failures,
"max": MAX_ATTEMPTS,
"limited": failures >= MAX_ATTEMPTS,
"window_seconds": WINDOW_SECONDS,
}
return {
"tracked_ips": _db_tracked("ip"),
"tracked_accounts": _db_tracked("account"),
"max_attempts": MAX_ATTEMPTS,
"account_max_attempts": ACCOUNT_MAX_ATTEMPTS,
"window_seconds": WINDOW_SECONDS,
"limited_ips": _db_limited_count("ip", MAX_ATTEMPTS),
}
_cleanup_expired()
if ip:
attempts = _ip_attempts.get(ip, [])
+207
View File
@@ -0,0 +1,207 @@
"""Markdown rendering pipeline (ROADMAP #85, tranche 9).
Helpers extraits de :mod:`backend.main` sans changement de comportement :
slugification des headings, IDs d'ancrage, rendu mistune singleton,
wikilinks, normalisation des sauts de ligne et pipeline complet
:func:`_render_markdown` (rendu + sanitizer XSS BUG-021).
Les noms gardent leur préfixe ``_`` d'origine pour un déplacement
strictement verbatim (tests et routers pointent ici désormais).
"""
from __future__ import annotations
import html as html_mod
import re
import unicodedata
from pathlib import Path
import mistune
from backend.image_processor import preprocess_images
from backend.indexer import find_file_in_index, get_vault_data
from backend.secret_redactor import redact_file_content
from backend.services.sanitizer import sanitize_html
def _heading_slugify(text: str) -> str:
"""Generate a URL-safe slug from heading text.
Matches the JavaScript slugify algorithm exactly using
Unicode-aware character classification:
1. Strip HTML tags (e.g. wikilink spans rendered inside headings)
2. Decode HTML entities (e.g. ``&amp;`` → ``&``)
3. Lowercase
4. NFD normalize + strip combining marks
5. Keep only Unicode letters, numbers, spaces, hyphens
6. Replace spaces with hyphens, collapse multiple hyphens
Args:
text: The heading text content (may contain inline HTML).
Returns:
A URL-safe slug string.
"""
# Strip any inline HTML so it does not pollute the slug
text = re.sub(r"<[^>]+>", "", text)
# Decode HTML entities so &amp; becomes & before slugification
text = html_mod.unescape(text)
text = text.lower()
text = unicodedata.normalize("NFD", text)
text = "".join(ch for ch in text if not unicodedata.combining(ch))
# Unicode-aware: keep letters (L*), numbers (N*), spaces, and hyphens
cleaned = []
for ch in text:
cat = unicodedata.category(ch)
if cat.startswith('L') or cat.startswith('N') or ch in (' ', '-'):
cleaned.append(ch)
text = "".join(cleaned)
text = re.sub(r"\s+", "-", text)
text = re.sub(r"-+", "-", text)
result = text.strip("-")
return result if result else "heading"
def _add_heading_ids(html: str) -> str:
"""Post-process rendered HTML to add IDs to heading tags.
Adds an ``id`` attribute to every ``<h1>`` through ``<h6>`` tag
using a slug generated from the heading's text content.
Duplicate slugs get a ``-2``, ``-3``, etc. suffix.
Args:
html: Rendered HTML string.
Returns:
HTML with heading IDs injected.
"""
used_ids: dict[str, int] = {}
def _replace_heading(match):
tag = match.group(1)
content = match.group(2)
slug = _heading_slugify(content)
count = used_ids.get(slug, 0)
used_ids[slug] = count + 1
if count > 0:
slug = f"{slug}-{count + 1}"
return f'<{tag} id="{slug}">{content}</{tag}>'
# Match h1-h6 tags with text content (no existing id attribute)
return re.sub(
r'<(h[1-6])>([^<]*(?:<(?!/?h[1-6])[^<]*)*)</h[1-6]>',
_replace_heading,
html,
)
# Cached mistune renderer — avoids re-creating on every request
_markdown_renderer = mistune.create_markdown(
escape=False,
plugins=["table", "strikethrough", "footnotes", "task_lists"],
)
def _convert_wikilinks(content: str, current_vault: str) -> str:
"""Convert ``[[wikilinks]]`` and ``[[target|display]]`` to clickable HTML.
Supports:
- Internal file links: ``[[My Note]]`` / ``[[My Note|display]]``
- Same-document anchors: ``[[#Heading]]`` / ``[[#Heading|display]]``
Resolved file links get a ``data-vault`` / ``data-path`` attribute pair.
Anchor links target the slugified heading ID in the current document.
Unresolved links are rendered as ``<span class="wikilink-missing">``.
Args:
content: Markdown string potentially containing wikilinks.
current_vault: Active vault name for resolution priority.
Returns:
Markdown string with wikilinks replaced by HTML anchors.
"""
def _replace(match):
target = match.group(1).strip()
display = match.group(2).strip() if match.group(2) else target
# Same-document anchor link: [[#Heading|display]]
if target.startswith("#"):
anchor_text = target[1:].strip()
anchor_slug = _heading_slugify(anchor_text)
link_display = display if display != target else anchor_text
return f'<a class="wikilink-anchor" href="#{anchor_slug}">{link_display}</a>'
found = find_file_in_index(target, current_vault)
if found:
return (
f'<a class="wikilink" href="#" '
f'data-vault="{found["vault"]}" '
f'data-path="{found["path"]}">{display}</a>'
)
return f'<span class="wikilink-missing">{display}</span>'
pattern = r'\[\[([^\]|]+)(?:\|([^\]]+))?\]\]'
return re.sub(pattern, _replace, content)
def _normalize_line_breaks(text: str) -> str:
"""Convert single newlines to hard breaks (matching Obsidian default behavior).
In standard Markdown, a single ``\\n`` is a "soft break" — it renders as a space,
not a visible line break. Obsidian defaults to treating single newlines as hard
breaks (equivalent to ``<br>``). This function pre-processes the Markdown source
so that mistune renders standalone lines on separate rows, while still honouring
blank lines as paragraph separators.
Fenced code blocks (`` ``` ``) are left untouched so their internal newlines are
preserved verbatim.
"""
parts = re.split(r"(```[\s\S]*?```)", text)
for i, part in enumerate(parts):
if part.startswith("```"):
continue # Protect fenced code blocks
# Single \n (not preceded or followed by another \n) → two spaces + \n
parts[i] = re.sub(r"(?<!\n)\n(?!\n)", " \n", part)
return "".join(parts)
def _render_markdown(raw_md: str, vault_name: str, current_file_path: Path | None = None) -> str:
"""Render a markdown string to HTML with wikilink and image support.
Uses the cached singleton mistune renderer for performance.
Args:
raw_md: Raw markdown text (frontmatter already stripped).
vault_name: Current vault for wikilink resolution context.
current_file_path: Absolute path to the current markdown file.
Returns:
HTML string.
"""
# Get vault data for image resolution
vault_data = get_vault_data(vault_name)
vault_root = Path(vault_data["path"]) if vault_data else None
attachments_path = vault_data.get("config", {}).get("attachmentsPath") if vault_data else None
# Redact secrets before rendering (P0 security)
raw_md = redact_file_content(raw_md, str(current_file_path) if current_file_path else "")
# Preprocess images first
if vault_root:
raw_md = preprocess_images(raw_md, vault_name, vault_root, current_file_path, attachments_path)
# Convert wikilinks
converted = _convert_wikilinks(raw_md, vault_name)
# Normalize line breaks to match Obsidian behavior (single \n → hard break)
converted = _normalize_line_breaks(converted)
rendered = _markdown_renderer(converted)
# Add heading IDs for TOC navigation
rendered = _add_heading_ids(rendered)
# Sanitize: raw HTML in vault content must never reach the DOM (BUG-021).
rendered = sanitize_html(rendered)
return rendered
+531
View File
@@ -0,0 +1,531 @@
"""Configuration, AI keys, diagnostics & dashboard endpoints (ROADMAP #85, tranche 7).
Handlers déplacés depuis :mod:`backend.main` sans changement de
comportement : mêmes chemins (``/api/config*``, ``/api/diagnostics``,
``/api/dashboard``), mêmes modèles de réponse, mêmes dépendances
d'authentification.
Adaptations strictement équivalentes :
- ``_load_config`` / ``_save_config`` / ``_DEFAULT_CONFIG`` /
``_CONFIG_PATH`` / ``_BASE_DIR`` ont déménagé ici : ``main`` les
réimporte pour son lifespan (pas de cycle : ce module ne dépend pas de
``main``).
- ``AI_KEYS_FILE`` / ``_write_ai_keys`` / ``_FALLBACK_MODELS`` ont déménagé
ici (``AI_KEYS_FILE`` garde son chemin relatif ``data/api_keys.json``,
résolu depuis le même CWD au runtime).
"""
import json as _json
import logging
import os
import urllib.request
from pathlib import Path
from fastapi import APIRouter, Body, Depends, HTTPException, Query
from backend.ai import PROVIDERS, _read_ai_keys, get_ai_key
from backend.auth.middleware import require_admin, require_auth
from backend.indexer import index
from backend.media_types import IMAGE_EXTENSIONS
from backend.schemas import (
AIKeyDeleteResponse,
AIKeysResponse,
AIModelsResponse,
AITestResponse,
AppConfigResponse,
DashboardResponse,
DiagnosticsResponse,
StatusResponse,
)
from backend.search_executor import get_search_executor
from backend.tools.secrets import (
TOOL_KEY_NAMES as _TOOL_KEY_NAMES,
)
from backend.tools.secrets import (
delete_tool_key as _delete_tool_key,
)
from backend.tools.secrets import (
get_tool_key as _get_tool_key,
)
from backend.tools.secrets import (
mask_value as _mask_tool_value,
)
from backend.tools.secrets import (
set_tool_key as _set_tool_key,
)
logger = logging.getLogger("obsigate")
router = APIRouter(tags=["System"])
_BASE_DIR = Path(__file__).resolve().parent.parent.parent
_CONFIG_PATH = _BASE_DIR / "data" / "config.json"
_DEFAULT_CONFIG = {
"search_workers": 2,
"debounce_ms": 300,
"results_per_page": 50,
"min_query_length": 2,
"search_timeout_ms": 30000,
"max_content_size": 100000,
"snippet_context_chars": 120,
"max_snippet_highlights": 5,
"title_boost": 3.0,
"path_boost": 1.5,
"watcher_enabled": True,
"watcher_use_polling": False,
"watcher_polling_interval": 5.0,
"watcher_debounce": 2.0,
"tag_boost": 2.0,
"prefix_max_expansions": 50,
"recent_files_limit": 20,
"max_backups_per_file": 10,
"ai_default_provider": "deepseek",
"ai_default_models": {},
}
def _load_config() -> dict:
"""Load config from disk, merging with defaults."""
config = dict(_DEFAULT_CONFIG)
if _CONFIG_PATH.exists():
try:
stored = _json.loads(_CONFIG_PATH.read_text(encoding="utf-8"))
config.update(stored)
except Exception as e:
logger.warning(f"Failed to read config.json: {e}")
return config
def _save_config(config: dict) -> None:
"""Persist config to disk."""
try:
_CONFIG_PATH.write_text(
_json.dumps(config, indent=2, ensure_ascii=False),
encoding="utf-8",
)
except Exception as e:
logger.error(f"Failed to write config.json: {e}")
raise HTTPException(status_code=500, detail=f"Failed to save config: {e}")
AI_KEYS_FILE = Path("data/api_keys.json")
def _write_ai_keys(data: dict):
AI_KEYS_FILE.parent.mkdir(parents=True, exist_ok=True)
tmp = AI_KEYS_FILE.with_suffix(".tmp")
tmp.write_text(_json.dumps(data, indent=2), encoding="utf-8")
tmp.replace(AI_KEYS_FILE)
@router.get("/api/config", response_model=AppConfigResponse)
async def api_get_config(current_user=Depends(require_auth)):
"""Return current configuration with defaults for missing keys."""
return _load_config()
@router.post("/api/config", response_model=AppConfigResponse)
async def api_set_config(body: dict = Body(...), current_user=Depends(require_admin)):
"""Update configuration. Only known keys are accepted.
Keys matching ``_DEFAULT_CONFIG`` are validated and persisted.
Unknown keys are silently ignored.
Returns the full merged config after update.
"""
current = _load_config()
updated_keys = []
for key, value in body.items():
if key in _DEFAULT_CONFIG:
expected_type = type(_DEFAULT_CONFIG[key])
if isinstance(value, expected_type) or (expected_type is float and isinstance(value, (int, float))):
current[key] = value
updated_keys.append(key)
else:
raise HTTPException(
status_code=400,
detail=f"Invalid type for '{key}': expected {expected_type.__name__}, got {type(value).__name__}",
)
_save_config(current)
if any(k.startswith("ai_") for k in updated_keys):
try:
from backend.ai import reload_ai_config
reload_ai_config()
except Exception as e:
logger.warning(f"Failed to reload AI config: {e}")
logger.info(f"Config updated: {updated_keys}")
return current
@router.get("/api/config/ai-keys", response_model=AIKeysResponse)
async def api_get_ai_keys(current_user=Depends(require_admin)):
"""Return stored AI keys (values masked)."""
keys = _read_ai_keys()
masked = {}
for k in ["DEEPSEEK_API_KEY", "OPENROUTER_API_KEY", "GEMINI_API_KEY", "NVIDIA_API_KEY", "QWENCLOUD_API_KEY", "XIAOMI_API_KEY", "MISTRAL_API_KEY"]:
val = keys.get(k, "") or os.environ.get(k, "")
if val:
masked[k] = val[:4] + "..." + val[-4:] if len(val) > 8 else "***"
else:
masked[k] = ""
return masked
@router.post("/api/config/ai-keys", response_model=StatusResponse)
async def api_set_ai_keys(body: dict = Body(...), current_user=Depends(require_admin)):
"""Save AI keys. Pass {"DEEPSEEK_API_KEY":"sk-...","OPENROUTER_API_KEY":"...","GEMINI_API_KEY":"..."}"""
keys = _read_ai_keys()
for k in ["DEEPSEEK_API_KEY", "OPENROUTER_API_KEY", "GEMINI_API_KEY", "NVIDIA_API_KEY", "QWENCLOUD_API_KEY", "XIAOMI_API_KEY", "MISTRAL_API_KEY"]:
if body.get(k):
keys[k] = body[k]
_write_ai_keys(keys)
logger.info("AI keys updated")
return {"status": "ok"}
@router.delete("/api/config/ai-keys/{provider_env}", response_model=AIKeyDeleteResponse)
async def api_delete_ai_key(provider_env: str, current_user=Depends(require_admin)):
"""Delete a specific AI provider key from storage."""
allowed = {"DEEPSEEK_API_KEY", "OPENROUTER_API_KEY", "GEMINI_API_KEY",
"NVIDIA_API_KEY", "QWENCLOUD_API_KEY", "XIAOMI_API_KEY", "MISTRAL_API_KEY"}
key_name = provider_env.upper()
if key_name not in allowed:
raise HTTPException(status_code=400, detail=f"Clé inconnue: {provider_env}")
keys = _read_ai_keys()
if key_name in keys:
del keys[key_name]
_write_ai_keys(keys)
# Also clear from env at runtime so get_ai_key() no longer finds it
os.environ.pop(key_name, None)
logger.info(f"AI key deleted: {key_name}")
return {"status": "deleted", "key": key_name}
@router.get("/api/config/tool-keys", response_model=AIKeysResponse)
async def api_get_tool_keys(current_user=Depends(require_admin)):
"""Return tool/connected-source configuration (tokens masked, URLs clear)."""
masked = {}
for name in _TOOL_KEY_NAMES:
masked[name] = _mask_tool_value(name, _get_tool_key(name))
return masked
@router.post("/api/config/tool-keys", response_model=StatusResponse)
async def api_set_tool_keys(body: dict = Body(...), current_user=Depends(require_admin)):
"""Save tool/connected-source keys.
Only whitelisted names (``backend.tools.secrets.TOOL_KEY_NAMES``) are
accepted: Tavily/Brave/SerpAPI/Exa API keys, Gitea URL + token, GitHub
token. Empty values delete the stored entry.
"""
updated = []
for name, value in body.items():
if name not in _TOOL_KEY_NAMES:
raise HTTPException(status_code=400, detail=f"Clé inconnue: {name}")
if value is not None and not isinstance(value, str):
raise HTTPException(status_code=400, detail=f"Type invalide pour {name}")
_set_tool_key(name, value or "")
updated.append(name)
logger.info(f"Tool keys updated: {updated}")
return {"status": "ok"}
@router.delete("/api/config/tool-keys/{name}", response_model=AIKeyDeleteResponse)
async def api_delete_tool_key(name: str, current_user=Depends(require_admin)):
"""Delete a stored tool key (the environment fallback still applies)."""
key_name = name.upper()
try:
existed = _delete_tool_key(key_name)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
logger.info(f"Tool key deleted: {key_name} (existed={existed})")
return {"status": "deleted", "key": key_name}
@router.post("/api/config/ai-keys/test", response_model=AITestResponse)
async def api_test_ai_keys(current_user=Depends(require_admin)):
"""Test which AI providers are configured.
Each provider has a dedicated (URL, header-name) test pair.
- Most OpenAI-compatible APIs use `Authorization: Bearer KEY`
- Xiaomi MiMo uses `api-key: KEY`
- Gemini uses a query-string key
"""
results = {}
for key_name, label, test_url_tmpl, header_name in [
# OpenAI-compatible — Authorization: Bearer
("DEEPSEEK_API_KEY", "deepseek", "https://api.deepseek.com/v1/models", "Authorization"),
("OPENROUTER_API_KEY","openrouter", "https://openrouter.ai/api/v1/models", "Authorization"),
("NVIDIA_API_KEY", "nvidia", "https://integrate.api.nvidia.com/v1/models", "Authorization"),
("QWENCLOUD_API_KEY", "qwencloud", "https://dashscope.aliyuncs.com/compatible-mode/v1/models", "Authorization"),
("MISTRAL_API_KEY", "mistral", "https://api.mistral.ai/v1/models", "Authorization"),
# Xiaomi MiMo — dedicated api-key header (NOT Authorization: Bearer)
("XIAOMI_API_KEY", "xiaomi", "https://api.xiaomimimo.com/v1/models", "api-key"),
# Gemini — key in query string
("GEMINI_API_KEY", "gemini", "https://generativelanguage.googleapis.com/v1beta/models?key={key}", None),
]:
key = get_ai_key(key_name)
if not key:
results[label] = "non configuré"
continue
try:
url = test_url_tmpl.replace("{key}", key) if "{key}" in test_url_tmpl else test_url_tmpl
if header_name:
req = urllib.request.Request(url, headers={header_name: key})
else:
req = urllib.request.Request(url)
urllib.request.urlopen(req, timeout=5)
results[label] = "ok"
except Exception as e:
# Truncate the error to keep the response small.
results[label] = "erreur: " + str(e)[:80]
return results
@router.get("/api/config/ai-models", response_model=AIModelsResponse)
async def api_list_ai_models(provider: str = Query(...), current_user=Depends(require_admin)):
"""List available models for a given AI provider.
Strategy:
1. Try the provider's public models endpoint (OpenAI-compatible /v1/models or Gemini).
2. If the network call fails (timeout, 4xx, 5xx, DNS, etc.), fall back to a
curated static list of known-good models for that provider.
3. Always return a non-empty list when the provider is known, so the UI
dropdown is never empty.
"""
provider = provider.lower()
from backend.model_capabilities import get_capabilities_for_models
from backend.provider_capabilities import remember_declared_capabilities
all_providers = ("deepseek", "openrouter", "gemini", "nvidia", "qwencloud", "xiaomi", "mistral")
if provider not in all_providers:
return {"models": [], "error": f"Unknown provider: {provider}", "source": "validation"}
key_name = f"{provider.upper()}_API_KEY"
key = get_ai_key(key_name)
if not key:
# No key configured — return curated fallback list so the UI can
# still show what WOULD be available once a key is set.
fallback = _FALLBACK_MODELS.get(provider, [])
return {"models": fallback, "source": "fallback",
"capabilities": get_capabilities_for_models(provider, fallback),
"note": "API key not configured — showing default model list"}
# Build URL
if provider == "gemini":
url = f"https://generativelanguage.googleapis.com/v1beta/models?key={key}"
elif provider == "deepseek":
url = "https://api.deepseek.com/v1/models"
elif provider == "openrouter":
url = "https://openrouter.ai/api/v1/models"
elif provider == "nvidia":
url = "https://integrate.api.nvidia.com/v1/models"
elif provider == "qwencloud":
url = "https://dashscope.aliyuncs.com/compatible-mode/v1/models"
elif provider == "xiaomi":
# Xiaomi MiMo — dedicated api-key header (NOT Authorization: Bearer).
# Endpoint: https://api.xiaomimimo.com/v1/models
url = "https://api.xiaomimimo.com/v1/models"
models = [] # parsed below with the custom header
elif provider == "mistral":
url = "https://api.mistral.ai/v1/models"
try:
if provider == "gemini":
req = urllib.request.Request(url)
elif provider == "xiaomi":
# Xiaomi MiMo uses a dedicated api-key header.
req = urllib.request.Request(url, headers={"api-key": key})
else:
req = urllib.request.Request(url, headers={"Authorization": "Bearer " + key})
with urllib.request.urlopen(req, timeout=10) as resp:
data = _json.loads(resp.read().decode())
if provider == "gemini":
models = [m.get("name", "") for m in data.get("models", []) if m.get("name")]
# Gemini returns names like "models/gemini-1.5-flash" — strip prefix
models = [m.replace("models/", "") for m in models]
else:
models = [m.get("id", "") for m in data.get("data", []) if m.get("id")]
# Cache the capabilities the provider declares for these models
# (BUG-044) — get_capabilities_for_models() below then returns the
# provider's own truth for the flags it declares, the curated table
# for the rest. Providers that declare nothing are left untouched.
remember_declared_capabilities(provider, data)
if models:
# Prepend the configured default if not already present
default = PROVIDERS.get(provider, {}).get("model")
if default and default not in models:
models = [default] + models
return {"models": models, "source": "live", "count": len(models),
"capabilities": get_capabilities_for_models(provider, models)}
# Empty list from API — fall through to fallback
raise ValueError("empty model list from provider API")
except Exception as e:
# Network error, auth error, parsing error — use curated fallback
fallback = _FALLBACK_MODELS.get(provider, [])
return {"models": fallback, "source": "fallback", "error": str(e)[:200],
"capabilities": get_capabilities_for_models(provider, fallback),
"note": "Could not reach provider API — showing default model list"}
# ── Curated fallback model lists ──────────────────────────────────────────
# Used when the provider API is unreachable or returns empty.
# Keep these short and focused on models known to work with the
# OpenAI-compatible chat completions interface (or Gemini's generateContent).
_FALLBACK_MODELS: dict[str, list[str]] = {
"deepseek": [
"deepseek-chat",
"deepseek-reasoner",
],
"openrouter": [
"openai/gpt-4o-mini",
"openai/gpt-4o",
"anthropic/claude-3.5-sonnet",
"anthropic/claude-3-haiku",
"google/gemini-2.0-flash-exp:free",
"meta-llama/llama-3.1-70b-instruct",
"meta-llama/llama-3.1-8b-instruct:free",
"mistralai/mistral-large-latest",
],
"gemini": [
"gemini-2.0-flash",
"gemini-2.0-flash-exp",
"gemini-1.5-pro",
"gemini-1.5-flash",
"gemini-1.5-flash-8b",
],
"nvidia": [
"meta/llama-3.1-405b-instruct",
"meta/llama-3.1-70b-instruct",
"meta/llama-3.1-8b-instruct",
"mistralai/mistral-large",
"google/gemma-2-27b-it",
"nvidia/llama-3.1-nemotron-70b-instruct",
],
"qwencloud": [
"qwen-max",
"qwen-plus",
"qwen-turbo",
"qwen-long",
"qwen-vl-max",
"qwen-vl-plus",
],
"xiaomi": [
# Xiaomi MiMo models — the public /v1/models endpoint requires the
# `api-key` custom header (NOT Authorization: Bearer), so the live
# call often fails with 401 even with the right key. We ship a
# known-good list as fallback. See https://mimo.mi.com/docs/
"mimo-v2.5-pro",
"mimo-v2.5",
"mimo-v2.5-asr",
"mimo-v2.5-tts",
"mimo-v2.5-tts-voiceclone",
"mimo-v2.5-tts-voicedesign",
],
"mistral": [
"mistral-large-latest",
"mistral-medium-latest",
"mistral-small-latest",
"open-mistral-7b",
"open-mixtral-8x7b",
"codestral-latest",
],
}
@router.get("/api/diagnostics", response_model=DiagnosticsResponse)
async def api_diagnostics(current_user=Depends(require_admin)):
"""Return index statistics and system diagnostics.
Includes document counts, token counts, memory estimates,
and inverted index status.
"""
import sys
from backend.search import get_inverted_index
inv = get_inverted_index()
# Per-vault stats
vault_stats = {}
total_files = 0
total_tags = 0
# Snapshot both dicts first: the indexer mutates them from background
# threads, and iterating a live dict raises "dictionary changed size".
for vname, vdata in list(index.items()):
file_count = len(vdata.get("files", []))
tag_count = len(vdata.get("tags", {}))
vault_stats[vname] = {"file_count": file_count, "tag_count": tag_count}
total_files += file_count
total_tags += tag_count
# Memory estimate for inverted index
word_index = inv.word_index.copy()
word_index_entries = sum(len(docs) for docs in word_index.values())
mem_estimate_mb = round(
(sys.getsizeof(inv.word_index) + word_index_entries * 80
+ len(inv.doc_info) * 200
+ len(inv._sorted_tokens) * 60) / (1024 * 1024), 2
)
return {
"index": {
"total_files": total_files,
"total_tags": total_tags,
"vaults": vault_stats,
},
"inverted_index": {
"unique_tokens": len(word_index),
"total_postings": word_index_entries,
"documents": inv.doc_count,
"sorted_tokens": len(inv._sorted_tokens),
"is_stale": inv.is_stale(),
"memory_estimate_mb": mem_estimate_mb,
},
"config": _load_config(),
"search_executor": {
"active": get_search_executor() is not None,
"max_workers": get_search_executor()._max_workers if get_search_executor() else 0,
},
}
@router.get("/api/dashboard", response_model=DashboardResponse)
async def api_dashboard(current_user=Depends(require_auth)):
"""Aggregated dashboard statistics across all accessible vaults."""
user_vaults = current_user.get("_token_vaults") or current_user.get("vaults", [])
vault_stats = []
total_files = 0
total_tags = set()
total_size = 0
total_images = 0
for vname, vdata in index.items():
if "*" not in user_vaults and vname not in user_vaults:
continue
files = vdata.get("files", [])
fc = len(files)
total_files += fc
vtags = set()
vsize = 0
vimages = 0
for f in files:
vtags.update(f.get("tags", []))
vsize += f.get("size", 0)
if (f.get("extension") or "").lower() in IMAGE_EXTENSIONS:
vimages += 1
total_tags.update(vtags)
total_size += vsize
total_images += vimages
vault_stats.append({
"name": vname, "file_count": fc, "tag_count": len(vtags),
"total_size_bytes": vsize, "image_count": vimages,
})
return {
"vaults": vault_stats,
"total_files": total_files,
"total_tags": len(total_tags),
"total_size_bytes": total_size,
"total_images": total_images,
}
+73
View File
@@ -0,0 +1,73 @@
"""Syncthing conflict endpoints (ROADMAP #85, tranche 8).
Handlers déplacés depuis :mod:`backend.main` sans changement de
comportement : mêmes chemins (``/api/conflicts*``), mêmes modèles de
réponse, mêmes dépendances d'authentification.
Adaptations strictement équivalentes :
- ``_resolve_safe_path`` / ``_backup_file`` → :mod:`backend.services.paths`
et :mod:`backend.services.backups` (pass-through).
"""
import logging
import shutil
from pathlib import Path
from fastapi import APIRouter, Body, Depends, HTTPException
from backend.audit import log_file_delete
from backend.auth.middleware import check_vault_access, require_auth
from backend.indexer import get_conflicts, get_vault_data, remove_single_file
from backend.schemas import ConflictResolveResponse, ConflictsResponse
from backend.services.backups import create_backup
from backend.services.paths import resolve_safe_path
from backend.sse import sse_manager
logger = logging.getLogger("obsigate")
router = APIRouter(tags=["conflicts"])
@router.get("/api/conflicts", response_model=ConflictsResponse)
async def api_conflicts(current_user=Depends(require_auth)):
"""List sync-conflict files across accessible vaults."""
user_vaults = current_user.get("_token_vaults") or current_user.get("vaults", [])
all_conflicts = get_conflicts()
if "*" not in user_vaults:
all_conflicts = [c for c in all_conflicts if c["vault"] in user_vaults]
return {"conflicts": all_conflicts, "total": len(all_conflicts)}
@router.post("/api/conflicts/resolve", response_model=ConflictResolveResponse)
async def api_conflict_resolve(body: dict = Body(...), current_user=Depends(require_auth)):
"""Resolve a conflict: keep_local (delete conflict file) or keep_conflict (replace original)."""
vault_name = body.get("vault")
conflict_path = body.get("conflict_path")
original_path = body.get("original_path")
action = body.get("action") # "keep_local" or "keep_conflict"
# mypy: narrow down from dict values
assert isinstance(vault_name, str), "'vault' is required and must be a string"
assert isinstance(conflict_path, str), "'conflict_path' is required and must be a string"
assert isinstance(original_path, str), "'original_path' is required and must be a string"
if not check_vault_access(vault_name, current_user):
raise HTTPException(403, f"Accès refusé à la vault '{vault_name}'")
vault_data = get_vault_data(vault_name)
if not vault_data:
raise HTTPException(404, "Vault not found")
vault_root = Path(vault_data["path"])
conf_file = resolve_safe_path(vault_root, conflict_path)
orig_file = resolve_safe_path(vault_root, original_path)
if not conf_file.exists():
raise HTTPException(404, "Conflict file not found")
try:
if action == "keep_conflict":
create_backup(orig_file, vault_name, original_path)
shutil.copy2(conf_file, orig_file)
logger.info(f"Conflict resolved (keep_conflict): {conflict_path} → {original_path}")
conf_file.unlink()
await remove_single_file(vault_name, conflict_path)
log_file_delete(current_user["username"], vault_name, conflict_path)
await sse_manager.broadcast("file_deleted", {"vault": vault_name, "path": conflict_path})
return {"status": "resolved", "action": action}
except Exception as e:
raise HTTPException(500, f"Error resolving conflict: {e!s}")
+3 -3
View File
@@ -9,7 +9,8 @@ d'authentification.
Adaptations strictement équivalentes :
- ``_resolve_safe_path`` → :mod:`backend.services.paths` (pass-through).
- ``_render_markdown`` reste dans ``main`` (import différé).
- ``_render_markdown`` vient de :mod:`backend.render` (#85 T9, sans cycle
d'import).
- ``_resolve_export_target`` / ``_safe_export_name`` (export uniquement)
sont définis ici ; ``stream_file_with_range`` vit dans
:mod:`backend.routers.helpers` (partagé).
@@ -29,6 +30,7 @@ from backend.history import record_open
from backend.indexer import get_vault_data, index, parse_markdown_file
from backend.media_thumbs import generate_thumbnail, is_decodable
from backend.media_types import is_audio, is_image, is_video, media_mime_type
from backend.render import _render_markdown
from backend.routers.helpers import media_max_inline_bytes, stream_file_with_range
from backend.schemas import (
AllVaultSettingsResponse,
@@ -85,8 +87,6 @@ def _safe_export_name(name: str) -> str:
)
async def api_file_pdf(vault_name: str, path: str = Query(..., description="Relative path to file"), current_user=Depends(require_auth)):
"""Download a markdown file as PDF."""
from backend.main import _render_markdown # différé : évite l'import circulaire (#85)
if generate_pdf is None:
raise HTTPException(501, "PDF export unavailable (WeasyPrint/GTK not available)")
if not check_vault_access(vault_name, current_user):
+3 -4
View File
@@ -7,8 +7,8 @@ lecture), mêmes modèles de réponse (déménagés dans
Adaptations strictement équivalentes :
- ``_resolve_safe_path`` → :mod:`backend.services.paths` (pass-through).
- ``_render_markdown`` reste dans ``main`` (import différé, extraction
prévue dans une tranche ultérieure).
- ``_render_markdown`` vient de :mod:`backend.render` (#85 T9, sans cycle
d'import).
- ``_content_disposition`` / ``_media_max_inline_bytes`` / ``EXT_TO_LANG``
ont déménagé : helpers partagés dans :mod:`backend.routers.helpers`
(``EXT_TO_LANG`` n'était utilisé que par la vue fichier).
@@ -31,6 +31,7 @@ from backend.indexer import (
parse_markdown_file,
)
from backend.media_types import is_audio, is_image, is_video, media_mime_type
from backend.render import _render_markdown
from backend.routers.helpers import media_max_inline_bytes
from backend.schemas import (
BacklinksResponse,
@@ -192,8 +193,6 @@ async def api_file(vault_name: str, path: str = Query(..., description="Relative
Returns:
``FileContentResponse`` with HTML, metadata, and tags.
"""
from backend.main import _render_markdown # différé : évite l'import circulaire (#85)
if not check_vault_access(vault_name, current_user):
raise HTTPException(status_code=403, detail=f"Accès refusé à la vault '{vault_name}'")
vault_data = get_vault_data(vault_name)
+160
View File
@@ -0,0 +1,160 @@
"""History endpoints — recent, bookmarks, saved searches (ROADMAP #85, tranche 8).
Handlers déplacés depuis :mod:`backend.main` sans changement de
comportement : mêmes chemins, mêmes modèles (``BookmarkToggleRequest``
déménagé dans :mod:`backend.schemas`), mêmes dépendances
d'authentification.
Adaptations strictement équivalentes :
- ``_resolve_safe_path`` / ``_backup_file`` → :mod:`backend.services.paths`
et :mod:`backend.services.backups` (pass-through).
- ``_load_config`` vient de :mod:`backend.routers.config`.
"""
import logging
from pathlib import Path
import frontmatter
from fastapi import APIRouter, Body, Depends, HTTPException, Query
from backend.auth.middleware import check_vault_access, require_auth
from backend.history import get_bookmarks, toggle_bookmark
from backend.indexer import find_file_in_index, get_vault_data, update_single_file
from backend.routers.config import _load_config
from backend.saved_searches import delete_saved, get_saved, save_search
from backend.schemas import (
BookmarksResponse,
BookmarkToggleRequest,
BookmarkToggleResponse,
RecentResponse,
SavedSearch,
StatusResponse,
)
from backend.services.backups import create_backup
from backend.services.paths import resolve_safe_path
from backend.services.recent import humanize_mtime, list_recent
logger = logging.getLogger("obsigate")
router = APIRouter(tags=["Bookmarks"])
@router.get("/api/recent", response_model=RecentResponse)
async def api_recent(limit: int | None = Query(None), vault: str | None = Query(None), mode: str | None = Query("opened"), current_user=Depends(require_auth)):
config = _load_config()
actual_limit = limit if limit is not None else config.get("recent_files_limit", 20)
username = current_user.get("username")
user_vaults = current_user.get("_token_vaults") or current_user.get("vaults", [])
return list_recent(
username,
user_vaults,
vault=vault,
limit=actual_limit,
mode=mode or "opened",
)
@router.get("/api/bookmarks", response_model=BookmarksResponse)
async def api_bookmarks(vault: str | None = Query(None), current_user=Depends(require_auth)):
username = current_user.get("username")
user_vaults = current_user.get("_token_vaults") or current_user.get("vaults", [])
if not username:
return {"files": []}
history = get_bookmarks(username, vault_filter=vault)
files_resp = []
for item in history:
v_name = item["vault"]
if "*" not in user_vaults and v_name not in user_vaults:
continue
# Find in index to get metadata
f_idx = find_file_in_index(item["path"], v_name)
if f_idx:
files_resp.append({
"path": f_idx["path"],
"title": f_idx.get("title") or item["path"].split("/")[-1],
"vault": v_name,
"mtime": item["bookmarked_at"],
"mtime_human": humanize_mtime(item["bookmarked_at"]),
"size_bytes": f_idx.get("size", 0),
"tags": [f"#{t}" for t in f_idx.get("tags", [])][:5],
"bookmarked": True
})
else:
files_resp.append({
"path": item["path"],
"title": item.get("title") or item["path"].split("/")[-1],
"vault": v_name,
"mtime": item["bookmarked_at"],
"mtime_human": humanize_mtime(item["bookmarked_at"]),
"tags": [],
"bookmarked": True
})
return {
"files": files_resp,
"total": len(files_resp)
}
@router.post("/api/bookmarks/toggle", response_model=BookmarkToggleResponse)
async def api_toggle_bookmark(req: BookmarkToggleRequest, current_user=Depends(require_auth)):
username = current_user.get("username")
if not username:
raise HTTPException(status_code=401, detail="Not authenticated")
# Check vault access
if not check_vault_access(req.vault, current_user):
raise HTTPException(status_code=403, detail="Access denied to vault")
is_now_bookmarked = toggle_bookmark(username, req.vault, req.path, req.title or "")
# Update the file's YAML frontmatter: favoris: true/false
vault_data = get_vault_data(req.vault)
if vault_data:
file_path = resolve_safe_path(Path(vault_data["path"]), req.path)
if file_path.exists() and file_path.suffix == ".md":
try:
raw = file_path.read_text(encoding="utf-8", errors="replace")
post = frontmatter.loads(raw)
if is_now_bookmarked:
post.metadata["favoris"] = True
elif "favoris" in post.metadata:
del post.metadata["favoris"]
new_raw = frontmatter.dumps(post)
create_backup(file_path, req.vault, req.path)
file_path.write_text(new_raw, encoding="utf-8")
await update_single_file(req.vault, str(file_path))
except Exception as e:
logger.warning(f"Failed to update favoris metadata on {req.vault}/{req.path}: {e}")
return {"bookmarked": is_now_bookmarked}
@router.get("/api/saved-searches", response_model=list[SavedSearch])
async def api_saved_searches(current_user=Depends(require_auth)):
username = current_user.get("username")
if not username:
raise HTTPException(401)
return get_saved(username)
@router.post("/api/saved-searches", response_model=SavedSearch)
async def api_save_search(body: dict = Body(...), current_user=Depends(require_auth)):
username = current_user.get("username")
if not username:
raise HTTPException(401)
return save_search(username, body)
@router.delete("/api/saved-searches/{search_id}", response_model=StatusResponse)
async def api_delete_saved_search(search_id: str, current_user=Depends(require_auth)):
username = current_user.get("username")
if not username:
raise HTTPException(401)
if not delete_saved(username, search_id):
raise HTTPException(404, "Not found")
return {"status": "deleted"}
+105
View File
@@ -0,0 +1,105 @@
"""Real-time endpoints — SSE stream & collaboration WebSocket (ROADMAP #85, tranche 9).
Handlers déplacés depuis :mod:`backend.main` sans changement de
comportement : mêmes chemins (``/api/events``,
``/ws/collab/{vault}/{path}``), même authentification (Depend pour le SSE,
manuelle pour le WebSocket — les ``Depends`` FastAPI ne s'exécutent pas sur
les routes WebSocket).
Pas de tags déclarés : assignation par chemin via
``openapi_docs.tag_for_path`` comme avant (``/api/events`` → System).
"""
import asyncio
import json as _json
from fastapi import APIRouter, Depends, WebSocket
from fastapi.responses import StreamingResponse
from backend.auth.middleware import check_vault_access, require_auth
from backend.collab import authenticate_websocket, collab_manager
from backend.services.paths import resolve_safe_path
from backend.services.vaults import get_vault_root
from backend.sse import sse_manager
router = APIRouter()
@router.get(
"/api/events",
response_class=StreamingResponse,
responses={200: {"content": {"text/event-stream": {}}, "description": "Server-Sent Events stream"}},
)
async def api_events(current_user=Depends(require_auth)):
"""SSE stream for real-time index update notifications.
Sends keepalive comments every 30s. Events:
- ``index_updated``: partial index change (file create/modify/delete/move)
- ``index_reloaded``: full re-index completed
- ``vault_added``: new vault added dynamically
- ``vault_removed``: vault removed dynamically
"""
queue = await sse_manager.connect()
async def event_generator():
try:
# Send initial connection event
yield f"event: connected\ndata: {_json.dumps({'sse_clients': sse_manager.client_count})}\n\n"
while True:
try:
msg = await asyncio.wait_for(queue.get(), timeout=30.0)
yield f"event: {msg['event']}\ndata: {msg['data']}\n\n"
except asyncio.TimeoutError:
# Keepalive comment
yield ": keepalive\n\n"
except asyncio.CancelledError:
break
finally:
sse_manager.disconnect(queue)
return StreamingResponse(
event_generator(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
},
)
@router.websocket("/ws/collab/{vault_name}/{path:path}")
async def collab_websocket(websocket: WebSocket, vault_name: str, path: str):
"""Real-time collaborative editing over WebSocket (ROADMAP #62).
One *room* is created per ``vault::path``; all clients editing the same
file share Yjs/CRDT updates, awareness (cursors/selection) and a debounced
server-side persistence of the markdown content.
Authentication is performed manually (FastAPI ``Depends`` do not run for
WebSocket routes) and vault access is enforced per connection.
"""
from backend.services.errors import ServiceError
user = authenticate_websocket(websocket)
if user is None:
await websocket.close(code=4401)
return
if not check_vault_access(vault_name, user):
await websocket.close(code=4403)
return
try:
vault_root = get_vault_root(vault_name)
file_path = resolve_safe_path(vault_root, path)
except ServiceError:
await websocket.close(code=4404)
return
if not file_path.exists() or not file_path.is_file():
await websocket.close(code=4404)
return
await websocket.accept()
await collab_manager.connect(websocket, vault_name, path, file_path, user)
+3 -7
View File
@@ -11,9 +11,8 @@ Adaptations strictement équivalentes (pas de changement de comportement) :
wrappers directs : appelés ici via :mod:`backend.services.paths` et
:mod:`backend.services.backups` (mêmes signatures, mêmes exceptions
``ServiceError`` toujours mappées par le handler global de ``main``).
- ``_render_markdown`` reste défini dans ``main`` (extraction prévue dans
une tranche ultérieure) : import différé à l'intérieur des handlers, donc
sans import circulaire au chargement.
- ``_render_markdown`` vient de :mod:`backend.render` (#85 T9, sans cycle
d'import).
"""
import html as html_mod
@@ -27,6 +26,7 @@ from fastapi.responses import FileResponse, HTMLResponse, Response
from backend.auth.middleware import check_vault_access, require_auth
from backend.indexer import get_vault_data, parse_markdown_file, update_single_file
from backend.render import _render_markdown
from backend.schemas import ShareModel, StatusResponse
from backend.secret_redactor import redact_file_content
from backend.services.backups import create_backup
@@ -115,8 +115,6 @@ async def api_share_revoke(share_id: str, current_user=Depends(require_auth)):
)
async def public_share_pdf_download(token: str):
"""Download shared document as real PDF via WeasyPrint."""
from backend.main import _render_markdown # différé : évite l'import circulaire (#85)
if generate_pdf is None:
raise HTTPException(501, "PDF export unavailable (WeasyPrint/GTK not available)")
share = get_share_by_token(token)
@@ -168,8 +166,6 @@ async def public_share_raw(token: str):
@router.get("/s/{token}", response_class=HTMLResponse)
async def public_share_view(token: str):
"""Public share view — no authentication required."""
from backend.main import _render_markdown # différé : évite l'import circulaire (#85)
share = get_share_by_token(token)
if not share:
raise HTTPException(404, "Share not found or expired")
+107
View File
@@ -0,0 +1,107 @@
"""Vault management endpoints (ROADMAP #85, tranche 8).
Handlers déplacés depuis :mod:`backend.main` sans changement de
comportement : mêmes chemins (``/api/vaults*``), mêmes modèles de réponse
(``VaultInfo`` déménagé dans :mod:`backend.schemas`), mêmes dépendances
d'authentification.
Le handle du file-watcher vit désormais dans :mod:`backend.watcher_state`
(partagé avec le lifespan de ``main``) au lieu du global de ``main``.
"""
from pathlib import Path
from fastapi import APIRouter, Body, Depends, HTTPException
from backend.auth.middleware import require_admin, require_auth
from backend.indexer import add_vault_to_index, index, remove_vault_from_index
from backend.schemas import VaultActionResponse, VaultInfo, VaultsStatusResponse, VaultStatsResponse
from backend.services.vaults import list_accessible_vaults
from backend.sse import sse_manager
from backend.watcher_state import get_watcher
router = APIRouter(tags=["vaults"])
@router.get("/api/vaults", response_model=list[VaultInfo])
async def api_vaults(current_user=Depends(require_auth)):
"""List configured vaults the user has access to.
Returns:
List of vault summary objects filtered by user permissions.
"""
return list_accessible_vaults(current_user)
@router.post("/api/vaults/add", response_model=VaultStatsResponse)
async def api_add_vault(body: dict = Body(...), current_user=Depends(require_admin)):
"""Add a new vault dynamically without restarting.
Body:
name: Display name for the vault.
path: Absolute filesystem path to the vault directory.
"""
name = body.get("name", "").strip()
vault_path = body.get("path", "").strip()
if not name or not vault_path:
raise HTTPException(status_code=400, detail="Both 'name' and 'path' are required")
if name in index:
raise HTTPException(status_code=409, detail=f"Vault '{name}' already exists")
if not Path(vault_path).exists():
raise HTTPException(status_code=400, detail=f"Path does not exist: {vault_path}")
stats = await add_vault_to_index(name, vault_path)
# Start watching the new vault
watcher = get_watcher()
if watcher:
await watcher.add_vault(name, vault_path)
await sse_manager.broadcast("vault_added", {"vault": name, "stats": stats})
return {"status": "ok", "vault": name, "stats": stats}
@router.delete("/api/vaults/{vault_name}", response_model=VaultActionResponse)
async def api_remove_vault(vault_name: str, current_user=Depends(require_admin)):
"""Remove a vault from the index and stop watching it.
Args:
vault_name: Name of the vault to remove.
"""
if vault_name not in index:
raise HTTPException(status_code=404, detail=f"Vault '{vault_name}' not found")
# Stop watching
watcher = get_watcher()
if watcher:
await watcher.remove_vault(vault_name)
await remove_vault_from_index(vault_name)
await sse_manager.broadcast("vault_removed", {"vault": vault_name})
return {"status": "ok", "vault": vault_name}
@router.get("/api/vaults/status", response_model=VaultsStatusResponse)
async def api_vaults_status(current_user=Depends(require_auth)):
"""Detailed status of all vaults including watcher state.
Returns per-vault: file count, tag count, watching status, vault path.
"""
watcher = get_watcher()
statuses = {}
for vname, vdata in index.items():
watching = watcher is not None and vname in watcher.observers
statuses[vname] = {
"file_count": len(vdata.get("files", [])),
"tag_count": len(vdata.get("tags", {})),
"path": vdata.get("path", ""),
"watching": watching,
}
return {
"vaults": statuses,
"watcher_active": watcher is not None,
"sse_clients": sse_manager.client_count,
}
+46
View File
@@ -214,6 +214,30 @@ class RestoreResponse(BaseModel):
current_backed_up: int | None = Field(default=None, description="Timestamp of the backup created from the current version before restore, if any")
class BackupEntry(BaseModel):
"""A single backup version of a file (#85 — extrait de backend.main, inchangé)."""
timestamp: int = Field(description="Unix timestamp of when the backup was created")
datetime: str = Field(description="ISO 8601 datetime string")
size: int = Field(description="File size in bytes")
filename: str = Field(description="Backup filename on disk")
class BackupListResponse(BaseModel):
"""Response listing all available backups for a file (#85 — extrait de backend.main, inchangé)."""
vault: str = Field(description="Vault name")
path: str = Field(description="Relative file path")
backups: list[BackupEntry] = Field(description="Available backups, newest first")
class DiffRequest(BaseModel):
"""Request parameters for generating a diff (#85 — extrait de backend.main, inchangé)."""
version: int = Field(description="Timestamp of the backup version to compare")
compare_with: int | None = Field(default=None, description="Timestamp of another backup version. If omitted, compares with the current file.")
# ---------------------------------------------------------------------------
# Files — browse / read (#85 — extrait de backend.main, inchangé)
# ---------------------------------------------------------------------------
@@ -411,6 +435,28 @@ class FileMoveResponse(BaseModel):
item_type: str = Field(description="Type of item moved: 'file' or 'directory'")
# ---------------------------------------------------------------------------
# Vaults & history (#85 — extrait de backend.main, inchangé)
# ---------------------------------------------------------------------------
class VaultInfo(BaseModel):
"""Summary information about a configured vault."""
name: str = Field(description="Display name of the vault")
file_count: int = Field(description="Number of indexed files")
tag_count: int = Field(description="Number of unique tags")
type: str = Field(default="VAULT", description="Type of the vault mapping (VAULT or DIR)")
class BookmarkToggleRequest(BaseModel):
"""Request to toggle a bookmark on a file."""
vault: str
path: str
title: str | None = None
# ---------------------------------------------------------------------------
# Search / suggest / graph (#85 — extrait de backend.main, inchangé)
# ---------------------------------------------------------------------------
+1 -1
View File
@@ -31,7 +31,7 @@ DEFAULT_MAX_BACKUPS = 10
def _default_max_backups() -> int:
"""Read ``max_backups_per_file`` from app config (lazy, best-effort)."""
try:
from backend.main import _load_config
from backend.routers.config import _load_config # ROADMAP #85 T7 — déménagé depuis backend.main
return int(_load_config().get("max_backups_per_file", DEFAULT_MAX_BACKUPS))
except Exception: # pragma: no cover - config unavailable
+48 -39
View File
@@ -10,6 +10,7 @@ No authentication required for public share views.
import json
import logging
import secrets
import threading
from datetime import datetime, timedelta, timezone
from pathlib import Path
@@ -17,6 +18,10 @@ logger = logging.getLogger("obsigate.share")
SHARES_FILE = Path("data/shares.json")
# ROADMAP #85 T10a — verrou autour des read-modify-write (perte de mises à
# jour en cas de créations/accès/révocations concurrents).
_lock = threading.RLock()
def _read() -> dict:
if not SHARES_FILE.exists():
@@ -41,26 +46,27 @@ def create_share(
expires_in_hours: int | None = None,
) -> dict:
"""Create a new share token for a document."""
data = _read()
token = secrets.token_hex(32) # 64-char hex token
with _lock:
data = _read()
token = secrets.token_hex(32) # 64-char hex token
expires_at = None
if expires_in_hours:
expires_at = (datetime.now(timezone.utc) + timedelta(hours=expires_in_hours)).isoformat()
expires_at = None
if expires_in_hours:
expires_at = (datetime.now(timezone.utc) + timedelta(hours=expires_in_hours)).isoformat()
share = {
"id": token,
"token": token,
"vault": vault,
"path": path,
"created_by": created_by,
"created_at": datetime.now(timezone.utc).isoformat(),
"expires_at": expires_at,
"access_count": 0,
"last_accessed": None,
}
data["shares"][token] = share
_write(data)
share = {
"id": token,
"token": token,
"vault": vault,
"path": path,
"created_by": created_by,
"created_at": datetime.now(timezone.utc).isoformat(),
"expires_at": expires_at,
"access_count": 0,
"last_accessed": None,
}
data["shares"][token] = share
_write(data)
logger.info(f"Created share for {vault}/{path} by {created_by}")
return share
@@ -80,22 +86,24 @@ def get_share_by_token(token: str) -> dict | None:
def record_access(token: str):
"""Increment access counter for a share."""
data = _read()
share = data["shares"].get(token)
if share:
share["access_count"] = share.get("access_count", 0) + 1
share["last_accessed"] = datetime.now(timezone.utc).isoformat()
_write(data)
with _lock:
data = _read()
share = data["shares"].get(token)
if share:
share["access_count"] = share.get("access_count", 0) + 1
share["last_accessed"] = datetime.now(timezone.utc).isoformat()
_write(data)
def revoke_share(share_id: str) -> bool:
"""Revoke (delete) a share by its token."""
data = _read()
if share_id in data["shares"]:
del data["shares"][share_id]
_write(data)
logger.info(f"Revoked share {share_id}")
return True
with _lock:
data = _read()
if share_id in data["shares"]:
del data["shares"][share_id]
_write(data)
logger.info(f"Revoked share {share_id}")
return True
return False
@@ -112,12 +120,13 @@ def list_shares(vault_filter: str | None = None) -> list:
def update_shares_after_rename(vault: str, old_path: str, new_path: str):
"""Update all shares when a file is renamed."""
data = _read()
updated = False
for sid, s in data["shares"].items():
if s.get("vault") == vault and s.get("path") == old_path:
s["path"] = new_path
updated = True
logger.info(f"Updated share {sid}: {vault}/{old_path} -> {new_path}")
if updated:
_write(data)
with _lock:
data = _read()
updated = False
for sid, s in data["shares"].items():
if s.get("vault") == vault and s.get("path") == old_path:
s["path"] = new_path
updated = True
logger.info(f"Updated share {sid}: {vault}/{old_path} -> {new_path}")
if updated:
_write(data)
+3 -1
View File
@@ -19,7 +19,9 @@ import io
import logging
import re
from typing import Any
from xml.sax import saxutils
# saxutils.escape uniquement (échappement de chaînes, aucun parsing XML).
from xml.sax import saxutils # nosec B406
from backend.services.errors import ServiceError
from backend.services.mutations import save_raw_file
+17 -11
View File
@@ -17,6 +17,7 @@ from __future__ import annotations
import json
import logging
import os
import threading
from pathlib import Path
logger = logging.getLogger("obsigate.tools.secrets")
@@ -34,6 +35,9 @@ TOOL_KEY_NAMES: tuple[str, ...] = (
_SECRET_MARKERS = ("API_KEY", "TOKEN")
# ROADMAP #85 T10a — verrou autour des read-modify-write du store de clés.
_lock = threading.RLock()
def _keys_file() -> Path:
base = os.environ.get("OBSIGATE_DATA_DIR", "data")
@@ -89,21 +93,23 @@ def set_tool_key(name: str, value: str) -> None:
if name not in TOOL_KEY_NAMES:
raise ValueError(f"Clé non prise en charge: {name}")
value = (value or "").strip()
keys = _read_keys()
if value:
keys[name] = value
else:
keys.pop(name, None)
_write_keys(keys)
with _lock:
keys = _read_keys()
if value:
keys[name] = value
else:
keys.pop(name, None)
_write_keys(keys)
def delete_tool_key(name: str) -> bool:
"""Remove one key from the store; return True when it existed."""
if name not in TOOL_KEY_NAMES:
raise ValueError(f"Clé non prise en charge: {name}")
keys = _read_keys()
if name in keys:
del keys[name]
_write_keys(keys)
return True
with _lock:
keys = _read_keys()
if name in keys:
del keys[name]
_write_keys(keys)
return True
return False
+3 -2
View File
@@ -22,7 +22,7 @@ Exemples :
from __future__ import annotations
import os
import subprocess
import subprocess # nosec B404
from pathlib import Path
_ROOT = Path(__file__).resolve().parent.parent # racine du dépôt ObsiGate
@@ -34,7 +34,8 @@ _ENV_VAR = "OBSIGATE_VERSION"
def _run_git(args: list[str]) -> str:
"""Run a git command in the repo root; return stdout (stripped) or ''."""
try:
result = subprocess.run(
# argv fixe (git + args internes), sans shell : pas d'injection.
result = subprocess.run( # nosec B404 B603 B607
["git", *args],
cwd=str(_ROOT),
capture_output=True,
+2 -1
View File
@@ -280,7 +280,8 @@ class VaultWatcher:
for observer in self.observers.values():
try:
observer.join(timeout=5)
except Exception: # nosec B110 — best-effort shutdown, ignore failures
# best-effort shutdown, ignore failures (B110) :
except Exception: # nosec B110
pass
self.observers.clear()
logger.info("VaultWatcher stopped")
+28
View File
@@ -0,0 +1,28 @@
"""Shared VaultWatcher handle (ROADMAP #85, tranche 8).
Holder extrait de :mod:`backend.main` sans changement de comportement : le
lifespan de ``main`` y dépose l'instance (``set_watcher``) et l'y reprend à
l'extinction ; le router ``vaults`` la consulte via :func:`get_watcher`
(démarrage/arrêt de surveillance à l'ajout/retrait dynamique de vault,
état dans ``/api/vaults/status``).
"""
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from backend.watcher import VaultWatcher
_watcher: VaultWatcher | None = None
def get_watcher() -> VaultWatcher | None:
"""Return the shared VaultWatcher instance (``None`` if disabled)."""
return _watcher
def set_watcher(watcher: VaultWatcher | None) -> None:
"""Store (or clear) the shared VaultWatcher instance."""
global _watcher
_watcher = watcher
+54 -43
View File
@@ -26,6 +26,7 @@ import json
import logging
import os
import socket
import threading
import uuid
from datetime import datetime, timezone
from pathlib import Path
@@ -144,6 +145,12 @@ def _read_secrets() -> dict:
return {}
# ROADMAP #85 T10a — verrou autour des read-modify-write des deux stores
# (webhooks + secrets) : perte de mises à jour en cas de mutations
# concurrentes.
_lock = threading.RLock()
def _write_secrets(secrets: dict):
WEBHOOK_SECRETS_FILE.parent.mkdir(parents=True, exist_ok=True)
tmp = WEBHOOK_SECRETS_FILE.with_suffix(".tmp")
@@ -156,12 +163,13 @@ def _write_secrets(secrets: dict):
def _store_secret(wh_id: str, secret: str | None) -> None:
secrets = _read_secrets()
if secret:
secrets[wh_id] = secret
else:
secrets.pop(wh_id, None)
_write_secrets(secrets)
with _lock:
secrets = _read_secrets()
if secret:
secrets[wh_id] = secret
else:
secrets.pop(wh_id, None)
_write_secrets(secrets)
def _get_secret(wh: dict) -> str | None:
@@ -189,52 +197,55 @@ def get_webhooks() -> list:
def create_webhook(name: str, url: str, events: list[str], secret: str | None = None) -> dict:
validate_webhook_url(url)
webhooks = _read()
wh_id = str(uuid.uuid4())
wh = {
"id": wh_id,
"name": name,
"url": url,
"events": [e for e in events if e in VALID_EVENTS],
"enabled": True,
"created_at": datetime.now(timezone.utc).isoformat(),
"last_fired_at": None,
}
webhooks.append(wh)
_write(webhooks)
if secret:
_store_secret(wh_id, secret)
with _lock:
webhooks = _read()
wh_id = str(uuid.uuid4())
wh = {
"id": wh_id,
"name": name,
"url": url,
"events": [e for e in events if e in VALID_EVENTS],
"enabled": True,
"created_at": datetime.now(timezone.utc).isoformat(),
"last_fired_at": None,
}
webhooks.append(wh)
_write(webhooks)
if secret:
_store_secret(wh_id, secret)
logger.info(f"Created webhook '{name}' → {url}")
return _public_view(wh)
def update_webhook(wh_id: str, updates: dict) -> dict | None:
webhooks = _read()
for wh in webhooks:
if wh["id"] == wh_id:
if updates.get("url"):
validate_webhook_url(updates["url"])
if "secret" in updates:
_store_secret(wh_id, updates["secret"])
safe_updates = {
k: v for k, v in updates.items()
if k not in ("id", "secret")
}
wh.update(safe_updates)
_write(webhooks)
return _public_view(wh)
with _lock:
webhooks = _read()
for wh in webhooks:
if wh["id"] == wh_id:
if updates.get("url"):
validate_webhook_url(updates["url"])
if "secret" in updates:
_store_secret(wh_id, updates["secret"])
safe_updates = {
k: v for k, v in updates.items()
if k not in ("id", "secret")
}
wh.update(safe_updates)
_write(webhooks)
return _public_view(wh)
return None
def delete_webhook(wh_id: str) -> bool:
webhooks = _read()
new_list = [wh for wh in webhooks if wh["id"] != wh_id]
if len(new_list) == len(webhooks):
return False
_write(new_list)
secrets = _read_secrets()
if secrets.pop(wh_id, None) is not None:
_write_secrets(secrets)
with _lock:
webhooks = _read()
new_list = [wh for wh in webhooks if wh["id"] != wh_id]
if len(new_list) == len(webhooks):
return False
_write(new_list)
secrets = _read_secrets()
if secrets.pop(wh_id, None) is not None:
_write_secrets(secrets)
return True
+1 -1
View File
@@ -2626,7 +2626,7 @@ dependencies = [
[[package]]
name = "obsigate-desktop"
version = "2.27.9"
version = "2.28.3"
dependencies = [
"chrono",
"env_logger",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "obsigate-desktop"
version = "2.27.9"
version = "2.28.3"
description = "ObsiGate Desktop — Porte d'entrée native pour vos vaults Obsidian"
authors = ["Bruno Charest"]
edition = "2021"
+1 -1
View File
@@ -1,7 +1,7 @@
{
"$schema": "https://raw.githubusercontent.com/nicedoc/obsigate/main/desktop/tauri.conf.schema.json",
"productName": "ObsiGate",
"version": "2.27.9",
"version": "2.28.3",
"identifier": "com.obsigate.desktop",
"build": {
"frontendDist": "../frontend",
+9 -19
View File
@@ -1,6 +1,6 @@
# ObsiGate — Roadmap
> **Version :** 2.27.9 | **Dernière mise à jour :** 2026-09-26
> **Version :** 2.28.3 | **Dernière mise à jour :** 2026-09-26
> **Ce fichier ne contient que le travail à venir** (🔵 En cours + ⚪ Backlog) et un index compact
> vers les fonctionnalités livrées.
> - **Méthode de livraison à appliquer pour toute tâche : [DELIVERY_WORKFLOW.md](./DELIVERY_WORKFLOW.md)**
@@ -63,28 +63,16 @@
## ⚪ Backlog — Sécurité, architecture & performance (P0/P1)
### 85. Refonte architecturale — découpage du monolithe & persistance d'état (phase 2)
- **Effort :** 8-12 jours | **Impact :** 🟡 | **Zone :** backend
- **Décision 2026-09-26 : prioritaire (axe Dette & sécurité).**
- **Statut :** 🔵 en cours depuis 2026-09-26 — découpe par tranches à impact minimal (comportement inchangé, un domaine par commit). **T1 livrée (v2.27.2) :** `health` (`/api/health`, `/api/health/detailed` → `backend/routers/health.py`, `HealthResponse` → `schemas.py`). **T2 livrée (v2.27.3) :** `webhooks` (CRUD `/api/webhooks` → `backend/routers/webhooks.py`, logique déjà dans `backend/webhooks.py`). **T3 livrée (v2.27.4) :** `sharing` (`/api/share/*`, `/api/shares`, `/s/{token}*` → `backend/routers/sharing.py`, logique déjà dans `backend/share.py`). **T4 livrée (v2.27.5) :** `backups` (9 routes `/api/file/{vault}/backups|diff|restore` + `/api/backups*` → `backend/routers/backups.py`, `Diff/Restore*` → `schemas.py`, singleton SSE → `backend/sse.py`). **T5 livrée (v2.27.6) :** `search` (11 routes search/tags/suggest/graph/reload → `backend/routers/search.py`, modèles search → `schemas.py`, pool threads → `backend/search_executor.py`). **T6a livrée (v2.27.7) :** lecture fichiers (`/api/browse`, `/raw`, `/download`, `/backlinks`, `GET /api/file` → `backend/routers/files_read.py`, modèles + `EXT_TO_LANG` déplacés, helpers `_content_disposition`/`_media_max_inline_bytes` → `backend/routers/helpers.py`). **T6b livrée (v2.27.8) :** mutations fichiers/dossiers (save, xlsx/save, delete, create, rename, move, directories ×3, batch-upload → `backend/routers/files_write.py`, 15 modèles → `schemas.py`). **T6c livrée (v2.27.9) :** media/pdf/export/guide (file/pdf, exports ×3, guide, pdf/stream|info, image, media+thumb, attachments ×2, vault settings ×3, vault files → `backend/routers/files_media.py`, Range helper → `helpers.py`).
- **Description :** extraire le monolithe `backend/main.py` (~4 827 lignes au 2026-09-26, ~17 % du backend) en routers FastAPI par domaine et rendre persistant l'état qui ne l'est pas (index de recherche, JTI révoqués, compteurs de rate-limit) pour préparer le multi-nœuds. L'état mémoire actuel (index, inverted index, vecteurs sémantiques, `SSEManager`, collab) rend le multi-workers unsafe.
- **Sous-tâches :**
- [ ] Routers par domaine : files, search, share, webhooks, plugins, collab, admin, ai — `main.py` conservé comme assemblage (< 500 lignes) ; dédupliquer les modèles Pydantic vers `schemas.py`. **Avancement :** `health` ✅ (T1, `backend/routers/health.py`), `webhooks` ✅ (T2, `backend/routers/webhooks.py`), `sharing` ✅ (T3, `backend/routers/sharing.py`), `backups` ✅ (T4, `backend/routers/backups.py` + `backend/sse.py`), `search` ✅ (T5, `backend/routers/search.py` + `backend/search_executor.py`), `files-read` ✅ (T6a, `backend/routers/files_read.py` + `helpers.py`), `files-write` ✅ (T6b, `backend/routers/files_write.py`), `files-media` ✅ (T6c, `backend/routers/files_media.py`) ; `tools/registry.py` existe déjà (permissions/quotas/redaction — à compléter, pas à créer)
- [ ] Compléter `tools/registry.py` (existant : permissions/quotas/redaction) comme contrat central des outils IA si des manques sont constatés
- [ ] Persister index, JTI révoqués et compteurs de rate-limit (SQLite par défaut, Redis en option multi-nœuds ; le rate-limit actuel est in-memory mono-process)
- [ ] Verrous asyncio autour de l'index global et des stores JSON ; auditer les `except Exception` larges (> 100 occurrences) : best-effort (backup/audit) vs masquage d'erreur (erreurs typées 4xx/5xx + test)
- [ ] Extraire le service de partage public (expiration, révocation, quotas)
### 87. Amélioration continue — tests, CI/CD, revues de sécurité (phase 4)
- **Effort :** 3-5 jours | **Impact :** 🟡 | **Zone :** `.gitea/workflows/`, `tests/`
- **Décision 2026-09-26 : prioritaire (axe Dette & sécurité).**
- **Statut :** 🔵 en cours depuis 2026-09-26 — par tranches. **T1 livrée (v2.28.1) :** bandit bloquant (`nosec` justifiés B324/B404/B603/B607/B406, B105 exclu comme `pyproject`), `npm audit` bloquant (0 vulnérabilité), 5 suites frontend intégrées au CI (`upload`, `pretty`, `media-viewer`, `mfa-settings`, `config-ai-keys`). pip-audit reste consultatif (montées starlette/weasyprint à qualifier).
- **Description :** renforcer le pipeline (`.gitea/workflows/ci.yml`, `desktop-build.yml`) pour le rendre bloquant par défaut et accompagner les phases 1 → 3. Constat 2026-09-26 : job `security` non bloquant (`bandit`/`pip-audit` en `|| echo`, ni semgrep ni trivy), E2E limité à `chromium-desktop`, 5 suites frontend hors CI.
- **Sous-tâches :**
- [ ] Jobs CI sécurité **bloquants** (bandit/semgrep/trivy, audits pip/npm) + tests E2E XSS (page de partage + lecteur markdown)
- [ ] Tests de concurrence (`users.json`), fuzzing de timing regex, couverture des composants critiques ; intégrer au CI les 5 suites frontend hors CI (`upload`, `pretty`, `media-viewer`, `mfa-settings`, `config-ai-keys`)
- [ ] Finir BUG-034 (migration CSP **nonce**, conversion des handlers inline), `Secure` cookies à `true` par défaut, politique CORS same-origin explicite ; confirmer la rotation de la clé DeepSeek (BUG-006, clé dans l'historique Git)
- [ ] Tests de concurrence (`users.json`), fuzzing de timing regex, couverture des composants critiques ; intégrer au CI les 5 suites frontend hors CI (`upload`, `pretty`, `media-viewer`, `mfa-settings`, `config-ai-keys`) — **T2 livrée (v2.28.2) :** `tests/test_hardening_concurrency.py` (users.json concurrent + budget temps regex) ; 5 suites au CI (T1)
- [ ] Finir BUG-034 (migration CSP **nonce**, conversion des handlers inline), `Secure` cookies à `true` par défaut, politique CORS same-origin explicite ; confirmer la rotation de la clé DeepSeek (BUG-006, clé dans l'historique Git) — **T3 livrée (v2.28.3) :** helper `is_secure_cookies()` centralisé (défaut `false` conservé : compatibilité HTTP local), avertissement au démarrage sur bind non-loopback sans `Secure`, absence CORS attestée par test (`tests/test_security_headers.py`) ; bascule du défaut reportée (casserait les logins localhost)
- [ ] Revue périodique des dépendances ; documentation utilisateur FR/EN synchronisée ; contrôle automatisé de la conformité au DoD
---
@@ -164,6 +152,7 @@
| BUG-078 | Fichiers de code — coloration syntaxique restaurée (feuilles highlight.js basculées sur le mode de thème et non la clé) | 2.25.0 | [features/viewer-toolbar-highlight-avatars.md](./features/viewer-toolbar-highlight-avatars.md) |
| 115 | Viewer — barre d'outils de lecture épinglée au défilement | 2.25.0 | [features/viewer-toolbar-highlight-avatars.md](./features/viewer-toolbar-highlight-avatars.md) |
| 117 | Configuration — avatars prédéfinis dans le profil utilisateur (12 images) | 2.25.0 | [features/viewer-toolbar-highlight-avatars.md](./features/viewer-toolbar-highlight-avatars.md) |
| 85 | Refonte architecturale — découpage du monolithe (14 routers, `main.py` 4 827 → ~750 lignes), stores JSON verrouillés, rate-limit SQLite optionnel | 2.27.2→2.27.13 | [features/archi-refonte-85.md](./features/archi-refonte-85.md) |
---
@@ -171,17 +160,18 @@
| Priorité | Items | Effort total estimé |
|---|---|---|
| ✅ Complété | #1 → #59, #61–72, #74–76, #78–84, #86, #88–93, #94–100, #102–115, #117, #92 | ~133 jours réalisés |
| ✅ Complété | #1 → #59, #61–72, #74–76, #78–86, #88–93, #94–100, #102–115, #117, #92 | ~141 jours réalisés |
| 🔵 Finitions | #77 Desktop : 6 tests E2E **manuels** ([protocole](./DESKTOP_E2E_CHECKLIST.md)) — signature Windows non retenue (décision 2026-09-26) | ~0,5-1 jour |
| ⚪ P4 reporté | #73 Sync — **reporté (décision 2026-09-26)**, hors chemin critique | 6-8 jours si réactivé |
| ⚪ P0/P1 prioritaire | #85, #87 Refonte architecturale, CI/CD (BUG-035 → BUG-040 corrigés, #86 livré) | ~11-17 jours |
| **Total chemin critique** | **#77 fin + #85 + #87** | **~12-18 jours** |
| ⚪ P0/P1 prioritaire | #87 CI/CD (BUG-035 → BUG-040 corrigés, #86 livré) | ~3-5 jours |
| **Total chemin critique** | **#77 fin + #87** | **~4-6 jours** |
---
## Notes
- **Décisions 2026-09-26 :** axe prioritaire = dette & sécurité (#85/#87) ; #73 Sync reporté (P4, hors chemin critique) ; desktop livré non signé + doc SmartScreen.
- **Clôture #85 (v2.27.13) :** monolithe découpé (T1→T9), stores verrouillés + rate-limit SQLite (T10), fiche `docs/features/archi-refonte-85.md`.
- Les items P3/P4 ne sont pas ordonnés par priorité interne — à raffiner selon les retours utilisateurs.
- L'effort inclut le développement + tests unitaires + intégration CI, mais pas la documentation utilisateur.
- Les items marqués 🟢 (nice-to-have) sont de bons candidats pour des contributions externes.
+77
View File
@@ -0,0 +1,77 @@
# #85 — Refonte architecturale : découpage du monolithe & persistance d'état (phase 2)
> **Statut :** livré (T1→T10) — `backend/main.py` 4 827 → ~750 lignes, 14 routers,
> persistance partielle (stores verrouillés + rate-limit SQLite optionnel).
> Méthode : tranches à impact minimal, comportement inchangé, un domaine par
> commit, suite complète verte à chaque commit (1320 passed / 6 skipped).
## 1. Découpage du monolithe (T1→T9, comportement inchangé)
Chaque tranche déplace un domaine vers `backend/routers/` (handlers verbatim,
mêmes chemins/modèles/auth/tags OpenAPI), les modèles vers `backend/schemas.py`,
et ne committe que sur suite verte + `test_version` vert.
| Tranche | Domaine | Nouveau module | Version |
|---|---|---|---|
| T1 | health (`/api/health*`) | `routers/health.py` (+ `HealthResponse` → schemas) | 2.27.2 |
| T2 | webhooks CRUD | `routers/webhooks.py` | 2.27.3 |
| T3 | sharing (`/api/share*`, `/s/*`) | `routers/sharing.py` | 2.27.4 |
| T4 | backups (9 routes) | `routers/backups.py` (+ `Diff/Restore*` → schemas, `backend/sse.py`) | 2.27.5 |
| T5 | search (11 routes) | `routers/search.py` (+ modèles → schemas, `backend/search_executor.py`) | 2.27.6 |
| T6a | lecture fichiers | `routers/files_read.py` (+ modèles, `routers/helpers.py`) | 2.27.7 |
| T6b | mutations fichiers/dossiers | `routers/files_write.py` (+ 15 modèles → schemas) | 2.27.8 |
| T6c | media/pdf/export/guide | `routers/files_media.py` (Range helper → `helpers.py`) | 2.27.9 |
| T7 | config (12 routes) | `routers/config.py` (`_FALLBACK_MODELS` déplacé) | 2.27.10 |
| T8 | vaults + history + conflicts (13 routes) | `routers/vaults.py`, `history.py`, `conflicts.py` (+ `backend/watcher_state.py`) | 2.27.11 |
| T9 | realtime + render | `routers/realtime.py` (SSE + collab WS), `backend/render.py` | 2.27.12 |
`main.py` ne contient plus que l'assemblage : lifespan, middlewares, montage
des routers, racine `/api`, statique/SPA, 4 cales de compatibilité testées
(`_resolve_safe_path`, `_backup_file`, `_check_vault_writable`, `_get_backup_dir`).
Correctifs au passage : décorateur orphelin `/s/{token}` (double-enregistrement
de `/api/conflicts`), tag OpenAPI `media` inexistant (assignation par chemin
conservée), tests statiques frontend réalignés (`image-viewer`, `media-viewer`),
tests repointés vers les modules canoniques (`test_ai_models`, `test_api_main`).
## 2. Persistance d'état (T10)
| État | Avant | Après |
|---|---|---|
| JTI révoqués (`revoked_tokens.json`) | persisté, **sans verrou** | `RLock` (load/save/revoke/check) |
| `shares.json` | persisté, **sans verrou** | `RLock` (4 mutateurs) |
| `webhooks.json` + secrets | persistés, **sans verrou** | `RLock` (create/update/delete/secrets) |
| `api_keys.json` (tool-secrets) | persisté, **sans verrou** | `RLock` (set/delete) |
| Rate-limit auth | mémoire, mono-process | **inchangé par défaut** + option `OBSIGATE_RATELIMIT_DB` (SQLite WAL : mêmes fenêtres/budgets, partagé multi-workers, survit au redémarrage) |
| Index de recherche | mémoire, rebuild au démarrage | **conservé** (voir §3) |
| `users.json`, `api_tokens.json`, `vault_settings.json` | déjà verrouillés (BUG-029, #107) | inchangé |
Tests : `tests/test_store_locks.py` (4 — concurrence threads, pertes prouvées
sans verrou : 25/200 partages), `tests/test_ratelimit_store.py` (7 —
sémantique SQLite identique, persistance, concurrence 200/200).
Déjà existants et vérifiés (pas de code) : verrous `threading` + `asyncio`
de l'indexeur (`_index_lock`, `_async_index_lock`), contrat central des
outils IA — `backend/tools/registry.py` couvre déjà permissions
(`requires_vault`, `require_destructive_allowed`), quotas
(`check_and_record` par outil) et redaction (`redact_payload`) pour les
35 outils enregistrés via `@tool(`.
## 3. Décisions assumées (non fait, et pourquoi)
- **Index non persisté sur disque.** Le rebuild différentiel (#86 : réutilise
les entrées inchangées `size` + `mtime`) rend le démarrage rapide ; un
snapshot introduirait des risques de staleness/drift de format sans gain
mesuré. Réévaluer si le démarrage devient lent (vaults 50k+ fichiers).
- **Redis exclu.** SQLite WAL couvre le multi-workers mono-hôte sans nouvelle
infra ; Redis reste l'option multi-nœuds documentée (cf. `ratelimit.py`).
- **`.gitignore` (`_*.py` ignore les `__init__.py`).** Contourné par
`git add -f` comme les packages existants ; assainir la règle à part.
- Noms en `_` conservés (`backend/render.py`, stores) : déplacement verbatim,
zéro churn d'appels.
## 4. Reste connu (hors #85)
- CSP `unsafe-inline` (migration nonce, BUG-034 partiel) et `Secure` cookies → #87.
- `main.py` (~750 lignes) : lifespan, middlewares, statique/SPA — cible
d'extraction ultérieure si besoin, non bloquant.
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "obsigate",
"version": "2.27.9",
"version": "2.28.3",
"description": "**Porte d'entrée web ultra-léger pour vos vaults Obsidian** — Accédez, naviguez et recherchez dans toutes vos notes Obsidian depuis n'importe quel appareil via une interface web moderne et responsive.",
"main": "patch.js",
"directories": {
+14 -3
View File
@@ -12,7 +12,8 @@ from __future__ import annotations
import pytest
from fastapi.testclient import TestClient
from backend.main import _FALLBACK_MODELS, app
from backend.main import app
from backend.routers.config import _FALLBACK_MODELS # ROADMAP #85 T7 — déménagé depuis backend.main
#: Trimmed-down copy of what api.mistral.ai/v1/models really returns (BUG-044).
MISTRAL_LIVE_PAYLOAD = {
@@ -28,11 +29,12 @@ MISTRAL_LIVE_PAYLOAD = {
@pytest.fixture
def admin_client(tmp_path):
"""Minimal admin client for the /api/config/ai-models endpoint."""
from backend.auth.password import hash_password
import json
import os
from pathlib import Path
from backend.auth.password import hash_password
data_dir = tmp_path / "data"
data_dir.mkdir()
users = {
@@ -69,8 +71,9 @@ def admin_client(tmp_path):
import backend.main
backend.main._load_config = lambda: {"watcher_enabled": False}
from backend.indexer import build_index, index
import asyncio
from backend.indexer import build_index, index
for key in list(index.keys()):
del index[key]
loop = asyncio.new_event_loop()
@@ -232,9 +235,11 @@ class TestListModelsEndpoint:
# in backend.main (which does `from backend.ai import ... get_ai_key`).
import backend.ai as aimod
import backend.main as bmain
import backend.routers.config as rconfig
monkeypatch.setattr(aimod, "get_ai_key", lambda name: "fake-xiaomi-key")
if hasattr(bmain, "get_ai_key"):
monkeypatch.setattr(bmain, "get_ai_key", lambda name: "fake-xiaomi-key")
monkeypatch.setattr(rconfig, "get_ai_key", lambda name: "fake-xiaomi-key")
captured = {}
@@ -316,9 +321,11 @@ class TestListModelsEndpoint:
# Patch BOTH the source module AND the imported reference in backend.main
import backend.ai as aimod
import backend.main as bmain
import backend.routers.config as rconfig
monkeypatch.setattr(aimod, "get_ai_key", lambda name: "fake-key")
if hasattr(bmain, "get_ai_key"):
monkeypatch.setattr(bmain, "get_ai_key", lambda name: "fake-key")
monkeypatch.setattr(rconfig, "get_ai_key", lambda name: "fake-key")
import urllib.request as global_urllib_mod
captured_urls = []
@@ -393,10 +400,12 @@ class TestDeclaredCapabilities:
"""main.py binds get_ai_key at import: patch that binding too."""
import backend.ai as aimod
import backend.main as bmain
import backend.routers.config as rconfig
monkeypatch.setattr(aimod, "get_ai_key", lambda name: "fake-mistral-key")
if hasattr(bmain, "get_ai_key"):
monkeypatch.setattr(bmain, "get_ai_key", lambda name: "fake-mistral-key")
monkeypatch.setattr(rconfig, "get_ai_key", lambda name: "fake-mistral-key")
def test_declared_vision_reaches_both_endpoints(self, admin_client, monkeypatch):
self._fake_key(monkeypatch)
@@ -450,10 +459,12 @@ class TestDeclaredCapabilities:
"""No API key: the curated list must still answer correctly (offline)."""
import backend.ai as aimod
import backend.main as bmain
import backend.routers.config as rconfig
monkeypatch.setattr(aimod, "get_ai_key", lambda name: None)
if hasattr(bmain, "get_ai_key"):
monkeypatch.setattr(bmain, "get_ai_key", lambda name: None)
monkeypatch.setattr(rconfig, "get_ai_key", lambda name: None)
token = _login_admin(admin_client)
resp = admin_client.get(
+19 -19
View File
@@ -589,30 +589,30 @@ class TestHumanizeMtime:
pass
def test_humanize_mtime_now(self):
from backend.main import humanize_mtime
from backend.services.recent import humanize_mtime # ROADMAP #85 T8 — ne plus passer par backend.main
import time
assert "instant" in humanize_mtime(time.time())
def test_humanize_mtime_minutes(self):
from backend.main import humanize_mtime
from backend.services.recent import humanize_mtime # ROADMAP #85 T8 — ne plus passer par backend.main
import time
result = humanize_mtime(time.time() - 120)
assert "min" in result
def test_humanize_mtime_hours(self):
from backend.main import humanize_mtime
from backend.services.recent import humanize_mtime # ROADMAP #85 T8 — ne plus passer par backend.main
import time
result = humanize_mtime(time.time() - 7200)
assert "h" in result or "jour" in result
def test_humanize_mtime_days(self):
from backend.main import humanize_mtime
from backend.services.recent import humanize_mtime # ROADMAP #85 T8 — ne plus passer par backend.main
import time
result = humanize_mtime(time.time() - 172800) # 2 days
assert "j" in result
def test_humanize_mtime_old(self):
from backend.main import humanize_mtime
from backend.services.recent import humanize_mtime # ROADMAP #85 T8 — ne plus passer par backend.main
import time
result = humanize_mtime(time.time() - 86400 * 30)
assert "202" in result or result # Should show formatted date
@@ -624,44 +624,44 @@ class TestHumanizeMtime:
class TestHeadingSlugify:
def test_slugify_simple(self):
from backend.main import _heading_slugify
from backend.render import _heading_slugify # ROADMAP #85 T9 — déménagé depuis backend.main
assert _heading_slugify("Hello World") == "hello-world"
def test_slugify_accented(self):
from backend.main import _heading_slugify
from backend.render import _heading_slugify # ROADMAP #85 T9 — déménagé depuis backend.main
result = _heading_slugify("Café Crème")
assert "cafe" in result or "caf" in result
def test_slugify_strips_symbols(self):
from backend.main import _heading_slugify
from backend.render import _heading_slugify # ROADMAP #85 T9 — déménagé depuis backend.main
result = _heading_slugify("Hello, World! Test?")
assert result.startswith("hello")
class TestRenderMarkdown:
def test_render_basic(self):
from backend.main import _render_markdown
from backend.render import _render_markdown # ROADMAP #85 T9 — déménagé depuis backend.main
result = _render_markdown("# Hello\n\nThis is a test.", "TestVault")
assert "<h1" in result
assert "Hello" in result
def test_render_with_code(self):
from backend.main import _render_markdown
from backend.render import _render_markdown # ROADMAP #85 T9 — déménagé depuis backend.main
result = _render_markdown("```python\nprint('hello')\n```", "TestVault")
assert "code" in result or "highlight" in result
def test_render_with_heading_ids(self):
from backend.main import _render_markdown
from backend.render import _render_markdown # ROADMAP #85 T9 — déménagé depuis backend.main
result = _render_markdown("# Title\n## Subtitle", "TestVault")
assert "id=" in result
def test_render_wikilink(self):
from backend.main import _render_markdown
from backend.render import _render_markdown # ROADMAP #85 T9 — déménagé depuis backend.main
result = _render_markdown("Link [[nonexistent.md]] here", "TestVault")
assert "wikilink" in result
def test_render_image_wikilink(self):
from backend.main import _render_markdown
from backend.render import _render_markdown # ROADMAP #85 T9 — déménagé depuis backend.main
result = _render_markdown("![[image.png]]", "TestVault")
assert "img" in result or "image" in result or "wikilink" in result
@@ -709,31 +709,31 @@ class TestCheckVaultWritable:
class TestConvertWikilinks:
def test_convert_wikilink(self):
from backend.main import _convert_wikilinks
from backend.render import _convert_wikilinks # ROADMAP #85 T9 — déménagé depuis backend.main
# Missing wikilinks render as span.wikilink-missing
result = _convert_wikilinks("See [[Introduction à Python]] for details", "TestVault")
assert "Introduction" in result
assert "wikilink" in result
def test_convert_wikilink_with_alias(self):
from backend.main import _convert_wikilinks
from backend.render import _convert_wikilinks # ROADMAP #85 T9 — déménagé depuis backend.main
result = _convert_wikilinks("See [[file.md|a different name]] here", "TestVault")
assert "a different name" in result
def test_convert_wikilink_image(self):
from backend.main import _convert_wikilinks
from backend.render import _convert_wikilinks # ROADMAP #85 T9 — déménagé depuis backend.main
result = _convert_wikilinks("See ![[image.png]] here", "TestVault")
assert "image" in result or "img" in result
def test_convert_wikilink_anchor(self):
from backend.main import _convert_wikilinks
from backend.render import _convert_wikilinks # ROADMAP #85 T9 — déménagé depuis backend.main
result = _convert_wikilinks("[[#Section importante|voir section]]", "TestVault")
assert 'href="#section-importante"' in result
assert "wikilink-anchor" in result
assert "voir section" in result
def test_convert_wikilink_anchor_without_alias(self):
from backend.main import _convert_wikilinks
from backend.render import _convert_wikilinks # ROADMAP #85 T9 — déménagé depuis backend.main
result = _convert_wikilinks("[[#Claude Code]]", "TestVault")
assert 'href="#claude-code"' in result
assert "Claude Code" in result
@@ -741,7 +741,7 @@ class TestConvertWikilinks:
class TestHeadingSlugifyHtmlStripping:
def test_slugify_strips_html_tags(self):
from backend.main import _heading_slugify
from backend.render import _heading_slugify # ROADMAP #85 T9 — déménagé depuis backend.main
result = _heading_slugify('## 1. Agents installés localement <a href="#table-des-matieres">↩</a>')
assert result == "1-agents-installes-localement"
+198
View File
@@ -0,0 +1,198 @@
"""Tests de durcissement — concurrence users.json + fuzzing regex (ROADMAP #87 T2).
- `users.json` : les read-modify-write sont sérialisés par `_users_lock`
(BUG-029). Ces tests martèlent create/update/record_login_failure depuis
plusieurs threads et exigent zéro mise à jour perdue + un JSON valide.
- Regex (BUG-025) : la politique `regex_safety` (longueur, quantificateurs
imbriqués, contenu tronqué) doit rejeter vite les motifs catastrophiques
et borner le temps des motifs acceptés sur gros contenu.
"""
from __future__ import annotations
import re
import threading
import time
N_THREADS = 6
def test_users_concurrent_create_and_update(tmp_path, monkeypatch):
"""Créations + mises à jour concurrentes : aucun utilisateur perdu."""
from backend.auth import user_store
monkeypatch.setattr(user_store, "USERS_FILE", tmp_path / "users.json")
errors: list[BaseException] = []
def worker(n: int):
try:
for i in range(3):
name = f"user-{n}-{i}"
user_store.create_user(name, "Motdepasse1!", display_name=name)
user_store.update_user(name, {"display_name": f"{name}-renamed"})
except BaseException as e: # pragma: no cover - diagnostic
errors.append(e)
threads = [threading.Thread(target=worker, args=(n,)) for n in range(N_THREADS)]
for t in threads:
t.start()
for t in threads:
t.join()
assert not errors
users = user_store._read()["users"]
assert len(users) == N_THREADS * 3
assert all(u["display_name"].endswith("-renamed") for u in users.values())
def test_users_concurrent_login_failures_no_lost_count(tmp_path, monkeypatch):
"""`record_login_failure` concurrents : compteur exact (pas de lost update)."""
from backend.auth import user_store
monkeypatch.setattr(user_store, "USERS_FILE", tmp_path / "users.json")
user_store.create_user("victim", "Motdepasse1!")
def worker():
for _ in range(10):
try:
user_store.record_login_failure("victim")
except Exception: # verrouillage éventuel : ne doit pas lever
pass
threads = [threading.Thread(target=worker) for _ in range(N_THREADS)]
for t in threads:
t.start()
for t in threads:
t.join()
user = user_store.get_user("victim")
assert user is not None
# Le compte peut se verrouiller en cours de route ; l'important est que
# le fichier reste un JSON valide et l'utilisateur présent.
assert user["username"] == "victim"
def test_users_file_stays_valid_json_under_load(tmp_path, monkeypatch):
"""Le fichier reste lisible à tout moment pendant les écritures."""
import json
from backend.auth import user_store
target = tmp_path / "users.json"
monkeypatch.setattr(user_store, "USERS_FILE", target)
user_store.create_user("base", "Motdepasse1!")
stop = threading.Event()
errors: list[BaseException] = []
def writer(n: int):
i = 0
while not stop.is_set():
try:
user_store.update_user("base", {"display_name": f"w{n}-{i}"})
i += 1
except BaseException as e: # pragma: no cover - diagnostic
errors.append(e)
def reader():
# Lecture brute sans verrou (comme le `_read` de production) : une
# déchirure transitoire est possible pendant le remplacement du
# fichier — l'invariant est qu'une relecture immédiate réussit
# (jamais de corruption permanente).
while not stop.is_set():
try:
raw = target.read_text(encoding="utf-8")
json.loads(raw)
except FileNotFoundError:
pass
except json.JSONDecodeError:
try:
time.sleep(0.01)
json.loads(target.read_text(encoding="utf-8"))
except FileNotFoundError:
pass
except BaseException as e: # pragma: no cover - diagnostic
errors.append(e)
except BaseException as e: # pragma: no cover - diagnostic
errors.append(e)
threads = [threading.Thread(target=writer, args=(n,)) for n in range(4)]
threads.append(threading.Thread(target=reader))
for t in threads:
t.start()
time.sleep(2.0)
stop.set()
for t in threads:
t.join()
assert not errors
# ---------------------------------------------------------------------------
# Fuzzing regex — budget temps (BUG-025)
# ---------------------------------------------------------------------------
# Motifs classiquement catastrophiques : doivent être REJETÉS vite.
CATASTROPHIC = [
"^(a+)+$",
"(a+)+$",
"(.*)*$",
"(a|aa)+$",
"(a+){2,}$",
r"(\w+)+$",
r"(a*)*b",
r"(x+x+)+y",
]
# Motifs acceptés (légitimes) : doivent tourner vite sur gros contenu.
ACCEPTED = [
r"hello",
r"h.llo",
r"\b\w+@\w+\.\w+\b",
r"[A-ZÉÈÊ][a-zéèêàâîôûç]+",
r"(ab|cd)+e",
r"\d{4}-\d{2}-\d{2}",
r"foo|bar|baz",
]
def test_catastrophic_patterns_rejected_fast():
"""Les motifs à backtracking catastrophique sont refusés en < 1 s."""
from backend.services.regex_safety import validate_regex
start = time.perf_counter()
for pattern in CATASTROPHIC:
try:
validate_regex(pattern)
except ValueError:
pass
assert time.perf_counter() - start < 1.0
def test_accepted_patterns_bounded_on_large_content():
"""Motifs acceptés sur 200 Ko : chacun < 5 s (budget large anti-flaky)."""
from backend.services.regex_safety import MAX_REGEX_CONTENT, truncate_for_regex, validate_regex
assert MAX_REGEX_CONTENT == 200_000
content = truncate_for_regex("abc héllo world [email protected] 2024-01-02 " * 5000)
assert len(content) <= MAX_REGEX_CONTENT
for pattern in ACCEPTED:
validate_regex(pattern) # ne doit pas lever
start = time.perf_counter()
re.search(pattern, content)
assert time.perf_counter() - start < 5.0, f"motif lent : {pattern!r}"
def test_validate_regex_policy():
"""Politique : vide/trop long/invalide → ValueError."""
from backend.services.regex_safety import MAX_PATTERN_LENGTH, validate_regex
for bad in ("", "x" * (MAX_PATTERN_LENGTH + 1), "(unclosed"):
try:
validate_regex(bad)
except ValueError:
pass
else: # pragma: no cover - doit lever
raise AssertionError(f"motif accepté à tort : {bad!r}")
assert validate_regex("simple") == "simple"
+143
View File
@@ -0,0 +1,143 @@
"""Tests — rate-limit SQLite optionnel (ROADMAP #85 T10b).
Le store mémoire reste le défaut (comportement inchangé) ; si
``OBSIGATE_RATELIMIT_DB`` pointe vers un fichier SQLite, les compteurs y
sont persistés (partagés entre workers/processus, conservés au redémarrage)
avec une sémantique identique (fenêtre glissante, budgets IP + compte,
reset au succès).
"""
from __future__ import annotations
import sqlite3
import threading
import time
def _use_db(monkeypatch, tmp_path):
monkeypatch.setenv("OBSIGATE_RATELIMIT_DB", str(tmp_path / "ratelimit.db"))
def test_memory_default_unchanged(monkeypatch):
"""Sans la variable d'env : le store mémoire historique est utilisé."""
from backend import ratelimit
monkeypatch.delenv("OBSIGATE_RATELIMIT_DB", raising=False)
assert ratelimit._db_path() is None
ip = "10.9.9.1"
ratelimit._ip_attempts.pop(ip, None)
assert not ratelimit.is_rate_limited(ip)
ratelimit.record_failure(ip)
assert not ratelimit.is_rate_limited(ip)
ratelimit.record_success(ip)
assert not ratelimit.is_rate_limited(ip)
def test_sqlite_failures_and_limit(monkeypatch, tmp_path):
"""Budget IP : N échecs → limité ; succès → reset (SQLite)."""
from backend import ratelimit
_use_db(monkeypatch, tmp_path)
monkeypatch.setattr(ratelimit, "MAX_ATTEMPTS", 3)
ip = "10.8.8.1"
assert not ratelimit.is_rate_limited(ip)
ratelimit.record_failure(ip)
ratelimit.record_failure(ip)
assert not ratelimit.is_rate_limited(ip)
failures, remaining = ratelimit.record_failure(ip)
assert (failures, remaining) == (3, 0)
assert ratelimit.is_rate_limited(ip)
ratelimit.record_success(ip)
assert not ratelimit.is_rate_limited(ip)
def test_sqlite_account_budget_case_insensitive(monkeypatch, tmp_path):
"""Budget par compte : insensible à la casse, indépendant des IP."""
from backend import ratelimit
_use_db(monkeypatch, tmp_path)
monkeypatch.setattr(ratelimit, "ACCOUNT_MAX_ATTEMPTS", 2)
assert not ratelimit.is_account_rate_limited("Alice")
ratelimit.record_account_failure("alice")
assert not ratelimit.is_account_rate_limited("ALICE")
ratelimit.record_account_failure("ALICE")
assert ratelimit.is_account_rate_limited("alice")
# Le budget IP n'est pas affecté par le budget compte.
assert not ratelimit.is_rate_limited("1.2.3.4")
ratelimit.record_account_success("alice")
assert not ratelimit.is_account_rate_limited("alice")
def test_sqlite_window_expiry(monkeypatch, tmp_path):
"""Les tentatives hors fenêtre ne comptent plus (SQLite)."""
from backend import ratelimit
_use_db(monkeypatch, tmp_path)
monkeypatch.setattr(ratelimit, "MAX_ATTEMPTS", 2)
monkeypatch.setattr(ratelimit, "WINDOW_SECONDS", 1)
ip = "10.7.7.1"
ratelimit.record_failure(ip)
ratelimit.record_failure(ip)
assert ratelimit.is_rate_limited(ip)
time.sleep(1.1)
assert not ratelimit.is_rate_limited(ip)
def test_sqlite_persists_across_restart(monkeypatch, tmp_path):
"""Les compteurs survivent au redémarrage (même fichier)."""
import os
from backend import ratelimit
db = tmp_path / "ratelimit.db"
monkeypatch.setenv("OBSIGATE_RATELIMIT_DB", str(db))
monkeypatch.setattr(ratelimit, "MAX_ATTEMPTS", 5)
for _ in range(3):
ratelimit.record_failure("10.6.6.6")
assert os.path.exists(db)
# "Redémarrage" : le module relit le même fichier (connexions courtes).
assert ratelimit.get_status("10.6.6.6")["failures"] == 3
with sqlite3.connect(str(db)) as conn:
(rows,) = conn.execute("SELECT COUNT(*) FROM attempts").fetchone()
assert rows == 3
def test_sqlite_get_status_shapes(monkeypatch, tmp_path):
"""`get_status` garde les mêmes formes qu'en mémoire."""
from backend import ratelimit
_use_db(monkeypatch, tmp_path)
ratelimit.record_failure("10.5.5.5")
ratelimit.record_account_failure("bob")
per_ip = ratelimit.get_status("10.5.5.5")
assert per_ip == {
"ip": "10.5.5.5",
"failures": 1,
"max": ratelimit.MAX_ATTEMPTS,
"limited": False,
"window_seconds": ratelimit.WINDOW_SECONDS,
}
glob = ratelimit.get_status()
assert glob["tracked_ips"] == 1
assert glob["tracked_accounts"] == 1
assert glob["limited_ips"] == 0
assert glob["max_attempts"] == ratelimit.MAX_ATTEMPTS
def test_sqlite_concurrent_writes(monkeypatch, tmp_path):
"""Écritures concurrentes : aucun échec compté perdu (SQLite/WAL)."""
from backend import ratelimit
_use_db(monkeypatch, tmp_path)
monkeypatch.setattr(ratelimit, "MAX_ATTEMPTS", 10_000)
def worker(n: int):
for _ in range(25):
ratelimit.record_failure("10.4.4.4")
threads = [threading.Thread(target=worker, args=(n,)) for n in range(8)]
for t in threads:
t.start()
for t in threads:
t.join()
assert ratelimit.get_status("10.4.4.4")["failures"] == 200
+52
View File
@@ -0,0 +1,52 @@
"""Tests — cookies Secure, CORS same-origin implicite, avertissement bind (ROADMAP #87 T3).
- `is_secure_cookies()` suit `OBSIGATE_SECURE_COOKIES` (défaut `false` :
compatibilité logins en HTTP local — les navigateurs ignorent les cookies
`Secure` en clair).
- Aucun en-tête CORS permissif n'est émis : sans `CORSMiddleware`, les
navigateurs appliquent le same-origin par défaut (politique explicite par
l'absence — vérifiée ici pour qu'un ajout futur de CORS soit conscient).
"""
from __future__ import annotations
def test_secure_cookies_default_false(monkeypatch):
"""Défaut `false` (logins HTTP locaux préservés)."""
from backend.auth.router import is_secure_cookies
monkeypatch.delenv("OBSIGATE_SECURE_COOKIES", raising=False)
assert is_secure_cookies() is False
def test_secure_cookies_opt_in(monkeypatch):
"""`OBSIGATE_SECURE_COOKIES=true` → flag actif (insensible à la casse)."""
from backend.auth.router import is_secure_cookies
for value in ("true", "True", "TRUE", "1", "yes"):
monkeypatch.setenv("OBSIGATE_SECURE_COOKIES", value)
assert is_secure_cookies() is (value.lower() == "true")
def test_no_cors_headers_on_api(client):
"""Pas de `Access-Control-Allow-Origin` : same-origin imposé par le navigateur."""
resp = client.get("/api/health")
assert resp.status_code == 200
assert "access-control-allow-origin" not in {k.lower() for k in resp.headers}
def test_no_cors_headers_on_public_share(client):
"""Idem sur la page publique de partage."""
resp = client.get("/s/jeton-inexistant")
assert resp.status_code == 404
assert "access-control-allow-origin" not in {k.lower() for k in resp.headers}
def test_security_headers_present(client):
"""En-têtes de durcissement posés par le middleware (non-régression)."""
resp = client.get("/api/health")
assert resp.headers.get("x-content-type-options") == "nosniff"
assert resp.headers.get("x-frame-options") == "SAMEORIGIN"
csp = resp.headers.get("content-security-policy", "")
assert "object-src 'none'" in csp
assert "frame-ancestors 'self'" in csp
+123
View File
@@ -0,0 +1,123 @@
"""Tests de non-régression — verrous des stores JSON (ROADMAP #85 T10a).
Sans verrou, les read-modify-write concurrents (créations de partages,
révocations de jetons) perdent des mises à jour : deux threads lisent le
même état, chacun écrit le sien, la première écriture est écrasée. Ces
tests martèlent les stores depuis plusieurs threads et exigent un compte
exact à la fin (aucune mise à jour perdue).
"""
from __future__ import annotations
import threading
N_THREADS = 8
N_OPS = 25
def test_concurrent_share_creations_lose_nothing(tmp_path, monkeypatch):
"""N threads × N partages → le store final contient tout."""
from backend import share as share_mod
monkeypatch.setattr(share_mod, "SHARES_FILE", tmp_path / "shares.json")
errors: list[BaseException] = []
def worker(n: int):
try:
for i in range(N_OPS):
share_mod.create_share("V", f"doc-{n}-{i}.md", "tester")
except BaseException as e: # pragma: no cover - diagnostic
errors.append(e)
threads = [threading.Thread(target=worker, args=(n,)) for n in range(N_THREADS)]
for t in threads:
t.start()
for t in threads:
t.join()
assert not errors
assert len(share_mod.list_shares()) == N_THREADS * N_OPS
def test_concurrent_share_access_and_revoke(tmp_path, monkeypatch):
"""Accès + révocations concurrents : compteurs et suppressions cohérents."""
from backend import share as share_mod
monkeypatch.setattr(share_mod, "SHARES_FILE", tmp_path / "shares.json")
tokens = [share_mod.create_share("V", f"doc-{i}.md", "tester")["token"] for i in range(20)]
def worker(n: int):
share_mod.record_access(tokens[2 * n])
share_mod.record_access(tokens[2 * n + 1])
share_mod.revoke_share(tokens[2 * n])
threads = [threading.Thread(target=worker, args=(n,)) for n in range(10)]
for t in threads:
t.start()
for t in threads:
t.join()
# Les 10 tokens pairs ont été révoqués ; les impairs subsistent avec
# leurs compteurs d'accès (aucune écriture perdue).
remaining = {s["token"] for s in share_mod.list_shares()}
assert len(remaining) == 10
assert all(share_mod.get_share_by_token(t)["access_count"] >= 1 for t in remaining)
def test_concurrent_token_revokes_lose_nothing(tmp_path, monkeypatch):
"""N threads × N révocations → toutes les JTIs sont persistées."""
import json
from backend.auth import jwt_handler
revoked_file = tmp_path / "revoked_tokens.json"
monkeypatch.setattr(jwt_handler, "REVOKED_TOKENS_FILE", revoked_file)
monkeypatch.setattr(jwt_handler, "_revoked_map", {})
monkeypatch.setattr(jwt_handler, "_revoked_loaded", True)
errors: list[BaseException] = []
def worker(n: int):
try:
for i in range(N_OPS):
jwt_handler.revoke_token(f"jti-{n}-{i}")
except BaseException as e: # pragma: no cover - diagnostic
errors.append(e)
threads = [threading.Thread(target=worker, args=(n,)) for n in range(N_THREADS)]
for t in threads:
t.start()
for t in threads:
t.join()
assert not errors
assert len(jwt_handler._revoked_map) == N_THREADS * N_OPS
# Le fichier reflète l'état mémoire (aucune écriture perdue).
on_disk = json.loads(revoked_file.read_text(encoding="utf-8"))
assert len(on_disk) == N_THREADS * N_OPS
assert jwt_handler.is_token_revoked("jti-0-0")
assert not jwt_handler.is_token_revoked("jti-absent")
def test_concurrent_webhook_and_tool_key_writes(tmp_path, monkeypatch):
"""Créations de webhooks + clés d'outils concurrentes : rien de perdu."""
from backend import webhooks as wh_mod
from backend.tools import secrets as sec_mod
monkeypatch.setattr(wh_mod, "WEBHOOKS_FILE", tmp_path / "webhooks.json")
monkeypatch.setattr(wh_mod, "WEBHOOK_SECRETS_FILE", tmp_path / "webhook_secrets.json")
monkeypatch.setattr(sec_mod, "_keys_file", lambda: tmp_path / "api_keys.json")
def worker(n: int):
for i in range(N_OPS):
wh_mod.create_webhook(f"hook-{n}-{i}", "https://example.com/hook", ["file_created"])
sec_mod.set_tool_key("OBSIGATE_GITHUB_TOKEN", f"tok-{n}-{i}")
threads = [threading.Thread(target=worker, args=(n,)) for n in range(N_THREADS)]
for t in threads:
t.start()
for t in threads:
t.join()
assert len(wh_mod.get_webhooks()) == N_THREADS * N_OPS