Compare commits

..
7 Commits
37 changed files with 1742 additions and 502 deletions
+5
View File
@@ -13,6 +13,9 @@ OBSIGATE_ADMIN_PASSWORD=chab30
# OBSIGATE_ALLOW_INSECURE=false # OBSIGATE_ALLOW_INSECURE=false
# Sécurité des cookies (activer si derrière HTTPS) # Sécurité des cookies (activer si derrière HTTPS)
# false par défaut : les navigateurs ignorent les cookies `Secure` en HTTP,
# ce qui casserait les logins en local. En production (TLS + bind réseau),
# posez true — un avertissement est loggé au démarrage sinon (#87).
# OBSIGATE_SECURE_COOKIES=false # OBSIGATE_SECURE_COOKIES=false
# Tokens TTL en secondes # Tokens TTL en secondes
@@ -23,6 +26,8 @@ OBSIGATE_ADMIN_PASSWORD=chab30
# OBSIGATE_LOGIN_MAX_ATTEMPTS=10 # OBSIGATE_LOGIN_MAX_ATTEMPTS=10
# OBSIGATE_ACCOUNT_MAX_ATTEMPTS=10 # OBSIGATE_ACCOUNT_MAX_ATTEMPTS=10
# OBSIGATE_LOGIN_WINDOW_SECONDS=900 # OBSIGATE_LOGIN_WINDOW_SECONDS=900
# Compteurs partagés/persistants (SQLite WAL, multi-workers) — défaut : mémoire.
# OBSIGATE_RATELIMIT_DB=data/ratelimit.db
# IP client derrière un reverse proxy (fait confiance à X-Forwarded-For) # IP client derrière un reverse proxy (fait confiance à X-Forwarded-For)
# OBSIGATE_TRUST_PROXY=false # OBSIGATE_TRUST_PROXY=false
+18 -4
View File
@@ -44,6 +44,11 @@ jobs:
node tests/frontend/config-mobile.test.mjs node tests/frontend/config-mobile.test.mjs
node tests/frontend/settings-order-avatar.test.mjs node tests/frontend/settings-order-avatar.test.mjs
node tests/frontend/mobile-toolbar.test.mjs node tests/frontend/mobile-toolbar.test.mjs
node tests/frontend/upload.test.mjs
node tests/frontend/pretty.test.mjs
node tests/frontend/media-viewer.test.mjs
node tests/frontend/mfa-settings.test.mjs
node tests/frontend/config-ai-keys.test.mjs
- name: Frontend JSDOM tests (PaneManager + Excalidraw + Plugins + AI + SW + Collab + Mobile + Semantic + Desktop + Inline edition) - name: Frontend JSDOM tests (PaneManager + Excalidraw + Plugins + AI + SW + Collab + Mobile + Semantic + Desktop + Inline edition)
run: | run: |
@@ -126,11 +131,17 @@ jobs:
pip install bandit pip-audit pip install bandit pip-audit
pip install -r backend/requirements.txt pip install -r backend/requirements.txt
- name: Bandit (SAST) - name: Bandit (SAST, bloquant — #87)
run: bandit -r backend/ --skip B101,B110,B310 || echo "bandit found issues (non-blocking)" # B105 est exclu (aligné avec [tool.bandit] de pyproject.toml :
# faux positifs systématiques sur les noms de variables) ; les rares
# vrais positifs restants portent un `# nosec` justifié inline.
run: bandit -r backend/ --skip B101,B105,B110,B310
- name: Pip-audit (dependency vulnerabilities) - name: Pip-audit (consultatif — #87)
run: pip-audit || echo "pip-audit found vulnerabilities (non-blocking)" # Reste non bloquant tant que les montées de version requises
# (starlette via fastapi, weasyprint) ne sont pas qualifiées :
# upgrade FastAPI = chantier de régression dédié, hors périmètre.
run: pip-audit || echo "pip-audit found vulnerabilities (non-blocking, see #87)"
# ── Docker build ────────────────────────────────────────────────── # ── Docker build ──────────────────────────────────────────────────
build: build:
@@ -192,6 +203,9 @@ jobs:
npm ci npm ci
npx playwright install --with-deps chromium npx playwright install --with-deps chromium
- name: Npm audit (bloquant — #87, 0 dépendance prod hors Playwright)
run: npm audit --omit=dev
- name: Start ObsiGate - name: Start ObsiGate
run: | run: |
docker rm -f obsigate-e2e 2>/dev/null || true docker rm -f obsigate-e2e 2>/dev/null || true
+79 -1
View File
@@ -6,7 +6,7 @@ Format basé sur [Keep a Changelog](https://keepachangelog.com/fr/1.1.0/),
et [Semantic Versioning](https://semver.org/spec/v2.0.0.html). et [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
> **En cours de développement** : les changements à venir sont listés dans la section > **En cours de développement** : les changements à venir sont listés dans la section
> [Unreleased](#unreleased). La dernière version livrée est **2.27.11**. > [Unreleased](#unreleased). La dernière version livrée est **2.28.5**.
--- ---
@@ -14,6 +14,67 @@ et [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
--- ---
## [2.28.5] — 2026-09-26
---
## [2.28.4] — 2026-09-26
---
## [2.28.3] — 2026-09-26
---
## [2.28.2] — 2026-09-26
### Ajouté
- **#87 (T4) — E2E XSS et serveur E2E piloté.**
`tests/e2e/xss.spec.js` : page publique `/s/{token}` (titre/frontmatter
hostile échappé, JSON neutralisé, aucun JS exécuté) et lecteur markdown
(sanitizer, aucun `on*`/`javascript:` vivant) — 2/2 verts en local.
`scripts/e2e-server.ps1` : `start|stop|status|logs` avec progression
visible et fichier PID (fini les serveurs orphelins sur le port 2029).
- **#87 (T3) — cookies `Secure` et CORS explicites.**
Helper `is_secure_cookies()` centralisé (défaut `false` conservé pour ne
pas casser les logins HTTP locaux) + avertissement au démarrage sur bind
non-loopback sans `Secure` ; `tests/test_security_headers.py` atteste
l'absence de CORS permissif (same-origin par défaut du navigateur) et les
en-têtes de durcissement.
- **#87 (T2) — tests de durcissement : concurrence et regex.**
`tests/test_hardening_concurrency.py` : créations/mises à jour/`login
failures` concurrents sur `users.json` (zéro mise à jour perdue, JSON
toujours récupérable) et budget temps de la politique ReDoS (motifs
catastrophiques rejetés en < 1 s, motifs acceptés < 5 s sur 200 Ko).
---
## [2.28.1] — 2026-09-26
### Modifié
- **#87 (T1) — CI sécurité durcie.**
`bandit` passe en bloquant (`# nosec` justifiés : SHA1 non-crypto,
subprocess git à argv fixe, `saxutils.escape` sans parsing ; B105 exclu
comme `pyproject.toml`) ; `npm audit --omit=dev` bloquant (0 faille) ;
les 5 suites frontend hors CI (`upload`, `pretty`, `media-viewer`,
`mfa-settings`, `config-ai-keys`, vertes en local) rejoignent le job
`lint`. `pip-audit` reste consultatif (upgrades starlette/weasyprint à
qualifier, chantier dédié).
---
## [2.28.0] — 2026-09-26
---
## [2.27.12] — 2026-09-26
---
## [2.27.11] — 2026-09-26 ## [2.27.11] — 2026-09-26
--- ---
@@ -54,6 +115,23 @@ et [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
### Modifié ### Modifié
- **#85 (T10) — persistance d'état et clôture de la refonte architecturale.**
Verrous `RLock` sur les stores JSON sans protection (`revoked_tokens`,
`shares`, `webhooks` + secrets, clés d'outils) avec tests de concurrence
(`tests/test_store_locks.py` — pertes prouvées sans verrou) ; rate-limit
auth persisté en option (`OBSIGATE_RATELIMIT_DB`, SQLite WAL, sémantique
identique, défaut mémoire inchangé, `tests/test_ratelimit_store.py`).
Contrat `tools/registry.py` audité (permissions/quotas/redaction déjà
câblés, rien à coder). Index non persisté : rebuild différentiel #86
suffisant (décision documentée). Fiche `docs/features/archi-refonte-85.md`,
#85 sorti du backlog (index roadmap).
- **#85 (T9) — extraction realtime + render hors du monolithe `backend/main.py`.**
Le stream SSE `/api/events` et le WebSocket `/ws/collab/*` sont servis par
`backend/routers/realtime.py`, le pipeline markdown (mistune, wikilinks,
slugs, sanitizer) par `backend/render.py` (imports directs, plus de
couplage différé). `main.py` (4 827 → ~760 lignes) ne contient plus que
l'assemblage : lifespan, middlewares, montage des 16 routers, racine
`/api`, statique/SPA et cales de compatibilité testées.
- **#85 (T8) — extraction vaults/history/conflicts hors du monolithe `backend/main.py`.** - **#85 (T8) — extraction vaults/history/conflicts hors du monolithe `backend/main.py`.**
13 routes servies par `backend/routers/vaults.py`, `history.py` et 13 routes servies par `backend/routers/vaults.py`, `history.py` et
`conflicts.py` ; `VaultInfo`/`BookmarkToggleRequest` dans `schemas.py`, `conflicts.py` ; `VaultInfo`/`BookmarkToggleRequest` dans `schemas.py`,
+3 -3
View File
@@ -4,7 +4,7 @@
**Porte d'entrée web ultra-léger pour vos vaults Obsidian** — Accédez, naviguez et recherchez dans toutes vos notes Obsidian depuis n'importe quel appareil via une interface web moderne et responsive. **Porte d'entrée web ultra-léger pour vos vaults Obsidian** — Accédez, naviguez et recherchez dans toutes vos notes Obsidian depuis n'importe quel appareil via une interface web moderne et responsive.
[![Version](https://img.shields.io/badge/Version-2.27.11-blue.svg)]() [![Version](https://img.shields.io/badge/Version-2.28.5-blue.svg)]()
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Docker](https://img.shields.io/badge/Docker-Ready-blue.svg)](https://www.docker.com/) [![Docker](https://img.shields.io/badge/Docker-Ready-blue.svg)](https://www.docker.com/)
[![Python](https://img.shields.io/badge/Python-3.11+-green.svg)](https://www.python.org/) [![Python](https://img.shields.io/badge/Python-3.11+-green.svg)](https://www.python.org/)
@@ -976,8 +976,8 @@ Ce projet est sous licence **MIT** — voir le fichier [LICENSE](LICENSE) pour l
## 📝 Changelog ## 📝 Changelog
Consultez le [CHANGELOG.md](./CHANGELOG.md) pour l'historique complet de toutes les versions (v1.0.0 → v2.27.11). Consultez le [CHANGELOG.md](./CHANGELOG.md) pour l'historique complet de toutes les versions (v1.0.0 → v2.28.5).
--- ---
*Projet : ObsiGate | Version : 2.27.11 | Dernière mise à jour : Septembre 2026* *Projet : ObsiGate | Version : 2.28.5 | Dernière mise à jour : Septembre 2026*
+3 -3
View File
@@ -2,7 +2,7 @@
**Ultra-light web gateway for your Obsidian vaults** — Access, browse, and search all your Obsidian notes from any device via a modern, responsive web interface. **Ultra-light web gateway for your Obsidian vaults** — Access, browse, and search all your Obsidian notes from any device via a modern, responsive web interface.
[![Version](https://img.shields.io/badge/Version-2.27.11-blue.svg)]() [![Version](https://img.shields.io/badge/Version-2.28.5-blue.svg)]()
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Docker](https://img.shields.io/badge/Docker-Ready-blue.svg)](https://www.docker.com/) [![Docker](https://img.shields.io/badge/Docker-Ready-blue.svg)](https://www.docker.com/)
[![Python](https://img.shields.io/badge/Python-3.11+-green.svg)](https://www.python.org/) [![Python](https://img.shields.io/badge/Python-3.11+-green.svg)](https://www.python.org/)
@@ -1151,8 +1151,8 @@ This project is licensed under the **MIT License** - see the [LICENSE](LICENSE)
## 📝 Changelog ## 📝 Changelog
See [CHANGELOG.md](./CHANGELOG.md) for the complete version history (v1.0.0 → v2.27.11). See [CHANGELOG.md](./CHANGELOG.md) for the complete version history (v1.0.0 → v2.28.5).
--- ---
*Project: ObsiGate | Version: 2.27.11 | Last updated: September 2026* *Project: ObsiGate | Version: 2.28.5 | Last updated: September 2026*
+1 -1
View File
@@ -1 +1 @@
2.27.11 2.28.5
+33 -26
View File
@@ -119,25 +119,30 @@ def decode_token(token: str) -> dict | None:
_revoked_map: dict[str, int] = {} _revoked_map: dict[str, int] = {}
_revoked_loaded = False _revoked_loaded = False
# ROADMAP #85 T10a — verrou autour du read-modify-write du store de
# révocation (perte de révocations en cas de logouts concurrents).
_revoked_lock = threading.RLock()
def _load_revoked(): def _load_revoked():
"""Load revoked token JTIs from disk into memory (once).""" """Load revoked token JTIs from disk into memory (once)."""
global _revoked_loaded, _revoked_map global _revoked_loaded, _revoked_map
if _revoked_loaded: with _revoked_lock:
return if _revoked_loaded:
if REVOKED_TOKENS_FILE.exists(): return
try: if REVOKED_TOKENS_FILE.exists():
data = json.loads(REVOKED_TOKENS_FILE.read_text()) try:
# Drop entries whose underlying token has itself expired. data = json.loads(REVOKED_TOKENS_FILE.read_text())
now = int(time.time()) # Drop entries whose underlying token has itself expired.
_revoked_map = { now = int(time.time())
jti: int(exp) for jti, exp in data.items() _revoked_map = {
if int(exp) > now jti: int(exp) for jti, exp in data.items()
} if int(exp) > now
except Exception as e: }
logger.warning(f"Failed to load revoked tokens: {e}") except Exception as e:
_revoked_map = {} logger.warning(f"Failed to load revoked tokens: {e}")
_revoked_loaded = True _revoked_map = {}
_revoked_loaded = True
def _save_revoked(): def _save_revoked():
@@ -154,24 +159,26 @@ def revoke_token(jti: str, expires_at: int | None = None):
``expires_at`` is the revoked token's own ``exp`` (unix seconds) — the ``expires_at`` is the revoked token's own ``exp`` (unix seconds) — the
record is kept at least that long so a long-lived API token cannot record is kept at least that long so a long-lived API token cannot
outlive its revocation. ``None`` means the token never expires (API/MCP outlive its revocation. ``None`` means the token never expires (API/MCP
"sans fin") → the record is kept forever (capped at ~100 years, the JWT "sans fin") → the record is kept forever (capped at ~100 years, the JWT
store's practical infinity). Default keeps 7 days (session tokens). store's practical infinity). Default keeps 7 days (session tokens).
""" """
_load_revoked() with _revoked_lock:
now = int(time.time()) _load_revoked()
if expires_at is None: now = int(time.time())
until = now + 100 * 365 * 24 * 3600 if expires_at is None:
else: until = now + 100 * 365 * 24 * 3600
until = max(int(expires_at), now + REFRESH_TOKEN_EXPIRE_SECONDS) else:
_revoked_map[jti] = until until = max(int(expires_at), now + REFRESH_TOKEN_EXPIRE_SECONDS)
_save_revoked() _revoked_map[jti] = until
_save_revoked()
logger.debug(f"Revoked token JTI: {jti[:8]}...") logger.debug(f"Revoked token JTI: {jti[:8]}...")
def is_token_revoked(jti: str) -> bool: def is_token_revoked(jti: str) -> bool:
"""Check if a token JTI has been revoked.""" """Check if a token JTI has been revoked."""
_load_revoked() with _revoked_lock:
return jti in _revoked_map _load_revoked()
return jti in _revoked_map
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
+18 -7
View File
@@ -5,6 +5,7 @@
import base64 import base64
import binascii import binascii
import logging import logging
import os
import re import re
from fastapi import APIRouter, Body, Depends, HTTPException, Request, Response from fastapi import APIRouter, Body, Depends, HTTPException, Request, Response
@@ -56,6 +57,17 @@ logger = logging.getLogger("obsigate.auth.router")
router = APIRouter(prefix="/api/auth", tags=["auth"]) router = APIRouter(prefix="/api/auth", tags=["auth"])
def is_secure_cookies() -> bool:
"""True when auth cookies must carry the ``Secure`` flag (#87 T3).
Opt-in via ``OBSIGATE_SECURE_COOKIES=true`` (required behind TLS).
Default stays ``false`` so logins keep working over plain HTTP on
trusted loopback deployments — browsers drop ``Secure`` cookies sent
over HTTP, which would silently break localhost logins.
"""
return os.environ.get("OBSIGATE_SECURE_COOKIES", "false").lower() == "true"
# ── Pydantic request models ────────────────────────────────────────── # ── Pydantic request models ──────────────────────────────────────────
class LoginRequest(BaseModel): class LoginRequest(BaseModel):
@@ -229,9 +241,8 @@ def _issue_tokens(user: dict, username: str, remember_me: bool, response: Respon
access_token = create_access_token(user) access_token = create_access_token(user)
refresh_token, refresh_jti = create_refresh_token(username, remember=remember_me) refresh_token, refresh_jti = create_refresh_token(username, remember=remember_me)
import os
max_age = 2592000 if remember_me else 604800 # 30d or 7d max_age = 2592000 if remember_me else 604800 # 30d or 7d
secure = os.environ.get("OBSIGATE_SECURE_COOKIES", "false").lower() == "true" secure = is_secure_cookies()
response.set_cookie( response.set_cookie(
key="refresh_token", key="refresh_token",
value=refresh_token, value=refresh_token,
@@ -253,7 +264,8 @@ def _issue_tokens(user: dict, username: str, remember_me: bool, response: Respon
) )
return { return {
"access_token": access_token, "access_token": access_token,
"token_type": "bearer", # nosec B105 — OAuth2 token_type, pas un mot de passe # OAuth2 token_type, pas un mot de passe (B105) :
"token_type": "bearer", # nosec B105
"expires_in": ACCESS_TOKEN_EXPIRE_SECONDS, "expires_in": ACCESS_TOKEN_EXPIRE_SECONDS,
"user": { "user": {
"username": user["username"], "username": user["username"],
@@ -299,9 +311,7 @@ async def refresh_token_endpoint(request: Request, response: Response):
if stale: if stale:
raise HTTPException(401, "Session expirée, veuillez vous reconnecter") raise HTTPException(401, "Session expirée, veuillez vous reconnecter")
import os secure = is_secure_cookies()
secure = os.environ.get("OBSIGATE_SECURE_COOKIES", "false").lower() == "true"
remember_me = bool(payload.get("remember", False)) remember_me = bool(payload.get("remember", False))
# BUG-027: rotate the refresh token — the old one is now single-use. # BUG-027: rotate the refresh token — the old one is now single-use.
@@ -332,7 +342,8 @@ async def refresh_token_endpoint(request: Request, response: Response):
return { return {
"access_token": new_access_token, "access_token": new_access_token,
"token_type": "bearer", # nosec B105 — OAuth2 token_type, pas un mot de passe # OAuth2 token_type, pas un mot de passe (B105) :
"token_type": "bearer", # nosec B105
"expires_in": ACCESS_TOKEN_EXPIRE_SECONDS, "expires_in": ACCESS_TOKEN_EXPIRE_SECONDS,
} }
+2 -1
View File
@@ -35,7 +35,8 @@ def diagram_png_for(code: str) -> Path | None:
Mermaid, ou None. Le hash doit rester synchrone avec le script de build : Mermaid, ou None. Le hash doit rester synchrone avec le script de build :
sha1(unescape(code).strip())[:16].""" sha1(unescape(code).strip())[:16]."""
normalized = html.unescape(code).strip() normalized = html.unescape(code).strip()
sha = hashlib.sha1(normalized.encode("utf-8")).hexdigest()[:16] # Identifiant de cache déterministe (pas un usage sécurité).
sha = hashlib.sha1(normalized.encode("utf-8")).hexdigest()[:16] # nosec B324
png = DIAGRAMS_DIR / (sha + ".png") png = DIAGRAMS_DIR / (sha + ".png")
return png if png.exists() else None return png if png.exists() else None
+20 -305
View File
@@ -1,27 +1,19 @@
import asyncio import asyncio
import html as html_mod
import json as _json
import logging import logging
import os import os
import re
import secrets import secrets
import string import string
from contextlib import asynccontextmanager from contextlib import asynccontextmanager
from pathlib import Path from pathlib import Path
import mistune from fastapi import Depends, FastAPI, HTTPException, Request
from fastapi import Depends, FastAPI, HTTPException, Request, WebSocket from fastapi.responses import FileResponse, HTMLResponse, JSONResponse
from fastapi.responses import FileResponse, HTMLResponse, JSONResponse, StreamingResponse
from fastapi.staticfiles import StaticFiles from fastapi.staticfiles import StaticFiles
from pydantic import BaseModel, Field
from starlette.middleware.base import BaseHTTPMiddleware from starlette.middleware.base import BaseHTTPMiddleware
from backend.collab import authenticate_websocket, collab_manager from backend.collab import collab_manager
from backend.image_processor import preprocess_images
from backend.indexer import ( from backend.indexer import (
build_index, build_index,
find_file_in_index,
get_vault_data,
handle_file_move, handle_file_move,
remove_single_file, remove_single_file,
update_single_file, update_single_file,
@@ -38,7 +30,6 @@ from backend.search import (
from backend.semantic_search import init_semantic_index from backend.semantic_search import init_semantic_index
from backend.services.backups import get_backup_dir as service_get_backup_dir from backend.services.backups import get_backup_dir as service_get_backup_dir
from backend.services.errors import ServiceError from backend.services.errors import ServiceError
from backend.services.sanitizer import sanitize_html
logging.basicConfig( logging.basicConfig(
level=logging.INFO, level=logging.INFO,
@@ -48,33 +39,10 @@ logger = logging.getLogger("obsigate")
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Pydantic response models : voir backend.schemas (vaults/history : #85 T8) # Pydantic models : voir backend.schemas (#85 T5→T9)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Filesystem mutation + search / suggest / graph models : voir backend.schemas (#85 T5, T6b)
class BackupEntry(BaseModel):
"""A single backup version of a file."""
timestamp: int = Field(description="Unix timestamp of when the backup was created")
datetime: str = Field(description="ISO 8601 datetime string")
size: int = Field(description="File size in bytes")
filename: str = Field(description="Backup filename on disk")
class BackupListResponse(BaseModel):
"""Response listing all available backups for a file."""
vault: str = Field(description="Vault name")
path: str = Field(description="Relative file path")
backups: list[BackupEntry] = Field(description="Available backups, newest first")
class DiffRequest(BaseModel):
"""Request parameters for generating a diff."""
version: int = Field(description="Timestamp of the backup version to compare")
compare_with: int | None = Field(default=None, description="Timestamp of another backup version. If omitted, compares with the current file.")
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# SSE Manager — voir backend.sse (ROADMAP #85 T4, instance partagée) # SSE Manager — voir backend.sse (ROADMAP #85 T4, instance partagée)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -281,6 +249,17 @@ async def lifespan(app: FastAPI):
# BUG-037: refuse to expose an unauthenticated instance on a public bind. # BUG-037: refuse to expose an unauthenticated instance on a public bind.
_guard_insecure_auth() _guard_insecure_auth()
# #87 T3 : avertir quand les cookies d'auth circulent sans flag Secure
# sur un bind non-loopback (transactions observables en clair).
from backend.auth.middleware import bind_host_from_argv, is_auth_enabled, is_loopback_host
from backend.auth.router import is_secure_cookies
if is_auth_enabled() and not is_secure_cookies() and not is_loopback_host(bind_host_from_argv()):
logger.warning(
"Cookies d'authentification sans flag `Secure` sur un bind non-loopback : "
"activez TLS et posez OBSIGATE_SECURE_COOKIES=true en production."
)
# Bootstrap admin account if needed # Bootstrap admin account if needed
bootstrap_admin() bootstrap_admin()
@@ -432,9 +411,7 @@ app.add_middleware(SecurityHeadersMiddleware)
# Multi-format export (HTML / MD bundle / ePub) — voir backend.routers.files_media (#85 T6c). # Multi-format export (HTML / MD bundle / ePub) — voir backend.routers.files_media (#85 T6c).
from backend.ai_routes import router as ai_router from backend.ai_routes import router as ai_router
from backend.auth.middleware import ( from backend.auth.middleware import (
check_vault_access,
require_admin, require_admin,
require_auth,
) )
from backend.auth.router import router as auth_router from backend.auth.router import router as auth_router
from backend.bookslm_routes import router as bookslm_router from backend.bookslm_routes import router as bookslm_router
@@ -447,11 +424,11 @@ from backend.routers.files_read import router as files_read_router
from backend.routers.files_write import router as files_write_router from backend.routers.files_write import router as files_write_router
from backend.routers.health import router as health_router from backend.routers.health import router as health_router
from backend.routers.history import router as history_router from backend.routers.history import router as history_router
from backend.routers.realtime import router as realtime_router
from backend.routers.search import router as search_router from backend.routers.search import router as search_router
from backend.routers.sharing import router as sharing_router from backend.routers.sharing import router as sharing_router
from backend.routers.vaults import router as vaults_router from backend.routers.vaults import router as vaults_router
from backend.routers.webhooks import router as webhooks_router from backend.routers.webhooks import router as webhooks_router
from backend.secret_redactor import redact_file_content
from backend.skills_routes import router as skills_router from backend.skills_routes import router as skills_router
app.include_router(auth_router) app.include_router(auth_router)
@@ -460,6 +437,7 @@ app.include_router(bookslm_router)
app.include_router(skills_router) app.include_router(skills_router)
app.include_router(health_router) # ROADMAP #85 T1 — System / health app.include_router(health_router) # ROADMAP #85 T1 — System / health
app.include_router(history_router) # ROADMAP #85 T8 — History app.include_router(history_router) # ROADMAP #85 T8 — History
app.include_router(realtime_router) # ROADMAP #85 T9 — SSE + collab WS
app.include_router(search_router) # ROADMAP #85 T5 — Search app.include_router(search_router) # ROADMAP #85 T5 — Search
app.include_router(backups_router) # ROADMAP #85 T4 — Backups app.include_router(backups_router) # ROADMAP #85 T4 — Backups
app.include_router(conflicts_router) # ROADMAP #85 T8 — Conflicts app.include_router(conflicts_router) # ROADMAP #85 T8 — Conflicts
@@ -569,193 +547,9 @@ def _check_vault_writable(vault_root: Path) -> bool:
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Markdown rendering helpers (singleton renderer) # Markdown rendering helpers : voir backend.render (#85 T9)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
import unicodedata
def _heading_slugify(text: str) -> str:
"""Generate a URL-safe slug from heading text.
Matches the JavaScript slugify algorithm exactly using
Unicode-aware character classification:
1. Strip HTML tags (e.g. wikilink spans rendered inside headings)
2. Decode HTML entities (e.g. ``&amp;`` → ``&``)
3. Lowercase
4. NFD normalize + strip combining marks
5. Keep only Unicode letters, numbers, spaces, hyphens
6. Replace spaces with hyphens, collapse multiple hyphens
Args:
text: The heading text content (may contain inline HTML).
Returns:
A URL-safe slug string.
"""
# Strip any inline HTML so it does not pollute the slug
text = re.sub(r"<[^>]+>", "", text)
# Decode HTML entities so &amp; becomes & before slugification
text = html_mod.unescape(text)
text = text.lower()
text = unicodedata.normalize("NFD", text)
text = "".join(ch for ch in text if not unicodedata.combining(ch))
# Unicode-aware: keep letters (L*), numbers (N*), spaces, and hyphens
cleaned = []
for ch in text:
cat = unicodedata.category(ch)
if cat.startswith('L') or cat.startswith('N') or ch in (' ', '-'):
cleaned.append(ch)
text = "".join(cleaned)
text = re.sub(r"\s+", "-", text)
text = re.sub(r"-+", "-", text)
result = text.strip("-")
return result if result else "heading"
def _add_heading_ids(html: str) -> str:
"""Post-process rendered HTML to add IDs to heading tags.
Adds an ``id`` attribute to every ``<h1>`` through ``<h6>`` tag
using a slug generated from the heading's text content.
Duplicate slugs get a ``-2``, ``-3``, etc. suffix.
Args:
html: Rendered HTML string.
Returns:
HTML with heading IDs injected.
"""
used_ids: dict[str, int] = {}
def _replace_heading(match):
tag = match.group(1)
content = match.group(2)
slug = _heading_slugify(content)
count = used_ids.get(slug, 0)
used_ids[slug] = count + 1
if count > 0:
slug = f"{slug}-{count + 1}"
return f'<{tag} id="{slug}">{content}</{tag}>'
# Match h1-h6 tags with text content (no existing id attribute)
return re.sub(
r'<(h[1-6])>([^<]*(?:<(?!/?h[1-6])[^<]*)*)</h[1-6]>',
_replace_heading,
html,
)
# Cached mistune renderer — avoids re-creating on every request
_markdown_renderer = mistune.create_markdown(
escape=False,
plugins=["table", "strikethrough", "footnotes", "task_lists"],
)
def _convert_wikilinks(content: str, current_vault: str) -> str:
"""Convert ``[[wikilinks]]`` and ``[[target|display]]`` to clickable HTML.
Supports:
- Internal file links: ``[[My Note]]`` / ``[[My Note|display]]``
- Same-document anchors: ``[[#Heading]]`` / ``[[#Heading|display]]``
Resolved file links get a ``data-vault`` / ``data-path`` attribute pair.
Anchor links target the slugified heading ID in the current document.
Unresolved links are rendered as ``<span class="wikilink-missing">``.
Args:
content: Markdown string potentially containing wikilinks.
current_vault: Active vault name for resolution priority.
Returns:
Markdown string with wikilinks replaced by HTML anchors.
"""
def _replace(match):
target = match.group(1).strip()
display = match.group(2).strip() if match.group(2) else target
# Same-document anchor link: [[#Heading|display]]
if target.startswith("#"):
anchor_text = target[1:].strip()
anchor_slug = _heading_slugify(anchor_text)
link_display = display if display != target else anchor_text
return f'<a class="wikilink-anchor" href="#{anchor_slug}">{link_display}</a>'
found = find_file_in_index(target, current_vault)
if found:
return (
f'<a class="wikilink" href="#" '
f'data-vault="{found["vault"]}" '
f'data-path="{found["path"]}">{display}</a>'
)
return f'<span class="wikilink-missing">{display}</span>'
pattern = r'\[\[([^\]|]+)(?:\|([^\]]+))?\]\]'
return re.sub(pattern, _replace, content)
def _normalize_line_breaks(text: str) -> str:
"""Convert single newlines to hard breaks (matching Obsidian default behavior).
In standard Markdown, a single ``\\n`` is a "soft break" — it renders as a space,
not a visible line break. Obsidian defaults to treating single newlines as hard
breaks (equivalent to ``<br>``). This function pre-processes the Markdown source
so that mistune renders standalone lines on separate rows, while still honouring
blank lines as paragraph separators.
Fenced code blocks (`` ``` ``) are left untouched so their internal newlines are
preserved verbatim.
"""
parts = re.split(r"(```[\s\S]*?```)", text)
for i, part in enumerate(parts):
if part.startswith("```"):
continue # Protect fenced code blocks
# Single \n (not preceded or followed by another \n) → two spaces + \n
parts[i] = re.sub(r"(?<!\n)\n(?!\n)", " \n", part)
return "".join(parts)
def _render_markdown(raw_md: str, vault_name: str, current_file_path: Path | None = None) -> str:
"""Render a markdown string to HTML with wikilink and image support.
Uses the cached singleton mistune renderer for performance.
Args:
raw_md: Raw markdown text (frontmatter already stripped).
vault_name: Current vault for wikilink resolution context.
current_file_path: Absolute path to the current markdown file.
Returns:
HTML string.
"""
# Get vault data for image resolution
vault_data = get_vault_data(vault_name)
vault_root = Path(vault_data["path"]) if vault_data else None
attachments_path = vault_data.get("config", {}).get("attachmentsPath") if vault_data else None
# Redact secrets before rendering (P0 security)
raw_md = redact_file_content(raw_md, str(current_file_path) if current_file_path else "")
# Preprocess images first
if vault_root:
raw_md = preprocess_images(raw_md, vault_name, vault_root, current_file_path, attachments_path)
# Convert wikilinks
converted = _convert_wikilinks(raw_md, vault_name)
# Normalize line breaks to match Obsidian behavior (single \n → hard break)
converted = _normalize_line_breaks(converted)
rendered = _markdown_renderer(converted)
# Add heading IDs for TOC navigation
rendered = _add_heading_ids(rendered)
# Sanitize: raw HTML in vault content must never reach the DOM (BUG-021).
rendered = sanitize_html(rendered)
return rendered
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -826,51 +620,9 @@ def _get_backup_dir(vault_name: str, relative_path: str) -> Path:
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# SSE endpoint — Server-Sent Events stream # SSE endpoint : voir backend.routers.realtime (#85 T9)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@app.get(
"/api/events",
response_class=StreamingResponse,
responses={200: {"content": {"text/event-stream": {}}, "description": "Server-Sent Events stream"}},
)
async def api_events(current_user=Depends(require_auth)):
"""SSE stream for real-time index update notifications.
Sends keepalive comments every 30s. Events:
- ``index_updated``: partial index change (file create/modify/delete/move)
- ``index_reloaded``: full re-index completed
- ``vault_added``: new vault added dynamically
- ``vault_removed``: vault removed dynamically
"""
queue = await sse_manager.connect()
async def event_generator():
try:
# Send initial connection event
yield f"event: connected\ndata: {_json.dumps({'sse_clients': sse_manager.client_count})}\n\n"
while True:
try:
msg = await asyncio.wait_for(queue.get(), timeout=30.0)
yield f"event: {msg['event']}\ndata: {msg['data']}\n\n"
except asyncio.TimeoutError:
# Keepalive comment
yield ": keepalive\n\n"
except asyncio.CancelledError:
break
finally:
sse_manager.disconnect(queue)
return StreamingResponse(
event_generator(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
},
)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Dynamic vault management endpoints : voir backend.routers.vaults (#85 T8) # Dynamic vault management endpoints : voir backend.routers.vaults (#85 T8)
@@ -940,46 +692,9 @@ async def api_events(current_user=Depends(require_auth)):
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Real-time collaboration — WebSocket endpoint (ROADMAP #62) # Real-time collaboration — WebSocket endpoint : voir backend.routers.realtime (#85 T9, ROADMAP #62)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@app.websocket("/ws/collab/{vault_name}/{path:path}")
async def collab_websocket(websocket: WebSocket, vault_name: str, path: str):
"""Real-time collaborative editing over WebSocket (ROADMAP #62).
One *room* is created per ``vault::path``; all clients editing the same
file share Yjs/CRDT updates, awareness (cursors/selection) and a debounced
server-side persistence of the markdown content.
Authentication is performed manually (FastAPI ``Depends`` do not run for
WebSocket routes) and vault access is enforced per connection.
"""
from backend.services.errors import ServiceError
from backend.services.vaults import get_vault_root
user = authenticate_websocket(websocket)
if user is None:
await websocket.close(code=4401)
return
if not check_vault_access(vault_name, user):
await websocket.close(code=4403)
return
try:
vault_root = get_vault_root(vault_name)
file_path = _resolve_safe_path(vault_root, path)
except ServiceError:
await websocket.close(code=4404)
return
if not file_path.exists() or not file_path.is_file():
await websocket.close(code=4404)
return
await websocket.accept()
await collab_manager.connect(websocket, vault_name, path, file_path, user)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Static files & SPA fallback # Static files & SPA fallback
+2 -1
View File
@@ -32,7 +32,8 @@ def thumb_cache_path(file_path: Path, size: int) -> Path:
stamp = f"{st.st_mtime_ns}:{st.st_size}" stamp = f"{st.st_mtime_ns}:{st.st_size}"
except OSError: except OSError:
stamp = "0:0" stamp = "0:0"
key = hashlib.sha1(f"{file_path}:{stamp}:{size}".encode()).hexdigest() # Clé de cache miniature (pas un usage sécurité).
key = hashlib.sha1(f"{file_path}:{stamp}:{size}".encode()).hexdigest() # nosec B324
return thumbs_cache_dir() / f"{key}.webp" return thumbs_cache_dir() / f"{key}.webp"
+172 -1
View File
@@ -12,14 +12,24 @@ the per-account lockout in ``user_store.py``.
deployment, front this service with a shared store (Redis) or a single deployment, front this service with a shared store (Redis) or a single
worker. This limitation is intentional and documented (BUG-031). worker. This limitation is intentional and documented (BUG-031).
Opt-in persistence (ROADMAP #85 T10b) : if ``OBSIGATE_RATELIMIT_DB`` points
to a SQLite file, counters are stored there instead (WAL mode, one short
connection per call — safe across threads, processes and restarts sharing
the same file). Semantics (windows, budgets, success reset) are identical
to the in-memory store, which remains the default when the variable is
unset.
Configuration via environment variables: Configuration via environment variables:
OBSIGATE_LOGIN_MAX_ATTEMPTS Max failures per IP (default: 10) OBSIGATE_LOGIN_MAX_ATTEMPTS Max failures per IP (default: 10)
OBSIGATE_ACCOUNT_MAX_ATTEMPTS Max failures per account (default: 10) OBSIGATE_ACCOUNT_MAX_ATTEMPTS Max failures per account (default: 10)
OBSIGATE_LOGIN_WINDOW_SECONDS Lockout window in seconds (default: 900) OBSIGATE_LOGIN_WINDOW_SECONDS Lockout window in seconds (default: 900)
OBSIGATE_RATELIMIT_DB SQLite file for shared/persistent counters (default: unset = memory)
""" """
import logging import logging
import os import os
import sqlite3
import threading
import time import time
from collections import defaultdict from collections import defaultdict
@@ -37,6 +47,127 @@ _last_cleanup = time.time()
CLEANUP_INTERVAL = 60 # seconds CLEANUP_INTERVAL = 60 # seconds
def _db_path() -> str | None:
"""SQLite file for shared counters, or ``None`` for the in-memory store."""
path = os.environ.get("OBSIGATE_RATELIMIT_DB", "").strip()
return path or None
def _db_connect(path: str) -> sqlite3.Connection:
"""Open a short-lived connection (WAL + busy timeout for concurrent workers)."""
_db_ensure_schema(path)
conn = sqlite3.connect(path, timeout=10.0)
conn.execute("PRAGMA busy_timeout=10000")
return conn
_schema_ready: set[str] = set()
_schema_lock = threading.Lock()
def _db_ensure_schema(path: str) -> None:
"""Create the store schema once per file (DDL under a process-wide lock)."""
with _schema_lock:
if path in _schema_ready:
return
conn = sqlite3.connect(path, timeout=10.0)
try:
conn.execute("PRAGMA journal_mode=WAL")
conn.execute(
"CREATE TABLE IF NOT EXISTS attempts"
" (kind TEXT NOT NULL, key TEXT NOT NULL, ts REAL NOT NULL, success INTEGER NOT NULL)"
)
conn.execute(
"CREATE INDEX IF NOT EXISTS idx_attempts_kind_key_ts"
" ON attempts (kind, key, ts)"
)
conn.commit()
finally:
conn.close()
_schema_ready.add(path)
def _db_write(fn, *args):
"""Run a write op, retrying once on lock contention (concurrent workers)."""
try:
return fn(*args)
except sqlite3.OperationalError as e:
if "locked" not in str(e).lower():
raise
time.sleep(0.05)
return fn(*args)
def _db_prune(conn: sqlite3.Connection, cutoff: float) -> None:
"""Drop expired entries (best-effort cap on disk growth)."""
conn.execute("DELETE FROM attempts WHERE ts <= ?", (cutoff,))
def _db_record(kind: str, key: str, success: bool) -> int:
"""Record one attempt in SQLite; return the live failure count."""
path = _db_path()
assert path is not None
now = time.time()
cutoff = now - WINDOW_SECONDS
def _write() -> int:
with _db_connect(path) as conn:
_db_prune(conn, cutoff)
if success:
# Mirror the in-memory reset: replace history with one success.
conn.execute("DELETE FROM attempts WHERE kind = ? AND key = ?", (kind, key))
conn.execute(
"INSERT INTO attempts (kind, key, ts, success) VALUES (?, ?, ?, ?)",
(kind, key, now, int(success)),
)
conn.commit()
(failures,) = conn.execute(
"SELECT COUNT(*) FROM attempts WHERE kind = ? AND key = ? AND ts > ? AND success = 0",
(kind, key, cutoff),
).fetchone()
return failures
return _db_write(_write)
def _db_failures(kind: str, key: str) -> int:
"""Live failure count in SQLite (expired entries never count)."""
path = _db_path()
assert path is not None
cutoff = time.time() - WINDOW_SECONDS
with _db_connect(path) as conn:
(failures,) = conn.execute(
"SELECT COUNT(*) FROM attempts WHERE kind = ? AND key = ? AND ts > ? AND success = 0",
(kind, key, cutoff),
).fetchone()
return failures
def _db_tracked(kind: str) -> int:
"""Number of distinct keys ever seen for one budget (SQLite)."""
path = _db_path()
assert path is not None
with _db_connect(path) as conn:
(n,) = conn.execute(
"SELECT COUNT(DISTINCT key) FROM attempts WHERE kind = ?", (kind,)
).fetchone()
return n
def _db_limited_count(kind: str, max_attempts: int) -> int:
"""Number of keys currently over budget (SQLite)."""
path = _db_path()
assert path is not None
cutoff = time.time() - WINDOW_SECONDS
with _db_connect(path) as conn:
rows = conn.execute(
"SELECT key, COUNT(*) FROM attempts"
" WHERE kind = ? AND ts > ? AND success = 0 GROUP BY key",
(kind, cutoff),
).fetchall()
return sum(1 for _, n in rows if n >= max_attempts)
def _prune(store: dict[str, list], cutoff: float) -> None: def _prune(store: dict[str, list], cutoff: float) -> None:
"""Drop expired entries from one store in place.""" """Drop expired entries from one store in place."""
expired = [] expired = []
@@ -66,6 +197,12 @@ def record_failure(ip: str) -> tuple[int, int]:
Returns: Returns:
(current_failure_count, remaining_attempts) (current_failure_count, remaining_attempts)
""" """
if _db_path() is not None:
failures = _db_record("ip", ip, False)
remaining = max(0, MAX_ATTEMPTS - failures)
if failures >= MAX_ATTEMPTS:
logger.warning(f"IP {ip} rate-limited after {failures} failed logins")
return failures, remaining
_cleanup_expired() _cleanup_expired()
_ip_attempts[ip].append((time.time(), False)) _ip_attempts[ip].append((time.time(), False))
failures = sum(1 for _, success in _ip_attempts[ip] if not success) failures = sum(1 for _, success in _ip_attempts[ip] if not success)
@@ -77,12 +214,17 @@ def record_failure(ip: str) -> tuple[int, int]:
def record_success(ip: str): def record_success(ip: str):
"""Clear rate limit state for an IP after successful login.""" """Clear rate limit state for an IP after successful login."""
if _db_path() is not None:
_db_record("ip", ip, True)
return
_cleanup_expired() _cleanup_expired()
_ip_attempts[ip] = [(time.time(), True)] _ip_attempts[ip] = [(time.time(), True)]
def is_rate_limited(ip: str) -> bool: def is_rate_limited(ip: str) -> bool:
"""Check if an IP has exceeded the rate limit.""" """Check if an IP has exceeded the rate limit."""
if _db_path() is not None:
return _db_failures("ip", ip) >= MAX_ATTEMPTS
_cleanup_expired() _cleanup_expired()
failures = sum(1 for _, success in _ip_attempts.get(ip, []) if not success) failures = sum(1 for _, success in _ip_attempts.get(ip, []) if not success)
return failures >= MAX_ATTEMPTS return failures >= MAX_ATTEMPTS
@@ -94,8 +236,14 @@ def record_account_failure(account: str) -> tuple[int, int]:
Returns: Returns:
(current_failure_count, remaining_attempts) (current_failure_count, remaining_attempts)
""" """
_cleanup_expired()
key = account.lower() key = account.lower()
if _db_path() is not None:
failures = _db_record("account", key, False)
remaining = max(0, ACCOUNT_MAX_ATTEMPTS - failures)
if failures >= ACCOUNT_MAX_ATTEMPTS:
logger.warning(f"Account {account} rate-limited after {failures} failed attempts")
return failures, remaining
_cleanup_expired()
_account_attempts[key].append((time.time(), False)) _account_attempts[key].append((time.time(), False))
failures = sum(1 for _, success in _account_attempts[key] if not success) failures = sum(1 for _, success in _account_attempts[key] if not success)
remaining = max(0, ACCOUNT_MAX_ATTEMPTS - failures) remaining = max(0, ACCOUNT_MAX_ATTEMPTS - failures)
@@ -106,12 +254,17 @@ def record_account_failure(account: str) -> tuple[int, int]:
def record_account_success(account: str): def record_account_success(account: str):
"""Clear the per-account rate limit state after a successful login.""" """Clear the per-account rate limit state after a successful login."""
if _db_path() is not None:
_db_record("account", account.lower(), True)
return
_cleanup_expired() _cleanup_expired()
_account_attempts[account.lower()] = [(time.time(), True)] _account_attempts[account.lower()] = [(time.time(), True)]
def is_account_rate_limited(account: str) -> bool: def is_account_rate_limited(account: str) -> bool:
"""Check if an account has exceeded the per-account rate limit.""" """Check if an account has exceeded the per-account rate limit."""
if _db_path() is not None:
return _db_failures("account", account.lower()) >= ACCOUNT_MAX_ATTEMPTS
_cleanup_expired() _cleanup_expired()
failures = sum( failures = sum(
1 for _, success in _account_attempts.get(account.lower(), []) if not success 1 for _, success in _account_attempts.get(account.lower(), []) if not success
@@ -121,6 +274,24 @@ def is_account_rate_limited(account: str) -> bool:
def get_status(ip: str | None = None) -> dict: def get_status(ip: str | None = None) -> dict:
"""Get rate limit status for an IP (for diagnostics).""" """Get rate limit status for an IP (for diagnostics)."""
if _db_path() is not None:
if ip:
failures = _db_failures("ip", ip)
return {
"ip": ip,
"failures": failures,
"max": MAX_ATTEMPTS,
"limited": failures >= MAX_ATTEMPTS,
"window_seconds": WINDOW_SECONDS,
}
return {
"tracked_ips": _db_tracked("ip"),
"tracked_accounts": _db_tracked("account"),
"max_attempts": MAX_ATTEMPTS,
"account_max_attempts": ACCOUNT_MAX_ATTEMPTS,
"window_seconds": WINDOW_SECONDS,
"limited_ips": _db_limited_count("ip", MAX_ATTEMPTS),
}
_cleanup_expired() _cleanup_expired()
if ip: if ip:
attempts = _ip_attempts.get(ip, []) attempts = _ip_attempts.get(ip, [])
+207
View File
@@ -0,0 +1,207 @@
"""Markdown rendering pipeline (ROADMAP #85, tranche 9).
Helpers extraits de :mod:`backend.main` sans changement de comportement :
slugification des headings, IDs d'ancrage, rendu mistune singleton,
wikilinks, normalisation des sauts de ligne et pipeline complet
:func:`_render_markdown` (rendu + sanitizer XSS BUG-021).
Les noms gardent leur préfixe ``_`` d'origine pour un déplacement
strictement verbatim (tests et routers pointent ici désormais).
"""
from __future__ import annotations
import html as html_mod
import re
import unicodedata
from pathlib import Path
import mistune
from backend.image_processor import preprocess_images
from backend.indexer import find_file_in_index, get_vault_data
from backend.secret_redactor import redact_file_content
from backend.services.sanitizer import sanitize_html
def _heading_slugify(text: str) -> str:
"""Generate a URL-safe slug from heading text.
Matches the JavaScript slugify algorithm exactly using
Unicode-aware character classification:
1. Strip HTML tags (e.g. wikilink spans rendered inside headings)
2. Decode HTML entities (e.g. ``&amp;`` → ``&``)
3. Lowercase
4. NFD normalize + strip combining marks
5. Keep only Unicode letters, numbers, spaces, hyphens
6. Replace spaces with hyphens, collapse multiple hyphens
Args:
text: The heading text content (may contain inline HTML).
Returns:
A URL-safe slug string.
"""
# Strip any inline HTML so it does not pollute the slug
text = re.sub(r"<[^>]+>", "", text)
# Decode HTML entities so &amp; becomes & before slugification
text = html_mod.unescape(text)
text = text.lower()
text = unicodedata.normalize("NFD", text)
text = "".join(ch for ch in text if not unicodedata.combining(ch))
# Unicode-aware: keep letters (L*), numbers (N*), spaces, and hyphens
cleaned = []
for ch in text:
cat = unicodedata.category(ch)
if cat.startswith('L') or cat.startswith('N') or ch in (' ', '-'):
cleaned.append(ch)
text = "".join(cleaned)
text = re.sub(r"\s+", "-", text)
text = re.sub(r"-+", "-", text)
result = text.strip("-")
return result if result else "heading"
def _add_heading_ids(html: str) -> str:
"""Post-process rendered HTML to add IDs to heading tags.
Adds an ``id`` attribute to every ``<h1>`` through ``<h6>`` tag
using a slug generated from the heading's text content.
Duplicate slugs get a ``-2``, ``-3``, etc. suffix.
Args:
html: Rendered HTML string.
Returns:
HTML with heading IDs injected.
"""
used_ids: dict[str, int] = {}
def _replace_heading(match):
tag = match.group(1)
content = match.group(2)
slug = _heading_slugify(content)
count = used_ids.get(slug, 0)
used_ids[slug] = count + 1
if count > 0:
slug = f"{slug}-{count + 1}"
return f'<{tag} id="{slug}">{content}</{tag}>'
# Match h1-h6 tags with text content (no existing id attribute)
return re.sub(
r'<(h[1-6])>([^<]*(?:<(?!/?h[1-6])[^<]*)*)</h[1-6]>',
_replace_heading,
html,
)
# Cached mistune renderer — avoids re-creating on every request
_markdown_renderer = mistune.create_markdown(
escape=False,
plugins=["table", "strikethrough", "footnotes", "task_lists"],
)
def _convert_wikilinks(content: str, current_vault: str) -> str:
"""Convert ``[[wikilinks]]`` and ``[[target|display]]`` to clickable HTML.
Supports:
- Internal file links: ``[[My Note]]`` / ``[[My Note|display]]``
- Same-document anchors: ``[[#Heading]]`` / ``[[#Heading|display]]``
Resolved file links get a ``data-vault`` / ``data-path`` attribute pair.
Anchor links target the slugified heading ID in the current document.
Unresolved links are rendered as ``<span class="wikilink-missing">``.
Args:
content: Markdown string potentially containing wikilinks.
current_vault: Active vault name for resolution priority.
Returns:
Markdown string with wikilinks replaced by HTML anchors.
"""
def _replace(match):
target = match.group(1).strip()
display = match.group(2).strip() if match.group(2) else target
# Same-document anchor link: [[#Heading|display]]
if target.startswith("#"):
anchor_text = target[1:].strip()
anchor_slug = _heading_slugify(anchor_text)
link_display = display if display != target else anchor_text
return f'<a class="wikilink-anchor" href="#{anchor_slug}">{link_display}</a>'
found = find_file_in_index(target, current_vault)
if found:
return (
f'<a class="wikilink" href="#" '
f'data-vault="{found["vault"]}" '
f'data-path="{found["path"]}">{display}</a>'
)
return f'<span class="wikilink-missing">{display}</span>'
pattern = r'\[\[([^\]|]+)(?:\|([^\]]+))?\]\]'
return re.sub(pattern, _replace, content)
def _normalize_line_breaks(text: str) -> str:
"""Convert single newlines to hard breaks (matching Obsidian default behavior).
In standard Markdown, a single ``\\n`` is a "soft break" — it renders as a space,
not a visible line break. Obsidian defaults to treating single newlines as hard
breaks (equivalent to ``<br>``). This function pre-processes the Markdown source
so that mistune renders standalone lines on separate rows, while still honouring
blank lines as paragraph separators.
Fenced code blocks (`` ``` ``) are left untouched so their internal newlines are
preserved verbatim.
"""
parts = re.split(r"(```[\s\S]*?```)", text)
for i, part in enumerate(parts):
if part.startswith("```"):
continue # Protect fenced code blocks
# Single \n (not preceded or followed by another \n) → two spaces + \n
parts[i] = re.sub(r"(?<!\n)\n(?!\n)", " \n", part)
return "".join(parts)
def _render_markdown(raw_md: str, vault_name: str, current_file_path: Path | None = None) -> str:
"""Render a markdown string to HTML with wikilink and image support.
Uses the cached singleton mistune renderer for performance.
Args:
raw_md: Raw markdown text (frontmatter already stripped).
vault_name: Current vault for wikilink resolution context.
current_file_path: Absolute path to the current markdown file.
Returns:
HTML string.
"""
# Get vault data for image resolution
vault_data = get_vault_data(vault_name)
vault_root = Path(vault_data["path"]) if vault_data else None
attachments_path = vault_data.get("config", {}).get("attachmentsPath") if vault_data else None
# Redact secrets before rendering (P0 security)
raw_md = redact_file_content(raw_md, str(current_file_path) if current_file_path else "")
# Preprocess images first
if vault_root:
raw_md = preprocess_images(raw_md, vault_name, vault_root, current_file_path, attachments_path)
# Convert wikilinks
converted = _convert_wikilinks(raw_md, vault_name)
# Normalize line breaks to match Obsidian behavior (single \n → hard break)
converted = _normalize_line_breaks(converted)
rendered = _markdown_renderer(converted)
# Add heading IDs for TOC navigation
rendered = _add_heading_ids(rendered)
# Sanitize: raw HTML in vault content must never reach the DOM (BUG-021).
rendered = sanitize_html(rendered)
return rendered
+3 -3
View File
@@ -9,7 +9,8 @@ d'authentification.
Adaptations strictement équivalentes : Adaptations strictement équivalentes :
- ``_resolve_safe_path`` → :mod:`backend.services.paths` (pass-through). - ``_resolve_safe_path`` → :mod:`backend.services.paths` (pass-through).
- ``_render_markdown`` reste dans ``main`` (import différé). - ``_render_markdown`` vient de :mod:`backend.render` (#85 T9, sans cycle
d'import).
- ``_resolve_export_target`` / ``_safe_export_name`` (export uniquement) - ``_resolve_export_target`` / ``_safe_export_name`` (export uniquement)
sont définis ici ; ``stream_file_with_range`` vit dans sont définis ici ; ``stream_file_with_range`` vit dans
:mod:`backend.routers.helpers` (partagé). :mod:`backend.routers.helpers` (partagé).
@@ -29,6 +30,7 @@ from backend.history import record_open
from backend.indexer import get_vault_data, index, parse_markdown_file from backend.indexer import get_vault_data, index, parse_markdown_file
from backend.media_thumbs import generate_thumbnail, is_decodable from backend.media_thumbs import generate_thumbnail, is_decodable
from backend.media_types import is_audio, is_image, is_video, media_mime_type from backend.media_types import is_audio, is_image, is_video, media_mime_type
from backend.render import _render_markdown
from backend.routers.helpers import media_max_inline_bytes, stream_file_with_range from backend.routers.helpers import media_max_inline_bytes, stream_file_with_range
from backend.schemas import ( from backend.schemas import (
AllVaultSettingsResponse, AllVaultSettingsResponse,
@@ -85,8 +87,6 @@ def _safe_export_name(name: str) -> str:
) )
async def api_file_pdf(vault_name: str, path: str = Query(..., description="Relative path to file"), current_user=Depends(require_auth)): async def api_file_pdf(vault_name: str, path: str = Query(..., description="Relative path to file"), current_user=Depends(require_auth)):
"""Download a markdown file as PDF.""" """Download a markdown file as PDF."""
from backend.main import _render_markdown # différé : évite l'import circulaire (#85)
if generate_pdf is None: if generate_pdf is None:
raise HTTPException(501, "PDF export unavailable (WeasyPrint/GTK not available)") raise HTTPException(501, "PDF export unavailable (WeasyPrint/GTK not available)")
if not check_vault_access(vault_name, current_user): if not check_vault_access(vault_name, current_user):
+3 -4
View File
@@ -7,8 +7,8 @@ lecture), mêmes modèles de réponse (déménagés dans
Adaptations strictement équivalentes : Adaptations strictement équivalentes :
- ``_resolve_safe_path`` → :mod:`backend.services.paths` (pass-through). - ``_resolve_safe_path`` → :mod:`backend.services.paths` (pass-through).
- ``_render_markdown`` reste dans ``main`` (import différé, extraction - ``_render_markdown`` vient de :mod:`backend.render` (#85 T9, sans cycle
prévue dans une tranche ultérieure). d'import).
- ``_content_disposition`` / ``_media_max_inline_bytes`` / ``EXT_TO_LANG`` - ``_content_disposition`` / ``_media_max_inline_bytes`` / ``EXT_TO_LANG``
ont déménagé : helpers partagés dans :mod:`backend.routers.helpers` ont déménagé : helpers partagés dans :mod:`backend.routers.helpers`
(``EXT_TO_LANG`` n'était utilisé que par la vue fichier). (``EXT_TO_LANG`` n'était utilisé que par la vue fichier).
@@ -31,6 +31,7 @@ from backend.indexer import (
parse_markdown_file, parse_markdown_file,
) )
from backend.media_types import is_audio, is_image, is_video, media_mime_type from backend.media_types import is_audio, is_image, is_video, media_mime_type
from backend.render import _render_markdown
from backend.routers.helpers import media_max_inline_bytes from backend.routers.helpers import media_max_inline_bytes
from backend.schemas import ( from backend.schemas import (
BacklinksResponse, BacklinksResponse,
@@ -192,8 +193,6 @@ async def api_file(vault_name: str, path: str = Query(..., description="Relative
Returns: Returns:
``FileContentResponse`` with HTML, metadata, and tags. ``FileContentResponse`` with HTML, metadata, and tags.
""" """
from backend.main import _render_markdown # différé : évite l'import circulaire (#85)
if not check_vault_access(vault_name, current_user): if not check_vault_access(vault_name, current_user):
raise HTTPException(status_code=403, detail=f"Accès refusé à la vault '{vault_name}'") raise HTTPException(status_code=403, detail=f"Accès refusé à la vault '{vault_name}'")
vault_data = get_vault_data(vault_name) vault_data = get_vault_data(vault_name)
+105
View File
@@ -0,0 +1,105 @@
"""Real-time endpoints — SSE stream & collaboration WebSocket (ROADMAP #85, tranche 9).
Handlers déplacés depuis :mod:`backend.main` sans changement de
comportement : mêmes chemins (``/api/events``,
``/ws/collab/{vault}/{path}``), même authentification (Depend pour le SSE,
manuelle pour le WebSocket — les ``Depends`` FastAPI ne s'exécutent pas sur
les routes WebSocket).
Pas de tags déclarés : assignation par chemin via
``openapi_docs.tag_for_path`` comme avant (``/api/events`` → System).
"""
import asyncio
import json as _json
from fastapi import APIRouter, Depends, WebSocket
from fastapi.responses import StreamingResponse
from backend.auth.middleware import check_vault_access, require_auth
from backend.collab import authenticate_websocket, collab_manager
from backend.services.paths import resolve_safe_path
from backend.services.vaults import get_vault_root
from backend.sse import sse_manager
router = APIRouter()
@router.get(
"/api/events",
response_class=StreamingResponse,
responses={200: {"content": {"text/event-stream": {}}, "description": "Server-Sent Events stream"}},
)
async def api_events(current_user=Depends(require_auth)):
"""SSE stream for real-time index update notifications.
Sends keepalive comments every 30s. Events:
- ``index_updated``: partial index change (file create/modify/delete/move)
- ``index_reloaded``: full re-index completed
- ``vault_added``: new vault added dynamically
- ``vault_removed``: vault removed dynamically
"""
queue = await sse_manager.connect()
async def event_generator():
try:
# Send initial connection event
yield f"event: connected\ndata: {_json.dumps({'sse_clients': sse_manager.client_count})}\n\n"
while True:
try:
msg = await asyncio.wait_for(queue.get(), timeout=30.0)
yield f"event: {msg['event']}\ndata: {msg['data']}\n\n"
except asyncio.TimeoutError:
# Keepalive comment
yield ": keepalive\n\n"
except asyncio.CancelledError:
break
finally:
sse_manager.disconnect(queue)
return StreamingResponse(
event_generator(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
},
)
@router.websocket("/ws/collab/{vault_name}/{path:path}")
async def collab_websocket(websocket: WebSocket, vault_name: str, path: str):
"""Real-time collaborative editing over WebSocket (ROADMAP #62).
One *room* is created per ``vault::path``; all clients editing the same
file share Yjs/CRDT updates, awareness (cursors/selection) and a debounced
server-side persistence of the markdown content.
Authentication is performed manually (FastAPI ``Depends`` do not run for
WebSocket routes) and vault access is enforced per connection.
"""
from backend.services.errors import ServiceError
user = authenticate_websocket(websocket)
if user is None:
await websocket.close(code=4401)
return
if not check_vault_access(vault_name, user):
await websocket.close(code=4403)
return
try:
vault_root = get_vault_root(vault_name)
file_path = resolve_safe_path(vault_root, path)
except ServiceError:
await websocket.close(code=4404)
return
if not file_path.exists() or not file_path.is_file():
await websocket.close(code=4404)
return
await websocket.accept()
await collab_manager.connect(websocket, vault_name, path, file_path, user)
+3 -7
View File
@@ -11,9 +11,8 @@ Adaptations strictement équivalentes (pas de changement de comportement) :
wrappers directs : appelés ici via :mod:`backend.services.paths` et wrappers directs : appelés ici via :mod:`backend.services.paths` et
:mod:`backend.services.backups` (mêmes signatures, mêmes exceptions :mod:`backend.services.backups` (mêmes signatures, mêmes exceptions
``ServiceError`` toujours mappées par le handler global de ``main``). ``ServiceError`` toujours mappées par le handler global de ``main``).
- ``_render_markdown`` reste défini dans ``main`` (extraction prévue dans - ``_render_markdown`` vient de :mod:`backend.render` (#85 T9, sans cycle
une tranche ultérieure) : import différé à l'intérieur des handlers, donc d'import).
sans import circulaire au chargement.
""" """
import html as html_mod import html as html_mod
@@ -27,6 +26,7 @@ from fastapi.responses import FileResponse, HTMLResponse, Response
from backend.auth.middleware import check_vault_access, require_auth from backend.auth.middleware import check_vault_access, require_auth
from backend.indexer import get_vault_data, parse_markdown_file, update_single_file from backend.indexer import get_vault_data, parse_markdown_file, update_single_file
from backend.render import _render_markdown
from backend.schemas import ShareModel, StatusResponse from backend.schemas import ShareModel, StatusResponse
from backend.secret_redactor import redact_file_content from backend.secret_redactor import redact_file_content
from backend.services.backups import create_backup from backend.services.backups import create_backup
@@ -115,8 +115,6 @@ async def api_share_revoke(share_id: str, current_user=Depends(require_auth)):
) )
async def public_share_pdf_download(token: str): async def public_share_pdf_download(token: str):
"""Download shared document as real PDF via WeasyPrint.""" """Download shared document as real PDF via WeasyPrint."""
from backend.main import _render_markdown # différé : évite l'import circulaire (#85)
if generate_pdf is None: if generate_pdf is None:
raise HTTPException(501, "PDF export unavailable (WeasyPrint/GTK not available)") raise HTTPException(501, "PDF export unavailable (WeasyPrint/GTK not available)")
share = get_share_by_token(token) share = get_share_by_token(token)
@@ -168,8 +166,6 @@ async def public_share_raw(token: str):
@router.get("/s/{token}", response_class=HTMLResponse) @router.get("/s/{token}", response_class=HTMLResponse)
async def public_share_view(token: str): async def public_share_view(token: str):
"""Public share view — no authentication required.""" """Public share view — no authentication required."""
from backend.main import _render_markdown # différé : évite l'import circulaire (#85)
share = get_share_by_token(token) share = get_share_by_token(token)
if not share: if not share:
raise HTTPException(404, "Share not found or expired") raise HTTPException(404, "Share not found or expired")
+24
View File
@@ -214,6 +214,30 @@ class RestoreResponse(BaseModel):
current_backed_up: int | None = Field(default=None, description="Timestamp of the backup created from the current version before restore, if any") current_backed_up: int | None = Field(default=None, description="Timestamp of the backup created from the current version before restore, if any")
class BackupEntry(BaseModel):
"""A single backup version of a file (#85 — extrait de backend.main, inchangé)."""
timestamp: int = Field(description="Unix timestamp of when the backup was created")
datetime: str = Field(description="ISO 8601 datetime string")
size: int = Field(description="File size in bytes")
filename: str = Field(description="Backup filename on disk")
class BackupListResponse(BaseModel):
"""Response listing all available backups for a file (#85 — extrait de backend.main, inchangé)."""
vault: str = Field(description="Vault name")
path: str = Field(description="Relative file path")
backups: list[BackupEntry] = Field(description="Available backups, newest first")
class DiffRequest(BaseModel):
"""Request parameters for generating a diff (#85 — extrait de backend.main, inchangé)."""
version: int = Field(description="Timestamp of the backup version to compare")
compare_with: int | None = Field(default=None, description="Timestamp of another backup version. If omitted, compares with the current file.")
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Files — browse / read (#85 — extrait de backend.main, inchangé) # Files — browse / read (#85 — extrait de backend.main, inchangé)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
+48 -39
View File
@@ -10,6 +10,7 @@ No authentication required for public share views.
import json import json
import logging import logging
import secrets import secrets
import threading
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from pathlib import Path from pathlib import Path
@@ -17,6 +18,10 @@ logger = logging.getLogger("obsigate.share")
SHARES_FILE = Path("data/shares.json") SHARES_FILE = Path("data/shares.json")
# ROADMAP #85 T10a — verrou autour des read-modify-write (perte de mises à
# jour en cas de créations/accès/révocations concurrents).
_lock = threading.RLock()
def _read() -> dict: def _read() -> dict:
if not SHARES_FILE.exists(): if not SHARES_FILE.exists():
@@ -41,26 +46,27 @@ def create_share(
expires_in_hours: int | None = None, expires_in_hours: int | None = None,
) -> dict: ) -> dict:
"""Create a new share token for a document.""" """Create a new share token for a document."""
data = _read() with _lock:
token = secrets.token_hex(32) # 64-char hex token data = _read()
token = secrets.token_hex(32) # 64-char hex token
expires_at = None expires_at = None
if expires_in_hours: if expires_in_hours:
expires_at = (datetime.now(timezone.utc) + timedelta(hours=expires_in_hours)).isoformat() expires_at = (datetime.now(timezone.utc) + timedelta(hours=expires_in_hours)).isoformat()
share = { share = {
"id": token, "id": token,
"token": token, "token": token,
"vault": vault, "vault": vault,
"path": path, "path": path,
"created_by": created_by, "created_by": created_by,
"created_at": datetime.now(timezone.utc).isoformat(), "created_at": datetime.now(timezone.utc).isoformat(),
"expires_at": expires_at, "expires_at": expires_at,
"access_count": 0, "access_count": 0,
"last_accessed": None, "last_accessed": None,
} }
data["shares"][token] = share data["shares"][token] = share
_write(data) _write(data)
logger.info(f"Created share for {vault}/{path} by {created_by}") logger.info(f"Created share for {vault}/{path} by {created_by}")
return share return share
@@ -80,22 +86,24 @@ def get_share_by_token(token: str) -> dict | None:
def record_access(token: str): def record_access(token: str):
"""Increment access counter for a share.""" """Increment access counter for a share."""
data = _read() with _lock:
share = data["shares"].get(token) data = _read()
if share: share = data["shares"].get(token)
share["access_count"] = share.get("access_count", 0) + 1 if share:
share["last_accessed"] = datetime.now(timezone.utc).isoformat() share["access_count"] = share.get("access_count", 0) + 1
_write(data) share["last_accessed"] = datetime.now(timezone.utc).isoformat()
_write(data)
def revoke_share(share_id: str) -> bool: def revoke_share(share_id: str) -> bool:
"""Revoke (delete) a share by its token.""" """Revoke (delete) a share by its token."""
data = _read() with _lock:
if share_id in data["shares"]: data = _read()
del data["shares"][share_id] if share_id in data["shares"]:
_write(data) del data["shares"][share_id]
logger.info(f"Revoked share {share_id}") _write(data)
return True logger.info(f"Revoked share {share_id}")
return True
return False return False
@@ -112,12 +120,13 @@ def list_shares(vault_filter: str | None = None) -> list:
def update_shares_after_rename(vault: str, old_path: str, new_path: str): def update_shares_after_rename(vault: str, old_path: str, new_path: str):
"""Update all shares when a file is renamed.""" """Update all shares when a file is renamed."""
data = _read() with _lock:
updated = False data = _read()
for sid, s in data["shares"].items(): updated = False
if s.get("vault") == vault and s.get("path") == old_path: for sid, s in data["shares"].items():
s["path"] = new_path if s.get("vault") == vault and s.get("path") == old_path:
updated = True s["path"] = new_path
logger.info(f"Updated share {sid}: {vault}/{old_path} -> {new_path}") updated = True
if updated: logger.info(f"Updated share {sid}: {vault}/{old_path} -> {new_path}")
_write(data) if updated:
_write(data)
+3 -1
View File
@@ -19,7 +19,9 @@ import io
import logging import logging
import re import re
from typing import Any from typing import Any
from xml.sax import saxutils
# saxutils.escape uniquement (échappement de chaînes, aucun parsing XML).
from xml.sax import saxutils # nosec B406
from backend.services.errors import ServiceError from backend.services.errors import ServiceError
from backend.services.mutations import save_raw_file from backend.services.mutations import save_raw_file
+17 -11
View File
@@ -17,6 +17,7 @@ from __future__ import annotations
import json import json
import logging import logging
import os import os
import threading
from pathlib import Path from pathlib import Path
logger = logging.getLogger("obsigate.tools.secrets") logger = logging.getLogger("obsigate.tools.secrets")
@@ -34,6 +35,9 @@ TOOL_KEY_NAMES: tuple[str, ...] = (
_SECRET_MARKERS = ("API_KEY", "TOKEN") _SECRET_MARKERS = ("API_KEY", "TOKEN")
# ROADMAP #85 T10a — verrou autour des read-modify-write du store de clés.
_lock = threading.RLock()
def _keys_file() -> Path: def _keys_file() -> Path:
base = os.environ.get("OBSIGATE_DATA_DIR", "data") base = os.environ.get("OBSIGATE_DATA_DIR", "data")
@@ -89,21 +93,23 @@ def set_tool_key(name: str, value: str) -> None:
if name not in TOOL_KEY_NAMES: if name not in TOOL_KEY_NAMES:
raise ValueError(f"Clé non prise en charge: {name}") raise ValueError(f"Clé non prise en charge: {name}")
value = (value or "").strip() value = (value or "").strip()
keys = _read_keys() with _lock:
if value: keys = _read_keys()
keys[name] = value if value:
else: keys[name] = value
keys.pop(name, None) else:
_write_keys(keys) keys.pop(name, None)
_write_keys(keys)
def delete_tool_key(name: str) -> bool: def delete_tool_key(name: str) -> bool:
"""Remove one key from the store; return True when it existed.""" """Remove one key from the store; return True when it existed."""
if name not in TOOL_KEY_NAMES: if name not in TOOL_KEY_NAMES:
raise ValueError(f"Clé non prise en charge: {name}") raise ValueError(f"Clé non prise en charge: {name}")
keys = _read_keys() with _lock:
if name in keys: keys = _read_keys()
del keys[name] if name in keys:
_write_keys(keys) del keys[name]
return True _write_keys(keys)
return True
return False return False
+3 -2
View File
@@ -22,7 +22,7 @@ Exemples :
from __future__ import annotations from __future__ import annotations
import os import os
import subprocess import subprocess # nosec B404
from pathlib import Path from pathlib import Path
_ROOT = Path(__file__).resolve().parent.parent # racine du dépôt ObsiGate _ROOT = Path(__file__).resolve().parent.parent # racine du dépôt ObsiGate
@@ -34,7 +34,8 @@ _ENV_VAR = "OBSIGATE_VERSION"
def _run_git(args: list[str]) -> str: def _run_git(args: list[str]) -> str:
"""Run a git command in the repo root; return stdout (stripped) or ''.""" """Run a git command in the repo root; return stdout (stripped) or ''."""
try: try:
result = subprocess.run( # argv fixe (git + args internes), sans shell : pas d'injection.
result = subprocess.run( # nosec B404 B603 B607
["git", *args], ["git", *args],
cwd=str(_ROOT), cwd=str(_ROOT),
capture_output=True, capture_output=True,
+2 -1
View File
@@ -280,7 +280,8 @@ class VaultWatcher:
for observer in self.observers.values(): for observer in self.observers.values():
try: try:
observer.join(timeout=5) observer.join(timeout=5)
except Exception: # nosec B110 — best-effort shutdown, ignore failures # best-effort shutdown, ignore failures (B110) :
except Exception: # nosec B110
pass pass
self.observers.clear() self.observers.clear()
logger.info("VaultWatcher stopped") logger.info("VaultWatcher stopped")
+54 -43
View File
@@ -26,6 +26,7 @@ import json
import logging import logging
import os import os
import socket import socket
import threading
import uuid import uuid
from datetime import datetime, timezone from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
@@ -144,6 +145,12 @@ def _read_secrets() -> dict:
return {} return {}
# ROADMAP #85 T10a — verrou autour des read-modify-write des deux stores
# (webhooks + secrets) : perte de mises à jour en cas de mutations
# concurrentes.
_lock = threading.RLock()
def _write_secrets(secrets: dict): def _write_secrets(secrets: dict):
WEBHOOK_SECRETS_FILE.parent.mkdir(parents=True, exist_ok=True) WEBHOOK_SECRETS_FILE.parent.mkdir(parents=True, exist_ok=True)
tmp = WEBHOOK_SECRETS_FILE.with_suffix(".tmp") tmp = WEBHOOK_SECRETS_FILE.with_suffix(".tmp")
@@ -156,12 +163,13 @@ def _write_secrets(secrets: dict):
def _store_secret(wh_id: str, secret: str | None) -> None: def _store_secret(wh_id: str, secret: str | None) -> None:
secrets = _read_secrets() with _lock:
if secret: secrets = _read_secrets()
secrets[wh_id] = secret if secret:
else: secrets[wh_id] = secret
secrets.pop(wh_id, None) else:
_write_secrets(secrets) secrets.pop(wh_id, None)
_write_secrets(secrets)
def _get_secret(wh: dict) -> str | None: def _get_secret(wh: dict) -> str | None:
@@ -189,52 +197,55 @@ def get_webhooks() -> list:
def create_webhook(name: str, url: str, events: list[str], secret: str | None = None) -> dict: def create_webhook(name: str, url: str, events: list[str], secret: str | None = None) -> dict:
validate_webhook_url(url) validate_webhook_url(url)
webhooks = _read() with _lock:
wh_id = str(uuid.uuid4()) webhooks = _read()
wh = { wh_id = str(uuid.uuid4())
"id": wh_id, wh = {
"name": name, "id": wh_id,
"url": url, "name": name,
"events": [e for e in events if e in VALID_EVENTS], "url": url,
"enabled": True, "events": [e for e in events if e in VALID_EVENTS],
"created_at": datetime.now(timezone.utc).isoformat(), "enabled": True,
"last_fired_at": None, "created_at": datetime.now(timezone.utc).isoformat(),
} "last_fired_at": None,
webhooks.append(wh) }
_write(webhooks) webhooks.append(wh)
if secret: _write(webhooks)
_store_secret(wh_id, secret) if secret:
_store_secret(wh_id, secret)
logger.info(f"Created webhook '{name}' → {url}") logger.info(f"Created webhook '{name}' → {url}")
return _public_view(wh) return _public_view(wh)
def update_webhook(wh_id: str, updates: dict) -> dict | None: def update_webhook(wh_id: str, updates: dict) -> dict | None:
webhooks = _read() with _lock:
for wh in webhooks: webhooks = _read()
if wh["id"] == wh_id: for wh in webhooks:
if updates.get("url"): if wh["id"] == wh_id:
validate_webhook_url(updates["url"]) if updates.get("url"):
if "secret" in updates: validate_webhook_url(updates["url"])
_store_secret(wh_id, updates["secret"]) if "secret" in updates:
safe_updates = { _store_secret(wh_id, updates["secret"])
k: v for k, v in updates.items() safe_updates = {
if k not in ("id", "secret") k: v for k, v in updates.items()
} if k not in ("id", "secret")
wh.update(safe_updates) }
_write(webhooks) wh.update(safe_updates)
return _public_view(wh) _write(webhooks)
return _public_view(wh)
return None return None
def delete_webhook(wh_id: str) -> bool: def delete_webhook(wh_id: str) -> bool:
webhooks = _read() with _lock:
new_list = [wh for wh in webhooks if wh["id"] != wh_id] webhooks = _read()
if len(new_list) == len(webhooks): new_list = [wh for wh in webhooks if wh["id"] != wh_id]
return False if len(new_list) == len(webhooks):
_write(new_list) return False
secrets = _read_secrets() _write(new_list)
if secrets.pop(wh_id, None) is not None: secrets = _read_secrets()
_write_secrets(secrets) if secrets.pop(wh_id, None) is not None:
_write_secrets(secrets)
return True return True
+1 -1
View File
@@ -2626,7 +2626,7 @@ dependencies = [
[[package]] [[package]]
name = "obsigate-desktop" name = "obsigate-desktop"
version = "2.27.11" version = "2.28.5"
dependencies = [ dependencies = [
"chrono", "chrono",
"env_logger", "env_logger",
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "obsigate-desktop" name = "obsigate-desktop"
version = "2.27.11" version = "2.28.5"
description = "ObsiGate Desktop — Porte d'entrée native pour vos vaults Obsidian" description = "ObsiGate Desktop — Porte d'entrée native pour vos vaults Obsidian"
authors = ["Bruno Charest"] authors = ["Bruno Charest"]
edition = "2021" edition = "2021"
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"$schema": "https://raw.githubusercontent.com/nicedoc/obsigate/main/desktop/tauri.conf.schema.json", "$schema": "https://raw.githubusercontent.com/nicedoc/obsigate/main/desktop/tauri.conf.schema.json",
"productName": "ObsiGate", "productName": "ObsiGate",
"version": "2.27.11", "version": "2.28.5",
"identifier": "com.obsigate.desktop", "identifier": "com.obsigate.desktop",
"build": { "build": {
"frontendDist": "../frontend", "frontendDist": "../frontend",
+10 -20
View File
@@ -1,6 +1,6 @@
# ObsiGate — Roadmap # ObsiGate — Roadmap
> **Version :** 2.27.11 | **Dernière mise à jour :** 2026-09-26 > **Version :** 2.28.5 | **Dernière mise à jour :** 2026-09-26
> **Ce fichier ne contient que le travail à venir** (🔵 En cours + ⚪ Backlog) et un index compact > **Ce fichier ne contient que le travail à venir** (🔵 En cours + ⚪ Backlog) et un index compact
> vers les fonctionnalités livrées. > vers les fonctionnalités livrées.
> - **Méthode de livraison à appliquer pour toute tâche : [DELIVERY_WORKFLOW.md](./DELIVERY_WORKFLOW.md)** > - **Méthode de livraison à appliquer pour toute tâche : [DELIVERY_WORKFLOW.md](./DELIVERY_WORKFLOW.md)**
@@ -63,28 +63,16 @@
## ⚪ Backlog — Sécurité, architecture & performance (P0/P1) ## ⚪ Backlog — Sécurité, architecture & performance (P0/P1)
### 85. Refonte architecturale — découpage du monolithe & persistance d'état (phase 2)
- **Effort :** 8-12 jours | **Impact :** 🟡 | **Zone :** backend
- **Décision 2026-09-26 : prioritaire (axe Dette & sécurité).**
- **Statut :** 🔵 en cours depuis 2026-09-26 — découpe par tranches à impact minimal (comportement inchangé, un domaine par commit). **T1 livrée (v2.27.2) :** `health` (`/api/health`, `/api/health/detailed` → `backend/routers/health.py`, `HealthResponse` → `schemas.py`). **T2 livrée (v2.27.3) :** `webhooks` (CRUD `/api/webhooks` → `backend/routers/webhooks.py`, logique déjà dans `backend/webhooks.py`). **T3 livrée (v2.27.4) :** `sharing` (`/api/share/*`, `/api/shares`, `/s/{token}*` → `backend/routers/sharing.py`, logique déjà dans `backend/share.py`). **T4 livrée (v2.27.5) :** `backups` (9 routes `/api/file/{vault}/backups|diff|restore` + `/api/backups*` → `backend/routers/backups.py`, `Diff/Restore*` → `schemas.py`, singleton SSE → `backend/sse.py`). **T5 livrée (v2.27.6) :** `search` (11 routes search/tags/suggest/graph/reload → `backend/routers/search.py`, modèles search → `schemas.py`, pool threads → `backend/search_executor.py`). **T6a livrée (v2.27.7) :** lecture fichiers (`/api/browse`, `/raw`, `/download`, `/backlinks`, `GET /api/file` → `backend/routers/files_read.py`, modèles + `EXT_TO_LANG` déplacés, helpers `_content_disposition`/`_media_max_inline_bytes` → `backend/routers/helpers.py`). **T6b livrée (v2.27.8) :** mutations fichiers/dossiers (save, xlsx/save, delete, create, rename, move, directories ×3, batch-upload → `backend/routers/files_write.py`, 15 modèles → `schemas.py`). **T6c livrée (v2.27.9) :** media/pdf/export/guide (file/pdf, exports ×3, guide, pdf/stream|info, image, media+thumb, attachments ×2, vault settings ×3, vault files → `backend/routers/files_media.py`, Range helper → `helpers.py`). **T7 livrée (v2.27.10) :** config (app/ai-keys/tool-keys/ai-models/diagnostics/dashboard → `backend/routers/config.py`, `_FALLBACK_MODELS` + clés déplacés, `test_ai_models` réaligné). **T8 livrée (v2.27.11) :** vaults + history + conflicts (→ `backend/routers/vaults.py|history.py|conflicts.py`, `VaultInfo`/`BookmarkToggleRequest` → `schemas.py`, watcher → `backend/watcher_state.py`).
- **Description :** extraire le monolithe `backend/main.py` (~4 827 lignes au 2026-09-26, ~17 % du backend) en routers FastAPI par domaine et rendre persistant l'état qui ne l'est pas (index de recherche, JTI révoqués, compteurs de rate-limit) pour préparer le multi-nœuds. L'état mémoire actuel (index, inverted index, vecteurs sémantiques, `SSEManager`, collab) rend le multi-workers unsafe.
- **Sous-tâches :**
- [ ] Routers par domaine : files, search, share, webhooks, plugins, collab, admin, ai — `main.py` conservé comme assemblage (< 500 lignes) ; dédupliquer les modèles Pydantic vers `schemas.py`. **Avancement :** `health` ✅ (T1, `backend/routers/health.py`), `webhooks` ✅ (T2, `backend/routers/webhooks.py`), `sharing` ✅ (T3, `backend/routers/sharing.py`), `backups` ✅ (T4, `backend/routers/backups.py` + `backend/sse.py`), `search` ✅ (T5, `backend/routers/search.py` + `backend/search_executor.py`), `files-read` ✅ (T6a, `backend/routers/files_read.py` + `helpers.py`), `files-write` ✅ (T6b, `backend/routers/files_write.py`), `files-media` ✅ (T6c, `backend/routers/files_media.py`), `config` ✅ (T7, `backend/routers/config.py`), `vaults|history|conflicts` ✅ (T8, 3 routers + `watcher_state.py`) ; `tools/registry.py` existe déjà (permissions/quotas/redaction — à compléter, pas à créer)
- [ ] Compléter `tools/registry.py` (existant : permissions/quotas/redaction) comme contrat central des outils IA si des manques sont constatés
- [ ] Persister index, JTI révoqués et compteurs de rate-limit (SQLite par défaut, Redis en option multi-nœuds ; le rate-limit actuel est in-memory mono-process)
- [ ] Verrous asyncio autour de l'index global et des stores JSON ; auditer les `except Exception` larges (> 100 occurrences) : best-effort (backup/audit) vs masquage d'erreur (erreurs typées 4xx/5xx + test)
- [ ] Extraire le service de partage public (expiration, révocation, quotas)
### 87. Amélioration continue — tests, CI/CD, revues de sécurité (phase 4) ### 87. Amélioration continue — tests, CI/CD, revues de sécurité (phase 4)
- **Effort :** 3-5 jours | **Impact :** 🟡 | **Zone :** `.gitea/workflows/`, `tests/` - **Effort :** 3-5 jours | **Impact :** 🟡 | **Zone :** `.gitea/workflows/`, `tests/`
- **Décision 2026-09-26 : prioritaire (axe Dette & sécurité).** - **Décision 2026-09-26 : prioritaire (axe Dette & sécurité).**
- **Statut :** 🔵 en cours depuis 2026-09-26 — par tranches. **T1 livrée (v2.28.1) :** bandit bloquant (`nosec` justifiés B324/B404/B603/B607/B406, B105 exclu comme `pyproject`), `npm audit` bloquant (0 vulnérabilité), 5 suites frontend intégrées au CI (`upload`, `pretty`, `media-viewer`, `mfa-settings`, `config-ai-keys`). pip-audit reste consultatif (montées starlette/weasyprint à qualifier).
- **Description :** renforcer le pipeline (`.gitea/workflows/ci.yml`, `desktop-build.yml`) pour le rendre bloquant par défaut et accompagner les phases 1 → 3. Constat 2026-09-26 : job `security` non bloquant (`bandit`/`pip-audit` en `|| echo`, ni semgrep ni trivy), E2E limité à `chromium-desktop`, 5 suites frontend hors CI. - **Description :** renforcer le pipeline (`.gitea/workflows/ci.yml`, `desktop-build.yml`) pour le rendre bloquant par défaut et accompagner les phases 1 → 3. Constat 2026-09-26 : job `security` non bloquant (`bandit`/`pip-audit` en `|| echo`, ni semgrep ni trivy), E2E limité à `chromium-desktop`, 5 suites frontend hors CI.
- **Sous-tâches :** - **Sous-tâches :**
- [ ] Jobs CI sécurité **bloquants** (bandit/semgrep/trivy, audits pip/npm) + tests E2E XSS (page de partage + lecteur markdown) - [ ] Jobs CI sécurité **bloquants** (bandit/semgrep/trivy, audits pip/npm) + tests E2E XSS (page de partage + lecteur markdown) — **T4 livrée :** `tests/e2e/xss.spec.js` (BUG-021/022, 2/2 vert) + `scripts/e2e-server.ps1` (cycle de vie serveur E2E avec progression `start|stop|status|logs`) + validation locale projet `chromium-desktop` : **108/108 verts** (obsigate 44, split 37, viewers 24, xss/header 3), mobiles ciblés 10/10
- [ ] Tests de concurrence (`users.json`), fuzzing de timing regex, couverture des composants critiques ; intégrer au CI les 5 suites frontend hors CI (`upload`, `pretty`, `media-viewer`, `mfa-settings`, `config-ai-keys`) - [ ] Tests de concurrence (`users.json`), fuzzing de timing regex, couverture des composants critiques ; intégrer au CI les 5 suites frontend hors CI (`upload`, `pretty`, `media-viewer`, `mfa-settings`, `config-ai-keys`) — **T2 livrée (v2.28.2) :** `tests/test_hardening_concurrency.py` (users.json concurrent + budget temps regex) ; 5 suites au CI (T1)
- [ ] Finir BUG-034 (migration CSP **nonce**, conversion des handlers inline), `Secure` cookies à `true` par défaut, politique CORS same-origin explicite ; confirmer la rotation de la clé DeepSeek (BUG-006, clé dans l'historique Git) - [ ] Finir BUG-034 (migration CSP **nonce**, conversion des handlers inline), `Secure` cookies à `true` par défaut, politique CORS same-origin explicite ; confirmer la rotation de la clé DeepSeek (BUG-006, clé dans l'historique Git) — **T3 livrée (v2.28.3) :** helper `is_secure_cookies()` centralisé (défaut `false` conservé : compatibilité HTTP local), avertissement au démarrage sur bind non-loopback sans `Secure`, absence CORS attestée par test (`tests/test_security_headers.py`) ; bascule du défaut reportée (casserait les logins localhost)
- [ ] Revue périodique des dépendances ; documentation utilisateur FR/EN synchronisée ; contrôle automatisé de la conformité au DoD - [ ] Revue périodique des dépendances ; documentation utilisateur FR/EN synchronisée ; contrôle automatisé de la conformité au DoD
--- ---
@@ -164,6 +152,7 @@
| BUG-078 | Fichiers de code — coloration syntaxique restaurée (feuilles highlight.js basculées sur le mode de thème et non la clé) | 2.25.0 | [features/viewer-toolbar-highlight-avatars.md](./features/viewer-toolbar-highlight-avatars.md) | | BUG-078 | Fichiers de code — coloration syntaxique restaurée (feuilles highlight.js basculées sur le mode de thème et non la clé) | 2.25.0 | [features/viewer-toolbar-highlight-avatars.md](./features/viewer-toolbar-highlight-avatars.md) |
| 115 | Viewer — barre d'outils de lecture épinglée au défilement | 2.25.0 | [features/viewer-toolbar-highlight-avatars.md](./features/viewer-toolbar-highlight-avatars.md) | | 115 | Viewer — barre d'outils de lecture épinglée au défilement | 2.25.0 | [features/viewer-toolbar-highlight-avatars.md](./features/viewer-toolbar-highlight-avatars.md) |
| 117 | Configuration — avatars prédéfinis dans le profil utilisateur (12 images) | 2.25.0 | [features/viewer-toolbar-highlight-avatars.md](./features/viewer-toolbar-highlight-avatars.md) | | 117 | Configuration — avatars prédéfinis dans le profil utilisateur (12 images) | 2.25.0 | [features/viewer-toolbar-highlight-avatars.md](./features/viewer-toolbar-highlight-avatars.md) |
| 85 | Refonte architecturale — découpage du monolithe (14 routers, `main.py` 4 827 → ~750 lignes), stores JSON verrouillés, rate-limit SQLite optionnel | 2.27.2→2.27.13 | [features/archi-refonte-85.md](./features/archi-refonte-85.md) |
--- ---
@@ -171,17 +160,18 @@
| Priorité | Items | Effort total estimé | | Priorité | Items | Effort total estimé |
|---|---|---| |---|---|---|
| ✅ Complété | #1 → #59, #61–72, #74–76, #78–84, #86, #88–93, #94–100, #102–115, #117, #92 | ~133 jours réalisés | | ✅ Complété | #1 → #59, #61–72, #74–76, #78–86, #88–93, #94–100, #102–115, #117, #92 | ~141 jours réalisés |
| 🔵 Finitions | #77 Desktop : 6 tests E2E **manuels** ([protocole](./DESKTOP_E2E_CHECKLIST.md)) — signature Windows non retenue (décision 2026-09-26) | ~0,5-1 jour | | 🔵 Finitions | #77 Desktop : 6 tests E2E **manuels** ([protocole](./DESKTOP_E2E_CHECKLIST.md)) — signature Windows non retenue (décision 2026-09-26) | ~0,5-1 jour |
| ⚪ P4 reporté | #73 Sync — **reporté (décision 2026-09-26)**, hors chemin critique | 6-8 jours si réactivé | | ⚪ P4 reporté | #73 Sync — **reporté (décision 2026-09-26)**, hors chemin critique | 6-8 jours si réactivé |
| ⚪ P0/P1 prioritaire | #85, #87 Refonte architecturale, CI/CD (BUG-035 → BUG-040 corrigés, #86 livré) | ~11-17 jours | | ⚪ P0/P1 prioritaire | #87 CI/CD (BUG-035 → BUG-040 corrigés, #86 livré) | ~3-5 jours |
| **Total chemin critique** | **#77 fin + #85 + #87** | **~12-18 jours** | | **Total chemin critique** | **#77 fin + #87** | **~4-6 jours** |
--- ---
## Notes ## Notes
- **Décisions 2026-09-26 :** axe prioritaire = dette & sécurité (#85/#87) ; #73 Sync reporté (P4, hors chemin critique) ; desktop livré non signé + doc SmartScreen. - **Décisions 2026-09-26 :** axe prioritaire = dette & sécurité (#85/#87) ; #73 Sync reporté (P4, hors chemin critique) ; desktop livré non signé + doc SmartScreen.
- **Clôture #85 (v2.27.13) :** monolithe découpé (T1→T9), stores verrouillés + rate-limit SQLite (T10), fiche `docs/features/archi-refonte-85.md`.
- Les items P3/P4 ne sont pas ordonnés par priorité interne — à raffiner selon les retours utilisateurs. - Les items P3/P4 ne sont pas ordonnés par priorité interne — à raffiner selon les retours utilisateurs.
- L'effort inclut le développement + tests unitaires + intégration CI, mais pas la documentation utilisateur. - L'effort inclut le développement + tests unitaires + intégration CI, mais pas la documentation utilisateur.
- Les items marqués 🟢 (nice-to-have) sont de bons candidats pour des contributions externes. - Les items marqués 🟢 (nice-to-have) sont de bons candidats pour des contributions externes.
+77
View File
@@ -0,0 +1,77 @@
# #85 — Refonte architecturale : découpage du monolithe & persistance d'état (phase 2)
> **Statut :** livré (T1→T10) — `backend/main.py` 4 827 → ~750 lignes, 14 routers,
> persistance partielle (stores verrouillés + rate-limit SQLite optionnel).
> Méthode : tranches à impact minimal, comportement inchangé, un domaine par
> commit, suite complète verte à chaque commit (1320 passed / 6 skipped).
## 1. Découpage du monolithe (T1→T9, comportement inchangé)
Chaque tranche déplace un domaine vers `backend/routers/` (handlers verbatim,
mêmes chemins/modèles/auth/tags OpenAPI), les modèles vers `backend/schemas.py`,
et ne committe que sur suite verte + `test_version` vert.
| Tranche | Domaine | Nouveau module | Version |
|---|---|---|---|
| T1 | health (`/api/health*`) | `routers/health.py` (+ `HealthResponse` → schemas) | 2.27.2 |
| T2 | webhooks CRUD | `routers/webhooks.py` | 2.27.3 |
| T3 | sharing (`/api/share*`, `/s/*`) | `routers/sharing.py` | 2.27.4 |
| T4 | backups (9 routes) | `routers/backups.py` (+ `Diff/Restore*` → schemas, `backend/sse.py`) | 2.27.5 |
| T5 | search (11 routes) | `routers/search.py` (+ modèles → schemas, `backend/search_executor.py`) | 2.27.6 |
| T6a | lecture fichiers | `routers/files_read.py` (+ modèles, `routers/helpers.py`) | 2.27.7 |
| T6b | mutations fichiers/dossiers | `routers/files_write.py` (+ 15 modèles → schemas) | 2.27.8 |
| T6c | media/pdf/export/guide | `routers/files_media.py` (Range helper → `helpers.py`) | 2.27.9 |
| T7 | config (12 routes) | `routers/config.py` (`_FALLBACK_MODELS` déplacé) | 2.27.10 |
| T8 | vaults + history + conflicts (13 routes) | `routers/vaults.py`, `history.py`, `conflicts.py` (+ `backend/watcher_state.py`) | 2.27.11 |
| T9 | realtime + render | `routers/realtime.py` (SSE + collab WS), `backend/render.py` | 2.27.12 |
`main.py` ne contient plus que l'assemblage : lifespan, middlewares, montage
des routers, racine `/api`, statique/SPA, 4 cales de compatibilité testées
(`_resolve_safe_path`, `_backup_file`, `_check_vault_writable`, `_get_backup_dir`).
Correctifs au passage : décorateur orphelin `/s/{token}` (double-enregistrement
de `/api/conflicts`), tag OpenAPI `media` inexistant (assignation par chemin
conservée), tests statiques frontend réalignés (`image-viewer`, `media-viewer`),
tests repointés vers les modules canoniques (`test_ai_models`, `test_api_main`).
## 2. Persistance d'état (T10)
| État | Avant | Après |
|---|---|---|
| JTI révoqués (`revoked_tokens.json`) | persisté, **sans verrou** | `RLock` (load/save/revoke/check) |
| `shares.json` | persisté, **sans verrou** | `RLock` (4 mutateurs) |
| `webhooks.json` + secrets | persistés, **sans verrou** | `RLock` (create/update/delete/secrets) |
| `api_keys.json` (tool-secrets) | persisté, **sans verrou** | `RLock` (set/delete) |
| Rate-limit auth | mémoire, mono-process | **inchangé par défaut** + option `OBSIGATE_RATELIMIT_DB` (SQLite WAL : mêmes fenêtres/budgets, partagé multi-workers, survit au redémarrage) |
| Index de recherche | mémoire, rebuild au démarrage | **conservé** (voir §3) |
| `users.json`, `api_tokens.json`, `vault_settings.json` | déjà verrouillés (BUG-029, #107) | inchangé |
Tests : `tests/test_store_locks.py` (4 — concurrence threads, pertes prouvées
sans verrou : 25/200 partages), `tests/test_ratelimit_store.py` (7 —
sémantique SQLite identique, persistance, concurrence 200/200).
Déjà existants et vérifiés (pas de code) : verrous `threading` + `asyncio`
de l'indexeur (`_index_lock`, `_async_index_lock`), contrat central des
outils IA — `backend/tools/registry.py` couvre déjà permissions
(`requires_vault`, `require_destructive_allowed`), quotas
(`check_and_record` par outil) et redaction (`redact_payload`) pour les
35 outils enregistrés via `@tool(`.
## 3. Décisions assumées (non fait, et pourquoi)
- **Index non persisté sur disque.** Le rebuild différentiel (#86 : réutilise
les entrées inchangées `size` + `mtime`) rend le démarrage rapide ; un
snapshot introduirait des risques de staleness/drift de format sans gain
mesuré. Réévaluer si le démarrage devient lent (vaults 50k+ fichiers).
- **Redis exclu.** SQLite WAL couvre le multi-workers mono-hôte sans nouvelle
infra ; Redis reste l'option multi-nœuds documentée (cf. `ratelimit.py`).
- **`.gitignore` (`_*.py` ignore les `__init__.py`).** Contourné par
`git add -f` comme les packages existants ; assainir la règle à part.
- Noms en `_` conservés (`backend/render.py`, stores) : déplacement verbatim,
zéro churn d'appels.
## 4. Reste connu (hors #85)
- CSP `unsafe-inline` (migration nonce, BUG-034 partiel) et `Secure` cookies → #87.
- `main.py` (~750 lignes) : lifespan, middlewares, statique/SPA — cible
d'extraction ultérieure si besoin, non bloquant.
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "obsigate", "name": "obsigate",
"version": "2.27.11", "version": "2.28.5",
"description": "**Porte d'entrée web ultra-léger pour vos vaults Obsidian** — Accédez, naviguez et recherchez dans toutes vos notes Obsidian depuis n'importe quel appareil via une interface web moderne et responsive.", "description": "**Porte d'entrée web ultra-léger pour vos vaults Obsidian** — Accédez, naviguez et recherchez dans toutes vos notes Obsidian depuis n'importe quel appareil via une interface web moderne et responsive.",
"main": "patch.js", "main": "patch.js",
"directories": { "directories": {
+151
View File
@@ -0,0 +1,151 @@
<#
.SYNOPSIS
ObsiGate — cycle de vie du serveur E2E local, avec progression visible.
.DESCRIPTION
Remplace le one-liner opaque de démarrage : chaque étape affiche sa
progression (port, PID, attente du health check seconde par seconde,
version servie). Memes conditions que le job CI `e2e` et que
`scripts/run-e2e-local.ps1` : uvicorn natif, auth désactivée, fixtures
TestVault/TestDir, port 2029.
Le PID est persisté dans `data/e2e-server.pid` pour un arrêt propre
(`stop`) — plus de serveurs orphelins qui squattent le port.
.EXAMPLE
./scripts/e2e-server.ps1 start # démarre + attend READY (défaut)
./scripts/e2e-server.ps1 status # port, PID, version servie
./scripts/e2e-server.ps1 logs # queues des logs serveur
./scripts/e2e-server.ps1 stop # arrête le serveur + libère le port
#>
[CmdletBinding()]
param(
[Parameter(Position = 0)]
[ValidateSet("start", "stop", "status", "logs")]
[string]$Command = "start",
[string]$Port = $(if ($env:E2E_PORT) { $env:E2E_PORT } else { "2029" })
)
$ErrorActionPreference = "Stop"
$Root = Split-Path -Parent $PSScriptRoot
Set-Location -LiteralPath $Root
$BaseUrl = "http://127.0.0.1:$Port"
$Python = ".\.venv\Scripts\python.exe"
$PidFile = "data/e2e-server.pid"
$OutLog = "data/e2e-server.log"
$ErrLog = "data/e2e-server.err.log"
function Get-PortOwner {
$conn = Get-NetTCPConnection -LocalPort $Port -State Listen -ErrorAction SilentlyContinue |
Select-Object -First 1
if (-not $conn) { return $null }
$proc = Get-Process -Id $conn.OwningProcess -ErrorAction SilentlyContinue
return @{ Pid = $conn.OwningProcess; Name = $(if ($proc) { $proc.ProcessName } else { "?" }) }
}
function Stop-Server {
param([string]$Why = "")
$killed = @()
if (Test-Path -LiteralPath $PidFile) {
$srvPid = (Get-Content -LiteralPath $PidFile -TotalCount 1).Trim()
if ($srvPid -match '^\d+$') {
Stop-Process -Id $srvPid -Force -ErrorAction SilentlyContinue
$killed += $srvPid
}
Remove-Item -LiteralPath $PidFile -Force -ErrorAction SilentlyContinue
}
$owner = Get-PortOwner
if ($owner) {
Stop-Process -Id $owner.Pid -Force -ErrorAction SilentlyContinue
$killed += $owner.Pid
}
if ($killed.Count) { Write-Host "[OK] Serveur arrêté (PID $($killed -join ', ')). $Why" }
else { Write-Host "[OK] Aucun serveur en cours (port $Port libre)." }
}
switch ($Command) {
"stop" {
Write-Host "[1/1] Arrêt du serveur E2E (port $Port)..."
Stop-Server
}
"status" {
$owner = Get-PortOwner
if (-not $owner) { Write-Host "[INFO] Port $Port libre, aucun serveur."; break }
Write-Host "[INFO] Port $Port occupé par PID $($owner.Pid) ($($owner.Name))."
try {
$health = Invoke-WebRequest -Uri "$BaseUrl/api/health" -TimeoutSec 5 -UseBasicParsing |
Select-Object -ExpandProperty Content | ConvertFrom-Json
Write-Host "[OK] Health 200 — version $($health.version), $($health.total_files) fichiers indexés."
} catch {
Write-Host "[WARN] Processus présent mais health injoignable : $($_.Exception.Message)"
}
}
"logs" {
Write-Host "===== $OutLog (stdout) ====="
Get-Content -LiteralPath $OutLog -Tail 15 -ErrorAction SilentlyContinue
Write-Host "===== $ErrLog (stderr) ====="
Get-Content -LiteralPath $ErrLog -Tail 25 -ErrorAction SilentlyContinue
}
"start" {
Write-Host "[1/4] Port $Port..."
$owner = Get-PortOwner
if ($owner) {
Write-Host "[ERR] Port $Port déjà occupé par PID $($owner.Pid) ($($owner.Name))."
Write-Host " Lancez d'abord : ./scripts/e2e-server.ps1 stop"
exit 1
}
Write-Host " libre."
Write-Host "[2/4] Interpréteur $Python..."
if (-not (Test-Path -LiteralPath $Python)) {
Write-Host "[ERR] $Python introuvable. Créez le venv (voir AGENTS.md)."
exit 1
}
Write-Host " présent."
New-Item -ItemType Directory -Force -Path "data" | Out-Null
Write-Host "[3/4] Démarrage uvicorn (auth désactivée, TestVault/TestDir)..."
$env:OBSIGATE_AUTH_ENABLED = "false"
$env:VAULT_1_NAME = "TestVault"
$env:VAULT_1_PATH = (Resolve-Path -LiteralPath "test_vault").Path
$env:DIR_1_NAME = "TestDir"
$env:DIR_1_PATH = (Resolve-Path -LiteralPath "test_dir").Path
$server = Start-Process -FilePath $Python `
-ArgumentList "-m", "uvicorn", "backend.main:app", "--host", "127.0.0.1", "--port", $Port `
-RedirectStandardOutput $OutLog -RedirectStandardError $ErrLog `
-PassThru -WindowStyle Hidden
$server.Id | Set-Content -LiteralPath $PidFile
Write-Host " PID $($server.Id) (logs : $OutLog / $ErrLog)."
Write-Host "[4/4] Attente du health check (30 s max)..."
$ready = $false
for ($i = 1; $i -le 30; $i++) {
try {
Invoke-WebRequest -Uri "$BaseUrl/api/health" -TimeoutSec 2 -UseBasicParsing | Out-Null
$ready = $true
break
} catch {
if ($server.HasExited) {
Write-Host "[ERR] Le serveur a quitté (code $($server.ExitCode)). Fin du log :"
Get-Content -LiteralPath $ErrLog -Tail 15 -ErrorAction SilentlyContinue
exit 1
}
if ($i % 5 -eq 0) { Write-Host " ... $i/30 s (indexation en cours, voir $ErrLog)" }
Start-Sleep -Seconds 1
}
}
if (-not $ready) {
Write-Host "[ERR] Injoignable après 30 s. Fin du log :"
Get-Content -LiteralPath $ErrLog -Tail 15 -ErrorAction SilentlyContinue
exit 1
}
$health = Invoke-WebRequest -Uri "$BaseUrl/api/health" -TimeoutSec 5 -UseBasicParsing |
Select-Object -ExpandProperty Content | ConvertFrom-Json
Write-Host "[OK] READY — ObsiGate v$($health.version) sur $BaseUrl ($($health.total_files) fichiers)."
}
}
+142
View File
@@ -0,0 +1,142 @@
/**
* E2E tests — XSS stocké : page publique de partage + lecteur markdown (#87 T4).
*
* Non-régression BUG-021 (sanitizer serveur du rendu markdown) et BUG-022
* (échappement `title`/frontmatter + neutralisation `</script>` sur `/s/{token}`).
* Chaque test arme un guetteur de dialogues `alert` : le moindre JS exécuté
* fait échouer le test, en plus des assertions DOM (contenu échappé, aucun
* attribut `on*` vivant).
*
* Run (local, instance de test port 2029, auth désactivée — cf. scripts/run-e2e-local.ps1) :
* BASE_URL=http://localhost:2029 npx playwright test tests/e2e/xss.spec.js --project=chromium-desktop
*/
import { test, expect } from '@playwright/test';
const BASE = process.env.BASE_URL || 'http://localhost:2029';
const VAULT = 'TestVault';
const XSS_FILE = 'e2e-xss-probe.md';
const XSS_TITLE = '<img src=x onerror="window.__xss_title=1">';
const XSS_BODY = [
'# Sonde XSS',
'',
'<img src=x onerror="window.__xss_body=1">',
'',
'<script>window.__xss_script=1</script>',
'',
'[xss](javascript:window.__xss_js=1)',
].join('\n');
async function api(request, method, path, data) {
const resp = await request.fetch(`${BASE}${path}`, {
method,
data,
headers: { 'Content-Type': 'application/json' },
});
if (!resp.ok()) {
throw new Error(`${method} ${path} → ${resp.status()} ${await resp.text()}`);
}
return resp.json();
}
async function precleanProbeFile(request) {
// Idempotence : un run précédent interrompu a pu laisser le fichier sonde.
await request.fetch(`${BASE}/api/file/${VAULT}?path=${encodeURIComponent(XSS_FILE)}`, {
method: 'DELETE',
}).catch(() => {});
}
async function armAlertTrap(page) {
const dialogs = [];
page.on('dialog', async (d) => {
dialogs.push(d.message());
await d.dismiss();
});
return dialogs;
}
async function openFile(page, vault, filePath) {
const treeItem = page.locator(`.tree-item[data-vault="${vault}"][data-path="${filePath}"]`);
if (!(await treeItem.count())) {
await page.locator(`.tree-item.vault-item[data-vault="${vault}"]`).first().click();
await treeItem.waitFor({ state: 'attached', timeout: 8000 });
}
await treeItem.dblclick({ timeout: 5000 });
}
test.describe('XSS — page publique de partage (/s/{token}, BUG-022)', () => {
test('un titre/frontmatter hostile est échappé et aucun script ne tourne', async ({ page, request }) => {
const dialogs = await armAlertTrap(page);
await precleanProbeFile(request);
await api(request, 'POST', `/api/file/${VAULT}`, {
path: XSS_FILE,
// Titre entre quotes simples YAML (les doubles quotes internes restent
// des caractères ordinaires et arrivent intactes au backend).
content: `---\ntitle: '${XSS_TITLE}'\n---\n\n${XSS_BODY}\n`,
});
const share = await api(request, 'POST', `/api/share/${VAULT}`, { path: XSS_FILE });
await page.goto(`${BASE}/s/${share.token}`);
await expect(page.locator('.share-banner')).toBeVisible({ timeout: 10000 });
// Le titre affiché est le texte brut (balise neutralisée), pas un <img> vivant.
await expect(page.locator('.toolbar-title')).toContainText('<img src=x', { timeout: 5000 });
expect(await page.locator('.toolbar-title img').count()).toBe(0);
expect(await page.locator('img[onerror]').count()).toBe(0);
// Les 2 <script> de la page sont son code statique : le JSON embarqué
// (`#raw-content`) doit être neutralisé (aucun `</script>` littéral).
const rawEmbedded = await page.evaluate(() => {
const el = document.getElementById('raw-content');
return { text: el ? el.textContent : null };
});
expect(rawEmbedded.text).not.toBeNull();
expect(rawEmbedded.text).not.toContain('</script');
expect(rawEmbedded.text).toContain('\\u003c');
// Aucun payload n'a tourné (titre, corps, bloc script, lien javascript:).
const flags = await page.evaluate(() => ({
title: window.__xss_title,
body: window.__xss_body,
script: window.__xss_script,
js: window.__xss_js,
}));
expect(flags).toEqual({ title: undefined, body: undefined, script: undefined, js: undefined });
expect(dialogs).toEqual([]);
await api(request, 'DELETE', `/api/share/${share.id}`);
await api(request, 'DELETE', `/api/file/${VAULT}?path=${encodeURIComponent(XSS_FILE)}`);
});
});
test.describe('XSS — lecteur markdown (BUG-021)', () => {
test('le HTML injecté dans une note est neutralisé à l\'affichage', async ({ page, request }) => {
const dialogs = await armAlertTrap(page);
await precleanProbeFile(request);
await api(request, 'POST', `/api/file/${VAULT}`, { path: XSS_FILE, content: `${XSS_BODY}\n` });
await page.goto(BASE);
await page.waitForFunction(() => window.__OBSIGATE_BOOTED === true, { timeout: 20000 });
await openFile(page, VAULT, XSS_FILE);
const content = page.locator('#content-area');
await expect(content).toContainText('Sonde XSS', { timeout: 10000 });
// Le sanitizer serveur a retiré les vecteurs : pas d'onerror, pas de script,
// pas de lien javascript: exécutable dans la zone de lecture.
expect(await content.locator('img[onerror]').count()).toBe(0);
expect(await content.locator('script').count()).toBe(0);
expect(await content.locator('a[href^="javascript:"]').count()).toBe(0);
const flags = await page.evaluate(() => ({
body: window.__xss_body,
script: window.__xss_script,
js: window.__xss_js,
}));
expect(flags).toEqual({ body: undefined, script: undefined, js: undefined });
expect(dialogs).toEqual([]);
await api(request, 'DELETE', `/api/file/${VAULT}?path=${encodeURIComponent(XSS_FILE)}`);
});
});
+14 -14
View File
@@ -624,44 +624,44 @@ class TestHumanizeMtime:
class TestHeadingSlugify: class TestHeadingSlugify:
def test_slugify_simple(self): def test_slugify_simple(self):
from backend.main import _heading_slugify from backend.render import _heading_slugify # ROADMAP #85 T9 — déménagé depuis backend.main
assert _heading_slugify("Hello World") == "hello-world" assert _heading_slugify("Hello World") == "hello-world"
def test_slugify_accented(self): def test_slugify_accented(self):
from backend.main import _heading_slugify from backend.render import _heading_slugify # ROADMAP #85 T9 — déménagé depuis backend.main
result = _heading_slugify("Café Crème") result = _heading_slugify("Café Crème")
assert "cafe" in result or "caf" in result assert "cafe" in result or "caf" in result
def test_slugify_strips_symbols(self): def test_slugify_strips_symbols(self):
from backend.main import _heading_slugify from backend.render import _heading_slugify # ROADMAP #85 T9 — déménagé depuis backend.main
result = _heading_slugify("Hello, World! Test?") result = _heading_slugify("Hello, World! Test?")
assert result.startswith("hello") assert result.startswith("hello")
class TestRenderMarkdown: class TestRenderMarkdown:
def test_render_basic(self): def test_render_basic(self):
from backend.main import _render_markdown from backend.render import _render_markdown # ROADMAP #85 T9 — déménagé depuis backend.main
result = _render_markdown("# Hello\n\nThis is a test.", "TestVault") result = _render_markdown("# Hello\n\nThis is a test.", "TestVault")
assert "<h1" in result assert "<h1" in result
assert "Hello" in result assert "Hello" in result
def test_render_with_code(self): def test_render_with_code(self):
from backend.main import _render_markdown from backend.render import _render_markdown # ROADMAP #85 T9 — déménagé depuis backend.main
result = _render_markdown("```python\nprint('hello')\n```", "TestVault") result = _render_markdown("```python\nprint('hello')\n```", "TestVault")
assert "code" in result or "highlight" in result assert "code" in result or "highlight" in result
def test_render_with_heading_ids(self): def test_render_with_heading_ids(self):
from backend.main import _render_markdown from backend.render import _render_markdown # ROADMAP #85 T9 — déménagé depuis backend.main
result = _render_markdown("# Title\n## Subtitle", "TestVault") result = _render_markdown("# Title\n## Subtitle", "TestVault")
assert "id=" in result assert "id=" in result
def test_render_wikilink(self): def test_render_wikilink(self):
from backend.main import _render_markdown from backend.render import _render_markdown # ROADMAP #85 T9 — déménagé depuis backend.main
result = _render_markdown("Link [[nonexistent.md]] here", "TestVault") result = _render_markdown("Link [[nonexistent.md]] here", "TestVault")
assert "wikilink" in result assert "wikilink" in result
def test_render_image_wikilink(self): def test_render_image_wikilink(self):
from backend.main import _render_markdown from backend.render import _render_markdown # ROADMAP #85 T9 — déménagé depuis backend.main
result = _render_markdown("![[image.png]]", "TestVault") result = _render_markdown("![[image.png]]", "TestVault")
assert "img" in result or "image" in result or "wikilink" in result assert "img" in result or "image" in result or "wikilink" in result
@@ -709,31 +709,31 @@ class TestCheckVaultWritable:
class TestConvertWikilinks: class TestConvertWikilinks:
def test_convert_wikilink(self): def test_convert_wikilink(self):
from backend.main import _convert_wikilinks from backend.render import _convert_wikilinks # ROADMAP #85 T9 — déménagé depuis backend.main
# Missing wikilinks render as span.wikilink-missing # Missing wikilinks render as span.wikilink-missing
result = _convert_wikilinks("See [[Introduction à Python]] for details", "TestVault") result = _convert_wikilinks("See [[Introduction à Python]] for details", "TestVault")
assert "Introduction" in result assert "Introduction" in result
assert "wikilink" in result assert "wikilink" in result
def test_convert_wikilink_with_alias(self): def test_convert_wikilink_with_alias(self):
from backend.main import _convert_wikilinks from backend.render import _convert_wikilinks # ROADMAP #85 T9 — déménagé depuis backend.main
result = _convert_wikilinks("See [[file.md|a different name]] here", "TestVault") result = _convert_wikilinks("See [[file.md|a different name]] here", "TestVault")
assert "a different name" in result assert "a different name" in result
def test_convert_wikilink_image(self): def test_convert_wikilink_image(self):
from backend.main import _convert_wikilinks from backend.render import _convert_wikilinks # ROADMAP #85 T9 — déménagé depuis backend.main
result = _convert_wikilinks("See ![[image.png]] here", "TestVault") result = _convert_wikilinks("See ![[image.png]] here", "TestVault")
assert "image" in result or "img" in result assert "image" in result or "img" in result
def test_convert_wikilink_anchor(self): def test_convert_wikilink_anchor(self):
from backend.main import _convert_wikilinks from backend.render import _convert_wikilinks # ROADMAP #85 T9 — déménagé depuis backend.main
result = _convert_wikilinks("[[#Section importante|voir section]]", "TestVault") result = _convert_wikilinks("[[#Section importante|voir section]]", "TestVault")
assert 'href="#section-importante"' in result assert 'href="#section-importante"' in result
assert "wikilink-anchor" in result assert "wikilink-anchor" in result
assert "voir section" in result assert "voir section" in result
def test_convert_wikilink_anchor_without_alias(self): def test_convert_wikilink_anchor_without_alias(self):
from backend.main import _convert_wikilinks from backend.render import _convert_wikilinks # ROADMAP #85 T9 — déménagé depuis backend.main
result = _convert_wikilinks("[[#Claude Code]]", "TestVault") result = _convert_wikilinks("[[#Claude Code]]", "TestVault")
assert 'href="#claude-code"' in result assert 'href="#claude-code"' in result
assert "Claude Code" in result assert "Claude Code" in result
@@ -741,7 +741,7 @@ class TestConvertWikilinks:
class TestHeadingSlugifyHtmlStripping: class TestHeadingSlugifyHtmlStripping:
def test_slugify_strips_html_tags(self): def test_slugify_strips_html_tags(self):
from backend.main import _heading_slugify from backend.render import _heading_slugify # ROADMAP #85 T9 — déménagé depuis backend.main
result = _heading_slugify('## 1. Agents installés localement <a href="#table-des-matieres">↩</a>') result = _heading_slugify('## 1. Agents installés localement <a href="#table-des-matieres">↩</a>')
assert result == "1-agents-installes-localement" assert result == "1-agents-installes-localement"
+198
View File
@@ -0,0 +1,198 @@
"""Tests de durcissement — concurrence users.json + fuzzing regex (ROADMAP #87 T2).
- `users.json` : les read-modify-write sont sérialisés par `_users_lock`
(BUG-029). Ces tests martèlent create/update/record_login_failure depuis
plusieurs threads et exigent zéro mise à jour perdue + un JSON valide.
- Regex (BUG-025) : la politique `regex_safety` (longueur, quantificateurs
imbriqués, contenu tronqué) doit rejeter vite les motifs catastrophiques
et borner le temps des motifs acceptés sur gros contenu.
"""
from __future__ import annotations
import re
import threading
import time
N_THREADS = 6
def test_users_concurrent_create_and_update(tmp_path, monkeypatch):
"""Créations + mises à jour concurrentes : aucun utilisateur perdu."""
from backend.auth import user_store
monkeypatch.setattr(user_store, "USERS_FILE", tmp_path / "users.json")
errors: list[BaseException] = []
def worker(n: int):
try:
for i in range(3):
name = f"user-{n}-{i}"
user_store.create_user(name, "Motdepasse1!", display_name=name)
user_store.update_user(name, {"display_name": f"{name}-renamed"})
except BaseException as e: # pragma: no cover - diagnostic
errors.append(e)
threads = [threading.Thread(target=worker, args=(n,)) for n in range(N_THREADS)]
for t in threads:
t.start()
for t in threads:
t.join()
assert not errors
users = user_store._read()["users"]
assert len(users) == N_THREADS * 3
assert all(u["display_name"].endswith("-renamed") for u in users.values())
def test_users_concurrent_login_failures_no_lost_count(tmp_path, monkeypatch):
"""`record_login_failure` concurrents : compteur exact (pas de lost update)."""
from backend.auth import user_store
monkeypatch.setattr(user_store, "USERS_FILE", tmp_path / "users.json")
user_store.create_user("victim", "Motdepasse1!")
def worker():
for _ in range(10):
try:
user_store.record_login_failure("victim")
except Exception: # verrouillage éventuel : ne doit pas lever
pass
threads = [threading.Thread(target=worker) for _ in range(N_THREADS)]
for t in threads:
t.start()
for t in threads:
t.join()
user = user_store.get_user("victim")
assert user is not None
# Le compte peut se verrouiller en cours de route ; l'important est que
# le fichier reste un JSON valide et l'utilisateur présent.
assert user["username"] == "victim"
def test_users_file_stays_valid_json_under_load(tmp_path, monkeypatch):
"""Le fichier reste lisible à tout moment pendant les écritures."""
import json
from backend.auth import user_store
target = tmp_path / "users.json"
monkeypatch.setattr(user_store, "USERS_FILE", target)
user_store.create_user("base", "Motdepasse1!")
stop = threading.Event()
errors: list[BaseException] = []
def writer(n: int):
i = 0
while not stop.is_set():
try:
user_store.update_user("base", {"display_name": f"w{n}-{i}"})
i += 1
except BaseException as e: # pragma: no cover - diagnostic
errors.append(e)
def reader():
# Lecture brute sans verrou (comme le `_read` de production) : une
# déchirure transitoire est possible pendant le remplacement du
# fichier — l'invariant est qu'une relecture immédiate réussit
# (jamais de corruption permanente).
while not stop.is_set():
try:
raw = target.read_text(encoding="utf-8")
json.loads(raw)
except FileNotFoundError:
pass
except json.JSONDecodeError:
try:
time.sleep(0.01)
json.loads(target.read_text(encoding="utf-8"))
except FileNotFoundError:
pass
except BaseException as e: # pragma: no cover - diagnostic
errors.append(e)
except BaseException as e: # pragma: no cover - diagnostic
errors.append(e)
threads = [threading.Thread(target=writer, args=(n,)) for n in range(4)]
threads.append(threading.Thread(target=reader))
for t in threads:
t.start()
time.sleep(2.0)
stop.set()
for t in threads:
t.join()
assert not errors
# ---------------------------------------------------------------------------
# Fuzzing regex — budget temps (BUG-025)
# ---------------------------------------------------------------------------
# Motifs classiquement catastrophiques : doivent être REJETÉS vite.
CATASTROPHIC = [
"^(a+)+$",
"(a+)+$",
"(.*)*$",
"(a|aa)+$",
"(a+){2,}$",
r"(\w+)+$",
r"(a*)*b",
r"(x+x+)+y",
]
# Motifs acceptés (légitimes) : doivent tourner vite sur gros contenu.
ACCEPTED = [
r"hello",
r"h.llo",
r"\b\w+@\w+\.\w+\b",
r"[A-ZÉÈÊ][a-zéèêàâîôûç]+",
r"(ab|cd)+e",
r"\d{4}-\d{2}-\d{2}",
r"foo|bar|baz",
]
def test_catastrophic_patterns_rejected_fast():
"""Les motifs à backtracking catastrophique sont refusés en < 1 s."""
from backend.services.regex_safety import validate_regex
start = time.perf_counter()
for pattern in CATASTROPHIC:
try:
validate_regex(pattern)
except ValueError:
pass
assert time.perf_counter() - start < 1.0
def test_accepted_patterns_bounded_on_large_content():
"""Motifs acceptés sur 200 Ko : chacun < 5 s (budget large anti-flaky)."""
from backend.services.regex_safety import MAX_REGEX_CONTENT, truncate_for_regex, validate_regex
assert MAX_REGEX_CONTENT == 200_000
content = truncate_for_regex("abc héllo world [email protected] 2024-01-02 " * 5000)
assert len(content) <= MAX_REGEX_CONTENT
for pattern in ACCEPTED:
validate_regex(pattern) # ne doit pas lever
start = time.perf_counter()
re.search(pattern, content)
assert time.perf_counter() - start < 5.0, f"motif lent : {pattern!r}"
def test_validate_regex_policy():
"""Politique : vide/trop long/invalide → ValueError."""
from backend.services.regex_safety import MAX_PATTERN_LENGTH, validate_regex
for bad in ("", "x" * (MAX_PATTERN_LENGTH + 1), "(unclosed"):
try:
validate_regex(bad)
except ValueError:
pass
else: # pragma: no cover - doit lever
raise AssertionError(f"motif accepté à tort : {bad!r}")
assert validate_regex("simple") == "simple"
+143
View File
@@ -0,0 +1,143 @@
"""Tests — rate-limit SQLite optionnel (ROADMAP #85 T10b).
Le store mémoire reste le défaut (comportement inchangé) ; si
``OBSIGATE_RATELIMIT_DB`` pointe vers un fichier SQLite, les compteurs y
sont persistés (partagés entre workers/processus, conservés au redémarrage)
avec une sémantique identique (fenêtre glissante, budgets IP + compte,
reset au succès).
"""
from __future__ import annotations
import sqlite3
import threading
import time
def _use_db(monkeypatch, tmp_path):
monkeypatch.setenv("OBSIGATE_RATELIMIT_DB", str(tmp_path / "ratelimit.db"))
def test_memory_default_unchanged(monkeypatch):
"""Sans la variable d'env : le store mémoire historique est utilisé."""
from backend import ratelimit
monkeypatch.delenv("OBSIGATE_RATELIMIT_DB", raising=False)
assert ratelimit._db_path() is None
ip = "10.9.9.1"
ratelimit._ip_attempts.pop(ip, None)
assert not ratelimit.is_rate_limited(ip)
ratelimit.record_failure(ip)
assert not ratelimit.is_rate_limited(ip)
ratelimit.record_success(ip)
assert not ratelimit.is_rate_limited(ip)
def test_sqlite_failures_and_limit(monkeypatch, tmp_path):
"""Budget IP : N échecs → limité ; succès → reset (SQLite)."""
from backend import ratelimit
_use_db(monkeypatch, tmp_path)
monkeypatch.setattr(ratelimit, "MAX_ATTEMPTS", 3)
ip = "10.8.8.1"
assert not ratelimit.is_rate_limited(ip)
ratelimit.record_failure(ip)
ratelimit.record_failure(ip)
assert not ratelimit.is_rate_limited(ip)
failures, remaining = ratelimit.record_failure(ip)
assert (failures, remaining) == (3, 0)
assert ratelimit.is_rate_limited(ip)
ratelimit.record_success(ip)
assert not ratelimit.is_rate_limited(ip)
def test_sqlite_account_budget_case_insensitive(monkeypatch, tmp_path):
"""Budget par compte : insensible à la casse, indépendant des IP."""
from backend import ratelimit
_use_db(monkeypatch, tmp_path)
monkeypatch.setattr(ratelimit, "ACCOUNT_MAX_ATTEMPTS", 2)
assert not ratelimit.is_account_rate_limited("Alice")
ratelimit.record_account_failure("alice")
assert not ratelimit.is_account_rate_limited("ALICE")
ratelimit.record_account_failure("ALICE")
assert ratelimit.is_account_rate_limited("alice")
# Le budget IP n'est pas affecté par le budget compte.
assert not ratelimit.is_rate_limited("1.2.3.4")
ratelimit.record_account_success("alice")
assert not ratelimit.is_account_rate_limited("alice")
def test_sqlite_window_expiry(monkeypatch, tmp_path):
"""Les tentatives hors fenêtre ne comptent plus (SQLite)."""
from backend import ratelimit
_use_db(monkeypatch, tmp_path)
monkeypatch.setattr(ratelimit, "MAX_ATTEMPTS", 2)
monkeypatch.setattr(ratelimit, "WINDOW_SECONDS", 1)
ip = "10.7.7.1"
ratelimit.record_failure(ip)
ratelimit.record_failure(ip)
assert ratelimit.is_rate_limited(ip)
time.sleep(1.1)
assert not ratelimit.is_rate_limited(ip)
def test_sqlite_persists_across_restart(monkeypatch, tmp_path):
"""Les compteurs survivent au redémarrage (même fichier)."""
import os
from backend import ratelimit
db = tmp_path / "ratelimit.db"
monkeypatch.setenv("OBSIGATE_RATELIMIT_DB", str(db))
monkeypatch.setattr(ratelimit, "MAX_ATTEMPTS", 5)
for _ in range(3):
ratelimit.record_failure("10.6.6.6")
assert os.path.exists(db)
# "Redémarrage" : le module relit le même fichier (connexions courtes).
assert ratelimit.get_status("10.6.6.6")["failures"] == 3
with sqlite3.connect(str(db)) as conn:
(rows,) = conn.execute("SELECT COUNT(*) FROM attempts").fetchone()
assert rows == 3
def test_sqlite_get_status_shapes(monkeypatch, tmp_path):
"""`get_status` garde les mêmes formes qu'en mémoire."""
from backend import ratelimit
_use_db(monkeypatch, tmp_path)
ratelimit.record_failure("10.5.5.5")
ratelimit.record_account_failure("bob")
per_ip = ratelimit.get_status("10.5.5.5")
assert per_ip == {
"ip": "10.5.5.5",
"failures": 1,
"max": ratelimit.MAX_ATTEMPTS,
"limited": False,
"window_seconds": ratelimit.WINDOW_SECONDS,
}
glob = ratelimit.get_status()
assert glob["tracked_ips"] == 1
assert glob["tracked_accounts"] == 1
assert glob["limited_ips"] == 0
assert glob["max_attempts"] == ratelimit.MAX_ATTEMPTS
def test_sqlite_concurrent_writes(monkeypatch, tmp_path):
"""Écritures concurrentes : aucun échec compté perdu (SQLite/WAL)."""
from backend import ratelimit
_use_db(monkeypatch, tmp_path)
monkeypatch.setattr(ratelimit, "MAX_ATTEMPTS", 10_000)
def worker(n: int):
for _ in range(25):
ratelimit.record_failure("10.4.4.4")
threads = [threading.Thread(target=worker, args=(n,)) for n in range(8)]
for t in threads:
t.start()
for t in threads:
t.join()
assert ratelimit.get_status("10.4.4.4")["failures"] == 200
+52
View File
@@ -0,0 +1,52 @@
"""Tests — cookies Secure, CORS same-origin implicite, avertissement bind (ROADMAP #87 T3).
- `is_secure_cookies()` suit `OBSIGATE_SECURE_COOKIES` (défaut `false` :
compatibilité logins en HTTP local — les navigateurs ignorent les cookies
`Secure` en clair).
- Aucun en-tête CORS permissif n'est émis : sans `CORSMiddleware`, les
navigateurs appliquent le same-origin par défaut (politique explicite par
l'absence — vérifiée ici pour qu'un ajout futur de CORS soit conscient).
"""
from __future__ import annotations
def test_secure_cookies_default_false(monkeypatch):
"""Défaut `false` (logins HTTP locaux préservés)."""
from backend.auth.router import is_secure_cookies
monkeypatch.delenv("OBSIGATE_SECURE_COOKIES", raising=False)
assert is_secure_cookies() is False
def test_secure_cookies_opt_in(monkeypatch):
"""`OBSIGATE_SECURE_COOKIES=true` → flag actif (insensible à la casse)."""
from backend.auth.router import is_secure_cookies
for value in ("true", "True", "TRUE", "1", "yes"):
monkeypatch.setenv("OBSIGATE_SECURE_COOKIES", value)
assert is_secure_cookies() is (value.lower() == "true")
def test_no_cors_headers_on_api(client):
"""Pas de `Access-Control-Allow-Origin` : same-origin imposé par le navigateur."""
resp = client.get("/api/health")
assert resp.status_code == 200
assert "access-control-allow-origin" not in {k.lower() for k in resp.headers}
def test_no_cors_headers_on_public_share(client):
"""Idem sur la page publique de partage."""
resp = client.get("/s/jeton-inexistant")
assert resp.status_code == 404
assert "access-control-allow-origin" not in {k.lower() for k in resp.headers}
def test_security_headers_present(client):
"""En-têtes de durcissement posés par le middleware (non-régression)."""
resp = client.get("/api/health")
assert resp.headers.get("x-content-type-options") == "nosniff"
assert resp.headers.get("x-frame-options") == "SAMEORIGIN"
csp = resp.headers.get("content-security-policy", "")
assert "object-src 'none'" in csp
assert "frame-ancestors 'self'" in csp
+123
View File
@@ -0,0 +1,123 @@
"""Tests de non-régression — verrous des stores JSON (ROADMAP #85 T10a).
Sans verrou, les read-modify-write concurrents (créations de partages,
révocations de jetons) perdent des mises à jour : deux threads lisent le
même état, chacun écrit le sien, la première écriture est écrasée. Ces
tests martèlent les stores depuis plusieurs threads et exigent un compte
exact à la fin (aucune mise à jour perdue).
"""
from __future__ import annotations
import threading
N_THREADS = 8
N_OPS = 25
def test_concurrent_share_creations_lose_nothing(tmp_path, monkeypatch):
"""N threads × N partages → le store final contient tout."""
from backend import share as share_mod
monkeypatch.setattr(share_mod, "SHARES_FILE", tmp_path / "shares.json")
errors: list[BaseException] = []
def worker(n: int):
try:
for i in range(N_OPS):
share_mod.create_share("V", f"doc-{n}-{i}.md", "tester")
except BaseException as e: # pragma: no cover - diagnostic
errors.append(e)
threads = [threading.Thread(target=worker, args=(n,)) for n in range(N_THREADS)]
for t in threads:
t.start()
for t in threads:
t.join()
assert not errors
assert len(share_mod.list_shares()) == N_THREADS * N_OPS
def test_concurrent_share_access_and_revoke(tmp_path, monkeypatch):
"""Accès + révocations concurrents : compteurs et suppressions cohérents."""
from backend import share as share_mod
monkeypatch.setattr(share_mod, "SHARES_FILE", tmp_path / "shares.json")
tokens = [share_mod.create_share("V", f"doc-{i}.md", "tester")["token"] for i in range(20)]
def worker(n: int):
share_mod.record_access(tokens[2 * n])
share_mod.record_access(tokens[2 * n + 1])
share_mod.revoke_share(tokens[2 * n])
threads = [threading.Thread(target=worker, args=(n,)) for n in range(10)]
for t in threads:
t.start()
for t in threads:
t.join()
# Les 10 tokens pairs ont été révoqués ; les impairs subsistent avec
# leurs compteurs d'accès (aucune écriture perdue).
remaining = {s["token"] for s in share_mod.list_shares()}
assert len(remaining) == 10
assert all(share_mod.get_share_by_token(t)["access_count"] >= 1 for t in remaining)
def test_concurrent_token_revokes_lose_nothing(tmp_path, monkeypatch):
"""N threads × N révocations → toutes les JTIs sont persistées."""
import json
from backend.auth import jwt_handler
revoked_file = tmp_path / "revoked_tokens.json"
monkeypatch.setattr(jwt_handler, "REVOKED_TOKENS_FILE", revoked_file)
monkeypatch.setattr(jwt_handler, "_revoked_map", {})
monkeypatch.setattr(jwt_handler, "_revoked_loaded", True)
errors: list[BaseException] = []
def worker(n: int):
try:
for i in range(N_OPS):
jwt_handler.revoke_token(f"jti-{n}-{i}")
except BaseException as e: # pragma: no cover - diagnostic
errors.append(e)
threads = [threading.Thread(target=worker, args=(n,)) for n in range(N_THREADS)]
for t in threads:
t.start()
for t in threads:
t.join()
assert not errors
assert len(jwt_handler._revoked_map) == N_THREADS * N_OPS
# Le fichier reflète l'état mémoire (aucune écriture perdue).
on_disk = json.loads(revoked_file.read_text(encoding="utf-8"))
assert len(on_disk) == N_THREADS * N_OPS
assert jwt_handler.is_token_revoked("jti-0-0")
assert not jwt_handler.is_token_revoked("jti-absent")
def test_concurrent_webhook_and_tool_key_writes(tmp_path, monkeypatch):
"""Créations de webhooks + clés d'outils concurrentes : rien de perdu."""
from backend import webhooks as wh_mod
from backend.tools import secrets as sec_mod
monkeypatch.setattr(wh_mod, "WEBHOOKS_FILE", tmp_path / "webhooks.json")
monkeypatch.setattr(wh_mod, "WEBHOOK_SECRETS_FILE", tmp_path / "webhook_secrets.json")
monkeypatch.setattr(sec_mod, "_keys_file", lambda: tmp_path / "api_keys.json")
def worker(n: int):
for i in range(N_OPS):
wh_mod.create_webhook(f"hook-{n}-{i}", "https://example.com/hook", ["file_created"])
sec_mod.set_tool_key("OBSIGATE_GITHUB_TOKEN", f"tok-{n}-{i}")
threads = [threading.Thread(target=worker, args=(n,)) for n in range(N_THREADS)]
for t in threads:
t.start()
for t in threads:
t.join()
assert len(wh_mod.get_webhooks()) == N_THREADS * N_OPS