feat: #85 T10 persistance etat (verrous stores, ratelimit SQLite) et cloture refonte

This commit is contained in:
2026-09-26 18:10:10 -04:00
parent 58312e64da
commit d6cca2b1af
18 changed files with 701 additions and 149 deletions
+2
View File
@@ -23,6 +23,8 @@ OBSIGATE_ADMIN_PASSWORD=chab30
# OBSIGATE_LOGIN_MAX_ATTEMPTS=10 # OBSIGATE_LOGIN_MAX_ATTEMPTS=10
# OBSIGATE_ACCOUNT_MAX_ATTEMPTS=10 # OBSIGATE_ACCOUNT_MAX_ATTEMPTS=10
# OBSIGATE_LOGIN_WINDOW_SECONDS=900 # OBSIGATE_LOGIN_WINDOW_SECONDS=900
# Compteurs partagés/persistants (SQLite WAL, multi-workers) — défaut : mémoire.
# OBSIGATE_RATELIMIT_DB=data/ratelimit.db
# IP client derrière un reverse proxy (fait confiance à X-Forwarded-For) # IP client derrière un reverse proxy (fait confiance à X-Forwarded-For)
# OBSIGATE_TRUST_PROXY=false # OBSIGATE_TRUST_PROXY=false
+15 -1
View File
@@ -6,7 +6,7 @@ Format basé sur [Keep a Changelog](https://keepachangelog.com/fr/1.1.0/),
et [Semantic Versioning](https://semver.org/spec/v2.0.0.html). et [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
> **En cours de développement** : les changements à venir sont listés dans la section > **En cours de développement** : les changements à venir sont listés dans la section
> [Unreleased](#unreleased). La dernière version livrée est **2.27.12**. > [Unreleased](#unreleased). La dernière version livrée est **2.28.0**.
--- ---
@@ -14,6 +14,10 @@ et [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
--- ---
## [2.28.0] — 2026-09-26
---
## [2.27.12] — 2026-09-26 ## [2.27.12] — 2026-09-26
--- ---
@@ -58,6 +62,16 @@ et [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
### Modifié ### Modifié
- **#85 (T10) — persistance d'état et clôture de la refonte architecturale.**
Verrous `RLock` sur les stores JSON sans protection (`revoked_tokens`,
`shares`, `webhooks` + secrets, clés d'outils) avec tests de concurrence
(`tests/test_store_locks.py` — pertes prouvées sans verrou) ; rate-limit
auth persisté en option (`OBSIGATE_RATELIMIT_DB`, SQLite WAL, sémantique
identique, défaut mémoire inchangé, `tests/test_ratelimit_store.py`).
Contrat `tools/registry.py` audité (permissions/quotas/redaction déjà
câblés, rien à coder). Index non persisté : rebuild différentiel #86
suffisant (décision documentée). Fiche `docs/features/archi-refonte-85.md`,
#85 sorti du backlog (index roadmap).
- **#85 (T9) — extraction realtime + render hors du monolithe `backend/main.py`.** - **#85 (T9) — extraction realtime + render hors du monolithe `backend/main.py`.**
Le stream SSE `/api/events` et le WebSocket `/ws/collab/*` sont servis par Le stream SSE `/api/events` et le WebSocket `/ws/collab/*` sont servis par
`backend/routers/realtime.py`, le pipeline markdown (mistune, wikilinks, `backend/routers/realtime.py`, le pipeline markdown (mistune, wikilinks,
+3 -3
View File
@@ -4,7 +4,7 @@
**Porte d'entrée web ultra-léger pour vos vaults Obsidian** — Accédez, naviguez et recherchez dans toutes vos notes Obsidian depuis n'importe quel appareil via une interface web moderne et responsive. **Porte d'entrée web ultra-léger pour vos vaults Obsidian** — Accédez, naviguez et recherchez dans toutes vos notes Obsidian depuis n'importe quel appareil via une interface web moderne et responsive.
[![Version](https://img.shields.io/badge/Version-2.27.12-blue.svg)]() [![Version](https://img.shields.io/badge/Version-2.28.0-blue.svg)]()
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Docker](https://img.shields.io/badge/Docker-Ready-blue.svg)](https://www.docker.com/) [![Docker](https://img.shields.io/badge/Docker-Ready-blue.svg)](https://www.docker.com/)
[![Python](https://img.shields.io/badge/Python-3.11+-green.svg)](https://www.python.org/) [![Python](https://img.shields.io/badge/Python-3.11+-green.svg)](https://www.python.org/)
@@ -976,8 +976,8 @@ Ce projet est sous licence **MIT** — voir le fichier [LICENSE](LICENSE) pour l
## 📝 Changelog ## 📝 Changelog
Consultez le [CHANGELOG.md](./CHANGELOG.md) pour l'historique complet de toutes les versions (v1.0.0 → v2.27.12). Consultez le [CHANGELOG.md](./CHANGELOG.md) pour l'historique complet de toutes les versions (v1.0.0 → v2.28.0).
--- ---
*Projet : ObsiGate | Version : 2.27.12 | Dernière mise à jour : Septembre 2026* *Projet : ObsiGate | Version : 2.28.0 | Dernière mise à jour : Septembre 2026*
+3 -3
View File
@@ -2,7 +2,7 @@
**Ultra-light web gateway for your Obsidian vaults** — Access, browse, and search all your Obsidian notes from any device via a modern, responsive web interface. **Ultra-light web gateway for your Obsidian vaults** — Access, browse, and search all your Obsidian notes from any device via a modern, responsive web interface.
[![Version](https://img.shields.io/badge/Version-2.27.12-blue.svg)]() [![Version](https://img.shields.io/badge/Version-2.28.0-blue.svg)]()
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Docker](https://img.shields.io/badge/Docker-Ready-blue.svg)](https://www.docker.com/) [![Docker](https://img.shields.io/badge/Docker-Ready-blue.svg)](https://www.docker.com/)
[![Python](https://img.shields.io/badge/Python-3.11+-green.svg)](https://www.python.org/) [![Python](https://img.shields.io/badge/Python-3.11+-green.svg)](https://www.python.org/)
@@ -1151,8 +1151,8 @@ This project is licensed under the **MIT License** - see the [LICENSE](LICENSE)
## 📝 Changelog ## 📝 Changelog
See [CHANGELOG.md](./CHANGELOG.md) for the complete version history (v1.0.0 → v2.27.12). See [CHANGELOG.md](./CHANGELOG.md) for the complete version history (v1.0.0 → v2.28.0).
--- ---
*Project: ObsiGate | Version: 2.27.12 | Last updated: September 2026* *Project: ObsiGate | Version: 2.28.0 | Last updated: September 2026*
+1 -1
View File
@@ -1 +1 @@
2.27.12 2.28.0
+33 -26
View File
@@ -119,25 +119,30 @@ def decode_token(token: str) -> dict | None:
_revoked_map: dict[str, int] = {} _revoked_map: dict[str, int] = {}
_revoked_loaded = False _revoked_loaded = False
# ROADMAP #85 T10a — verrou autour du read-modify-write du store de
# révocation (perte de révocations en cas de logouts concurrents).
_revoked_lock = threading.RLock()
def _load_revoked(): def _load_revoked():
"""Load revoked token JTIs from disk into memory (once).""" """Load revoked token JTIs from disk into memory (once)."""
global _revoked_loaded, _revoked_map global _revoked_loaded, _revoked_map
if _revoked_loaded: with _revoked_lock:
return if _revoked_loaded:
if REVOKED_TOKENS_FILE.exists(): return
try: if REVOKED_TOKENS_FILE.exists():
data = json.loads(REVOKED_TOKENS_FILE.read_text()) try:
# Drop entries whose underlying token has itself expired. data = json.loads(REVOKED_TOKENS_FILE.read_text())
now = int(time.time()) # Drop entries whose underlying token has itself expired.
_revoked_map = { now = int(time.time())
jti: int(exp) for jti, exp in data.items() _revoked_map = {
if int(exp) > now jti: int(exp) for jti, exp in data.items()
} if int(exp) > now
except Exception as e: }
logger.warning(f"Failed to load revoked tokens: {e}") except Exception as e:
_revoked_map = {} logger.warning(f"Failed to load revoked tokens: {e}")
_revoked_loaded = True _revoked_map = {}
_revoked_loaded = True
def _save_revoked(): def _save_revoked():
@@ -154,24 +159,26 @@ def revoke_token(jti: str, expires_at: int | None = None):
``expires_at`` is the revoked token's own ``exp`` (unix seconds) — the ``expires_at`` is the revoked token's own ``exp`` (unix seconds) — the
record is kept at least that long so a long-lived API token cannot record is kept at least that long so a long-lived API token cannot
outlive its revocation. ``None`` means the token never expires (API/MCP outlive its revocation. ``None`` means the token never expires (API/MCP
"sans fin") → the record is kept forever (capped at ~100 years, the JWT "sans fin") → the record is kept forever (capped at ~100 years, the JWT
store's practical infinity). Default keeps 7 days (session tokens). store's practical infinity). Default keeps 7 days (session tokens).
""" """
_load_revoked() with _revoked_lock:
now = int(time.time()) _load_revoked()
if expires_at is None: now = int(time.time())
until = now + 100 * 365 * 24 * 3600 if expires_at is None:
else: until = now + 100 * 365 * 24 * 3600
until = max(int(expires_at), now + REFRESH_TOKEN_EXPIRE_SECONDS) else:
_revoked_map[jti] = until until = max(int(expires_at), now + REFRESH_TOKEN_EXPIRE_SECONDS)
_save_revoked() _revoked_map[jti] = until
_save_revoked()
logger.debug(f"Revoked token JTI: {jti[:8]}...") logger.debug(f"Revoked token JTI: {jti[:8]}...")
def is_token_revoked(jti: str) -> bool: def is_token_revoked(jti: str) -> bool:
"""Check if a token JTI has been revoked.""" """Check if a token JTI has been revoked."""
_load_revoked() with _revoked_lock:
return jti in _revoked_map _load_revoked()
return jti in _revoked_map
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
+172 -1
View File
@@ -12,14 +12,24 @@ the per-account lockout in ``user_store.py``.
deployment, front this service with a shared store (Redis) or a single deployment, front this service with a shared store (Redis) or a single
worker. This limitation is intentional and documented (BUG-031). worker. This limitation is intentional and documented (BUG-031).
Opt-in persistence (ROADMAP #85 T10b) : if ``OBSIGATE_RATELIMIT_DB`` points
to a SQLite file, counters are stored there instead (WAL mode, one short
connection per call — safe across threads, processes and restarts sharing
the same file). Semantics (windows, budgets, success reset) are identical
to the in-memory store, which remains the default when the variable is
unset.
Configuration via environment variables: Configuration via environment variables:
OBSIGATE_LOGIN_MAX_ATTEMPTS Max failures per IP (default: 10) OBSIGATE_LOGIN_MAX_ATTEMPTS Max failures per IP (default: 10)
OBSIGATE_ACCOUNT_MAX_ATTEMPTS Max failures per account (default: 10) OBSIGATE_ACCOUNT_MAX_ATTEMPTS Max failures per account (default: 10)
OBSIGATE_LOGIN_WINDOW_SECONDS Lockout window in seconds (default: 900) OBSIGATE_LOGIN_WINDOW_SECONDS Lockout window in seconds (default: 900)
OBSIGATE_RATELIMIT_DB SQLite file for shared/persistent counters (default: unset = memory)
""" """
import logging import logging
import os import os
import sqlite3
import threading
import time import time
from collections import defaultdict from collections import defaultdict
@@ -37,6 +47,127 @@ _last_cleanup = time.time()
CLEANUP_INTERVAL = 60 # seconds CLEANUP_INTERVAL = 60 # seconds
def _db_path() -> str | None:
"""SQLite file for shared counters, or ``None`` for the in-memory store."""
path = os.environ.get("OBSIGATE_RATELIMIT_DB", "").strip()
return path or None
def _db_connect(path: str) -> sqlite3.Connection:
"""Open a short-lived connection (WAL + busy timeout for concurrent workers)."""
_db_ensure_schema(path)
conn = sqlite3.connect(path, timeout=10.0)
conn.execute("PRAGMA busy_timeout=10000")
return conn
_schema_ready: set[str] = set()
_schema_lock = threading.Lock()
def _db_ensure_schema(path: str) -> None:
"""Create the store schema once per file (DDL under a process-wide lock)."""
with _schema_lock:
if path in _schema_ready:
return
conn = sqlite3.connect(path, timeout=10.0)
try:
conn.execute("PRAGMA journal_mode=WAL")
conn.execute(
"CREATE TABLE IF NOT EXISTS attempts"
" (kind TEXT NOT NULL, key TEXT NOT NULL, ts REAL NOT NULL, success INTEGER NOT NULL)"
)
conn.execute(
"CREATE INDEX IF NOT EXISTS idx_attempts_kind_key_ts"
" ON attempts (kind, key, ts)"
)
conn.commit()
finally:
conn.close()
_schema_ready.add(path)
def _db_write(fn, *args):
"""Run a write op, retrying once on lock contention (concurrent workers)."""
try:
return fn(*args)
except sqlite3.OperationalError as e:
if "locked" not in str(e).lower():
raise
time.sleep(0.05)
return fn(*args)
def _db_prune(conn: sqlite3.Connection, cutoff: float) -> None:
"""Drop expired entries (best-effort cap on disk growth)."""
conn.execute("DELETE FROM attempts WHERE ts <= ?", (cutoff,))
def _db_record(kind: str, key: str, success: bool) -> int:
"""Record one attempt in SQLite; return the live failure count."""
path = _db_path()
assert path is not None
now = time.time()
cutoff = now - WINDOW_SECONDS
def _write() -> int:
with _db_connect(path) as conn:
_db_prune(conn, cutoff)
if success:
# Mirror the in-memory reset: replace history with one success.
conn.execute("DELETE FROM attempts WHERE kind = ? AND key = ?", (kind, key))
conn.execute(
"INSERT INTO attempts (kind, key, ts, success) VALUES (?, ?, ?, ?)",
(kind, key, now, int(success)),
)
conn.commit()
(failures,) = conn.execute(
"SELECT COUNT(*) FROM attempts WHERE kind = ? AND key = ? AND ts > ? AND success = 0",
(kind, key, cutoff),
).fetchone()
return failures
return _db_write(_write)
def _db_failures(kind: str, key: str) -> int:
"""Live failure count in SQLite (expired entries never count)."""
path = _db_path()
assert path is not None
cutoff = time.time() - WINDOW_SECONDS
with _db_connect(path) as conn:
(failures,) = conn.execute(
"SELECT COUNT(*) FROM attempts WHERE kind = ? AND key = ? AND ts > ? AND success = 0",
(kind, key, cutoff),
).fetchone()
return failures
def _db_tracked(kind: str) -> int:
"""Number of distinct keys ever seen for one budget (SQLite)."""
path = _db_path()
assert path is not None
with _db_connect(path) as conn:
(n,) = conn.execute(
"SELECT COUNT(DISTINCT key) FROM attempts WHERE kind = ?", (kind,)
).fetchone()
return n
def _db_limited_count(kind: str, max_attempts: int) -> int:
"""Number of keys currently over budget (SQLite)."""
path = _db_path()
assert path is not None
cutoff = time.time() - WINDOW_SECONDS
with _db_connect(path) as conn:
rows = conn.execute(
"SELECT key, COUNT(*) FROM attempts"
" WHERE kind = ? AND ts > ? AND success = 0 GROUP BY key",
(kind, cutoff),
).fetchall()
return sum(1 for _, n in rows if n >= max_attempts)
def _prune(store: dict[str, list], cutoff: float) -> None: def _prune(store: dict[str, list], cutoff: float) -> None:
"""Drop expired entries from one store in place.""" """Drop expired entries from one store in place."""
expired = [] expired = []
@@ -66,6 +197,12 @@ def record_failure(ip: str) -> tuple[int, int]:
Returns: Returns:
(current_failure_count, remaining_attempts) (current_failure_count, remaining_attempts)
""" """
if _db_path() is not None:
failures = _db_record("ip", ip, False)
remaining = max(0, MAX_ATTEMPTS - failures)
if failures >= MAX_ATTEMPTS:
logger.warning(f"IP {ip} rate-limited after {failures} failed logins")
return failures, remaining
_cleanup_expired() _cleanup_expired()
_ip_attempts[ip].append((time.time(), False)) _ip_attempts[ip].append((time.time(), False))
failures = sum(1 for _, success in _ip_attempts[ip] if not success) failures = sum(1 for _, success in _ip_attempts[ip] if not success)
@@ -77,12 +214,17 @@ def record_failure(ip: str) -> tuple[int, int]:
def record_success(ip: str): def record_success(ip: str):
"""Clear rate limit state for an IP after successful login.""" """Clear rate limit state for an IP after successful login."""
if _db_path() is not None:
_db_record("ip", ip, True)
return
_cleanup_expired() _cleanup_expired()
_ip_attempts[ip] = [(time.time(), True)] _ip_attempts[ip] = [(time.time(), True)]
def is_rate_limited(ip: str) -> bool: def is_rate_limited(ip: str) -> bool:
"""Check if an IP has exceeded the rate limit.""" """Check if an IP has exceeded the rate limit."""
if _db_path() is not None:
return _db_failures("ip", ip) >= MAX_ATTEMPTS
_cleanup_expired() _cleanup_expired()
failures = sum(1 for _, success in _ip_attempts.get(ip, []) if not success) failures = sum(1 for _, success in _ip_attempts.get(ip, []) if not success)
return failures >= MAX_ATTEMPTS return failures >= MAX_ATTEMPTS
@@ -94,8 +236,14 @@ def record_account_failure(account: str) -> tuple[int, int]:
Returns: Returns:
(current_failure_count, remaining_attempts) (current_failure_count, remaining_attempts)
""" """
_cleanup_expired()
key = account.lower() key = account.lower()
if _db_path() is not None:
failures = _db_record("account", key, False)
remaining = max(0, ACCOUNT_MAX_ATTEMPTS - failures)
if failures >= ACCOUNT_MAX_ATTEMPTS:
logger.warning(f"Account {account} rate-limited after {failures} failed attempts")
return failures, remaining
_cleanup_expired()
_account_attempts[key].append((time.time(), False)) _account_attempts[key].append((time.time(), False))
failures = sum(1 for _, success in _account_attempts[key] if not success) failures = sum(1 for _, success in _account_attempts[key] if not success)
remaining = max(0, ACCOUNT_MAX_ATTEMPTS - failures) remaining = max(0, ACCOUNT_MAX_ATTEMPTS - failures)
@@ -106,12 +254,17 @@ def record_account_failure(account: str) -> tuple[int, int]:
def record_account_success(account: str): def record_account_success(account: str):
"""Clear the per-account rate limit state after a successful login.""" """Clear the per-account rate limit state after a successful login."""
if _db_path() is not None:
_db_record("account", account.lower(), True)
return
_cleanup_expired() _cleanup_expired()
_account_attempts[account.lower()] = [(time.time(), True)] _account_attempts[account.lower()] = [(time.time(), True)]
def is_account_rate_limited(account: str) -> bool: def is_account_rate_limited(account: str) -> bool:
"""Check if an account has exceeded the per-account rate limit.""" """Check if an account has exceeded the per-account rate limit."""
if _db_path() is not None:
return _db_failures("account", account.lower()) >= ACCOUNT_MAX_ATTEMPTS
_cleanup_expired() _cleanup_expired()
failures = sum( failures = sum(
1 for _, success in _account_attempts.get(account.lower(), []) if not success 1 for _, success in _account_attempts.get(account.lower(), []) if not success
@@ -121,6 +274,24 @@ def is_account_rate_limited(account: str) -> bool:
def get_status(ip: str | None = None) -> dict: def get_status(ip: str | None = None) -> dict:
"""Get rate limit status for an IP (for diagnostics).""" """Get rate limit status for an IP (for diagnostics)."""
if _db_path() is not None:
if ip:
failures = _db_failures("ip", ip)
return {
"ip": ip,
"failures": failures,
"max": MAX_ATTEMPTS,
"limited": failures >= MAX_ATTEMPTS,
"window_seconds": WINDOW_SECONDS,
}
return {
"tracked_ips": _db_tracked("ip"),
"tracked_accounts": _db_tracked("account"),
"max_attempts": MAX_ATTEMPTS,
"account_max_attempts": ACCOUNT_MAX_ATTEMPTS,
"window_seconds": WINDOW_SECONDS,
"limited_ips": _db_limited_count("ip", MAX_ATTEMPTS),
}
_cleanup_expired() _cleanup_expired()
if ip: if ip:
attempts = _ip_attempts.get(ip, []) attempts = _ip_attempts.get(ip, [])
+48 -39
View File
@@ -10,6 +10,7 @@ No authentication required for public share views.
import json import json
import logging import logging
import secrets import secrets
import threading
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from pathlib import Path from pathlib import Path
@@ -17,6 +18,10 @@ logger = logging.getLogger("obsigate.share")
SHARES_FILE = Path("data/shares.json") SHARES_FILE = Path("data/shares.json")
# ROADMAP #85 T10a — verrou autour des read-modify-write (perte de mises à
# jour en cas de créations/accès/révocations concurrents).
_lock = threading.RLock()
def _read() -> dict: def _read() -> dict:
if not SHARES_FILE.exists(): if not SHARES_FILE.exists():
@@ -41,26 +46,27 @@ def create_share(
expires_in_hours: int | None = None, expires_in_hours: int | None = None,
) -> dict: ) -> dict:
"""Create a new share token for a document.""" """Create a new share token for a document."""
data = _read() with _lock:
token = secrets.token_hex(32) # 64-char hex token data = _read()
token = secrets.token_hex(32) # 64-char hex token
expires_at = None expires_at = None
if expires_in_hours: if expires_in_hours:
expires_at = (datetime.now(timezone.utc) + timedelta(hours=expires_in_hours)).isoformat() expires_at = (datetime.now(timezone.utc) + timedelta(hours=expires_in_hours)).isoformat()
share = { share = {
"id": token, "id": token,
"token": token, "token": token,
"vault": vault, "vault": vault,
"path": path, "path": path,
"created_by": created_by, "created_by": created_by,
"created_at": datetime.now(timezone.utc).isoformat(), "created_at": datetime.now(timezone.utc).isoformat(),
"expires_at": expires_at, "expires_at": expires_at,
"access_count": 0, "access_count": 0,
"last_accessed": None, "last_accessed": None,
} }
data["shares"][token] = share data["shares"][token] = share
_write(data) _write(data)
logger.info(f"Created share for {vault}/{path} by {created_by}") logger.info(f"Created share for {vault}/{path} by {created_by}")
return share return share
@@ -80,22 +86,24 @@ def get_share_by_token(token: str) -> dict | None:
def record_access(token: str): def record_access(token: str):
"""Increment access counter for a share.""" """Increment access counter for a share."""
data = _read() with _lock:
share = data["shares"].get(token) data = _read()
if share: share = data["shares"].get(token)
share["access_count"] = share.get("access_count", 0) + 1 if share:
share["last_accessed"] = datetime.now(timezone.utc).isoformat() share["access_count"] = share.get("access_count", 0) + 1
_write(data) share["last_accessed"] = datetime.now(timezone.utc).isoformat()
_write(data)
def revoke_share(share_id: str) -> bool: def revoke_share(share_id: str) -> bool:
"""Revoke (delete) a share by its token.""" """Revoke (delete) a share by its token."""
data = _read() with _lock:
if share_id in data["shares"]: data = _read()
del data["shares"][share_id] if share_id in data["shares"]:
_write(data) del data["shares"][share_id]
logger.info(f"Revoked share {share_id}") _write(data)
return True logger.info(f"Revoked share {share_id}")
return True
return False return False
@@ -112,12 +120,13 @@ def list_shares(vault_filter: str | None = None) -> list:
def update_shares_after_rename(vault: str, old_path: str, new_path: str): def update_shares_after_rename(vault: str, old_path: str, new_path: str):
"""Update all shares when a file is renamed.""" """Update all shares when a file is renamed."""
data = _read() with _lock:
updated = False data = _read()
for sid, s in data["shares"].items(): updated = False
if s.get("vault") == vault and s.get("path") == old_path: for sid, s in data["shares"].items():
s["path"] = new_path if s.get("vault") == vault and s.get("path") == old_path:
updated = True s["path"] = new_path
logger.info(f"Updated share {sid}: {vault}/{old_path} -> {new_path}") updated = True
if updated: logger.info(f"Updated share {sid}: {vault}/{old_path} -> {new_path}")
_write(data) if updated:
_write(data)
+17 -11
View File
@@ -17,6 +17,7 @@ from __future__ import annotations
import json import json
import logging import logging
import os import os
import threading
from pathlib import Path from pathlib import Path
logger = logging.getLogger("obsigate.tools.secrets") logger = logging.getLogger("obsigate.tools.secrets")
@@ -34,6 +35,9 @@ TOOL_KEY_NAMES: tuple[str, ...] = (
_SECRET_MARKERS = ("API_KEY", "TOKEN") _SECRET_MARKERS = ("API_KEY", "TOKEN")
# ROADMAP #85 T10a — verrou autour des read-modify-write du store de clés.
_lock = threading.RLock()
def _keys_file() -> Path: def _keys_file() -> Path:
base = os.environ.get("OBSIGATE_DATA_DIR", "data") base = os.environ.get("OBSIGATE_DATA_DIR", "data")
@@ -89,21 +93,23 @@ def set_tool_key(name: str, value: str) -> None:
if name not in TOOL_KEY_NAMES: if name not in TOOL_KEY_NAMES:
raise ValueError(f"Clé non prise en charge: {name}") raise ValueError(f"Clé non prise en charge: {name}")
value = (value or "").strip() value = (value or "").strip()
keys = _read_keys() with _lock:
if value: keys = _read_keys()
keys[name] = value if value:
else: keys[name] = value
keys.pop(name, None) else:
_write_keys(keys) keys.pop(name, None)
_write_keys(keys)
def delete_tool_key(name: str) -> bool: def delete_tool_key(name: str) -> bool:
"""Remove one key from the store; return True when it existed.""" """Remove one key from the store; return True when it existed."""
if name not in TOOL_KEY_NAMES: if name not in TOOL_KEY_NAMES:
raise ValueError(f"Clé non prise en charge: {name}") raise ValueError(f"Clé non prise en charge: {name}")
keys = _read_keys() with _lock:
if name in keys: keys = _read_keys()
del keys[name] if name in keys:
_write_keys(keys) del keys[name]
return True _write_keys(keys)
return True
return False return False
+54 -43
View File
@@ -26,6 +26,7 @@ import json
import logging import logging
import os import os
import socket import socket
import threading
import uuid import uuid
from datetime import datetime, timezone from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
@@ -144,6 +145,12 @@ def _read_secrets() -> dict:
return {} return {}
# ROADMAP #85 T10a — verrou autour des read-modify-write des deux stores
# (webhooks + secrets) : perte de mises à jour en cas de mutations
# concurrentes.
_lock = threading.RLock()
def _write_secrets(secrets: dict): def _write_secrets(secrets: dict):
WEBHOOK_SECRETS_FILE.parent.mkdir(parents=True, exist_ok=True) WEBHOOK_SECRETS_FILE.parent.mkdir(parents=True, exist_ok=True)
tmp = WEBHOOK_SECRETS_FILE.with_suffix(".tmp") tmp = WEBHOOK_SECRETS_FILE.with_suffix(".tmp")
@@ -156,12 +163,13 @@ def _write_secrets(secrets: dict):
def _store_secret(wh_id: str, secret: str | None) -> None: def _store_secret(wh_id: str, secret: str | None) -> None:
secrets = _read_secrets() with _lock:
if secret: secrets = _read_secrets()
secrets[wh_id] = secret if secret:
else: secrets[wh_id] = secret
secrets.pop(wh_id, None) else:
_write_secrets(secrets) secrets.pop(wh_id, None)
_write_secrets(secrets)
def _get_secret(wh: dict) -> str | None: def _get_secret(wh: dict) -> str | None:
@@ -189,52 +197,55 @@ def get_webhooks() -> list:
def create_webhook(name: str, url: str, events: list[str], secret: str | None = None) -> dict: def create_webhook(name: str, url: str, events: list[str], secret: str | None = None) -> dict:
validate_webhook_url(url) validate_webhook_url(url)
webhooks = _read() with _lock:
wh_id = str(uuid.uuid4()) webhooks = _read()
wh = { wh_id = str(uuid.uuid4())
"id": wh_id, wh = {
"name": name, "id": wh_id,
"url": url, "name": name,
"events": [e for e in events if e in VALID_EVENTS], "url": url,
"enabled": True, "events": [e for e in events if e in VALID_EVENTS],
"created_at": datetime.now(timezone.utc).isoformat(), "enabled": True,
"last_fired_at": None, "created_at": datetime.now(timezone.utc).isoformat(),
} "last_fired_at": None,
webhooks.append(wh) }
_write(webhooks) webhooks.append(wh)
if secret: _write(webhooks)
_store_secret(wh_id, secret) if secret:
_store_secret(wh_id, secret)
logger.info(f"Created webhook '{name}' → {url}") logger.info(f"Created webhook '{name}' → {url}")
return _public_view(wh) return _public_view(wh)
def update_webhook(wh_id: str, updates: dict) -> dict | None: def update_webhook(wh_id: str, updates: dict) -> dict | None:
webhooks = _read() with _lock:
for wh in webhooks: webhooks = _read()
if wh["id"] == wh_id: for wh in webhooks:
if updates.get("url"): if wh["id"] == wh_id:
validate_webhook_url(updates["url"]) if updates.get("url"):
if "secret" in updates: validate_webhook_url(updates["url"])
_store_secret(wh_id, updates["secret"]) if "secret" in updates:
safe_updates = { _store_secret(wh_id, updates["secret"])
k: v for k, v in updates.items() safe_updates = {
if k not in ("id", "secret") k: v for k, v in updates.items()
} if k not in ("id", "secret")
wh.update(safe_updates) }
_write(webhooks) wh.update(safe_updates)
return _public_view(wh) _write(webhooks)
return _public_view(wh)
return None return None
def delete_webhook(wh_id: str) -> bool: def delete_webhook(wh_id: str) -> bool:
webhooks = _read() with _lock:
new_list = [wh for wh in webhooks if wh["id"] != wh_id] webhooks = _read()
if len(new_list) == len(webhooks): new_list = [wh for wh in webhooks if wh["id"] != wh_id]
return False if len(new_list) == len(webhooks):
_write(new_list) return False
secrets = _read_secrets() _write(new_list)
if secrets.pop(wh_id, None) is not None: secrets = _read_secrets()
_write_secrets(secrets) if secrets.pop(wh_id, None) is not None:
_write_secrets(secrets)
return True return True
+1 -1
View File
@@ -2626,7 +2626,7 @@ dependencies = [
[[package]] [[package]]
name = "obsigate-desktop" name = "obsigate-desktop"
version = "2.27.12" version = "2.28.0"
dependencies = [ dependencies = [
"chrono", "chrono",
"env_logger", "env_logger",
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "obsigate-desktop" name = "obsigate-desktop"
version = "2.27.12" version = "2.28.0"
description = "ObsiGate Desktop — Porte d'entrée native pour vos vaults Obsidian" description = "ObsiGate Desktop — Porte d'entrée native pour vos vaults Obsidian"
authors = ["Bruno Charest"] authors = ["Bruno Charest"]
edition = "2021" edition = "2021"
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"$schema": "https://raw.githubusercontent.com/nicedoc/obsigate/main/desktop/tauri.conf.schema.json", "$schema": "https://raw.githubusercontent.com/nicedoc/obsigate/main/desktop/tauri.conf.schema.json",
"productName": "ObsiGate", "productName": "ObsiGate",
"version": "2.27.12", "version": "2.28.0",
"identifier": "com.obsigate.desktop", "identifier": "com.obsigate.desktop",
"build": { "build": {
"frontendDist": "../frontend", "frontendDist": "../frontend",
+6 -17
View File
@@ -1,6 +1,6 @@
# ObsiGate — Roadmap # ObsiGate — Roadmap
> **Version :** 2.27.12 | **Dernière mise à jour :** 2026-09-26 > **Version :** 2.28.0 | **Dernière mise à jour :** 2026-09-26
> **Ce fichier ne contient que le travail à venir** (🔵 En cours + ⚪ Backlog) et un index compact > **Ce fichier ne contient que le travail à venir** (🔵 En cours + ⚪ Backlog) et un index compact
> vers les fonctionnalités livrées. > vers les fonctionnalités livrées.
> - **Méthode de livraison à appliquer pour toute tâche : [DELIVERY_WORKFLOW.md](./DELIVERY_WORKFLOW.md)** > - **Méthode de livraison à appliquer pour toute tâche : [DELIVERY_WORKFLOW.md](./DELIVERY_WORKFLOW.md)**
@@ -63,19 +63,6 @@
## ⚪ Backlog — Sécurité, architecture & performance (P0/P1) ## ⚪ Backlog — Sécurité, architecture & performance (P0/P1)
### 85. Refonte architecturale — découpage du monolithe & persistance d'état (phase 2)
- **Effort :** 8-12 jours | **Impact :** 🟡 | **Zone :** backend
- **Décision 2026-09-26 : prioritaire (axe Dette & sécurité).**
- **Statut :** 🔵 en cours depuis 2026-09-26 — découpe par tranches à impact minimal (comportement inchangé, un domaine par commit). **T1 livrée (v2.27.2) :** `health` (`/api/health`, `/api/health/detailed` → `backend/routers/health.py`, `HealthResponse` → `schemas.py`). **T2 livrée (v2.27.3) :** `webhooks` (CRUD `/api/webhooks` → `backend/routers/webhooks.py`, logique déjà dans `backend/webhooks.py`). **T3 livrée (v2.27.4) :** `sharing` (`/api/share/*`, `/api/shares`, `/s/{token}*` → `backend/routers/sharing.py`, logique déjà dans `backend/share.py`). **T4 livrée (v2.27.5) :** `backups` (9 routes `/api/file/{vault}/backups|diff|restore` + `/api/backups*` → `backend/routers/backups.py`, `Diff/Restore*` → `schemas.py`, singleton SSE → `backend/sse.py`). **T5 livrée (v2.27.6) :** `search` (11 routes search/tags/suggest/graph/reload → `backend/routers/search.py`, modèles search → `schemas.py`, pool threads → `backend/search_executor.py`). **T6a livrée (v2.27.7) :** lecture fichiers (`/api/browse`, `/raw`, `/download`, `/backlinks`, `GET /api/file` → `backend/routers/files_read.py`, modèles + `EXT_TO_LANG` déplacés, helpers `_content_disposition`/`_media_max_inline_bytes` → `backend/routers/helpers.py`). **T6b livrée (v2.27.8) :** mutations fichiers/dossiers (save, xlsx/save, delete, create, rename, move, directories ×3, batch-upload → `backend/routers/files_write.py`, 15 modèles → `schemas.py`). **T6c livrée (v2.27.9) :** media/pdf/export/guide (file/pdf, exports ×3, guide, pdf/stream|info, image, media+thumb, attachments ×2, vault settings ×3, vault files → `backend/routers/files_media.py`, Range helper → `helpers.py`). **T7 livrée (v2.27.10) :** config (app/ai-keys/tool-keys/ai-models/diagnostics/dashboard → `backend/routers/config.py`, `_FALLBACK_MODELS` + clés déplacés, `test_ai_models` réaligné). **T8 livrée (v2.27.11) :** vaults + history + conflicts (→ `backend/routers/vaults.py|history.py|conflicts.py`, `VaultInfo`/`BookmarkToggleRequest` → `schemas.py`, watcher → `backend/watcher_state.py`). **T9 livrée (v2.27.12) :** realtime + render (events SSE + collab WS → `backend/routers/realtime.py`, pipeline markdown → `backend/render.py`, derniers modèles → `schemas.py`).
- **Description :** extraire le monolithe `backend/main.py` (~4 827 lignes au 2026-09-26, ~17 % du backend) en routers FastAPI par domaine et rendre persistant l'état qui ne l'est pas (index de recherche, JTI révoqués, compteurs de rate-limit) pour préparer le multi-nœuds. L'état mémoire actuel (index, inverted index, vecteurs sémantiques, `SSEManager`, collab) rend le multi-workers unsafe.
- **Sous-tâches :**
- [ ] Routers par domaine : files, search, share, webhooks, plugins, collab, admin, ai — `main.py` conservé comme assemblage (< 500 lignes) ; dédupliquer les modèles Pydantic vers `schemas.py`. **Avancement :** `health` ✅ (T1, `backend/routers/health.py`), `webhooks` ✅ (T2, `backend/routers/webhooks.py`), `sharing` ✅ (T3, `backend/routers/sharing.py`), `backups` ✅ (T4, `backend/routers/backups.py` + `backend/sse.py`), `search` ✅ (T5, `backend/routers/search.py` + `backend/search_executor.py`), `files-read` ✅ (T6a, `backend/routers/files_read.py` + `helpers.py`), `files-write` ✅ (T6b, `backend/routers/files_write.py`), `files-media` ✅ (T6c, `backend/routers/files_media.py`), `config` ✅ (T7, `backend/routers/config.py`), `vaults|history|conflicts` ✅ (T8, 3 routers + `watcher_state.py`), `realtime|render` ✅ (T9, `routers/realtime.py` + `render.py`) ; reste T10 (persistance + verrous + registry)
- [ ] Compléter `tools/registry.py` (existant : permissions/quotas/redaction) comme contrat central des outils IA si des manques sont constatés
- [ ] Persister index, JTI révoqués et compteurs de rate-limit (SQLite par défaut, Redis en option multi-nœuds ; le rate-limit actuel est in-memory mono-process)
- [ ] Verrous asyncio autour de l'index global et des stores JSON ; auditer les `except Exception` larges (> 100 occurrences) : best-effort (backup/audit) vs masquage d'erreur (erreurs typées 4xx/5xx + test)
- [ ] Extraire le service de partage public (expiration, révocation, quotas)
### 87. Amélioration continue — tests, CI/CD, revues de sécurité (phase 4) ### 87. Amélioration continue — tests, CI/CD, revues de sécurité (phase 4)
- **Effort :** 3-5 jours | **Impact :** 🟡 | **Zone :** `.gitea/workflows/`, `tests/` - **Effort :** 3-5 jours | **Impact :** 🟡 | **Zone :** `.gitea/workflows/`, `tests/`
@@ -164,6 +151,7 @@
| BUG-078 | Fichiers de code — coloration syntaxique restaurée (feuilles highlight.js basculées sur le mode de thème et non la clé) | 2.25.0 | [features/viewer-toolbar-highlight-avatars.md](./features/viewer-toolbar-highlight-avatars.md) | | BUG-078 | Fichiers de code — coloration syntaxique restaurée (feuilles highlight.js basculées sur le mode de thème et non la clé) | 2.25.0 | [features/viewer-toolbar-highlight-avatars.md](./features/viewer-toolbar-highlight-avatars.md) |
| 115 | Viewer — barre d'outils de lecture épinglée au défilement | 2.25.0 | [features/viewer-toolbar-highlight-avatars.md](./features/viewer-toolbar-highlight-avatars.md) | | 115 | Viewer — barre d'outils de lecture épinglée au défilement | 2.25.0 | [features/viewer-toolbar-highlight-avatars.md](./features/viewer-toolbar-highlight-avatars.md) |
| 117 | Configuration — avatars prédéfinis dans le profil utilisateur (12 images) | 2.25.0 | [features/viewer-toolbar-highlight-avatars.md](./features/viewer-toolbar-highlight-avatars.md) | | 117 | Configuration — avatars prédéfinis dans le profil utilisateur (12 images) | 2.25.0 | [features/viewer-toolbar-highlight-avatars.md](./features/viewer-toolbar-highlight-avatars.md) |
| 85 | Refonte architecturale — découpage du monolithe (14 routers, `main.py` 4 827 → ~750 lignes), stores JSON verrouillés, rate-limit SQLite optionnel | 2.27.2→2.27.13 | [features/archi-refonte-85.md](./features/archi-refonte-85.md) |
--- ---
@@ -171,17 +159,18 @@
| Priorité | Items | Effort total estimé | | Priorité | Items | Effort total estimé |
|---|---|---| |---|---|---|
| ✅ Complété | #1 → #59, #61–72, #74–76, #78–84, #86, #88–93, #94–100, #102–115, #117, #92 | ~133 jours réalisés | | ✅ Complété | #1 → #59, #61–72, #74–76, #78–86, #88–93, #94–100, #102–115, #117, #92 | ~141 jours réalisés |
| 🔵 Finitions | #77 Desktop : 6 tests E2E **manuels** ([protocole](./DESKTOP_E2E_CHECKLIST.md)) — signature Windows non retenue (décision 2026-09-26) | ~0,5-1 jour | | 🔵 Finitions | #77 Desktop : 6 tests E2E **manuels** ([protocole](./DESKTOP_E2E_CHECKLIST.md)) — signature Windows non retenue (décision 2026-09-26) | ~0,5-1 jour |
| ⚪ P4 reporté | #73 Sync — **reporté (décision 2026-09-26)**, hors chemin critique | 6-8 jours si réactivé | | ⚪ P4 reporté | #73 Sync — **reporté (décision 2026-09-26)**, hors chemin critique | 6-8 jours si réactivé |
| ⚪ P0/P1 prioritaire | #85, #87 Refonte architecturale, CI/CD (BUG-035 → BUG-040 corrigés, #86 livré) | ~11-17 jours | | ⚪ P0/P1 prioritaire | #87 CI/CD (BUG-035 → BUG-040 corrigés, #86 livré) | ~3-5 jours |
| **Total chemin critique** | **#77 fin + #85 + #87** | **~12-18 jours** | | **Total chemin critique** | **#77 fin + #87** | **~4-6 jours** |
--- ---
## Notes ## Notes
- **Décisions 2026-09-26 :** axe prioritaire = dette & sécurité (#85/#87) ; #73 Sync reporté (P4, hors chemin critique) ; desktop livré non signé + doc SmartScreen. - **Décisions 2026-09-26 :** axe prioritaire = dette & sécurité (#85/#87) ; #73 Sync reporté (P4, hors chemin critique) ; desktop livré non signé + doc SmartScreen.
- **Clôture #85 (v2.27.13) :** monolithe découpé (T1→T9), stores verrouillés + rate-limit SQLite (T10), fiche `docs/features/archi-refonte-85.md`.
- Les items P3/P4 ne sont pas ordonnés par priorité interne — à raffiner selon les retours utilisateurs. - Les items P3/P4 ne sont pas ordonnés par priorité interne — à raffiner selon les retours utilisateurs.
- L'effort inclut le développement + tests unitaires + intégration CI, mais pas la documentation utilisateur. - L'effort inclut le développement + tests unitaires + intégration CI, mais pas la documentation utilisateur.
- Les items marqués 🟢 (nice-to-have) sont de bons candidats pour des contributions externes. - Les items marqués 🟢 (nice-to-have) sont de bons candidats pour des contributions externes.
+77
View File
@@ -0,0 +1,77 @@
# #85 — Refonte architecturale : découpage du monolithe & persistance d'état (phase 2)
> **Statut :** livré (T1→T10) — `backend/main.py` 4 827 → ~750 lignes, 14 routers,
> persistance partielle (stores verrouillés + rate-limit SQLite optionnel).
> Méthode : tranches à impact minimal, comportement inchangé, un domaine par
> commit, suite complète verte à chaque commit (1320 passed / 6 skipped).
## 1. Découpage du monolithe (T1→T9, comportement inchangé)
Chaque tranche déplace un domaine vers `backend/routers/` (handlers verbatim,
mêmes chemins/modèles/auth/tags OpenAPI), les modèles vers `backend/schemas.py`,
et ne committe que sur suite verte + `test_version` vert.
| Tranche | Domaine | Nouveau module | Version |
|---|---|---|---|
| T1 | health (`/api/health*`) | `routers/health.py` (+ `HealthResponse` → schemas) | 2.27.2 |
| T2 | webhooks CRUD | `routers/webhooks.py` | 2.27.3 |
| T3 | sharing (`/api/share*`, `/s/*`) | `routers/sharing.py` | 2.27.4 |
| T4 | backups (9 routes) | `routers/backups.py` (+ `Diff/Restore*` → schemas, `backend/sse.py`) | 2.27.5 |
| T5 | search (11 routes) | `routers/search.py` (+ modèles → schemas, `backend/search_executor.py`) | 2.27.6 |
| T6a | lecture fichiers | `routers/files_read.py` (+ modèles, `routers/helpers.py`) | 2.27.7 |
| T6b | mutations fichiers/dossiers | `routers/files_write.py` (+ 15 modèles → schemas) | 2.27.8 |
| T6c | media/pdf/export/guide | `routers/files_media.py` (Range helper → `helpers.py`) | 2.27.9 |
| T7 | config (12 routes) | `routers/config.py` (`_FALLBACK_MODELS` déplacé) | 2.27.10 |
| T8 | vaults + history + conflicts (13 routes) | `routers/vaults.py`, `history.py`, `conflicts.py` (+ `backend/watcher_state.py`) | 2.27.11 |
| T9 | realtime + render | `routers/realtime.py` (SSE + collab WS), `backend/render.py` | 2.27.12 |
`main.py` ne contient plus que l'assemblage : lifespan, middlewares, montage
des routers, racine `/api`, statique/SPA, 4 cales de compatibilité testées
(`_resolve_safe_path`, `_backup_file`, `_check_vault_writable`, `_get_backup_dir`).
Correctifs au passage : décorateur orphelin `/s/{token}` (double-enregistrement
de `/api/conflicts`), tag OpenAPI `media` inexistant (assignation par chemin
conservée), tests statiques frontend réalignés (`image-viewer`, `media-viewer`),
tests repointés vers les modules canoniques (`test_ai_models`, `test_api_main`).
## 2. Persistance d'état (T10)
| État | Avant | Après |
|---|---|---|
| JTI révoqués (`revoked_tokens.json`) | persisté, **sans verrou** | `RLock` (load/save/revoke/check) |
| `shares.json` | persisté, **sans verrou** | `RLock` (4 mutateurs) |
| `webhooks.json` + secrets | persistés, **sans verrou** | `RLock` (create/update/delete/secrets) |
| `api_keys.json` (tool-secrets) | persisté, **sans verrou** | `RLock` (set/delete) |
| Rate-limit auth | mémoire, mono-process | **inchangé par défaut** + option `OBSIGATE_RATELIMIT_DB` (SQLite WAL : mêmes fenêtres/budgets, partagé multi-workers, survit au redémarrage) |
| Index de recherche | mémoire, rebuild au démarrage | **conservé** (voir §3) |
| `users.json`, `api_tokens.json`, `vault_settings.json` | déjà verrouillés (BUG-029, #107) | inchangé |
Tests : `tests/test_store_locks.py` (4 — concurrence threads, pertes prouvées
sans verrou : 25/200 partages), `tests/test_ratelimit_store.py` (7 —
sémantique SQLite identique, persistance, concurrence 200/200).
Déjà existants et vérifiés (pas de code) : verrous `threading` + `asyncio`
de l'indexeur (`_index_lock`, `_async_index_lock`), contrat central des
outils IA — `backend/tools/registry.py` couvre déjà permissions
(`requires_vault`, `require_destructive_allowed`), quotas
(`check_and_record` par outil) et redaction (`redact_payload`) pour les
35 outils enregistrés via `@tool(`.
## 3. Décisions assumées (non fait, et pourquoi)
- **Index non persisté sur disque.** Le rebuild différentiel (#86 : réutilise
les entrées inchangées `size` + `mtime`) rend le démarrage rapide ; un
snapshot introduirait des risques de staleness/drift de format sans gain
mesuré. Réévaluer si le démarrage devient lent (vaults 50k+ fichiers).
- **Redis exclu.** SQLite WAL couvre le multi-workers mono-hôte sans nouvelle
infra ; Redis reste l'option multi-nœuds documentée (cf. `ratelimit.py`).
- **`.gitignore` (`_*.py` ignore les `__init__.py`).** Contourné par
`git add -f` comme les packages existants ; assainir la règle à part.
- Noms en `_` conservés (`backend/render.py`, stores) : déplacement verbatim,
zéro churn d'appels.
## 4. Reste connu (hors #85)
- CSP `unsafe-inline` (migration nonce, BUG-034 partiel) et `Secure` cookies → #87.
- `main.py` (~750 lignes) : lifespan, middlewares, statique/SPA — cible
d'extraction ultérieure si besoin, non bloquant.
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "obsigate", "name": "obsigate",
"version": "2.27.12", "version": "2.28.0",
"description": "**Porte d'entrée web ultra-léger pour vos vaults Obsidian** — Accédez, naviguez et recherchez dans toutes vos notes Obsidian depuis n'importe quel appareil via une interface web moderne et responsive.", "description": "**Porte d'entrée web ultra-léger pour vos vaults Obsidian** — Accédez, naviguez et recherchez dans toutes vos notes Obsidian depuis n'importe quel appareil via une interface web moderne et responsive.",
"main": "patch.js", "main": "patch.js",
"directories": { "directories": {
+143
View File
@@ -0,0 +1,143 @@
"""Tests — rate-limit SQLite optionnel (ROADMAP #85 T10b).
Le store mémoire reste le défaut (comportement inchangé) ; si
``OBSIGATE_RATELIMIT_DB`` pointe vers un fichier SQLite, les compteurs y
sont persistés (partagés entre workers/processus, conservés au redémarrage)
avec une sémantique identique (fenêtre glissante, budgets IP + compte,
reset au succès).
"""
from __future__ import annotations
import sqlite3
import threading
import time
def _use_db(monkeypatch, tmp_path):
monkeypatch.setenv("OBSIGATE_RATELIMIT_DB", str(tmp_path / "ratelimit.db"))
def test_memory_default_unchanged(monkeypatch):
"""Sans la variable d'env : le store mémoire historique est utilisé."""
from backend import ratelimit
monkeypatch.delenv("OBSIGATE_RATELIMIT_DB", raising=False)
assert ratelimit._db_path() is None
ip = "10.9.9.1"
ratelimit._ip_attempts.pop(ip, None)
assert not ratelimit.is_rate_limited(ip)
ratelimit.record_failure(ip)
assert not ratelimit.is_rate_limited(ip)
ratelimit.record_success(ip)
assert not ratelimit.is_rate_limited(ip)
def test_sqlite_failures_and_limit(monkeypatch, tmp_path):
"""Budget IP : N échecs → limité ; succès → reset (SQLite)."""
from backend import ratelimit
_use_db(monkeypatch, tmp_path)
monkeypatch.setattr(ratelimit, "MAX_ATTEMPTS", 3)
ip = "10.8.8.1"
assert not ratelimit.is_rate_limited(ip)
ratelimit.record_failure(ip)
ratelimit.record_failure(ip)
assert not ratelimit.is_rate_limited(ip)
failures, remaining = ratelimit.record_failure(ip)
assert (failures, remaining) == (3, 0)
assert ratelimit.is_rate_limited(ip)
ratelimit.record_success(ip)
assert not ratelimit.is_rate_limited(ip)
def test_sqlite_account_budget_case_insensitive(monkeypatch, tmp_path):
"""Budget par compte : insensible à la casse, indépendant des IP."""
from backend import ratelimit
_use_db(monkeypatch, tmp_path)
monkeypatch.setattr(ratelimit, "ACCOUNT_MAX_ATTEMPTS", 2)
assert not ratelimit.is_account_rate_limited("Alice")
ratelimit.record_account_failure("alice")
assert not ratelimit.is_account_rate_limited("ALICE")
ratelimit.record_account_failure("ALICE")
assert ratelimit.is_account_rate_limited("alice")
# Le budget IP n'est pas affecté par le budget compte.
assert not ratelimit.is_rate_limited("1.2.3.4")
ratelimit.record_account_success("alice")
assert not ratelimit.is_account_rate_limited("alice")
def test_sqlite_window_expiry(monkeypatch, tmp_path):
"""Les tentatives hors fenêtre ne comptent plus (SQLite)."""
from backend import ratelimit
_use_db(monkeypatch, tmp_path)
monkeypatch.setattr(ratelimit, "MAX_ATTEMPTS", 2)
monkeypatch.setattr(ratelimit, "WINDOW_SECONDS", 1)
ip = "10.7.7.1"
ratelimit.record_failure(ip)
ratelimit.record_failure(ip)
assert ratelimit.is_rate_limited(ip)
time.sleep(1.1)
assert not ratelimit.is_rate_limited(ip)
def test_sqlite_persists_across_restart(monkeypatch, tmp_path):
"""Les compteurs survivent au redémarrage (même fichier)."""
import os
from backend import ratelimit
db = tmp_path / "ratelimit.db"
monkeypatch.setenv("OBSIGATE_RATELIMIT_DB", str(db))
monkeypatch.setattr(ratelimit, "MAX_ATTEMPTS", 5)
for _ in range(3):
ratelimit.record_failure("10.6.6.6")
assert os.path.exists(db)
# "Redémarrage" : le module relit le même fichier (connexions courtes).
assert ratelimit.get_status("10.6.6.6")["failures"] == 3
with sqlite3.connect(str(db)) as conn:
(rows,) = conn.execute("SELECT COUNT(*) FROM attempts").fetchone()
assert rows == 3
def test_sqlite_get_status_shapes(monkeypatch, tmp_path):
"""`get_status` garde les mêmes formes qu'en mémoire."""
from backend import ratelimit
_use_db(monkeypatch, tmp_path)
ratelimit.record_failure("10.5.5.5")
ratelimit.record_account_failure("bob")
per_ip = ratelimit.get_status("10.5.5.5")
assert per_ip == {
"ip": "10.5.5.5",
"failures": 1,
"max": ratelimit.MAX_ATTEMPTS,
"limited": False,
"window_seconds": ratelimit.WINDOW_SECONDS,
}
glob = ratelimit.get_status()
assert glob["tracked_ips"] == 1
assert glob["tracked_accounts"] == 1
assert glob["limited_ips"] == 0
assert glob["max_attempts"] == ratelimit.MAX_ATTEMPTS
def test_sqlite_concurrent_writes(monkeypatch, tmp_path):
"""Écritures concurrentes : aucun échec compté perdu (SQLite/WAL)."""
from backend import ratelimit
_use_db(monkeypatch, tmp_path)
monkeypatch.setattr(ratelimit, "MAX_ATTEMPTS", 10_000)
def worker(n: int):
for _ in range(25):
ratelimit.record_failure("10.4.4.4")
threads = [threading.Thread(target=worker, args=(n,)) for n in range(8)]
for t in threads:
t.start()
for t in threads:
t.join()
assert ratelimit.get_status("10.4.4.4")["failures"] == 200
+123
View File
@@ -0,0 +1,123 @@
"""Tests de non-régression — verrous des stores JSON (ROADMAP #85 T10a).
Sans verrou, les read-modify-write concurrents (créations de partages,
révocations de jetons) perdent des mises à jour : deux threads lisent le
même état, chacun écrit le sien, la première écriture est écrasée. Ces
tests martèlent les stores depuis plusieurs threads et exigent un compte
exact à la fin (aucune mise à jour perdue).
"""
from __future__ import annotations
import threading
N_THREADS = 8
N_OPS = 25
def test_concurrent_share_creations_lose_nothing(tmp_path, monkeypatch):
"""N threads × N partages → le store final contient tout."""
from backend import share as share_mod
monkeypatch.setattr(share_mod, "SHARES_FILE", tmp_path / "shares.json")
errors: list[BaseException] = []
def worker(n: int):
try:
for i in range(N_OPS):
share_mod.create_share("V", f"doc-{n}-{i}.md", "tester")
except BaseException as e: # pragma: no cover - diagnostic
errors.append(e)
threads = [threading.Thread(target=worker, args=(n,)) for n in range(N_THREADS)]
for t in threads:
t.start()
for t in threads:
t.join()
assert not errors
assert len(share_mod.list_shares()) == N_THREADS * N_OPS
def test_concurrent_share_access_and_revoke(tmp_path, monkeypatch):
"""Accès + révocations concurrents : compteurs et suppressions cohérents."""
from backend import share as share_mod
monkeypatch.setattr(share_mod, "SHARES_FILE", tmp_path / "shares.json")
tokens = [share_mod.create_share("V", f"doc-{i}.md", "tester")["token"] for i in range(20)]
def worker(n: int):
share_mod.record_access(tokens[2 * n])
share_mod.record_access(tokens[2 * n + 1])
share_mod.revoke_share(tokens[2 * n])
threads = [threading.Thread(target=worker, args=(n,)) for n in range(10)]
for t in threads:
t.start()
for t in threads:
t.join()
# Les 10 tokens pairs ont été révoqués ; les impairs subsistent avec
# leurs compteurs d'accès (aucune écriture perdue).
remaining = {s["token"] for s in share_mod.list_shares()}
assert len(remaining) == 10
assert all(share_mod.get_share_by_token(t)["access_count"] >= 1 for t in remaining)
def test_concurrent_token_revokes_lose_nothing(tmp_path, monkeypatch):
"""N threads × N révocations → toutes les JTIs sont persistées."""
import json
from backend.auth import jwt_handler
revoked_file = tmp_path / "revoked_tokens.json"
monkeypatch.setattr(jwt_handler, "REVOKED_TOKENS_FILE", revoked_file)
monkeypatch.setattr(jwt_handler, "_revoked_map", {})
monkeypatch.setattr(jwt_handler, "_revoked_loaded", True)
errors: list[BaseException] = []
def worker(n: int):
try:
for i in range(N_OPS):
jwt_handler.revoke_token(f"jti-{n}-{i}")
except BaseException as e: # pragma: no cover - diagnostic
errors.append(e)
threads = [threading.Thread(target=worker, args=(n,)) for n in range(N_THREADS)]
for t in threads:
t.start()
for t in threads:
t.join()
assert not errors
assert len(jwt_handler._revoked_map) == N_THREADS * N_OPS
# Le fichier reflète l'état mémoire (aucune écriture perdue).
on_disk = json.loads(revoked_file.read_text(encoding="utf-8"))
assert len(on_disk) == N_THREADS * N_OPS
assert jwt_handler.is_token_revoked("jti-0-0")
assert not jwt_handler.is_token_revoked("jti-absent")
def test_concurrent_webhook_and_tool_key_writes(tmp_path, monkeypatch):
"""Créations de webhooks + clés d'outils concurrentes : rien de perdu."""
from backend import webhooks as wh_mod
from backend.tools import secrets as sec_mod
monkeypatch.setattr(wh_mod, "WEBHOOKS_FILE", tmp_path / "webhooks.json")
monkeypatch.setattr(wh_mod, "WEBHOOK_SECRETS_FILE", tmp_path / "webhook_secrets.json")
monkeypatch.setattr(sec_mod, "_keys_file", lambda: tmp_path / "api_keys.json")
def worker(n: int):
for i in range(N_OPS):
wh_mod.create_webhook(f"hook-{n}-{i}", "https://example.com/hook", ["file_created"])
sec_mod.set_tool_key("OBSIGATE_GITHUB_TOKEN", f"tok-{n}-{i}")
threads = [threading.Thread(target=worker, args=(n,)) for n in range(N_THREADS)]
for t in threads:
t.start()
for t in threads:
t.join()
assert len(wh_mod.get_webhooks()) == N_THREADS * N_OPS