feat(ai): durcissement phase F (#79) - rate limit, redaction, OpenAPI/MCP, E2E
This commit is contained in:
@@ -14,6 +14,19 @@ et [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|||||||
|
|
||||||
### Ajouté
|
### Ajouté
|
||||||
|
|
||||||
|
- **#79 Phase F — Durcissement & documentation (rate limiting, redaction, OpenAPI/MCP, E2E)** —
|
||||||
|
clôture de la feature #79. **Rate limiting** par jeton et par outil
|
||||||
|
(`backend/tools/ratelimit.py`, fenêtre glissante ; identité = JTI du jeton sinon id/username ;
|
||||||
|
`OBSIGATE_TOOL_RATE_LIMIT`, `OBSIGATE_TOOL_RATE_LIMIT_PER_TOOL`, `OBSIGATE_TOOL_RATE_WINDOW` ;
|
||||||
|
erreur `rate_limited`). **Quotas** `BOOKSLM_MAX_*` : `BOOKSLM_MAX_TOOL_CALLS` plafonne les appels
|
||||||
|
d'outils par run d'agent (`stopped="quota_exceeded"`) et `BOOKSLM_MAX_TOOL_READ_BYTES` plafonne
|
||||||
|
`read_file`. **Redaction** systématique de **tout** résultat d'outil avant retour au LLM
|
||||||
|
(`backend/tools/redaction.py`, récursif : diffs, extraits de recherche, lectures). **OpenAPI** :
|
||||||
|
tag `MCP` + injection du path `/mcp` (Streamable HTTP) dans le schéma ; nouveau
|
||||||
|
[`docs/MCP_GUIDE.md`](./docs/MCP_GUIDE.md) (config Claude Desktop/Cursor, tools/resources/prompts,
|
||||||
|
sécurité, variables, dépannage). Tests E2E : `tests/test_ai_e2e.py` (agent read→confirm→write,
|
||||||
|
quota, rate limit, redaction, flux MCP `read`→`propose`→`apply`→`resources/read`).
|
||||||
|
|
||||||
- **#79 Phase E — Serveur MCP (Streamable HTTP)** — nouvel endpoint `/mcp` exposant ObsiGate à des
|
- **#79 Phase E — Serveur MCP (Streamable HTTP)** — nouvel endpoint `/mcp` exposant ObsiGate à des
|
||||||
clients MCP externes (Claude Desktop, Cursor…) via le SDK Python `mcp==1.9.4`. Les deux fronts
|
clients MCP externes (Claude Desktop, Cursor…) via le SDK Python `mcp==1.9.4`. Les deux fronts
|
||||||
(assistant in-app et MCP) consomment la même couche d'outils. **Tools** : outils de lecture/recherche
|
(assistant in-app et MCP) consomment la même couche d'outils. **Tools** : outils de lecture/recherche
|
||||||
|
|||||||
@@ -55,6 +55,7 @@
|
|||||||
## ✨ Fonctionnalités
|
## ✨ Fonctionnalités
|
||||||
|
|
||||||
- **🤖 AI Editor intégré** — Éditeur CodeMirror 6 avec toolbar IA : amélioration, correction, traduction, génération, réécriture personnalisée, toolbox (liste, tableau, frontmatter, canvas) — multi-provider DeepSeek/OpenRouter/Gemini
|
- **🤖 AI Editor intégré** — Éditeur CodeMirror 6 avec toolbar IA : amélioration, correction, traduction, génération, réécriture personnalisée, toolbox (liste, tableau, frontmatter, canvas) — multi-provider DeepSeek/OpenRouter/Gemini
|
||||||
|
- **🧩 Serveur MCP & agent IA** — Serveur Model Context Protocol intégré (`/mcp`) et assistant avec function calling : lisez, cherchez et modifiez vos vaults depuis Claude Desktop, Cursor… avec confirmations two-step, permissions par vault, rate limiting et redaction des secrets ([guide](docs/MCP_GUIDE.md))
|
||||||
- **🗺️ Vue graphe interactive** — Canvas force-directed avec Barnes-Hut O(n log n), filtres (tag, type), profondeur, mode focus, historique de navigation ←→↑, export PNG, aperçu au survol (Ctrl+click)
|
- **🗺️ Vue graphe interactive** — Canvas force-directed avec Barnes-Hut O(n log n), filtres (tag, type), profondeur, mode focus, historique de navigation ←→↑, export PNG, aperçu au survol (Ctrl+click)
|
||||||
- **🗂️ Multi-vault** : Visualisez plusieurs vaults Obsidian simultanément
|
- **🗂️ Multi-vault** : Visualisez plusieurs vaults Obsidian simultanément
|
||||||
- **🌳 Navigation arborescente** : Parcourez vos dossiers et fichiers dans la sidebar
|
- **🌳 Navigation arborescente** : Parcourez vos dossiers et fichiers dans la sidebar
|
||||||
|
|||||||
@@ -48,6 +48,7 @@
|
|||||||
## ✨ Features
|
## ✨ Features
|
||||||
|
|
||||||
- **🤖 Integrated AI Editor** — CodeMirror 6 editor with AI toolbar: improve, correct, translate, generate, custom rewrite, toolbox (list, table, frontmatter, canvas) — multi-provider DeepSeek/OpenRouter/Gemini
|
- **🤖 Integrated AI Editor** — CodeMirror 6 editor with AI toolbar: improve, correct, translate, generate, custom rewrite, toolbox (list, table, frontmatter, canvas) — multi-provider DeepSeek/OpenRouter/Gemini
|
||||||
|
- **🧩 MCP Server & AI Agent** — Built-in Model Context Protocol server (`/mcp`) and tool-calling assistant: read, search and edit your vaults from Claude Desktop, Cursor… with two-step confirmations, per-vault permissions, rate limiting and secret redaction ([guide](docs/MCP_GUIDE.md))
|
||||||
- **🗺️ Interactive Graph View** — Canvas force-directed with Barnes-Hut O(n log n), filters (tag, type), depth, focus mode, navigation history ←→↑, export PNG, preview on hover (Ctrl+click)
|
- **🗺️ Interactive Graph View** — Canvas force-directed with Barnes-Hut O(n log n), filters (tag, type), depth, focus mode, navigation history ←→↑, export PNG, preview on hover (Ctrl+click)
|
||||||
- **🗂️ Multi-vault** : View multiple Obsidian vaults simultaneously
|
- **🗂️ Multi-vault** : View multiple Obsidian vaults simultaneously
|
||||||
- **🌳 Tree Navigation** : Browse your folders and files in the sidebar
|
- **🌳 Tree Navigation** : Browse your folders and files in the sidebar
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
|
import os
|
||||||
from collections.abc import Callable
|
from collections.abc import Callable
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from typing import Any
|
from typing import Any
|
||||||
@@ -35,11 +36,14 @@ logger = logging.getLogger("obsigate.agent.loop")
|
|||||||
DEFAULT_MAX_ITERATIONS = 10
|
DEFAULT_MAX_ITERATIONS = 10
|
||||||
# Cap the size of a tool result fed back to the model (chars).
|
# Cap the size of a tool result fed back to the model (chars).
|
||||||
MAX_TOOL_RESULT_CHARS = 100_000
|
MAX_TOOL_RESULT_CHARS = 100_000
|
||||||
|
# Quota: maximum tool calls executed per agent run (``BOOKSLM_MAX_TOOL_CALLS``).
|
||||||
|
DEFAULT_MAX_TOOL_CALLS = int(os.environ.get("BOOKSLM_MAX_TOOL_CALLS", "25"))
|
||||||
|
|
||||||
# Stopping reasons
|
# Stopping reasons
|
||||||
STOP_DONE = "done"
|
STOP_DONE = "done"
|
||||||
STOP_MAX_ITERATIONS = "max_iterations"
|
STOP_MAX_ITERATIONS = "max_iterations"
|
||||||
STOP_CONFIRMATION_REQUIRED = "confirmation_required"
|
STOP_CONFIRMATION_REQUIRED = "confirmation_required"
|
||||||
|
STOP_QUOTA_EXCEEDED = "quota_exceeded"
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
@@ -158,6 +162,7 @@ async def run_agent(
|
|||||||
llm: Callable[..., Any] | None = None,
|
llm: Callable[..., Any] | None = None,
|
||||||
tools: list[dict[str, Any]] | None = None,
|
tools: list[dict[str, Any]] | None = None,
|
||||||
max_iterations: int = DEFAULT_MAX_ITERATIONS,
|
max_iterations: int = DEFAULT_MAX_ITERATIONS,
|
||||||
|
max_tool_calls: int | None = None,
|
||||||
on_tool_call: Callable[[ToolCallRecord], None] | None = None,
|
on_tool_call: Callable[[ToolCallRecord], None] | None = None,
|
||||||
resume_messages: list[dict[str, Any]] | None = None,
|
resume_messages: list[dict[str, Any]] | None = None,
|
||||||
confirm_pending: dict[str, Any] | None = None,
|
confirm_pending: dict[str, Any] | None = None,
|
||||||
@@ -173,6 +178,8 @@ async def run_agent(
|
|||||||
tools: Tool schemas to expose. ``None`` exposes all in-app tools;
|
tools: Tool schemas to expose. ``None`` exposes all in-app tools;
|
||||||
pass ``[]`` to disable tool calling (plain chat).
|
pass ``[]`` to disable tool calling (plain chat).
|
||||||
max_iterations: Hard cap on LLM round-trips.
|
max_iterations: Hard cap on LLM round-trips.
|
||||||
|
max_tool_calls: Hard cap on the total number of executed tool calls
|
||||||
|
(quota, defaults to ``BOOKSLM_MAX_TOOL_CALLS``).
|
||||||
on_tool_call: Optional callback invoked after each executed tool call.
|
on_tool_call: Optional callback invoked after each executed tool call.
|
||||||
resume_messages: Conversation snapshot from a paused run (returned with
|
resume_messages: Conversation snapshot from a paused run (returned with
|
||||||
a ``confirmation`` event). When set, the loop resumes from it.
|
a ``confirmation`` event). When set, the loop resumes from it.
|
||||||
@@ -187,11 +194,20 @@ async def run_agent(
|
|||||||
llm = llm or provider_llm
|
llm = llm or provider_llm
|
||||||
if tools is None:
|
if tools is None:
|
||||||
tools = get_tool_schemas(scope=ToolScope.IN_APP)
|
tools = get_tool_schemas(scope=ToolScope.IN_APP)
|
||||||
|
quota = DEFAULT_MAX_TOOL_CALLS if max_tool_calls is None else max_tool_calls
|
||||||
|
|
||||||
convo = [dict(m) for m in (resume_messages if resume_messages is not None else messages)]
|
convo = [dict(m) for m in (resume_messages if resume_messages is not None else messages)]
|
||||||
executed: list[ToolCallRecord] = []
|
executed: list[ToolCallRecord] = []
|
||||||
|
|
||||||
if confirm_pending:
|
if confirm_pending:
|
||||||
|
if quota is not None and len(executed) >= quota:
|
||||||
|
return AgentResult(
|
||||||
|
content="",
|
||||||
|
messages=convo,
|
||||||
|
tool_calls=executed,
|
||||||
|
iterations=0,
|
||||||
|
stopped=STOP_QUOTA_EXCEEDED,
|
||||||
|
)
|
||||||
_execute_confirmed(ctx, confirm_pending, convo, executed, on_tool_call)
|
_execute_confirmed(ctx, confirm_pending, convo, executed, on_tool_call)
|
||||||
|
|
||||||
for iteration in range(1, max_iterations + 1):
|
for iteration in range(1, max_iterations + 1):
|
||||||
@@ -209,6 +225,15 @@ async def run_agent(
|
|||||||
convo.append(_assistant_tool_message(response.content, response.tool_calls))
|
convo.append(_assistant_tool_message(response.content, response.tool_calls))
|
||||||
|
|
||||||
for call in response.tool_calls:
|
for call in response.tool_calls:
|
||||||
|
if quota is not None and len(executed) >= quota:
|
||||||
|
logger.warning(f"Agent reached the tool-call quota ({quota})")
|
||||||
|
return AgentResult(
|
||||||
|
content=response.content or "",
|
||||||
|
messages=convo,
|
||||||
|
tool_calls=executed,
|
||||||
|
iterations=iteration,
|
||||||
|
stopped=STOP_QUOTA_EXCEEDED,
|
||||||
|
)
|
||||||
try:
|
try:
|
||||||
result = call_tool(call.name, ctx, call.arguments)
|
result = call_tool(call.name, ctx, call.arguments)
|
||||||
payload = result.data
|
payload = result.data
|
||||||
|
|||||||
@@ -63,6 +63,8 @@ def get_current_user(
|
|||||||
|
|
||||||
# Attach vault permissions from the token (snapshot at login time)
|
# Attach vault permissions from the token (snapshot at login time)
|
||||||
user["_token_vaults"] = payload.get("vaults", [])
|
user["_token_vaults"] = payload.get("vaults", [])
|
||||||
|
# Attach the token id for per-token rate limiting (AI tool layer).
|
||||||
|
user["_token_jti"] = payload.get("jti")
|
||||||
return user
|
return user
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -32,6 +32,7 @@ TAGS_METADATA: list[dict[str, str]] = [
|
|||||||
{"name": "Export", "description": "Export notes or whole vaults to HTML, Markdown bundle or ePub."},
|
{"name": "Export", "description": "Export notes or whole vaults to HTML, Markdown bundle or ePub."},
|
||||||
{"name": "AI", "description": "AI-powered editor actions, provider status and model discovery."},
|
{"name": "AI", "description": "AI-powered editor actions, provider status and model discovery."},
|
||||||
{"name": "BooksLM", "description": "Directory-scoped AI chat (NotebookLM-style) over a vault folder."},
|
{"name": "BooksLM", "description": "Directory-scoped AI chat (NotebookLM-style) over a vault folder."},
|
||||||
|
{"name": "MCP", "description": "Model Context Protocol server (Streamable HTTP) exposing the shared AI tool layer to external clients (Claude Desktop, Cursor…)."},
|
||||||
{"name": "Sharing", "description": "Create and manage public read-only share links for documents."},
|
{"name": "Sharing", "description": "Create and manage public read-only share links for documents."},
|
||||||
{"name": "Webhooks", "description": "HTTP callbacks signed with HMAC-SHA256 for file events."},
|
{"name": "Webhooks", "description": "HTTP callbacks signed with HMAC-SHA256 for file events."},
|
||||||
{"name": "Conflicts", "description": "Detect and resolve Syncthing sync-conflict files."},
|
{"name": "Conflicts", "description": "Detect and resolve Syncthing sync-conflict files."},
|
||||||
@@ -79,6 +80,7 @@ _TAG_RULES: list[tuple[re.Pattern[str], str]] = [
|
|||||||
(re.compile(r"^/api/push"), "Push"),
|
(re.compile(r"^/api/push"), "Push"),
|
||||||
(re.compile(r"^/api/ai/bookslm"), "BooksLM"),
|
(re.compile(r"^/api/ai/bookslm"), "BooksLM"),
|
||||||
(re.compile(r"^/api/ai"), "AI"),
|
(re.compile(r"^/api/ai"), "AI"),
|
||||||
|
(re.compile(r"^/mcp"), "MCP"),
|
||||||
(re.compile(r"^/api/config/ai-"), "AI"),
|
(re.compile(r"^/api/config/ai-"), "AI"),
|
||||||
(re.compile(r"^/api/share"), "Sharing"),
|
(re.compile(r"^/api/share"), "Sharing"),
|
||||||
(re.compile(r"^/api/shares"), "Sharing"),
|
(re.compile(r"^/api/shares"), "Sharing"),
|
||||||
@@ -141,6 +143,7 @@ _TAG_ALIASES: dict[str, str] = {
|
|||||||
"frontend": "Frontend",
|
"frontend": "Frontend",
|
||||||
"ai": "AI",
|
"ai": "AI",
|
||||||
"bookslm": "BooksLM",
|
"bookslm": "BooksLM",
|
||||||
|
"mcp": "MCP",
|
||||||
"pdf": "PDF",
|
"pdf": "PDF",
|
||||||
"bookmarks": "Bookmarks",
|
"bookmarks": "Bookmarks",
|
||||||
}
|
}
|
||||||
@@ -224,6 +227,67 @@ def _is_binary_operation(operation: dict[str, Any]) -> bool:
|
|||||||
return any(media.startswith(_BINARY_MEDIA) for media in content)
|
return any(media.startswith(_BINARY_MEDIA) for media in content)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# MCP endpoint (not a FastAPI route: custom ASGI mount) — documented manually
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
_MCP_DESCRIPTION = (
|
||||||
|
"**Model Context Protocol** server over Streamable HTTP (JSON-RPC 2.0). "
|
||||||
|
"Exposes the shared AI tool layer to external MCP clients (Claude Desktop, "
|
||||||
|
"Cursor…). Authentication uses `Authorization: Bearer <JWT>` (the same "
|
||||||
|
"token as the REST API).\n\n"
|
||||||
|
"Primitives: read/search **tools** directly; write/destructive tools as a "
|
||||||
|
"two-step `propose_<tool>` / `apply_<tool>` pair (signed, single-use "
|
||||||
|
"confirmation token); **resources** `vault://<name>` and "
|
||||||
|
"`vault://<name>/<path>` (read-only, secrets redacted); **prompts** "
|
||||||
|
"`summarize-directory`, `generate-note`, `find-related`.\n\n"
|
||||||
|
"See `docs/MCP_GUIDE.md` for client setup."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _inject_mcp_path(schema: dict[str, Any]) -> None:
|
||||||
|
"""Add the MCP Streamable HTTP endpoint to the schema (idempotent)."""
|
||||||
|
paths = schema.setdefault("paths", {})
|
||||||
|
if "/mcp" in paths:
|
||||||
|
return
|
||||||
|
paths["/mcp"] = {
|
||||||
|
"post": {
|
||||||
|
"tags": ["MCP"],
|
||||||
|
"summary": "MCP Streamable HTTP endpoint (JSON-RPC 2.0)",
|
||||||
|
"operationId": "mcp_streamable_http",
|
||||||
|
"description": _MCP_DESCRIPTION,
|
||||||
|
"requestBody": {
|
||||||
|
"required": True,
|
||||||
|
"content": {
|
||||||
|
"application/json": {
|
||||||
|
"example": {
|
||||||
|
"jsonrpc": "2.0",
|
||||||
|
"id": 1,
|
||||||
|
"method": "tools/list",
|
||||||
|
"params": {},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"responses": {
|
||||||
|
"200": {
|
||||||
|
"description": "JSON-RPC response (or 202 for notifications)",
|
||||||
|
"content": {
|
||||||
|
"application/json": {
|
||||||
|
"example": {
|
||||||
|
"jsonrpc": "2.0",
|
||||||
|
"id": 1,
|
||||||
|
"result": {"tools": []},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"security": [{"bearerAuth": []}],
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def enrich_openapi_schema(schema: dict[str, Any]) -> dict[str, Any]:
|
def enrich_openapi_schema(schema: dict[str, Any]) -> dict[str, Any]:
|
||||||
"""Enrich a FastAPI-generated OpenAPI schema in place and return it.
|
"""Enrich a FastAPI-generated OpenAPI schema in place and return it.
|
||||||
|
|
||||||
@@ -242,6 +306,8 @@ def enrich_openapi_schema(schema: dict[str, Any]) -> dict[str, Any]:
|
|||||||
}
|
}
|
||||||
schema["tags"] = TAGS_METADATA
|
schema["tags"] = TAGS_METADATA
|
||||||
|
|
||||||
|
_inject_mcp_path(schema)
|
||||||
|
|
||||||
components = schema.setdefault("components", {})
|
components = schema.setdefault("components", {})
|
||||||
security_schemes = components.setdefault("securitySchemes", {})
|
security_schemes = components.setdefault("securitySchemes", {})
|
||||||
security_schemes.setdefault("bearerAuth", {
|
security_schemes.setdefault("bearerAuth", {
|
||||||
|
|||||||
@@ -17,11 +17,22 @@ from backend.tools.context import (
|
|||||||
ToolMode,
|
ToolMode,
|
||||||
ToolNotFoundError,
|
ToolNotFoundError,
|
||||||
ToolPermissionError,
|
ToolPermissionError,
|
||||||
|
ToolRateLimitError,
|
||||||
ToolRisk,
|
ToolRisk,
|
||||||
ToolScope,
|
ToolScope,
|
||||||
ToolValidationError,
|
ToolValidationError,
|
||||||
resolve_safe_path,
|
resolve_safe_path,
|
||||||
)
|
)
|
||||||
|
from backend.tools.ratelimit import (
|
||||||
|
check_and_record as check_tool_rate_limit,
|
||||||
|
)
|
||||||
|
from backend.tools.ratelimit import (
|
||||||
|
get_status as get_tool_rate_limit_status,
|
||||||
|
)
|
||||||
|
from backend.tools.ratelimit import (
|
||||||
|
reset as reset_tool_rate_limit,
|
||||||
|
)
|
||||||
|
from backend.tools.redaction import redact_payload
|
||||||
from backend.tools.registry import (
|
from backend.tools.registry import (
|
||||||
ToolSpec,
|
ToolSpec,
|
||||||
call_tool,
|
call_tool,
|
||||||
@@ -39,15 +50,20 @@ __all__ = [
|
|||||||
"ToolMode",
|
"ToolMode",
|
||||||
"ToolNotFoundError",
|
"ToolNotFoundError",
|
||||||
"ToolPermissionError",
|
"ToolPermissionError",
|
||||||
|
"ToolRateLimitError",
|
||||||
"ToolResult",
|
"ToolResult",
|
||||||
"ToolRisk",
|
"ToolRisk",
|
||||||
"ToolScope",
|
"ToolScope",
|
||||||
"ToolSpec",
|
"ToolSpec",
|
||||||
"ToolValidationError",
|
"ToolValidationError",
|
||||||
"call_tool",
|
"call_tool",
|
||||||
|
"check_tool_rate_limit",
|
||||||
"get_tool",
|
"get_tool",
|
||||||
|
"get_tool_rate_limit_status",
|
||||||
"get_tool_schemas",
|
"get_tool_schemas",
|
||||||
"list_tools",
|
"list_tools",
|
||||||
|
"redact_payload",
|
||||||
|
"reset_tool_rate_limit",
|
||||||
"resolve_safe_path",
|
"resolve_safe_path",
|
||||||
"tool",
|
"tool",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -102,6 +102,21 @@ class ToolConfirmationRequired(ToolError):
|
|||||||
return payload
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
class ToolRateLimitError(ToolError):
|
||||||
|
"""Raised when an identity exceeds its tool-call rate limit (Phase F)."""
|
||||||
|
|
||||||
|
code = "rate_limited"
|
||||||
|
|
||||||
|
def __init__(self, tool: str, retry_after: int = 0, message: str | None = None):
|
||||||
|
super().__init__(
|
||||||
|
message or f"Rate limit exceeded for tool '{tool}'",
|
||||||
|
code="rate_limited",
|
||||||
|
details={"tool": tool, "retry_after": retry_after},
|
||||||
|
)
|
||||||
|
self.tool = tool
|
||||||
|
self.retry_after = retry_after
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class ToolContext:
|
class ToolContext:
|
||||||
"""Carries the caller identity and execution options for a tool call.
|
"""Carries the caller identity and execution options for a tool call.
|
||||||
@@ -126,6 +141,21 @@ class ToolContext:
|
|||||||
def username(self) -> str:
|
def username(self) -> str:
|
||||||
return self.user.get("username", "unknown")
|
return self.user.get("username", "unknown")
|
||||||
|
|
||||||
|
@property
|
||||||
|
def rate_limit_identity(self) -> str:
|
||||||
|
"""Identity used by the per-token/per-tool rate limiter.
|
||||||
|
|
||||||
|
Prefers the JWT id (``_token_jti``, attached by the auth middleware),
|
||||||
|
then an explicit ``token_id`` in :attr:`metadata`, then the user id and
|
||||||
|
finally the username. This yields per-token limiting when a token id is
|
||||||
|
available and per-account limiting otherwise.
|
||||||
|
"""
|
||||||
|
token_id = self.metadata.get("token_id") or self.user.get("_token_jti")
|
||||||
|
if token_id:
|
||||||
|
return f"token:{token_id}"
|
||||||
|
user_id = self.user.get("id") or self.user.get("username")
|
||||||
|
return f"user:{user_id or 'anonymous'}"
|
||||||
|
|
||||||
def has_vault_access(self, vault: str) -> bool:
|
def has_vault_access(self, vault: str) -> bool:
|
||||||
"""Return True if the caller may access *vault*."""
|
"""Return True if the caller may access *vault*."""
|
||||||
return bool(vault) and check_vault_access(vault, self.user)
|
return bool(vault) and check_vault_access(vault, self.user)
|
||||||
|
|||||||
@@ -0,0 +1,153 @@
|
|||||||
|
"""Rate limiting for the AI tool layer (Phase F).
|
||||||
|
|
||||||
|
Complements the IP-based login limiter (``backend.ratelimit``) with a
|
||||||
|
per-identity, per-tool sliding-window limiter applied to every tool call,
|
||||||
|
whether it originates from the in-app assistant or the MCP server.
|
||||||
|
|
||||||
|
Two counters are maintained per identity:
|
||||||
|
|
||||||
|
* a **global** counter (all tools combined), capped by
|
||||||
|
``OBSIGATE_TOOL_RATE_LIMIT``;
|
||||||
|
* a **per-tool** counter, capped by ``OBSIGATE_TOOL_RATE_LIMIT_PER_TOOL``
|
||||||
|
(defaults to the global cap) so a single expensive tool cannot consume the
|
||||||
|
whole budget.
|
||||||
|
|
||||||
|
The window length is ``OBSIGATE_TOOL_RATE_WINDOW`` seconds (default 60).
|
||||||
|
|
||||||
|
The identity is the token JTI when available (``_token_jti`` attached by the
|
||||||
|
auth middleware), otherwise the user id/username. This gives "per token / per
|
||||||
|
tool" limiting while remaining meaningful for anonymous/disabled-auth mode.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import time
|
||||||
|
from collections import deque
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
logger = logging.getLogger("obsigate.tools.ratelimit")
|
||||||
|
|
||||||
|
# --- Configuration (read at call time so tests can monkeypatch env) ---
|
||||||
|
DEFAULT_RATE_LIMIT = 60
|
||||||
|
DEFAULT_RATE_WINDOW = 60
|
||||||
|
|
||||||
|
|
||||||
|
def _env_int(name: str, default: int) -> int:
|
||||||
|
try:
|
||||||
|
return int(os.environ.get(name, str(default)))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return default
|
||||||
|
|
||||||
|
|
||||||
|
def _global_limit() -> int:
|
||||||
|
return _env_int("OBSIGATE_TOOL_RATE_LIMIT", DEFAULT_RATE_LIMIT)
|
||||||
|
|
||||||
|
|
||||||
|
def _per_tool_limit() -> int:
|
||||||
|
return _env_int("OBSIGATE_TOOL_RATE_LIMIT_PER_TOOL", _global_limit())
|
||||||
|
|
||||||
|
|
||||||
|
def _window() -> int:
|
||||||
|
return max(1, _env_int("OBSIGATE_TOOL_RATE_WINDOW", DEFAULT_RATE_WINDOW))
|
||||||
|
|
||||||
|
|
||||||
|
# --- In-memory store: {counter_key: deque[timestamp]} ---
|
||||||
|
_calls: dict[str, deque] = {}
|
||||||
|
|
||||||
|
|
||||||
|
def _identity_key(identity: str | None) -> str:
|
||||||
|
return identity or "anonymous"
|
||||||
|
|
||||||
|
|
||||||
|
def _counter_key(identity: str | None, tool: str | None) -> str:
|
||||||
|
base = _identity_key(identity)
|
||||||
|
return f"{base}\x00{tool}" if tool else base
|
||||||
|
|
||||||
|
|
||||||
|
def _prune(entries: deque, now: float, window: int) -> None:
|
||||||
|
cutoff = now - window
|
||||||
|
while entries and entries[0] <= cutoff:
|
||||||
|
entries.popleft()
|
||||||
|
|
||||||
|
|
||||||
|
def check_and_record(identity: str | None, tool: str, *, now: float | None = None) -> tuple[bool, int]:
|
||||||
|
"""Record one call and report whether it is allowed.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
identity: Rate-limit identity (token JTI or username).
|
||||||
|
tool: Tool name (used for the per-tool counter).
|
||||||
|
now: Optional timestamp override (tests).
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
``(allowed, retry_after)``. When ``allowed`` is False, ``retry_after``
|
||||||
|
is the number of seconds until the oldest call leaves the window.
|
||||||
|
"""
|
||||||
|
now = time.time() if now is None else now
|
||||||
|
window = _window()
|
||||||
|
global_limit = _global_limit()
|
||||||
|
tool_limit = _per_tool_limit()
|
||||||
|
|
||||||
|
global_entries = _calls.setdefault(_counter_key(identity, None), deque())
|
||||||
|
tool_entries = _calls.setdefault(_counter_key(identity, tool), deque())
|
||||||
|
_prune(global_entries, now, window)
|
||||||
|
_prune(tool_entries, now, window)
|
||||||
|
|
||||||
|
if len(global_entries) >= global_limit or len(tool_entries) >= tool_limit:
|
||||||
|
oldest = min(
|
||||||
|
global_entries[0] if len(global_entries) >= global_limit else now,
|
||||||
|
tool_entries[0] if len(tool_entries) >= tool_limit else now,
|
||||||
|
)
|
||||||
|
retry_after = max(1, int(oldest + window - now) + 1)
|
||||||
|
logger.warning(f"Tool rate limit exceeded for '{_identity_key(identity)}' on '{tool}'")
|
||||||
|
return False, retry_after
|
||||||
|
|
||||||
|
global_entries.append(now)
|
||||||
|
tool_entries.append(now)
|
||||||
|
return True, 0
|
||||||
|
|
||||||
|
|
||||||
|
def remaining(identity: str | None, tool: str | None = None) -> int:
|
||||||
|
"""Return the number of calls still allowed in the current window."""
|
||||||
|
now = time.time()
|
||||||
|
window = _window()
|
||||||
|
if tool:
|
||||||
|
entries = _calls.get(_counter_key(identity, tool))
|
||||||
|
if entries is None:
|
||||||
|
return _per_tool_limit()
|
||||||
|
_prune(entries, now, window)
|
||||||
|
return max(0, _per_tool_limit() - len(entries))
|
||||||
|
entries = _calls.get(_counter_key(identity, None))
|
||||||
|
if entries is None:
|
||||||
|
return _global_limit()
|
||||||
|
_prune(entries, now, window)
|
||||||
|
return max(0, _global_limit() - len(entries))
|
||||||
|
|
||||||
|
|
||||||
|
def reset(identity: str | None = None) -> None:
|
||||||
|
"""Clear rate-limit state (all identities, or a single one)."""
|
||||||
|
if identity is None:
|
||||||
|
_calls.clear()
|
||||||
|
return
|
||||||
|
prefix = f"{_identity_key(identity)}\x00"
|
||||||
|
for key in [k for k in _calls if k == _identity_key(identity) or k.startswith(prefix)]:
|
||||||
|
del _calls[key]
|
||||||
|
|
||||||
|
|
||||||
|
def get_status(identity: str | None = None) -> dict[str, Any]:
|
||||||
|
"""Diagnostic snapshot of the limiter (for tests / admin tooling)."""
|
||||||
|
if identity is None:
|
||||||
|
return {
|
||||||
|
"tracked_identities": len({k.split("\x00", 1)[0] for k in _calls}),
|
||||||
|
"global_limit": _global_limit(),
|
||||||
|
"per_tool_limit": _per_tool_limit(),
|
||||||
|
"window_seconds": _window(),
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
"identity": _identity_key(identity),
|
||||||
|
"remaining_global": remaining(identity),
|
||||||
|
"global_limit": _global_limit(),
|
||||||
|
"per_tool_limit": _per_tool_limit(),
|
||||||
|
"window_seconds": _window(),
|
||||||
|
}
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
"""Secret redaction for tool results (Phase F).
|
||||||
|
|
||||||
|
Tool results are fed back to the LLM (in-app agent loop) or returned to an
|
||||||
|
external MCP client, so they must never leak credentials. ``read_file`` and
|
||||||
|
``read_file_raw`` already redact through the shared file service, but other
|
||||||
|
tools (``diff_backup``, ``search_*``) return content that has not been through
|
||||||
|
the redactor. This module applies :func:`backend.secret_redactor.redact` to
|
||||||
|
every string in a tool payload, recursively.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from backend.secret_redactor import redact
|
||||||
|
|
||||||
|
_MAX_DEPTH = 12
|
||||||
|
|
||||||
|
|
||||||
|
def redact_payload(payload: Any, _depth: int = 0) -> Any:
|
||||||
|
"""Return a copy of *payload* with every string secret-redacted.
|
||||||
|
|
||||||
|
Walks dicts, lists and tuples; scalars are returned unchanged. Strings are
|
||||||
|
run through :func:`backend.secret_redactor.redact`. A recursion cap avoids
|
||||||
|
pathological/cyclic structures (tool results are JSON-serialisable, so
|
||||||
|
cycles should not occur, but the guard keeps this safe).
|
||||||
|
"""
|
||||||
|
if _depth > _MAX_DEPTH:
|
||||||
|
return payload
|
||||||
|
if isinstance(payload, str):
|
||||||
|
redacted, count = redact(payload)
|
||||||
|
return redacted if count else payload
|
||||||
|
if isinstance(payload, dict):
|
||||||
|
return {key: redact_payload(value, _depth + 1) for key, value in payload.items()}
|
||||||
|
if isinstance(payload, list):
|
||||||
|
return [redact_payload(item, _depth + 1) for item in payload]
|
||||||
|
if isinstance(payload, tuple):
|
||||||
|
return tuple(redact_payload(item, _depth + 1) for item in payload)
|
||||||
|
return payload
|
||||||
@@ -26,10 +26,13 @@ from backend.tools.context import (
|
|||||||
ToolError,
|
ToolError,
|
||||||
ToolNotFoundError,
|
ToolNotFoundError,
|
||||||
ToolPermissionError,
|
ToolPermissionError,
|
||||||
|
ToolRateLimitError,
|
||||||
ToolRisk,
|
ToolRisk,
|
||||||
ToolScope,
|
ToolScope,
|
||||||
ToolValidationError,
|
ToolValidationError,
|
||||||
)
|
)
|
||||||
|
from backend.tools.ratelimit import check_and_record
|
||||||
|
from backend.tools.redaction import redact_payload
|
||||||
from backend.tools.schemas import ToolResult
|
from backend.tools.schemas import ToolResult
|
||||||
|
|
||||||
logger = logging.getLogger("obsigate.tools.registry")
|
logger = logging.getLogger("obsigate.tools.registry")
|
||||||
@@ -186,6 +189,12 @@ def call_tool(
|
|||||||
_audit(ctx, spec, arguments, ok=False, error=error.code)
|
_audit(ctx, spec, arguments, ok=False, error=error.code)
|
||||||
raise error from e
|
raise error from e
|
||||||
|
|
||||||
|
allowed, retry_after = check_and_record(ctx.rate_limit_identity, spec.name)
|
||||||
|
if not allowed:
|
||||||
|
rate_error = ToolRateLimitError(spec.name, retry_after)
|
||||||
|
_audit(ctx, spec, arguments, ok=False, error=rate_error.code)
|
||||||
|
raise rate_error
|
||||||
|
|
||||||
vault = getattr(params, "vault", None)
|
vault = getattr(params, "vault", None)
|
||||||
if spec.requires_vault and vault:
|
if spec.requires_vault and vault:
|
||||||
try:
|
try:
|
||||||
@@ -223,4 +232,6 @@ def call_tool(
|
|||||||
duration_ms = round((time.perf_counter() - started) * 1000, 2)
|
duration_ms = round((time.perf_counter() - started) * 1000, 2)
|
||||||
logger.debug(f"Tool '{name}' ok in {duration_ms}ms")
|
logger.debug(f"Tool '{name}' ok in {duration_ms}ms")
|
||||||
_audit(ctx, spec, arguments, ok=True)
|
_audit(ctx, spec, arguments, ok=True)
|
||||||
return ToolResult(ok=True, data=data)
|
# Never forward raw secrets to the model (defense in depth: some tools —
|
||||||
|
# diffs, search snippets — return content that was not pre-redacted).
|
||||||
|
return ToolResult(ok=True, data=redact_payload(data))
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ confirmation mechanism (two-step propose/apply).
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
|
import os
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from backend.indexer import get_backlinks as _get_backlinks
|
from backend.indexer import get_backlinks as _get_backlinks
|
||||||
@@ -90,8 +91,9 @@ from backend.tools.schemas import (
|
|||||||
|
|
||||||
logger = logging.getLogger("obsigate.tools.service")
|
logger = logging.getLogger("obsigate.tools.service")
|
||||||
|
|
||||||
# Maximum file size returned by ``read_file`` (bytes).
|
# Maximum file size returned by ``read_file`` (bytes). Quota configurable via
|
||||||
TOOL_MAX_READ_BYTES = 200_000
|
# ``BOOKSLM_MAX_TOOL_READ_BYTES``.
|
||||||
|
TOOL_MAX_READ_BYTES = int(os.environ.get("BOOKSLM_MAX_TOOL_READ_BYTES", "200000"))
|
||||||
|
|
||||||
|
|
||||||
# ── C1. Vaults / navigation ────────────────────────────────────────────────
|
# ── C1. Vaults / navigation ────────────────────────────────────────────────
|
||||||
|
|||||||
@@ -252,9 +252,14 @@ Tout outil reçoit un `ToolContext` et applique **systématiquement** :
|
|||||||
1. `check_vault_access(vault, user)` — `backend/auth/middleware.py:87`
|
1. `check_vault_access(vault, user)` — `backend/auth/middleware.py:87`
|
||||||
2. `_resolve_safe_path(vault_root, path)` — `backend/main.py:867`
|
2. `_resolve_safe_path(vault_root, path)` — `backend/main.py:867`
|
||||||
3. Confirmation utilisateur pour `risk in (write, dangerous)` (UI in-app : carte Apply ; MCP : two-step `propose`/`apply`).
|
3. Confirmation utilisateur pour `risk in (write, dangerous)` (UI in-app : carte Apply ; MCP : two-step `propose`/`apply`).
|
||||||
4. Redaction des secrets avant tout envoi au LLM — `redact_file_content()`.
|
4. Redaction des secrets avant tout envoi au LLM — `redact_file_content()` puis
|
||||||
|
`backend/tools/redaction.py` sur **tout** résultat d'outil (diffs, extraits de recherche).
|
||||||
5. Journalisation dans l'audit (`data/audit.log`).
|
5. Journalisation dans l'audit (`data/audit.log`).
|
||||||
6. Rate limiting par token/outil ; quotas réutilisant `BOOKSLM_MAX_*`.
|
6. **Rate limiting par jeton/outil** (`backend/tools/ratelimit.py` :
|
||||||
|
`OBSIGATE_TOOL_RATE_LIMIT`, `OBSIGATE_TOOL_RATE_LIMIT_PER_TOOL`,
|
||||||
|
`OBSIGATE_TOOL_RATE_WINDOW`) et **quotas** réutilisant `BOOKSLM_MAX_*`
|
||||||
|
(`BOOKSLM_MAX_TOOL_CALLS` par run d'agent, `BOOKSLM_MAX_TOOL_READ_BYTES`
|
||||||
|
pour `read_file`).
|
||||||
|
|
||||||
### Confirmations MCP — décision : two-step `propose`/`apply`
|
### Confirmations MCP — décision : two-step `propose`/`apply`
|
||||||
|
|
||||||
@@ -299,7 +304,7 @@ Pour répondre au besoin de cibler un fournisseur/modèle sans dépendre uniquem
|
|||||||
| **0 — Fondations** | `backend/tools/` (registry, context, service, audit) + extraction des services métier + tests unitaires | Couche d'outils testable sans IA |
|
| **0 — Fondations** | `backend/tools/` (registry, context, service, audit) + extraction des services métier + tests unitaires | Couche d'outils testable sans IA |
|
||||||
| **1 — Function calling in-app** | Abstraction tool-calling multi-provider, agent loop, confirmations UI, SSE réel, outils de navigation | Assistant qui lit/cherche/lit/ouvre/modifie avec confirmation |
|
| **1 — Function calling in-app** | Abstraction tool-calling multi-provider, agent loop, confirmations UI, SSE réel, outils de navigation | Assistant qui lit/cherche/lit/ouvre/modifie avec confirmation |
|
||||||
| **2 — Serveur MCP** | `backend/mcp/server.py` (tools + resources + prompts), **Streamable HTTP** (`/mcp`, auth JWT), confirmation two-step | ObsiGate accessible comme serveur MCP (local + distant, multi-utilisateur) |
|
| **2 — Serveur MCP** | `backend/mcp/server.py` (tools + resources + prompts), **Streamable HTTP** (`/mcp`, auth JWT), confirmation two-step | ObsiGate accessible comme serveur MCP (local + distant, multi-utilisateur) |
|
||||||
| **3 — Durcissement** | Rate limiting, quotas, redaction, tests, doc OpenAPI/MCP | Observabilité et sécurité complètes |
|
| **3 — Durcissement** ✅ | Rate limiting (`backend/tools/ratelimit.py`), quotas `BOOKSLM_MAX_*`, redaction systématique des résultats (`backend/tools/redaction.py`), doc OpenAPI (tag/path MCP) + [guide MCP](./MCP_GUIDE.md), tests E2E | Observabilité et sécurité complètes |
|
||||||
|
|
||||||
Voir `docs/ROADMAP.md` (item dédié) pour le détail des activités.
|
Voir `docs/ROADMAP.md` (item dédié) pour le détail des activités.
|
||||||
|
|
||||||
@@ -330,6 +335,9 @@ Voir `docs/ROADMAP.md` (item dédié) pour le détail des activités.
|
|||||||
- `backend/agent/loop.py` — agent loop in-app (multi-étapes, LLM injectable)
|
- `backend/agent/loop.py` — agent loop in-app (multi-étapes, LLM injectable)
|
||||||
- `backend/mcp/server.py` — serveur MCP (Streamable HTTP `/mcp`, tools/resources/prompts)
|
- `backend/mcp/server.py` — serveur MCP (Streamable HTTP `/mcp`, tools/resources/prompts)
|
||||||
- `backend/mcp/confirmations.py` — jetons de confirmation signés (two-step, anti-rejeu)
|
- `backend/mcp/confirmations.py` — jetons de confirmation signés (two-step, anti-rejeu)
|
||||||
|
- `backend/tools/ratelimit.py` — rate limiting par jeton/outil (phase F)
|
||||||
|
- `backend/tools/redaction.py` — redaction récursive des résultats d'outils (phase F)
|
||||||
|
- `docs/MCP_GUIDE.md` — guide d'installation et d'utilisation des clients MCP
|
||||||
- `backend/bookslm.py`, `backend/bookslm_routes.py` — assistant contextuel (+ endpoint `/agent`)
|
- `backend/bookslm.py`, `backend/bookslm_routes.py` — assistant contextuel (+ endpoint `/agent`)
|
||||||
- `frontend/js/ai.js`, `frontend/js/bookslm.js` — UI IA
|
- `frontend/js/ai.js`, `frontend/js/bookslm.js` — UI IA
|
||||||
- `backend/auth/middleware.py` — permissions
|
- `backend/auth/middleware.py` — permissions
|
||||||
|
|||||||
@@ -0,0 +1,182 @@
|
|||||||
|
# ObsiGate — Guide MCP (Model Context Protocol)
|
||||||
|
|
||||||
|
> **Statut :** livré (#79 phase E + F) · **Dernière mise à jour :** 2026-09-11
|
||||||
|
> **Voir aussi :** [AI_ARCHITECTURE_GUIDE.md](./AI_ARCHITECTURE_GUIDE.md) ·
|
||||||
|
> [features/ai-tools-mcp.md](./features/ai-tools-mcp.md) · [ROADMAP.md](./ROADMAP.md)
|
||||||
|
|
||||||
|
ObsiGate expose ses vaults à des **clients MCP externes** (Claude Desktop, Cursor,
|
||||||
|
tout client compatible MCP) via un serveur **Streamable HTTP** monté sur `/mcp`.
|
||||||
|
Les outils sont les **mêmes** que ceux de l'assistant in-app : la couche
|
||||||
|
`backend/tools/` est la source unique de vérité.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Prérequis
|
||||||
|
|
||||||
|
1. Une instance ObsiGate accessible (locale ou distante).
|
||||||
|
2. Un **jeton JWT** valide (`Authorization: Bearer <token>`), obtenu via
|
||||||
|
`POST /api/auth/login` (ou une clé API). Le jeton porte les permissions par
|
||||||
|
vault de l'utilisateur — l'autorisation MCP réutilise `get_current_user`.
|
||||||
|
3. Si l'authentification est désactivée (`OBSIGATE_AUTH_ENABLED=false`), le
|
||||||
|
serveur MCP accepte un utilisateur anonyme disposant de tous les vaults.
|
||||||
|
|
||||||
|
> Le transport `stdio` n'est **pas** encore supporté ; utilisez le transport
|
||||||
|
> HTTP (un pont local type `mcp-remote` si votre client ne gère pas nativement
|
||||||
|
> le Streamable HTTP distant).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Endpoint & protocole
|
||||||
|
|
||||||
|
| Élément | Valeur |
|
||||||
|
|---|---|
|
||||||
|
| URL | `https://<obsigate>/mcp` |
|
||||||
|
| Transport | Streamable HTTP (`POST` JSON-RPC 2.0, `Accept: application/json, text/event-stream`) |
|
||||||
|
| Auth | `Authorization: Bearer <JWT>` |
|
||||||
|
| Protocole MCP | `2025-03-26` (négocié à l'`initialize`) |
|
||||||
|
| Réponses | JSON (`json_response=True`) |
|
||||||
|
|
||||||
|
Handshake minimal :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -sS https://obsigate.example/mcp \
|
||||||
|
-H "Authorization: Bearer $TOKEN" \
|
||||||
|
-H "Accept: application/json, text/event-stream" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{
|
||||||
|
"protocolVersion":"2025-03-26","capabilities":{},
|
||||||
|
"clientInfo":{"name":"curl","version":"1.0"}}}'
|
||||||
|
```
|
||||||
|
|
||||||
|
La réponse contient l'en-tête `Mcp-Session-Id` à réutiliser pour les appels
|
||||||
|
suivants (`tools/list`, `tools/call`, `resources/read`, …).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Configuration des clients
|
||||||
|
|
||||||
|
### Claude Desktop (via pont `mcp-remote`)
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"mcpServers": {
|
||||||
|
"obsigate": {
|
||||||
|
"command": "npx",
|
||||||
|
"args": [
|
||||||
|
"-y", "mcp-remote",
|
||||||
|
"https://obsigate.example/mcp",
|
||||||
|
"--header", "Authorization: Bearer ${OBSIGATE_TOKEN}"
|
||||||
|
],
|
||||||
|
"env": { "OBSIGATE_TOKEN": "eyJ..." }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Cursor
|
||||||
|
|
||||||
|
`.cursor/mcp.json` :
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"mcpServers": {
|
||||||
|
"obsigate": {
|
||||||
|
"url": "https://obsigate.example/mcp",
|
||||||
|
"headers": { "Authorization": "Bearer eyJ..." }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Primitives exposées
|
||||||
|
|
||||||
|
### 4.1 Tools
|
||||||
|
|
||||||
|
Les outils de **lecture/recherche** sont exposés directement. Les outils
|
||||||
|
**d'écriture/destructifs** sont exposés via une paire **two-step** :
|
||||||
|
`propose_<tool>` (aperçu + jeton de confirmation, aucune modification) puis
|
||||||
|
`apply_<tool>` (consomme le jeton et exécute).
|
||||||
|
|
||||||
|
| Catégorie | Outils |
|
||||||
|
|---|---|
|
||||||
|
| Vaults / navigation | `list_vaults`, `list_directory`, `list_all_files` |
|
||||||
|
| Lecture | `read_file`, `read_file_raw`, `get_backlinks`, `list_backups`, `diff_backup`, `get_graph` |
|
||||||
|
| Recherche | `search_fulltext`, `search_advanced`, `search_paths`, `list_tags`, `suggest_tags`, `list_recent` |
|
||||||
|
| Écriture (propose/apply) | `create_file`, `create_directory`, `edit_file`, `append_to_file`, `restore_backup` |
|
||||||
|
| Destructif (propose/apply) | `rename_file`, `rename_directory`, `move_path`, `replace_in_files`, `delete_file`, `delete_directory` |
|
||||||
|
|
||||||
|
Flux d'une mutation :
|
||||||
|
|
||||||
|
```text
|
||||||
|
1. tools/call { name: "propose_edit_file",
|
||||||
|
arguments: { vault, path, content } }
|
||||||
|
→ { tool, arguments, diff, confirmation_token, expires_in }
|
||||||
|
|
||||||
|
2. (l'utilisateur / l'agent valide)
|
||||||
|
|
||||||
|
3. tools/call { name: "apply_edit_file",
|
||||||
|
arguments: { confirmation_token } }
|
||||||
|
→ { ok: true, data: { ... } }
|
||||||
|
```
|
||||||
|
|
||||||
|
Le jeton est **signé (JWT), à usage unique et à durée de vie limitée**
|
||||||
|
(`OBSIGATE_MCP_CONFIRMATION_TTL`, défaut 300 s). Un rejeu renvoie
|
||||||
|
`token_reused`.
|
||||||
|
|
||||||
|
### 4.2 Resources
|
||||||
|
|
||||||
|
| URI | Contenu |
|
||||||
|
|---|---|
|
||||||
|
| `vault://<name>` | Vault accessible (métadonnées, nombre de fichiers) |
|
||||||
|
| `vault://<name>/<path>` | Contenu d'un fichier (lecture seule, **secrets redactés**) |
|
||||||
|
|
||||||
|
### 4.3 Prompts
|
||||||
|
|
||||||
|
`summarize-directory`, `generate-note`, `find-related`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Sécurité
|
||||||
|
|
||||||
|
- **Permissions par vault** : `check_vault_access` est appliqué à chaque outil
|
||||||
|
et chaque resource ; un utilisateur ne voit que ses vaults.
|
||||||
|
- **Anti path-traversal** : `resolve_safe_path` rejette tout chemin hors du vault.
|
||||||
|
- **Confirmation two-step** pour toute mutation (jeton signé, usage unique).
|
||||||
|
- **Toggle par vault** `aiDestructiveTools` (défaut : activé) : le désactiver
|
||||||
|
bloque rename/move/replace/delete tout en laissant create/edit/append.
|
||||||
|
- **Backup automatique** avant chaque opération destructive.
|
||||||
|
- **Rate limiting** : par jeton et par outil
|
||||||
|
(`OBSIGATE_TOOL_RATE_LIMIT`, `OBSIGATE_TOOL_RATE_LIMIT_PER_TOOL`,
|
||||||
|
`OBSIGATE_TOOL_RATE_WINDOW`). Une limite dépassée renvoie le code
|
||||||
|
`rate_limited`.
|
||||||
|
- **Redaction des secrets** : les résultats d'outils (lectures, diffs,
|
||||||
|
extraits de recherche) sont nettoyés avant tout retour au client.
|
||||||
|
- **Audit** : chaque appel est journalisé (`data/audit.log`, action
|
||||||
|
`ai_tool_call`) avec arguments sensibles résumés.
|
||||||
|
|
||||||
|
### Variables d'environnement
|
||||||
|
|
||||||
|
| Variable | Défaut | Rôle |
|
||||||
|
|---|---|---|
|
||||||
|
| `OBSIGATE_MCP_CONFIRMATION_TTL` | `300` | Durée de vie (s) des jetons de confirmation |
|
||||||
|
| `OBSIGATE_TOOL_RATE_LIMIT` | `60` | Appels d'outils max par identité et par fenêtre |
|
||||||
|
| `OBSIGATE_TOOL_RATE_LIMIT_PER_TOOL` | = global | Appels max par outil et par fenêtre |
|
||||||
|
| `OBSIGATE_TOOL_RATE_WINDOW` | `60` | Longueur de la fenêtre (s) |
|
||||||
|
| `BOOKSLM_MAX_TOOL_CALLS` | `25` | Quota d'appels d'outils par run d'agent |
|
||||||
|
| `BOOKSLM_MAX_TOOL_READ_BYTES` | `200000` | Taille max renvoyée par `read_file` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Dépannage
|
||||||
|
|
||||||
|
| Symptôme | Cause probable / remède |
|
||||||
|
|---|---|
|
||||||
|
| `401 Authentification requise` | En-tête `Authorization: Bearer` absent ou jeton expiré |
|
||||||
|
| `vault_access_denied` | Le jeton n'a pas accès à ce vault (`vaults` / `_token_vaults`) |
|
||||||
|
| `destructive_tools_disabled` | `aiDestructiveTools=false` pour ce vault |
|
||||||
|
| `confirmation_required` | Appeler d'abord `propose_<tool>` puis `apply_<tool>` |
|
||||||
|
| `token_reused` / `invalid_confirmation` | Jeton déjà consommé ou expiré → refaire un `propose_` |
|
||||||
|
| `rate_limited` | Quota dépassé ; respecter `retry_after` |
|
||||||
|
| Le client ne se connecte pas | Vérifier le transport Streamable HTTP / le pont `mcp-remote` |
|
||||||
+4
-15
@@ -40,18 +40,6 @@
|
|||||||
- [ ] Jumplist vaults récents dans le menu Démarrer (optionnel)
|
- [ ] Jumplist vaults récents dans le menu Démarrer (optionnel)
|
||||||
- [ ] 6 tests E2E **manuels** : installation, tray icon, notifications natives, association `.md`, auto-update, désinstallation
|
- [ ] 6 tests E2E **manuels** : installation, tray icon, notifications natives, association `.md`, auto-update, désinstallation
|
||||||
|
|
||||||
### 79. Assistant IA — Outils (function calling) & serveur MCP
|
|
||||||
|
|
||||||
- **Effort :** 10-15 jours | **Impact :** 🟡
|
|
||||||
- **Statut :** 🔵 Phase 0 + A2 + B1/B2/B3/B4/B5/B6/B7 + C + D + E + G livrés (2026-09-11). Détail complet : [features/ai-tools-mcp.md](./features/ai-tools-mcp.md)
|
|
||||||
- **Description :** Transformer l'assistant BooksLM en agent (lire, chercher, lister, ouvrir, modifier) via **function calling natif**, puis exposer ObsiGate à des **clients MCP externes** (Claude Desktop, Cursor…). Les deux fronts consomment une **couche d'outils partagée**.
|
|
||||||
- **Reste à faire :**
|
|
||||||
- [x] **C.** Catalogue lecture & recherche (vaults, read, backlinks, backups, search, tags)
|
|
||||||
- [x] **D.** Catalogue mutations (create/edit/append/rename/move/delete) + confirmations two-step
|
|
||||||
- [x] **E.** Serveur MCP (Streamable HTTP `/mcp`, auth Bearer JWT, `propose`/`apply`, toggle par vault)
|
|
||||||
- [ ] **F.** Durcissement (rate limiting, redaction secrets, OpenAPI + guide MCP, E2E)
|
|
||||||
- **Documentation :** [AI_ARCHITECTURE_GUIDE.md](./AI_ARCHITECTURE_GUIDE.md) (architecture, catalogue d'outils, sécurité, phases).
|
|
||||||
|
|
||||||
### 80. Assistant IA — Rendu Markdown, liens fichiers/paths & sessions
|
### 80. Assistant IA — Rendu Markdown, liens fichiers/paths & sessions
|
||||||
|
|
||||||
- **Effort :** 1-2 jours | **Impact :** 🟡
|
- **Effort :** 1-2 jours | **Impact :** 🟡
|
||||||
@@ -169,6 +157,7 @@
|
|||||||
| 72 | API publique documentée — OpenAPI 3.1 | 2.2.0 | [archive](./archive/COMPLETED_v1-v2.md) |
|
| 72 | API publique documentée — OpenAPI 3.1 | 2.2.0 | [archive](./archive/COMPLETED_v1-v2.md) |
|
||||||
| 76 | BooksLM — Console AI contextuelle par répertoire | 2.2.0 | [features/bookslm.md](./features/bookslm.md) |
|
| 76 | BooksLM — Console AI contextuelle par répertoire | 2.2.0 | [features/bookslm.md](./features/bookslm.md) |
|
||||||
| 78 | Éditeur Excalidraw | 2.2.0 | [features/excalidraw.md](./features/excalidraw.md) |
|
| 78 | Éditeur Excalidraw | 2.2.0 | [features/excalidraw.md](./features/excalidraw.md) |
|
||||||
|
| 79 | Assistant IA — Outils (function calling) & serveur MCP | 2.3.0 | [features/ai-tools-mcp.md](./features/ai-tools-mcp.md) |
|
||||||
| 77 | Application Desktop native — Tauri | 🔵 en cours | [features/desktop-tauri.md](./features/desktop-tauri.md) |
|
| 77 | Application Desktop native — Tauri | 🔵 en cours | [features/desktop-tauri.md](./features/desktop-tauri.md) |
|
||||||
| 80 | Assistant IA — Rendu Markdown, liens fichiers/paths & sessions | 🔵 en cours | [features/ai-assistant-ux.md](./features/ai-assistant-ux.md) |
|
| 80 | Assistant IA — Rendu Markdown, liens fichiers/paths & sessions | 🔵 en cours | [features/ai-assistant-ux.md](./features/ai-assistant-ux.md) |
|
||||||
|
|
||||||
@@ -178,11 +167,11 @@
|
|||||||
|
|
||||||
| Priorité | Items | Effort total estimé |
|
| Priorité | Items | Effort total estimé |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| ✅ Complété | #1 → #59, #61, #63–68, #71, #72, #74–76, #78 | ~82 jours réalisés |
|
| ✅ Complété | #1 → #59, #61, #63–68, #71, #72, #74–79 | ~92 jours réalisés |
|
||||||
| 🔵 P2 restant | #77 Desktop : signature code (optionnel), wizard 1er lancement (optionnel), 6 tests E2E **manuels** | ~1-2 jours |
|
| 🔵 P2 restant | #77 Desktop : signature code (optionnel), wizard 1er lancement (optionnel), 6 tests E2E **manuels** | ~1-2 jours |
|
||||||
| ⚪ P3 restant | #62 Collaboration Yjs (5-7j) · #79 Assistant IA outils + MCP (10-15j, Phase 0 + A2 + B + C + D + E + G livrés) · #80 Assistant IA UX (livré, en attente vérif) | ~15-22 jours |
|
| ⚪ P3 restant | #62 Collaboration Yjs (5-7j) · #80 Assistant IA UX (livré, en attente vérif) | ~5-7 jours |
|
||||||
| ⚪ P4 restant | #69 Mobile éditeur (2-3j) · #70 Sémantique (4-5j) · #73 Sync (6-8j) | 11-16 jours |
|
| ⚪ P4 restant | #69 Mobile éditeur (2-3j) · #70 Sémantique (4-5j) · #73 Sync (6-8j) | 11-16 jours |
|
||||||
| **Total restant** | **6 items + finitions** | **~27-39 jours** |
|
| **Total restant** | **5 items + finitions** | **~17-25 jours** |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# #79 — Assistant IA — Outils (function calling) & serveur MCP
|
# #79 — Assistant IA — Outils (function calling) & serveur MCP
|
||||||
|
|
||||||
> **Statut :** 🔵 En cours — Phase 0 + A2 + B1/B2/B3/B4/B5/B6/B7 + C + D + E + G livrés (2026-09-11)
|
> **Statut :** ✅ Livré — Phase 0 + A2 + B1/B2/B3/B4/B5/B6/B7 + C + D + E + F + G livrés (2026-09-11)
|
||||||
> **Effort :** 10-15 jours | **Impact :** 🟡
|
> **Effort :** 10-15 jours | **Impact :** 🟡
|
||||||
> **Références :** [Roadmap](../ROADMAP.md) · [Guide d'architecture IA](../AI_ARCHITECTURE_GUIDE.md)
|
> **Références :** [Roadmap](../ROADMAP.md) · [Guide d'architecture IA](../AI_ARCHITECTURE_GUIDE.md)
|
||||||
|
|
||||||
@@ -53,11 +53,25 @@
|
|||||||
matcher `/mcp` exact, démarrage paresseux du manager pour les tests). Dépendances ajoutées :
|
matcher `/mcp` exact, démarrage paresseux du manager pour les tests). Dépendances ajoutées :
|
||||||
`mcp==1.9.4`, `sse-starlette==2.1.3` (compatibles avec FastAPI 0.110 / starlette 0.37).
|
`mcp==1.9.4`, `sse-starlette==2.1.3` (compatibles avec FastAPI 0.110 / starlette 0.37).
|
||||||
|
|
||||||
## F. Durcissement & documentation (1 jour)
|
## F. Durcissement & documentation (1 jour) — ✅ livré (2026-09-11)
|
||||||
- [ ] **F1.** Rate limiting par token/outil + quotas (`BOOKSLM_MAX_*`)
|
- [x] **F1.** Rate limiting par token/outil + quotas — `backend/tools/ratelimit.py`
|
||||||
- [ ] **F2.** Redaction des secrets avant retour au LLM
|
(fenêtre glissante par identité **et** par outil ; identité = JTI du jeton
|
||||||
- [ ] **F3.** Documentation OpenAPI (`backend/openapi_docs.py`) + guide MCP
|
(`_token_jti`, ajouté par `backend/auth/middleware.py`) sinon id/username).
|
||||||
- [ ] **F4.** Tests E2E de bout en bout
|
Env : `OBSIGATE_TOOL_RATE_LIMIT` (60), `OBSIGATE_TOOL_RATE_LIMIT_PER_TOOL`,
|
||||||
|
`OBSIGATE_TOOL_RATE_WINDOW` (60 s) ; erreur `ToolRateLimitError` (`rate_limited`).
|
||||||
|
Quotas `BOOKSLM_MAX_*` : `BOOKSLM_MAX_TOOL_CALLS` (25) plafonne les appels d'outils
|
||||||
|
par run d'agent (`backend/agent/loop.py`, `stopped="quota_exceeded"`) et
|
||||||
|
`BOOKSLM_MAX_TOOL_READ_BYTES` (200000) plafonne `read_file`.
|
||||||
|
- [x] **F2.** Redaction des secrets avant retour au LLM — `backend/tools/redaction.py`
|
||||||
|
(`redact_payload`, récursif) appliqué à **tout** résultat d'outil dans
|
||||||
|
`call_tool` (couvre les diffs, extraits de recherche et lectures non pré-redactées).
|
||||||
|
- [x] **F3.** Documentation OpenAPI + guide MCP — `backend/openapi_docs.py` : tag `MCP`,
|
||||||
|
règle `/mcp`, injection du path `/mcp` (Streamable HTTP, JSON-RPC) dans le schéma ;
|
||||||
|
nouveau [`docs/MCP_GUIDE.md`](../MCP_GUIDE.md) (endpoint, auth, config Claude Desktop /
|
||||||
|
Cursor, tools/resources/prompts, sécurité, variables, dépannage).
|
||||||
|
- [x] **F4.** Tests E2E de bout en bout — `tests/test_ai_e2e.py` : agent in-app
|
||||||
|
read→confirmation→write, quota d'outils, rate limiting, redaction, et flux MCP complet
|
||||||
|
`read` → `propose_edit_file` → `apply_edit_file` → `resources/read`.
|
||||||
|
|
||||||
## G. Sélection fournisseur/modèle par défaut — ✅ livré
|
## G. Sélection fournisseur/modèle par défaut — ✅ livré
|
||||||
- [x] **G1.** Persistance `ai_default_provider` + `ai_default_models` dans `data/config.json` (`_DEFAULT_CONFIG`)
|
- [x] **G1.** Persistance `ai_default_provider` + `ai_default_models` dans `data/config.json` (`_DEFAULT_CONFIG`)
|
||||||
|
|||||||
@@ -12,6 +12,16 @@ from fastapi.testclient import TestClient
|
|||||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _reset_tool_ratelimit():
|
||||||
|
"""Isolate the per-identity tool rate limiter between tests."""
|
||||||
|
from backend.tools import ratelimit
|
||||||
|
|
||||||
|
ratelimit.reset()
|
||||||
|
yield
|
||||||
|
ratelimit.reset()
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
@pytest.fixture(autouse=True)
|
||||||
def _clean_env():
|
def _clean_env():
|
||||||
"""Ensure no vault env vars leak between tests — but preserve test vault config."""
|
"""Ensure no vault env vars leak between tests — but preserve test vault config."""
|
||||||
|
|||||||
@@ -0,0 +1,265 @@
|
|||||||
|
# tests/test_ai_e2e.py — End-to-end tests for the AI tool layer (Phase F)
|
||||||
|
"""End-to-end coverage of the AI assistant, combining the shared tool layer,
|
||||||
|
the in-app agent loop and the MCP server.
|
||||||
|
|
||||||
|
The LLM is scripted (no network) so the tests are deterministic; everything
|
||||||
|
else (permissions, services, confirmations, redaction, rate limiting, audit) is
|
||||||
|
the real production code path.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from backend.agent.loop import (
|
||||||
|
STOP_CONFIRMATION_REQUIRED,
|
||||||
|
STOP_DONE,
|
||||||
|
STOP_QUOTA_EXCEEDED,
|
||||||
|
run_agent,
|
||||||
|
)
|
||||||
|
from backend.ai_chat import LLMResponse, ToolCall
|
||||||
|
from backend.tools.api import (
|
||||||
|
ToolContext,
|
||||||
|
ToolRateLimitError,
|
||||||
|
ToolRisk,
|
||||||
|
call_tool,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _ctx(vaults=None) -> ToolContext:
|
||||||
|
return ToolContext(
|
||||||
|
user={"username": "e2e", "role": "admin", "vaults": vaults or ["*"]},
|
||||||
|
audit_enabled=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _vault_path() -> Path:
|
||||||
|
from backend.indexer import get_vault_data
|
||||||
|
|
||||||
|
return Path(get_vault_data("TestVault")["path"])
|
||||||
|
|
||||||
|
|
||||||
|
class ScriptedLLM:
|
||||||
|
"""Async LLM returning pre-scripted responses and recording invocations."""
|
||||||
|
|
||||||
|
def __init__(self, responses):
|
||||||
|
self.responses = list(responses)
|
||||||
|
self.calls = []
|
||||||
|
|
||||||
|
async def __call__(self, messages, tools):
|
||||||
|
self.calls.append({"messages": [dict(m) for m in messages], "tools": tools})
|
||||||
|
return self.responses.pop(0)
|
||||||
|
|
||||||
|
|
||||||
|
# ═══════════════════════════════════════════════════════════════════
|
||||||
|
# In-app agent — read then confirmed write, end to end
|
||||||
|
# ═══════════════════════════════════════════════════════════════════
|
||||||
|
|
||||||
|
|
||||||
|
class TestAgentEndToEnd:
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_read_then_confirmed_create_file(self, client):
|
||||||
|
target = _vault_path() / "agent-note.md"
|
||||||
|
assert not target.exists()
|
||||||
|
|
||||||
|
llm = ScriptedLLM([
|
||||||
|
LLMResponse(tool_calls=[ToolCall(
|
||||||
|
id="1", name="read_file", arguments={"vault": "TestVault", "path": "note1.md"},
|
||||||
|
)]),
|
||||||
|
LLMResponse(tool_calls=[ToolCall(
|
||||||
|
id="2",
|
||||||
|
name="create_file",
|
||||||
|
arguments={"vault": "TestVault", "path": "agent-note.md", "content": "# Agent\n"},
|
||||||
|
)]),
|
||||||
|
])
|
||||||
|
|
||||||
|
paused = await run_agent([{"role": "user", "content": "read then create"}], ctx=_ctx(), llm=llm)
|
||||||
|
assert paused.stopped == STOP_CONFIRMATION_REQUIRED
|
||||||
|
# The read ran, the write is pending confirmation.
|
||||||
|
assert len(paused.tool_calls) == 1
|
||||||
|
assert paused.tool_calls[0].name == "read_file"
|
||||||
|
assert paused.pending["error"]["tool"] == "create_file"
|
||||||
|
assert not target.exists()
|
||||||
|
|
||||||
|
llm2 = ScriptedLLM([LLMResponse(content="Note créée.")])
|
||||||
|
resumed = await run_agent(
|
||||||
|
[{"role": "user", "content": "read then create"}],
|
||||||
|
ctx=_ctx(),
|
||||||
|
llm=llm2,
|
||||||
|
resume_messages=paused.messages,
|
||||||
|
confirm_pending=paused.pending,
|
||||||
|
)
|
||||||
|
assert resumed.stopped == STOP_DONE
|
||||||
|
assert resumed.content == "Note créée."
|
||||||
|
assert target.read_text(encoding="utf-8") == "# Agent\n"
|
||||||
|
assert any(r.name == "create_file" and r.ok for r in resumed.tool_calls)
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_tool_call_quota_stops_run(self, client):
|
||||||
|
llm = ScriptedLLM([
|
||||||
|
LLMResponse(tool_calls=[
|
||||||
|
ToolCall(id="1", name="list_vaults", arguments={}),
|
||||||
|
ToolCall(id="2", name="list_vaults", arguments={}),
|
||||||
|
]),
|
||||||
|
])
|
||||||
|
result = await run_agent(
|
||||||
|
[{"role": "user", "content": "loop"}],
|
||||||
|
ctx=_ctx(),
|
||||||
|
llm=llm,
|
||||||
|
max_tool_calls=1,
|
||||||
|
)
|
||||||
|
assert result.stopped == STOP_QUOTA_EXCEEDED
|
||||||
|
assert len(result.tool_calls) == 1
|
||||||
|
|
||||||
|
|
||||||
|
# ═══════════════════════════════════════════════════════════════════
|
||||||
|
# Hardening — rate limiting & redaction
|
||||||
|
# ═══════════════════════════════════════════════════════════════════
|
||||||
|
|
||||||
|
|
||||||
|
class TestHardening:
|
||||||
|
def test_rate_limit_blocks_excess_calls(self, client, monkeypatch):
|
||||||
|
monkeypatch.setenv("OBSIGATE_TOOL_RATE_LIMIT", "1")
|
||||||
|
monkeypatch.setenv("OBSIGATE_TOOL_RATE_LIMIT_PER_TOOL", "1")
|
||||||
|
|
||||||
|
first = call_tool("list_vaults", _ctx(), {})
|
||||||
|
assert first.ok
|
||||||
|
|
||||||
|
with pytest.raises(ToolRateLimitError) as exc:
|
||||||
|
call_tool("list_vaults", _ctx(), {})
|
||||||
|
assert exc.value.code == "rate_limited"
|
||||||
|
assert exc.value.retry_after >= 1
|
||||||
|
|
||||||
|
def test_rate_limit_is_per_tool(self, client, monkeypatch):
|
||||||
|
monkeypatch.setenv("OBSIGATE_TOOL_RATE_LIMIT", "100")
|
||||||
|
monkeypatch.setenv("OBSIGATE_TOOL_RATE_LIMIT_PER_TOOL", "1")
|
||||||
|
|
||||||
|
call_tool("list_vaults", _ctx(), {})
|
||||||
|
# A different tool still has budget.
|
||||||
|
assert call_tool("list_tags", _ctx(), {}).ok
|
||||||
|
with pytest.raises(ToolRateLimitError):
|
||||||
|
call_tool("list_vaults", _ctx(), {})
|
||||||
|
|
||||||
|
def test_tool_results_are_redacted(self, client, monkeypatch):
|
||||||
|
from backend.tools import registry
|
||||||
|
from backend.tools.registry import ToolSpec
|
||||||
|
from backend.tools.schemas import ListVaultsInput
|
||||||
|
|
||||||
|
fake_jwt = "eyJ" + "a" * 30 + "." + "b" * 30 + "." + "c" * 30
|
||||||
|
spec = ToolSpec(
|
||||||
|
name="_leak",
|
||||||
|
description="returns a secret for tests",
|
||||||
|
input_model=ListVaultsInput,
|
||||||
|
handler=lambda ctx, params: {"content": f"token: {fake_jwt}", "nested": [fake_jwt]},
|
||||||
|
risk=ToolRisk.READ,
|
||||||
|
)
|
||||||
|
monkeypatch.setitem(registry._REGISTRY, "_leak", spec)
|
||||||
|
|
||||||
|
result = call_tool("_leak", _ctx(), {})
|
||||||
|
assert fake_jwt not in json.dumps(result.data)
|
||||||
|
assert "[JWT MASQUÉ]" in result.data["content"]
|
||||||
|
assert "[JWT MASQUÉ]" in result.data["nested"][0]
|
||||||
|
|
||||||
|
|
||||||
|
# ═══════════════════════════════════════════════════════════════════
|
||||||
|
# MCP — full propose/apply flow over Streamable HTTP
|
||||||
|
# ═══════════════════════════════════════════════════════════════════
|
||||||
|
|
||||||
|
ACCEPT = "application/json, text/event-stream"
|
||||||
|
PROTOCOL = "2025-03-26"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def mcp_client(app_with_vault, monkeypatch):
|
||||||
|
from backend import main
|
||||||
|
|
||||||
|
async def _noop_build(*args, **kwargs):
|
||||||
|
return None
|
||||||
|
|
||||||
|
monkeypatch.setattr(main, "build_index", _noop_build)
|
||||||
|
monkeypatch.setattr(main, "init_inverted_index", lambda: None)
|
||||||
|
|
||||||
|
main.mcp_app._manager = None
|
||||||
|
main.mcp_app._run_task = None
|
||||||
|
main.mcp_app._start_lock = None
|
||||||
|
|
||||||
|
with TestClient(main.app) as client:
|
||||||
|
yield client
|
||||||
|
|
||||||
|
|
||||||
|
def _mcp_post(client, payload, session=None):
|
||||||
|
headers = {"Accept": ACCEPT, "Content-Type": "application/json"}
|
||||||
|
if session:
|
||||||
|
headers["Mcp-Session-Id"] = session
|
||||||
|
return client.post("/mcp", content=json.dumps(payload), headers=headers)
|
||||||
|
|
||||||
|
|
||||||
|
def _mcp_call(client, session, method, params=None, req_id=2):
|
||||||
|
resp = _mcp_post(
|
||||||
|
client,
|
||||||
|
{"jsonrpc": "2.0", "id": req_id, "method": method, "params": params or {}},
|
||||||
|
session=session,
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200, resp.text
|
||||||
|
return resp.json()
|
||||||
|
|
||||||
|
|
||||||
|
def _text_json(result):
|
||||||
|
return json.loads(result["content"][0]["text"])
|
||||||
|
|
||||||
|
|
||||||
|
class TestMcpEndToEnd:
|
||||||
|
def test_read_propose_apply(self, mcp_client):
|
||||||
|
init = _mcp_post(
|
||||||
|
mcp_client,
|
||||||
|
{
|
||||||
|
"jsonrpc": "2.0",
|
||||||
|
"id": 1,
|
||||||
|
"method": "initialize",
|
||||||
|
"params": {
|
||||||
|
"protocolVersion": PROTOCOL,
|
||||||
|
"capabilities": {},
|
||||||
|
"clientInfo": {"name": "pytest-e2e", "version": "1.0"},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assert init.status_code == 200
|
||||||
|
session = init.headers["mcp-session-id"]
|
||||||
|
_mcp_post(mcp_client, {"jsonrpc": "2.0", "method": "notifications/initialized"}, session)
|
||||||
|
|
||||||
|
# 1. Read the current content.
|
||||||
|
read = _mcp_call(
|
||||||
|
mcp_client, session, "tools/call",
|
||||||
|
{"name": "read_file", "arguments": {"vault": "TestVault", "path": "note1.md"}},
|
||||||
|
)
|
||||||
|
assert "Python" in _text_json(read["result"])["data"]["content"]
|
||||||
|
|
||||||
|
# 2. Propose an edit (no change yet).
|
||||||
|
proposed = _mcp_call(
|
||||||
|
mcp_client, session, "tools/call",
|
||||||
|
{
|
||||||
|
"name": "propose_edit_file",
|
||||||
|
"arguments": {"vault": "TestVault", "path": "note1.md", "content": "# E2E\n"},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
payload = _text_json(proposed["result"])
|
||||||
|
token = payload["confirmation_token"]
|
||||||
|
assert payload["tool"] == "edit_file"
|
||||||
|
assert "diff" in payload
|
||||||
|
|
||||||
|
# 3. Apply it.
|
||||||
|
applied = _mcp_call(
|
||||||
|
mcp_client, session, "tools/call",
|
||||||
|
{"name": "apply_edit_file", "arguments": {"confirmation_token": token}},
|
||||||
|
req_id=3,
|
||||||
|
)
|
||||||
|
assert _text_json(applied["result"])["ok"] is True
|
||||||
|
assert (_vault_path() / "note1.md").read_text(encoding="utf-8") == "# E2E\n"
|
||||||
|
|
||||||
|
# 4. Resource read reflects the new content.
|
||||||
|
resource = _mcp_call(
|
||||||
|
mcp_client, session, "resources/read", {"uri": "vault://TestVault/note1.md"}, req_id=4
|
||||||
|
)
|
||||||
|
assert "E2E" in resource["result"]["contents"][0]["text"]
|
||||||
Reference in New Issue
Block a user