Files

T

bruno 3f52b56251 refactor: extraction des modules xlsx, lien dashboard-grille et inspecteur redimensionnable #154

2026-09-29 15:25:38 -04:00

47 KiB

Raw Blame History

ObsiGate

Ultra-light web gateway for your Obsidian vaults — Access, browse, and search all your Obsidian notes from any device via a modern, responsive web interface.

ObsiGate web interface: multi-vault sidebar, global search, dashboard stats and shortcuts.

📚 Guides

Step-by-step user guides live in docs/GUIDES/:

Guide	What it covers
🚀 Getting Started	First run, interface, navigation, vaults, shortcuts
🔍 Search, PDF, Excel & Excalidraw	Query syntax, semantic search, PDF/Excel viewers, diagrams
🤖 AI Assistant & Forge	Providers, AI editor, BooksLM, Forge, `@` / `/` commands
📝 Editing & Collaboration	Simultaneous editing, remote cursors, persistence
📱 PWA & Offline	Install as an app, offline cache, sync queue, push
🔌 REST API	Authentication, API keys, endpoints, `curl` examples, SSE
🧩 MCP Server	Connect Claude Desktop, Cursor, Cline… to your vaults
🔒 Auth & Security	Users, MFA, per-vault permissions, hardening
🐳 Docker Deployment	`docker-compose`, volumes, reverse proxy, updates
🖥️ Desktop (Tauri)	Install, first run, build from source, troubleshooting

All guides are currently written in French. See the full index: docs/GUIDES/README.md.

📋 Table of Contents

✨ Features
📚 Guides
🚀 Prerequisites
⚡ Quick Installation
⚙️ Detailed Configuration
🌍 Environment Variables
🔒 Authentication
➕ Adding a New Vault
🔨 Build & Deployment with build.sh
🖼️ Obsidian Image Rendering
🖥️ Desktop (Tauri) — Native Application
📖 Usage
👥 Real-time Collaboration
🔌 API
🔍 Advanced Search
🛡️ Security
⚡ Performance
🔧 Troubleshooting
🏗️ Tech Stack
🏠 Architecture
📝 Development
📄 License
🤝 Support
📝 Changelog

✨ Features

🤖 Integrated AI Editor — CodeMirror 6 editor with AI toolbar: improve, correct, translate, generate, custom rewrite, toolbox (list, table, frontmatter, canvas) — multi-provider DeepSeek/OpenRouter/Gemini
🧩 MCP Server & AI Agent — Built-in Model Context Protocol server (/mcp) and tool-calling assistant: read, search and edit your vaults from Claude Desktop, Cursor… with two-step confirmations, per-vault permissions, rate limiting and secret redaction (guide)
👥 Real-time Collaboration — Simultaneous editing of the same document (Yjs/CRDT): colored remote cursors, presence indicator, conflict-free merge, automatic reconnection and server-side persistence (details)
📖 Built-in User Guide — Complete FR/EN help from the Options menu: interface, navigation, search, files, AI, security, API & integrations (OpenAPI, MCP), offline, collaboration, desktop, plus an Architecture section with a Mermaid diagram; downloadable as Markdown and PDF in the current language (details)
📱 Native Mobile Editor — Touch-optimised editing: floating Markdown toolbar (bold/italic/code/list/link), persistent Paste button (iOS workaround), pinch-zoom font & adjustable height, swipe shortcuts (backlinks / table of contents) and a full-screen reading mode with page navigation (details)
🗺️ Interactive Graph View — Canvas force-directed with Barnes-Hut O(n log n), filters (tag, type), depth, focus mode, navigation history ←→↑, export PNG, preview on hover (Ctrl+click)
🗂️ Multi-vault : View multiple Obsidian vaults simultaneously
🌳 Tree Navigation : Browse your folders and files in the sidebar
🔍 Advanced Search : TF-IDF search engine with French stemming, accent normalization, highlighted snippets, facets, pagination, and sorting — plus an optional semantic search (embeddings via all-MiniLM-L6-v2, hybrid TF-IDF + RRF fusion) toggled with ~ (details)
💡 Smart Autocomplete : Suggestions for files, tags, and history with keyboard navigation
🧩 Query Syntax : Operators tag:, #, vault:, title:, path:, ext: with visual chips
📜 Search History : Persisted in localStorage (max 50 entries, LIFO, deduplicated)
🏷️ Tag Cloud : Filtering by tags extracted from YAML frontmatters
🔗 Wikilinks : [[internal links]] from Obsidian are clickable
🖼️ Obsidian Images : Full support for all Obsidian image syntaxes with intelligent resolution
🎬 Audio & video : Built-in HTML5 players (.mp3 .wav .flac .mp4 .webm…) with HTTP Range streaming (play, seek, fullscreen) and persistent playback (floating mini-player / mini video window, return to media or stop anytime, lock-screen controls via Media Session), falling back to download when the format is not playable in the browser
🎨 Excalidraw Diagrams : Native viewer/editor for .excalidraw and .excalidraw.md files (sandboxed iframe, autosave, dark/light theme, diagram text indexed for search)
📊 Excel Spreadsheets : .xlsx and .xlsm files open in a dedicated viewer — one table per sheet with tabs, A1 headers and inline cell editing (PUT /api/file/{vault}/xlsx/save, automatic backup, atomic write), plus download of the original file. The viewer renders fonts, colors, merged cells and frozen panes, offers keyboard navigation, a formula bar, sort/filter/find, CSV export, sheet & row/column structure editing and a workbook dashboard (named ranges, charts/pivot detection, per-sheet stats); .csv is edited in the same grid (RFC 4180) while .xls and .ods open read-only. Workbooks holding elements ObsiGate cannot preserve (cached values, slicers, form controls, signature…) show a warning and ask for confirmation before saving; a value starting with = or @ is stored as text unless the f(x) toggle is enabled. The AI assistant can list sheets, dump a bounded table to its context, update cells and append rows
🎨 Syntax Highlight : Syntax highlighting for code blocks
🌓 Light/Dark Theme : Toggle persisted in localStorage
📡 Real-time Sync : Automatic file monitoring via watchdog with incremental index updates
📡 Server-Sent Events : SSE notifications for index changes with automatic reconnection
➕ Dynamic Vault Management : Add/remove vaults via API without restart
🐳 Multi-platform Docker : linux/amd64, linux/arm64, linux/arm/v7, linux/386
🔒 Authentication : JWT + Argon2id, persistent sessions, per-vault access control
🛡️ Security : Rate limiting, audit log, automatic backup, secret redaction, CSP headers, path traversal protection, non-root user
⚡ Performance : GZip compression, Cache-Control immutable, incremental inverted index, search with no disk I/O
❤️ Healthcheck : /api/health endpoint integrated for Docker and monitoring

🚀 Prerequisites

System Requirements

Docker >= 20.10
docker-compose >= 2.0
Disk space : ~200MB for the Docker image

Supported Systems

Linux (Ubuntu, Debian, CentOS, etc.)
macOS (Intel and Apple Silicon)
Windows (with Docker Desktop)
Docker-compatible NAS (Synology, QNAP, etc.)

⚡ Quick Installation

1. Clone the Repository

git clone https://git.dracodev.net/Projets/ObsiGate.git
cd ObsiGate

2. Configure Your Vaults and Secrets

Edit the docker-compose.yml file to add your Obsidian vaults:

volumes:
  - /absolute/path/to/your/vault:/vaults/YourVaultName:ro

Important

: The path must be absolute and the volume must be read-only (:ro)

Create your .env file for authentication and secrets:

cp .env.example .env
# Edit .env to configure your passwords and options

Never commit .env! It is in .gitignore. Use .env.example as a reference.

3. Launch the Application

# Make the script executable (once)
chmod +x build.sh

# Build + deploy in one command
./build.sh

That's it! build.sh checks Docker, validates your volumes, builds the image, and starts the container automatically.

Useful options: ./build.sh --help for help, ./build.sh --cache for a faster rebuild, ./build.sh --build-only to build without starting.

4. Access the Interface

Open your browser at: http://localhost:2020

⚙️ Detailed Configuration

Step 1: Prepare Your Vaults

Locate your Obsidian vaults on your system
Note the absolute paths to each .obsidian folder
Check permissions : Docker must be able to read these folders

Step 2: Configure docker-compose.yml

Here's a complete example:

services:
  obsigate:
    build:
      context: .
    image: obsigate:latest
    container_name: obsigate
    restart: unless-stopped
    ports:
      - "2020:8080"  # Local port 2020 → Container port 8080
    volumes:
      # Vault examples (adapt to your paths)
      - /home/user/Documents/Obsidian-Recipes:/vaults/Recipes:ro
      - /home/user/Documents/Obsidian-IT:/vaults/IT:ro
      - /home/user/Documents/Obsidian-Personal:/vaults/Personal:ro
      # Auth data persistence
      - ./data:/app/data
    environment:
      # Vault configuration
      - VAULT_1_NAME=Recipes
      - VAULT_1_PATH=/vaults/Recipes
      - VAULT_2_NAME=IT
      - VAULT_2_PATH=/vaults/IT
      - VAULT_3_NAME=Personal
      - VAULT_3_PATH=/vaults/Personal
      # Auth (secrets are in .env)
      - OBSIGATE_AUTH_ENABLED=true
      - OBSIGATE_ADMIN_USER=admin
    env_file:
      - .env  # Contains OBSIGATE_ADMIN_PASSWORD and other secrets

Step 3: Build & Deploy

# Use the automated build script (recommended)
chmod +x build.sh
./build.sh

The build.sh script handles everything: prerequisite checks, volume validation, image building, and startup.

Manual alternative:

docker compose build --no-cache
docker compose up -d

Docker Compatibility : The image uses the minimal uvicorn variant and a compatible fastapi version (0.110.3) to avoid certain optional native dependencies (watchfiles, uvloop, httptools, fastapi-cli, etc.) that may fail to build on some platforms like Alpine, ARM, or i386.

🌍 Environment Variables

Vaults are configured using pairs of VAULT_N_NAME / VAULT_N_PATH variables (N = 1, 2, 3…):

Variable	Description	Example
`VAULT_1_NAME`	Display name of the vault	`Recipes`
`VAULT_1_PATH`	Path inside the container	`/vaults/Obsidian-RECIPES`
`VAULT_1_ATTACHMENTS_PATH`	Relative path to the attachments folder (optional)	`06_Toolbox/6.2_Attachments`
`VAULT_1_SCAN_ATTACHMENTS`	Enable image scanning on startup (optional, default: true)	`true`
`VAULT_2_NAME`	Display name of the vault	`IT`
`VAULT_2_PATH`	Path inside the container	`/vaults/Obsidian_IT`

Naming rules:

Use only letters, numbers, and hyphens
No spaces or special characters
The name must match the path inside the container

🔒 Authentication

Disabled by default — Compatible with all existing installations.

ObsiGate supports an optional authentication system based on JWT + Argon2id with per-vault access control.

Enable Authentication

Copy the .env template :
```
cp .env.example .env
```

Edit .env and uncomment/configure the variables :

OBSIGATE_AUTH_ENABLED=true
OBSIGATE_ADMIN_USER=admin
OBSIGATE_ADMIN_PASSWORD=your_password  # Leave empty = auto-generated (see logs)
# OBSIGATE_SECURE_COOKIES=false              # true if behind HTTPS

In docker-compose.yml, make sure you have :
```
env_file:
  - .env
```

Never put passwords in docker-compose.yml! Always use .env.

First Startup

If no user exists, ObsiGate automatically creates an admin account and displays the password in the logs :

docker-compose logs obsigate | grep -A4 "FIRST"

============================================================
FIRST STARTUP — Admin account created automatically
  Username : admin
  Password : xK9mQ3pLr7wN2jT5
CHANGE THIS PASSWORD on first login!
============================================================

User Management via CLI

# Create a user
docker exec obsigate python backend/create_admin.py create alice MyPassword --role user --vaults Recipes IT

# Create an admin with full access
docker exec obsigate python backend/create_admin.py create bob SecretPass --role admin --vaults "*"

# List users
docker exec obsigate python backend/create_admin.py list

# Delete a user
docker exec obsigate python backend/create_admin.py delete alice

Admin Interface

When an admin account is logged in, a 🛡️ icon appears in the header. Clicking it opens the admin panel allowing you to:

List all users
Create / edit / delete users
Assign accessible vaults per user
Enable/disable accounts

Per-Vault Access Control

vaults value	Access
`["*"]`	All vaults (including future ones) — admin default
`["Recipes", "IT"]`	Only these vaults
`[]`	No access

Auth Environment Variables

Variable	Description	Default
`OBSIGATE_AUTH_ENABLED`	Enable authentication	`false`
`OBSIGATE_ADMIN_USER`	Auto-created admin username	`admin`
`OBSIGATE_ADMIN_PASSWORD`	Admin password (empty = auto-generated)	(auto)
`OBSIGATE_SECURE_COOKIES`	`Secure` cookie (HTTPS only)	`false`
`OBSIGATE_ACCESS_TOKEN_TTL`	JWT token lifetime (seconds)	`3600`
`OBSIGATE_REFRESH_TOKEN_TTL`	Refresh token lifetime (seconds)	`2592000`
`OBSIGATE_LOGIN_MAX_ATTEMPTS`	Max login attempts per IP	`10`
`OBSIGATE_ACCOUNT_MAX_ATTEMPTS`	Max login attempts per account	`10`
`OBSIGATE_LOGIN_WINDOW_SECONDS`	Rate limiting window (seconds)	`900`
`OBSIGATE_TRUST_PROXY`	Trust `X-Forwarded-For` for the client IP (reverse proxy)	`false`
`OBSIGATE_WEBHOOK_ALLOW_HTTP`	Allow non-HTTPS webhook targets	`false`
`OBSIGATE_WEBHOOK_ALLOW_PRIVATE`	Allow webhooks to private/loopback addresses	`false`
`OBSIGATE_PDF_MAX_SIZE_MB`	Max PDF size for text extraction	`50`
`OBSIGATE_MEDIA_MAX_INLINE_MB`	Max size for inline audio/video playback (above: download)	`500`
`OBSIGATE_PDF_EXTRACT_TIMEOUT`	PDF extraction timeout (seconds)	`30`
`OBSIGATE_TAVILY_API_KEY` / `OBSIGATE_BRAVE_API_KEY` / `OBSIGATE_SERPAPI_API_KEY` / `OBSIGATE_EXA_API_KEY`	Keyed web-search providers (tried before SearXNG)	—
`OBSIGATE_WEB_PROVIDERS`	Search provider order (e.g. `brave,searxng`)	—
`OBSIGATE_WEB_RETRY`	Web tools network retries (house-made backoff)	`1`
`OBSIGATE_WEB_CACHE_TTL`	SQLite cache TTL for web results (seconds, `0` = off)	`900`
`OBSIGATE_GITEA_URL` / `OBSIGATE_GITEA_TOKEN`	Gitea connected source (`git_list_repos`…)	—
`OBSIGATE_GITHUB_TOKEN`	GitHub token (`git_list_repos`…)	—

These keys can also be entered from the UI (menu → Configurations → "Connected sources & search"): the stored value goes to data/api_keys.json and takes precedence over the environment variable.

All these variables are documented in .env.example.

Volume for Persistence

Auth data (users.json, secret.key) is stored in /app/data. Add a volume to persist it:

volumes:
  # vaults...
  - ./data:/app/data  # Persist users and JWT key

➕ Adding a New Vault

Method 1: Direct Editing

Stop the container :
```
docker-compose down
```

Add a volume in docker-compose.yml :

volumes:
  - /new/vault/path:/vaults/NewVault:ro

Add the environment variables :

environment:
  - VAULT_4_NAME=NewVault
  - VAULT_4_PATH=/vaults/NewVault

Restart :
```
./build.sh
```

Method 2: Hot-reload (recommended)

Add the volume and variables as above
Apply the changes :
```
./build.sh
```
Reload the index via the interface or the API :
```
curl http://localhost:2020/api/index/reload
```

Method 3: Dynamic API (without restart)

Add a vault on the fly via the API (the volume must already be mounted):

curl -X POST http://localhost:2020/api/vaults/add \
  -H "Content-Type: application/json" \
  -d '{"name": "NewVault", "path": "/vaults/NewVault"}'

Remove a vault:

curl -X DELETE http://localhost:2020/api/vaults/NewVault

🔨 Build & Deployment with build.sh

The build.sh script is the recommended way to build and deploy ObsiGate.

Basic Usage

chmod +x build.sh   # once
./build.sh          # build from scratch + start

Available Options

Option	Description
`--help`, `-h`	Display full help
`--build-only`	Build the image without starting the container
`--no-cache`	Full rebuild without Docker cache (default)
`--cache`	Use Docker cache (faster if few changes)
`--progress=plain`	Verbose output (recommended for debugging)
`--progress=tty`	Interactive output with progress bars

What the Script Does

Checks that Docker and Docker Compose are installed (with version numbers)
Validates the presence and syntax of the docker-compose.yml file
Checks each mounted volume : warns if a source directory does not exist
Builds the Docker image via docker compose build (multi-stage, ~180MB)
Starts the container via docker compose up -d
Displays the container status then real-time logs

Examples

# Clean build + start (recommended after code changes)
./build.sh

# Fast rebuild with cache (minor frontend changes only)
./build.sh --cache

# Test the build without starting (useful for pre-prod testing)
./build.sh --build-only

# Verbose output to debug a build failure
./build.sh --progress=plain

Stopping / Restarting

docker compose down      # Stop the container
docker compose up -d     # Restart without rebuild
docker compose logs -f   # View logs

🖼️ Obsidian Image Rendering

ObsiGate supports all Obsidian image syntaxes with an intelligent multi-strategy resolution system.

Supported Syntaxes

Standard Markdown with HTML attributes (Obsidian-compatible) :

[<img width="180" height="60" src="path/to/image.svg"/>](https://example.com)

Wiki-link embed with full path :

![[06_Toolbox/6.2_Attachments/image.svg]]

Wiki-link embed with filename only :
```
![[image.svg]]
```
Standard Markdown :
```
![alt text](path/to/image.png)
```

Intelligent Path Resolution

ObsiGate uses 7 resolution strategies in order of priority:

Absolute path : If the path is absolute and exists
Configured attachments folder : Via VAULT_N_ATTACHMENTS_PATH
Startup index (unique match) : Search by filename in the index
Same directory : Relative to the current markdown file
Vault root : Relative to the vault root
Startup index (closest match) : If multiple files have the same name
Fallback : Display a styled placeholder [image not found: filename.ext]

Viewer & file tree

Images are first-class vault files: they appear in the tree, are indexed (name + metadata, never the bytes) and open in a dedicated viewer — wheel zoom 0.1×–8×, drag pan, double-click to reset, ←/→ navigation between images in the same folder (WebP thumbnail filmstrip), metadata panel, full-screen lightbox, open original and download. The ext:png/ext:jpg search filter is available. Decodable formats: PNG, JPEG, GIF, WebP, BMP, ICO, SVG (SVG served with a sandbox CSP). HEIC/HEIF (iPhone) is not decodable by browsers and is not supported.

Configuration

To optimize resolution, configure the attachments folder for each vault:

environment:
  - VAULT_1_NAME=MyVault
  - VAULT_1_PATH=/vaults/MyVault
  - VAULT_1_ATTACHMENTS_PATH=Assets/Images  # Relative path
  - VAULT_1_SCAN_ATTACHMENTS=true           # Enable scanning (default)

Manual Rescan

To rescan images in a vault after adding/removing:

curl -X POST http://localhost:2020/api/attachments/rescan/MyVault

🖥️ Desktop (Tauri) — Native Application

📖 Full guide: Desktop (Tauri)

ObsiGate Desktop is a native application built with Tauri (Rust + system webview). It embeds the Python backend and frontend in a standalone executable — zero Docker, zero command line.

🚧 Version 2.0.0 — binaries are being stabilized. For now, building from source is recommended.

Native Desktop Features

Feature	Web	Desktop
Local file access	Via upload	Native (folder picker)
System theme	Manual	Auto (follows OS dark/light)
Notifications	Service Worker	Native OS
`.md` associations	❌	✅ "Open with ObsiGate"
Tray icon	❌	✅ Taskbar
Auto-update	❌	✅ Checks Gitea releases
Offline mode	Limited	Full (local backend)

Download (Pre-built Binaries)

Releases are published on Gitea :

Platform	Format
Linux	`.deb` + `.AppImage`
Windows	`.msi` + `.exe` (NSIS)

Linux

# .deb (Debian/Ubuntu/Deepin)
sudo dpkg -i obsigate_2.0.0_amd64.deb
# Launch : ObsiGate from the applications menu or `obsigate-desktop`

# .AppImage (any distro)
chmod +x ObsiGate_2.0.0_amd64.AppImage
./ObsiGate_2.0.0_amd64.AppImage

Windows

:: Double-click ObsiGate_2.0.0_x64.msi
:: Or launch ObsiGate from the Start menu

Getting Started

Launch the application from the menu or command line
The Python backend starts automatically on 127.0.0.1:17890
The window opens and loads the ObsiGate interface
First launch : select your Obsidian vaults folder via the native picker
To close : tray icon → Quit (clean backend shutdown)

Build from Source

Detailed guide: desktop/README.md.

Common Prerequisites

Tool	Version	Installation
Rust (cargo)	≥ 1.75	`rustup`
Tauri CLI	≥ 2.0	`cargo install tauri-cli`
Git	—	—
Linux system dependencies	—	`sudo apt install libwebkit2gtk-4.1-dev libgtk-3-dev libayatana-appindicator3-dev`

Important — staging : tauri.conf.json embeds backend/** and frontend/** from the desktop/ folder. The build scripts automatically copy ../backend and ../frontend into desktop/ before cargo tauri build. Without this staging, the build fails with "glob pattern backend/**/* path not found".

🪟 Windows — `build-windows.bat`

REM Prerequisites (via Scoop) : rustup, curl, git
scoop install rustup curl git
rustup default stable
cargo install tauri-cli

cd desktop
build-windows.bat

Script steps:

Kills residual Python processes (taskkill /F /IM python.exe)
Downloads Python 3.11 embed (python.org) → desktop\python-embed\ + enables pip (python311._pth)
pip install -r ..\backend\requirements.txt in the embed
Staging : copies ..\backend and ..\frontend to desktop\
cargo tauri build --target x86_64-pc-windows-msvc --bundles nsis
Copies python-embed next to the executable (target\x86_64-pc-windows-msvc\release\) for local dev mode
Cleans the staged folders

→ Artifact : desktop\target\x86_64-pc-windows-msvc\release\bundle\nsis\ObsiGate_2.0.0_x64-setup.exe

🐧 Linux — `build-linux.sh`

cd desktop
chmod +x build-linux.sh
./build-linux.sh

Script steps:

Checks Rust + Tauri CLI, installs system dependencies (apt)
Creates a Python venv desktop/python-embed/venv + pip install -r ../backend/requirements.txt
Staging : copies ../backend and ../frontend to desktop/
cargo tauri build --target x86_64-unknown-linux-gnu --bundles deb,appimage
Copies the runtime (python-embed/, backend/, frontend/) next to the executable

→ Artifacts :

desktop/target/x86_64-unknown-linux-gnu/release/bundle/deb/obsigate_2.0.0_amd64.deb
desktop/target/x86_64-unknown-linux-gnu/release/bundle/appimage/ObsiGate_2.0.0_amd64.AppImage

🤖 CI/CD Builds — Automatic Artifacts

Yes — the workflow .gitea/workflows/desktop-build.yml builds desktop binaries on every push to main touching desktop/**, frontend/**, or backend/** (and manually via workflow_dispatch), on self-hosted runners :

Job	Runner	Artifacts (retained 30 days)
`build-windows`	`[self-hosted, windows, desktop]`	`desktop/target/release/bundle/msi/*.msi`
`build-linux`	`[self-hosted, linux, desktop]`	`.AppImage` + `.deb`

Artifacts are downloadable from the Actions page of the Gitea run.
Publishing to Gitea Release is planned for tags v* (Publish to Gitea Release step).
The web workflow .gitea/workflows/ci.yml separately handles lint → tests → security → Docker build → e2e Playwright.

Desktop Architecture

┌────────────────────────────────────────────┐
│ Tauri (Rust)                               │
│  ├─ Webview (system webview)              │
│  │   └─ Frontend (HTML/JS/CSS)            │
│  └─ Sidecar Python                         │
│      └─ uvicorn backend.main:app          │
│           └─ port 127.0.0.1:17890         │
└────────────────────────────────────────────┘

Lifecycle: Tauri spawns the Python backend → health check → opens the webview. On close: SIGTERM → clean shutdown → cleanup.

📖 Usage

Web Interface

Navigation : Click on vaults in the sidebar to expand them
Search : Use the search bar to search across all vaults
Tags : Click on tags to filter content
Wikilinks : [[page]] links are clickable and navigable
Images : All Obsidian image syntaxes are rendered automatically
Theme : Toggle between light/dark theme with the 🌙/☀️ icon

Keyboard Shortcuts

Action	Shortcut
Search	`Ctrl + K` or `/`
Toggle theme	`Ctrl + T`
Focus search	`Esc`

👥 Real-time Collaboration

📖 Full guide: Editing & Collaboration

Multiple users can edit the same markdown document simultaneously (Google Docs style):

Conflict-free merge via Yjs (CRDT): two people can type in the same place, no change is lost.
Colored remote cursors and visible selections in CodeMirror, labelled with each user's name.
Presence indicator in the editor header (avatars + connection status).
Automatic reconnection (exponential backoff): state is merged on return.
Server-side persistence: the document is written to disk 2 s after the last change.
Transport: WebSocket ws(s)://<host>/ws/collab/{vault}/{path}, authenticated via the access_token cookie (or ?token=) and subject to per-vault access control.

No configuration is required: open the same file in two browsers (or two windows) to see it live.

🔌 API

📖 Full guide: REST API · MCP Server

ObsiGate exposes a complete REST API :

Endpoint	Description	Method	Auth
`/api/health`	Health check (status, version, stats)	GET	No
`/api/auth/status`	Auth status (enabled, users present)	GET	No
`/api/auth/login`	Login (returns access token + refresh cookie)	POST	No
`/api/auth/refresh`	Refresh access token via refresh cookie	POST	Cookie
`/api/auth/logout`	Logout + revoke refresh token	POST	Yes
`/api/auth/me`	Current user info	GET	Yes
`/api/auth/change-password`	Change password	POST	Yes
`/api/auth/admin/users`	List / create users	GET/POST	Admin
`/api/auth/admin/users/{u}`	Modify / delete a user	PATCH/DELETE	Admin
`/api/vaults`	List vaults (filtered by permissions)	GET	Yes
`/api/browse/{vault}?path=`	Browse folders	GET	Yes
`/api/file/{vault}?path=`	Rendered content of a file	GET	Yes
`/api/file/{vault}/raw?path=`	Raw content of a file	GET	Yes
`/api/file/{vault}/download?path=`	Download a file	GET	Yes
`/api/file/{vault}/save?path=`	Save a file	PUT	Yes
`/api/file/{vault}?path=`	Delete a file	DELETE	Yes
`/api/search/advanced`	Advanced TF-IDF search (+ `semantic=true` for hybrid)	GET	Yes
`/api/suggest` / `/api/tags/suggest`	Autocomplete	GET	Yes
`/api/tags?vault=`	Unique tags with counters	GET	Yes
`/api/index/reload`	Force a rescan of vaults	GET	Admin
`/api/events`	Real-time SSE stream	GET	Yes
`/api/vaults/add` / `/api/vaults/{name}`	Dynamic vault management	POST/DELETE	Admin
`/api/image/{vault}?path=`	Serve an image	GET	Yes
`/api/media/{vault}/thumb?path=&size=`	WebP thumbnail (disk cache)	GET	Yes
`/api/config`	Read / write configuration	GET/POST	Yes/Admin
`/api/diagnostics`	Index and memory statistics	GET	Admin

When OBSIGATE_AUTH_ENABLED=false, all endpoints are accessible without a token.

All endpoints expose documented Pydantic schemas. Interactive docs are available at /docs (Swagger UI).

Usage Example :

# Health check
curl http://localhost:2020/api/health

# List vaults
curl http://localhost:2020/api/vaults

# Simple search (legacy)
curl "http://localhost:2020/api/search?q=recipe&vault=all"

# Advanced search with TF-IDF, facets, and pagination
curl "http://localhost:2020/api/search/advanced?q=recipe%20tag:cuisine&vault=all&limit=20&offset=0&sort=relevance"

# Title autocomplete
curl "http://localhost:2020/api/suggest?q=piz&vault=all"

# Tag autocomplete
curl "http://localhost:2020/api/tags/suggest?q=rec&vault=all"

# Get a file
curl "http://localhost:2020/api/file/Recipes?path=pizza.md"

🔍 Advanced Search

📖 Full guide: Search, PDF, Excel & Excalidraw

Query Syntax

Operator	Description	Example
`tag:<name>`	Filter by tag	`tag:recipe docker`
`#<name>`	Tag shortcut	`#linux server`
`vault:<name>`	Filter by vault	`vault:IT kubernetes`
`title:<text>`	Filter by title	`title:pizza`
`path:<text>`	Filter by path	`path:recipes/soups`
`ext:<type>`	Filter by file type	`ext:md kubernetes`
`"exact phrase"`	Phrase search	`tag:"multiple words"`

Extension filter examples: ext:sh for bash scripts, ext:py for Python scripts, ext:md for Markdown files, ext:pdf for PDF documents, ext:excalidraw for Excalidraw diagrams (text-extracted content is indexed).

PDF support

PDF files in your vaults are rendered inline in the browser via the native PDF viewer (iframe + <embed>). The viewer streams the file over HTTP Range requests (206 Partial Content), so large PDFs load progressively. Text is extracted on indexing (pypdf / pymupdf) so PDF content is searchable via the full-text search. Filter with ext:pdf to restrict results to PDFs. PDF metadata (pages, title, author) is available via GET /api/file/{vault}/pdf/info without transferring the document.

Limitations: no OCR (scanned PDFs aren't searchable), no annotation, no editing of the PDF itself.

Excalidraw diagrams

.excalidraw and .excalidraw.md files open in a full visual Excalidraw editor embedded in a sandboxed iframe — draw, edit and save without leaving ObsiGate. Changes are saved automatically (2s debounce) or with Ctrl+S; the editor follows the light/dark theme. Text labels inside the diagram elements are extracted on indexing, so diagram content is searchable via the full-text search (ext:excalidraw). Files created with the Obsidian Excalidraw plugin (including the compressed .excalidraw.md format) are compatible.

Operators are combinable: tag:linux vault:IT ext:md server web searches for "server web" in Markdown files of the IT vault with the linux tag.

Keyboard Shortcuts

Shortcut	Action
`Ctrl+K` / `Cmd+K`	Focus search bar
`/`	Focus search (outside text fields)
`↑` / `↓`	Navigate suggestions
`Enter`	Select active suggestion or launch search
`Escape`	Close suggestions / exit search

Features

TF-IDF : Scoring based on term frequency weighted by inverse document frequency
Title boost : Title matches receive 3× higher score
Accent normalization : resume finds résumé, elephant finds éléphant
Highlighted snippets : Found terms are wrapped in <mark> in excerpts
Facets : Counters by vault and by tag in results
Pagination : Page navigation with 50 results per page
Sorting : By relevance (TF-IDF) or modification date
Visual chips : Active filters are shown as removable colored chips
History : Last 50 searches are stored in localStorage
Semantic search (optional) : Toggle ~ (or Alt+S) fuses the TF-IDF ranking with an embedding-based ranking (RRF). Works out of the box with a dependency-free hashing embedder; install backend/requirements-semantic.txt and/or set OBSIGATE_EMBEDDING_* for real all-MiniLM-L6-v2 embeddings. See docs/features/semantic-search.md.

🔧 Troubleshooting

Common Issues

Port already in use:

# Check who is using the port
sudo netstat -tulpn | grep 2020
# Change the port in docker-compose.yml
ports:
  - "2021:8080"

Vault not found:

Verify paths are absolute
Ensure read permissions
Restart the container after modifications

Build fails:

# Clean Docker cache and rebuild
docker system prune -f
docker compose down
./build.sh --progress=plain

# If failure persists, check build logs
./build.sh --progress=plain 2>&1 | tee build.log

Logs for debugging:

# Real-time logs
docker compose logs -f obsigate

# Detailed logs
docker compose logs --tail=100 obsigate

⚡ Performance

Metric	Estimate
Indexing	~1–2s for 1,000 markdown files
Advanced Search	< 10ms for most queries (inverted index + TF-IDF)
Wikilink Resolution	O(1) via lookup table
Memory	~80–150MB per 1,000 files (content capped at 100 KB/file)
Docker Image	~180MB (multi-stage, no build tools)
CPU	Non-blocking; search offloaded to dedicated thread pool

Recommended Settings by Vault Size

Size	Files	`search_workers`	`prefix_max_expansions`	`max_content_size`
Small	< 500	1	50	100,000
Medium	500–5,000	2	50	100,000
Large	5,000+	4	30	50,000

These parameters are configurable via the interface (Settings) or the /api/config API.

Key Optimizations (v1.2.0)

Inverted index with set-intersection : Search uses posting lists for O(k × postings) retrieval instead of O(N) full scan
Prefix matching by binary search : O(log V + k) instead of O(V) linear vocabulary scan
ThreadPoolExecutor : CPU-bound search functions are offloaded from the asyncio event loop
Race condition guard : currentSearchId + AbortController prevent stale result rendering
Progress bar : Animated progress bar during search
Search timeout : Automatic abort after 30s (configurable)
Query time display : Server time shown in results (query_time_ms)
Staleness detection fix : Generation counter instead of id(index) to detect index changes

Optimizations v1.5.0 (Quick Wins 2026-05-27)

GZip Compression : FastAPI middleware compresses all responses >1KB (~70% bandwidth saved)
Cache-Control immutable : Static assets (/static/*) are cached by the browser for 1 year
.dockerignore : Minimal Docker build context (no .git, docs/, *.md, __pycache__)
Incremental InvertedIndex : add_document/remove_document hooks, no more O(N) rebuild on every mutation

Optimizations v1.1.0

Search without I/O : File content is cached in memory index
Multi-factor scoring : exact title (+20), partial title (+10), path (+5), tag (+3), content frequency (x1 per occurrence, capped at 10)
Markdown renderer singleton : The mistune renderer is instantiated once
AbortController : Stale search requests are aborted client-side
Debounced icon rendering : lucide.createIcons() is batched via requestAnimationFrame

🛡️ Security

📖 Full guide: Auth & Security

Path traversal : All file endpoints validate that the resolved path stays within the vault
Rate limiting : 10 login attempts max per IP over 15 minutes + per-account lockout (5 attempts)
Audit log : All writes, deletions, and config changes are logged in data/audit.log (JSON lines, 10 MB rotation)
Automatic backup : Every file modification or deletion is saved in .obsigate-backup/ with timestamp
Secret redaction : Automatic masking of JWTs, API keys, tokens, and connection strings in previews
Non-root user : The Docker container runs under user obsigate (UID 1000)
Read-only volumes : Vaults are mounted as :ro by default in docker-compose
Secrets in .env : Passwords and tokens are never in docker-compose.yml
Atomic writes : Data files (users.json, shares.json, webhooks.json) use tmp+replace

🏗️ Tech Stack

Backend : Python 3.11 + FastAPI 0.110 + Uvicorn
Auth : python-jose (JWT HS256) + argon2-cffi (Argon2id)
Security : Rate limiting (IP + account lockout), audit logging, secret redaction, atomic writes
File Watcher : watchdog 4.x (native inotify + fallback polling)
Frontend : Vanilla JS + HTML + CSS (zero framework, zero build)
Markdown Rendering : mistune 3.x
PDF Export : WeasyPrint 60+
Compression : GZip middleware (FastAPI), Cache-Control immutable for assets
Docker Image : python:3.11-slim (multi-stage), .dockerignore for minimal context
User Storage : Local JSON (data/users.json) — no database
Architecture : SPA + REST API + SSE

🏠 Architecture

┌─────────────────┐     ┌─────────────────────────────────────────┐
│   Browser       │◄───►│  FastAPI (backend/main.py)             │
│   (SPA)         │ REST │                                         │
│                 │      │  ┌──────────────┐  ┌──────────────┐  │
│  app.js         │      │  │ indexer.py  │  │  search.py    │  │
│  style.css      │      │  │ (scan+cache)│  │ (in-memory)  │  │
│  index.html     │      │  └───────┬──────┘  └──────┬───────┘  │
└─────────────────┘      │        │              │              │
                         │        └──────┬───────┘              │
                         │             │                        │
                         │    ┌────────┴─────────┐            │
                         │    │ In-Memory Index │            │
                         │    │ (files, tags,   │            │
                         │    │  content, lookup)│            │
                         │    └──────────────────┘            │
                         └─────────────────────────────────────────┘
         ┌───────────────────────────────┐
         │  Filesystem (mounted vaults)  │
         │  /vaults/Recipes  (ro)        │
         │  /vaults/IT         (ro)      │
         └───────────────────────────────┘

Data Flow:

On startup, indexer.py scans all vaults in parallel (thread pool)
Content, tags (YAML + inline), and metadata are cached in memory
An O(1) lookup table is built for wikilink resolution
watcher.py starts file monitoring (native watchdog or polling)
Detected changes trigger incremental index updates
Changes are notified to the frontend via Server-Sent Events (SSE)
Search queries use the in-memory index (zero disk I/O)
The SPA frontend communicates via REST + SSE and manages state client-side

📝 Development

🧪 Tests and CI — Mandatory Rules (Before Each Push)

⚠️ Absolute rule — human contributors and AI agents: never git push without having locally executed the entire sequence below with 100 % success. Push is not meant to discover failures: CI is a safety net, not a diagnostic tool.

Local Checks Before Commit

Step	Command	Equivalent CI Job
Backend lint	`ruff check backend/`	`lint`
Frontend import validation	`node tests/frontend/validate-imports.mjs`	`lint`
Frontend unit tests	`node tests/frontend/unit.test.mjs`	`lint`
Backend tests	`pytest tests/ -q`	`test`
E2E Playwright	`npm run test:e2e` (~10 min)	`e2e`

Local E2E Tests (`npm run test:e2e`)

The scripts/run-e2e-local.sh script faithfully reproduces the e2e CI job conditions (.gitea/workflows/ci.yml):

backend started natively (venv .venv-e2e, Python 3.11 via uv) ;
authentication disabled (OBSIGATE_AUTH_ENABLED=false) ;
fixtures test_vault (TestVault) and test_dir (TestDir) ;
port 2029 (baseURL from playwright.config.ts) ;
Playwright project chromium-desktop only (same as CI).

Prerequisites: uv, Node.js ≥ 20, Playwright browsers (npx playwright install chromium). The venv is created automatically on first run. Docker is not required.

npm run test:e2e                                  # full suite
bash scripts/run-e2e-local.sh --headed            # visible browser
bash scripts/run-e2e-local.sh -g "reset panes"    # filter on a test

On Windows, when bash is unusable (WSL unavailable, git-bash blocked by an Application Control policy), use the equivalent PowerShell launcher:

npm run test:e2e:ps
pwsh -File scripts/run-e2e-local.ps1 -PlaywrightArgs @('-g','reset panes')

The suite must end with all tests passing (60 currently), with no failure or reliance on retries. In case of failure: fix and re-run locally until 100 %, then only commit.

Test Writing Rules (Lesson Learned)

Check every selector in the actual DOM before using it in a test. Real bug encountered: a test was targeting #cp-input while the command palette input only has the .cp-input class (no id) — the test could not pass in any environment.
Any new or modified test must be run locally (at least with -g "test name") before push.
No workarounds that mask flakiness (arbitrary waitForTimeout, silent fallbacks): fix the root cause.

Commits and Push

Commit message: imperative subject ≤ 50 chars; body (72 col.) only if it adds useful information.
One commit = one logical change (do not mix fix and refactor).
Push only when all 5 local steps are green. CI pipeline: lint → test → security → build → e2e.

Project Structure

ObsiGate/
├── backend/              # FastAPI API
│   ├── main.py          # Endpoints, Pydantic models, markdown rendering
│   ├── indexer.py       # Vault scanning, in-memory index, lookup table
│   ├── search.py        # Full-text search engine with scoring
│   ├── watcher.py       # File monitoring (watchdog + debounce)
│   ├── auth/            # Authentication module
│   │   ├── password.py  # Argon2id hashing
│   │   ├── jwt_handler.py # JWT tokens + revocation
│   │   ├── user_store.py  # users.json CRUD (atomic)
│   │   ├── middleware.py  # FastAPI auth dependencies
│   │   └── router.py    # /api/auth/* endpoints
│   ├── create_admin.py  # User management CLI
│   └── requirements.txt
├── frontend/            # Web interface (Vanilla JS, zero framework)
│   ├── index.html       # SPA page + login screen
│   ├── app.js           # SPA logic, AuthManager, AdminPanel
│   └── style.css        # Styles (CSS variables, themes, responsive)
├── data/                # Persistent data (created on startup)
│   ├── users.json       # Users (Argon2id hashed)
│   └── secret.key       # JWT secret key (512 bits)
├── Dockerfile           # Multi-stage, healthcheck, non-root
├── docker-compose.yml   # Deployment with healthcheck and auth env vars
├── build.sh             # Automated build & deployment (docker compose build + up)
└── docs/
    ├── GUIDES/          # User guides (getting started, API, MCP, desktop…)
    └── CONTRIBUTING.md  # Contribution guide

Contributing

See CONTRIBUTING.md for code standards and docs/DELIVERY_WORKFLOW.md for the mandatory delivery process.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🤝 Support

Issues : git.dracodev.net/Projets/ObsiGate/issues
Documentation : git.dracodev.net/Projets/ObsiGate/wiki
Author : Bruno Beloeil

📝 Changelog

See CHANGELOG.md for the complete version history (v1.0.0 → v2.43.1).

Project: ObsiGate | Version: 2.43.1 | Last updated: September 2026

47 KiB Raw Blame History Unescape Escape

ObsiGate

📚 Guides

📋 Table of Contents

✨ Features

🚀 Prerequisites

System Requirements

Supported Systems

⚡ Quick Installation

1. Clone the Repository

2. Configure Your Vaults and Secrets

3. Launch the Application

4. Access the Interface

⚙️ Detailed Configuration

Step 1: Prepare Your Vaults

Step 2: Configure docker-compose.yml

Step 3: Build & Deploy

🌍 Environment Variables

🔒 Authentication

Enable Authentication

First Startup

User Management via CLI

Admin Interface

Per-Vault Access Control

Auth Environment Variables

Volume for Persistence

➕ Adding a New Vault

Method 1: Direct Editing

Method 2: Hot-reload (recommended)

Method 3: Dynamic API (without restart)

🔨 Build & Deployment with build.sh

Basic Usage

Available Options

What the Script Does

Examples

Stopping / Restarting

🖼️ Obsidian Image Rendering

Supported Syntaxes

Intelligent Path Resolution

Viewer & file tree

Configuration

Manual Rescan

🖥️ Desktop (Tauri) — Native Application

Native Desktop Features

Download (Pre-built Binaries)

Linux

Windows

Getting Started

Build from Source

Common Prerequisites

🪟 Windows — build-windows.bat

🐧 Linux — build-linux.sh

🤖 CI/CD Builds — Automatic Artifacts

Desktop Architecture

📖 Usage

Web Interface

Keyboard Shortcuts

👥 Real-time Collaboration

🔌 API

🔍 Advanced Search

Query Syntax

PDF support

Excalidraw diagrams

Keyboard Shortcuts

Features

🔧 Troubleshooting

Common Issues

⚡ Performance

Recommended Settings by Vault Size

Key Optimizations (v1.2.0)

Optimizations v1.5.0 (Quick Wins 2026-05-27)

Optimizations v1.1.0

🛡️ Security

🏗️ Tech Stack

🏠 Architecture

📝 Development

🧪 Tests and CI — Mandatory Rules (Before Each Push)

Local Checks Before Commit

Local E2E Tests (npm run test:e2e)

Test Writing Rules (Lesson Learned)

47 KiB

Raw Blame History

🪟 Windows — `build-windows.bat`

🐧 Linux — `build-linux.sh`

Local E2E Tests (`npm run test:e2e`)