ObsiGate/context.md
Bruno Charest 482937fb30 Add audit logging, rate limiting, secret redactor, and backlinks
Implement several security and feature improvements across the backend
and frontend:
- New IP-based rate limiter for authentication endpoints
- New audit logging system for sensitive operations
- New secret redactor to mask sensitive patterns in rendered content
- Configurable token TTL and IGNORED_DIRS via environment variables
- Add backlink index and API endpoint
- Add preview tab support with single/double-click behavior in tree
- Add file backup before write/delete operations
2026-05-26 10:27:00 -04:00

14 KiB
Raw Blame History

Code Context

Files Retrieved

  1. C:/dev/git/python/ObsiGate/backend/main.py (lines 1-2504) - Core API endpoints, markdown rendering, SSE
  2. C:/dev/git/python/ObsiGate/backend/auth/router.py (full file, 263 lines) - Auth endpoints (login, logout, refresh, admin CRUD)
  3. C:/dev/git/python/ObsiGate/backend/auth/jwt_handler.py (full file, 153 lines) - JWT token creation, validation, revocation
  4. C:/dev/git/python/ObsiGate/backend/indexer.py (full file, ~728 lines) - File indexing, vault config, file lookup
  5. C:/dev/git/python/ObsiGate/backend/search.py (full file, ~700 lines) - Full-text search, TF-IDF, suggestions
  6. C:/dev/git/python/ObsiGate/frontend/app.js (lines 1-8046) - Frontend SPA (TOC, tabs, tree, search)
  7. C:/dev/git/python/ObsiGate/frontend/style.css (lines 5379-5476) - Tab bar styles

1. Backend: main.py

PUT endpoint for saving files

Location: Line 717 (@app.put("/api/file/{vault_name}/save", response_model=FileSaveResponse))

  • Function: api_file_save (line 718)
  • Body expects {"content": "..."}
  • Ends at line ~750 with return {"status": "ok", "vault": vault_name, "path": path, "size": len(content)}

DELETE endpoint for files

Location: Line 753 (@app.delete("/api/file/{vault_name}", response_model=FileDeleteResponse))

  • Function: api_file_delete (line 754)
  • Path provided as query parameter: path: str = Query(...)
  • Also calls remove_single_file() and broadcasts SSE file_deleted event
  • Ends at line ~797

_heading_slugify function

Location: Lines 476503 (inside the Markdown rendering helpers section)

def _heading_slugify(text: str) -> str:
  • Matches the JavaScript slugify() exactly:
    1. Lowercase
    2. NFD normalize + strip combining marks
    3. Keep only Unicode letters, numbers, spaces, hyphens
    4. Spaces → hyphens, collapse multiple hyphens
    5. Strip leading/trailing hyphens, fallback to "heading"

_add_heading_ids function

Location: Lines 506527

  • Post-processes HTML to inject id="" attributes on <h1><h6> tags
  • Handles duplicate slugs with -2, -3 suffix

Health endpoint

Location: Lines 562571 (@app.get("/api/health", response_model=HealthResponse))

  • Returns { status, version, vaults, total_files }
  • No authentication required
  • _convert_wikilinks(): lines 528549 — converts [[target]] / [[target|display]] to clickable HTML anchors
  • _render_markdown(): lines 552577 — master renderer: preprocesses images → converts wikilinks → renders with mistune → adds heading IDs
  • Wikilinks render as <a class="wikilink" data-vault="..." data-path="..."> when resolved, <span class="wikilink-missing"> otherwise

2. Backend: auth/router.py

Login endpoint

Location: Line 97 (@router.post("/login"))

  • Function: login (line 98)
  • Accepts LoginRequest with username, password, remember_me
  • Rate limiting via lockout:
    • is_locked() check at line 108 → returns 429 after too many failures
    • record_login_failure() at line 112 → increments failure counter
    • Lockout message: "Compte temporairement verrouillé (15min)" (line 109)
  • Success: calls create_access_token() + create_refresh_token(), sets cookies

Rate limiting

Location: Lines 108117 (inside login endpoint)

  • There is NO decorator-based or middleware rate limiting. Rate limiting is manual, login-only:
    • Checks is_locked() (line 108) — if true, raises HTTP 429
    • Calls record_login_failure() (line 112) on bad password
    • Shows remaining attempts when <= 2 (line 114115)
  • Implementation lives in backend/auth/user_store.py:
    • record_login_failure() at line 142
    • is_locked() at line 167
  • No rate limiting on other endpoints (no slowapi, no middleware, no global limiter)

3. Backend: auth/jwt_handler.py

JWT TTL / expiration settings

Location: Lines 2223

ACCESS_TOKEN_EXPIRE_SECONDS = 3600      # 1 hour
REFRESH_TOKEN_EXPIRE_SECONDS = 604800   # 7 days
  • Algorithm: HS256 (line 20)
  • Secret key auto-generated to data/secret.key on first run (line 29)
  • Refresh cookie max_age in router.py: 30 days if remember_me, else 7 days (line 131)

create_access_token function

Location: Lines 4860

def create_access_token(user: dict) -> str:
  • Payload: { sub, role, vaults, jti, iat, exp, type: "access" }
  • Encoded with jwt.encode() using HS256 and the secret key from get_secret_key()

create_refresh_token function

Location: Lines 6373

  • Returns (token_string, jti) tuple
  • Payload: { sub, jti, iat, exp, type: "refresh" }
  • Uses REFRESH_TOKEN_EXPIRE_SECONDS

4. Backend: indexer.py

IGNORED_DIRS or similar

There is NO IGNORED_DIRS constant. The indexer indexes everything including hidden files (starting with .). This is stated explicitly in the docstring at line 206:

"All files and directories are indexed, including hidden files (starting with '.')."

Hidden-file filtering is handled at the UI/browse level via vault settings (hideHiddenFiles) in main.py and vault_settings.py.

vault_config handling

Location: Lines 1516 (global)

vault_config: Dict[str, Dict[str, Any]] = {}
  • Type: {name: {path, attachmentsPath, scanAttachmentsOnStartup, type}}
  • Populated by load_vault_config() at lines 50104
  • Reads VAULT_N_NAME/VAULT_N_PATH and DIR_N_NAME/DIR_N_PATH env vars
  • Also has vault_config.update(load_vault_config()) in build_index() at line 312

Key data structures

  • index: dict of vaults → {files, tags, path, paths, config} (line 11)
  • _file_lookup: {filename_lower: [{vault, path}, ...]} — O(1) wikilink resolution (line 22)
  • path_index: {vault_name: [{path, name, type}, ...]} — tree filtering (line 25)
  • _index_lock: threading.Lock() (line 18)
  • _index_generation: int counter for staleness detection (line 24)

5. Backend: search.py

There are NO wikilink or backlink functions in search.py. The file handles:

  • Full-text search with TF-IDF via InvertedIndex class (line 218)
  • advanced_search() (line 426) — supports operators: tag:, vault:, title:, path:, ext:
  • Title suggestions: suggest_titles() (line 594)
  • Tag suggestions: suggest_tags() (line 620)

Wikilink resolution lives in backend/indexer.py via find_file_in_index() (line 653) using _file_lookup. Wikilink rendering lives in backend/main.py via _convert_wikilinks() (line 528). Backlinks do not exist anywhere in the codebase — no function computes "what links to this file."


6. Frontend: app.js

Tree item click handler (sidebar file opening)

Primary location: Lines 23492360 (inside _renderDirectoryInContainer during tree rendering)

fileItem.addEventListener("click", () => {
  scrollTreeItemIntoView(fileItem, false);
  openFile(vaultName, item.path);
  closeMobileSidebar();
});

Second location (search results): Lines 26372642 — same pattern in a different tree-rendering path. Third location (tree search filter results): Lines 27902795 — filter results click handler.

openFile function

Location: Lines 30853106 (original openFile)

  • Sets currentVault, currentPath, fetches /api/file/{vault}?path={path}
  • Calls renderFile(data) which builds breadcrumb, tags, action buttons, then renders HTML

Overridden at line 7604:

openFile = function(vault, path) {
  TabManager.open(vault, path);
};

This wraps the original to use tab-based navigation. TabManager.open() creates/focuses a tab.

Tab management functions (TabManager)

Location: Lines 72347598 (const TabManager = { ... })

  • init() — line 7243 — grabs DOM refs
  • open(vault, path, options) — line 7247 — opens a file in a new/focused tab
  • activate(tabId) — line 7274 — switches to a tab, saves/restores state
  • close(tabId) — line 7330 — closes a tab, switches to adjacent
  • closeAll() — line 7348 — closes all, shows dashboard
  • closeRight(tabId) — line 7358 — closes tabs to the right
  • closeOthers(tabId) — line 7374 — closes all except current
  • moveTab(fromIdx, toIdx) — line 7387 — drag-and-drop reorder
  • _renderTabs() — line 7438 — DOM rendering of tab bar with icons, names, close buttons, drag & drop
  • _showTabContextMenu(x, y, tabId) — line 7558 — right-click menu (Close, Close Others, Close Right, Close All)

TOC slugify function

Location: Lines 766776 (inside OutlineManager)

slugify(text) {
  return text
    .toLowerCase()
    .normalize("NFD")
    .replace(/[\u0300-\u036f]/g, "")
    .replace(/[^\p{L}\p{N}\s-]/gu, "")
    .replace(/\s+/g, "-")
    .replace(/-+/g, "-")
    .trim() || "heading";
}

There is NO backlinks UI or functionality anywhere in the frontend.

  • No backlink string found in app.js, style.css, or index.html
  • No "Links to this page" panel, no backlink section in the editor, no backlink search in the sidebar

7. Frontend: style.css

Location: Lines 53795480 (.tab-bar through .tab-drop-indicator)

  • .tab-bar (line 5379): flex container, 36px min-height, border-bottom
  • .tab-bar[hidden] (line 5389): display: none
  • .tab-list (line 5393): horizontal flex with overflow-x auto
  • .tab-item (line 5406): padding 6px 12px, 0.8rem, border-right, transitions
  • .tab-item:hover (line 5424): bg hover, color change
  • .tab-item.active (line 5429): bg primary, bottom accent border
  • .tab-item .tab-icon (line 5436): 14×14, flex-shrink
  • .tab-item .tab-name (line 5443): overflow ellipsis, max-width 150px
  • .tab-item .tab-close (line 5449): 16×16, hidden by default (opacity: 0)
  • .tab-item:hover .tab-close, .tab-item.active .tab-close (lines 54615462): opacity 0.6
  • .tab-item .tab-close:hover (line 5466): opacity 1
  • .tab-item.dragging (line 5471): opacity 0.5
  • .tab-drop-indicator (line 5476): 2px accent bar for drag-drop

Sidebar tab styles (sidebar-tab, not content-tab)

Location: Lines 744802

  • .sidebar-tabs (line 745)
  • .sidebar-tab (line 754): uppercase, accent border on active
  • .sidebar-tab-panel (line 793): display none, scrollable

Architecture Summary

┌─────────────────────────────────────────────────────┐
│ Frontend (app.js ~8000 lines)                       │
│  ┌──────────┐ ┌──────────────┐ ┌─────────────────┐ │
│  │ Sidebar  │ │ Content Area │ │ Right Sidebar   │ │
│  │ Tree     │ │ TabManager   │ │ TOC/Outline     │ │
│  │ Tags     │ │ renderFile() │ │ (slugify)       │ │
│  │ Filter   │ │ Breadcrumbs  │ │ ReadingProgress │ │
│  └──────────┘ └──────────────┘ └─────────────────┘ │
│  ← openFile() → TabManager.open() → api() → backend│
└─────────────────────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────┐
│ Backend (FastAPI)                                   │
│  main.py:                                           │
│    PUT /api/file/{vault}/save   (line 717)          │
│    DELETE /api/file/{vault}     (line 753)          │
│    GET  /api/file/{vault}       (rendered HTML)     │
│    GET  /api/health             (line 562)          │
│    _heading_slugify()           (line 476)          │
│    _convert_wikilinks()         (line 528)          │
│                                                      │
│  auth/router.py:                                    │
│    POST /api/auth/login         (line 97)           │
│    Rate limiting: lockout only (lines 108-117)      │
│                                                      │
│  auth/jwt_handler.py:                               │
│    ACCESS_TOKEN_EXPIRE_SECONDS = 3600  (line 22)    │
│    REFRESH_TOKEN_EXPIRE_SECONDS = 604800 (line 23)  │
│    create_access_token()        (line 48)           │
│                                                      │
│  indexer.py:                                        │
│    vault_config {}              (line 15-16)         │
│    load_vault_config()          (line 50)            │
│    ⚠ No IGNORED_DIRS — indexes everything          │
│                                                      │
│  search.py:                                         │
│    ⚠ No wikilink/backlink functions                │
│    Wikilinks resolved via indexer.find_file_in_index│
└─────────────────────────────────────────────────────┘

Start Here

For any feature work, start with C:/dev/git/python/ObsiGate/backend/main.py — it contains the API surface, markdown rendering (wikilinks, heading IDs, slugify), and all the endpoint definitions that tie the frontend to the backend index/search/auth subsystems.

Key Findings / Gaps

  • No rate limiting except manual lockout on login
  • No backlinks — neither computed in backend nor displayed in frontend
  • No IGNORED_DIRS — the indexer indexes everything; hidden-file hiding is at the UI layer
  • No wikilink/backlink in search.py — wikilink resolution is in indexer.py, rendering in main.py
  • TabManager is a self-contained singleton at the end of app.js (line 7234) wrapping the original openFile