Projets/ObsiGate

Fork 0

Bruno Charest b38f3f16e4 Coalesce index generation increments and add rebuild cooldown

2026-05-26 11:42:47 -04:00

12 KiB

Raw Permalink Blame History

Code Context — ObsiGate Bug Investigation

Files Retrieved

frontend/app.js (lines 5585–5665) — showWelcome() rebuilds dashboard HTML with only bookmarks + recent sections
frontend/index.html (lines 360–406) — Initial dashboard DOM has all 4 sections: stats, bookmarks, conflicts, recent
frontend/app.js (lines 7640–7672) — TabManager.activate() hides dashboard; _showDashboard() restores it
frontend/app.js (lines 7497–7597) — TabManager.openPreview() / openPersistent() definitions
frontend/app.js (lines 7778–7830) — TabManager._renderTabs() — missing preview CSS class
frontend/style.css (lines 5450–5457) — CSS rules for .tab-item.preview (italic)
frontend/app.js (lines 2340–2376) — Tree click handlers in _renderDirectoryInContainer
frontend/app.js (lines 3144–3310) — renderFile() overwrites content-area.innerHTML, destroying dashboard DOM
frontend/app.js (lines 3347–3380) — DashboardStatsWidget relies on dashboard-stats-grid element
frontend/app.js (lines 3381–3434) — DashboardConflictsWidget relies on dashboard-conflicts-section element
backend/search.py (lines 239–425) — InvertedIndex class, is_stale(), rebuild(), get_inverted_index()
backend/indexer.py (lines 28, 330–601, 634–667, 770–839) — _index_generation counter and all increment sites
backend/watcher.py (lines 191–226) — Debounce: dispatches batched events after debounce_seconds (2s default)
backend/main.py (lines 376–420) — _on_vault_change() calls update_single_file per event → each increments _index_generation
backend/search.py (lines 670–700, 844–875) — advanced_search() and suggest_titles() call get_inverted_index()

Issue 1: Dashboard Layout — Sections Disappearing

Root Cause: `showWelcome()` Rebuild Kills Stats & Conflicts Sections

showWelcome() (frontend/app.js, lines 5585–5665):

It checks if dashboard-home or dashboard-bookmarks-section don't exist.
If either is missing, it replaces content-area.innerHTML with only two sections: bookmarks and recent.
Stats and Conflicts sections are NOT included in this rebuilt HTML.

// line 5595 — only rebuilds if home or bookmarksSection is missing
if (area && (!home || !bookmarksSection)) {
  area.innerHTML = `
    <div id="dashboard-home" ...>
      <div id="dashboard-bookmarks-section" class="dashboard-section">...</div>
      <div id="dashboard-recent-section" class="dashboard-section">...</div>
    </div>`;
  // NOTE: No #dashboard-stats-section or #dashboard-conflicts-section!
}

The original dashboard (frontend/index.html, lines 360–406) has 4 sections:

#dashboard-stats-section
#dashboard-bookmarks-section
#dashboard-conflicts-section
#dashboard-recent-section

When does the dashboard get destroyed?

renderFile() at line 3302: area.innerHTML = ""; — this wipes content-area including #dashboard-home (which is a child of content-area).
After the file renders, activate() at line 7642 hides the dashboard: dashboard.style.display = "none" — but at this point the dashboard DOM elements have been removed from the DOM.

Later, when _showDashboard() (line 7764) is called:

_showDashboard() {
  const area = document.getElementById("content-area");
  area.innerHTML = "";          // clear file content
  const dashboard = document.getElementById("dashboard-home");
  if (dashboard) {
    dashboard.style.display = "";  // show it
    area.appendChild(dashboard);   // move back into DOM
  }
}

This works only if dashboard-home still exists in the document. But renderFile() called area.innerHTML = "" which destroyed it.

Next time showWelcome() is called (e.g., after goHome()), it detects !home → rebuilds with only 2 sections.

Why "Recently opened" stays visible when stats/bookmarks disappear:

The rebuilt HTML in showWelcome() includes #dashboard-recent-section and #dashboard-bookmarks-section, but not #dashboard-stats-section or #dashboard-conflicts-section.
DashboardStatsWidget.load() (line 3348) looks for document.getElementById("dashboard-stats-grid") — if not found, it silently returns.
DashboardConflictsWidget.load() (line 3383) looks for document.getElementById("dashboard-conflicts-section") — if not found, silently returns.
So stats and conflicts sections vanish, while bookmarks and recent survive the rebuild.

How the Dashboard Visibility Toggle Works

TabManager.activate() (line 7642): dashboard.style.display = "none" — hides dashboard when a file tab is shown.
TabManager._showDashboard() (line 7764): dashboard.style.display = "" + appends to content-area — shows dashboard when all tabs close.

CSS for Dashboard Sections

frontend/style.css (lines 4739–4742):

.dashboard-section {
  display: flex;
  flex-direction: column;
}

All .dashboard-section elements are just flex columns — there's no show/hide logic in CSS beyond what JavaScript sets via inline style.display.

Issue 2: Click/Double-Click Handling

Tree Click Handlers — Correctly Wired

In _renderDirectoryInContainer (frontend/app.js, lines 2354–2360):

fileItem.addEventListener("click", () => {
  scrollTreeItemIntoView(fileItem, false);
  TabManager.openPreview(vaultName, item.path);   // single click → preview
  closeMobileSidebar();
});
fileItem.addEventListener("dblclick", (e) => {
  e.preventDefault();
  TabManager.openPersistent(vaultName, item.path); // double click → persistent
});

Same pattern at lines 3132–3133 (sidebar recent), 4909–4910 and 5090–5091 (search results). Wiring is correct.

openPreview() and openPersistent() — Properly Defined

openPreview() (line 7497): Creates tab with preview: true, closes existing preview, activates.
openPersistent() (line 7533): If a preview tab exists for this file, converts it to persistent (previewTab.preview = false). If already persistent, just activates. Otherwise creates new persistent tab via this.open().

Both are correctly implemented.

_renderTabs() — BUG: Missing `preview` CSS Class

_renderTabs() at line 7792:

el.className = "tab-item" + (tab.id === this._activeTabId ? " active" : "");

The preview class is NEVER added. The CSS has rules for .tab-item.preview:

/* style.css line 5450 */
.tab-item.preview .tab-name { font-style: italic; opacity: 0.75; }
.tab-item.preview { background: var(--bg-hover); }

But since the JS never adds el.classList.add("preview") when tab.preview is true, preview tabs do not render in italic.

The fix should be:

el.className = "tab-item" + (tab.id === this._activeTabId ? " active" : "") + (tab.preview ? " preview" : "");

Additionally, the click handler on tabs at line 7824 calls this.activate(tab.id) — activating a preview tab should work fine since activate() already properly handles loading file content.

Issue 3: Search Performance — "Rebuilding Inverted Index..."

InvertedIndex Architecture

backend/search.py (lines 239–425):

InvertedIndex is a singleton stored in module variable _inverted_index (line 415).
is_stale() (line 270): returns True when _indexer._index_generation != self._source_generation.
rebuild() (line 278): logs "Rebuilding inverted index...", iterates over the entire index dict (all vaults, all files), rebuilds all internal structures from scratch. Sets self._source_generation = _indexer._index_generation at line 345.

get_inverted_index() (line 418): check-then-rebuild every call:

def get_inverted_index() -> InvertedIndex:
    if _inverted_index.is_stale():
        _inverted_index.rebuild()
    return _inverted_index

What triggers `_index_generation` increments?

backend/indexer.py — the counter increments at 7 sites, every time the index mutates:

Line	Trigger
336	`rebuild_index()` — full index rebuild on startup or manual reindex
375	Per-vault processing within `rebuild_index()` (once per vault)
468	`reindex_vault()` — single vault reindex
601	`_remove_file_from_structures()` — single file deletion
667	`_add_file_to_structures()` — single file addition
794	`remove_vault_from_index()` — vault removal
839	`add_vault_to_index()` — vault addition

Frequent Rebuild Trigger: File Watcher Hot-Reload

backend/main.py (lines 376–420) — _on_vault_change():

Each file watcher event (create/modify/delete/move) calls update_single_file(), remove_single_file(), or handle_file_move().
These call _remove_file_from_structures() + _add_file_to_structures(), each incrementing _index_generation.
The watcher debounces for 2 seconds (backend/watcher.py line 100), then dispatches all batched events at once (line 207–210).
Each event in the batch still increments _index_generation independently.

Result: If you save 5 files simultaneously, _index_generation jumps by up to 5. The next search/autocomplete call triggers a full inverted index rebuild. On a vault with thousands of files, each rebuild is expensive.

Which API calls trigger the rebuild?

get_inverted_index() is called from:

advanced_search() at search.py line 684 (the TF-IDF advanced search)
suggest_titles() at search.py line 862 (autocomplete)
main.py line 2518 (stats/diagnostic endpoint)

The basic search() function (line 430+) does NOT use get_inverted_index() — it scans the raw index dict directly. But suggest_titles() is called on every keystroke in the search bar.

Why frequent saves cause repeated rebuilds

User saves file → watcher fires modified event → _on_vault_change → update_single_file → _index_generation += 1 (or +2 for add+remove).
User types in search bar → suggest_titles() → get_inverted_index() → is_stale() returns True → full rebuild.
User saves again → generation increments → next keystroke triggers another full rebuild.

This is the core performance issue: single-file index mutations invalidate the entire inverted index, and it gets rebuilt from scratch on the very next query.

Architecture Summary

Data Flow for Dashboard

index.html (full dashboard DOM)
  → renderFile() destroys dashboard-home via innerHTML
  → activate() hides remaining dashboard ref
  → _showDashboard() tries to re-append dashboard-home
  → showWelcome() rebuilds from scratch (only 2/4 sections)
  → Stats/Conflicts widgets silently no-op (target elements missing)

Data Flow for Search Performance

File change → watcher debounce (2s) → _on_vault_change()
  → update_single_file() → _remove + _add → _index_generation += 2
  → OR remove_single_file() → _index_generation += 1
User types → suggest_titles() → get_inverted_index()
  → is_stale() checks generation counter
  → if counter changed: full rebuild of InvertedIndex (O(N) over all files)

Start Here

Open frontend/app.js at line 5585 (showWelcome()) — this is the dashboard rebuild function and the primary cause of Issue 1. The fix needs to:

Include all 4 dashboard sections in showWelcome()'s rebuild HTML
OR: store the dashboard DOM before renderFile() wipes it, and restore it in _showDashboard()

For Issue 2, open frontend/app.js at line 7792 (_renderTabs) — add the missing preview class.

For Issue 3, open backend/search.py at line 418 (get_inverted_index) and backend/indexer.py lines 600–667 — consider whether single-file incremental updates to the inverted index are feasible, or whether the generation counter should be debounced/coalesced for file watcher batches.

12 KiB Raw Permalink Blame History Unescape Escape