ObsiGate/context.md

# Code Context — ObsiGate Bug Investigation

## Files Retrieved
1. `frontend/app.js` (lines 5585–5665) — `showWelcome()` rebuilds dashboard HTML with only bookmarks + recent sections
2. `frontend/index.html` (lines 360–406) — Initial dashboard DOM has all 4 sections: stats, bookmarks, conflicts, recent
3. `frontend/app.js` (lines 7640–7672) — `TabManager.activate()` hides dashboard; `_showDashboard()` restores it
4. `frontend/app.js` (lines 7497–7597) — `TabManager.openPreview()` / `openPersistent()` definitions
5. `frontend/app.js` (lines 7778–7830) — `TabManager._renderTabs()` — **missing `preview` CSS class**
6. `frontend/style.css` (lines 5450–5457) — CSS rules for `.tab-item.preview` (italic)
7. `frontend/app.js` (lines 2340–2376) — Tree click handlers in `_renderDirectoryInContainer`
8. `frontend/app.js` (lines 3144–3310) — `renderFile()` overwrites `content-area.innerHTML`, destroying dashboard DOM
9. `frontend/app.js` (lines 3347–3380) — `DashboardStatsWidget` relies on `dashboard-stats-grid` element
10. `frontend/app.js` (lines 3381–3434) — `DashboardConflictsWidget` relies on `dashboard-conflicts-section` element
11. `backend/search.py` (lines 239–425) — `InvertedIndex` class, `is_stale()`, `rebuild()`, `get_inverted_index()`
12. `backend/indexer.py` (lines 28, 330–601, 634–667, 770–839) — `_index_generation` counter and all increment sites
13. `backend/watcher.py` (lines 191–226) — Debounce: dispatches batched events after `debounce_seconds` (2s default)
14. `backend/main.py` (lines 376–420) — `_on_vault_change()` calls `update_single_file` per event → each increments `_index_generation`
15. `backend/search.py` (lines 670–700, 844–875) — `advanced_search()` and `suggest_titles()` call `get_inverted_index()`

---

## Issue 1: Dashboard Layout — Sections Disappearing

### Root Cause: `showWelcome()` Rebuild Kills Stats & Conflicts Sections

**`showWelcome()`** (`frontend/app.js`, lines 5585–5665):
- It checks if `dashboard-home` or `dashboard-bookmarks-section` don't exist.
- If either is missing, it replaces `content-area.innerHTML` with **only two sections**: bookmarks and recent.
- **Stats and Conflicts sections are NOT included** in this rebuilt HTML.

```js
// line 5595 — only rebuilds if home or bookmarksSection is missing
if (area && (!home || !bookmarksSection)) {
  area.innerHTML = `
    <div id="dashboard-home" ...>
      <div id="dashboard-bookmarks-section" class="dashboard-section">...</div>
      <div id="dashboard-recent-section" class="dashboard-section">...</div>
    </div>`;
  // NOTE: No #dashboard-stats-section or #dashboard-conflicts-section!
}
```

**The original dashboard** (`frontend/index.html`, lines 360–406) has 4 sections:
1. `#dashboard-stats-section`
2. `#dashboard-bookmarks-section`
3. `#dashboard-conflicts-section`
4. `#dashboard-recent-section`

**When does the dashboard get destroyed?**
- `renderFile()` at line 3302: `area.innerHTML = "";` — this wipes `content-area` including `#dashboard-home` (which is a child of `content-area`).
- After the file renders, `activate()` at line 7642 hides the dashboard: `dashboard.style.display = "none"` — but at this point the dashboard DOM elements have been **removed from the DOM**.
- Later, when `_showDashboard()` (line 7764) is called:
  ```js
  _showDashboard() {
    const area = document.getElementById("content-area");
    area.innerHTML = "";          // clear file content
    const dashboard = document.getElementById("dashboard-home");
    if (dashboard) {
      dashboard.style.display = "";  // show it
      area.appendChild(dashboard);   // move back into DOM
    }
  }
  ```
  This works **only if** `dashboard-home` still exists in the document. But `renderFile()` called `area.innerHTML = ""` which destroyed it.

- Next time `showWelcome()` is called (e.g., after `goHome()`), it detects `!home` → rebuilds with **only 2 sections**.

**Why "Recently opened" stays visible when stats/bookmarks disappear:**
- The rebuilt HTML in `showWelcome()` **includes** `#dashboard-recent-section` and `#dashboard-bookmarks-section`, but **not** `#dashboard-stats-section` or `#dashboard-conflicts-section`.
- `DashboardStatsWidget.load()` (line 3348) looks for `document.getElementById("dashboard-stats-grid")` — if not found, it silently returns.
- `DashboardConflictsWidget.load()` (line 3383) looks for `document.getElementById("dashboard-conflicts-section")` — if not found, silently returns.
- So stats and conflicts sections vanish, while bookmarks and recent survive the rebuild.

### How the Dashboard Visibility Toggle Works
- `TabManager.activate()` (line 7642): `dashboard.style.display = "none"` — hides dashboard when a file tab is shown.
- `TabManager._showDashboard()` (line 7764): `dashboard.style.display = ""` + appends to `content-area` — shows dashboard when all tabs close.

### CSS for Dashboard Sections
`frontend/style.css` (lines 4739–4742):
```css
.dashboard-section {
  display: flex;
  flex-direction: column;
}
```
All `.dashboard-section` elements are just flex columns — there's no show/hide logic in CSS beyond what JavaScript sets via inline `style.display`.

---

## Issue 2: Click/Double-Click Handling

### Tree Click Handlers — Correctly Wired
In `_renderDirectoryInContainer` (`frontend/app.js`, lines 2354–2360):
```js
fileItem.addEventListener("click", () => {
  scrollTreeItemIntoView(fileItem, false);
  TabManager.openPreview(vaultName, item.path);   // single click → preview
  closeMobileSidebar();
});
fileItem.addEventListener("dblclick", (e) => {
  e.preventDefault();
  TabManager.openPersistent(vaultName, item.path); // double click → persistent
});
```
Same pattern at lines 3132–3133 (sidebar recent), 4909–4910 and 5090–5091 (search results). **Wiring is correct.**

### openPreview() and openPersistent() — Properly Defined
- `openPreview()` (line 7497): Creates tab with `preview: true`, closes existing preview, activates.
- `openPersistent()` (line 7533): If a preview tab exists for this file, converts it to persistent (`previewTab.preview = false`). If already persistent, just activates. Otherwise creates new persistent tab via `this.open()`.

**Both are correctly implemented.**

### _renderTabs() — BUG: Missing `preview` CSS Class
`_renderTabs()` at line 7792:
```js
el.className = "tab-item" + (tab.id === this._activeTabId ? " active" : "");
```
**The `preview` class is NEVER added.** The CSS has rules for `.tab-item.preview`:
```css
/* style.css line 5450 */
.tab-item.preview .tab-name { font-style: italic; opacity: 0.75; }
.tab-item.preview { background: var(--bg-hover); }
```
But since the JS never adds `el.classList.add("preview")` when `tab.preview` is true, **preview tabs do not render in italic**.

The fix should be:
```js
el.className = "tab-item" + (tab.id === this._activeTabId ? " active" : "") + (tab.preview ? " preview" : "");
```

Additionally, the `click` handler on tabs at line 7824 calls `this.activate(tab.id)` — activating a preview tab should work fine since `activate()` already properly handles loading file content.

---

## Issue 3: Search Performance — "Rebuilding Inverted Index..."

### InvertedIndex Architecture
`backend/search.py` (lines 239–425):
- `InvertedIndex` is a singleton stored in module variable `_inverted_index` (line 415).
- `is_stale()` (line 270): returns `True` when `_indexer._index_generation != self._source_generation`.
- `rebuild()` (line 278): logs `"Rebuilding inverted index..."`, iterates over the **entire** `index` dict (all vaults, all files), rebuilds all internal structures from scratch. Sets `self._source_generation = _indexer._index_generation` at line 345.
- `get_inverted_index()` (line 418): **check-then-rebuild every call**:
  ```python
  def get_inverted_index() -> InvertedIndex:
      if _inverted_index.is_stale():
          _inverted_index.rebuild()
      return _inverted_index
  ```

### What triggers `_index_generation` increments?
`backend/indexer.py` — the counter increments at **7 sites**, every time the index mutates:
| Line | Trigger |
|------|---------|
| 336 | `rebuild_index()` — full index rebuild on startup or manual reindex |
| 375 | Per-vault processing within `rebuild_index()` (once per vault) |
| 468 | `reindex_vault()` — single vault reindex |
| 601 | `_remove_file_from_structures()` — single file deletion |
| 667 | `_add_file_to_structures()` — single file addition |
| 794 | `remove_vault_from_index()` — vault removal |
| 839 | `add_vault_to_index()` — vault addition |

### Frequent Rebuild Trigger: File Watcher Hot-Reload
`backend/main.py` (lines 376–420) — `_on_vault_change()`:
- Each file watcher event (create/modify/delete/move) calls `update_single_file()`, `remove_single_file()`, or `handle_file_move()`.
- These call `_remove_file_from_structures()` + `_add_file_to_structures()`, **each incrementing `_index_generation`**.
- The watcher debounces for 2 seconds (`backend/watcher.py` line 100), then dispatches **all batched events at once** (line 207–210).
- Each event in the batch still increments `_index_generation` independently.

**Result**: If you save 5 files simultaneously, `_index_generation` jumps by up to 5. The next search/autocomplete call triggers a full inverted index rebuild. On a vault with thousands of files, each rebuild is expensive.

### Which API calls trigger the rebuild?
`get_inverted_index()` is called from:
- `advanced_search()` at `search.py` line 684 (the TF-IDF advanced search)
- `suggest_titles()` at `search.py` line 862 (autocomplete)
- `main.py` line 2518 (stats/diagnostic endpoint)

The basic `search()` function (line 430+) does **NOT** use `get_inverted_index()` — it scans the raw `index` dict directly. But `suggest_titles()` is called on every keystroke in the search bar.

### Why frequent saves cause repeated rebuilds
1. User saves file → watcher fires `modified` event → `_on_vault_change` → `update_single_file` → `_index_generation += 1` (or +2 for add+remove).
2. User types in search bar → `suggest_titles()` → `get_inverted_index()` → `is_stale()` returns `True` → **full rebuild**.
3. User saves again → generation increments → next keystroke triggers another full rebuild.

This is the core performance issue: **single-file index mutations invalidate the entire inverted index**, and it gets rebuilt from scratch on the very next query.

---

## Architecture Summary

### Data Flow for Dashboard
```
index.html (full dashboard DOM)
  → renderFile() destroys dashboard-home via innerHTML
  → activate() hides remaining dashboard ref
  → _showDashboard() tries to re-append dashboard-home
  → showWelcome() rebuilds from scratch (only 2/4 sections)
  → Stats/Conflicts widgets silently no-op (target elements missing)
```

### Data Flow for Search Performance
```
File change → watcher debounce (2s) → _on_vault_change()
  → update_single_file() → _remove + _add → _index_generation += 2
  → OR remove_single_file() → _index_generation += 1
User types → suggest_titles() → get_inverted_index()
  → is_stale() checks generation counter
  → if counter changed: full rebuild of InvertedIndex (O(N) over all files)
```

---

## Start Here
Open `frontend/app.js` at line 5585 (`showWelcome()`) — this is the dashboard rebuild function and the primary cause of Issue 1. The fix needs to:
1. Include all 4 dashboard sections in `showWelcome()`'s rebuild HTML
2. OR: store the dashboard DOM before `renderFile()` wipes it, and restore it in `_showDashboard()`

For Issue 2, open `frontend/app.js` at line 7792 (`_renderTabs`) — add the missing `preview` class.

For Issue 3, open `backend/search.py` at line 418 (`get_inverted_index`) and `backend/indexer.py` lines 600–667 — consider whether single-file incremental updates to the inverted index are feasible, or whether the generation counter should be debounced/coalesced for file watcher batches.