Bruno Charest bb365c60c1 ajout des docs architecture

2025-10-06 11:19:55 -04:00

17 KiB

Raw Blame History

🏗️ ARCHITECTURE CIBLE & SCHÉMAS

1. Diagramme Architecture Globale

┌─────────────────────────────────────────────────────────────────┐
│                         FRONTEND (Angular 20)                    │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐         │
│  │   UI Layer   │  │ Main Thread  │  │ Web Workers  │         │
│  │              │  │              │  │              │         │
│  │ • Components │  │ • Services   │  │ • Markdown   │         │
│  │ • OnPush CD  │  │ • Signals    │  │   Parser     │         │
│  │ • Virtual    │  │ • HTTP       │  │ • Search     │         │
│  │   Scroll     │  │ • State Mgmt │  │   Index      │         │
│  │              │  │              │  │ • Graph      │         │
│  └──────────────┘  └──────────────┘  │   Layout     │         │
│         │                  │          └──────────────┘         │
│         └──────────────────┴─────────────────┘                 │
│                            │                                    │
└────────────────────────────┼────────────────────────────────────┘
                             │ HTTP/JSON
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│                    BACKEND (Node.js Express)                     │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐         │
│  │  API Routes  │  │   Services   │  │   Watchers   │         │
│  │              │  │              │  │              │         │
│  │ • /api/vault │  │ • VaultSvc   │  │ • Chokidar   │         │
│  │ • /api/search│  │ • SearchSvc  │  │ • SSE Events │         │
│  │ • /api/log   │  │ • LogSvc     │  │              │         │
│  │ • /api/health│  │ • CacheSvc   │  │              │         │
│  └──────────────┘  └──────────────┘  └──────────────┘         │
│         │                  │                                    │
│         └──────────────────┴─────────────────┐                 │
│                                               ▼                 │
│                                    ┌──────────────────┐         │
│                                    │   Meilisearch    │         │
│                                    │   (Search Engine)│         │
│                                    │                  │         │
│                                    │ • Index vault_{} │         │
│                                    │ • Typo-tolerance │         │
│                                    │ • Facets         │         │
│                                    │ • Highlights     │         │
│                                    └──────────────────┘         │
└─────────────────────────────────────────────────────────────────┘
                             │
                             ▼
              ┌──────────────────────────┐
              │   Filesystem / Volumes   │
              │                          │
              │  • /app/vault (Obsidian) │
              │  • /app/db (logs, cache) │
              │  • /app/tmp              │
              └──────────────────────────┘

2. Schéma d'Index Meilisearch

Index par Voûte: `vault_{vaultId}`

Configuration Index:

{
  "primaryKey": "docId",
  "searchableAttributes": [
    "title",
    "content",
    "headings",
    "tags",
    "fileName",
    "path"
  ],
  "filterableAttributes": [
    "tags",
    "folder",
    "ext",
    "hasAttachment",
    "mtime",
    "size"
  ],
  "sortableAttributes": [
    "mtime",
    "title",
    "size"
  ],
  "rankingRules": [
    "words",
    "typo",
    "proximity",
    "attribute",
    "sort",
    "exactness"
  ],
  "typoTolerance": {
    "enabled": true,
    "minWordSizeForTypos": {
      "oneTypo": 4,
      "twoTypos": 8
    },
    "disableOnWords": [],
    "disableOnAttributes": []
  },
  "faceting": {
    "maxValuesPerFacet": 100
  },
  "pagination": {
    "maxTotalHits": 1000
  }
}

Structure Document:

interface MeilisearchDocument {
  docId: string;              // Format: {vaultId}:{relativePath}
  vaultId: string;            // Ex: "primary", "work", "personal"
  title: string;              // Note title
  fileName: string;           // Ex: "home.md"
  path: string;               // Ex: "folder1/subfolder/note"
  folder: string;             // Ex: "folder1/subfolder"
  ext: string;                // Ex: "md"
  content: string;            // Body markdown (stripped frontmatter)
  contentPreview: string;     // First 200 chars
  headings: string[];         // ["# Heading 1", "## Heading 2"]
  tags: string[];             // ["#project", "#important"]
  properties: Record<string, any>; // Frontmatter key-value
  hasAttachment: boolean;     // Contains ![[image.png]]
  linkCount: number;          // Number of outgoing links
  backlinksCount: number;     // Number of incoming links
  mtime: number;              // Unix timestamp
  size: number;               // Bytes
  createdAt: string;          // ISO 8601
  updatedAt: string;          // ISO 8601
}

3. Mapping Opérateurs Obsidian → Meilisearch

Opérateur Obsidian	Meilisearch Query/Filter	Exemple
`tag:#project`	`filter: "tags = '#project'"`	`?filter=tags%20%3D%20%27%23project%27`
`path:folder1/`	`filter: "folder = 'folder1'"`	`?filter=folder%20%3D%20%27folder1%27`
`file:home`	`filter: "fileName = 'home.md'"`	`?filter=fileName%20%3D%20%27home.md%27`
`-tag:#archive`	`filter: "tags != '#archive'"`	`?filter=tags%20!%3D%20%27%23archive%27`
`content:search term`	`q=search term&attributesToSearchOn=content`	Default search
`section:"My Heading"`	`q=My Heading&attributesToSearchOn=headings`	Section-specific
`task:TODO`	Custom backend filter (tasks extracted)	-
`has:attachment`	`filter: "hasAttachment = true"`	`?filter=hasAttachment%20%3D%20true`

Query combinée exemple:

Obsidian: tag:#project -tag:#archive path:work/ important

Meilisearch:
POST /indexes/vault_primary/search
{
  "q": "important",
  "filter": [
    "tags = '#project'",
    "tags != '#archive'",
    "folder = 'work'"
  ],
  "attributesToHighlight": ["content", "title"],
  "highlightPreTag": "<mark>",
  "highlightPostTag": "</mark>",
  "limit": 50
}

4. Routes API Backend

`/api/search` - Recherche unifiée

Endpoint: POST /api/search

Request:

interface SearchRequest {
  query: string;               // Raw Obsidian query
  vaultId?: string;            // Default: "primary"
  options?: {
    limit?: number;            // Default: 50
    offset?: number;           // Pagination
    attributesToRetrieve?: string[];
    attributesToHighlight?: string[];
  };
}

Response:

interface SearchResponse {
  hits: Array<{
    docId: string;
    title: string;
    path: string;
    _formatted?: {             // With highlights
      title: string;
      content: string;
    };
    _matchesPosition?: MatchRange[];
  }>;
  estimatedTotalHits: number;
  processingTimeMs: number;
  query: string;
  facetDistribution?: Record<string, Record<string, number>>;
}

`/api/search/suggest` - Autocomplete

Endpoint: GET /api/search/suggest?q=term&type=tag

Response:

interface SuggestResponse {
  suggestions: string[];
  type: 'tag' | 'file' | 'path' | 'property';
}

`/api/search/facets` - Facettes disponibles

Endpoint: GET /api/search/facets

Response:

interface FacetsResponse {
  tags: Array<{ name: string; count: number }>;
  folders: Array<{ name: string; count: number }>;
  extensions: Array<{ name: string; count: number }>;
}

`/api/search/reindex` - Réindexation manuelle

Endpoint: POST /api/search/reindex

Request:

interface ReindexRequest {
  vaultId: string;
  full?: boolean;              // true = rebuild complet, false = delta
}

Response:

interface ReindexResponse {
  taskId: string;              // Meilisearch task UID
  status: 'enqueued' | 'processing' | 'succeeded' | 'failed';
  indexedDocuments?: number;
  durationMs?: number;
}

`/api/log` - Logging endpoint

Endpoint: POST /api/log

Request:

interface LogBatchRequest {
  records: LogRecord[];
}

interface LogRecord {
  ts: string;                  // ISO 8601
  level: 'debug' | 'info' | 'warn' | 'error';
  app: string;                 // "ObsiViewer"
  sessionId: string;           // UUID
  userAgent: string;
  context: {
    version: string;
    route?: string;
    theme?: 'light' | 'dark';
    vault?: string;
  };
  event: LogEvent;
  data: Record<string, unknown>;
}

Response:

interface LogResponse {
  accepted: number;
  rejected: number;
  errors?: string[];
}

5. Plan /api/log - Événements Standardisés (12+)

Événements Frontend

Event	Trigger	Data Fields	Exemple
`APP_START`	App bootstrap	`viewport: {width, height}`	User opens app
`APP_STOP`	beforeunload	-	User closes tab
`PAGE_VIEW`	Route change	`route: string, previousRoute?: string`	Navigate to /graph
`SEARCH_EXECUTED`	Search submit	`query: string, queryLength: number, resultsCount?: number`	User searches "tag:#project"
`SEARCH_OPTIONS_APPLIED`	Filter toggle	`options: {caseSensitive, regex, wholeWord}`	Enable case-sensitive
`GRAPH_VIEW_OPEN`	Graph tab click	-	User opens graph view
`GRAPH_INTERACTION`	Node click/hover	`action: 'click'\|'hover', nodeId: string`	Click node in graph
`BOOKMARKS_OPEN`	Bookmarks panel	-	Open bookmarks
`BOOKMARKS_MODIFY`	Add/edit/delete	`action: 'add'\|'update'\|'delete', path: string`	Bookmark note
`CALENDAR_SEARCH_EXECUTED`	Calendar date select	`resultsCount: number, dateRange?: string`	Select date range
`ERROR_BOUNDARY`	Uncaught error	`message: string, stack?: string, componentStack?: string`	React error boundary
`PERFORMANCE_METRIC`	Web Vitals	`metric: 'LCP'\|'FID'\|'CLS', value: number`	Lighthouse metrics

Événements Backend (à implémenter)

Event	Trigger	Data Fields
`VAULT_INDEXED`	Meilisearch indexation complète	`vaultId: string, documentsCount: number, durationMs: number`
`VAULT_WATCH_ERROR`	Chokidar error	`vaultId: string, error: string`
`SEARCH_BACKEND_EXECUTED`	Meilisearch query	`query: string, hitsCount: number, processingTimeMs: number`

6. Stratégie Worker/WebGL pour Graph - Critères Anti-Gel

Problème actuel

Graph layout calcul dans Web Worker ✅ (bon)
Rendu Canvas dans main thread ❌ (peut bloquer)
Pas de LOD (Level of Detail) ❌
Pas de capping nodes/links ❌

Solution cible

A) Web Worker pour Layout (déjà implémenté)

graph-layout.worker.ts calcule positions avec d3-force
Communication via postMessage ✅
Garde-fou: Timeout 30s, max iterations 1000

B) Canvas Rendering Optimisé

Throttle redraw:

private lastDrawTime = 0;
private readonly MIN_FRAME_INTERVAL = 16; // 60fps max

private scheduleRedraw(): void {
  const now = performance.now();
  if (now - this.lastDrawTime < this.MIN_FRAME_INTERVAL) {
    return; // Skip frame
  }
  requestAnimationFrame(() => {
    this.draw();
    this.lastDrawTime = performance.now();
  });
}

LOD (Level of Detail):

private draw(): void {
  const zoom = this.transform().k;
  
  // Adaptive rendering based on zoom
  if (zoom < 0.5) {
    // Far view: circles only, no labels, no arrows
    this.drawNodesSimple(ctx, nodes);
  } else if (zoom < 1.5) {
    // Medium: circles + labels
    this.drawNodes(ctx, nodes);
    this.drawLabels(ctx, nodes);
  } else {
    // Close: full detail
    this.drawLinks(ctx, links, settings);
    this.drawNodes(ctx, nodes);
    this.drawLabels(ctx, nodes);
  }
}

Capping nodes/links:

private readonly MAX_VISIBLE_NODES = 500;
private readonly MAX_VISIBLE_LINKS = 1000;

private cullNodes(nodes: SimulationNode[]): SimulationNode[] {
  // Show only visible nodes in viewport + margin
  const visible = nodes.filter(n => this.isInViewport(n));
  
  if (visible.length > this.MAX_VISIBLE_NODES) {
    // Sort by importance (link count, selection state)
    return visible
      .sort((a, b) => b.importance - a.importance)
      .slice(0, this.MAX_VISIBLE_NODES);
  }
  
  return visible;
}

Clustering pour grands graphes (>1000 nodes):

interface ClusterNode {
  id: string;
  type: 'cluster';
  children: string[]; // Node IDs in cluster
  x: number;
  y: number;
  radius: number;
}

// Use force-cluster layout for >1000 nodes
if (nodes.length > 1000) {
  const clusters = this.clusterByFolder(nodes, settings.clusterThreshold);
  this.renderClusters(clusters);
}

Critères d'arrêt du gel UI:

✅ Aucune frame >50ms (pas de janky scroll)
✅ Redraw budget: <16ms par frame (60fps)
✅ Worker layout timeout: 30s max
✅ Progressive rendering: 100 nodes/frame si >500 total
✅ User interaction priority: stopper animation si clic détecté

7. Docker Multi-Stage + Healthcheck

Dockerfile optimisé:

# syntax=docker/dockerfile:1

# ========== Stage 1: Builder ==========
FROM node:20-alpine AS builder

WORKDIR /build

# Install dependencies (leverage layer cache)
COPY package*.json ./
RUN npm ci --only=production=false

# Copy source
COPY . .

# Build Angular (production)
RUN npx ng build --configuration=production

# Prune dev dependencies
RUN npm prune --omit=dev

# ========== Stage 2: Runtime ==========
FROM node:20-alpine AS runtime

# Security: non-root user
RUN addgroup -g 1001 -S nodejs && \
    adduser -S nodejs -u 1001

WORKDIR /app

# Install curl for healthcheck
RUN apk add --no-cache curl

# Copy artifacts from builder
COPY --from=builder --chown=nodejs:nodejs /build/dist ./dist
COPY --from=builder --chown=nodejs:nodejs /build/server ./server
COPY --from=builder --chown=nodejs:nodejs /build/node_modules ./node_modules
COPY --from=builder --chown=nodejs:nodejs /build/package*.json ./
COPY --from=builder --chown=nodejs:nodejs /build/db ./db

# Create volumes
RUN mkdir -p /app/vault /app/tmp /app/logs && \
    chown -R nodejs:nodejs /app

USER nodejs

# Expose port
EXPOSE 4000

# Healthcheck
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
  CMD curl -f http://localhost:4000/api/health || exit 1

# Environment defaults
ENV NODE_ENV=production \
    PORT=4000 \
    VAULT_PATH=/app/vault \
    LOG_LEVEL=info

# Start server
CMD ["node", "./server/index.mjs"]

Variables d'env clés:

# .env.example
PORT=4000
NODE_ENV=production
VAULT_PATH=/app/vault
VAULT_ID=primary
MEILISEARCH_URL=http://meilisearch:7700
MEILISEARCH_KEY=masterKey
LOG_LEVEL=info
LOG_ENDPOINT=http://localhost:4000/api/log
ENABLE_SEARCH_CACHE=true
CACHE_TTL_SECONDS=300

17 KiB Raw Blame History

🏗️ ARCHITECTURE CIBLE & SCHÉMAS

1. Diagramme Architecture Globale

2. Schéma d'Index Meilisearch

Index par Voûte: vault_{vaultId}

3. Mapping Opérateurs Obsidian → Meilisearch

4. Routes API Backend

/api/search - Recherche unifiée

/api/search/suggest - Autocomplete

/api/search/facets - Facettes disponibles

/api/search/reindex - Réindexation manuelle

/api/log - Logging endpoint

5. Plan /api/log - Événements Standardisés (12+)

Événements Frontend

Événements Backend (à implémenter)

6. Stratégie Worker/WebGL pour Graph - Critères Anti-Gel

Problème actuel

Solution cible

A) Web Worker pour Layout (déjà implémenté)

B) Canvas Rendering Optimisé

7. Docker Multi-Stage + Healthcheck

17 KiB

Raw Blame History

Index par Voûte: `vault_{vaultId}`

`/api/search` - Recherche unifiée

`/api/search/suggest` - Autocomplete

`/api/search/facets` - Facettes disponibles

`/api/search/reindex` - Réindexation manuelle

`/api/log` - Logging endpoint