Projets/ObsiViewer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
ObsiViewer/docs/SEARCH/MEILISEARCH_SETUP.md

7.8 KiB
Raw Blame History

🔍 Meilisearch Setup Guide

Complete guide for testing and using the Meilisearch integration in ObsiViewer.

Quick Start (5 minutes)

1. Start Meilisearch

npm run meili:up

This launches Meilisearch in Docker on port 7700.

2. Index Your Vault

npm run meili:reindex

This indexes all .md files from your vault. Progress will be logged to console.

3. Test the API

# Simple search
curl "http://localhost:4000/api/search?q=obsidian"

# With operators
curl "http://localhost:4000/api/search?q=tag:work path:Projects"

# All notes
curl "http://localhost:4000/api/search?q=*&limit=5"

4. Enable in Angular (Optional)

Edit src/core/logging/environment.ts:

export const environment = {
  USE_MEILI: true,  // ← Change this to true
  // ...
};

Then restart your Angular dev server: npm run dev

Testing Checklist

✅ Backend Tests

  • Docker started: docker ps shows obsiviewer-meilisearch running
  • Indexing works: npm run meili:reindex completes without errors
  • Basic search: curl "http://localhost:4000/api/search?q=*" returns hits
  • Tag filter: curl "http://localhost:4000/api/search?q=tag:yourTag" works
  • Path filter: curl "http://localhost:4000/api/search?q=path:YourFolder" works
  • File filter: curl "http://localhost:4000/api/search?q=file:readme" works
  • Highlights: Response includes _formatted with <mark> tags
  • Facets: Response includes facetDistribution with tags, parentDirs
  • Performance: npm run bench:search shows P95 < 150ms

✅ Incremental Updates

Test that Chokidar automatically updates Meilisearch:

  1. Start server: node server/index.mjs
  2. Create a new .md file in vault with tag #test-meilisearch
  3. Search: curl "http://localhost:4000/api/search?q=tag:test-meilisearch"
  4. Verify the new file appears in results
  5. Edit the file, search again - changes should be reflected
  6. Delete the file, search again - should no longer appear

✅ Angular Integration

With USE_MEILI: true:

  1. Open app in browser: http://localhost:4200
  2. Use search bar
  3. Open browser DevTools Network tab
  4. Verify requests go to /api/search instead of local search
  5. Results should show server-side highlights

Query Examples

Basic Searches

# Find all notes
curl "http://localhost:4000/api/search?q=*"

# Simple text search
curl "http://localhost:4000/api/search?q=obsidian"

# Phrase search
curl "http://localhost:4000/api/search?q=search%20engine"

Operators

# Single tag
curl "http://localhost:4000/api/search?q=tag:dev"

# Multiple tags (both must match)
curl "http://localhost:4000/api/search?q=tag:dev tag:angular"

# Path filter
curl "http://localhost:4000/api/search?q=path:Projects/Angular"

# File name search
curl "http://localhost:4000/api/search?q=file:readme"

# Combined: tag + path + text
curl "http://localhost:4000/api/search?q=tag:dev path:Projects typescript"

Typo Tolerance

# Intentional typos (should still match)
curl "http://localhost:4000/api/search?q=obsever"  # → "observer"
curl "http://localhost:4000/api/search?q=searh"    # → "search"

Pagination & Sorting

# Limit results
curl "http://localhost:4000/api/search?q=*&limit=10"

# Offset (page 2)
curl "http://localhost:4000/api/search?q=*&limit=10&offset=10"

# Sort by date (newest first)
curl "http://localhost:4000/api/search?q=*&sort=updatedAt:desc"

# Sort by title
curl "http://localhost:4000/api/search?q=*&sort=title:asc"

Performance Benchmarking

Run the benchmark suite:

npm run bench:search

This tests 5 different queries with 20 concurrent connections for 10 seconds each.

Expected Results (on modern dev machine with 1000 notes):

  • P95: < 150ms ✅
  • Average: 50-100ms
  • Throughput: 200+ req/sec

If P95 exceeds 150ms, check:

  1. Is Meilisearch running? docker ps
  2. Is the index populated? Check /health endpoint
  3. System resources (CPU/RAM)
  4. Adjust typoTolerance or searchableAttributes in meilisearch.client.mjs

Troubleshooting

Meilisearch won't start

# Check if port 7700 is already in use
netstat -an | findstr 7700  # Windows
lsof -i :7700               # macOS/Linux

# View Meilisearch logs
docker logs obsiviewer-meilisearch

# Restart Meilisearch
npm run meili:down
npm run meili:up

Search returns empty results

# Check if index exists and has documents
curl http://localhost:7700/indexes/notes_vault/stats \
  -H "Authorization: Bearer dev_meili_master_key_change_me"

# Reindex if needed
npm run meili:reindex

"Connection refused" errors

Ensure Meilisearch is running:

docker ps | grep meilisearch
# Should show: obsiviewer-meilisearch ... Up ... 0.0.0.0:7700->7700/tcp

Changes not reflected immediately

Wait 1-2 seconds after file changes for Chokidar to process. Check server logs:

[Meili] Upserted: Projects/MyNote.md

Performance issues

  1. Reduce typo tolerance: Edit server/meilisearch.client.mjs

    typoTolerance: {
  enabled: true,
  minWordSizeForTypos: {
    oneTypo: 5,    // Was 3
    twoTypos: 9    // Was 6
  }
}

  2. Limit searchable attributes: Remove headings or properties.* if not needed

  3. Increase batch size: Edit server/meilisearch-indexer.mjs

    const batchSize = 1000;  // Was 750

Environment Variables

Development (local)

.env or shell:

VAULT_PATH=./vault
MEILI_HOST=http://127.0.0.1:7700
MEILI_API_KEY=dev_meili_master_key_change_me

Docker Compose

docker-compose/.env:

MEILI_MASTER_KEY=dev_meili_master_key_change_me
MEILI_ENV=development

Production

Important: Change the master key in production!

MEILI_MASTER_KEY=$(openssl rand -base64 32)
MEILI_ENV=production

API Reference

GET /api/search

Query Parameters:

  • q (string, required): Search query with optional operators
  • limit (number, default: 20): Max results to return
  • offset (number, default: 0): Pagination offset
  • sort (string): Sort field and direction, e.g., updatedAt:desc
  • highlight (boolean, default: true): Include <mark> highlights

Response:

{
  "hits": [
    {
      "id": "Projects/MyNote.md",
      "title": "My Note",
      "path": "Projects/MyNote.md",
      "file": "MyNote.md",
      "tags": ["dev", "typescript"],
      "excerpt": "First 500 chars...",
      "_formatted": {
        "title": "My <mark>Note</mark>",
        "content": "...search <mark>term</mark>..."
      }
    }
  ],
  "estimatedTotalHits": 42,
  "facetDistribution": {
    "tags": { "dev": 15, "typescript": 8 },
    "parentDirs": { "Projects": 42 }
  },
  "processingTimeMs": 12,
  "query": "note"
}

POST /api/reindex

Triggers a full reindex of all markdown files in the vault.

Response:

{
  "ok": true,
  "indexed": true,
  "count": 1247,
  "elapsedMs": 3421
}

Next Steps

P1 Features (Future)

  • Multi-language support (configure locale in Meilisearch)
  • Synonyms configuration
  • Stop words for common terms
  • Advanced operators: line:, section:, [property]:value
  • Faceted search UI (tag/folder selectors)
  • Search result pagination in Angular
  • Search analytics and logging
  • Incremental indexing optimization (delta updates)

Performance Optimization

  • Redis cache layer for frequent queries
  • CDN for static assets
  • Index sharding for very large vaults (10k+ notes)
  • Lazy loading of highlights (fetch on demand)

Support

If you encounter issues:

  1. Check server logs: node server/index.mjs (console output)
  2. Check Meilisearch logs: docker logs obsiviewer-meilisearch
  3. Verify environment variables are set correctly
  4. Ensure vault path is correct and contains .md files
  5. Test with curl before testing in Angular