# Excalidraw Implementation - Obsidian Format Support

## Overview

ObsiViewer now fully supports Obsidian's Excalidraw plugin format, including:
- ✅ Reading `.excalidraw.md` files with LZ-String compressed data
- ✅ Writing files in Obsidian-compatible format
- ✅ Preserving front matter and metadata
- ✅ Round-trip compatibility (ObsiViewer ⇄ Obsidian)
- ✅ Backward compatibility with legacy flat JSON format
- ✅ Migration tool for converting old files

## File Format

### Obsidian Format (`.excalidraw.md`)

```markdown
---
excalidraw-plugin: parsed
tags: [excalidraw]
---
==⚠  Switch to EXCALIDRAW VIEW in the MORE OPTIONS menu of this document. ⚠==

# Excalidraw Data

## Text Elements
%%
## Drawing
```compressed-json
<LZ-STRING_BASE64_COMPRESSED_DATA>
```
%%
```

The `compressed-json` block contains the Excalidraw scene data compressed using **LZ-String** (`compressToBase64`/`decompressFromBase64`).

### Legacy Format (`.excalidraw`, `.json`)

Plain JSON format:
```json
{
  "elements": [...],
  "appState": {...},
  "files": {...}
}
```

## Architecture

### Backend (`server/`)

#### `excalidraw-obsidian.mjs`
Core utilities for parsing and serializing Excalidraw files:

- **`parseObsidianExcalidrawMd(md)`** - Parse Obsidian format, decompress LZ-String
- **`parseFlatJson(text)`** - Parse legacy JSON format
- **`toObsidianExcalidrawMd(data, frontMatter?)`** - Convert to Obsidian format
- **`extractFrontMatter(md)`** - Extract YAML front matter
- **`parseExcalidrawAny(text)`** - Auto-detect and parse any format
- **`isValidExcalidrawScene(data)`** - Validate scene structure

#### API Routes (`server/index.mjs`)

**GET `/api/files?path=<file>`**
- Accepts `path` as query parameter (properly URL-encoded)
- Returns parsed Excalidraw scene as JSON
- Supports `.excalidraw.md`, `.excalidraw`, `.json`
- Sets `ETag` header for conflict detection

**PUT `/api/files?path=<file>`**
- Accepts `path` as query parameter
- Content-Type: `application/json` (Excalidraw scene)
- Automatically converts to Obsidian format for `.excalidraw.md` files
- Preserves existing front matter
- Supports `If-Match` header for conflict detection
- Atomic write with backup (`.bak`)

**PUT `/api/files/blob?path=<file>`**
- For binary sidecars (PNG/SVG exports)
- Accepts `path` as query parameter

### Frontend (`src/app/features/drawings/`)

#### `excalidraw-io.service.ts`
Frontend parsing and serialization service:

- **`parseObsidianMd(md)`** - Parse Obsidian format
- **`parseFlatJson(text)`** - Parse legacy format
- **`toObsidianMd(data, frontMatter?)`** - Convert to Obsidian format
- **`parseAny(text)`** - Auto-detect format
- **`isValidScene(data)`** - Validate scene

#### `drawings-file.service.ts`
HTTP service for file operations:

- **`get(path)`** - Load Excalidraw file
- **`put(path, scene)`** - Save with conflict detection
- **`putForce(path, scene)`** - Force overwrite
- **`putBinary(path, blob, mime)`** - Save binary sidecar

All methods use query parameters for proper URL encoding.

#### `drawings-editor.component.ts`
Editor component with:

- Auto-save (debounced)
- Conflict detection and resolution
- Manual save (Ctrl+S)
- Export to PNG/SVG
- Theme synchronization

## Usage

### Opening Files

Files are automatically detected and parsed:

```typescript
// Backend automatically detects format
GET /api/files?path=drawing.excalidraw.md
// Returns: { elements: [...], appState: {...}, files: {...} }
```

### Saving Files

The backend automatically converts to Obsidian format:

```typescript
// Frontend sends JSON
PUT /api/files?path=drawing.excalidraw.md
Content-Type: application/json
{ elements: [...], appState: {...}, files: {...} }

// Backend writes Obsidian format with LZ-String compression
```

### Migration

Convert old flat JSON files to Obsidian format:

```bash
# Dry run (preview changes)
npm run migrate:excalidraw:dry

# Apply migration
npm run migrate:excalidraw

# Custom vault path
node server/migrate-excalidraw.mjs --vault-path=/path/to/vault
```

The migration script:
- Scans for `.excalidraw` and `.json` files
- Validates Excalidraw structure
- Converts to `.excalidraw.md` with Obsidian format
- Creates `.bak` backups
- Removes original files

## Testing

### Unit Tests

```bash
# Run backend utility tests
npm run test:excalidraw
```

Tests cover:
- ✅ Front matter extraction
- ✅ Obsidian format parsing
- ✅ LZ-String compression/decompression
- ✅ Round-trip conversion
- ✅ Legacy JSON parsing
- ✅ Edge cases (empty scenes, special characters, large files)

### E2E Tests

```bash
# Run Playwright tests
npm run test:e2e -- excalidraw.spec.ts
```

Tests cover:
- ✅ Editor loading
- ✅ API endpoints with query params
- ✅ Obsidian format parsing
- ✅ File structure validation

## Compatibility

### Obsidian → ObsiViewer
✅ Open `.excalidraw.md` files created in Obsidian  
✅ Render drawings correctly  
✅ Preserve all metadata and front matter

### ObsiViewer → Obsidian
✅ Save in Obsidian-compatible format  
✅ Files open correctly in Obsidian  
✅ No warnings or data loss  
✅ Front matter preserved

### Legacy Support
✅ Open old flat JSON files  
✅ Automatically convert on save  
✅ Migration tool available

## Key Changes from Previous Implementation

### Backend
- ❌ Removed `zlib` decompression (wrong algorithm)
- ✅ Added **LZ-String** support (`lz-string` package)
- ❌ Removed splat routes (`/api/files/*splat`)
- ✅ Added query param routes (`/api/files?path=...`)
- ✅ Proper URL encoding/decoding
- ✅ Front matter preservation
- ✅ Atomic writes with backups

### Frontend
- ✅ Created `ExcalidrawIoService` for parsing
- ✅ Updated all HTTP calls to use query params
- ✅ Proper `encodeURIComponent` usage
- ✅ No more 400 errors on special characters

## Troubleshooting

### 400 Bad Request
**Cause**: Path not properly encoded or missing query parameter  
**Fix**: Ensure using `?path=` query param, not URL path

### Invalid Excalidraw Format
**Cause**: Corrupted compressed data or wrong compression algorithm  
**Fix**: Check file was created with LZ-String, not zlib

### Conflict Detected (409)
**Cause**: File modified externally while editing  
**Fix**: Use "Reload from disk" or "Overwrite" buttons in UI

### File Opens in Obsidian but Not ObsiViewer
**Cause**: Unsupported Excalidraw version or corrupted data  
**Fix**: Check console for parsing errors, validate JSON structure

## Dependencies

- **Backend**: `lz-string` (^1.5.0)
- **Frontend**: `lz-string` (^1.5.0)

## Future Enhancements

- [ ] Support for Excalidraw libraries
- [ ] Embedded images optimization
- [ ] Collaborative editing
- [ ] Version history
- [ ] Template system

## References

- [Obsidian Excalidraw Plugin](https://github.com/zsviczian/obsidian-excalidraw-plugin)
- [LZ-String](https://github.com/pieroxy/lz-string)
- [Excalidraw](https://excalidraw.com/)