Compare commits

...
Author SHA1 Message Date
bruno ffa1fa89ab fix: A25 + A21 (partiel) — plus d'exception muque, transaction protégée (v7.3.8)
FlowDeck CI / lint (push) Successful in 1m51s
FlowDeck CI / docker (push) Canceled after 0s
FlowDeck CI / test (push) Canceled after 10m12s
- A25 — 84 `except Exception: pass/…` → `logger.exception("<fonction>")`
  (19 fichiers : api_v2 30, dashboard 10, board 7, sites 5, workspace 5,
  api_v2_helpers 5, …) ; `logger` ajouté là où il manquait (api_v2_helpers,
  sites + `import logging`)
- A25 critique — les `try` autour de `materialize_properties` supprimés dans
  `create_collection_v2` ET `apply_db_template_v2` : un échec interrompt la
  transaction au lieu de commiter une collection sans schéma
- test `test_collection_rollback_when_materialize_fails` (Bearer v2, monkeypatch
  qui lève, assertions : RuntimeError + 0 ligne commitée)
- A21 partiel — `PRAGMA busy_timeout=5000` dans `get_conn()` (point d'entrée
  unique) ; commentaire `ponytail:` : le wrapper async + les 510 call sites
  restent à migrer module par module
- suite **1028/1028** · `ruff check app tests` OK
2026-10-01 08:16:42 -04:00
bruno 3ad2605c9e fix: A14 — fin du fallback « row admin » sur l'agent (v7.3.7)
FlowDeck CI / lint (push) Successful in 1m55s
FlowDeck CI / test (push) Successful in 20m33s
FlowDeck CI / docker (push) Canceled after 0s
- `_current_user_id` : 401 sans session (24 sites) au lieu de retomber sur
  `SELECT id FROM users WHERE login='admin'`
- `_current_admin` : suppression du même fallback — `PATCH /api/agent/providers`
  et `POST /api/agent/providers/test` (donc `LLMClient.ping(api_base=…)`)
  exigent une session admin : 401 sans session, 403 non-admin
- `_check_api_base()` sur les 2 routes : scheme http(s), pas d'identifiants
  dans l'URL (400) ; hôtes privés maintenus — Ollama `localhost:11434` est le
  provider par défaut du produit (commentaire `ponytail:` pour la fermeture)
- +1 test de non-régression → suite **1027/1027**, `ruff check app tests` OK
2026-10-01 07:53:06 -04:00
bruno 1f705ce512 fix: A19 terminé — plus aucun préfixe cookie-auth exempt du CSRF (v7.3.6)
FlowDeck CI / test (push) Successful in 20m10s
FlowDeck CI / lint (push) Successful in 1m51s
FlowDeck CI / docker (push) Canceled after 0s
- 46 appels non-GET équipés de `X-CSRF-Token` (expression cookie en ligne,
  portée indifférente) : agent_panel (9), settings (12), local_workspace (15),
  gitea_workspace (4), workspace (2), workspaces (5), library (2), welcome (2 en v7.3.5)
- 5 derniers préfixes sortis d'`EXCLUDED_PATHS` : /api/workspace (+/api/workspaces),
  /api/local-workspace, /api/settings, /api/gitea, /api/agent
- il ne reste exempté que : Bearer (/api/webhook, /api/v1, /api/v2, /scim/v2),
  callbacks /auth/*, pages publiques (/s/, /f/), /api/csrf-token et /api/frontend-error
- vérif : `node --check` des <script> des 39 templates → 0 échec (avant et après)
- tests : `anon_csrf()` sur les 5 tests d'anonymat devenus CSRF-first
- suite **1026/1026** · `ruff check app tests` OK
2026-10-01 07:41:23 -04:00
bruno cf76e00f12 docs(roadmap): A19 — 49 fetch restants (compte exact) au lieu de 51
FlowDeck CI / docker (push) Successful in 1m48s
FlowDeck CI / lint (push) Successful in 1m49s
FlowDeck CI / test (push) Successful in 21m14s
2026-09-30 23:38:28 -04:00
bruno 0861f1fdbf fix: A19 (partiel) — CSRF réservé aux vrais cas d'exemption (v7.3.5)
- Scan de tous les appels `fetch` non-GET du front par préfixe : 12 préfixes
  n'ont AUCUN call site sans `X-CSRF-Token` → sortis de `EXCLUDED_PATHS`
  (`/db/`, `/workspace`, `/workspace/automations`, `/board/api/favorites`,
  `/api/pages`, `/api/recents`, `/api/notifications`, `/api/comments`,
  `/api/automations`, `/api/github`, `/api/admin`, `/api/onboarding`)
- `welcome.html` : les 2 POST `/api/onboarding/*` reçoivent le header
  (`adminFetch` prouve que `/api/admin` était déjà couvert)
- reste A19 (front à équiper) : `/api/workspace` (+`/api/workspaces`),
  `/api/local-workspace`, `/api/settings`, `/api/gitea`, `/api/agent` — 49 fetch
- tests : helper `anon_csrf()` (anonyme + CSRF valide) pour isoler les 401 de
  route du 403 middleware — 4 tests d'anonymat ajustés
- suite **1026/1026** · `ruff check app tests` OK
2026-09-30 23:38:03 -04:00
bruno 72fcef2ba9 fix: A16 — ACL sur l'export et les pièces jointes (v7.3.4)
FlowDeck CI / docker (push) Successful in 1m44s
FlowDeck CI / lint (push) Successful in 1m49s
FlowDeck CI / test (push) Successful in 21m5s
- `export.py` : `_load_page_or_404(request, page_id)` — 401 sans session,
  404 si `PermissionManager.can_view_page` refuse (les 4 formats d'export)
- `dashboard.py` : helper `_require_page_view` sur `GET /api/pages/{id}/download`
  et `GET /api/pages/{id}/file-content`
- +1 test de non-régression → suite **1026/1026**, `ruff check app tests` OK
2026-09-30 23:20:26 -04:00
bruno 5a537f5dc3 fix: A12–A24 — SSRF, auth routes legacy, uploads, N+1 et routes doublonnes (v7.3.3)
FlowDeck CI / docker (push) Successful in 1m44s
FlowDeck CI / lint (push) Successful in 1m53s
FlowDeck CI / test (push) Successful in 20m45s
- A12 — `og_fetcher` : GET sans `follow_redirects`, `_is_public_host` revérifié à
  chaque saut (max 5) ; `POST /board/api/og/metadata` → 400 sur hôte privé/loopback
- A13 — router automations sous `Depends(_require_session)` (CRUD, run,
  press-button) + `created_by` sans fallback ; action `webhook` validée par
  `_is_public_host` avant POST (SSRF)
- A15 — webhooks sortants : `_require_admin` sur GET/POST/DELETE + `_is_public_host`
  sur l'URL en création
- A17 — router legacy `/api` sous `Depends(_require_session_or_bearer)` (session ou
  Bearer `/api/v1`), allowlist explicite `/api/health` + `/api/frontend-error`
- A22 — les 2 uploads locales : session exigée (`_require_user_id`) + `validate_upload`
  branché (taille + extension) + `FLOWDECK_DATA_DIR` au lieu de `/data` codé en dur
- A23 — N+1 : COUNT→`GROUP BY` (dashboard), cards→`executemany` (board sync),
  duplicata de propriétés→`executemany` + remap des ids par SELECT (collections)
- A24 — 2 routes écrasées supprimées : `GET /api/projects` (api.py) et
  `GET /workspace` (workspace.py) + test « aucun doublon méthode+chemin »
- Tests : +9 dans `tests/test_audit_p0_fixes.py` (SSRF, 401s, validate_upload,
  doublons de routes) ; tests OG sur hôtes résolubles (la garde fait du DNS)
- suite **1025/1025** · `ruff check app tests` OK
2026-09-30 23:12:20 -04:00
bruno 8ab6569974 fix: A11 + A18 — path traversal avatar et XSS/flags sur la vue publique (v7.3.2)
FlowDeck CI / docker (push) Successful in 1m44s
FlowDeck CI / lint (push) Successful in 1m49s
FlowDeck CI / test (push) Successful in 22m0s
- A11 : `GET /api/settings/avatar/{filename:path}` → `resolve()` + `relative_to()` (motif de `serve_uploaded_file`), 403 hors de `/data/avatars`
- A18 : `GET /workspace/public/{id}` → 404 HTML explicite pour `permission_type` restricted/private, `html.escape` sur le nom, l'icône et les titres de lignes (le f-string HTML ne passe pas par Jinja2)
- `tests/test_audit_p0_fixes.py` : 3 tests de non-régression (traversal, échappement, hidden restricted)
- ROADMAP A11/A18 cochés · CHANGELOG/WORKLOAD/VERSION → 7.3.2 · suite **1019/1019** · `ruff check app tests` OK
2026-09-30 22:40:34 -04:00
bruno 69a0aceba6 fix: A10 — autoescape Jinja2 activé partout via un Environment partagé
FlowDeck CI / docker (push) Successful in 1m44s
FlowDeck CI / lint (push) Successful in 1m49s
FlowDeck CI / test (push) Successful in 21m54s
- `app/templating.py` : `ENV = Environment(loader=FileSystemLoader("app/templates"), autoescape=select_autoescape(["html"]))`
- les 29 instantiations `Environment(loader=FileSystemLoader("app/templates"))` (9 routers) remplacées par `env = ENV` — plus aucune interpolation `{{ … }}` servie crue, les `|safe` redeviennent efficaces
- re-tri des `|safe` : `card_detail.html` corps d'issue et commentaires échappés (XSS stocké), placeholder de description sorti du ternaire, `sidebar_config` passé en dict + `|tojson` (échappement `</script>` en contexte script)
- `|safe` conservé sur `right_actions` (HTML fabriqué dans les templates, fiable)
- `ruff check app tests` OK · suite **1016/1016 verts**
2026-09-30 22:28:48 -04:00
bruno d76d7943fc docs(roadmap): A3-A8 cochés — bloc fallback admin corrigé, suite 1016/1016
FlowDeck CI / lint (push) Successful in 2m0s
FlowDeck CI / test (push) Successful in 22m31s
FlowDeck CI / docker (push) Successful in 1m45s
Commit d125eb3 (code + tests).
2026-09-30 22:05:06 -04:00
bruno d125eb399e fix: A3–A8 — bloc « fallback admin » de l'audit sécurité (401 sans session, ACL, CSRF)
FlowDeck CI / lint (push) Successful in 1m49s
FlowDeck CI / test (push) Successful in 21m47s
FlowDeck CI / docker (push) Successful in 1m45s
- A3 : helper `_require_user_id()` (401 sans session) sur profile/password/token/forge ; `PUT /api/user/password` exige `current_password` vérifié ; `/api/user` sorti de la liste CSRF exemptée
- A4 : `POST /api/v1/token` → 401 sans session, chemin legacy `user_id=0` supprimé
- A5 : CRUD membres d'espace (POST/PUT/DELETE) : session + rôle admin de l'espace (ou admin global), placeholder user créé en `is_admin=0`, GET membres sans session → 401
- A6 : `_require_view` → 404 et `_require_edit` → 401 quand il n'y a pas de session (fin du legacy single-user sur les collections)
- A7 : création ET lecture de page → 401 sans session (PermissionManager conservé) ; `/board/api/pages` sorti de `EXCLUDED_PATHS` ; header CSRF manquant ajouté sur setItemIcon (local_workspace)
- A8 : seed admin sans mot de passe codé en dur — aléatoire au premier boot loggé une fois, ou `FLOWDECK_ADMIN_PASSWORD` ; re-seed seulement si absent
- tests : client connecté par défaut via `_TestSessionAuth` (session + CSRF injectés à la volée, jamais dans le cookie jar → plus de CookieConflict), helper `anon()` sur les 40 tests d'anonymat ; 1016/1016 verts, `ruff check app tests` OK
2026-09-30 22:04:13 -04:00
bruno e6c1f7dbb3 docs(roadmap): A1/A2/A9 cochés — deps, cycle commit+tag v7.3.0, désindexation .db + rotation secret
FlowDeck CI / lint (push) Successful in 1m49s
FlowDeck CI / test (push) Successful in 21m17s
FlowDeck CI / docker (push) Successful in 2m13s
Suite 1016/1016 verts.
2026-09-30 20:20:19 -04:00
bruno 465853ac59 fix(tests): A1 — fin du rebinding app.config.settings dans test_v54 (isolation rétablie, 1016/1016 verts)
FlowDeck CI / lint (push) Successful in 1m53s
FlowDeck CI / test (push) Failing after 3h8m45s
FlowDeck CI / docker (push) Skipped
Le rebind (`app.config.settings = Settings()`) laissait tous les modules déjà
importés (sso_provisioning, trash, …) sur un objet périmé : le test
test_v67_sso::test_env_config_fallback_when_table_empty échouait dès qu'il
tournait après test_v54 dans le même worker (-n auto). Mutation sur place
comme le préconise conftest.py.
2026-09-30 20:19:09 -04:00
bruno 1706ad1ee9 feat: v7.3.0 — cycle v6.8.0→v7.3.0 (Sites, Search, Automations, Calendar, SCIM, Wiki) + audit A9
FlowDeck CI / lint (push) Successful in 1m48s
FlowDeck CI / test (push) Failing after 21m19s
FlowDeck CI / docker (push) Skipped
- v6.8.0 Sites & Forms publics (migrations 24)
- v6.9.0 Recherche sémantique hybride + Ask AI (migration 25)
- v7.0.0 Automations v2 multi-étapes + Workers sandboxés (migration 26)
- v7.1.0 Calendar sync Google/CalDAV + Meeting Notes (migration 27)
- v7.2.0 Enterprise : SCIM 2.0, 2FA TOTP/passkeys, audit UI, agent approvals (migration 28)
- v7.3.0 Wiki/Teamspaces, verified pages, collab polish, charts, unfurl (migration 29)
- docs V68→V73, ROADMAP/CHANGELOG/WORKLOAD à jour, VERSION 7.3.0
- A9 : flowdeck.db, flowdeck_dev.db, test-commit.md, upload_test.txt et e2e/{node_modules,shots,test-results} désindexés + ignorés (.gitignore/.dockerignore)
2026-09-30 20:02:57 -04:00
bruno d074689b18 feat: v6.7.0 — SSO/SAML + OIDC entreprise (SP SAML signé + OIDC PKCE, auto-provisioning, groupes IdP → rôles workspace, SSO only, onglet Settings « SSO / Enterprise », API /api/v2/sso/*, help, migration 23, docs + OpenAPI 439 chemins) · 802 tests verts
FlowDeck CI / lint (push) Successful in 1m28s
FlowDeck CI / test (push) Successful in 11m52s
FlowDeck CI / docker (push) Successful in 1m51s
2026-09-24 13:32:17 -04:00
bruno 9562f30366 feat: v6.6.0 — Agent phase 5 : API publique agent (/api/v2/agents, run synchrone JSON) + marketplace skills (export/import portable + galerie de 6 presets, palette / du panneau) + webhooks agent.run.started/failed · 764 tests verts
FlowDeck CI / docker (push) Successful in 1m21s
FlowDeck CI / lint (push) Successful in 1m27s
FlowDeck CI / test (push) Successful in 11m5s
2026-09-24 10:16:24 -04:00
bruno 6dfd6d718e feat: v6.5.1 — 7 tests webhooks_v2 dé-skipés (0 skip, 749 verts) + roadmap rattrapée (sync.py Bearer coché)
FlowDeck CI / docker (push) Successful in 1m20s
FlowDeck CI / lint (push) Successful in 1m27s
FlowDeck CI / test (push) Successful in 10m59s
2026-09-24 09:10:08 -04:00
bruno 401d0b17ca merge: feat/v6.5.0-synced-db → main (v6.5.0 Synced blocks production)
FlowDeck CI / docker (push) Successful in 1m21s
FlowDeck CI / lint (push) Successful in 1m28s
FlowDeck CI / test (push) Successful in 11m0s
2026-09-24 08:28:33 -04:00
bruno 5951c707eb feat: v6.5.0 Synced blocks production — pages contenu par lignes de DB, résolution serveur à chaque lecture, propagation écrite réelle
FlowDeck CI / lint (push) Successful in 1m27s
FlowDeck CI / test (push) Successful in 10m51s
FlowDeck CI / docker (push) Successful in 1m21s
2026-09-24 08:28:25 -04:00
bruno f2f2f3209e feat: v6.4.0 Realtime production — merge 3-voix (au-delà du LWW) + broadcast non bloquant
FlowDeck CI / lint (push) Successful in 1m25s
FlowDeck CI / test (push) Successful in 10m20s
FlowDeck CI / docker (push) Successful in 1m21s
- app/services/realtime_merge.py : merge à 3 voix diff3-lite, regions disjointes conservees, conflit par champ + drapeau
- protocole base (client embarque la base de sa saisie) ; sans base -> LWW historique (retro-compat)
- ack renvoie le bloc fusionne + conflict ; adoption cote client + toast ; broadcast du resultat fusionne
- broadcast non bloquant : file sortante + tache writer par connexion, coalescence des curseurs
- clients trop lents deconnectes (4413), budget ops anti-flood (400/10s)
- fix fuite room 4404 + room_state() sur page inexistante
- GET /api/realtime/stats (observabilite)
- 26 tests test_realtime_v64.py ; suite 725 verte ; ruff + eslint OK ; version 6.4.0
2026-09-23 23:55:39 -04:00
bruno f2e5684e4e fix(test): declare pytest-asyncio dep for webhook async tests
FlowDeck CI / lint (push) Successful in 1m18s
FlowDeck CI / test (push) Successful in 10m6s
FlowDeck CI / docker (push) Successful in 1m18s
2026-09-22 00:27:34 -04:00
bruno b56b181c3e chore: fix ruff lint (webhooks v2) + Windows-safe test teardown
FlowDeck CI / lint (push) Successful in 1m21s
FlowDeck CI / test (push) Failing after 10m1s
FlowDeck CI / docker (push) Skipped
2026-09-21 21:58:47 -04:00
bruno 2fceed0da2 docs: add v5.15.0 Webhooks v2 to roadmap
FlowDeck CI / lint (push) Failing after 1m12s
FlowDeck CI / test (push) Failing after 10m8s
FlowDeck CI / docker (push) Skipped
2026-09-21 21:36:58 -04:00
bruno 436898d86d feat: add Webhooks v2 with HMAC signature, retries (2s/10s/60s), and 20+ new events
FlowDeck CI / lint (push) Failing after 1m11s
FlowDeck CI / test (push) Failing after 9m59s
FlowDeck CI / docker (push) Skipped
- Add HMAC SHA-256 signature verification for webhook payloads
- Implement retry logic with delays (2s, 10s, 60s) and max 4 attempts
- Add 20+ new events (total ~50 events) covering pages, blocks, users, etc.
- Add API v2 endpoints for testing HMAC signature and retrying deliveries
- Add comprehensive test suite for webhooks v2 functionality

Generated by opencode.
2026-09-21 21:30:57 -04:00
bruno e0237e576f Fire automation events across API routers
FlowDeck CI / lint (push) Failing after 1m12s
FlowDeck CI / test (push) Failing after 3h3m3s
FlowDeck CI / docker (push) Skipped
2026-09-21 20:30:05 -04:00
bruno 189ed5bbca chore: track opencode.json + e2e diag shots
FlowDeck CI / lint (push) Failing after 1m10s
FlowDeck CI / test (push) Failing after 3h8m55s
FlowDeck CI / docker (push) Skipped
2026-09-21 08:18:43 -04:00
bruno ce0d561ade feat(share,permissions): partage de page par groupes (page_shares)
FlowDeck CI / lint (push) Failing after 1m10s
FlowDeck CI / test (push) Successful in 9m55s
FlowDeck CI / docker (push) Successful in 1m11s
- app/db.py: ajout colonne shared_with_group_id (FK user_groups, migration
  backfill v5.x) dans page_shares
- app/routers/sharing.py: POST /api/pages/{id}/share accepte group_id
  (upsert, verif FK groupe), GET /shares expose kind/group_name, synchro
  bidirectionnelle avec page_permissions (mirror grant/revoke) pour que
  PermissionManager donne un acces effectif (view/comment/edit) aux membres
  du groupe; PUT/DELETE gardent le miroir a jour
- app/routers/board.py, library.py: received/made incluent les partages via
  groupes (JOIN group_members)
- app/templates/_page_editor_content.html, _page_editor_scripts.html:
  dialogue Share — invite groups (fetch /api/v2/groups, filtre deja partages),
  pickInviteGroup, shareInvite(group_id), rendu accessList avec avatar groupe
- tests/test_share_groups.py: 10 tests (CRUD groupe, kind, miroir ACL)

Chore: inclut evolutions v6.4.0 deja en working copy (webhooks prod,
migrations, sync, config/main) pour garder l'arbre coherent.
2026-09-21 06:38:02 -04:00
bruno 95bc861cdb feat: v6.3.0 API publique complete v2 (REST /api/v2, scopes, OpenAPI)
FlowDeck CI / lint (push) Successful in 1m13s
FlowDeck CI / test (push) Successful in 9m20s
FlowDeck CI / docker (push) Successful in 1m10s
- Router api_v2.py (~100 endpoints) : tokens, users, workspaces/members,
  collections, pages, proprietes, vues/dashboards, commentaires/mentions,
  notifications, favoris/tags/recents, partage/publish, historique, sprints,
  templates, export/import, forges, recherche FTS, admin, webhooks CRUD
- Helpers api_v2_helpers.py : Bearer unifie (sha256/expires_at/extension_devices),
  scopes hierarchiques read<write<admin, pagination + X-Total-Count, ISO-8601,
  RFC 7807, idempotence, audit, rate-limit par token
- Migration 20 : api_tokens.scopes/expires_at, webhook_deliveries,
  api_audit_log, idempotency_keys
- main.py : handler d'erreurs unifie StarletteHTTPException, /docs + /redoc
- config : PUBLIC_API_INSECURE_OK (dev only), API_V2_RATE_LIMIT_PER_TOKEN
- OpenAPI docs/openapi-v2.json (402 chemins), tests/test_public_api_v2.py (24)
- Docs : CHANGELOG (v6.2.0/6.2.1 clipper + v6.3.0), ROADMAP, API_GUIDE_V6,
  V6_Web_Clipper, README, ARCHITECTURE, /help
- Suite complete 668 verte, ruff OK
2026-09-20 13:19:29 -04:00
bruno ea19d1d050 fix(web-clipper): bouton rond transparent draggable + toggle affichage + refresh auto sidebar + fix bloc bookmark (v6.2.1)
FlowDeck CI / lint (push) Successful in 1m14s
FlowDeck CI / test (push) Failing after 8m44s
FlowDeck CI / docker (push) Skipped
- bouton flottant rond (44px), semi-transparent blur, hover plus fonce, deplacable souris (pos persistee storage)
- option afficher/cacher dans popup extension (showButton)
- clipper: notification instantanee des onglets FlowDeck via tabs.sendMessage + polling 15s + BroadcastChannel + visibilitychange
- fix bookmark template literal '+title+' -> \/\/\/\/\ (_page_editor_scripts.html)
- sync static/extension + zip
2026-09-20 11:02:49 -04:00
bruno 7f998faf7b fix(editor): corriger SyntaxError duplicate inner dans _page_editor_scripts.html (v6.2.0)
FlowDeck CI / lint (push) Successful in 1m11s
FlowDeck CI / test (push) Failing after 8m39s
FlowDeck CI / docker (push) Skipped
- Renomme var inner -> syncedInner dans le bloc synced pour lever Uncaught SyntaxError Identifier inner has already been declared
- Ce SyntaxError cassait le parsing de tout le script editor -> editorState/loadCoverIcon/etc. not defined
- Les pages clippees paraissaient vides a cause du JS casse; apres fix le rendu des blocks (bookmark/image/paragraph) fonctionne
- Rebuild Docker OK (v6.2.0, migration 19 deja appliquee)
2026-09-20 00:22:10 -04:00
bruno 0b251649e5 feat: v6.2.0 Web Clipper — extension navigateur (capture article/selection/bookmark/screenshot)
FlowDeck CI / lint (push) Successful in 1m11s
FlowDeck CI / test (push) Failing after 8m32s
FlowDeck CI / docker (push) Skipped
- Service app/services/web_clipper.py: sanitize HTML, html->blocks, extraction article, creation page workspace-aware, rate limit 50/h, device registration
- Router app/routers/web_clipper.py: POST /api/v2/web-clipper/clip, GET /status, POST /auth/verify, GET/DELETE /devices, GET /extensions (download page), auth via session ou Bearer (api_tokens / extension_devices)
- Migration 19: extension_devices + extension_clips (+ indexes)
- Extension Manifest V3: content.js (floating button, selection), background.js (clip + contextMenus), popup.html/js, clipper.css, icons
- Settings UI: onglet Extensions (liste devices, revoke, test clip, liens download), page /extensions
- Tests: 16 tests web_clipper (sanitize, blocks, article/bookmark/selection/screenshot, bearer, rate-limit, devices, extensions page)
- Bump version 6.1.0 -> 6.2.0
2026-09-19 23:26:03 -04:00
bruno 13d5f8625a feat: v6.1.0 granular permissions (page/collection/property ACL + groups + audit)
FlowDeck CI / lint (push) Failing after 1m8s
FlowDeck CI / test (push) Failing after 8m5s
FlowDeck CI / docker (push) Skipped
- Migration 18: 6 tables + 3 colonnes permission_type + indexes
- PermissionManager: heritage page->collection->workspace, least privilege, groups, cache 60s
- API /api/v2: pages/collections/properties/groups/users/audit (401/403/404/400)
- Guards board.py + collections.py (404/403, admin/owner bypass)
- Tests 21/21 (inherit/restricted/private, grant, revoke, batch, group, audit)
- Docs + ROADMAP + CHANGELOG + VERSION 6.1.0
2026-09-19 22:54:16 -04:00
bruno d707a6850a merge: feat/v6.0 into main (v6.0.0 PWA offline support)
FlowDeck CI / lint (push) Successful in 1m6s
FlowDeck CI / test (push) Successful in 7m7s
FlowDeck CI / docker (push) Successful in 1m0s
2026-09-18 13:34:20 -04:00
bruno f1ce34a8a6 fix: lire la version du log de demarrage depuis VERSION
FlowDeck CI / lint (push) Successful in 1m7s
FlowDeck CI / test (push) Successful in 7m7s
FlowDeck CI / docker (push) Successful in 1m1s
2026-09-18 13:15:46 -04:00
bruno b5207216f1 feat: v6.0.0 PWA offline support
- manifest + icones, service worker (precache, network-first, Background Sync)

- module client FlowOffline (IndexedDB, queue, delta, flush) + hook editeur

- endpoints /api/v2/sync/{delta,batch,status} + moteur de sync (conflits LWW/orpheline/copie offline)

- migrations offline_sync_queue + sync_version (triggers)

- UI offline (banner, badge sync, toasts, icone dirty) + doc /help

- tests pytest (sync, migrations, SW, offline) + E2E Playwright; bump 6.0.0
2026-09-18 13:05:40 -04:00
bruno 62620ef884 docs: marquer v5.14.0 Synced blocks comme COMPLETÉ
FlowDeck CI / lint (push) Successful in 1m5s
FlowDeck CI / test (push) Successful in 6m25s
FlowDeck CI / docker (push) Successful in 59s
2026-09-18 07:41:40 -04:00
bruno b0cb3a3923 fix: corriger erreurs Ruff lint I001 et W292
FlowDeck CI / lint (push) Successful in 1m4s
FlowDeck CI / test (push) Successful in 6m28s
FlowDeck CI / docker (push) Successful in 58s
2026-09-17 20:38:37 -04:00
bruno e6afa004d0 fix: update FastAPI version to 5.14.0
FlowDeck CI / lint (push) Failing after 58s
FlowDeck CI / test (push) Successful in 6m24s
FlowDeck CI / docker (push) Successful in 58s
2026-09-17 20:13:05 -04:00
brunoandFlowDeck bf3d1ac0cc feat: v5.14.0 Synced blocks - block created once, edited everywhere
FlowDeck CI / lint (push) Failing after 57s
FlowDeck CI / test (push) Successful in 6m34s
FlowDeck CI / docker (push) Successful in 59s
- Table synced_blocks (source of truth) + page_synced_blocks (references)
- Migration 15 + service app/services/synced_blocks.py
- API endpoints: CRUD synced blocks, add/remove page references
- Editor: /synced slash command, synced block rendering with badge
- Realtime: _propagate_synced broadcasts updates to all referencing rooms
- Export: synced blocks resolved in Markdown/HTML/PDF
- 18 tests in tests/test_v514_synced_blocks.py

Co-authored-by: FlowDeck <[email protected]>
2026-09-17 20:08:09 -04:00
bruno f9da57c9e0 fix(agent): remonter le corps de reponse des erreurs HTTP LLM
FlowDeck CI / lint (push) Successful in 1m1s
FlowDeck CI / test (push) Successful in 5m51s
FlowDeck CI / docker (push) Successful in 47s
Un 4xx (ex. 403 Mistral) affichait seulement 'Client error 403 Forbidden'. Le message d'erreur inclut desormais le corps renvoye par le fournisseur (modele non autorise, region bloquee, etc.) pour le test de connexion et la recuperation des modeles.
2026-09-14 23:21:21 -04:00
bruno 88e4ae1db8 fix(agent): purge les api_base LLM obsoletes (Mistral/Cohere /v2)
FlowDeck CI / lint (push) Successful in 53s
FlowDeck CI / test (push) Successful in 5m46s
FlowDeck CI / docker (push) Successful in 48s
Un api_base stocke (ex. https://api.mistral.ai/v2) ecrasait l'URL par defaut corrigee et faisait echouer le test de connexion. Desormais: normalisation a l'ecriture (une base egale au defaut n'est pas stockee), possibilite de vider le champ (api_base='' vs None), et migration 14 qui efface les bases obsoletes/redondantes dans user_llm_keys et llm_config.

Tests de non-regression ajoutes.
2026-09-14 23:04:39 -04:00
bruno a0db4d6e65 fix(agent): URLs OpenAI-compatibles validees pour les fournisseurs LLM
FlowDeck CI / lint (push) Successful in 57s
FlowDeck CI / test (push) Successful in 6m5s
FlowDeck CI / docker (push) Successful in 48s
Cohere -> /compatibility/v1, Google Gemini -> /v1beta/openai, Perplexity -> host racine, Chutes -> llm.chutes.ai/v1, SenseNova -> compatible-mode/v1. Mise a jour des modeles par defaut (xAI grok-4.6, Fireworks deepseek-v4-pro-0813, Cohere command-a-plus).

Retrait de LTX Studio (aucune API chat-completions) et MemTensor/MemOS (API memoire non OpenAI-compatible, auth Token + /chat). Google utilise desormais l'auth Bearer pour lister les modeles. Test de non-regression ajoute.
2026-09-14 22:50:36 -04:00
bruno fb14c7e709 feat(agent): 24 fournisseurs LLM + refonte du panneau Agent & IA
FlowDeck CI / lint (push) Successful in 53s
FlowDeck CI / test (push) Successful in 5m47s
FlowDeck CI / docker (push) Successful in 48s
Enregistre OpenAI, Anthropic, Mistral, Cohere, Google Gemini, Groq, DeepSeek, OpenRouter, NVIDIA NIM, Together, Perplexity, xAI, DashScope, MiniMax, Morph, Fireworks, Cerebras, SambaNova, Chutes, Xiaomi, LTX, SEA-LION, SenseNova et MemTensor (URLs de base + presets de modeles + libelles).

Settings: panneau Agent & IA repense en maitre/detail (liste rechercheable + volet de configuration) pour tenir avec des dizaines de fournisseurs, en conservant tous les boutons/fonctions.
2026-09-14 22:25:57 -04:00
bruno 0d475c3d2d fix(workspace): compteurs de tags dynamiques + counts scoped par workspace
FlowDeck CI / lint (push) Successful in 53s
FlowDeck CI / test (push) Successful in 5m47s
FlowDeck CI / docker (push) Successful in 48s
2026-09-14 20:26:27 -04:00
bruno 98af112ba1 fix(workspace): tags list inclut tous les tags utilisateur (count 0 workspace) pour menu contextuel
FlowDeck CI / lint (push) Successful in 55s
FlowDeck CI / test (push) Successful in 5m52s
FlowDeck CI / docker (push) Successful in 47s
2026-09-14 19:51:21 -04:00
bruno 7096707b3e fix(workspace): tags count update dynamique via /api/local-workspace/tags
FlowDeck CI / lint (push) Successful in 53s
FlowDeck CI / test (push) Successful in 5m46s
FlowDeck CI / docker (push) Successful in 49s
2026-09-14 19:33:54 -04:00
bruno 9ab47d8113 fix(workspace): menu contextuel après navigation partielle + sync sidebar/header au rename
FlowDeck CI / lint (push) Successful in 52s
FlowDeck CI / test (push) Failing after 3h14m19s
FlowDeck CI / docker (push) Skipped
2026-09-14 18:20:43 -04:00
bruno 41c1d315d3 feat(workspace,editor): creation dans le dossier courant + sync live du titre (sidebar/header)
FlowDeck CI / lint (push) Successful in 54s
FlowDeck CI / test (push) Successful in 5m50s
FlowDeck CI / docker (push) Successful in 47s
- creation fichier/dossier a la racine du dossier courant ou d'un dossier cible (context-menu, boutons de survol) via createPage/showCreateFolderModal(parentId)
- instances de modeles en tant qu'enfant d'un dossier (parent_id) ; nom vide -> 'Untitled'
- sidebar de la librairie rafraichi apres delete/move/duplicate/rename (_syncSidebar)
- editeur: le titre se synchronise en direct dans le sidebar, le breadcrumb et l'onglet (pages et fichiers) et persiste via PUT /board/api/pages/{id}
2026-09-14 17:05:03 -04:00
bruno 4038e9bdad fix(editor): repair block identity for template pages (duplicate/reordered lines)
FlowDeck CI / lint (push) Successful in 52s
FlowDeck CI / test (push) Successful in 5m46s
FlowDeck CI / docker (push) Successful in 47s
Template-created pages (weekly report, project doc, meeting notes, to-do
list, ...) were persisted without block ids. The editor assigned ids
client-side, but the realtime room loaded the raw id-less content and sent
it back on 'sync'; applySync then merged id-less server blocks with local
blocks, producing data-bid='undefined' collisions and duplicated/shuffled
lines as soon as the user edited. Editing an empty page was unaffected
because the server state was empty.

Fixes:
- board.use_page_template: materialize unique block ids (recursively) when
  instantiating built-in or user templates.
- realtime_server: unique block_id() (uuid) + ensure_block_ids() on room
  load and on insert ops.
- editor: recursive ensureBlockIds() in init; gtTok() now restores
  [[fddate:...]] tokens (date chips survived as labels before).
- realtime client: applySync() normalizes ids, no longer appends unknown
  local blocks (duplication), and keeps local content when server is empty.

Tests: pytest (templates + realtime) and Playwright e2e covering to-do
list, weekly report date chip, Enter ordering and legacy id-less repair.
2026-09-14 11:45:44 -04:00
bruno 334a937507 fix(templates): persist workspace context so template pages actually save
FlowDeck CI / lint (push) Successful in 51s
FlowDeck CI / test (push) Successful in 5m46s
FlowDeck CI / docker (push) Successful in 46s
Pages created via the page-template picker (weekly report, project doc,
meeting notes, ...) previously got workspace_id=NULL and workspace=login,
so they never appeared in the active local/Gitea workspace tree — they
looked like they 'didn't save' even though the row existed.

Frontend: _useTemplate now sends the active workspace context (workspace_id
or workspace key), mirroring _createPlainPage.
Backend: use_page_template resolves/validates the workspace and persists
both workspace and workspace_id on the new page.
2026-09-14 11:03:21 -04:00
bruno c7d4fd901f feat(e2e): ajouter tests Playwright flux utilisateur core (login, workspace, palette, dashboard)
FlowDeck CI / lint (push) Successful in 51s
FlowDeck CI / test (push) Successful in 5m55s
FlowDeck CI / docker (push) Successful in 47s
- e2e/flowdeck_e2e_final.spec.js: 4 tests E2E validés
  1. Login UI local (#email, #password, .btn-primary)
  2. Créer workspace local + entrer (modal .dialog-input)
  3. Ouvrir palette Ctrl+K et valider 'Créer une collection'
  4. Dashboard accessible
- e2e/ : package.json, playwright.config.js, install chromium
2026-09-14 09:53:31 -04:00
bruno 7794a03934 fix(tests): rendre la suite hermétique — pin oauth_redirect_uri + FLOWDECK_DATA_DIR temporel
FlowDeck CI / lint (push) Successful in 55s
FlowDeck CI / test (push) Successful in 6m7s
FlowDeck CI / docker (push) Successful in 44s
- test_get_redirect_uri_from_host_header: épingle oauth_redirect_uri à vide
  pour tester la dérivation Host de façon isolée (le .env du projet définit
  OAUTH_REDIRECT_URI, qui passe prioritaire par design)
- conftest: pose FLOWDECK_DATA_DIR vers un répertoire temporel inscriptible
  pour les tests emoji/docx/covers qui dépendaient de /data (conteneur)
- ajoute scripts/audit_functional.py: audit de bout-en-bout des processus
  (notes, DB, tâches, partage, publication, export, recherche, agents)

Suite locale: 538 passed
2026-09-14 07:45:33 -04:00
bruno 9dfc38706c feat(wiki,templates): v5.11.0 wiki-links & mentions + v5.12.0 templates & page lock (release 5.12.0)
FlowDeck CI / lint (push) Successful in 50s
FlowDeck CI / test (push) Successful in 5m41s
FlowDeck CI / docker (push) Successful in 45s
v5.11.0 Wiki-links & mentions de page :
- tokens [[fdpage:ID]] / [[fddate:ISO]] dans le texte des blocs,
  service app/services/wiki_links.py (labels, rendu HTML, extraction)
- taper [[ ouvre le picker de pages (recherche floue, clavier) ;
  le menu @ gagne les sections Pages et Date (today/tomorrow/YYYY-MM-DD)
- chips atomiques contenteditable=false relues en tokens par gtTok()
  (autosave/drag/undo preservent les liens) ; renommage propage via
  GET /board/api/wiki/titles ; backlinks reconnaissent les tokens ;
  page publique rend les chips (echopee)

v5.12.0 Templates & verrouillage :
- template picker global sur + New page : 5 built-in
  (app/services/block_templates.py) + templates perso
  (table page_global_templates, migration 13)
- POST /board/api/page-templates (save current page) + /{id}/use
  (instantiate, id 0 = built-in par cle)
- page lock : POST /api/pages/{id}/lock, garde _ensure_page_editable
  -> 423 en ecriture pour les non-privileged, deblocage par
  locked_by ou admin seulement (403 sinon), banniere + read-only UI
- full-width / small text par page (pages.full_width/font_small,
  POST /api/pages/{id}/options, classes CSS)
- migration 13 : is_locked, locked_by, full_width, font_small,
  page_global_templates

Tests : tests/test_v511_v512_wiki_templates.py (15) ; suite complete
538 verte ; ruff OK ; node --check des templates JS OK.
2026-09-14 06:38:09 -04:00
bruno d4adf89db5 feat(calendar): v5.8.0 calendrier & rappels + v5.7.0 database avancee (pt.2)
FlowDeck CI / lint (push) Successful in 49s
FlowDeck CI / test (push) Successful in 5m26s
FlowDeck CI / docker (push) Successful in 43s
v5.8.0 (release 5.11.7) — Calendrier & Rappels :
- moteur de recurrence RRULE subset (daily/weekly/monthly, interval,
  count, until, byweekday, timezone) — app/services/recurrence.py
- vues calendar Jour / Semaine / Mois avec expansion des occurrences
  cote serveur (GET /db/{id}/calendar/api) et popover evenement
  (Time / Timezone / Repeat / Remind)
- rappels avant echeance (scan 60 s, table reminder_log, in-app +
  email, cible = personnes assignees) — app/services/reminders.py
- fuseaux horaires : users.timezone + reglages in-app, timezone par
  evenement, liste de zones (GET /db/timezones/api)
- notifications d'assignation sur PUT /db/pages/{id}/api et
  preferences etendues (reminders, assignments)
- template Meeting notes enrichi (Agenda, Notes — migration 12)
- migrations 11-12 ; 20 tests dedies ; suite complete 523 verte

v5.7.0 (release 5.11.6, termine avant cette session, reste dans
l'arbre sans commit) — Database Avancée Pt.2 :
- proprietes person + auto-proprietes (created/last-edited time & by)
- groupes de proprietes, vues sauvegardees par utilisateur
- swimlanes, WIP limits, cartes configurables, calendar drag & drop,
  gallery couvertures ; 12 tests dedies
2026-09-13 22:49:14 -04:00
bruno 055956a351 fix(palette): corriger les options de la palette de recherche inutilisables au clic (le re-rendu au survol detruisait l'element sous le curseur)
FlowDeck CI / lint (push) Successful in 48s
FlowDeck CI / test (push) Successful in 5m10s
FlowDeck CI / docker (push) Successful in 44s
2026-09-13 20:31:30 -04:00
bruno 3b3e95e23a fix(cloudflare): proteger les scripts inline des pages du Rocket Loader
FlowDeck CI / lint (push) Successful in 48s
FlowDeck CI / test (push) Successful in 5m2s
FlowDeck CI / docker (push) Successful in 42s
Rocket Loader (Cloudflare) reecrit les balises script inline en type=...-text/javascript et les execute de facon differee, ce qui casse l'enregistrement des composants Alpine et l'init des pages lors d'un acces direct via le tunnel (ex. /settings ne se chargeait pas au complet). Ajout de data-cfasync=false sur les scripts inline des pages completes et des partiels du shell (settings, accounts, trash, workspace, board, gitea_workspace, welcome, import, page_editor_collection/embed, _database_table_scripts, _notification_bell, public_page, local_workspace). Les fragments charges via HTMX restent inchanges.
2026-09-13 12:35:28 -04:00
bruno 37337a5de7 fix(settings): corriger le chargement du panneau via navigation partielle HTMX
FlowDeck CI / lint (push) Successful in 47s
FlowDeck CI / test (push) Successful in 5m0s
FlowDeck CI / docker (push) Successful in 41s
settingsInit est declare comme fonction globale au lieu d'etre enregistre dans alpine:init : l'evenement alpine:init ne se redeclenche pas lors d'un swap HTMX de .main-wrapper, ce qui laissait settingsInit non enregistre et provoquait des ReferenceError (llmSaving, llmTesting, llmDefaultProvider, llmMsgOk) au chargement des Reglages depuis un bouton du topbar. Le bloc du token API cree passe aussi de x-show a template x-if pour eviter le dereferencement de newToken null.
2026-09-13 11:50:17 -04:00
bruno e8797afa05 merge: feat/v5.6.0-import into main (v5.6.0 data import, phases 0-5)
FlowDeck CI / lint (push) Successful in 48s
FlowDeck CI / test (push) Successful in 4m55s
FlowDeck CI / docker (push) Successful in 1m8s
2026-09-13 10:43:27 -04:00
bruno 3b00cbc371 feat(import): v5.6.0 unified data import (phases 0-5)
- unified importer framework (app/services/importers/): normalized model,
  registry, common pipeline (hierarchy, attachments, collections, dedup),
  async jobs, dry-run preview, column->type mapping
- Phase 1: Obsidian, Notion, Logseq/Roam, HTML (Apple Notes/Bear/Ulysses/
  OneNote), Google Keep, generic Markdown
- Phase 2: typed CSV/TSV, Excel (openpyxl), generic JSON
- Phase 3: Word .docx (python-docx), PDF (pypdf), HTML folders
- Phase 4: Raindrop, Pocket, Readwise, Shaarli, Netscape bookmarks, .ics,
  OPML, Standard Notes, Gitea/GitHub issues (+labels/milestones)
- Phase 5: incremental re-sync (skip/update/duplicate), partial-error resume,
  forge repo files, URL web clipper (SSRF guard), batch multi-file + UI queue,
  Notion relation resolution, exportable JSON reports
- /import wizard, API /api/import/*, migration 9 (import_items, import_jobs)
- fix: property values stored by property id (correct DB view rendering)
- deps: openpyxl, beautifulsoup4, PyYAML, python-docx, pypdf
- 43 import tests; full suite 491 green; ruff clean
- bump version 5.11.5
2026-09-13 10:43:20 -04:00
bruno d986959927 fix(cloudflare): protect agent panel, workspaces & shell scripts from Rocket Loader; trust proxy headers
FlowDeck CI / lint (push) Successful in 47s
FlowDeck CI / test (push) Successful in 4m17s
FlowDeck CI / docker (push) Successful in 45s
2026-09-13 09:51:54 -04:00
bruno 714642363b style(sidebar): round floating peek corners and increase top/bottom inset
FlowDeck CI / lint (push) Successful in 45s
FlowDeck CI / test (push) Successful in 4m19s
FlowDeck CI / docker (push) Successful in 40s
2026-09-12 23:54:45 -04:00
bruno df78badd0c feat(sidebar): Notion-style floating peek card, pin toggle & drag-resize
FlowDeck CI / lint (push) Successful in 46s
FlowDeck CI / test (push) Successful in 4m24s
FlowDeck CI / docker (push) Successful in 44s
2026-09-12 23:46:17 -04:00
bruno 9836c85e24 feat(sidebar): Notion-style collapse/peek, desktop reopen button, Ctrl+\\ toggle
FlowDeck CI / lint (push) Successful in 45s
FlowDeck CI / test (push) Successful in 4m20s
FlowDeck CI / docker (push) Successful in 40s
Stop collapsing the sidebar on navigation (was hiding the nav when opening Shared pages). Show the hamburger in the top-left on desktop when collapsed, add a left-edge hover zone that floats the sidebar in and auto-collapses on mouse leave, and bind Ctrl/Cmd+\\ to toggle it.
2026-09-12 22:41:22 -04:00
bruno 61a33a7891 fix(alpine): resolve console errors on editor and agent panel
FlowDeck CI / lint (push) Successful in 45s
FlowDeck CI / test (push) Failing after 3h0m32s
FlowDeck CI / docker (push) Skipped
Initialize coverOpen/coverLoading in editorState, render import file info with x-if instead of x-show (null deref), split x-if/x-for on the agent mention menu, guard m.steps.length and use unique keys for the icon picker. Also reduce htmx swap responses to .main-wrapper so shell scripts (agent panel, base inline) are not re-executed on partial navigation.
2026-09-12 21:55:22 -04:00
bruno e707becbf8 feat(nav): partial HTMX navigation — swap main content without full reload
FlowDeck CI / lint (push) Successful in 45s
FlowDeck CI / test (push) Successful in 4m17s
FlowDeck CI / docker (push) Successful in 41s
Intercept internal links and load only .main-wrapper via htmx.ajax, keeping the sidebar/header shell in place. Adds history pushState/popstate handling, sidebar active-state resync and document title update. Moves page script blocks inside #main-content so they run on swap, and makes editor/board scripts idempotent. Routes navigateTo/breadcrumb/library/local-workspace through window.fdNavigate and fixes the duplicated navigateTo definition.
2026-09-12 21:26:56 -04:00
bruno da3e09c215 feat(icon): Notion-style icon picker + custom workspace emojis; fix side peek file content & ctx menu overflow
FlowDeck CI / lint (push) Successful in 45s
FlowDeck CI / test (push) Failing after 3h6m55s
FlowDeck CI / docker (push) Skipped
2026-09-12 20:05:53 -04:00
bruno 19e8ba0e4a fix: editor file mode — single-root x-if more menu + pasteBlocks brace + fileUrl init
FlowDeck CI / lint (push) Successful in 45s
FlowDeck CI / test (push) Successful in 4m9s
FlowDeck CI / docker (push) Successful in 38s
2026-09-12 14:54:44 -04:00
bruno 7a6c66a609 feat: add download + copy file content to context menus and editor
FlowDeck CI / lint (push) Successful in 45s
FlowDeck CI / test (push) Successful in 4m8s
FlowDeck CI / docker (push) Successful in 38s
- Add /api/pages/{id}/download and /api/pages/{id}/file-content endpoints
- Add Download + Copy content to shared context menu (_ctx_menu.html)
- Wire handlers in local-workspace and library context menus
- Add Download + Copy content to editor '...' menu for file pages
- Multi-block selection copy/cut/paste with Ctrl+C/X/V shortcuts
- Align page title with blocks start (CSS pseudo-element offset)
2026-09-12 12:53:17 -04:00
bruno 3bb17eb984 Merge branch 'feat/v5.5.0-embeds-media' into main
FlowDeck CI / lint (push) Successful in 44s
FlowDeck CI / test (push) Successful in 4m16s
FlowDeck CI / docker (push) Successful in 37s
2026-09-12 09:41:04 -04:00
bruno bdc15c7328 feat(v5.5.0): Embeds & Media riche - universal embeds, bookmark cards, lightbox, inline previews, cover & icon
FlowDeck CI / lint (push) Successful in 44s
FlowDeck CI / test (push) Successful in 4m10s
FlowDeck CI / docker (push) Successful in 37s
- embeds.py: provider detection/rewrite (YouTube, Vimeo, Figma, Maps, Docs,
  Loom, CodePen, Miro, Spotify, SoundCloud, Twitch, X/Twitter, Pinterest,
  Office) + resolve_embed/inline_kind/provider; POST /board/api/embed/resolve
- editor resolves pasted URLs and caches embed_src (persisted); renderer,
  public pages and MD/HTML/PDF exports prefer embed_src
- og_fetcher.py: robust meta parsing (any attribute order), favicon,
  injectable transport, network-safe fallback
- image lightbox with keyboard nav (arrows/Esc) in editor and public pages
- inline PDF/video/audio previews
- cover (URL or upload) & page icon endpoints
- fix broken editor API paths (/api/pages -> /board/api/pages) for cover,
  icon, versions, backlinks, import, move and OG metadata
- 47 tests in tests/test_v55.py; full suite 444 green; ruff clean
- version 5.11.2
2026-09-12 09:40:50 -04:00
bruno c435e277f2 Merge branch 'docs/roadmap-v5.4.0-completed' into main
FlowDeck CI / lint (push) Successful in 44s
FlowDeck CI / test (push) Successful in 4m2s
FlowDeck CI / docker (push) Successful in 37s
2026-09-12 09:23:21 -04:00
253 changed files with 72142 additions and 1456 deletions
+7
View File
@@ -14,3 +14,10 @@ build/
node_modules/
Dockerfile
.dockerignore
# A9 — jamais de DB ni de fichiers de test dans l'image
*.db
*.db-*
test-commit.md
upload_test.txt
e2e/
+29
View File
@@ -44,6 +44,11 @@ BACKUP_KEEP=30
PROJECT_SYNC_ENABLED=true
PROJECT_SYNC_INTERVAL_HOURS=1
# ── Public API v2 (v6.3.0) ──
# PUBLIC_API_INSECURE_OK=true autorise le token de dev fd-public-key (jamais en prod).
PUBLIC_API_INSECURE_OK=false
API_V2_RATE_LIMIT_PER_TOKEN=300
# ── Email notifications (v4.9.0) ──
# Laisser SMTP_HOST vide = pas d'envoi d'email (seulement les notifications in-app).
SMTP_HOST=
@@ -53,3 +58,27 @@ SMTP_PASSWORD=
SMTP_FROM=FlowDeck <[email protected]>
SMTP_USE_TLS=true
APP_BASE_URL=http://localhost:8080
# ── SSO / Enterprise (v6.7.0) ──
# Fallback de démarrage uniquement : dès qu'un admin enregistre une configuration
# dans Settings → Admin → SSO / Enterprise, la table `sso_config` prime sur le .env.
# Le bouton SSO n'apparaît sur la page de connexion que si une config est active.
# SSO_PROVIDER=saml # saml | oidc (vide = SSO désactivé)
# SSO_NAME=Company SSO # libellé du bouton
# SSO_ONLY=false # true = refuser le login local (les admins gardent le leur)
# SSO_AUTO_PROVISION=true # créer le compte au premier login SSO
# SAML :
# SSO_ENTITY_ID=https://idp.example.com/saml/metadata
# SSO_SSO_URL=https://idp.example.com/saml/sso
# SSO_SLO_URL=https://idp.example.com/saml/slo
# SSO_X509_CERTIFICATE=-----BEGIN CERTIFICATE-----
# SSO_SIGN_REQUESTS=false # signer les AuthnRequests / LogoutRequest
# OIDC :
# SSO_ISSUER_URL=https://auth.example.com/realms/flowdeck
# SSO_CLIENT_ID=
# SSO_CLIENT_SECRET=
# SSO_SCOPE=openid profile email
# Mapping (JSON) :
# SSO_ATTRIBUTE_MAPPING={"email":"email","full_name":"name","groups":"groups"}
# SSO_GROUPS_MAPPING=[{"sso_group":"FlowDeck Admins","workspace_role":"admin","workspace_id":1}]
# SSO_DEFAULT_WORKSPACE_ID=0
+9
View File
@@ -17,3 +17,12 @@ dist/
.ua/.trash-*/
.ua/.understandignore
uv.lock
# A9 — jamais de DB ni de fichiers de test dans git
*.db
*.db-*
test-commit.md
upload_test.txt
e2e/node_modules/
e2e/shots/
e2e/test-results/
+8 -3
View File
@@ -109,8 +109,11 @@ FlowDeck est un **clone de Notion** intégré à Gitea. Il recrée l'expérience
│ │ ├─ pages.py — /pages/... Pages CRUD │ │
│ │ ├─ collections.py — /db/... Collections │ │
│ │ ├─ editor.py — /api/editor/... Block editor │ │
│ │ ├─ private.py — /api/private/* Section privée │ │
│ │ ├─ public_api.py — /api/public/* Public API │ │
│ │ ├─ public_api.py — /api/v1 Public API v1 │ │
│ │ ├─ api_v2.py — /api/v2 Public API v2 │ │
│ │ │ — Bearer + scopes, CRUD complet │ │
│ │ ├─ web_clipper.py — /api/v2/web-clipper Web Clipper │ │
│ │ ├─ permissions.py — /api/v2 (ACL) Permissions │ │
│ │ ├─ workspace.py — Workspaces API + Gitea projets │ │
│ │ ├─ webhooks.py — /webhooks/... Gitea hooks │ │
│ │ └─ admin.py — /api/admin/* Admin users │ │
@@ -1705,6 +1708,8 @@ docker compose restart flowdeck
- **Automatisations** — Règles déclenchées sur événements (Notion-style)
- **Base de données avancée** — Relations inter-collections, rollups
- **Kanban flexible** — Colonnes custom, WIP limits
- **API publique REST** — Tokens d'accès pour intégrations tierces
- **API publique REST v2** — `/api/v2` (v6.3.0) : Bearer + scopes `read/write/admin`, CRUD complet, pagination, RFC 7807, idempotence, audit, OpenAPI (`/docs`, `docs/openapi-v2.json`) ; `/api/v1` lecture seule (compat)
- **API agent publique** — `/api/v2/agents/*` + `/api/v2/skills/*` (v6.6.0, agent phase 5) : wrappers Bearer sur `AgentEngine` (run synchrone JSON, journal + rollback, trigger externe) et marketplace de skills (export/import portable, galerie de presets) — `app/routers/api_v2_agent.py`, `app/services/skill_gallery.py`
- **SSO / SAML + OIDC entreprise** — v6.7.0 (Enterprise Auth, dernière feature v6.0.0) : SP SAML (`python3-saml`) + OIDC PKCE (`authlib`), auto-provisioning + mapping groupes IdP → rôles workspace, mode « SSO only », onglet admin « SSO / Enterprise », migration 23 (`sso_config`, `sso_login_history`, `sso_requests`), `/help` section SSO — `app/routers/sso.py`, `app/services/sso_provisioning.py`, `app/auth/providers/{saml,oidc}_provider.py`
- **Volume Docker persistant** — `/data` monté pour survie des données
- **PostgreSQL** — Migration optionnelle pour scaling
+1072
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -41,4 +41,4 @@ EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=5s --start-period=5s --retries=3 \
CMD curl -f http://localhost:8080/api/health || exit 1
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080", "--proxy-headers", "--forwarded-allow-ips", "*"]
+10 -6
View File
@@ -2,7 +2,7 @@
Clone complet de **Notion** intégré nativement à **Gitea** — Databases, Pages, Kanban, Calendar, Gallery, Timeline, List, Multi-Users.
> **v2.1.0** — API publique, Webhooks sortants, PWA
> **v6.7.0** — SSO / SAML + OIDC entreprise (auth fédérée IdP, auto-provisioning, group mapping, mode SSO only) · avant : v6.6.x agent API + marketplace, v6.5.x synced blocks, v6.4.0 realtime, PWA offline
## Quick Start
@@ -49,10 +49,14 @@ docker compose up -d
- **CSV Import/Export**
- **Public Sharing**: lien de partage lecture seule
### API & Intégrations (v2.1)
- **API publique REST**: `/api/v1` avec token auth
- **Webhooks sortants**: gestion + dispatcher d'événements
- **PWA**: manifest.json, prêt pour installation mobile
### API & Intégrations (v6.3–v6.6)
- **API publique REST v2**: `/api/v2` — CRUD complet, Bearer + scopes `read/write/admin`, pagination, filtres, erreurs RFC 7807, idempotence, audit — [guide](docs/API_GUIDE_V6.md) · OpenAPI `/docs`
- **API agent publique (v6.6)**: `/api/v2/agents/*` — agents, conversations, **run synchrone JSON**, journal d'actions + rollback, `trigger` externe
- **Marketplace de skills (v6.6)**: export/import portable + galerie de 6 presets installables (`/api/v2/skills/*`), section « Galerie » dans la palette `/` de l'agent
- **API publique v1**: `/api/v1` (lecture seule, compat)
- **Webhooks sortants**: gestion + dispatcher d'événements (CRUD v2)
- **Web Clipper**: extension navigateur Manifest V3 (article/sélection/bookmark/screenshot)
- **PWA**: manifest.json + service worker, offline support
### UI Notion-Style (v1.1–v1.2)
- Sidebar gauche avec sections hiérarchiques
@@ -85,7 +89,7 @@ DATABASE_URL=sqlite:////data/flowdeck.db
## Tests
```bash
python3 -m pytest tests/ -v # 73/73 passent
python3 -m pytest tests/ -v # 764/764 passent (0 skip)
```
## Roadmap
+605 -74
View File
@@ -404,11 +404,11 @@ app/
- **Rédiger un rapport hebdomadaire** — agrège les pages modifiées, génère le rapport
#### Plan de migration (5 phases)
1. **Phase 1 — Core Agent** : AgentEngine + LLMClient + 3 outils (search, read, create) + SSE streaming
1. **Phase 1 — Core Agent** ✅ (v4.10.0) : AgentEngine + LLMClient + 3 outils (search, read, create) + SSE streaming
2. **Phase 2 — UI** : agent_panel.html, historique conversations, sélecteur de modèle ✅ (v4.10.1)
3. **Phase 3 — Outils avancés** : 7 outils supplémentaires (views, properties, Gitea, uploads)
4. **Phase 4 — Autonomie** : custom agents, skills, déclencheurs planifiés
5. **Phase 5 — Plateforme** : API publique agent → intégrations tierces, marketplace skills
3. **Phase 3 — Outils avancés** ✅ (v4.10.0) : 7 outils supplémentaires (views, properties, Gitea, uploads)
4. **Phase 4 — Autonomie** ✅ (v4.10.0) : custom agents, skills, déclencheurs planifiés
5. **Phase 5 — Plateforme** ✅ (v6.6.0) : API publique agent → intégrations tierces, marketplace skills
#### Limitations (v1)
- Pas de modification concurrente (lock optimiste DB)
@@ -523,43 +523,132 @@ app/
- [x] **Import** — `POST /api/pages/import` (markdown) + `POST /api/pages/import/file` (.md/.txt/.zip d'export Notion) ; import CSV collections `POST /workspace/collections/{id}/import/csv`
- [x] **17 tests** `tests/test_v54.py` ; suite complète **397 verte**
### v5.5.0 — Embeds & Média Riche
### v5.5.0 — Embeds & Média Riche ✅ (2026-09-12)
> **Objectif** : parité avec les 60+ embeds Notion — contenu tiers rendu dans la page.
> **Source** : analyse Notion clone (delta v4 → v5.5, 2026-09-04).
> **Source** : analyse Notion clone (delta v4 → v5.5, 2026-09-04). **COMPLETED**.
- [ ] **Bloc Embed universel** — `/embed` : YouTube, Vimeo, Figma, Google Maps, Loom, X/Twitter, CodePen, Miro, Spotify, SoundCloud, Twitch, Office…
- [ ] **Bookmark cards** — aperçu riche des URLs (métadonnées OG : titre, image, description)
- [ ] **Image lightbox** — clic pour agrandir, navigation clavier/consultation plein écran
- [ ] **Previews inline** — PDF, vidéo, audio rendus directement dans la page
- [ ] **Cover & icône de page** — upload image de couverture + emoji/icône custom (déjà partiel ?)
- [x] **Bloc Embed universel** — `/embed` : YouTube (watch/shorts/youtu.be), Vimeo, Figma, Google Maps, Google Docs/Sheets/Slides, Loom, X/Twitter, CodePen, Miro, Spotify, SoundCloud, Twitch, Pinterest, Office…
- [x] **Bookmark cards** — aperçu riche des URLs (métadonnées OG : titre, image, description, site, favicon)
- [x] **Image lightbox** — clic pour agrandir, navigation clavier (←/→) + plein écran dans l'éditeur **et** les pages publiques
- [x] **Previews inline** — PDF, vidéo, audio rendus directement dans la page
- [x] **Cover & icône de page** — upload image de couverture (fichier ou URL) + emoji/icône custom
### v5.6.0 — Import de données (étendu)
> **Objectif** : compléter l'import de base (Markdown/CSV/Notion, déjà dans v5.4.0) par les formats pro et l'inférence de types.
Détails livrés :
- Service `app/services/embeds.py` : détection multi-provider + réécriture d'URL, `resolve_embed()`, `inline_kind()`, `embed_html()` ; endpoint `POST /board/api/embed/resolve`
- Service `app/services/og_fetcher.py` : parseur OG robuste (ordre d'attributs libre), favicon, repli sans réseau ; endpoint `POST /board/api/og/metadata`
- Endpoints `POST/DELETE /board/api/pages/{id}/cover` (JSON URL ou upload image) et `POST /board/api/pages/{id}/icon`
- Éditeur : résolution d'embed à la saisie (URL d'origine conservée + `embed_src` mis en cache), lightbox multi-images navigable, préviews vidéo/audio/PDF
- Pages publiques `/p/<slug>` : cover + icône, embed résolu, lightbox clavier
- Export Markdown/HTML/PDF : nouveau bloc `embed`/`bookmark` avec `embed_src` résolu
- **Fix chemins API** : l'éditeur appelait `/api/pages/...` alors que les routes sont `/board/api/pages/...` (cover, icon, versions, backlinks, import, move, OG) — corrigé
- **47 tests** `tests/test_v55.py` ; suite complète **444 verte** ; `ruff check` OK
- [ ] **Import CSV typé** — inférence automatique des types de propriétés (texte, nombre, date, select)
- [ ] **Import Confluence / Evernote / Asana / Trello** — via leurs formats d'export (HTML/JSON)
### v5.6.0 — Import de données (étendu) ✅ (2026-09-13)
> **Objectif** : faire de FlowDeck la cible d'import universelle pour les outils réellement utilisés
> (notes Markdown, bureautique, tableaux, signets, dev), en complétant l'import de base
> (Markdown/CSV/Notion, déjà dans v5.4.0). **COMPLETED**.
> **Sources retenues** : Obsidian, Notion, Logseq/Roam, Apple Notes/Bear/Ulysses, Google Keep,
> OneNote, Word, Google Docs, HTML, PDF, CSV/Excel/Sheets/JSON, Gitea/GitHub, Raindrop, Pocket,
> Readwise, Shaarli, `.ics`, OPML, Standard Notes.
> **Sources exclues** : Confluence, Evernote, Asana, Trello (non utilisées).
### v5.7.0 — Database Avancée (Pt. 2)
> **Objectif** : compléter la parité sur les propriétés, les vues sauvegardées et le Kanban pro.
#### Phase 0 — Socle d'import unifié (prérequis) ✅
> Toutes les sources partagent le même pipeline ; à livrer avant les imports.
- [x] **Service `app/services/importers/`** — interface commune (`detect()`, `parse()`, `to_pages()`) + résultat normalisé (`ImportResult` : pages, pièces jointes, warnings, stats)
- [x] **Pipeline commun** — upload → parse → normalisation blocs + métadonnées → création (pages + hiérarchie `parent_id`) → rapport d'import
- [x] **Pièces jointes** — extraction (images/fichiers), upload, réécriture des liens dans les blocs
- [x] **Frontmatter YAML → propriétés** (title, tags, dates, champs custom)
- [x] **Assistant UI d'import** — choix de la source, mapping colonnes→types, preview/dry-run, barre de progression
- [x] **Import asynchrone** (job en arrière-plan + polling) pour les gros volumes (vaults, zips)
- [x] **Idempotence / déduplication** + reprise sur erreur partielle
- [x] **Tests** — harnais d'import + fixtures par source
- [x] **Réutilise** : `_md_to_blocks` (`app/services/export.py`), `PROPERTY_TYPES` + `validate_property_rule` (`property_types.py`), `db_templates.materialize_properties`, endpoints upload/cover existants
- [ ] **Types propriété manquants** — `person`, `created_time`, `created_by`, `last_edited_time`, `last_edited_by`
- [ ] **Groupes de propriétés** — sections pliables dans le header de DB
- [ ] **Vues sauvegardées par utilisateur** — save view (per-user, pas workspace-wide)
- [ ] **Swimlanes Kanban** — sous-groupes horizontaux (2e dimension de groupement)
- [ ] **WIP limits** — par colonne, alerte visuelle au dépassement
- [ ] **Cartes configurables** — propriétés affichées par carte, couverture (image/icône/couleur), mode compact / détaillé
- [ ] **Calendar avec drag & drop** — reschedule direct sur la grille
- [ ] **Gallery avec couvertures** — image/icône comme vignette de carte
#### Phase 1 — Notes & Markdown (réutilise `_md_to_blocks`) — faible effort ✅
- [x] **Obsidian** (vault `.zip`/dossier) — frontmatter YAML, `[[wikilinks]]`, `![[embeds]]`, attachments, hiérarchie de dossiers *(prépare v5.11.0)*
- [x] **Notion** (améliorer l'existant) — hiérarchie complète, databases (CSV) → collections, images
- [x] **Logseq / Roam Research** — markdown outliné (puces imbriquées), `((block refs))`, pages journal
- [x] **Apple Notes / Bear / Ulysses** — import export HTML/Markdown
- [x] **Google Keep** — Takeout JSON + HTML
- [x] **OneNote** — export HTML/PDF, best-effort (fidélité limitée)
### v5.8.0 — Calendrier & Rappels
> **Objectif** : calendrier complet + notifications proactives.
#### Phase 2 — Données & tableaux (fort ROI databases) — effort faible/moyen ✅
- [x] **CSV/TSV typé** — inférence auto (texte/nombre/date/bool/select/multi_select) + UI mapping + options
- [x] **Excel `.xlsx`** (openpyxl) — multi-feuilles → collections
- [x] **Google Sheets** — export CSV/XLSX
- [x] **JSON générique** — mapping configurable (JSONPath → propriétés)
- [ ] **Vues Jour / Semaine / Mois** — calendrier complet (actuellement mois seulement)
- [ ] **Récurrence d'événements** — daily/weekly/monthly/custom (RRULE)
- [ ] **Support timezone** — par utilisateur + par événement
- [ ] **Rappels** — in-app + email (avant échéance : N minutes/heures/jours)
- [ ] **Centre de notifications** — cloche in-app (mentions, assignations, commentaires, rappels)
- [ ] **Template Meeting Notes** — Attendees, Agenda, Notes, Action Items
#### Phase 3 — Documents & bureautique — effort moyen ✅
- [x] **Word `.docx`** (mammoth/pandoc) — titres, listes, tableaux, images
- [x] **Google Docs** — Takeout `.docx`/HTML
- [x] **HTML** (fichiers/dossier, web clipper) — conversion HTML → blocs
- [x] **PDF** — extraction texte + images (fidélité limitée)
#### Phase 4 — Signets, dev & divers ✅
- [x] **Gitea / GitHub issues + labels + milestones** → collection (via API — quasi natif)
- [x] **Raindrop.io** (CSV/HTML) → bookmark cards (réutilise v5.5.0)
- [x] **Pocket** (CSV/HTML)
- [x] **Readwise** (highlights CSV/Markdown)
- [x] **Shaarli** (API/export JSON) → bookmark cards
- [x] **Calendrier `.ics`** (Google/Outlook/Apple)
- [x] **OPML** (flux/outlines)
- [x] **Standard Notes / autres**
#### Phase 5 — Durcissement & finition ✅
> **Objectif** : industrialiser l'import (ré-import, lots, URL, forge, relations, rapports).
- [x] **Import incrémental / re-sync** — modes `skip` / `update` (upsert des pages + lignes par titre) / `duplicate`, via `import_items`
- [x] **Import de dépôt forge** — arborescence de fichiers Gitea/GitHub → pages (hiérarchie de dossiers, fichiers texte, code en blocs)
- [x] **Import par URL / web clipper** — `POST /api/import/url` : fetch (garde SSRF) + OG metadata → page (carte bookmark + contenu)
- [x] **Lot multi-fichiers + file d'attente** — `POST /api/import/run-batch` + file d'attente UI (statut par fichier)
- [x] **Relations Notion** — `POST /api/import/relations/resolve` : colonnes texte référençant une autre collection → propriétés `relation` (ids de pages) ; auto-exécuté après un import Notion
- [x] **Reprise / erreurs partielles + rapport exportable** — import qui continue par page en cas d'erreur (`status: partial`, liste `errors`) ; rapport JSON téléchargeable (`GET /api/import/jobs/{id}/report` + bouton UI)
- [x] **Valeurs de propriétés par id** — correction : les lignes importées stockent les valeurs par id de propriété (rendu correct dans les vues DB)
#### Priorisation
| Ordre | Phase | Effort | Impact |
|---|---|---|---|
| 1 | Phase 0 — Socle unifié | M | 🔴 prérequis |
| 2 | Phase 1 — Notes/Markdown | S–M | 🔴 fort (Obsidian, Notion) |
| 3 | Phase 2 — Données/tableaux | S–M | 🔴 fort (databases) |
| 4 | Phase 4 — Signets/dev/divers | S | 🟠 moyen (Gitea natif, Raindrop) |
| 5 | Phase 3 — Documents | M | 🟠 moyen (docx, HTML) |
### v5.7.0 — Database Avancée (Pt. 2) ✅ (2026-09-13)
> **Objectif** : compléter la parité sur les propriétés, les vues sauvegardées et le Kanban pro. **COMPLETED**.
- [x] **Types propriété** — `person`, `created_time`, `created_by`, `last_edited_time`, `last_edited_by` câblés : auto-valeurs calculées serveur (`apply_auto_properties`) à la création/mise à jour, sélecteur `person` (membres du workspace via `GET /db/{id}/members/api`), rendu chips/avatars et édition dédiée dans l'UI
- [x] **Groupes de propriétés** — colonne `collection_properties.group_name`, header de table avec sections pliables (`db-group-row` + toggle) et `POST /db/{id}/property-groups/api`
- [x] **Vues sauvegardées par utilisateur** — `collection_views.created_by` ; `GET /db/{id}/views/api` filtre par propriétaire (les vues partagées `NULL` restent visibles), `save-as`/`duplicate`/`DELETE` + renommage
- [x] **Swimlanes Kanban** — `sub_group_by` (2e dimension de groupement) : une rangée de colonnes par lane
- [x] **WIP limits** — `wip_limits` par colonne, alerte visuelle au dépassement (`wip-exceeded`)
- [x] **Cartes configurables** — `card_properties`, `card_size` (compact/détaillé), couverture `cover_mode` (`none`/`icon`/`color`/`property`) via `cover_property`
- [x] **Calendar avec drag & drop** — grille mensuelle, `date_property`, navigation mois/Today, drop d'une carte sur un jour = reschedule
- [x] **Gallery avec couvertures** — cartes avec vignette (image de propriété, icône, couleur, ou cover de ligne), tailles small/medium/large
Détails livrés :
- Migration 10 : `collection_properties.group_name`, `collection_views.created_by`/`updated_at`, `collection_pages.cover_url`
- `app/services/property_types.py` : `user_ref()`, `apply_auto_properties()`, validation `person`
- `collections.py` : auto-props create/update/sub-item, PATCH partiel fusionné, membres, groupes, vues per-user (list/save/duplicate/delete), config de vue étendue (`sub_group_by`, `wip_limits`, `card_size`, `cover_mode`, `card_properties`, `date_property`, `property_groups`)
- `dashboard.py` : auto-props à la création de ligne, `views` exposées dans `table-data` et le contexte de page
- `_database_table_scripts.html` réécrit en composant multi-vues (table/board/calendar/gallery/list) + barre de vues, éditeurs de cellules par type, picker personne, popovers de configuration
- **12 tests** `tests/test_v57_db_advanced.py` ; suite complète **503 verte** ; `ruff check` OK
### v5.8.0 — Calendrier & Rappels ✅ (2026-09-13)
> **Objectif** : calendrier complet + notifications proactives. **COMPLETED**.
- [x] **Vues Jour / Semaine / Mois** — sélecteur de mode dans la vue calendar (persisté en config de vue), navigation ‹/› et Today adaptées, vue Jour en agenda horodaté, Semaine en 7 colonnes ; événements servis par `GET /db/{id}/calendar/api` avec expansion serveur des occurrences
- [x] **Récurrence d'événements** — moteur `app/services/recurrence.py` (daily/weekly/monthly, intervalle, count, until, byweekday lundi=0, timezone) ; règle stockée dans `property_values_json.__recurrence__` ; popover événement (double-clic) avec Repeat/Every/Ends ; validation serveur (400)
- [x] **Support timezone** — colonne `users.timezone` + picker dans Settings → Notifications (`GET/POST /api/notifications/timezone`, zones via `GET /db/timezones/api`) ; timezone par événement (`__timezone__`) > règle de récurrence > préférence utilisateur
- [x] **Rappels** — `app/services/reminders.py` : lead minutes/heures/jours avant chaque occurrence, scan toutes les 60 s (`reminder_scheduler`), dédup `reminder_log`, notifie les personnes assignées (repli admin), in-app + email (pref `reminders`) ; stockés dans `property_values_json.__reminder__`, édités dans le popover événement
- [x] **Centre de notifications** — la cloche (v4.9.0) couvre désormais rappels, assignations et commentaires ; nouvelles préférences `reminders` + `assignments` ; notification d'assignation émise par `notify_assignment()` sur `PUT /db/pages/{id}/api` (comparaison avant/après des propriétés `person`)
- [x] **Template Meeting Notes** — propriétés `Agenda` et `Notes` ajoutées au template seed (migration 12, préserve les personnalisations) ; le schéma Attendees/Date/Status/Action items était déjà là
Détails livrés :
- Migrations 11 (`reminder_log`, `users.timezone`) et 12 (template Meeting notes)
- `app/config.py` : `reminders_enabled`, `reminder_scan_interval_seconds`
- `_database_table_scripts.html` : modes jour/semaine/mois, chips d'occurrences (badge ↻ + heure), popover Time/Timezone/Repeat/Remind, badges ↻/🔔 dans les cellules date, drag & drop de reschedule préservé
- **20 tests** `tests/test_v58_calendar_reminders.py` ; suite complète **523 verte** ; `ruff check` OK
- **Version** — 5.11.7
### v5.9.0 — AI Writing Assist (éditeur) ✅ (2026-09-10)
> **Objectif** : l'IA Notion dans l'éditeur, au-dessus du moteur v4.10.0 (Agent IA). **COMPLETED**.
@@ -575,47 +664,392 @@ app/
## v5.10.0 — Éditeur : interactions de bloc ✅ (livré — voir section Completed)
## v5.11.0 — Wiki-links & mentions de page ⬜ (non commencé)
## v5.11.0 — Wiki-links & mentions de page ✅ (2026-09-14)
> **Objectif** : le graphe de connaissances Notion. Complète les backlinks de v5.4.0
> (section « Lié depuis ») par la création des liens depuis l'éditeur.
> (section « Lié depuis ») par la création des liens depuis l'éditeur. **COMPLETED**.
- [ ] **Wiki-links `[[`** — taper `[[` ouvre un picker de pages (recherche fuzzy, toutes collections), Enter insère un lien interne rendu comme chip de page (icône + titre mis à jour dynamiquement)
- [ ] **Mention de page `@`** — dans le menu @ existant (utilisateurs v4.9.0), ajouter l'onglet « Pages » : `@` suivi d'un nom de page crée un lien inline
- [ ] **Mention de date `@`** — `@today`, `@tomorrow`, `@2026-10-01` rendus comme chips de date
- [ ] **Renommage propagé** — renommer une page met à jour le libellé affiché de tous ses liens internes (résolution au rendu via `page_id`, pas de texte dur)
- [x] **Wiki-links `[[`** — taper `[[` ouvre le picker de pages (`GET /board/api/wiki/pages`, recherche substring + sous-séquence floue), Entrée insère un lien interne rendu comme chip atomique (icône + titre, cliquable) ; module `WM` dans `_page_editor_scripts.html` ; tokens `[[fdpage:ID]]` stockés dans le texte des blocs et relus intacts par `gtTok()` (autosave, drag, undo/redo préservés)
- [x] **Mention de page `@`** — le menu `@` (utilisateurs v4.9.0) gagne la section « Pages » : `@nom` cherche les pages et insère le chip inline
- [x] **Mention de date `@`** — section « Date » : `@today`, `@tomorrow`, `@hier` ou `@YYYY-MM-DD` insèrent `[[fddate:…]]`, rendus comme chips de date lisibles (« Fri 25 Dec 2026 »)
- [x] **Renommage propagé** — le token ne stocke que `page_id` ; `GET /board/api/wiki/titles` résout les libellés au rendu (editeur + page publique) → renommer une page met à jour tous ses liens ; page supprimée → « Deleted page »
- [x] Bonus — le scanner de backlinks v5.4.0 reconnaît les tokens wiki ; `_render_blocks_public` rend les chips en page publiée (HTML échappé) ; service `app/services/wiki_links.py` ; 15 tests dédiés (voir v5.12.0)
## v5.12.0 — Templates & verrouillage de page ⬜ (non commencé)
## v5.12.0 — Templates & verrouillage de page ✅ (2026-09-14)
> **Objectif** : démarrage rapide productif et protection des pages stabilisées.
> Les `page_templates` existent (v2.0.0/v4.2.0) mais uniquement côté collections.
> Les `page_templates` existent (v2.0.0/v4.2.0) mais uniquement côté collections. **COMPLETED**.
- [ ] **Template picker global** — sur « + New page » : galerie de templates (Empty, Meeting notes, Weekly report, To-do list…) + templates custom utilisateur
- [ ] **Bouton « Use template »** — duplication du contenu du template dans la nouvelle page
- [ ] **Page lock** — toggle 🔒 dans le menu « … » : page en lecture seule (édition désactivée pour tous sauf owner/admin), indicateur visuel en topbar
- [ ] **Full-width mode** — toggle pour passer la page en pleine largeur (comme Notion)
- [ ] **Small text / typo options** — option de page : taille de police réduite, serif/mono
- [x] **Template picker global** — « + New page » (sidebar, footer, Ctrl+K) ouvre la galerie : 5 templates built-in (`app/services/block_templates.py` — Empty, Meeting notes, Weekly report, To-do list, Project doc) + templates custom utilisateur (`page_global_templates`, migration 13, cloisonnés par `created_by`)
- [x] **Bouton « Use template »** — `POST /board/api/page-templates/{id}/use` (id 0 = built-in par clé) duplique le contenu des blocs dans une nouvelle page ; « Empty » retombe sur la création classique ; menu « … » → « 📑 Save as template » capture la page courante
- [x] **Page lock** — toggle 🔒 dans le menu « … » : bannière sticky read-only, blocs/titre non éditables ; serveur `POST /board/api/pages/{id}/lock` + garde `_ensure_page_editable` → 423 sur `PUT /api/pages/{id}` et `POST /api/pages/{id}/blocks` ; déverrouillage réservé au poseur du lock (`locked_by`) ou admin (403 sinon) ; indicateur visuel = bannière en tête de page
- [x] **Full-width mode** — toggle dans le menu « … », persisté par page (`pages.full_width`), `POST /board/api/pages/{id}/options`, classe CSS `.full-width`
- [x] **Small text / typo options** — toggle « Aa Small text » (`pages.font_small`), classe `.small-text` (taille réduite ; serif/mono : non demandé au-delà du compact — volontairement hors scope)
## v5.13.0 — Collaboration temps réel ✅ (livré — voir section Completed)
## v5.14.0 — Synced blocks ⬜ (non commencé)
## v5.14.0 — Synced blocks ✅ (2026-09-15)
> **Objectif** : avancer depuis v6.0.0 un bloc Notion très utilisé (même contenu dans
> plusieurs pages, édité une fois).
> plusieurs pages, édité une fois). **COMPLETED**.
- [ ] **Bloc `synced_block`** — table `synced_blocks` (source de vérité) + références par page ; slash command `/synced`
- [ ] **Rendu** — ring rouge + badge « Synced » sur le bloc ; édition à un endroit => mise à jour partout (via rooms WS v5.13.0 si actives, sinon au reload)
- [ ] **Unsync** — action « Unsync » qui convertit l'instance en copie indépendante
- [ ] **Copy & sync across pages** — copier un bloc dans une autre page avec option « Paste and sync »
- [x] **Bloc `synced_block`** — table `synced_blocks` (source de vérité) + références par page ; slash command `/synced`
- [x] **Rendu** — ring rouge + badge « Synced » sur le bloc ; édition à un endroit => mise à jour partout (via rooms WS v5.13.0 si actives, sinon au reload)
- [x] **Unsync** — action « Unsync » qui convertit l'instance en copie indépendante
- [x] **Copy & sync across pages** — copier un bloc dans une autre page avec option « Paste and sync »
### v5.15.0 — Webhooks v2 ✅ (2026-09-21)
> **Objectif** : webhooks sécurisés, fiables et complets pour les intégrations tierces. **COMPLETED**.
- [x] **Signature HMAC SHA-256** — header `X-FlowDeck-Signature` sur chaque payload, vérification côté receveur
- [x] **Retries avec backoff** — 4 tentatives max, délais 2s / 10s / 60s, logs détaillés
- [x] **20+ nouveaux événements** — total ~50 événements (pages, blocs, utilisateurs, collections, workspaces, etc.)
- [x] **API v2 endpoints** — `/api/v2/webhooks/test-signature` (test HMAC), `/api/v2/webhooks/retry` (relancer les échecs)
- [x] **Tests** — 21 dédiés (`tests/test_webhooks_v2.py`) ; **7 tests d'intégration câblés** (retry échec→succès, échec après MAX_ATTEMPTS, event inconnu ignoré, aucun abonné, fan-out wildcard, flow complet signé bout en bout, `retry_due_deliveries` superseded→delivered) — fixture `db` = SQLite isolée par test + `httpx.MockTransport` + `RETRY_DELAYS` neutralisé ; suite complète **749 verts, 0 skip**
---
## v6.0.0 — Pro (futur)
## v6.0.0 — PWA : Progressive Web App, offline ✅ (2026-09-18)
> **Objectif** : support hors ligne complet (manifest, service worker, IndexedDB,
> queue de mutations, sync serveur + résolution de conflits). **COMPLETED**.
- [ ] **PWA** — Progressive Web App, offline support
- [ ] **SSO/SAML** — enterprise authentication
- [ ] **Granular permissions** — page-level, property-level access control
- [ ] **Web Clipper** — extension navigateur
- [ ] **API publique complète** — REST API documentée (OpenAPI) *(base existante : `public_api.py`, à étendre + documenter)* — voir [`docs/API_GUIDE_V6.md`](docs/API_GUIDE_V6.md) : référence complète (conventions, CRUD par ressource, webhooks, sécurité, checklist)
- [ ] **Realtime editing (production)** — voir **v5.13.0** (curseurs + présence déjà avancés ici) ; reste en v6 : conflits avancés, édition large échelle
- [ ] **Synced blocks (production)** — voir **v5.14.0** (bloc de base) ; reste en v6 : syncing côté databases/vues
- [x] **Manifest & icônes PWA** — `static/manifest.json`, `static/icons/*`, `scripts/generate_pwa_icons.py`
- [x] **Service Worker** — `static/sw.js` (precache shell, network-first HTML/API, page offline, Background Sync)
- [x] **IndexedDB client** — `static/js/offline.js` (`window.FlowOffline`) : queue, delta, flush, marqueurs dirty
- [x] **Endpoints sync** — `app/routers/sync.py` + `app/services/sync_engine.py` (`/api/v2/sync/delta|batch|status`)
- [x] **Migrations** — table `offline_sync_queue` + colonnes `sync_version` (triggers AFTER UPDATE)
- [x] **Conflits** — edit-edit (last-write-wins), edit-delete (page orpheline), create-create (« copie offline »)
- [x] **UI** — banner offline + pending count, badge de synchronisation, toasts, icône ⟳ sur pages dirty
- [x] **Durcissement** — max 100 mutations/batch, timeout 30 s, rétention queue 30 j
- [x] **Tests** — `test_sync.py`, `test_sync_migrations.py`, `test_service_worker.py`, `test_pwa_offline.py`, E2E `e2e/pwa_offline.spec.js`
- [x] **Doc** — section `/help` « Offline mode (PWA) »
---
## v6.1.0 — Granular Permissions ✅ (2026-09-19)
> **Objectif** : page-level, collection-level & property-level ACL + groupes + audit. **COMPLETED**.
- [x] **Page permissions** — modes `inherit|restricted|private` (`pages.permission_type`, `collections.permission_type`, `collection_pages.permission_type`), grants explicites user/group (`page_permissions` role viewer/commenter/editor/owner), héritage page→collection→workspace, 404 masqué pour non-grantees, owner/admin bypass
- [x] **Collection permissions** — `collection_permissions` + permission_type, `/db/{id}` et `/db/{id}/api` masqués (404→302), création page et delete collection gatés (viewer 403), editor bypass
- [x] **Property-level visibility** — `property_permissions` (viewer|editor), `GET /db/{id}/properties/api` filtré par `get_visible_properties()`, `GET /api/v2/collections/{id}/properties/visible` (visible/hidden), grant = owner collection uniquement
- [x] **Groupes réutilisables** — tables `user_groups` + `group_members` (workspace-scoped, UNIQUE(name)), CRUD `/api/v2/groups` + `/groups/{id}/members`, grant par `group_id` (page/collection/property), retrait membre révoque l'accès
- [x] **API** — `app/routers/permissions.py` : pages (list/mine, grant, batch, revoke, permission-type), collections (list/grant/revoke/type + visible), properties (list/grant/revoke), groups (list/create/update/delete + members), users picker, audit `GET /api/v2/audit/permissions`
- [x] **Guards** — `board.py` (`GET/PUT /board/api/pages/{id}`) + `collections.py` (`delete collection`, `GET/PUT/DELETE page`, `properties`), 403/404 conformes, is_admin/owner bypass + `_session_user()` (no admin fallback)
- [x] **PermissionManager** — `app/services/permission_manager.py` étendu : `_explicit_grant_role()` (best rank user+groups), `get_page/collection_permission()`, `can_view/edit_page|collection`, `can_view/edit_property()`, `get_visible_properties()`, groups, `log_permission_change()`, cache 60s + `invalidate()`
- [x] **Audit** — table `permission_audit_log` (resource_type, action grant/revoke/type_change/group_*), index, log sur tous les mutateurs, `GET /api/v2/audit/permissions` (owner/admin only, limit 500)
- [x] **Migration 18** — `migrations.py` : création 6 tables + 3 colonnes `permission_type` + indexes (idempotent)
- [x] **Tests** — `tests/test_v60_granular_permissions.py` **21 tests** (inherit/restricted/private, grant viewer/editor, revoke, batch, type via API, collection restricted+grant, property visibility/hidden, group inherits + revoke, audit, auth 401, validation 400/404)
## v6.2.0 — Web Clipper : extension navigateur ✅ (2026-09-19)
> **Objectif** : capturer n'importe quelle page web en page FlowDeck (article, sélection, bookmark, screenshot) depuis une extension Manifest V3 + API directe. **COMPLETED**.
- [x] **Extension Manifest V3** — `extension/` + `static/extension/` (manifest, `background.js`, `content.js`, `popup.html/js`, `clipper.css`, icônes 16/32/48/128, `flowdeck-clipper.zip` servi à `/static/extension/`)
- [x] **4 types de capture** — article (HTML complet → `sanitize_html` + `html_to_blocks`), sélection, bookmark (carte OG v5.5.0), screenshot (base64) ; cap 10 MB / 200 blocs
- [x] **Serveur** — `POST /api/v2/web-clipper/clip`, `GET /status`, `POST /auth/verify` (register device → token `fd_…` montré une fois), `GET /devices`, `DELETE /devices/{id}` + page HTML `GET /extensions` ; auth triple (session OU Bearer `api_tokens` OU Bearer `extension_devices` OU legacy `user_tokens`), rate-limit 50/h/device
- [x] **Tables migration 19** — `extension_devices`, `extension_clips` + index `idx_ext_*`
- [x] **Settings UI** — onglet Extensions (devices, clips count, revoke)
- [x] **16 tests** `tests/test_web_clipper.py` ; `VERSION` 6.2.0 ; wiring `app/main.py:50,158`
### v6.2.1 — Web Clipper polish ✅ (2026-09-20)
- [x] **Bouton flottant rond transparent draggable** — toggle d'affichage persistant, `clipper:clipped` refresh auto sidebar
- [x] **Fix bloc bookmark** — `create_page_from_clip()` émet un bloc `bookmark` fidèle (OG + `embed_src` résolu), rendu/correct en éditeur + `/p/<slug>` + exports
- [x] Fix `duplicate inner` SyntaxError dans `_page_editor_scripts.html`
- [x] `flowdeck-clipper.zip` régénéré
## v6.3.0 — API publique complète v2 ✅ (2026-09-21)
> **Objectif** : REST API documentée OpenAPI, CRUD complet, un seul chemin de code (wrappers sur les services internes). Parité `docs/API_GUIDE_V6.md` §4 (~80 endpoints) + scopes hiérarchiques `read < write < admin`. **COMPLETED**.
> **Route** : `feat/v6-api-v2` → `develop` → `main` — livraison **en une fois** (tous domaines).
> **Doc** : [`docs/API_GUIDE_V6.md`](/docs/API_GUIDE_V6.md) · OpenAPI : `/docs` + `docs/openapi-v2.json` (402 chemins)
#### Phase 0 — Roadmap & doc catch-up ✅
- [x] Tagguer v6.2.0/v6.2.1 dans `CHANGELOG.md` + `ROADMAP.md` + `docs/V6_Web_Clipper.md` → `COMPLETED`
- [x] Détailler v6.3.0 phases 1-8 dans `ROADMAP.md` (plan gelé)
#### Phase 1 — Migrations socles (v20) ✅
- [x] `api_tokens` : colonnes `scopes TEXT DEFAULT 'read,write'`, `expires_at TIMESTAMP` (idempotent, rétro-compat)
- [x] `webhook_deliveries` : `id, webhook_id FK, status, http_code, error, duration_ms, attempt, created_at`
- [x] `api_audit_log` : `id, user_id, token_id, action, resource_type, resource_id, ip, created_at`
- [x] `idempotency_keys` : `key TEXT PRIMARY KEY, user_id, response_json, created_at`
#### Phase 2 — Helpers & auth v2 unifiée (scopes hiérarchiques) ✅
- [x] `app/config.py` : `public_api_insecure_ok: bool = False` — `fd-public-key` accepté seulement si `True` (dev local)
- [x] `app/services/api_v2_helpers.py` : `parse_pagination()` (+`X-Total-Count`), `to_iso8601()`, handler RFC 7807 `application/problem+json`, `require_scope()` (hiérarchie `admin ⊇ write ⊇ read`), `resolve_bearer_token()` / `get_bearer_user()` (hash sha256, `revoked` + `expires_at` + scopes, `last_used_at`, `extension_devices`)
- [x] Auth Bearer unifié partagé (clipper + legacy `user_tokens` supportés) ; handler d'erreurs unifié sur `StarletteHTTPException`
#### Phase 3 — Tokens CRUD v2 ✅
- [x] `POST /api/v2/tokens` (`name, scopes, expires_at`) → `fd_{urlsafe(32)}` hashé, montré une fois
- [x] `GET /api/v2/tokens` (prefix only), `DELETE /api/v2/tokens/{id}` (revoke), `POST /api/v2/tokens/{id}/rotate`
#### Phase 4 — Wrappers read (pagination/filtres/tri/fields) ✅
- [x] `app/routers/api_v2.py` (`prefix="/api/v2"`, tag `api-v2`) — `GET /collections`, `GET /collections/{id}`, `GET /pages/{id}`, `GET /collections/{id}/pages?filter[]=&sort=&fields=&query=`, `GET /search?query=&workspace_id=&type=` (FTS5), header `X-Total-Count`, `filter[]` AND, `sort=prop/-prop`
#### Phase 5 — Wrappers write critiques (`Idempotency-Key`) ✅
- [x] Collections : `POST/GET/PATCH/DELETE /collections/{id}` + `/linked`, `/task`, `/sources`
- [x] Pages : `POST/GET/PATCH/DELETE /pages/{id}` + `/restore`, `/move`, `/sub-items`, `/dependencies`
- [x] Properties : `GET/POST /collections/{id}/properties`, `PATCH/DELETE /properties/{id}`, `POST .../relation`, `evaluate-formula`, `compute-rollup`
- [x] Views/Dashboards/Comments/Notifications/Favorites/Tags/Recents/Sharing/History/Sprints/Templates/Export/Workspaces/Users/Admin — regroupés par ressource
#### Phase 6 — Webhooks v2 — CRUD simple ✅
- [x] `GET/POST/PATCH/DELETE /api/v2/webhooks`, `POST /api/v2/webhooks/{id}/test` (ping)
- [x] `GET /api/v2/webhooks/{id}/deliveries` (journal basique)
- [x] *(reporté v6.4, livré plus tôt)* **v5.15.0** : signature HMAC `X-FlowDeck-Signature`, retry 2s/10s/60s, +20 events (Webhooks v2)
#### Phase 7 — Forges & ressources restantes ✅
- [x] Sprints, dashboards, templates, export/import, workspaces/members, users, admin
- [x] Forges : `GET /projects`, `/projects/{owner}/{repo}/tree` (best-effort via `gitea_client`)
- [x] *(reporté, livré v6.4.0)* : migration de `sync.py` vers Bearer — auth `Authorization: Bearer` (scopes read/write) via `get_bearer_user()` + repli cookie session conservé pour compat PWA offline (`app/routers/sync.py`)
#### Phase 8 — OpenAPI, tests & docs ✅
- [x] `docs_url="/docs"` + `redoc_url="/redoc"` activés ; `docs/openapi-v2.json` généré (402 chemins)
- [x] `tests/test_public_api_v2.py` — **24 tests** (auth scopes, pagination, filtres, RFC 7807, idempotency, webhooks deliveries, CRUD multi-domaines)
- [x] Vérif `ruff check app tests` + `pytest -n auto` → **668 verte**
## v6.4.0 — Realtime editing (production) ✅ (2026-09-22)
> **Objectif** : passer le realtime v5.13.0 en « production » — résolution de
> conflits au-delà du last-write-wins + édition à grande échelle (broadcast non
> bloquant, plusieurs rooms/pages, observabilité). **COMPLETED**.
#### Conflits au-delà du LWW
- [x] **Merge à 3 voix (diff3-lite)** — nouveau service `app/services/realtime_merge.py` : `merge_text_3way()` (merge de caractères) + `merge_block_3way()` (merge champ-par-champ), fonctions pures et testées sans WebSocket ni base
- [x] **Régions disjointes conservées** — deux utilisateurs tapant à des endroits différents du *même* bloc voient leurs deux saisies survivre (au lieu d'écraser l'une par l'autre) ; ordre d'arrivée indifférent
- [x] **Chevauchement réel → LWW par champ + drapeau** — un conflit n'est plus « tout le bloc perdu » mais limité au champ concerné ; `conflict: true` renvoyé dans l'`ack` et le broadcast
- [x] **Protocole `base`** — le client embarque dans chaque `update` la version du bloc dont dérive sa saisie ; le serveur fait `merge_block_3way(base, current, incoming)` ; sans `base` → LWW historique (rétro-compat ancien client)
- [x] **Adoption côté client** — l'`ack` renvoie le bloc fusionné ; le client met à jour sa `base`, adopte le résultat (hors bloc en cours d'édition) et affiche un toast en cas de conflit
- [x] **Broadcast du résultat fusionné** — le serveur diffuse toujours le bloc final fusionné (jamais la proposition brute) pour convergence garantie de tous les clients
#### Échelle & robustesse
- [x] **Broadcast non bloquant** — chaque connexion a une file sortante (`asyncio.Queue`) + une tâche `_writer` dédiée ; `_broadcast()` fait `put_nowait` et n'attend plus le socket → un client lent ne fige plus la room
- [x] **Coalescence des curseurs** — le writer réduit les messages `sel` empilés à la position la plus récente (seule la dernière compte), tout en préservant l'ordre des messages importants
- [x] **Déconnexion des clients trop lents** — file pleine (`MAX_OUT_QUEUE=512`) → fermeture 4413 + compteur `slow_disconnects` (évite qu'une room entière stagne sur un pair mortel)
- [x] **Anti-flood** — budget d'opérations par connexion (`OP_WINDOW_MAX=400` / 10 s) ; au-delà, réponse `ack stale` sans application
- [x] **Fix fuite de rooms** — une connexion 4404 n'enregistre plus de `Room` orpheline en mémoire ; `room_state()` sur page inexistante retourne un dict vide au lieu de planter sur `None`
- [x] **Observabilité** — `GET /api/realtime/stats` (authentifié) : rooms, connexions, ops, merges, conflits, déconnexions lentes + détail par page
Détails livrés :
- `app/services/realtime_merge.py` — merge 3-voix (nouveau, ~170 lignes)
- `app/services/realtime_server.py` — `RTConn` (file + writer + budget), `_apply()` avec merge, `_evict_slow()`, `stats()`, fix fuites
- `app/routers/realtime.py` — endpoint `GET /api/realtime/stats`
- `app/templates/_page_editor_realtime.html` — envoi de `base`, adoption du bloc fusionné, toast de conflit
- **26 tests** `tests/test_realtime_v64.py` (merge purs, protocole WS, convergence 2 clients, rétro-compat LWW, fuite 4404, coalescence, stats, anti-flood) ; **14 tests** `tests/test_realtime.py` préservés
- `ruff check app tests` OK · `eslint static/js` 0 problème
- [x] **Version** — 6.4.0
---
## v6.7.0 — SSO / SAML + OIDC entreprise (Enterprise Auth) ✅ (2026-09-24)
> **Objectif** : dernière feature de la roadmap v6.0.0 — authentification fédérée
> via un IdP d'entreprise. **COMPLETED**. Design : [`docs/V6_SSO_SAML_Enterprise_Auth.md`](docs/V6_SSO_SAML_Enterprise_Auth.md).
- [x] **SAML 2.0 SP** — `app/auth/providers/saml_provider.py` (python3-saml 1.16.0) : login SP-initié
(`/auth/saml/login`), ACS signé (signature/audience/destination/InResponseTo validés),
SP metadata public (`/auth/saml/metadata`), SLO (`/auth/saml/logout`) ; `allowSingleLabelDomains`
pour les hôts homelab/LAN
- [x] **OIDC + PKCE** — `app/auth/providers/oidc_provider.py` (authlib 1.8.0) : `state` +
`code_verifier` en DB, ID token vérifié via JWKS (aud/iss/nonce/exp), discovery + JWKS cachés 1 h
- [x] **Provisioning** — `app/services/sso_provisioning.py` : auto-provision au 1er login
(`auth_method=saml|oidc`, lien par email), rôle par défaut + `default_workspace_id`,
mapping groupes IdP → rôles workspace (re-synchronisé à chaque login), mode **SSO only**
(login local refusé, admins exceptés — design §7.1), secrets Fernet, anti-replay `sso_requests`,
historique `sso_login_history`
- [x] **Routeur** — `app/routers/sso.py` : parcours navigateur + API admin
`GET/POST/DELETE /api/v2/sso/config` (admin + CSRF), `GET /api/v2/sso/workspaces`,
`POST /api/v2/sso/sync` ; CSRF exclut `/auth/saml/*` + `/auth/oidc/*`
- [x] **Migration 23** — tables `sso_config`, `sso_login_history`, `sso_requests`
- [x] **UI admin** — onglet « SSO / Enterprise » dans Settings (config SAML/OIDC, provisioning,
groupes → rôles, URL métadonnées SP copiable, désactivation) ; bouton SSO sur la page de login
- [x] **`PermissionManager`** (design §7.2) — `is_sso_only_workspace()`, `get_sso_roles()`,
`sync_sso_permissions()`
- [x] **Help** — section « SSO (Enterprise) » + badge `.help-badge.sso`
- [x] **Docs** — CHANGELOG, WORKLOAD, README, ARCHITECTURE, `docs/API_GUIDE_V6.md` §2.5,
OpenAPI régénéré : `docs/openapi-v2.json` → **439 chemins**, `info.version = 6.7.0`
- [x] **Tests** — `tests/test_v67_sso.py` : **38 tests** (config admin + CSRF + secret préservé,
métadonnées SP, flots SAML/OIDC complets vs IdP mockés, négatifs aud/iss/nonce/replay/tamper,
group mapping, `sso_only`, SLO, historique, rate limit, extension `PermissionManager`)
- [x] **Version** — 6.7.0 · `ruff check app tests` OK · **suite 802 verts, 0 skip**
---
## v6.6.0 — Agent phase 5 : API publique agent & skill marketplace ✅ (2026-09-24)
> **Objectif** : dernière étape du plan d'igration en 5 phases de l'agent — « Plateforme » :
> exposer l'agent en API publique pour les intégrations tierces et rendre les skills
> partageables (marketplace). **COMPLETED**. Voir [`docs/API_GUIDE_V6.md` §2.4](docs/API_GUIDE_V6.md).
- [x] **API publique agent** — `app/routers/api_v2_agent.py` (15 routes `/api/v2/agents/*`,
Bearer + scopes `read`/`write`, rate limit par token, idempotence `Idempotency-Key`, `api_audit_log`) :
- agents CRUD, conversations + messages, **run synchrone JSON** (le flux SSE reste réservé à l'UI),
journal `agent_actions` + `POST /agents/actions/{id}/undo`, `POST /agents/{id}/trigger`
- propriété des conversations vérifiée par `user_id` : un token tiers reçoit `404` (pas de fuite)
- réponse de run : `{status, final, error, reasoning[], actions[], events[], duration_ms}`
- [x] **Marketplace skills** — `app/services/skill_gallery.py` (source unique interne + v2) :
- **export portable** `{format: "flowdeck-skill", version: 1, skill{name, description, prompt_template, allowed_tools}}`
(aucun id/workspace/auteur local ne fuit) + **import** avec contrôle de collision (`409`, `overwrite: true` pour écraser)
- **galerie de 6 presets** installables : rapport hebdo, CR de réunion, base CRM, OKR, analyse repo Gitea, résumé de document
(`GET /api/v2/skills/gallery`, `POST /api/v2/skills/gallery/{slug}/install`)
- routes jumelles côté session : `GET /api/agent/skills/gallery`, `POST .../gallery/{slug}/install`,
`POST /api/agent/skills/import`, `GET /api/agent/skills/{id}/export`, `DELETE /api/agent/skills/{id}`
(comble le trou CRUD : pas de suppression de skill avant)
- **UI branchée** : section « Galerie » dans la palette `/` du panneau agent → installation + épinglage automatique du chip
- [x] **Webhooks de cycle de vie agent** — `agent.run.started` et `agent.run.failed` émis pour la 1re fois
(seul `agent.run.finished` l'était) ; les 3 sont dans le catalogue `webhook_outbound` → abonnement `agent.*`
- [x] **Docs** — `docs/API_GUIDE_V6.md` (§2.4 + correction du nom d'événement `agent.run.completed` → `agent.run.finished`),
OpenAPI régénéré : `docs/openapi-v2.json` → **427 chemins**, `info.version = 6.6.0`
- [x] **Tests** — `tests/test_v66_agent_api.py` : **15 tests** (auth/scopes, CRUD agents, run synchrone,
ownership 404, trigger, export/import/validation, galerie + outils valides, routes internes, cycle de vie webhooks)
- [x] **Version** — 6.6.0 · `ruff check app tests` OK · `eslint static/js` 0 problème · **suite 764 verts, 0 skip**
---
## v6.5.0 — Synced blocks production (databases & vues) ✅ (2026-09-24)
> **Objectif** : passer les synced blocks en « production » (résolution +
> propagation fiables partout) et les faire vivre dans les databases/vues —
> le « reste en v6 » du point **v5.14.0**. **COMPLETED**.
#### Contenu des lignes de database (le point « databases »)
- [x] **Migration 22** — `pages.collection_row_id INTEGER REFERENCES collection_pages(id) ON DELETE CASCADE` + index partiel ; chaque ligne de database gagne une **page contenu** (page `blocks` standard) portant son éditeur complet
- [x] **Service `app/services/row_pages.py`** — `ensure_row_page()` : création lazy/réparation (page trashée restaurée, `parent_section='DbRow'`), workspace hérité de la page hôte (DB full-page → page parente inline → `collections.gitea_owner/repo`), `collection_id` hérité pour que les ACL de collection s'appliquent au contenu
- [x] **Endpoint `GET /db/pages/{row_id}/open/api`** — renvoie `{page_id}` (garde `_require_view` de la collection) ; utilisé par le peek de **toute** vue (table/board/gallery/list/calendar)
- [x] **Fix bug d'ouverture de ligne** — le peek appelait `/pages/{rowId}` qui lit la table `pages` → avec des ids croisés il ouvrait une **page au hasard** (ligne id 5 « Ask AI » → page id 5 « Données ») ; désormais résolution via l'endpoint puis iframe `/pages/{pageId}`
- [x] **Titres synchronisés bidirectionnellement** — renommer la ligne (`PUT /db/pages/{id}/api`) → page contenu ; renommer dans l'éditeur (`PUT /board/api/pages/{id}`, `POST .../blocks`, `PUT /api/pages/{id}/rename`) → ligne
- [x] **Cascade** — supprimer une ligne supprime sa page contenu (FK `ON DELETE CASCADE`, refs `page_synced_blocks` en cascade)
- [x] **Exclusions des listings** — `AND collection_row_id IS NULL` sur : arbre sidebar (`_build_page_tree`), racines workspace (`_load_workspace_pages`), liste des pages du board, projets builtins, arbre Library (`_build_tree_children` + API tree), `page_count` des workspaces ; `parent_section='DbRow'` les tient hors des sections Private/Trash
#### Résolution serveur + propagation (le point « production »)
- [x] **Résolution à chaque lecture** — `resolve_content_json()` (`app/services/synced_blocks.py`) appliqué sur : les 2 routes de rendu `/pages/{id}` (dashboard + board, dual-route), `GET /api/pages/{id}/content`, `GET /api/local-workspace/page-content/{id}`, page publique `/p/{slug}`, `GET /api/v2/pages/{id}`, et les rooms realtime (`load_room`) → le cache `_synced_content` périmé n'est plus jamais servi
- [x] **Propagation écrite réelle** — `PUT /api/synced-blocks/{id}` réécrit le contenu stocké de **chaque** page référente (`sync_synced_blocks_in_page`, jusqu'ici jamais appelé = code mort depuis v5.14.0) **avant** le broadcast WS ; récursif dans les `children` (colonnes, toggles)
- [x] **Suppression de source** — ids des pages référentes collectés **avant** la cascade FK, `mark_synced_block_deleted()` marque `_synced_deleted` dans le stocké, broadcast dédié `manager._broadcast_synced_to()`
- [x] **État « deleted »** — `resolve_synced_block()` marque `_synced_deleted` (source absente) ; rendu dédié dans l'éditeur (« Deleted synced block » au lieu d'un cache fantôme / « Loading… » éternel)
- [x] **Rendu public des synced blocks** — branche `synced` ajoutée à `_render_blocks_public` (avant : JSON brut en `<p>`) + état deleted
- [x] **API v2** — `content_page_id` sur `GET /api/v2/pages/{id}` pour une ligne (sans création lazy en lecture)
#### Tests
- [x] **`tests/test_v65_synced_db.py` — 17 tests** (migration, création/idempotence/cascade/restauration de la page contenu, sync des titres, exclusions de listings, résolution lecture éditeur/content/public, propagation écrite + récursivité + deleted, synced block dans une ligne de DB de bout en bout, `content_page_id` v2)
- [x] Suite complète **742 verts** (`pytest -n auto`) · `ruff check app tests` OK · `eslint static/js` 0 problème
- [x] **Version** — 6.5.0 (VERSION + `app/main.py`)
---
## v6.0.0 — Pro ✅ (2026-09-24 — cycle COMPLETED)
- [x] **PWA** — Progressive Web App, offline support ✅ (livré) — [📄 Conception détaillée](/docs/V6_PWA_Progressive_Web_App.md)
- [x] **Granular permissions** — page-level, property-level access control ✅ (livré v6.1.0) — [📄 Conception détaillée](/docs/V6_Granular_Permissions.md)
- [x] **Web Clipper** — extension navigateur ✅ (livré v6.2.0/6.2.1) — [📄 Conception détaillée](/docs/V6_Web_Clipper.md)
- [x] **API publique complète** — REST API documentée (OpenAPI) ✅ (livré v6.3.0) — [📄 API Guide v2](/docs/API_GUIDE_V6.md) · [📄 OpenAPI](/docs/openapi-v2.json)
- [x] **SSO/SAML** — enterprise authentication ✅ (livré v6.7.0) — [📄 Conception détaillée](/docs/V6_SSO_SAML_Enterprise_Auth.md)
- [x] **Realtime editing (production)** ✅ livré **v6.4.0** (merge 3-voix au-delà du LWW, broadcast non bloquant) ; voir **v5.13.0** pour le socle (curseurs + présence)
- [x] **Synced blocks (production)** ✅ livré **v6.5.0** (page contenu par ligne de database, résolution serveur à chaque lecture, propagation écrite réelle, état deleted, rendu public) ; socle : **v5.14.0** (bloc de base)
---
## v7.0.0 — Cycle « Publier, Retrouver, Automatiser » (planifié)
> **Contexte (2026-09-28)** : roadmap v6.0.0 COMPLETED (802 tests). Le core Notion est à parité
> (blocs, 11 vues, 21 props, realtime, synced, PWA, API v2, agent, SSO).
> Le cycle v7 comble les couches où Notion a poussé en 2025-2026 : **Sites, Forms,
> Search sémantique, Automations/Workers, Calendar sync, SCIM, MCP**.
> Référence : `notion.com/product/features` + `notion.com/releases` (Workers, Meeting Notes → agents, Developer bar, MCP).
### v6.8.0 — Sites & Forms publics ✅ (2026-09-28)
> **Objectif** : publier (site multi-pages) et collecter (formulaires anonymes).
> Parité Notion Sites + Forms. **Doc** : [`docs/V68_Sites_Forms.md`](docs/V68_Sites_Forms.md)
- [x] **Sites multi-pages** — table `sites` (`slug UNIQUE`, `root_page_id`, `title`, `theme light/dark`, `custom_domain`, `password_hash`, `expires_at`, `noindex`, `analytics_id`), table `site_pages` (arbre public ordonné) ; nav latérale auto ; réutilise `_render_blocks_public` (synced + wiki résolus)
- [x] **SEO & partage** — OG/Twitter cards par site/page, sitemap `/s/<slug>/sitemap.xml`, meta robots, preview sociale
- [x] **Gating public** — mot de passe (hash salé `password_utils`, cookie signé 24h), expiry (410), `noindex`, analytics vues (`site_views` : jour + compteur, pas d'IP brute)
- [x] **Routes** — `GET /s/<slug>` (home), `GET /s/<slug>/<page-slug>` (résolution slug + id), `GET|POST /s/<slug>/auth`, `GET/PATCH/DELETE /api/v2/sites*` (+ pages + stats), domaine custom via `Host` header
- [x] **Forms publics** — `form_config_json` par collection (`public_token f_*`, `fields`, `required`, `success_message`, `notify_user_ids`) ; `GET /f/<token>` (no-auth) + `POST /f/<token>` (anonyme, rate-limit 20/h/IP, honeypot, validation `validate_property_rule`) ; chaque soumission = `collection_pages` + log `form_responses` (ip_hash jour)
- [x] **Embed & notifs** — `?embed=1` sans chrome, notif in-app aux `notify_user_ids`, trigger `form.submitted` (branché v7.0)
- [x] **Migrations 24** — `sites`, `site_pages`, `site_views`, `form_responses`, colonne `collections.form_config_json`
- [x] **Tests** — `tests/test_v68_sites_forms.py` (**20 tests** : migration, CRUD, slug/conflit, auth, isolation, pages add/remove + root protégée, rendu home/subpage/blocs, 404, vues, password gate, expiry 410, sitemap, noindex, form config/submit/required/404/honeypot/rate-limit/embed)
- [x] **Version** — 6.8.0 (`VERSION` + `app/main.py`) · `ruff check` OK · CSRF exempt `/s/` + `/f/`
### v6.9.0 — Recherche sémantique + Ask AI ✅ (2026-09-28)
> **Objectif** : retrouver (hybride FTS + vectoriel) et demander (RAG avec citations).
> Parité Notion Enterprise Search + AI Q&A. **Doc** : [`docs/V69_Search_Ask_AI.md`](docs/V69_Search_Ask_AI.md)
- [x] **Embeddings** — tables `semantic_embeddings` (`resource_type`, `resource_id`, `chunk_id`, `chunk_text`, `embedding BLOB`, `model hash-256`) + `search_index_state` ; encodeur hashed-TF déterminstique (md5 % 256, L2, zéro dépendance, `embed_texts()` pluggable) ; job incrémental (`updated_at > indexed_at`, batch 50, scheduler 5 min dans le lifespan) + `purge_orphans()`
- [x] **Recherche hybride** — `GET /api/v2/search/hybrid` (lexical FTS5/LIKE via `search.py` + cosine, fusion RRF k=60, `X-Total-Count`, pagination) ; filtres `workspace_id`, scope membership + `PermissionManager` (pages restreintes masquées), `search_excluded` respecté partout
- [x] **Ask AI** — `POST /api/v2/search/ask` (`{question, workspace_id}` → `{answer_markdown, citations}`) : top-8 chunks autorisés (ACL avant prompt), `LLMClient.complete()` si provider configuré sinon extractif offline avec `[[fdpage:ID]]`, cache 10 min, rate-limit 30/min, audit
- [x] **Observabilité** — `GET /api/v2/search/index-status` (ressources indexées, vecteurs, modèle)
- [x] **Migrations 25** — `semantic_embeddings`, `semantic_index_state`, colonne `pages.search_excluded`
- [x] **Tests** — `tests/test_v69_search_ask.py` (**24 tests** : migration, chunk/overlap, déterminisme/norme, cosinus, index idempotent, exclusion, purge, rappel vectoriel partiel, hybride keyword/auth/400/pagination/isolation ACL/exclusion/collections, ask citations/auth/400/cache/ACL/rate-limit, index-status)
- [x] **Version** — 6.9.0 (`VERSION` + `app/main.py`) · `ruff check` OK
- [ ] **UI** — palette `Ctrl+K` onglets `Pages / Fichiers / ✨ Réponses IA` (reporté : backend livré, frontend en follow-up)
### v7.0.0 — Automations v2 + Workers ✅ (2026-09-28)
> **Objectif** : automatiser au-delà du if-this-then-that + custom code sandboxé.
> Parité Notion Automations + Workers (07/2026 : credits dashboard, partage équipe).
> **Doc** : [`docs/V70_Automations_Workers.md`](docs/V70_Automations_Workers.md)
- [x] **Automations multi-étapes** — table `automation_steps` (`automation_id`, `kind`, `position`, `config_json`) ; triggers `any` (défaut) / `all` (fenêtre 5 min, `PUT .../mode`) ; `form.submitted` branché de bout en bout ; conditions AND (réutilise `match_condition_props`) ; actions `slack` (incoming webhook, secret chiffré Fernet), `email` (via `mailer`, `user:` résolu), `forge_issue` (Gitea `GiteaClient` / GitHub API, token `user_oauth_tokens`), `agent_trigger` (conversation + run `AgentEngine`), `delay` (sleep plafonné 300s)
- [x] **Bouton DB natif** — type `button` dans `PROPERTY_TYPES`, colonne `button_automation_id`, `POST /api/automations/press-button` (CSRF-exempt, contexte ligne, erreurs 400 explicites)
- [x] **Workers lite** — `app/services/workers.py` : lint AST (imports `os/sys/subprocess/socket`, `open/exec/eval`, dunders), builtins restreints, thread + timeout 30s, stdout capturé, budget journalier `daily_budget_s`, fork des partagés ; `app/routers/workers.py` : CRUD `/api/v2/workers*` (session ou Bearer `write`, code masqué aux non-owners), run/runs/fork/usage ; crons dans la boucle 60s existante
- [x] **Compat legacy** — automations v5.1.0 sans steps inchangées ; matcher legacy ignore les automatisations à steps (pas de double run) ; triggers inconnus du catalogue webhook acceptés côté steps
- [x] **Migrations 26** — `automation_steps`, `workers`, `worker_runs`, `automations.trigger_mode`, `collection_properties.button_automation_id`
- [x] **Tests** — `tests/test_v70_automations_workers.py` (**31 tests** : migration, steps CRUD/validation/auth, mode any/all, `form.submitted`, chaînes + interpolation, condition, delay, slack + secret chiffré, email no-SMTP, forge mock + sans-token, agent mock + 404, button press/validation, legacy single-run, workers CRUD/auth/rejet code/run ok/error/timeout/budget/fork/privacy/usage/cron/masquage code)
- [x] **Version** — 7.0.0 (`VERSION` + `app/main.py`) · `ruff check` OK
- [ ] **Éditeur visuel** — canvas Settings → Automations (reporté : API steps livrée, UI en follow-up)
### v7.1.0 — Calendar sync + Meeting Notes ✅ (2026-09-28)
> **Objectif** : calendrier bidirectionnel + transcription → agents (cf. Notion 07/2026 : Meeting Notes trigger Custom Agents).
> **Doc** : [`docs/V71_Calendar_Meetings.md`](docs/V71_Calendar_Meetings.md)
- [x] **Sync externe** — table `calendar_links` (`user_id`, `provider google/caldav`, `tokens_enc` Fernet, `calendar_id`, `collection_id`, `date_property`, `sync_token`, `last_sync`) ; pull (event → ligne, date + `external_event_id`) + push (ligne → event) ; boucle 15 min dans le lifespan ; conflits (édité des 2 côtés → LWW + notif `calendar.conflict`, résolution par édition manuelle) ; passe pull marquée `touched` (jamais repoussée)
- [x] **Providers sans dépendance** — Google REST (Bearer, 401 → « relink » explicite), CalDAV brut REPORT/PUT + parseur multistatus minimal (UID/SUMMARY/DTSTART/DESCRIPTION) ; I/O module-level = monkeypatchables
- [x] **Meeting Notes v2** — upload audio (`.mp3/.wav/.m4a/.ogg/.flac/.aac`, 100 MB, `meetings_dir()`) → transcription `STT_COMMAND` ou transcript manuel (client-side STT) → résumé `ai_writing.summarize` (offline-capable) → trigger `meeting.summarized` (agents v7.0 : update tracker, post Slack, file tickets)
- [x] **Free/busy** — `GET /db/{id}/calendar/freebusy?from=&to=` (jours busy/free weekdays, récurrences expandues serveur, weekends exclus, 1..370 j)
- [x] **Migrations 27** — `calendar_links`, `meeting_transcripts`, colonne `collection_pages.external_event_id` + index
- [x] **Tests** — `tests/test_v71_calendar_meetings.py` (**15 tests** : migration, links CRUD/chiffrement/validation/auth/isolation, pull/push/idempotence/conflit LWW + notif, 502 token expiré, parseur CalDAV, freebusy + validations, upload + flow manuel + `meeting.summarized` → automation, validations audio, 404)
- [x] **Version** — 7.1.0 (`VERSION` + `app/main.py`) · `ruff check` OK
### v7.2.0 — Enterprise Admin : SCIM + 2FA + Audit UI ✅ (2026-09-29)
> **Objectif** : provisioning auto, durcissement auth, audit exploitable.
> Parité Notion SCIM + audit + domain verification. **Doc** : [`docs/V72_Enterprise_SCIM_2FA.md`](docs/V72_Enterprise_SCIM_2FA.md)
- [x] **SCIM 2.0** — `GET/POST/PUT/PATCH/DELETE /scim/v2/Users` (Bearer `scim_tokens`, mapping `userName→login`, `active→is_active`) ; auto-suspend (`is_active=0` → sessions révoquées, 401) ; tokens SHA-256 (affichés une fois, révocables) ; 404 `application/scim+json` + exemption CSRF (clients IdP sans cookie)
- [x] **2FA** — TOTP (`users.totp_secret_enc` chiffré Fernet, backup codes SHA-256 à usage unique, défi `pending` signé 5 min ; `local-login` → `2fa_required` sans session puis `local-verify`) + passkeys WebAuthn (`webauthn_credentials`, attestation vérifiée, connexion sans mot de passe avec anti-rejeu `sign_count`) ; `sso_only` + 2FA combinables
- [x] **Domain claim** — `domain_claims` (`domain` normalisé, `txt_token`, `auto_join_role`, `enforce_sso`) ; vérification `GET https://<domain>/.well-known/flowdeck-verify.txt` ; SSO forcé par domaine dans `local_login` (admins exemptés) ; jeton jamais renvoyé par le listing
- [x] **Audit UI** — `GET /api/v2/audit/logs` fusionne `api_audit_log` + `permission_audit_log` + `sso_login_history` (schéma commun), filtres `source`/`actor`/`action`, pagination, export `?format=csv` (admin ou Bearer `read:admin`) — *UI Settings → Audit : voir follow-up « Polish » v7.3*
- [x] **Gouvernance agents** — `agent_policies` (scope outils/workspace, `max_steps`, `require_approval` avant write) consultée par `AgentEngine` **avant** les ACL, file `agent_approvals` + décision `/api/v2/agent-approvals/{id}/decide`, événement `agent.run.approval_requested`
- [x] **Migrations 28** — `scim_tokens`, `domain_claims`, `webauthn_credentials`, `agent_policies`, `agent_approvals`, colonnes `users.totp_secret_enc`/`totp_backup_hashes` (`is_active` existait)
- [x] **Tests** — `tests/test_v72_enterprise.py` (**52 tests** : migration 28, SCIM tokens/CRUD/suspend/doublon/404, 2FA setup-activate-verify-backup-chiffrement-désactivation + challenges, domain claims, WebAuthn, audit multi-source + filtres + CSV + pagination, politiques + gate d'approbation)
- [x] **Version** — 7.2.0 (`VERSION` + `app/main.py`) · `ruff check app tests` OK · `test_agent.py`+`test_app.py` 261 verts
### v7.3.0 — Wiki / Teamspaces + Polish ✅ (2026-09-29)
> **Objectif** : connaissance vérifiée + finition collaborative. **Doc** : [`docs/V73_Wiki_Teamspaces_Polish.md`](docs/V73_Wiki_Teamspaces_Polish.md)
- [x] **Teamspaces** — `teamspaces` (`workspace_id`, `name`, `private`) + `teamspace_members` (rôles owner/editor/commenter/viewer) ; `pages.teamspace_id` + `collections.teamspace_id` ; `private` → 404 (comme collections restricted) ; un teamspace public reste cantonné à son workspace
- [x] **Verified pages** — `page_verifications` (badge ✅, `verified_by`, `note`, `expires_at` 90 j) ; index `/api/v2/wiki/verified` (exclut expirés + teamspaces privés) ; sweep admin `/api/v2/wiki/verify-expiry-sweep` → notif `page.verification_expiring` à J-7 ; Wiki Home `/api/v2/wiki/home` (teamspaces + verified + recents)
- [x] **Blocs manquants** — `mermaid` (SVG inline via `mmdc` si présent, sinon `<pre class="mermaid">` rendu client), `equation_inline` (KaTeX, source sanitizée : `<`/`>`/`\` retirés), `progress` (barre % agrégée sur `property_values_json`) ; rendus par `export.blocks_to_html` (donc export HTML/PDF) + `POST /api/v2/wiki/blocks/preview`
- [x] **Collab polish** — `comment_reactions` (agrégation emoji + toggle), `page_follows` (toggle + followers), `guest_shares` → `GET /g/{token>` sans compte (vue enregistrée, 404 HTML si expiré/révoqué), `page_views` (compteurs journaliers, séries sans trou)
- [x] **Migrations 29** — `teamspaces`, `teamspace_members`, `page_verifications`, `comment_reactions`, `page_follows`, `guest_shares`, `page_views` + colonnes `teamspace_id`
- [x] **Tests** — `tests/test_v73_wiki_polish.py` (**58 tests**)
- [x] **Version** — 7.3.0 (`VERSION` + `app/main.py`) · `ruff check app tests` OK · v7.2 + v7.3 = **110 verts**
- [x] **Sidebar par teamspace** — section `Teamspaces` dans `base.html` (état Alpine + `loadTeamspaces`/`openTeamspace`), `GET /api/v2/wiki/teamspaces` sans `workspace_id` (liste cross-workspace + `workspace_name`), page HTML `GET /wiki/teamspaces/{id}`, entrée `DEFAULT_CONFIG` sidebar
- [x] **Notif `page.updated` aux followers** — `notify_followers_of_page_update` (fenêtre 10 min), hooks `fire_event` + payloads `actor_id` (update_page/save_page_blocks), auto-follow à la création d'un commentaire
- [x] **Charts avancés** — type `number` (KPI count/sum/avg/min/max), dashboards multi-DB `GET /db/{id}/dashboards/{did}` (widgets à `collection_id`, 40 widgets max, 200 lignes/cap), 0 conservé dans les données (bug `val if val else 1`), sélection du view config selon `view_type`
- [x] **Link previews riches** — unfurl `gitea:owner/repo` / `github:owner/repo` dans `POST /board/api/og/metadata` (GiteaClient / GitHubAdapter, fallback API publique), champs bookmark persistés (`url/title/description/image/site_name` dans l'autosave)
- [x] **UI Settings → Audit** — `settings.html` onglet `admin-audit` branché sur `/api/v2/audit/logs` (filtres source/actor/action + pagination + export CSV)
- [x] **SSO 21 casses** — install `python3-saml==1.16.0` + `authlib==1.8.0` (xmlsec/isodate/joserfc) → `tests/test_v67_sso.py` **38/38**
- [x] **Tests follow-ups** — `tests/test_v73_wiki_polish.py` (58 → **72 tests**) : sidebar cross-workspace, page teamspace (owner 200 / outsider 404 API), auto-follow comment, notif throttlée, unfurl gitea/github + endpoint, KPI + dashboards multi-DB
- [x] **Suite complète** — `python -m pytest -n auto` → **1016 passed** (était 981 passed / 21 failed en SSO)
---
## 🎯 Ordre de priorité (état 2026-09-28 — cycle v7 ouvert)
---
@@ -628,14 +1062,33 @@ app/
---
## 🎯 Ordre de priorité recommandé (état 2026-09)
## 🎯 Ordre de priorité (état 2026-09-28 — cycle v7 ouvert)
### Cycle v6 — livré, pour mémoire
1. ~~**v5.2.0 → Infrastructure & Polish**~~ ✅ livré
2. ~~**v5.0.0 → Command palette + FTS5**~~ ✅ livré
3. ~~**v5.3.0 → Inline databases + templates + validation**~~ ✅ livré
4. ~~**v5.13.0 → Realtime (WS + présence)**~~ ✅ livré
5. ~~**v5.10.0 → Interactions de bloc**~~ ✅ livré
6. ~~**v5.4.0 → Expérience éditeur**~~ ✅ livré
7. ~~**v5.5.0 → Embeds & Média riche**~~ ✅ livré
8. ~~**v5.6.0 → Import de données (6 phases)**~~ ✅ livré
9. ~~**v5.7.0 → Database Avancée (Pt. 2)**~~ ✅ livré
10. ~~**v5.8.0 → Calendrier & Rappels**~~ ✅ livré
11. ~~**v5.11.0 → Wiki-links & mentions de page**~~ ✅ livré
12. ~~**v5.12.0 → Templates & verrouillage de page**~~ ✅ livré
13. ~~**v5.14.0 → Synced blocks**~~ ✅ livré · ~~**v6.0–v6.7 → PWA, Perms, Clipper, API v2, Realtime prod, Synced prod, Agent API, SSO**~~ ✅ livré
### Cycle v7 — à livrer
| Ordre | Version | Effort | Impact |
|---|---|---|---|
| 1 | **v6.8.0 → Sites & Forms publics** | M | 🔴 publier + collecter |
| 2 | **v6.9.0 → Search sémantique + Ask AI** | M | 🔴 retrouver |
| 3 | **v7.0.0 → Automations v2 + Workers** | L | 🔴 automatiser |
| 4 | **v7.1.0 → Calendar sync + Meeting Notes** | M | 🟠 |
| 5 | **v7.2.0 → SCIM + 2FA + Audit UI** | M | 🟠 enterprise |
| 6 | **v7.3.0 → Wiki/Teamspaces + Polish** | S–M | 🟢 finition |
1. ~~**v5.2.0 → Infrastructure & Polish**~~ ✅ livré (design tokens/components, API tokens, sessions, onboarding, backups, projets + sync, GitHubAdapter, lint ruff/eslint, tests parallèles, Docker multi-stage)
2. ~~**v5.0.0 → Command palette + FTS5**~~ ✅ livré (palette Ctrl+K + `GET /api/search`)
3. ~~**v5.3.0 → Inline databases + templates + validation**~~ ✅ livré (slash `/database`, 6 templates, validation propriétés)
4. ~~**v5.13.0 → Realtime (WS + présence)**~~ ✅ livré (`app/services/realtime_server.py` + `WS /ws/pages/{id}`, présence, curseurs live, merge LWW, 12 tests)
5. ~~**v5.10.0 → Interactions de bloc**~~ ✅ livré (drag&drop multi, undo/redo, duplicate, menu ⋮, en-têtes de tableau, 9 tests)
6. ~~**v5.4.0 → Expérience éditeur**~~ ✅ livré (backlinks, page/collection duplicate, corbeille globale + purge 30 j, historique de version UI, import Markdown/CSV/Notion)
---
## Résumé des phases
@@ -647,11 +1100,89 @@ Base + Kanban Éditeur + Gitea UX Pro MVP Onboard
+ UI Notion + Tags + Admin + Sharing COMPLETED
+ GitHub OAuth + Library
v4.0.2 ✅ v4.1–4.9 ✅ v4.10 ✅ v5.0–5.3 ✅ v5.13 ✅ · v5.10 ✅ v5.14 ⬜ v6.0 ⬜
Quality DB views, Agent IA Palette → Realtime + Synced Pro + Agent
& Tests Templates & COMPLETED Automations Interactions blocks
Collaboration Realtime, de bloc (undo/
DB avancée, redo, drag&drop,
Calendrier, AI duplicate)
v4.0.2 ✅ v4.1–4.9 ✅ v4.10 ✅ v5.0–5.3 ✅ v5.13 ✅ · v5.10 ✅ v5.5 ✅ · v5.7 ✅ v5.14 ✅ · v6.0 ✅
Quality DB views, Agent IA Palette → Realtime + Embeds & Synced PWA · Perms
& Tests Templates & COMPLETED Automations Interactions Média riche blocks Clipper · API v2
Collaboration Realtime, de bloc (undo/ (embed, + DB
DB avancée, redo, drag&drop, bookmark, avancée
Calendrier, AI duplicate) lightbox…) (Pt.2) v6.1 ✅ v6.2 ✅ v6.3 ✅
*Dernière mise à jour: 2026-09-12 — **v5.4.0 Expérience éditeur validé** (17 tests dédiés, 397 tests suite complète, ruff/eslint verts) ; reste v5.11 → v6.0*
*Dernière mise à jour: 2026-09-29 — **cycle v7 ouvert et backlog planifié** : v6.8.0 Sites & Forms ✅ → v6.9.0 Search + Ask AI ✅ → v7.0.0 Automations v2 + Workers ✅ → v7.1.0 Calendar sync + Meetings ✅ → **v7.2.0 SCIM + 2FA + Audit + gouvernance agents ✅ (52 tests)** → **v7.3.0 Wiki/Teamspaces + Polish ✅ (72 tests, suite 1016 verts)**. Version courante **7.3.0**. Follow-ups v7.3 livrés : sidebar teamspaces, notif `page.updated`, charts `number`/multi-DB, unfurl `gitea:`/`github:`, page Settings → Audit. Voir docs `V68`–`V73`.*
---
## 🔴 Anomalies & dette technique — audit complet 2026-09-30
> **Méthode** : `ruff check app tests` (clean) · `pytest -n auto` (524 s) · introspection des routes réelles (**680 routes / 40 routers**) · 3 audits parallèles (sécurité, backend, frontend) — **chaque item ci-dessous relu file:line dans le code**, rien n'est rapporté sur oui-dire.
> **Tests réels : 1002 passed / 14 failed** (les docs annoncent « 1016 verts » → A1).
> **Cause racine de la moitié des items sécurité** : pas de middleware d'auth global (`main.py` ne monte que Session/CSRF/CSP/RateLimit/CORS), `get_current_user` (`auth/session.py:178`) **renvoie `None` au lieu de lever**, et 4 fallbacks « single-user legacy » transforment un anonymous en admin : `dashboard.py:687` (`else 1`), `workspace.py:23` (`or {"login":"admin","id":1}`), `agent.py:95-101` et `agent.py:115-131` (ligne `admin`). **Supprimer ces 4 fallbacks + un garde de route partagé corrige ~15 items d'un coup.**
### 🔴 P0 — Critique (avant toute exposition réseau)
- [x] **A1 — 13 tests en échec = deps manquantes** : `pyotp`, `webauthn`, `cbor2` listés dans `requirements.txt` mais absents du `.venv` → 6 tests 2FA (`ModuleNotFoundError: No module named 'pyotp'`), 7 tests WebAuthn (501 « WebAuthn library not installed »). Le 14ᵉ échec **n'est pas fixe** (run 1 = `test_v67_sso::test_env_config_fallback_when_table_empty`, run 2 = `test_v69_search_ask::test_ask_rate_limit` « assert 200 == 429 ») → isolation cassée : compteurs de rate-limit en mémoire partagés par worker + `tests/test_v54.py:208` fait `app.config.settings = app.config.Settings()` (rebinding explicitement interdit par `conftest.py:36-41`), alors que 17 modules font l'import précoce. *Fix : `uv pip install -r requirements.txt -r requirements-dev.txt --python .venv` · virer le rebinding de test_v54 · compteur de rate-limit par test. Effort : **XS**.*
- [x] **A2 — Cycle v6.8→v7.3 jamais committé** : 22 fichiers modifiés + 33 nouveaux (≈ 8 580 lignes non suivies), dernier commit `v6.7.0` (2026-09-24), alors que `VERSION=7.3.0` et CHANGELOG/ROADMAP/WORKLOAD annoncent « livré ». *Fix : commit + push + tag `v7.3.0`. Effort : **XS**.*
- [x] **A3 — Takeover admin non authentifié** : `PUT /api/user/password` (`dashboard.py:703`) passe par `_get_user_id` (`dashboard.py:687-689`) qui finit en `... else 1` → sans aucun cookie : `UPDATE users SET password_hash=? WHERE id=1` = l'admin seedé. `/api/user` est en plus **exclu du CSRF**. *Fix : 401 sans session + exiger le mot de passe actuel ; supprimer le `else 1`. Effort : **S**.*
- [x] **A4 — Mint de tokens API non authentifié (×2)** : `POST /api/user/token` (`dashboard.py:717`) renvoie `fd_<hex>` lié à l'id 1 sans session ; `POST /api/v1/token` (`public_api.py:57-79`) écrit une ligne `user_tokens` valable sur tout `/api/v1/*` même sans cookie (« legacy shared token »), et `/api/v1` est exclu du CSRF. *Fix : 401 sans session/Bearer `write` ; supprimer le chemin `user_id=0`. Effort : **S**.*
- [x] **A5 — CRUD membres d'espace sans auth + création d'admin** : `POST /workspace/{id}/members` (`workspace.py:69-83`) n'a **aucune vérif de session** et fait `INSERT OR IGNORE INTO users (id, login, full_name, is_admin) VALUES (?,?,?,1)` ; idem `PUT .../members/{user_id}` (85) et `DELETE` (98) ; `/workspace` est exclu du CSRF. Un anonymous s'ajoute à n'importe quel espace et crée un admin. *Fix : session + rôle admin espace sur tout le router ; ne plus écrire `is_admin=1` par là. Effort : **S**.*
- [x] **A6 — ACL collections no-op pour anonymous** : `_require_edit` (`collections.py:59-65`) et `_require_view` commencent par `if not user: return` → l'absence de session = accès total en écriture ; utilisé par la création (2637) et la modif/suppression (544, 617) de pages ; `/db/` est exclu du CSRF. *Fix : `if not user: raise 403/404`. Effort : **XS**.*
- [x] **A7 — Création de pages sans session, CSRF-exempt** : `POST /board/api/pages` (`board.py:1381-1404`) lit la session mais **ne vérifie jamais `if not user`** (contrairement à `set_page_lock`, `board.py:128-130`), et `/board/api/pages` est exclu du CSRF ; même pattern « No session → legacy single-user behaviour » en lecture `board.py:1420-1424`. *Fix : 401 sans session + sortir `/board/api/pages` de la liste CSRF. Effort : **S**.*
- [x] **A8 — Mot de passe admin codé en dur, re-seedé à chaque boot** : `app/main.py:80` `hash_password("FlowDeck2026!")` puis `INSERT OR IGNORE ... 'admin' ... is_admin=1` (80-85). Literal commité + scannable + réappliqué si le hash est effacé. *Fix : mot de passe aléatoire au premier boot (affiché une fois) ou `FLOWDECK_ADMIN_PASSWORD` ; ne re-hasher qu'au premier démarrage. Effort : **XS**.*
- [x] **A9 — DB de prod trackée dans git** : `flowdeck.db` (11 users, e-mails, `password_hash`, 9 sessions actives), `flowdeck_dev.db`, `test-commit.md`, `upload_test.txt` sont dans l'index **et** absents de `.gitignore` **et** de `.dockerignore` → `COPY . .` les embarque dans l'image. *Fix : `git rm --cached` + ajouter `*.db`, `*.db-*`, `test-commit.md`, `upload_test.txt`, `e2e/node_modules/`, `e2e/shots/` à `.gitignore` **et** `.dockerignore` + rotation du `app_secret_key` (les sessions sont révoquées). Effort : **S**.*
- [x] **A10 — Jinja2 `autoescape` désactivé partout** : les 29 sites construisent `Environment(loader=FileSystemLoader("app/templates"))` sans `autoescape` (vérifié à l'exécution : `autoescape = False`, jinja2 3.1.6 ; `grep autoescape app/*.py` → 0 hit). Résultat : 326 interpolations `{{ … }}` brutes dans 39 templates, et **tous les `|safe` du codebase sont des no-op**. Pannes concrètes : `notes.html:16,25` (`<textarea>{{ content }}</textarea>` + preview), `public_page.html:7,161` (titre non échappé sur les pages publiques `/s/`), `card_detail.html:48,85` (`issue.body|safe`, `comment.body|safe`), `base.html:140` (nom de page injecté en JS inline → exécution), `base.html:1789` (`innerHTML + item.name` depuis l'arbre Gitea), `base.html:1333` (`safeName` n'échappe que les guillemets, pas `<`/`>`). *Fix **à la racine** : un seul `app/templating.py` avec `ENV = Environment(loader=..., autoescape=select_autoescape(["html"]))`, remplacer les 29 instantiations, puis re-trier les `|safe`. Effort : **M**.*
- [x] **A11 — Path traversal en lecture** : `GET /api/settings/avatar/{filename:path}` (`dashboard.py:1870-1877`) fait `Path("/data/avatars") / filename` puis `FileResponse` **sans `.resolve()` ni `relative_to()`** alors que le bon motif existe 40 lignes plus bas (`dashboard.py:1479-1484`). Le `:path` Starlette accepte les `/`. *Fix : copier la garde de `serve_uploaded_file`. Effort : **XS**.*
### 🟠 P1 — Hautes
- [x] **A12 — SSRF avec exfiltration via unfurl OG** : `POST /api/og/metadata` (`board.py:1887,1908`) sans auth, `fetch_og_metadata` (`og_fetcher.py:124-129`) fait `client.get(src, follow_redirects=True)` **sans contrôle d'hôte**, renvoie title/description (~400 car.) → `169.254.169.254`, `localhost` atteignables et partiellement lisibles. Le garde existe déjà : `app/services/importers/url_fetch.py:22 _is_public_host`. *Fix : le réutiliser + re-vérifier après redirection. Effort : **S**.*
- [x] **A13 — Automations CRUD sans auth + SSRF webhook** : `create_automation` (`automations.py:74`), `update_automation` (113), `delete_automation` (142), `run_automation_endpoint` (160) n'ont **aucune auth** ; `services/automations.py:167-179` poste `context` (données de page) vers `action["url"]` sans validation d'hôte ; `/api/automations` et `/workspace/automations` sont exclus du CSRF. *Fix : session (admin pour CUD) + `_is_public_host` sur l'action webhook. Effort : **S**.*
- [x] **A14 — Fallback ligne `admin` dans le router agent → SSRF `api_base`** : `_current_admin` (`agent.py:115-131`) et `_current_user_id` (95-101) retombent sur la row `admin` sans session ; `PATCH /api/agent/providers` et `POST .../providers/test` (1000-1043) envoient `api_base` dans `LLMClient(...).ping()` → anonymous = pointer le serveur vers une URL interne ; `/api/agent` exclu du CSRF. *Fix : 403 sans session + validation d'`api_base` (bloquer les hôtes privés). Effort : **M**.*
- [x] **A15 — Webhooks sortants créés sans auth** : `POST /workspace/webhooks` (`workspace.py:672-686`) : aucune auth, aucune validation d'URL, `DELETE` (689) idem → + le retry scheduler, le serveur POSTe chaque événement (titres, contenu) vers l'URL d'un attaquant. *Fix : session admin + `_is_public_host`. Effort : **S**.*
- [x] **A16 — Lectures de pages/export sans aucune ACL** : `export.py:53` (`_load_page_or_404` = simple `SELECT ... WHERE id=?`), `dashboard.py:1141-1186` (`download_page_file`, `page_file_content`), et la lecture legacy `board.py:1420-1424` → contenu de **toute** page énumérable par id, sans session. *Fix : passer par `PermissionManager.can_view_page` + 401 anonymous. Effort : **M**.*
- [x] **A17 — Router legacy `/api` qui mute sans auth** : `move_card` (`api.py:98`), `set_col_mapping` (177), `delete_col_mapping` (207), `create_issue`/`update_issue` (281/322), `delete_checklist[_item]` (522/531), `PUT /users/me` (558) → seul garde = `_check_rate_limit`. *Fix : un `dependencies=[Depends(...)]` au niveau du router (session **ou** Bearer). Effort : **S**.*
- [x] **A18 — Collection publiée quelconque + stocké XSS** : `GET /workspace/public/{collection_id}` (`workspace.py:699-719`) « no auth required », **ignore les flags `restricted/private`**, et interpole `coll['name']`/`p['title']` dans un `HTMLResponse(f"""…""")` sans `html.escape`. *Fix : respecter les flags de partage + `html.escape`. Effort : **S**.*
- [x] **A19 — Liste CSRF trop large (34 préfixes) — TERMINÉ 2026-09-30 : 17 préfixes retirés, aucun préfixe cookie-auth n'est plus exempté** (12 en v7.3.5 après scan + 5 en v7.3.6 après équipement des 46 call sites) (`/db/`, `/workspace`, `/workspace/automations`, `/board/api/favorites`, `/api/pages`, `/api/recents`, `/api/notifications`, `/api/comments`, `/api/automations`, `/api/github`, `/api/admin`, `/api/onboarding`) puis les 5 derniers en v7.3.6 (46 call sites équipés de `X-CSRF-Token` dans agent_panel, settings, local_workspace, library, gitea_workspace, workspace, workspaces, welcome) — il ne reste que du Bearer/callback/public/infra : `csrf.py:21,25` couvre `/api/v2`, `/api/admin`, `/db/`, `/workspace`, `/api/user`, `/api/settings`, `/board/api/pages`, `/api/local-workspace`, `/api/comments`, `/api/agent`, `/api/automations`, `/auth/2fa` — tous **cookie-auth**. Seul `/scim/v2` est justifié par le commentaire de la ligne 19-20. Bonus : `/api/workspace` exempt aussi `/api/workspaces/*`. Filet restant = `SameSite=Lax` par défaut (jamais déclaré explicitement dans `main.py:150`). *Fix : garder un petit ensemble SAFE (webhooks, `/api/v1`, `/api/v2` Bearer, `/scim/v2`, callbacks OAuth/SSO) + ancrer les préfixes ; ajouter le header sur les 49 `fetch()` concernés (helper `csrfFetch` existe déjà : `base.html:892`). Effort : **M**.*
- [ ] **A20 — CSP sans filet : `script-src 'unsafe-inline' 'unsafe-eval'`** (`security.py:67`) → aucun nonce/hash ; combiné à A10, chaque sink XSS ci-dessus tourne sans violation CSP. *Fix : externaliser le JS inline (A27), passer à `'nonce-…'`, retirer `'unsafe-eval'` (Alpine/HTMX n'en ont pas besoin par défaut), resserrer `img-src`/`connect-src`. Effort : **L**.*
- [ ] **A21 — `sqlite3` synchrone sur l'event loop — PARTIEL 2026-10-01 : `PRAGMA busy_timeout=5000` ajouté au point d'entrée unique `get_conn()` (db.py)** ; reste le wrapper async `anyio.to_thread` + la migration des 510 call sites : `get_conn()` (`db.py:833-843`) est synchrone et **510 des 689 `async def` de routes** l'appellent (805 occurrences au total ; 0 `run_in_threadpool`, 1 seul `asyncio.to_thread` dans tout le dépôt : `semantic_search.py:262`) ; connexion neuve par requête (`connect` + 2 PRAGMA), **aucun `busy_timeout`**. Chaque requête bloque la boucle. *Fix : wrapper async (`anyio.to_thread.run_sync`) partagé, migrer d'abord `api_v2`/`dashboard`/`collections`/`board` + `PRAGMA busy_timeout=5000`. Effort : **M**.*
- [x] **A22 — Validateur d'upload = code mort** : `validate_upload()` + `ALLOWED_EXTENSIONS` + `MAX_UPLOAD_SIZE` (`security.py:33-40`) n'ont **aucun appelant** ; `upload_local_workspace_file` (`dashboard.py:1507-1574`) fait `file_path.write_bytes(content)` après seulement `Path(filename).name` — ni taille, ni extension, ni auth, et `/api/local-workspace` est exclu du CSRF. Chemin aussi codé en dur `Path(f"/data/uploads/...")` au lieu de `FLOWDECK_DATA_DIR`. *Fix : appeler `validate_upload()` avant `read()` + session. Effort : **S**.*
- [x] **A23 — N+1 avérés** : `dashboard.py:905` (`COUNT(*) FROM pages` par page dans une boucle de 20), `board.py:2141-2146` (INSERT + `_extract_ai_keywords` par issue), `collections.py:378` (INSERT par propriété en boucle). *Fix : une agrégation `GROUP BY` / un `executemany`. Effort : **S**.*
- [x] **A24 — 2 routes silencieusement mortes (vérifié en servant l'app)** : `GET /workspace` → `dashboard.py:820` (HTML, inclus 1er, `main.py:158`) écrase `workspace.py:30` **et** `library.py:537` (JSON, inclus après) ; `GET /api/projects` → `projects.py:24` (inclus 161) écrase `api.py:82` (handler Gitea). *Fix : supprimer/renommer les doublons + assertion au boot « pas deux routes même méthode+chemin ». Effort : **XS**.*
- [x] **A25 — 92 `except Exception: pass` dont 68 dans des handlers async, 0 loggé** (471 `except Exception` au total, 102 dans `api_v2.py` seul) ; pire : `api_v2.py:504-511` avale l'échec de `materialize_properties` **dans la transaction de création de collection** (collection commitée sans schéma), `api_v2.py:859`, et tous les `await _fire_event(...)` (`1750`, `885`, `903`) → pipeline d'événements/webhooks cassé = invisible. *Fix : `logger.exception` sur les 68 sites ; supprimer le try autour des 2 writes de `create_collection_v2`. Effort : **M**.*
### 🟡 P2 — Moyennes
- [ ] **A26 — Config piège** : (a) `FLOWDECK_STANDALONE` documenté dans `config.py:26` mais **jamais lu** (le champ réel est `STANDALONE`, pas de `env_prefix`, `extra="ignore"` avale la coquille) ; (b) `.env.example` documente `postgresql://…` mais `db_path` (`config.py:133`) retombe silencieusement sur SQLite ; (c) `db_path` fait `Path("/" + p)` (`config.py:132`) → pour `sqlite:////data/flowdeck.db` le résultat est le chemin UNC `\\data\flowdeck.db` (vérifié : inexistant) ; (d) `app_secret_key="change-me-to-random"` (`config.py:36`) + `gitea_oauth_client_secret="test-secret"` sans aucun garde au boot alors qu'il signe `flowdeck_session` (`session.py:14`). *Fix : 1 normalisation de `db_path` + `raise` au boot si secret par défaut + corriger le commentaire/`env_prefix`. Effort : **S**.*
- [ ] **A27 — 13 900 lignes de JS inline dans 37 blocs**, ~3 800 livrées sur **chaque** page (`base.html` 1520 + `agent_panel` 1805 + `_icon_picker` 297 + `_header` 124 + `_notification_bell` 69), et **0 linté** : `eslint.config.mjs:50` ne couvre que `static/js/**/*.js` (soit `app.js` + `offline.js`), 2 blocs se neutralisent avec `/* eslint-disable */`. Grosseurs : `_page_editor_scripts` 2517, `local_workspace` 2030, `agent_panel` 1805, `base` 1520, `_database_table_scripts` 1323, `settings` 1093, `library` 1039. *Fix : extraire les gros partials vers `/static/js/*.js` (ils ne sont pas Jinja-interpolés) + ajouter les templates à eslint. Effort : **L**.*
- [ ] **A28 — Dette de découpe (god files)** : `api_v2.py` 115 routes / 131 Ko, `dashboard.py` 63 / 116 Ko (27 pages HTMLResponse + 50 JSON + I/O fichiers, 16 `Environment(...)` locaux), `collections.py` 53 / 112 Ko, `board.py` 53 / 93 Ko (page CRUD + `zipfile` + sync Gitea). *Fix : scinder par **concern** (`pages_html`, `files`, sous-modules `api_v2/*`) — mécanique, 0 changement d'URL. Effort : **L**.*
- [ ] **A29 — Endpoints dupliqués 2-3×** : publish/unpublish existe en 3 endroits (`sharing.py:304/345`, `board.py:1020/1039`, `api_v2.py:1743/1761`) avec slug et auth **différents** ; listing collections ×3 (`/api/v1/collections`, `/db/api`, `/api/v2/collections`) ; `/api/users/me` ×2. *Fix : un `services/publish.py` partagé, les routers déléguent. Effort : **M**.*
- [ ] **A30 — 16 fonctions top-level jamais référencées**, dont `require_scope` (`api_v2_helpers.py:213`, la factory FastAPI qui doit faire les scopes — les handlers font `has_scope(...)` à la main), `validate_upload`, `_get_user_or_redirect`, `_require_user_gitea`, `unsync_block`, `find_referring`… *Fix : câbler `validate_upload` (A22) + `require_scope`, supprimer le reste. Effort : **S**.*
- [ ] **A31 — Dette migrations** : `migrations.py` 1 522 lignes / 66 Ko, 28 migrations (versions 2-29, contiguës, bien version-gated), **25 copies du motif `PRAGMA table_info`** sans helper (`table_exists`/`column_exists` inexistants), 30 `ALTER TABLE`, et `fn(conn)` tourne **hors transaction** → un échec au milieu laisse du DDL partiel commité. *Fix : 3 helpers + transaction par migration. Effort : **M**.*
- [ ] **A32 — Couverture de tests par trou** : routers à **0 test** : `webhooks.py` (0/3), `notes.py` (0/2), `sidebar_config.py` (0/2), `github_routes.py` (0/2) ; quasi nuls : `library.py` 1/10, `api.py` 3/23 (move, col-mapping, board-config, CRUD issues), `dashboard.py` 17/63, `api_v2.py` 50/115. Points positifs vérifiés : 1 002 tests, **aucun sans `assert`**, aucun qui touche le réseau réel. *Fix : 1 smoke test par route non couverte (fixture TestClient existante). Effort : **M**.*
- [ ] **A33 — Rate limit incomplet et mal câblé** : `security.py:98` ne couvre que `/api/`, `/board/api/`, `/auth/` — pas `/scim/v2`, `/workspace`, `/db/`, `/s/{slug}/auth` (brute force du mot de passe de site, `sites.py:599`), ni `/f/` ; `max_requests=100` codé en dur alors que `settings.rate_limit_requests=60` n'est **jamais lu** ; clé = `request.client.host` (tous les users derrière 1 proxy = 1 seau) ; `_store` (`security.py:113,134-145`) **jamais épuré** → croissance mémoire par IP. `config.py:44` ment donc sur la valeur. *Fix : lire le settings, ajouter les préfixes, épurage, `X-Forwarded-For`. Effort : **S**.*
- [ ] **A34 — 10 schedulers sans observabilité** : `main.py:90-124` — les boucles **ont** bien un `try` interne (vérifié), mais **aucun `add_done_callback` ni restart** : une exception hors `try` tue la tâche en silence ; `calendar_sync.py:469` et `automations.py:484` loggent leurs échecs en `logger.debug` (invisibles à `LOG_LEVEL=INFO`) ; le `finally` (`main.py:136-139`) ne catch que `CancelledError` → un task mort re-raise à l'arrêt. *Fix : helper `spawn()` avec `add_done_callback` (log + recreé) + passer les 2 debug en warning. Effort : **S**.*
- [ ] **A35 — Docs/périmètre dérivés** : `docs/openapi-v2.json` = `info.version 6.7.0`, **439 chemins vs 511 réels** (v6.8→v7.3 non documentés) · `README.md:5` = v6.7.0 alors que `VERSION=7.3.0` · `API_GUIDE_V6.md:8` = « 427 chemins » · **ROADMAP titre dupliqué** `## 🎯 Ordre de priorité (état 2026-09-28)` aux lignes 1052 (vide) et 1065 · drift Python : Dockerfile/CI/README = 3.12, venv local = 3.13, `uv.lock` ≥3.13, ruff target py312. *Fix : régénérer l'OpenAPI à chaque bump (`app.openapi()`), une passe README, dédoublonner la section, aligner 3.13 partout. Effort : **S**.*
- [ ] **A36 — Chaîne de dépendances cassée** : `pyproject.toml` **sans `[project]` ni `dependencies`** (35 lignes, que pytest+ruff), `uv.lock` gitignoré (`.gitignore:19`) et réduit à 3 lignes → aucun verrouillage reproductible ; deps mortes dans `requirements.txt` : **`aiosqlite`, `slowapi`, `loguru`, `packaging` = 0 import** (le rate-limit maison a remplacé slowapi). *Fix : purger les 4 mortes, soit `[project].dependencies`, soit un lock réel. Effort : **S**.*
- [ ] **A37 — CORS `allow_origins=["*"], allow_methods=["*"], allow_headers=["*"]`** (`main.py:154`) alors que l'auth est cookie de session (avec A19 qui désactive le CSRF sur la plupart des routes) — et `allow_credentials` n'est pas posé. *Fix : origines explicites (`app_base_url` + frontends connus). Effort : **XS**.*
### 🟢 P3 — Basses / hygiène
- [ ] **A38 — Duplication front systémique** : helper CSRF réimplémenté **10× sous 5 noms** (`getCsrfToken` ×3 dont 2 corps différents dans `base.html:852,1732`, `getCsrf` ×2, `_getCsrf`, `csrf()`, `csrfTok()`) + ~25 `document.cookie.match(/csrf_token=…)` en dur ; 12 `function` définies dans 2+ templates (`onDoc` ×5, `escHtml`, `esc`, `getCsrf`, `openCardDetail`…) sans IIFE systématique → ombre silencieuse entre partials ; `library.html` et `local_workspace.html` partagent **21 noms de méthodes identiques** (1 039 + 2 030 lignes quasi jumelles). *Fix : un `window.FlowDeck.getCsrf` + wrappeur IIFE/`type="module"` + un `workspace-tree.js` partagé. Effort : **M**.*
- [ ] **A39 — HTMX chargé (50 Ko) pour 10 attributs** vs 265 `fetch(` manuels (36 `hx-*` dont 10 réels ; `settings.html` 42 fetch/0 hx, `_page_editor_scripts` 56/0). *Fix : soit drop `htmx.min.js` et convertir les 10, soit rien. Effort : **XS**.*
- [ ] **A40 — Assets** : `?v=` incohérent (`app.css?v=5.1.1` mais CSS modifié le 2026-09-14 > dernier bump 2026-09-12 → servi depuis le cache), la même liste d'assets est **dupliquée** dans `sw.js:19-31`, htmx/alpine/prism vendored **sans bannière de version ni SRI**, 3 `<script src>` sans `?v=` (`base.html:116-118`). *Fix : une source unique `{{ asset_version }}` lue par `base.html` et `sw.js` + versions notées dans `static/js/VENDOR.md`. Effort : **S**.*
- [ ] **A41 — ~10 Ko de CSS mort** : 75 classes d'`app.css` jamais référencées (97 règles = 10 082 octets) — `.sidebar-invite*`, `.skeleton-*`, `.toast-error|info`, `.slash-group*`, `.block-h1..h4`, `.ftable-*` (18 revérifiées une à une). *Fix : purge one-shot contre `app/templates/**` + `app.js`. Effort : **XS**.*
- [ ] **A42 — Duplication backend résiduelle** : `Jinja Environment` réinstancié **29 fois** dans 10 routers (16 dans `dashboard.py` seul) — même diff que A10 ; 52 `httpx.AsyncClient` créés à la demande (aucun client partagé) ; cache Gitea sans évacuation des entrées expirées (`gitea_client.py:26-38`) ; `_data_dir()` copié 7 fois (`board.py:1696`, `dashboard.py:1127,1478`, `emoji.py:25`, `export.py:96`, `pipeline.py:30`, `meetings.py:32,58`) + 2 `/data` codés en dur (`dashboard.py:1535,1874`). *Fix : `app/templating.py` + `settings.data_dir` + un client httpx partagé. Effort : **M**.*
- [ ] **A43 — Dette mineure** : 22 `datetime.utcnow()` dépréciés (warnings dans les tests), `health` (`api.py:49`) avale db **et** gitea sans log (« degraded » sans raison + 1 aller-retour réseau par probe), `base.html:120` sert le littéral `__CSRF_PLACEHOLDER__` rempli côté JS (fenêtre de course) et `base.html:2292` re-parse ce JSON **à chaque frappe** de la palette sur un GET (où le CSRF ne s'applique pas). *Effort : **XS**.*
### ✅ Vérifié non-problème (ne pas re-checker)
`sort`/`direction` de l'`ORDER BY` f-string **whitelisté** (`api_v2.py:696-703`) · `permissions._grant_common(table, …)` ne reçoit que des littéraux de ses 3 appelants · `admin.py` utilise bien `Depends(admin_required)` · `/api/v2/*` = `get_bearer_user` + scopes · SCIM Bearer-only (commenté `csrf.py:19-20`) · `serve_uploaded_file` et `_file_page_disk_path` ont la bonne garde de traversal · `url_fetch._is_public_host` est un vrai garde SSRF (à réutiliser) · aucun `password_hash`/valeur de token dans les réponses API (les tests le couvrent) · `.env` bien gitignoré, aucun secret en log · les 10 schedulers ont un `try` interne · 0 test sans `assert`, 0 test sur le réseau réel.
### ⚡ Correctifs immédiats (avant le prochain cycle — ~30 min au total)
1. **A2** : commit + push + tag du cycle v7 (sinon tout le reste risque de partir avec une réinit).
2. **A1** : `uv pip install -r requirements.txt -r requirements-dev.txt --python .venv` → les 13 échecs deps disparaissent, le 14ᵉ reste à isoler.
3. **A9** : `git rm --cached flowdeck.db flowdeck_dev.db test-commit.md upload_test.txt` + `.gitignore`/`.dockerignore` + rotation de `app_secret_key`.
→ Puis **A3–A8** (le bloc « fallback admin ») d'un seul tenant, puis **A10** (autoescape) qui débloque A18/A20.
*Audit produit le 2026-09-30 · 43 items · aucun code modifié ( ROADMAP seul ).*
→ **A1–A9 corrigés le 2026-09-30** : deps réinstallées (`pyotp`/`webauthn`/`cbor2`), rebinding de `settings` supprimé dans `test_v54.py` → **suite 1016/1016 verts**, cycle committé (`1706ad1`) + tag `v7.3.0` poussé, `.db`/fichiers de test désindexés, `APP_SECRET_KEY` roté dans `.env` (sessions révoquées) · **A3–A8 : 401 sans session sur les routes de compte (mdp actuel exigé), tokens `/api/v1` + `/api/user` sans session → 401, CRUD membres d'espace sous session+role admin, `_require_view`/`_require_edit` sans session → 404/401, création/lecture de page sous session, `/board/api/pages` + `/api/user` sortis du CSRF exempt, seed admin sans mdp en dur (aléatoire ou `FLOWDECK_ADMIN_PASSWORD`). Tests : client connecte par defaut (`_TestSessionAuth`), helper `anon()` sur les 40 tests d'anonymat → suite 1016/1016 + ruff OK, commit `d125eb3` · **A10 : `app/templating.py` (ENV partagé + autoescape `select_autoescape(["html"])`) remplace les 29 instantiations, `|safe` retriés (corps d'issue/commentaires echappes, `sidebar_config` en `|tojson`) → suite 1016/1016, version 7.3.1 · **A11 (traversal avatar) + A18 (vue publique : 404 restricted/private + html.escape)** : `tests/test_audit_p0_fixes.py`, suite 1019/1019, version 7.3.2 · **A12/A13/A15/A17/A22/A23/A24** : garde SSRF par hop, routers automations + /api sous session/Bearer, webhooks admin+URL publique, uploads validés, N+1 en GROUP BY/executemany, 2 doublons de routes supprimés → suite 1025/1025, version 7.3.3 · **A16** : exports + pièces jointes sous session + `can_view_page` → suite 1026/1026, version 7.3.4 · **A19 (partiel)** : 12 préfixes CSRF retirés après scan front (12 préfixes = tous ceux dont les appels portent déjà le header), reste 5 préfixes / 49 fetchs → suite 1026/1026, version 7.3.5 · **A19 terminé** : 46 call sites front équipés, plus aucun préfixe cookie-auth exempté → suite 1026/1026, version 7.3.6 · **A14** : fallback `admin` supprimé de `_current_user_id`/`_current_admin` (401 sans session), `api_base` validé scheme http(s) sans identifiants (hôtes privés acceptés = Ollama localhost, commentaire `ponytail:`) → suite 1027/1027, version 7.3.7 · **A25** : 84 `except Exception: pass` remplacés par `logger.exception(fn)` (19 fichiers, +`logger` manquants), try supprimé sur `materialize_properties` dans `create_collection_v2` et `apply_db_template_v2` (rollback au lieu d'un commit sans schéma), test de rollback · **A21 (partiel)** : `busy_timeout=5000` dans `get_conn()` → suite 1028/1028, version 7.3.8.**
+1 -1
View File
@@ -1 +1 @@
5.11.1
7.3.8
+19 -4
View File
@@ -1,7 +1,7 @@
# WORKLOAD — FlowDeck Notion Clone
> **Début**: 2026-07-08 | **Version**: v2.2.0 | **Statut**: EN COURS 🔄
> **Cible v3.0**: Multi-User, Multi-Forge (Gitea/GitHub), Standalone
> **Début**: 2026-07-08 | **Version**: v7.3.8 (audit sécurité — A25 + A21 partiel) | **Statut**: EN COURS 🔄
> **Cible**: parité Notion + intégration forge · **Follow-ups v7.3 livrés**: sidebar teamspaces, notif `page.updated`, charts `number` + dashboards multi-DB, unfurl forge, UI Settings → Audit — voir `ROADMAP.md § v7.3.0`
## Avancement Global
@@ -21,7 +21,22 @@
| v2.0 | Multi-User + Editor Complete | ✅ | 67/67 |
| v2.1 | Public API, Webhooks, PWA | ✅ | 73/73 |
| v2.2 | Share/Publish, Favorites, Library | ✅ | 73/73 |
| v3.0 | **Auth locale, Multi-Forge, Standalone** | 🔲 | — |
| v3.0 | Auth locale, Multi-Forge, Standalone | ✅ | — |
| v4.x–v5.x | MVP → Agent IA, palette, automations, import, calendrier, wiki-links, synced blocks | ✅ | 523+ |
| v6.0–v6.3 | PWA offline, permissions granulaires, web clipper, API publique v2 | ✅ | 668+ |
| **v6.4.0** | **Realtime production (merge 3-voix, broadcast non bloquant)** | ✅ | **749+** |
| **v6.5.0–v6.5.1** | **Synced blocks production (databases/vues) + webhooks v2 complets** | ✅ | **749** |
| **v6.6.0** | **Agent phase 5 — API publique agent & skill marketplace** | ✅ | **764+** |
| **v6.7.0** | **SSO / SAML + OIDC entreprise (Enterprise Auth)** | ✅ | **802** |
| v6.8.0 | Sites multi-pages + Forms publics ([doc](docs/V68_Sites_Forms.md)) | ✅ | 20 |
| v6.9.0 | Recherche hybride + Ask AI RAG ([doc](docs/V69_Search_Ask_AI.md)) | ✅ | 24 |
| v7.0.0 | Automations multi-étapes + Workers lite ([doc](docs/V70_Automations_Workers.md)) | ✅ | 31 |
| v7.1.0 | Calendar sync + Meeting Notes ([doc](docs/V71_Calendar_Meetings.md)) | ✅ | 15 |
| v7.2.0 | SCIM + 2FA + Audit UI + gouvernance agents ([doc](docs/V72_Enterprise_SCIM_2FA.md)) | ✅ | 52 |
| v7.3.0 | Teamspaces + Verified + Polish + follow-ups ([doc](docs/V73_Wiki_Teamspaces_Polish.md)) | ✅ | 72 |
> **Suites de régression** : `test_v72_enterprise.py` (52) + `test_v73_wiki_polish.py` (72) = **124 verts** · suite complète `-n auto` = **1016 passed**.
> **Follow-ups v7.3 livrés** (voir § v7.3.0 du `ROADMAP.md`): sidebar par teamspace, charts `number`/multi-DB, notif `page.updated` aux followers, unfurl `gitea:`/`github:`, page Settings → Audit — + 21 casses SSO corrigées (install `python3-saml`/`authlib`).
## Blocs Complétés
@@ -58,5 +73,5 @@ CRUD collections/pages, 5 vues HTML, relations/rollups/formulas, sub-items/depen
- **BDD**: SQLite WAL mode, 21 tables, foreign keys ON
- **Auth**: OAuth2 Gitea + sessions signed (itsdangerous) + token API
- **Déploiement**: Docker (python:3.12-slim), docker-compose, port 8080
- **Tests**: pytest, 73 tests, TestClient avec SQLite temporaire
- **Tests**: pytest, 764+ tests, TestClient avec SQLite temporaire
- **CI/CD**: Gitea Actions (.gitea/workflows/ci.yml)
+219
View File
@@ -0,0 +1,219 @@
"""OIDC provider — authorization code flow with PKCE (v6.7.0).
Discovery (``.well-known/openid-configuration``) is cached for an hour, the
ID token signature is verified against the issuer JWKS via authlib's JOSE
implementation, and ``iss`` / ``aud`` / ``exp`` / ``nonce`` are checked here
explicitly so the rules are visible and unit-testable.
"""
from __future__ import annotations
import base64
import hashlib
import json
import logging
import secrets
import time
import warnings
import httpx
logger = logging.getLogger(__name__)
#: Default attribute mapping (design doc §3.2) — OIDC claim names.
DEFAULT_OIDC_MAPPING: dict[str, str] = {
"login": "sub",
"email": "email",
"full_name": "name",
"avatar_url": "picture",
"groups": "groups",
}
_DISCOVERY_TTL = 3600.0
_discovery_cache: dict[str, tuple[float, dict]] = {}
class OIDCError(Exception):
"""OIDC processing failure — ``message`` is user-facing."""
def pkce_pair() -> tuple[str, str]:
"""Return ``(code_verifier, code_challenge)`` for the S256 method."""
verifier = secrets.token_urlsafe(64)
digest = hashlib.sha256(verifier.encode("ascii")).digest()
challenge = base64.urlsafe_b64encode(digest).rstrip(b"=").decode("ascii")
return verifier, challenge
def _b64url(data: bytes) -> str:
return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii")
def _b64url_decode(data: str) -> bytes:
return base64.urlsafe_b64decode(data + "=" * (-len(data) % 4))
async def discover(issuer_url: str) -> dict:
"""Fetch (and cache) the issuer's OIDC discovery document."""
issuer = issuer_url.rstrip("/")
url = f"{issuer}/.well-known/openid-configuration"
now = time.time()
hit = _discovery_cache.get(issuer)
if hit and now - hit[0] < _DISCOVERY_TTL:
return hit[1]
try:
async with httpx.AsyncClient(timeout=15) as client:
r = await client.get(url)
r.raise_for_status()
doc = r.json()
except Exception as err:
raise OIDCError(f"OIDC discovery failed ({url}): {err}") from err
if not doc.get("authorization_endpoint") or not doc.get("token_endpoint"):
raise OIDCError("OIDC discovery document is missing authorization/token endpoints")
_discovery_cache[issuer] = (now, doc)
return doc
def build_authorize_url(
doc: dict,
*,
client_id: str,
redirect_uri: str,
scope: str,
state: str,
nonce: str,
code_challenge: str,
) -> str:
from urllib.parse import urlencode
params = {
"client_id": client_id,
"redirect_uri": redirect_uri,
"response_type": "code",
"scope": scope or "openid profile email",
"state": state,
"nonce": nonce,
"code_challenge": code_challenge,
"code_challenge_method": "S256",
}
sep = "&" if "?" in doc["authorization_endpoint"] else "?"
return doc["authorization_endpoint"] + sep + urlencode(params)
async def exchange_code(
doc: dict, *, client_id: str, client_secret: str, code: str, redirect_uri: str, code_verifier: str
) -> dict:
"""Exchange the authorization code for tokens (PKCE, confidential client)."""
data = {
"grant_type": "authorization_code",
"code": code,
"redirect_uri": redirect_uri,
"client_id": client_id,
"code_verifier": code_verifier,
}
auth = None
if client_secret:
auth = (client_id, client_secret)
try:
async with httpx.AsyncClient(timeout=15) as client:
r = await client.post(doc["token_endpoint"], data=data, auth=auth)
except Exception as err:
raise OIDCError(f"OIDC token request failed: {err}") from err
if r.status_code != 200:
raise OIDCError(f"OIDC token endpoint returned {r.status_code}: {r.text[:300]}")
try:
tokens = r.json()
except Exception as err:
raise OIDCError(f"OIDC token endpoint returned a non-JSON body: {err}") from err
if "error" in tokens:
raise OIDCError(f"OIDC error: {tokens.get('error')} {tokens.get('error_description', '')}".strip())
return tokens
async def fetch_userinfo(doc: dict, access_token: str) -> dict:
"""Best-effort userinfo fetch (groups often only live there)."""
endpoint = doc.get("userinfo_endpoint")
if not endpoint or not access_token:
return {}
try:
async with httpx.AsyncClient(timeout=15) as client:
r = await client.get(endpoint, headers={"Authorization": f"Bearer {access_token}"})
if r.status_code != 200:
return {}
data = r.json()
return data if isinstance(data, dict) else {}
except Exception as err: # userinfo is optional enrichment
logger.debug("userinfo fetch failed: %s", err)
return {}
def validate_id_token(
id_token: str, *, issuer: str, client_id: str, nonce: str, jwks: dict
) -> dict:
"""Verify the ID token signature and claims. Returns the claims dict."""
with warnings.catch_warnings():
warnings.simplefilter("ignore", DeprecationWarning)
from authlib.jose import JsonWebKey
from authlib.jose import jwt as jose_jwt
if isinstance(id_token, bytes):
# authlib's jose.jwt.encode() returns bytes; IdP token endpoints send
# str — accept both instead of crashing on ``bytes.count(".")``.
id_token = id_token.decode()
if not id_token or id_token.count(".") != 2:
raise OIDCError("Missing or malformed ID token")
try:
keyset = JsonWebKey.import_key_set(jwks)
except Exception as err:
raise OIDCError(f"Invalid issuer JWKS: {err}") from err
try:
# Pick the key matching the token header (kid) when several are offered.
header = json.loads(_b64url_decode(id_token.split(".")[0]))
kid = header.get("kid")
key = keyset.get_by_kid(kid) if kid and hasattr(keyset, "get_by_kid") else None
token_obj = jose_jwt.decode(id_token, key or keyset)
except Exception as err:
raise OIDCError(f"ID token signature verification failed: {err}") from err
claims = dict(token_obj) # authlib's JWTClaims is a dict subclass
now = int(time.time())
if claims.get("iss") != issuer.rstrip("/") and claims.get("iss") != issuer:
raise OIDCError(f"ID token issuer mismatch: {claims.get('iss')!r}")
aud = claims.get("aud")
aud_list = aud if isinstance(aud, list) else [aud]
if client_id not in aud_list:
raise OIDCError("ID token audience does not include this client")
exp = claims.get("exp")
if not isinstance(exp, int) or exp < now:
raise OIDCError("ID token expired")
iat = claims.get("iat")
if isinstance(iat, int) and iat > now + 300:
raise OIDCError("ID token issued in the future")
if nonce and claims.get("nonce") != nonce:
raise OIDCError("ID token nonce mismatch")
if not claims.get("sub"):
raise OIDCError("ID token has no subject")
return claims
def claims_to_identity(claims: dict, mapping: dict | None = None) -> dict:
"""Map OIDC claims onto the shared ``{login, email, full_name, avatar_url, groups}`` shape."""
mapping = mapping or DEFAULT_OIDC_MAPPING
identity: dict = {"_raw": claims}
for field in ("login", "email", "full_name", "avatar_url"):
source = mapping.get(field) or field
value = claims.get(source, "")
if isinstance(value, list):
value = value[0] if value else ""
identity[field] = str(value or "").strip()
groups = claims.get(mapping.get("groups", "groups"), [])
if isinstance(groups, str):
groups = [groups]
identity["groups"] = [str(g) for g in groups if g]
if not identity["email"]:
identity["email"] = claims.get("email", "") or ""
if not identity["full_name"]:
identity["full_name"] = claims.get("name", "") or identity["email"]
return identity
+279
View File
@@ -0,0 +1,279 @@
"""SAML 2.0 Service Provider — wrapper around python3-saml (OneLogin toolkit).
v6.7.0. Adapts FastAPI's ``Request`` to the toolkit's flat ``request_data``
dict and builds the SP settings from the ``sso_config`` row.
What the toolkit validates in strict mode (all covered by tests):
XML schema, signature of the assertion and/or the message against the IdP
certificate, ``Conditions`` timestamps, ``Audience``, ``Destination``,
``Issuer``, ``Status``, "exactly one assertion", and ``InResponseTo``
against the AuthnRequest id we pass to ``process_response()`` — combined
with the single-use ``sso_requests`` store that makes replay impossible.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass, field
from fastapi import Request
logger = logging.getLogger(__name__)
BINDING_HTTP_REDIRECT = "urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect"
BINDING_HTTP_POST = "urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST"
NAMEID_FORMAT_EMAIL = "urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress"
#: Default attribute mapping (design doc §3.2). ``nameid`` = the assertion's
#: NameID; every other value is matched against attribute Name / FriendlyName
#: / URI local part (so ``email`` finds both ``email`` and
#: ``http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress``).
DEFAULT_SAML_MAPPING: dict[str, str] = {
"login": "nameid",
"email": "nameid",
"full_name": "displayName",
"avatar_url": "avatar",
"groups": "groups",
}
class SAMLError(Exception):
"""SAML processing failure — ``message`` is user-facing, ``reason`` is logged."""
@dataclass
class SAMLIdentity:
"""What a validated assertion tells us about the user."""
name_id: str
name_id_format: str = ""
session_index: str = ""
attributes: dict[str, list[str]] = field(default_factory=dict)
friendly_attributes: dict[str, list[str]] = field(default_factory=dict)
def resolve(self, source: str) -> str:
"""Resolve one mapped source (``nameid`` or an attribute name) → first value."""
if not source or source == "nameid":
return self.name_id or ""
if source in self.attributes and self.attributes[source]:
return (self.attributes[source][0] or "").strip()
# FriendlyName match (case-insensitive)
lower = {k.lower(): v for k, v in self.friendly_attributes.items()}
if source.lower() in lower and lower[source.lower()]:
return (lower[source.lower()][0] or "").strip()
# URI local part match: ".../claims/emailaddress" ~ "emailaddress", and
# a mapping of "email" must still find ".../claims/emailaddress".
want = source.lower().lstrip("./")
for name, values in self.attributes.items():
if not values:
continue
local = name.rsplit("/", 1)[-1].rsplit("}", 1)[-1].lower()
if local == want or local.endswith(want) or want.endswith(local):
return (values[0] or "").strip()
return ""
def external_base_url(request: Request) -> str:
"""Scheme://host the user actually used (proxy-aware, like OAuth redirects)."""
proto = request.headers.get("x-forwarded-proto", "")
scheme = proto.split(",")[0].strip() or request.url.scheme or "http"
fwd_host = request.headers.get("x-forwarded-host", "")
host = fwd_host.split(",")[0].strip() or request.headers.get("host", "localhost:8080")
return f"{scheme}://{host}"
def saml_endpoints(request: Request) -> dict[str, str]:
"""SP entity id + ACS/SLO/metadata URLs derived from the incoming request."""
base = external_base_url(request)
return {
"entity_id": f"{base}/auth/saml/metadata",
"acs": f"{base}/auth/saml/callback",
"slo": f"{base}/auth/saml/logout",
"metadata": f"{base}/auth/saml/metadata",
}
def build_settings(cfg: dict, endpoints: dict[str, str]) -> dict:
"""python3-saml settings dict built from a ``sso_config`` row."""
sign_requests = bool(cfg.get("sign_requests"))
sp: dict = {
"entityId": endpoints["entity_id"],
"assertionConsumerService": {
"url": endpoints["acs"],
"binding": BINDING_HTTP_POST,
},
"singleLogoutService": {
"url": endpoints["slo"],
"binding": BINDING_HTTP_REDIRECT,
},
"NameIDFormat": NAMEID_FORMAT_EMAIL,
}
if cfg.get("sp_private_key") and cfg.get("sp_certificate"):
sp["privateKey"] = cfg["sp_private_key"]
sp["x509cert"] = cfg["sp_certificate"]
idp: dict = {
"entityId": cfg.get("entity_id") or "",
"singleSignOnService": {
"url": cfg.get("sso_url") or "",
"binding": BINDING_HTTP_REDIRECT,
},
"x509cert": cfg.get("x509_certificate") or "",
}
if cfg.get("slo_url"):
idp["singleLogoutService"] = {"url": cfg["slo_url"], "binding": BINDING_HTTP_REDIRECT}
return {
"strict": True,
"debug": False,
"sp": sp,
"idp": idp,
"security": {
"authnRequestsSigned": sign_requests,
"logoutRequestSigned": sign_requests,
"logoutResponseSigned": False,
"wantMessagesSigned": False,
"wantAssertionsSigned": True,
"wantNameIdEncrypted": False,
"wantAssertionsEncrypted": False,
"wantXmlValidation": True,
"signatureAlgorithm": "http://www.w3.org/2001/04/xmldsig-more#rsa-sha256",
"digestAlgorithm": "http://www.w3.org/2001/04/xmlenc#sha256",
"rejectDeprecatedAlgorithm": True,
# FlowDeck is self-hosted: LAN/homelab deploys commonly reach the
# SP through single-label hosts (http://flowdeck/, docker service
# names). python3-saml rejects those URLs unless this is on.
"allowSingleLabelDomains": True,
},
}
def _request_data(request: Request, script_name: str, post_data: dict | None = None) -> dict:
"""Flat request dict expected by ``OneLogin_Saml2_Auth``."""
https = "on" if external_base_url(request).startswith("https") else "off"
return {
"https": https,
"http_host": request.headers.get("host", "localhost:8080"),
"script_name": script_name,
"request_uri": request.url.path,
"query_string": str(request.url.query or ""),
"get_data": dict(request.query_params),
"post_data": post_data or {},
}
def _auth(request: Request, cfg: dict, script_name: str, post_data: dict | None = None):
from onelogin.saml2.auth import OneLogin_Saml2_Auth
settings = build_settings(cfg, saml_endpoints(request))
try:
return OneLogin_Saml2_Auth(
_request_data(request, script_name, post_data=post_data), old_settings=settings
)
except Exception as err: # malformed IdP/SP config (bad cert, missing URL…)
raise SAMLError(f"Invalid SAML configuration: {err}") from err
def create_login(request: Request, cfg: dict, relay_state: str) -> tuple[str, str]:
"""Build the AuthnRequest. Returns ``(redirect_url, authn_request_id)``."""
auth = _auth(request, cfg, "/auth/saml/login")
try:
url = auth.login(return_to=relay_state)
except Exception as err:
raise SAMLError(f"Could not build the SAML AuthnRequest: {err}") from err
request_id = auth.get_last_request_id() or ""
if not request_id:
raise SAMLError("AuthnRequest was built without an id")
return url, request_id
def process_response(request: Request, cfg: dict, post_data: dict, request_id: str) -> SAMLIdentity:
"""Validate the IdP's SAMLResponse and extract the identity.
``request_id`` is the id of the AuthnRequest we issued (from the
single-use ``sso_requests`` row): the toolkit rejects any response whose
``InResponseTo`` does not match it.
"""
auth = _auth(request, cfg, "/auth/saml/callback", post_data=post_data)
try:
auth.process_response(request_id=request_id or None)
except Exception as err:
raise SAMLError(f"SAML response could not be processed: {err}") from err
errors = auth.get_errors()
if errors:
raise SAMLError(auth.get_last_error_reason() or f"SAML errors: {', '.join(errors)}")
if not auth.is_authenticated():
raise SAMLError("SAML response did not authenticate the user")
name_id = auth.get_nameid() or ""
if not name_id:
raise SAMLError("SAML assertion carries no NameID")
return SAMLIdentity(
name_id=name_id,
name_id_format=auth.get_nameid_format() or "",
session_index=auth.get_session_index() or "",
attributes=auth.get_attributes() or {},
friendly_attributes=auth.get_friendlyname_attributes() or {},
)
def metadata_xml(request: Request, cfg: dict) -> str:
"""SP metadata XML (for the IdP configuration screen)."""
from onelogin.saml2.settings import OneLogin_Saml2_Settings
settings = OneLogin_Saml2_Settings(
build_settings(cfg, saml_endpoints(request)), custom_base_path=None
)
try:
xml = settings.get_sp_metadata()
except Exception as err:
raise SAMLError(f"Could not build the SP metadata: {err}") from err
if isinstance(xml, bytes):
xml = xml.decode("utf-8")
return xml
def build_logout_url(request: Request, cfg: dict, return_to: str, name_id: str, session_index: str) -> str:
"""SP-initiated Single Logout (HTTP-Redirect LogoutRequest to the IdP)."""
auth = _auth(request, cfg, "/auth/saml/logout")
if not cfg.get("slo_url"):
raise SAMLError("The IdP has no Single Logout URL configured")
try:
return auth.logout(
return_to=return_to,
name_id=name_id or None,
session_index=session_index or None,
)
except Exception as err:
raise SAMLError(f"Could not build the SAML LogoutRequest: {err}") from err
def process_slo_form(request: Request, cfg: dict, form: dict, query: dict) -> tuple[str | None, list[str]]:
"""Process a LogoutRequest / LogoutResponse received from the IdP.
``form`` holds the POSTed fields, ``query`` the GET parameters (the
HTTP-Redirect binding delivers LogoutRequest/LogoutResponse there).
Returns ``(redirect_url, errors)``.
"""
from onelogin.saml2.auth import OneLogin_Saml2_Auth
settings = build_settings(cfg, saml_endpoints(request))
https = "on" if external_base_url(request).startswith("https") else "off"
post_data = {k: v for k, v in form.items() if k in ("SAMLRequest", "SAMLResponse", "RelayState")}
if not post_data:
post_data = {"SAMLResponse": query["SAMLResponse"]} if "SAMLResponse" in query else {}
req_data = {
"https": https,
"http_host": request.headers.get("host", "localhost:8080"),
"script_name": "/auth/saml/logout",
"request_uri": request.url.path,
"query_string": str(request.url.query or ""),
"get_data": dict(query),
"post_data": post_data,
}
auth = OneLogin_Saml2_Auth(req_data, old_settings=settings)
try:
url = auth.process_slo(keep_local_session=True)
except Exception as err:
raise SAMLError(f"SAML logout could not be processed: {err}") from err
return url, auth.get_errors()
+1 -1
View File
@@ -171,7 +171,7 @@ def _touch_session(sid: str) -> None:
)
conn.commit()
except Exception:
pass
logger.exception("_touch_session")
# FastAPI dependency
+39 -2
View File
@@ -43,6 +43,10 @@ class Settings(BaseSettings):
rate_limit_enabled: bool = True
rate_limit_requests: int = 60 # per minute
# Public API v2 (v6.3.0)
public_api_insecure_ok: bool = False # if True, fd-public-key is accepted (dev only)
api_v2_rate_limit_per_token: int = 300 # req/min per token for /api/v2
# Database
database_url: str = "sqlite:////data/flowdeck.db"
@@ -60,6 +64,14 @@ class Settings(BaseSettings):
project_sync_enabled: bool = True
project_sync_interval_hours: int = 1
# Reminders (v5.8.0) — background scan for due date reminders
reminders_enabled: bool = True
reminder_scan_interval_seconds: int = 60
# Webhooks outbound (v6.4.0) — retry of failed deliveries
webhook_retry_enabled: bool = True
webhook_retry_interval_seconds: int = 60
# Email / SMTP notifications (v4.9.0) — optional. If smtp_host is empty,
# email notifications are skipped (only in-app notifications are delivered).
smtp_host: str = ""
@@ -70,11 +82,36 @@ class Settings(BaseSettings):
smtp_use_tls: bool = True
app_base_url: str = "http://localhost:8080"
# SSO / SAML + OIDC (v6.7.0) — bootstrap fallback ONLY: as soon as an admin
# saves a configuration in Settings → Admin → SSO / Enterprise, the
# `sso_config` table wins (see app/services/sso_provisioning.py).
sso_provider: str = "" # 'saml' | 'oidc' | '' (disabled)
sso_name: str = "Company SSO" # button label on the login page
sso_entity_id: str = "" # SAML: IdP entity id
sso_sso_url: str = "" # SAML: IdP SSO URL (HTTP-Redirect)
sso_slo_url: str = "" # SAML: IdP Single Logout URL
sso_x509_certificate: str = "" # SAML: IdP signing certificate (PEM)
sso_issuer_url: str = "" # OIDC: issuer identifier
sso_client_id: str = "" # OIDC: client id
sso_client_secret: str = "" # OIDC: client secret (env only)
sso_scope: str = "openid profile email"
sso_attribute_mapping: str = "" # JSON, defaults per provider
sso_groups_mapping: str = "[]" # JSON [{sso_group, workspace_role, workspace_id}]
sso_auto_provision: bool = True
sso_only: bool = False # refuse local login when true
sso_sign_requests: bool = False # sign AuthnRequest / LogoutRequest
sso_default_workspace_id: int = 0
# FlowDeck Agent (v4.10.0) — multi-LLM. Empty keys → offline/mock mode
# (deterministic rule-based planner so the agent works without any API key).
agent_enabled: bool = True
llm_provider: str = "offline" # openai | anthropic | google | ollama |
# deepseek | qwencloud | nvidia | openrouter | offline
llm_provider: str = "offline" # any id from llm_client.PROVIDERS
# (openai, anthropic, mistral, cohere,
# google, groq, deepseek, openrouter,
# nvidia, together, perplexity, xai,
# qwencloud, minimax, morph, fireworks,
# cerebras, sambanova, chutes, xiaomi,
# sealion, sensenova, ollama, offline)
llm_model: str = "gpt-4o"
llm_api_key: str = ""
llm_api_base: str = "" # custom base URL (Ollama, OpenRouter, ...)
+11
View File
@@ -442,12 +442,18 @@ def init_db():
id INTEGER PRIMARY KEY AUTOINCREMENT,
page_id INTEGER NOT NULL REFERENCES pages(id) ON DELETE CASCADE,
shared_with_user_id INTEGER REFERENCES users(id),
shared_with_group_id INTEGER REFERENCES user_groups(id) ON DELETE CASCADE,
shared_with_email TEXT DEFAULT '',
permission TEXT NOT NULL DEFAULT 'view',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
created_by INTEGER REFERENCES users(id)
)
""")
# v5.x: migration — partage par groupes (colonne manquante sur DB existantes)
try:
conn.execute("ALTER TABLE page_shares ADD COLUMN shared_with_group_id INTEGER REFERENCES user_groups(id) ON DELETE CASCADE")
except sqlite3.OperationalError:
pass
# 4) recents table
conn.execute("""
CREATE TABLE IF NOT EXISTS recents (
@@ -831,6 +837,11 @@ def get_conn():
conn.row_factory = sqlite3.Row
conn.execute("PRAGMA journal_mode=WAL")
conn.execute("PRAGMA foreign_keys=ON")
# A21 (partiel) : un seul writer par DB — sans timeout la requête concurrente
# échoue immédiatement avec « database is locked » (tests xdist, schedulers).
# ponytail: le reste d'A21 (wrapper async + 510 call sites synchrones sur
# l'event loop) reste à migrer module par module.
conn.execute("PRAGMA busy_timeout=5000")
try:
yield conn
finally:
+134 -30
View File
@@ -8,6 +8,7 @@ from contextlib import asynccontextmanager
from fastapi import FastAPI, Request
from fastapi.middleware.cors import CORSMiddleware
from fastapi.staticfiles import StaticFiles
from starlette.exceptions import HTTPException as _StarHTTPException
from starlette.middleware.sessions import SessionMiddleware
from app.config import settings
@@ -33,15 +34,34 @@ from app.routers import (
security,
sharing,
sidebar_config,
sync,
webhooks,
workspace,
)
from app.routers.api_v2 import router as api_v2_router
from app.routers.api_v2_agent import router as api_v2_agent_router
from app.routers.audit import router as audit_router
from app.routers.automations import router as automations_router
from app.routers.collaboration import router as collaboration_router
from app.routers.emoji import router as emoji_router
from app.routers.gitea import router as gitea_router
from app.routers.github_routes import router as github_router
from app.routers.governance import router as governance_router
from app.routers.imports import page_router as import_page_router
from app.routers.imports import router as imports_router
from app.routers.meetings import router as meetings_router
from app.routers.notifications import router as notifications_router
from app.routers.permissions import router as permissions_router
from app.routers.realtime import router as realtime_router
from app.routers.scim import router as scim_router
from app.routers.search_ai import router as search_ai_router
from app.routers.sites import router as sites_router
from app.routers.sso import router as sso_router
from app.routers.web_clipper import api_router as web_clipper_api_router
from app.routers.web_clipper import router as web_clipper_router
from app.routers.webauthn import router as webauthn_router
from app.routers.wiki import router as wiki_router
from app.routers.workers import router as workers_router
from app.services.webhook_outbound import init_webhook_tables
logging.basicConfig(
@@ -55,15 +75,27 @@ logger = logging.getLogger(__name__)
async def lifespan(_app: FastAPI):
init_db()
init_webhook_tables()
import os
import secrets
from app.db import get_conn
from app.password_utils import hash_password
admin_hash = hash_password("FlowDeck2026!")
# A8 : plus de mot de passe admin codé en dur — mot de passe aléatoire au
# premier boot (affiché une fois) ou FLOWDECK_ADMIN_PASSWORD ; re-seed si absent.
with get_conn() as conn:
conn.execute(
"INSERT OR IGNORE INTO users (login, full_name, email, password_hash, is_admin) VALUES ('admin', 'Admin', '', ?, 1)",
(admin_hash,)
)
conn.commit()
if not conn.execute("SELECT 1 FROM users WHERE login='admin'").fetchone():
admin_pw = os.environ.get("FLOWDECK_ADMIN_PASSWORD") or secrets.token_urlsafe(12)
conn.execute(
"INSERT INTO users (login, full_name, email, password_hash, is_admin) VALUES ('admin', 'Admin', '', ?, 1)",
(hash_password(admin_pw),),
)
conn.commit()
logger.warning(
"Premier démarrage : compte admin créé, mot de passe = %s "
"(définir FLOWDECK_ADMIN_PASSWORD pour le fixer)",
admin_pw,
)
# ── FlowDeck Agent (v4.10.0): scheduled custom-agent triggers ──
from app.routers.agent import agent_scheduler
@@ -85,13 +117,34 @@ async def lifespan(_app: FastAPI):
from app.services.trash import trash_purge_scheduler
trash_task = asyncio.create_task(trash_purge_scheduler())
logger.info("FlowDeck v5.11.1 started on port %d", settings.app_port)
# ── Reminders (v5.8.0): due-reminder scan for database rows ──
from app.services.reminders import reminder_scheduler
reminder_task = asyncio.create_task(reminder_scheduler())
# ── Semantic search (v6.9.0): incremental vector indexing ──
from app.services.semantic_search import semantic_index_scheduler
semantic_task = asyncio.create_task(semantic_index_scheduler())
# ── Calendar sync (v7.1.0): external calendars every 15 min ──
from app.services.calendar_sync import calendar_sync_scheduler
calendar_task = asyncio.create_task(calendar_sync_scheduler())
# ── Webhooks outbound (v6.4.0): retry failed deliveries ──
from app.services.webhook_outbound import webhook_retry_scheduler
webhook_task = None
if settings.webhook_retry_enabled:
webhook_task = asyncio.create_task(webhook_retry_scheduler())
logger.info("FlowDeck v%s started on port %d", dashboard._get_app_version(), settings.app_port)
try:
yield
finally:
for task in (scheduler_task, automation_task, backup_task, projects_task, trash_task):
_tasks = (scheduler_task, automation_task, backup_task, projects_task, trash_task, reminder_task, semantic_task, calendar_task)
if webhook_task is not None:
_tasks = _tasks + (webhook_task,)
for task in _tasks:
task.cancel()
for task in (scheduler_task, automation_task, backup_task, projects_task, trash_task):
for task in _tasks:
try:
await task
except asyncio.CancelledError:
@@ -100,9 +153,9 @@ async def lifespan(_app: FastAPI):
app = FastAPI(
title="FlowDeck",
version="5.11.1",
docs_url="/docs" if settings.log_level == "DEBUG" else None,
redoc_url=None,
version="7.3.8",
docs_url="/docs",
redoc_url="/redoc",
lifespan=lifespan,
)
@@ -113,6 +166,7 @@ app.add_middleware(RateLimitMiddleware)
app.add_middleware(CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"])
app.include_router(auth.router)
app.include_router(sso_router)
app.include_router(dashboard.router)
app.include_router(board.router)
app.include_router(notes.router)
@@ -134,26 +188,47 @@ app.include_router(export.router)
app.include_router(notifications_router)
app.include_router(automations_router)
app.include_router(collaboration_router)
app.include_router(emoji_router)
app.include_router(realtime_router)
app.include_router(agent.router)
app.include_router(search.router)
app.include_router(security.router)
app.include_router(onboarding.router)
app.include_router(sync.router)
app.include_router(imports_router)
app.include_router(import_page_router)
app.include_router(permissions_router)
app.include_router(web_clipper_api_router)
app.include_router(web_clipper_router)
app.include_router(api_v2_router)
app.include_router(api_v2_agent_router)
app.include_router(sites_router)
app.include_router(search_ai_router)
app.include_router(workers_router)
app.include_router(meetings_router)
# v7.2.0 — enterprise admin
app.include_router(scim_router)
app.include_router(webauthn_router)
app.include_router(audit_router)
app.include_router(governance_router)
# v7.3.0 — teamspaces + verified wiki
app.include_router(wiki_router)
app.mount("/static", StaticFiles(directory="static"), name="static")
@app.get("/manifest.json")
async def pwa_manifest():
return {
"name": "FlowDeck",
"short_name": "FlowDeck",
"start_url": "/",
"display": "standalone",
"background_color": "#191919",
"theme_color": "#191919",
"icons": [{"src": "/static/icon-192.png", "sizes": "192x192", "type": "image/png"}],
}
"""Serve the static web manifest from disk (same URL as before v6.0.0)."""
from fastapi.responses import FileResponse
return FileResponse("static/manifest.json", media_type="application/manifest+json")
@app.get("/sw.js")
async def service_worker():
"""Serve the PWA service worker at top-level scope (/)."""
from fastapi.responses import FileResponse
return FileResponse("static/sw.js", media_type="application/javascript")
# ═══════════ API aliases (v4.0.1) ═══════════
@@ -218,12 +293,41 @@ body{font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',sans-serif;backgrou
</html>"""
@app.exception_handler(404)
async def not_found_handler(request: Request, exc):
"""Redirect 404 HTML pages to /workspaces. API routes still get JSON."""
# Preserve JSON 404 for all API-like paths (including /db/xxx/api)
if "/api" in request.url.path:
from fastapi.responses import JSONResponse
return JSONResponse({"detail": "Not found"}, status_code=404)
from fastapi.responses import RedirectResponse
return RedirectResponse("/workspaces", status_code=302)
@app.exception_handler(_StarHTTPException)
async def http_exception_handler(request: Request, exc: _StarHTTPException):
"""Unified handler: RFC7807 for /api/v2, JSON for other /api, redirect for HTML.
Registered on Starlette's HTTPException (the base class) so it catches both
raised exceptions and route-miss 404s.
"""
status = getattr(exc, "status_code", 500)
detail = getattr(exc, "detail", str(exc))
is_api_v2 = request.url.path.startswith("/api/v2")
# Programmatic API prefixes that must always answer JSON errors instead of
# being redirected to the HTML shell (SCIM 2.0 clients, WebAuthn fetch).
JSON_ERROR_PREFIXES = ("/api/v2", "/scim/v2", "/auth/webauthn")
is_json_api = request.url.path.startswith(JSON_ERROR_PREFIXES)
if status == 404:
if is_api_v2:
from app.services.api_v2_helpers import problem_response
return problem_response(request, exc)
if is_json_api and request.url.path.startswith("/scim/v2"):
from fastapi.responses import JSONResponse
return JSONResponse(
{"schemas": ["urn:ietf:params:scim:api:messages:2.0:Error"],
"detail": detail if isinstance(detail, str) else "Not found",
"status": "404"},
status_code=404,
headers={"Content-Type": "application/scim+json"},
)
if "/api" in request.url.path or is_json_api:
from fastapi.responses import JSONResponse
return JSONResponse({"detail": detail if isinstance(detail, str) else "Not found"}, status_code=404)
from fastapi.responses import RedirectResponse
return RedirectResponse("/workspaces", status_code=302)
# Non-404: RFC7807 for /api/v2
if is_api_v2:
from app.services.api_v2_helpers import problem_response
return problem_response(request, exc)
from fastapi.responses import JSONResponse
return JSONResponse({"detail": detail if isinstance(detail, str) else str(detail)}, status_code=status)
+17 -1
View File
@@ -16,7 +16,23 @@ class CSRFMiddleware(BaseHTTPMiddleware):
"""
SAFE_METHODS = {"GET", "HEAD", "OPTIONS"}
EXCLUDED_PATHS = {"/api/webhook", "/api/v1", "/auth/callback", "/auth/register", "/auth/local-login", "/api/user", "/board/api/pages", "/board/api/favorites", "/api/workspace", "/api/local-workspace", "/api/settings", "/db/", "/workspace", "/api/frontend-error", "/api/admin", "/api/gitea", "/api/github", "/api/pages", "/api/recents", "/api/csrf-token", "/api/notifications", "/api/comments", "/api/agent", "/api/automations", "/workspace/automations", "/api/onboarding"}
# NOTE: ``/scim/v2`` is Bearer-token only (no cookie auth), so CSRF does not
# apply — IdP SCIM clients (Okta, Entra) cannot send an X-CSRF-Token.
# A19 — LISTE FINALE : plus aucun préfixe cookie-auth n'est exempté.
# Tous les appels non-GET du front envoient désormais `X-CSRF-Token`
# (46 sites équipés en v7.3.6 : agent_panel, settings, local_workspace,
# library, gitea_workspace, workspace, workspaces, welcome).
# Ne restent que du machine-to-machine / hors session :
# - Bearer : /api/webhook, /api/v1, /api/v2, /scim/v2
# - callbacks : /auth/* (login, register, SSO, 2FA, WebAuthn)
# - publics : /s/ (sites), /f/ (forms)
# - infra/diag : /api/csrf-token (le jeton lui-même), /api/frontend-error
EXCLUDED_PATHS = {
"/api/webhook", "/api/v1", "/api/v2", "/scim/v2",
"/auth/callback", "/auth/register", "/auth/local-login", "/auth/local-verify",
"/auth/2fa", "/auth/saml", "/auth/oidc", "/auth/webauthn",
"/s/", "/f/", "/api/csrf-token", "/api/frontend-error",
}
async def dispatch(self, request: Request, call_next):
# Webhook receiver, OAuth callback, and internal API are exempt
+1164
View File
File diff suppressed because it is too large Load Diff
+121 -24
View File
@@ -14,6 +14,7 @@ from fastapi.responses import StreamingResponse
from app.auth.session import get_current_user
from app.config import settings
from app.db import get_conn
from app.services import skill_gallery
from app.services.agent_engine import AgentEngine, undo_action
from app.services.llm_client import PROVIDER_MODELS, PROVIDERS, LLMClient
from app.services.llm_config import (
@@ -91,13 +92,12 @@ async def agent_scheduler(interval_seconds: int = 60):
logger.exception("Agent scheduler tick failed")
async def _current_user_id(request: Request) -> int | None:
async def _current_user_id(request: Request) -> int:
"""A14 : plus de fallback sur la row `admin` — 401 sans session."""
user = await get_current_user(request)
if user and user.get("id"):
return user["id"]
with get_conn() as conn:
row = conn.execute("SELECT id FROM users WHERE login='admin' ORDER BY id LIMIT 1").fetchone()
return row["id"] if row else None
if not user or not user.get("id"):
raise HTTPException(status_code=401, detail="Authentication required")
return user["id"]
async def _workspace_id(request: Request) -> int | None:
@@ -112,22 +112,19 @@ async def _workspace_id(request: Request) -> int | None:
async def _current_admin(request: Request) -> dict:
"""Require an admin session. Falls back to the single admin row, matching
the agent router's unauthenticated convention (single-user deployments)."""
"""A14 : session obligatoire, puis admin. L'ancien fallback « row admin »
laissait un anonymous diriger `PATCH /api/agent/providers` (et donc le
`ping()` vers un `api_base` de son choix = SSRF)."""
user = await get_current_user(request)
if user:
if not user.get("is_admin"):
from app.db import get_conn as _gc
with _gc() as conn:
row = conn.execute("SELECT is_admin FROM users WHERE id=?", (user.get("id"),)).fetchone()
if not row or not row["is_admin"]:
raise HTTPException(status_code=403, detail="Accès administrateur requis")
return user
with get_conn() as conn:
row = conn.execute("SELECT * FROM users WHERE login='admin' ORDER BY id LIMIT 1").fetchone()
if not row or not row["is_admin"]:
raise HTTPException(status_code=403, detail="Accès administrateur requis")
return dict(row)
if not user:
raise HTTPException(status_code=401, detail="Authentication required")
if not user.get("is_admin"):
from app.db import get_conn as _gc
with _gc() as conn:
row = conn.execute("SELECT is_admin FROM users WHERE id=?", (user.get("id"),)).fetchone()
if not row or not row["is_admin"]:
raise HTTPException(status_code=403, detail="Accès administrateur requis")
return user
def _default_agent(conn, user_id: int) -> dict:
@@ -524,6 +521,81 @@ async def apply_skill(request: Request, skill_id: int):
return {"conversation_id": cur.lastrowid, "skill": skill["name"], "status": "ready"}
# ── Skill marketplace (v6.6.0, Agent phase 5) ──
# Galerie de presets + export/import portable — même implémentation que
# l'API publique (/api/v2/skills/*), via app.services.skill_gallery.
@router.get("/skills/gallery")
async def skills_gallery(request: Request):
presets = skill_gallery.list_gallery()
return {"gallery": presets, "total": len(presets),
"install": "POST /api/agent/skills/gallery/{slug}/install"}
@router.post("/skills/gallery/{slug}/install")
async def install_gallery_skill(request: Request, slug: str):
user_id = await _current_user_id(request)
ws = await _workspace_id(request)
preset = skill_gallery.get_gallery(slug)
if not preset:
raise HTTPException(status_code=404, detail=f"Skill inconnue dans la galerie: {slug}")
body = await request.json() if request.headers.get("content-type") else {}
try:
row, created = skill_gallery.upsert_skill(
skill_gallery.parse_payload(preset),
workspace_id=ws, created_by=user_id,
overwrite=bool(body.get("overwrite", True)),
)
except ValueError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
return {"slug": slug, "id": row.get("id"), "name": row.get("name"),
"status": "installed" if created else "updated", "skill": row}
@router.post("/skills/import")
async def import_skill(request: Request):
"""Importe un skill portable (JSON exporté depuis une autre instance)."""
user_id = await _current_user_id(request)
ws = await _workspace_id(request)
body = await request.json() if request.headers.get("content-type") else {}
payload = body.get("payload") if isinstance(body.get("payload"), dict) else body
try:
fields = skill_gallery.parse_payload(payload)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
try:
row, created = skill_gallery.upsert_skill(
fields, workspace_id=ws, created_by=user_id,
overwrite=bool(body.get("overwrite")),
)
except ValueError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
return {"id": row.get("id"), "name": fields["name"],
"status": "imported" if created else "updated", "skill": row}
@router.get("/skills/{skill_id}/export")
async def export_skill(request: Request, skill_id: int):
"""Document JSON portable — à rejouer sur /api/agent/skills/import."""
with get_conn() as conn:
row = conn.execute("SELECT * FROM agent_skills WHERE id=?", (skill_id,)).fetchone()
if not row:
raise HTTPException(status_code=404, detail="Skill introuvable")
return skill_gallery.export_skill(row)
@router.delete("/skills/{skill_id}")
async def delete_skill(request: Request, skill_id: int):
with get_conn() as conn:
row = conn.execute("SELECT name FROM agent_skills WHERE id=?", (skill_id,)).fetchone()
if not row:
raise HTTPException(status_code=404, detail="Skill introuvable")
conn.execute("DELETE FROM agent_skills WHERE id=?", (skill_id,))
conn.commit()
return {"id": skill_id, "status": "deleted"}
# ── Mentions (commande @ / +) & feedback (boutons 👍 / 👎) ──
@@ -810,11 +882,13 @@ async def save_llm_key(request: Request, llm_provider: str):
if provider not in PROVIDERS:
raise HTTPException(status_code=400, detail=f"Provider inconnu: {provider}")
body = await request.json() if request.headers.get("content-type") else {}
api_base_raw = body.get("api_base")
raw = upsert_user_llm_key(
user_id,
provider,
api_key=(body.get("api_key") or "").strip(),
api_base=(body.get("api_base") or "").strip(),
# None = keep the stored base, "" = reset to the provider default.
api_base=api_base_raw.strip() if isinstance(api_base_raw, str) else None,
default_model=(body.get("default_model") or "").strip(),
models=body.get("models"),
)
@@ -919,6 +993,29 @@ async def fetch_llm_models(request: Request, llm_provider: str):
return {"ok": False, "provider": provider, "error": str(exc)}
def _check_api_base(value: str) -> str:
"""A14 : `api_base` doit être une URL http(s) sans identifiants.
ponytail: les hôtes PRIVÉS restent acceptés — le provider par défaut du
produit est `http://localhost:11434/v1` (Ollama, `llm_client.PROVIDERS`) et
le verrou nommé par l'audit (un anonymous qui oriente le `ping()` du
serveur) est neutralisé par `_current_admin` (401 sans session / 403 non
admin). Pour verrouiller plus tard : allowlist des providers locaux ou un
settings `llm_allow_private=false`.
"""
url = (value or "").strip()
if not url:
return ""
from urllib.parse import urlparse
parsed = urlparse(url)
if parsed.scheme not in ("http", "https") or not parsed.netloc:
raise HTTPException(status_code=400, detail=f"api_base invalide: {url!r}")
if parsed.username or parsed.password:
raise HTTPException(status_code=400, detail="api_base ne doit pas contenir d'identifiants")
return url
@router.patch("/providers")
async def update_provider_config(request: Request):
await _current_admin(request)
@@ -930,7 +1027,7 @@ async def update_provider_config(request: Request):
provider=provider or None,
model=(body.get("model") or "").strip() or None,
api_key=body.get("api_key"),
api_base=(body.get("api_base") or "").strip() or None,
api_base=_check_api_base(body.get("api_base") or "") or None,
clear_keys=(provider == "offline"),
)
llm = LLMClient()
@@ -959,7 +1056,7 @@ async def test_provider_config(request: Request):
llm = LLMClient(
provider=provider,
api_key=body.get("api_key"),
api_base=(body.get("api_base") or "").strip() or None,
api_base=_check_api_base(body.get("api_base") or "") or None,
)
try:
resp = await llm.ping(model=(body.get("model") or "").strip() or None)
+26 -22
View File
@@ -5,7 +5,7 @@ import json
import logging
from datetime import datetime
from fastapi import APIRouter, HTTPException, Query, Request
from fastapi import APIRouter, Depends, HTTPException, Query, Request
from fastapi.responses import HTMLResponse
from app.auth.session import SessionManager
@@ -15,7 +15,27 @@ from app.routers.board import STATUS_COLORS, STATUS_LABELS, _issue_column, _map_
from app.services.gitea_client import gitea
logger = logging.getLogger(__name__)
router = APIRouter(tags=["api"], prefix="/api")
# A17 : ce router legacy ne sert plus un anonymous. Deux exceptions assumées —
# le probe d'infra (Docker HEALTHCHECK / CI) et la remontée d'erreur client.
_API_PUBLIC_PATHS = {"/api/health", "/api/frontend-error"}
async def _require_session_or_bearer(request: Request) -> None:
"""Session de cookie **ou** Bearer d'API valide sur toutes les routes /api."""
if request.url.path in _API_PUBLIC_PATHS:
return
if SessionManager.decode_session(request.cookies.get("flowdeck_session", "")):
return
auth = request.headers.get("Authorization", "")
if auth.startswith("Bearer "):
from app.routers.public_api import verify_token
verify_token(auth)
return
raise HTTPException(401, "Authentication required")
router = APIRouter(tags=["api"], prefix="/api", dependencies=[Depends(_require_session_or_bearer)])
# ── Simple rate limiter (in-memory, per-IP) ──
_rate_limit_store: dict[str, tuple[float, int]] = {}
@@ -47,12 +67,12 @@ async def health(request: Request):
conn.execute("SELECT 1")
db_ok = True
except Exception:
pass
logger.exception("health")
try:
await gitea.get_user_repos(page=1, limit=1)
gitea_ok = True
except Exception:
pass
logger.exception("health")
return {
"status": "ok" if (db_ok and gitea_ok) else "degraded",
@@ -79,22 +99,6 @@ async def stats():
}
@router.get("/projects")
async def projects(search: str = Query(default=""), show_archived: bool = Query(default=False)):
"""List Gitea projects (JSON)."""
try:
repos = await gitea.get_user_repos(page=1, limit=50)
if search:
q = search.lower()
repos = [r for r in repos if q in r.get("full_name", "").lower() or q in (r.get("description") or "").lower()]
if not show_archived:
repos = [r for r in repos if not r.get("archived", False)]
repos.sort(key=lambda r: r.get("updated_at", ""), reverse=True)
except Exception:
repos = []
return {"projects": repos}
@router.post("/move")
async def move_card(
request: Request,
@@ -440,8 +444,8 @@ async def get_issue_detail(owner: str, repo: str, issue_id: int, format: str = Q
"comments": comments,
"checklists": checklists,
}
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
template = env.get_template("card_detail.html")
return HTMLResponse(template.render(**ctx))
File diff suppressed because it is too large Load Diff
+657
View File
@@ -0,0 +1,657 @@
"""FlowDeck — Public API v2 : Agent & Skill marketplace (v6.6.0, phase 5).
Thin Bearer+scopes wrappers over the existing agent logic (AgentEngine,
`agent_skills`, the gallery service) so third-party integrations can drive
FlowDeck Agent without a browser session:
* ``/api/v2/agents`` — agents CRUD, conversations, synchronous runs (JSON,
the SSE stream stays an internal/UI concern), audit journal & rollback.
* ``/api/v2/skills`` — the skill marketplace: CRUD, portable export/import and
the built-in gallery of installable presets.
Rules honoured (see docs/API_GUIDE_V6.md): one code path (the engine and the
gallery service are reused, never re-implemented), JSON only, no secrets or
internal columns, rate limit + audit + idempotency on every mutation.
"""
from __future__ import annotations
import json
import time
from fastapi import APIRouter, Header, HTTPException, Request
from fastapi.responses import JSONResponse
from app.db import get_conn
from app.routers.agent import _default_agent
from app.services import skill_gallery
from app.services.agent_engine import AgentEngine, undo_action
from app.services.api_v2_helpers import (
audit_log,
check_idempotency,
check_v2_rate_limit,
get_bearer_user,
has_scope,
paginate_headers,
parse_pagination,
row_to_dict,
store_idempotency,
)
from app.services.llm_client import LLMClient
from app.services.llm_config import get_user_llm_key
router = APIRouter(prefix="/api/v2", tags=["api-v2-agent"])
# ── Shared guards ──────────────────────────────────────────────────────────
def _guard(request: Request, authorization: str | None, *, write: bool = False) -> dict:
"""Bearer auth + per-token rate limit (+ write scope when required)."""
user = get_bearer_user(request, authorization)
ip = request.client.host if request.client else "unknown"
if not check_v2_rate_limit(user.get("_token_hash"), ip):
raise HTTPException(429, "Rate limit exceeded: 300 req/min per token")
if write and not has_scope(user.get("_token_scopes"), "write"):
raise HTTPException(403, "Insufficient scope. Required: write")
return user
async def _json_body(request: Request) -> dict:
try:
body = await request.json()
except Exception: # noqa: BLE001
return {}
return body if isinstance(body, dict) else {}
def _workspace_of(request: Request, body: dict | None = None) -> int | None:
"""Workspace resolution mirrors the internal agent router: explicit param
wins, then the token's own workspace, else NULL (shared/global scope)."""
body = body or {}
raw = body.get("workspace_id") or request.query_params.get("workspace_id")
if raw is None:
return None
try:
return int(raw)
except (TypeError, ValueError):
return None
def _owned_conversation(conn, conversation_id: int, user_id: int):
"""Conversation visible to this token's user (ownership is enforced here,
unlike the session router where the browser is already authenticated)."""
return conn.execute(
"SELECT * FROM agent_conversations WHERE id=? AND user_id=?",
(conversation_id, user_id),
).fetchone()
def _engine_for(user_id: int, workspace_id: int | None, provider: str | None) -> AgentEngine:
engine = AgentEngine(user_id, workspace_id=workspace_id)
if provider:
user_key = get_user_llm_key(user_id, provider)
if user_key and user_key.get("api_key"):
engine.llm = LLMClient(
provider=provider,
api_key=user_key["api_key"],
api_base=user_key.get("api_base") or None,
)
else:
engine.llm = LLMClient(provider=provider)
return engine
# ── Agents ─────────────────────────────────────────────────────────────────
@router.get("/agents")
async def list_agents_v2(request: Request, authorization: str | None = Header(default=None)):
user = _guard(request, authorization)
limit, offset = parse_pagination(request)
ws = _workspace_of(request)
with get_conn() as conn:
_default_agent(conn, user["id"])
clause = "WHERE workspace_id IS ? OR workspace_id=?"
total = conn.execute(f"SELECT COUNT(*) FROM agents {clause}", (ws, ws)).fetchone()[0]
rows = conn.execute(
f"SELECT * FROM agents {clause} ORDER BY agent_type, name LIMIT ? OFFSET ?",
(ws, ws, limit, offset),
).fetchall()
return JSONResponse(
content={"agents": [row_to_dict(r) for r in rows], "total": total,
"limit": limit, "offset": offset},
headers=paginate_headers(total),
)
@router.post("/agents")
async def create_agent_v2(request: Request, authorization: str | None = Header(default=None)):
user = _guard(request, authorization, write=True)
idem = check_idempotency(request, user["id"])
if idem:
return JSONResponse(content=idem["data"], status_code=idem["status"])
body = await _json_body(request)
name = (body.get("name") or "").strip() or "Custom Agent"
ws = _workspace_of(request, body)
with get_conn() as conn:
try:
cur = conn.execute(
"""INSERT INTO agents (workspace_id, name, icon, agent_type, description,
system_instructions, model, scope_json, trigger_json, approval_mode, created_by)
VALUES (?,?,?,?,?,?,?,?,?,?,?)""",
(ws, name, body.get("icon", "🤖"), body.get("agent_type", "custom"),
body.get("description", ""), body.get("system_instructions", ""),
body.get("model", "gpt-4o"),
json.dumps(body.get("scope", {})), json.dumps(body.get("trigger", {})),
body.get("approval_mode", "auto"), user["id"]),
)
conn.commit()
agent_id = cur.lastrowid
row = conn.execute("SELECT * FROM agents WHERE id=?", (agent_id,)).fetchone()
except Exception as exc: # noqa: BLE001
raise HTTPException(409, f"Cannot create agent: {exc}") from exc
audit_log(user, "agent.create", "agent", agent_id, name, request)
data = {"id": agent_id, "name": name, "status": "created", "agent": row_to_dict(row)}
key = (request.headers.get("Idempotency-Key") or "").strip()
if key:
store_idempotency(key, user["id"], data, 201)
return JSONResponse(content=data, status_code=201)
@router.get("/agents/{agent_id}")
async def get_agent_v2(agent_id: int, request: Request, authorization: str | None = Header(default=None)):
_guard(request, authorization)
with get_conn() as conn:
row = conn.execute("SELECT * FROM agents WHERE id=?", (agent_id,)).fetchone()
if not row:
raise HTTPException(404, "Agent not found")
return row_to_dict(row)
@router.put("/agents/{agent_id}")
async def update_agent_v2(agent_id: int, request: Request, authorization: str | None = Header(default=None)):
user = _guard(request, authorization, write=True)
body = await _json_body(request)
with get_conn() as conn:
existing = conn.execute("SELECT * FROM agents WHERE id=?", (agent_id,)).fetchone()
if not existing:
raise HTTPException(404, "Agent not found")
sets, params = [], []
for col in ("name", "icon", "description", "system_instructions", "model",
"approval_mode", "is_active"):
if col in body:
sets.append(f"{col}=?")
params.append(body[col])
if "scope" in body:
sets.append("scope_json=?")
params.append(json.dumps(body["scope"]))
if "trigger" in body:
sets.append("trigger_json=?")
params.append(json.dumps(body["trigger"]))
if sets:
params.append(agent_id)
conn.execute(f"UPDATE agents SET {', '.join(sets)} WHERE id=?", params)
conn.commit()
audit_log(user, "agent.update", "agent", agent_id, "", request)
return {"id": agent_id, "status": "updated"}
@router.delete("/agents/{agent_id}")
async def delete_agent_v2(agent_id: int, request: Request, authorization: str | None = Header(default=None)):
user = _guard(request, authorization, write=True)
with get_conn() as conn:
if not conn.execute("SELECT id FROM agents WHERE id=?", (agent_id,)).fetchone():
raise HTTPException(404, "Agent not found")
conn.execute("DELETE FROM agents WHERE id=?", (agent_id,))
conn.commit()
audit_log(user, "agent.delete", "agent", agent_id, "", request)
return {"id": agent_id, "status": "deleted"}
# ── Conversations (static paths declared before /agents/{agent_id}) ────────
@router.get("/agents/conversations")
async def list_conversations_v2(request: Request, authorization: str | None = Header(default=None)):
user = _guard(request, authorization)
limit, offset = parse_pagination(request)
with get_conn() as conn:
total = conn.execute(
"SELECT COUNT(*) FROM agent_conversations WHERE user_id=?", (user["id"],)
).fetchone()[0]
rows = conn.execute(
"""SELECT * FROM agent_conversations WHERE user_id=?
ORDER BY updated_at DESC LIMIT ? OFFSET ?""",
(user["id"], limit, offset),
).fetchall()
return JSONResponse(
content={"conversations": [row_to_dict(r) for r in rows], "total": total,
"limit": limit, "offset": offset},
headers=paginate_headers(total),
)
@router.post("/agents/conversations")
async def create_conversation_v2(request: Request, authorization: str | None = Header(default=None)):
user = _guard(request, authorization, write=True)
idem = check_idempotency(request, user["id"])
if idem:
return JSONResponse(content=idem["data"], status_code=idem["status"])
body = await _json_body(request)
ws = _workspace_of(request, body)
agent_id = body.get("agent_id")
with get_conn() as conn:
if agent_id is not None:
agent = conn.execute("SELECT id FROM agents WHERE id=?", (agent_id,)).fetchone()
if not agent:
raise HTTPException(404, "Agent not found")
agent_id = agent["id"]
else:
agent_id = _default_agent(conn, user["id"])["id"]
cur = conn.execute(
"""INSERT INTO agent_conversations (agent_id, user_id, title, context_json, provider, model)
VALUES (?,?,?,?,?,?)""",
(agent_id, user["id"], body.get("title") or "New conversation",
json.dumps({"workspace_id": ws}),
body.get("provider") or "", body.get("model") or ""),
)
conv_id = cur.lastrowid
conn.commit()
row = conn.execute("SELECT * FROM agent_conversations WHERE id=?", (conv_id,)).fetchone()
audit_log(user, "agent.conversation.create", "agent_conversation", conv_id, "", request)
data = {"id": conv_id, "status": "created", "conversation": row_to_dict(row)}
key = (request.headers.get("Idempotency-Key") or "").strip()
if key:
store_idempotency(key, user["id"], data, 201)
return JSONResponse(content=data, status_code=201)
@router.get("/agents/conversations/{conversation_id}")
async def get_conversation_v2(conversation_id: int, request: Request,
authorization: str | None = Header(default=None)):
user = _guard(request, authorization)
with get_conn() as conn:
conv = _owned_conversation(conn, conversation_id, user["id"])
if not conv:
raise HTTPException(404, "Conversation not found")
messages = conn.execute(
"SELECT * FROM agent_messages WHERE conversation_id=? ORDER BY created_at, id",
(conversation_id,),
).fetchall()
return {"conversation": row_to_dict(conv), "messages": [row_to_dict(m) for m in messages]}
@router.delete("/agents/conversations/{conversation_id}")
async def delete_conversation_v2(conversation_id: int, request: Request,
authorization: str | None = Header(default=None)):
user = _guard(request, authorization, write=True)
with get_conn() as conn:
if not _owned_conversation(conn, conversation_id, user["id"]):
raise HTTPException(404, "Conversation not found")
conn.execute("DELETE FROM agent_conversations WHERE id=?", (conversation_id,))
conn.commit()
audit_log(user, "agent.conversation.delete", "agent_conversation", conversation_id, "", request)
return {"id": conversation_id, "status": "deleted"}
@router.get("/agents/conversations/{conversation_id}/actions")
async def list_actions_v2(conversation_id: int, request: Request,
authorization: str | None = Header(default=None)):
user = _guard(request, authorization)
with get_conn() as conn:
if not _owned_conversation(conn, conversation_id, user["id"]):
raise HTTPException(404, "Conversation not found")
rows = conn.execute(
"SELECT * FROM agent_actions WHERE conversation_id=? ORDER BY created_at, id",
(conversation_id,),
).fetchall()
return {"actions": [row_to_dict(r) for r in rows]}
@router.post("/agents/actions/{action_id}/undo")
async def undo_action_v2(action_id: int, request: Request,
authorization: str | None = Header(default=None)):
user = _guard(request, authorization, write=True)
with get_conn() as conn:
row = conn.execute(
"""SELECT a.id FROM agent_actions a
JOIN agent_conversations c ON c.id = a.conversation_id
WHERE a.id=? AND c.user_id=?""",
(action_id, user["id"]),
).fetchone()
if not row:
raise HTTPException(404, "Action not found")
try:
undo_action(action_id)
except ValueError as exc:
raise HTTPException(400, str(exc)) from exc
except Exception as exc: # noqa: BLE001
raise HTTPException(500, f"Rollback failed: {exc}") from exc
audit_log(user, "agent.action.undo", "agent_action", action_id, "", request)
return {"id": action_id, "status": "reverted"}
# ── Runs (JSON — the SSE stream stays internal) ────────────────────────────
def _collect_run_events(events: list[dict]) -> dict:
"""Aggregate an engine event stream into a JSON run result.
Engine events are flat (``{"type": "final", "content": ...}``), the same
shape the SSE panel consumes.
"""
final = None
reasoning = []
actions = []
error = None
for ev in events:
etype = ev.get("type")
if etype == "final":
final = ev.get("content") or final
elif etype == "reasoning":
reasoning.append(ev.get("content") or "")
elif etype == "action":
actions.append({k: v for k, v in ev.items() if k != "type"})
elif etype == "error":
error = ev.get("message") or "run failed"
return {
"status": "failed" if error else "completed",
"final": final,
"error": error,
"reasoning": reasoning,
"actions": actions,
}
@router.post("/agents/conversations/{conversation_id}/run")
async def run_conversation_v2(conversation_id: int, request: Request,
authorization: str | None = Header(default=None)):
"""Synchronous agent run: buffers the engine stream and returns JSON.
Third parties get one HTTP round-trip instead of an SSE subscription; the
same AgentEngine, permissions, journal and webhooks are used as the UI.
"""
user = _guard(request, authorization, write=True)
idem = check_idempotency(request, user["id"])
if idem:
return JSONResponse(content=idem["data"], status_code=idem["status"])
body = await _json_body(request)
objective = (body.get("message") or body.get("objective") or "").strip()
if not objective:
raise HTTPException(400, "message is required")
with get_conn() as conn:
conv = _owned_conversation(conn, conversation_id, user["id"])
if not conv:
raise HTTPException(404, "Conversation not found")
eff_provider = body.get("provider") or conv["provider"] or None
eff_model = body.get("model") or conv["model"] or None
if body.get("provider") is not None or body.get("model") is not None:
conn.execute(
"UPDATE agent_conversations SET provider=?, model=?, updated_at=CURRENT_TIMESTAMP WHERE id=?",
(body.get("provider", conv["provider"] or ""),
body.get("model", conv["model"] or ""), conversation_id),
)
conn.commit()
conv_context = {}
try:
conv_context = json.loads(conv["context_json"] or "{}") or {}
except (TypeError, ValueError):
conv_context = {}
ws = _workspace_of(request, body)
if ws is None:
ws = conv_context.get("workspace_id")
engine = _engine_for(user["id"], ws, eff_provider)
started = time.time()
events = [
ev async for ev in engine.run(
conversation_id, objective,
model=eff_model,
mentions=body.get("mentions"),
files=body.get("files"),
skill_id=body.get("skill_id"),
skill_ids=body.get("skill_ids"),
extra_context=body.get("context"),
)
]
result = _collect_run_events(events)
with get_conn() as conn:
actions = conn.execute(
"SELECT * FROM agent_actions WHERE conversation_id=? ORDER BY created_at, id",
(conversation_id,),
).fetchall()
payload = {
"conversation_id": conversation_id,
"status": result["status"],
"final": result["final"],
"error": result["error"],
"reasoning": result["reasoning"],
"actions": [row_to_dict(a) for a in actions],
"events": events,
"duration_ms": int((time.time() - started) * 1000),
}
audit_log(user, "agent.run", "agent_conversation", conversation_id, objective[:200], request)
status_code = 200 if result["status"] == "completed" else 500
data = payload
key = (request.headers.get("Idempotency-Key") or "").strip()
if key:
store_idempotency(key, user["id"], data, status_code)
return JSONResponse(content=data, status_code=status_code)
@router.post("/agents/{agent_id}/trigger")
async def trigger_agent_v2(agent_id: int, request: Request,
authorization: str | None = Header(default=None)):
"""Fire a custom agent from an external integration (JSON, synchronous)."""
user = _guard(request, authorization, write=True)
body = await _json_body(request)
ws = _workspace_of(request, body)
with get_conn() as conn:
agent = conn.execute("SELECT * FROM agents WHERE id=?", (agent_id,)).fetchone()
if not agent:
raise HTTPException(404, "Agent not found")
cur = conn.execute(
"""INSERT INTO agent_conversations (agent_id, user_id, title, context_json)
VALUES (?,?,?,?)""",
(agent_id, user["id"], f"Run: {agent['name']}", json.dumps({"workspace_id": ws})),
)
conv_id = cur.lastrowid
conn.commit()
objective = (agent["system_instructions"] or "").strip() or f"Exécute l'agent « {agent['name']} »."
if body.get("message"):
objective = f"{objective}\n\n{body['message']}"
engine = _engine_for(user["id"], ws, agent["model"] or None)
started = time.time()
events = [ev async for ev in engine.run(conv_id, objective, model=agent["model"])]
result = _collect_run_events(events)
payload = {
"conversation_id": conv_id,
"agent_id": agent_id,
"status": result["status"],
"final": result["final"],
"error": result["error"],
"reasoning": result["reasoning"],
"actions": result["actions"],
"duration_ms": int((time.time() - started) * 1000),
}
audit_log(user, "agent.trigger", "agent", agent_id, objective[:200], request)
return JSONResponse(content=payload, status_code=200 if result["status"] == "completed" else 500)
# ── Skill marketplace ──────────────────────────────────────────────────────
@router.get("/skills")
async def list_skills_v2(request: Request, authorization: str | None = Header(default=None)):
_guard(request, authorization)
limit, offset = parse_pagination(request)
ws = _workspace_of(request)
with get_conn() as conn:
total = conn.execute(
"SELECT COUNT(*) FROM agent_skills WHERE workspace_id IS ? OR workspace_id=?",
(ws, ws),
).fetchone()[0]
rows = conn.execute(
"""SELECT * FROM agent_skills WHERE workspace_id IS ? OR workspace_id=?
ORDER BY name LIMIT ? OFFSET ?""",
(ws, ws, limit, offset),
).fetchall()
return JSONResponse(
content={"skills": [row_to_dict(r) for r in rows], "total": total,
"limit": limit, "offset": offset},
headers=paginate_headers(total),
)
@router.post("/skills")
async def create_skill_v2(request: Request, authorization: str | None = Header(default=None)):
user = _guard(request, authorization, write=True)
idem = check_idempotency(request, user["id"])
if idem:
return JSONResponse(content=idem["data"], status_code=idem["status"])
body = await _json_body(request)
ws = _workspace_of(request, body)
try:
fields = skill_gallery.parse_payload(
{k: body[k] for k in ("name", "description", "prompt_template", "allowed_tools")
if k in body} | {"format": skill_gallery.EXPORT_FORMAT}
)
except ValueError as exc:
raise HTTPException(400, str(exc)) from exc
try:
row, created = skill_gallery.upsert_skill(
fields, workspace_id=ws, created_by=user["id"],
overwrite=bool(body.get("overwrite")),
)
except ValueError as exc:
raise HTTPException(409, str(exc)) from exc
audit_log(user, "skill.create", "skill", row.get("id"), fields["name"], request)
data = {"id": row.get("id"), "name": fields["name"],
"status": "created" if created else "updated", "skill": row_to_dict(row) if row else {}}
key = (request.headers.get("Idempotency-Key") or "").strip()
if key:
store_idempotency(key, user["id"], data, 201 if created else 200)
return JSONResponse(content=data, status_code=201 if created else 200)
# Gallery & import are static segments: declared before /skills/{skill_id} so
# FastAPI never tries to coerce "gallery" into an int path parameter.
@router.get("/skills/gallery")
async def skills_gallery_v2(request: Request, authorization: str | None = Header(default=None)):
_guard(request, authorization)
presets = skill_gallery.list_gallery()
return {"gallery": presets, "total": len(presets),
"install": "POST /api/v2/skills/gallery/{slug}/install"}
@router.post("/skills/gallery/{slug}/install")
async def install_gallery_skill_v2(slug: str, request: Request,
authorization: str | None = Header(default=None)):
user = _guard(request, authorization, write=True)
preset = skill_gallery.get_gallery(slug)
if not preset:
raise HTTPException(404, f"Unknown gallery skill: {slug}")
body = await _json_body(request)
ws = _workspace_of(request, body)
try:
row, created = skill_gallery.upsert_skill(
skill_gallery.parse_payload(preset),
workspace_id=ws, created_by=user["id"],
overwrite=bool(body.get("overwrite", True)),
)
except ValueError as exc:
raise HTTPException(409, str(exc)) from exc
audit_log(user, "skill.gallery.install", "skill", row.get("id"), slug, request)
data = {"slug": slug, "id": row.get("id"), "name": row.get("name"),
"status": "installed" if created else "updated", "skill": row_to_dict(row)}
return JSONResponse(content=data, status_code=201 if created else 200)
@router.post("/skills/import")
async def import_skill_v2(request: Request, authorization: str | None = Header(default=None)):
"""Import a portable skill document (from another FlowDeck instance)."""
user = _guard(request, authorization, write=True)
idem = check_idempotency(request, user["id"])
if idem:
return JSONResponse(content=idem["data"], status_code=idem["status"])
body = await _json_body(request)
payload = body.get("payload") if isinstance(body.get("payload"), dict) else body
try:
fields = skill_gallery.parse_payload(payload)
except ValueError as exc:
raise HTTPException(400, str(exc)) from exc
ws = _workspace_of(request, body)
try:
row, created = skill_gallery.upsert_skill(
fields, workspace_id=ws, created_by=user["id"],
overwrite=bool(body.get("overwrite")),
)
except ValueError as exc:
raise HTTPException(409, str(exc)) from exc
audit_log(user, "skill.import", "skill", row.get("id"), fields["name"], request)
data = {"id": row.get("id"), "name": fields["name"],
"status": "imported" if created else "updated", "skill": row_to_dict(row)}
key = (request.headers.get("Idempotency-Key") or "").strip()
if key:
store_idempotency(key, user["id"], data, 201 if created else 200)
return JSONResponse(content=data, status_code=201 if created else 200)
@router.get("/skills/{skill_id}")
async def get_skill_v2(skill_id: int, request: Request, authorization: str | None = Header(default=None)):
_guard(request, authorization)
with get_conn() as conn:
row = conn.execute("SELECT * FROM agent_skills WHERE id=?", (skill_id,)).fetchone()
if not row:
raise HTTPException(404, "Skill not found")
return row_to_dict(row)
@router.get("/skills/{skill_id}/export")
async def export_skill_v2(skill_id: int, request: Request,
authorization: str | None = Header(default=None)):
"""Portable JSON document — POST it to /api/v2/skills/import elsewhere."""
_guard(request, authorization)
with get_conn() as conn:
row = conn.execute("SELECT * FROM agent_skills WHERE id=?", (skill_id,)).fetchone()
if not row:
raise HTTPException(404, "Skill not found")
return skill_gallery.export_skill(row)
@router.delete("/skills/{skill_id}")
async def delete_skill_v2(skill_id: int, request: Request,
authorization: str | None = Header(default=None)):
user = _guard(request, authorization, write=True)
with get_conn() as conn:
row = conn.execute("SELECT name FROM agent_skills WHERE id=?", (skill_id,)).fetchone()
if not row:
raise HTTPException(404, "Skill not found")
conn.execute("DELETE FROM agent_skills WHERE id=?", (skill_id,))
conn.commit()
audit_log(user, "skill.delete", "skill", skill_id, row["name"] or "", request)
return {"id": skill_id, "status": "deleted"}
@router.post("/skills/{skill_id}/apply")
async def apply_skill_v2(skill_id: int, request: Request,
authorization: str | None = Header(default=None)):
"""Open a conversation pre-loaded with the skill (ready to run)."""
user = _guard(request, authorization, write=True)
body = await _json_body(request)
ws = _workspace_of(request, body)
with get_conn() as conn:
skill = conn.execute("SELECT * FROM agent_skills WHERE id=?", (skill_id,)).fetchone()
if not skill:
raise HTTPException(404, "Skill not found")
agent_id = _default_agent(conn, user["id"])["id"]
cur = conn.execute(
"""INSERT INTO agent_conversations (agent_id, user_id, title, context_json)
VALUES (?,?,?,?)""",
(agent_id, user["id"], skill["name"],
json.dumps({"workspace_id": ws if ws is not None else skill["workspace_id"],
"skill_id": skill_id})),
)
conv_id = cur.lastrowid
conn.commit()
audit_log(user, "skill.apply", "skill", skill_id, skill["name"], request)
return JSONResponse(content={"conversation_id": conv_id, "skill": skill["name"],
"status": "ready"}, status_code=201)
+124
View File
@@ -0,0 +1,124 @@
"""FlowDeck — unified audit log API (v7.2.0).
Merges ``api_audit_log`` + ``permission_audit_log`` + ``sso_login_history``
with actor/resource/date filters and CSV export (10k rows max, 365-day
retention note). Admin only. See ``docs/V72_Enterprise_SCIM_2FA.md``.
"""
from __future__ import annotations
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import JSONResponse, PlainTextResponse
from app.auth.session import SessionManager
from app.db import get_conn
from app.services.api_v2_helpers import (
has_scope,
parse_pagination,
resolve_bearer_token,
)
router = APIRouter(tags=["audit"])
def _admin_user(request: Request) -> dict:
sess = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if sess:
with get_conn() as conn:
row = conn.execute("SELECT is_admin FROM users WHERE id=?",
(sess.get("id"),)).fetchone()
if row and row["is_admin"]:
return sess
raise HTTPException(403, "Admin required")
auth = request.headers.get("authorization") or request.headers.get("Authorization") or ""
if auth.lower().startswith("bearer "):
user = resolve_bearer_token(auth[7:].strip())
if user and user.get("is_admin") and has_scope(
user.get("_token_scopes") or "read", "admin"):
return user
raise HTTPException(401, "Admin authentication required")
def _query(source: str, actor: str, action: str, limit: int, offset: int):
"""One source query → (rows, columns). All normalized to a common shape."""
with get_conn() as conn:
if source in ("api", "all"):
rows = conn.execute(
"""SELECT created_at AS at, user_id AS actor, action,
resource_type || ':' || resource_id AS resource,
ip_address AS ip, detail, 'api' AS source
FROM api_audit_log
WHERE (?='' OR CAST(user_id AS TEXT)=?)
AND (?='' OR action LIKE ?)
ORDER BY id DESC LIMIT ? OFFSET ?""",
(actor, actor, action, f"%{action}%" if action else "%", limit, offset)
).fetchall()
if source == "api":
return rows
api = [dict(r) for r in rows]
else:
api = []
if source in ("permissions", "all"):
rows = conn.execute(
"""SELECT created_at AS at, performed_by AS actor, action,
resource_type || ':' || resource_id AS resource,
ip_address AS ip,
('target=' || COALESCE(target_user_id, target_group_id, '')
|| ' ' || COALESCE(old_role,'') || '→' || COALESCE(new_role,'')) AS detail,
'permissions' AS source
FROM permission_audit_log
WHERE (?='' OR CAST(performed_by AS TEXT)=?)
AND (?='' OR action LIKE ?)
ORDER BY id DESC LIMIT ? OFFSET ?""",
(actor, actor, action, f"%{action}%" if action else "%", limit, offset)
).fetchall()
if source == "permissions":
return rows
perm = [dict(r) for r in rows]
else:
perm = []
if source in ("sso", "all"):
rows = conn.execute(
"""SELECT created_at AS at, user_id AS actor,
('sso_' || provider_type || '_' ||
CASE success WHEN 1 THEN 'success' ELSE 'failure' END) AS action,
provider_name AS resource, ip_address AS ip,
COALESCE(error_message, sso_identifier, '') AS detail,
'sso' AS source
FROM sso_login_history
WHERE (?='' OR CAST(user_id AS TEXT)=?)
ORDER BY id DESC LIMIT ? OFFSET ?""",
(actor, actor, limit, offset)).fetchall()
if source == "sso":
return rows
sso = [dict(r) for r in rows]
else:
sso = []
merged = sorted(api + perm + sso, key=lambda d: str(d.get("at") or ""),
reverse=True)
return merged[:limit]
@router.get("/api/v2/audit/logs")
async def audit_logs(request: Request):
_admin_user(request)
qp = request.query_params
source = (qp.get("source") or "all").lower()
if source not in ("all", "api", "permissions", "sso"):
raise HTTPException(400, "source must be all|api|permissions|sso")
limit, offset = parse_pagination(request, default_limit=50, max_limit=500)
rows = _query(source, qp.get("actor") or "", qp.get("action") or "", limit, offset)
rows = [dict(r) if not isinstance(r, dict) else r for r in rows]
if qp.get("format") == "csv":
import csv
import io
buf = io.StringIO()
writer = csv.DictWriter(buf, fieldnames=["at", "source", "actor", "action",
"resource", "ip", "detail"])
writer.writeheader()
for r in rows[:10000]:
writer.writerow({k: r.get(k, "") for k in writer.fieldnames})
return PlainTextResponse(buf.getvalue(), media_type="text/csv",
headers={"Content-Disposition":
"attachment; filename=audit.csv"})
return JSONResponse(content={"logs": rows, "source": source,
"limit": limit, "offset": offset})
+176 -6
View File
@@ -63,6 +63,7 @@ body{background:#191919;color:#fff;font-family:-apple-system,BlinkMacSystemFont,
.oauth-section{margin-top:20px;border-top:1px solid rgba(255,255,255,.08);padding-top:20px;}
.oauth-btn{display:flex;align-items:center;justify-content:center;gap:8px;width:100%;padding:10px;border-radius:8px;font-size:14px;cursor:pointer;border:1px solid rgba(255,255,255,.12);background:#2A2A2A;color:#fff;}
.oauth-btn:hover{background:#333;}
.sso-btn{border-color:rgba(35,131,226,.5);}
</style>
</head>
<body>
@@ -88,11 +89,17 @@ body{background:#191919;color:#fff;font-family:-apple-system,BlinkMacSystemFont,
<div class="form-group" id="name-group" style="display:none"><label>Name</label><input type="text" id="name"></div>
<button type="submit" class="btn btn-primary" id="submit-btn">Login</button>
</form>
<div class="oauth-section">
<div class="oauth-section" id="oauth-section">
<p style="font-size:13px;color:rgba(255,255,255,.4);margin-bottom:8px;">Or continue with</p>
<button class="oauth-btn" onclick="window.location='/auth/login?provider=gitea'" style="margin-bottom:8px">🔗 <span id="gitea-btn-label">Login</span> with Gitea</button>
<button class="oauth-btn" onclick="window.location='/auth/login?provider=github'">🐙 <span id="github-btn-label">Login</span> with GitHub</button>
</div>
<!-- SSO / SAML + OIDC (v6.7.0) — buttons injected by loadSsoProviders() -->
<div class="oauth-section" id="sso-section" style="display:none">
<p id="sso-divider" style="font-size:13px;color:rgba(255,255,255,.4);margin-bottom:8px;">Or continue with</p>
<div id="sso-buttons"></div>
<p id="sso-only-note" style="display:none;font-size:12px;color:rgba(255,255,255,.45);margin-top:12px;line-height:1.5;">This instance only accepts your organization account — local login is disabled.</p>
</div>
</div>
<script>
// Show session expired banner if ?expired=1 in URL
@@ -101,6 +108,38 @@ let mode='login';
function switchTab(t){mode=t;document.querySelectorAll('.tab').forEach(el=>el.classList.remove('active'));document.getElementById('tab-'+t).classList.add('active');document.getElementById('name-group').style.display=t==='register'?'block':'none';document.getElementById('submit-btn').textContent=t==='register'?'Register':'Login';document.getElementById('error-msg').style.display='none';document.getElementById('success-msg').style.display='none';var lbl=t==='register'?'Register':'Login';var e1=document.getElementById('gitea-btn-label');var e2=document.getElementById('github-btn-label');if(e1)e1.textContent=lbl;if(e2)e2.textContent=lbl;}
function togglePassword(){var pw=document.getElementById('password');var btn=pw.parentElement.querySelector('.pw-toggle');if(pw.type==='password'){pw.type='text';btn.textContent='🙈';}else{pw.type='password';btn.textContent='👁';}}
async function handleLogin(e){e.preventDefault();const email=document.getElementById('email').value;const password=document.getElementById('password').value;const name=document.getElementById('name').value;const url=mode==='register'?'/auth/register':'/auth/local-login';const body={email,password};if(mode==='register')body.name=name;try{const r=await fetch(url,{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(body)});const d=await r.json();if(!r.ok){document.getElementById('error-msg').textContent=d.error||'Error';document.getElementById('error-msg').style.display='block';return;}if(d.status==='ok'){window.location='/workspaces';}}catch(err){document.getElementById('error-msg').textContent='Network error';document.getElementById('error-msg').style.display='block';}}
// SSO buttons (v6.7.0) — rendered from /api/v2/sso/providers
(async function loadSsoProviders(){
try{
const r = await fetch('/api/v2/sso/providers');
if(!r.ok) return;
const d = await r.json();
const providers = d.providers || [];
if(!providers.length) return;
const wrap = document.getElementById('sso-buttons');
providers.forEach(function(p){
const b = document.createElement('button');
b.className = 'oauth-btn sso-btn';
b.style.marginBottom = '8px';
b.title = 'Sign in with ' + (p.name || 'SSO');
b.onclick = function(){ window.location = p.login_url; };
const icon = document.createElement('span'); icon.textContent = p.icon || '🏢';
const label = document.createElement('span');
label.textContent = (mode === 'register' ? 'Sign up' : 'Login') + ' with ' + (p.name || 'SSO');
b.appendChild(icon); b.appendChild(label);
wrap.appendChild(b);
});
document.getElementById('sso-section').style.display = 'block';
if(d.sso_only){
// Local auth is refused server-side too — don't show a dead form.
const form = document.getElementById('login-form'); if(form) form.style.display = 'none';
const tabs = document.querySelector('.tabs'); if(tabs) tabs.style.display = 'none';
const oauth = document.getElementById('oauth-section'); if(oauth) oauth.style.display = 'none';
const note = document.getElementById('sso-only-note'); if(note) note.style.display = 'block';
const intro = document.querySelector('.login-box p'); if(intro) intro.textContent = 'Sign in with your organization account to continue';
}
}catch(e){}
})();
</script>
</body>
</html>"""
@@ -191,6 +230,16 @@ async def register(request: Request):
from fastapi.responses import JSONResponse
return JSONResponse({"error": "Password must be at least 6 characters"}, status_code=400)
# SSO-only instance (v6.7.0): local registration is refused — accounts are
# auto-provisioned by the IdP instead (admins still come from Settings).
from app.services.sso_provisioning import is_sso_only
if is_sso_only():
from fastapi.responses import JSONResponse
return JSONResponse(
{"error": "Registration is disabled — sign in with your organization SSO"},
status_code=403,
)
with get_conn() as conn:
existing = conn.execute("SELECT id FROM users WHERE login=?", (email,)).fetchone()
if existing:
@@ -260,13 +309,41 @@ async def local_login(request: Request):
conn.commit()
return JSONResponse({"error": "Invalid credentials"}, status_code=401)
# Successful login
# Successful local login — SSO-only instances keep a way in for admins
# only (every other account must use the IdP, design §7.1).
from app.services.sso_provisioning import is_sso_only
if is_sso_only() and not ud.get("is_admin"):
return JSONResponse(
{"error": "Local login is disabled on this instance — sign in with SSO"},
status_code=403,
)
# v7.2.0: verified domain with SSO enforcement (admins keep local access).
if not ud.get("is_admin"):
with get_conn() as conn:
dom = (ud.get("email") or "").split("@")[-1].lower() if "@" in (ud.get("email") or "") else ""
if dom:
enforced = conn.execute(
"SELECT id FROM domain_claims WHERE domain=? AND verified=1"
" AND enforce_sso=1", (dom,)).fetchone()
if enforced:
return JSONResponse(
{"error": "Local login is disabled for your domain — sign in with SSO"},
status_code=403)
with get_conn() as conn:
conn.execute(
"UPDATE users SET login_attempts=0, locked_until=NULL, last_login=? WHERE id=?",
(str(time.time()), ud["id"]),
)
conn.commit()
# v7.2.0: TOTP 2FA — password OK, but hold the session until code check.
from app.services import two_factor as _2fa
if _2fa.is_enabled(ud["id"]):
return JSONResponse({"status": "2fa_required",
"pending": _2fa.mint_pending(ud["id"])})
session = SessionManager.create_session(ud, request)
_log_login(ud["id"], request)
response = JSONResponse({"status": "ok", "user": {"login": ud["login"], "name": ud["full_name"]}})
@@ -373,9 +450,21 @@ async def callback(
@router.get("/logout")
async def logout():
"""Clear session and redirect to login page."""
response = RedirectResponse(url="/auth/login?provider=local", status_code=302)
async def logout(request: Request):
"""Clear session and redirect to login page.
SAML sessions additionally hand over to the IdP's Single Logout when one
is configured (the actual cookie clearing happens on the SLO route).
"""
cookie = request.cookies.get("flowdeck_session", "")
user = SessionManager.decode_session(cookie) if cookie else None
local_target = "/auth/login?provider=local"
if user and user.get("_sso_name_id"):
# SSO session → let /auth/saml/logout revoke locally + notify the IdP.
return RedirectResponse(url=f"/auth/saml/logout?next={local_target}", status_code=302)
response = RedirectResponse(url=local_target, status_code=302)
response.delete_cookie("flowdeck_session")
return response
@@ -389,6 +478,87 @@ async def current_user(request: Request):
return {"authenticated": False}
return {"authenticated": True, "user": user}
# ── v7.2.0 — TOTP 2FA ─────────────────────────────────────────────────────
@router.post("/local-verify")
async def local_verify(request: Request):
"""Exchange a 2FA ``pending`` token + TOTP/backup code for a session."""
from fastapi.responses import JSONResponse
from app.db import get_conn
from app.services import two_factor as _2fa
try:
body = await request.json()
except Exception:
body = {}
user_id = _2fa.redeem_pending(body.get("pending", ""))
if not user_id:
return JSONResponse({"error": "Challenge expired — log in again"}, status_code=401)
if not _2fa.verify_code(user_id, body.get("code", "")):
return JSONResponse({"error": "Invalid code"}, status_code=401)
with get_conn() as conn:
row = conn.execute("SELECT * FROM users WHERE id=?", (user_id,)).fetchone()
if not row or not row["is_active"]:
return JSONResponse({"error": "Account disabled"}, status_code=403)
ud = dict(row)
session = SessionManager.create_session(ud, request)
_log_login(ud["id"], request)
response = JSONResponse({"status": "ok", "user": {"login": ud["login"]}})
response.set_cookie("flowdeck_session", session, httponly=True,
max_age=86400 * 7, samesite="lax", path="/")
return response
def _session_user_or_401(request: Request) -> dict:
from fastapi import HTTPException
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(401, "Authentication required")
return user
@router.get("/2fa/status")
async def twofa_status(request: Request):
from app.services import two_factor as _2fa
user = _session_user_or_401(request)
return {"enabled": _2fa.is_enabled(user["id"]),
"backup_remaining": _2fa.remaining_backup_codes(user["id"])}
@router.post("/2fa/setup")
async def twofa_setup(request: Request):
from app.services import two_factor as _2fa
user = _session_user_or_401(request)
return _2fa.setup_secret(user["id"])
@router.post("/2fa/activate")
async def twofa_activate(request: Request):
from fastapi.responses import JSONResponse
from app.services import two_factor as _2fa
user = _session_user_or_401(request)
try:
body = await request.json()
except Exception:
body = {}
try:
codes = _2fa.activate_secret(user["id"], body.get("secret", ""),
body.get("code", ""))
except ValueError:
return JSONResponse({"error": "Invalid code — secret not activated"},
status_code=400)
return {"status": "enabled", "backup_codes": codes}
@router.post("/2fa/disable")
async def twofa_disable(request: Request):
from app.services import two_factor as _2fa
user = _session_user_or_401(request)
_2fa.disable(user["id"])
return {"status": "disabled"}
# ── Helpers ──
def _log_login(user_id: int, request: Request):
"""Record login in history."""
@@ -403,4 +573,4 @@ def _log_login(user_id: int, request: Request):
)
conn.commit()
except Exception:
pass
logger.exception("_log_login")
+153 -4
View File
@@ -4,14 +4,28 @@ from __future__ import annotations
import json
import logging
from fastapi import APIRouter, HTTPException, Request
from fastapi import APIRouter, Depends, HTTPException, Request
from app.auth.session import SessionManager
from app.db import get_conn
from app.services.automations import get_page_context, run_automation
from app.services.automations import (
get_page_context,
get_steps,
press_button,
run_automation,
validate_step,
)
logger = logging.getLogger(__name__)
router = APIRouter(tags=["automations"])
def _require_session(request: Request) -> None:
"""A13 : toute la route (CRUD, run, press-button) exige une session."""
if not SessionManager.decode_session(request.cookies.get("flowdeck_session", "")):
raise HTTPException(status_code=401, detail="Authentication required")
router = APIRouter(tags=["automations"], dependencies=[Depends(_require_session)])
TRIGGER_TYPES = ("event", "cron", "button")
@@ -69,7 +83,7 @@ async def create_automation(request: Request):
body = await request.json() if request.headers.get("content-type") else {}
_validate_payload(body)
user = _current_user(request)
by = user.get("id") or 1
by = user["id"]
with get_conn() as conn:
cur = conn.execute(
"""INSERT INTO automations
@@ -172,3 +186,138 @@ async def automation_runs_history(request: Request, auto_id: int, limit: int = 5
(auto_id, limit),
).fetchall()
return {"runs": [dict(r) for r in rows]}
# ── v7.0.0 — chained steps (trigger/condition/delay/action) ───────────────
STEP_SECRET_FIELDS = {"webhook_url"}
def _require_session(request: Request) -> dict:
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(status_code=401, detail="Authentication required")
return user
def _get_auto(auto_id: int) -> dict | None:
with get_conn() as conn:
row = conn.execute("SELECT * FROM automations WHERE id=?", (auto_id,)).fetchone()
return dict(row) if row else None
def _auto_404():
# NOTE: return (not raise) — the global 404 handler redirects non-/api
# paths to /workspaces, which TestClient follows into a 200.
from fastapi.responses import JSONResponse
return JSONResponse({"detail": "Automation not found"}, status_code=404)
def _encrypt_step_config(config: dict) -> dict:
"""Encrypt secret fields at rest (empty = keep existing, like sso_config)."""
from app.services.sso_provisioning import encrypt_secret
cfg = dict(config or {})
for field in STEP_SECRET_FIELDS:
if field in cfg and cfg[field]:
val = str(cfg[field])
if not val.startswith("gAAAAA"):
cfg[field] = encrypt_secret(val)
return cfg
@router.get("/workspace/automations/{auto_id}/steps")
async def list_steps(request: Request, auto_id: int):
if _get_auto(auto_id) is None:
return _auto_404()
return {"automation_id": auto_id, "steps": get_steps(auto_id)}
@router.post("/workspace/automations/{auto_id}/steps")
async def create_step(request: Request, auto_id: int):
_require_session(request)
if _get_auto(auto_id) is None:
return _auto_404()
body = await request.json() if request.headers.get("content-type") else {}
kind = body.get("kind", "")
config = body.get("config", {}) or {}
validate_step(kind, config)
with get_conn() as conn:
pos = conn.execute(
"SELECT COALESCE(MAX(position), -1)+1 FROM automation_steps WHERE automation_id=?",
(auto_id,)).fetchone()[0]
cur = conn.execute(
"INSERT INTO automation_steps (automation_id, kind, position, config_json)"
" VALUES (?,?,?,?)",
(auto_id, kind, int(body.get("position", pos)),
json.dumps(_encrypt_step_config(config))))
conn.commit()
step_id = cur.lastrowid
return {"id": step_id, "status": "created"}
@router.put("/workspace/automations/steps/{step_id}")
async def update_step(request: Request, step_id: int):
_require_session(request)
body = await request.json() if request.headers.get("content-type") else {}
with get_conn() as conn:
row = conn.execute("SELECT * FROM automation_steps WHERE id=?", (step_id,)).fetchone()
if not row:
from fastapi.responses import JSONResponse
return JSONResponse({"detail": "Step not found"}, status_code=404)
kind = body.get("kind", row["kind"])
try:
config = body.get("config", json.loads(row["config_json"] or "{}"))
except (TypeError, json.JSONDecodeError):
config = {}
validate_step(kind, config if isinstance(config, dict) else {})
conn.execute(
"UPDATE automation_steps SET kind=?, position=?, config_json=? WHERE id=?",
(kind, int(body.get("position", row["position"])),
json.dumps(_encrypt_step_config(config)), step_id))
conn.commit()
return {"id": step_id, "status": "updated"}
@router.delete("/workspace/automations/steps/{step_id}")
async def delete_step(request: Request, step_id: int):
_require_session(request)
with get_conn() as conn:
conn.execute("DELETE FROM automation_steps WHERE id=?", (step_id,))
conn.commit()
return {"id": step_id, "status": "deleted"}
@router.put("/workspace/automations/{auto_id}/mode")
async def set_trigger_mode(request: Request, auto_id: int):
"""Set multi-trigger mode: any (default) or all (5-minute window)."""
_require_session(request)
if _get_auto(auto_id) is None:
return _auto_404()
body = await request.json() if request.headers.get("content-type") else {}
mode = (body.get("mode") or "any").lower()
if mode not in ("any", "all"):
raise HTTPException(status_code=400, detail="mode must be any or all")
with get_conn() as conn:
conn.execute("UPDATE automations SET trigger_mode=? WHERE id=?", (mode, auto_id))
conn.commit()
return {"id": auto_id, "trigger_mode": mode}
@router.post("/api/automations/press-button")
async def press_button_endpoint(request: Request):
"""Run the automation linked to a native DB button cell (CSRF-exempt)."""
body = await request.json() if request.headers.get("content-type") else {}
try:
collection_id = int(body.get("collection_id", 0))
row_id = int(body.get("row_id", 0))
except (TypeError, ValueError):
raise HTTPException(status_code=400, detail="collection_id + row_id required") from None
prop_ref = body.get("property", body.get("property_id", ""))
if not prop_ref:
raise HTTPException(status_code=400, detail="property required")
user = _current_user(request)
try:
result = await press_button(collection_id, row_id, prop_ref, user.get("id") or 1)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from None
return result
+661 -42
View File
@@ -3,6 +3,7 @@ from __future__ import annotations
import json
import logging
import re
from pathlib import Path
from fastapi import APIRouter, HTTPException, Query, Request
@@ -14,6 +15,7 @@ from app.routers.dashboard import _get_app_version
from app.routers.sidebar_config import get_sidebar_config_sync
from app.services.automations import fire_event
from app.services.gitea_client import gitea
from app.services.permission_manager import PermissionManager
logger = logging.getLogger(__name__)
router = APIRouter(tags=["board"], prefix="/board")
@@ -26,6 +28,312 @@ AI_KEYWORD_COLORS = [
"#E255A1", "#787774", "#6B4E3D", "#6374C4", "#5A9E4B",
]
def _ensure_page_editable(conn, page_id: int, user: dict | None) -> None:
"""v5.12.0: raise 423 when the page is locked and the actor may not edit.
Allowed to edit a locked page: admins and the user who locked it
(locked_by). Unauthenticated callers only pass when the page is unlocked.
"""
row = conn.execute("SELECT is_locked, locked_by FROM pages WHERE id=?", (page_id,)).fetchone()
if not row or not row["is_locked"]:
return
uid = (user or {}).get("id")
is_admin = bool((user or {}).get("is_admin"))
if is_admin or (uid and row["locked_by"] == uid):
return
raise HTTPException(423, "Page is locked — only the owner of the lock or an admin can edit")
def _ensure_block_ids(blocks) -> None:
"""Assign unique ids to blocks missing one, recursively.
Built-in page templates ship without ids (the editor used to assign them
client-side only). Without persisted ids, the realtime layer and the editor
disagree on block identity, which duplicated lines / shuffled blocks when
editing a template-created page. We now materialize ids at creation time.
"""
import uuid
if not isinstance(blocks, list):
return
for b in blocks:
if isinstance(b, dict):
if not b.get("id"):
b["id"] = "b" + uuid.uuid4().hex[:12]
if isinstance(b.get("children"), list):
_ensure_block_ids(b["children"])
@router.get("/api/wiki/pages")
async def wiki_page_search(request: Request, q: str = Query(default="")):
"""v5.11.0: page picker for [[ wiki-links. Fuzzy-ish title search across
every non-deleted page the current user can see (single source: pages)."""
q = (q or "").strip().lower()
with get_conn() as conn:
rows = conn.execute(
"""SELECT id, title, page_icon, workspace FROM pages
WHERE deleted_at IS NULL
ORDER BY updated_at DESC LIMIT 500"""
).fetchall()
results = []
for r in rows:
title = r["title"] or "Untitled"
if q:
# subsequence match ("mtg" → "Meeting notes") or plain substring.
hay = title.lower()
it = iter(hay)
subseq = all(ch in it for ch in q)
if q not in hay and not subseq:
continue
results.append({
"id": r["id"],
"title": title,
"icon": r["page_icon"] or "",
"workspace": r["workspace"] or "",
})
if len(results) >= 20:
break
return {"pages": results}
@router.get("/api/wiki/titles")
async def wiki_titles(request: Request, ids: str = Query(default="")):
"""v5.11.0: resolve page-id lists to current labels (rename propagation)."""
parsed: list[int] = []
for part in (ids or "").split(","):
part = part.strip()
if part.isdigit():
parsed.append(int(part))
parsed = parsed[:200]
out: dict[str, str] = {}
if parsed:
placeholders = ",".join("?" * len(parsed))
with get_conn() as conn:
rows = conn.execute(
f"SELECT id, title, page_icon, deleted_at FROM pages WHERE id IN ({placeholders})",
parsed,
).fetchall()
for r in rows:
if r["deleted_at"]:
out[str(r["id"])] = "Deleted page"
else:
icon = (r["page_icon"] or "")
out[str(r["id"])] = (icon + " " if icon else "") + (r["title"] or "Untitled")
return {"titles": out}
@router.post("/api/pages/{page_id}/lock")
async def set_page_lock(request: Request, page_id: int):
"""v5.12.0: lock/unlock a page (read-only for everyone except the locker,
admins and the page creator)."""
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(401, "Authentication required")
try:
body = await request.json()
except Exception:
body = {}
locked = bool(body.get("locked"))
with get_conn() as conn:
row = conn.execute("SELECT id, is_locked, locked_by, workspace FROM pages WHERE id=?",
(page_id,)).fetchone()
if not row:
raise HTTPException(404, "Page not found")
is_admin = 1 if user.get("is_admin") else 0
if row["is_locked"] and not is_admin and row["locked_by"] != user["id"]:
raise HTTPException(403, "Only the person who locked this page (or an admin) can unlock it")
conn.execute("UPDATE pages SET is_locked=?, locked_by=? WHERE id=?",
(1 if locked else 0, user["id"] if locked else None, page_id))
conn.commit()
await fire_event("page.locked" if locked else "page.unlocked",
{"page_id": page_id, "by": user["id"]})
return {"status": "ok", "is_locked": int(locked)}
@router.post("/api/pages/{page_id}/options")
async def set_page_options(request: Request, page_id: int):
"""v5.12.0: page layout options — full-width and compact typography."""
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(401, "Authentication required")
try:
body = await request.json()
except Exception:
body = {}
updates = {}
for key in ("full_width", "font_small"):
if key in body:
updates[key] = 1 if body[key] else 0
if not updates:
raise HTTPException(400, "nothing to update")
sets = ", ".join(f"{k}=?" for k in updates)
with get_conn() as conn:
row = conn.execute("SELECT id FROM pages WHERE id=?", (page_id,)).fetchone()
if not row:
raise HTTPException(404, "Page not found")
conn.execute(f"UPDATE pages SET {sets}, updated_at=CURRENT_TIMESTAMP WHERE id=?",
(*updates.values(), page_id))
conn.commit()
return {"status": "ok", **{k: bool(v) for k, v in updates.items()}}
@router.get("/api/page-templates")
async def list_page_templates_api(request: Request):
"""v5.12.0: built-in + user global page templates for the picker."""
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
uid = (user or {}).get("id")
from app.services.block_templates import template_list
with get_conn() as conn:
rows = conn.execute(
"""SELECT id, name, icon, description, created_by
FROM page_global_templates
WHERE created_by IS NULL OR created_by=?
ORDER BY created_at""",
(uid,),
).fetchall()
mine = [dict(r) for r in rows]
for t in mine:
t["builtin"] = False
return {"templates": template_list() + mine}
@router.post("/api/page-templates")
async def create_page_template(request: Request):
"""v5.12.0: save the current page (or a raw block list) as a personal
global template: {name, icon?, description?, page_id? | blocks?}."""
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(401, "Authentication required")
try:
body = await request.json()
except Exception:
body = {}
name = (body.get("name") or "").strip()
if not name:
raise HTTPException(400, "name is required")
blocks = body.get("blocks")
if body.get("page_id"):
with get_conn() as conn:
row = conn.execute("SELECT content, content_format FROM pages WHERE id=?",
(int(body["page_id"]),)).fetchone()
if not row:
raise HTTPException(404, "Page not found")
if row["content_format"] == "blocks" and row["content"]:
try:
blocks = json.loads(row["content"])
except (json.JSONDecodeError, TypeError):
raise HTTPException(400, "Page content is not block JSON") from None
if not isinstance(blocks, list) or not blocks:
raise HTTPException(400, "blocks (or page_id) required")
with get_conn() as conn:
cur = conn.execute(
"""INSERT INTO page_global_templates (name, icon, description, blocks_json, created_by)
VALUES (?, ?, ?, ?, ?)""",
(name, body.get("icon") or "📄", body.get("description") or "",
json.dumps(blocks), user["id"]),
)
conn.commit()
tid = cur.lastrowid
return {"status": "ok", "id": tid}
@router.post("/api/page-templates/{template_id}/use")
async def use_page_template(request: Request, template_id: int):
"""v5.12.0: instantiate a page from a template (built-in or user).
Body: {key?} for built-ins OR uses the row id for user templates.
Creates 'blocks'-format page in the caller's workspace and returns its id.
"""
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
try:
body = await request.json()
except Exception:
body = {}
title = (body.get("title") or "").strip()
blocks_json = None
if template_id == 0:
from app.services.block_templates import blocks_json_for
key = body.get("key") or "empty"
blocks_json = blocks_json_for(key)
name = key
if blocks_json is None:
raise HTTPException(404, "Unknown built-in template")
else:
with get_conn() as conn:
uid = (user or {}).get("id")
row = conn.execute(
"SELECT * FROM page_global_templates WHERE id=? AND (created_by IS NULL OR created_by=?)",
(template_id, uid),
).fetchone()
if not row:
raise HTTPException(404, "Template not found")
blocks_json = row["blocks_json"]
name = row["name"]
title = title or row["name"]
# Resolve the target workspace so the new page actually shows up in the
# active local workspace (bugfix: template pages previously got
# workspace_id = NULL and never appeared in the sidebar/tree).
uid = (user or {}).get("id")
ws_id_raw = body.get("workspace_id")
ws_id = None
if ws_id_raw is not None:
try:
ws_id = int(ws_id_raw)
except (TypeError, ValueError):
ws_id = None
ws_key = user.get("login", "Bruno") if user else "Bruno"
if ws_id is not None:
with get_conn() as conn:
ws_row = conn.execute(
"SELECT id, name, owner_id FROM workspaces WHERE id=?", (ws_id,)
).fetchone()
if ws_row and (uid is None or ws_row["owner_id"] == uid):
ws_key = ws_row["name"] or ws_key
else:
ws_id = None
else:
body_ws = (body.get("workspace") or "").strip()
if body_ws:
ws_key = body_ws
# Optional target folder: instantiate the template as a child of it.
parent_id = body.get("parent_id")
try:
parent_id = int(parent_id) if parent_id not in (None, "", 0, "0") else None
except (TypeError, ValueError):
parent_id = None
try:
parsed_blocks = json.loads(blocks_json)
except (json.JSONDecodeError, TypeError):
raise HTTPException(500, "Template content corrupted") from None
_ensure_block_ids(parsed_blocks)
blocks_json = json.dumps(parsed_blocks)
with get_conn() as conn:
if parent_id is not None:
next_order = conn.execute(
"SELECT COALESCE(MAX(sort_order), -1) + 1 FROM pages WHERE parent_id=?",
(parent_id,),
).fetchone()[0]
elif ws_id is not None:
next_order = conn.execute(
"SELECT COALESCE(MAX(sort_order), -1) + 1 FROM pages WHERE workspace_id=? AND parent_id IS NULL",
(ws_id,),
).fetchone()[0]
else:
next_order = conn.execute(
"SELECT COALESCE(MAX(sort_order), -1) + 1 FROM pages WHERE workspace=? AND parent_id IS NULL",
(ws_key,),
).fetchone()[0]
cur = conn.execute(
"""INSERT INTO pages (workspace, workspace_id, title, content, content_format, parent_section, parent_id, sort_order)
VALUES (?,?,?,?,?, 'Private', ?, ?)""",
(ws_key, ws_id, title or name, blocks_json, "blocks", parent_id, next_order),
)
conn.commit()
page_id = cur.lastrowid
await fire_event("page.created", {"page_id": page_id, "title": title or name,
"workspace": ws_key, "from_template": name})
return {"status": "ok", "id": page_id, "title": title or name}
# ── Core helpers ──
def _issue_column(issue: dict, columns: list[str], board_id: int) -> str:
@@ -124,7 +432,7 @@ def _build_page_tree(conn, parent_id: int | None, ws_key: str, depth: int = 0, m
if depth >= max_depth:
return []
rows = conn.execute(
"SELECT id, title, updated_at FROM pages WHERE workspace=? AND parent_id IS ? AND deleted_at IS NULL ORDER BY sort_order ASC, updated_at DESC",
"SELECT id, title, updated_at FROM pages WHERE workspace=? AND parent_id IS ? AND deleted_at IS NULL AND collection_row_id IS NULL ORDER BY sort_order ASC, updated_at DESC",
(ws_key, parent_id),
).fetchall()
items = []
@@ -192,7 +500,7 @@ def _load_workspace_pages(ws_cookie: str) -> list:
rows = conn.execute(
"SELECT id, title, parent_section, content_format, "
"is_shared, share_mode, COALESCE(published,0) AS published "
"FROM pages WHERE workspace_id=? AND parent_id IS NULL AND deleted_at IS NULL ORDER BY created_at DESC",
"FROM pages WHERE workspace_id=? AND parent_id IS NULL AND deleted_at IS NULL AND collection_row_id IS NULL ORDER BY created_at DESC",
(ws_id,),
).fetchall()
items = []
@@ -305,12 +613,21 @@ def _load_shared_sidebar_pages(user_id: int) -> tuple[list, list, list, list]:
f"ORDER BY updated_at DESC LIMIT 20",
scope_params,
).fetchall()
received_rows = conn.execute(
"SELECT DISTINCT p.id, p.title, p.workspace, p.updated_at FROM page_shares s "
"JOIN pages p ON p.id=s.page_id "
"WHERE s.shared_with_user_id=? AND p.deleted_at IS NULL",
(user_id,),
).fetchall()
try:
received_rows = conn.execute(
"SELECT DISTINCT p.id, p.title, p.workspace, p.updated_at FROM page_shares s "
"JOIN pages p ON p.id=s.page_id "
"LEFT JOIN group_members gm ON gm.group_id = s.shared_with_group_id AND gm.user_id=? "
"WHERE (s.shared_with_user_id=? OR gm.user_id=?) AND p.deleted_at IS NULL",
(user_id, user_id, user_id),
).fetchall()
except Exception:
received_rows = conn.execute(
"SELECT DISTINCT p.id, p.title, p.workspace, p.updated_at FROM page_shares s "
"JOIN pages p ON p.id=s.page_id "
"WHERE s.shared_with_user_id=? AND p.deleted_at IS NULL",
(user_id,),
).fetchall()
def _entry(r, icon):
return {
@@ -378,7 +695,7 @@ def _sidebar_data(request: Request, owner: str = "", repo: str = "") -> dict:
local_ws_id = row["id"]
workspace_pages = _load_workspace_pages(str(local_ws_id))
except Exception:
pass
logger.exception("_sidebar_data")
elif ws_cookie and user:
try:
wsi = int(ws_cookie)
@@ -464,7 +781,7 @@ def _sidebar_data(request: Request, owner: str = "", repo: str = "") -> dict:
elif t["provider"] == "github":
github_linked = True
except Exception:
pass
logger.exception("_sidebar_data")
return {"workspace_name": ws_name, "workspace_initial": ws_name[0].upper() if ws_name else "B",
"active_ws_name": active_ws_name,
@@ -487,7 +804,7 @@ def _sidebar_data(request: Request, owner: str = "", repo: str = "") -> dict:
"has_active_workspace": has_active_workspace,
"app_version": _get_app_version(),
"local_workspaces": _local_workspaces_for_user(user),
"sidebar_config": json.dumps(get_sidebar_config_sync(uid))}
"sidebar_config": get_sidebar_config_sync(uid)}
def _extract_ai_keywords(owner: str, repo: str, labels: list[dict], body: str = ""):
@@ -591,8 +908,8 @@ def _apply_sorts(cards: list[dict], sorts: str) -> list[dict]:
@router.get("/library", response_class=HTMLResponse)
async def library_page(request: Request, owner: str = Query(default=""), repo: str = Query(default="")):
"""Library page avec onglets Recents, Favorites, Shared, Private — scopé au workspace."""
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
sidebar = _sidebar_data(request, owner, repo)
# Load all pages for the workspace from DB
ws_key = f"{owner}/{repo}" if owner and repo else ""
@@ -600,13 +917,13 @@ async def library_page(request: Request, owner: str = Query(default=""), repo: s
if ws_key:
rows = conn.execute(
"SELECT id, title, workspace, updated_at FROM pages "
"WHERE workspace=? ORDER BY updated_at DESC",
"WHERE workspace=? AND collection_row_id IS NULL ORDER BY updated_at DESC",
(ws_key,),
).fetchall()
else:
rows = conn.execute(
"SELECT id, title, workspace, updated_at FROM pages "
"ORDER BY updated_at DESC",
"WHERE collection_row_id IS NULL ORDER BY updated_at DESC",
).fetchall()
all_pages = []
for r in rows:
@@ -662,6 +979,10 @@ async def add_favorite(request: Request, page_id: int):
(uid, page_id, pos),
)
conn.commit()
try:
await fire_event("favorite.added", {"page_id": page_id, "user_id": uid})
except Exception:
logger.exception("add_favorite")
return {"status": "added", "page_id": page_id}
@@ -673,6 +994,10 @@ async def remove_favorite(request: Request, page_id: int):
with get_conn() as conn:
conn.execute("DELETE FROM favorites WHERE user_id=? AND page_id=?", (uid, page_id))
conn.commit()
try:
await fire_event("favorite.removed", {"page_id": page_id, "user_id": uid})
except Exception:
logger.exception("remove_favorite")
return {"status": "removed", "page_id": page_id}
# ═══════════ Share API ═══════════
@@ -704,6 +1029,10 @@ async def publish_page(request: Request, page_id: int):
)
conn.commit()
row = conn.execute("SELECT title FROM pages WHERE id=?", (page_id,)).fetchone()
try:
await fire_event("page.published", {"page_id": page_id, "slug": slug})
except Exception:
logger.exception("publish_page")
return {"is_published": True, "publish_slug": slug, "title": row["title"] if row else ""}
@@ -716,6 +1045,10 @@ async def unpublish_page(request: Request, page_id: int):
(page_id,),
)
conn.commit()
try:
await fire_event("page.unpublished", {"page_id": page_id})
except Exception:
logger.exception("unpublish_page")
return {"is_published": False}
@@ -733,6 +1066,10 @@ async def restore_page(request: Request, page_id: int):
with get_conn() as conn:
conn.execute("UPDATE pages SET deleted_at=NULL WHERE id=?", (page_id,))
conn.commit()
try:
await fire_event("page.restored", {"page_id": page_id})
except Exception:
logger.exception("restore_page")
return {"status": "ok", "restored": page_id}
@@ -747,18 +1084,137 @@ async def permanent_delete(request: Request, page_id: int):
@router.get("/trash", response_class=HTMLResponse)
async def trash_page(request: Request):
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
template = env.get_template("trash.html")
return template.render(**_sidebar_data(request))
# ═══════════════ v5.14.0: Synced Blocks ─═══════════════════
# These routes MUST be registered before the catch-all /{owner}/{repo} below.
@router.get("/api/synced-blocks")
async def list_synced_blocks_api(request: Request, workspace: str = Query(default="")):
"""List synced blocks for a workspace."""
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
from app.services.synced_blocks import list_synced_blocks
return {"synced_blocks": list_synced_blocks(workspace or user.get("login", ""))}
@router.post("/api/synced-blocks")
async def create_synced_block_api(request: Request):
"""Create a new synced block."""
try:
body = await request.json()
except Exception:
raise HTTPException(400, "Invalid JSON body") from None
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
from app.services.synced_blocks import create_synced_block
sid = create_synced_block(
workspace=body.get("workspace", ""),
title=body.get("title", "Synced block"),
content=body.get("content", []),
created_by=user.get("id"),
)
return {"status": "ok", "synced_block_id": sid}
@router.put("/api/synced-blocks/{sid}")
async def update_synced_block_api(request: Request, sid: int):
"""Update a synced block's content (propagates to all pages)."""
try:
body = await request.json()
except Exception:
raise HTTPException(400, "Invalid JSON body") from None
from app.services.synced_blocks import (
get_synced_block,
page_ids_for_synced,
sync_synced_blocks_in_page,
update_synced_block,
)
sb = get_synced_block(sid)
if not sb:
raise HTTPException(404, "Synced block not found")
update_synced_block(sid, body.get("title", sb["title"]), body.get("content", []))
# v6.5.0: rewrite every referencing page's stored content first (DB row
# content pages included), THEN push the realtime update so open rooms
# reload the fresh content from the DB.
for pid in page_ids_for_synced(sid):
sync_synced_blocks_in_page(pid)
from app.services.realtime_server import manager
await manager._propagate_synced(sid)
return {"status": "ok"}
@router.delete("/api/synced-blocks/{sid}")
async def delete_synced_block_api(request: Request, sid: int):
"""Delete a synced block."""
from app.services.synced_blocks import (
delete_synced_block,
get_synced_block,
mark_synced_block_deleted,
page_ids_for_synced,
)
sb = get_synced_block(sid)
if not sb:
raise HTTPException(404, "Synced block not found")
# v6.5.0: collect referencing pages BEFORE the FK cascade wipes the
# refs, rewrite their stored content (deleted state), then broadcast.
pids = page_ids_for_synced(sid)
delete_synced_block(sid)
mark_synced_block_deleted(sid, pids)
from app.services.realtime_server import manager
await manager._broadcast_synced_to(pids, sid)
return {"status": "ok"}
@router.get("/api/synced-blocks/{sid}")
async def get_synced_block_api(sid: int):
"""Get a synced block by id."""
from app.services.synced_blocks import get_synced_block
sb = get_synced_block(sid)
if not sb:
raise HTTPException(404, "Synced block not found")
return dict(sb)
@router.post("/api/pages/{page_id}/synced")
async def add_synced_to_page(request: Request, page_id: int):
"""Add a synced block reference to a page."""
try:
body = await request.json()
except Exception:
raise HTTPException(400, "Invalid JSON body") from None
from app.services.synced_blocks import add_page_synced, get_synced_block
sid = body.get("synced_block_id")
sb = get_synced_block(sid)
if not sb:
raise HTTPException(404, "Synced block not found")
add_page_synced(page_id, sid, body.get("block_index", 0))
return {"status": "ok", "synced_block_id": sid}
@router.delete("/api/pages/{page_id}/synced/{sid}")
async def remove_synced_from_page(request: Request, page_id: int, sid: int):
"""Remove a synced block reference from a page (unsync)."""
from app.services.synced_blocks import remove_page_synced
remove_page_synced(page_id, sid)
return {"status": "ok"}
@router.get("/api/pages/{page_id}/synced")
async def get_page_synced_refs(request: Request, page_id: int):
"""Get all synced block references for a page."""
from app.services.synced_blocks import get_page_synced
return {"synced_blocks": get_page_synced(page_id)}
# ═══════════ Board page ═══════════
@router.get("/{owner}/{repo}", response_class=HTMLResponse)
async def board(request: Request, owner: str, repo: str, view: str = Query(default="")):
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
sidebar = _sidebar_data(request, owner, repo)
template = env.get_template("board.html")
return template.render(request=request, owner=owner, repo=repo, groups=[],
@@ -784,8 +1240,8 @@ async def board_view(
logger.error("Board view error: %s", e)
cards = []
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
# Dynamic groups from Gitea labels (fallback to hardcoded)
group_names = ["Design", "Engineering", "No Team"]
@@ -929,6 +1385,9 @@ async def create_page(request: Request, title: str = Query(default=""),
parent_id: int = Query(default=0)):
"""Create a new Markdown page, optionally as a sub-page."""
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
# A7 : la création de page exige une session (route sortue de la liste CSRF).
raise HTTPException(401, "Authentication required")
ws_key = project if project else (user.get("login", "Bruno") if user else "Bruno")
page_title = title.strip() if title else ""
try:
@@ -958,8 +1417,14 @@ async def create_page(request: Request, title: str = Query(default=""),
@router.get("/api/pages/{page_id}")
async def get_page(page_id: int):
async def get_page(request: Request, page_id: int):
"""Get a Markdown page."""
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(401, "Authentication required")
# v6.0.0: granular page permissions — 404 (not 403) hides restricted pages.
if not PermissionManager(user["id"], bool(user.get("is_admin"))).can_view_page(page_id):
raise HTTPException(404, "Page not found")
with get_conn() as conn:
row = conn.execute("SELECT * FROM pages WHERE id=?", (page_id,)).fetchone()
if not row:
@@ -972,16 +1437,31 @@ async def update_page(request: Request, page_id: int, title: str = Query(default
content: str = Query(default=""),
content_format: str = Query(default="")):
"""Update a page's title and/or content. Accepts JSON body for blocks."""
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(403, "Authentication required")
# v6.0.0: granular page permissions — hidden pages 404 (not 403); a visible
# page the caller cannot edit yields 403.
pm = PermissionManager(user["id"], bool(user.get("is_admin")))
if not pm.can_view_page(page_id):
raise HTTPException(404, "Page not found")
if not pm.can_edit_page(page_id):
raise HTTPException(403, "You don't have edit access to this page")
with get_conn() as conn:
_ensure_page_editable(conn, page_id, user)
if title:
conn.execute("UPDATE pages SET title=?, updated_at=CURRENT_TIMESTAMP WHERE id=?", (title, page_id))
# v6.5.0: renaming a database row's content page updates the row.
from app.services.row_pages import sync_page_title_to_row
sync_page_title_to_row(conn, page_id)
if content:
conn.execute("UPDATE pages SET content=?, updated_at=CURRENT_TIMESTAMP WHERE id=?", (content, page_id))
if content_format:
conn.execute("UPDATE pages SET content_format=?, updated_at=CURRENT_TIMESTAMP WHERE id=?", (content_format, page_id))
conn.commit()
await fire_event("page.updated", {"page_id": page_id, "title": title,
"content_format": content_format or "markdown"})
"content_format": content_format or "markdown",
"actor_id": user.get("id")})
return {"status": "ok"}
@@ -991,27 +1471,64 @@ async def save_page_blocks(request: Request, page_id: int):
v5.4.0: a version snapshot is recorded (if the block content actually
changed) so the UI can browse the version history and restore any of them.
v5.14.0: synced block references are tracked in page_synced_blocks.
"""
try:
body = await request.json()
except Exception:
raise HTTPException(400, "Invalid JSON body") from None
blocks_json = json.dumps(body.get("blocks", []))
blocks = body.get("blocks", [])
blocks_json = json.dumps(blocks)
title = body.get("title", "")
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
uid = (user or {}).get("id")
# No session → legacy single-user behaviour; otherwise enforce edit rights.
if uid and not PermissionManager(uid).can_edit_page(page_id):
raise HTTPException(403, "You don't have edit access to this page")
# Extract synced block ids from the blocks
def _extract_synced(blocks: list[dict]) -> set[int]:
ids: set[int] = set()
for b in blocks:
if b.get("type") == "synced" and b.get("synced_id"):
ids.add(b["synced_id"])
if isinstance(b.get("children"), list):
ids |= _extract_synced(b["children"])
return ids
synced_ids = _extract_synced(blocks)
with get_conn() as conn:
_ensure_page_editable(conn, page_id, user)
if title:
conn.execute("UPDATE pages SET title=?, updated_at=CURRENT_TIMESTAMP WHERE id=?", (title, page_id))
# v6.5.0: renaming a database row's content page updates the row.
from app.services.row_pages import sync_page_title_to_row
sync_page_title_to_row(conn, page_id)
conn.execute(
"UPDATE pages SET content=?, content_format='blocks', updated_at=CURRENT_TIMESTAMP WHERE id=?",
(blocks_json, page_id),
)
_record_version(conn, page_id, uid, title or "", blocks_json)
# Update synced block references
existing = {r["synced_block_id"] for r in conn.execute(
"SELECT synced_block_id FROM page_synced_blocks WHERE page_id=?", (page_id,)
).fetchall()}
for sid in synced_ids:
if sid not in existing:
conn.execute(
"INSERT OR IGNORE INTO page_synced_blocks (page_id, synced_block_id, block_index) VALUES (?, ?, 0)",
(page_id, sid),
)
for sid in existing - synced_ids:
conn.execute(
"DELETE FROM page_synced_blocks WHERE page_id=? AND synced_block_id=?",
(page_id, sid),
)
conn.commit()
await fire_event("page.updated", {"page_id": page_id, "title": title or "",
"content_format": "blocks"})
"content_format": "blocks",
"actor_id": uid})
return {"status": "ok", "id": page_id}
@@ -1055,6 +1572,7 @@ async def page_backlinks(request: Request, page_id: int):
reference to ``/pages/{page_id}`` or ``#fdblk-…`` inside ``/pages/{page_id}``.
"""
target = f"/pages/{page_id}" if page_id else None
wiki_target = f"[[fdpage:{page_id}]]" if page_id else None
backlinks = []
with get_conn() as conn:
rows = conn.execute(
@@ -1070,17 +1588,17 @@ async def page_backlinks(request: Request, page_id: int):
blocks = json.loads(r["content"])
for b in blocks if isinstance(blocks, list) else []:
for text in _block_texts(b):
if target and f"/pages/{page_id}" in text:
if target and (target in text or (wiki_target and wiki_target in text)):
hits = True
break
if hits:
break
except (json.JSONDecodeError, TypeError):
hits = target and f"/pages/{page_id}" in (r["content"] or "")
hits = target and (target in (r["content"] or "") or (wiki_target and wiki_target in (r["content"] or "")))
elif fmt == "markdown":
hits = target and f"/pages/{page_id}" in (r["content"] or "")
hits = target and (target in (r["content"] or "") or (wiki_target and wiki_target in (r["content"] or "")))
elif r["content"]:
hits = target and f"/pages/{page_id}" in json.dumps(r["content"])
hits = target and (target in json.dumps(r["content"]) or (wiki_target and wiki_target in json.dumps(r["content"])))
if not hits and target:
hits = f"/pages/{page_id}" in (r["content"] or "")
if hits:
@@ -1264,10 +1782,10 @@ async def remove_page_cover(request: Request, page_id: int):
@router.post("/api/pages/{page_id}/icon")
async def set_page_icon(request: Request, page_id: int):
"""v5.5.0: set a page emoji/icon label."""
"""v5.5.0: set a page emoji/icon label (or a custom-emoji image URL)."""
body = await request.json()
icon = (body.get("icon") or "").strip()
if len(icon) > 16:
if len(icon) > 512:
raise HTTPException(400, "icon too long")
with get_conn() as conn:
conn.execute("UPDATE pages SET page_icon=? WHERE id=?", (icon, page_id))
@@ -1371,7 +1889,11 @@ async def import_file(request: Request):
@router.post("/api/og/metadata")
async def og_metadata(request: Request):
"""v5.5.0: Open Graph metadata for a bookmark card."""
"""v5.5.0: Open Graph metadata for a bookmark card.
v7.3.0: ``gitea:owner/repo`` and ``github:owner/repo`` schemes are
unfurled straight from the forge API (no HTTP fetch of the HTML page).
"""
try:
body = await request.json()
except Exception:
@@ -1379,11 +1901,82 @@ async def og_metadata(request: Request):
url = (body.get("url") or "").strip()
if not url:
raise HTTPException(400, "url required")
m = _REPO_REF_RE.match(url)
if m:
forge, owner, repo = m.group(1).lower(), m.group(2), m.group(3)
data = await _unfurl_repo(forge, owner, repo)
if data:
return {"ok": True, **data}
from app.services.og_fetcher import fetch_og_metadata
data = await fetch_og_metadata(url)
try:
data = await fetch_og_metadata(url)
except ValueError as exc:
# A12 : SSRF — URL vers un hôte privé/loopback (ou redirection vers l'un).
raise HTTPException(400, str(exc)) from None
return {"ok": True, **data}
_REPO_REF_RE = re.compile(r"^(gitea|github):([\w\-\.]+)/([\w\-\.]+)$")
async def _unfurl_repo(forge: str, owner: str, repo: str):
"""Resolve a ``gitea:owner/repo`` / ``github:owner/repo`` ref."""
try:
if forge == "gitea":
from app.services.gitea_client import GiteaClient
info = await GiteaClient().get_repo_info(owner, repo)
site = "Gitea"
else:
from app.config import settings
from app.services.github_adapter import GitHubAdapter
token = getattr(settings, "github_token", None) or ""
if token:
info = await GitHubAdapter(access_token=token).get_repo_info(owner, repo)
else:
import httpx
async with httpx.AsyncClient(timeout=10) as client:
r = await client.get(
f"https://api.github.com/repos/{owner}/{repo}",
headers={"Accept": "application/vnd.github+json"},
)
r.raise_for_status()
info = r.json()
site = "GitHub"
except Exception as exc: # noqa: BLE001 — forge lookup is best-effort
logging.getLogger(__name__).debug("unfurl %s failed: %s", forge, exc)
return None
branch = info.get("default_branch") or "main"
return {
"url": info.get("html_url") or f"https://{forge}.com/{owner}/{repo}",
"title": info.get("full_name") or f"{owner}/{repo}",
"description": (info.get("description") or f"{site} repository "
f"{owner}/{repo} · default branch: {branch}"),
"image": "",
"site_name": site,
"language": info.get("language") or "",
}
@router.post("/api/embed/resolve")
async def resolve_embed(request: Request):
"""v5.5.0: rewrite a pasted URL to its provider embed src.
Powers the universal ``/embed`` block (YouTube, Vimeo, Figma, Maps,
Docs, Loom, CodePen, Miro, Spotify, SoundCloud, Twitch, X/Twitter…).
"""
try:
body = await request.json()
except Exception:
raise HTTPException(400, "Invalid JSON body") from None
url = (body.get("url") or "").strip()
if not url:
raise HTTPException(400, "url required")
from app.config import settings
from app.services.embeds import resolve_embed as _resolve
data = _resolve(url, parent=settings.app_base_url)
return {"ok": True, "url": url, **data}
@router.put("/api/pages/{page_id}/move")
async def move_page(request: Request, page_id: int):
"""Move a page to another workspace or reorder within tree.
@@ -1435,6 +2028,13 @@ async def move_page(request: Request, page_id: int):
@router.delete("/api/pages/{page_id}")
async def delete_page(request: Request, page_id: int):
"""Move a page to trash (soft delete)."""
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
uid = (user or {}).get("id")
if not uid:
raise HTTPException(403, "Authentication required")
# v6.0.0: granular page permissions — need at least edit access to trash.
if not PermissionManager(uid).can_edit_page(page_id):
raise HTTPException(403, "You don't have edit access to this page")
with get_conn() as conn:
row = conn.execute("SELECT id, title FROM pages WHERE id=? AND deleted_at IS NULL", (page_id,)).fetchone()
if not row:
@@ -1451,13 +2051,21 @@ async def view_page(request: Request, page_id: int):
"""Render a page as HTML, or a file viewer for uploaded files.
?embed=1 — minimal mode for side peek (editor only, no header)."""
embed = request.query_params.get("embed") == "1"
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
# v6.0.0: granular page permissions — hide restricted pages (404).
user_hdr = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if user_hdr and user_hdr.get("id") and not PermissionManager(user_hdr["id"]).can_view_page(page_id):
return HTMLResponse("<h2>Page not found</h2>", status_code=404)
with get_conn() as conn:
row = conn.execute("SELECT * FROM pages WHERE id=?", (page_id,)).fetchone()
if not row:
return HTMLResponse("<h2>Page not found</h2>", status_code=404)
page = dict(row)
# v6.5.0: synced blocks resolve server-side at read time (fresh content
# even when the stored cache is stale).
from app.services.synced_blocks import resolve_content_json
page["content"] = resolve_content_json(page.get("content", ""), page.get("content_format"))
ws = page.get("workspace", "")
parts = ws.split("/") if "/" in ws else ["", ""]
@@ -1472,7 +2080,15 @@ async def view_page(request: Request, page_id: int):
).fetchone()
# Build page_data, including file metadata for uploaded files
page_data = {"id": page_id, "title": page.get("title"), "content_format": page.get("content_format", "blocks"), "content": page.get("content", ""), "favorited": fav is not None, "share_mode": page.get("share_mode", "private"), "published": bool(page.get("published", 0)), "cover_url": page.get("cover_url", "") or "", "page_icon": page.get("page_icon", "") or ""}
_locked = bool(page.get("is_locked", 0))
_locked_by = page.get("locked_by") if "locked_by" in page else None
_can_edit = (not _locked) or bool(user and user.get("is_admin")) or (uid and _locked_by == uid)
page_data = {"id": page_id, "title": page.get("title"), "content_format": page.get("content_format", "blocks"), "content": page.get("content", ""), "favorited": fav is not None, "share_mode": page.get("share_mode", "private"), "published": bool(page.get("published", 0)), "cover_url": page.get("cover_url", "") or "", "page_icon": page.get("page_icon", "") or "",
"is_locked": _locked,
"locked_by": _locked_by,
"can_edit": _can_edit,
"full_width": bool(page.get("full_width", 0)) if "full_width" in page else False,
"font_small": bool(page.get("font_small", 0)) if "font_small" in page else False}
# For file pages, extract file metadata and add to page_data
if page.get("content_format") == "file":
@@ -1529,15 +2145,18 @@ async def sync_project(owner: str, repo: str):
if board:
board_id = board["id"]
columns = json.loads(board["columns_json"])
# A23 : un seul executemany pour toutes les cards.
conn.executemany(
"INSERT OR REPLACE INTO cards (board_id, gitea_issue_id, column_name) VALUES (?,?,?)",
[
(board_id, issue["number"], _issue_column(issue, columns, board_id))
for issue in issues_only
],
)
for issue in issues_only:
col = _issue_column(issue, columns, board_id)
conn.execute(
"INSERT OR REPLACE INTO cards (board_id, gitea_issue_id, column_name) VALUES (?,?,?)",
(board_id, issue["number"], col),
)
# Extract AI keywords from each issue
_extract_ai_keywords(owner, repo, issue.get("labels", []), issue.get("body", ""))
conn.commit()
return {"status": "ok", "issues_synced": len(issues_only)}
return {"status": "ok", "issues_synced": len(issues_only)}
except Exception as e:
raise HTTPException(500, str(e)) from e
+26
View File
@@ -13,6 +13,7 @@ from fastapi import APIRouter, HTTPException, Request
from app.auth.session import SessionManager
from app.db import get_conn
from app.services import notifications as notif
from app.services.automations import fire_event as _fire_event
logger = logging.getLogger(__name__)
router = APIRouter(tags=["collaboration"], prefix="/api")
@@ -106,6 +107,12 @@ async def add_comment(request: Request, page_id: int):
comment_id = cur.lastrowid
conn.commit()
# v7.3.0: commenting implies following — the author gets the
# (throttled) page.updated notifications like any other follower.
from app.services import wiki as wiki_svc
wiki_svc.ensure_follow(page_id, uid, conn=conn)
conn.commit()
# Notify users @-mentioned in the comment (skip the author).
url = _page_url(page_id)
title = f"New comment on “{page['title']}”"
@@ -116,6 +123,14 @@ async def add_comment(request: Request, page_id: int):
)
conn.commit()
try:
await _fire_event("comment.added", {"comment_id": comment_id, "page_id": page_id, "user_id": uid})
mentioned_ids = notif.extract_mentions(text)
if mentioned_ids:
await _fire_event("mention.added", {"page_id": page_id, "comment_id": comment_id, "count": len(mentioned_ids)})
except Exception:
logger.exception("add_comment")
return {"id": comment_id, "status": "created"}
@@ -140,6 +155,11 @@ async def notify_page_mentions(request: Request, page_id: int):
"page", page_id, url, conn=conn,
)
conn.commit()
if mentioned:
try:
await _fire_event("mention.added", {"page_id": page_id, "user_ids": mentioned, "count": len(mentioned)})
except Exception:
logger.exception("notify_page_mentions")
return {"mentioned": mentioned}
@@ -161,10 +181,16 @@ async def update_comment(request: Request, comment_id: int):
"UPDATE comments SET body=?, updated_at=CURRENT_TIMESTAMP WHERE id=?",
(body["body"].strip(), comment_id),
)
was_resolved = int(row["resolved"] or 0)
if "resolved" in body and body.get("resolved") is not None:
conn.execute("UPDATE comments SET resolved=? WHERE id=?",
(1 if body["resolved"] else 0, comment_id))
conn.commit()
if body.get("resolved") and not was_resolved:
try:
await _fire_event("comment.resolved", {"comment_id": comment_id, "page_id": row["page_id"]})
except Exception:
logger.exception("update_comment")
return {"id": comment_id, "status": "updated"}
File diff suppressed because it is too large Load Diff
+393 -92
View File
@@ -3,7 +3,7 @@ from __future__ import annotations
import logging
from fastapi import APIRouter, Query, Request
from fastapi import APIRouter, HTTPException, Query, Request
from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse
from app.auth.session import SessionManager
@@ -52,7 +52,7 @@ def _get_user_or_redirect(request: Request):
if count == 0:
return {"id": 1, "login": "admin", "full_name": "Admin", "is_admin": True}
except Exception:
pass
logger.exception("_get_user_or_redirect")
return RedirectResponse("/auth/login?provider=local&expired=1", status_code=302)
return user
@@ -88,7 +88,7 @@ def _sidebar_data(request: Request, repos: list[dict], include_workspace: bool =
avatar_url = row["avatar_url"] or ""
avatar_color = row["avatar_color"] or "#3A3A3A"
except Exception:
pass
logger.exception("_sidebar_data")
recent_pages = []
for repo in repos[:10]:
@@ -179,7 +179,7 @@ def _sidebar_data(request: Request, repos: list[dict], include_workspace: bool =
elif t["provider"] == "github":
github_linked = True
except Exception:
pass
logger.exception("_sidebar_data")
# Get local workspace ID for Gitea workspace mirror
local_ws_id = 0
@@ -193,7 +193,7 @@ def _sidebar_data(request: Request, repos: list[dict], include_workspace: bool =
if row:
local_ws_id = row["id"]
except Exception:
pass
logger.exception("_sidebar_data")
# Private pages for mirror workspace (when Gitea remote active)
private_pages = []
@@ -206,7 +206,7 @@ def _sidebar_data(request: Request, repos: list[dict], include_workspace: bool =
).fetchall()
private_pages = [{"id": r[0], "title": r[1] or "Untitled"} for r in pp_rows]
except Exception:
pass
logger.exception("_sidebar_data")
# Shared / received / published pages for the sidebar (Par moi / Avec moi)
shared_made_pages = []
@@ -252,10 +252,9 @@ def _sidebar_data(request: Request, repos: list[dict], include_workspace: bool =
@router.get("/trash", response_class=HTMLResponse)
async def trash_page(request: Request, owner: str = Query(default=""), repo: str = Query(default="")):
"""Trash page — scoped to workspace if owner/repo provided."""
from jinja2 import Environment, FileSystemLoader
from app.routers.board import _sidebar_data as board_sidebar
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
sidebar = board_sidebar(request, owner, repo)
with get_conn() as conn:
ws_key = f"{owner}/{repo}" if owner and repo else ""
@@ -283,10 +282,9 @@ async def library_page(request: Request, owner: str = Query(default=""), repo: s
Sidebar data is kept intact. Tab content is loaded client-side via /api/library/* endpoints.
"""
from jinja2 import Environment, FileSystemLoader
from app.routers.board import _sidebar_data as board_sidebar
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
sidebar = board_sidebar(request, owner, repo)
# Pass active workspace for breadcrumb nav menu
ws_key_ws = f"{owner}/{repo}" if owner and repo else ""
@@ -312,15 +310,17 @@ async def view_page_root(request: Request, page_id: int):
"""Render a Markdown page at root level with workspace context — or file viewer.
?embed=1 — minimal mode for side peek (editor only, no header)."""
embed = request.query_params.get("embed") == "1"
from jinja2 import Environment, FileSystemLoader
from app.routers.board import _sidebar_data as board_sidebar
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
with get_conn() as conn:
row = conn.execute("SELECT * FROM pages WHERE id=?", (page_id,)).fetchone()
if not row:
return RedirectResponse("/workspaces", status_code=302)
page = dict(row)
# v6.5.0: synced blocks resolve server-side at read time.
from app.services.synced_blocks import resolve_content_json
page["content"] = resolve_content_json(page.get("content", ""), page.get("content_format"))
ws = page.get("workspace", "")
parts = ws.split("/") if "/" in ws else ["", ""]
@@ -338,7 +338,17 @@ async def view_page_root(request: Request, page_id: int):
).fetchone()
# Build page_data, including file metadata for uploaded files
page_data = {"id": page_id, "title": page.get("title"), "content_format": page.get("content_format", "blocks"), "content": page.get("content", ""), "favorited": fav is not None, "share_mode": page.get("share_mode", "private"), "published": bool(page.get("published", 0))}
_locked = bool(page.get("is_locked", 0))
_locked_by = page.get("locked_by") if "locked_by" in page else None
_sess_user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
_uid = _sess_user.get("id") if _sess_user and _sess_user.get("id") else None
_can_edit = (not _locked) or bool(_sess_user and _sess_user.get("is_admin")) or (_locked_by and _uid and _locked_by == _uid)
page_data = {"id": page_id, "title": page.get("title"), "content_format": page.get("content_format", "blocks"), "content": page.get("content", ""), "favorited": fav is not None, "share_mode": page.get("share_mode", "private"), "published": bool(page.get("published", 0)),
"is_locked": _locked,
"locked_by": _locked_by,
"can_edit": _can_edit,
"full_width": bool(page.get("full_width", 0)) if "full_width" in page else False,
"font_small": bool(page.get("font_small", 0)) if "font_small" in page else False}
# For file pages, extract file metadata and add to page_data
if page.get("content_format") == "file":
@@ -385,10 +395,17 @@ async def view_page_root(request: Request, page_id: int):
(page["collection_id"],),
).fetchall()
]
cviews = [
dict(r) for r in conn.execute(
"SELECT * FROM collection_views WHERE collection_id=? ORDER BY position",
(page["collection_id"],),
).fetchall()
]
collection_data = {
"collection": dict(col),
"properties": props,
"pages": cpages,
"views": cviews,
}
page_data["collection_id"] = page["collection_id"]
@@ -417,8 +434,8 @@ async def view_page_root(request: Request, page_id: int):
@router.get("/accounts", response_class=HTMLResponse)
async def accounts_page(request: Request):
"""Account management panel."""
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
sidebar = _sidebar_data(request, [])
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
with get_conn() as conn:
@@ -431,8 +448,8 @@ async def accounts_page(request: Request):
@router.get("/help", response_class=HTMLResponse)
async def help_page(request: Request):
"""Comprehensive help & documentation page."""
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
sidebar = _sidebar_data(request, [])
# Render via a block-based template so content_html lands in {% block content %}
block_tpl = env.from_string(
@@ -469,6 +486,7 @@ async def help_page(request: Request):
.help-badge.local{background:rgba(35,131,226,.15);color:#2C8CEB;}
.help-badge.gitea{background:rgba(0,200,100,.15);color:#00CC66;}
.help-badge.github{background:rgba(130,80,220,.15);color:#A060F0;}
.help-badge.sso{background:rgba(217,115,13,.18);color:#E0952B;}
</style>
<div class="help-page">
<div class="help-hero">
@@ -537,6 +555,16 @@ async def help_page(request: Request):
<li><b>My Tasks</b> — Aggregated tasks from all collections</li>
</ul>
</div>
<div class="help-card">
<h3><span class="icon">📶</span>Offline & PWA</h3>
<p>Install FlowDeck as an app and keep working without a connection.</p>
<ul>
<li><b>Install</b> — browser menu → <i>Install app</i> / <i>Add to Home Screen</i></li>
<li>Edits made offline are queued locally and synced automatically</li>
<li>A <b>⟳</b> marker shows pages with pending changes</li>
</ul>
</div>
</div>
<div class="help-section">
@@ -560,6 +588,51 @@ FlowDeck supports three authentication methods:<br>
<span class="help-badge github">GitHub OAuth</span> Login or link your GitHub account in Settings → Integrations.<br><br>
<b>Tip:</b> You can connect Gitea/GitHub to an existing local account — your identity stays as your local user.
</p>
<p style="color:var(--text-dim);font-size:14px;line-height:1.6;">
<span class="help-badge sso">SSO</span> <b>Enterprise SSO</b> (v6.7.0) — sign in with your organization account.<br>
<i>For administrators:</i> open <b>Settings → Admin → SSO / Enterprise</b> and pick a provider:<br>
• <b>SAML 2.0</b> — paste the IdP <i>Entity ID</i>, <i>SSO URL</i> and signing certificate, then give the IdP this
<code>/auth/saml/metadata</code> link (it contains the SP Entity ID, ACS URL and certificate).<br>
• <b>OpenID Connect</b> — paste the <i>Issuer URL</i>, <i>Client ID</i> and <i>Client Secret</i> (PKCE is used, scopes default to <code>openid profile email</code>).<br>
• <b>Provisioning</b> — accounts are created automatically on first login, groups from the IdP map to workspace roles,
and <i>SSO only</i> disables local login (admins keep their local door). Every attempt is audited in
<b>Settings → Admin → SSO / Enterprise</b> (login history).
</p>
</div>
<div class="help-section">
<h2>📶 Offline mode (PWA)</h2>
<p style="color:var(--text-dim);font-size:14px;line-height:1.6;">
FlowDeck is a Progressive Web App: pages you visited stay available offline and your
edits are saved locally, then synchronised when the connection returns.<br><br>
<b>Install:</b> open your browser menu and choose <i>Install app</i> (Chrome/Edge) or
<i>Add to Home Screen</i> (Safari/iOS). FlowDeck then opens in its own window.<br>
<b>Offline editing:</b> while offline, the editor stores changes in the browser
(IndexedDB) and shows an offline banner with the number of pending changes. A
<b>⟳</b> icon appears next to pages that have unsynced edits.<br>
<b>Reconnection:</b> the queue is replayed automatically (and via Background Sync).
A spinner badge appears while syncing, followed by a confirmation toast.<br>
<b>Conflicts:</b> if a page changed on the server, the latest edit wins and a notice is
shown. If a page was deleted server-side, your offline copy is recreated as an orphan
page. If a page with the same title already exists, the offline copy is renamed
<i>“Title (copie offline)”</i>.
</p>
</div>
<div class="help-section">
<h2>🔌 API publique v2</h2>
<p style="color:var(--text-dim);font-size:14px;line-height:1.6;">
FlowDeck exposes a full REST API under <b>/api/v2</b> for third-party integrations.<br>
<b>Auth:</b> create a token in Settings → API tokens, then send it as
<code>Authorization: Bearer &lt;token&gt;</code>. Tokens carry scopes
<code>read</code>, <code>write</code> or <code>admin</code> (a higher scope implies the lower ones).<br>
<b>Features:</b> CRUD on collections, pages, properties, views, comments, notifications,
favorites, tags, sharing, sprints and templates; pagination (<code>?limit=&amp;offset=</code> +
<code>X-Total-Count</code>), filters (<code>filter[prop]=value</code>), sorting, full-text search
(<code>/api/v2/search</code>), idempotency (<code>Idempotency-Key</code>) and RFC 7807 error bodies.<br>
<b>Reference:</b> interactive OpenAPI docs at <a href="/docs" target="_blank" rel="noopener">/docs</a>
(also <code>/redoc</code>, <code>docs/openapi-v2.json</code>).
</p>
</div>
<div class="help-section">
@@ -580,8 +653,8 @@ FlowDeck supports three authentication methods:<br>
@router.get("/accounts/settings", response_class=HTMLResponse)
async def settings_page(request: Request):
"""User settings page — profile, forges, tokens."""
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
sidebar = _sidebar_data(request, [])
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user:
@@ -613,11 +686,20 @@ def _get_user_id(request: Request) -> int:
return user["id"] if user and user.get("id") else 1
def _require_user_id(request: Request) -> int:
"""A3/A4 — 401 sans session (les routes qui mutent un compte ne tolèrent
plus le fallback « legacy single-user » → id 1 = l'admin seedé)."""
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(401, "Authentication required")
return user["id"]
@router.put("/api/user/profile")
async def update_profile(request: Request):
body = await request.json()
full_name = body.get("full_name", "").strip()
uid = _get_user_id(request)
uid = _require_user_id(request)
with get_conn() as conn:
conn.execute("UPDATE users SET full_name=? WHERE id=?", (full_name, uid))
conn.commit()
@@ -626,13 +708,18 @@ async def update_profile(request: Request):
@router.put("/api/user/password")
async def update_password(request: Request):
from app.password_utils import hash_password
from app.password_utils import hash_password, verify_password
body = await request.json()
password = body.get("password", "").strip()
if len(password) < 6:
return {"error": "Password must be at least 6 characters"}
uid = _get_user_id(request)
uid = _require_user_id(request)
# A3 : le mot de passe actuel est exigé (session volée ≠ droit de changer le mdp).
current = body.get("current_password", "")
with get_conn() as conn:
row = conn.execute("SELECT password_hash FROM users WHERE id=?", (uid,)).fetchone()
if not row or not verify_password(current, row["password_hash"]):
raise HTTPException(403, "Current password is incorrect")
conn.execute("UPDATE users SET password_hash=? WHERE id=?", (hash_password(password), uid))
conn.commit()
return {"status": "ok"}
@@ -641,7 +728,7 @@ async def update_password(request: Request):
@router.post("/api/user/token")
async def generate_token(request: Request):
import secrets
uid = _get_user_id(request)
uid = _require_user_id(request)
token = secrets.token_hex(32)
with get_conn() as conn:
conn.execute(
@@ -654,7 +741,7 @@ async def generate_token(request: Request):
@router.delete("/api/user/forge/{provider}")
async def disconnect_forge(request: Request, provider: str):
uid = _get_user_id(request)
uid = _require_user_id(request)
with get_conn() as conn:
conn.execute(
"DELETE FROM user_oauth_tokens WHERE user_id=? AND provider=?", (uid, provider)
@@ -679,14 +766,14 @@ async def dashboard(
with get_conn() as conn:
count = conn.execute("SELECT COUNT(*) FROM users").fetchone()[0]
if count == 0:
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
template = env.get_template("landing.html")
return template.render()
except Exception:
pass
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
logger.exception("dashboard")
from app.templating import ENV
env = ENV
template = env.get_template("landing.html")
return template.render()
@@ -701,7 +788,7 @@ async def dashboard(
).fetchone()
has_gitea = bool(tok)
except Exception:
pass
logger.exception("dashboard")
if not has_gitea:
# Check if user has any workspace
@@ -714,7 +801,7 @@ async def dashboard(
# v5.2.0: first-launch → onboarding wizard
return RedirectResponse("/welcome", status_code=302)
except Exception:
pass
logger.exception("dashboard")
return RedirectResponse("/local-workspace", status_code=302)
# ── Gitea user → full dashboard ──
@@ -731,8 +818,8 @@ async def dashboard(
logger.error("Dashboard error: %s", e)
repos = []
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
sidebar = _sidebar_data(request, repos)
template = env.get_template("dashboard.html")
return template.render(request=request, repos=repos, search=search,
@@ -744,8 +831,8 @@ async def dashboard(
@router.get("/workspace", response_class=HTMLResponse)
async def workspace_page(request: Request):
"""Unified workspace showing all projects."""
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
sidebar = _sidebar_data(request, [])
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user:
@@ -763,8 +850,8 @@ async def gitea_workspace_page(request: Request):
"""Gitea workspace — browse repo files."""
import json
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
sidebar = _sidebar_data(request, [])
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user:
@@ -823,11 +910,20 @@ async def list_workspace_projects(request: Request):
builtin = []
with get_conn() as conn:
rows = conn.execute(
"SELECT id, title FROM pages WHERE parent_id IS NULL ORDER BY updated_at DESC LIMIT 20"
"SELECT id, title FROM pages WHERE parent_id IS NULL AND collection_row_id IS NULL ORDER BY updated_at DESC LIMIT 20"
).fetchall()
# A23 : une seule agrégation GROUP BY au lieu d'un COUNT par ligne.
counts = {}
if rows:
for c in conn.execute(
"SELECT parent_id, COUNT(*) AS c FROM pages WHERE parent_id IN ({}) GROUP BY parent_id".format(
",".join("?" * len(rows))
),
[r["id"] for r in rows],
).fetchall():
counts[c["parent_id"]] = c["c"]
for r in rows:
count = conn.execute("SELECT COUNT(*) FROM pages WHERE parent_id=?", (r["id"],)).fetchone()[0]
builtin.append({"id": str(r["id"]), "name": r["title"] or "Untitled", "pageCount": count, "forge": "builtin"})
builtin.append({"id": str(r["id"]), "name": r["title"] or "Untitled", "pageCount": counts.get(r["id"], 0), "forge": "builtin"})
gitea_repos = []
# Use per-user token if available, otherwise return empty
@@ -846,7 +942,7 @@ async def list_workspace_projects(request: Request):
"forge": "gitea",
})
except Exception:
pass
logger.exception("list_workspace_projects")
return {"builtin": builtin, "gitea": gitea_repos, "github": []}
@@ -940,8 +1036,8 @@ async def local_workspace_page(request: Request, folder: int = None):
If ?folder=ID is provided, shows that folder's contents with breadcrumb.
"""
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
sidebar = _sidebar_data(request, [])
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user:
@@ -969,6 +1065,7 @@ async def local_workspace_page(request: Request, folder: int = None):
"workspace_id": ws_id or 0,
"nav_workspace_id": ws_id or 0,
"breadcrumb": breadcrumb,
"breadcrumbs": breadcrumb,
}
template = env.get_template("local_workspace.html")
return HTMLResponse(
@@ -1018,7 +1115,109 @@ async def get_page_content(page_id: int):
fmt = row["content_format"]
if fmt == "file":
return JSONResponse({"content": "(uploaded file)", "format": fmt})
return {"content": row["content"] or "", "format": fmt}
# v6.5.0: resolve synced blocks server-side (fresh content on read).
from app.services.synced_blocks import resolve_content_json
return {"content": resolve_content_json(row["content"] or "", fmt), "format": fmt}
def _file_page_disk_path(page: dict):
"""Resolve the on-disk file behind a ``content_format == 'file'`` page.
Returns ``(abs_path: Path, filename: str, mime: str, size: int)`` or None
when the row is not a file page, references a non-textual/missing file, or
the path escapes the data root (path-traversal guard).
"""
if (page.get("content_format") or "") != "file":
return None
import json as _json
try:
meta = _json.loads(page.get("content", "{}"))
except (_json.JSONDecodeError, TypeError):
meta = {}
if not isinstance(meta, dict):
return None
rel = (meta.get("file_path") or "").replace("\\", "/").strip()
if not rel or not rel.startswith("uploads/"):
return None
parts = rel.split("/")
if ".." in parts or "." in parts:
return None
import os as _os
from pathlib import Path
root = Path(_os.environ.get("FLOWDECK_DATA_DIR", "/data")).resolve()
full = (root / rel).resolve()
try:
full.relative_to(root)
except ValueError:
return None
if not full.exists() or not full.is_file():
return None
filename = parts[-1] or page.get("title", "file")
mime = meta.get("mime_type") or "application/octet-stream"
size = meta.get("size") or 0
return (full, filename, mime, size)
def _require_page_view(request: Request, page_id: int) -> None:
"""A16 : lecture d'une pièce jointe = session + `can_view_page` (404 sinon)."""
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(401, "Authentication required")
from app.services.permission_manager import PermissionManager
if not PermissionManager(user["id"], bool(user.get("is_admin"))).can_view_page(page_id):
raise HTTPException(404, "Page not found")
@router.get("/api/pages/{page_id}/download")
async def download_page_file(request: Request, page_id: int):
"""Download the original uploaded file of a ``file`` page (attachment)."""
_require_page_view(request, page_id)
with get_conn() as conn:
row = conn.execute(
"SELECT id, title, content, content_format FROM pages "
"WHERE id=? AND deleted_at IS NULL",
(page_id,),
).fetchone()
if not row:
return JSONResponse({"error": "Not found"}, status_code=404)
resolved = _file_page_disk_path(dict(row))
if not resolved:
return JSONResponse({"error": "No downloadable file"}, status_code=404)
full, filename, mime, _size = resolved
from fastapi.responses import FileResponse
return FileResponse(
str(full), media_type=mime or "application/octet-stream",
filename=filename, content_disposition_type="attachment",
)
@router.get("/api/pages/{page_id}/file-content")
async def page_file_content(request: Request, page_id: int):
"""Return the textual content of a ``file`` page (for copy to clipboard).
Binary files (PDF, images…) answer ``{ok: false}`` — the clipboard copy is
only meaningful for plain-text / code / markdown files.
"""
from app.services.export import _file_text
_require_page_view(request, page_id)
with get_conn() as conn:
row = conn.execute(
"SELECT id, title, content, content_format FROM pages "
"WHERE id=? AND deleted_at IS NULL",
(page_id,),
).fetchone()
if not row:
return JSONResponse({"error": "Not found"}, status_code=404)
page = dict(row)
text = _file_text(page)
if text is None:
return JSONResponse(
{"ok": False, "error": "Not a textual file", "name": page.get("title", "")},
status_code=415,
)
return {"ok": True, "name": page.get("title") or "File", "content": text}
@router.get("/api/local-workspace/breadcrumb")
@@ -1036,7 +1235,7 @@ def _build_tree_children(conn, parent_id: int | None, ws_id: int, uid: int | Non
"SELECT id, title, parent_section, content_format, content, page_icon, "
"(is_shared OR share_mode != 'private' OR COALESCE(published,0)) as is_shared, "
"created_at, updated_at FROM pages "
"WHERE parent_id IS NULL AND workspace_id=? AND deleted_at IS NULL "
"WHERE parent_id IS NULL AND workspace_id=? AND deleted_at IS NULL AND collection_row_id IS NULL "
"ORDER BY created_at DESC",
(ws_id,),
).fetchall()
@@ -1215,7 +1414,7 @@ async def nav_menu(request: Request, workspace_id: int = None, parent_id: int =
else:
rows = conn.execute(
"SELECT id, title, parent_section, content_format FROM pages "
"WHERE parent_id IS NULL AND workspace_id=? AND deleted_at IS NULL "
"WHERE parent_id IS NULL AND workspace_id=? AND deleted_at IS NULL AND collection_row_id IS NULL "
"ORDER BY sort_order ASC, created_at DESC",
(ws_id,),
).fetchall()
@@ -1249,7 +1448,7 @@ async def nav_menu(request: Request, workspace_id: int = None, parent_id: int =
async def create_local_workspace_item(request: Request):
"""Create a new file in the active workspace."""
body = await request.json()
name = body.get("name", "Untitled").strip()
name = (body.get("name") or "").strip() or "Untitled"
item_type = body.get("type", "page")
parent_id = body.get("parent_id")
explicit_ws_id = body.get("workspace_id")
@@ -1308,8 +1507,10 @@ async def restore_local_workspace_item(request: Request, item_id: int):
async def serve_uploaded_file(ws_id: int, filename: str):
"""Serve an uploaded file from disk."""
import mimetypes
import os
from pathlib import Path
base_dir = Path(f"/data/uploads/workspace_{ws_id}").resolve()
root = Path(os.environ.get("FLOWDECK_DATA_DIR", "/data"))
base_dir = (root / f"uploads/workspace_{ws_id}").resolve()
fp = (base_dir / filename).resolve()
try:
fp.relative_to(base_dir)
@@ -1348,7 +1549,8 @@ async def upload_local_workspace_file(request: Request):
import json
from pathlib import Path
ws = _get_active_workspace(request, user_id=_get_user_id(request))
uid = _require_user_id(request) # A22 : pas d'upload anonyme
ws = _get_active_workspace(request, user_id=uid)
ws_id = ws["id"] if ws else None
if not ws_id:
return JSONResponse({"error": "No active workspace"}, status_code=400)
@@ -1365,7 +1567,12 @@ async def upload_local_workspace_file(request: Request):
if not files:
return JSONResponse({"error": "No files provided"}, status_code=400)
upload_dir = Path(f"/data/uploads/workspace_{ws_id}")
import os
from app.middleware.security import validate_upload
data_root = Path(os.environ.get("FLOWDECK_DATA_DIR", "/data"))
upload_dir = data_root / f"uploads/workspace_{ws_id}"
upload_dir.mkdir(parents=True, exist_ok=True)
results = []
@@ -1386,10 +1593,14 @@ async def upload_local_workspace_file(request: Request):
counter += 1
content = await f.read()
err = validate_upload(safe_name, len(content)) # A22 : taille + extension
if err:
results.append({"name": safe_name, "error": err})
continue
file_path.write_bytes(content)
# Determine if this is a folder marker or actual file
rel_path = str(file_path.relative_to("/data"))
rel_path = str(file_path.relative_to(data_root))
size = len(content)
mime = f.content_type or "application/octet-stream"
@@ -1421,7 +1632,8 @@ async def upload_local_workspace_folder(request: Request):
import json
from pathlib import Path
ws = _get_active_workspace(request, user_id=_get_user_id(request))
uid = _require_user_id(request) # A22 : pas d'upload anonyme
ws = _get_active_workspace(request, user_id=uid)
ws_id = ws["id"] if ws else None
if not ws_id:
return JSONResponse({"error": "No active workspace"}, status_code=400)
@@ -1443,7 +1655,12 @@ async def upload_local_workspace_folder(request: Request):
except json.JSONDecodeError:
return JSONResponse({"error": "Invalid structure JSON"}, status_code=400)
upload_dir = Path(f"/data/uploads/workspace_{ws_id}")
import os
from app.middleware.security import validate_upload
data_root = Path(os.environ.get("FLOWDECK_DATA_DIR", "/data"))
upload_dir = data_root / f"uploads/workspace_{ws_id}"
upload_dir.mkdir(parents=True, exist_ok=True)
results = []
@@ -1502,9 +1719,13 @@ async def upload_local_workspace_folder(request: Request):
counter += 1
content = await matched.read()
err = validate_upload(safe_name, len(content)) # A22 : taille + extension
if err:
results.append({"name": safe_name, "error": err})
continue
file_path.write_bytes(content)
rel_path = str(file_path.relative_to("/data"))
rel_path = str(file_path.relative_to(data_root))
cursor = conn.execute(
"""INSERT INTO pages (workspace, workspace_id, title, content, content_format, parent_section, parent_id)
VALUES ('', ?, ?, ?, 'file', 'Private', ?)""",
@@ -1554,8 +1775,8 @@ def _get_active_workspace(request: Request, user_id: int = None) -> dict | None:
@router.get("/workspaces", response_class=HTMLResponse)
async def workspaces_page(request: Request):
"""Workspaces list page."""
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
sidebar = _sidebar_data(request, [], include_workspace=False)
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user:
@@ -1575,7 +1796,7 @@ async def list_workspaces(request: Request):
uid = user["id"] if user and user.get("id") else 1
with get_conn() as conn:
rows = conn.execute(
"SELECT w.*, (SELECT COUNT(*) FROM pages WHERE workspace_id=w.id) as page_count "
"SELECT w.*, (SELECT COUNT(*) FROM pages WHERE workspace_id=w.id AND collection_row_id IS NULL) as page_count "
"FROM workspaces w WHERE w.owner_id=? ORDER BY w.created_at DESC",
(uid,),
).fetchall()
@@ -1657,8 +1878,8 @@ async def select_workspace(request: Request, ws_id: int):
@router.get("/settings", response_class=HTMLResponse)
async def app_settings_page(request: Request):
"""Settings & configuration page."""
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
sidebar = _sidebar_data(request, [])
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user:
@@ -1706,7 +1927,14 @@ async def serve_avatar_file(filename: str):
from pathlib import Path
from fastapi.responses import FileResponse
filepath = Path("/data/avatars") / filename
# A11 : garde path traversal (motif de serve_uploaded_file) — `:path` Starlette
# accepte les `/`, donc `..%2f` ressortirait du dossier avatars.
base_dir = Path("/data/avatars").resolve()
filepath = (base_dir / filename).resolve()
try:
filepath.relative_to(base_dir)
except ValueError:
return JSONResponse({"error": "Path traversal denied"}, status_code=403)
if not filepath.is_file():
return JSONResponse({"error": "Not found"}, status_code=404)
return FileResponse(filepath)
@@ -1794,14 +2022,17 @@ async def list_all_tags_global(request: Request):
"WHERE t.user_id = ? GROUP BY t.id ORDER BY t.name",
(uid,),
).fetchall()
return {"tags": [dict(r) for r in rows]}
return JSONResponse(
{"tags": [dict(r) for r in rows]},
headers={"Cache-Control": "no-store"},
)
# ═══════════ Workspace Tags API ═══════════
@router.get("/api/local-workspace/tags")
async def list_tags(request: Request):
"""List tags used in the active workspace (current user's tags only)."""
"""List ALL user tags with counts scoped to the active workspace."""
ws = _get_active_workspace(request, user_id=_get_user_id(request))
ws_id = ws["id"] if ws else None
uid = _get_user_id(request)
@@ -1809,14 +2040,17 @@ async def list_tags(request: Request):
return {"tags": []}
with get_conn() as conn:
rows = conn.execute(
"SELECT DISTINCT t.id, t.name, t.color, COUNT(pt.page_id) as count "
"FROM tags t "
"JOIN page_tags pt ON pt.tag_id = t.id "
"JOIN pages p ON p.id = pt.page_id AND p.workspace_id = ? "
"WHERE t.user_id = ? GROUP BY t.id ORDER BY t.name",
"SELECT t.id, t.name, t.color, "
"(SELECT COUNT(*) FROM page_tags pt "
" JOIN pages p ON p.id = pt.page_id AND p.workspace_id = ? "
" WHERE pt.tag_id = t.id) as count "
"FROM tags t WHERE t.user_id = ? ORDER BY t.name",
(ws_id, uid),
).fetchall()
return {"tags": [dict(r) for r in rows]}
return JSONResponse(
{"tags": [dict(r) for r in rows]},
headers={"Cache-Control": "no-store"},
)
@router.get("/api/local-workspace/items/{item_id:int}/tags")
@@ -1863,7 +2097,7 @@ async def add_item_tag(request: Request, item_id: int):
)
conn.commit()
except Exception:
pass
logger.exception("add_item_tag")
return {"tag": {"id": tag["id"], "name": tag["name"], "color": tag["color"]}}
@@ -1969,9 +2203,8 @@ async def sidebar_workspace_tree(request: Request):
Called by appState().refreshSidebarTree() after CRUD operations
in the main content area to keep the sidebar in sync.
"""
from jinja2 import Environment, FileSystemLoader
from app.routers.board import _load_workspace_pages
from app.templating import ENV
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user:
@@ -2007,7 +2240,7 @@ async def sidebar_workspace_tree(request: Request):
)
# Render the tree using the extracted macro
env = Environment(loader=FileSystemLoader("app/templates"))
env = ENV
template = env.from_string(
"{% from '_workspace_tree_macro.html' import render_workspace_tree %}"
"{{ render_workspace_tree(pages) }}"
@@ -2028,7 +2261,7 @@ async def sidebar_workspace_tree(request: Request):
@router.get("/p/{slug}", response_class=HTMLResponse)
async def public_published_page(request: Request, slug: str):
"""Serve a published page at /p/<slug> — no auth required."""
from jinja2 import Environment, FileSystemLoader
from app.templating import ENV
with get_conn() as conn:
row = conn.execute(
@@ -2048,7 +2281,7 @@ h1{font-size:3rem;opacity:.3}</style></head><body><h1>404</h1></body></html>""",
)
page = dict(row)
env = Environment(loader=FileSystemLoader("app/templates"))
env = ENV
# Convert blocks to HTML for rendering
content_html = ""
@@ -2056,7 +2289,13 @@ h1{font-size:3rem;opacity:.3}</style></head><body><h1>404</h1></body></html>""",
import json as _json
try:
blocks = _json.loads(page["content"])
content_html = _render_blocks_public(blocks)
from app.services.synced_blocks import resolve_synced_block
blocks = resolve_synced_block(blocks)
from app.db import get_conn as _gc
from app.services.wiki_links import token_labels
with _gc() as conn:
wiki_titles_map = token_labels(conn, page["content"])
content_html = _render_blocks_public(blocks, wiki_titles_map)
except (_json.JSONDecodeError, Exception):
content_html = f"<p>{page.get('content', '')}</p>"
elif page.get("content"):
@@ -2082,12 +2321,23 @@ def _sanitize_id(block_id: str) -> str:
return "".join(ch for ch in str(block_id) if ch.isalnum())
def _render_blocks_public(blocks: list) -> str:
"""Render FlowDeck blocks as plain HTML for public pages."""
def _render_blocks_public(blocks: list, titles: dict | None = None) -> str:
"""Render FlowDeck blocks as plain HTML for public pages.
v5.11.0: ``titles`` (token → label, see app.services.wiki_links) turns
``[[fdpage:ID]]`` / ``[[fddate:...]]`` tokens into chips/links.
"""
html_parts = []
def _wiki(c: str) -> str:
if titles and ("[[fdpage:" in c or "[[fddate:" in c):
from app.services.wiki_links import resolve_tokens_html
return resolve_tokens_html(c, titles)
return c
for b in blocks:
t = b.get("type", "paragraph")
c = b.get("content", "") or ""
c = _wiki(b.get("content", "") or "")
if t == "heading_1":
html_parts.append(f'<h1 id="h-{_sanitize_id(b.get("id",""))}" style="font-size:2.5rem;font-weight:700;margin:32px 0 8px;">{c}</h1>')
elif t == "heading_2":
@@ -2113,7 +2363,7 @@ def _render_blocks_public(blocks: list) -> str:
children_html = ""
if b.get("children"):
children_html = '<div style="margin-left:22px;padding-left:12px;border-left:1px solid rgba(255,255,255,.1);margin-top:4px;">'
children_html += _render_blocks_public(b["children"])
children_html += _render_blocks_public(b["children"], titles)
children_html += "</div>"
html_parts.append(
f'<details style="margin:8px 0;" open><summary style="cursor:pointer;font-weight:500;">{c}</summary>{children_html}</details>'
@@ -2151,7 +2401,7 @@ def _render_blocks_public(blocks: list) -> str:
cols_html += (
'<div style="flex:1;min-width:0;padding:10px 12px;background:rgba(255,255,255,.05);'
'border-radius:8px;box-sizing:border-box;">'
+ _render_blocks_public([child]) + "</div>"
+ _render_blocks_public([child], titles) + "</div>"
)
html_parts.append(
f'<div style="display:flex;gap:12px;margin:8px 0 16px;align-items:stretch;">{cols_html}</div>'
@@ -2230,7 +2480,7 @@ def _render_blocks_public(blocks: list) -> str:
)
elif url:
from app.services.embeds import embed_src
src = embed_src(url) or url
src = b.get("embed_src") or embed_src(url) or url
height = b.get("height") or 520
try:
height = int(height)
@@ -2241,6 +2491,31 @@ def _render_blocks_public(blocks: list) -> str:
f'background:#0a0a0a;"><iframe src="{src}" loading="lazy" frameborder="0" '
f'style="position:absolute;inset:0;width:100%;height:100%;" allowfullscreen allow="autoplay; encrypted-media; picture-in-picture"></iframe></div>'
)
elif t == "synced":
# v6.5.0: render synced block instances (resolved server-side).
if b.get("_synced_deleted"):
html_parts.append(
'<div style="margin:8px 0;padding:8px 12px;border-left:3px solid #e05e5e;'
'background:rgba(224,94,94,.08);border-radius:4px;font-size:13px;opacity:.8;">'
'Deleted synced block</div>'
)
else:
inner = b.get("_synced_content")
if not isinstance(inner, list) or not inner:
try:
import json as _sj
parsed = _sj.loads(b.get("content") or "[]")
inner = parsed if isinstance(parsed, list) else []
except (ValueError, TypeError):
inner = []
inner = [{"type": "paragraph", "content": str(x)} if not isinstance(x, dict) else x
for x in inner]
if inner:
html_parts.append(
'<div style="margin:8px 0;padding-left:12px;'
'border-left:3px solid var(--accent,#4c9aff);">'
+ _render_blocks_public(inner, titles) + '</div>'
)
else:
html_parts.append(f'<p style="margin:4px 0;line-height:1.7;">{c}</p>')
return "\n".join(html_parts)
@@ -2258,9 +2533,11 @@ async def api_page_content(page_id: int):
).fetchone()
if not row:
return JSONResponse({"error": "Not found"}, status_code=404)
# v6.5.0: resolve synced blocks server-side (fresh content on read).
from app.services.synced_blocks import resolve_content_json
return {
"title": row["title"],
"content": row["content"],
"content": resolve_content_json(row["content"], row["content_format"]),
"format": row["content_format"] or "blocks",
}
@@ -2277,6 +2554,9 @@ async def api_rename_page(page_id: int, request: Request):
"UPDATE pages SET title=?, updated_at=CURRENT_TIMESTAMP WHERE id=? AND deleted_at IS NULL",
(title, page_id),
)
# v6.5.0: renaming a database row's content page updates the row.
from app.services.row_pages import sync_page_title_to_row
sync_page_title_to_row(conn, page_id)
conn.commit()
return {"status": "ok", "title": title}
@@ -2383,10 +2663,18 @@ async def api_collection_table_data(collection_id: int):
).fetchall()
]
views = [
dict(r) for r in conn.execute(
"SELECT * FROM collection_views WHERE collection_id=? ORDER BY position",
(collection_id,),
).fetchall()
]
return {
"collection": dict(coll),
"properties": properties,
"pages": pages,
"views": views,
}
@@ -2415,20 +2703,31 @@ async def api_create_collection_page(collection_id: int, request: Request):
).fetchone()[0]
# Load default property values from collection properties
props = conn.execute(
"SELECT id, name, prop_type FROM collection_properties WHERE collection_id=? ORDER BY position",
(collection_id,),
).fetchall()
props = [
dict(r) for r in conn.execute(
"SELECT id, name, prop_type FROM collection_properties WHERE collection_id=? ORDER BY position",
(collection_id,),
).fetchall()
]
default_values = {}
for p in props:
if p["prop_type"] == "title":
default_values[str(p["id"])] = title
# Caller-provided values (e.g. board "add card in column X") win.
incoming = body.get("properties") or {}
if isinstance(incoming, dict):
default_values.update(incoming)
from app.services.property_types import apply_auto_properties
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", "")) or {"login": "admin", "id": 1}
apply_auto_properties(props, default_values, user, is_create=True)
cur = conn.execute(
"""INSERT INTO collection_pages
(collection_id, title, icon, position, property_values_json)
VALUES (?, ?, ?, ?, ?)""",
(collection_id, title, icon, max_pos, _json.dumps(default_values)),
(collection_id, title, icon, cover_url, position, property_values_json)
VALUES (?, ?, ?, ?, ?, ?)""",
(collection_id, title, icon, body.get("cover_url", ""), max_pos,
_json.dumps(default_values)),
)
page_id = cur.lastrowid
conn.commit()
@@ -2436,6 +2735,8 @@ async def api_create_collection_page(collection_id: int, request: Request):
return {
"id": page_id,
"title": title,
"icon": icon,
"position": max_pos,
"property_values_json": default_values,
"status": "created",
}
+100
View File
@@ -0,0 +1,100 @@
"""FlowDeck — custom workspace emojis (v5.6.0).
Uploaded emoji images stored per-workspace and usable as page icons. Kept on a
prefix-less router so the paths stay ``/api/custom-emojis`` (the board router's
``/{owner}/{repo}`` HTML catch-all would otherwise shadow them).
"""
from __future__ import annotations
import datetime
import re
from pathlib import Path
from fastapi import APIRouter, HTTPException, Request
from app.db import get_conn
router = APIRouter(tags=["emojis"])
_IMAGE_EXTS = {"png", "jpg", "jpeg", "gif", "webp", "svg", "bmp", "ico", "avif"}
def _upload_root() -> Path:
import os
return Path(os.environ.get("FLOWDECK_DATA_DIR", "/data"))
def _active_ws(request: Request) -> int:
"""Workspace id from the active-workspace cookie, fallback 1."""
try:
ws_id = int(request.cookies.get("flowdeck_workspace", "") or 0)
if ws_id > 0:
return ws_id
except (ValueError, TypeError):
pass
return 1
@router.get("/api/custom-emojis")
async def list_custom_emojis(request: Request):
"""List the current workspace's custom emojis."""
ws_id = _active_ws(request)
with get_conn() as conn:
rows = conn.execute(
"SELECT id, name, url FROM custom_emojis WHERE workspace_id=? ORDER BY created_at DESC",
(ws_id,),
).fetchall()
return {"status": "ok", "emojis": [dict(r) for r in rows]}
@router.post("/api/custom-emojis")
async def create_custom_emoji(request: Request):
"""Upload a custom emoji image (multipart: ``name`` + ``file``)."""
form = await request.form()
name = (form.get("name") or "").strip()[:40] or "emoji"
upload = form.get("file")
if upload is None or not hasattr(upload, "filename"):
raise HTTPException(400, "file field required")
original = (upload.filename or "emoji.png").replace("\\", "/").rsplit("/", 1)[-1]
safe = re.sub(r"[^A-Za-z0-9._-]", "_", original)[:80]
ext = safe.rsplit(".", 1)[-1].lower() if "." in safe else "png"
if ext not in _IMAGE_EXTS:
raise HTTPException(400, "Unsupported image format")
ws_id = _active_ws(request)
stamp = datetime.datetime.utcnow().strftime("%Y%m%d%H%M%S%f")
folder = _upload_root() / f"uploads/workspace_{ws_id}"
folder.mkdir(parents=True, exist_ok=True)
final = f"emoji_{stamp}_{safe}"
(folder / final).write_bytes(await upload.read())
url = f"/api/files/{ws_id}/{final}"
with get_conn() as conn:
cur = conn.execute(
"INSERT INTO custom_emojis (workspace_id, name, url) VALUES (?, ?, ?)",
(ws_id, name, url),
)
conn.commit()
emoji_id = cur.lastrowid
return {"status": "ok", "emoji": {"id": emoji_id, "name": name, "url": url}}
@router.delete("/api/custom-emojis/{emoji_id}")
async def delete_custom_emoji(request: Request, emoji_id: int):
"""Delete a custom emoji (and its stored file)."""
ws_id = _active_ws(request)
with get_conn() as conn:
row = conn.execute(
"SELECT url FROM custom_emojis WHERE id=? AND workspace_id=?",
(emoji_id, ws_id),
).fetchone()
if not row:
raise HTTPException(404, "emoji not found")
conn.execute("DELETE FROM custom_emojis WHERE id=?", (emoji_id,))
conn.commit()
try:
fname = (row["url"] or "").rsplit("/", 1)[-1]
if fname:
(_upload_root() / f"uploads/workspace_{ws_id}" / fname).unlink(missing_ok=True)
except OSError:
pass
return {"status": "ok", "id": emoji_id}
+15 -5
View File
@@ -23,7 +23,15 @@ logger = logging.getLogger(__name__)
router = APIRouter(tags=["export"], prefix="/api/export")
def _load_page_or_404(page_id: int) -> dict:
def _load_page_or_404(request: Request, page_id: int) -> dict:
"""A16 : session obligatoire + `PermissionManager.can_view_page` — l'export ne
doit pas délivrer le contenu d'une page énumérable par id."""
from app.auth.session import SessionManager
from app.services.permission_manager import PermissionManager
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(status_code=401, detail="Authentication required")
with get_conn() as conn:
row = conn.execute(
"SELECT * FROM pages WHERE id=? AND deleted_at IS NULL",
@@ -31,6 +39,8 @@ def _load_page_or_404(page_id: int) -> dict:
).fetchone()
if not row:
raise HTTPException(status_code=404, detail="Page not found")
if not PermissionManager(user["id"], bool(user.get("is_admin"))).can_view_page(page_id):
raise HTTPException(status_code=404, detail="Page not found")
return dict(row)
@@ -52,7 +62,7 @@ def _safe_filename(page: dict, ext: str) -> str:
@router.get("/markdown/{page_id}")
async def export_markdown(page_id: int, request: Request):
page = _load_page_or_404(page_id)
page = _load_page_or_404(request, page_id)
md = page_to_markdown(page)
filename = _safe_filename(page, "md")
headers = _download_header(filename, "text/markdown")
@@ -61,7 +71,7 @@ async def export_markdown(page_id: int, request: Request):
@router.get("/html/{page_id}")
async def export_html(page_id: int, request: Request):
page = _load_page_or_404(page_id)
page = _load_page_or_404(request, page_id)
html = page_to_standalone_html(page)
filename = _safe_filename(page, "html")
headers = _download_header(filename, "text/html")
@@ -70,7 +80,7 @@ async def export_html(page_id: int, request: Request):
@router.get("/pdf/{page_id}")
async def export_pdf(page_id: int, request: Request):
page = _load_page_or_404(page_id)
page = _load_page_or_404(request, page_id)
try:
pdf_bytes = page_to_pdf_bytes(page)
except ImportError:
@@ -85,7 +95,7 @@ async def export_pdf(page_id: int, request: Request):
@router.get("/site/{page_id}")
async def export_site(page_id: int, request: Request):
page = _load_page_or_404(page_id)
page = _load_page_or_404(request, page_id)
site_bytes = build_static_site_bytes(page)
title = _safe_filename(page, "site").replace(".site", "") or "flowdeck-site"
filename = f"{title}_site.zip"
+98
View File
@@ -0,0 +1,98 @@
"""FlowDeck — agent governance API (v7.2.0): policies + approval queue."""
from __future__ import annotations
import json
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import JSONResponse
from app.auth.session import SessionManager
from app.db import get_conn
from app.services import agent_policies as policies
from app.services.api_v2_helpers import audit_log
router = APIRouter(tags=["governance"])
def _owner_or_admin(request: Request) -> dict:
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(401, "Authentication required")
with get_conn() as conn:
row = conn.execute("SELECT is_admin FROM users WHERE id=?", (user["id"],)).fetchone()
is_admin = bool(row and row["is_admin"])
if not is_admin and request.query_params.get("workspace_id"):
member = conn.execute(
"SELECT role FROM workspace_members WHERE workspace_id=? AND user_id=?",
(request.query_params.get("workspace_id"), user["id"])).fetchone()
owner = conn.execute("SELECT id FROM workspaces WHERE id=? AND owner_id=?",
(request.query_params.get("workspace_id"),
user["id"])).fetchone()
if not member and not owner:
raise HTTPException(403, "Workspace access required")
if member and member["role"] not in ("admin", "editor", "owner"):
raise HTTPException(403, "Editor role required")
user["is_admin"] = is_admin
return user
@router.get("/api/v2/agent-policies")
async def list_policies(request: Request):
_owner_or_admin(request)
with get_conn() as conn:
rows = conn.execute("SELECT * FROM agent_policies ORDER BY workspace_id").fetchall()
return {"policies": [dict(r) for r in rows]}
@router.post("/api/v2/agent-policies")
async def upsert_policy(request: Request):
user = _owner_or_admin(request)
try:
body = await request.json()
except Exception:
body = {}
wid = body.get("workspace_id")
tools = body.get("allowed_tools")
if tools is not None and not isinstance(tools, list):
raise HTTPException(400, "allowed_tools must be a list or null")
with get_conn() as conn:
conn.execute(
"""INSERT INTO agent_policies (workspace_id, allowed_tools_json, max_steps,
require_approval)
VALUES (?,?,?,?)
ON CONFLICT(workspace_id) DO UPDATE SET
allowed_tools_json=excluded.allowed_tools_json,
max_steps=excluded.max_steps, require_approval=excluded.require_approval""",
(wid, json.dumps(tools) if tools is not None else None,
max(1, min(int(body.get("max_steps") or 12), 50)),
1 if body.get("require_approval") else 0))
conn.commit()
row = conn.execute("SELECT * FROM agent_policies WHERE workspace_id IS ?",
(wid,)).fetchone()
audit_log(user, "agent.policy.upsert", "workspace", wid or 0, "", request)
return JSONResponse(status_code=201, content=dict(row))
@router.get("/api/v2/agent-approvals")
async def list_approvals(request: Request):
_owner_or_admin(request)
status = request.query_params.get("status", "pending")
with get_conn() as conn:
rows = conn.execute("SELECT * FROM agent_approvals WHERE status=? ORDER BY id DESC"
" LIMIT 100", (status,)).fetchall()
return {"approvals": [dict(r) for r in rows]}
@router.post("/api/v2/agent-approvals/{approval_id}/decide")
async def decide_approval(approval_id: int, request: Request):
user = _owner_or_admin(request)
try:
body = await request.json()
except Exception:
body = {}
out = policies.decide_approval(approval_id, user["id"], bool(body.get("approve")))
if out is None:
raise HTTPException(404, "Pending approval not found")
audit_log(user, "agent.approval.decide", "agent_approval", approval_id,
out["status"], request)
return out
+378
View File
@@ -0,0 +1,378 @@
"""FlowDeck — unified import API (v5.6.0, Phase 0/1/2).
Exposes the importer registry, a dry-run preview, a synchronous run and an
optional background job with polling. Works with the existing workspace cookie
(``flowdeck_workspace``) and session.
"""
from __future__ import annotations
import json
import logging
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import HTMLResponse, RedirectResponse, Response
from app.auth.session import SessionManager
from app.db import get_conn
from app.services.automations import fire_event
from app.services.importers import (
get_job,
list_jobs,
list_sources,
parse_upload,
preview_result,
resolve_relations,
run_import,
start_import_job,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/import", tags=["import"])
page_router = APIRouter(tags=["import"])
MAX_UPLOAD_BYTES = 200 * 1024 * 1024
def _current_user(request: Request) -> dict:
return SessionManager.decode_session(request.cookies.get("flowdeck_session", "")) or {}
@page_router.get("/import", response_class=HTMLResponse)
async def import_page(request: Request):
"""Standalone import wizard (source picker, dry-run, mapping, progress)."""
user = _current_user(request)
if not user:
return RedirectResponse("/auth/login?provider=local", status_code=302)
from app.templating import ENV
env = ENV
return HTMLResponse(content=env.get_template("import.html").render(user=user))
def _workspace(request: Request) -> tuple[int | None, str]:
"""Resolve (workspace_id, login) from the workspace cookie + session."""
ws_id: int | None = None
cookie = request.cookies.get("flowdeck_workspace", "")
try:
value = int(cookie)
if value > 0:
ws_id = value
except (ValueError, TypeError):
pass
user = _current_user(request)
login = user.get("login", "") if user else ""
return ws_id, login
async def _read_upload(request: Request) -> tuple[str, bytes, str | None]:
form = await request.form()
upload = form.get("file")
if upload is None or not hasattr(upload, "filename"):
raise HTTPException(400, "file field required")
filename = (upload.filename or "import").replace("\\", "/").rsplit("/", 1)[-1]
data = await upload.read()
if len(data) > MAX_UPLOAD_BYTES:
raise HTTPException(413, "File too large (max 200 MB)")
source_id = form.get("source") or None
return filename, data, source_id
@router.get("/sources")
async def import_sources(request: Request):
"""List every available importer for the UI source picker."""
return {"sources": list_sources()}
@router.post("/preview")
async def import_preview(request: Request):
"""Dry-run: parse the upload and describe what would be created."""
filename, data, source_id = await _read_upload(request)
imp, result = parse_upload(filename, data, source_id)
if imp is None:
raise HTTPException(400, "Format non reconnu — choisissez une source")
out = preview_result(result)
out["detected_source"] = imp.source_id
out["source_label"] = imp.label
return out
@router.post("/run")
async def import_run(request: Request):
"""Import an upload (synchronously, or as a background job when async=true)."""
form = await request.form()
upload = form.get("file")
if upload is None or not hasattr(upload, "filename"):
raise HTTPException(400, "file field required")
filename = (upload.filename or "import").replace("\\", "/").rsplit("/", 1)[-1]
data = await upload.read()
if len(data) > MAX_UPLOAD_BYTES:
raise HTTPException(413, "File too large (max 200 MB)")
source_id = form.get("source") or None
parent_id = _int_or_none(form.get("parent_id"))
target = _int_or_none(form.get("target_collection_id"))
dedup = str(form.get("dedup", "true")).lower() not in ("false", "0", "no")
async_mode = str(form.get("async", "false")).lower() in ("true", "1", "yes")
mapping = _parse_mapping(form.get("mapping"))
mode = _parse_mode(form.get("mode"))
ws_id, login = _workspace(request)
if async_mode:
job = start_import_job(
filename=filename, data=data, source_id=source_id,
workspace_id=ws_id, workspace_name=login, user_login=login,
parent_page_id=parent_id, target_collection_id=target, dedup=dedup,
mapping=mapping, mode=mode,
)
return {"status": "queued", "job_id": job["id"]}
imp, result = parse_upload(filename, data, source_id)
if imp is None:
raise HTTPException(400, "Format non reconnu — choisissez une source")
report = run_import(
result, workspace_id=ws_id, workspace_name=login, user_login=login,
parent_page_id=parent_id, target_collection_id=target, dedup=dedup,
mapping=mapping, mode=mode,
)
report["detected_source"] = imp.source_id
if imp.source_id == "notion":
with get_conn() as conn:
report["relations"] = resolve_relations(conn, ws_id)
for page_id in report.get("page_ids", [])[:100]:
try:
await fire_event("page.created", {"page_id": page_id, "title": "", "workspace": login})
except Exception: # noqa: BLE001 - events are best-effort
pass
return report
@router.post("/forge")
async def import_forge(request: Request):
"""Import a forge repo's issues (+ labels/milestones) into collections."""
try:
body = await request.json()
except Exception:
raise HTTPException(400, "Invalid JSON body") from None
provider = str(body.get("provider") or "gitea").lower()
owner = str(body.get("owner") or "").strip()
repo = str(body.get("repo") or "").strip()
if not owner or not repo:
raise HTTPException(400, "owner and repo are required")
state = str(body.get("state") or "all")
include_labels = bool(body.get("include_labels", True))
include_milestones = bool(body.get("include_milestones", True))
ws_id, login = _workspace(request)
if provider == "gitea":
from app.services.gitea_client import get_user_gitea_client
from app.services.importers.forge import GiteaForgeAdapter
client = get_user_gitea_client(request)
if client is None:
raise HTTPException(400, "Gitea non connecté")
adapter = GiteaForgeAdapter(client)
elif provider == "github":
from app.services.github_adapter import GitHubAdapter
token = _user_oauth_token(request, "github")
if not token:
raise HTTPException(400, "GitHub non connecté")
adapter = GitHubAdapter(token)
else:
raise HTTPException(400, "provider must be 'gitea' or 'github'")
from app.services.importers.forge import fetch_forge_issues
result = await fetch_forge_issues(
adapter, owner, repo, provider=provider, state=state,
include_labels=include_labels, include_milestones=include_milestones,
)
report = run_import(
result, workspace_id=ws_id, workspace_name=login, user_login=login,
)
report["detected_source"] = f"forge:{provider}"
return report
@router.post("/forge-repo")
async def import_forge_repo(request: Request):
"""Import a forge repo's text files as pages (folder hierarchy preserved)."""
try:
body = await request.json()
except Exception:
raise HTTPException(400, "Invalid JSON body") from None
provider = str(body.get("provider") or "gitea").lower()
owner = str(body.get("owner") or "").strip()
repo = str(body.get("repo") or "").strip()
if not owner or not repo:
raise HTTPException(400, "owner and repo are required")
path = str(body.get("path") or "")
max_files = min(int(body.get("max_files") or 200), 1000)
ws_id, login = _workspace(request)
adapter = _forge_adapter(request, provider)
from app.services.importers.forge_repo import fetch_forge_repo
result = await fetch_forge_repo(
adapter, owner, repo, provider=provider, path=path, max_files=max_files,
)
report = run_import(result, workspace_id=ws_id, workspace_name=login, user_login=login)
report["detected_source"] = f"forge-repo:{provider}"
return report
@router.post("/url")
async def import_url(request: Request):
"""Web clipper: fetch a URL and create a page (bookmark card + content)."""
try:
body = await request.json()
except Exception:
raise HTTPException(400, "Invalid JSON body") from None
url = str(body.get("url") or "").strip()
if not url:
raise HTTPException(400, "url is required")
ws_id, login = _workspace(request)
from app.services.importers.url_fetch import fetch_url_result
try:
result = await fetch_url_result(url)
except ValueError as exc:
raise HTTPException(400, str(exc)) from None
if not result.pages:
raise HTTPException(422, "; ".join(result.warnings) or "Page introuvable")
report = run_import(result, workspace_id=ws_id, workspace_name=login, user_login=login)
report["detected_source"] = "url"
return report
@router.post("/run-batch")
async def import_run_batch(request: Request):
"""Import several uploaded files sequentially, returning one report each."""
form = await request.form()
uploads = form.getlist("file")
if not uploads:
raise HTTPException(400, "file field required")
source_id = form.get("source") or None
parent_id = _int_or_none(form.get("parent_id"))
target = _int_or_none(form.get("target_collection_id"))
dedup = str(form.get("dedup", "true")).lower() not in ("false", "0", "no")
mode = _parse_mode(form.get("mode"))
mapping = _parse_mapping(form.get("mapping"))
ws_id, login = _workspace(request)
results: list[dict] = []
summary = {"files": 0, "pages_created": 0, "rows_created": 0, "errors": 0}
for upload in uploads:
filename = (getattr(upload, "filename", "") or "import").replace("\\", "/").rsplit("/", 1)[-1]
data = await upload.read()
if len(data) > MAX_UPLOAD_BYTES:
results.append({"filename": filename, "report": {"status": "error",
"errors": [{"title": filename, "error": "File too large"}]}})
summary["errors"] += 1
continue
imp, result = parse_upload(filename, data, source_id)
if imp is None:
results.append({"filename": filename, "report": {"status": "error",
"errors": [{"title": filename, "error": "Format non reconnu"}]}})
summary["errors"] += 1
continue
report = run_import(
result, workspace_id=ws_id, workspace_name=login, user_login=login,
parent_page_id=parent_id, target_collection_id=target, dedup=dedup,
mapping=mapping, mode=mode,
)
report["detected_source"] = imp.source_id
results.append({"filename": filename, "report": report})
summary["files"] += 1
summary["pages_created"] += report.get("pages_created", 0)
summary["rows_created"] += report.get("rows_created", 0)
summary["errors"] += len(report.get("errors", []))
return {"status": "ok", "summary": summary, "results": results}
@router.post("/relations/resolve")
async def import_resolve_relations(request: Request):
"""Convert text columns referencing another collection into relation props."""
ws_id, _ = _workspace(request)
with get_conn() as conn:
return resolve_relations(conn, ws_id)
@router.get("/jobs")
async def import_jobs(request: Request):
return {"jobs": list_jobs()}
@router.get("/jobs/{job_id}")
async def import_job(job_id: str):
job = get_job(job_id)
if not job:
raise HTTPException(404, "Job not found")
return job
@router.get("/jobs/{job_id}/report")
async def import_job_report(job_id: str):
"""Download a job's import report as JSON."""
job = get_job(job_id)
if not job:
raise HTTPException(404, "Job not found")
payload = json.dumps(job.get("report") or {}, ensure_ascii=False, indent=2)
return Response(
content=payload,
media_type="application/json",
headers={"Content-Disposition": f'attachment; filename="import-{job_id}.json"'},
)
def _forge_adapter(request: Request, provider: str):
if provider == "gitea":
from app.services.gitea_client import get_user_gitea_client
from app.services.importers.forge import GiteaForgeAdapter
client = get_user_gitea_client(request)
if client is None:
raise HTTPException(400, "Gitea non connecté")
return GiteaForgeAdapter(client)
if provider == "github":
from app.services.github_adapter import GitHubAdapter
token = _user_oauth_token(request, "github")
if not token:
raise HTTPException(400, "GitHub non connecté")
return GitHubAdapter(token)
raise HTTPException(400, "provider must be 'gitea' or 'github'")
def _int_or_none(value) -> int | None:
try:
ivalue = int(value)
return ivalue if ivalue > 0 else None
except (ValueError, TypeError):
return None
def _parse_mapping(value) -> dict[str, str] | None:
if not value:
return None
try:
parsed = json.loads(value)
except (ValueError, TypeError):
return None
if isinstance(parsed, dict):
return {str(k): str(v) for k, v in parsed.items() if v}
return None
def _parse_mode(value) -> str | None:
mode = str(value or "").strip().lower()
return mode if mode in ("skip", "update", "duplicate") else None
def _user_oauth_token(request: Request, provider: str) -> str:
user = _current_user(request)
if not user or not user.get("id"):
return ""
with get_conn() as conn:
row = conn.execute(
"SELECT access_token FROM user_oauth_tokens WHERE user_id=? AND provider=? "
"ORDER BY updated_at DESC LIMIT 1",
(user["id"], provider),
).fetchone()
return row["access_token"] if row else ""
+13 -4
View File
@@ -235,15 +235,24 @@ async def library_shared(
uid = _get_user_id(request)
# Page ids the user shares toward others (nominal page_shares) or receives
# (direct shares + group shares via group_members)
with get_conn() as conn:
made_rows = conn.execute(
"SELECT DISTINCT s.page_id FROM page_shares s WHERE s.created_by=?",
(uid,),
).fetchall()
recv_rows = conn.execute(
"SELECT DISTINCT s.page_id FROM page_shares s WHERE s.shared_with_user_id=?",
(uid,),
).fetchall()
try:
recv_rows = conn.execute(
"""SELECT DISTINCT s.page_id FROM page_shares s
LEFT JOIN group_members gm ON gm.group_id = s.shared_with_group_id AND gm.user_id=?
WHERE s.shared_with_user_id=? OR gm.user_id=?""",
(uid, uid, uid),
).fetchall()
except Exception:
recv_rows = conn.execute(
"SELECT DISTINCT s.page_id FROM page_shares s WHERE s.shared_with_user_id=?",
(uid,),
).fetchall()
made_ids = {r[0] for r in made_rows}
recv_ids = {r[0] for r in recv_rows}
+206
View File
@@ -0,0 +1,206 @@
"""FlowDeck — Calendar links + Meetings API (v7.1.0).
``/api/v2/calendar-links*`` — Google/CalDAV link CRUD (session or Bearer
``write``), manual sync trigger. ``GET /db/{id}/calendar/freebusy`` —
weekday availability. ``/api/v2/meetings/*`` — audio upload, manual
transcript, AI summary (fires ``meeting.summarized``).
See ``docs/V71_Calendar_Meetings.md``.
"""
from __future__ import annotations
import secrets
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import JSONResponse
from app.auth.session import SessionManager
from app.db import get_conn
from app.services import calendar_sync as cal
from app.services import meetings as meet
from app.services.api_v2_helpers import (
audit_log,
has_scope,
resolve_bearer_token,
row_to_dict,
)
router = APIRouter(tags=["calendar-meetings"])
def _auth_user(request: Request, *, require_write: bool = False) -> dict:
sess = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if sess:
return sess
auth = request.headers.get("authorization") or request.headers.get("Authorization") or ""
if auth.lower().startswith("bearer "):
user = resolve_bearer_token(auth[7:].strip())
if not user:
raise HTTPException(401, "Invalid or expired API token")
if require_write and not has_scope(user.get("_token_scopes") or "read", "write"):
raise HTTPException(403, "Insufficient scope. Required: write")
return user
raise HTTPException(401, "Authentication required")
# ── calendar links ─────────────────────────────────────────────────────────
@router.post("/api/v2/calendar-links")
async def create_link(request: Request):
user = _auth_user(request, require_write=True)
try:
body = await request.json()
except Exception:
body = {}
provider = (body.get("provider") or "").lower()
if provider not in cal.PROVIDERS:
raise HTTPException(400, "provider must be google|caldav")
try:
collection_id = int(body.get("collection_id", 0))
except (TypeError, ValueError):
raise HTTPException(400, "collection_id required") from None
creds = body.get("credentials") or {}
if provider == "google" and not creds.get("access_token"):
raise HTTPException(400, "google needs credentials.access_token")
if provider == "caldav" and not creds.get("url"):
raise HTTPException(400, "caldav needs credentials.url")
try:
out = cal.save_link(user["id"], provider, collection_id, creds,
body.get("calendar_id") or "primary",
body.get("date_property") or "")
except ValueError as exc:
raise HTTPException(400, str(exc)) from None
audit_log(user, "calendar.link", "collection", collection_id, provider, request)
return JSONResponse(status_code=201, content=out)
@router.get("/api/v2/calendar-links")
async def get_links(request: Request):
user = _auth_user(request)
return {"links": cal.list_links(user["id"])}
@router.delete("/api/v2/calendar-links/{link_id}")
async def remove_link(link_id: int, request: Request):
user = _auth_user(request, require_write=True)
if not cal.delete_link(user["id"], link_id):
raise HTTPException(404, "Link not found")
audit_log(user, "calendar.unlink", "calendar_link", link_id, "", request)
return {"status": "deleted", "id": link_id}
@router.post("/api/v2/calendar-links/{link_id}/sync")
async def sync_now(link_id: int, request: Request):
user = _auth_user(request, require_write=True)
with get_conn() as conn:
row = conn.execute("SELECT * FROM calendar_links WHERE id=?", (link_id,)).fetchone()
if not row or (row["user_id"] != user["id"] and not user.get("is_admin")):
raise HTTPException(404, "Link not found")
try:
stats = await cal.sync_link(link_id)
except (cal.SyncError, ValueError) as exc:
raise HTTPException(502 if isinstance(exc, cal.SyncError) else 400,
str(exc)) from None
audit_log(user, "calendar.sync", "calendar_link", link_id, str(stats), request)
return {"link_id": link_id, **stats}
# ── free/busy ──────────────────────────────────────────────────────────────
@router.get("/db/{collection_id}/calendar/freebusy")
async def freebusy(collection_id: int, request: Request):
_auth_user(request)
qp = request.query_params
try:
out = cal.freebusy(collection_id, qp.get("from", ""), qp.get("to", ""),
qp.get("date_property", ""))
except ValueError as exc:
raise HTTPException(400, str(exc)) from None
return out
# ── meetings ───────────────────────────────────────────────────────────────
@router.post("/api/v2/meetings/transcribe")
async def upload_and_transcribe(request: Request):
user = _auth_user(request, require_write=True)
try:
form = await request.form()
except Exception:
raise HTTPException(400, "multipart upload required") from None
upload = form.get("audio")
try:
page_id = int(form.get("page_id", 0))
except (TypeError, ValueError):
raise HTTPException(400, "page_id required") from None
language = (form.get("language") or "fr")[:10]
manual = (form.get("transcript") or "").strip()
if upload is None and not manual:
raise HTTPException(400, "audio file or transcript required")
audio_path = ""
if upload is not None:
filename = (upload.filename or "").lower()
ext = filename.rsplit(".", 1)[-1] if "." in filename else ""
if ext not in meet.AUDIO_EXTENSIONS:
raise HTTPException(400, f"audio must be one of {sorted(meet.AUDIO_EXTENSIONS)}")
data = await upload.read()
if len(data) > meet.MAX_AUDIO_BYTES:
raise HTTPException(413, "audio exceeds 100 MB")
if not data:
raise HTTPException(400, "empty audio file")
audio_path = str(meet.meetings_dir()
/ f"{page_id}_{secrets.token_hex(8)}.{ext}")
with open(audio_path, "wb") as fh:
fh.write(data)
transcript = manual
if not transcript and audio_path:
try:
transcript = meet.transcribe_audio(audio_path, language)
except meet.TranscriptionUnavailable as exc:
transcript = "" # stored; client transcribes or posts manual text later
_ = exc
try:
tid = meet.save_transcript(page_id, transcript, language, audio_path)
except ValueError as exc:
raise HTTPException(404, str(exc)) from None
with get_conn() as conn:
row = conn.execute("SELECT * FROM meeting_transcripts WHERE id=?", (tid,)).fetchone()
audit_log(user, "meeting.transcribe", "page", page_id, f"transcript={tid}", request)
return JSONResponse(status_code=201, content={
**row_to_dict(row), "transcribed": bool(transcript)})
@router.post("/api/v2/meetings/transcripts/{transcript_id}/text")
async def set_transcript_text(transcript_id: int, request: Request):
"""Store a client-side (manual) transcript on an existing row."""
_auth_user(request, require_write=True)
try:
body = await request.json()
except Exception:
body = {}
text = (body.get("transcript") or "").strip()
if not text:
raise HTTPException(400, "transcript required")
with get_conn() as conn:
if not conn.execute("SELECT id FROM meeting_transcripts WHERE id=?",
(transcript_id,)).fetchone():
raise HTTPException(404, "Transcript not found")
conn.execute("UPDATE meeting_transcripts SET transcript=? WHERE id=?",
(text, transcript_id))
conn.commit()
row = conn.execute("SELECT * FROM meeting_transcripts WHERE id=?",
(transcript_id,)).fetchone()
return row_to_dict(row)
@router.post("/api/v2/meetings/transcripts/{transcript_id}/summarize")
async def summarize(transcript_id: int, request: Request):
user = _auth_user(request, require_write=True)
try:
out = await meet.summarize_transcript(transcript_id, user.get("id"))
except ValueError as exc:
raise HTTPException(400, str(exc)) from None
except RuntimeError as exc:
raise HTTPException(502, str(exc)) from None
audit_log(user, "meeting.summarize", "transcript", transcript_id, "", request)
return out
+2 -2
View File
@@ -6,10 +6,10 @@ import logging
from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse
from jinja2 import Environment, FileSystemLoader
from app.auth.session import SessionManager
from app.db import get_conn
from app.templating import ENV
logger = logging.getLogger(__name__)
router = APIRouter(tags=["my-tasks"], prefix="/my-tasks")
@@ -102,7 +102,7 @@ async def my_tasks_dashboard(request: Request, view: str = "all", days: int = 7)
</div>"""
from app.routers.dashboard import _sidebar_data
env = Environment(loader=FileSystemLoader("app/templates"))
env = ENV
sidebar = _sidebar_data(request, [])
block_tpl = env.from_string(
'{% extends "base.html" %}{% block content %}{{ content_html|safe }}{% endblock %}'
+4 -6
View File
@@ -21,10 +21,9 @@ async def get_notes(request: Request, owner: str, repo: str):
).fetchone()
content = row["content"] if row else ""
from jinja2 import Environment, FileSystemLoader
from app.auth.session import SessionManager
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
template = env.get_template("notes.html")
return template.render(request=request, owner=owner, repo=repo, content=content, user=user)
@@ -45,10 +44,9 @@ async def save_notes(request: Request, owner: str, repo: str):
)
conn.commit()
from jinja2 import Environment, FileSystemLoader
from app.auth.session import SessionManager
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
template = env.get_template("notes.html")
return template.render(request=request, owner=owner, repo=repo, content=content, user=user)
+27 -1
View File
@@ -97,13 +97,39 @@ async def set_prefs(request: Request):
from app.services import notifications as notif
body = await request.json() if request.headers.get("content-type") else {}
prefs = notif.get_user_prefs(user["id"])
for key in ("comments", "mentions"):
for key in ("comments", "mentions", "reminders", "assignments"):
if key in body:
prefs[key] = bool(body[key])
notif.set_user_prefs(user["id"], prefs)
return {"status": "ok", "prefs": prefs}
@router.get("/timezone")
async def get_timezone(request: Request):
"""Return the current user's IANA timezone ('' = UTC)."""
user = _current_user(request)
with get_conn() as conn:
row = conn.execute("SELECT timezone FROM users WHERE id=?", (user["id"],)).fetchone()
tz = (row["timezone"] if row and "timezone" in row.keys() else "") or ""
from app.services.recurrence import common_timezones
return {"timezone": tz, "zones": common_timezones()}
@router.post("/timezone")
async def set_timezone(request: Request):
"""Update the current user's IANA timezone (empty string = UTC)."""
user = _current_user(request)
body = await request.json() if request.headers.get("content-type") else {}
tz = (body.get("timezone") or "").strip()
from app.services.recurrence import is_valid_timezone
if tz and not is_valid_timezone(tz):
raise HTTPException(status_code=400, detail=f"Unknown timezone '{tz}'")
with get_conn() as conn:
conn.execute("UPDATE users SET timezone=? WHERE id=?", (tz, user["id"]))
conn.commit()
return {"status": "ok", "timezone": tz}
@router.get("/users/search")
async def search_users(request: Request, q: str = ""):
"""User autocomplete for @mentions."""
+2 -2
View File
@@ -41,8 +41,8 @@ async def onboarding_page(request: Request):
if ws_count > 0:
return RedirectResponse("/workspaces", status_code=302)
from jinja2 import Environment, FileSystemLoader
env = Environment(loader=FileSystemLoader("app/templates"))
from app.templating import ENV
env = ENV
template = env.get_template("welcome.html")
return HTMLResponse(content=template.render(
user=user,
+525
View File
@@ -0,0 +1,525 @@
"""FlowDeck — v6.0.0 Granular permissions API (page/collection/property ACL).
Backend for the page-editor "Permissions" panel, database property visibility
and user-group management. Grants are stored in ``page_permissions`` /
``collection_permissions`` / ``property_permissions``; every mutation is logged
into ``permission_audit_log`` for the admin audit view.
"""
from __future__ import annotations
import logging
from fastapi import APIRouter, HTTPException, Request
from app.auth.session import SessionManager
from app.db import get_conn
from app.services.permission_manager import PermissionManager
logger = logging.getLogger(__name__)
router = APIRouter(tags=["permissions"], prefix="/api/v2")
PAGE_ROLES = ("viewer", "commenter", "editor", "owner")
COLLECTION_ROLES = ("viewer", "commenter", "editor", "owner")
PROPERTY_ROLES = ("viewer", "editor")
PERMISSION_TYPES = ("inherit", "restricted", "private")
def _require_user(request: Request) -> dict:
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(status_code=401, detail="Not authenticated")
return user
def _pm(request: Request) -> PermissionManager:
return PermissionManager(_require_user(request)["id"])
def _client_ip(request: Request) -> str:
try:
return request.client.host if request.client else ""
except Exception:
return ""
def _perm_list(conn, table: str, fk: str, resource_id: int) -> list[dict]:
rows = conn.execute(
f"""SELECT p.*,
u.login AS user_login, u.full_name AS user_name,
g.name AS group_name
FROM {table} p
LEFT JOIN users u ON u.id = p.user_id
LEFT JOIN user_groups g ON g.id = p.group_id
WHERE p.{fk}=? ORDER BY p.id""",
(resource_id,),
).fetchall()
out = []
for r in rows:
d = dict(r)
if d.get("user_id"):
d["name"] = d["user_name"] or d["user_login"] or f"User #{d['user_id']}"
d["kind"] = "user"
else:
d["name"] = d["group_name"] or f"Group #{d['group_id']}"
d["kind"] = "group"
out.append(d)
return out
def _grant_common(request: Request, pm: PermissionManager, resource_type: str,
resource_id: int, body: dict, table: str, fk: str,
allowed_roles: tuple[str, ...],
extra_cols: dict | None = None) -> dict:
user_id = body.get("user_id")
group_id = body.get("group_id")
role = (body.get("role") or "").strip()
if role not in allowed_roles:
raise HTTPException(400, f"role must be one of {', '.join(allowed_roles)}")
if not user_id and not group_id:
raise HTTPException(400, "Provide either user_id or group_id")
if user_id and not isinstance(user_id, int):
raise HTTPException(400, "user_id must be an integer")
if group_id and not isinstance(group_id, int):
raise HTTPException(400, "group_id must be an integer")
actor = _require_user(request)["id"]
with get_conn() as conn:
if user_id:
exists = conn.execute("SELECT id FROM users WHERE id=?", (user_id,)).fetchone()
if not exists:
raise HTTPException(404, "User not found")
if group_id:
exists = conn.execute("SELECT id FROM user_groups WHERE id=?", (group_id,)).fetchone()
if not exists:
raise HTTPException(404, "Group not found")
existing = conn.execute(
f"SELECT id, role FROM {table} WHERE {fk}=? AND user_id IS ? AND group_id IS ?",
(resource_id, user_id, group_id),
).fetchone()
if existing:
conn.execute(f"UPDATE {table} SET role=? WHERE id=?",
(role, existing["id"]))
old_role = existing["role"]
perm_id = existing["id"]
else:
cols = [fk, "user_id", "group_id", "role", "granted_by"]
vals: list = [resource_id, user_id, group_id, role, actor]
for col, val in (extra_cols or {}).items():
cols.append(col)
vals.append(val)
placeholders = ", ".join("?" for _ in cols)
cur = conn.execute(
f"INSERT INTO {table} ({', '.join(cols)}) VALUES ({placeholders})",
tuple(vals),
)
perm_id = cur.lastrowid
old_role = None
conn.commit()
pm.invalidate()
pm.log_permission_change(resource_type, resource_id, "grant",
target_user_id=user_id, target_group_id=group_id,
old_role=old_role, new_role=role, ip_address=_client_ip(request))
return {"status": "ok", "id": perm_id, "role": role, "user_id": user_id, "group_id": group_id}
def _revoke_common(request: Request, pm: PermissionManager, resource_type: str,
resource_id: int, table: str, fk: str, perm_id: int) -> dict:
with get_conn() as conn:
row = conn.execute(
f"SELECT user_id, group_id, role FROM {table} WHERE id=? AND {fk}=?",
(perm_id, resource_id),
).fetchone()
if not row:
raise HTTPException(404, "Permission not found")
conn.execute(f"DELETE FROM {table} WHERE id=?", (perm_id,))
conn.commit()
pm.invalidate()
pm.log_permission_change(resource_type, resource_id, "revoke",
target_user_id=row["user_id"], target_group_id=row["group_id"],
old_role=row["role"], new_role=None, ip_address=_client_ip(request))
return {"status": "revoked"}
def _set_permission_type(request: Request, pm: PermissionManager, resource_type: str,
resource_id: int, table: str, body: dict) -> dict:
ptype = (body.get("permission_type") or "").strip()
if ptype not in PERMISSION_TYPES:
raise HTTPException(400, f"permission_type must be one of {', '.join(PERMISSION_TYPES)}")
with get_conn() as conn:
conn.execute(f"UPDATE {table} SET permission_type=? WHERE id=?", (ptype, resource_id))
conn.commit()
pm.invalidate()
pm.log_permission_change(resource_type, resource_id, "type_change",
new_role=ptype, ip_address=_client_ip(request))
return {"status": "ok", "permission_type": ptype}
# ═══════════════ Page permissions ═══════════════
@router.get("/pages/{page_id}/permissions")
async def list_page_permissions(page_id: int, request: Request):
"""List explicit page grants + the caller's effective role."""
pm = _pm(request)
if not pm.can_view_page(page_id):
raise HTTPException(404, "Page not found")
with get_conn() as conn:
grants = _perm_list(conn, "page_permissions", "page_id", page_id)
return {
"permissions": grants,
"mine": pm.get_page_permission(page_id),
"permission_type": _page_type(page_id),
"can_manage": pm.can_manage_page_permissions(page_id),
}
@router.get("/pages/{page_id}/permissions/mine")
async def my_page_permission(page_id: int, request: Request):
"""Effective role of the current user on a page (UI gating)."""
pm = _pm(request)
if not pm.can_view_page(page_id):
raise HTTPException(404, "Page not found")
return {
"role": pm.get_page_permission(page_id),
"can_edit": pm.can_edit_page(page_id),
"can_comment": pm.can_comment_page(page_id),
"can_manage": pm.can_manage_page_permissions(page_id),
"permission_type": _page_type(page_id),
}
def _page_type(page_id: int) -> str:
with get_conn() as conn:
row = conn.execute(
"SELECT permission_type FROM pages WHERE id=?", (page_id,)
).fetchone()
return (row["permission_type"] if row else "inherit") or "inherit"
@router.post("/pages/{page_id}/permissions")
async def grant_page_permission(page_id: int, request: Request):
pm = _pm(request)
if not pm.can_manage_page_permissions(page_id):
raise HTTPException(403, "Only a page owner can manage its permissions")
body = await request.json()
return _grant_common(request, pm, "page", page_id, body,
"page_permissions", "page_id", PAGE_ROLES)
@router.post("/pages/{page_id}/permissions/batch")
async def batch_page_permissions(page_id: int, request: Request):
"""Grant several permissions in one call: {grants: [{user_id|group_id, role}, ...]}."""
pm = _pm(request)
if not pm.can_manage_page_permissions(page_id):
raise HTTPException(403, "Only a page owner can manage its permissions")
body = await request.json()
grants = body.get("grants") or []
if not isinstance(grants, list) or not grants:
raise HTTPException(400, "grants must be a non-empty list")
results = []
for g in grants:
results.append(_grant_common(request, pm, "page", page_id, g,
"page_permissions", "page_id", PAGE_ROLES))
return {"status": "ok", "granted": results}
@router.delete("/pages/{page_id}/permissions/{perm_id}")
async def revoke_page_permission(page_id: int, perm_id: int, request: Request):
pm = _pm(request)
if not pm.can_manage_page_permissions(page_id):
raise HTTPException(403, "Only a page owner can manage its permissions")
return _revoke_common(request, pm, "page", page_id, "page_permissions", "page_id", perm_id)
@router.post("/pages/{page_id}/permission-type")
async def set_page_permission_type(page_id: int, request: Request):
pm = _pm(request)
if not pm.can_manage_page_permissions(page_id):
raise HTTPException(403, "Only a page owner can manage its permissions")
return _set_permission_type(request, pm, "page", page_id, "pages", await request.json())
# ═══════════════ Collection permissions ═══════════════
@router.get("/collections/{collection_id}/permissions")
async def list_collection_permissions(collection_id: int, request: Request):
pm = _pm(request)
if not pm.can_view_collection(collection_id):
raise HTTPException(404, "Collection not found")
with get_conn() as conn:
grants = _perm_list(conn, "collection_permissions", "collection_id", collection_id)
return {
"permissions": grants,
"mine": pm.get_collection_permission(collection_id),
"permission_type": _collection_type(collection_id),
"can_manage": pm.can_manage_collection_permissions(collection_id),
}
def _collection_type(collection_id: int) -> str:
with get_conn() as conn:
row = conn.execute(
"SELECT permission_type FROM collections WHERE id=?", (collection_id,)
).fetchone()
return (row["permission_type"] if row else "inherit") or "inherit"
@router.post("/collections/{collection_id}/permissions")
async def grant_collection_permission(collection_id: int, request: Request):
pm = _pm(request)
if not pm.can_manage_collection_permissions(collection_id):
raise HTTPException(403, "Only a collection owner can manage its permissions")
body = await request.json()
return _grant_common(request, pm, "collection", collection_id, body,
"collection_permissions", "collection_id", COLLECTION_ROLES)
@router.delete("/collections/{collection_id}/permissions/{perm_id}")
async def revoke_collection_permission(collection_id: int, perm_id: int, request: Request):
pm = _pm(request)
if not pm.can_manage_collection_permissions(collection_id):
raise HTTPException(403, "Only a collection owner can manage its permissions")
return _revoke_common(request, pm, "collection", collection_id,
"collection_permissions", "collection_id", perm_id)
@router.post("/collections/{collection_id}/permission-type")
async def set_collection_permission_type(collection_id: int, request: Request):
pm = _pm(request)
if not pm.can_manage_collection_permissions(collection_id):
raise HTTPException(403, "Only a collection owner can manage its permissions")
return _set_permission_type(request, pm, "collection", collection_id,
"collections", await request.json())
@router.get("/collections/{collection_id}/properties/visible")
async def visible_properties(collection_id: int, request: Request):
"""Split property ids into visible / hidden for the current user."""
pm = _pm(request)
if not pm.can_view_collection(collection_id):
raise HTTPException(404, "Collection not found")
visible = pm.get_visible_properties(collection_id)
with get_conn() as conn:
all_ids = [r["id"] for r in conn.execute(
"SELECT id FROM collection_properties WHERE collection_id=?",
(collection_id,),
).fetchall()]
return {
"visible": visible,
"hidden": [pid for pid in all_ids if pid not in visible],
"can_edit": pm.can_edit_collection(collection_id),
}
# ═══════════════ Property permissions ═══════════════
@router.get("/collections/{collection_id}/properties/{property_id}/permissions")
async def list_property_permissions(collection_id: int, property_id: int, request: Request):
pm = _pm(request)
if not pm.can_view_collection(collection_id):
raise HTTPException(404, "Collection not found")
with get_conn() as conn:
grants = _perm_list(conn, "property_permissions", "property_id", property_id)
return {
"permissions": grants,
"mine_view": pm.can_view_property(collection_id, property_id),
"mine_edit": pm.can_edit_property(collection_id, property_id),
"can_manage": pm.can_manage_collection_permissions(collection_id),
}
@router.post("/collections/{collection_id}/properties/{property_id}/permissions")
async def grant_property_permission(collection_id: int, property_id: int, request: Request):
pm = _pm(request)
if not pm.can_manage_collection_permissions(collection_id):
raise HTTPException(403, "Only a collection owner can manage property permissions")
body = await request.json()
with get_conn() as conn:
prop = conn.execute(
"SELECT id FROM collection_properties WHERE id=? AND collection_id=?",
(property_id, collection_id),
).fetchone()
if not prop:
raise HTTPException(404, "Property not found")
return _grant_common(request, pm, "property", property_id, body,
"property_permissions", "property_id", PROPERTY_ROLES,
extra_cols={"collection_id": collection_id})
@router.delete("/collections/{collection_id}/properties/{property_id}/permissions/{perm_id}")
async def revoke_property_permission(collection_id: int, property_id: int,
perm_id: int, request: Request):
pm = _pm(request)
if not pm.can_manage_collection_permissions(collection_id):
raise HTTPException(403, "Only a collection owner can manage property permissions")
return _revoke_common(request, pm, "property", property_id,
"property_permissions", "property_id", perm_id)
# ═══════════════ Groups ═══════════════
@router.get("/groups")
async def list_groups(request: Request, workspace_id: int | None = None):
user = _require_user(request)
pm = PermissionManager(user["id"])
return {"groups": pm.get_groups_for_workspace(workspace_id)}
@router.post("/groups")
async def create_group(request: Request):
pm = _pm(request)
body = await request.json()
ws_id = body.get("workspace_id")
gid = pm.create_group(ws_id, body.get("name") or "", body.get("description") or "",
created_by=pm.user_id)
pm.log_permission_change("group", gid, "group_create",
target_group_id=gid, new_role="",
ip_address=_client_ip(request))
return {"status": "ok", "id": gid}
@router.put("/groups/{group_id}")
async def update_group(group_id: int, request: Request):
pm = _pm(request)
body = await request.json()
name = (body.get("name") or "").strip()
if not name:
raise HTTPException(400, "name is required")
with get_conn() as conn:
row = conn.execute(
"SELECT workspace_id FROM user_groups WHERE id=?", (group_id,)
).fetchone()
if not row:
raise HTTPException(404, "Group not found")
if not pm.is_workspace_admin(row["workspace_id"]):
raise HTTPException(403, "Only a workspace owner or admin can edit groups")
conn.execute(
"UPDATE user_groups SET name=?, description=? WHERE id=?",
(name, body.get("description") or "", group_id),
)
conn.commit()
return {"status": "ok"}
@router.delete("/groups/{group_id}")
async def delete_group(group_id: int, request: Request):
pm = _pm(request)
with get_conn() as conn:
row = conn.execute(
"SELECT workspace_id FROM user_groups WHERE id=?", (group_id,)
).fetchone()
if not row:
raise HTTPException(404, "Group not found")
if not pm.is_workspace_admin(row["workspace_id"]):
raise HTTPException(403, "Only a workspace owner or admin can delete groups")
pm.delete_group(group_id)
pm.log_permission_change("group", group_id, "group_delete",
target_group_id=group_id, ip_address=_client_ip(request))
return {"status": "deleted"}
@router.get("/groups/{group_id}/members")
async def list_group_members(group_id: int, request: Request):
user = _require_user(request)
return {"members": PermissionManager(user["id"]).get_group_members(group_id)}
@router.post("/groups/{group_id}/members")
async def add_group_member(group_id: int, request: Request):
pm = _pm(request)
body = await request.json()
user_id = body.get("user_id")
if not user_id or not isinstance(user_id, int):
raise HTTPException(400, "user_id is required")
with get_conn() as conn:
row = conn.execute(
"SELECT workspace_id FROM user_groups WHERE id=?", (group_id,)
).fetchone()
if not row:
raise HTTPException(404, "Group not found")
if not pm.is_workspace_admin(row["workspace_id"]):
raise HTTPException(403, "Only a workspace owner or admin can manage groups")
pm.add_user_to_group(group_id, user_id)
pm.invalidate()
pm.log_permission_change("group", group_id, "member_add",
target_user_id=user_id, target_group_id=group_id,
ip_address=_client_ip(request))
return {"status": "ok"}
@router.delete("/groups/{group_id}/members/{user_id}")
async def remove_group_member(group_id: int, user_id: int, request: Request):
pm = _pm(request)
with get_conn() as conn:
row = conn.execute(
"SELECT workspace_id FROM user_groups WHERE id=?", (group_id,)
).fetchone()
if not row:
raise HTTPException(404, "Group not found")
if not pm.is_workspace_admin(row["workspace_id"]):
raise HTTPException(403, "Only a workspace owner or admin can manage groups")
pm.remove_user_from_group(group_id, user_id)
pm.invalidate()
pm.log_permission_change("group", group_id, "member_remove",
target_user_id=user_id, target_group_id=group_id,
ip_address=_client_ip(request))
return {"status": "ok"}
# ═══════════════ Users (access pickers) + audit ═══════════════
@router.get("/users")
async def list_users(request: Request, workspace_id: int | None = None, q: str = ""):
"""Workspace members (+ admins) for the grant pickers."""
_require_user(request)
q = (q or "").strip().lower()
with get_conn() as conn:
if workspace_id:
rows = conn.execute(
"""SELECT DISTINCT u.id, u.login, u.full_name, u.email, u.avatar_color
FROM users u
LEFT JOIN workspace_members wm ON wm.user_id=u.id AND wm.workspace_id=?
WHERE u.is_admin=1 OR wm.id IS NOT NULL
ORDER BY u.login""",
(workspace_id,),
).fetchall()
else:
rows = conn.execute(
"SELECT id, login, full_name, email, avatar_color FROM users ORDER BY login"
).fetchall()
users = []
for r in rows:
d = dict(r)
if q and q not in (d["login"].lower(), d["full_name"].lower(),
d["email"].lower()):
continue
users.append({"id": d["id"], "login": d["login"], "name": d["full_name"] or d["login"],
"email": d["email"], "avatar_color": d["avatar_color"]})
return {"users": users}
@router.get("/audit/permissions")
async def permission_audit(request: Request, limit: int = 100):
"""Full permission change history — workspace owner/admin only."""
user = _require_user(request)
uid = user["id"]
is_admin = bool(user.get("is_admin"))
limit = max(1, min(int(limit), 500))
with get_conn() as conn:
if not is_admin:
owned = conn.execute(
"SELECT id FROM workspaces WHERE owner_id=?", (uid,)
).fetchall()
if not owned:
raise HTTPException(403, "Only a workspace owner or admin can view the audit log")
rows = conn.execute(
"""SELECT a.*, u.login AS actor_login
FROM permission_audit_log a LEFT JOIN users u ON u.id=a.performed_by
ORDER BY a.created_at DESC, a.id DESC LIMIT ?""",
(limit,),
).fetchall()
return {"events": [dict(r) for r in rows]}
+11 -16
View File
@@ -40,6 +40,9 @@ def verify_token(authorization: str | None = Header(None)):
raise HTTPException(401, "API token required. Generate one via Settings → API tokens.")
token = authorization[7:] # strip "Bearer "
if token == DEFAULT_TOKEN:
from app.config import settings as _s
if not _s.public_api_insecure_ok:
raise HTTPException(401, "Default token disabled. Set PUBLIC_API_INSECURE_OK=true in dev or use a real Bearer token.")
return token
with get_conn() as conn:
row = conn.execute("SELECT 1 FROM user_tokens WHERE gitea_token=?", (token,)).fetchone()
@@ -53,25 +56,17 @@ def verify_token(authorization: str | None = Header(None)):
@router.post("/token")
async def generate_token(request: Request):
"""Generate a public API access token.
When an authenticated session is present the token is bound to that user
(revocable from Settings → API tokens); otherwise a legacy shared token is
created for backward compatibility.
"""
"""Generate a public API access token (A4 : session obligatoire — plus de
« legacy shared token » `user_id=0` créable par un anonymous)."""
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(401, "Authentication required")
token = f"fd_{token_urlsafe(24)}"
with get_conn() as conn:
if user and user.get("id"):
conn.execute(
"INSERT INTO api_tokens (user_id, name, token_hash, token_prefix) VALUES (?, ?, ?, ?)",
(user["id"], "API token", _hash_token(token), token[:12]),
)
else:
conn.execute(
"INSERT OR REPLACE INTO user_tokens (gitea_user_id, gitea_token, updated_at) VALUES (?, ?, CURRENT_TIMESTAMP)",
(0, token),
)
conn.execute(
"INSERT INTO api_tokens (user_id, name, token_hash, token_prefix) VALUES (?, ?, ?, ?)",
(user["id"], "API token", _hash_token(token), token[:12]),
)
conn.commit()
return {"token": token, "note": "Use as: Authorization: Bearer <token>"}
+16 -2
View File
@@ -7,7 +7,7 @@ from __future__ import annotations
import json
import logging
from fastapi import APIRouter, WebSocket
from fastapi import APIRouter, Request, WebSocket
from starlette.websockets import WebSocketDisconnect
from app.auth.session import SessionManager
@@ -17,6 +17,20 @@ logger = logging.getLogger(__name__)
router = APIRouter(tags=["realtime"])
@router.get("/api/realtime/stats")
async def realtime_stats(request: Request):
"""Observabilité realtime v6.4.0 : rooms, connexions, ops, merges, conflits.
Réservé aux utilisateurs authentifiés (données d'activité internes).
"""
user = SessionManager.decode_session(
request.cookies.get("flowdeck_session", "")
)
if not user or not user.get("id"):
return {"error": "unauthorized"}
return manager.stats()
@router.websocket("/ws/pages/{page_id}")
async def ws_page(websocket: WebSocket, page_id: int):
await websocket.accept()
@@ -27,7 +41,7 @@ async def ws_page(websocket: WebSocket, page_id: int):
try:
await websocket.close(code=4401)
except Exception:
pass
logger.exception("ws_page")
return
conn = await manager.connect(websocket, page_id, user)
+297
View File
@@ -0,0 +1,297 @@
"""FlowDeck — SCIM 2.0 provisioning + domain claims (v7.2.0).
``/scim/v2/Users`` (Bearer ``scim_tokens``, admin) : IT systems provision and
deprovision accounts. Suspend (``active=false``) flips ``users.is_active`` and
revokes ``user_sessions``. Domain claims: ``/.well-known`` HTTP verification +
optional local-login enforcement per email domain.
See ``docs/V72_Enterprise_SCIM_2FA.md``.
"""
from __future__ import annotations
import hashlib
import secrets
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import JSONResponse
from app.auth.session import SessionManager
from app.db import get_conn
from app.services.api_v2_helpers import audit_log
router = APIRouter(tags=["scim"])
SCIM_SCHEMAS = ["urn:ietf:params:scim:schemas:core:2.0:User"]
# ── auth ───────────────────────────────────────────────────────────────────
def _scim_guard(request: Request) -> dict:
auth = request.headers.get("authorization") or request.headers.get("Authorization") or ""
if auth.lower().startswith("bearer "):
digest = hashlib.sha256(auth[7:].strip().encode()).hexdigest()
with get_conn() as conn:
row = conn.execute("SELECT * FROM scim_tokens WHERE token_hash=? AND revoked=0",
(digest,)).fetchone()
if row:
return {"scim_token_id": row["id"], "name": row["name"]}
raise HTTPException(401, "SCIM token required")
def _admin_session(request: Request) -> dict:
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(401, "Authentication required")
with get_conn() as conn:
row = conn.execute("SELECT is_admin FROM users WHERE id=?", (user["id"],)).fetchone()
if not row or not row["is_admin"]:
raise HTTPException(403, "Admin required")
return user
def _scim_user(row) -> dict:
d = dict(row)
return {"schemas": SCIM_SCHEMAS, "id": str(d["id"]), "userName": d["login"],
"name": {"formatted": d.get("full_name") or d["login"]},
"emails": [{"value": d.get("email") or "", "primary": True}],
"active": bool(d.get("is_active", 1)),
"meta": {"resourceType": "User"}}
# ── SCIM resources ─────────────────────────────────────────────────────────
@router.get("/scim/v2/Users")
async def scim_list(request: Request):
_scim_guard(request)
with get_conn() as conn:
rows = conn.execute("SELECT * FROM users ORDER BY id LIMIT 100").fetchall()
items = [_scim_user(r) for r in rows]
return {"schemas": ["urn:ietf:params:scim:api:messages:2.0:ListResponse"],
"totalResults": len(items), "Resources": items}
@router.post("/scim/v2/Users")
async def scim_create(request: Request):
_scim_guard(request)
try:
body = await request.json()
except Exception:
body = {}
username = (body.get("userName") or "").strip()
if not username:
raise HTTPException(400, "userName required")
email = ""
for em in body.get("emails") or []:
if isinstance(em, dict) and em.get("value"):
email = em["value"]
break
name = ((body.get("name") or {}).get("formatted") or username)[:200]
active = body.get("active", True)
with get_conn() as conn:
if conn.execute("SELECT id FROM users WHERE login=?", (username,)).fetchone():
raise HTTPException(409, "User already exists")
cur = conn.execute(
"INSERT INTO users (login, full_name, email, is_active, auth_method)"
" VALUES (?,?,?,?,'saml')",
(username, name, email, 1 if active else 0))
conn.commit()
row = conn.execute("SELECT * FROM users WHERE id=?", (cur.lastrowid,)).fetchone()
return JSONResponse(status_code=201, content=_scim_user(row))
@router.get("/scim/v2/Users/{user_id}")
async def scim_get(user_id: str, request: Request):
_scim_guard(request)
with get_conn() as conn:
row = conn.execute("SELECT * FROM users WHERE id=?", (user_id,)).fetchone()
if not row:
raise HTTPException(404, "User not found")
return _scim_user(row)
def _apply_scim_update(conn, user_id: str, body: dict) -> None:
row = conn.execute("SELECT * FROM users WHERE id=?", (user_id,)).fetchone()
if not row:
raise HTTPException(404, "User not found")
updates: dict = {}
if "userName" in body and body["userName"]:
updates["login"] = body["userName"].strip()
if isinstance(body.get("name"), dict) and body["name"].get("formatted"):
updates["full_name"] = body["name"]["formatted"][:200]
if isinstance(body.get("emails"), list):
for em in body["emails"]:
if isinstance(em, dict) and em.get("value"):
updates["email"] = em["value"][:200]
break
if "active" in body:
updates["is_active"] = 1 if body["active"] else 0
if updates:
sets = ", ".join(f"{k}=?" for k in updates)
conn.execute(f"UPDATE users SET {sets} WHERE id=?", (*updates.values(), user_id))
if body.get("active") is False:
conn.execute("UPDATE user_sessions SET revoked=1 WHERE user_id=?", (user_id,))
conn.commit()
@router.put("/scim/v2/Users/{user_id}")
async def scim_replace(user_id: str, request: Request):
_scim_guard(request)
try:
body = await request.json()
except Exception:
body = {}
with get_conn() as conn:
_apply_scim_update(conn, user_id, body)
row = conn.execute("SELECT * FROM users WHERE id=?", (user_id,)).fetchone()
return _scim_user(row)
@router.patch("/scim/v2/Users/{user_id}")
async def scim_patch(user_id: str, request: Request):
_scim_guard(request)
try:
body = await request.json()
except Exception:
body = {}
flat: dict = {}
for op in body.get("Operations") or []:
path = (op.get("path") or "").lower()
if path in ("username", "active"):
flat["userName" if path == "username" else "active"] = op.get("value")
with get_conn() as conn:
_apply_scim_update(conn, user_id, {**body, **flat})
row = conn.execute("SELECT * FROM users WHERE id=?", (user_id,)).fetchone()
return _scim_user(row)
@router.delete("/scim/v2/Users/{user_id}")
async def scim_delete(user_id: str, request: Request):
_scim_guard(request)
with get_conn() as conn:
row = conn.execute("SELECT id FROM users WHERE id=?", (user_id,)).fetchone()
if not row:
raise HTTPException(404, "User not found")
# Deprovision = suspend (keeps content + audit trail).
conn.execute("UPDATE users SET is_active=0 WHERE id=?", (user_id,))
conn.execute("UPDATE user_sessions SET revoked=1 WHERE user_id=?", (user_id,))
conn.commit()
return JSONResponse(status_code=204, content=None)
# ── SCIM token management (admin, session) ─────────────────────────────────
@router.post("/api/v2/scim/tokens")
async def create_scim_token(request: Request):
admin = _admin_session(request)
try:
body = await request.json()
except Exception:
body = {}
raw = f"scim_{secrets.token_urlsafe(32)}"
digest = hashlib.sha256(raw.encode()).hexdigest()
with get_conn() as conn:
cur = conn.execute("INSERT INTO scim_tokens (token_hash, name, created_by)"
" VALUES (?,?,?)",
(digest, str(body.get("name") or "SCIM")[:120], admin["id"]))
conn.commit()
audit_log(admin, "scim.token.create", "scim_token", cur.lastrowid, "", request)
return JSONResponse(status_code=201,
content={"id": cur.lastrowid, "token": raw,
"warning": "shown once"})
@router.get("/api/v2/scim/tokens")
async def list_scim_tokens(request: Request):
_admin_session(request)
with get_conn() as conn:
rows = conn.execute("SELECT id, name, created_by, revoked, created_at"
" FROM scim_tokens ORDER BY id DESC").fetchall()
return {"tokens": [dict(r) for r in rows]}
@router.delete("/api/v2/scim/tokens/{token_id}")
async def revoke_scim_token(token_id: int, request: Request):
admin = _admin_session(request)
with get_conn() as conn:
conn.execute("UPDATE scim_tokens SET revoked=1 WHERE id=?", (token_id,))
conn.commit()
audit_log(admin, "scim.token.revoke", "scim_token", token_id, "", request)
return {"status": "revoked", "id": token_id}
# ── domain claims ──────────────────────────────────────────────────────────
@router.get("/api/v2/domain-claims")
async def list_domains(request: Request):
_admin_session(request)
with get_conn() as conn:
rows = conn.execute("SELECT * FROM domain_claims ORDER BY domain").fetchall()
out = []
for r in rows:
d = dict(r)
d.pop("txt_token", None)
out.append(d)
return {"domains": out}
@router.post("/api/v2/domain-claims")
async def create_domain(request: Request):
admin = _admin_session(request)
try:
body = await request.json()
except Exception:
body = {}
domain = (body.get("domain") or "").strip().lower()
if not domain or "." not in domain or "/" in domain:
raise HTTPException(400, "valid domain required")
token = f"flowdeck-verify={secrets.token_hex(16)}"
with get_conn() as conn:
try:
cur = conn.execute(
"""INSERT INTO domain_claims
(domain, txt_token, auto_join_role, enforce_sso, workspace_id)
VALUES (?,?,?,?,?)""",
(domain, token, body.get("auto_join_role") or "viewer",
1 if body.get("enforce_sso") else 0, body.get("workspace_id")))
conn.commit()
except Exception:
raise HTTPException(409, "Domain already claimed") from None
did = cur.lastrowid
audit_log(admin, "domain.claim", "domain", did, domain, request)
return JSONResponse(status_code=201, content={
"id": did, "domain": domain,
"verify_url": f"https://{domain}/.well-known/flowdeck-verify.txt",
"expected_content": token})
@router.post("/api/v2/domain-claims/{domain_id}/verify")
async def verify_domain(domain_id: int, request: Request):
admin = _admin_session(request)
import httpx
with get_conn() as conn:
row = conn.execute("SELECT * FROM domain_claims WHERE id=?", (domain_id,)).fetchone()
if not row:
raise HTTPException(404, "Domain not found")
claim = dict(row)
url = f"https://{claim['domain']}/.well-known/flowdeck-verify.txt"
try:
async with httpx.AsyncClient(timeout=10, follow_redirects=True) as client:
resp = await client.get(url)
ok = resp.status_code == 200 and claim["txt_token"] in (resp.text or "")
except Exception: # noqa: BLE001 — unreachable domain = not verified
ok = False
if ok:
with get_conn() as conn:
conn.execute("UPDATE domain_claims SET verified=1 WHERE id=?", (domain_id,))
conn.commit()
audit_log(admin, "domain.verify", "domain", domain_id, str(ok), request)
return {"id": domain_id, "verified": ok}
@router.delete("/api/v2/domain-claims/{domain_id}")
async def delete_domain(domain_id: int, request: Request):
admin = _admin_session(request)
with get_conn() as conn:
conn.execute("DELETE FROM domain_claims WHERE id=?", (domain_id,))
conn.commit()
audit_log(admin, "domain.delete", "domain", domain_id, "", request)
return {"status": "deleted", "id": domain_id}
+95
View File
@@ -0,0 +1,95 @@
"""FlowDeck — hybrid search + Ask AI API (v6.9.0).
``GET /api/v2/search/hybrid`` — lexical (FTS5/LIKE) fused with vector cosine
(RRF), workspace-scoped, ACL-filtered, paginated with ``X-Total-Count``.
``POST /api/v2/search/ask`` — RAG answer with ``[[fdpage:ID]]`` citations
(LLM when configured, extractive offline fallback), cached 10 min.
Auth: session cookie first, Bearer fallback (``read`` scope suffices).
See ``docs/V69_Search_Ask_AI.md``.
"""
from __future__ import annotations
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import JSONResponse
from app.auth.session import SessionManager
from app.db import get_conn
from app.services import semantic_search as sem
from app.services.api_v2_helpers import (
audit_log,
has_scope,
paginate_headers,
parse_pagination,
resolve_bearer_token,
)
router = APIRouter(tags=["search-ai"])
def _auth_user(request: Request) -> dict:
sess = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if sess:
return sess
auth = request.headers.get("authorization") or request.headers.get("Authorization") or ""
if auth.lower().startswith("bearer "):
user = resolve_bearer_token(auth[7:].strip())
if not user:
raise HTTPException(401, "Invalid or expired API token")
if not has_scope(user.get("_token_scopes") or "read", "read"):
raise HTTPException(403, "Insufficient scope. Required: read")
return user
raise HTTPException(401, "Authentication required")
@router.get("/api/v2/search/hybrid")
async def hybrid(request: Request):
user = _auth_user(request)
q = (request.query_params.get("q") or request.query_params.get("query") or "").strip()
if not q:
raise HTTPException(400, "q is required")
limit, offset = parse_pagination(request)
ws_raw = request.query_params.get("workspace_id")
workspace_id = int(ws_raw) if ws_raw and str(ws_raw).isdigit() else None
results, _total = sem.hybrid_search(q, user, limit=limit + offset,
workspace_id=workspace_id)
page = results[offset:offset + limit]
# Index-on-read: a fresh page may not be indexed yet (scheduler runs every
# 5 min). Best-effort is handled by tests calling index_resource directly.
resp = JSONResponse({"query": q, "results": page,
"total": len(results), "limit": limit, "offset": offset})
for k, v in paginate_headers(len(results)).items():
resp.headers[k] = v
return resp
@router.post("/api/v2/search/ask")
async def ask_ai(request: Request):
user = _auth_user(request)
try:
body = await request.json()
except Exception:
body = {}
question = (body.get("question") or body.get("q") or "").strip()
if not question:
raise HTTPException(400, "question is required")
ws = body.get("workspace_id")
workspace_id = int(ws) if isinstance(ws, int) or (isinstance(ws, str) and ws.isdigit()) else None
out = await sem.ask(question, user, workspace_id)
audit_log(user, "search.ask", "search", "", question[:200], request)
return {"question": question, "workspace_id": workspace_id, **out}
@router.get("/api/v2/search/index-status")
async def index_status(request: Request):
"""How many resources are indexed vs pending (owner/admin visibility)."""
user = _auth_user(request)
with get_conn() as conn:
indexed = conn.execute("SELECT COUNT(*) FROM semantic_index_state").fetchone()[0]
vectors = conn.execute("SELECT COUNT(*) FROM semantic_embeddings").fetchone()[0]
pages_total = conn.execute(
"SELECT COUNT(*) FROM pages WHERE (deleted_at IS NULL OR deleted_at='') "
"AND COALESCE(search_excluded, 0)=0").fetchone()[0]
return {"indexed_resources": indexed, "vectors": vectors,
"indexable_pages": pages_total, "model": sem.MODEL, "dim": sem.DIM,
"user_id": user.get("id")}
+1 -1
View File
@@ -117,5 +117,5 @@ async def revoke_session(sid: str, request: Request):
try:
request.session.clear()
except Exception:
pass
logger.exception("revoke_session")
return {"status": "revoked"}
+134 -21
View File
@@ -10,6 +10,7 @@ from fastapi import APIRouter, HTTPException, Request
from app.auth.session import SessionManager
from app.db import get_conn
from app.services.automations import fire_event as _fire_event
logger = logging.getLogger(__name__)
router = APIRouter(tags=["sharing"], prefix="/api")
@@ -36,18 +37,23 @@ def _slugify(title: str) -> str:
@router.post("/pages/{page_id}/share")
async def share_page(page_id: int, request: Request):
"""Invite a user or email to a page."""
"""Invite a user, an email, or a group to a page."""
user = _require_auth(request)
body = await request.json() if request.headers.get("content-type") else {}
target_user_id = body.get("user_id")
target_group_id = body.get("group_id")
email = body.get("email", "")
permission = body.get("permission", "view")
if permission not in ("view", "comment", "edit"):
raise HTTPException(400, "Invalid permission. Use view, comment, or edit.")
if not target_user_id and not email:
raise HTTPException(400, "Provide user_id or email to share with.")
if not target_user_id and not target_group_id and not email:
raise HTTPException(400, "Provide user_id, group_id or email to share with.")
# Bridge share permission (view/comment/edit) → granular role
# (viewer/commenter/editor) so page_permissions grants stay in sync.
_SHARE_TO_ROLE = {"view": "viewer", "comment": "commenter", "edit": "editor"}
with get_conn() as conn:
# Verify page exists
@@ -61,18 +67,29 @@ async def share_page(page_id: int, request: Request):
if not target:
raise HTTPException(404, "Target user not found")
# Verify target group exists if group_id given
if target_group_id:
gtarget = conn.execute("SELECT id FROM user_groups WHERE id=?", (target_group_id,)).fetchone()
if not gtarget:
raise HTTPException(404, "Target group not found")
# Upsert to avoid duplicates: update the existing permission if the same
# target (user or email) is already shared on this page.
# target (user, group or email) is already shared on this page.
target_row = None
if target_user_id:
target_row = conn.execute(
"SELECT id FROM page_shares WHERE page_id=? AND shared_with_user_id=?",
(page_id, target_user_id),
).fetchone()
elif target_group_id:
target_row = conn.execute(
"SELECT id FROM page_shares WHERE page_id=? AND shared_with_group_id=?",
(page_id, target_group_id),
).fetchone()
elif email:
target_row = conn.execute(
"""SELECT id FROM page_shares
WHERE page_id=? AND shared_with_email=? AND shared_with_user_id IS NULL""",
WHERE page_id=? AND shared_with_email=? AND shared_with_user_id IS NULL AND shared_with_group_id IS NULL""",
(page_id, email.strip()),
).fetchone()
@@ -85,29 +102,80 @@ async def share_page(page_id: int, request: Request):
else:
if not email:
email = ""
cur = conn.execute(
"""INSERT INTO page_shares (page_id, shared_with_user_id, shared_with_email, permission, created_by)
VALUES (?, ?, ?, ?, ?)""",
(page_id, target_user_id, email.strip(), permission, user["id"]),
)
try:
cur = conn.execute(
"""INSERT INTO page_shares (page_id, shared_with_user_id, shared_with_group_id, shared_with_email, permission, created_by)
VALUES (?, ?, ?, ?, ?, ?)""",
(page_id, target_user_id, target_group_id, email.strip(), permission, user["id"]),
)
except Exception:
# Fallback for DBs where the migration has not run yet
cur = conn.execute(
"""INSERT INTO page_shares (page_id, shared_with_user_id, shared_with_email, permission, created_by)
VALUES (?, ?, ?, ?, ?)""",
(page_id, target_user_id, email.strip(), permission, user["id"]),
)
share_id = cur.lastrowid
conn.execute("UPDATE pages SET is_shared=1 WHERE id=?", (page_id,))
# ── Mirror group shares into page_permissions so the ACL used by
# PermissionManager (can_view/edit/comment) grants real access to
# every group member. Best-effort: never break legacy page_shares.
if target_group_id:
try:
_mirror_share_grant(conn, page_id, target_group_id, _SHARE_TO_ROLE[permission], user["id"])
except Exception:
logger.warning("share→page_permissions mirror failed (page=%s group=%s)", page_id, target_group_id)
conn.commit()
try:
await _fire_event("page.shared", {"page_id": page_id, "share_id": share_id, "permission": permission})
except Exception:
logger.exception("share_page")
return {
"id": share_id,
"page_id": page_id,
"shared_with_user_id": target_user_id,
"shared_with_group_id": target_group_id,
"shared_with_email": email,
"permission": permission,
"status": "shared",
}
def _mirror_share_grant(conn, page_id: int, group_id: int, role: str, granted_by: int) -> None:
"""Upsert a ``page_permissions`` grant mirroring a group ``page_shares`` row.
Keeps the granular ACL (used by ``PermissionManager``) in sync with what
the share dialog shows, so invited groups get effective view/edit rights.
"""
existing = conn.execute(
"SELECT id FROM page_permissions WHERE page_id=? AND user_id IS NULL AND group_id=?",
(page_id, group_id),
).fetchone()
if existing:
conn.execute("UPDATE page_permissions SET role=?, granted_by=? WHERE id=?",
(role, granted_by, existing["id"]))
else:
conn.execute(
"INSERT INTO page_permissions (page_id, user_id, group_id, role, granted_by) "
"VALUES (?, NULL, ?, ?, ?)",
(page_id, group_id, role, granted_by),
)
def _mirror_share_revoke(conn, page_id: int, group_id: int) -> None:
"""Remove the mirrored grant when a group share is updated away or deleted."""
conn.execute(
"DELETE FROM page_permissions WHERE page_id=? AND user_id IS NULL AND group_id=?",
(page_id, group_id),
)
@router.put("/pages/{page_id}/share/{share_id}", description="Update a share's permission.")
async def update_share_permission(page_id: int, share_id: int, request: Request):
"""Change the permission level of an existing share entry."""
_require_auth(request)
user = _require_auth(request)
body = await request.json() if request.headers.get("content-type") else {}
permission = body.get("permission", "")
@@ -117,7 +185,7 @@ async def update_share_permission(page_id: int, share_id: int, request: Request)
with get_conn() as conn:
row = conn.execute(
"SELECT id FROM page_shares WHERE id=? AND page_id=?",
"SELECT id, shared_with_group_id FROM page_shares WHERE id=? AND page_id=?",
(share_id, page_id),
).fetchone()
if not row:
@@ -127,6 +195,18 @@ async def update_share_permission(page_id: int, share_id: int, request: Request)
"UPDATE page_shares SET permission=? WHERE id=?",
(permission, share_id),
)
# Keep the mirrored ACL grant in sync for group shares.
try:
gid = row["shared_with_group_id"] if "shared_with_group_id" in row.keys() else None
except Exception:
gid = None
if gid:
try:
_mirror_share_grant(conn, page_id, gid,
{"view": "viewer", "comment": "commenter", "edit": "editor"}[permission],
user["id"])
except Exception:
logger.warning("share→page_permissions mirror failed (share=%s)", share_id)
conn.commit()
return {"status": "updated", "share_id": share_id, "permission": permission}
@@ -139,13 +219,22 @@ async def remove_share(page_id: int, share_id: int, request: Request):
with get_conn() as conn:
row = conn.execute(
"SELECT id FROM page_shares WHERE id=? AND page_id=?",
"SELECT id, shared_with_group_id FROM page_shares WHERE id=? AND page_id=?",
(share_id, page_id),
).fetchone()
if not row:
raise HTTPException(404, "Share entry not found")
conn.execute("DELETE FROM page_shares WHERE id=?", (share_id,))
try:
gid = row["shared_with_group_id"] if "shared_with_group_id" in row.keys() else None
except Exception:
gid = None
if gid:
try:
_mirror_share_revoke(conn, page_id, gid)
except Exception:
logger.warning("share→page_permissions revoke failed (share=%s)", share_id)
# If no more shares, unset is_shared
remaining = conn.execute(
"SELECT COUNT(*) AS c FROM page_shares WHERE page_id=?", (page_id,)
@@ -167,14 +256,25 @@ async def list_shares(page_id: int, request: Request):
if not page:
raise HTTPException(404, "Page not found")
rows = conn.execute(
"""SELECT s.*, u.login, u.full_name, u.avatar_url
FROM page_shares s
LEFT JOIN users u ON s.shared_with_user_id = u.id
WHERE s.page_id=?
ORDER BY s.created_at DESC""",
(page_id,),
).fetchall()
try:
rows = conn.execute(
"""SELECT s.*, u.login, u.full_name, u.avatar_url, g.name AS group_name
FROM page_shares s
LEFT JOIN users u ON s.shared_with_user_id = u.id
LEFT JOIN user_groups g ON s.shared_with_group_id = g.id
WHERE s.page_id=?
ORDER BY s.created_at DESC""",
(page_id,),
).fetchall()
except Exception:
rows = conn.execute(
"""SELECT s.*, u.login, u.full_name, u.avatar_url
FROM page_shares s
LEFT JOIN users u ON s.shared_with_user_id = u.id
WHERE s.page_id=?
ORDER BY s.created_at DESC""",
(page_id,),
).fetchall()
return {
"page_id": page_id,
@@ -182,6 +282,7 @@ async def list_shares(page_id: int, request: Request):
{
"id": r["id"],
"shared_with_user_id": r["shared_with_user_id"],
"shared_with_group_id": r["shared_with_group_id"] if "shared_with_group_id" in r.keys() else None,
"shared_with_email": r["shared_with_email"],
"permission": r["permission"],
"created_at": r["created_at"],
@@ -189,6 +290,8 @@ async def list_shares(page_id: int, request: Request):
"user_login": r["login"],
"user_full_name": r["full_name"],
"user_avatar_url": r["avatar_url"],
"group_name": r["group_name"] if "group_name" in r.keys() else None,
"kind": "group" if (("shared_with_group_id" in r.keys() and r["shared_with_group_id"]) or ("group_name" in r.keys() and r["group_name"])) else "user",
}
for r in rows
],
@@ -226,6 +329,11 @@ async def publish_page(page_id: int, request: Request):
)
conn.commit()
try:
await _fire_event("page.published", {"page_id": page_id, "slug": slug})
except Exception:
logger.exception("publish_page")
return {
"page_id": page_id,
"is_published": True,
@@ -252,6 +360,11 @@ async def unpublish_page(page_id: int, request: Request):
)
conn.commit()
try:
await _fire_event("page.unpublished", {"page_id": page_id})
except Exception:
logger.exception("unpublish_page")
return {
"page_id": page_id,
"is_published": False,
+4 -3
View File
@@ -27,9 +27,10 @@ DEFAULT_CONFIG = {
"recents": {"visible": True, "order": 3, "show_count": 10},
"favorites": {"visible": True, "order": 4, "show_count": 10},
"agents": {"visible": True, "order": 5, "show_count": None},
"shared": {"visible": True, "order": 6, "show_count": 10},
"published": {"visible": True, "order": 7, "show_count": 10},
"private": {"visible": True, "order": 8, "show_count": None},
"teamspaces": {"visible": True, "order": 6, "show_count": None},
"shared": {"visible": True, "order": 7, "show_count": 10},
"published": {"visible": True, "order": 8, "show_count": 10},
"private": {"visible": True, "order": 9, "show_count": None},
}
+838
View File
@@ -0,0 +1,838 @@
"""FlowDeck — Sites & public Forms (v6.8.0).
Notion Sites + Forms parity: multi-page public sites (/s/<slug>) with nav,
password/expiry gating, SEO + view stats, and anonymous collection forms
(/f/<token>) with rate limiting, validation and notifications.
Auth: session cookie first, Bearer token fallback (api_tokens,
extension_devices, legacy user_tokens) via api_v2_helpers.
"""
from __future__ import annotations
import hashlib
import html
import json
import logging
import re
import secrets
import time
import unicodedata
from datetime import UTC, datetime
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import HTMLResponse, JSONResponse, PlainTextResponse
from app.auth.session import SessionManager
from app.config import settings
from app.db import get_conn
from app.password_utils import hash_password, verify_password
from app.services.api_v2_helpers import (
audit_log,
get_bearer_user,
has_scope,
paginate_headers,
parse_pagination,
resolve_bearer_token,
row_to_dict,
)
logger = logging.getLogger(__name__)
router = APIRouter(tags=["sites"])
_SLUG_RE = re.compile(r"^[a-z0-9-]{3,50}$")
_FORM_TOKEN_RE = re.compile(r"^f_[A-Za-z0-9_-]{6,64}$")
# In-memory rate limiting for anonymous form posts: ip -> (window_start, count).
_form_rate: dict[str, tuple[float, int]] = {}
_FORM_RATE_MAX = 20
_FORM_RATE_WINDOW = 3600.0
# ── helpers ────────────────────────────────────────────────────────────────
def _slugify(title: str) -> str:
slug = unicodedata.normalize("NFKD", title or "").encode("ascii", "ignore").decode("ascii")
slug = re.sub(r"[^\w\s-]", "", slug.lower())
slug = re.sub(r"[-\s]+", "-", slug).strip("-")
return slug or "untitled"
def _check_slug(slug: str) -> None:
if not _SLUG_RE.match(slug or ""):
raise HTTPException(400, "Invalid slug: 3-50 chars, lowercase letters, digits, dashes.")
def _auth_user(request: Request, *, require_write: bool = False) -> dict:
"""Session-first auth, Bearer fallback. Enforces scope for Bearer tokens."""
sess = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if sess:
return sess
auth = request.headers.get("authorization") or request.headers.get("Authorization") or ""
if auth.lower().startswith("bearer "):
user = resolve_bearer_token(auth[7:].strip())
if not user:
raise HTTPException(401, "Invalid or expired API token")
if require_write and not has_scope(user.get("_token_scopes") or "read", "write"):
raise HTTPException(403, "Insufficient scope. Required: write")
return user
raise HTTPException(401, "Authentication required")
def _site_auth_cookie(site_id: int) -> str:
return f"site_auth_{site_id}"
def _site_unlocked(request: Request, site: dict) -> bool:
if not site.get("password_hash"):
return True
from itsdangerous import BadSignature, URLSafeTimedSerializer
ser = URLSafeTimedSerializer(settings.app_secret_key, salt="site-auth")
try:
val = ser.loads(request.cookies.get(_site_auth_cookie(site["id"]), ""), max_age=86400)
return val == site["id"]
except BadSignature:
return False
except Exception:
return False
def _site_expired(site: dict) -> bool:
exp = site.get("expires_at")
if not exp:
return False
try:
dt = datetime.fromisoformat(str(exp).replace("Z", "+00:00"))
if dt.tzinfo is None:
dt = dt.replace(tzinfo=UTC)
return dt.timestamp() < time.time()
except Exception:
return False
def _resolve_site(conn, *, slug: str = "", host: str = "") -> dict | None:
row = None
if slug:
row = conn.execute("SELECT * FROM sites WHERE slug=?", (slug,)).fetchone()
elif host:
row = conn.execute("SELECT * FROM sites WHERE custom_domain=?", (host.split(":")[0],)).fetchone()
return dict(row) if row else None
def _site_pages(conn, site_id: int) -> list[dict]:
rows = conn.execute(
"""SELECT p.id, p.title, p.page_icon, p.cover_url, p.updated_at, sp.position
FROM site_pages sp JOIN pages p ON p.id = sp.page_id
WHERE sp.site_id=? AND (p.deleted_at IS NULL OR p.deleted_at='')
ORDER BY sp.position, p.id""",
(site_id,),
).fetchall()
out = []
for r in rows:
d = dict(r)
d["slug"] = _slugify(d.get("title") or "untitled") or f"page-{d['id']}"
out.append(d)
return out
def _find_site_page(pages: list[dict], ref: str) -> dict | None:
ref = (ref or "").strip()
if ref.isdigit():
for p in pages:
if p["id"] == int(ref):
return p
for p in pages:
if p["slug"] == ref:
return p
# slug with -<id> suffix fallback
m = re.search(r"-(\d+)$", ref)
if m:
for p in pages:
if p["id"] == int(m.group(1)):
return p
return None
def _render_page_html(page: dict) -> str:
"""Render a pages row to HTML (blocks → public renderer, else <pre>)."""
if page.get("content_format") == "blocks" and page.get("content"):
try:
from app.routers import dashboard as _dash
blocks = json.loads(page["content"])
try:
from app.services.synced_blocks import resolve_synced_block
blocks = resolve_synced_block(blocks)
except Exception:
logger.exception("_render_page_html")
titles: dict = {}
try:
from app.db import get_conn as _gc
from app.services.wiki_links import token_labels
with _gc() as _c:
titles = token_labels(_c, page["content"])
except Exception:
titles = {}
return _dash._render_blocks_public(blocks, titles)
except Exception:
return f"<p>{html.escape(str(page.get('content', '')))}</p>"
if page.get("content"):
return (
"<pre style='white-space:pre-wrap;font-family:system-ui;"
f"font-size:16px;line-height:1.6;'>{html.escape(page['content'])}</pre>"
)
return "<p style='color:#999'>Empty page.</p>"
def _site_shell(*, site: dict, pages: list[dict], current_id: int, title: str,
body_html: str, noindex: bool = False) -> str:
nav = "".join(
f"<a href='/s/{site['slug']}/{p['slug']}'"
f" style='display:block;padding:6px 10px;border-radius:6px;text-decoration:none;"
f"color:{'#fff' if p['id'] == current_id else '#bbb'};"
f"background:{'#333' if p['id'] == current_id else 'transparent'}'>"
f"{html.escape((p.get('page_icon') or '') + ' ' + (p.get('title') or 'Untitled'))}</a>"
for p in pages
)
robots = "noindex,nofollow" if (noindex or site.get("noindex")) else "index,follow"
desc = html.escape((site.get("title") or title)[:160])
theme_bg = "#191919" if site.get("theme", "dark") == "dark" else "#ffffff"
theme_fg = "#e0e0e0" if site.get("theme", "dark") == "dark" else "#222222"
return f"""<!DOCTYPE html><html lang="en"><head><meta charset="UTF-8">
<meta name="viewport" content="width=device-width,initial-scale=1.0">
<meta name="robots" content="{robots}">
<meta name="description" content="{desc}">
<meta property="og:title" content="{html.escape(title)}">
<meta property="og:description" content="{desc}">
<meta name="twitter:card" content="summary">
<title>{html.escape(title)} — {html.escape(site.get('title') or 'FlowDeck Site')}</title>
<style>body{{font-family:system-ui,sans-serif;background:{theme_bg};color:{theme_fg};margin:0}}
.layout{{display:flex;min-height:100vh}}.nav{{width:240px;padding:16px;border-right:1px solid #333}}
.main{{flex:1;padding:32px;max-width:860px}}a{{color:#4c9aff}}
@media(max-width:700px){{.nav{{display:none}}.main{{padding:16px}}}}</style></head>
<body><div class="layout"><nav class="nav">
<a href="/s/{site['slug']}" style="font-weight:700;color:{theme_fg};text-decoration:none">
{html.escape(site.get('title') or 'Site')}</a><div style="height:12px"></div>{nav}</nav>
<main class="main">{body_html}</main></div></body></html>"""
def _track_view(site_id: int) -> None:
day = datetime.now(UTC).strftime("%Y-%m-%d")
try:
with get_conn() as conn:
conn.execute(
"""INSERT INTO site_views (site_id, day, views) VALUES (?, ?, 1)
ON CONFLICT(site_id, day) DO UPDATE SET views=views+1""",
(site_id, day),
)
conn.commit()
except Exception:
logger.exception("_track_view")
def _form_config(conn, collection_id: int) -> dict:
row = conn.execute(
"SELECT id, name, form_config_json FROM collections WHERE id=?", (collection_id,)
).fetchone()
if not row:
raise HTTPException(404, "Collection not found")
try:
cfg = json.loads(row["form_config_json"] or "{}")
except Exception:
cfg = {}
return {"id": row["id"], "name": row["name"], "config": cfg}
def _check_form_rate(ip: str) -> None:
now = time.time()
start, count = _form_rate.get(ip, (now, 0))
if now - start > _FORM_RATE_WINDOW:
_form_rate[ip] = (now, 1)
return
if count >= _FORM_RATE_MAX:
raise HTTPException(429, "Too many submissions. Try again later.")
_form_rate[ip] = (start, count + 1)
# ── Sites CRUD (session or Bearer) ─────────────────────────────────────────
@router.post("/api/v2/sites")
async def create_site(request: Request):
user = _auth_user(request, require_write=True)
try:
body = await request.json()
except Exception:
body = {}
root_page_id = body.get("root_page_id")
if not root_page_id:
raise HTTPException(400, "root_page_id is required")
slug = (body.get("slug") or "").strip().lower() or None
with get_conn() as conn:
page = conn.execute("SELECT id, title FROM pages WHERE id=?", (root_page_id,)).fetchone()
if not page:
raise HTTPException(404, "Root page not found")
if not slug:
slug = _slugify(page["title"])
base, i = slug, 1
while conn.execute("SELECT id FROM sites WHERE slug=?", (slug,)).fetchone():
slug = f"{base}-{i}"
i += 1
else:
_check_slug(slug)
if conn.execute("SELECT id FROM sites WHERE slug=?", (slug,)).fetchone():
raise HTTPException(409, "Slug already taken")
theme = body.get("theme", "dark")
if theme not in ("light", "dark"):
raise HTTPException(400, "theme must be light or dark")
custom_domain = (body.get("custom_domain") or "").strip() or None
if custom_domain and conn.execute(
"SELECT id FROM sites WHERE custom_domain=?", (custom_domain,)
).fetchone():
raise HTTPException(409, "Domain already linked to another site")
expires_at = body.get("expires_at")
if expires_at:
try:
datetime.fromisoformat(str(expires_at).replace("Z", "+00:00"))
except Exception:
raise HTTPException(400, "Invalid expires_at (use ISO-8601)") from None
cur = conn.execute(
"""INSERT INTO sites (slug, root_page_id, title, theme, custom_domain,
expires_at, noindex, analytics_id, created_by)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)""",
(slug, root_page_id, body.get("title") or page["title"],
theme, custom_domain, expires_at,
1 if body.get("noindex") else 0,
(body.get("analytics_id") or "")[:120], user["id"]),
)
site_id = cur.lastrowid
conn.execute(
"INSERT OR IGNORE INTO site_pages (site_id, page_id, position) VALUES (?, ?, 0)",
(site_id, root_page_id),
)
conn.commit()
site = conn.execute("SELECT * FROM sites WHERE id=?", (site_id,)).fetchone()
audit_log(user, "site.create", "site", site_id, f"slug={slug}", request)
return JSONResponse(status_code=201, content=row_to_dict(site))
@router.get("/api/v2/sites")
async def list_sites(request: Request):
user = _auth_user(request)
limit, offset = parse_pagination(request)
with get_conn() as conn:
total = conn.execute(
"SELECT COUNT(*) FROM sites WHERE created_by=?", (user["id"],)
).fetchone()[0]
rows = conn.execute(
"SELECT * FROM sites WHERE created_by=? ORDER BY id DESC LIMIT ? OFFSET ?",
(user["id"], limit, offset),
).fetchall()
resp = JSONResponse([row_to_dict(r) for r in rows])
for k, v in paginate_headers(total).items():
resp.headers[k] = v
return resp
@router.get("/api/v2/sites/{site_id}")
async def get_site(site_id: int, request: Request):
user = _auth_user(request)
with get_conn() as conn:
row = conn.execute("SELECT * FROM sites WHERE id=?", (site_id,)).fetchone()
if not row:
raise HTTPException(404, "Site not found")
site = dict(row)
if site.get("created_by") != user["id"] and not user.get("is_admin"):
raise HTTPException(404, "Site not found")
pages = _site_pages(conn, site_id)
out = row_to_dict(row)
out["pages"] = pages
return out
@router.patch("/api/v2/sites/{site_id}")
async def update_site(site_id: int, request: Request):
user = _auth_user(request, require_write=True)
try:
body = await request.json()
except Exception:
body = {}
with get_conn() as conn:
row = conn.execute("SELECT * FROM sites WHERE id=?", (site_id,)).fetchone()
if not row:
raise HTTPException(404, "Site not found")
site = dict(row)
if site.get("created_by") != user["id"] and not user.get("is_admin"):
raise HTTPException(404, "Site not found")
updates: dict = {}
if "title" in body:
updates["title"] = str(body["title"] or "")[:200]
if "theme" in body:
if body["theme"] not in ("light", "dark"):
raise HTTPException(400, "theme must be light or dark")
updates["theme"] = body["theme"]
if "slug" in body and body["slug"] != site["slug"]:
_check_slug(str(body["slug"]).lower())
if conn.execute(
"SELECT id FROM sites WHERE slug=? AND id!=?", (body["slug"].lower(), site_id)
).fetchone():
raise HTTPException(409, "Slug already taken")
updates["slug"] = str(body["slug"]).lower()
if "custom_domain" in body:
dom = (body["custom_domain"] or "").strip() or None
if dom and conn.execute(
"SELECT id FROM sites WHERE custom_domain=? AND id!=?", (dom, site_id)
).fetchone():
raise HTTPException(409, "Domain already linked to another site")
updates["custom_domain"] = dom
if "expires_at" in body:
if body["expires_at"]:
try:
datetime.fromisoformat(str(body["expires_at"]).replace("Z", "+00:00"))
except Exception:
raise HTTPException(400, "Invalid expires_at (use ISO-8601)") from None
updates["expires_at"] = body["expires_at"]
if "noindex" in body:
updates["noindex"] = 1 if body["noindex"] else 0
if "analytics_id" in body:
updates["analytics_id"] = str(body["analytics_id"] or "")[:120]
if "password" in body:
updates["password_hash"] = hash_password(str(body["password"])) if body["password"] else ""
if updates:
updates["updated_at"] = datetime.now(UTC).strftime("%Y-%m-%d %H:%M:%S")
sets = ", ".join(f"{k}=?" for k in updates)
conn.execute(f"UPDATE sites SET {sets} WHERE id=?", (*updates.values(), site_id))
conn.commit()
site = conn.execute("SELECT * FROM sites WHERE id=?", (site_id,)).fetchone()
audit_log(user, "site.update", "site", site_id, ",".join(updates), request)
return row_to_dict(site)
@router.delete("/api/v2/sites/{site_id}")
async def delete_site(site_id: int, request: Request):
user = _auth_user(request, require_write=True)
with get_conn() as conn:
row = conn.execute("SELECT * FROM sites WHERE id=?", (site_id,)).fetchone()
if not row:
raise HTTPException(404, "Site not found")
if row["created_by"] != user["id"] and not user.get("is_admin"):
raise HTTPException(404, "Site not found")
conn.execute("DELETE FROM sites WHERE id=?", (site_id,))
conn.commit()
audit_log(user, "site.delete", "site", site_id, "", request)
return {"status": "deleted", "id": site_id}
@router.get("/api/v2/sites/{site_id}/pages")
async def list_site_pages(site_id: int, request: Request):
user = _auth_user(request)
with get_conn() as conn:
row = conn.execute("SELECT * FROM sites WHERE id=?", (site_id,)).fetchone()
if not row or (row["created_by"] != user["id"] and not user.get("is_admin")):
raise HTTPException(404, "Site not found")
return {"site_id": site_id, "pages": _site_pages(conn, site_id)}
@router.post("/api/v2/sites/{site_id}/pages")
async def add_site_page(site_id: int, request: Request):
user = _auth_user(request, require_write=True)
try:
body = await request.json()
except Exception:
body = {}
page_id = body.get("page_id")
if not page_id:
raise HTTPException(400, "page_id is required")
with get_conn() as conn:
row = conn.execute("SELECT * FROM sites WHERE id=?", (site_id,)).fetchone()
if not row or (row["created_by"] != user["id"] and not user.get("is_admin")):
raise HTTPException(404, "Site not found")
if not conn.execute("SELECT id FROM pages WHERE id=?", (page_id,)).fetchone():
raise HTTPException(404, "Page not found")
pos = conn.execute(
"SELECT COALESCE(MAX(position), -1)+1 FROM site_pages WHERE site_id=?", (site_id,)
).fetchone()[0]
conn.execute(
"INSERT OR IGNORE INTO site_pages (site_id, page_id, position) VALUES (?, ?, ?)",
(site_id, page_id, pos),
)
conn.commit()
pages = _site_pages(conn, site_id)
audit_log(user, "site.page.add", "site", site_id, f"page={page_id}", request)
return {"site_id": site_id, "pages": pages}
@router.delete("/api/v2/sites/{site_id}/pages/{page_id}")
async def remove_site_page(site_id: int, page_id: int, request: Request):
user = _auth_user(request, require_write=True)
with get_conn() as conn:
row = conn.execute("SELECT * FROM sites WHERE id=?", (site_id,)).fetchone()
if not row or (row["created_by"] != user["id"] and not user.get("is_admin")):
raise HTTPException(404, "Site not found")
if page_id == row["root_page_id"]:
raise HTTPException(400, "Cannot remove the root page")
conn.execute(
"DELETE FROM site_pages WHERE site_id=? AND page_id=?", (site_id, page_id)
)
conn.commit()
audit_log(user, "site.page.remove", "site", site_id, f"page={page_id}", request)
return {"status": "removed", "site_id": site_id, "page_id": page_id}
@router.get("/api/v2/sites/{site_id}/stats")
async def site_stats(site_id: int, request: Request, days: int = 30):
user = _auth_user(request)
days = max(1, min(int(days or 30), 365))
with get_conn() as conn:
row = conn.execute("SELECT * FROM sites WHERE id=?", (site_id,)).fetchone()
if not row or (row["created_by"] != user["id"] and not user.get("is_admin")):
raise HTTPException(404, "Site not found")
rows = conn.execute(
"SELECT day, views FROM site_views WHERE site_id=? ORDER BY day DESC LIMIT ?",
(site_id, days),
).fetchall()
total = conn.execute(
"SELECT COALESCE(SUM(views), 0) FROM site_views WHERE site_id=?", (site_id,)
).fetchone()[0]
return {"site_id": site_id, "total_views": total,
"days": [{"day": r["day"], "views": r["views"]} for r in rows]}
# ── Public site rendering ──────────────────────────────────────────────────
def _public_guard(site: dict, request: Request):
if _site_expired(site):
return HTMLResponse("<h1>410 — Site expired.</h1>", status_code=410)
if site.get("password_hash") and not _site_unlocked(request, site):
return HTMLResponse(
f"""<!DOCTYPE html><html><body style="font-family:system-ui;background:#191919;color:#eee;
display:flex;align-items:center;justify-content:center;height:100vh">
<form method="post" action="/s/{site['slug']}/auth">
<h2>🔒 {html.escape(site.get('title') or 'Protected site')}</h2>
<input type="password" name="password" placeholder="Password"
style="padding:8px;border-radius:6px;border:1px solid #444;background:#222;color:#fff">
<button style="padding:8px 14px;border-radius:6px">Unlock</button></form></body></html>""",
status_code=401,
)
return None
@router.get("/s/{slug}", response_class=HTMLResponse)
async def public_site_home(request: Request, slug: str):
with get_conn() as conn:
site = _resolve_site(conn, slug=slug,
host=request.headers.get("host", ""))
if not site:
return HTMLResponse("<h1>404 — Site not found.</h1>", status_code=404)
guard = _public_guard(site, request)
if guard:
return guard
pages = _site_pages(conn, site["id"])
page = conn.execute("SELECT * FROM pages WHERE id=?", (site["root_page_id"],)).fetchone()
if not page:
return HTMLResponse("<h1>404 — Root page removed.</h1>", status_code=404)
page = dict(page)
_track_view(site["id"])
body = f"<h1>{html.escape(page.get('title') or 'Untitled')}</h1>" + _render_page_html(page)
return _site_shell(site=site, pages=pages, current_id=page["id"],
title=page.get("title") or "Untitled", body_html=body)
@router.get("/s/{slug}/sitemap.xml", response_class=PlainTextResponse)
async def site_sitemap(request: Request, slug: str):
with get_conn() as conn:
site = _resolve_site(conn, slug=slug)
if not site or _site_expired(site) or site.get("password_hash"):
return PlainTextResponse("Not found", status_code=404)
pages = _site_pages(conn, site["id"])
base = str(request.base_url).rstrip("/")
urls = [f"<url><loc>{base}/s/{slug}</loc></url>"] + [
f"<url><loc>{base}/s/{slug}/{p['slug']}</loc></url>" for p in pages
]
return PlainTextResponse(
"<?xml version='1.0' encoding='UTF-8'?>"
"<urlset xmlns='http://www.sitemaps.org/schemas/sitemap/0.9'>"
f"{''.join(urls)}</urlset>",
media_type="application/xml",
)
@router.get("/s/{slug}/{page_ref}", response_class=HTMLResponse)
async def public_site_page(request: Request, slug: str, page_ref: str):
with get_conn() as conn:
site = _resolve_site(conn, slug=slug, host=request.headers.get("host", ""))
if not site:
return HTMLResponse("<h1>404 — Site not found.</h1>", status_code=404)
guard = _public_guard(site, request)
if guard:
return guard
pages = _site_pages(conn, site["id"])
target = _find_site_page(pages, page_ref)
if not target:
return HTMLResponse("<h1>404 — Page not in this site.</h1>", status_code=404)
page = conn.execute("SELECT * FROM pages WHERE id=?", (target["id"],)).fetchone()
if not page:
return HTMLResponse("<h1>404 — Page removed.</h1>", status_code=404)
page = dict(page)
_track_view(site["id"])
body = f"<h1>{html.escape(page.get('title') or 'Untitled')}</h1>" + _render_page_html(page)
return _site_shell(site=site, pages=pages, current_id=page["id"],
title=page.get("title") or "Untitled", body_html=body)
@router.post("/s/{slug}/auth")
async def public_site_auth(request: Request, slug: str):
with get_conn() as conn:
site = _resolve_site(conn, slug=slug)
if not site:
return JSONResponse({"detail": "Site not found"}, status_code=404)
if not site.get("password_hash"):
return {"status": "public"}
ctype = request.headers.get("content-type", "")
password = ""
if "application/json" in ctype:
try:
password = (await request.json()).get("password", "")
except Exception:
logger.exception("public_site_auth")
password = ""
else:
try:
form = await request.form()
password = form.get("password", "")
except Exception:
logger.exception("public_site_auth")
password = ""
if not verify_password(password or "", site["password_hash"] or ""):
raise HTTPException(401, "Wrong password")
from itsdangerous import URLSafeTimedSerializer
ser = URLSafeTimedSerializer(settings.app_secret_key, salt="site-auth")
resp = JSONResponse({"status": "unlocked"})
resp.set_cookie(_site_auth_cookie(site["id"]), ser.dumps(site["id"]),
httponly=True, samesite="lax", max_age=86400, path="/")
return resp
# ── Public Forms ───────────────────────────────────────────────────────────
@router.get("/api/v2/collections/{collection_id}/form")
async def get_form_config(collection_id: int, request: Request):
_auth_user(request)
with get_conn() as conn:
info = _form_config(conn, collection_id)
return {"collection_id": collection_id, "name": info["name"], "form": info["config"]}
@router.put("/api/v2/collections/{collection_id}/form")
async def put_form_config(collection_id: int, request: Request):
user = _auth_user(request, require_write=True)
try:
body = await request.json()
except Exception:
body = {}
with get_conn() as conn:
info = _form_config(conn, collection_id)
cfg = info["config"] if isinstance(info["config"], dict) else {}
if "enabled" in body:
cfg["enabled"] = bool(body["enabled"])
for key in ("title", "success_message"):
if key in body:
cfg[key] = str(body[key] or "")[:300]
for key in ("fields", "required", "notify_user_ids"):
if key in body and isinstance(body[key], list):
cfg[key] = body[key][:50]
if "public_token" in body and body["public_token"]:
tok = str(body["public_token"])
if not _FORM_TOKEN_RE.match(tok):
raise HTTPException(400, "Invalid public_token (f_ + 6-64 chars)")
cfg["public_token"] = tok
if cfg.get("enabled") and not cfg.get("public_token"):
cfg["public_token"] = "f_" + secrets.token_urlsafe(9)
conn.execute(
"UPDATE collections SET form_config_json=? WHERE id=?",
(json.dumps(cfg), collection_id),
)
conn.commit()
audit_log(user, "form.config", "collection", collection_id, "", request)
return {"collection_id": collection_id, "form": cfg}
def _collection_props(conn, collection_id: int) -> list[dict]:
return [dict(r) for r in conn.execute(
"SELECT * FROM collection_properties WHERE collection_id=? ORDER BY position",
(collection_id,)).fetchall()]
@router.get("/f/{token}", response_class=HTMLResponse)
async def public_form(request: Request, token: str):
embed = request.query_params.get("embed") == "1"
with get_conn() as conn:
row = conn.execute("SELECT * FROM collections").fetchone()
target = None
if _FORM_TOKEN_RE.match(token or ""):
for c in conn.execute("SELECT id, name, form_config_json FROM collections").fetchall():
try:
cfg = json.loads(c["form_config_json"] or "{}")
except Exception:
continue
if cfg.get("enabled") and cfg.get("public_token") == token:
target = (c, cfg)
break
_ = row
if not target:
return HTMLResponse("<h1>404 — Form not found.</h1>", status_code=404)
coll, cfg = target
props = _collection_props(conn, coll["id"])
fields = cfg.get("fields") or [p["name"] for p in props if p["prop_type"] != "formula"][:10]
required = set(cfg.get("required") or [])
inputs = ""
for name in fields:
prop = next((p for p in props if p["name"] == name), None)
ptype = (prop or {}).get("prop_type", "text")
itype = {"number": "number", "email": "email", "url": "url",
"date": "date", "phone": "tel"}.get(ptype, "text")
req = "required" if name in required else ""
if ptype in ("select", "status") and prop:
try:
opts = json.loads(prop.get("options_json") or "[]")
except Exception:
opts = []
opts_html = "".join(
f"<option>{html.escape(o.get('name', ''))}</option>" for o in opts)
inputs += (f"<label>{html.escape(name)}{'*' if req else ''}</label>"
f"<select name='{html.escape(name)}' {req}>{opts_html}</select>")
elif ptype == "checkbox":
inputs += (f"<label><input type='checkbox' name='{html.escape(name)}'> "
f"{html.escape(name)}</label>")
else:
inputs += (f"<label>{html.escape(name)}{'*' if req else ''}</label>"
f"<input type='{itype}' name='{html.escape(name)}' {req}>")
chrome = "" if embed else f"<h1>{html.escape(cfg.get('title') or coll['name'])}</h1>"
return HTMLResponse(
f"""<!DOCTYPE html><html><head><meta charset="UTF-8">
<meta name="viewport" content="width=device-width,initial-scale=1.0">
<title>{html.escape(cfg.get('title') or coll['name'])}</title>
<style>body{{font-family:system-ui;background:#191919;color:#eee;margin:0;padding:24px}}
form{{max-width:520px;margin:auto}}label{{display:block;margin:12px 0 4px}}
input,select,textarea{{width:100%;padding:8px;border-radius:6px;border:1px solid #444;background:#222;color:#fff}}
button{{margin-top:16px;padding:10px 18px;border-radius:6px;border:0;background:#2383E2;color:#fff}}</style>
</head><body>{chrome}
<form method="post" action="/f/{token}">
<input type="text" name="__hp" style="display:none" tabindex="-1" autocomplete="off">
{inputs}<button>Submit</button></form></body></html>"""
)
@router.post("/f/{token}")
async def submit_form(request: Request, token: str):
ip = request.client.host if request.client else "unknown"
_check_form_rate(ip or "unknown")
ctype = request.headers.get("content-type", "")
data: dict = {}
if "application/json" in ctype:
try:
data = await request.json()
except Exception:
data = {}
else:
try:
form = await request.form()
data = dict(form)
except Exception:
data = {}
if data.get("__hp"):
raise HTTPException(400, "Spam detected")
with get_conn() as conn:
target = None
if _FORM_TOKEN_RE.match(token or ""):
for c in conn.execute("SELECT id, name, form_config_json FROM collections").fetchall():
try:
cfg = json.loads(c["form_config_json"] or "{}")
except Exception:
continue
if cfg.get("enabled") and cfg.get("public_token") == token:
target = (c, cfg)
break
if not target:
# NOTE: return (not raise) — the global 404 handler redirects
# non-/api paths to /workspaces, which would turn this into a 200.
return JSONResponse({"detail": "Form not found"}, status_code=404)
coll, cfg = target
props = _collection_props(conn, coll["id"])
by_name = {p["name"]: p for p in props}
fields = cfg.get("fields") or list(by_name)[:10]
required = set(cfg.get("required") or [])
values: dict = {}
for name in fields:
prop = by_name.get(name)
if not prop:
continue
raw = data.get(name, "")
if prop["prop_type"] == "checkbox":
raw = True if raw in (True, "on", "true", "1", "checked") else False
if name in required and (raw is None or raw == "" or raw is False):
raise HTTPException(400, f"Field required: {name}")
values[str(prop["id"])] = raw
# Validate via property_types.validate_property_rule
try:
from app.services.property_types import validate_property_rule
for name in fields:
prop = by_name.get(name)
if not prop:
continue
ok, _msg = validate_property_rule(
prop.get("prop_type", "text"), values.get(str(prop["id"])),
prop.get("validation_json") or prop.get("options_json") or "")
if not ok:
raise HTTPException(400, f"Invalid value for {name}: {_msg}")
except HTTPException:
raise
except Exception:
logger.exception("submit_form")
title = str(data.get(by_name[fields[0]]["name"], "Form response") if fields else "Form response")[:200]
cur = conn.execute(
"""INSERT INTO collection_pages (collection_id, title, property_values_json)
VALUES (?, ?, ?)""",
(coll["id"], title or "Form response", json.dumps(values)),
)
row_id = cur.lastrowid
ip_hash = hashlib.sha256(f"{ip}|{datetime.now(UTC).strftime('%Y-%m-%d')}".encode()).hexdigest()
conn.execute(
"INSERT INTO form_responses (collection_id, row_id, ip_hash) VALUES (?, ?, ?)",
(coll["id"], row_id, ip_hash),
)
conn.commit()
notify_ids = cfg.get("notify_user_ids") or []
# Notify (never throws the submission)
try:
from app.services.notifications import create_notification
for uid in notify_ids[:20]:
try:
create_notification(int(uid), None, "form_response",
f"New response: {coll['name']}",
f"{title}", "collection", coll["id"],
f"/db/{coll['id']}")
except Exception:
continue
except Exception:
logger.exception("submit_form")
try:
from app.services.automations import fire_event as _fire
await _fire("form.submitted", {"collection_id": coll["id"], "row_id": row_id})
except Exception:
logger.exception("submit_form")
if "application/json" in ctype:
return {"status": "ok", "row_id": row_id,
"message": cfg.get("success_message") or "Merci !"}
return HTMLResponse(
f"""<!DOCTYPE html><html><body style="font-family:system-ui;background:#191919;color:#eee;
display:flex;align-items:center;justify-content:center;height:100vh">
<p>{html.escape(cfg.get('success_message') or 'Merci !')}</p></body></html>"""
)
# used by tests to reset the anonymous rate limiter
def _reset_form_rate() -> None:
_form_rate.clear()
# Backwards-compat alias for tests importing ``get_bearer_user`` from here.
__all__ = ["router", "get_bearer_user"]
+656
View File
@@ -0,0 +1,656 @@
"""FlowDeck — v6.7.0 SSO: SAML 2.0 + OIDC endpoints and admin config API.
Two families of routes:
* ``/auth/saml/*`` and ``/auth/oidc/*`` — the browser flows (login redirect,
ACS callback, SP metadata, Single Logout). The callback endpoints are
CSRF-exempt (cross-site POST from the IdP) and instead protected by the
single-use ``sso_requests`` relay token + full assertion validation.
* ``/api/v2/sso/*`` — admin configuration API (session admin or Bearer token
with write scope), consumed by Settings → Admin → SSO / Enterprise.
Every attempt — success or rejection — lands in ``sso_login_history``.
"""
from __future__ import annotations
import logging
import secrets
import time
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import HTMLResponse, RedirectResponse
from app.auth.providers import oidc_provider, saml_provider
from app.auth.session import SessionManager
from app.services import sso_provisioning as sso
from app.services.api_v2_helpers import has_scope, resolve_bearer_token
logger = logging.getLogger(__name__)
router = APIRouter(tags=["sso"])
DEFAULT_NEXT = "/workspaces"
# ── Rate limiting (design §5.2: 5 SSO attempts / minute / IP) ──────────────
_RATE_WINDOW = 60.0
_RATE_MAX = 5
_rate_store: dict[str, tuple[float, int]] = {}
def _rate_ok(request: Request, bucket: str = "sso") -> bool:
from app.config import settings
if not settings.rate_limit_enabled:
return True
ip = request.client.host if request.client else "unknown"
key = f"{bucket}:{ip}"
now = time.time()
window, count = _rate_store.get(key, (0.0, 0))
if now - window > _RATE_WINDOW:
_rate_store[key] = (now, 1)
return True
if count >= _RATE_MAX:
return False
_rate_store[key] = (window, count + 1)
return True
def _page(title: str, body: str, status: int = 200) -> HTMLResponse:
"""Small standalone error/info page (same styling as the login page)."""
return HTMLResponse(
f"""<!DOCTYPE html><html lang="en"><head><meta charset="UTF-8">
<title>FlowDeck — {title}</title><style>
*{{margin:0;padding:0;box-sizing:border-box}}
body{{background:#191919;color:#fff;font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',sans-serif;
display:flex;align-items:center;justify-content:center;min-height:100vh;}}
.box{{background:#222;border:1px solid rgba(255,255,255,.08);border-radius:12px;padding:40px;max-width:460px;text-align:center;}}
h1{{font-size:20px;margin-bottom:12px}}p{{color:rgba(255,255,255,.55);font-size:14px;margin-bottom:10px;line-height:1.5;word-break:break-word}}
a{{color:#2383E2;font-size:14px;text-decoration:none}}a:hover{{text-decoration:underline}}
</style></head><body><div class="box"><h1>{title}</h1>{body}</div></body></html>""",
status_code=status,
)
def _sso_config_or_error() -> dict | None:
cfg = sso.get_sso_config()
return sso.normalize_config(cfg) if cfg else None
def _session_cookie(user_data: dict, request: Request):
"""Signed, revocable session cookie (same shape as local/OAuth logins)."""
return SessionManager.create_session(user_data, request)
def _login_error(message: str, *, cfg: dict | None, identifier: str = "", request=None) -> HTMLResponse:
provider_type = (cfg or {}).get("provider_type", "saml")
sso.log_sso_login(
user_id=None,
provider_type=provider_type,
provider_name=(cfg or {}).get("name") or "SSO",
identifier=identifier,
request=request,
success=False,
error=message,
)
logger.warning("SSO login rejected: %s", message)
safe = (
message.replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;")[:400]
)
return _page(
"SSO sign-in failed",
f"<p>{safe}</p><p><a href=\"/auth/login?provider=local\">↩ Back to login</a></p>",
status=403,
)
# ═══════════════════════════════ SAML 2.0 ════════════════════════════════
@router.get("/auth/saml/login")
async def saml_login(request: Request, next: str = DEFAULT_NEXT):
"""SP-initiated SSO: issue an AuthnRequest and redirect to the IdP."""
if not _rate_ok(request, "saml"):
return _page("Too many attempts", "<p>Please wait a minute and try again.</p>", status=429)
cfg = _sso_config_or_error()
if not cfg or cfg["provider_type"] != "saml":
return _page(
"SAML not configured",
"<p>Single Sign-On has not been set up by the server administrator.</p>"
"<p><a href=\"/auth/login?provider=local\">↩ Use local login</a></p>",
status=404,
)
cfg = sso.ensure_sp_keypair(cfg)
# RelayState = "<AuthnRequest id>.<CSRF token>" — both checked at the ACS.
csrf_token = secrets.token_hex(16)
# The id is only known after building the request, so build it first with a
# placeholder relay state, then re-issue with the real one? python3-saml
# builds the AuthnRequest inside login(); we instead create the row right
# after login() returns the URL — but the RelayState is already embedded.
# So: generate the request id ourselves is not possible → build the URL,
# then patch the RelayState by rebuilding with the known id.
from urllib.parse import parse_qs, urlencode, urlparse
provisional = saml_provider.create_login(request, cfg, relay_state="_pending_")
authn_id = provisional[1]
relay = f"{authn_id}.{csrf_token}"
sso.create_request(
"saml_authn",
request_id=authn_id,
relay_state=csrf_token,
next_path=sso.safe_next_path(next),
)
# Replace the placeholder RelayState with the real token (same SAMLRequest).
parsed = urlparse(provisional[0])
params = parse_qs(parsed.query)
params["RelayState"] = [relay]
flat = [(k, v) for k, values in params.items() for v in values]
url = f"{parsed.scheme}://{parsed.netloc}{parsed.path}?{urlencode(flat)}"
return RedirectResponse(url, status_code=302)
@router.post("/auth/saml/callback")
async def saml_callback(request: Request):
"""Assertion Consumer Service — validate the SAMLResponse and open a session."""
if not _rate_ok(request, "saml-cb"):
return _page("Too many attempts", "<p>Please wait a minute and try again.</p>", status=429)
form = await request.form()
saml_response = str(form.get("SAMLResponse") or "")
relay_state = str(form.get("RelayState") or "")
if not saml_response:
return _login_error("Missing SAMLResponse", cfg=None, request=request)
cfg = _sso_config_or_error()
if not cfg or cfg["provider_type"] != "saml":
return _login_error("SAML is not configured", cfg=None, request=request)
authn_id, _, csrf_token = relay_state.partition(".")
pending = sso.peek_request("saml_authn", authn_id)
if not pending and sso.was_consumed("saml_authn", authn_id):
# Same assertion twice: the single-use row is already spent.
return _login_error(
"Replayed SAML response rejected", cfg=cfg, identifier=authn_id, request=request
)
if not pending or not csrf_token or not secrets.compare_digest(
pending.get("relay_state", ""), csrf_token
):
return _login_error(
"Unknown or expired login request (start again from the login page)",
cfg=cfg, request=request,
)
try:
identity = saml_provider.process_response(
request, cfg, {"SAMLResponse": saml_response, "RelayState": relay_state}, authn_id
)
except saml_provider.SAMLError as err:
return _login_error(str(err), cfg=cfg, identifier=authn_id, request=request)
# Single-use: the same AuthnRequest id can never authenticate twice.
consumed = sso.consume_request("saml_authn", authn_id, csrf_token)
if not consumed:
return _login_error(
"Replayed SAML response rejected", cfg=cfg, identifier=authn_id, request=request
)
claims = sso.identity_from_saml(identity, cfg)
identifier = sso.sso_identifier_field(claims)
try:
user = sso.handle_sso_login(claims, provider_type="saml", cfg=cfg, request=request)
except sso.SSOProvisioningError as err:
# _login_error() below records the failed attempt itself.
return _login_error(str(err), cfg=cfg, identifier=identifier, request=request)
sso.log_sso_login(
user_id=user["id"], provider_type="saml",
provider_name=cfg.get("name") or "SSO", identifier=identifier,
request=request, success=True,
)
user_data = dict(user)
user_data["_sso_name_id"] = identity.name_id
user_data["_sso_session_index"] = identity.session_index
response = RedirectResponse(consumed.get("next_path") or DEFAULT_NEXT, status_code=302)
response.set_cookie(
"flowdeck_session", _session_cookie(user_data, request),
httponly=True, max_age=86400 * 7, samesite="lax", path="/",
)
return response
@router.get("/auth/saml/metadata")
async def saml_metadata(request: Request):
"""SP metadata XML — paste into the IdP (Azure AD / Okta / Keycloak…)."""
cfg = _sso_config_or_error()
if not cfg or cfg["provider_type"] != "saml":
return _page("SAML not configured", "<p>No SAML configuration found.</p>", status=404)
cfg = sso.ensure_sp_keypair(cfg)
try:
xml = saml_provider.metadata_xml(request, cfg)
except saml_provider.SAMLError as err:
return _page("Metadata error", f"<p>{err}</p>", status=500)
return HTMLResponse(xml, media_type="application/samlmetadata+xml")
async def _saml_logout(request: Request, next: str = "/auth/login?provider=local"):
"""Single Logout: SP-initiated (our logout button) or IdP-initiated.
* no SAML payload → build a LogoutRequest to the IdP (after revoking the
local session);
* ``SAMLRequest`` / ``SAMLResponse`` present → process it (LogoutResponse
of our own SLO, or a LogoutRequest issued by the IdP).
"""
form = dict(await request.form()) if request.method == "POST" else {}
query = dict(request.query_params)
cfg = _sso_config_or_error()
if not cfg or cfg["provider_type"] != "saml":
response = RedirectResponse(next, status_code=302)
response.delete_cookie("flowdeck_session")
return response
payload = form.get("SAMLRequest") or form.get("SAMLResponse") or query.get("SAMLResponse")
if payload:
try:
url, errors = saml_provider.process_slo_form(request, cfg, form, query)
except saml_provider.SAMLError as err:
logger.warning("SLO processing failed: %s", err)
return _login_error(str(err), cfg=cfg, request=request)
if errors:
return _login_error(
"; ".join(errors)[:300], cfg=cfg, request=request
)
response = RedirectResponse(url or next, status_code=302)
response.delete_cookie("flowdeck_session")
return response
# SP-initiated
cookie = request.cookies.get("flowdeck_session", "")
user = SessionManager.decode_session(cookie) if cookie else None
response = RedirectResponse(next, status_code=302)
if cookie:
sid = SessionManager.session_id(cookie)
if sid:
SessionManager.revoke_session(sid)
response.delete_cookie("flowdeck_session")
if user and cfg.get("slo_url") and user.get("_sso_name_id"):
try:
logout_url = saml_provider.build_logout_url(
request, cfg,
return_to=sso.safe_next_path(next),
name_id=user.get("_sso_name_id", ""),
session_index=user.get("_sso_session_index", ""),
)
# Keep the cookie-clearing headers built above: hand the browser
# to the IdP with our local session already dead.
response = RedirectResponse(logout_url, status_code=302)
response.delete_cookie("flowdeck_session")
return response
except saml_provider.SAMLError as err:
logger.warning("SP-initiated SLO failed: %s", err)
return response
@router.get("/auth/saml/logout")
async def saml_logout(request: Request, next: str = "/auth/login?provider=local"):
"""SP-initiated Single Logout (GET) — hands the browser to the IdP."""
return await _saml_logout(request, next)
@router.post("/auth/saml/logout")
async def saml_logout_post(request: Request, next: str = "/auth/login?provider=local"):
"""IdP-initiated Single Logout (POST with SAMLRequest/SAMLResponse)."""
return await _saml_logout(request, next)
# ═════════════════════════════════ OIDC ═══════════════════════════════════
@router.get("/auth/oidc/login")
async def oidc_login(request: Request, next: str = DEFAULT_NEXT):
"""Redirect to the OIDC provider (authorization code + PKCE)."""
if not _rate_ok(request, "oidc"):
return _page("Too many attempts", "<p>Please wait a minute and try again.</p>", status=429)
cfg = _sso_config_or_error()
if not cfg or cfg["provider_type"] != "oidc":
return _page(
"OIDC not configured",
"<p>Single Sign-On has not been set up by the server administrator.</p>"
"<p><a href=\"/auth/login?provider=local\">↩ Use local login</a></p>",
status=404,
)
try:
doc = await oidc_provider.discover(cfg["issuer_url"])
except oidc_provider.OIDCError as err:
return _login_error(str(err), cfg=cfg, request=request)
from app.auth.providers.saml_provider import external_base_url
state = secrets.token_hex(32)
nonce = secrets.token_hex(16)
verifier, challenge = oidc_provider.pkce_pair()
sso.create_request(
"oidc",
request_id=state,
relay_state=nonce,
code_verifier=verifier,
next_path=sso.safe_next_path(next),
)
url = oidc_provider.build_authorize_url(
doc,
client_id=cfg["client_id"],
redirect_uri=f"{external_base_url(request)}/auth/oidc/callback",
scope=cfg.get("scope") or "openid profile email",
state=state,
nonce=nonce,
code_challenge=challenge,
)
return RedirectResponse(url, status_code=302)
async def _oidc_callback(request: Request):
"""OIDC callback: exchange the code, validate the ID token, open a session."""
if not _rate_ok(request, "oidc-cb"):
return _page("Too many attempts", "<p>Please wait a minute and try again.</p>", status=429)
params = dict(request.query_params)
if request.method == "POST":
params.update({k: str(v) for k, v in (await request.form()).items()})
cfg = _sso_config_or_error()
if not cfg or cfg["provider_type"] != "oidc":
return _login_error("OIDC is not configured", cfg=None, request=request)
if params.get("error"):
return _login_error(
f"Provider error: {params.get('error')} {params.get('error_description', '')}".strip(),
cfg=cfg, request=request,
)
code, state = params.get("code", ""), params.get("state", "")
pending = sso.consume_request("oidc", state)
if not code or not pending:
return _login_error(
"Unknown or expired OIDC state (start again from the login page)",
cfg=cfg, request=request,
)
from app.auth.providers.saml_provider import external_base_url
try:
doc = await oidc_provider.discover(cfg["issuer_url"])
tokens = await oidc_provider.exchange_code(
doc,
client_id=cfg["client_id"],
client_secret=sso.client_secret_value(cfg),
code=code,
redirect_uri=f"{external_base_url(request)}/auth/oidc/callback",
code_verifier=pending.get("code_verifier", ""),
)
jwks = await _fetch_jwks(doc)
claims = oidc_provider.validate_id_token(
tokens.get("id_token", ""),
issuer=cfg["issuer_url"],
client_id=cfg["client_id"],
nonce=pending.get("relay_state", ""),
jwks=jwks,
)
userinfo = await oidc_provider.fetch_userinfo(doc, tokens.get("access_token", ""))
except oidc_provider.OIDCError as err:
return _login_error(str(err), cfg=cfg, identifier=state, request=request)
merged = {**claims, **userinfo}
identity = oidc_provider.claims_to_identity(merged, cfg.get("attribute_mapping") or None)
identifier = sso.sso_identifier_field(identity)
try:
user = sso.handle_sso_login(identity, provider_type="oidc", cfg=cfg, request=request)
except sso.SSOProvisioningError as err:
# _login_error() below records the failed attempt itself.
return _login_error(str(err), cfg=cfg, identifier=identifier, request=request)
sso.log_sso_login(
user_id=user["id"], provider_type="oidc",
provider_name=cfg.get("name") or "SSO", identifier=identifier,
request=request, success=True,
)
response = RedirectResponse(pending.get("next_path") or DEFAULT_NEXT, status_code=302)
response.set_cookie(
"flowdeck_session", _session_cookie(dict(user), request),
httponly=True, max_age=86400 * 7, samesite="lax", path="/",
)
return response
@router.get("/auth/oidc/callback")
async def oidc_callback(request: Request):
"""OIDC callback (GET, authorization code in the query string)."""
return await _oidc_callback(request)
@router.post("/auth/oidc/callback")
async def oidc_callback_post(request: Request):
"""OIDC callback (POST, form_post response mode)."""
return await _oidc_callback(request)
async def _fetch_jwks(doc: dict) -> dict:
url = doc.get("jwks_uri")
if not url:
raise oidc_provider.OIDCError("Discovery document has no jwks_uri")
import httpx
try:
async with httpx.AsyncClient(timeout=15) as client:
r = await client.get(url)
r.raise_for_status()
data = r.json()
except Exception as err:
raise oidc_provider.OIDCError(f"Could not fetch the issuer JWKS: {err}") from err
if not isinstance(data, dict) or not data.get("keys"):
raise oidc_provider.OIDCError("Issuer JWKS contains no keys")
return data
async def _oidc_logout(request: Request, next: str = "/auth/login?provider=local"):
"""Local logout + RP-initiated logout at the provider when supported."""
cfg = _sso_config_or_error()
response = RedirectResponse(next, status_code=302)
cookie = request.cookies.get("flowdeck_session", "")
if cookie:
sid = SessionManager.session_id(cookie)
if sid:
SessionManager.revoke_session(sid)
response.delete_cookie("flowdeck_session")
if cfg and cfg["provider_type"] == "oidc":
try:
doc = await oidc_provider.discover(cfg["issuer_url"])
end_session = doc.get("end_session_endpoint")
if end_session:
from urllib.parse import urlencode
from app.auth.providers.saml_provider import external_base_url
qs = urlencode({
"client_id": cfg["client_id"],
"post_logout_redirect_uri": external_base_url(request) + next,
})
sep = "&" if "?" in end_session else "?"
return RedirectResponse(f"{end_session}{sep}{qs}", status_code=302)
except oidc_provider.OIDCError as err:
logger.debug("RP-initiated logout skipped: %s", err)
return response
@router.get("/auth/oidc/logout")
async def oidc_logout(request: Request, next: str = "/auth/login?provider=local"):
"""OIDC logout (GET) — local session first, then the IdP end-session URL."""
return await _oidc_logout(request, next)
@router.post("/auth/oidc/logout")
async def oidc_logout_post(request: Request, next: str = "/auth/login?provider=local"):
"""OIDC logout (POST)."""
return await _oidc_logout(request, next)
# ═══════════════════════ Admin configuration API ══════════════════════════
async def _require_admin(request: Request, *, write: bool) -> dict:
"""Admin identity: Bearer token (scope read/write) or an admin session.
Session-authenticated writes also need the CSRF header — ``/api/v2`` is
exempted in the middleware, so the check lives here for this router.
"""
auth_header = request.headers.get("authorization") or ""
if auth_header.lower().startswith("bearer "):
user = resolve_bearer_token(auth_header[7:].strip())
if not user:
raise HTTPException(status_code=401, detail="Invalid or expired token")
scopes = user.get("_token_scopes") or ""
need = "write" if write else "read"
if not (has_scope(scopes, need) or has_scope(scopes, "admin")):
raise HTTPException(status_code=403, detail=f"Insufficient scope. Required: {need}")
if not user.get("is_admin"):
raise HTTPException(status_code=403, detail="Admin access required")
return user
from app.db import get_conn
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(status_code=401, detail="Not authenticated")
with get_conn() as conn:
row = conn.execute(
"SELECT id, login, full_name, email, is_admin FROM users WHERE id=?",
(user["id"],),
).fetchone()
if not row or not row["is_admin"]:
raise HTTPException(status_code=403, detail="Admin access required")
if write and request.method in ("POST", "PUT", "PATCH", "DELETE"):
cookie = request.cookies.get("csrf_token", "")
header = request.headers.get("X-CSRF-Token", "")
if not cookie or not header or not secrets.compare_digest(cookie, header):
raise HTTPException(status_code=403, detail="CSRF validation failed")
return dict(row)
@router.get("/api/v2/sso/providers")
async def sso_providers(request: Request):
"""Public: what the login page should show (button list + sso_only flag)."""
cfg = _sso_config_or_error()
if not cfg:
return {"providers": [], "sso_only": False}
from app.auth.providers.saml_provider import external_base_url
base = external_base_url(request)
login_path = "/auth/saml/login" if cfg["provider_type"] == "saml" else "/auth/oidc/login"
return {
"providers": [{
"type": cfg["provider_type"],
"name": cfg.get("name") or "Company SSO",
"icon": "🏢",
"login_url": f"{login_path}?next={DEFAULT_NEXT}",
}],
"sso_only": bool(cfg.get("sso_only")),
"base_url": base,
}
@router.get("/api/v2/sso/config")
async def get_sso_config_api(request: Request):
"""Read the current SSO configuration (secrets never returned)."""
await _require_admin(request, write=False)
cfg = _sso_config_or_error()
return sso.public_config_view(cfg)
@router.post("/api/v2/sso/config")
@router.put("/api/v2/sso/config")
async def save_sso_config_api(request: Request):
"""Create/replace the SSO configuration (admin, scope write)."""
admin = await _require_admin(request, write=True)
try:
payload = await request.json()
except Exception as err:
raise HTTPException(status_code=400, detail="Invalid JSON body") from err
try:
saved = sso.save_sso_config(payload, created_by=admin.get("id"))
except sso.SSOConfigError as err:
raise HTTPException(status_code=400, detail=str(err)) from err
from app.services.api_v2_helpers import audit_log
audit_log(admin, "sso.config.save", "sso_config", saved.get("id", 0),
f"provider={saved.get('provider_type')}", request)
return sso.public_config_view(saved)
@router.delete("/api/v2/sso/config")
async def delete_sso_config_api(request: Request):
"""Disable SSO — local logins keep working (design §8 « SSO disable »)."""
admin = await _require_admin(request, write=True)
removed = sso.delete_sso_config()
from app.services.api_v2_helpers import audit_log
audit_log(admin, "sso.config.disable", "sso_config", 0, "", request)
return {"status": "ok", "disabled": removed}
@router.get("/api/v2/sso/workspaces")
async def sso_workspaces(request: Request):
"""Workspaces available for default assignment / group mapping."""
await _require_admin(request, write=False)
from app.db import get_conn
with get_conn() as conn:
rows = conn.execute(
"SELECT id, name, owner_id FROM workspaces ORDER BY name"
).fetchall()
cfg = _sso_config_or_error()
return {
"workspaces": [dict(r) for r in rows],
"default_workspace_id": (cfg or {}).get("default_workspace_id"),
"sso_only": bool((cfg or {}).get("sso_only")),
"provisioned_users": sso.provisioned_count(),
}
@router.post("/api/v2/sso/sync")
async def sso_sync(request: Request):
"""Re-apply group → workspace role mapping for every SSO user."""
admin = await _require_admin(request, write=True)
try:
result = sso.force_sync_all_groups()
except sso.SSOProvisioningError as err:
raise HTTPException(status_code=400, detail=str(err)) from err
from app.services.api_v2_helpers import audit_log
audit_log(admin, "sso.sync", "sso_config", 0, str(result), request)
return {"status": "ok", **result}
@router.get("/api/v2/sso/history")
async def sso_history(request: Request, limit: int = 50):
"""Audit trail of SSO login attempts (successes and rejections)."""
await _require_admin(request, write=False)
from app.db import get_conn
limit = max(1, min(int(limit or 50), 200))
with get_conn() as conn:
rows = conn.execute(
"""SELECT h.id, h.user_id, u.login, h.provider_type, h.provider_name,
h.sso_identifier, h.ip_address, h.success, h.error_message,
h.created_at
FROM sso_login_history h LEFT JOIN users u ON u.id = h.user_id
ORDER BY h.id DESC LIMIT ?""",
(limit,),
).fetchall()
out = []
for r in rows:
d = dict(r)
ident = d.get("sso_identifier") or ""
if "|" in ident: # drop the stored group list from the UI payload
d["sso_identifier"] = ident.split("|", 1)[0]
d["success"] = bool(d["success"])
out.append(d)
return {"history": out}
+108
View File
@@ -0,0 +1,108 @@
"""FlowDeck — /api/v2/sync endpoints (v6.0.0 PWA offline sync, Bearer v6.4.0).
Auth: ``Authorization: Bearer <token>`` (scopes ``read`` for delta/status,
``write`` for batch). The legacy ``flowdeck_session`` cookie is still accepted
as a fallback so the installed PWA/service worker keeps syncing.
"""
from __future__ import annotations
import logging
from fastapi import APIRouter, Header, HTTPException, Query, Request
from fastapi.responses import JSONResponse
from app.auth.session import SessionManager
from app.services.api_v2_helpers import get_bearer_user, has_scope
from app.services.sync_engine import SyncEngine
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v2/sync", tags=["sync"])
_engine = SyncEngine()
def _user(request: Request, authorization: str | None = None,
*, required_scope: str = "read") -> dict:
"""Bearer-first auth with session-cookie fallback (offline.js compat)."""
auth = authorization or request.headers.get("authorization") or ""
if auth and auth.lower().startswith("bearer "):
try:
user = get_bearer_user(request, authorization)
except HTTPException:
raise HTTPException(
status_code=401, detail="Invalid or expired API token"
) from None
if not has_scope(user.get("_token_scopes"), required_scope):
raise HTTPException(
status_code=403,
detail=f"Insufficient scope. Required: {required_scope}",
)
return user
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user:
raise HTTPException(status_code=401, detail="Authentication required")
return user
@router.get("/delta")
async def sync_delta(
request: Request,
since: float = Query(default=0, description="Epoch seconds (ou ms) du dernier sync"),
workspace_id: int = Query(default=None),
authorization: str | None = Header(default=None),
):
"""Pull server-side changes since `since` (for the given workspace)."""
user = _user(request, authorization, required_scope="read")
if workspace_id is None:
raise HTTPException(status_code=400, detail="workspace_id is required")
result = await _engine.get_delta(user["id"], since, workspace_id)
if result.get("error") == "forbidden":
return JSONResponse({"detail": "Forbidden"}, status_code=403)
return result
@router.post("/batch")
async def sync_batch(request: Request, authorization: str | None = Header(default=None)):
"""Apply a batch of offline mutations and return per-mutation results."""
user = _user(request, authorization, required_scope="write")
try:
body = await request.json()
except Exception:
raise HTTPException(status_code=400, detail="Invalid JSON body") from None
mutations = body.get("mutations") or []
device_id = body.get("device_id") or "unknown"
if not isinstance(mutations, list) or not mutations:
return {"results": [], "conflicts": [], "server_time": SyncEngine._now_epoch()}
result = await _engine.apply_batch(user["id"], mutations, device_id)
result["server_time"] = SyncEngine._now_epoch()
return result
@router.get("/status")
async def sync_status(request: Request, workspace_id: int = Query(default=None),
authorization: str | None = Header(default=None)):
"""Synchronization status for the workspace (pending server queue, last sync)."""
user = _user(request, authorization, required_scope="read")
from app.db import get_conn
with get_conn() as conn:
if not SyncEngine._can_access(conn, user["id"], workspace_id):
return JSONResponse({"detail": "Forbidden"}, status_code=403)
pending = conn.execute(
"SELECT COUNT(*) AS n FROM offline_sync_queue WHERE user_id=? AND status='pending'",
(user["id"],),
).fetchone()["n"]
last = conn.execute(
"SELECT MAX(created_at) AS last FROM offline_sync_queue "
"WHERE user_id=? AND status='synced'",
(user["id"],),
).fetchone()["last"]
return {
"pending_count": pending,
"last_sync": last,
"is_syncing": False,
"server_time": SyncEngine._now_epoch(),
"workspace_id": workspace_id,
}
+315
View File
@@ -0,0 +1,315 @@
"""FlowDeck — Web Clipper router (v6.0.0).
Endpoints:
GET /api/v2/web-clipper/status
POST /api/v2/web-clipper/auth/verify
POST /api/v2/web-clipper/clip
GET /api/v2/web-clipper/devices
DELETE /api/v2/web-clipper/devices/{id}
GET /extensions (HTML download page)
Auth: session cookie OR Bearer api_token OR Bearer extension device token.
"""
from __future__ import annotations
import hashlib
import logging
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import HTMLResponse
from app.auth.session import SessionManager
from app.db import get_conn
from app.services.web_clipper import (
MAX_CLIP_BYTES,
_check_rate_limit,
create_page_from_clip,
list_devices,
log_clip,
register_device,
revoke_device,
sanitize_html,
)
logger = logging.getLogger(__name__)
router = APIRouter(tags=["web-clipper"])
api_router = APIRouter(prefix="/api/v2/web-clipper", tags=["web-clipper"])
def _hash(token: str) -> str:
return hashlib.sha256(token.encode()).hexdigest()
def _user_from_request(request: Request) -> dict | None:
# 1) session cookie
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if user:
return user
# 2) Authorization Bearer
auth = request.headers.get("authorization") or request.headers.get("Authorization") or ""
if auth.lower().startswith("bearer "):
token = auth[7:].strip()
if not token:
return None
th = _hash(token)
with get_conn() as conn:
# api_tokens (Settings → API tokens)
row = conn.execute(
"SELECT user_id FROM api_tokens WHERE token_hash=? AND revoked=0", (th,)
).fetchone()
if row:
u = conn.execute("SELECT id, login, full_name, email, is_admin FROM users WHERE id=?", (row["user_id"],)).fetchone()
if u:
return dict(u)
# extension_devices
row = conn.execute(
"SELECT user_id FROM extension_devices WHERE token_hash=? AND revoked=0", (th,)
).fetchone()
if row:
u = conn.execute("SELECT id, login, full_name, email, is_admin FROM users WHERE id=?", (row["user_id"],)).fetchone()
if u:
return dict(u)
# legacy user_tokens
row = conn.execute("SELECT gitea_user_id FROM user_tokens WHERE gitea_token=?", (token,)).fetchone()
if row:
u = conn.execute("SELECT id, login, full_name, email, is_admin FROM users WHERE id=?", (row["gitea_user_id"],)).fetchone()
if u:
return dict(u)
return None
def _require_user(request: Request) -> dict:
user = _user_from_request(request)
if not user:
raise HTTPException(status_code=401, detail="Authentication required")
return user
# ── API: status ──
@api_router.get("/status")
async def clipper_status(request: Request):
user = _user_from_request(request)
if not user:
return {"authenticated": False}
with get_conn() as conn:
dev_cnt = conn.execute("SELECT COUNT(*) FROM extension_devices WHERE user_id=? AND revoked=0", (user["id"],)).fetchone()[0]
clip_cnt = conn.execute("SELECT COUNT(*) FROM extension_clips WHERE user_id=?", (user["id"],)).fetchone()[0]
return {"authenticated": True, "user": {"id": user["id"], "login": user.get("login")}, "devices": dev_cnt, "clips": clip_cnt}
# ── API: auth verify / device registration ──
@api_router.post("/auth/verify")
async def auth_verify(request: Request):
user = _require_user(request)
try:
body = await request.json()
except Exception:
body = {}
device_id = (body.get("device_id") or request.headers.get("x-device-id") or "").strip()
device_name = (body.get("device_name") or body.get("deviceName") or "").strip()[:200]
extension_name = (body.get("extension_name") or body.get("extensionName") or "chrome").strip()[:20].lower()
if not device_id:
raise HTTPException(status_code=400, detail="device_id required")
if len(device_id) > 128:
raise HTTPException(status_code=400, detail="device_id too long")
try:
res = register_device(user["id"], device_id, device_name, extension_name)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e)) from None
if res["existing"]:
return {"status": "ok", "device_id": device_id, "existing": True, "message": "Device already registered"}
return {"status": "ok", "device_id": device_id, "token": res["token"], "note": "Copy token now — shown once. Use as Authorization: Bearer <token>"}
@api_router.post("/clip")
async def clip_page(request: Request):
user = _require_user(request)
# Enforce max body size early (10 MB)
clen = request.headers.get("content-length")
if clen:
try:
if int(clen) > MAX_CLIP_BYTES + 1024:
raise HTTPException(status_code=413, detail="Clip too large (max 10 MB)")
except ValueError:
pass
try:
body = await request.json()
except Exception:
raise HTTPException(status_code=400, detail="Invalid JSON") from None
# Device identification for rate limiting and logging
device_id = (body.get("device_id") or request.headers.get("x-device-id") or "web").strip()[:128] or "web"
# Rate limit 50/hour per device
if not _check_rate_limit(f"{user['id']}:{device_id}"):
raise HTTPException(status_code=429, detail="Rate limit: max 50 clips/hour per device")
url = (body.get("url") or body.get("source_url") or "").strip()
title = (body.get("title") or "").strip()
content = body.get("content") or body.get("html") or ""
clip_type = (body.get("content_type") or body.get("clip_type") or "article").strip().lower()
if clip_type not in ("article", "selection", "bookmark", "screenshot"):
clip_type = "article"
if not url and not title and not content:
raise HTTPException(status_code=400, detail="url, title or content required")
# Validate url if present
if url:
if not (url.startswith("http://") or url.startswith("https://")):
# allow bare domain? reject javascript:
if url.lower().startswith("javascript:") or url.lower().startswith("data:"):
raise HTTPException(status_code=400, detail="Invalid URL")
# Cap content bytes
if content and len(content.encode("utf-8")) > MAX_CLIP_BYTES:
raise HTTPException(status_code=413, detail="Content too large (max 10 MB)")
# Sanitize HTML content if present
if content and "<" in content:
# sanitize but keep structure for blocks converter
content = sanitize_html(content)[: MAX_CLIP_BYTES]
# Prepare payload for service
_img_b64 = body.get("image_base64") or body.get("screenshot") or ""
if not _img_b64 and body.get("images"):
try:
_imgs = body.get("images")
if isinstance(_imgs, list) and _imgs:
_img_b64 = _imgs[0].get("base64") or _imgs[0].get("src") or ""
except Exception:
logger.exception("clip_page")
clip_data = {
"url": url,
"title": title[:200],
"content": content,
"content_type": clip_type,
"selection_html": body.get("selection_html") or body.get("selection") or "",
"image_base64": _img_b64,
"tags": body.get("tags") or [],
"target_workspace_id": body.get("target_workspace_id") or body.get("workspace_id"),
"target_page_id": body.get("target_page_id") or body.get("parent_page_id"),
"metadata": body.get("metadata") or {},
}
try:
result = create_page_from_clip(clip_data, user["id"])
except Exception as e:
logger.exception("clip creation failed: %s", e)
raise HTTPException(status_code=500, detail="Failed to create page") from None
# Log clip
try:
log_clip(user["id"], device_id, clip_type, url, result["page_id"], result["workspace_id"], result["title"])
except Exception:
logger.exception("clip_page")
return {"status": "ok", "page_id": result["page_id"], "title": result["title"], "workspace_id": result["workspace_id"], "url": f"/pages/{result['page_id']}"}
@api_router.get("/devices")
async def list_extension_devices(request: Request):
user = _require_user(request)
devices = list_devices(user["id"])
return {"devices": devices}
@api_router.delete("/devices/{device_id}")
async def revoke_extension_device(device_id: int, request: Request):
user = _require_user(request)
ok = revoke_device(user["id"], device_id)
if not ok:
raise HTTPException(status_code=404, detail="Device not found")
return {"status": "revoked"}
# ── HTML: /extensions download page ──
@router.get("/extensions", response_class=HTMLResponse)
async def extensions_page(request: Request):
from app.routers.dashboard import _sidebar_data
from app.templating import ENV
env = ENV
try:
sidebar = _sidebar_data(request, [])
except Exception:
sidebar = {}
# Simple standalone page reusing base.html
block_tpl = env.from_string(
'{% extends "base.html" %}{% block content %}{{ content_html|safe }}{% endblock %}'
)
user = _user_from_request(request)
# Count for auth user
devices = []
clips = 0
if user:
try:
devices = list_devices(user["id"])
clips = sum(d.get("clips_count", 0) for d in devices)
except Exception:
logger.exception("extensions_page")
content_html = f"""
<style>
.ext-page{{max-width:900px;margin:0 auto;padding:32px 24px 80px;}}
.ext-hero{{text-align:center;padding:28px 0 8px;}}
.ext-hero h1{{font-size:30px;font-weight:800;margin:0 0 6px;}}
.ext-hero p{{color:var(--text-dim);font-size:14px;max-width:560px;margin:0 auto;line-height:1.6;}}
.ext-grid{{display:grid;grid-template-columns:repeat(auto-fit,minmax(240px,1fr));gap:14px;margin:28px 0;}}
.ext-card{{border:1px solid var(--border);border-radius:12px;padding:18px;background:var(--bg-card);}}
.ext-card h3{{font-size:15px;margin:0 0 6px;display:flex;align-items:center;gap:8px;}}
.ext-card p{{font-size:12.5px;color:var(--text-dim);line-height:1.5;margin:0 0 10px;}}
.ext-card a{{font-size:13px;color:var(--accent);text-decoration:none;}}
.ext-card a:hover{{text-decoration:underline;}}
.ext-section{{margin:28px 0;}}
.ext-section h2{{font-size:18px;font-weight:700;margin:0 0 10px;}}
.ext-steps{{counter-reset:step;list-style:none;padding:0;margin:0;}}
.ext-steps li{{display:flex;gap:12px;padding:10px 0;border-bottom:1px solid var(--border);font-size:13px;color:var(--text-dim);}}
.ext-steps li::before{{counter-increment:step;content:counter(step);flex:0 0 26px;height:26px;display:flex;align-items:center;justify-content:center;background:var(--accent);color:#fff;border-radius:50%;font-size:12px;font-weight:600;}}
.ext-dev-list{{margin-top:12px;}}
.ext-dev-item{{display:flex;align-items:center;justify-content:space-between;padding:10px 12px;border:1px solid var(--border);border-radius:8px;margin-bottom:6px;background:var(--bg-tertiary);}}
.ext-badge{{font-size:10px;padding:2px 8px;border-radius:99px;background:rgba(46,160,67,.14);color:#2ea043;font-weight:600;}}
</style>
<div class="ext-page">
<div class="ext-hero">
<h1>🧩 FlowDeck Web Clipper</h1>
<p>Capture any web page — article, selection, bookmark or screenshot — directly into FlowDeck. Install the browser extension, connect it once, then clip in one click.</p>
</div>
<div class="ext-grid">
<div class="ext-card">
<h3>🟢 Chrome / Edge</h3>
<p>Manifest V3 — Chrome 88+, Edge 88+.</p>
<a href="/static/extension/flowdeck-clipper.zip" download>Download .zip</a> · <span style="font-size:11px;color:var(--text-dim);">load unpacked in chrome://extensions</span>
</div>
<div class="ext-card">
<h3>🟠 Firefox</h3>
<p>Firefox 109+ (Manifest V2 compat).</p>
<a href="/static/extension/flowdeck-clipper.zip" download>Download .zip</a> · <span style="font-size:11px;color:var(--text-dim);">load temporary add-on</span>
</div>
<div class="ext-card">
<h3>⌨️ Sans extension</h3>
<p>API directe — <code>POST /api/v2/web-clipper/clip</code> avec Bearer token.</p>
<a href="/help">Docs /help</a>
</div>
</div>
<div class="ext-section">
<h2>How it works</h2>
<ol class="ext-steps">
<li>Install the extension (.zip) → enable in your browser.</li>
<li>Open FlowDeck, go to <b>Settings → Extensions</b> and copy a Bearer token (or the clipper verifies via your session cookie).</li>
<li>On any web page, click <b>📌 Clip to FlowDeck</b> (floating button, right-click selection, or extension popup).</li>
<li>Choose type: Article (full), Selection, Bookmark or Screenshot — the page is created instantly in your workspace.</li>
</ol>
</div>
<div class="ext-section">
<h2>Captures on this account</h2>
<p style="font-size:12px;color:var(--text-dim);">{len(devices)} device(s) · {clips} clip(s) total</p>
<div class="ext-dev-list">
{"".join(f'<div class="ext-dev-item"><span><b>{d.get("device_name") or d.get("extension_name")}</b> <code style="font-size:11px;color:var(--text-dim);">{d.get("device_id")[:24]}</code></span><span><span class="ext-badge">{d.get("clips_count",0)} clips</span> <span style="font-size:11px;color:var(--text-dim);">{d.get("last_clip_at") or ""}</span></span></div>' for d in devices[:10]) or '<p style="font-size:13px;color:var(--text-dim);">No devices yet — clip your first page from the extension to appear here.</p>'}
</div>
<p style="margin-top:10px;"><a href="/accounts/settings" style="font-size:13px;color:var(--accent);">Manage in Settings → Extensions</a></p>
</div>
</div>
"""
return HTMLResponse(block_tpl.render(**sidebar, request=request, page_title="Extensions", title_prefix="Extensions", page_icon="🧩", content_html=content_html))
+226
View File
@@ -0,0 +1,226 @@
"""FlowDeck — Passkeys / WebAuthn (v7.2.0).
Registration + passwordless login via the ``webauthn`` package (pinned in
requirements). Challenges live in a short-lived in-memory store (5 min,
single-process — same tradeoff as the SSE rooms). RP ID is derived from the
request host. See ``docs/V72_Enterprise_SCIM_2FA.md``.
"""
from __future__ import annotations
import secrets
import time
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import JSONResponse
from app.auth.session import SessionManager
from app.db import get_conn
router = APIRouter(tags=["webauthn"], prefix="/auth/webauthn")
# key -> (challenge bytes, expires_at). key = f"reg:{user_id}" | f"login:{login}".
_challenges: dict[str, tuple[bytes, float]] = {}
_CHALLENGE_TTL = 300.0
def _require_lib():
try:
import webauthn # noqa: F401
return True
except ImportError:
return False
def _store_challenge(key: str, challenge: bytes) -> None:
_challenges[key] = (challenge, time.time() + _CHALLENGE_TTL)
def _take_challenge(key: str) -> bytes | None:
item = _challenges.pop(key, None)
if not item:
return None
challenge, exp = item
return challenge if exp > time.time() else None
def _rp(request: Request) -> tuple[str, str]:
host = (request.url.hostname or "localhost").split(":")[0]
return host, f"{request.url.scheme}://{request.headers.get('host', host)}"
def _session_user(request: Request) -> dict:
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(401, "Authentication required")
return user
@router.post("/register/begin")
async def register_begin(request: Request):
if not _require_lib():
raise HTTPException(501, "WebAuthn library not installed")
from webauthn import generate_registration_options, options_to_json
user = _session_user(request)
rp_id, _origin = _rp(request)
with get_conn() as conn:
existing = conn.execute("SELECT credential_id FROM webauthn_credentials WHERE user_id=?",
(user["id"],)).fetchall()
from webauthn.helpers.structs import PublicKeyCredentialDescriptor
exclude = [PublicKeyCredentialDescriptor(id=_b64url_to_bytes(r["credential_id"]))
for r in existing]
options = generate_registration_options(
rp_id=rp_id, rp_name="FlowDeck", user_name=user.get("login", f"user{user['id']}"),
user_id=str(user["id"]).encode(), exclude_credentials=exclude or None)
_store_challenge(f"reg:{user['id']}", options.challenge)
return JSONResponse(content=__import__("json").loads(options_to_json(options)))
@router.post("/register/finish")
async def register_finish(request: Request):
if not _require_lib():
raise HTTPException(501, "WebAuthn library not installed")
from webauthn import verify_registration_response
user = _session_user(request)
try:
body = await request.json()
except Exception:
body = {}
challenge = _take_challenge(f"reg:{user['id']}")
if not challenge:
raise HTTPException(400, "Challenge expired — begin again")
rp_id, origin = _rp(request)
try:
verified = verify_registration_response(
credential=body.get("credential") or {},
expected_challenge=challenge, expected_rp_id=rp_id, expected_origin=origin,
require_user_verification=False)
except Exception as exc: # noqa: BLE001 — invalid attestation → 400, never 500
raise HTTPException(400, f"Registration rejected: {exc}") from None
import base64
cred_id = base64.urlsafe_b64encode(verified.credential_id).decode().rstrip("=")
pubkey = base64.b64encode(bytes(verified.credential_public_key)).decode()
with get_conn() as conn:
try:
cur = conn.execute(
"""INSERT INTO webauthn_credentials
(user_id, credential_id, public_key, sign_count, name)
VALUES (?,?,?,?,?)""",
(user["id"], cred_id, pubkey, verified.sign_count,
str(body.get("name") or "Passkey")[:80]))
conn.commit()
except Exception:
raise HTTPException(409, "Credential already registered") from None
kid = cur.lastrowid
return {"id": kid, "status": "registered"}
@router.post("/login/begin")
async def login_begin(request: Request):
if not _require_lib():
raise HTTPException(501, "WebAuthn library not installed")
from webauthn import generate_authentication_options, options_to_json
try:
body = await request.json()
except Exception:
body = {}
login = (body.get("login") or "").strip()
if not login:
raise HTTPException(400, "login required")
with get_conn() as conn:
user = conn.execute("SELECT * FROM users WHERE login=?", (login,)).fetchone()
if not user or not user["is_active"]:
raise HTTPException(401, "Invalid credentials")
creds = conn.execute("SELECT credential_id FROM webauthn_credentials WHERE user_id=?",
(user["id"],)).fetchall()
if not creds:
raise HTTPException(400, "No passkeys for this account")
rp_id, _origin = _rp(request)
from webauthn.helpers.structs import PublicKeyCredentialDescriptor
allow = [PublicKeyCredentialDescriptor(id=_b64url_to_bytes(r["credential_id"]))
for r in creds]
options = generate_authentication_options(rp_id=rp_id, allow_credentials=allow)
_store_challenge(f"login:{login}", options.challenge)
return JSONResponse(content=__import__("json").loads(options_to_json(options)))
@router.post("/login/finish")
async def login_finish(request: Request):
if not _require_lib():
raise HTTPException(501, "WebAuthn library not installed")
from webauthn import verify_authentication_response
try:
body = await request.json()
except Exception:
body = {}
login = (body.get("login") or "").strip()
challenge = _take_challenge(f"login:{login}")
if not login or not challenge:
raise HTTPException(400, "Challenge expired — begin again")
with get_conn() as conn:
user = conn.execute("SELECT * FROM users WHERE login=?", (login,)).fetchone()
if not user or not user["is_active"]:
raise HTTPException(401, "Invalid credentials")
stored = conn.execute("SELECT * FROM webauthn_credentials WHERE user_id=?",
(user["id"],)).fetchall()
rp_id, origin = _rp(request)
credential = body.get("credential") or {}
cred_id = (credential.get("id") or "").rstrip("=")
match = next((dict(r) for r in stored if r["credential_id"].rstrip("=") == cred_id), None)
if not match:
raise HTTPException(401, "Unknown credential")
import base64
try:
verified = verify_authentication_response(
credential=credential, expected_challenge=challenge,
expected_origin=origin, expected_rp_id=rp_id,
credential_public_key=base64.b64decode(match["public_key"]),
credential_current_sign_count=match["sign_count"],
require_user_verification=False)
except Exception as exc: # noqa: BLE001
raise HTTPException(401, f"Authentication rejected: {exc}") from None
with get_conn() as conn:
conn.execute("UPDATE webauthn_credentials SET sign_count=? WHERE id=?",
(verified.new_sign_count, match["id"]))
conn.execute("UPDATE users SET last_login=? WHERE id=?",
(str(time.time()), user["id"]))
conn.commit()
ud = dict(conn.execute("SELECT * FROM users WHERE id=?", (user["id"],)).fetchone())
session = SessionManager.create_session(ud, request)
response = JSONResponse({"status": "ok", "user": {"login": ud["login"]}})
response.set_cookie("flowdeck_session", session, httponly=True,
max_age=86400 * 7, samesite="lax", path="/")
return response
@router.get("/keys")
async def list_keys(request: Request):
user = _session_user(request)
with get_conn() as conn:
rows = conn.execute("SELECT id, name, sign_count, created_at FROM webauthn_credentials"
" WHERE user_id=? ORDER BY id", (user["id"],)).fetchall()
return {"keys": [dict(r) for r in rows]}
@router.delete("/keys/{key_id}")
async def delete_key(key_id: int, request: Request):
user = _session_user(request)
with get_conn() as conn:
cur = conn.execute("DELETE FROM webauthn_credentials WHERE id=? AND user_id=?",
(key_id, user["id"]))
conn.commit()
if not cur.rowcount:
raise HTTPException(404, "Key not found")
return {"status": "deleted", "id": key_id}
def _b64url_to_bytes(data: str) -> bytes:
import base64
padded = data + "=" * (-len(data) % 4)
return base64.urlsafe_b64decode(padded)
def reset_challenges() -> None:
_challenges.clear()
__all__ = ["router", "reset_challenges", "secrets"]
+537
View File
@@ -0,0 +1,537 @@
"""FlowDeck — teamspaces, verified pages, wiki home, collab polish (v7.3.0).
Routes under ``/api/v2/wiki`` plus the guest entry point ``/g/{token}``.
Design : ``docs/V73_Wiki_Teamspaces_Polish.md``.
"""
from __future__ import annotations
import html
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import HTMLResponse, JSONResponse
from app.auth.session import SessionManager
from app.db import get_conn
from app.services import wiki
from app.services.api_v2_helpers import audit_log
from app.services.notifications import create_notification
from app.templating import ENV
router = APIRouter(tags=["wiki"])
def _esc(value) -> str:
return html.escape(str(value))
def _user(request: Request) -> dict:
sess = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not sess or not sess.get("id"):
raise HTTPException(401, "Authentication required")
with get_conn() as conn:
row = conn.execute("SELECT is_admin, is_active FROM users WHERE id=?",
(sess["id"],)).fetchone()
if not row or not row["is_active"]:
raise HTTPException(403, "Account disabled")
return sess
def _workspace_id(request: Request) -> int:
wid = request.query_params.get("workspace_id")
if not wid:
raise HTTPException(400, "workspace_id required")
try:
wid = int(wid)
except (TypeError, ValueError):
raise HTTPException(400, "invalid workspace_id") from None
with get_conn() as conn:
if not conn.execute("SELECT 1 FROM workspaces WHERE id=?", (wid,)).fetchone():
raise HTTPException(404, "Workspace not found")
return wid
def _teamspace_or_404(teamspace_id: int, user_id: int) -> dict:
with get_conn() as conn:
row = conn.execute("SELECT * FROM teamspaces WHERE id=?", (teamspace_id,)).fetchone()
if not row:
raise HTTPException(404, "Teamspace not found")
if not wiki.can_read_teamspace(user_id, teamspace_id):
# private teamspace → 404 (not 403), same as restricted collections
raise HTTPException(404, "Teamspace not found")
return dict(row)
def _page_or_404(page_id: int) -> dict:
with get_conn() as conn:
row = conn.execute(
"SELECT id, title, workspace_id, teamspace_id, deleted_at FROM pages WHERE id=?",
(page_id,)).fetchone()
if not row or row["deleted_at"]:
raise HTTPException(404, "Page not found")
return dict(row)
def _is_admin(user: dict) -> bool:
with get_conn() as conn:
row = conn.execute("SELECT is_admin FROM users WHERE id=?", (user["id"],)).fetchone()
return bool(row and row["is_admin"])
def _can_verify(user: dict, page: dict) -> bool:
"""Admin, or an editor/owner of the teamspace / workspace holding the page."""
if _is_admin(user):
return True
if page.get("teamspace_id"):
return wiki.can_write_teamspace(user["id"], page["teamspace_id"])
wid = page.get("workspace_id")
if not wid:
return False
with get_conn() as conn:
owner = conn.execute("SELECT owner_id FROM workspaces WHERE id=?", (wid,)).fetchone()
if owner and owner["owner_id"] == user["id"]:
return True
member = conn.execute("SELECT role FROM workspace_members WHERE workspace_id=? AND user_id=?",
(wid, user["id"])).fetchone()
return bool(member and member["role"] in ("owner", "admin", "editor"))
# ── teamspaces ─────────────────────────────────────────────────────────────
@router.get("/api/v2/wiki/teamspaces")
async def list_teamspaces(request: Request):
user = _user(request)
wid = request.query_params.get("workspace_id")
if wid:
try:
wid = int(wid)
except (TypeError, ValueError):
raise HTTPException(400, "invalid workspace_id") from None
else:
wid = None
return {"teamspaces": wiki.list_teamspaces(user["id"], wid)}
@router.get("/wiki/teamspaces/{teamspace_id}", response_class=HTMLResponse)
async def teamspace_page(teamspace_id: int, request: Request):
"""Teamspace detail HTML page — sidebar entry point."""
user = _user(request)
ts = _teamspace_or_404(teamspace_id, user["id"])
ts["role"] = wiki.get_teamspace_role(user["id"], teamspace_id)
with get_conn() as conn:
ws = conn.execute("SELECT name FROM workspaces WHERE id=?",
(ts["workspace_id"],)).fetchone()
pages = wiki.teamspace_pages(teamspace_id)
collections = wiki.teamspace_collections(teamspace_id)
page_rows = "\n".join(
f'<a class="ts-row" href="/pages/{p["id"]}" style="display:flex;align-items:center;gap:8px;'
f'padding:8px 10px;border-radius:6px;color:inherit;text-decoration:none;">'
f'<span>📄</span><span>{_esc(p["title"] or "Untitled")}</span></a>'
for p in pages)
coll_rows = "\n".join(
f'<a class="ts-row" href="/db/{c["id"]}" style="display:flex;align-items:center;gap:8px;'
f'padding:8px 10px;border-radius:6px;color:inherit;text-decoration:none;">'
f'<span>{_esc(c["icon"] or "🗄️")}</span><span>{_esc(c["name"] or "Untitled")}</span></a>'
for c in collections)
content_html = f"""
<div style="max-width:860px;margin:0 auto;padding:40px 24px;">
<h1 style="font-size:26px;display:flex;align-items:center;gap:10px;">
{_esc(ts['name'])}{' <span style="font-size:13px;padding:2px 8px;border-radius:10px;background:rgba(76,154,255,.15);color:#4c9aff;">🔒 private</span>' if ts['private'] else ''}
</h1>
<p style="color:var(--text-dim);">{_esc(ts.get('description') or '')}</p>
<div style="display:flex;gap:10px;font-size:12px;color:var(--text-dim);margin-bottom:24px;flex-wrap:wrap;">
<span>Workspace: {_esc((ws["name"]) if ws else '')}</span>
<span>·</span><span>Role: {_esc(ts['role'])}</span>
<span>·</span><span>{len(pages) + len(collections)} items</span>
</div>
<h2 style="font-size:16px;margin:20px 0 8px;">Pages</h2>
<div style="display:flex;flex-direction:column;gap:4px;">
{page_rows or '<p style="color:var(--text-dim);font-size:13px;">No pages yet.</p>'}
</div>
<h2 style="font-size:16px;margin:24px 0 8px;">Databases</h2>
<div style="display:flex;flex-direction:column;gap:4px;">
{coll_rows or '<p style="color:var(--text-dim);font-size:13px;">No databases yet.</p>'}
</div>
</div>
<style>
.ts-row:hover{{background:var(--bg-hover);}}
</style>"""
from app.routers.dashboard import _sidebar_data
env = ENV
sidebar = _sidebar_data(request, [])
block_tpl = env.from_string(
'{% extends "base.html" %}{% block content %}{{ content_html|safe }}{% endblock %}'
)
return block_tpl.render(
**sidebar,
request=request,
content_html=content_html,
page_title=ts["name"],
title_prefix="Teamspace",
page_icon="🏛️",
)
@router.post("/api/v2/wiki/teamspaces")
async def create_teamspace(request: Request):
user = _user(request)
try:
body = await request.json()
except Exception:
body = {}
name = (body.get("name") or "").strip()
if not name or len(name) > 120:
raise HTTPException(400, "name required (max 120 chars)")
wid = int(body.get("workspace_id") or 0)
with get_conn() as conn:
ws = conn.execute("SELECT owner_id FROM workspaces WHERE id=?", (wid,)).fetchone()
if not ws:
raise HTTPException(404, "Workspace not found")
admin = conn.execute("SELECT is_admin FROM users WHERE id=?", (user["id"],)).fetchone()
member = conn.execute("SELECT role FROM workspace_members WHERE workspace_id=? AND user_id=?",
(wid, user["id"])).fetchone()
allowed = (ws["owner_id"] == user["id"] or (admin and admin["is_admin"])
or (member and member["role"] in ("admin", "editor", "owner")))
if not allowed:
raise HTTPException(403, "Editor role required in the workspace")
try:
tsid = wiki.create_teamspace(wid, name, user["id"],
description=body.get("description") or "",
private=bool(body.get("private")))
except ValueError as exc:
raise HTTPException(409, str(exc)) from None
audit_log(user, "teamspace.create", "teamspace", tsid, name, request)
return JSONResponse(status_code=201, content={"id": tsid, "name": name})
@router.get("/api/v2/wiki/teamspaces/{teamspace_id}")
async def get_teamspace(teamspace_id: int, request: Request):
user = _user(request)
ts = _teamspace_or_404(teamspace_id, user["id"])
ts["role"] = wiki.get_teamspace_role(user["id"], teamspace_id)
ts["member_count"] = len(wiki.teamspace_member_ids(teamspace_id))
return ts
@router.get("/api/v2/wiki/teamspaces/{teamspace_id}/members")
async def list_members(teamspace_id: int, request: Request):
user = _user(request)
_teamspace_or_404(teamspace_id, user["id"])
with get_conn() as conn:
rows = conn.execute(
"""SELECT m.user_id, m.role, u.login, u.full_name FROM teamspace_members m
JOIN users u ON u.id = m.user_id WHERE m.teamspace_id=? ORDER BY u.login""",
(teamspace_id,)).fetchall()
return {"members": [dict(r) for r in rows]}
@router.put("/api/v2/wiki/teamspaces/{teamspace_id}/members/{member_id}")
async def set_member(teamspace_id: int, member_id: int, request: Request):
user = _user(request)
_teamspace_or_404(teamspace_id, user["id"])
if not wiki.can_write_teamspace(user["id"], teamspace_id):
raise HTTPException(403, "Editor role required")
try:
body = await request.json()
except Exception:
body = {}
role = body.get("role")
if role not in wiki.TEAMSPACE_ROLES:
raise HTTPException(400, f"role must be one of {', '.join(wiki.TEAMSPACE_ROLES)}")
with get_conn() as conn:
if not conn.execute("SELECT 1 FROM users WHERE id=?", (member_id,)).fetchone():
raise HTTPException(404, "User not found")
conn.execute(
"""INSERT INTO teamspace_members (teamspace_id, user_id, role) VALUES (?,?,?)
ON CONFLICT(teamspace_id, user_id) DO UPDATE SET role=excluded.role""",
(teamspace_id, member_id, role))
conn.commit()
audit_log(user, "teamspace.member.set", "teamspace", teamspace_id,
f"u{member_id}={role}", request)
return {"status": "ok", "user_id": member_id, "role": role}
@router.delete("/api/v2/wiki/teamspaces/{teamspace_id}/members/{member_id}")
async def remove_member(teamspace_id: int, member_id: int, request: Request):
user = _user(request)
_teamspace_or_404(teamspace_id, user["id"])
if not wiki.can_write_teamspace(user["id"], teamspace_id):
raise HTTPException(403, "Editor role required")
with get_conn() as conn:
cur = conn.execute("DELETE FROM teamspace_members WHERE teamspace_id=? AND user_id=?",
(teamspace_id, member_id))
conn.commit()
if not cur.rowcount:
raise HTTPException(404, "Not a member")
return {"status": "removed", "user_id": member_id}
# ── verified pages ─────────────────────────────────────────────────────────
@router.get("/api/v2/wiki/pages/{page_id}/verification")
async def get_verification(page_id: int, request: Request):
_user(request)
_page_or_404(page_id)
return {"verification": wiki.verification(page_id)}
@router.post("/api/v2/wiki/pages/{page_id}/verify")
async def verify_page(page_id: int, request: Request):
user = _user(request)
page = _page_or_404(page_id)
if not _can_verify(user, page):
raise HTTPException(403, "Editor role required to verify a page")
try:
body = await request.json()
except Exception:
body = {}
out = wiki.verify_page(page_id, user["id"],
days=body.get("days") or wiki.VERIFICATION_DAYS_DEFAULT,
note=body.get("note") or "")
audit_log(user, "page.verify", "page", page_id, out.get("expires_at") or "", request)
return {"verification": out}
@router.delete("/api/v2/wiki/pages/{page_id}/verify")
async def unverify_page(page_id: int, request: Request):
user = _user(request)
_page_or_404(page_id)
if not wiki.unverify_page(page_id):
raise HTTPException(404, "Page is not verified")
audit_log(user, "page.unverify", "page", page_id, "", request)
return {"status": "unverified", "page_id": page_id}
@router.get("/api/v2/wiki/verified")
async def list_verified(request: Request):
"""Verified (non-expired) pages of a workspace — the ✅ wiki index."""
user = _user(request)
wid = _workspace_id(request)
with get_conn() as conn:
rows = conn.execute(
"""SELECT p.id, p.title, p.page_icon, p.teamspace_id,
v.verified_at, v.expires_at, v.note, u.login
FROM page_verifications v
JOIN pages p ON p.id = v.page_id
LEFT JOIN users u ON u.id = v.verified_by
WHERE p.workspace_id=? AND p.deleted_at IS NULL""",
(wid,)).fetchall()
out = []
for r in rows:
item = dict(r)
if wiki.is_expired(r):
continue # badge lapsed → not listed
if item["teamspace_id"] and not wiki.can_read_teamspace(user["id"],
item["teamspace_id"]):
continue # private teamspace → hidden
item["active"] = True
out.append(item)
return {"pages": out}
# ── follows ────────────────────────────────────────────────────────────────
@router.post("/api/v2/wiki/pages/{page_id}/follow")
async def follow_page(page_id: int, request: Request):
user = _user(request)
_page_or_404(page_id)
now = wiki.toggle_follow(page_id, user["id"])
return {"page_id": page_id, "following": now}
@router.get("/api/v2/wiki/pages/{page_id}/followers")
async def list_followers(page_id: int, request: Request):
_user(request)
_page_or_404(page_id)
ids = wiki.followers(page_id)
if not ids:
return {"followers": []}
with get_conn() as conn:
rows = conn.execute(
f"SELECT id, login, full_name FROM users WHERE id IN ({','.join('?' * len(ids))})",
ids).fetchall()
return {"followers": [dict(r) for r in rows]}
# ── comment reactions ──────────────────────────────────────────────────────
@router.post("/api/v2/wiki/comments/{comment_id}/reactions")
async def react(comment_id: int, request: Request):
user = _user(request)
try:
body = await request.json()
except Exception:
body = {}
emoji = (body.get("emoji") or "").strip()
if not emoji:
raise HTTPException(400, "emoji required")
try:
counts = wiki.toggle_reaction(comment_id, user["id"], emoji)
except LookupError:
raise HTTPException(404, "Comment not found") from None
return {"comment_id": comment_id, "reactions": counts}
@router.get("/api/v2/wiki/comments/{comment_id}/reactions")
async def list_reactions(comment_id: int, request: Request):
_user(request)
return {"comment_id": comment_id, "reactions": wiki.reactions(comment_id)}
# ── guest shares ───────────────────────────────────────────────────────────
@router.post("/api/v2/wiki/pages/{page_id}/guests")
async def create_guest(page_id: int, request: Request):
user = _user(request)
page = _page_or_404(page_id)
if page.get("teamspace_id") and not wiki.can_write_teamspace(user["id"], page["teamspace_id"]):
raise HTTPException(403, "Editor role required to share")
try:
body = await request.json()
except Exception:
body = {}
try:
share = wiki.create_guest_share(page_id, body.get("email") or "",
body.get("role") or "viewer",
user["id"], days=body.get("days", 30))
except ValueError as exc:
raise HTTPException(400, str(exc)) from None
audit_log(user, "page.guest_share", "page", page_id, share["email"], request)
return JSONResponse(status_code=201, content={
"id": share["id"], "token": share["token"], "role": share["role"],
"expires_at": share["expires_at"], "url": f"/g/{share['token']}"})
@router.get("/api/v2/wiki/pages/{page_id}/guests")
async def list_guests(page_id: int, request: Request):
_user(request)
_page_or_404(page_id)
with get_conn() as conn:
rows = conn.execute(
"SELECT id, email, role, expires_at, revoked, created_at FROM guest_shares"
" WHERE page_id=? ORDER BY id DESC", (page_id,)).fetchall()
return {"guests": [dict(r) for r in rows]}
@router.delete("/api/v2/wiki/guests/{share_id}")
async def revoke_guest(share_id: int, request: Request):
user = _user(request)
with get_conn() as conn:
if not conn.execute("SELECT 1 FROM guest_shares WHERE id=?", (share_id,)).fetchone():
raise HTTPException(404, "Guest share not found")
conn.execute("UPDATE guest_shares SET revoked=1 WHERE id=?", (share_id,))
conn.commit()
audit_log(user, "page.guest_revoke", "guest_share", share_id, "", request)
return {"status": "revoked", "id": share_id}
_GUEST_404 = """<!DOCTYPE html><html lang="en"><head><meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>Link unavailable — FlowDeck</title>
<style>body{font-family:-apple-system,Segoe UI,Roboto,sans-serif;max-width:520px;
margin:80px auto;padding:0 20px;color:#1f2328;text-align:center}
h1{font-size:20px} p{color:#656d76;line-height:1.6}</style></head><body>
<h1>This link is unavailable</h1>
<p>It may have expired, been revoked, or never existed.<br>
Ask the person who shared it with you for a new link.</p></body></html>"""
@router.get("/g/{token}", response_class=HTMLResponse)
async def guest_page(token: str, request: Request):
"""Account-less page access (read-only or commenter). 404 if inactive."""
share = wiki.resolve_guest_share(token)
if not share:
return HTMLResponse(_GUEST_404, status_code=404)
with get_conn() as conn:
page = conn.execute("SELECT id, title, content, created_at, updated_at, deleted_at"
" FROM pages WHERE id=?", (share["page_id"],)).fetchone()
if not page or page["deleted_at"]:
return HTMLResponse(_GUEST_404, status_code=404)
wiki.record_view(share["page_id"])
return f"""<!DOCTYPE html><html lang="en"><head><meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>{page['title']} — FlowDeck guest</title>
<style>body{{font-family:-apple-system,Segoe UI,Roboto,sans-serif;max-width:760px;
margin:40px auto;padding:0 20px;line-height:1.6;color:#1f2328}}
.guest-banner{{background:#fff4e5;border:1px solid #ffd8a8;padding:10px 14px;
border-radius:8px;margin-bottom:24px;font-size:14px}}
pre{{background:#f6f8fa;padding:14px;border-radius:8px;overflow:auto;
white-space:pre-wrap;word-break:break-word}}</style></head><body>
<div class="guest-banner">You are viewing this page as a guest
({share['role']}{' — expires ' + str(share['expires_at']) if share['expires_at'] else ''}).
Editing is disabled.</div>
<h1>{page['title']}</h1><pre>{page['content'] or ''}</pre></body></html>"""
# ── page views ─────────────────────────────────────────────────────────────
@router.get("/api/v2/wiki/pages/{page_id}/views")
async def page_views(page_id: int, request: Request):
user = _user(request)
page = _page_or_404(page_id)
if page.get("teamspace_id") and not wiki.can_write_teamspace(user["id"], page["teamspace_id"]):
raise HTTPException(403, "Editor role required to read analytics")
return wiki.view_stats(page_id, days=request.query_params.get("days", 30))
# ── wiki home ──────────────────────────────────────────────────────────────
@router.get("/api/v2/wiki/home")
async def wiki_home(request: Request):
"""Aggregated knowledge home: verified pages + recents + teamspaces."""
user = _user(request)
wid = _workspace_id(request)
with get_conn() as conn:
recents = conn.execute(
"""SELECT id, title, page_icon, updated_at FROM pages
WHERE workspace_id=? AND deleted_at IS NULL
ORDER BY updated_at DESC LIMIT 20""", (wid,)).fetchall()
verified = conn.execute(
"""SELECT v.page_id, v.verified_at, v.expires_at FROM page_verifications v
JOIN pages p ON p.id = v.page_id
WHERE p.workspace_id=? AND p.deleted_at IS NULL
AND (v.expires_at IS NULL OR v.expires_at > ?)""",
(wid, __import__("datetime").datetime.now(
__import__("datetime").timezone.utc).replace(microsecond=0).isoformat()),
).fetchall()
return {"workspace_id": wid,
"teamspaces": wiki.list_teamspaces(user["id"], wid),
"verified": [dict(r) for r in verified],
"recents": [dict(r) for r in recents]}
@router.post("/api/v2/wiki/verify-expiry-sweep")
async def sweep_expiry(request: Request):
"""Notify verifiers whose ✅ expires within 7 days (idempotent-ish job)."""
user = _user(request)
with get_conn() as conn:
if not conn.execute("SELECT is_admin FROM users WHERE id=?", (user["id"],)).fetchone()[0]:
raise HTTPException(403, "Admin required")
sent = 0
for row in wiki.expiring_verifications(days=7):
create_notification(row["owner_id"], None, "page.verification_expiring",
"Verification expiring soon",
f"“{row['title']}” loses its verified badge on {row['expires_at']}.",
resource_type="page", resource_id=row["page_id"])
sent += 1
return {"notified": sent}
# ── blocks (mermaid / equation_inline / progress) ───────────────────────────
@router.post("/api/v2/wiki/blocks/preview")
async def preview_blocks(request: Request):
"""Render v7.3 blocks to HTML (same renderer used by the export pipeline)."""
_user(request)
try:
body = await request.json()
except Exception:
body = {}
blocks = body.get("blocks")
if not isinstance(blocks, list):
raise HTTPException(400, "blocks must be a list")
if len(blocks) > 200:
raise HTTPException(400, "max 200 blocks per preview")
from app.services.wiki_blocks import mmdc_available, render_block
out = [{"type": b.get("type"), "html": render_block(b)} for b in blocks
if isinstance(b, dict) and b.get("type") in ("mermaid", "equation_inline", "progress")]
return {"rendered": out, "mmdc_available": mmdc_available()}
+216
View File
@@ -0,0 +1,216 @@
"""FlowDeck — Workers API (v7.0.0): CRUD, manual run, history, fork, usage."""
from __future__ import annotations
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import JSONResponse
from app.auth.session import SessionManager
from app.db import get_conn
from app.services import workers as worker_service
from app.services.api_v2_helpers import (
audit_log,
has_scope,
paginate_headers,
parse_pagination,
resolve_bearer_token,
row_to_dict,
)
router = APIRouter(tags=["workers"])
def _auth_user(request: Request, *, require_write: bool = False) -> dict:
sess = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if sess:
return sess
auth = request.headers.get("authorization") or request.headers.get("Authorization") or ""
if auth.lower().startswith("bearer "):
user = resolve_bearer_token(auth[7:].strip())
if not user:
raise HTTPException(401, "Invalid or expired API token")
if require_write and not has_scope(user.get("_token_scopes") or "read", "write"):
raise HTTPException(403, "Insufficient scope. Required: write")
return user
raise HTTPException(401, "Authentication required")
def _row_to_api(row) -> dict:
d = row_to_dict(row)
d.pop("code_py", None) # code only via ?include_code=1 or owner fetch
return d
@router.post("/api/v2/workers")
async def create_worker(request: Request):
user = _auth_user(request, require_write=True)
try:
body = await request.json()
except Exception:
body = {}
name = (body.get("name") or "Untitled worker").strip()[:200]
code = body.get("code_py") or ""
try:
worker_service.validate_code(code)
except worker_service.WorkerRejected as exc:
raise HTTPException(400, f"code rejected: {exc}") from None
slug = worker_service.unique_slug(body.get("slug") or name)
with get_conn() as conn:
cur = conn.execute(
"""INSERT INTO workers (slug, workspace_id, name, code_py, schedule_cron,
shared, daily_budget_s, created_by)
VALUES (?,?,?,?,?,?,?,?)""",
(slug, body.get("workspace_id"), name, code,
(body.get("schedule_cron") or "")[:60],
1 if body.get("shared") else 0,
max(1, min(int(body.get("daily_budget_s") or 60), 3600)),
user["id"]))
conn.commit()
wid = cur.lastrowid
row = conn.execute("SELECT * FROM workers WHERE id=?", (wid,)).fetchone()
audit_log(user, "worker.create", "worker", wid, slug, request)
return JSONResponse(status_code=201, content={**_row_to_api(row), "code_py": code})
@router.get("/api/v2/workers")
async def list_workers(request: Request):
_auth_user(request)
limit, offset = parse_pagination(request)
with get_conn() as conn:
total = conn.execute("SELECT COUNT(*) FROM workers").fetchone()[0]
rows = conn.execute(
"SELECT * FROM workers ORDER BY id DESC LIMIT ? OFFSET ?",
(limit, offset)).fetchall()
resp = JSONResponse([_row_to_api(r) for r in rows])
for k, v in paginate_headers(total).items():
resp.headers[k] = v
return resp
@router.get("/api/v2/workers/{worker_id}")
async def get_worker(worker_id: int, request: Request):
user = _auth_user(request)
with get_conn() as conn:
row = conn.execute("SELECT * FROM workers WHERE id=?", (worker_id,)).fetchone()
if not row:
raise HTTPException(404, "Worker not found")
out = _row_to_api(row)
if (request.query_params.get("include_code") == "1" or row["created_by"] == user["id"]
or user.get("is_admin")):
out["code_py"] = row["code_py"]
return out
@router.patch("/api/v2/workers/{worker_id}")
async def update_worker(worker_id: int, request: Request):
user = _auth_user(request, require_write=True)
try:
body = await request.json()
except Exception:
body = {}
with get_conn() as conn:
row = conn.execute("SELECT * FROM workers WHERE id=?", (worker_id,)).fetchone()
if not row:
raise HTTPException(404, "Worker not found")
if row["created_by"] != user["id"] and not user.get("is_admin"):
raise HTTPException(403, "Only the owner can update this worker")
updates: dict = {}
if "name" in body:
updates["name"] = str(body["name"] or "")[:200]
if "code_py" in body:
try:
worker_service.validate_code(body["code_py"] or "")
except worker_service.WorkerRejected as exc:
raise HTTPException(400, f"code rejected: {exc}") from None
updates["code_py"] = body["code_py"] or ""
if "schedule_cron" in body:
updates["schedule_cron"] = str(body["schedule_cron"] or "")[:60]
if "shared" in body:
updates["shared"] = 1 if body["shared"] else 0
if "daily_budget_s" in body:
updates["daily_budget_s"] = max(1, min(int(body["daily_budget_s"] or 60), 3600))
if "slug" in body and body["slug"] != row["slug"]:
if not worker_service._SLUG_RE.match(str(body["slug"] or "")):
raise HTTPException(400, "Invalid slug")
if conn.execute("SELECT id FROM workers WHERE slug=? AND id!=?",
(body["slug"], worker_id)).fetchone():
raise HTTPException(409, "Slug already taken")
updates["slug"] = body["slug"]
if updates:
sets = ", ".join(f"{k}=?" for k in updates)
conn.execute(f"UPDATE workers SET {sets}, updated_at=CURRENT_TIMESTAMP WHERE id=?",
(*updates.values(), worker_id))
conn.commit()
row = conn.execute("SELECT * FROM workers WHERE id=?", (worker_id,)).fetchone()
audit_log(user, "worker.update", "worker", worker_id, ",".join(updates), request)
return _row_to_api(row)
@router.delete("/api/v2/workers/{worker_id}")
async def delete_worker(worker_id: int, request: Request):
user = _auth_user(request, require_write=True)
with get_conn() as conn:
row = conn.execute("SELECT * FROM workers WHERE id=?", (worker_id,)).fetchone()
if not row:
raise HTTPException(404, "Worker not found")
if row["created_by"] != user["id"] and not user.get("is_admin"):
raise HTTPException(403, "Only the owner can delete this worker")
conn.execute("DELETE FROM workers WHERE id=?", (worker_id,))
conn.commit()
audit_log(user, "worker.delete", "worker", worker_id, "", request)
return {"status": "deleted", "id": worker_id}
@router.post("/api/v2/workers/{worker_id}/run")
async def run_worker_endpoint(worker_id: int, request: Request):
user = _auth_user(request, require_write=True)
try:
body = await request.json() if request.headers.get("content-type") else {}
except Exception:
body = {}
with get_conn() as conn:
row = conn.execute("SELECT * FROM workers WHERE id=?", (worker_id,)).fetchone()
if not row:
raise HTTPException(404, "Worker not found")
if (row["created_by"] != user["id"] and not row["shared"]
and not user.get("is_admin")):
raise HTTPException(403, "Worker is private")
import asyncio
loop = asyncio.get_running_loop()
try:
out = await loop.run_in_executor(
None, worker_service.run_worker, worker_id, body.get("ctx") or {})
except HTTPException:
raise
audit_log(user, "worker.run", "worker", worker_id, out.get("status", ""), request)
return out
@router.get("/api/v2/workers/{worker_id}/runs")
async def worker_runs(worker_id: int, request: Request):
_auth_user(request)
limit, _offset = parse_pagination(request, default_limit=20)
with get_conn() as conn:
if not conn.execute("SELECT id FROM workers WHERE id=?", (worker_id,)).fetchone():
raise HTTPException(404, "Worker not found")
rows = conn.execute(
"SELECT * FROM worker_runs WHERE worker_id=? ORDER BY id DESC LIMIT ?",
(worker_id, limit)).fetchall()
return {"worker_id": worker_id, "runs": [row_to_dict(r) for r in rows]}
@router.post("/api/v2/workers/{worker_id}/fork")
async def fork_worker_endpoint(worker_id: int, request: Request):
user = _auth_user(request, require_write=True)
out = worker_service.fork_worker(worker_id, user["id"])
audit_log(user, "worker.fork", "worker", worker_id, "", request)
return JSONResponse(status_code=201, content=out)
@router.get("/api/v2/workers-usage")
async def workers_usage(request: Request):
user = _auth_user(request)
ws_raw = request.query_params.get("workspace_id")
wid = int(ws_raw) if ws_raw and str(ws_raw).isdigit() else None
return {"workspace_id": wid,
"used_seconds_today": round(worker_service.daily_usage_s(wid), 2),
"user_id": user.get("id")}
+86 -10
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import csv
import html
import io
import json
import logging
@@ -25,15 +26,36 @@ def _current_user(request: Request) -> dict:
return SessionManager.decode_session(s) or {"login": "admin", "id": 1}
# ── Workspaces ──
def _require_admin(request: Request) -> dict:
"""A15 : webhooks sortants = admin — le serveur POSTe le contenu des pages."""
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(401, "Authentication required")
if not user.get("is_admin"):
raise HTTPException(403, "Admin only")
return user
@router.get("")
async def list_workspaces(request: Request):
def _require_ws_admin(request: Request, ws_id: int) -> None:
"""A5 — CRUD des membres : session obligatoire + rôle admin de l'espace
(ou admin global). Un anonymous ne peut plus s'ajouter lui-même ni se
promouvoir admin."""
user = SessionManager.decode_session(request.cookies.get("flowdeck_session", ""))
if not user or not user.get("id"):
raise HTTPException(401, "Authentication required")
with get_conn() as conn:
rows = conn.execute("SELECT * FROM workspaces ORDER BY name").fetchall()
return {"workspaces": [dict(r) for r in rows]}
if conn.execute("SELECT 1 FROM users WHERE id=? AND is_admin=1", (user["id"],)).fetchone():
return
row = conn.execute(
"SELECT role FROM workspace_members WHERE workspace_id=? AND user_id=?",
(ws_id, user["id"]),
).fetchone()
if not row or row["role"] != "admin":
raise HTTPException(403, "Workspace admin role required")
# ── Workspaces ──
@router.post("")
async def create_workspace(request: Request):
body = await request.json() if request.headers.get("content-type") else {}
@@ -58,6 +80,8 @@ async def create_workspace(request: Request):
@router.get("/{ws_id}/members")
async def list_members(request: Request, ws_id: int):
if not SessionManager.decode_session(request.cookies.get("flowdeck_session", "")):
raise HTTPException(401, "Authentication required")
with get_conn() as conn:
rows = conn.execute(
"SELECT wm.*, u.login, u.full_name, u.avatar_url FROM workspace_members wm JOIN users u ON wm.user_id=u.id WHERE wm.workspace_id=?",
@@ -68,13 +92,14 @@ async def list_members(request: Request, ws_id: int):
@router.post("/{ws_id}/members")
async def add_member(request: Request, ws_id: int):
_require_ws_admin(request, ws_id)
body = await request.json() if request.headers.get("content-type") else {}
user_id = body.get("user_id")
role = body.get("role", "editor")
if role not in ROLES:
raise HTTPException(400, f"Invalid role: {role}")
with get_conn() as conn:
conn.execute("INSERT OR IGNORE INTO users (id, login, full_name, is_admin) VALUES (?,?,?,1)",
conn.execute("INSERT OR IGNORE INTO users (id, login, full_name, is_admin) VALUES (?,?,?,0)",
(user_id, f"user_{user_id}", f"User {user_id}"))
conn.execute("INSERT INTO workspace_members (workspace_id, user_id, role) VALUES (?,?,?)",
(ws_id, user_id, role))
@@ -84,6 +109,7 @@ async def add_member(request: Request, ws_id: int):
@router.put("/{ws_id}/members/{user_id}")
async def update_member_role(request: Request, ws_id: int, user_id: int):
_require_ws_admin(request, ws_id)
body = await request.json() if request.headers.get("content-type") else {}
role = body.get("role", "editor")
if role not in ROLES:
@@ -97,6 +123,7 @@ async def update_member_role(request: Request, ws_id: int, user_id: int):
@router.delete("/{ws_id}/members/{user_id}")
async def remove_member(request: Request, ws_id: int, user_id: int):
_require_ws_admin(request, ws_id)
with get_conn() as conn:
conn.execute("DELETE FROM workspace_members WHERE workspace_id=? AND user_id=?", (ws_id, user_id))
conn.commit()
@@ -130,6 +157,10 @@ async def add_comment(request: Request, page_id: int):
cur = conn.execute("INSERT INTO comments (page_id, user_id, body, parent_id) VALUES (?,?,?,?)",
(page_id, uid, b, parent_id))
conn.commit()
try:
await fire_event("comment.added", {"comment_id": cur.lastrowid, "page_id": page_id, "user_id": uid})
except Exception:
logger.exception("add_comment")
return {"id": cur.lastrowid, "status": "created"}
@@ -139,11 +170,17 @@ async def update_comment(request: Request, comment_id: int):
b = body.get("body")
resolved = body.get("resolved")
with get_conn() as conn:
row = conn.execute("SELECT page_id, resolved FROM comments WHERE id=?", (comment_id,)).fetchone()
if b is not None:
conn.execute("UPDATE comments SET body=?, updated_at=CURRENT_TIMESTAMP WHERE id=?", (b, comment_id))
if resolved is not None:
conn.execute("UPDATE comments SET resolved=? WHERE id=?", (int(resolved), comment_id))
conn.commit()
if resolved and row and not int(row["resolved"] or 0):
try:
await fire_event("comment.resolved", {"comment_id": comment_id, "page_id": row["page_id"]})
except Exception:
logger.exception("update_comment")
return {"status": "updated"}
@@ -206,6 +243,10 @@ async def add_favorite(request: Request):
(uid, page_id, collection_id),
)
conn.commit()
try:
await fire_event("favorite.added", {"page_id": page_id, "collection_id": collection_id, "user_id": uid})
except Exception:
logger.exception("add_favorite")
return {"status": "favorited"}
@@ -449,6 +490,10 @@ async def create_sprint(request: Request, collection_id: int):
(collection_id, name, start_date, end_date, goal, status, auto_complete),
)
conn.commit()
try:
await fire_event("sprint.created", {"sprint_id": cur.lastrowid, "collection_id": collection_id, "name": name})
except Exception:
logger.exception("create_sprint")
return {"id": cur.lastrowid, "name": name, "status": "created"}
@@ -475,6 +520,10 @@ async def update_sprint(request: Request, collection_id: int, sid: int):
(name, start_date, end_date, goal, status, auto_complete, sid),
)
conn.commit()
try:
await fire_event("sprint.updated", {"sprint_id": sid, "collection_id": collection_id, "name": name, "status": status})
except Exception:
logger.exception("update_sprint")
return {"id": sid, "status": "updated"}
@@ -642,6 +691,7 @@ async def export_csv(request: Request, collection_id: int):
@router.get("/webhooks")
async def list_webhooks(request: Request):
_require_admin(request)
with get_conn() as conn:
rows = conn.execute("SELECT * FROM webhook_subscriptions ORDER BY created_at DESC").fetchall()
return {"webhooks": [dict(r) for r in rows]}
@@ -649,12 +699,20 @@ async def list_webhooks(request: Request):
@router.post("/webhooks")
async def create_webhook(request: Request):
_require_admin(request)
body = await request.json() if request.headers.get("content-type") else {}
url = body.get("url", "").strip()
event = body.get("event", "page.created")
secret = body.get("secret", "")
if not url:
raise HTTPException(400, "url required")
# A15 : SSRF — le scheduler POSTe le contenu des pages vers cette URL.
from urllib.parse import urlparse
from app.services.importers.url_fetch import _is_public_host
parsed = urlparse(url)
if parsed.scheme not in ("http", "https") or not parsed.hostname or not _is_public_host(parsed.hostname):
raise HTTPException(400, f"url non autorisée: {parsed.hostname}")
with get_conn() as conn:
cur = conn.execute(
"INSERT INTO webhook_subscriptions (url, event, secret) VALUES (?,?,?)",
@@ -666,6 +724,7 @@ async def create_webhook(request: Request):
@router.delete("/webhooks/{wh_id}")
async def delete_webhook(request: Request, wh_id: int):
_require_admin(request)
with get_conn() as conn:
conn.execute("DELETE FROM webhook_subscriptions WHERE id=?", (wh_id,))
conn.commit()
@@ -676,22 +735,39 @@ async def delete_webhook(request: Request, wh_id: int):
@router.get("/public/{collection_id}")
async def public_view(request: Request, collection_id: int):
"""Simple public read-only view — no auth required."""
"""Simple public read-only view — no auth required.
A18 : les bases ``restricted``/``private`` (``permission_type``) restent
masquées (404) et toute interpolation part dans ``html.escape`` (XSS stocké
sur le titre de la base ou d'une ligne).
"""
with get_conn() as conn:
coll = conn.execute("SELECT * FROM collections WHERE id=?", (collection_id,)).fetchone()
if not coll:
raise HTTPException(404, "Collection not found")
ptype = coll["permission_type"] if "permission_type" in coll.keys() else "inherit"
if ptype in ("restricted", "private"):
# 404 explicite : le handler global transformerait un HTTPException(404)
# en redirection 302 → login pour un chemin HTML.
return HTMLResponse(
"<!DOCTYPE html><html><head><meta charset=\"utf-8\"><title>404</title></head>"
"<body><h1>404 — Not found</h1></body></html>",
status_code=404,
)
pages = conn.execute(
"SELECT id, title, icon, property_values_json FROM collection_pages WHERE collection_id=? ORDER BY position",
(collection_id,),
).fetchall()
esc = html.escape
name = esc(str(coll["name"] or ""))
icon = esc(str(coll["icon"] or ""))
items = "".join(
f"<li>{p['icon']} <b>{p['title']}</b></li>"
f"<li>{esc(str(p['icon'] or ''))} <b>{esc(str(p['title'] or ''))}</b></li>"
for p in pages
)
return HTMLResponse(f"""<!DOCTYPE html>
<html><head><meta charset="utf-8"><title>{coll['name']} — FlowDeck Public</title>
<html><head><meta charset="utf-8"><title>{name} — FlowDeck Public</title>
<style>body{{font-family:system-ui;background:#191919;color:#fff;padding:20px}}
h1{{font-size:24px}} li{{padding:8px;border-bottom:1px solid #333}}</style></head>
<body><h1>{coll['icon']} {coll['name']}</h1><ul>{items}</ul><p>{len(pages)} items</p></body></html>""")
<body><h1>{icon} {name}</h1><ul>{items}</ul><p>{len(pages)} items</p></body></html>""")
+60 -3
View File
@@ -18,15 +18,30 @@ import re
from app.config import settings
from app.db import get_conn
from app.services.agent_policies import check_tool, get_policy
from app.services.context_builder import ContextBuilder
from app.services.llm_client import LLMClient
from app.services.permission_manager import PermissionManager
from app.services.permission_manager import WRITE_TOOLS, PermissionManager
from app.services.tool_registry import ToolRegistry
logger = logging.getLogger(__name__)
MAX_ITERATIONS = 12
async def _fire_agent_webhook(event: str, payload: dict) -> None:
"""Dispatch an outbound agent lifecycle event (never raises).
Lifecycle: ``agent.run.started`` → ``agent.run.finished`` | ``agent.run.failed``.
All three are in the webhook_outbound catalogue, so integrations can subscribe
to `agent.*` and drive FlowDeck Agent from outside (Agent phase 5).
"""
try:
from app.services.webhook_outbound import fire_event as _fire_wh
await _fire_wh(event, payload)
except Exception: # noqa: BLE001
logger.debug("%s webhook dispatch failed", event)
# Compact in-app guide so the LLM can answer « comment faire… ? » questions even
# when no document is attached to the conversation (generic help / onboarding).
APP_GUIDE = """## Guide de l'utilisateur FlowDeck (sert à répondre aux questions « comment … ? »)
@@ -151,6 +166,11 @@ class AgentEngine:
self._persist_message(conversation_id, "user", objective)
self._update_conversation(conversation_id, status="running")
await _fire_agent_webhook("agent.run.started", {
"conversation_id": conversation_id,
"objective": objective[:500],
"model": model or "",
})
# Update the history title right away (before the run finishes) and
# refine it once we have the final answer (_autotitle below).
@@ -166,7 +186,10 @@ class AgentEngine:
used_model = model or "" # peut être ajusté par un repli de modèle (404/410)
try:
for _step in range(settings.agent_max_iterations or MAX_ITERATIONS):
# v7.2.0 — the workspace policy may cap iterations below the global max.
policy_max = get_policy(self.workspace_id).get("max_steps") or MAX_ITERATIONS
iterations = min(settings.agent_max_iterations or MAX_ITERATIONS, policy_max)
for _step in range(iterations):
if self._tokens >= settings.agent_max_tokens_budget:
yield self._event("error", {"message": "Budget de tokens dépassé"})
break
@@ -216,8 +239,30 @@ class AgentEngine:
tool, args = call["name"], call.get("arguments") or {}
call_id = tool_specs[idx]["id"]
denied = False
# v7.2.0 — workspace tool scope + human approval gate, checked
# *before* permissions so a scoped-out tool never reaches ACLs.
gov = check_tool(self.user_id, self.workspace_id, tool,
is_write=tool in WRITE_TOOLS,
conversation_id=conversation_id)
if not gov.get("allowed"):
detail = gov.get("reason") or "Refusé par la politique agent"
if gov.get("approval_id"):
detail = (f"Approbation requise (demande #{gov['approval_id']}) "
f"— action suspendue")
yield self._event("action", {
"tool": tool, "status": "approval_required" if gov.get("approval_id")
else "error", "detail": detail,
"approval_id": gov.get("approval_id")})
self._log_action(conversation_id, tool, args, {}, "error", detail=detail)
messages.append({
"role": "tool", "tool_call_id": call_id,
"content": json.dumps({"status": "error", "message": detail},
ensure_ascii=False),
})
denied = True
try:
self.perms.assert_can(tool, args, self.workspace_id, approval_mode)
if not denied:
self.perms.assert_can(tool, args, self.workspace_id, approval_mode)
except Exception as exc: # permission / approval guard
detail = self._exc_detail(exc)
yield self._event("action", {"tool": tool, "status": "error", "detail": detail})
@@ -259,9 +304,21 @@ class AgentEngine:
self._persist_message(conversation_id, "assistant", final_text,
model=used_model, tokens=self._tokens)
await self._autotitle(conversation_id, objective, final_text)
# v6.4.0: emit agent.run.finished (outbound webhooks only).
await _fire_agent_webhook("agent.run.finished", {
"conversation_id": conversation_id,
"objective": objective[:500],
"model": used_model,
"tokens": self._tokens,
})
except Exception as exc: # noqa: BLE001
logger.exception("AgentEngine run failed")
await _fire_agent_webhook("agent.run.failed", {
"conversation_id": conversation_id,
"objective": objective[:500],
"error": str(exc)[:500],
})
yield self._event("error", {"message": f"Erreur interne: {exc}"})
finally:
self._update_conversation(conversation_id, status="idle")
+89
View File
@@ -0,0 +1,89 @@
"""FlowDeck — agent governance (v7.2.0): workspace tool scope + approval gate.
``agent_policies``: ``allowed_tools_json`` (null = all tools), ``max_steps``,
``require_approval`` (write tools pause for a human). ``check_tool()`` is
consulted by ``AgentEngine`` before ``PermissionManager.assert_can``.
``agent.run.approval_requested`` is emitted on the outbound webhook bus so
external systems can subscribe. See ``docs/V72_Enterprise_SCIM_2FA.md``.
"""
from __future__ import annotations
import json
from app.db import get_conn
def get_policy(workspace_id: int | None) -> dict:
"""Effective policy (workspace row, else global row, else defaults)."""
with get_conn() as conn:
row = None
if workspace_id is not None:
row = conn.execute("SELECT * FROM agent_policies WHERE workspace_id=?",
(workspace_id,)).fetchone()
if row is None:
row = conn.execute("SELECT * FROM agent_policies WHERE workspace_id IS NULL"
).fetchone()
if not row:
return {"allowed_tools": None, "max_steps": 12, "require_approval": False}
d = dict(row)
try:
allowed = json.loads(d.get("allowed_tools_json")) if d.get("allowed_tools_json") else None
except (TypeError, ValueError):
allowed = None
return {"allowed_tools": allowed, "max_steps": d.get("max_steps") or 12,
"require_approval": bool(d.get("require_approval"))}
def check_tool(user_id: int, workspace_id: int | None, tool: str,
is_write: bool, conversation_id: int = 0) -> dict:
"""Policy gate for one tool call.
Returns {allowed: bool, approval_id: int|None}. Denied tools and gated
writes (pending approval) return allowed=False; the engine renders both
as action errors without executing.
"""
from app.services.permission_manager import WRITE_TOOLS
policy = get_policy(workspace_id)
allowed = policy["allowed_tools"]
if allowed is not None and tool not in set(allowed):
return {"allowed": False, "approval_id": None, "reason": "tool not in policy scope"}
if is_write or tool in WRITE_TOOLS:
if policy["require_approval"]:
with get_conn() as conn:
cur = conn.execute(
"""INSERT INTO agent_approvals
(conversation_id, tool, args_json, status, requester_id)
VALUES (?,?,?,?,?)""",
(conversation_id, tool, "{}", "pending", user_id))
conn.commit()
approval_id = cur.lastrowid
try:
import asyncio
from app.services.webhook_outbound import fire_event as _fire
try:
loop = asyncio.get_running_loop()
except RuntimeError:
loop = None
if loop is not None:
loop.create_task(_fire("agent.run.approval_requested", {
"approval_id": approval_id, "tool": tool,
"conversation_id": conversation_id}))
except Exception: # noqa: BLE001 — webhook never blocks policy
pass
return {"allowed": False, "approval_id": approval_id,
"reason": "approval requested"}
return {"allowed": True, "approval_id": None, "reason": ""}
def decide_approval(approval_id: int, approver_id: int, approve: bool) -> dict | None:
with get_conn() as conn:
row = conn.execute("SELECT * FROM agent_approvals WHERE id=?",
(approval_id,)).fetchone()
if not row or row["status"] != "pending":
return None
conn.execute("UPDATE agent_approvals SET status=?, approver_id=? WHERE id=?",
("approved" if approve else "rejected", approver_id, approval_id))
conn.commit()
return dict(conn.execute("SELECT * FROM agent_approvals WHERE id=?",
(approval_id,)).fetchone())
+313
View File
@@ -0,0 +1,313 @@
"""FlowDeck — helpers for API v2 (v6.3.0).
Pagination, ISO-8601, RFC7807 errors, hierarchical scopes, Bearer auth.
No duplication: thin wrappers over existing services.
"""
from __future__ import annotations
import hashlib
import json
import logging
import time
from datetime import UTC, datetime
from typing import Any
from fastapi import Header, HTTPException, Request
from fastapi.responses import JSONResponse
from app.config import settings
from app.db import get_conn
logger = logging.getLogger(__name__)
# ── ISO-8601 ──────────────────────────────────────────────────────────────
def to_iso8601(value: str | None) -> str | None:
if not value:
return None
# SQLite stores "YYYY-MM-DD HH:MM:SS" or with T; convert to UTC Z
try:
# try with seconds
for fmt in ("%Y-%m-%d %H:%M:%S", "%Y-%m-%dT%H:%M:%S", "%Y-%m-%d %H:%M:%S.%f", "%Y-%m-%dT%H:%M:%S.%f"):
try:
dt = datetime.strptime(value[:19], fmt[:8] if "." in value else fmt)
# SQLite has no tz => assume UTC
dt = dt.replace(tzinfo=UTC)
return dt.isoformat().replace("+00:00", "Z")
except ValueError:
continue
# fallback: if already ISO with T/Z, return as-is
if "T" in value:
return value
return value
except Exception:
return value
def row_to_dict(row, *, iso_fields: tuple[str, ...] = ("created_at", "updated_at", "created_at_ts", "last_login", "joined_at", "accessed_at", "fired_at", "start_date", "end_date", "logged_at", "last_seen_at", "last_used_at", "verified_at", "last_login_at")) -> dict:
if row is None:
return {}
d = dict(row)
for k in list(d.keys()):
if k in iso_fields and d[k]:
iso = to_iso8601(str(d[k]))
if iso:
d[k] = iso
# parse *_json columns
if k.endswith("_json") and isinstance(d[k], str):
try:
d[k] = json.loads(d[k] or "{}" if d[k].strip().startswith("{") or d[k].strip().startswith("[") else d[k])
except Exception:
logger.exception("row_to_dict")
return d
# ── Pagination ────────────────────────────────────────────────────────────
def parse_pagination(request: Request, default_limit: int = 30, max_limit: int = 100) -> tuple[int, int]:
try:
limit = int(request.query_params.get("limit", str(default_limit)))
except ValueError:
limit = default_limit
try:
offset = int(request.query_params.get("offset", "0"))
except ValueError:
offset = 0
limit = max(1, min(limit, max_limit))
offset = max(0, offset)
return limit, offset
def paginate_headers(total: int) -> dict[str, str]:
return {"X-Total-Count": str(total)}
# ── Scopes (hierarchical: read < write < admin) ──────────────────────────
SCOPE_RANK = {"read": 1, "write": 2, "admin": 3}
VALID_SCOPES = set(SCOPE_RANK.keys())
def normalize_scopes(raw: str | None) -> set[str]:
if not raw:
return set()
parts = [p.strip().lower() for p in raw.split(",") if p.strip()]
return {p for p in parts if p in VALID_SCOPES}
def has_scope(token_scopes: str | None, required: str) -> bool:
req_rank = SCOPE_RANK.get(required, 99)
# token with higher rank satisfies lower requirement
# admin => write => read
token_set = normalize_scopes(token_scopes)
if not token_set:
return False
# effective rank = max rank among token scopes
eff = max((SCOPE_RANK.get(s, 0) for s in token_set), default=0)
return eff >= req_rank
def validate_scopes_input(scopes_raw: str | None) -> str:
if not scopes_raw:
return "read"
parts = [p.strip().lower() for p in scopes_raw.split(",") if p.strip()]
for p in parts:
if p not in VALID_SCOPES:
raise HTTPException(status_code=400, detail=f"Invalid scope: {p}. Valid: read, write, admin")
if not parts:
return "read"
# dedup preserve order
seen = []
for p in parts:
if p not in seen:
seen.append(p)
return ",".join(seen)
# ── Bearer auth (unified) ─────────────────────────────────────────────────
def _hash_token(token: str) -> str:
return hashlib.sha256(token.encode()).hexdigest()
def resolve_bearer_token(token: str) -> dict | None:
"""Resolve Bearer token to user dict. Returns None if invalid/expired/revoked.
Supports api_tokens (hashed), extension_devices (hashed), and legacy user_tokens (plain).
"""
if not token:
return None
# dev-only fallback
if token == "fd-public-key":
if not settings.public_api_insecure_ok:
return None
# return a synthetic admin-like user? Use first admin or id 1
with get_conn() as conn:
row = conn.execute("SELECT id, login, full_name, email, is_admin FROM users WHERE is_admin=1 ORDER BY id LIMIT 1").fetchone()
if row:
d = dict(row)
d["_token_id"] = None
d["_token_scopes"] = "read,write,admin"
d["_token_hash"] = None
return d
row = conn.execute("SELECT id, login, full_name, email, is_admin FROM users ORDER BY id LIMIT 1").fetchone()
if row:
d = dict(row)
d["_token_id"] = None
d["_token_scopes"] = "read,write,admin"
d["_token_hash"] = None
return d
return None
th = _hash_token(token)
with get_conn() as conn:
# 1) api_tokens
row = conn.execute("SELECT id, user_id, scopes, expires_at, revoked FROM api_tokens WHERE token_hash=?", (th,)).fetchone()
if row:
if row["revoked"]:
return None
exp = row["expires_at"]
if exp:
try:
# compare as timestamp; SQLite format "YYYY-MM-DD HH:MM:SS"
# parse to epoch
dt = datetime.fromisoformat(str(exp).replace("Z", "+00:00")) if "T" in str(exp) else datetime.strptime(str(exp)[:19], "%Y-%m-%d %H:%M:%S")
if dt.tzinfo is None:
dt = dt.replace(tzinfo=UTC)
if dt.timestamp() < time.time():
return None
except Exception:
logger.exception("resolve_bearer_token")
u = conn.execute("SELECT id, login, full_name, email, is_admin FROM users WHERE id=?", (row["user_id"],)).fetchone()
if u:
d = dict(u)
d["_token_id"] = row["id"]
d["_token_scopes"] = row["scopes"] or "read,write"
d["_token_hash"] = th
# touch last_used_at best-effort
try:
conn.execute("UPDATE api_tokens SET last_used_at=CURRENT_TIMESTAMP WHERE id=?", (row["id"],))
conn.commit()
except Exception:
logger.exception("resolve_bearer_token")
return d
# 2) extension_devices
row = conn.execute("SELECT user_id, scopes FROM extension_devices WHERE token_hash=? AND revoked=0", (th,)).fetchone()
if row:
u = conn.execute("SELECT id, login, full_name, email, is_admin FROM users WHERE id=?", (row["user_id"],)).fetchone()
if u:
d = dict(u)
d["_token_id"] = None
d["_token_scopes"] = row["scopes"] or "read,write"
d["_token_hash"] = th
return d
# 3) legacy user_tokens (plain storage)
row = conn.execute("SELECT gitea_user_id FROM user_tokens WHERE gitea_token=?", (token,)).fetchone()
if row:
u = conn.execute("SELECT id, login, full_name, email, is_admin FROM users WHERE id=?", (row["gitea_user_id"],)).fetchone()
if u:
d = dict(u)
d["_token_id"] = None
d["_token_scopes"] = "read,write"
d["_token_hash"] = th
return d
return None
def get_bearer_user(request: Request, authorization: str | None = Header(default=None)) -> dict:
# Prefer explicit Authorization header, fallback to lowercase
auth = authorization or request.headers.get("authorization") or request.headers.get("Authorization") or ""
if not auth or not auth.lower().startswith("bearer "):
raise HTTPException(status_code=401, detail="API token required. Use Authorization: Bearer <token>")
token = auth[7:].strip()
user = resolve_bearer_token(token)
if not user:
raise HTTPException(status_code=401, detail="Invalid or expired API token")
return user
def require_scope(required: str):
def _dep(request: Request, authorization: str | None = Header(default=None)) -> dict:
user = get_bearer_user(request, authorization)
scopes = user.get("_token_scopes") or "read"
if not has_scope(scopes, required):
raise HTTPException(status_code=403, detail=f"Insufficient scope. Required: {required}, token scopes: {scopes}")
return user
return _dep
# ── RFC 7807 ──────────────────────────────────────────────────────────────
def problem_response(request: Request, exc: HTTPException) -> JSONResponse:
title_map = {
400: "Bad Request",
401: "Unauthorized",
403: "Forbidden",
404: "Not Found",
409: "Conflict",
422: "Unprocessable Entity",
429: "Too Many Requests",
500: "Internal Server Error",
}
status = exc.status_code
detail = exc.detail if isinstance(exc.detail, str) else str(exc.detail)
body = {
"type": f"https://flowdeck/api/errors/{status}",
"title": title_map.get(status, "Error"),
"status": status,
"detail": detail,
"instance": str(request.url.path),
}
return JSONResponse(status_code=status, content=body, media_type="application/problem+json")
# ── Audit ─────────────────────────────────────────────────────────────────
def audit_log(user: dict, action: str, resource_type: str = "", resource_id: str | int = "", detail: str = "", request: Request | None = None) -> None:
try:
ip = ""
if request and request.client:
ip = request.client.host or ""
with get_conn() as conn:
conn.execute(
"INSERT INTO api_audit_log (user_id, token_id, action, resource_type, resource_id, ip_address, detail) VALUES (?, ?, ?, ?, ?, ?, ?)",
(user.get("id"), user.get("_token_id"), action, resource_type, str(resource_id), ip, detail[:1000]),
)
conn.commit()
except Exception:
logger.exception("audit_log")
# ── Rate limit per token (in-memory) ─────────────────────────────────────
_v2_rate_store: dict[str, tuple[float, int]] = {}
def check_v2_rate_limit(token_hash: str | None, ip: str) -> bool:
"""Return True if allowed, False if 429. Uses api_v2_rate_limit_per_token."""
key = token_hash or f"ip:{ip}"
now = time.time()
window = 60.0
max_req = settings.api_v2_rate_limit_per_token
start, count = _v2_rate_store.get(key, (now, 0))
if now - start > window:
_v2_rate_store[key] = (now, 1)
return True
if count >= max_req:
return False
_v2_rate_store[key] = (start, count + 1)
return True
# ── Idempotency ───────────────────────────────────────────────────────────
def check_idempotency(request: Request, user_id: int) -> dict | None:
key = request.headers.get("Idempotency-Key") or request.headers.get("idempotency-key")
if not key:
return None
key = key.strip()[:200]
if not key:
return None
with get_conn() as conn:
row = conn.execute("SELECT response_json, status_code FROM idempotency_keys WHERE key=? AND user_id=?", (key, user_id)).fetchone()
if row:
try:
data = json.loads(row["response_json"])
return {"data": data, "status": row["status_code"], "key": key}
except Exception:
return None
return None
def store_idempotency(key: str, user_id: int, data: Any, status_code: int = 200) -> None:
if not key:
return
try:
with get_conn() as conn:
conn.execute(
"INSERT OR IGNORE INTO idempotency_keys (key, user_id, response_json, status_code) VALUES (?, ?, ?, ?)",
(key.strip()[:200], user_id, json.dumps(data), status_code),
)
conn.commit()
except Exception:
logger.exception("store_idempotency")
+435
View File
@@ -23,6 +23,7 @@ from __future__ import annotations
import asyncio
import json
import logging
import time
from datetime import datetime, timedelta
import httpx
@@ -167,6 +168,13 @@ async def _run_action(action: dict, context: dict, trigger_source: str) -> str:
url = action.get("url", "").strip()
if not url:
raise ValueError("webhook action requires a url")
# A13 : SSRF — même garde que l'importer URL (loopback/privé refusé).
from urllib.parse import urlparse as _urlparse
from app.services.importers.url_fetch import _is_public_host
_parsed = _urlparse(url)
if _parsed.scheme not in ("http", "https") or not _parsed.hostname or not _is_public_host(_parsed.hostname):
raise ValueError(f"webhook url non autorisée: {_parsed.hostname!r}")
secret = action.get("secret", "")
headers = {"Content-Type": "application/json", "X-FlowDeck-Event": context.get("event", "")}
if secret:
@@ -246,6 +254,43 @@ async def _run_action(action: dict, context: dict, trigger_source: str) -> str:
)
return f"notified user {user_id}"
if atype == "slack":
url = _secret_value(action.get("webhook_url") or action.get("url") or "")
if not url:
raise ValueError("slack action requires a webhook_url")
_, text = _maybe_convert_prediction(
action.get("text") or action.get("message") or "Automation fired", context)
return await _post_slack(url, text)
if atype == "email":
to = action.get("to", "")
_, subject = _maybe_convert_prediction(action.get("subject", "FlowDeck automation"), context)
_, body = _maybe_convert_prediction(action.get("body", action.get("message", "")), context)
return await _send_email_action(to, subject, body, context)
if atype == "forge_issue":
provider = (action.get("provider") or "gitea").lower()
owner = action.get("owner", "")
repo = action.get("repo", "")
if not owner or not repo:
raise ValueError("forge_issue requires owner + repo")
_, title = _maybe_convert_prediction(action.get("title", "Automation issue"), context)
_, body = _maybe_convert_prediction(action.get("body", ""), context)
return await _create_forge_issue(
provider, owner, repo, title, body,
labels=action.get("labels") or [],
user_id=context.get("created_by"),
)
if atype == "agent_trigger":
agent_id = action.get("agent_id")
if not agent_id:
raise ValueError("agent_trigger requires an agent_id")
_, message = _maybe_convert_prediction(action.get("message", ""), context)
return await _run_linked_agent(
int(agent_id), context.get("created_by") or 1,
context.get("workspace_id"), message, context)
raise ValueError(f"unknown action type: {atype!r}")
@@ -260,6 +305,11 @@ async def run_automation(automation_id: int, trigger_source: str, context: dict)
if not auto["enabled"]:
return {"status": "skipped", "detail": "automation disabled"}
# v7.0.0: chained steps take over when present (legacy path otherwise).
stepped = await _maybe_run_stepped(auto, trigger_source, context)
if stepped is not None:
return stepped
props = context.get("properties")
before = context.get("before_properties")
if not evaluate_conditions(auto["condition_json"], props, before):
@@ -283,6 +333,20 @@ async def run_automation(automation_id: int, trigger_source: str, context: dict)
detail = "; ".join(results)
_save_run(automation_id, trigger_source, "fired", detail,
ctx.get("collection_id"), ctx.get("page_id"))
# v6.4.0: emit automation.fired (goes through fire_event → outbound
# webhooks, but NOT back through automations to avoid recursion).
try:
from app.services.webhook_outbound import fire_event as _fire_wh
await _fire_wh("automation.fired", {
"automation_id": automation_id,
"name": auto["name"],
"trigger": trigger_source,
"collection_id": ctx.get("collection_id"),
"page_id": ctx.get("page_id"),
"detail": detail,
})
except Exception: # noqa: BLE001
logger.debug("automation.fired webhook dispatch failed")
return {"status": "fired", "detail": detail}
except Exception as exc: # noqa: BLE001 — record every failure in history
logger.warning("Automation %s failed: %s", automation_id, exc)
@@ -293,6 +357,15 @@ async def run_automation(automation_id: int, trigger_source: str, context: dict)
async def fire_event(event: str, payload: dict):
"""Dispatch an event to outbound webhooks and matching automations."""
# v7.3.0: page.updated → in-app notification to followers (throttled).
if event == "page.updated":
try:
from app.services.wiki import notify_followers_of_page_update
notify_followers_of_page_update(
payload.get("page_id"), payload.get("actor_id"),
payload.get("title") or "")
except Exception: # noqa: BLE001 — notifications are best-effort
logger.debug("followers notification failed for page.updated")
# Outbound webhooks (v2.1.0 machinery, previously called nowhere).
try:
from app.services.webhook_outbound import fire_event as fire_webhooks
@@ -306,14 +379,29 @@ async def fire_event(event: str, payload: dict):
WHERE trigger_type='event' AND event=? AND enabled=1""",
(event,),
).fetchall()
stepped_ids: set[int] = set()
try:
with get_conn() as _c:
stepped_ids = {r[0] for r in _c.execute(
"SELECT DISTINCT automation_id FROM automation_steps").fetchall()}
except Exception: # noqa: BLE001 — table missing on very old DBs
pass
for row in rows:
auto = dict(row)
if auto["id"] in stepped_ids:
continue # v7.0.0: handled by fire_stepped_event below (no double run)
if auto["collection_id"] and payload.get("collection_id") != auto["collection_id"]:
continue
context = dict(payload)
context["event"] = event
await run_automation(auto["id"], "event", context)
# v7.0.0: step-based automations (multi-trigger any/all, chains).
try:
await fire_stepped_event(event, payload)
except Exception: # noqa: BLE001
logger.debug("stepped dispatch failed for %s", event)
# ═══════════ Cron scheduling (trigger_type='cron') ═══════════
@@ -395,6 +483,353 @@ async def automation_scheduler():
await run_automation(auto["id"], "cron", context)
except Exception: # noqa: BLE001
logger.warning("Cron automation %s errored", auto["id"])
# v7.0.0: workers on a cron schedule share the same 60s loop.
try:
from app.services.workers import run_due_workers
await run_due_workers()
except Exception: # noqa: BLE001
logger.debug("worker cron iteration failed")
except Exception: # noqa: BLE001
logger.warning("automation_scheduler iteration failed")
await asyncio.sleep(60)
# ═══════════ v7.0.0 — multi-step automations (triggers/conditions/delay) ══
STEP_KINDS = ("trigger", "condition", "delay", "action")
STEP_ACTION_TYPES = ("webhook", "set_property", "create_page", "notify",
"slack", "email", "forge_issue", "agent_trigger")
ALL_MODE_WINDOW_S = 300.0
# mode=all bookkeeping (single-process): automation_id -> {event: timestamp}.
_ALL_PENDING: dict[int, dict[str, float]] = {}
def reset_all_pending() -> None:
"""Test helper: clear the mode=all arrival window."""
_ALL_PENDING.clear()
def _secret_value(stored: str | None) -> str:
"""Decrypt a Fernet secret, falling back to raw plaintext (legacy/tests)."""
if not stored:
return ""
try:
from app.services.sso_provisioning import decrypt_secret
decrypted = decrypt_secret(stored)
if decrypted:
return decrypted
except Exception: # noqa: BLE001
pass
if isinstance(stored, str) and not stored.startswith("gAAAAA"):
return stored
return ""
def validate_step(kind: str, config: dict) -> None:
"""Validate a step payload. Raises ValueError with a human message."""
from fastapi import HTTPException
if kind not in STEP_KINDS:
raise HTTPException(400, f"invalid kind: {kind!r} (want trigger|condition|delay|action)")
config = config or {}
if kind == "trigger":
if not config.get("event"):
raise HTTPException(400, "trigger step requires an event")
elif kind == "condition":
if config.get("op", "eq") not in COND_OPS:
raise HTTPException(400, f"invalid op: {config.get('op')!r}")
elif kind == "delay":
try:
seconds = int(config.get("seconds", 0))
except (TypeError, ValueError):
raise HTTPException(400, "delay step requires integer seconds") from None
if seconds < 0 or seconds > 86400:
raise HTTPException(400, "delay seconds must be 0..86400")
elif kind == "action":
if config.get("type") not in STEP_ACTION_TYPES:
raise HTTPException(400, f"invalid action type: {config.get('type')!r}")
def get_steps(automation_id: int) -> list[dict]:
"""Ordered steps of an automation (empty when legacy single-mode)."""
with get_conn() as conn:
rows = conn.execute(
"SELECT * FROM automation_steps WHERE automation_id=? ORDER BY position, id",
(automation_id,),
).fetchall()
out = []
for r in rows:
d = dict(r)
try:
d["config"] = json.loads(d.get("config_json") or "{}")
except (TypeError, json.JSONDecodeError):
d["config"] = {}
out.append(d)
return out
def _steps_by_kind(steps: list[dict]) -> dict[str, list[dict]]:
grouped: dict[str, list[dict]] = {"trigger": [], "condition": [],
"delay": [], "action": []}
for s in steps:
if s.get("kind") in grouped:
grouped[s["kind"]].append(s)
return grouped
def _step_trigger_matches(step_cfg: dict, event: str, payload: dict,
automation_collection_id: int | None) -> bool:
if step_cfg.get("event") != event:
return False
want_coll = step_cfg.get("collection_id") or automation_collection_id
if want_coll and payload.get("collection_id") != want_coll:
return False
return True
async def _run_with_steps(auto: dict, steps: list[dict], trigger_source: str,
context: dict) -> dict:
"""Execute a chained automation. Records one run row with per-step detail."""
grouped = _steps_by_kind(steps)
props = context.get("properties")
before = context.get("before_properties")
# Legacy single condition still applies on top of step conditions.
if not evaluate_conditions(auto.get("condition_json") or "[]", props, before):
_save_run(auto["id"], trigger_source, "skipped", "condition not met",
context.get("collection_id"), context.get("page_id"))
return {"status": "skipped", "detail": "condition not met"}
for cond in grouped["condition"]:
cfg = cond.get("config") or {}
if not match_condition_props(props, before, {
"property": cfg.get("property"), "op": cfg.get("op", "eq"),
"value": cfg.get("value")}):
_save_run(auto["id"], trigger_source, "skipped",
f"step condition not met: {cfg.get('property')}",
context.get("collection_id"), context.get("page_id"))
return {"status": "skipped", "detail": "step condition not met"}
ctx = dict(context)
ctx["automation_name"] = auto["name"]
ctx["created_by"] = auto["created_by"] or ctx.get("created_by")
ordered = sorted(steps, key=lambda s: (s.get("position", 0), s.get("id", 0)))
results = []
try:
for step in ordered:
kind = step.get("kind")
cfg = step.get("config") or {}
if kind in ("trigger", "condition"):
continue
if kind == "delay":
seconds = max(0, min(int(cfg.get("seconds", 0)), 300))
if seconds:
await asyncio.sleep(seconds)
results.append(f"delay {cfg.get('seconds', 0)}s")
elif kind == "action":
summary = await _run_action({"type": cfg.get("type"), **cfg}, ctx,
trigger_source)
results.append(summary)
detail = "; ".join(results) or "no steps executed"
_save_run(auto["id"], trigger_source, "fired", detail,
ctx.get("collection_id"), ctx.get("page_id"))
try:
from app.services.webhook_outbound import fire_event as _fire_wh
await _fire_wh("automation.fired", {
"automation_id": auto["id"], "name": auto["name"],
"trigger": trigger_source, "collection_id": ctx.get("collection_id"),
"page_id": ctx.get("page_id"), "detail": detail})
except Exception: # noqa: BLE001
logger.debug("automation.fired webhook dispatch failed")
return {"status": "fired", "detail": detail}
except Exception as exc: # noqa: BLE001
logger.warning("Automation %s (steps) failed: %s", auto["id"], exc)
_save_run(auto["id"], trigger_source, "error", str(exc),
ctx.get("collection_id"), ctx.get("page_id"))
return {"status": "error", "detail": str(exc)}
async def _maybe_run_stepped(auto: dict, trigger_source: str, context: dict) -> dict | None:
"""Run via steps when the automation has any; None → use legacy path."""
steps = get_steps(auto["id"])
if not steps:
return None
return await _run_with_steps(auto, steps, trigger_source, context)
def _match_stepped_automations(event: str, payload: dict) -> list[tuple[dict, list[dict]]]:
"""Automations (enabled) whose trigger steps match ``event`` + collection."""
with get_conn() as conn:
rows = conn.execute(
"""SELECT a.* FROM automations a
JOIN automation_steps s ON s.automation_id = a.id
WHERE a.enabled=1 AND s.kind='trigger' GROUP BY a.id"""
).fetchall()
matched = []
for row in rows:
auto = dict(row)
steps = get_steps(auto["id"])
triggers = [s for s in steps if s.get("kind") == "trigger"]
if any(_step_trigger_matches(t.get("config") or {}, event, payload,
auto.get("collection_id")) for t in triggers):
matched.append((auto, triggers))
return matched
async def fire_stepped_event(event: str, payload: dict) -> None:
"""Dispatch ``event`` to step-based automations (mode any/all).
Called from :func:`fire_event` after the legacy matcher. Unknown events
(not in the webhook catalogue) still work here — steps are independent
from outbound webhooks.
"""
now = time.time()
for auto, triggers in _match_stepped_automations(event, payload):
# Skip automations already handled by the legacy matcher to avoid
# double runs (legacy = trigger_type event + no steps).
if not get_steps(auto["id"]):
continue
mode = (auto.get("trigger_mode") or "any").lower()
if mode == "all":
pending = _ALL_PENDING.setdefault(auto["id"], {})
pending[event] = now
# Expire arrivals outside the window.
for ev in [e for e, ts in pending.items() if now - ts > ALL_MODE_WINDOW_S]:
del pending[ev]
wanted = {t.get("config", {}).get("event") for t in triggers}
if not wanted <= set(pending):
continue
_ALL_PENDING.pop(auto["id"], None)
context = dict(payload)
context["event"] = event
await run_automation(auto["id"], "event", context)
# ── v7.0.0 action backends (module-level = monkeypatchable in tests) ───────
async def _post_slack(webhook_url: str, text: str) -> str:
async with httpx.AsyncClient(timeout=10) as client:
resp = await client.post(webhook_url, json={"text": text})
if resp.status_code >= 400:
raise RuntimeError(f"slack webhook returned HTTP {resp.status_code}")
return f"slack → ({resp.status_code})"
async def _send_email_action(to: str, subject: str, body: str, context: dict) -> str:
from app.services import mailer
address = (to or "").strip()
if address.startswith("user:"):
try:
uid = int(address.split(":", 1)[1])
except ValueError:
raise ValueError(f"bad email target: {to!r}") from None
with get_conn() as conn:
row = conn.execute("SELECT email FROM users WHERE id=?", (uid,)).fetchone()
address = (row["email"] if row and row["email"] else "")
if not address:
raise ValueError(f"user {uid} has no email")
if not address:
address = None
with get_conn() as conn:
row = conn.execute("SELECT email FROM users WHERE id=?",
(context.get("created_by") or 1,)).fetchone()
if row and row["email"]:
address = row["email"]
if not address:
return "email skipped (no recipient)"
ok = mailer.send_email(address, subject or "FlowDeck automation", body or "")
return f"email → {address}" if ok else "email skipped (SMTP not configured)"
async def _create_forge_issue(provider: str, owner: str, repo: str, title: str,
body: str, labels: list | None = None,
user_id: int | None = None) -> str:
token = ""
if user_id:
with get_conn() as conn:
row = conn.execute(
"SELECT access_token FROM user_oauth_tokens WHERE user_id=? AND provider=?",
(user_id, provider)).fetchone()
token = (row["access_token"] if row else "") or ""
if provider == "github":
if not token:
raise ValueError("github action needs a linked GitHub account (token)")
async with httpx.AsyncClient(timeout=15) as client:
resp = await client.post(
f"https://api.github.com/repos/{owner}/{repo}/issues",
headers={"Authorization": f"Bearer {token}",
"Accept": "application/vnd.github+json"},
json={"title": title, "body": body,
"labels": labels or []} if labels else {"title": title, "body": body},
)
if resp.status_code >= 400:
raise RuntimeError(f"github returned HTTP {resp.status_code}")
return f"github issue #{resp.json().get('number')} in {owner}/{repo}"
# gitea (default)
from app.services.gitea_client import GiteaClient
gitea = GiteaClient(user_token=token or None)
issue = await gitea.create_issue(owner, repo, title, body)
return f"gitea issue #{issue.get('number')} in {owner}/{repo}"
async def _run_linked_agent(agent_id: int, user_id: int, workspace_id: int | None,
message: str, context: dict) -> str:
from app.services.agent_engine import AgentEngine
with get_conn() as conn:
agent = conn.execute("SELECT * FROM agents WHERE id=?", (agent_id,)).fetchone()
if not agent:
raise ValueError(f"agent {agent_id} not found")
cur = conn.execute(
"""INSERT INTO agent_conversations (agent_id, user_id, title, context_json)
VALUES (?,?,?,?)""",
(agent_id, user_id,
f"Automation: {context.get('automation_name', 'run')}",
json.dumps({"workspace_id": workspace_id})),
)
conv_id = cur.lastrowid
conn.commit()
objective = ((agent["system_instructions"] or "").strip()
or f"Exécute l'agent « {agent['name']} ».")
if message:
objective = f"{objective}\n\n{message}"
engine = AgentEngine(user_id, workspace_id, agent["model"] or None)
final = ""
async for _ev in engine.run(conv_id, objective, model=agent["model"]):
pass
with get_conn() as conn:
row = conn.execute(
"SELECT content FROM agent_messages WHERE conversation_id=? AND role='assistant'"
" ORDER BY id DESC LIMIT 1", (conv_id,)).fetchone()
final = (row["content"][:300] if row and row["content"] else "")
return f"agent « {agent['name']} » ran (conversation {conv_id})" + (f": {final}" if final else "")
# ── v7.0.0 native DB button ────────────────────────────────────────────────
async def press_button(collection_id: int, row_id: int, prop_ref: str | int,
user_id: int) -> dict:
"""Run the automation linked to a ``button`` property cell."""
with get_conn() as conn:
if isinstance(prop_ref, int) or str(prop_ref).isdigit():
prop = conn.execute(
"SELECT * FROM collection_properties WHERE id=? AND collection_id=?",
(int(prop_ref), collection_id)).fetchone()
else:
prop = conn.execute(
"SELECT * FROM collection_properties WHERE collection_id=? AND name=?",
(collection_id, prop_ref)).fetchone()
if not prop:
raise ValueError("button property not found")
prop = dict(prop)
if prop.get("prop_type") != "button":
raise ValueError("property is not a button")
auto_id = prop.get("button_automation_id")
if not auto_id:
raise ValueError("button has no linked automation")
row = conn.execute(
"SELECT id FROM collection_pages WHERE id=? AND collection_id=?",
(row_id, collection_id)).fetchone()
if not row:
raise ValueError("row not found")
context = get_page_context(row_id, collection_id)
context["created_by"] = user_id
return await run_automation(auto_id, "button", context)
+1 -1
View File
@@ -41,7 +41,7 @@ def backup_db(now: datetime | None = None) -> str | None:
with sqlite3.connect(str(db_path)) as conn:
conn.execute("PRAGMA wal_checkpoint(TRUNCATE)")
except Exception:
pass
logger.exception("backup_db")
dest_dir = _backup_dir()
filename = f"flowdeck-{now:%Y%m%d-%H%M%S}.db"
+103
View File
@@ -0,0 +1,103 @@
"""FlowDeck — Built-in page (block) templates (v5.12.0).
Global page templates used by the « + New page » picker. Built-ins live here
(code, versioned); user templates live in the ``page_global_templates``
table. Block shapes match the editor's storage format (see
``app/routers/board.py::save_page_blocks``) — ids are assigned client-side.
"""
from __future__ import annotations
import json
def _b(btype: str, content: str = "", **extra) -> dict:
out = {"type": btype, "content": content}
out.update(extra)
return out
BUILTIN_TEMPLATES: dict[str, dict] = {
"empty": {
"name": "Empty",
"icon": "📄",
"description": "A blank page.",
"blocks": [_b("paragraph")],
},
"meeting_notes": {
"name": "Meeting notes",
"icon": "🗒️",
"description": "Attendees, agenda, notes, action items.",
"blocks": [
_b("heading_1", "Meeting notes"),
_b("callout", "Date: · Time: · Attendees: ", icon="📅"),
_b("heading_2", "Agenda"),
_b("bulleted_list", "Topic 1"),
_b("bulleted_list", "Topic 2"),
_b("heading_2", "Notes"),
_b("paragraph"),
_b("heading_2", "Decisions"),
_b("bulleted_list"),
_b("heading_2", "Action items"),
_b("to_do", "Owner — due date", checked=False),
_b("to_do", "", checked=False),
],
},
"weekly_report": {
"name": "Weekly report",
"icon": "📊",
"description": "Wins, in progress, blockers, next week.",
"blocks": [
_b("heading_1", "Week of [[fddate:2026-01-05]]"),
_b("heading_2", "🎉 Wins"),
_b("bulleted_list"),
_b("heading_2", "🚧 In progress"),
_b("bulleted_list"),
_b("heading_2", "⛔ Blockers"),
_b("bulleted_list"),
_b("heading_2", "🗓️ Next week"),
_b("to_do", "", checked=False),
],
},
"todo_list": {
"name": "To-do list",
"icon": "✅",
"description": "A simple checklist.",
"blocks": [
_b("heading_1", "To-do"),
_b("to_do", "", checked=False),
_b("to_do", "", checked=False),
_b("to_do", "", checked=False),
],
},
"project_doc": {
"name": "Project doc",
"icon": "🚀",
"description": "Goals, status, team, links.",
"blocks": [
_b("heading_1", "Project title"),
_b("callout", "One-line description of the project.", icon="💡"),
_b("heading_2", "Goals"),
_b("numbered_list"),
_b("heading_2", "Status"),
_b("toggle", "This week", expanded=True, children=[_b("paragraph")]),
_b("heading_2", "Team"),
_b("bulleted_list"),
_b("heading_2", "Resources"),
_b("bulleted_list"),
],
},
}
def template_list() -> list[dict]:
"""Public shape of the built-in templates for the picker UI."""
return [
{"key": key, "name": t["name"], "icon": t["icon"],
"description": t["description"], "builtin": True}
for key, t in BUILTIN_TEMPLATES.items()
]
def blocks_json_for(key: str) -> str | None:
t = BUILTIN_TEMPLATES.get(key)
return json.dumps(t["blocks"]) if t else None
+528
View File
@@ -0,0 +1,528 @@
"""FlowDeck — external calendar sync (v7.1.0).
Bidirectional sync between a collection (date property) and an external
calendar: Google Calendar (REST) or any CalDAV server (raw REPORT/PUT, no
extra dependency). Tokens are Fernet-encrypted at rest.
Matching: ``collection_pages.external_event_id`` ↔ remote event id.
Conflicts (both sides changed since ``last_sync``): last-write-wins +
in-app ``calendar.conflict`` notification (manual edit resolves).
See ``docs/V71_Calendar_Meetings.md``.
"""
from __future__ import annotations
import asyncio
import json
import logging
import time
import uuid
from datetime import UTC, datetime, timedelta
import httpx
from app.db import get_conn
logger = logging.getLogger(__name__)
PROVIDERS = ("google", "caldav")
SYNC_LOOKBACK_DAYS = 30
SYNC_LOOKAHEAD_DAYS = 90
class SyncError(RuntimeError):
"""Raised when the remote calendar cannot be reached/authorized."""
# ── links ──────────────────────────────────────────────────────────────────
def _encrypt_tokens(creds: dict) -> str:
from app.services.sso_provisioning import encrypt_secret
return encrypt_secret(json.dumps(creds or {}))
def _decrypt_tokens(tokens_enc: str) -> dict:
if not tokens_enc:
return {}
try:
from app.services.sso_provisioning import decrypt_secret
raw = decrypt_secret(tokens_enc)
if raw:
return json.loads(raw)
except Exception: # noqa: BLE001
pass
try: # legacy plaintext (tests)
data = json.loads(tokens_enc)
return data if isinstance(data, dict) else {}
except Exception: # noqa: BLE001
return {}
def save_link(user_id: int, provider: str, collection_id: int,
credentials: dict, calendar_id: str = "primary",
date_property: str = "") -> dict:
if provider not in PROVIDERS:
raise ValueError(f"provider must be google|caldav, got {provider!r}")
with get_conn() as conn:
if not conn.execute("SELECT id FROM collections WHERE id=?",
(collection_id,)).fetchone():
raise ValueError("collection not found")
cur = conn.execute(
"""INSERT INTO calendar_links
(user_id, provider, tokens_enc, calendar_id, collection_id, date_property)
VALUES (?,?,?,?,?,?)
ON CONFLICT(user_id, provider, calendar_id) DO UPDATE SET
tokens_enc=excluded.tokens_enc, collection_id=excluded.collection_id,
date_property=excluded.date_property""",
(user_id, provider, _encrypt_tokens(credentials),
calendar_id or "primary", collection_id, date_property or ""))
conn.commit()
row = conn.execute(
"SELECT * FROM calendar_links WHERE user_id=? AND provider=? AND calendar_id=?",
(user_id, provider, calendar_id or "primary")).fetchone()
_ = cur
out = dict(row)
out.pop("tokens_enc", None)
return out
def list_links(user_id: int) -> list[dict]:
with get_conn() as conn:
rows = conn.execute(
"SELECT id, user_id, provider, calendar_id, collection_id,"
" date_property, last_sync, created_at FROM calendar_links WHERE user_id=?"
" ORDER BY id", (user_id,)).fetchall()
return [dict(r) for r in rows]
def delete_link(user_id: int, link_id: int) -> bool:
with get_conn() as conn:
cur = conn.execute("DELETE FROM calendar_links WHERE id=? AND user_id=?",
(link_id, user_id))
conn.commit()
return cur.rowcount > 0
def _load_link(link_id: int) -> dict | None:
with get_conn() as conn:
row = conn.execute("SELECT * FROM calendar_links WHERE id=?", (link_id,)).fetchone()
return dict(row) if row else None
# ── remote I/O (module-level = monkeypatchable) ────────────────────────────
def _remote_event(eid: str, title: str, start: str, description: str = "",
updated: str = "") -> dict:
return {"id": str(eid), "title": title or "Untitled", "start": start,
"description": description or "", "updated": updated or ""}
async def google_list_events(tokens: dict, calendar_id: str,
time_min: str, time_max: str) -> list[dict]:
access = tokens.get("access_token", "")
if not access:
raise SyncError("google link has no access_token — relink the calendar")
url = (f"https://www.googleapis.com/calendar/v3/calendars/{calendar_id}"
f"/events?singleEvents=true&orderBy=startTime"
f"&timeMin={time_min}&timeMax={time_max}")
async with httpx.AsyncClient(timeout=15) as client:
resp = await client.get(url, headers={"Authorization": f"Bearer {access}"})
if resp.status_code == 401:
raise SyncError("google token expired — relink the calendar")
if resp.status_code >= 400:
raise SyncError(f"google returned HTTP {resp.status_code}")
out = []
for item in resp.json().get("items", []):
start = (item.get("start") or {}).get("dateTime") or (item.get("start") or {}).get("date") or ""
out.append(_remote_event(item.get("id", ""), item.get("summary", ""),
start, item.get("description", ""),
item.get("updated", "")))
return out
async def google_push_event(tokens: dict, calendar_id: str, event: dict,
remote_id: str = "") -> str:
access = tokens.get("access_token", "")
if not access:
raise SyncError("google link has no access_token — relink the calendar")
body = {"summary": event.get("title", ""),
"description": event.get("description", ""),
"start": {"date": event.get("start", "")[:10]},
"end": {"date": event.get("start", "")[:10]}}
base = f"https://www.googleapis.com/calendar/v3/calendars/{calendar_id}/events"
async with httpx.AsyncClient(timeout=15) as client:
if remote_id:
resp = await client.patch(f"{base}/{remote_id}",
headers={"Authorization": f"Bearer {access}"}, json=body)
else:
resp = await client.post(base, headers={"Authorization": f"Bearer {access}"},
json=body)
if resp.status_code == 401:
raise SyncError("google token expired — relink the calendar")
if resp.status_code >= 400:
raise SyncError(f"google returned HTTP {resp.status_code}")
return str(resp.json().get("id", remote_id or ""))
_CALDAV_REPORT = """<?xml version="1.0" encoding="utf-8" ?>
<C:calendar-query xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
<D:prop><D:getetag/><C:calendar-data/></D:prop>
<C:filter><C:comp-filter name="VCALENDAR"><C:comp-filter name="VEVENT">
<C:time-range start="{start}" end="{end}"/>
</C:comp-filter></C:comp-filter></C:filter>
</C:calendar-query>"""
def _parse_caldav_events(xml_text: str) -> list[dict]:
"""Minimal multistatus → event parser (UID/SUMMARY/DTSTART/DESCRIPTION)."""
import re
import xml.etree.ElementTree as ET
events = []
try:
root = ET.fromstring(xml_text)
except ET.ParseError:
return []
ns = {"D": "DAV:", "C": "urn:ietf:params:xml:ns:caldav"}
for resp in root.findall("D:response", ns):
href = resp.findtext("D:href", default="", namespaces=ns)
data_el = resp.find(".//{urn:ietf:params:xml:ns:caldav}calendar-data")
if data_el is None or not data_el.text:
continue
ics = data_el.text
uid = re.search(r"^UID:(.+)$", ics, re.M)
summary = re.search(r"^SUMMARY:(.+)$", ics, re.M)
dtstart = re.search(r"^DTSTART(?:;[^:]*)?:(.+)$", ics, re.M)
desc = re.search(r"^DESCRIPTION:(.+)$", ics, re.M)
events.append(_remote_event(
(uid.group(1).strip() if uid else href.strip("/").split("/")[-1]),
summary.group(1).strip() if summary else "Untitled",
_ics_to_date(dtstart.group(1).strip()) if dtstart else "",
desc.group(1).strip() if desc else ""))
return events
def _ics_to_date(value: str) -> str:
value = value.strip()
if len(value) >= 8 and value[:8].isdigit():
return f"{value[:4]}-{value[4:6]}-{value[6:8]}"
return value[:10]
def _event_to_ics(uid: str, title: str, date: str, description: str = "") -> str:
stamp = datetime.now(UTC).strftime("%Y%m%dT%H%M%SZ")
day = (date or "")[:10].replace("-", "")
return (f"BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//FlowDeck//Sync//EN\r\n"
f"BEGIN:VEVENT\r\nUID:{uid}\r\nDTSTAMP:{stamp}\r\nDTSTART;VALUE=DATE:{day}\r\n"
f"SUMMARY:{title}\r\nDESCRIPTION:{description}\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n")
async def caldav_list_events(creds: dict, time_min: str, time_max: str) -> list[dict]:
url = creds.get("url", "")
if not url:
raise SyncError("caldav link needs a calendar url")
auth = (creds.get("username", ""), creds.get("password", ""))
body = _CALDAV_REPORT.format(
start=time_min.replace("-", "").split("T")[0] + "T000000Z",
end=time_max.replace("-", "").split("T")[0] + "T000000Z")
async with httpx.AsyncClient(timeout=15, auth=auth if auth[0] else None) as client:
resp = await client.request("REPORT", url, content=body,
headers={"Depth": "1",
"Content-Type": "application/xml"})
if resp.status_code == 401:
raise SyncError("caldav rejected credentials")
if resp.status_code >= 400:
raise SyncError(f"caldav returned HTTP {resp.status_code}")
return _parse_caldav_events(resp.text)
async def caldav_push_event(creds: dict, event: dict, remote_id: str = "") -> str:
url = (creds.get("url", "") or "").rstrip("/")
if not url:
raise SyncError("caldav link needs a calendar url")
auth = (creds.get("username", ""), creds.get("password", ""))
uid = remote_id or f"flowdeck-{uuid.uuid4().hex}@flowdeck"
href = f"{url}/{uid}.ics" if not remote_id else (
remote_id if remote_id.startswith("http") else f"{url}/{remote_id}")
ics = _event_to_ics(uid.split("@")[0], event.get("title", ""),
event.get("start", ""), event.get("description", ""))
async with httpx.AsyncClient(timeout=15, auth=auth if auth[0] else None) as client:
resp = await client.put(href, content=ics, headers={"Content-Type": "text/calendar"})
if resp.status_code >= 400:
raise SyncError(f"caldav returned HTTP {resp.status_code}")
return uid
# ── mapping + sync ─────────────────────────────────────────────────────────
def _date_prop_id(conn, collection_id: int, wanted: str = "") -> tuple[str, str] | None:
props = conn.execute(
"SELECT id, name FROM collection_properties WHERE collection_id=? ORDER BY position",
(collection_id,)).fetchall()
if wanted:
for p in props:
if str(p["id"]) == str(wanted) or p["name"] == wanted:
return str(p["id"]), p["name"]
return None
for p in props:
# prop_type lives in the row; fetch full rows only when needed
full = conn.execute("SELECT prop_type FROM collection_properties WHERE id=?",
(p["id"],)).fetchone()
if full and full["prop_type"] == "date":
return str(p["id"]), p["name"]
return None
def _row_date(values: dict, prop_id: str, prop_name: str) -> str:
raw = values.get(prop_id, values.get(prop_name, ""))
if isinstance(raw, dict):
raw = raw.get("date") or raw.get("value") or ""
return str(raw or "")
def _to_epoch(value: str | None) -> float:
if not value:
return 0.0
text = str(value).strip()
try:
if text.endswith("Z"):
dt = datetime.fromisoformat(text.replace("Z", "+00:00"))
else:
dt = datetime.fromisoformat(text[:19] if "T" in text else text[:19])
if dt.tzinfo is None:
dt = dt.replace(tzinfo=UTC)
return dt.timestamp()
except Exception: # noqa: BLE001
try:
return time.mktime(time.strptime(text[:10], "%Y-%m-%d"))
except Exception: # noqa: BLE001
return 0.0
def _window() -> tuple[str, str]:
now = datetime.now(UTC)
start = (now - timedelta(days=SYNC_LOOKBACK_DAYS)).strftime("%Y-%m-%dT00:00:00Z")
end = (now + timedelta(days=SYNC_LOOKAHEAD_DAYS)).strftime("%Y-%m-%dT00:00:00Z")
return start, end
async def sync_link(link_id: int) -> dict:
"""One bidirectional sync pass. Returns {pulled, pushed, conflicts}."""
link = _load_link(link_id)
if not link:
raise ValueError("link not found")
creds = _decrypt_tokens(link.get("tokens_enc") or "")
collection_id = link.get("collection_id")
if not collection_id:
raise ValueError("link has no collection")
with get_conn() as conn:
date_prop = _date_prop_id(conn, collection_id, link.get("date_property") or "")
if not date_prop:
raise ValueError("collection has no date property")
prop_id, prop_name = date_prop
tmin, tmax = _window()
if link["provider"] == "google":
remote = await google_list_events(creds, link.get("calendar_id") or "primary",
tmin, tmax)
else:
remote = await caldav_list_events(creds, tmin, tmax)
last_sync = _to_epoch(link.get("last_sync"))
pulled = pushed = conflicts = 0
with get_conn() as conn:
rows = conn.execute(
"SELECT id, title, property_values_json, updated_at,"
" COALESCE(external_event_id, '') AS xid FROM collection_pages"
" WHERE collection_id=?", (collection_id,)).fetchall()
local = {r["xid"]: dict(r) for r in rows if r["xid"]}
seen_remote: set[str] = set()
touched: set[int] = set() # rows written by this pull pass — never push back
for ev in remote:
eid = ev.get("id", "")
if not eid:
continue
seen_remote.add(eid)
day = (ev.get("start") or "")[:10]
if eid not in local:
values: dict = {}
values[prop_id] = day
max_pos = conn.execute(
"SELECT COALESCE(MAX(position), -1)+1 FROM collection_pages"
" WHERE collection_id=?", (collection_id,)).fetchone()[0]
conn.execute(
"""INSERT INTO collection_pages
(collection_id, title, position, property_values_json, external_event_id)
VALUES (?,?,?,?,?)""",
(collection_id, ev.get("title") or "Untitled", max_pos,
json.dumps(values), eid))
pulled += 1
continue
row = local[eid]
try:
values = json.loads(row["property_values_json"] or "{}")
except (TypeError, json.JSONDecodeError):
values = {}
local_day = _row_date(values, prop_id, prop_name)[:10]
remote_newer = _to_epoch(ev.get("updated")) > _to_epoch(row["updated_at"])
local_dirty = _to_epoch(row["updated_at"]) > last_sync and local_day != day
if remote_newer and local_dirty and local_day and day and local_day != day:
# Conflict: both sides moved → last-write-wins + notify.
if _to_epoch(ev.get("updated")) >= _to_epoch(row["updated_at"]):
values[prop_id] = day
conn.execute(
"UPDATE collection_pages SET property_values_json=?,"
" updated_at=CURRENT_TIMESTAMP WHERE id=?",
(json.dumps(values), row["id"]))
touched.add(row["id"])
conflicts += 1
_notify_conflict(conn, link, row, ev)
elif day and day != local_day:
values[prop_id] = day
conn.execute(
"UPDATE collection_pages SET property_values_json=?,"
" updated_at=CURRENT_TIMESTAMP WHERE id=?",
(json.dumps(values), row["id"]))
touched.add(row["id"])
pulled += 1
# Push local changes (created locally or edited after last_sync).
for xid, row in local.items():
if row["id"] in touched:
continue
if xid in seen_remote:
# Edited locally since last sync and remote unchanged → push.
if last_sync and _to_epoch(row["updated_at"]) > last_sync:
await _push(link, creds, row, prop_id, prop_name, xid)
pushed += 1
continue
# Remote deleted the event → drop the local id (keep the row).
conn.execute("UPDATE collection_pages SET external_event_id='' WHERE id=?",
(row["id"],))
# Rows never linked and recently touched → create remotely.
fresh = conn.execute(
"SELECT id, title, property_values_json, updated_at FROM collection_pages"
" WHERE collection_id=? AND COALESCE(external_event_id, '')=''",
(collection_id,)).fetchall()
for row in fresh:
try:
values = json.loads(row["property_values_json"] or "{}")
except (TypeError, json.JSONDecodeError):
values = {}
day = _row_date(values, prop_id, prop_name)[:10]
if not day:
continue
new_id = await _push(link, creds, dict(row), prop_id, prop_name, "")
conn.execute("UPDATE collection_pages SET external_event_id=? WHERE id=?",
(new_id, row["id"]))
pushed += 1
conn.execute("UPDATE calendar_links SET last_sync=CURRENT_TIMESTAMP WHERE id=?",
(link_id,))
conn.commit()
return {"pulled": pulled, "pushed": pushed, "conflicts": conflicts}
async def _push(link: dict, creds: dict, row: dict, prop_id: str,
prop_name: str, remote_id: str) -> str:
try:
values = json.loads(row.get("property_values_json") or "{}")
except (TypeError, json.JSONDecodeError):
values = {}
event = {"title": row.get("title") or "Untitled",
"start": _row_date(values, prop_id, prop_name),
"description": ""}
if link["provider"] == "google":
return await google_push_event(creds, link.get("calendar_id") or "primary",
event, remote_id)
return await caldav_push_event(creds, event, remote_id)
def _notify_conflict(conn, link: dict, row: dict, ev: dict) -> None:
try:
from app.services.notifications import create_notification
create_notification(
link["user_id"], link["user_id"], "calendar",
"Calendar sync conflict",
f"« {row.get('title') or 'Untitled'} » changed on both sides;"
f" kept the newest ({ev.get('start', '')[:10]}). Edit the row to resolve.",
resource_type="collection", resource_id=link.get("collection_id") or 0,
url=f"/db/{link.get('collection_id')}", conn=conn, commit=False)
except Exception: # noqa: BLE001 — notify must never break sync
pass
async def calendar_sync_scheduler(interval_seconds: int = 900) -> None:
"""Background loop: sync every link with a collection (15 min default)."""
while True:
try:
with get_conn() as conn:
ids = [r["id"] for r in conn.execute(
"SELECT id FROM calendar_links WHERE collection_id IS NOT NULL"
).fetchall()]
for link_id in ids:
try:
await sync_link(link_id)
except Exception as exc: # noqa: BLE001 — one link must not kill the loop
logger.debug("calendar sync link %s failed: %s", link_id, exc)
except Exception as exc: # noqa: BLE001
logger.debug("calendar_sync_scheduler: %s", exc)
await asyncio.sleep(interval_seconds)
# ── free/busy ──────────────────────────────────────────────────────────────
def freebusy(collection_id: int, date_from: str, date_to: str,
date_property: str = "") -> dict:
"""Busy/free weekdays in [date_from, date_to] (day granularity).
Expands recurrence rules server-side (``recurrence.expand_rule``).
"""
from app.services import recurrence as _rec
try:
start = datetime.strptime(date_from[:10], "%Y-%m-%d").date()
end = datetime.strptime(date_to[:10], "%Y-%m-%d").date()
except ValueError:
raise ValueError("use YYYY-MM-DD dates") from None
if end < start or (end - start).days > 370:
raise ValueError("range must be 1..370 days")
with get_conn() as conn:
date_prop = _date_prop_id(conn, collection_id, date_property)
if not date_prop:
raise ValueError("collection has no date property")
prop_id, prop_name = date_prop
rows = conn.execute(
"SELECT property_values_json FROM collection_pages WHERE collection_id=?",
(collection_id,)).fetchall()
busy: set[str] = set()
for r in rows:
try:
values = json.loads(r["property_values_json"] or "{}")
except (TypeError, json.JSONDecodeError):
continue
base = _row_date(values, prop_id, prop_name)
if not base:
continue
rec = (values.get("__recurrence__") or {})
rule = rec.get(prop_id) or rec.get(prop_name)
if rule:
try:
for occ in _rec.expand_rule(
base, rule, start.isoformat(), end.isoformat()):
busy.add(occ[:10])
except Exception: # noqa: BLE001 — bad rule, use base date only
busy.add(base[:10])
else:
if start.isoformat() <= base[:10] <= end.isoformat():
busy.add(base[:10])
days, free = [], []
day = start
while day <= end:
iso = day.isoformat()
days.append({"date": iso, "busy": iso in busy,
"weekend": day.weekday() >= 5})
if iso not in busy and day.weekday() < 5:
free.append(iso)
day += timedelta(days=1)
return {"collection_id": collection_id, "from": start.isoformat(),
"to": end.isoformat(), "days": days, "free_weekdays": free}
+3 -1
View File
@@ -95,7 +95,7 @@ SEED_TEMPLATES: list[dict] = [
{
"name": "Meeting notes",
"icon": "🗒️",
"description": "Notes de réunion avec participants et décisions.",
"description": "Notes de réunion avec participants, agenda, notes et actions.",
"schema": [
{"name": "Title", "type": "title"},
{"name": "Date", "type": "date"},
@@ -104,6 +104,8 @@ SEED_TEMPLATES: list[dict] = [
{"name": "Scheduled", "color": "gray"},
{"name": "Done", "color": "green"},
]},
{"name": "Agenda", "type": "text"},
{"name": "Notes", "type": "text"},
{"name": "Action items", "type": "multi_select"},
],
},
+146 -62
View File
@@ -1,9 +1,10 @@
"""FlowDeck — Media embeds (v5.5.0): provider detection + iframe rewriting.
Maps a raw http(s) URL to a provider-specific embed URL so that one generic
``embed`` block can render YouTube, Vimeo, Figma, Google Maps, Loom,
CodePen, Miro, Spotify, SoundCloud, Twitch, X/Twitter, Pinterest, Office
docs… exactly like Notion's universal embed.
``embed`` block can render YouTube, Vimeo, Figma, Google Maps, Google
Docs/Sheets/Slides, Loom, CodePen, Miro, Spotify, SoundCloud, Twitch,
X/Twitter, Pinterest, Microsoft Office docs… exactly like Notion's universal
embed.
Unknown/showable URLs (PDF, images, direct video/audio files, plain http)
fall back to a plain iframe so the link is still visible inline.
@@ -11,11 +12,7 @@ fall back to a plain iframe so the link is still visible inline.
from __future__ import annotations
import re
from urllib.parse import parse_qs, urlparse
_PROVIDERS = (
# (host pattern, extra path check, to_embed(url, path, query) -> str | None)
)
from urllib.parse import parse_qs, quote, urlparse
def _q(params, key):
@@ -23,9 +20,21 @@ def _q(params, key):
return vals[0] if vals else ""
def _embed_youtube(url: str, path: str, params) -> str | None:
m = re.search(r"((?:v|shorts|embed)/|be/)([A-Za-z0-9_-]{6,20})", path)
vid = m.group(2) if m else _q(params, "v")
def _host_matches(netloc: str, host: str) -> bool:
"""True when ``netloc`` is ``host`` or one of its subdomains."""
netloc = (netloc or "").lower().split(":")[0]
host = host.lower()
return netloc == host or netloc.endswith("." + host)
def _embed_youtube(url: str, path: str, params, ctx: dict) -> str | None:
m = re.search(r"/(?:v|shorts|embed|live)/([A-Za-z0-9_-]{6,20})", path)
vid = m.group(1) if m else _q(params, "v")
if not vid:
# youtu.be/<id> (short link) — the id is the first path segment.
seg = path.strip("/").split("/")[0]
if re.fullmatch(r"[A-Za-z0-9_-]{6,20}", seg or ""):
vid = seg
if not vid:
return None
start = _q(params, "t") or _q(params, "start")
@@ -33,98 +42,122 @@ def _embed_youtube(url: str, path: str, params) -> str | None:
return f"https://www.youtube.com/embed/{vid}{frag}"
def _embed_vimeo(url: str, path: str, params) -> str | None:
def _embed_vimeo(url: str, path: str, params, ctx: dict) -> str | None:
m = re.search(r"/(\d{6,12})", path)
if not m:
return None
return f"https://player.vimeo.com/video/{m.group(1)}"
def _embed_loom(url: str, path: str, params) -> str | None:
m = re.search(r"/(embed/)?([0-9a-f]{32})", path)
def _embed_loom(url: str, path: str, params, ctx: dict) -> str | None:
m = re.search(r"/(?:embed/|share/)?([0-9a-f]{32})", path)
if not m:
return None
return f"https://www.loom.com/embed/{m.group(2)}"
return f"https://www.loom.com/embed/{m.group(1)}"
def _embed_figma(url: str, path: str, params) -> str | None:
if "figma.com/file/" not in url and "figma.com/proto/" not in url:
def _embed_figma(url: str, path: str, params, ctx: dict) -> str | None:
clean = url.split("?", 1)[0]
if "figma.com/file/" not in clean and "figma.com/proto/" not in clean and "figma.com/design/" not in clean:
return None
m = re.search(r"(/[^/]+/[^?]+)", url.split("?", 1)[0])
if not m:
return None
return "https://www.figma.com/embed?embed_host=flowdeck&url=" + url.split("?", 1)[0]
return "https://www.figma.com/embed?embed_host=flowdeck&url=" + quote(clean, safe="")
def _embed_map(url: str, path: str, params) -> str | None:
def _embed_map(url: str, path: str, params, ctx: dict) -> str | None:
if "google.com/maps" not in url and "maps.app.goo.gl" not in url:
return None
return f"https://www.google.com/maps?output=embed&q={url}"
return "https://maps.google.com/maps?q=" + quote(url, safe="") + "&output=embed"
def _embed_codepen(url: str, path: str, params) -> str | None:
def _embed_gdocs(url: str, path: str, params, ctx: dict) -> str | None:
m = re.search(
r"docs\.google\.com/(document|spreadsheets|presentation|forms)/d/([A-Za-z0-9_-]+)", url
)
if not m:
return None
kind, doc_id = m.group(1), m.group(2)
if kind == "forms":
return f"https://docs.google.com/forms/d/{doc_id}/viewform?embedded=true"
return f"https://docs.google.com/{kind}/d/{doc_id}/preview"
def _embed_codepen(url: str, path: str, params, ctx: dict) -> str | None:
m = re.search(r"codepen\.io/([^/]+)/pen/([^/?#]+)", url)
if not m:
return None
return f"https://codepen.io/{m.group(1)}/embed/{m.group(2)}?default-tab=result"
def _embed_miro(url: str, path: str, params) -> str | None:
m = re.search(r"miro\.com/app/board/([^/?#]+)", url)
def _embed_miro(url: str, path: str, params, ctx: dict) -> str | None:
m = re.search(r"miro\.com/app/(?:board|live-embed)/([^/?#]+)", url)
if not m:
return None
return f"https://miro.com/app/live-embed/{m.group(1)}"
def _embed_spotify(url: str, path: str, params) -> str | None:
def _embed_spotify(url: str, path: str, params, ctx: dict) -> str | None:
m = re.search(r"/(track|playlist|album|episode|show|artist)/([A-Za-z0-9]+)", url)
if not m:
return None
return f"https://open.spotify.com/embed/{m.group(1)}/{m.group(2)}"
def _embed_soundcloud(url: str, path: str, params) -> str | None:
m = re.search(r"(?:soundcloud\.com/[^/]+/[^/?#]+)", url)
if not m:
def _embed_soundcloud(url: str, path: str, params, ctx: dict) -> str | None:
if "soundcloud.com" not in url:
return None
return f"https://w.soundcloud.com/player/?url={url}&color=%2300aaff"
return "https://w.soundcloud.com/player/?url=" + quote(url, safe="") + "&color=%2300aaff"
def _embed_twitch(url: str, path: str, params) -> str | None:
def _embed_twitch(url: str, path: str, params, ctx: dict) -> str | None:
if "twitch.tv" not in url:
return None
parent = (ctx.get("parent") or "localhost").replace("https://", "").replace("http://", "").split("/")[0]
video = re.search(r"twitch\.tv/videos/(\d+)", url)
if video:
return f"https://player.twitch.tv/?video={video.group(1)}&parent={parent}"
m = re.search(r"twitch\.tv/([^/?#]+)", url)
if not m:
if not m or m.group(1) in ("videos", "directory"):
return None
return f"https://player.twitch.tv/?channel={m.group(1)}&parent="
return f"https://player.twitch.tv/?channel={m.group(1)}&parent={parent}"
def _embed_twitter(url: str, path: str, params) -> str | None:
def _embed_twitter(url: str, path: str, params, ctx: dict) -> str | None:
if "twitter.com" not in url and "x.com" not in url:
return None
m = re.search(r"/([^/?#]+)/status/(\d+)", url)
m = re.search(r"/status(?:es)?/(\d+)", url)
if not m:
return f"https://platform.twitter.com/embed/Tweet.html?url={url}"
return f"https://platform.twitter.com/embed/Tweet.html?id={m.group(2)}"
return f"https://platform.twitter.com/embed/Tweet.html?url={quote(url, safe='')}"
return f"https://platform.twitter.com/embed/Tweet.html?id={m.group(1)}"
def _embed_pinterest(url: str, path: str, params) -> str | None:
def _embed_pinterest(url: str, path: str, params, ctx: dict) -> str | None:
if "pinterest" not in url:
return None
return f"https://pinterest.com/pin/embed?url={url}"
return f"https://pinterest.com/pin/embed?url={quote(url, safe='')}"
def _embed_files(url: str, path: str, params) -> str | None:
def _embed_office(url: str, path: str, params, ctx: dict) -> str | None:
low = url.lower().split("?", 1)[0]
if low.endswith((".doc", ".docx", ".xls", ".xlsx", ".ppt", ".pptx", ".odt", ".ods", ".odp")):
return "https://view.officeapps.live.com/op/embed.aspx?src=" + quote(url, safe="")
if "officeapps.live.com" in low or "sharepoint.com" in low or "1drv.ms" in low:
return "https://view.officeapps.live.com/op/embed.aspx?src=" + quote(url, safe="")
return None
def _embed_files(url: str, path: str, params, ctx: dict) -> str | None:
"""Direct media: PDF/images/videos/audio can live in a plain iframe."""
return url
# (host, handler) — order matters: more specific hosts first.
_HANDLERS = (
("youtube.com", _embed_youtube),
("youtu.be", _embed_youtube),
("vimeo.com", _embed_vimeo),
("loom.com", _embed_loom),
("figma.com", _embed_figma),
("docs.google.com", _embed_gdocs),
("google.com/maps", _embed_map),
("maps.app.goo.gl", _embed_map),
("codepen.io", _embed_codepen),
@@ -135,27 +168,54 @@ _HANDLERS = (
("twitch.tv", _embed_twitch),
("twitter.com", _embed_twitter),
("x.com", _embed_twitter),
("pinterest", _embed_pinterest),
("pinterest.", _embed_pinterest),
("office.com", _embed_office),
("officeapps.live.com", _embed_office),
("sharepoint.com", _embed_office),
("1drv.ms", _embed_office),
)
def embed_src(url: str) -> str | None:
_SCHEME_RE = re.compile(r"^([a-zA-Z][a-zA-Z0-9+.-]*):")
def _parse(url: str):
raw = url.strip()
if not raw:
return None, None, None
m = _SCHEME_RE.match(raw)
if m:
if m.group(1).lower() not in ("http", "https"):
return None, None, None # mailto:, tel:, javascript:, data:…
else:
raw = "https://" + raw
u = urlparse(raw)
if u.scheme not in ("http", "https") or not u.netloc:
return None, None, None
host = u.hostname or ""
if "." not in host and host != "localhost":
return None, None, None # a bare word is not a URL
return raw, u, parse_qs(u.query)
def embed_src(url: str, *, parent: str = "") -> str | None:
"""Return the embeddable iframe src for a URL, or None if it can't embed."""
if not url:
raw, u, params = _parse(url)
if raw is None:
return None
url = url.strip()
u = urlparse(url if url.startswith("http") else "https://" + url)
if u.scheme not in ("http", "https"):
return None
(u.hostname or "").lower().replace("www.", "")
ctx = {"parent": parent}
netloc = (u.netloc or "").lower()
params = parse_qs(u.query)
# Google Maps share links encode the query in the path (…&q=/maps/…)
encoded = url
for needle, handler in _HANDLERS:
if needle in netloc:
return handler(encoded, u.path, params)
return _embed_files(encoded, u.path, params)
if "/" in needle or needle.endswith("."):
if needle in raw.lower():
return handler(raw, u.path, params, ctx)
elif _host_matches(netloc, needle):
return handler(raw, u.path, params, ctx)
# Office documents hosted on arbitrary domains.
office = _embed_office(raw, u.path, params, ctx)
if office:
return office
return _embed_files(raw, u.path, params, ctx)
_IMAGE_EXT = re.compile(r"\.(png|jpe?g|gif|webp|svg|bmp|ico|avif)$", re.I)
@@ -165,12 +225,12 @@ _AUDIO_EXT = re.compile(r"\.(mp3|wav|ogg|oga|m4a|flac|aac)$", re.I)
def inline_kind(url: str) -> str | None:
"""Best inline renderer for a URL: 'iframe' | 'image' | 'pdf' | 'video' |
"""Best inline renderer for a URL: 'iframe' | 'image' | 'pdf' | 'video'
| 'audio'. Returns None when the URL should open in a new tab."""
if not url:
raw, u, _params = _parse(url)
if raw is None:
return None
url = url.strip()
path = urlparse(url).path
path = u.path or ""
if _IMAGE_EXT.search(path):
return "image"
if _PDF_EXT.search(path):
@@ -179,9 +239,33 @@ def inline_kind(url: str) -> str | None:
return "video"
if _AUDIO_EXT.search(path):
return "audio"
if url.startswith(("http://", "https://")):
return "iframe"
return None
return "iframe"
def provider(url: str) -> str:
"""Human-readable provider name for a URL (used by the editor)."""
raw, u, _params = _parse(url)
if raw is None:
return ""
netloc = (u.netloc or "").lower()
for needle, _handler in _HANDLERS:
if "/" in needle or needle.endswith("."):
if needle in raw.lower():
return needle.split(".")[0].rstrip(".")
elif _host_matches(netloc, needle):
name = needle.split(".")[0]
return "youtube" if name == "youtu" else name
return ""
def resolve_embed(url: str, *, parent: str = "") -> dict:
"""Resolve a URL to ``{src, kind, provider}`` for the generic embed block."""
kind = inline_kind(url)
return {
"src": embed_src(url, parent=parent) or "",
"kind": kind or "",
"provider": provider(url),
}
def embed_html(src: str, *, height: int = 520) -> str:
+47 -16
View File
@@ -390,6 +390,20 @@ def _md_to_blocks(md: str) -> list:
blocks.append({"type": "divider", "content": ""})
i += 1
continue
if re.match(r"^\s*[-*+]\s+\[[ xX]\]\s+", line):
flush_para()
while i < n:
s = lines[i].strip()
m2 = re.match(r"^[-*+]\s+\[([ xX])\]\s+(.*)$", s)
if not m2:
break
blocks.append({
"type": "to_do",
"content": m2.group(2).strip(),
"checked": m2.group(1).lower() == "x",
})
i += 1
continue
if re.match(r"^\s*[-*+]\s+", line):
flush_para()
while i < n:
@@ -410,21 +424,6 @@ def _md_to_blocks(md: str) -> list:
blocks.append({"type": "numbered_list", "content": m2.group(1).strip()})
i += 1
continue
mtodo = re.match(r"^\s*[-*+]\s+\[([ xX])\]\s+(.*)$", stripped)
if mtodo:
flush_para()
while i < n:
s = lines[i].strip()
m2 = re.match(r"^[-*+]\s+\[([ xX])\]\s+(.*)$", s)
if not m2:
break
blocks.append({
"type": "to_do",
"content": m2.group(2).strip(),
"checked": m2.group(1).lower() == "x",
})
i += 1
continue
mq = re.match(r"^>\s?(.*)$", stripped)
if mq and line == stripped:
flush_para()
@@ -530,6 +529,20 @@ def blocks_to_markdown(blocks: list) -> str:
out.append(f"[{url}]({url})" if url else "[embed]")
elif t == "table":
out.append(_table_to_markdown(b))
elif t == "synced":
synced_id = b.get("synced_id")
if synced_id:
try:
from app.services.synced_blocks import get_synced_block
sb = get_synced_block(synced_id)
if sb and sb.get("content"):
resolved = json.loads(sb["content"])
if isinstance(resolved, list):
out.append(blocks_to_markdown(resolved))
else:
out.append(str(resolved))
except Exception:
out.append(f"[Synced block {synced_id}]")
else:
out.append(c)
return "\n\n".join(filter(None, out))
@@ -626,6 +639,10 @@ def blocks_to_html(blocks: list) -> str:
icon = b.get("icon") or "💡"
bg = (b.get("style") or {}).get("bgColor", "#eef2ff")
parts.append(f'<div class="callout" style="background:{bg}"><span>{_text(icon, escape=False)}</span><div>{c}</div></div>')
elif t in ("mermaid", "equation_inline", "progress"):
# v7.3.0 blocks — server-rendered so export embeds real content
from app.services.wiki_blocks import render_block
parts.append(render_block(b))
elif t == "image":
src = b.get("src") or ""
alt = _text(b.get("alt"))
@@ -665,7 +682,7 @@ def blocks_to_html(blocks: list) -> str:
parts.append(f'<iframe src="{_text(url)}" style="width:100%;height:70vh;border:none;border-radius:8px;"></iframe>')
elif url:
from app.services.embeds import embed_src
src = embed_src(url) or url
src = b.get("embed_src") or embed_src(url) or url
height = 520
if b.get("height"):
try:
@@ -679,6 +696,20 @@ def blocks_to_html(blocks: list) -> str:
)
elif t == "table":
parts.append(_table_to_html(b))
elif t == "synced":
synced_id = b.get("synced_id")
if synced_id:
try:
from app.services.synced_blocks import get_synced_block
sb = get_synced_block(synced_id)
if sb and sb.get("content"):
resolved = json.loads(sb["content"])
if isinstance(resolved, list):
parts.append(blocks_to_html(resolved))
else:
parts.append(f"<p>{_text(resolved)}</p>")
except Exception:
parts.append(f"<p>[Synced block {synced_id}]</p>")
else:
parts.append(f"<p>{c}</p>")
return "\n".join(parts)
+27
View File
@@ -73,6 +73,33 @@ class GiteaClient:
self._set_cache(cache_key, data)
return data
async def get_repo_info(self, owner: str, repo: str) -> dict:
"""Repository metadata for an unfurl card (owner/name/branch/…)."""
cache_key = f"repo_info:{owner}:{repo}"
cached = self._cached(cache_key)
if cached:
return cached
async with httpx.AsyncClient(timeout=15) as client:
resp = await client.get(
f"{self._base}/repos/{owner}/{repo}",
headers=self._headers,
)
resp.raise_for_status()
info = resp.json()
repo_info = {
"id": info.get("id"),
"name": info.get("name"),
"owner": (info.get("owner") or {}).get("login", owner),
"full_name": info.get("full_name") or f"{owner}/{repo}",
"clone_url": info.get("clone_url", ""),
"html_url": info.get("html_url", ""),
"default_branch": info.get("default_branch", "main"),
"description": info.get("description") or "",
"language": info.get("language") or "",
}
self._set_cache(cache_key, repo_info)
return repo_info
async def get_user_orgs(self) -> list[dict]:
cache_key = "user_orgs"
cached = self._cached(cache_key)
+55 -1
View File
@@ -194,7 +194,7 @@ class GitHubAdapter(ForgeAdapter):
if langs:
repo_info["language"] = max(langs, key=langs.get)
except Exception:
pass
logger.exception("get_repo_info")
self._set_cache(cache_key, repo_info)
return repo_info
@@ -209,6 +209,60 @@ class GitHubAdapter(ForgeAdapter):
resp.raise_for_status()
return resp.json()
# ── issues / labels / milestones ──
async def list_issues(self, owner: str, repo: str, state: str = "all") -> list[dict]:
"""List issues (pull requests are filtered out)."""
issues: list[dict] = []
for page in range(1, 6):
async with self._client() as client:
resp = await client.get(
f"{self._base}/repos/{owner}/{repo}/issues",
headers=self._headers,
params={"state": state, "per_page": 100, "page": page},
)
resp.raise_for_status()
batch = resp.json()
if not batch:
break
issues.extend(i for i in batch if "pull_request" not in i)
if len(batch) < 100:
break
return issues
async def list_labels(self, owner: str, repo: str) -> list[dict]:
async with self._client() as client:
resp = await client.get(
f"{self._base}/repos/{owner}/{repo}/labels",
headers=self._headers,
params={"per_page": 100},
)
resp.raise_for_status()
return resp.json()
async def list_milestones(self, owner: str, repo: str, state: str = "all") -> list[dict]:
async with self._client() as client:
resp = await client.get(
f"{self._base}/repos/{owner}/{repo}/milestones",
headers=self._headers,
params={"state": state, "per_page": 100},
)
resp.raise_for_status()
return resp.json()
async def list_repo_files(self, owner: str, repo: str, path: str = "") -> list[dict]:
"""Flatten the repo tree into file entries (``path``, ``size``)."""
tree = await self.get_repo_tree(owner, repo)
prefix = path.strip("/")
files = [
{"path": item["path"], "size": item.get("size", 0)}
for item in tree
if item.get("type") == "blob" and item.get("path")
]
if prefix:
files = [f for f in files if f["path"].startswith(prefix + "/") or f["path"] == prefix]
return files
# ── token validation ──
async def validate_token(self) -> bool:
+58
View File
@@ -0,0 +1,58 @@
"""FlowDeck — importers package (v5.6.0).
Importing this package registers every built-in importer. Use
:func:`parse_upload` to detect a source and :func:`run_import` to persist it.
"""
from __future__ import annotations
from app.services.importers import ( # noqa: F401 - registration side effects
bookmarks,
calendar,
docx,
html_notes,
markdown,
notion,
obsidian,
opml,
outline,
pdf,
standard_notes,
tabular,
)
from app.services.importers.base import ( # noqa: F401
ImportAttachment,
Importer,
ImportPage,
ImportResult,
all_importers,
detect_importer,
get_importer,
list_sources,
)
from app.services.importers.jobs import ( # noqa: F401
create_job,
get_job,
list_jobs,
parse_upload,
start_import_job,
)
from app.services.importers.pipeline import preview_result, resolve_relations, run_import # noqa: F401
__all__ = [
"ImportAttachment",
"ImportPage",
"ImportResult",
"Importer",
"all_importers",
"detect_importer",
"get_importer",
"list_sources",
"create_job",
"get_job",
"list_jobs",
"parse_upload",
"start_import_job",
"preview_result",
"run_import",
"resolve_relations",
]
+106
View File
@@ -0,0 +1,106 @@
"""FlowDeck — shared helpers for note importers (frontmatter, wikilinks)."""
from __future__ import annotations
import re
from typing import Any
try: # PyYAML ships transitively via uvicorn[standard]
import yaml
except Exception: # pragma: no cover - fallback parser below
yaml = None
_FRONTMATTER_RE = re.compile(r"^\ufeff?---\s*\n(.*?)\n---\s*\n?", re.DOTALL)
_WIKILINK_RE = re.compile(r"(!?)\[\[([^\]|#]+)(?:#[^\]|]+)?(?:\|([^\]]+))?\]\]")
def split_frontmatter(text: str) -> tuple[dict[str, Any], str]:
"""Split YAML frontmatter from the body. Returns ``(metadata, body)``."""
m = _FRONTMATTER_RE.match(text)
if not m:
return {}, text
raw = m.group(1)
body = text[m.end():]
if yaml is not None:
try:
meta = yaml.safe_load(raw)
if isinstance(meta, dict):
return meta, body
except Exception: # noqa: BLE001 - fall back to the simple parser
pass
return _simple_yaml(raw), body
def _simple_yaml(raw: str) -> dict[str, Any]:
"""Minimal YAML subset parser (scalars, inline lists, block lists)."""
meta: dict[str, Any] = {}
current: str | None = None
for line in raw.splitlines():
if not line.strip() or line.lstrip().startswith("#"):
continue
if line.lstrip().startswith("- ") and current:
meta.setdefault(current, [])
if isinstance(meta[current], list):
meta[current].append(_scalar(line.lstrip()[2:].strip()))
continue
if ":" in line:
key, _, value = line.partition(":")
key = key.strip()
value = value.strip()
current = key
if not value:
meta[key] = []
elif value.startswith("[") and value.endswith("]"):
inner = value[1:-1].strip()
meta[key] = [_scalar(v.strip()) for v in inner.split(",") if v.strip()] if inner else []
else:
meta[key] = _scalar(value)
return meta
def _scalar(value: str) -> Any:
v = value.strip().strip('"').strip("'")
if v.lower() in ("true", "false"):
return v.lower() == "true"
if re.fullmatch(r"-?\d+", v):
return int(v)
if re.fullmatch(r"-?\d+\.\d+", v):
return float(v)
return v
def convert_wikilinks(text: str, *, embeds: bool = True) -> str:
"""Turn Obsidian/Logseq ``[[link]]`` into Markdown links and ``![[img]]``
into Markdown images so the block converter can render them."""
def repl(m: re.Match) -> str:
bang, target, alias = m.group(1), m.group(2).strip(), m.group(3)
label = (alias or target).strip()
if bang == "!":
return f"![{label}]({target})" if embeds else label
return f"[{label}]({target})"
return _WIKILINK_RE.sub(repl, text)
def normalize_title(value: Any) -> str:
return str(value).strip() if value is not None else ""
def coerce_tags(value: Any) -> list[str]:
if value is None:
return []
if isinstance(value, list):
return [str(v).strip().lstrip("#") for v in value if str(v).strip()]
if isinstance(value, str):
parts = re.split(r"[,\s]+", value)
return [p.strip().lstrip("#") for p in parts if p.strip()]
return [str(value)]
def strip_markdown(text: str) -> str:
text = re.sub(r"`{1,3}([^`]*)`{1,3}", r"\1", text)
text = re.sub(r"!\[[^\]]*\]\([^)]*\)", "", text)
text = re.sub(r"\[([^\]]*)\]\([^)]*\)", r"\1", text)
text = re.sub(r"[*_~#>]+", "", text)
return text.strip()
+147
View File
@@ -0,0 +1,147 @@
"""FlowDeck — unified import framework (v5.6.0, Phase 0).
Defines the normalized data model shared by every importer and the registry
used to auto-detect a source. An :class:`Importer` turns an uploaded file into
an :class:`ImportResult` (pages, attachments, warnings, stats) which the
pipeline (:mod:`app.services.importers.pipeline`) persists into FlowDeck.
"""
from __future__ import annotations
from abc import ABC, abstractmethod
from dataclasses import dataclass, field
from typing import Any
def decode_text(data: bytes) -> str:
"""Best-effort decode of uploaded bytes (BOM aware, latin-1 fallback)."""
for enc in ("utf-8-sig", "utf-8", "utf-16", "latin-1"):
try:
return data.decode(enc)
except (UnicodeDecodeError, UnicodeError):
continue
return data.decode("utf-8", errors="replace")
@dataclass
class ImportAttachment:
"""A binary asset extracted from an archive/vault."""
source_path: str
filename: str
data: bytes = b""
mime: str = ""
@dataclass
class ImportPage:
"""One page to create. ``markdown`` is converted to blocks by the pipeline
unless ``blocks`` is already provided. ``collection`` marks a database
(Notion database, Excel sheet…) whose rows become ``collection_pages``."""
title: str = "Untitled"
markdown: str = ""
blocks: list[dict] = field(default_factory=list)
source_path: str = ""
parent_path: str = ""
properties: dict[str, Any] = field(default_factory=dict)
collection: dict[str, Any] | None = None
external_id: str = ""
page_id: int | None = None
@dataclass
class ImportResult:
"""Normalized output of any importer."""
source: str = ""
pages: list[ImportPage] = field(default_factory=list)
attachments: list[ImportAttachment] = field(default_factory=list)
warnings: list[str] = field(default_factory=list)
stats: dict[str, Any] = field(default_factory=dict)
def warn(self, message: str) -> None:
if message and message not in self.warnings:
self.warnings.append(message)
def finalize(self) -> ImportResult:
self.stats.setdefault("pages", len(self.pages))
self.stats.setdefault("collections", sum(1 for p in self.pages if p.collection))
self.stats.setdefault("attachments", len(self.attachments))
self.stats.setdefault("warnings", len(self.warnings))
return self
class Importer(ABC):
"""Base class for a source importer."""
source_id: str = ""
label: str = ""
description: str = ""
extensions: tuple[str, ...] = ()
order: int = 100
def detect(self, filename: str, data: bytes) -> bool:
"""Return True when this importer recognizes the uploaded file."""
return False
@abstractmethod
def parse(self, filename: str, data: bytes) -> ImportResult:
"""Parse the upload into a normalized :class:`ImportResult`."""
def info(self) -> dict[str, Any]:
return {
"source_id": self.source_id,
"label": self.label,
"description": self.description,
"extensions": list(self.extensions),
}
REGISTRY: list[Importer] = []
def register_importer(cls: type[Importer]) -> type[Importer]:
"""Class decorator registering an importer instance."""
REGISTRY.append(cls())
REGISTRY.sort(key=lambda i: i.order)
return cls
def all_importers() -> list[Importer]:
return list(REGISTRY)
def get_importer(source_id: str) -> Importer | None:
for imp in REGISTRY:
if imp.source_id == source_id:
return imp
return None
def detect_importer(filename: str, data: bytes) -> Importer | None:
"""First importer that recognizes the file, else None."""
for imp in REGISTRY:
try:
if imp.detect(filename, data):
return imp
except Exception: # noqa: BLE001 - a broken detector must not break detection
continue
return None
def list_sources() -> list[dict[str, Any]]:
return [imp.info() for imp in REGISTRY]
def make_collection(name: str, schema: list[dict], rows: list[dict],
*, source_path: str = "", external_id: str = "") -> ImportPage:
"""Build an ImportPage carrying a collection spec (database import).
``rows`` entries are ``{"title": str, "properties": {name: value}}``.
"""
return ImportPage(
title=name or "Imported database",
collection={"name": name or "Imported database", "schema": schema, "rows": rows},
source_path=source_path or name,
external_id=external_id or source_path or name,
)
+284
View File
@@ -0,0 +1,284 @@
"""FlowDeck — bookmark importers (v5.6.0, Phase 4).
Raindrop.io, Pocket, Readwise, Shaarli and generic Netscape bookmark files are
normalized into a FlowDeck collection (URL, description, tags, created date).
"""
from __future__ import annotations
import csv
import io
import json
import re
from datetime import UTC, datetime
from typing import Any
from app.services.importers._common import coerce_tags
from app.services.importers.base import (
Importer,
ImportResult,
decode_text,
make_collection,
register_importer,
)
_SCHEMA = [
{"name": "Title", "type": "title"},
{"name": "URL", "type": "url"},
{"name": "Description", "type": "text"},
{"name": "Tags", "type": "multi_select"},
{"name": "Created", "type": "date"},
]
_TAG_RE = re.compile(
r"<h3[^>]*>(?P<folder>.*?)</h3>|<a\s+(?P<attrs>[^>]*?)>(?P<title>.*?)</a>",
re.IGNORECASE | re.DOTALL,
)
_ATTR_RE = re.compile(r'([a-zA-Z_:-]+)\s*=\s*"([^"]*)"')
def _strip_tags(value: str) -> str:
return re.sub(r"<[^>]+>", "", value or "").strip()
def _iso_from_epoch(value: Any) -> str:
try:
return datetime.fromtimestamp(int(str(value)[:10]), tz=UTC).date().isoformat()
except (ValueError, TypeError, OSError, OverflowError):
return ""
def _iso(value: Any) -> str:
text = str(value or "").strip()
if not text:
return ""
if re.fullmatch(r"\d{10}", text):
return _iso_from_epoch(text)
return text[:10] if re.match(r"^\d{4}-\d{2}-\d{2}", text) else text
def _row(title: str, url: str, description: str = "", tags=None, created: str = "") -> dict:
props: dict[str, Any] = {}
if url:
props["URL"] = url
if description:
props["Description"] = description
tag_list = coerce_tags(tags)
if tag_list:
props["Tags"] = tag_list
if created:
props["Created"] = created
return {"title": (title or url or "Bookmark").strip()[:200], "properties": props}
def parse_netscape(html: str) -> list[dict]:
"""Parse a Netscape bookmark file (browser / Pocket / Raindrop HTML)."""
rows: list[dict] = []
folder = ""
for match in _TAG_RE.finditer(html):
if match.group("folder") is not None:
folder = _strip_tags(match.group("folder"))
continue
attrs = dict(_ATTR_RE.findall(match.group("attrs") or ""))
url = attrs.get("href") or attrs.get("HREF") or ""
if not url:
continue
title = _strip_tags(match.group("title"))
tags = attrs.get("tags") or attrs.get("TAGS") or folder
rows.append(_row(title, url, tags=tags, created=_iso_from_epoch(attrs.get("add_date", ""))))
return rows
def _csv_rows(text: str) -> list[dict]:
reader = csv.DictReader(io.StringIO(text))
return [{(k or "").strip().lower(): v for k, v in row.items()} for row in reader]
class _BookmarkBase(Importer):
source_id = "bookmarks"
label = "Signets"
description = ""
order = 44
keywords: tuple[str, ...] = ()
def _hint(self, filename: str, text: str) -> bool:
low = filename.lower()
return any(k in low or k in text.lower() for k in self.keywords)
@register_importer
class RaindropImporter(_BookmarkBase):
source_id = "raindrop"
label = "Raindrop.io"
description = "Export Raindrop.io (CSV ou HTML) → collection de signets."
extensions = (".csv", ".html", ".htm")
order = 46
keywords = ("raindrop",)
def detect(self, filename: str, data: bytes) -> bool:
text = decode_text(data)
if not self._hint(filename, text):
return False
return filename.lower().endswith((".csv", ".html", ".htm"))
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
text = decode_text(data)
rows: list[dict] = []
if filename.lower().endswith(".csv"):
for r in _csv_rows(text):
rows.append(_row(
r.get("title", ""), r.get("url", ""),
r.get("note") or r.get("excerpt") or "",
r.get("tags", ""), _iso(r.get("created", "")),
))
else:
rows = parse_netscape(text)
result.pages.append(make_collection("Raindrop", _SCHEMA, rows, source_path=filename))
result.stats["rows"] = len(rows)
return result.finalize()
@register_importer
class PocketImporter(_BookmarkBase):
source_id = "pocket"
label = "Pocket"
description = "Export Pocket (CSV ou HTML) → collection de signets."
extensions = (".csv", ".html", ".htm")
order = 45
keywords = ("pocket",)
def detect(self, filename: str, data: bytes) -> bool:
text = decode_text(data)
if not self._hint(filename, text):
return False
return filename.lower().endswith((".csv", ".html", ".htm"))
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
text = decode_text(data)
rows: list[dict] = []
if filename.lower().endswith(".csv"):
for r in _csv_rows(text):
rows.append(_row(
r.get("title", ""), r.get("url", ""),
"", r.get("tags", ""), _iso(r.get("time_added", "")),
))
else:
rows = parse_netscape(text)
result.pages.append(make_collection("Pocket", _SCHEMA, rows, source_path=filename))
result.stats["rows"] = len(rows)
return result.finalize()
@register_importer
class ReadwiseImporter(_BookmarkBase):
source_id = "readwise"
label = "Readwise"
description = "Export Readwise (highlights CSV) → collection de surlignages."
extensions = (".csv", ".md", ".markdown")
order = 47
keywords = ("readwise",)
def detect(self, filename: str, data: bytes) -> bool:
text = decode_text(data)
if not self._hint(filename, text):
return False
if filename.lower().endswith(".csv"):
header = text.splitlines()[0].lower() if text.strip() else ""
return "highlight" in header or "book title" in header
return True
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
text = decode_text(data)
schema = [
{"name": "Highlight", "type": "title"},
{"name": "Book", "type": "text"},
{"name": "Author", "type": "text"},
{"name": "Note", "type": "text"},
{"name": "Tags", "type": "multi_select"},
{"name": "Highlighted at", "type": "date"},
]
rows: list[dict] = []
if filename.lower().endswith(".csv"):
for r in _csv_rows(text):
props: dict[str, Any] = {}
if r.get("book title"):
props["Book"] = r["book title"]
if r.get("book author"):
props["Author"] = r["book author"]
if r.get("note"):
props["Note"] = r["note"]
tags = coerce_tags(r.get("document tags") or r.get("tags"))
if tags:
props["Tags"] = tags
created = _iso(r.get("highlighted at", ""))
if created:
props["Highlighted at"] = created
rows.append({"title": (r.get("highlight") or "Highlight").strip()[:200], "properties": props})
else:
rows = [{"title": ln.lstrip("-* ").strip()[:200], "properties": {}} for ln in text.splitlines() if ln.strip()]
result.pages.append(make_collection("Readwise", schema, rows, source_path=filename))
result.stats["rows"] = len(rows)
return result.finalize()
@register_importer
class ShaarliImporter(_BookmarkBase):
source_id = "shaarli"
label = "Shaarli"
description = "Export Shaarli (JSON) → collection de signets."
extensions = (".json",)
order = 48
keywords = ("shaarli",)
def _records(self, data: bytes) -> list[dict] | None:
try:
obj = json.loads(decode_text(data))
except Exception: # noqa: BLE001
return None
if isinstance(obj, dict) and isinstance(obj.get("links"), list):
obj = obj["links"]
if isinstance(obj, list) and obj and all(isinstance(x, dict) and x.get("url") for x in obj):
return obj
return None
def detect(self, filename: str, data: bytes) -> bool:
if not filename.lower().endswith(".json"):
return False
return self._records(data) is not None
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
records = self._records(data) or []
rows = [
_row(r.get("title", ""), r.get("url", ""), r.get("description", ""),
r.get("tags", ""), _iso(r.get("created", "")))
for r in records
]
result.pages.append(make_collection("Shaarli", _SCHEMA, rows, source_path=filename))
result.stats["rows"] = len(rows)
return result.finalize()
@register_importer
class BookmarksImporter(_BookmarkBase):
source_id = "bookmarks"
label = "Signets HTML (navigateur)"
description = "Fichier de signets Netscape HTML (Chrome/Firefox/Edge…)."
extensions = (".html", ".htm")
order = 49
keywords = ()
def detect(self, filename: str, data: bytes) -> bool:
if not filename.lower().endswith((".html", ".htm")):
return False
text = decode_text(data)[:4000].lower()
return "netscape-bookmark-file" in text or "<dt><a href" in text
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
rows = parse_netscape(decode_text(data))
result.pages.append(make_collection("Bookmarks", _SCHEMA, rows, source_path=filename))
result.stats["rows"] = len(rows)
return result.finalize()
+138
View File
@@ -0,0 +1,138 @@
"""FlowDeck — iCalendar (.ics) importer (v5.6.0, Phase 4).
Parses VEVENT blocks (RFC 5545, best-effort) into a FlowDeck calendar
collection (start/end, all-day, location, description).
"""
from __future__ import annotations
import re
from typing import Any
from app.services.importers.base import (
Importer,
ImportResult,
decode_text,
make_collection,
register_importer,
)
_SCHEMA = [
{"name": "Title", "type": "title"},
{"name": "Start", "type": "date"},
{"name": "End", "type": "date"},
{"name": "All day", "type": "checkbox"},
{"name": "Location", "type": "text"},
{"name": "Description", "type": "text"},
{"name": "Calendar", "type": "text"},
]
_UNESCAPE = [("\\n", "\n"), ("\\N", "\n"), ("\\,", ","), ("\\;", ";"), ("\\\\", "\\")]
def _unfold(text: str) -> list[str]:
lines: list[str] = []
for raw in text.replace("\r\n", "\n").replace("\r", "\n").split("\n"):
if raw[:1] in (" ", "\t") and lines:
lines[-1] += raw[1:]
else:
lines.append(raw)
return lines
def _unescape(value: str) -> str:
for src, dst in _UNESCAPE:
value = value.replace(src, dst)
return value.strip()
def _parse_prop(line: str) -> tuple[str, dict[str, str], str]:
if ":" not in line:
return "", {}, ""
head, _, value = line.partition(":")
parts = head.split(";")
name = parts[0].upper()
params: dict[str, str] = {}
for p in parts[1:]:
if "=" in p:
k, _, v = p.partition("=")
params[k.upper()] = v
return name, params, value
def _iso_datetime(value: str, params: dict[str, str]) -> tuple[str, bool]:
"""Return (iso, is_all_day)."""
v = value.strip()
if params.get("VALUE") == "DATE" or re.fullmatch(r"\d{8}", v):
m = re.fullmatch(r"(\d{4})(\d{2})(\d{2})", v)
return (f"{m.group(1)}-{m.group(2)}-{m.group(3)}", True) if m else ("", True)
m = re.fullmatch(r"(\d{4})(\d{2})(\d{2})T(\d{2})(\d{2})(\d{2})(Z?)", v)
if not m:
return v, False
date = f"{m.group(1)}-{m.group(2)}-{m.group(3)}T{m.group(4)}:{m.group(5)}:{m.group(6)}"
return (date + "+00:00" if m.group(7) else date), False
@register_importer
class IcsImporter(Importer):
source_id = "ics"
label = "Calendrier (.ics)"
description = "Export iCalendar (Google/Outlook/Apple) → collection d'événements."
extensions = (".ics", ".ical")
order = 33
def detect(self, filename: str, data: bytes) -> bool:
if filename.lower().endswith((".ics", ".ical")):
return True
return "BEGIN:VCALENDAR" in decode_text(data)[:2000]
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
lines = _unfold(decode_text(data))
calendar = ""
rows: list[dict] = []
event: dict[str, Any] | None = None
for line in lines:
upper = line.strip().upper()
if upper == "BEGIN:VEVENT":
event = {}
continue
if upper == "END:VEVENT":
if event is not None:
rows.append(_event_row(event, calendar))
event = None
continue
name, params, value = _parse_prop(line.strip())
if name == "X-WR-CALNAME" and not event:
calendar = _unescape(value)
if event is None:
continue
if name == "SUMMARY":
event["title"] = _unescape(value)
elif name == "DTSTART":
event["start"], event["all_day"] = _iso_datetime(value, params)
elif name == "DTEND":
event["end"], _ = _iso_datetime(value, params)
elif name == "LOCATION":
event["location"] = _unescape(value)
elif name == "DESCRIPTION":
event["description"] = _unescape(value)
elif name == "UID":
event["uid"] = value
result.pages.append(make_collection("Calendar", _SCHEMA, rows, source_path=filename))
result.stats["rows"] = len(rows)
return result.finalize()
def _event_row(event: dict[str, Any], calendar: str) -> dict:
props: dict[str, Any] = {}
if event.get("start"):
props["Start"] = event["start"]
if event.get("end"):
props["End"] = event["end"]
props["All day"] = bool(event.get("all_day"))
if event.get("location"):
props["Location"] = event["location"]
if event.get("description"):
props["Description"] = event["description"]
if calendar:
props["Calendar"] = calendar
return {"title": (event.get("title") or "Event").strip()[:200], "properties": props}
+122
View File
@@ -0,0 +1,122 @@
"""FlowDeck — Word (.docx) importer (v5.6.0, Phase 3).
Converts a Word document (including Google Docs Takeout ``.docx`` exports) into
a FlowDeck page: headings, lists, tables and inline images.
"""
from __future__ import annotations
import io
import re
from pathlib import Path
from app.services.importers.base import (
ImportAttachment,
Importer,
ImportResult,
register_importer,
)
_HEADING_STYLES = {
"title": 1, "heading 1": 1, "heading 2": 2, "heading 3": 3,
"heading 4": 4, "heading 5": 4, "heading 6": 4,
}
_MIME = {
".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg",
".gif": "image/gif", ".webp": "image/webp", ".bmp": "image/bmp",
".emf": "image/emf", ".wmf": "image/wmf", ".tiff": "image/tiff",
}
def _escape_cell(text: str) -> str:
return text.strip().replace("|", "\\|").replace("\n", " ")
def _table_markdown(table) -> str:
rows: list[list[str]] = []
for row in table.rows:
rows.append([_escape_cell(cell.text) for cell in row.cells])
if not rows:
return ""
width = max(len(r) for r in rows)
rows = [r + [""] * (width - len(r)) for r in rows]
header = "| " + " | ".join(rows[0]) + " |"
sep = "| " + " | ".join(["---"] * width) + " |"
body = "\n".join("| " + " | ".join(r) + " |" for r in rows[1:])
return "\n".join(x for x in (header, sep, body) if x)
@register_importer
class DocxImporter(Importer):
source_id = "docx"
label = "Word / Google Docs (.docx)"
description = "Document Word : titres, listes, tableaux et images."
extensions = (".docx", ".docm")
order = 36
def detect(self, filename: str, data: bytes) -> bool:
return filename.lower().endswith((".docx", ".docm"))
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
try:
from docx import Document
from docx.oxml.ns import qn
except ImportError:
result.warn("python-docx n'est pas installé : import Word indisponible")
return result.finalize()
try:
doc = Document(io.BytesIO(data))
except Exception as exc: # noqa: BLE001
result.warn(f"Document illisible : {exc}")
return result.finalize()
lines: list[str] = []
for para in doc.paragraphs:
text = para.text.strip()
style = (para.style.name or "").lower() if para.style else ""
images = self._paragraph_images(doc, para, qn, result)
if text:
level = _HEADING_STYLES.get(style)
if level:
lines.append("#" * level + " " + text)
elif "list bullet" in style or "list paragraph" in style:
lines.append("- " + text)
elif "list number" in style:
lines.append("1. " + text)
elif style == "quote":
lines.append("> " + text)
else:
lines.append(text)
lines.extend(images)
for table in doc.tables:
md = _table_markdown(table)
if md:
lines.append(md)
markdown = re.sub(r"\n{3,}", "\n\n", "\n\n".join(lines)).strip()
title = Path(filename).stem or "Document"
result.pages.append(_page(title, markdown, filename))
return result.finalize()
def _paragraph_images(self, doc, para, qn, result: ImportResult) -> list[str]:
images: list[str] = []
for blip in para._p.iter(qn("a:blip")):
rid = blip.get(qn("r:embed")) or blip.get(qn("r:link"))
if not rid:
continue
part = doc.part.related_parts.get(rid)
if part is None or not hasattr(part, "blob"):
continue
name = Path(str(part.partname)).name or f"image_{len(result.attachments)}.png"
result.attachments.append(ImportAttachment(
source_path=name, filename=name, data=part.blob,
mime=_MIME.get(Path(name).suffix.lower(), "application/octet-stream"),
))
images.append(f"![{name}]({name})")
return images
def _page(title: str, markdown: str, filename: str):
from app.services.importers.base import ImportPage
return ImportPage(title=title, markdown=markdown, source_path=filename, external_id=filename)
+232
View File
@@ -0,0 +1,232 @@
"""FlowDeck — forge (Gitea/GitHub) issues importer (v5.6.0, Phase 4).
Pulls a repository's issues, labels and milestones through a forge adapter and
normalizes them into FlowDeck collections.
"""
from __future__ import annotations
from typing import Any
from app.services.importers.base import ImportResult, make_collection
def _label_names(issue: dict) -> list[str]:
labels = issue.get("labels") or []
names: list[str] = []
for label in labels:
if isinstance(label, dict):
name = label.get("name") or label.get("title")
else:
name = str(label)
if name and name not in names:
names.append(name)
return names
def _milestone_name(issue: dict) -> str:
milestone = issue.get("milestone")
if isinstance(milestone, dict):
return str(milestone.get("title") or milestone.get("name") or "")
return str(milestone or "")
def _assignees(issue: dict) -> list[str]:
out: list[str] = []
for key in ("assignees", "assignee"):
value = issue.get(key)
if isinstance(value, list):
for a in value:
login = a.get("login") if isinstance(a, dict) else str(a)
if login and login not in out:
out.append(login)
elif isinstance(value, dict):
login = value.get("login")
if login and login not in out:
out.append(login)
elif isinstance(value, str) and value and value not in out:
out.append(value)
return out
_ISSUE_SCHEMA = [
{"name": "Title", "type": "title"},
{"name": "Number", "type": "number"},
{"name": "State", "type": "select", "options": [
{"name": "open", "color": "green"}, {"name": "closed", "color": "red"},
]},
{"name": "Labels", "type": "multi_select"},
{"name": "Milestone", "type": "text"},
{"name": "Assignee", "type": "text"},
{"name": "Created", "type": "date"},
{"name": "Updated", "type": "date"},
{"name": "URL", "type": "url"},
{"name": "Body", "type": "text"},
]
def build_issues_result(
issues: list[dict],
*,
labels: list[dict] | None = None,
milestones: list[dict] | None = None,
owner: str = "",
repo: str = "",
provider: str = "",
) -> ImportResult:
"""Normalize forge issues/labels/milestones into an ImportResult."""
result = ImportResult(source=f"forge:{provider}" if provider else "forge")
name = f"{owner}/{repo} issues".strip("/ ") or "Issues"
rows: list[dict] = []
for issue in issues:
if issue.get("pull_request"):
continue
props: dict[str, Any] = {}
if issue.get("number") is not None:
props["Number"] = issue["number"]
if issue.get("state"):
props["State"] = issue["state"]
label_names = _label_names(issue)
if label_names:
props["Labels"] = label_names
milestone = _milestone_name(issue)
if milestone:
props["Milestone"] = milestone
assignees = _assignees(issue)
if assignees:
props["Assignee"] = ", ".join(assignees)
if issue.get("created_at"):
props["Created"] = issue["created_at"]
if issue.get("updated_at"):
props["Updated"] = issue["updated_at"]
if issue.get("html_url"):
props["URL"] = issue["html_url"]
if issue.get("body"):
props["Body"] = issue["body"]
title = issue.get("title") or f"#{issue.get('number', '')}".strip()
rows.append({"title": title[:200], "properties": props})
result.pages.append(make_collection(
name, _ISSUE_SCHEMA, rows,
source_path=f"{provider}:{owner}/{repo}:issues",
external_id=f"{provider}:{owner}/{repo}:issues",
))
result.stats["rows"] = len(rows)
if labels:
label_schema = [
{"name": "Name", "type": "title"},
{"name": "Color", "type": "text"},
{"name": "Description", "type": "text"},
]
label_rows = [{
"title": (lbl.get("name") or lbl.get("title") or "Label")[:200],
"properties": {
k: v for k, v in (
("Color", lbl.get("color")),
("Description", lbl.get("description")),
) if v
},
} for lbl in labels]
result.pages.append(make_collection(
f"{owner}/{repo} labels".strip("/ "), label_schema, label_rows,
source_path=f"{provider}:{owner}/{repo}:labels",
external_id=f"{provider}:{owner}/{repo}:labels",
))
if milestones:
ms_schema = [
{"name": "Title", "type": "title"},
{"name": "State", "type": "select", "options": [
{"name": "open", "color": "green"}, {"name": "closed", "color": "red"},
]},
{"name": "Due date", "type": "date"},
{"name": "Description", "type": "text"},
]
ms_rows = []
for ms in milestones:
props: dict[str, Any] = {}
if ms.get("state"):
props["State"] = ms["state"]
if ms.get("due_on"):
props["Due date"] = ms["due_on"]
if ms.get("description"):
props["Description"] = ms["description"]
ms_rows.append({"title": (ms.get("title") or "Milestone")[:200], "properties": props})
result.pages.append(make_collection(
f"{owner}/{repo} milestones".strip("/ "), ms_schema, ms_rows,
source_path=f"{provider}:{owner}/{repo}:milestones",
external_id=f"{provider}:{owner}/{repo}:milestones",
))
return result.finalize()
class GiteaForgeAdapter:
"""Adapts a :class:`GiteaClient` to the ``list_*`` interface used here."""
def __init__(self, client) -> None:
self._client = client
async def list_issues(self, owner: str, repo: str, state: str = "all") -> list[dict]:
issues: list[dict] = []
for page in range(1, 6):
batch = await self._client.get_issues(owner, repo, state=state, page=page, limit=50)
if not batch:
break
issues.extend(batch)
if len(batch) < 50:
break
return issues
async def list_labels(self, owner: str, repo: str) -> list[dict]:
return await self._client.get_labels(owner, repo)
async def list_milestones(self, owner: str, repo: str, state: str = "all") -> list[dict]:
return await self._client.get_milestones(owner, repo, state=state)
async def list_repo_files(self, owner: str, repo: str, path: str = "") -> list[dict]:
"""Recursively flatten Gitea repo contents into file entries."""
files: list[dict] = []
pending = [path.strip("/")]
while pending and len(files) < 5000:
current = pending.pop()
items = await self._client.get_repo_contents(owner, repo, current)
for item in items:
if item.get("type") == "dir":
pending.append(item.get("path") or item.get("name"))
elif item.get("type") == "file":
files.append({"path": item.get("path") or item.get("name"), "size": item.get("size", 0)})
return files
async def get_file_content(self, owner: str, repo: str, path: str) -> str:
return await self._client.get_file_content(owner, repo, path)
async def fetch_forge_issues(
adapter,
owner: str,
repo: str,
*,
provider: str = "",
state: str = "all",
include_labels: bool = True,
include_milestones: bool = True,
) -> ImportResult:
"""Fetch issues (and optionally labels/milestones) then normalize them."""
issues = await adapter.list_issues(owner, repo, state)
labels = None
milestones = None
if include_labels:
try:
labels = await adapter.list_labels(owner, repo)
except Exception: # noqa: BLE001 - labels are optional
labels = None
if include_milestones:
try:
milestones = await adapter.list_milestones(owner, repo, state)
except Exception: # noqa: BLE001
milestones = None
return build_issues_result(
issues, labels=labels, milestones=milestones,
owner=owner, repo=repo, provider=provider,
)
+85
View File
@@ -0,0 +1,85 @@
"""FlowDeck — forge repository file importer (v5.6.0, Phase 5).
Imports a Gitea/GitHub repository's text files as pages, preserving the folder
hierarchy. Markdown files become pages; other text files become code blocks.
"""
from __future__ import annotations
from pathlib import Path
from app.services.export import _CODE_LANG, _TEXTUAL_EXTS
from app.services.importers.base import ImportPage, ImportResult
_MD_EXTS = {"md", "markdown"}
def _ext(path: str) -> str:
return Path(path).suffix.lower().lstrip(".")
def build_repo_result(
files: list[tuple[str, str]],
*,
owner: str,
repo: str,
provider: str = "",
) -> ImportResult:
"""Turn ``[(path, content)]`` into pages with folder hierarchy."""
result = ImportResult(source=f"forge-repo:{provider}" if provider else "forge-repo")
for path, content in files:
clean = path.replace("\\", "/").strip("/")
if not clean:
continue
ext = _ext(clean)
if ext in _MD_EXTS:
markdown = content
else:
lang = _CODE_LANG.get(ext, "")
markdown = f"```{lang}\n{content.rstrip()}\n```"
parts = clean.split("/")
result.pages.append(ImportPage(
title=parts[-1] or clean,
markdown=markdown,
source_path=clean,
parent_path="/".join(parts[:-1]),
external_id=f"{provider}:{owner}/{repo}:{clean}",
))
result.stats["rows"] = len(result.pages)
return result.finalize()
async def fetch_forge_repo(
adapter,
owner: str,
repo: str,
*,
provider: str = "",
path: str = "",
max_files: int = 200,
max_file_bytes: int = 512_000,
) -> ImportResult:
"""List a repo's files and fetch the textual ones."""
try:
metas = await adapter.list_repo_files(owner, repo, path)
except Exception as exc: # noqa: BLE001
result = ImportResult(source=f"forge-repo:{provider}" if provider else "forge-repo")
result.warn(f"Arborescence illisible : {exc}")
return result.finalize()
files: list[tuple[str, str]] = []
for meta in metas:
file_path = meta.get("path") or ""
if _ext(file_path) not in _TEXTUAL_EXTS:
continue
if int(meta.get("size") or 0) > max_file_bytes:
continue
if len(files) >= max_files:
break
try:
content = await adapter.get_file_content(owner, repo, file_path)
except Exception: # noqa: BLE001 - skip unreadable files
continue
if not content or content == "[binary file]":
continue
files.append((file_path, content))
return build_repo_result(files, owner=owner, repo=repo, provider=provider)
+300
View File
@@ -0,0 +1,300 @@
"""FlowDeck — HTML notes & Google Keep importer (v5.6.0, Phase 1).
Covers HTML exports from Apple Notes, Bear, Ulysses and OneNote, plus the
Google Takeout ``Keep`` JSON/HTML format. HTML is converted to Markdown and then
to FlowDeck blocks by the pipeline.
"""
from __future__ import annotations
import io
import json
import re
import zipfile
from bs4 import BeautifulSoup, NavigableString, Tag
from app.services.importers._common import coerce_tags, normalize_title
from app.services.importers.base import (
ImportAttachment,
Importer,
ImportPage,
ImportResult,
decode_text,
register_importer,
)
_HTML_EXTS = (".html", ".htm")
def _inline(node: Tag) -> str:
out: list[str] = []
for child in node.children:
if isinstance(child, NavigableString):
out.append(str(child))
elif isinstance(child, Tag):
name = child.name.lower()
if name in ("strong", "b"):
out.append(f"**{_inline(child).strip()}**")
elif name in ("em", "i"):
out.append(f"*{_inline(child).strip()}*")
elif name == "code":
out.append(f"`{child.get_text()}`")
elif name == "br":
out.append("\n")
elif name == "a":
href = child.get("href", "")
label = _inline(child).strip() or href
out.append(f"[{label}]({href})" if href else label)
elif name == "img":
src = child.get("src", "")
alt = child.get("alt", "")
out.append(f"![{alt}]({src})" if src else "")
elif name in ("del", "s", "strike"):
out.append(f"~~{_inline(child).strip()}~~")
else:
out.append(_inline(child))
return re.sub(r"[ \t]+", " ", "".join(out))
def _table(node: Tag) -> str:
rows: list[list[str]] = []
for tr in node.find_all("tr"):
cells = tr.find_all(["th", "td"])
rows.append([_inline(c).strip().replace("|", "\\|") for c in cells])
if not rows:
return ""
width = max(len(r) for r in rows)
rows = [r + [""] * (width - len(r)) for r in rows]
header = "| " + " | ".join(rows[0]) + " |"
sep = "| " + " | ".join(["---"] * width) + " |"
body = "\n".join("| " + " | ".join(r) + " |" for r in rows[1:])
return "\n".join(x for x in (header, sep, body) if x)
def _block(node: Tag, depth: int = 0) -> str:
name = node.name.lower()
if name in ("h1", "h2", "h3", "h4", "h5", "h6"):
return "#" * int(name[1]) + " " + _inline(node).strip()
if name == "p":
return _inline(node).strip()
if name in ("ul", "ol"):
lines = []
for i, li in enumerate(node.find_all("li", recursive=False)):
marker = f"{i + 1}." if name == "ol" else "-"
text = _inline(li).strip()
lines.append(f"{' ' * depth}{marker} {text}")
return "\n".join(lines)
if name == "blockquote":
return "\n".join(f"> {ln}" for ln in _inline(node).strip().splitlines())
if name == "pre":
code = node.get_text()
lang = ""
cls = " ".join(node.get("class", [])) if node.get("class") else ""
m = re.search(r"(?:language|lang)-([\w+-]+)", cls)
if m:
lang = m.group(1)
return f"```{lang}\n{code.rstrip()}\n```"
if name == "hr":
return "---"
if name == "table":
return _table(node)
if name == "img":
src = node.get("src", "")
return f"![{node.get('alt', '')}]({src})" if src else ""
if name in ("div", "section", "article", "body", "main", "html", "span", "font", "center"):
inner = "\n\n".join(
_block(c, depth) for c in node.children if isinstance(c, Tag)
).strip()
if inner:
return inner
text = _inline(node).strip()
return text
return _inline(node).strip()
def _html_to_markdown(html: str) -> str:
soup = BeautifulSoup(html, "html.parser")
for tag in soup(["script", "style", "head", "nav", "footer"]):
tag.decompose()
root = soup.body or soup
blocks = [_block(c) for c in root.children if isinstance(c, Tag)]
md = "\n\n".join(b for b in blocks if b and b.strip())
return re.sub(r"\n{3,}", "\n\n", md).strip()
def _title_from_html(html: str, fallback: str) -> str:
soup = BeautifulSoup(html, "html.parser")
if soup.title and soup.title.string:
return soup.title.string.strip()
h1 = soup.find(["h1", "h2"])
if h1:
return h1.get_text().strip()
return fallback
@register_importer
class HtmlNotesImporter(Importer):
source_id = "html_notes"
label = "HTML (Apple Notes, Bear, Ulysses, OneNote)"
description = "Fichiers HTML ou archive .zip (notes exportées en HTML)."
extensions = (".html", ".htm", ".zip")
order = 50
def detect(self, filename: str, data: bytes) -> bool:
low = filename.lower()
if low.endswith(_HTML_EXTS):
return True
if low.endswith(".zip"):
try:
zf = zipfile.ZipFile(io.BytesIO(data))
except (zipfile.BadZipFile, OSError):
return False
names = [n for n in zf.namelist() if not n.endswith("/")]
return any(n.lower().endswith(_HTML_EXTS) for n in names)
return False
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
entries: list[tuple[str, bytes]] = []
if filename.lower().endswith(".zip"):
try:
zf = zipfile.ZipFile(io.BytesIO(data))
except (zipfile.BadZipFile, OSError) as exc:
result.warn(f"Archive invalide : {exc}")
return result.finalize()
for name in zf.namelist():
if name.endswith("/"):
continue
clean = name.replace("\\", "/")
if clean.lower().endswith(_HTML_EXTS):
entries.append((clean, zf.read(name)))
else:
result.attachments.append(ImportAttachment(
source_path=clean,
filename=clean.rsplit("/", 1)[-1],
data=zf.read(name),
))
else:
entries.append((filename, data))
for name, payload in entries:
html = decode_text(payload)
fallback = name.replace("\\", "/").rsplit("/", 1)[-1].rsplit(".", 1)[0]
parts = name.replace("\\", "/").split("/")
result.pages.append(ImportPage(
title=_title_from_html(html, fallback) or "Untitled",
markdown=_html_to_markdown(html),
source_path=name,
parent_path="/".join(parts[:-1]),
external_id=name,
))
return result.finalize()
@register_importer
class GoogleKeepImporter(Importer):
source_id = "google_keep"
label = "Google Keep (Takeout)"
description = "Export Google Takeout : Keep/*.json (notes, listes, labels, pièces jointes)."
extensions = (".json", ".zip")
order = 40
def _is_keep_json(self, data: bytes) -> bool:
try:
obj = json.loads(decode_text(data))
except Exception: # noqa: BLE001
return False
return isinstance(obj, dict) and any(
k in obj for k in ("textContent", "listContent", "isTrashed", "color")
)
def detect(self, filename: str, data: bytes) -> bool:
low = filename.lower()
if low.endswith(".json"):
return self._is_keep_json(data)
if low.endswith(".zip"):
try:
zf = zipfile.ZipFile(io.BytesIO(data))
except (zipfile.BadZipFile, OSError):
return False
for n in zf.namelist():
if n.lower().endswith(".json") and "keep" in n.lower():
try:
if self._is_keep_json(zf.read(n)):
return True
except Exception: # noqa: BLE001
continue
return False
def _page_from_keep(self, obj: dict, name: str) -> ImportPage | None:
if obj.get("isTrashed"):
return None
title = normalize_title(obj.get("title"))
lines: list[str] = []
for item in obj.get("listContent") or []:
mark = "x" if item.get("isChecked") else " "
lines.append(f"- [{mark}] {item.get('text', '')}")
if obj.get("textContent"):
lines.insert(0, obj["textContent"])
body = "\n\n".join(lines)
if not title:
first = next((ln for ln in body.splitlines() if ln.strip()), "")
first = re.sub(r"^[-*+]\s*(\[[ xX]\]\s*)?", "", first).strip()
title = first[:60] or "Note"
labels = coerce_tags(obj.get("labels"))
props = {"tags": labels} if labels else {}
return ImportPage(
title=title,
markdown=body,
source_path=name,
parent_path="",
properties=props,
external_id=name,
)
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
if filename.lower().endswith(".zip"):
try:
zf = zipfile.ZipFile(io.BytesIO(data))
except (zipfile.BadZipFile, OSError) as exc:
result.warn(f"Archive invalide : {exc}")
return result.finalize()
for name in zf.namelist():
if name.endswith("/"):
continue
clean = name.replace("\\", "/")
if clean.lower().endswith(".json") and "keep" in clean.lower():
try:
obj = json.loads(decode_text(zf.read(name)))
except Exception: # noqa: BLE001
continue
if not isinstance(obj, dict):
continue
page = self._page_from_keep(obj, clean)
if page:
result.pages.append(page)
elif "/keep/" in clean.lower() and not clean.lower().endswith(".json"):
result.attachments.append(ImportAttachment(
source_path=clean,
filename=clean.rsplit("/", 1)[-1],
data=zf.read(name),
))
return result.finalize()
try:
obj = json.loads(decode_text(data))
except Exception as exc: # noqa: BLE001
result.warn(f"JSON invalide : {exc}")
return result.finalize()
if isinstance(obj, list):
for i, item in enumerate(obj):
if isinstance(item, dict):
page = self._page_from_keep(item, f"{filename}#{i}")
if page:
result.pages.append(page)
else:
page = self._page_from_keep(obj, filename)
if page:
result.pages.append(page)
return result.finalize()
+150
View File
@@ -0,0 +1,150 @@
"""FlowDeck — background import jobs (v5.6.0, Phase 0).
Small in-process job manager used for large uploads (vaults, zips): the upload
is parsed and persisted in a worker thread while the UI polls job status.
"""
from __future__ import annotations
import threading
import time
import traceback
import uuid
from typing import Any
from app.db import get_conn
from app.services.importers.base import (
Importer,
ImportResult,
detect_importer,
get_importer,
)
from app.services.importers.pipeline import run_import
_JOBS: dict[str, dict[str, Any]] = {}
_LOCK = threading.Lock()
def parse_upload(filename: str, data: bytes, source_id: str | None = None) -> tuple[Importer | None, ImportResult]:
"""Detect (or use) an importer and parse the upload synchronously."""
imp = get_importer(source_id) if source_id else None
if imp is None:
imp = detect_importer(filename, data)
if imp is None:
return None, ImportResult(source=source_id or "unknown", warnings=["Format non reconnu"])
return imp, imp.parse(filename, data)
def _record(job: dict, *, status: str | None = None, error: str = "",
report: dict | None = None, progress: int | None = None) -> None:
with _LOCK:
if status:
job["status"] = status
if error:
job["error"] = error
if report is not None:
job["report"] = report
if progress is not None:
job["progress"] = progress
job["updated_at"] = time.time()
_persist(job)
def _persist(job: dict) -> None:
try:
with get_conn() as conn:
conn.execute(
"INSERT INTO import_jobs (id, source, filename, status, error, report_json, created_at, updated_at) "
"VALUES (?,?,?,?,?,?,?,?) "
"ON CONFLICT(id) DO UPDATE SET status=excluded.status, error=excluded.error, "
"report_json=excluded.report_json, updated_at=excluded.updated_at",
(job["id"], job["source"], job["filename"], job["status"], job.get("error", ""),
_json(job.get("report")), job["created_at"], job["updated_at"]),
)
conn.commit()
except Exception: # noqa: BLE001 - persistence is best-effort
pass
def _json(value: Any) -> str:
import json
try:
return json.dumps(value, ensure_ascii=False)
except (TypeError, ValueError):
return "{}"
def create_job(source: str, filename: str) -> dict:
job = {
"id": uuid.uuid4().hex[:16],
"source": source,
"filename": filename,
"status": "queued",
"progress": 0,
"error": "",
"report": None,
"created_at": time.time(),
"updated_at": time.time(),
}
with _LOCK:
_JOBS[job["id"]] = job
_persist(job)
return job
def get_job(job_id: str) -> dict | None:
with _LOCK:
job = _JOBS.get(job_id)
return dict(job) if job else None
def list_jobs(limit: int = 50) -> list[dict]:
with _LOCK:
jobs = sorted(_JOBS.values(), key=lambda j: j["created_at"], reverse=True)
return [dict(j) for j in jobs[:limit]]
def start_import_job(
*,
filename: str,
data: bytes,
source_id: str | None,
workspace_id: int | None,
workspace_name: str | None,
user_login: str,
parent_page_id: int | None,
target_collection_id: int | None,
dedup: bool = True,
mapping: dict[str, str] | None = None,
mode: str | None = None,
) -> dict:
"""Create a job and run parse + persist in a background thread."""
job = create_job(source_id or "auto", filename)
_record(job, status="running", progress=5)
def worker() -> None:
try:
imp, result = parse_upload(filename, data, source_id)
if imp is None:
_record(job, status="error", error="Format non reconnu")
return
_record(job, progress=40)
report = run_import(
result,
workspace_id=workspace_id,
workspace_name=workspace_name,
user_login=user_login,
parent_page_id=parent_page_id,
target_collection_id=target_collection_id,
dedup=dedup,
mapping=mapping,
mode=mode,
)
_record(job, status="done", progress=100, report=report)
except Exception as exc: # noqa: BLE001 - surface the error to the UI
_record(job, status="error", error=f"{exc}", report={
"traceback": traceback.format_exc()[-2000:],
})
threading.Thread(target=worker, name=f"import-{job['id']}", daemon=True).start()
return job
+87
View File
@@ -0,0 +1,87 @@
"""FlowDeck — generic Markdown / text importer (v5.6.0, Phase 1)."""
from __future__ import annotations
import io
import zipfile
from app.services.importers.base import (
Importer,
ImportPage,
ImportResult,
decode_text,
register_importer,
)
_MD_EXTS = (".md", ".markdown", ".txt", ".mdx")
def _title_from_name(name: str) -> str:
base = name.replace("\\", "/").rsplit("/", 1)[-1]
for ext in (".markdown", ".markdown", ".mdx", ".md", ".txt"):
if base.lower().endswith(ext):
base = base[: -len(ext)]
break
return base.strip() or "Untitled"
@register_importer
class MarkdownImporter(Importer):
source_id = "markdown"
label = "Markdown / texte"
description = "Fichiers .md/.markdown/.txt ou archive .zip de fichiers Markdown."
extensions = (".md", ".markdown", ".txt", ".zip")
order = 90
def detect(self, filename: str, data: bytes) -> bool:
low = filename.lower()
if low.endswith(_MD_EXTS):
return True
if low.endswith(".zip"):
try:
zf = zipfile.ZipFile(io.BytesIO(data))
except (zipfile.BadZipFile, OSError):
return False
names = [n for n in zf.namelist() if not n.endswith("/")]
return bool(names) and all(n.lower().endswith(_MD_EXTS) for n in names)
return False
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
if filename.lower().endswith(".zip"):
try:
zf = zipfile.ZipFile(io.BytesIO(data))
except (zipfile.BadZipFile, OSError) as exc:
result.warn(f"Archive invalide : {exc}")
return result.finalize()
entries = sorted(
(n for n in zf.namelist()
if not n.endswith("/") and n.lower().endswith(_MD_EXTS)),
key=lambda n: (n.count("/"), n.lower()),
)
if not entries:
result.warn("Aucun fichier Markdown trouvé dans l'archive")
return result.finalize()
for name in entries:
try:
text = decode_text(zf.read(name))
except Exception as exc: # noqa: BLE001
result.warn(f"Lecture impossible : {name} ({exc})")
continue
parts = name.replace("\\", "/").split("/")
result.pages.append(ImportPage(
title=_title_from_name(name),
markdown=text,
source_path=name,
parent_path="/".join(parts[:-1]),
external_id=name,
))
return result.finalize()
text = decode_text(data)
result.pages.append(ImportPage(
title=_title_from_name(filename),
markdown=text,
source_path=filename,
external_id=filename,
))
return result.finalize()
+135
View File
@@ -0,0 +1,135 @@
"""FlowDeck — Notion export importer (v5.6.0, Phase 1, amélioration v5.4.0).
Imports a Notion "Export as Markdown & CSV" ``.zip``: complete page hierarchy,
databases (``.csv``) turned into FlowDeck collections, and image attachments.
"""
from __future__ import annotations
import csv
import io
import re
import zipfile
from urllib.parse import unquote
from app.services.importers._common import split_frontmatter
from app.services.importers.base import (
ImportAttachment,
Importer,
ImportPage,
ImportResult,
decode_text,
register_importer,
)
from app.services.importers.tabular import rows_to_collection
_HASH_RE = re.compile(r"\s+[0-9a-f]{32}$")
_MD_LINK_RE = re.compile(r"\]\(([^)]+)\.md\)")
def _clean_name(name: str) -> str:
base = unquote(name.replace("\\", "/").rsplit("/", 1)[-1])
base = re.sub(r"\.(md|csv|markdown)$", "", base, flags=re.IGNORECASE)
return _HASH_RE.sub("", base).strip() or "Untitled"
def _strip_hash_link(match: re.Match) -> str:
target = unquote(match.group(1)).strip()
return f"]({_HASH_RE.sub('', target).strip() or target})"
@register_importer
class NotionImporter(Importer):
source_id = "notion"
label = "Notion (export .zip)"
description = "Export Notion Markdown & CSV : hiérarchie, databases → collections, images."
extensions = (".zip",)
order = 20
def detect(self, filename: str, data: bytes) -> bool:
if not filename.lower().endswith(".zip"):
return False
try:
zf = zipfile.ZipFile(io.BytesIO(data))
except (zipfile.BadZipFile, OSError):
return False
names = [n for n in zf.namelist() if not n.endswith("/")]
md = [n for n in names if n.lower().endswith(".md")]
csvs = [n for n in names if n.lower().endswith(".csv")]
if not md:
return False
if csvs:
return True
return any(_HASH_RE.search(unquote(n.rsplit("/", 1)[-1])) for n in md)
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
try:
zf = zipfile.ZipFile(io.BytesIO(data))
except (zipfile.BadZipFile, OSError) as exc:
result.warn(f"Archive invalide : {exc}")
return result.finalize()
names = [n for n in zf.namelist() if not n.endswith("/")]
md_names = [n for n in names if n.lower().endswith(".md")]
csv_names = [n for n in names if n.lower().endswith(".csv")]
pages_by_title: dict[str, ImportPage] = {}
for name in sorted(md_names, key=lambda n: (n.count("/"), n.lower())):
try:
text = decode_text(zf.read(name))
except Exception as exc: # noqa: BLE001
result.warn(f"Lecture impossible : {name} ({exc})")
continue
meta, body = split_frontmatter(text)
title = _clean_name(name)
body = _MD_LINK_RE.sub(_strip_hash_link, body)
first_line = body.lstrip().splitlines()[0] if body.strip() else ""
if first_line.strip().startswith("# ") and first_line.strip()[2:].strip() == title:
body = "\n".join(body.lstrip().splitlines()[1:]).lstrip("\n")
parts = unquote(name).replace("\\", "/").split("/")
page = ImportPage(
title=title,
markdown=body,
source_path=name,
parent_path="/".join(parts[:-1]),
properties={k: v for k, v in meta.items() if k != "title"},
external_id=name,
)
pages_by_title.setdefault(title, page)
result.pages.append(page)
for name in sorted(csv_names, key=lambda n: (n.count("/"), n.lower())):
try:
text = decode_text(zf.read(name))
except Exception as exc: # noqa: BLE001
result.warn(f"Lecture impossible : {name} ({exc})")
continue
reader = csv.DictReader(io.StringIO(text))
headers = [h for h in (reader.fieldnames or []) if h is not None]
rows = [dict(r) for r in reader]
title = _clean_name(name)
spec_page = rows_to_collection(title, headers, rows)
parts = unquote(name).replace("\\", "/").split("/")
existing = pages_by_title.get(title)
if existing is not None:
existing.collection = spec_page.collection
existing.source_path = existing.source_path or name
else:
spec_page.parent_path = "/".join(parts[:-1])
spec_page.source_path = name
spec_page.external_id = name
result.pages.append(spec_page)
for name in names:
low = name.lower()
if low.endswith((".md", ".csv")):
continue
try:
result.attachments.append(ImportAttachment(
source_path=name,
filename=unquote(name).replace("\\", "/").rsplit("/", 1)[-1],
data=zf.read(name),
))
except Exception: # noqa: BLE001
continue
return result.finalize()
+118
View File
@@ -0,0 +1,118 @@
"""FlowDeck — Obsidian vault importer (v5.6.0, Phase 1).
Imports a vault exported as a ``.zip``: Markdown notes (with YAML frontmatter),
the folder hierarchy, ``[[wikilinks]]``/``![[embeds]]`` and binary attachments.
"""
from __future__ import annotations
import io
import zipfile
from app.services.importers._common import (
coerce_tags,
convert_wikilinks,
normalize_title,
split_frontmatter,
)
from app.services.importers.base import (
ImportAttachment,
Importer,
ImportPage,
ImportResult,
decode_text,
register_importer,
)
_SKIP_DIRS = (".obsidian/", ".trash/", ".git/", ".DS_Store")
def _mime_for(name: str) -> str:
import mimetypes
return mimetypes.guess_type(name)[0] or "application/octet-stream"
@register_importer
class ObsidianImporter(Importer):
source_id = "obsidian"
label = "Obsidian (vault .zip)"
description = "Vault Obsidian : notes Markdown, frontmatter YAML, wikilinks, pièces jointes."
extensions = (".zip",)
order = 10
def detect(self, filename: str, data: bytes) -> bool:
if not filename.lower().endswith(".zip"):
return False
try:
zf = zipfile.ZipFile(io.BytesIO(data))
except (zipfile.BadZipFile, OSError):
return False
names = zf.namelist()
if any("/.obsidian/" in n or n.startswith(".obsidian/") for n in names):
return True
# Heuristic: mostly-markdown archive containing wikilinks.
md = [n for n in names if n.lower().endswith(".md")]
if not md:
return False
for n in md[:20]:
try:
if "[[" in decode_text(zf.read(n)):
return True
except Exception: # noqa: BLE001
continue
return False
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
try:
zf = zipfile.ZipFile(io.BytesIO(data))
except (zipfile.BadZipFile, OSError) as exc:
result.warn(f"Archive invalide : {exc}")
return result.finalize()
names = [n for n in zf.namelist() if not n.endswith("/")]
notes = [n for n in names if n.lower().endswith(".md")]
assets = [n for n in names if not n.lower().endswith(".md")]
for name in assets:
clean = name.replace("\\", "/")
if any(part in clean for part in _SKIP_DIRS) or clean.split("/")[-1].startswith("."):
continue
try:
payload = zf.read(name)
except Exception: # noqa: BLE001
continue
result.attachments.append(ImportAttachment(
source_path=name,
filename=clean.rsplit("/", 1)[-1],
data=payload,
mime=_mime_for(name),
))
for name in sorted(notes, key=lambda n: (n.count("/"), n.lower())):
clean = name.replace("\\", "/")
if any(part in clean for part in _SKIP_DIRS):
continue
try:
text = decode_text(zf.read(name))
except Exception as exc: # noqa: BLE001
result.warn(f"Lecture impossible : {name} ({exc})")
continue
meta, body = split_frontmatter(text)
body = convert_wikilinks(body)
parts = clean.split("/")
title = normalize_title(meta.get("title")) or parts[-1][:-3]
props = dict(meta)
props.pop("title", None)
tags = coerce_tags(meta.get("tags"))
if tags:
props["tags"] = tags
result.pages.append(ImportPage(
title=title or "Untitled",
markdown=body,
source_path=clean,
parent_path="/".join(parts[:-1]),
properties=props,
external_id=clean,
))
return result.finalize()
+86
View File
@@ -0,0 +1,86 @@
"""FlowDeck — OPML importer (v5.6.0, Phase 4).
Imports an OPML outline (RSS readers, feed lists) as a collection of feeds.
"""
from __future__ import annotations
import xml.etree.ElementTree as ET
from typing import Any
from app.services.importers.base import (
Importer,
ImportResult,
decode_text,
make_collection,
register_importer,
)
_SCHEMA = [
{"name": "Title", "type": "title"},
{"name": "Feed URL", "type": "url"},
{"name": "Site URL", "type": "url"},
{"name": "Type", "type": "select", "options": [
{"name": "rss", "color": "orange"},
{"name": "folder", "color": "gray"},
]},
{"name": "Folder", "type": "text"},
]
@register_importer
class OpmlImporter(Importer):
source_id = "opml"
label = "OPML (flux RSS)"
description = "Outline OPML → collection de flux (titre, URL, dossier)."
extensions = (".opml", ".xml")
order = 34
def detect(self, filename: str, data: bytes) -> bool:
if filename.lower().endswith(".opml"):
return True
head = decode_text(data)[:1000].lower()
return "<opml" in head and "<outline" in head
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
rows: list[dict] = []
try:
root = ET.fromstring(decode_text(data))
except ET.ParseError as exc:
result.warn(f"OPML invalide : {exc}")
return result.finalize()
for outline in root.iter("outline"):
attrs = {k.lower(): v for k, v in outline.attrib.items()}
feed = attrs.get("xmlurl")
title = attrs.get("title") or attrs.get("text") or feed or ""
if not feed and not title:
continue
props: dict[str, Any] = {}
if feed:
props["Feed URL"] = feed
props["Type"] = "rss"
else:
props["Type"] = "folder"
if attrs.get("htmlurl"):
props["Site URL"] = attrs["htmlurl"]
folder = _folder_of(outline, root)
if folder:
props["Folder"] = folder
rows.append({"title": title[:200] or "Feed", "properties": props})
result.pages.append(make_collection("OPML feeds", _SCHEMA, rows, source_path=filename))
result.stats["rows"] = len(rows)
return result.finalize()
def _folder_of(node: ET.Element, root: ET.Element) -> str:
parents = {child: parent for parent in root.iter() for child in parent}
parts: list[str] = []
current = parents.get(node)
while current is not None:
attrs = {k.lower(): v for k, v in current.attrib.items()}
if not attrs.get("xmlurl"):
label = attrs.get("title") or attrs.get("text")
if label:
parts.append(label)
current = parents.get(current)
return " / ".join(reversed(parts))
+164
View File
@@ -0,0 +1,164 @@
"""FlowDeck — Logseq / Roam Research outliner importer (v5.6.0, Phase 1)."""
from __future__ import annotations
import io
import re
import zipfile
from app.services.importers._common import (
coerce_tags,
convert_wikilinks,
normalize_title,
split_frontmatter,
)
from app.services.importers.base import (
ImportAttachment,
Importer,
ImportPage,
ImportResult,
decode_text,
register_importer,
)
_LOGSEQ_MARKERS = ("property::", "logseq/", "journals/")
_ROAM_MARKERS = ("{{[[TODO]]}}", "{{[[DONE]]}}", "{{[[query]]}}")
_PROP_RE = re.compile(r"^\s*([a-zA-Z][\w-]*)::\s*(.*)$")
def _journal_title(name: str) -> str:
m = re.match(r"^(\d{4})[_-](\d{2})[_-](\d{2})", name)
if m:
return f"{m.group(1)}-{m.group(2)}-{m.group(3)}"
return name
def _clean_outline(text: str) -> tuple[dict, str]:
meta, body = split_frontmatter(text)
lines_out: list[str] = []
for line in body.splitlines():
m = _PROP_RE.match(line)
if m and line.lstrip().startswith("-"):
continue
# Logseq properties appear as bare ``key:: value`` lines too.
m2 = _PROP_RE.match(line)
if m2 and not line.lstrip().startswith(("-", "*", "#", "|")):
key = m2.group(1)
if key not in meta:
meta[key] = m2.group(2).strip()
continue
lines_out.append(line)
body = "\n".join(lines_out)
# Roam task markers → GFM checkboxes.
body = body.replace("{{[[TODO]]}}", "[ ] ").replace("{{[[DONE]]}}", "[x] ")
# Block references ((uuid)) → plain anchors.
body = re.sub(r"\(\(([0-9a-fA-F-]{6,})\)\)", r"[[\1]]", body)
body = convert_wikilinks(body)
return meta, body
class _OutlineBase(Importer):
markers: tuple[str, ...] = ()
property_syntax = False
source_id = "outline"
label = "Outliner"
description = ""
order = 30
def _text_matches(self, text: str) -> bool:
if any(m in text for m in self.markers if not m.endswith("/")):
return True
return bool(self.property_syntax and re.search(r"^\s*[a-zA-Z][\w-]*::", text, re.M))
def _looks_like(self, filename: str, data: bytes) -> bool:
low = filename.lower()
if low.endswith((".md", ".markdown", ".txt")):
return self._text_matches(decode_text(data))
if low.endswith(".zip"):
try:
zf = zipfile.ZipFile(io.BytesIO(data))
except (zipfile.BadZipFile, OSError):
return False
names = zf.namelist()
if any(m in name for m in self.markers if m.endswith("/") for name in names):
return True
for n in [x for x in names if x.lower().endswith(".md")][:10]:
try:
if self._text_matches(decode_text(zf.read(n))):
return True
except Exception: # noqa: BLE001
continue
return False
def detect(self, filename: str, data: bytes) -> bool:
return self._looks_like(filename, data)
def _emit(self, result: ImportResult, name: str, text: str, is_journal: bool) -> None:
meta, body = _clean_outline(text)
parts = name.replace("\\", "/").split("/")
raw_title = parts[-1].rsplit(".", 1)[0]
title = normalize_title(meta.get("title")) or (
_journal_title(raw_title) if is_journal else raw_title
)
props = {k: v for k, v in meta.items() if k != "title"}
tags = coerce_tags(meta.get("tags"))
if tags:
props["tags"] = tags
result.pages.append(ImportPage(
title=title or "Untitled",
markdown=body,
source_path=name,
parent_path="/".join(parts[:-1]),
properties=props,
external_id=name,
))
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
if filename.lower().endswith(".zip"):
try:
zf = zipfile.ZipFile(io.BytesIO(data))
except (zipfile.BadZipFile, OSError) as exc:
result.warn(f"Archive invalide : {exc}")
return result.finalize()
names = [n for n in zf.namelist() if not n.endswith("/")]
for name in [n for n in names if not n.lower().endswith(".md")]:
try:
result.attachments.append(ImportAttachment(
source_path=name,
filename=name.replace("\\", "/").rsplit("/", 1)[-1],
data=zf.read(name),
))
except Exception: # noqa: BLE001
continue
for name in sorted(
(n for n in names if n.lower().endswith(".md")),
key=lambda n: (n.count("/"), n.lower()),
):
try:
text = decode_text(zf.read(name))
except Exception as exc: # noqa: BLE001
result.warn(f"Lecture impossible : {name} ({exc})")
continue
self._emit(result, name, text, is_journal="journal" in name.lower())
return result.finalize()
text = decode_text(data)
self._emit(result, filename, text, is_journal=False)
return result.finalize()
@register_importer
class LogseqImporter(_OutlineBase):
source_id = "logseq"
label = "Logseq"
description = "Outliner Logseq : pages/journal, propriétés `key:: value`, block refs."
markers = ("property::", "logseq/", "journals/")
property_syntax = True
@register_importer
class RoamImporter(_OutlineBase):
source_id = "roam"
label = "Roam Research"
description = "Outliner Roam : `{{[[TODO]]}}`, block refs, wikilinks."
markers = _ROAM_MARKERS
+94
View File
@@ -0,0 +1,94 @@
"""FlowDeck — PDF importer (v5.6.0, Phase 3).
Best-effort text + image extraction from a PDF into a FlowDeck page (fidelity
depends on the source PDF; scanned documents have no text layer).
"""
from __future__ import annotations
import io
import re
from pathlib import Path
from app.services.importers.base import (
ImportAttachment,
Importer,
ImportPage,
ImportResult,
register_importer,
)
_MIME = {".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg",
".gif": "image/gif", ".webp": "image/webp", ".bmp": "image/bmp",
".tiff": "image/tiff", ".tif": "image/tiff"}
@register_importer
class PdfImporter(Importer):
source_id = "pdf"
label = "PDF"
description = "Extraction texte + images d'un PDF (fidélité limitée)."
extensions = (".pdf",)
order = 37
def detect(self, filename: str, data: bytes) -> bool:
return filename.lower().endswith(".pdf")
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
try:
from pypdf import PdfReader
except ImportError:
result.warn("pypdf n'est pas installé : import PDF indisponible")
return result.finalize()
try:
reader = PdfReader(io.BytesIO(data))
except Exception as exc: # noqa: BLE001
result.warn(f"PDF illisible : {exc}")
return result.finalize()
chunks: list[str] = []
empty_pages = 0
for index, page in enumerate(reader.pages, start=1):
try:
text = (page.extract_text() or "").strip()
except Exception: # noqa: BLE001
text = ""
if text:
if len(reader.pages) > 1:
chunks.append(f"## Page {index}\n\n{text}")
else:
chunks.append(text)
else:
empty_pages += 1
chunks.extend(self._page_images(page, index, result))
if empty_pages:
result.warn(f"{empty_pages} page(s) sans couche texte (document scanné ?)")
markdown = re.sub(r"\n{3,}", "\n\n", "\n\n".join(chunks)).strip()
title = Path(filename).stem or "Document"
result.pages.append(ImportPage(
title=title, markdown=markdown, source_path=filename, external_id=filename,
))
return result.finalize()
def _page_images(self, page, index: int, result: ImportResult) -> list[str]:
images: list[str] = []
try:
page_images = list(page.images)
except Exception: # noqa: BLE001
return images
for i, image in enumerate(page_images, start=1):
name = getattr(image, "name", "") or f"page{index}_img{i}.png"
name = Path(name).name
try:
payload = image.data
except Exception: # noqa: BLE001
continue
if not payload:
continue
result.attachments.append(ImportAttachment(
source_path=f"page{index}/{name}", filename=name, data=payload,
mime=_MIME.get(Path(name).suffix.lower(), "application/octet-stream"),
))
images.append(f"![{name}]({name})")
return images
+566
View File
@@ -0,0 +1,566 @@
"""FlowDeck — common import pipeline (v5.6.0, Phase 0).
Persists an :class:`~app.services.importers.base.ImportResult` into FlowDeck:
resolves the workspace, rebuilds the folder hierarchy (``parent_id``), stores
attachments, rewrites links, creates collections + rows, and records imported
items for idempotent re-imports.
"""
from __future__ import annotations
import hashlib
import json
import logging
import os
import re
from pathlib import Path
from typing import Any
from app.db import get_conn
from app.services.db_templates import materialize_properties
from app.services.export import markdown_to_blocks
from app.services.importers.base import ImportPage, ImportResult
from app.services.importers.tabular import apply_type_mapping
logger = logging.getLogger(__name__)
_IMG_RE = re.compile(r"!\[([^\]]*)\]\(([^)\s]+)(?:\s+\"[^\"]*\")?\)")
def _data_dir() -> Path:
return Path(os.environ.get("FLOWDECK_DATA_DIR", "/data"))
def _safe_filename(name: str) -> str:
base = Path(name.replace("\\", "/")).name
base = re.sub(r"[^\w.\- ()]+", "_", base).strip() or "file"
return base[:150]
def _sha1(*parts: str) -> str:
return hashlib.sha1("||".join(parts).encode("utf-8")).hexdigest()
def _resolve_workspace(conn, workspace_id: int | None, workspace_name: str | None,
user_login: str) -> tuple[int | None, str]:
if workspace_id:
row = conn.execute("SELECT id, name FROM workspaces WHERE id=?", (workspace_id,)).fetchone()
if row:
return row["id"], row["name"]
name = workspace_name or user_login
if name:
row = conn.execute("SELECT id, name FROM workspaces WHERE name=?", (name,)).fetchone()
if row:
return row["id"], row["name"]
return workspace_id, name or ""
def _rewrite_links(text: str, attachment_map: dict[str, str]) -> str:
"""Point Markdown links/images at uploaded attachment URLs."""
def repl(m: re.Match) -> str:
alt, target = m.group(1), m.group(2)
url = attachment_map.get(target) or attachment_map.get(Path(target).name.lower())
return f"![{alt}]({url})" if url else m.group(0)
text = re.sub(r"!\[([^\]]*)\]\(([^)\s]+)(?:\s+\"[^\"]*\")?\)", repl, text)
return text
def _extract_media(blocks: list[dict]) -> list[dict]:
"""Split inline image markdown out of paragraphs into real image blocks."""
out: list[dict] = []
for block in blocks:
if block.get("type") != "paragraph":
out.append(block)
continue
content = str(block.get("content", ""))
pos = 0
found = False
for match in _IMG_RE.finditer(content):
found = True
before = content[pos:match.start()].strip()
if before:
out.append({"type": "paragraph", "content": before})
out.append({"type": "image", "src": match.group(2), "alt": match.group(1)})
pos = match.end()
if not found:
out.append(block)
continue
tail = content[pos:].strip()
if tail:
out.append({"type": "paragraph", "content": tail})
return out
def _properties_callout(props: dict) -> dict | None:
if not props:
return None
lines = [f"**{k}** : {', '.join(map(str, v)) if isinstance(v, list) else v}" for k, v in props.items()]
return {"type": "callout", "icon": "ℹ️", "content": "\n".join(lines)}
def _blocks_for(page: ImportPage, attachment_map: dict[str, str], *, include_properties: bool) -> list[dict]:
if page.blocks:
return _extract_media(page.blocks)
md = _rewrite_links(page.markdown or "", attachment_map)
blocks = _extract_media(markdown_to_blocks(md))
if include_properties and page.properties:
callout = _properties_callout(page.properties)
if callout:
blocks.insert(0, callout)
return blocks
def preview_result(result: ImportResult) -> dict[str, Any]:
"""Dry-run preview: what would be created, without touching the database."""
pages = []
for page in result.pages:
if page.collection:
kind = "collection"
rows = len(page.collection.get("rows", []))
blocks = len(_blocks_for(page, {}, include_properties=False))
schema = page.collection.get("schema", [])
else:
kind = "page"
rows = 0
blocks = len(_blocks_for(page, {}, include_properties=False))
schema = []
pages.append({
"title": page.title,
"type": kind,
"source_path": page.source_path,
"parent_path": page.parent_path,
"properties": list(page.properties.keys()),
"rows": rows,
"blocks": blocks,
"schema": schema,
})
return {
"source": result.source,
"dry_run": True,
"pages": pages,
"stats": {
**result.stats,
"pages": len(result.pages),
"collections": sum(1 for p in result.pages if p.collection),
"attachments": len(result.attachments),
"warnings": len(result.warnings),
},
"warnings": result.warnings,
}
def _insert_page(conn, *, workspace: str, workspace_id: int | None, title: str,
blocks: list[dict], parent_id: int | None, sort_order: int,
content_format: str = "blocks") -> int:
cur = conn.execute(
"INSERT INTO pages (workspace, title, content, content_format, parent_section, "
"sort_order, workspace_id, parent_id) VALUES (?,?,?,?,'Private',?,?,?)",
(workspace, title or "Untitled", json.dumps(blocks), content_format,
sort_order, workspace_id, parent_id),
)
return cur.lastrowid
def _prop_id_map(conn, collection_id: int) -> dict[str, int]:
return {
r["name"]: r["id"]
for r in conn.execute(
"SELECT id, name FROM collection_properties WHERE collection_id=?",
(collection_id,),
).fetchall()
}
def _rows_to_values(rows: list[dict], id_map: dict[str, int]) -> list[tuple[str, dict]]:
"""Key row properties by property id (the shape the editor reads)."""
out: list[tuple[str, dict]] = []
for row in rows:
values: dict[str, Any] = {}
for name, value in (row.get("properties") or {}).items():
prop_id = id_map.get(name)
if prop_id is not None:
values[str(prop_id)] = value
out.append((row.get("title") or "Untitled", values))
return out
def _ensure_default_view(conn, collection_id: int) -> None:
conn.execute(
"INSERT INTO collection_views (collection_id, name, view_type, config_json) "
"VALUES (?,?,?,?)",
(collection_id, "Default View", "table",
json.dumps({"visible_properties": ["Title"], "sorts": [], "filters": []})),
)
def _insert_collection(conn, page: ImportPage, *, workspace_id: int | None,
parent_page_id: int | None) -> tuple[int, int]:
spec = page.collection or {}
schema = spec.get("schema", [])
cur = conn.execute(
"INSERT INTO collections (name, description, icon, schema_json, is_inline, "
"parent_page_id, workspace_id) VALUES (?,?,?,?,1,?,?)",
(page.title or spec.get("name") or "Imported database", "", "📥",
json.dumps(schema), parent_page_id, workspace_id),
)
collection_id = cur.lastrowid
materialize_properties(conn, collection_id, schema)
_ensure_default_view(conn, collection_id)
id_map = _prop_id_map(conn, collection_id)
position = 0
for title, values in _rows_to_values(spec.get("rows", []), id_map):
conn.execute(
"INSERT INTO collection_pages (collection_id, title, position, property_values_json) "
"VALUES (?,?,?,?)",
(collection_id, title, position, json.dumps(values)),
)
position += 1
return collection_id, position
def _upsert_collection_rows(conn, collection_id: int, rows: list[dict]) -> tuple[int, int]:
"""Update existing rows by title, insert the new ones. Returns (created, updated)."""
id_map = _prop_id_map(conn, collection_id)
existing = {
(r["title"] or "").strip(): r["id"]
for r in conn.execute(
"SELECT id, title FROM collection_pages WHERE collection_id=?", (collection_id,)
).fetchall()
}
max_pos = conn.execute(
"SELECT COALESCE(MAX(position), -1) FROM collection_pages WHERE collection_id=?",
(collection_id,),
).fetchone()[0]
created = updated = 0
for title, values in _rows_to_values(rows, id_map):
pid = existing.get((title or "").strip())
if pid:
conn.execute(
"UPDATE collection_pages SET property_values_json=?, updated_at=CURRENT_TIMESTAMP WHERE id=?",
(json.dumps(values), pid),
)
updated += 1
else:
max_pos += 1
conn.execute(
"INSERT INTO collection_pages (collection_id, title, position, property_values_json) "
"VALUES (?,?,?,?)",
(collection_id, title, max_pos, json.dumps(values)),
)
created += 1
return created, updated
def _update_page(conn, page_id: int, title: str, blocks: list[dict]) -> None:
conn.execute(
"UPDATE pages SET title=?, content=?, content_format='blocks', "
"updated_at=CURRENT_TIMESTAMP WHERE id=?",
(title or "Untitled", json.dumps(blocks), page_id),
)
def run_import(
result: ImportResult,
*,
workspace_id: int | None = None,
workspace_name: str | None = None,
user_login: str = "",
parent_page_id: int | None = None,
target_collection_id: int | None = None,
dry_run: bool = False,
dedup: bool = True,
include_properties: bool = True,
mapping: dict[str, str] | None = None,
mode: str | None = None,
) -> dict[str, Any]:
"""Persist an import result. Returns a report dict.
``mode`` controls re-import behaviour for items already imported (matched by
``import_items``): ``skip`` (default), ``update`` (re-sync in place) or
``duplicate`` (always create a new page).
"""
if dry_run:
return preview_result(result)
if mapping:
for page in result.pages:
if page.collection:
apply_type_mapping(page.collection, mapping)
if not mode:
mode = "skip" if dedup else "duplicate"
report: dict[str, Any] = {
"source": result.source,
"status": "ok",
"mode": mode,
"pages_created": 0,
"pages_updated": 0,
"collections_created": 0,
"rows_created": 0,
"rows_updated": 0,
"attachments": 0,
"skipped": 0,
"page_ids": [],
"errors": [],
"warnings": list(result.warnings),
}
with get_conn() as conn:
ws_id, ws_name = _resolve_workspace(conn, workspace_id, workspace_name, user_login)
if not ws_name:
report["status"] = "error"
report["warnings"].append("Workspace introuvable")
return report
attachment_map: dict[str, str] = {}
if result.attachments and ws_id:
dest_dir = _data_dir() / "uploads" / f"workspace_{ws_id}" / "import"
dest_dir.mkdir(parents=True, exist_ok=True)
for att in result.attachments:
safe = _safe_filename(att.filename)
target = dest_dir / safe
if target.exists():
target = dest_dir / f"{_sha1(att.source_path)[:8]}_{safe}"
try:
target.write_bytes(att.data)
except OSError:
continue
url = f"/api/files/{ws_id}/import/{target.name}"
attachment_map[att.source_path] = url
attachment_map[att.source_path.lower()] = url
attachment_map[Path(att.source_path).name.lower()] = url
report["attachments"] += 1
if target_collection_id:
return _import_into_collection(
conn, result, target_collection_id, report, attachment_map,
dedup=dedup, workspace_id=ws_id, include_properties=include_properties,
)
next_order = conn.execute(
"SELECT COALESCE(MAX(sort_order), -1) + 1 FROM pages WHERE workspace=? AND parent_id IS NULL",
(ws_name,),
).fetchone()[0]
order_counter = [next_order]
path_to_page: dict[str, int] = {}
folder_cache: dict[str, int | None] = {}
def ensure_folder(path: str, _depth: int = 0) -> int | None:
path = (path or "").strip("/")
if not path:
return parent_page_id
if path in folder_cache:
return folder_cache[path]
parent_path = "/".join(path.split("/")[:-1])
parent_id = ensure_folder(parent_path, _depth + 1)
title = path.split("/")[-1] or "Folder"
pid = _insert_page(
conn, workspace=ws_name, workspace_id=ws_id, title=title,
blocks=[], parent_id=parent_id, sort_order=order_counter[0],
)
order_counter[0] += 1
folder_cache[path] = pid
path_to_page[path] = pid
report["pages_created"] += 1
report["page_ids"].append(pid)
return pid
ordered = sorted(
result.pages,
key=lambda p: (p.parent_path.count("/") if p.parent_path else -1, p.source_path.lower()),
)
for page in ordered:
external = page.external_id or page.source_path or page.title
existing_pid = None
if external:
row = conn.execute(
"SELECT page_id FROM import_items WHERE workspace_id IS ? AND source=? AND external_id=?",
(ws_id, result.source, external),
).fetchone()
if row:
existing_pid = row["page_id"]
if existing_pid and mode == "skip":
report["skipped"] += 1
continue
try:
page_parent = ensure_folder(page.parent_path) if page.parent_path else parent_page_id
blocks = _blocks_for(page, attachment_map, include_properties=include_properties)
if existing_pid and mode == "update":
if page.collection:
cid = _collection_for_page(conn, existing_pid)
if cid:
materialize_properties(conn, cid, (page.collection or {}).get("schema", []))
created, updated = _upsert_collection_rows(
conn, cid, (page.collection or {}).get("rows", []))
report["rows_created"] += created
report["rows_updated"] += updated
blocks = blocks + [{"type": "embed", "embed_type": "collection",
"collection_id": cid, "content": ""}]
_update_page(conn, existing_pid, page.title, blocks)
report["pages_updated"] += 1
report["page_ids"].append(existing_pid)
path_to_page[page.source_path] = existing_pid
continue
if page.collection:
pid = _insert_page(
conn, workspace=ws_name, workspace_id=ws_id, title=page.title,
blocks=blocks, parent_id=page_parent, sort_order=order_counter[0],
)
order_counter[0] += 1
cid, rows = _insert_collection(conn, page, workspace_id=ws_id, parent_page_id=pid)
conn.execute(
"UPDATE pages SET content=? WHERE id=?",
(json.dumps(blocks + [{"type": "embed", "embed_type": "collection",
"collection_id": cid, "content": ""}]), pid),
)
report["collections_created"] += 1
report["rows_created"] += rows
report["pages_created"] += 1
report["page_ids"].append(pid)
else:
pid = _insert_page(
conn, workspace=ws_name, workspace_id=ws_id, title=page.title,
blocks=blocks, parent_id=page_parent, sort_order=order_counter[0],
)
order_counter[0] += 1
report["pages_created"] += 1
report["page_ids"].append(pid)
path_to_page[page.source_path] = pid
if external:
conn.execute(
"INSERT OR IGNORE INTO import_items (workspace_id, source, external_id, page_id) VALUES (?,?,?,?)",
(ws_id, result.source, external, pid),
)
except Exception as exc: # noqa: BLE001 - partial import must keep going
logger.warning("import failed for %r: %s", page.title, exc)
report["errors"].append({"title": page.title, "error": str(exc)})
conn.commit()
if report["errors"]:
report["status"] = "partial"
return report
def _collection_for_page(conn, page_id: int) -> int | None:
row = conn.execute(
"SELECT id FROM collections WHERE parent_page_id=? ORDER BY id LIMIT 1", (page_id,)
).fetchone()
return row["id"] if row else None
def _import_into_collection(conn, result: ImportResult, collection_id: int, report: dict,
attachment_map: dict[str, str], *, dedup: bool,
workspace_id: int | None, include_properties: bool) -> dict:
exists = conn.execute("SELECT id FROM collections WHERE id=?", (collection_id,)).fetchone()
if not exists:
report["status"] = "error"
report["warnings"].append("Collection cible introuvable")
return report
max_pos = conn.execute(
"SELECT COALESCE(MAX(position), -1) + 1 FROM collection_pages WHERE collection_id=?",
(collection_id,),
).fetchone()[0]
id_map = _prop_id_map(conn, collection_id)
for page in result.pages:
external = page.external_id or page.source_path or page.title
if dedup and external:
row = conn.execute(
"SELECT id FROM collection_pages WHERE collection_id=? AND title=?",
(collection_id, page.title),
).fetchone()
if row:
report["skipped"] += 1
continue
props = {
str(id_map[name]): value
for name, value in page.properties.items()
if name in id_map
}
conn.execute(
"INSERT INTO collection_pages (collection_id, title, position, property_values_json) "
"VALUES (?,?,?,?)",
(collection_id, page.title, max_pos, json.dumps(props)),
)
max_pos += 1
report["rows_created"] += 1
conn.commit()
return report
def resolve_relations(conn, workspace_id: int | None) -> dict[str, Any]:
"""Convert text columns that reference another imported collection's titles
into real ``relation`` properties (array of ``collection_pages`` ids)."""
collections = conn.execute(
"SELECT id, name FROM collections WHERE workspace_id IS ?", (workspace_id,)
).fetchall()
if not collections:
return {"relations_resolved": 0, "details": []}
titles: dict[int, dict[str, int]] = {}
for coll in collections:
rows = conn.execute(
"SELECT id, title FROM collection_pages WHERE collection_id=?", (coll["id"],)
).fetchall()
titles[coll["id"]] = {
(r["title"] or "").strip(): r["id"]
for r in rows if (r["title"] or "").strip()
}
resolved = 0
details: list[dict] = []
for coll in collections:
props = conn.execute(
"SELECT id, name, prop_type FROM collection_properties WHERE collection_id=?",
(coll["id"],),
).fetchall()
pages = conn.execute(
"SELECT id, property_values_json FROM collection_pages WHERE collection_id=?",
(coll["id"],),
).fetchall()
for prop in props:
if prop["prop_type"] not in ("text", "select", "multi_select"):
continue
key = str(prop["id"])
values: list[str] = []
for page in pages:
pv = json.loads(page["property_values_json"] or "{}")
value = pv.get(key)
if value in (None, "", []):
continue
items = value if isinstance(value, list) else [value]
values.extend(str(v) for v in items)
if not values:
continue
for target in collections:
if target["id"] == coll["id"]:
continue
target_titles = titles.get(target["id"]) or {}
if target_titles and all(v in target_titles for v in values):
conn.execute(
"UPDATE collection_properties SET prop_type='relation', "
"related_collection_id=? WHERE id=?",
(target["id"], prop["id"]),
)
for page in pages:
pv = json.loads(page["property_values_json"] or "{}")
value = pv.get(key)
if value in (None, "", []):
continue
items = value if isinstance(value, list) else [value]
pv[key] = [target_titles[str(v)] for v in items if str(v) in target_titles]
conn.execute(
"UPDATE collection_pages SET property_values_json=? WHERE id=?",
(json.dumps(pv), page["id"]),
)
resolved += 1
details.append({
"collection": coll["name"], "property": prop["name"],
"related": target["name"],
})
break
conn.commit()
return {"relations_resolved": resolved, "details": details}
+96
View File
@@ -0,0 +1,96 @@
"""FlowDeck — Standard Notes importer (v5.6.0, Phase 4).
Imports a Standard Notes backup (``.json``): each non-encrypted note becomes a
FlowDeck page. Encrypted notes are reported as warnings.
"""
from __future__ import annotations
import json
from typing import Any
from app.services.importers.base import (
Importer,
ImportPage,
ImportResult,
decode_text,
register_importer,
)
_NOTE_TYPES = ("note", "org.standardnotes.sn", "org.standardnotes.plain-text")
def _note_body(content: Any) -> tuple[str, str, bool]:
"""Return (title, markdown, encrypted)."""
if isinstance(content, dict):
if content.get("encrypted"):
return "", "", True
text = content.get("text") or content.get("preview_plain") or ""
title = content.get("title") or ""
return title, text, False
if isinstance(content, str):
stripped = content.strip()
if stripped.startswith("{"):
try:
parsed = json.loads(stripped)
if isinstance(parsed, dict) and ("encrypted" in parsed or "000" in parsed):
return "", "", True
if isinstance(parsed, dict):
return parsed.get("title", ""), parsed.get("text", "") or parsed.get("preview_plain", ""), False
except json.JSONDecodeError:
pass
return "", content, False
return "", "", False
@register_importer
class StandardNotesImporter(Importer):
source_id = "standard_notes"
label = "Standard Notes"
description = "Sauvegarde JSON Standard Notes → pages (notes chiffrées ignorées)."
extensions = (".json",)
order = 39
def _items(self, data: bytes) -> list[dict] | None:
try:
obj = json.loads(decode_text(data))
except Exception: # noqa: BLE001
return None
if isinstance(obj, dict) and isinstance(obj.get("items"), list):
return [x for x in obj["items"] if isinstance(x, dict)]
return None
def detect(self, filename: str, data: bytes) -> bool:
if not filename.lower().endswith(".json"):
return False
items = self._items(data)
if not items:
return False
return any("content_type" in it for it in items)
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
items = self._items(data) or []
count = 0
for item in items:
if item.get("deleted"):
continue
ctype = str(item.get("content_type", "")).lower()
if ctype and not any(t in ctype for t in _NOTE_TYPES):
continue
title, body, encrypted = _note_body(item.get("content"))
if encrypted:
result.warn("Note chiffrée ignorée (déchiffrement non pris en charge)")
continue
if not body.strip():
continue
if not title:
title = next((ln.strip(" #") for ln in body.splitlines() if ln.strip()), "Note")
result.pages.append(ImportPage(
title=title[:200] or "Note",
markdown=body,
source_path=item.get("uuid") or filename,
external_id=item.get("uuid") or f"{filename}#{count}",
))
count += 1
result.stats["rows"] = count
return result.finalize()
+305
View File
@@ -0,0 +1,305 @@
"""FlowDeck — tabular importers: typed CSV/TSV, Excel, generic JSON (v5.6.0, Phase 2).
Each source becomes a FlowDeck collection (database): columns are inferred from
the data (text/number/date/checkbox/email/url/select/multi_select) and rows are
inserted as ``collection_pages``.
"""
from __future__ import annotations
import csv
import io
import json
import re
from typing import Any
from app.services.importers.base import (
Importer,
ImportPage,
ImportResult,
decode_text,
register_importer,
)
_TITLE_HEADERS = ("title", "name", "task", "nom", "titre", "subject", "label")
_DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}([T ]\d{2}:\d{2}(:\d{2})?)?")
_EMAIL_RE = re.compile(r"^[^@\s]+@[^@\s]+\.[^@\s]+$")
_URL_RE = re.compile(r"^https?://\S+$", re.IGNORECASE)
_BOOL_TRUE = {"true", "yes", "oui", "1", "x", "vrai"}
_BOOL_FALSE = {"false", "no", "non", "0", "faux", ""}
def _is_number(value: str) -> bool:
try:
float(str(value).replace(",", ".").replace(" ", ""))
return True
except (ValueError, TypeError):
return False
def _is_bool(value: str) -> bool:
return str(value).strip().lower() in _BOOL_TRUE | _BOOL_FALSE
def infer_column_type(values: list[str]) -> str:
"""Infer the FlowDeck property type from a list of raw string cells."""
sample = [str(v).strip() for v in values if str(v).strip()]
if not sample:
return "text"
if all(_is_bool(v) for v in sample):
return "checkbox"
if all(_is_number(v) for v in sample):
return "number"
if all(_DATE_RE.match(v) for v in sample):
return "date"
if all(_EMAIL_RE.match(v) for v in sample):
return "email"
if all(_URL_RE.match(v) for v in sample):
return "url"
unique = {v for v in sample}
if len(unique) <= 20 and len(unique) <= max(2, len(sample) // 2):
if any(("," in v or ";" in v) for v in sample):
return "multi_select"
return "select"
return "text"
def _split_multi(value: str) -> list[str]:
return [p.strip() for p in re.split(r"[;,]", value) if p.strip()]
def coerce_value(prop_type: str, value: Any) -> Any:
if value is None:
return None
raw = str(value).strip()
if raw == "":
return None
if prop_type == "number":
try:
num = float(raw.replace(",", ".").replace(" ", ""))
return int(num) if num.is_integer() else num
except (ValueError, TypeError):
return raw
if prop_type == "checkbox":
return raw.lower() in _BOOL_TRUE
if prop_type == "multi_select":
return _split_multi(raw)
return raw
def build_schema(headers: list[str], rows: list[dict[str, Any]]) -> tuple[list[dict], str]:
"""Return ``(schema, title_header)`` from headers + row dicts."""
title_header = ""
for h in headers:
if h and h.strip().lower() in _TITLE_HEADERS:
title_header = h
break
schema: list[dict] = []
for h in headers:
if not h or h == title_header:
continue
ptype = infer_column_type([r.get(h, "") for r in rows])
entry: dict[str, Any] = {"name": h, "type": ptype}
if ptype in ("select", "status", "multi_select"):
seen: list[str] = []
for r in rows:
vals = _split_multi(str(r.get(h, ""))) if ptype == "multi_select" else [str(r.get(h, "")).strip()]
for v in vals:
if v and v not in seen:
seen.append(v)
entry["options"] = [{"name": v, "color": "gray"} for v in seen[:100]]
schema.append(entry)
if title_header:
schema.insert(0, {"name": title_header, "type": "title"})
return schema, title_header
def rows_to_collection(name: str, headers: list[str], raw_rows: list[dict[str, Any]]) -> ImportPage:
"""Normalize parsed rows into an ImportPage carrying a collection spec."""
schema, title_header = build_schema(headers, raw_rows)
rows = _normalize_rows(headers, raw_rows, schema, title_header)
return ImportPage(
title=name or "Imported database",
collection={
"name": name or "Imported database",
"schema": schema,
"rows": rows,
"headers": headers,
"title_header": title_header,
"raw_rows": raw_rows,
},
source_path=name,
external_id=name,
)
def _normalize_rows(headers: list[str], raw_rows: list[dict[str, Any]],
schema: list[dict], title_header: str) -> list[dict[str, Any]]:
types = {s["name"]: s["type"] for s in schema}
rows: list[dict[str, Any]] = []
for raw in raw_rows:
title = ""
if title_header:
title = str(raw.get(title_header, "")).strip()
if not title:
for h in headers:
if h and str(raw.get(h, "")).strip():
title = str(raw[h]).strip()
break
props: dict[str, Any] = {}
for h in headers:
if not h or h == title_header:
continue
val = coerce_value(types.get(h, "text"), raw.get(h))
if val is not None and val != "":
props[h] = val
rows.append({"title": title or "Untitled", "properties": props})
return rows
def apply_type_mapping(spec: dict, mapping: dict[str, str]) -> dict:
"""Override inferred column types (UI mapping) and re-coerce the rows."""
if not mapping:
return spec
for entry in spec.get("schema", []):
if entry.get("name") in mapping:
entry["type"] = mapping[entry["name"]]
headers = spec.get("headers")
raw_rows = spec.get("raw_rows")
if headers is not None and raw_rows is not None:
spec["rows"] = _normalize_rows(headers, raw_rows, spec.get("schema", []),
spec.get("title_header", ""))
return spec
def _sniff_delimiter(sample: str) -> str:
try:
return csv.Sniffer().sniff(sample, delimiters=",;\t|").delimiter
except csv.Error:
return "\t" if sample.count("\t") > sample.count(",") else ","
@register_importer
class CsvImporter(Importer):
source_id = "csv"
label = "CSV / TSV (typé)"
description = "Tableur CSV/TSV : types inférés automatiquement, une collection par fichier."
extensions = (".csv", ".tsv")
order = 60
def detect(self, filename: str, data: bytes) -> bool:
return filename.lower().endswith((".csv", ".tsv"))
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
text = decode_text(data)
if not text.strip():
result.warn("Fichier vide")
return result.finalize()
delimiter = "\t" if filename.lower().endswith(".tsv") else _sniff_delimiter(text[:4096])
reader = csv.DictReader(io.StringIO(text), delimiter=delimiter)
headers = [h for h in (reader.fieldnames or []) if h is not None]
rows = [dict(r) for r in reader]
name = filename.replace("\\", "/").rsplit("/", 1)[-1].rsplit(".", 1)[0]
result.pages.append(rows_to_collection(name, headers, rows))
result.stats["rows"] = len(rows)
return result.finalize()
@register_importer
class ExcelImporter(Importer):
source_id = "excel"
label = "Excel (.xlsx)"
description = "Classeur Excel : une collection par feuille (openpyxl)."
extensions = (".xlsx", ".xlsm")
order = 61
def detect(self, filename: str, data: bytes) -> bool:
low = filename.lower()
if low.endswith((".xlsx", ".xlsm")):
return True
return low.endswith(".xls")
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
try:
from openpyxl import load_workbook
except ImportError:
result.warn("openpyxl n'est pas installé : import Excel indisponible")
return result.finalize()
try:
wb = load_workbook(io.BytesIO(data), read_only=True, data_only=True)
except Exception as exc: # noqa: BLE001
result.warn(f"Classeur illisible : {exc}")
return result.finalize()
base = filename.replace("\\", "/").rsplit("/", 1)[-1].rsplit(".", 1)[0]
total_rows = 0
for ws in wb.worksheets:
values = list(ws.iter_rows(values_only=True))
if not values:
continue
headers = [str(h).strip() if h is not None else f"Column {i + 1}" for i, h in enumerate(values[0])]
rows: list[dict[str, Any]] = []
for row in values[1:]:
if row is None or all(c is None or str(c).strip() == "" for c in row):
continue
rows.append({headers[i]: row[i] for i in range(min(len(headers), len(row)))})
if not rows:
continue
name = f"{base} — {ws.title}" if len(wb.worksheets) > 1 else (base or ws.title)
page = rows_to_collection(name, headers, rows)
page.source_path = f"{filename}#{ws.title}"
page.external_id = page.source_path
result.pages.append(page)
total_rows += len(rows)
result.stats["rows"] = total_rows
return result.finalize()
@register_importer
class JsonImporter(Importer):
source_id = "json"
label = "JSON (mapping générique)"
description = "Tableau d'objets JSON → collection (union des clés)."
extensions = (".json",)
order = 65
def _records(self, data: bytes) -> list[dict] | None:
try:
obj = json.loads(decode_text(data))
except Exception: # noqa: BLE001
return None
if isinstance(obj, list) and obj and all(isinstance(x, dict) for x in obj):
return obj
if isinstance(obj, dict):
for value in obj.values():
if isinstance(value, list) and value and all(isinstance(x, dict) for x in value):
return value
return None
def detect(self, filename: str, data: bytes) -> bool:
if not filename.lower().endswith(".json"):
return False
return self._records(data) is not None
def parse(self, filename: str, data: bytes) -> ImportResult:
result = ImportResult(source=self.source_id)
records = self._records(data)
if not records:
result.warn("Aucun tableau d'objets JSON détecté")
return result.finalize()
headers: list[str] = []
for rec in records:
for key in rec:
if key not in headers:
headers.append(key)
flat: list[dict[str, Any]] = []
for rec in records:
row = {}
for h in headers:
v = rec.get(h)
row[h] = json.dumps(v, ensure_ascii=False) if isinstance(v, (dict, list)) else v
flat.append(row)
name = filename.replace("\\", "/").rsplit("/", 1)[-1].rsplit(".", 1)[0]
result.pages.append(rows_to_collection(name, headers, flat))
result.stats["rows"] = len(flat)
return result.finalize()
+94
View File
@@ -0,0 +1,94 @@
"""FlowDeck — URL / web clipper importer (v5.6.0, Phase 5).
Fetches a web page and turns it into a page: a bookmark card (OG metadata)
followed by the article converted to FlowDeck blocks.
"""
from __future__ import annotations
import ipaddress
import socket
from urllib.parse import urlparse
import httpx
from app.services.export import markdown_to_blocks
from app.services.importers.base import ImportPage, ImportResult
from app.services.importers.html_notes import _html_to_markdown
_BLOCKED_HOSTS = {"localhost", "localhost.localdomain"}
_MAX_BYTES = 3_000_000
def _is_public_host(host: str) -> bool:
"""SSRF guard: reject loopback/private/link-local/reserved addresses."""
if not host or host.lower() in _BLOCKED_HOSTS:
return False
try:
infos = socket.getaddrinfo(host, None)
except socket.gaierror:
return False
for info in infos:
address = info[4][0]
try:
ip = ipaddress.ip_address(address)
except ValueError:
return False
if (ip.is_private or ip.is_loopback or ip.is_link_local
or ip.is_reserved or ip.is_multicast or ip.is_unspecified):
return False
return True
def _validate_url(url: str) -> str:
parsed = urlparse(url.strip())
if parsed.scheme not in ("http", "https"):
raise ValueError("Seules les URLs http(s) sont autorisées")
if not parsed.hostname or not _is_public_host(parsed.hostname):
raise ValueError("Hôte non autorisé")
return url.strip()
def _bookmark_block(url: str, meta: dict) -> dict:
block = {"type": "bookmark", "url": url}
for key in ("title", "description", "image", "site_name"):
if meta.get(key):
block[key] = meta[key]
return block
async def fetch_url_result(url: str, *, transport: httpx.BaseTransport | None = None) -> ImportResult:
"""Fetch ``url`` and build a single-page ImportResult (raises on bad URL)."""
safe_url = _validate_url(url)
result = ImportResult(source="url")
try:
async with httpx.AsyncClient(
timeout=15, follow_redirects=True, transport=transport,
headers={"User-Agent": "FlowDeck-Importer/1.0"},
) as client:
response = await client.get(safe_url)
response.raise_for_status()
if response.url.host and not _is_public_host(response.url.host):
raise ValueError("Redirection vers un hôte non autorisé")
content_type = response.headers.get("content-type", "")
if "html" not in content_type.lower():
raise ValueError("La ressource n'est pas une page HTML")
body = response.text[:_MAX_BYTES]
except httpx.HTTPError as exc:
result.warn(f"Échec du téléchargement : {exc}")
return result.finalize()
from app.services.og_fetcher import parse_og
meta = parse_og(body, safe_url)
title = (meta.get("title") or urlparse(safe_url).hostname or "Page").strip()
markdown = _html_to_markdown(body)
blocks = [_bookmark_block(safe_url, meta)]
if markdown:
blocks.extend(markdown_to_blocks(markdown))
result.pages.append(ImportPage(
title=title[:200],
blocks=blocks,
source_path=safe_url,
external_id=safe_url,
))
return result.finalize()
+106 -10
View File
@@ -5,8 +5,11 @@ directly — it emits *tool intentions* (function calls) that AgentEngine turns
into guarded internal actions.
Supported providers (OpenAI-compatible chat-completions JSON response):
openai, anthropic*, google*, deepseek, qwencloud, nvidia, openrouter, ollama.
(* routed through an OpenAI-compatible gateway / any configured api_base)
openai, anthropic, mistral, cohere, google, groq, deepseek, openrouter,
nvidia, together, perplexity, xai, qwencloud, minimax, morph, fireworks,
cerebras, sambanova, chutes, xiaomi, sealion, sensenova,
ollama (local, no key). Anthropic, Google (`/v1beta/openai`) and Cohere
(`/compatibility/v1`) expose an OpenAI-compatible surface at their base URL.
When no API key is configured (or provider == "offline") the client falls back
to a deterministic, dependency-free *mock planner*. This keeps the whole agent
@@ -28,30 +31,104 @@ from app.config import settings
logger = logging.getLogger(__name__)
# Provider → default model + base URL when llm_model/api_base are empty.
# All entries speak the OpenAI-compatible chat-completions protocol (Anthropic,
# Google and Cohere expose an OpenAI-compatible surface at their given base).
PROVIDERS = {
"openai": ("https://api.openai.com/v1", "gpt-4o"),
"anthropic": ("https://api.anthropic.com/v1", "claude-opus-4-8"),
"google": ("https://generativelanguage.googleapis.com/v1beta", "gemini-2.0-pro"),
"mistral": ("https://api.mistral.ai/v1", "mistral-large-latest"),
"cohere": ("https://api.cohere.ai/compatibility/v1", "command-a-plus-05-2026"),
"google": ("https://generativelanguage.googleapis.com/v1beta/openai", "gemini-2.0-flash"),
"groq": ("https://api.groq.com/openai/v1", "llama-3.3-70b-versatile"),
"deepseek": ("https://api.deepseek.com/v1", "deepseek-chat"),
"qwencloud": ("https://dashscope.aliyuncs.com/compatible-mode/v1", "qwen-max"),
"nvidia": ("https://integrate.api.nvidia.com/v1", "nvidia/nemotron-3-super-120b-a12b"),
"openrouter": ("https://openrouter.ai/api/v1", "meta-llama/llama-3.3-70b-instruct"),
"nvidia": ("https://integrate.api.nvidia.com/v1", "nvidia/nemotron-3-super-120b-a12b"),
"together": ("https://api.together.xyz/v1", "meta-llama/Llama-3.3-70B-Instruct-Turbo"),
"perplexity": ("https://api.perplexity.ai", "sonar-pro"),
"xai": ("https://api.x.ai/v1", "grok-4.6"),
"qwencloud": ("https://dashscope-intl.aliyuncs.com/compatible-mode/v1", "qwen-max"),
"minimax": ("https://api.minimax.chat/v1", "MiniMax-Text-01"),
"morph": ("https://api.morphllm.com/v1", "morph-v3-large"),
"fireworks": ("https://api.fireworks.ai/inference/v1",
"accounts/fireworks/models/deepseek-v4-pro-0813"),
"cerebras": ("https://api.cerebras.ai/v1", "llama-3.3-70b"),
"sambanova": ("https://api.sambanova.ai/v1", "Meta-Llama-3.3-70B-Instruct"),
"chutes": ("https://llm.chutes.ai/v1", "deepseek-ai/DeepSeek-V3"),
"xiaomi": ("https://api.xiaomimimo.com/v1", "mimo-7b-rl"),
"sealion": ("https://api.sea-lion.ai/v1", "aisingapore/Llama-SEA-LION-v3-70B-IT"),
"sensenova": ("https://api.sensenova.cn/compatible-mode/v1", "SenseChat-5"),
"ollama": ("http://localhost:11434/v1", "llama3.1"),
"offline": (None, None),
}
# Friendly display names for the Settings / Agent UIs.
PROVIDER_LABELS: dict[str, str] = {
"openai": "OpenAI",
"anthropic": "Anthropic",
"mistral": "Mistral",
"cohere": "Cohere",
"google": "Google Gemini",
"groq": "Groq",
"deepseek": "DeepSeek",
"openrouter": "OpenRouter",
"nvidia": "NVIDIA NIM",
"together": "Together AI",
"perplexity": "Perplexity",
"xai": "xAI (Grok)",
"qwencloud": "DashScope (Alibaba)",
"minimax": "MiniMax",
"morph": "Morph",
"fireworks": "Fireworks AI",
"cerebras": "Cerebras",
"sambanova": "SambaNova",
"chutes": "Chutes AI",
"xiaomi": "Xiaomi (MiMo)",
"sealion": "SEA-LION",
"sensenova": "SenseNova",
"ollama": "Ollama (local)",
"offline": "Hors-ligne (mock)",
}
# Curated model presets surfaced by /api/agent/providers for the UI selectors.
PROVIDER_MODELS: dict[str, list[str]] = {
"openai": ["gpt-4o", "gpt-4o-mini", "gpt-4.1", "gpt-4.1-mini", "o3-mini", "gpt-4-turbo"],
"anthropic": ["claude-opus-4-8", "claude-sonnet-4-5", "claude-3-5-sonnet", "claude-haiku-4-5"],
"google": ["gemini-2.0-pro", "gemini-2.0-flash", "gemini-1.5-pro", "gemini-1.5-flash"],
"mistral": ["mistral-large-latest", "mistral-medium-latest", "mistral-small-latest",
"codestral-latest", "open-mistral-nemo", "pixtral-large-latest"],
"cohere": ["command-a-plus-05-2026", "command-r-plus", "command-r", "command-a-03-2025"],
"google": ["gemini-2.0-flash", "gemini-2.0-flash-lite", "gemini-1.5-pro", "gemini-1.5-flash"],
"groq": ["llama-3.3-70b-versatile", "llama-3.1-8b-instant",
"mixtral-8x7b-32768", "gemma2-9b-it"],
"deepseek": ["deepseek-chat", "deepseek-reasoner"],
"qwencloud": ["qwen-max", "qwen-plus", "qwen-turbo", "qwen-long"],
"openrouter": ["meta-llama/llama-3.3-70b-instruct", "anthropic/claude-3.5-sonnet",
"openai/gpt-4o", "mistralai/mistral-large"],
"nvidia": ["nvidia/nemotron-3-super-120b-a12b", "nvidia/nemotron-3-nano-30b-a3b",
"meta/llama-3.1-70b-instruct", "nvidia/llama-3.3-nemotron-super-49b-v1.5",
"deepseek-ai/deepseek-v4-pro", "z-ai/glm-5.2"],
"openrouter": ["meta-llama/llama-3.3-70b-instruct", "anthropic/claude-3.5-sonnet",
"openai/gpt-4o", "mistralai/mistral-large"],
"together": ["meta-llama/Llama-3.3-70B-Instruct-Turbo",
"meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo",
"Qwen/Qwen2.5-72B-Instruct-Turbo", "mistralai/Mixtral-8x7B-Instruct-v0.1"],
"perplexity": ["sonar-pro", "sonar", "sonar-reasoning", "sonar-deep-research"],
"xai": ["grok-4.6", "grok-4.5", "grok-4.3", "grok-4.20-0309-reasoning",
"grok-build-0.1"],
"qwencloud": ["qwen-max", "qwen-plus", "qwen-turbo", "qwen-long"],
"minimax": ["MiniMax-Text-01", "abab6.5s-chat", "abab6.5-chat"],
"morph": ["morph-v3-large", "morph-v3-fast"],
"fireworks": ["accounts/fireworks/models/deepseek-v4-pro-0813",
"accounts/fireworks/models/kimi-k2p6",
"accounts/fireworks/models/glm-5p2",
"accounts/fireworks/models/minimax-m3",
"accounts/fireworks/models/gpt-oss-120b",
"accounts/fireworks/models/qwen3-8b"],
"cerebras": ["llama-3.3-70b", "llama3.1-8b", "llama-3.1-70b"],
"sambanova": ["Meta-Llama-3.3-70B-Instruct", "Meta-Llama-3.1-405B-Instruct",
"Qwen2.5-72B-Instruct"],
"chutes": ["deepseek-ai/DeepSeek-V3", "deepseek-ai/DeepSeek-R1",
"Qwen/Qwen2.5-72B-Instruct"],
"xiaomi": ["mimo-7b-rl", "mimo-7b"],
"sealion": ["aisingapore/Llama-SEA-LION-v3-70B-IT",
"aisingapore/Gemma-SEA-LION-v3-9B-IT"],
"sensenova": ["SenseChat-5", "SenseChat-5-Cantonese", "SenseChat-Turbo"],
"ollama": ["llama3.1", "llama3", "mistral", "qwen2.5", "gemma2", "mixtral"],
"offline": [],
}
@@ -162,6 +239,24 @@ class LLMClient:
def _endpoint(self) -> str:
return f"{self.api_base.rstrip('/')}/chat/completions"
@staticmethod
def _http_error_detail(exc: httpx.HTTPStatusError) -> str:
"""Human-readable HTTP error including the provider's response body.
Providers return actionable JSON on 4xx (e.g. « model not allowed »,
« country not supported »); surfacing it makes the Settings test
debuggable instead of a bare « 403 Forbidden ».
"""
resp = exc.response
try:
body = (resp.text or "").strip()
except Exception: # noqa: BLE001 — body already consumed / undecodable
body = ""
if len(body) > 500:
body = body[:500] + "…"
base = f"{resp.status_code} {resp.reason_phrase} ({resp.url})"
return f"{base}: {body}" if body else base
async def _http_complete(self, messages, model, tools, *, _noticer: str = "") -> LLMResponse:
payload: dict = {
"model": model,
@@ -195,7 +290,8 @@ class LLMClient:
f"Le modèle « {model} » n'est plus disponible ({exc.response.status_code}). "
f"Réponse générée avec « {self.default_model} » à la place."
))
raise
# Surface the provider's own error body (403 « forbidden », 400 …).
raise RuntimeError(self._http_error_detail(exc)) from exc
response = self._parse_response(data, model)
response.notice = _noticer or ""
+47 -12
View File
@@ -14,7 +14,7 @@ import json
import time
from app.config import settings
from app.services.llm_client import PROVIDER_MODELS, PROVIDERS
from app.services.llm_client import PROVIDER_LABELS, PROVIDER_MODELS, PROVIDERS
# Providers whose /v1/models lists far more entries than /v1/chat/completions
# actually serves. The fetched list is validated (name filter + live probe)
@@ -101,8 +101,12 @@ def set_llm_config(*, provider: str | None = None, model: str | None = None,
cfg["verified"] = 0
cfg["verified_model"] = ""
cfg["last_error"] = ""
if api_base is not None and api_base.strip():
cfg["api_base"] = api_base.strip()
if api_base is not None:
# Normalise so a base equal to the provider default is stored as "use
# default" — a later correction of PROVIDERS then applies automatically.
cfg["api_base"] = _normalize_api_base(
provider or cfg.get("provider") or "offline", api_base
)
if clear_keys:
cfg["api_key"] = ""
cfg["api_base"] = ""
@@ -165,7 +169,7 @@ def provider_info() -> list[dict]:
models.insert(0, default_model)
out.append({
"id": name,
"name": name.capitalize(),
"name": PROVIDER_LABELS.get(name) or name.replace("_", " ").title(),
"default_model": default_model,
"models": models,
"base_url": (base or ""),
@@ -222,18 +226,44 @@ def list_user_llm_keys(user_id: int) -> list[dict]:
return [_mask_key(r) for r in rows]
def _default_api_base(provider: str) -> str:
"""The OpenAI-compatible base URL the app uses by default for a provider."""
base = (PROVIDERS.get(provider.lower()) or (None, None))[0]
return (base or "").rstrip("/")
def _normalize_api_base(provider: str, api_base: str) -> str:
"""Drop an api_base that just repeats the provider default.
Storing the default as a per-user override freezes it: a later correction
of the provider URL (e.g. Cohere `/v2` → `/compatibility/v1`) would never
apply. An empty value means "use the provider default".
"""
value = (api_base or "").strip()
if value.rstrip("/") == _default_api_base(provider):
return ""
return value
def upsert_user_llm_key(user_id: int, provider: str, *, api_key: str = "",
api_base: str = "", default_model: str = "",
api_base: str | None = None, default_model: str = "",
models: list[str] | None = None) -> dict:
"""Upsert a user's provider key. Empty api_key keeps the existing one
(allows saving model/base without re-typing the key). Saving a *different*
key resets the `verified` flag so the provider must pass a test again."""
key resets the `verified` flag so the provider must pass a test again.
``api_base`` uses ``None`` to mean "keep the stored value" and an empty
string to explicitly reset it to the provider default.
"""
from app.db import get_conn
provider = provider.lower()
existing = get_user_llm_key(user_id, provider)
new_key = api_key if api_key else (existing.get("api_key", "") if existing else "")
new_base = api_base if api_base else (existing.get("api_base", "") if existing else "")
if api_base is None:
new_base = (existing.get("api_base", "") if existing else "")
else:
new_base = _normalize_api_base(provider, api_base)
new_model = default_model if default_model else (existing.get("default_model", "") if existing else "")
new_models = models if models is not None else (
json.loads(existing["models_json"]) if existing and existing.get("models_json") else []
@@ -296,8 +326,9 @@ async def fetch_provider_models(provider: str, *, api_key: str = "",
api_base: str = "", timeout: int = 20) -> list[str]:
"""Fetch the live model list from a provider (best-effort, no mock).
OpenAI-compatible providers use `GET {base}/models` with a Bearer token;
Anthropic uses `x-api-key` + `anthropic-version`; Gemini an `x-goog-api-key`.
OpenAI-compatible providers (including Google's `/openai` surface and
Cohere's compatibility API) use `GET {base}/models` with a Bearer token;
Anthropic's native model listing uses `x-api-key` + `anthropic-version`.
Returns a de-duplicated list capped at 300 models.
"""
import httpx
@@ -311,14 +342,18 @@ async def fetch_provider_models(provider: str, *, api_key: str = "",
headers: dict = {}
if provider == "anthropic":
headers = {"x-api-key": api_key, "anthropic-version": "2023-06-01"}
elif provider == "google":
headers = {"x-goog-api-key": api_key}
elif api_key:
headers = {"Authorization": f"Bearer {api_key}"}
async with httpx.AsyncClient(timeout=timeout) as client:
resp = await client.get(f"{base_url}/models", headers=headers)
resp.raise_for_status()
if resp.status_code >= 400:
body = (resp.text or "").strip()
if len(body) > 500:
body = body[:500] + "…"
raise RuntimeError(
f"{resp.status_code} {resp.reason_phrase} ({resp.url}): {body}"
)
data = resp.json()
ids: list[str] = []
+112
View File
@@ -0,0 +1,112 @@
"""FlowDeck — AI Meeting Notes v2 (v7.1.0).
Upload audio → optional server transcription (``STT_COMMAND``, e.g. whisper)
→ AI summary (``AIWritingService.summarize``, offline-capable) → fires
``meeting.summarized`` so custom agents pick it up (Notion 07/2026 pattern).
Without ``STT_COMMAND`` the server stores the audio and accepts a manual
``transcript`` (client-side transcription). Nothing here requires new pip
dependencies. See ``docs/V71_Calendar_Meetings.md``.
"""
from __future__ import annotations
import logging
import os
import shutil
import subprocess
from pathlib import Path
from app.db import get_conn
logger = logging.getLogger(__name__)
AUDIO_EXTENSIONS = {"mp3", "wav", "m4a", "ogg", "flac", "aac"}
MAX_AUDIO_BYTES = 100 * 1024 * 1024
class TranscriptionUnavailable(RuntimeError):
"""Raised when no transcription backend is configured."""
def meetings_dir() -> Path:
root = Path(os.environ.get("FLOWDECK_DATA_DIR", "/data"))
d = root / "uploads" / "meetings"
d.mkdir(parents=True, exist_ok=True)
return d
def save_transcript(page_id: int, transcript: str, language: str = "fr",
audio_path: str = "") -> int:
with get_conn() as conn:
if not conn.execute("SELECT id FROM pages WHERE id=?", (page_id,)).fetchone():
raise ValueError("page not found")
cur = conn.execute(
"""INSERT INTO meeting_transcripts (page_id, audio_path, transcript, language)
VALUES (?,?,?,?)""",
(page_id, audio_path, transcript or "", language or "fr"))
conn.commit()
return cur.lastrowid
def transcribe_audio(audio_path: str, language: str = "fr") -> str:
"""Transcribe with ``STT_COMMAND`` (``{cmd} {file}` → stdout text).
Example: ``STT_COMMAND="whisper --language fr --output_format txt --output_dir /tmp"``
(command must print or be adapted — stdout is preferred). Raises
:class:`TranscriptionUnavailable` when unconfigured.
"""
cmd_template = os.environ.get("STT_COMMAND", "").strip()
if not cmd_template:
raise TranscriptionUnavailable(
"no transcription backend (set STT_COMMAND or POST a manual transcript)")
if shutil.which(cmd_template.split()[0]) is None:
raise TranscriptionUnavailable(f"STT command not found: {cmd_template.split()[0]}")
try:
proc = subprocess.run(cmd_template.split() + [audio_path], # noqa: S603 — admin-configured
capture_output=True, text=True, timeout=600)
except subprocess.TimeoutExpired as exc:
raise TranscriptionUnavailable("transcription timed out") from exc
text = (proc.stdout or "").strip()
if proc.returncode != 0 or not text:
raise TranscriptionUnavailable(
f"transcription failed: {(proc.stderr or '')[:300]}")
return text
async def summarize_transcript(transcript_id: int, user_id: int | None = None) -> dict:
"""Summarize a stored transcript + fire ``meeting.summarized``.
Returns {transcript_id, summary, offline}. Emits the automation event so
custom agents (update tracker, post recap, file tickets) trigger.
"""
from app.services.ai_writing import AIWritingService
with get_conn() as conn:
row = conn.execute("SELECT * FROM meeting_transcripts WHERE id=?",
(transcript_id,)).fetchone()
if not row:
raise ValueError("transcript not found")
tr = dict(row)
if not (tr.get("transcript") or "").strip():
raise ValueError("transcript is empty — transcribe first")
svc = AIWritingService(user_id=user_id)
res = await svc.run("summarize", context=tr["transcript"])
summary = (res.get("text") or "").strip() if res.get("ok") else ""
if not summary:
raise RuntimeError(f"summarization failed: {res.get('error', 'unknown')}")
with get_conn() as conn:
conn.execute("UPDATE meeting_transcripts SET summary=? WHERE id=?",
(summary, transcript_id))
page = conn.execute("SELECT id FROM pages WHERE id=?", (tr["page_id"],)).fetchone()
conn.commit()
try:
from app.services.automations import fire_event
await fire_event("meeting.summarized", {
"page_id": tr["page_id"] if page else 0,
"transcript_id": transcript_id,
"language": tr.get("language") or "fr",
})
except Exception as exc: # noqa: BLE001 — summary stands even if dispatch fails
logger.debug("meeting.summarized dispatch failed: %s", exc)
return {"transcript_id": transcript_id, "summary": summary,
"offline": bool(res.get("offline"))}
+68 -2
View File
@@ -121,13 +121,15 @@ def get_user_prefs(user_id: int, conn=None) -> dict:
"SELECT notification_prefs FROM users WHERE id=?", (user_id,)
).fetchone()
if not row or not row["notification_prefs"]:
return {"comments": True, "mentions": True}
return {"comments": True, "mentions": True, "reminders": True, "assignments": True}
try:
prefs = json.loads(row["notification_prefs"])
except (TypeError, json.JSONDecodeError):
prefs = {}
return {"comments": bool(prefs.get("comments", True)),
"mentions": bool(prefs.get("mentions", True))}
"mentions": bool(prefs.get("mentions", True)),
"reminders": bool(prefs.get("reminders", True)),
"assignments": bool(prefs.get("assignments", True))}
def set_user_prefs(user_id: int, prefs: dict, conn=None) -> dict:
@@ -145,3 +147,67 @@ def set_user_prefs(user_id: int, prefs: dict, conn=None) -> dict:
)
conn.commit()
return prefs
def _person_ids(value) -> list[int]:
"""Extract user ids from a stored ``person`` property value."""
ids: list[int] = []
if isinstance(value, dict):
value = [value]
if isinstance(value, list):
for person in value:
if isinstance(person, dict) and person.get("id"):
try:
ids.append(int(person["id"]))
except (TypeError, ValueError):
pass
return ids
def notify_assignment(collection_id: int, page_id: int, page_title: str,
old_props: dict, new_props: dict, actor_id: int | None,
conn=None) -> list[int]:
"""Notify users newly assigned via a ``person`` property.
Compares old vs new property values keyed by property id; users present
in the new value but not the old one get an in-app + (opt-in) email
notification. Returns the notified user ids.
"""
def _run(c):
props = c.execute(
"SELECT id, name FROM collection_properties "
"WHERE collection_id=? AND prop_type='person'",
(collection_id,),
).fetchall()
newly: list[int] = []
for p in props:
key = str(p["id"])
before = set(_person_ids((old_props or {}).get(key)))
after = _person_ids((new_props or {}).get(key))
for uid in after:
if uid not in before and uid != actor_id and uid not in newly:
newly.append(uid)
for uid in newly:
create_notification(
uid, actor_id, "assignment",
title="Assigned to you",
message=page_title or "Untitled",
resource_type="db_page",
resource_id=page_id,
url=f"/collections/{collection_id}",
conn=c, commit=False,
)
mailer.notify_user(
uid, subject="[FlowDeck] Assigned to you",
body_text=page_title or "Untitled",
cta_url=f"/collections/{collection_id}",
prefs_key="assignments",
)
return newly
if conn is not None:
return _run(conn)
with get_conn() as c:
result = _run(c)
c.commit()
return result
+106 -58
View File
@@ -7,39 +7,44 @@ slow/unreachable hosts.
"""
from __future__ import annotations
import html as htmlmod
import logging
import re
from urllib.parse import urljoin, urlparse
logger = logging.getLogger(__name__)
_META_RE = re.compile(
r'<meta\b[^>]*?(?:content=[\'"]([^\'"]*)[\'"]|property=[\'"]([^\'"]*)[\'"]|name=[\'"]([^\'"]*)[\'"]|content=[\'"]([^\'"]*)[\'"])[^>]*?>',
re.I,
)
_META_TAG_RE = re.compile(r"<meta\b[^>]*?>", re.I)
_ATTR_RE = re.compile(r"([A-Za-z_:][-A-Za-z0-9_:.]*)\s*=\s*[\"']([^\"']*)[\"']")
_TITLE_RE = re.compile(r"<title[^>]*>(.*?)</title>", re.I | re.S)
_FAVICON_RE = re.compile(r"<link\b[^>]*?>", re.I)
_ICON_REL = re.compile(r"\b(?:shortcut\s+)?icon\b", re.I)
# Property/name keys we look for, in priority order, mapped to our payload keys.
_OG_TITLE = ("og:title", "twitter:title", "title", "og:site_name")
_OG_DESC = ("og:description", "twitter:description", "description")
_OG_IMG = ("og:image", "twitter:image", "twitter:image:src")
_OG_IMG = ("og:image", "twitter:image", "twitter:image:src", "image")
_OG_SITE = ("og:site_name", "twitter:site", "application-name")
def _extract_og(html: str) -> dict:
text = html[:400_000] # only scan the beginning — that's where <head> lives
def _attrs(tag: str) -> dict:
return {k.lower(): v for k, v in _ATTR_RE.findall(tag)}
def _extract_og(body: str) -> dict:
"""Parse all ``<meta>`` tags into a ``{key: content}`` dict.
Attributes may appear in any order (``content`` before or after
``property``/``name``), which the previous implementation mishandled.
First value wins so the most specific tag (top of document) is kept.
"""
props: dict[str, str] = {}
for m in _META_RE.finditer(text):
m.groups()
content = ""
prop = ""
for s in re.findall(r"(?:content|property|name)=[\"']([^\"']*)[\"']", m.group(0)):
if prop == "":
prop = s
else:
content = s
break
if prop:
props[prop.lower()] = content
for tag in _META_TAG_RE.finditer(body[:400_000]):
attrs = _attrs(tag.group(0))
key = (attrs.get("property") or attrs.get("name") or attrs.get("itemprop") or "").lower()
content = attrs.get("content")
if key and content is not None and key not in props:
props[key] = content
return props
@@ -51,12 +56,12 @@ def _pick(props: dict, keys: tuple) -> str:
return ""
def _title_of(props: dict, raw_title: str) -> str:
def _title_of(props: dict, body: str) -> str:
t = _pick(props, _OG_TITLE)
if t:
return t
m = re.search(r"<title[^>]*>(.*?)</title>", raw_title[:200_000], re.I | re.S)
return (m.group(1).strip() if m else "") or urlparse(props.get("_url", "")).netloc
m = _TITLE_RE.search(body[:200_000])
return m.group(1).strip() if m else ""
def _site_name(url: str) -> str:
@@ -64,22 +69,94 @@ def _site_name(url: str) -> str:
return host.split(".")[0].capitalize() if host else ""
async def fetch_og_metadata(url: str, timeout: float = 6.0) -> dict:
def _favicon(body: str, base_url: str) -> str:
for tag in _FAVICON_RE.finditer(body):
attrs = _attrs(tag.group(0))
rel = attrs.get("rel", "")
href = attrs.get("href", "")
if href and _ICON_REL.search(rel):
return urljoin(base_url, htmlmod.unescape(href))
return ""
def parse_og(body: str, url: str) -> dict:
"""Pure HTML → bookmark payload (no network). ``url`` is the base URL."""
src = url.strip()
if not src.startswith(("http://", "https://")):
src = "https://" + src
props = _extract_og(body)
title = htmlmod.unescape(_title_of(props, body))
desc = htmlmod.unescape(_pick(props, _OG_DESC))
img = _pick(props, _OG_IMG)
site = htmlmod.unescape(_pick(props, _OG_SITE)) or _site_name(src)
def abs_url(u: str) -> str:
return urljoin(src, htmlmod.unescape(u)) if u else ""
return {
"url": src,
"title": title.strip()[:200] or urlparse(src).netloc or src,
"description": desc.strip()[:400],
"image": abs_url(img),
"site_name": site.strip()[:100],
"favicon": _favicon(body, src),
}
_MAX_REDIRECTS = 5
async def _get_checked(client, url: str, headers: dict):
"""GET avec re-vérification de l'hôte à CHAQUE saut de redirection (A12 SSRF).
`follow_redirects=True` laisserait une URL publique rediriger vers
169.254.169.254 / localhost — la garde doit donc tourner à chaque hop.
"""
from app.services.importers.url_fetch import _is_public_host
current = url
for _ in range(_MAX_REDIRECTS + 1):
parsed = urlparse(current)
if parsed.scheme not in ("http", "https") or not parsed.hostname or not _is_public_host(parsed.hostname):
raise ValueError(f"hôte non autorisé: {parsed.hostname!r}")
r = await client.get(current, headers=headers, follow_redirects=False)
if r.status_code in (301, 302, 303, 307, 308):
loc = r.headers.get("location")
if not loc:
return r
current = urljoin(current, loc)
continue
r.raise_for_status()
return r
raise ValueError("trop de redirections")
async def fetch_og_metadata(url: str, timeout: float = 6.0, transport=None) -> dict:
"""Fetch ``url`` and return {url, title, description, image, site_name,
favicon}. Empty strings are omitted. Never raises for network errors."""
favicon}. Empty strings are omitted. Never raises for network errors.
``transport`` is an optional ``httpx`` transport (used by tests to mock
HTTP without hitting the network).
"""
src = url.strip()
if not src.startswith(("http://", "https://")):
src = "https://" + src
base = {"url": src, "title": "", "description": "", "image": "", "site_name": "", "favicon": ""}
try:
import httpx
headers = {
"User-Agent": "FlowDeck/5.10 bookmark-fetcher (+https://flowdeck.dracodev.net)",
"User-Agent": "FlowDeck/5.5 bookmark-fetcher (+https://flowdeck.dracodev.net)",
"Accept": "text/html,application/xhtml+xml",
}
async with httpx.AsyncClient(follow_redirects=True, timeout=timeout) as client:
resp = await client.get(src, headers=headers)
resp.raise_for_status()
kwargs = {"timeout": timeout}
if transport is not None:
kwargs["transport"] = transport
async with httpx.AsyncClient(**kwargs) as client:
resp = await _get_checked(client, src, headers)
except ValueError:
# A12 : hôte privé/loopback ou trop de redirections → refus explicite.
raise
except Exception as exc: # noqa: BLE001 - network/parse failures are non-fatal
logger.debug("og fetch failed for %s: %s", src, exc)
base["title"] = urlparse(src).netloc or src
@@ -92,33 +169,4 @@ async def fetch_og_metadata(url: str, timeout: float = 6.0) -> dict:
base["site_name"] = _site_name(src)
return base
import html as htmlmod
body = resp.text
props = _extract_og(body)
props["_url"] = src
title = htmlmod.unescape(_title_of(props, body))
desc = htmlmod.unescape(_pick(props, _OG_DESC))
img = _pick(props, _OG_IMG)
site = htmlmod.unescape(_pick(props, _OG_SITE)) or _site_name(src)
def abs_url(u: str) -> str:
if not u:
return ""
return urljoin(src, htmlmod.unescape(u))
favicon = ""
fm = re.search(r"<link[^>]+rel=[\"'](?:shortcut )?icon[\"'][^>]+href=[\"']([^\"']+)[\"']", body, re.I)
if not fm:
fm = re.search(r"<link[^>]+href=[\"']([^\"']+)[\"'][^>]+rel=[\"'](?:shortcut )?icon[\"']", body, re.I)
if fm:
favicon = abs_url(fm.group(1))
return {
"url": src,
"title": title.strip()[:200] or urlparse(src).netloc or src,
"description": desc.strip()[:400],
"image": abs_url(img),
"site_name": site.strip()[:100],
"favicon": favicon,
}
return parse_og(resp.text, src)
+424 -6
View File
@@ -1,12 +1,27 @@
"""FlowDeck — Agent permission guard (v4.10.0).
"""FlowDeck — Permission manager: workspace roles + granular ACL (v6.0.0).
The agent always acts with *at most* the permissions of the invoking user
(Notion Agent principle). This manager resolves the user's role in the active
workspace and gates tool execution before any write reaches the database.
Two layers:
1. **Workspace roles** (v4.10.0, agent guard): every user has a single role in
each workspace (owner > owner-membership > editor > commenter > viewer).
The FlowDeck Agent always acts with *at most* the permissions of the
invoking user (Notion Agent principle).
2. **Granular permissions** (v6.0.0): explicit page / collection / property
grants plus reusable user groups. Resolution follows the least-privilege
rule — an explicit grant on a resource overrides the inherited chain
(page → collection → workspace), while ``restricted`` / ``private``
resources deny access unless a grant (or the workspace owner / admin)
applies.
Resolution results are cached for 60 s to keep the hot paths (sidebar, view
rendering, route guards) < 10 ms per check; ``PermissionManager.invalidate()``
drops the cache after any grant/revoke/type change.
"""
from __future__ import annotations
import logging
import time
from fastapi import HTTPException
@@ -19,6 +34,12 @@ READ_ROLES = {"viewer", "commenter", "editor", "admin", "owner"}
WRITE_ROLES = {"editor", "admin", "owner"}
DESTRUCTIVE_ROLES = {"admin", "owner"}
# Granular resource roles (ranked, least → most privileged).
_GRANULAR_ROLES = ("viewer", "commenter", "editor", "owner")
_ROLE_RANK = {role: i for i, role in enumerate(_GRANULAR_ROLES)}
_PROPERTY_ROLES = ("viewer", "editor")
_PROPERTY_RANK = {"viewer": 0, "editor": 1}
# Tools that mutate state and therefore require at least an editor role.
WRITE_TOOLS = {
"create_collection", "create_view", "create_page", "update_page",
@@ -35,10 +56,27 @@ DESTRUCTIVE_TOOLS = {
class PermissionManager:
"""Resolves workspace role and gates agent tool calls."""
"""Resolves workspace role and gates agent + granular ACL checks."""
def __init__(self, user_id: int):
def __init__(self, user_id: int, is_admin: bool = False):
self.user_id = user_id
self._is_admin_override = bool(is_admin)
self._cache: dict[str, tuple[float, object]] = {}
# ── Cache helpers ──
def _cached(self, key: str, ttl: float, fn):
now = time.monotonic()
hit = self._cache.get(key)
if hit and now - hit[0] < ttl:
return hit[1]
val = fn()
self._cache[key] = (now, val)
return val
def invalidate(self) -> None:
"""Drop the resolution cache after a grant/revoke/type change."""
self._cache.clear()
# ── Role resolution ──
@@ -60,6 +98,74 @@ class PermissionManager:
).fetchone()
return "owner" if owner else "viewer"
# ── SSO (v6.7.0, design §7.2) ─────────────────────────────────────────
def is_sso_only_workspace(self, workspace_id: int | None = None) -> bool:
"""True when that workspace can only be reached through SSO.
FlowDeck keeps a single instance-wide SSO-only switch (design §7.1 /
§4.2): when it is on, local login is refused for every non-admin, so
every workspace on the instance is effectively SSO-only.
``workspace_id`` is accepted to mirror the design's per-workspace API.
"""
from app.services.sso_provisioning import is_sso_only
return is_sso_only()
def _user_auth_method(self) -> str:
with get_conn() as conn:
row = conn.execute(
"SELECT auth_method FROM users WHERE id=?", (self.user_id,)
).fetchone()
return (row["auth_method"] or "local") if row else "local"
def get_sso_roles(
self, user_id: int | None = None, workspace_id: int | None = None
) -> list[str]:
"""Roles granted to that user through SSO group mapping (design §7.2).
SSO grants land in the regular ``workspace_members`` row (the mapping
is re-applied at every SSO login), so the answer is the explicit
membership role of a non-local account — local accounts and users
without an explicit grant (the implicit *viewer* fallback is not an
SSO grant) get ``[]``.
"""
if workspace_id is None:
return []
pm = PermissionManager(int(user_id)) if (user_id and int(user_id) != self.user_id) else self
if pm._user_auth_method() == "local":
return []
with get_conn() as conn:
row = conn.execute(
"SELECT role FROM workspace_members WHERE workspace_id=? AND user_id=?",
(int(workspace_id), pm.user_id),
).fetchone()
return [row["role"]] if row else []
def sync_sso_permissions(
self,
user_id: int | None,
sso_groups: list[str],
workspace_id: int | None = None,
) -> list[int]:
"""Re-apply the group → workspace role mapping (design §7.2).
Delegates to ``sso_provisioning.sync_sso_groups`` (the single source
of truth used at login and by ``POST /api/v2/sso/sync``). Returns the
touched workspace ids, narrowed to ``workspace_id`` when given.
"""
from app.services.sso_provisioning import get_sso_config, sync_sso_groups
cfg = get_sso_config()
if not cfg:
return []
touched = sync_sso_groups(int(user_id or self.user_id), list(sso_groups or []), cfg)
if workspace_id is not None:
touched = [w for w in touched if int(w) == int(workspace_id)]
if touched:
self.invalidate()
return touched
def can_read(self, workspace_id: int | None) -> bool:
return self.role_in_workspace(workspace_id) in READ_ROLES
@@ -100,3 +206,315 @@ class PermissionManager:
# A viewer can always read; editor can read+write.
if role not in READ_ROLES:
raise HTTPException(status_code=403, detail="User has no access to this workspace")
# ═══════════════════════════════════════════════════════════════════════
# Granular permissions (v6.0.0)
# ═══════════════════════════════════════════════════════════════════════
def _is_admin(self, conn) -> bool:
if self._is_admin_override:
return True
row = conn.execute(
"SELECT is_admin FROM users WHERE id=?", (self.user_id,)
).fetchone()
return bool(row and row["is_admin"])
def _owns_workspace(self, conn, workspace_id: int | None) -> bool:
if workspace_id is None:
# No workspace → single-user semantics: the actor is the owner.
return True
row = conn.execute(
"SELECT id FROM workspaces WHERE id=? AND owner_id=?",
(workspace_id, self.user_id),
).fetchone()
return bool(row)
def user_group_ids(self, conn) -> list[int]:
return [
r["group_id"]
for r in conn.execute(
"SELECT group_id FROM group_members WHERE user_id=?", (self.user_id,)
).fetchall()
]
def _explicit_grant_role(self, conn, table: str, fk: str, resource_id: int,
role_rank: dict[str, int] | None = None) -> str | None:
"""Most-privileged explicit role on ``table`` for the user / groups."""
rank = role_rank or _ROLE_RANK
groups = self.user_group_ids(conn)
if groups:
placeholders = ", ".join("?" * len(groups))
rows = conn.execute(
f"SELECT role FROM {table} WHERE {fk}=? "
f"AND (user_id=? OR group_id IN ({placeholders}))",
(resource_id, self.user_id, *groups),
).fetchall()
else:
rows = conn.execute(
f"SELECT role FROM {table} WHERE {fk}=? AND user_id=?",
(resource_id, self.user_id),
).fetchall()
best = max((rank.get(r["role"], -1) for r in rows), default=-1)
if best < 0:
return None
rev = {rank[k]: k for k in rank}
return rev[best]
# ── Page-level ──
def get_page_permission(self, page_id: int) -> str | None:
"""Effective page role for ``self.user_id`` (least privilege).
Chain: explicit page grant > explicit collection grant > workspace
role. ``restricted`` / ``private`` pages ignore the inherited chain.
Returns ``None`` when the user must not see the page at all.
"""
def _resolve() -> str | None:
with get_conn() as conn:
page = conn.execute(
"SELECT permission_type, workspace_id, collection_id FROM pages WHERE id=?",
(page_id,),
).fetchone()
if not page:
return None
if self._is_admin(conn) or self._owns_workspace(conn, page["workspace_id"]):
return "owner"
explicit = self._explicit_grant_role(
conn, "page_permissions", "page_id", page_id
)
if explicit:
return explicit
ptype = page["permission_type"] or "inherit"
if ptype in ("restricted", "private"):
return None
if page["collection_id"]:
coll_role = self._collection_role(conn, page["collection_id"])
if coll_role:
return coll_role
return self.role_in_workspace(page["workspace_id"])
return self._cached(f"page:{page_id}", 60, _resolve)
def can_view_page(self, page_id: int) -> bool:
return self.get_page_permission(page_id) is not None
def can_edit_page(self, page_id: int) -> bool:
role = self.get_page_permission(page_id)
return bool(role and _ROLE_RANK[role] >= _ROLE_RANK["editor"])
def can_comment_page(self, page_id: int) -> bool:
role = self.get_page_permission(page_id)
return bool(role and _ROLE_RANK[role] >= _ROLE_RANK["commenter"])
def can_manage_page_permissions(self, page_id: int) -> bool:
role = self.get_page_permission(page_id)
return bool(role and _ROLE_RANK[role] >= _ROLE_RANK["owner"])
# ── Collection-level ──
def _collection_role(self, conn, collection_id: int) -> str | None:
coll = conn.execute(
"SELECT permission_type, workspace_id FROM collections WHERE id=?",
(collection_id,),
).fetchone()
if not coll:
return None
if self._is_admin(conn) or self._owns_workspace(conn, coll["workspace_id"]):
return "owner"
explicit = self._explicit_grant_role(
conn, "collection_permissions", "collection_id", collection_id
)
if explicit:
return explicit
ptype = coll["permission_type"] or "inherit"
if ptype in ("restricted", "private"):
return None
return self.role_in_workspace(coll["workspace_id"])
def get_collection_permission(self, collection_id: int) -> str | None:
def _resolve() -> str | None:
with get_conn() as conn:
return self._collection_role(conn, collection_id)
return self._cached(f"collection:{collection_id}", 60, _resolve)
def can_view_collection(self, collection_id: int) -> bool:
return self.get_collection_permission(collection_id) is not None
def can_edit_collection(self, collection_id: int) -> bool:
role = self.get_collection_permission(collection_id)
return bool(role and _ROLE_RANK[role] >= _ROLE_RANK["editor"])
def can_manage_collection_permissions(self, collection_id: int) -> bool:
role = self.get_collection_permission(collection_id)
return bool(role and _ROLE_RANK[role] >= _ROLE_RANK["owner"])
# ── Property-level ──
def _property_grants_exist(self, conn, property_id: int) -> bool:
row = conn.execute(
"SELECT 1 FROM property_permissions WHERE property_id=? LIMIT 1",
(property_id,),
).fetchone()
return row is not None
def _has_property_grant(self, conn, property_id: int, min_rank: int) -> bool:
groups = self.user_group_ids(conn)
if groups:
placeholders = ", ".join("?" * len(groups))
rows = conn.execute(
f"SELECT role FROM property_permissions WHERE property_id=? "
f"AND (user_id=? OR group_id IN ({placeholders}))",
(property_id, self.user_id, *groups),
).fetchall()
else:
rows = conn.execute(
"SELECT role FROM property_permissions WHERE property_id=? AND user_id=?",
(property_id, self.user_id),
).fetchall()
return any(_PROPERTY_RANK.get(r["role"], -1) >= min_rank for r in rows)
def can_view_property(self, collection_id: int, property_id: int) -> bool:
"""A property is visible unless it carries explicit grants excluding
the user; without any grant it inherits from the collection. Collection
owners/admins always see every property."""
if not self.can_view_collection(collection_id):
return False
return self._cached(
f"prop:{property_id}", 60, lambda: self._property_visible(collection_id, property_id)
)
def _collection_workspace_id(self, conn, collection_id: int) -> int | None:
row = conn.execute(
"SELECT workspace_id FROM collections WHERE id=?", (collection_id,)
).fetchone()
return row["workspace_id"] if row else None
def _property_visible(self, collection_id: int, property_id: int) -> bool:
with get_conn() as conn:
workspace_id = self._collection_workspace_id(conn, collection_id)
if self._is_admin(conn) or self._owns_workspace(conn, workspace_id):
return True
if self.can_manage_collection_permissions(collection_id):
return True
if not self._property_grants_exist(conn, property_id):
return True
return self._has_property_grant(conn, property_id, _PROPERTY_RANK["viewer"])
def can_edit_property(self, collection_id: int, property_id: int) -> bool:
if not self.can_edit_collection(collection_id):
return False
with get_conn() as conn:
workspace_id = self._collection_workspace_id(conn, collection_id)
if self._is_admin(conn) or self._owns_workspace(conn, workspace_id):
return True
if self.can_manage_collection_permissions(collection_id):
return True
if not self._property_grants_exist(conn, property_id):
return True
return self._has_property_grant(conn, property_id, _PROPERTY_RANK["editor"])
def get_visible_properties(self, collection_id: int) -> list[int]:
def _resolve() -> list[int]:
with get_conn() as conn:
props = conn.execute(
"SELECT id FROM collection_properties WHERE collection_id=?",
(collection_id,),
).fetchall()
return [p["id"] for p in props if self.can_view_property(collection_id, p["id"])]
return self._cached(f"visible_props:{collection_id}", 60, _resolve)
# ── Groups ──
def is_workspace_admin(self, workspace_id: int | None) -> bool:
with get_conn() as conn:
return self._is_admin(conn) or self._owns_workspace(conn, workspace_id)
def create_group(self, workspace_id: int | None, name: str,
description: str = "", created_by: int | None = None) -> int:
if not self.is_workspace_admin(workspace_id):
raise HTTPException(403, "Only a workspace owner or admin can create groups")
if not name.strip():
raise HTTPException(400, "name is required")
with get_conn() as conn:
dupe = conn.execute(
"SELECT id FROM user_groups WHERE workspace_id IS ? AND name=?",
(workspace_id, name.strip()),
).fetchone()
if dupe:
raise HTTPException(400, "A group with this name already exists")
cur = conn.execute(
"INSERT INTO user_groups (workspace_id, name, description, created_by) "
"VALUES (?, ?, ?, ?)",
(workspace_id, name.strip(), description or "", created_by),
)
conn.commit()
return cur.lastrowid
def add_user_to_group(self, group_id: int, user_id: int) -> None:
with get_conn() as conn:
group = conn.execute(
"SELECT workspace_id FROM user_groups WHERE id=?", (group_id,)
).fetchone()
if not group:
raise HTTPException(404, "Group not found")
conn.execute(
"INSERT OR IGNORE INTO group_members (group_id, user_id) VALUES (?, ?)",
(group_id, user_id),
)
conn.commit()
def remove_user_from_group(self, group_id: int, user_id: int) -> None:
with get_conn() as conn:
conn.execute(
"DELETE FROM group_members WHERE group_id=? AND user_id=?",
(group_id, user_id),
)
conn.commit()
def delete_group(self, group_id: int) -> None:
with get_conn() as conn:
conn.execute("DELETE FROM user_groups WHERE id=?", (group_id,))
conn.commit()
def get_groups_for_workspace(self, workspace_id: int | None) -> list[dict]:
with get_conn() as conn:
rows = conn.execute(
"""SELECT g.id, g.name, g.description, g.created_by, g.created_at,
(SELECT COUNT(*) FROM group_members m WHERE m.group_id=g.id) AS member_count
FROM user_groups g WHERE g.workspace_id IS ? ORDER BY g.name""",
(workspace_id,),
).fetchall()
return [dict(r) for r in rows]
def get_group_members(self, group_id: int) -> list[dict]:
with get_conn() as conn:
rows = conn.execute(
"""SELECT u.id, u.login, u.full_name, u.email, m.joined_at
FROM group_members m JOIN users u ON u.id=m.user_id
WHERE m.group_id=? ORDER BY u.login""",
(group_id,),
).fetchall()
return [dict(r) for r in rows]
# ── Audit log ──
def log_permission_change(self, resource_type: str, resource_id: int, action: str,
target_user_id: int | None = None,
target_group_id: int | None = None,
old_role: str | None = None,
new_role: str | None = None,
ip_address: str = "") -> None:
"""Write one immutable audit row for a permission change."""
try:
with get_conn() as conn:
conn.execute(
"""INSERT INTO permission_audit_log
(resource_type, resource_id, action, target_user_id, target_group_id,
old_role, new_role, performed_by, ip_address)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)""",
(resource_type, resource_id, action, target_user_id, target_group_id,
old_role, new_role, self.user_id, ip_address),
)
conn.commit()
except Exception as exc: # audit must never break the caller
logger.warning("permission audit log failed: %s", exc)
+56 -3
View File
@@ -96,6 +96,10 @@ PROPERTY_TYPES: dict[str, dict] = {
"storage": "auto — {id, login}",
"default": None,
},
"button": {
"storage": "none — runs linked automation (button_automation_id)",
"default": None,
},
}
# CSV-friendly subset (no relation/rollup/formula)
@@ -136,6 +140,9 @@ def validate_property_value(prop_type: str, value: Any, options: list | None = N
datetime.fromisoformat(value.replace("Z", "+00:00"))
except (ValueError, TypeError):
return False, f"'{value}' is not a valid ISO 8601 date"
elif prop_type == "person":
if not isinstance(value, list):
return False, "Person must be a list of workspace members"
elif prop_type == "url":
if not isinstance(value, str):
return False, "URL must be a string"
@@ -227,17 +234,63 @@ def validate_property_rule(
return True, ""
def user_ref(user: dict | None) -> dict | None:
"""Normalize a session user dict into the stored ``person`` value shape."""
if not user:
return None
return {
"id": user.get("id"),
"login": user.get("login") or user.get("full_name") or "",
"full_name": user.get("full_name") or "",
"avatar_url": user.get("avatar_url") or "",
"avatar_color": user.get("avatar_color") or "#3A3A3A",
}
def get_auto_property_value(prop_type: str, user: dict | None = None) -> Any:
"""Compute the value of an auto-property."""
if prop_type == "created_time" or prop_type == "last_edited_time":
return datetime.now(UTC).isoformat()
if prop_type == "created_by" or prop_type == "last_edited_by":
if user:
return {"id": user.get("id"), "login": user.get("login")}
return None
return user_ref(user)
return None
def apply_auto_properties(
properties: list[dict],
values: dict,
user: dict | None = None,
*,
is_create: bool = False,
now: str | None = None,
) -> dict:
"""Fill/refresh auto-property values (``created_time``, ``created_by``,
``last_edited_time``, ``last_edited_by``) in ``values`` (keyed by property id).
``created_*`` are only written on creation (or when missing); ``last_edited_*``
are refreshed on every call. Returns the mutated dict.
"""
if values is None:
values = {}
stamp = now or datetime.now(UTC).isoformat()
for prop in properties or []:
ptype = prop.get("prop_type")
if ptype not in AUTO_TYPES:
continue
pid = str(prop.get("id"))
if ptype == "created_time":
if is_create or pid not in values or values.get(pid) in (None, ""):
values[pid] = stamp
elif ptype == "last_edited_time":
values[pid] = stamp
elif ptype == "created_by":
if is_create or pid not in values or values.get(pid) in (None, ""):
values[pid] = user_ref(user)
elif ptype == "last_edited_by":
values[pid] = user_ref(user)
return values
def get_next_unique_id(collection_id: int, conn) -> int:
"""Get the next unique_id for a collection (max + 1)."""
row = conn.execute(
+164
View File
@@ -0,0 +1,164 @@
"""FlowDeck — v6.4.0 Realtime: résolution de conflits au-delà du last-write-wins.
Le LWW par bloc (v5.13.0) écrase intégralement le bloc du dernier arrivé : si deux
utilisateurs tapent dans le *même* bloc, la saisie du premier est perdue. Ce module
implémente un vrai *merge à trois versions* (diff3-lite) :
base = l'état du bloc dont le client dérive sa saisie (envoyé avec l'op)
current = l'état actuel du bloc côté serveur (déjà mis à jour par d'autres)
incoming = la nouvelle proposition du client
Règle champ-par-champ :
* incoming == base → le client n'a pas touché ce champ → on garde current
* current == base → le serveur n'a pas touché ce champ → on garde incoming
* current == incoming → les deux ont fait la même chose → sans conflit
* sinon (conflit)
- champ texte (str) : merge de caractères. Les régions modifiées qui ne se
chevauchent pas sont *toutes conservées* (les deux saisies survivent) ;
chevauchement réel → LWW sur ce champ + drapeau de conflit.
- autre type (bool, nombre…) : LWW sur ce champ + drapeau de conflit.
Toutes les fonctions sont pures et testables sans WebSocket ni base de données.
"""
from __future__ import annotations
from typing import Any
__all__ = ["merge_text_3way", "merge_block_3way", "changed_region"]
def changed_region(base: str, other: str) -> tuple[int, int, str] | None:
"""Région de `base` remplacée par `other` (trim préfixe/suffixe commun).
Retourne ``(start, end, replacement)`` tel que ``base[:start] + replacement +
base[end:] == other``, ou ``None`` si ``other == base`` (aucun changement).
"""
if base == other:
return None
minlen = min(len(base), len(other))
prefix = 0
while prefix < minlen and base[prefix] == other[prefix]:
prefix += 1
suffix = 0
# ne jamais chevaucher le préfixe déjà consommé
while (suffix < len(base) - prefix and suffix < len(other) - prefix
and base[len(base) - 1 - suffix] == other[len(other) - 1 - suffix]):
suffix += 1
return prefix, len(base) - suffix, other[prefix:len(other) - suffix]
def merge_text_3way(base: str, current: str, incoming: str) -> tuple[str, bool]:
"""Merge à trois versions d'une chaîne. Retourne ``(texte, conflit)``.
Les éditions qui ne se chevauchent pas sont toutes les deux conservées ;
un chevauchement réel retombe en LWW (``incoming`` gagne) et signale le conflit.
"""
if current == incoming:
return current, False
if current == base:
return incoming, False
if incoming == base:
return current, False
rc = changed_region(base, current)
ri = changed_region(base, incoming)
if rc is None:
return incoming, False
if ri is None:
return current, False
c_start, c_end, c_text = rc
i_start, i_end, i_text = ri
# Régions disjointes (ou juste adjacentes) → appliquer les deux sur base.
if c_end <= i_start or i_end <= c_start:
edits = sorted([(c_start, c_end, c_text), (i_start, i_end, i_text)],
key=lambda e: e[0])
out: list[str] = []
pos = 0
for start, end, text in edits:
if start < pos:
continue # sécurité: ne jamais réappliquer par-dessus
out.append(base[pos:start])
out.append(text)
pos = end
out.append(base[pos:])
return "".join(out), False
# Chevauchement réel → LWW sur ce champ, conflit signalé.
return incoming, True
def _scalar_conflict(base: Any, current: Any, incoming: Any) -> tuple[Any, bool]:
"""Conflit sur un champ non-texte : LWW (incoming gagne)."""
if current == incoming:
return current, False
if current == base:
return incoming, False
if incoming == base:
return current, False
return incoming, True
def merge_block_3way(base_blk: Any, current_blk: Any,
incoming_blk: Any) -> tuple[dict, list[str]]:
"""Merge à trois versions d'un bloc entier.
Retourne ``(bloc fusionné, champs en conflit)``. Ne lève jamais d'exception :
une entrée non-dict retombe en LWW (``incoming``) avec conflit signalé sur
``__block__`` pour que l'appelant puisse journaliser.
"""
if not isinstance(base_blk, dict):
base_blk = {}
if not isinstance(current_blk, dict):
current_blk = {}
if not isinstance(incoming_blk, dict):
# proposition invalide → on garde l'état serveur
return dict(current_blk), ["__block__"]
keys = set(base_blk) | set(current_blk) | set(incoming_blk)
merged: dict[str, Any] = {}
conflicts: list[str] = []
for key in keys:
b = base_blk.get(key)
c = current_blk.get(key)
i = incoming_blk.get(key)
if i == b:
# le client n'a pas modifié ce champ → valeur serveur. Si le serveur
# a *supprimé* le champ (absent de current) alors la suppression doit
# gagner : on n'insère pas de clé fantôme value=None.
if key not in current_blk:
continue
merged[key] = c
elif c == b:
# le serveur n'a pas modifié ce champ → valeur client
merged[key] = i
elif c == i:
merged[key] = c
else:
# les deux ont changé, différemment
if isinstance(b, str) and isinstance(c, str) and isinstance(i, str):
text, conflict = merge_text_3way(b, c, i)
merged[key] = text
if conflict:
conflicts.append(key)
else:
value, conflict = _scalar_conflict(b, c, i)
merged[key] = value
if conflict:
conflicts.append(key)
# Un champ supprimé par le serveur et absent de la proposition client doit
# rester supprimé (pas de réapparition d'une valeur None fantôme).
merged = {k: v for k, v in merged.items()
if not (v is None and k not in incoming_blk)}
if "id" not in merged:
# garantir l'identité du bloc même si base était vide
ident = incoming_blk.get("id") or current_blk.get("id")
if ident:
merged["id"] = ident
return merged, conflicts
+309 -73
View File
@@ -1,5 +1,23 @@
"""FlowDeck — v5.13.0 Realtime: WebSocket gateway, présences, curseurs live,
merge des opérations de blocs (last-write-wins par bloc) + version de page.
"""FlowDeck — v6.4.0 Realtime: WebSocket gateway, présences, curseurs live,
merge à trois versions des opérations de blocs (au-delà du last-write-wins) +
version de page.
Améliorations « production » par rapport à v5.13.0 :
* **Conflits** — les mises à jour de bloc embarquent la ``base`` dont dérive la
saisie du client ; le serveur fait un merge 3-voix champ-par-champ + texte
(voir ``realtime_merge``) au lieu d'écraser le bloc entier. Les deux saisies
disjointes survivent, les chevauchements réels retombent en LWW *par champ*
avec drapeau de conflit renvoyé au client.
* **Échelle** — chaque connexion possède une file sortante + une tâche writer
dédiée ; le broadcast devient non bloquant (un client lent ne bloque plus la
room), les mises à jour de curseur se coalescent (une seule par flush), et un
client trop lent (file pleine) est déconnecté proprement (4413).
* **Anti-flood** — budget d'opérations par connexion (fenêtre glissante).
* **Fuites corrigées** — une room 4404 n'est plus enregistrée ; ``room_state``
ne crée plus d'objet None ; les rooms vides sont évacuées.
* **Observabilité** — compteurs (rooms, conns, ops, merges, conflits, déconnexions
lentes) exposés par ``stats()``.
Rooms in-memory (un seul worker uvicorn). Persistance en base (page.content)
avec debounce. Fallback polling côté client si le WS est indisponible.
@@ -9,34 +27,66 @@ from __future__ import annotations
import asyncio
import json
import logging
import time
from fastapi import WebSocket
from app.db import get_conn
from app.services.realtime_merge import merge_block_3way
logger = logging.getLogger(__name__)
COLORS = ["#2383E2", "#46A758", "#E5484D", "#F76B15", "#8E4EC6", "#12A594",
"#FFC53D", "#D6409F", "#0091FF", "#3E63DD", "#30A46C", "#FF3333"]
# File sortante maximale par connexion au-delà de laquelle le client est
# considéré comme trop lent et déconnecté (évite qu'une room entière stagne).
MAX_OUT_QUEUE = 512
# Nombre maximal d'opérations acceptées par connexion et par fenêtre (anti-flood).
OP_WINDOW_SECONDS = 10.0
OP_WINDOW_MAX = 400
def color_for(uid: int) -> str:
return COLORS[(uid or 0) % len(COLORS)]
def block_id() -> str:
import time
return f"b{int(time.time()*1000)}"
import uuid
return "b" + uuid.uuid4().hex[:12]
def ensure_block_ids(blocks: list[dict]) -> list[dict]:
"""Assign unique ids to blocks missing one, recursively.
Template-created pages may have been persisted without block ids; without
them the room state is not addressable by ops and the editor ends up with
``data-bid="undefined"`` blocks (duplicated / reordered lines).
"""
if not isinstance(blocks, list):
return blocks
for b in blocks:
if isinstance(b, dict):
if not b.get("id"):
b["id"] = block_id()
if isinstance(b.get("children"), list):
ensure_block_ids(b["children"])
return blocks
def apply_op(blocks: list[dict], op: dict) -> list[dict]:
"""Apply one block op (insert/update/delete/move) — LWW par bloc."""
"""Apply one block op (insert/update/delete/move) — LWW par bloc.
Conservé pour la compatibilité : tests et chemins sans ``base`` continuent
de fonctionner. Le merge 3-voix vit dans ``RealtimeManager._apply``.
"""
t = op.get("type")
if t == "insert":
blk = op.get("block") or {}
if not blk.get("id"):
blk = dict(blk)
blk["id"] = block_id()
ensure_block_ids([blk])
idx = op.get("index")
if not isinstance(idx, int):
idx = len(blocks)
@@ -71,9 +121,40 @@ def merge_ops(blocks: list[dict], ops: list[dict]) -> list[dict]:
return out
class RTConn:
"""Une connexion WS : file sortante + tâche writer dédiée.
Le broadcast ne fait que ``put_nowait`` dans la file ; c'est la tâche writer
qui consomme et écrit sur le socket. Un client lent n'empêche donc jamais
les autres membres de la room de recevoir les messages.
"""
__slots__ = ("ws", "user", "page_id", "out_q", "writer", "closed",
"_ops_count", "_ops_window_start", "created_at")
def __init__(self, ws: WebSocket, user: dict, page_id: int):
self.ws = ws
self.user = user
self.page_id = page_id
self.out_q: asyncio.Queue = asyncio.Queue(maxsize=MAX_OUT_QUEUE)
self.writer: asyncio.Task | None = None
self.closed = False
self._ops_count = 0
self._ops_window_start = time.monotonic()
self.created_at = time.monotonic()
def op_budget_ok(self) -> bool:
"""Fenêtre glissante simple anti-flood d'opérations."""
now = time.monotonic()
if now - self._ops_window_start > OP_WINDOW_SECONDS:
self._ops_window_start = now
self._ops_count = 0
self._ops_count += 1
return self._ops_count <= OP_WINDOW_MAX
class Room:
__slots__ = ("page_id", "blocks", "title", "version", "conns",
"persist_task", "dirty")
"persist_task", "dirty", "merge_count", "conflict_count")
def __init__(self, page_id: int):
self.page_id = page_id
@@ -83,21 +164,21 @@ class Room:
self.conns: set[RTConn] = set()
self.persist_task: asyncio.Task | None = None
self.dirty = False
class RTConn:
__slots__ = ("ws", "user", "page_id")
def __init__(self, ws: WebSocket, user: dict, page_id: int):
self.ws = ws
self.user = user
self.page_id = page_id
self.merge_count = 0
self.conflict_count = 0
class RealtimeManager:
def __init__(self):
self._rooms: dict[int, Room] = {}
# compteurs globaux (observabilité)
self.stat_ops = 0
self.stat_merges = 0
self.stat_conflicts = 0
self.stat_slow_disconnects = 0
self.stat_connections_total = 0
# ── rooms ────────────────────────────────────────────────────────────
def room(self, page_id: int) -> Room:
return self._rooms.setdefault(page_id, Room(page_id))
@@ -126,47 +207,95 @@ class RealtimeManager:
room.title = row["title"] or ""
if (row["content_format"] or "") == "blocks" and row["content"]:
try:
room.blocks = json.loads(row["content"])
blocks = ensure_block_ids(json.loads(row["content"]))
# v6.5.0: rooms serve server-resolved synced blocks so a
# sync_req never returns a stale cached copy.
from app.services.synced_blocks import resolve_synced_block
room.blocks = resolve_synced_block(blocks)
except Exception:
room.blocks = []
return True
# ── entrées / sorties ────────────────────────────────────────────────
async def connect(self, ws: WebSocket, page_id: int, user: dict) -> RTConn | None:
room = self.room(page_id)
# Ne JAMAIS enregistrer la room avant d'avoir validé l'existence de la
# page : avant, un 4404 laissait une Room orpheline en mémoire pour
# toujours (fuite).
existing = self._rooms.get(page_id)
room = existing if existing is not None else Room(page_id)
if not room.conns and not await self.load_room(room):
await ws.close(code=4404)
return None
return None # room non enregistrée → pas de fuite
if existing is None:
self._rooms[page_id] = room
conn = RTConn(ws, user, page_id)
self.stat_connections_total += 1
conn.writer = asyncio.create_task(self._writer(conn))
room.conns.add(conn)
me = self._peer(user)
peers = [self._peer(c.user) for c in room.conns if c is not conn]
await ws.send_json({"t": "welcome", "self": me,
"peers": peers, "color": me["color"]})
await ws.send_json({"t": "sync", "blocks": room.blocks,
"title": room.title, "version": room.version})
for c in room.conns:
if c is not conn:
try:
await c.ws.send_json({"t": "peer_join", "peer": me})
except Exception:
pass
await self._broadcast(room, {"t": "peer_join", "peer": me}, exclude=conn)
return conn
async def disconnect(self, conn: RTConn):
room = self._rooms.get(conn.page_id)
if not room:
if room is None:
self._close_writer(conn)
return
room.conns.discard(conn)
for c in room.conns:
try:
await c.ws.send_json({"t": "peer_leave",
"id": conn.user.get("id") or 0})
except Exception:
pass
self._close_writer(conn)
await self._broadcast(room, {"t": "peer_leave",
"id": conn.user.get("id") or 0})
if not room.conns:
await self.flush(room)
self._rooms.pop(conn.page_id, None)
def _close_writer(self, conn: RTConn):
conn.closed = True
t = conn.writer
if t is not None and not t.done():
t.cancel()
conn.writer = None
async def _writer(self, conn: RTConn):
"""Tâche dédiée : consomme la file sortante et écrit sur le socket.
Sert aussi de filet de coalescence : après chaque message consommé, les
mises à jour de curseur (``sel``) déjà empilées sont réduites à la
dernière (les curseurs n'ont pas besoin d'être ordonnés entre eux, seule
la position la plus récente compte).
"""
try:
while not conn.closed:
msg = await conn.out_q.get()
if msg is None:
return
await conn.ws.send_json(msg)
# coalescence des curseurs en attente
pending_sel = []
while not conn.out_q.empty():
nxt = conn.out_q.get_nowait()
if nxt is None:
return
if isinstance(nxt, dict) and nxt.get("t") == "sel":
pending_sel.append(nxt)
else:
await conn.ws.send_json(nxt)
if pending_sel:
await conn.ws.send_json(pending_sel[-1])
except asyncio.CancelledError:
raise
except Exception as e: # socket mort → on arrête proprement
logger.debug("writer stopped: %s", e)
conn.closed = True
# ── persistance ──────────────────────────────────────────────────────
async def flush(self, room: Room):
"""Write current room state to DB (sync, used on idle + disconnect)."""
if room.persist_task and not room.persist_task.done():
@@ -199,67 +328,102 @@ class RealtimeManager:
room.persist_task = asyncio.create_task(_run())
# ── broadcast non bloquant ───────────────────────────────────────────
def _enqueue(self, conn: RTConn, msg: dict) -> bool:
"""Pose le message dans la file du client. Retourne False si trop lent."""
if conn.closed:
return False
try:
conn.out_q.put_nowait(msg)
return True
except asyncio.QueueFull:
return False
async def _evict_slow(self, room: Room, conn: RTConn):
"""Client dépassé : on le déconnecte pour ne pas figer la room."""
self.stat_slow_disconnects += 1
logger.info("realtime: evicting slow client (user=%s page=%s)",
conn.user.get("login"), conn.page_id)
room.conns.discard(conn)
self._close_writer(conn)
try:
await conn.ws.close(code=4413)
except Exception:
logger.exception("_evict_slow")
async def _broadcast(self, room: Room, msg: dict, exclude: RTConn | None = None):
for c in room.conns:
"""Enfile ``msg`` chez chaque membre — jamais d'attente sur le socket."""
slow: list[RTConn] = []
for c in list(room.conns):
if c is exclude:
continue
try:
await c.ws.send_json(msg)
except Exception:
pass
if not self._enqueue(c, msg):
slow.append(c)
for c in slow:
await self._evict_slow(room, c)
async def _send(self, conn: RTConn, msg: dict):
if not self._enqueue(conn, msg):
room = self._rooms.get(conn.page_id)
if room is not None:
await self._evict_slow(room, conn)
# ── protocole ────────────────────────────────────────────────────────
async def handle(self, conn: RTConn, msg: dict):
room = self._rooms.get(conn.page_id)
if not room:
if room is None:
return
t = msg.get("t")
me = (conn.user.get("id") or 0)
if t == "hello":
try:
await conn.ws.send_json({"t": "sync", "blocks": room.blocks,
"title": room.title, "version": room.version})
except Exception:
pass
await self._send(conn, {"t": "sync", "blocks": room.blocks,
"title": room.title, "version": room.version})
return
if t == "sync_req":
try:
await conn.ws.send_json({"t": "sync", "blocks": room.blocks,
"title": room.title, "version": room.version})
except Exception:
pass
await self._send(conn, {"t": "sync", "blocks": room.blocks,
"title": room.title, "version": room.version})
return
if t == "ping":
try:
await conn.ws.send_json({"t": "pong"})
except Exception:
pass
await self._send(conn, {"t": "pong"})
return
if t == "op":
if not conn.op_budget_ok():
# trop d'ops : on ignore silencieusement (le client resync)
await self._send(conn, {"t": "ack", "v": room.version,
"stale": True})
return
op = msg.get("op") or {}
client_v = msg.get("v", 0)
room.blocks = apply_op(room.blocks, op)
result = self._apply(room, op)
room.version += 1
self.stat_ops += 1
self._schedule_persist(room)
stale = client_v < room.version - 1
await self._broadcast(room, {"t": "op", "op": op, "from": me,
"v": room.version}, exclude=conn)
try:
await conn.ws.send_json({"t": "ack", "v": room.version,
"stale": stale})
except Exception:
pass
# broadcast : on diffuse TOUJOURS le bloc final fusionné (pas la
# proposition brute) pour que tous les clients convergent.
out_op = dict(op)
if result.get("merged") is not None:
out_op["block"] = result["merged"]
out_op["merged"] = True
await self._broadcast(room, {"t": "op", "op": out_op, "from": me,
"v": room.version,
"conflict": result.get("conflict", False)},
exclude=conn)
ack = {"t": "ack", "v": room.version, "stale": stale}
if result.get("merged") is not None:
ack["merged"] = result["merged"]
ack["conflict"] = result.get("conflict", False)
await self._send(conn, ack)
if stale:
try:
await conn.ws.send_json({"t": "sync",
"blocks": room.blocks,
"title": room.title,
"version": room.version})
except Exception:
pass
await self._send(conn, {"t": "sync", "blocks": room.blocks,
"title": room.title, "version": room.version})
return
if t == "title":
@@ -270,11 +434,7 @@ class RealtimeManager:
self._schedule_persist(room)
await self._broadcast(room, {"t": "title", "title": room.title,
"from": me, "v": room.version}, exclude=conn)
try:
await conn.ws.send_json({"t": "ack", "v": room.version,
"stale": False})
except Exception:
pass
await self._send(conn, {"t": "ack", "v": room.version, "stale": False})
return
if t == "sel":
@@ -284,11 +444,87 @@ class RealtimeManager:
"offset": msg.get("offset", 0)}, exclude=conn)
return
def _apply(self, room: Room, op: dict) -> dict:
"""Applique une opération en mergeant à 3 voix si le client fournit
une ``base``. Retourne ``{"merged": bloc|None, "conflict": bool}``.
* ``update`` avec ``base`` → merge 3-voix (au-delà du LWW).
* le reste (insert/delete/move, ou update sans base) → LWW historique.
"""
if op.get("type") != "update" or "base" not in op:
room.blocks = apply_op(room.blocks, op)
return {"merged": None, "conflict": False}
incoming = op.get("block") or {}
base = op.get("base")
bid = incoming.get("id")
if not bid:
return {"merged": None, "conflict": False}
current = next((b for b in room.blocks if b.get("id") == bid), None)
if current is None:
# bloc introuvable : LWW historique = pas d'update possible
return {"merged": None, "conflict": False}
merged, conflicts = merge_block_3way(base, current, incoming)
room.merge_count += 1
conflict = bool(conflicts)
if conflict:
room.conflict_count += 1
self.stat_conflicts += 1
logger.debug("realtime conflict page=%s block=%s fields=%s",
room.page_id, bid, conflicts)
self.stat_merges += 1
room.blocks = [merged if b.get("id") == bid else b for b in room.blocks]
return {"merged": merged, "conflict": conflict}
# ── observabilité ────────────────────────────────────────────────────
async def room_state(self, page_id: int) -> dict:
room = self.room(page_id)
if not room.conns and not room.blocks:
room = self._rooms.get(page_id)
if room is None:
# ne pas créer d'objet None : on charge dans une room jetable
room = Room(page_id)
if not await self.load_room(room):
return {"blocks": [], "title": "", "version": 0}
elif not room.conns and not room.blocks:
await self.load_room(room)
return {"blocks": room.blocks, "title": room.title, "version": room.version}
def stats(self) -> dict:
rooms = len(self._rooms)
conns = sum(len(r.conns) for r in self._rooms.values())
return {
"rooms": rooms,
"connections": conns,
"connections_total": self.stat_connections_total,
"ops": self.stat_ops,
"merges": self.stat_merges,
"conflicts": self.stat_conflicts,
"slow_disconnects": self.stat_slow_disconnects,
"pages": [{"page_id": r.page_id, "conns": len(r.conns),
"version": r.version, "merges": r.merge_count,
"conflicts": r.conflict_count}
for r in self._rooms.values()],
}
async def _broadcast_synced_to(self, pages: list[int], synced_id: int) -> None:
"""Broadcast a synced-block event to the given pages' open rooms."""
for pid in pages:
room = self._rooms.get(pid)
if room and room.conns:
await self.load_room(room)
await self._broadcast(room, {"t": "synced_update",
"synced_id": synced_id,
"version": room.version})
async def _propagate_synced(self, synced_id: int) -> None:
"""Broadcast a synced-block update to all rooms that reference it."""
from app.services.synced_blocks import page_ids_for_synced
try:
pages = page_ids_for_synced(synced_id)
except Exception:
return
await self._broadcast_synced_to(pages, synced_id)
manager = RealtimeManager()
+273
View File
@@ -0,0 +1,273 @@
"""FlowDeck — Recurring events engine (v5.8.0 Calendrier & Rappels).
A recurring row stores its rule in ``property_values_json`` under the special
key ``__recurrence__``::
"__recurrence__": { "<date_prop_id>": {"freq": "weekly", "interval": 1,
"count": null, "until": null, "byweekday": [0,2,4],
"timezone": "Europe/Paris"} }
Only a RRULE subset is supported (matching the roadmap: daily/weekly/monthly
+ custom interval, COUNT, UNTIL and BYDAY for weekly). Expansion is computed
on the fly for a visible window — occurrences are virtual, never persisted.
All pure-Python (stdlib only) so it is trivially testable.
"""
from __future__ import annotations
import calendar as _cal
import datetime as dt
import logging
logger = logging.getLogger(__name__)
FREQS = ("daily", "weekly", "monthly")
# Monday-first weekday numbering (matches the UI calendar grid).
DOW_NAMES = {"mo": 0, "tu": 1, "we": 2, "th": 3, "fr": 4, "sa": 5, "su": 6}
RECURRENCE_KEY = "__recurrence__"
TIMEZONE_KEY = "__timezone__"
try: # Python >= 3.9 ships zoneinfo; keep a graceful fallback anyway.
from zoneinfo import ZoneInfo, available_timezones
def _zone(tz_name: str):
try:
return ZoneInfo(tz_name)
except Exception:
return None
def is_valid_timezone(tz_name: str) -> bool:
if not tz_name:
return True
try:
return tz_name in available_timezones() or tz_name == "UTC"
except Exception:
return False
except Exception: # pragma: no cover - environment without tz database
def _zone(tz_name: str):
return None
def is_valid_timezone(tz_name: str) -> bool:
return True # no tz database → accept anything (zoneinfo absent)
def parse_date(value) -> dt.date | None:
"""Parse a stored date property value ('YYYY-MM-DD[THH:MM]') into a date."""
s = str(value or "").strip()
if not s:
return None
s = s.replace(" ", "T")
for fmt in ("%Y-%m-%dT%H:%M:%S", "%Y-%m-%dT%H:%M", "%Y-%m-%d"):
try:
return dt.datetime.strptime(s[:19], fmt).date()
except ValueError:
continue
try:
return dt.date.fromisoformat(s[:10])
except ValueError:
return None
def validate_rule(rule) -> tuple[bool, str]:
"""Validate a recurrence rule dict. Returns (ok, error_message)."""
if rule is None:
return True, ""
if not isinstance(rule, dict):
return False, "Recurrence must be an object"
freq = rule.get("freq")
if freq not in FREQS:
return False, f"freq must be one of {FREQS}"
interval = rule.get("interval", 1)
if not isinstance(interval, int) or isinstance(interval, bool) or interval < 1 or interval > 365:
return False, "interval must be an integer between 1 and 365"
count = rule.get("count")
if count is not None:
if not isinstance(count, int) or isinstance(count, bool) or count < 1 or count > 1000:
return False, "count must be an integer between 1 and 1000"
until = rule.get("until")
if until not in (None, "") and parse_date(until) is None:
return False, "until must be a YYYY-MM-DD date"
if count is not None and until not in (None, ""):
return False, "count and until are mutually exclusive"
bywd = rule.get("byweekday")
if bywd not in (None, []):
if not isinstance(bywd, list) or any(
(not isinstance(d, int) or isinstance(d, bool) or d < 0 or d > 6) for d in bywd
):
return False, "byweekday must be a list of integers 0-6 (Monday=0)"
if freq != "weekly":
return False, "byweekday is only valid for weekly recurrence"
tz = rule.get("timezone")
if tz not in (None, "") and not is_valid_timezone(tz):
return False, f"unknown timezone '{tz}'"
return True, ""
def _generate_occurrences(base: dt.date, rule: dict):
"""Yield every occurrence date in chronological order, honouring
freq/interval/count/until/byweekday. Infinite rules yield forever —
the caller must bound the consumption (see expand_rule)."""
freq = rule.get("freq", "daily")
interval = max(1, int(rule.get("interval") or 1))
count = rule.get("count")
until = parse_date(rule.get("until")) if rule.get("until") else None
weekdays = sorted(set(rule.get("byweekday") or [])) if freq == "weekly" else []
produced = 0
def limit_hit(d: dt.date) -> bool:
return (count is not None and produced >= count) or bool(until and d > until)
if freq == "daily":
d = base
while not limit_hit(d):
yield d
produced += 1
d += dt.timedelta(days=interval)
elif freq == "weekly":
# Monday=0 weekday numbering (matches the UI calendar grid).
days = weekdays or [(base.isoweekday() - 1) % 7]
week0 = base - dt.timedelta(days=(base.isoweekday() - 1) % 7)
w = week0
while True:
for off in days:
d = w + dt.timedelta(days=off)
if d < base or limit_hit(d):
continue
yield d
produced += 1
w += dt.timedelta(weeks=interval)
if limit_hit(w):
return
else: # monthly
i = 0
while True:
d = _nth_month_occurrence(base, i * interval)
if limit_hit(d):
return
yield d
produced += 1
i += 1
def expand_rule(base_value, rule: dict, window_start: dt.date, window_end: dt.date,
max_occurrences: int = 500) -> list[str]:
"""Return ISO dates ('YYYY-MM-DD') of occurrences inside the inclusive
window [window_start, window_end] for a rule anchored at ``base_value``.
Occurrences before the window are skipped (but still count against
``count``); the expansion is hard-capped so pathological rules can never
blow up a request.
"""
base = parse_date(base_value)
ok, _err = validate_rule(rule)
if base is None or rule is None or not ok:
return []
out: list[str] = []
scanned = 0
for d in _generate_occurrences(base, rule or {}):
if d > window_end:
break
scanned += 1
if scanned > 200000: # safety valve for old, very long-running rules
break
if d >= window_start:
out.append(d.isoformat())
if len(out) >= max_occurrences:
break
return out
def _nth_month_occurrence(base: dt.date, months_offset: int) -> dt.date:
month_index = (base.year * 12 + (base.month - 1)) + months_offset
year, month0 = divmod(month_index, 12)
month = month0 + 1
day = min(base.day, _cal.monthrange(year, month)[1])
return dt.date(year, month, day)
def now_in_tz(tz_name: str | None = None) -> dt.datetime:
"""Current time in the given IANA timezone (fallback: UTC)."""
zone = _zone(tz_name) if tz_name else None
if zone is not None:
return dt.datetime.now(zone)
return dt.datetime.now(dt.UTC)
def local_date_in_tz(tz_name: str | None = None) -> dt.date:
"""'Today' from the point of view of ``tz_name`` (fallback UTC)."""
return now_in_tz(tz_name).date()
def _zone_dt(d: dt.date, time_str: str, tz_name: str | None):
"""Combine an occurrence date + 'HH:MM' time into an aware datetime in
``tz_name`` (fallback naive→UTC). Default time 09:00 when none stored."""
hh, mm = 9, 0
if time_str and len(time_str) >= 16:
try:
hh, mm = int(time_str[11:13]), int(time_str[14:16])
except ValueError:
pass
zone = _zone(tz_name) if tz_name else None
naive = dt.datetime(d.year, d.month, d.day, hh, mm)
if zone is not None:
return naive.replace(tzinfo=zone)
return naive.replace(tzinfo=dt.UTC)
def next_occurrences(base_value, rule: dict, tz_name: str | None,
after: dt.datetime, horizon_days: int = 365,
max_count: int = 50) -> list[tuple[dt.date, dt.datetime]]:
"""Upcoming occurrences (date, aware event datetime) strictly on/after
``after`` for a row anchored at ``base_value``. Non-recurring rules
(rule falsy) yield the single base occurrence. Bounded by horizon."""
base = parse_date(base_value)
if base is None:
return []
time_str = str(base_value or "")
start = after.date()
if not rule:
event = _zone_dt(base, time_str, tz_name)
return [(base, event)] if event >= after else []
ok, _ = validate_rule(rule)
if not ok:
return []
out: list[tuple[dt.date, dt.datetime]] = []
window_end = start + dt.timedelta(days=horizon_days)
for iso in expand_rule(base_value, rule, start, window_end, max_occurrences=max_count):
d = dt.date.fromisoformat(iso)
event = _zone_dt(d, time_str, tz_name)
if event >= after:
out.append((d, event))
if len(out) >= max_count:
break
return out
# A pragmatic list for pickers (zoneinfo's full list is ~600 entries).
COMMON_TIMEZONES = [
"UTC", "Europe/Paris", "Europe/London", "Europe/Berlin", "Europe/Madrid",
"Europe/Rome", "Europe/Amsterdam", "Europe/Lisbon", "Europe/Zurich",
"Europe/Stockholm", "Europe/Warsaw", "Europe/Moscow", "Europe/Istanbul",
"America/New_York", "America/Chicago", "America/Denver", "America/Los_Angeles",
"America/Toronto", "America/Vancouver", "America/Mexico_City", "America/Sao_Paulo",
"America/Buenos_Aires", "Asia/Dubai", "Asia/Karachi", "Asia/Kolkata",
"Asia/Bangkok", "Asia/Singapore", "Asia/Shanghai", "Asia/Tokyo", "Asia/Seoul",
"Australia/Sydney", "Australia/Perth", "Pacific/Auckland", "Africa/Cairo",
"Africa/Lagos", "Africa/Johannesburg", "Africa/Algiers", "Africa/Casablanca",
]
def common_timezones() -> list[str]:
"""Timezone list for pickers: curated common zones + all installed ones
whose region matches, deduplicated and sorted."""
try:
from zoneinfo import available_timezones
all_zones = sorted(available_timezones())
# Merge: keep curated first, then everything else available.
extra = [z for z in all_zones if z not in COMMON_TIMEZONES]
return COMMON_TIMEZONES + extra
except Exception:
return list(COMMON_TIMEZONES)
+225
View File
@@ -0,0 +1,225 @@
"""FlowDeck — Reminder service (v5.8.0 Calendrier & Rappels).
Reminders live on database rows, next to the recurrence rule, inside
``property_values_json`` under the special key ``__reminder__``::
"__reminder__": { "<date_prop_id>": {"value": 30, "unit": "minutes"} }
A background loop scans rows with reminders, computes the next occurrence
(recurrence-aware, timezone-aware), and fires an in-app notification (+ email
when the user opted in) at ``event - lead``. Fired reminders are deduplicated
through the ``reminder_log`` table keyed by (page_id, occurrence date).
The scan is a plain synchronous function (:func:`scan_and_fire`) so tests can
drive it deterministically with an injected ``now``.
"""
from __future__ import annotations
import datetime as dt
import json
import logging
from app.db import get_conn
from app.services.recurrence import (
RECURRENCE_KEY,
TIMEZONE_KEY,
next_occurrences,
parse_date,
)
logger = logging.getLogger(__name__)
REMINDER_KEY = "__reminder__"
_UNITS = {"minutes": 60, "hours": 3600, "days": 86400}
# Fire late is still useful; ignore events that happened long ago.
FIRE_GRACE_SECONDS = 3600
def parse_lead(reminder) -> int | None:
"""Return the lead in seconds for a reminder dict, or None if disabled."""
if not isinstance(reminder, dict):
return None
unit = reminder.get("unit", "none")
if unit not in _UNITS:
return None
try:
value = int(reminder.get("value", 0))
except (TypeError, ValueError):
return None
if value <= 0:
return None
return value * _UNITS[unit]
def lead_human(reminder: dict) -> str:
try:
value = int(reminder.get("value", 0))
except (TypeError, ValueError):
return ""
unit = reminder.get("unit", "")
if value == 1 and unit.endswith("s"):
unit = unit[:-1]
return f"{value} {unit}"
def _person_targets(conn, collection_id: int, props: dict) -> list[int]:
"""User ids assigned via any ``person`` property of the row."""
targets: list[int] = []
person_props = conn.execute(
"SELECT id FROM collection_properties WHERE collection_id=? AND prop_type='person'",
(collection_id,),
).fetchall()
for pp in person_props:
value = props.get(str(pp["id"])) or props.get(pp["name"] if "name" in pp.keys() else None)
if isinstance(value, list):
for person in value:
if isinstance(person, dict) and person.get("id"):
targets.append(int(person["id"]))
return targets
def _row_tz(conn, collection_id: int, props: dict) -> str:
"""Effective timezone of a row: per-event ``__timezone__`` first, then the
recurrence rule tz, then the created_by user's personal timezone,
then '' (UTC)."""
tzmap = props.get(TIMEZONE_KEY)
if isinstance(tzmap, dict):
for value in tzmap.values():
if value:
return str(value)
rec = props.get(RECURRENCE_KEY) or {}
for rule in rec.values():
if isinstance(rule, dict) and rule.get("timezone"):
return rule["timezone"]
auto_props = conn.execute(
"SELECT id FROM collection_properties WHERE collection_id=? AND prop_type='created_by'",
(collection_id,),
).fetchall()
for ap in auto_props:
value = props.get(str(ap["id"]))
people = value if isinstance(value, list) else ([value] if isinstance(value, dict) else [])
for person in people:
if isinstance(person, dict) and person.get("id"):
row = conn.execute(
"SELECT timezone FROM users WHERE id=?", (person["id"],)
).fetchone()
if row and row["timezone"]:
return row["timezone"]
return ""
def scan_and_fire(now: dt.datetime | None = None) -> int:
"""Fire every reminder that is due. Returns the number of notifications
created. Safe to call repeatedly (dedup via reminder_log)."""
now = now or dt.datetime.now(dt.UTC)
fired = 0
with get_conn() as conn:
pages = conn.execute(
"SELECT id, collection_id, title, property_values_json FROM collection_pages"
).fetchall()
for page in pages:
try:
props = json.loads(page["property_values_json"] or "{}")
except (json.JSONDecodeError, TypeError):
continue
reminders = props.get(REMINDER_KEY)
if not isinstance(reminders, dict):
continue
rec_all = props.get(RECURRENCE_KEY) if isinstance(props.get(RECURRENCE_KEY), dict) else {}
for prop_key, reminder in reminders.items():
lead = parse_lead(reminder)
if lead is None:
continue
base_value = props.get(str(prop_key))
if not base_value and str(prop_key).isdigit():
# fall back to name-keyed storage
pname = conn.execute(
"SELECT name FROM collection_properties WHERE id=?", (int(prop_key),)
).fetchone()
if pname:
base_value = props.get(pname["name"])
if parse_date(base_value) is None:
continue
rule = rec_all.get(str(prop_key))
if rule is None and not str(prop_key).isdigit():
pid = conn.execute(
"SELECT id FROM collection_properties WHERE collection_id=? AND name=?",
(page["collection_id"], str(prop_key)),
).fetchone()
if pid:
rule = rec_all.get(str(pid["id"]))
tz_name = _row_tz(conn, page["collection_id"], props)
# Look for the occurrence whose reminder moment is in
# [now - grace, now].
events = next_occurrences(
base_value, rule, tz_name or None,
after=now - dt.timedelta(seconds=lead + FIRE_GRACE_SECONDS),
horizon_days=400, max_count=2000,
)
due = None
for _d, event in events:
fire_at = event - dt.timedelta(seconds=lead)
if fire_at <= now and now < event + dt.timedelta(seconds=FIRE_GRACE_SECONDS):
due = (event, fire_at)
break
if due is None:
continue
event, _fire_at = due
if event < now - dt.timedelta(seconds=FIRE_GRACE_SECONDS):
continue
target_ids = _person_targets(conn, page["collection_id"], props)
if not target_ids:
row = conn.execute(
"SELECT id FROM users WHERE is_admin=1 ORDER BY id LIMIT 1"
).fetchone()
target_ids = [row["id"]] if row else []
for uid in dict.fromkeys(target_ids):
cur = conn.execute(
"INSERT OR IGNORE INTO reminder_log (page_id, occurrence_date) VALUES (?, ?)",
(page["id"], event.date().isoformat()),
)
if cur.rowcount != 1:
continue
from app.services import mailer, notifications
lead_text = lead_human(reminder)
notifications.create_notification(
uid, None, "reminder",
title=f"Reminder: {page['title'] or 'Untitled'}",
message=(f"Due in {lead_text} — {event.strftime('%Y-%m-%d %H:%M')}"
if lead else
f"Happening now — {event.strftime('%Y-%m-%d %H:%M')}"),
resource_type="db_page",
resource_id=page["id"],
url=f"/collections/{page['collection_id']}",
conn=conn, commit=False,
)
mailer.notify_user(
uid,
subject=f"[FlowDeck] Reminder: {page['title'] or 'Untitled'}",
body_text=f"« {page['title'] or 'Untitled'} » le {event.strftime('%Y-%m-%d %H:%M')} ({tz_name or 'UTC'}).",
cta_url=f"/collections/{page['collection_id']}",
prefs_key="reminders",
)
fired += 1
conn.commit()
return fired
async def reminder_scheduler(interval_seconds: int = 60):
"""Background loop: scan for due reminders once a minute."""
import asyncio
from app.config import settings
if not getattr(settings, "reminders_enabled", True):
logger.info("Reminder scheduler disabled via settings")
return
await asyncio.sleep(5)
while True:
try:
n = scan_and_fire()
if n:
logger.info("Reminders: fired %d notification(s)", n)
except Exception as exc: # pragma: no cover - defensive only
logger.warning("Reminder scan failed: %s", exc)
await asyncio.sleep(max(10, int(getattr(settings, "reminder_scan_interval_seconds", interval_seconds))))
+123
View File
@@ -0,0 +1,123 @@
"""FlowDeck — v6.5.0 database row content pages (synced blocks production).
A database row (``collection_pages``) has no block content of its own:
its Notion-style content lives in a shadow ``pages`` row linked via
``pages.collection_row_id``. The full page editor (synced blocks,
versions, realtime, permissions) is reused unchanged on that page.
``ensure_row_page()`` lazily creates (or heals) the shadow page and is
the single entry point used by the row-open endpoint.
"""
from __future__ import annotations
import logging
from app.db import get_conn
logger = logging.getLogger(__name__)
def _host_workspace(conn, coll: dict) -> tuple[str, int | None]:
"""Workspace (key, id) the shadow page should live in.
Resolution order: full-page database host page → inline parent page →
the collection's own ``workspace_id`` (key left empty).
"""
host = conn.execute(
"SELECT workspace, workspace_id FROM pages "
"WHERE collection_id=? AND deleted_at IS NULL "
"AND collection_row_id IS NULL LIMIT 1",
(coll.get("id"),),
).fetchone()
if not host and coll.get("parent_page_id"):
host = conn.execute(
"SELECT workspace, workspace_id FROM pages WHERE id=?",
(coll["parent_page_id"],),
).fetchone()
if host:
return host["workspace"] or "", host["workspace_id"]
if coll.get("gitea_owner") and coll.get("gitea_repo"):
return f"{coll['gitea_owner']}/{coll['gitea_repo']}", coll.get("workspace_id")
return "", coll.get("workspace_id")
def ensure_row_page(row_id: int) -> int:
"""Return the content page id for a database row, creating it if needed.
- existing live page → returned as-is;
- soft-deleted page → restored (title resynced);
- missing → created with the row's title and the collection workspace.
Raises ``KeyError`` when the row does not exist.
"""
with get_conn() as conn:
row = conn.execute(
"SELECT id, collection_id, title FROM collection_pages WHERE id=?",
(row_id,),
).fetchone()
if not row:
raise KeyError(f"row {row_id} not found")
coll_row = conn.execute(
"SELECT id, workspace_id, parent_page_id, gitea_owner, gitea_repo "
"FROM collections WHERE id=?",
(row["collection_id"],),
).fetchone()
coll = dict(coll_row) if coll_row else {"id": row["collection_id"]}
existing = conn.execute(
"SELECT id, deleted_at, title FROM pages WHERE collection_row_id=?",
(row_id,),
).fetchone()
if existing:
if existing["deleted_at"]:
conn.execute(
"UPDATE pages SET deleted_at=NULL, parent_section='DbRow', "
"title=?, updated_at=CURRENT_TIMESTAMP WHERE id=?",
(row["title"] or existing["title"] or "Untitled", existing["id"]),
)
conn.commit()
return existing["id"]
ws_key, ws_id = _host_workspace(conn, coll)
cur = conn.execute(
"""INSERT INTO pages
(workspace, workspace_id, title, content, content_format,
parent_section, collection_id, collection_row_id)
VALUES (?, ?, ?, '', 'blocks', 'DbRow', ?, ?)""",
(ws_key, ws_id, row["title"] or "Untitled",
row["collection_id"], row_id),
)
page_id = cur.lastrowid
conn.commit()
logger.debug("created content page %s for row %s", page_id, row_id)
return page_id
def sync_row_title_to_page(conn, row_id: int) -> None:
"""Row → page: copy the row's title onto its content page (if any)."""
conn.execute(
"""UPDATE pages SET title=(
SELECT title FROM collection_pages WHERE id=pages.collection_row_id
), updated_at=CURRENT_TIMESTAMP
WHERE collection_row_id=? AND EXISTS (
SELECT 1 FROM collection_pages cp
WHERE cp.id=pages.collection_row_id AND cp.title<>pages.title
)""",
(row_id,),
)
def sync_page_title_to_row(conn, page_id: int) -> None:
"""Page → row: a rename inside the editor propagates to the row."""
conn.execute(
"""UPDATE collection_pages SET title=(
SELECT title FROM pages WHERE id=?
), updated_at=CURRENT_TIMESTAMP
WHERE id=(SELECT collection_row_id FROM pages WHERE id=?)
AND EXISTS (
SELECT 1 FROM pages p
WHERE p.id=? AND p.collection_row_id IS NOT NULL
AND p.title<>collection_pages.title
)""",
(page_id, page_id, page_id),
)

Some files were not shown because too many files have changed in this diff Show More