Initial commit: 25 scripts Python sauvegardés

Date: 2026-08-28 11:43:34 EDT
2026-08-28 11:43:34 -04:00
parent 0ba9701e0f
commit 994ea86d47
204 changed files with 70104 additions and 2 deletions
@@ -0,0 +1,226 @@
+# Citation Management & Hallucination Prevention
+
+This reference provides a complete workflow for managing citations programmatically, preventing AI-generated citation hallucinations, and maintaining clean bibliographies.
+
+---
+
+## Contents
+
+- [Why Citation Verification Matters](#why-citation-verification-matters)
+- [Citation APIs Overview](#citation-apis-overview)
+- [Verified Citation Workflow](#verified-citation-workflow)
+- [Python Implementation](#python-implementation)
+- [BibTeX Management](#bibtex-management)
+- [Common Citation Formats](#common-citation-formats)
+- [Troubleshooting](#troubleshooting)
+
+---
+
+## Why Citation Verification Matters
+
+### The Hallucination Problem
+
+Research has documented significant issues with AI-generated citations:
+- **~40% error rate** in AI-generated citations (Enago Academy research)
+- NeurIPS 2025 found **100+ hallucinated citations** slipped through review
+- Common errors include:
+  - Fabricated paper titles with real author names
+  - Wrong publication venues or years
+  - Non-existent papers with plausible metadata
+  - Incorrect DOIs or arXiv IDs
+
+### Consequences
+
+- Desk rejection at some venues
+- Loss of credibility with reviewers
+- Potential retraction if published
+- Wasted time chasing non-existent sources
+
+### Solution
+
+**Never generate citations from memory—always verify programmatically.**
+
+---
+
+## Citation APIs Overview
+
+### Primary APIs
+
+| API | Coverage | Rate Limits | Best For |
+|-----|----------|-------------|----------|
+| **Semantic Scholar** | 214M papers | 1 RPS (free key) | ML/AI papers, citation graphs |
+| **CrossRef** | 140M+ DOIs | Polite pool with mailto | DOI lookup, BibTeX retrieval |
+| **arXiv** | Preprints | 3-second delays | ML preprints, PDF access |
+| **OpenAlex** | 240M+ works | 100K/day, 10 RPS | Open alternative to MAG |
+
+### API Selection Guide
+
+```
+Need ML paper search? → Semantic Scholar
+Have DOI, need BibTeX? → CrossRef content negotiation
+Looking for preprint? → arXiv API
+Need open data, bulk access? → OpenAlex
+```
+
+### No Official Google Scholar API
+
+Google Scholar has no official API. Scraping violates ToS. Use SerpApi ($75-275/month) only if Semantic Scholar coverage is insufficient.
+
+---
+
+## Verified Citation Workflow
+
+### 5-Step Process
+
+```
+1. SEARCH → Query Semantic Scholar with specific keywords
+     ↓
+2. VERIFY → Confirm paper exists in 2+ sources
+     ↓
+3. RETRIEVE → Get BibTeX via DOI content negotiation
+     ↓
+4. VALIDATE → Confirm the claim appears in source
+     ↓
+5. ADD → Add verified entry to .bib file
+```
+
+### Step 1: Search
+
+Use Semantic Scholar for ML/AI papers:
+
+```python
+from semanticscholar import SemanticScholar
+
+sch = SemanticScholar()
+results = sch.search_paper("transformer attention mechanism", limit=10)
+
+for paper in results:
+    print(f"Title: {paper.title}")
+    print(f"Year: {paper.year}")
+    print(f"DOI: {paper.externalIds.get('DOI', 'N/A')}")
+    print(f"arXiv: {paper.externalIds.get('ArXiv', 'N/A')}")
+    print(f"Citation count: {paper.citationCount}")
+    print("---")
+```
+
+### Step 2: Verify Existence
+
+Confirm paper exists in at least two sources:
+
+```python
+import requests
+
+def verify_paper(doi=None, arxiv_id=None, title=None):
+    """Verify paper exists in multiple sources."""
+    sources_found = []
+
+    # Check Semantic Scholar
+    sch = SemanticScholar()
+    if doi:
+        paper = sch.get_paper(f"DOI:{doi}")
+        if paper:
+            sources_found.append("Semantic Scholar")
+
+    # Check CrossRef (via DOI)
+    if doi:
+        resp = requests.get(f"https://api.crossref.org/works/{doi}")
+        if resp.status_code == 200:
+            sources_found.append("CrossRef")
+
+    # Check arXiv
+    if arxiv_id:
+        resp = requests.get(
+            f"http://export.arxiv.org/api/query?id_list={arxiv_id}"
+        )
+        if "<entry>" in resp.text:
+            sources_found.append("arXiv")
+
+    return len(sources_found) >= 2, sources_found
+```
+
+### Step 3: Retrieve BibTeX
+
+Use DOI content negotiation for guaranteed accuracy:
+
+```python
+import requests
+
+def doi_to_bibtex(doi: str) -> str:
+    """Get verified BibTeX from DOI via CrossRef content negotiation."""
+    response = requests.get(
+        f"https://doi.org/{doi}",
+        headers={"Accept": "application/x-bibtex"},
+        allow_redirects=True
+    )
+    response.raise_for_status()
+    return response.text
+
+# Example: "Attention Is All You Need"
+bibtex = doi_to_bibtex("10.48550/arXiv.1706.03762")
+print(bibtex)
+```
+
+### Step 4: Validate Claims
+
+Before citing a paper for a specific claim, verify the claim exists:
+
+```python
+def get_paper_abstract(doi):
+    """Get abstract to verify claims."""
+    sch = SemanticScholar()
+    paper = sch.get_paper(f"DOI:{doi}")
+    return paper.abstract if paper else None
+
+# Verify claim appears in abstract
+abstract = get_paper_abstract("10.48550/arXiv.1706.03762")
+claim = "attention mechanism"
+if claim.lower() in abstract.lower():
+    print("Claim appears in paper")
+```
+
+### Step 5: Add to Bibliography
+
+Add verified entry to your .bib file with consistent key format:
+
+```python
+def generate_citation_key(bibtex: str) -> str:
+    """Generate consistent citation key: author_year_firstword."""
+    import re
+
+    # Extract author
+    author_match = re.search(r'author\s*=\s*\{([^}]+)\}', bibtex, re.I)
+    if author_match:
+        first_author = author_match.group(1).split(',')[0].split()[-1]
+    else:
+        first_author = "unknown"
+
+    # Extract year
+    year_match = re.search(r'year\s*=\s*\{?(\d{4})\}?', bibtex, re.I)
+    year = year_match.group(1) if year_match else "0000"
+
+    # Extract title first word
+    title_match = re.search(r'title\s*=\s*\{([^}]+)\}', bibtex, re.I)
+    if title_match:
+        first_word = title_match.group(1).split()[0].lower()
+        first_word = re.sub(r'[^a-z]', '', first_word)
+    else:
+        first_word = "paper"
+
+    return f"{first_author.lower()}_{year}_{first_word}"
+```
+
+---
+
+## Python Implementation
+
+### Complete Citation Manager Class
+
+{% raw %}
+```python
+"""
+Citation Manager - Verified citation workflow for ML papers.
+"""
+
+import requests
+import time
+from typing import Opt