WITCHCRAFT AI · PANTHEON · GENOMIC ANALYSIS DIVISION
In an internal 200-question test drawn from six public AI benchmarks, our Hecate method raised Gemini 4 Argon's score from 58% to 85% on the same questions. Internal and not independently audited; some questions were reformatted, so these figures are not comparable to official leaderboards.
In a follow-up open-ended math test (HMMT, BRUMO and CMIMC 2025), the Hecate method scored 84% versus 72% for Gemini 4 Argon answering without code tools. Internal, not independently audited.
The laboratory sequences. She reads what came back. ATHENA is our whole-genome analysis division: rigorous, structured analysis of whole-genome sequencing data, with a proposed workflow from quality control through annotation to expert-reviewed interpretation support.
Begin with whole-genome data, its source, reference build and processing history. ATHENA focuses on data analysis; sample collection and sequencing are outside this product brief.
Coverage, contamination, call rate. The dataset argues for its own fitness before any question is asked of it. Weak data is named, not smoothed over.
Every genomic interval transformation between UCSC BED (0-based half-open [b, e)) and Sequence Ontology GFF3 v1.26 (1-based inclusive [s, e]) is mediated by our dual-verified deterministic CoordinateLedger engine (5,050/5,050 round-trips, 100.00% exact agreement across 3,522 evaluated LLM episodes, deterministic abstention on liftover, circular, discontinuous, and zero-length insertion traps) before variant annotation.
Findings are organised into structured, human-reviewable reports: what was observed, what the literature supports, what remains uncertain. Confidence is stated, never implied.
Build review and provenance checks into the proposed workflow. Expert review and reproducibility must be demonstrated before clinical use is considered.
Awarded and executed under the Iacon Research Grants / Arkenos Vector Programme (G-0064), CoordinateLedger tests whether pairing a large language model with a deterministic validation and coordinate-conversion ledger improves exact genomic interval agreement, edge-case abstention, and token efficiency between UCSC BED (0-based, half-open [b, e)) and Sequence Ontology GFF3 v1.26 (1-based, inclusive [s, e]) across 360 synthetic fixtures (180 valid conversions, 120 schema-invalid inputs, and 60 legitimate unsupported representations; zero patient DNA).
| cell_0.deterministic_baseline | 270 / 270 (100.00%, Wilson 95% CI [98.60%, 100.00%]) on held-out and 90 / 90 (100.00%) on development, backed by 5,050 / 5,050 (100.00%) bijective BED <-> GFF3 round-trips and 360 / 360 (100.00%) agreement between two independent algebraic solvers (converter_tool.py & reference_checker.py). |
|---|---|
| cell_1_vs_2.gemini_3_8_flash | 810 / 810 (100.00%, [99.53%, 100.00%]) with CoordinateLedger tool mediation vs 720 / 810 (88.89%, [86.54%, 90.87%]) instructions-only Intention-to-Treat — a paired lift of +11.11 percentage points (10,000x fixture-clustered bootstrap 95% CI [+9.26%, +12.96%], exact McNemar p = 1.62 × 10-27, Cohen's h = 0.680), eliminating a 90-case CoT reasoning-token exhaustion failure (66,596 output tokens burned in-model before turn cutoff; Per-Protocol retry sensitivity 810/810) while reducing output tokens by 99.39% (527,173 → 3,219; 163.8× compression). |
| cell_3_vs_4.gemini_4_argon | 810 / 810 (100.00%, [99.53%, 100.00%]) in both tool-mediated and instructions-only arms across 3 held-out repetitions, with CoordinateLedger cutting output tokens by 98.36% (254,611 → 4,165; 61.1× compression) and input tokens by 93.60% (1,054,042 → 67,430). |
| pilot_96 & unhinted_ablation_96 | 96 / 96 (100.00%) on the 4-cell development pilot and 96 / 96 (100.00%) on the 4-cell unhinted raw-record ablation battery (zero inline formula hints), cutting ablation output tokens from 51,143 (instructions-only) to 3,985 (tool-mediated). |
| open.reproducibility_pack | All new CoordinateLedger code is released under MIT and all 360 synthetic fixtures & specifications under CC BY 4.0 (zero proprietary ATHENA prompts or clinical/patient data disclosed). |
| dataset.provenance | Laboratory source, reference build, pipeline versions. The report states where every letter came from. |
|---|---|
| quality.summary | The QC verdict in plain numbers, with any dataset flagged and why. |
| findings.structured | Observations organised by category, each carrying its evidence source. |
| confidence.stated | What is supported, what is uncertain, and what would change the reading. |
| human.review | A place for the reviewing expert, review date and disposition in the proposed report. |
A claim without a traceable source does not enter the report. Charm is not a data type.
The same rigour that finds what is present must be applied to what is not yet known. Uncertainty is reported, not buried.
The intended audience includes geneticists, counsellors and researchers. Interpretation support must preserve their judgment and responsibility.