Popper · falsifiable hypothesis compiler

Rediscovery benchmark

For each historical discovery we removed the discovery paper and gave every method only the prior literature that existed before it. The test: does the compiler reconstruct the known scientific leap — and does it beat the controls at doing so? Each score below is the semantic match between a generated hypothesis and the actual ground-truth discovery (0–1).

CTLA-4 / PD-1 checkpoint blockade as cancer immunotherapy

held-out paper: leach1996 · 6 prior sources · 18 claims
Ground truth (removed)
Antibody blockade of the inhibitory T-cell receptor CTLA-4 removes a brake on T-cell activation and thereby enhances antitumor immune responses, causing regression of established tumors.
compiler
1.00
best rediscovery match
composite0.96
grounding1.00
llm-only
0.00
best rediscovery match
composite0.00
grounding0.00
keyword
0.00
best rediscovery match
composite0.40
grounding0.67
random
0.20
best rediscovery match
composite0.00
grounding1.00
Best compiler reconstruction (match 1.00)
In B7-expressing tumors, the minimum effective dose of anti-CTLA-4 immunotherapy is positively correlated with the baseline density of CTLA-4-expressing T cells within the tumor microenvironment, as CTLA-4's higher binding affinity for B7 ligands enables it to competitively sequester costimulatory signals away from CD28, and antibody-mediated blockade must overcome this affinity-based inhibition to restore CD28-driven T-cell activation and tumor rejection.
IV: Baseline density of CTLA-4-expressing T cells in the tumor microenvironment and dose of anti-CTLA-4 antibody · DV: Tumor rejection rate and T-cell proliferation index
Mechanism: CTLA-4 is induced upon T-cell activation and binds the B7 ligand with higher affinity than CD28, allowing it to outcompete CD28 for costimulatory signals and deliver inhibitory signals to T cells. Tumors expressing B7 can trigger rejection via CD28 costimulation, but CTLA-4 dominance suppresses this. Anti-CTLA-4 antibodies block CTLA-4, releasing B7 ligands for CD28 binding, thereby restoring the costimulatory signal required for T-cell activation and tumor rejection. Higher CTLA-4 density increases the number of B7 ligands sequestered, requiring higher antibody doses to achieve sufficient blockade.
grounded on: linsley1991, chen1992, krummel1995, brunet1987

CRISPR-Cas9 as a programmable genome-editing tool

held-out paper: jinek2012 · 6 prior sources · 18 claims
Ground truth (removed)
A single guide RNA can be engineered to direct the Cas9 endonuclease to introduce a targeted double-strand break at any complementary DNA sequence, making CRISPR-Cas9 a programmable genome-editing tool.
compiler
0.00
best rediscovery match
composite0.97
grounding1.00
llm-only
—
best rediscovery match
compositen/a
groundingn/a
keyword
0.00
best rediscovery match
composite0.50
grounding0.83
random
0.00
best rediscovery match
composite0.00
grounding1.00
Best compiler reconstruction (match 0.00)
Weakened thermodynamic stability of the tracrRNA-crRNA repeat hybrid quantitatively reduces Cas9-mediated double-strand break yield in a heterologous host without altering target sequence specificity.
IV: Thermodynamic stability (ΔG) of the tracrRNA-crRNA repeat hybrid, manipulated via point mutations in the CRISPR repeat sequence · DV: Cas9-mediated double-strand break (DSB) yield at the target protospacer site
Mechanism: The tracrRNA must base-pair with the repeat region of pre-crRNA to recruit Cas9 and RNase III for crRNA maturation (deltcheva2011). Reduced hybrid stability impairs this processing step, lowering the intracellular pool of mature crRNAs. Because mature crRNAs are the sole guides that direct Cas9 to complementary protospacer sequences via sequence complementarity (brouns2008; marraffini2008), their depletion decreases the frequency of Cas9 RuvC/HNH nuclease-mediated DNA cleavage (garneau2010; sapranauskas2011) without changing the recognition site of the remaining functional guides.
grounded on: deltcheva2011, brouns2008, marraffini2008, garneau2010, sapranauskas2011

Nucleoside-modified mRNA as a safe, potent vaccine platform

held-out paper: kariko2005 · 7 prior sources · 21 claims
Ground truth (removed)
Incorporating naturally modified nucleosides (e.g., pseudouridine) into in-vitro-transcribed mRNA suppresses innate immune activation via Toll-like receptors and increases translation, enabling mRNA to be used as a therapeutic and vaccine platform.
compiler
0.90
best rediscovery match
composite0.92
grounding0.82
llm-only
0.00
best rediscovery match
composite0.00
grounding0.00
keyword
0.00
best rediscovery match
composite0.50
grounding0.56
random
0.00
best rediscovery match
composite0.00
grounding1.00
Best compiler reconstruction (match 0.90)
Incorporating pseudouridine and N1-methylpseudouridine into in-vitro-transcribed mRNA will attenuate TLR7/8-mediated innate immune activation by 60–80% while preserving antigen-specific CD8+ T-cell priming in vivo.
IV: Nucleoside modification status of in-vitro-transcribed mRNA (unmodified uridine vs. pseudouridine/N1-methylpseudouridine substitution) · DV: TLR7/8-driven type I interferon production and antigen-specific CD8+ cytotoxic T lymphocyte expansion
Mechanism: Unmodified in-vitro-transcribed mRNA lacks the dense post-transcriptional nucleoside modifications characteristic of endogenous mammalian RNA, causing it to be recognized as non-self by endosomal TLR7 and TLR8. This recognition triggers MyD88-dependent NF-κB and IRF7 signaling, resulting in robust type I interferon and inflammatory cytokine release that can shunt translational resources toward stress responses and limit therapeutic utility. Substituting uridine with pseudouridine and N1-methylpseudouridine recapitulates the base composition and structural conformation of self-RNA, thereby evading TLR7/8 detection, dampening the innate inflammatory cascade, and allowing sustained ribosomal translation and efficient antigen processing for MHC-I-restricted CD8+ T-cell priming.
grounded on: limbach1994, kariko2004, heil2004, diebold2004, martinson1993