Post

Post 1: Follow-up: ran my AI red teaming pipeline against all 3 GiveWell interventions with published AI output. Signal rates: 84%, 100%, 82%. Zero hallucinated citations. The interesting part is what showed up across all three. 🧵

Same structural patterns in every model: dynamic phenomena treated as static, meaningful variation collapsed into aggregates, and for the malaria interventions, no modeling of biological adaptation by the target organism.

First run cost $30 — double target. Diagnosed it: Verifier stage was 70% of budget. Batched critiques by thread, cut verification cost ~70%. Phases 2 and 3 came in at $16 each. $34 total for all three interventions.

0 reposts0 likes