Open Source AI & Bioterrorism Risk

The author examines the biorisk citations in "Open-Sourcing Highly Capable Foundation Models," a Center for the Governance of AI policy paper that recommends against open-sourcing highly capable foundation models, and concludes that they do not supply good evidence for that recommendation. The strongest cited material is Anthropic statements, which rest on data outside Anthropic cannot see, and the remaining citations rest mainly on one experiment without a control group. The author also argues that the paper never shows that LLMs supply information humans cannot easily find elsewhere, a principle the paper itself acknowledges, and that it never shows that missing knowledge is a significant barrier to bioweapons creation.