Categorizing Variants of Goodhart's Law

Manheim and Garrabrant categorize several distinct failure modes of overoptimizing systems based on metrics, a class often called Goodhart's Law, and argue that the distinctions between them are often ignored in discussion. The paper expands on an earlier Garrabrant discussion that identified "(at least) four different mechanisms" relating to Goodhart's Law and aims to explore these mechanisms further and specify how they occur. The authors say the analysis applies to economic regulation, public policy, machine learning, and AI alignment. They also state that the importance of Goodhart effects depends on the optimization power directed at the proxy, which makes artificial intelligence especially critical for that field.