Software 2.0

Andrej Karpathy argues that neural networks represent a fundamental shift in how software is developed, which he calls Software 2.0, rather than just another classifier in the machine learning toolbox. In this stack, programs are found by optimization against a dataset and an evaluation criterion, not written line by line as in Software 1.0. He points to visual recognition, speech recognition, machine translation, Go, and database indexing as areas already moving to this approach, including AlphaGo Zero and "The Case for Learned Index Structures", which reported outperforming cache-optimized B-Trees by up to 70% in speed. He also notes drawbacks, such as networks that are hard to interpret and can fail silently through biases in their training data.