Evaluating Prediction Markets

Simon argues that prediction market supporters treat these markets as infallible and that they need better scoring methods than calibration alone. He points to survivor bias: a 70% outcome that later comes true is held up as proof of accuracy, a 15% outcome that flips true is framed as the mainstream media missing it, and failed predictions are forgotten. He notes that Brier scores, where 0 is perfect and 1 is worst, ignore timing, and that Philip Tetlock describes predictions that are well calibrated but hover near 50%, which are not useful. Polymarket alone recorded $213 million of volume in July.