Forecasting ML Benchmarks in 2023

Thanks to Collin Burns, Ruiqi Zhong, Cassidy Laidlaw, Jean-Stanislas Denain, and Erik Jones, who generated most of the considerations discussed in this post. Previously [], I evaluated the accuracy

AI Forecasting: One Year In

Last August, my research group created a forecasting contest [] to predict AI progress on four benchmarks. Forecasts were asked to predict state-of-the-art performance (SOTA) on each benchmark for