Things we learned about LLMs in 2024

In 2024 the GPT-4 barrier was broken, with 18 organizations now holding models that rank above the original GPT-4 (GPT-4-0314) on the Chatbot Arena Leaderboard, 70 models in total, beginning with Google's Gemini 1.5 Pro in February. Prices for top hosted models fell sharply, driven by competition and efficiency gains, with GPT-4o mini at $0.15 per million input tokens, about 200 times cheaper than GPT-4. Meta's Llama 3.3 70B, released in December, is a GPT-4 class model that runs on a 64GB laptop.