What the AI boom is getting wrong (and right), according to Hugging Face’s head

Irene Solaiman, Hugging Face's head of global policy, says the AI industry is behind in handling consent, bias and low-resource languages, and that these problems need to be addressed at the level of the whole system. She describes Hugging Face as advising regulators from the U.S. to the European Union while leading its own work on bias assessment and watermarking. On language, she says training and generating non-English text costs more per token, though OpenAI's GPT-4o launch and Cohere's Command R have cut tokenization costs for many languages. She also describes Hugging Face's work with the Indian government's Bhashini program on a Hinglish model and an evaluation leaderboard launched in Arabic and Korean.