# AI and the Problem of Knowledge Collapse

Source: https://arxiv.org/abs/2404.03502
Kiwi News: https://news.kiwistand.com/stories/AI-and-the-Problem-of-Knowledge-Collapse?index=0x661cc1f686a3ed9ce7ad371630420465a045dd7ac3fec9b1005cefa97704bd1750e0e08e
Submitted by misha on 2024-04-15 UTC, 1 upvote, 0 comments

## Summary

The article argues that widespread reliance on AI systems could cause "knowledge collapse," a process in which reduced access costs for certain knowledge paradoxically harm public understanding, innovation and culture. Andrew J. Peterson explains that large language models tend to generate output toward the center of their training data distribution, which is useful but can narrow the diversity of knowledge. The author's model of a community of learners choosing between traditional methods and a discounted AI-assisted process finds that a 20% discount on AI-generated content produces public beliefs 2.3 times further from the truth than no discount. The article also notes that humans, unlike AI models, may seek out diverse knowledge if they see it as worthwhile.

## Article

AI and the Problem of Knowledge Collapse

Authors: Andrew J. Peterson

While artificial intelligence has the potential to process vast amounts of data, generate new insights, and unlock greater productivity, its widespread adoption may entail unforeseen consequences. We identify conditions under which AI, by reducing the cost of access to certain modes of knowledge, can paradoxically harm public understanding. While large language models are trained on vast amounts of diverse data, they naturally generate output towards the 'center' of the distribution. This is generally useful, but widespread reliance on recursive AI systems could lead to a process we define as "knowledge collapse", and argue this could harm innovation and the richness of human understanding and culture. However, unlike AI models that cannot choose what data they are trained on, humans may strategically seek out diverse forms of knowledge if they perceive them to be worthwhile. To investigate this, we provide a simple model in which a community of learners or innovators choose to use traditional methods or to rely on a discounted AI-assisted process and identify conditions under which knowledge collapse occurs. In our default model, a 20% discount on AI-generated content generates public beliefs 2.3 times further from the truth than when there is no discount. An empirical approach to measuring the distribution of LLM outputs is provided in theoretical terms and illustrated through a specific example comparing the diversity of outputs across different models and prompting styles. Finally, based on the results, we consider further research directions to counteract such outcomes.