# Microsoft and Reddit Are Fighting About Why Bing’s Crawler Is Blocked on Reddit

Source: https://www.404media.co/microsoft-and-reddit-are-fighting-about-why-bings-crawler-is-blocked-on-reddit?ref=daily-stories-newsletter
Kiwi News: https://news.kiwistand.com/stories/Microsoft-and-Reddit-Are-Fighting-About-Why-Bings-Crawler-Is-Blocked-on-Reddit?index=0x66abd963935252d0047a53fd96236f3625543752862dbeea608afea8d06e5dcf760f4806
Submitted by naomiii.eth on 2024-08-01 UTC, 2 upvotes, 0 comments

## Summary

Microsoft and Reddit are giving conflicting explanations for why Bing is currently blocked from crawling Reddit and showing its links in search results. Reddit, which now requires payment from anyone crawling its site and using its data to train AI products, says Bing's crawler is being used to power AI products. Microsoft says it has made it easy for any site to block the crawler used for AI products while still allowing a search-only crawler, and that Reddit's block is "impacting competition" in search. Presented as further evidence, the dispute reflects how indiscriminate scraping for AI training data is eroding trust and making the internet less open.

## Article

Microsoft and Reddit are offering conflicting explanations for why Microsoft’s search engine, Bing, is currently blocked from crawling Reddit and offering links from the site in its search results.

Reddit, which now demands payment from anyone crawling the site and using its data to train AI products, claims that Bing’s crawler is being used to power AI products. Microsoft claims it has made it easy for any site to block its crawler that’s used for AI products, while still allowing a crawler that is only used for search results, and that Reddit’s decision to block Bing is “impacting competition” in the search engine space.

The conflicting reasonings behind the block are further proof that the massive, indiscriminate scraping of the internet to create AI training data in a way that violates long-respected norms about how to access information on the web are eroding trust, making the internet less open, and causing tech companies to beef about this issue in public.

This post is for paid members only

Become a paid member for unlimited ad-free access to articles, bonus podcast content, and more.

Subscribe

Sign up for free access to this post

Free members get access to posts like this one along with an email round-up of our week's stories.

Subscribe

Already have an account? Sign in