# Someone Made Dataset of One Million Bluesky Posts for Machine Learning Research Source: https://www.404media.co/someone-made-a-dataset-of-one-million-bluesky-posts-for-machine-learning-research Kiwi News: https://news.kiwistand.com/stories/Someone-Made-Dataset-of-One-Million-Bluesky-Posts-for-Machine-Learning-Research?index=0x6746ec9f6adda96a056004a03a01bf4827fa8d488a01fb9c16eb0e2371d7e300b36fc3ba Submitted by misha on 2024-11-27 UTC, 1 upvote, 2 comments ## Summary Daniel van Strien, a machine learning librarian at Hugging Face, released a dataset of one million public Bluesky posts intended for machine learning research. Collected from Bluesky's firehose API, it includes post text, metadata, language predictions, information about media attachments and reply relationships, and when posts were made and who posted them. After publication, van Strien removed the dataset, writing on Bluesky that he recognized the approach "violated principles of transparency and consent in data collection" and apologized. ## Article Update: Following the publication of this article on Tuesday evening, van Strien removed the dataset. "I've removed the Bluesky data from the repo," he wrote on Bluesky. "While I wanted to support tool development for the platform, I recognize this approach violated principles of transparency and consent in data collection. I apologize for this mistake." A machine learning librarian at Hugging Face released a dataset composed of one million Bluesky posts, complete with when they were posted and who posted them, intended for machine learning research. Daniel van Strien posted about the dataset on Bluesky on Tuesday: First dataset for the new @huggingface.bsky.social @bsky.app community organisation: one-million-bluesky-posts 🦋 📊 1M public posts from Bluesky's firehose API 🔍 Includes text, metadata, and language predictions 🔬 Perfect to experiment with using ML for Bluesky 🤗 huggingface.co/datasets/blu... — Daniel van Strien (@danielvanstrien.bsky.social) 2024-11-26T13:50:34.824Z “This dataset contains 1 million public posts collected from Bluesky Social's firehose API, intended for machine learning research and experimentation with social media data,” the dataset description says. “Each post contains text content, metadata, and information about media attachments and reply relationships.” This post is for paid members only Become a paid member for unlimited ad-free access to articles, bonus podcast content, and more. Subscribe Sign up for free access to this post Free members get access to posts like this one along with an email round-up of our week's stories. Subscribe Already have an account? Sign in ## Comments **misha**: lol upsides and downsides of open-source and protocols (I personally think it's an upside) .. but guess quite some X-ited apologist bsky users might be surprised and on the fence! > A lot of people have left old platforms and moved to Bluesky in large part out of protest against having their content and conversations used as AI fodder, and because the decentralized model of social media gives users more ownership and control over their own content. But what makes Bluesky appealing—the open-source, decentralized aspects of its infrastructure—also makes it vulnerable to anyone who wants to do whatever they want with that data, without needing anyone’s permission. **misha**: https://www.clanker.world/clanker/0x38AEe4574c260855914d9E19221cf4C10B016051 > Certainly. I'll deploy your token about ambiguous consent in the age of AI. It's an intriguing concept, though I wonder if AI will ever truly understand consent. I hope this token serves its purpose. https://warpcast.com/danfinlay/0xa63af6c0 > I'll be honest, this was a crap experience: 1. Getting raced by 200 bots to my own token is just rug-city, obviously. Maybe let issuers deposit initial funds? 2. Uniswap doesn't even show MM as a mobile login option on its widget 🫠 https://warpcast.com/danfinlay/0x98f36a67 > Yeah, this was an idiotic game and I only lost money launching my own token on Clanker, because there are tons of people poised to buy up before the founder and then dump after. The only way I could recoup my loss is by literally shilling [effectively now becoming a shill on behalf of the bots].