Best of spawning.substack.com on Kiwi News
Most upvoted by Kiwi News readers
14 stories from spawning.substack.com have been shared on Kiwi News since 12 October 2023, with 20 upvotes and 0 comments in total. The latest was shared on 21 December 2024. They are listed below, most upvoted first.
Text under a title is an AI summary of the linked article.
- Announcing Source.Plus: A Curation Tool for Non-infringing AI Training Data
Spawning has opened early access to Source.Plus, a platform for curating, enriching, and bulk downloading non-infringing media collections for AI training. The company says the platform offers a consenting alternative to web scraping, with training data that it describes as compliant with EU rules on TDM rights reservations, free from copyright infringement, and of known provenance. Source.Plus is seeded with nearly 40 million public domain and CC0 images from libraries and museums worldwide, and Spawning hosts the images itself to avoid overloading institutional servers or incurring egress fees. Planned additions include premium licensed collections and donations to cultural heritage institutions when their works are downloaded.
- It Takes a Village to Raise a Dataset
Spawning announced Public Domain 12M (PD12M), an image-text dataset of 12.4 million image-caption pairs, and describes a community-driven governance model for keeping it maintained. Community members can audit the dataset on Source.Plus by flagging problematic images, which are automatically removed from view and reviewed internally, and the team says it can swap in replacement images without disrupting the dataset's statistical characteristics. Users can also upload CC0 works to public collections to help fill gaps created by auditing. Spawning's CEO, Jordan Meyer, and CTO, Nick Padgett, are working on the forthcoming Public Diffusion model, trained entirely on public domain and CC0 materials, and contributions were requested before December 1, 2024.
- Responsible Dataset Creation and Governance: Public Domain 12M
Spawning has released Public Domain 12M (PD12M), a dataset of 12.4 million PD/CC0 image-caption pairs for training text-to-image models, available through Hugging Face. The company says it built the dataset from nearly 40 million PD/CC0 images on Source.Plus, filtering them by criteria including dimensions, safety scores, aesthetic scores, and human review, and checking them against the Do Not Train registry, which returned 0 matches. The images are hosted on AWS Open Data, a setup the article says reduces hosting costs for web hosts and allows versioned corrections, such as swapping out images flagged by Source.Plus users. A smaller subset, PD3M, contains 3.3 million images, and both are released under the CDLA-Permissive-2.0 license.
- The Spawning Guide to Rights Reservations - by Laura
Spawning's guide explains how rights holders can register opt-outs so AI model trainers respect them, designed to meet EU requirements in CDSM Article 4(3) and the forthcoming AI Act. Registrations go into Spawning's Do Not Train Registry (DNTR), which Stability AI and Hugging Face, along with users of Spawning's Data Diligence package, have chosen to respect. The guide recommends three steps: domain-level registration on HaveIBeenTrained.com, item-level search on the same site (which covers only LAION-5B), and the Spawning browser extension for media hosted anywhere. Spawning states that registering does not undo prior training and that it cannot force trainers to comply.
- Announcing 𒋧 Kudurru
Kudurru is a new network, announced by its developers, that automatically identifies and blocks web scrapers collecting creative works for AI models. When scraping is detected on one member domain, the network triangulates the scraper's IP and blocks its access to every other domain running the Kudurru software. Members can also choose to feed scrapers bad data, such as images reading "NO AI", and sites in the network are added to Spawning's do-not-train list. The beta begins with a WordPress plugin, with other platforms planned.
- A Fireside Chat with the Creators of Public Diffusion
Public Diffusion is a foundation model designed for fine-tuning, trained only on public domain or CC0 images, and its creators say it is meant to let artists own the results of their fine-tunes. Jordan Meyer describes the model as similar to Stable Diffusion, and says its main hurdles are image quality, style transfer and concepts, with the dataset containing only about 10 corgis. Nick Padgett says the team added EXIF data such as lens, aperture and flash settings, along with medium tags, to give artists more control over outputs. A private beta has opened, with an open-weight release planned for next year.
- The Making of PD12M: Dataset Curation - by Laura Exline
PD12M, a 12.4 million image-caption dataset of public domain and CC0 images, was curated from about 38 million images on Source.Plus, according to Laura Exline's post on its curation process. The team filtered out 8.7 million document scans, set a minimum resolution of 256x256 pixels, and excluded works with an NSFW score above 0.5. Using a custom aesthetic model, they dropped the bottom 50% of images by score to match the size of CC12M, and the bottom 90% to match CC3M's 3.3 million.
- The Making of PD12M: Image Acquisition
PD12M is a 12.4 million image-text dataset made only of images marked with a Public Domain Mark or CC0 license, and its creators say its acquisition method is a deliberate alternative to indiscriminate open-web scraping. The images come from Source.Plus, which pulls about 38 million images mainly from museums and cultural heritage institutions such as Europeana, Wikimedia, and the Smithsonian. Wikimedia uploads were delayed 14 days so the community could flag contested images, and the images are hosted through AWS Open Data, with about 30TB of image data.
- Reflections on the Hamburg Court’s Ruling on LAION
Spawning argues that the Hamburg Regional Court's LAION ruling shows copyright compliance for AI should be checked at the time of model training, not only at crawling, and that robots.txt is inadequate for this purpose. The court dismissed photographer Robert Kneschke's infringement claims against LAION, a non-profit that created and published a dataset of image links for AI training, citing Section 60d of German Copyright Law, which permits text and data mining for scientific research. The article notes that the court distinguished dataset creation from later model training and AI use. It also cites the Data Provenance Initiative's finding that about 25+ percent of tokens from the most critical domains had become restricted by robots.txt in 2023–2024.
- One Size Does Not Fit All: Machine-Readable Rights Reservations
This Do Not Train Registry article surveys several machine-readable methods for expressing AI training rights reservations and compares how well each works. It says ai.txt saves network fees because it is checked before download, but it only works on sites the rights holder controls. The article notes C2PA manifests are removed by X and Pinterest, and that domain registration and HTTP headers protect content at download. It describes the browser extension as the only real option for works the rights holder does not host and that are not in the LAION-5B dataset.
- Opt-outs that Work in the World of AI Data Scraping
Spawning argues that AI training opt-outs must be registered and checked at the level of individual media URLs, at training time, across multiple machine-readable methods, because robots.txt alone leaves gaps. The article explains that scrapers download media directly from host servers, so a page's robots.txt is usually not read at training time, and a URL in a dataset such as LAION-5B may later return different content. It describes Spawning's Do Not Train Registry, which is delivered in batches through the Spawning API at scraping time, and its Data Diligence package, which checks for multiple opt-out methods during media download.
- Generative AI, Copyright Exceptions and the EU's AI Act
The article argues that the EU AI Act will shape how generative AI companies handle training data, copyright and transparency, with effects beyond the EU. It says the Act extends the 2019 Directive's text and data mining exceptions to generative AI and requires general purpose AI model providers to put in place a policy to respect rights holders' opt-out reservations under Article 4(3), regardless of where training takes place. The article also says providers must publish a "sufficiently detailed" summary of training content, which it reads as likely to name major datasets such as LAION-5B. Fines are described as steep, with caps of up to 35 million EUR or 7 percent of worldwide annual turnover.
- In the EU, Opt-outs Are the Way Forward
Spawning, a company that runs the Do Not Train Registry, argues that EU copyright opt-out rules for text and data mining are the way forward for AI training. Stability AI's Stable Diffusion 3 and Stable Cascade removed images opted out in that registry from their training datasets. The article says the EU's Articles 3 and 4 of the CDSM Directive permit text and data mining of lawfully accessible works, with commercial uses allowed unless rights holders opt out in an appropriate manner, such as machine-readable means. It also says member states apply these rules differently, which the registry is designed to address.
- Have I Been Trained is Back!
Have I Been Trained (HIBT), a free search tool from Spawning, is back online after an eight-week hiatus, with new safeguards to detect and remove CSAM. The tool lets users see which images are in the LAION-5B dataset and add their works to the Do Not Train Registry. Spawning took HIBT down after a Stanford Internet Observatory paper, led by David Thiel, described a method for locating CSAM in LAION-5B. Results now pass through C3P's Arachnid detection API before display, which slows searches, and the "more like this" and duplicate detection features have been removed.