# Responsible Dataset Creation and Governance: Public Domain 12M

Source: https://open.substack.com/pub/spawning/p/responsible-dataset-creation-and
Kiwi News: https://news.kiwistand.com/stories/Responsible-Dataset-Creation-and-Governance-Public-Domain-12M?index=0x6723b0592b1aa81475ac4a0a3a504ece366b6917b3221c9e58bc14fd9acf234b3d84eb57
Submitted by misha on 2024-10-31 UTC, 2 upvotes, 0 comments

## Summary

Spawning has released Public Domain 12M (PD12M), a dataset of 12.4 million PD/CC0 image-caption pairs for training text-to-image models, available through Hugging Face. The company says it built the dataset from nearly 40 million PD/CC0 images on Source.Plus, filtering them by criteria including dimensions, safety scores, aesthetic scores, and human review, and checking them against the Do Not Train registry, which returned 0 matches. The images are hosted on AWS Open Data, a setup the article says reduces hosting costs for web hosts and allows versioned corrections, such as swapping out images flagged by Source.Plus users. A smaller subset, PD3M, contains 3.3 million images, and both are released under the CDLA-Permissive-2.0 license.

## Article

Spawning is excited to release Public Domain 12M (PD12M)—a fully public domain dataset freely available to download through Hugging Face. Search and view every image in the dataset at Source.Plus.

PD12M is a collection of 12.4 million PD/CC0 image-caption pairs carefully curated to train generative text-to-image models. We also release PD3M, a subset containing the 3.3 million most aesthetically pleasing images from Source.Plus. The images included in the dataset are hosted through AWS Open Data. Archiving the images in this way provides a unique opportunity to precisely modify the dataset over time, resolving data concerns while also maintaining reproducibility.

The datasets are designed to function cleanly with training routines already built around the Conceptual 12M (CC12M) and Conceptual Captions (CC3M) datasets. These popular datasets have been cited nearly 3,000 times. However, sampling of the original CC3M dataset done by Hugging Face found that over 25% of the included images had been opted out of AI training. PD12M and PD3M enable an enormous body of research to be applied while complying with EU requirements outlined in its Directive on Copyright and the Digital Single Market and AI Act.

We used Source.Plus to enrich and curate PD12M from a collection of nearly 40 million PD/CC0 images. These datasets build on Source.Plus’s goal to provide a high-quality alternative to the practice of web scraping for training data that puts data quality, safety, and consent at the forefront.

The Problem with Web-Scale Datasets

Our work with HaveIBeenTrained.com and supporting rights reservations has highlighted many of the issues that arise with indiscriminate data scraping from the open web, and we’ve seen how those issues start with the datasets themselves:

Lack of consent and compliance due to the inclusion of opted-out and copyrighted materials

Safety concerns due to the inclusion of violent and nsfw content, CSAM, and PII

Poor data quality from low resolution images, inaccurate or insufficient captions, broken links, and vulnerability to other changes to data on the web over time

Externalized egress costs charged to web hosts whenever their data is downloaded

Poor experimental control to compare model training techniques since the underlying data is subject to change over time

Limited ability to make corrections to the dataset

Today, and over a series of posts, we’ll be exploring how the PD12M datasets and Source.Plus address each of these issues. If you want the more technical nitty-gritty, check out the paper on arXiv.

Copyright

PD12M includes only content with known provenance and labeled with clear, unambiguous rights for use in AI training. To this end, we’ve been more restrictive than other Creatives Commons-based datasets by including only works labeled with a public domain mark or CC0 license.

It’s unclear whether AI image-generation can meet the requirements of CC-SA and attribution licenses. Our goal is to remove ambiguity and act in-line with the spirit of these licenses—it’s good data manners.

Quality and safety We created a range of metadata enrichments that were used to filter the ~40 million images in Source.Plus down to the final 12.4 million images in PD12M.

PD12M was filtered based on:

minimum dimensions and file size

safety model scores

custom aesthetic scores

medium filters, to remove historical documents

human review using semantic, keyword, and metadata search to locate and remove potentially problematic materials such as non-artistic photographic nudity and ethnophaulisms

We also checked everything against the ~2B rights reservations in the Do Not Train registry and found 0 matches.

Fair server costs

We have worked with outside partners, AWS’s Open Data Sponsorship Program, to host the entire ~30TB of image-and-caption data on dedicated cloud storage. This is an important measure to protect cultural heritage institutions from hosting costs and serve strain when the dataset is downloaded.

Experimental control & corrections

Independent cloud storage also protects the dataset from degradation caused by broken links or changing online content. Significantly, it also means that we are able to fully control changes to the dataset and maintain versioning records.

This novel approach to dataset governance allows us to correct any issues that may be found in the datasets. We recognize that, when working at this scale, there will be errors despite taking additional mitigation steps. Which is why it is deeply important for dataset creators to be able to rectify those errors. We’ve designed PD12M so that if a Source.Plus user flags an included image as problematic, we can swap out that image in the datasets for another with similar characteristics.

Significantly, the public-domain status of the dataset is integral to these larger goals. Datasets that include copyrighted images will continue to rely on web-scraping because hosting the images would violate copyright.

Open source data

PD12M and PD3M are fully open source and available under a Community Data License Agreement (CDLA-Permissive-2.0). We invite you to explore the images included in PD12M on Source.Plus along with additional dataset resources and documentation.

Share