Show HN: DELN – An interactive atlas of AI training datasets
DELN is an interactive atlas that visualizes major web crawl AI training datasets, showing how they relate to and influence one another via a map interface with dropdown filters.
Background
- AI models like GPT, Llama, and Stable Diffusion are trained on massive collections of text and images scraped from the public internet — these collections are called "web crawl datasets." The best-known example is **Common Crawl**, a free archive of billions of web pages that has been the foundation for most major AI training runs.
- Companies and researchers rarely train directly on raw web data; instead they create refined subsets (e.g., **C4**, **RefinedWeb**, **Dolma**, **FineWeb**) by filtering, cleaning, or deduplicating Common Crawl snapshots. Which dataset you pick affects what your model learns — and what biases, errors, or copyrighted material it absorbs.
- DELN is an interactive visualization tool (not a dataset itself) that lets you explore how these datasets are related — which ones fork from others, which are subsets, and how the lineage flows. It's meant to make a normally opaque part of AI infrastructure visible to practitioners and the curious.