80TB+ of astronomy for the HDD-poor: crossmatch the Universe from your laptop
Researchers released a 450GB subset of the 80TB+ Multimodal Universe dataset, allowing astronomers to crossmatch data from multiple large sky surveys on a standard laptop. The dataset includes curated data from seven major surveys like Euclid, DESI, and Gaia, aiming to democratize access to big-data astronomy for machine learning research.
Background
- The **Multimodal Universe** is a massive open dataset (~80TB) combining astronomical images, spectra, and catalogs from major sky surveys (e.g., DESI, Hubble, JWST, Gaia, Euclid, SDSS, Rubin/LSST). It is designed to train machine learning models like Vision–Language Models (VLMs) for astronomy.
- Hugging Face (HF) is the leading platform for sharing ML models and datasets, analogous to GitHub for AI. This blog post is on Hugging Face's "Hugging Science" channel, which promotes open science and ML in research.
- The dataset is "crossmatched" — astronomical objects (stars, galaxies, quasars) are linked across different surveys so one can query, say, “show me all quasars observed by both Hubble and DESI” without downloading 80TB.
- **Why it matters**: Astronomy now produces petabytes of data; individual researchers cannot download everything. The Multimodal Universe enables anyone with a laptop to work with state-of-the-art multi-survey data for ML and science, democratizing access that was previously limited to large institutions.
- The blog aligns with the broader open-science movement and efforts to make frontier research more reproducible and accessible via community tools.