You can't rely on LLMs to understand the grain of your data
LLMs struggle to grasp the granularity (grain) of data, often misunderstanding whether records represent individual transactions, daily summaries, or aggregated totals. This leads to flawed analysis, as LLMs may assume a finer or coarser grain than actually exists. The article warns that relying on LLMs without explicit grain verification can produce misleading results.
Background
- "Grain" is a key data-engineering term: the level of detail at which each row in a dataset is recorded (e.g., one row per customer vs. one row per transaction). Getting grain wrong leads to double-counting, nonsense aggregations, or hidden correlations.
- LLMs (Large Language Models) like GPT-4 and Claude are great at generating SQL or data-analysis code from plain-English prompts, but they have no real-world understanding of your specific dataset's structure or business rules.
- A common failure: an LLM might naively sum revenue by customer without realizing the table's grain is "one row per order" with multiple orders per customer, producing inflated totals.
- The article argues that human domain knowledge — knowing which columns define uniqueness, what the data-generating process looks like, and which joins are valid — remains essential. LLMs are a productivity tool, not a replacement for data literacy.