Judgment Cannot Be Prompted
The article argues that AI lacks true judgment because it operates through prompting and pattern matching rather than genuine understanding. Real human judgment requires subjective experience, ethical reasoning, and contextual awareness that algorithms cannot replicate. Over-reliance on AI for nuanced decisions is cautioned against.
Background
- The article argues that large language models (GPT-4, Claude, Gemini) cannot exercise genuine judgment — they only predict the next likely word, not weigh what is true, important, or appropriate.
- "Prompt engineering" (crafting detailed instructions to steer AI output) is presented as fundamentally limited because judgment requires discretion, sensing stakes, and balancing competing values — things statistical token prediction cannot do.
- The piece draws on the philosophical distinction between following rules (calculation) and knowing when and how to apply them (judgment), associated with Aristotle's "phronesis" (practical wisdom) and philosopher Hans-Georg Gadamer.
- The author pushes back against tech-industry optimism that better prompting or chain-of-thought reasoning will make AI reliably wise, suggesting that genuine judgment requires embodiment, experience, and accountability — things LLMs lack.
- Relevant to debates about AI reliability, the limits of prompting, the "alignment" problem, and whether scaling models produces understanding or just more convincing mimicry.