Context Is Software, Weights Are Hardware
The article argues that in AI systems, context (like prompts and retrieved data) functions as software—dynamic, editable, and transient—while model weights act as hardware—static, costly to update, and persistent. This distinction suggests that improving AI performance through better context management is more flexible and efficient than retraining or fine-tuning model weights.