Guide to Using Large Language Models and Generative AI in Economic History
This NBER paper provides a practical guide for economic historians on using large language models (LLMs) and generative AI tools for research tasks such as transcription, classification, data extraction, and text analysis, while discussing methodological considerations and best practices.
Background
- The National Bureau of Economic Research (NBER) is the leading US nonprofit for economic research; its working papers are widely read by academics and policymakers before formal journal publication.
- This is a methodological guide — it does not present new findings but rather shows economic historians how to apply LLMs (e.g., GPT-4, Claude) to historical sources such as digitized newspapers, census records, or archival texts.
- Key techniques covered: optical character recognition (OCR) correction, named-entity recognition, text classification, data extraction from tables, and synthetic data generation — all adapted for historical language and non-standard spelling.
- Prior context: economic historians have traditionally used manual or rule-based coding; this paper reflects a broader push to use generative AI to scale up analysis of unstructured historical documents, while cautioning about anachronism, hallucinations, and data privacy.