Writing an LLM from scratch, part 32l -- Interventions: updated instruction fine-tuning results
Updated instruction fine-tuning tests on GPT-2-style models show OpenAI's models performed best. Some custom models with similar test loss scores showed unexpected variations in instruction-following ability, with no clear pattern emerging across all tested models.