Instruction Retrieval at Inference Time for Small Language Models

Kenan Alkiek, David Jurgens, Vinod Vydiswaran

The facts a language model stores are tied to its parameter count, so small models that fit on edge devices fail on expert problems, which need specialized knowledge and follow mul…