Showing cs.CLShow all
2 papers · 1 filter
cs.CL2024
STLM Engineering Report: Dropout
Dylan Hillier, Leon Guertler, Bobby Cheng +1
In this work we explore the relevance of dropout for modern language models, particularly in the context of models on the scale of <100M parameters. We explore it's relevance first…
cs.CL2024
Super Tiny Language Models
Dylan Hillier, Leon Guertler, Cheston Tan +3
The rapid advancement of large language models (LLMs) has led to significant improvements in natural language processing but also poses challenges due to their high computational a…