3 papers
cs.CL2024
STLM Engineering Report: Dropout
Dylan Hillier, Leon Guertler, Bobby Cheng +1
In this work we explore the relevance of dropout for modern language models, particularly in the context of models on the scale of <100M parameters. We explore it's relevance first…
cs.LG2024
Social Learning through Interactions with Other Agents: A Survey
Dylan Hillier, Cheston Tan, Jing Jiang
Social learning plays an important role in the development of human intelligence. As children, we imitate our parents' speech patterns until we are able to produce sounds; we learn…
cs.CL2024
Super Tiny Language Models
Dylan Hillier, Leon Guertler, Cheston Tan +3
The rapid advancement of large language models (LLMs) has led to significant improvements in natural language processing but also poses challenges due to their high computational a…