2 citations · 20 across the 53 of their papers we have counts for
4 papers · 1 filter
Decoders Laugh as Loud as Encoders
Eli Borodach, Raj Dandekar, Rajat Dandekar +1
From the dawn of the computer, Allen Turing dreamed of a robot that could communicate using language as a human being. The recent advances in the field of Large Language Models (LL…
Latent Multi-Head Attention for Small Language Models
Sushant Mehta, Raj Dandekar, Rajat Dandekar +1
We present the first comprehensive study of latent multi-head attention (MLA) for small language models, revealing interesting efficiency-quality trade-offs. Training 30M-parameter…
Regional Tiny Stories: Using Small Models to Compare Language Learning and Tokenizer Performance
Nirvan Patil, Malhar Abhay Inamdar, Agnivo Gosai +7
Small Language Models (SLMs) offer efficient alternatives to LLMs for specific domains. The 2023 TinyStories study developed an English dataset that allows SLMs with 1 to 10 millio…
CBEval: A framework for evaluating and interpreting cognitive biases in LLMs
Ammar Shaikh, Raj Abhijit Dandekar, Sreedath Panat +1
Rapid advancements in Large Language models (LLMs) has significantly enhanced their reasoning capabilities. Despite improved performance on benchmarks, LLMs exhibit notable gaps in…