4 papers
State-of-the-art Small Language Coder Model: Mify-Coder
Abhinav Parmar, Abhisek Panigrahi, Abhishek Kumar Dwivedi +93
We present Mify-Coder, a 2.5B-parameter code model trained on 4.2T tokens using a compute-optimal strategy built on the Mify-2.5B foundation model. Mify-Coder achieves comparable a…
Conformal Transformations for Symmetric Power Transformers
Saurabh Kumar, Jacob Buckman, Carles Gelada +1
Transformers with linear attention offer significant computational advantages over softmax-based transformers but often suffer from degraded performance. The symmetric power (sympo…
The Need for a Big World Simulator: A Scientific Challenge for Continual Learning
Saurabh Kumar, Hong Jun Jeon, Alex Lewandowski +1
The "small agent, big world" frame offers a conceptual view that motivates the need for continual learning. The idea is that a small agent operating in a much bigger world cannot s…
Satisficing Exploration for Deep Reinforcement Learning
Dilip Arumugam, Saurabh Kumar, Ramki Gummadi +1
A default assumption in the design of reinforcement-learning algorithms is that a decision-making agent always explores to learn optimal behavior. In sufficiently complex environme…