41 citations · 174 across the 34 of their papers we have counts for
3 papers · 1 filter
Fractional Rotation, Full Potential? Investigating Performance and Convergence of Partial RoPE
Mohammad Aflah Khan, Krishna P. Gummadi, Manish Gupta +1
Rotary Positional Embedding (RoPE) is widely used in transformers to encode relative positional information, yet the effect of applying RoPE to only a subset of hidden dimensions r…
Revisiting the Past: Data Unlearning with Model State History
Keivan Rezaei, Mehrdad Saberi, Abhilasha Ravichander +1
Large language models are trained on massive corpora of web data, which may include private data, copyrighted material, factually inaccurate data, or data that degrades model perfo…
Preserving Intermediate Objectives: One Simple Trick to Improve Learning for Hierarchical Models
Abhilasha Ravichander, Shruti Rijhwani, Rajat Kulshreshtha +3
Hierarchical models are utilized in a wide variety of problems which are characterized by task hierarchies, where predictions on smaller subtasks are useful for trying to predict a…