4 citations · 6 across the 5 of their papers we have counts for
7 papers
Fine-Tuning of Transformer models with Frames
Harshavardhan Adepu, Li Zhang, Sanjiv Kumar +1
Parameter-Efficient Fine-Tuning (PEFT) strategies such as Low-Rank Adaptation (LoRA) are effective solutions for fine-tuning large-scale pre-trained models; however, their memory r…
Scalable In-context Ranking with Generative Models
Nilesh Gupta, Chong You, Srinadh Bhojanapalli +3
In-context Ranking (ICR) is an emerging paradigm for Information Retrieval (IR), which leverages contextual understanding of LLMs by directly incorporating the task description, ca…
Hierarchical Retrieval: The Geometry and a Pretrain-Finetune Recipe
Chong You, Rajesh Jayaram, Ananda Theertha Suresh +3
Dual encoder (DE) models, where a pair of matching query and document are embedded into similar vector representations, are widely used in information retrieval due to their simpli…
LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization
Jui-Nan Yen, Si Si, Zhao Meng +5
Low-rank adaption (LoRA) is a widely used parameter-efficient finetuning method for LLM that reduces memory requirements. However, current LoRA optimizers lack transformation invar…
ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agent
Renat Aksitov, Sobhan Miryoosefi, Zonglin Li +10
Answering complex natural language questions often necessitates multi-step reasoning and integrating external information. Several systems have combined knowledge retrieval with a…
It's an Alignment, Not a Trade-off: Revisiting Bias and Variance in Deep Models
Lin Chen, Michal Lukasik, Wittawat Jitkrittum +2
Classical wisdom in machine learning holds that the generalization error can be decomposed into bias and variance, and these two terms exhibit a \emph{trade-off}. However, in this…
Functional Interpolation for Relative Positions Improves Long Context Transformers
Shanda Li, Chong You, Guru Guruganesh +7
Preventing the performance decay of Transformers on inputs longer than those used for training has been an important challenge in extending the context length of these models. Thou…