5 papers · 1 filter
When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework
Zhen Xu, Shang Zhu, Jue Wang +5
We investigate the challenge of applying Large Language Models (LLMs) to long texts. We propose a theoretical framework that distinguishes the failure modes of long context tasks i…
Memorization Dynamics in Knowledge Distillation for Language Models
Jaydeep Borkar, Karan Chadha, Niloofar Mireshghallah +6
Knowledge Distillation (KD) is increasingly adopted to transfer capabilities from large language models to smaller ones, offering significant improvements in efficiency and utility…
Enhancing LLM-Based Data Annotation with Error Decomposition
Zhen Xu, Vedant Khatri, Yijun Dai +4
Large language models offer a scalable alternative to human coding for data annotation tasks, enabling the scale-up of research across data-intensive domains. While LLMs are alread…
Scalable In-Context Learning on Tabular Data via Retrieval-Augmented Large Language Models
Xumeng Wen, Shun Zheng, Zhen Xu +2
Recent studies have shown that large language models (LLMs), when customized with post-training on tabular data, can acquire general tabular in-context learning (TabICL) capabiliti…
Usefulness of LLMs as an Author Checklist Assistant for Scientific Papers: NeurIPS'24 Experiment
Alexander Goldberg, Ihsan Ullah, Thanh Gia Hieu Khuong +4
Large language models (LLMs) represent a promising, but controversial, tool in aiding scientific peer review. This study evaluates the usefulness of LLMs in a conference setting as…