4 papers
Beyond Length: Quantifying Long-Range Information for Long-Context LLM Pretraining Data
Haoran Deng, Yingyu Lin, Zhenghao Lin +4
Long-context language models unlock advanced capabilities in reasoning, code generation, and document summarization by leveraging dependencies across extended spans of text. Howeve…
Lossless Token Sequence Compression via Meta-Tokens
John Harvill, Ziwei Fan, Hao Wang +4
Existing work on prompt compression for Large Language Models (LLM) focuses on lossy methods that try to maximize the retention of semantic information that is relevant to downstre…
Agree to Disagree? A Meta-Evaluation of LLM Misgendering
Arjun Subramonian, Vagrant Gautam, Preethi Seshadri +3
Numerous methods have been proposed to measure LLM misgendering, including probability-based evaluations (e.g., automatically with templatic sentences) and generation-based evaluat…
Theoretical and Empirical Insights into the Origins of Degree Bias in Graph Neural Networks
Arjun Subramonian, Jian Kang, Yizhou Sun
Graph Neural Networks (GNNs) often perform better for high-degree nodes than low-degree nodes on node classification tasks. This degree bias can reinforce social marginalization by…