2 citations · 2 across the 5 of their papers we have counts for
8 papers
JevGPT: Thinking Fast and Slow in Autonomous Research
Yasheng Sun, Yanning Dai, Yimeng Chen +9
Autonomous research agents must acquire and reason over evidence within a limited budget. Yet relevant sources can remain uninspected or unused, leaving a gap between retrieval cov…
History Replays Itself: Unleashing Long-Context Capability in Linear Attention
Li Nanbo, Firas Laakom, Yilan Zhang +10
Linear attention promises the efficiency needed for long-context language modelling, yet its fixed-size recurrent state must simultaneously support ongoing computation and preserve…
Planning to Explore: Curiosity-Driven Planning for LLM Test Generation
Alfonso Amayuelas, Firas Laakom, Piotr Piękos +5
The use of LLMs for code generation has naturally extended to code testing and evaluation. As codebases grow in size and complexity, so does the need for automated test generation.…
PDR: A Plug-and-Play Positional Decay Framework for LLM Pre-training Data Detection
Jinhan Liu, Yibo Yang, Ruiying Lu +4
Detecting pre-training data in Large Language Models (LLMs) is crucial for auditing data privacy and copyright compliance, yet it remains challenging in black-box, zero-shot settin…
PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priors
Yimeng Chen, Piotr Piȩkos, Mateusz Ostaszewski +2
Evaluating the scientific discovery capabilities of large language model based agents, particularly how they cope with varying environmental complexity and utilize prior knowledge,…
Hyperbolic Residual Quantization: Discrete Representations for Data with Latent Hierarchies
Piotr Piękos, Subhradeep Kayal, Alexandros Karatzoglou
Hierarchical data arise in countless domains, from biological taxonomies and organizational charts to legal codes and knowledge graphs. Residual Quantization (RQ) is widely used to…
Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via Expert-Choice Routing
Piotr Piękos, Róbert Csordás, Jürgen Schmidhuber
Recent advances in large language models highlighted the excessive quadratic cost of self-attention. Despite the significant research efforts, subquadratic attention methods still…
SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention
Róbert Csordás, Piotr Piękos, Kazuki Irie +1
Despite many recent works on Mixture of Experts (MoEs) for resource-efficient Transformer language models, existing methods mostly focus on MoEs for feedforward layers. Previous at…