7 papers
How Much Can We Trust LLM Search Agents? Measuring Endorsement Vulnerability to Web Content Manipulation
Yimeng Chen, Zhe Ren, Firas Laakom +3
Large language model (LLM)-based search agents synthesize open-web content into actionable recommendations on behalf of users, creating a risk that attacker-published pages are tra…
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
Planning to Explore: Curiosity-Driven Planning for LLM Test Generation
Alfonso Amayuelas, Firas Laakom, Piotr PiÄkos +5
The use of LLMs for code generation has naturally extended to code testing and evaluation. As codebases grow in size and complexity, so does the need for automated test generation.…
Huxley-Gödel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine
Wenyi Wang, Piotr PiÄkos, Li Nanbo +5
Recent studies operationalize self-improvement through coding agents that edit their own codebases. They grow a tree of self-modifications through expansion strategies that favor h…
PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priors
Yimeng Chen, Piotr Piȩkos, Mateusz Ostaszewski +2
Evaluating the scientific discovery capabilities of large language model based agents, particularly how they cope with varying environmental complexity and utilize prior knowledge,…
Fairness Overfitting in Machine Learning: An Information-Theoretic Perspective
Firas Laakom, Haobo Chen, Jürgen Schmidhuber +1
Despite substantial progress in promoting fairness in high-stake applications using machine learning models, existing methods often modify the training process, such as through reg…