6 citations · 18 across the 21 of their papers we have counts for
23 papers
SkillGym: Training Skill-Use Agents with Automatic Verifiable Environment Generation
Renxi Wang, Mingshan Hee, Fajri Koto +2
Skills equip LLM agents with professional knowledge and guidance to complete long-horizon and complex tasks. Although skills have been widely adopted in recent agent paradigms and…
From Leaky Thoughts to Private Reasoning: Controlling What LRMs Say to Themselves
Haritz Puerto, Haonan Li, Xudong Han +2
Large reasoning models (LRMs) produce reasoning traces (RTs) that often contain sensitive information. These leaky thoughts are difficult to control and frequently violate explicit…
Training and Benchmarking Code Generation for Physics-Inspired Animations
Yanan Wang, Renxi Wang, Yongxin Wang +5
Large language models (LLMs) have been widely studied in areas such as mathematical reasoning, complex coding, and scientific problem solving. However, their ability to generate ex…
Cross-Cultural Transfer of Commonsense Reasoning in LLMs: Evidence from the Arab World
Saeed Almheiri, Rania Hossam, Mena Attia +4
Large language models (LLMs) often reflect Western-centric biases, limiting their effectiveness in diverse cultural contexts. Although some work has explored cultural alignment, th…
AgentFly: Scaling Agentic Reinforcement Learning with Unified Resource System
Renxi Wang, Rifo Ahmad Genadi, Bilal El Bouardi +6
Methods to build LLM agents have evolved from prompt engineering and supervised finetuning to agentic reinforcement learning (agentic RL). However, agentic RL remains bottlenecked…
RuozhiBench: Evaluating LLMs with Logical Fallacies and Misleading Premises
Zenan Zhai, Hao Li, Xudong Han +4
Recent advances in large language models (LLMs) have shown that they can answer questions requiring complex reasoning. However, their ability to identify and respond to text contai…
SCALAR: Scientific Citation-based Live Assessment of Long-context Academic Reasoning
Renxi Wang, Honglin Mu, Liqun Ma +5
Long-context understanding has emerged as a critical capability for large language models (LLMs). However, evaluating this ability remains challenging. We present SCALAR, a benchma…
Qorgau: Evaluating LLM Safety in Kazakh-Russian Bilingual Contexts
Maiya Goloburda, Nurkhan Laiyk, Diana Turmakhan +11
Large language models (LLMs) are known to have the potential to generate harmful content, posing risks to users. While significant progress has been made in developing taxonomies f…
Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety
Xingjun Ma, Yifeng Gao, Yixu Wang +45
The rapid advancement of large models, driven by their exceptional abilities in learning and generalization through large-scale pre-training, has reshaped the landscape of Artifici…
Loki: An Open-Source Tool for Fact Verification
Haonan Li, Xudong Han, Hao Wang +7
We introduce Loki, an open-source tool designed to address the growing problem of misinformation. Loki adopts a human-centered approach, striking a balance between the quality of f…
Arabic Dataset for LLM Safeguard Evaluation
Yasser Ashraf, Yuxia Wang, Bin Gu +2
The growing use of large language models (LLMs) has raised concerns regarding their safety. While many studies have focused on English, the safety of LLMs in Arabic, with its lingu…
ToolGen: Unified Tool Retrieval and Calling via Generation
Renxi Wang, Xudong Han, Lei Ji +3
As large language models (LLMs) advance, their inability to autonomously execute tasks by directly interacting with external tools remains a critical limitation. Traditional method…
Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs
Sukmin Yun, Haokun Lin, Rusiru Thushara +14
Multimodal large language models (MLLMs) have shown impressive success across modalities such as image, video, and audio in a variety of understanding and generation tasks. However…
IndoCulture: Exploring Geographically-Influenced Cultural Commonsense Reasoning Across Eleven Indonesian Provinces
Fajri Koto, Rahmad Mahendra, Nurul Aisyah +1
Although commonsense reasoning is greatly shaped by cultural and geographical factors, previous studies have predominantly centered on cultures grounded in the English language, po…
Against The Achilles' Heel: A Survey on Red Teaming for Generative Models
Lizhi Lin, Honglin Mu, Zenan Zhai +9
Generative models are rapidly gaining popularity and being integrated into everyday applications, raising concerns over their safe use as various vulnerabilities are exposed. In li…
Zero-shot Sentiment Analysis in Low-Resource Languages Using a Multilingual Sentiment Lexicon
Fajri Koto, Tilman Beck, Zeerak Talat +2
Improving multilingual language models capabilities in low-resource languages is generally difficult due to the scarcity of large-scale data in those languages. In this paper, we r…
ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic
Fajri Koto, Haonan Li, Sara Shatnawi +10
The focus of language model evaluation has transitioned towards reasoning and knowledge-intensive tasks, driven by advancements in pretraining large models. While state-of-the-art…
A Chinese Dataset for Evaluating the Safeguards in Large Language Models
Yuxia Wang, Zenan Zhai, Haonan Li +6
Many studies have demonstrated that large language models (LLMs) can produce harmful responses, exposing users to unexpected risks when LLMs are deployed. Previous studies have pro…
Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language Models as Agents
Renxi Wang, Haonan Li, Xudong Han +2
Large language models (LLMs) have achieved success in acting as agents, which interact with environments through tools such as search engines. However, LLMs are optimized for langu…
Location Aware Modular Biencoder for Tourism Question Answering
Haonan Li, Martin Tomko, Timothy Baldwin
Answering real-world tourism questions that seek Point-of-Interest (POI) recommendations is challenging, as it requires both spatial and non-spatial reasoning, over a large candida…
LLM360: Towards Fully Transparent Open-Source LLMs
Zhengzhong Liu, Aurick Qiao, Willie Neiswanger +25
The recent surge in open-source Large Language Models (LLMs), such as LLaMA, Falcon, and Mistral, provides diverse options for AI practitioners and researchers. However, most LLMs…
Demystifying Instruction Mixing for Fine-tuning Large Language Models
Renxi Wang, Haonan Li, Minghao Wu +4
Instruction tuning significantly enhances the performance of large language models (LLMs) across various tasks. However, the procedure to optimizing the mixing of instruction datas…
Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU
Fajri Koto, Nurul Aisyah, Haonan Li +1
Although large language models (LLMs) are often pre-trained on large-scale multilingual texts, their reasoning abilities and real-world knowledge are mainly evaluated based on Engl…