7 papers
HumanoidToolBench: Benchmarking Humanoid Tool Use from Selection to Mobile Execution
Kyochul Jang, Seohyeon Park, Ohchul Kwon +9
As robotic hardware and learning methods advance, humanoids need tools to perform tasks beyond their inherent physical limits. Successful tool use requires selecting a suitable too…
SafeBranch: Branch-Pair Safety Alignment for Embodied Agents
Hyunse Lee, Jiwoo Jeong, Haneul Lee +3
Vision-language-model-based embodied agents can complete instructed tasks but often violate safety constraints in the process, a problem recently framed as interactive safety. Trai…
K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts
Nahyun Lee, Dongkeun Yoon, Guijin Son +12
Frontier model evaluations are shifting from foundational capabilities (e.g., instruction following and reasoning) toward compositional, agentic ones, but Korean agentic benchmarks…
InFerActive: Interactive Tree-Based Exploration of LLM Sampling for Safety Evaluation
Junhyeong Hwangbo, Soohyun Lee, Hyeon Jeon +4
Even LLMs that appear safe during evaluation can still produce harmful responses in deployment. Because stochastic sampling yields different responses to the same prompt, low-proba…
DICE-BENCH: Evaluating the Tool-Use Capabilities of Large Language Models in Multi-Round, Multi-Party Dialogues
Kyochul Jang, Donghyeon Lee, Kyusik Kim +4
Existing function-calling benchmarks focus on single-turn interactions. However, they overlook the complexity of real-world scenarios. To quantify how existing benchmarks address p…
ETHIC: Evaluating Large Language Models on Long-Context Tasks with High Information Coverage
Taewhoo Lee, Chanwoong Yoon, Kyochul Jang +4
Recent advancements in large language models (LLM) capable of processing extremely long texts highlight the need for a dedicated evaluation benchmark to assess their long-context c…
KU-DMIS at EHRSQL 2024:Generating SQL query via question templatization in EHR
Hajung Kim, Chanhwi Kim, Hoonick Lee +5
Transforming natural language questions into SQL queries is crucial for precise data retrieval from electronic health record (EHR) databases. A significant challenge in this proces…