4 papers
J-tau: A Fully Localized Japanese Benchmark for Tool-Agent-User Interaction
Chihiro Yano, Jun Hirako, Hiroaki Yamagiwa +1
We present J-tau, the first publicly available Japanese benchmark for evaluating tool-using LLM agents in interactive customer-support scenarios. While tau-bench is widely used to…
FrameBench:A Language Understanding Benchmark Based on Frame Semantics
Chihiro Yano, Ryohei Sasano
In frame semantics, sentence comprehension is assumed to proceed by relating lexical meaning to background knowledge called semantic frames, thereby enabling readers to implicitly…
FrameEOL: Semantic Frame Induction using Causal Language Models
Chihiro Yano, Kosuke Yamada, Hayato Tsukagoshi +2
Semantic frame induction is the task of clustering frame-evoking words according to the semantic frames they evoke. In recent years, leveraging embeddings of frame-evoking words th…
Multilingual Sentence-T5: Scalable Sentence Encoders for Multilingual Applications
Chihiro Yano, Akihiko Fukuchi, Shoko Fukasawa +2
Prior work on multilingual sentence embedding has demonstrated that the efficient use of natural language inference (NLI) data to build high-performance models can outperform conve…