4 citations · 6 across the 14 of their papers we have counts for
16 papers
J-tau: A Fully Localized Japanese Benchmark for Tool-Agent-User Interaction
Chihiro Yano, Jun Hirako, Hiroaki Yamagiwa +1
We present J-tau, the first publicly available Japanese benchmark for evaluating tool-using LLM agents in interactive customer-support scenarios. While tau-bench is widely used to…
Domain Mixture Design via Log-Likelihood Differences for Aligning Language Models with a Target Model
Ryo Kishino, Riku Shiomi, Hiroaki Yamagiwa +2
Instead of directly distilling a language model, this study addresses the problem of aligning a base model with a target model in distribution by designing the domain mixture of tr…
Measuring Affinity between Attention-Head Weight Subspaces via the Projection Kernel
Hiroaki Yamagiwa, Yusuke Takase, Hidetoshi Shimodaira
Understanding relationships between attention heads is essential for interpreting the internal structure of Transformers, yet existing metrics do not capture this structure well. W…
Likelihood Variance as Text Importance for Resampling Texts to Map Language Models
Momose Oyama, Ryo Kishino, Hiroaki Yamagiwa +1
We address the computational cost of constructing a model map, which embeds diverse language models into a common space for comparison via KL divergence. The map relies on log-like…
Establishing a Scale for Kullback-Leibler Divergence in Language Models Across Various Settings
Ryo Kishino, Yusuke Takase, Momose Oyama +2
Log-likelihood vectors define a common space for comparing language models as probability distributions, enabling unified comparisons across heterogeneous settings. We extend this…
Mapping 1,000+ Language Models via the Log-Likelihood Vector
Momose Oyama, Hiroaki Yamagiwa, Yusuke Takase +1
To compare autoregressive language models at scale, we propose using log-likelihood vectors computed on a predefined text set as model features. This approach has a solid theoretic…
Quantifying Lexical Semantic Shift via Unbalanced Optimal Transport
Ryo Kishino, Hiroaki Yamagiwa, Ryo Nagata +2
Lexical semantic change detection aims to identify shifts in word meanings over time. While existing methods using embeddings from a diachronic corpus pair estimate the degree of c…
Understanding Higher-Order Correlations Among Semantic Components in Embeddings
Momose Oyama, Hiroaki Yamagiwa, Hidetoshi Shimodaira
Independent Component Analysis (ICA) offers interpretable semantic components of embeddings. While ICA theory assumes that embeddings can be linearly decomposed into independent co…
Norm of Mean Contextualized Embeddings Determines their Variance
Hiroaki Yamagiwa, Hidetoshi Shimodaira
Contextualized embeddings vary by context, even for the same token, and form a distribution in the embedding space. To analyze this distribution, we focus on the norm of the mean e…
Shimo Lab at "Discharge Me!": Discharge Summarization by Prompt-Driven Concatenation of Electronic Health Record Sections
Yunzhen He, Hiroaki Yamagiwa, Hidetoshi Shimodaira
In this paper, we present our approach to the shared task "Discharge Me!" at the BioNLP Workshop 2024. The primary goal of this task is to reduce the time and effort clinicians spe…
Revisiting Cosine Similarity via Normalized ICA-transformed Embeddings
Hiroaki Yamagiwa, Momose Oyama, Hidetoshi Shimodaira
Cosine similarity is widely used to measure the similarity between two embeddings, while interpretations based on angle and correlation coefficient are common. In this study, we fo…
Predicting drug-gene relations via analogy tasks with word embeddings
Hiroaki Yamagiwa, Ryoma Hashimoto, Kiwamu Arakane +6
Natural language processing (NLP) is utilized in a wide range of fields, where words in text are typically transformed into feature vectors called embeddings. BioConceptVec is a sp…
Axis Tour: Word Tour Determines the Order of Axes in ICA-transformed Embeddings
Hiroaki Yamagiwa, Yusuke Takase, Hidetoshi Shimodaira
Word embedding is one of the most important components in natural language processing, but interpreting high-dimensional embeddings remains a challenging problem. To address this p…
Zero-Shot Edge Detection with SCESAME: Spectral Clustering-based Ensemble for Segment Anything Model Estimation
Hiroaki Yamagiwa, Yusuke Takase, Hiroyuki Kambe +1
This paper proposes a novel zero-shot edge detection with SCESAME, which stands for Spectral Clustering-based Ensemble for Segment Anything Model Estimation, based on the recently…
Discovering Universal Geometry in Embeddings with ICA
Hiroaki Yamagiwa, Momose Oyama, Hidetoshi Shimodaira
This study utilizes Independent Component Analysis (ICA) to unveil a consistent semantic structure within embeddings of words or images. Our approach extracts independent semantic…
Improving word mover's distance by leveraging self-attention matrix
Hiroaki Yamagiwa, Sho Yokoi, Hidetoshi Shimodaira
Measuring the semantic similarity between two sentences is still an important task. The word mover's distance (WMD) computes the similarity via the optimal alignment between the se…