4 papers
Domain Mixture Design via Log-Likelihood Differences for Aligning Language Models with a Target Model
Ryo Kishino, Riku Shiomi, Hiroaki Yamagiwa +2
Instead of directly distilling a language model, this study addresses the problem of aligning a base model with a target model in distribution by designing the domain mixture of tr…
Likelihood Variance as Text Importance for Resampling Texts to Map Language Models
Momose Oyama, Ryo Kishino, Hiroaki Yamagiwa +1
We address the computational cost of constructing a model map, which embeds diverse language models into a common space for comparison via KL divergence. The map relies on log-like…
Establishing a Scale for Kullback-Leibler Divergence in Language Models Across Various Settings
Ryo Kishino, Yusuke Takase, Momose Oyama +2
Log-likelihood vectors define a common space for comparing language models as probability distributions, enabling unified comparisons across heterogeneous settings. We extend this…
Quantifying Lexical Semantic Shift via Unbalanced Optimal Transport
Ryo Kishino, Hiroaki Yamagiwa, Ryo Nagata +2
Lexical semantic change detection aims to identify shifts in word meanings over time. While existing methods using embeddings from a diachronic corpus pair estimate the degree of c…