8 papers
Establishing a Scale for Kullback-Leibler Divergence in Language Models Across Various Settings
Ryo Kishino, Yusuke Takase, Momose Oyama +2
Log-likelihood vectors define a common space for comparing language models as probability distributions, enabling unified comparisons across heterogeneous settings. We extend this…
Language Model Maps for Prompt-Response Distributions via Log-Likelihood Vectors
Yusuke Takase, Momose Oyama, Hidetoshi Shimodaira
We propose a method that represents language models by log-likelihood vectors over prompt-response pairs and constructs model maps for comparing their conditional distributions. In…
Domain Mixture Design via Log-Likelihood Differences for Aligning Language Models with a Target Model
Ryo Kishino, Riku Shiomi, Hiroaki Yamagiwa +2
Instead of directly distilling a language model, this study addresses the problem of aligning a base model with a target model in distribution by designing the domain mixture of tr…
Mapping 1,000+ Language Models via the Log-Likelihood Vector
Momose Oyama, Hiroaki Yamagiwa, Yusuke Takase +1
To compare autoregressive language models at scale, we propose using log-likelihood vectors computed on a predefined text set as model features. This approach has a solid theoretic…
Predicting drug-gene relations via analogy tasks with word embeddings
Hiroaki Yamagiwa, Ryoma Hashimoto, Kiwamu Arakane +6
Natural language processing (NLP) is utilized in a wide range of fields, where words in text are typically transformed into feature vectors called embeddings. BioConceptVec is a sp…
Likelihood Variance as Text Importance for Resampling Texts to Map Language Models
Momose Oyama, Ryo Kishino, Hiroaki Yamagiwa +1
We address the computational cost of constructing a model map, which embeds diverse language models into a common space for comparison via KL divergence. The map relies on log-like…