8 citations · 8 across the 8 of their papers we have counts for
4 papers · 1 filter
Contrastive Learning-based Sentence Encoders Implicitly Weight Informative Words
Hiroto Kurita, Goro Kobayashi, Sho Yokoi +1
The performance of sentence encoders can be significantly improved through the simple practice of fine-tuning using contrastive loss. A natural question arises: what characteristic…
Assessing Step-by-Step Reasoning against Lexical Negation: A Case Study on Syllogism
Mengyu Ye, Tatsuki Kuribayashi, Jun Suzuki +2
Large language models (LLMs) take advantage of step-by-step reasoning instructions, e.g., chain-of-thought (CoT) prompting. Building on this, their ability to perform CoT-style rea…
Transformer Language Models Handle Word Frequency in Prediction Head
Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi +1
Prediction head is a crucial component of Transformer language models. Despite its direct impact on prediction, this component has often been overlooked in analyzing Transformers.…
Analyzing Feed-Forward Blocks in Transformers through the Lens of Attention Maps
Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi +1
Transformers are ubiquitous in wide tasks. Interpreting their internals is a pivotal goal. Nevertheless, their particular components, feed-forward (FF) blocks, have typically been…