12 citations · 25 across the 7 of their papers we have counts for
12 papers
Diverse Lottery Tickets Boost Ensemble from a Single Pretrained Model
Sosuke Kobayashi, Shun Kiyono, Jun Suzuki +1
Ensembling is a popular method used to improve performance as a last resort. However, ensembling multiple models finetuned from a single pretrained model has been not very effectiv…
SHAPE: Shifted Absolute Position Embedding for Transformers
Shun Kiyono, Sosuke Kobayashi, Jun Suzuki +1
Position representation is crucial for building position-aware representations in Transformers. Existing position representations suffer from a lack of generalization to test data…
Rethinking Perturbations in Encoder-Decoders for Fast Training
Sho Takase, Shun Kiyono
We often use perturbations to regularize neural models. For neural encoder-decoders, previous studies applied the scheduled sampling (Bengio et al., 2015) and adversarial perturbat…
Pseudo Zero Pronoun Resolution Improves Zero Anaphora Resolution
Ryuto Konno, Shun Kiyono, Yuichiroh Matsubayashi +2
Masked language models (MLMs) have contributed to drastic performance improvements with regard to zero anaphora resolution (ZAR). To further improve this approach, in this study, w…
An Empirical Study of Contextual Data Augmentation for Japanese Zero Anaphora Resolution
Ryuto Konno, Yuichiroh Matsubayashi, Shun Kiyono +3
One critical issue of zero anaphora resolution (ZAR) is the scarcity of labeled data. This study explores how effectively this problem can be alleviated by data augmentation. We ad…
A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction
Masato Mita, Shun Kiyono, Masahiro Kaneko +2
Existing approaches for grammatical error correction (GEC) largely rely on supervised learning with manually created GEC datasets. However, there has been little focus on verifying…