19 citations · 96 across the 38 of their papers we have counts for
36 papers · 1 filter
Low-Latency Real-Time Audio Game Commentary System via LLM-Based Parallel Text Generation
Ryota Kawamatsu, Anum Afzal, Yuki Saito +5
We present a low-latency real-time audio game commentary system that generates spoken commentary directly from live gameplay video. In this end-to-end setting, a key bottleneck is…
Top-down string-to-dependency Neural Machine Translation
Shuhei Kondo, Katsuhito Sudoh, Yuji Matsumoto
Most of modern neural machine translation (NMT) models are based on an encoder-decoder framework with an attention mechanism. While they perform well on standard datasets, they can…
Real-Time Generation of Game Video Commentary with Multimodal LLMs: Pause-Aware Decoding Approaches
Anum Afzal, Yuki Saito, Hiroya Takamura +5
Real-time video commentary generation provides textual descriptions of ongoing events in videos. It supports accessibility and engagement in domains such as sports, esports, and li…
Findings of the IWSLT 2024 Evaluation Campaign
Ibrahim Said Ahmad, Antonios Anastasopoulos, Ondřej Bojar +42
This paper reports on the shared tasks organized by the 21st IWSLT Conference. The shared tasks address 7 scientific challenges in spoken language translation: simultaneous and off…
FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data
Haoran Sun, Renren Jin, Shaoyang Xu +10
Large language models (LLMs) have demonstrated prowess in a wide range of tasks. However, many LLMs exhibit significant performance discrepancies between high- and low-resource lan…
Word Order in English-Japanese Simultaneous Interpretation: Analyses and Evaluation using Chunk-wise Monotonic Translation
Kosuke Doi, Yuka Ko, Mana Makinae +2
This paper analyzes the features of monotonic translations, which follow the word order of the source language, in simultaneous interpreting (SI). Word order differences are one of…
An Automatic Quality Metric for Evaluating Simultaneous Interpretation
Mana Makinae, Katsuhito Sudoh, Masaru Yamada +1
Simultaneous interpretation (SI), the translation of one language to another in real time, starts translation before the original speech has finished. Its evaluation needs to consi…
NAIST Simultaneous Speech Translation System for IWSLT 2024
Yuka Ko, Ryo Fukuda, Yuta Nishikawa +9
This paper describes NAIST's submission to the simultaneous track of the IWSLT 2024 Evaluation Campaign: English-to-{German, Japanese, Chinese} speech-to-text translation and Engli…
LLMs Are Zero-Shot Context-Aware Simultaneous Translators
Roman Koshkin, Katsuhito Sudoh, Satoshi Nakamura
The advent of transformers has fueled progress in machine translation. More recently large language models (LLMs) have come to the spotlight thanks to their generality and strong p…
Automated Essay Scoring Using Grammatical Variety and Errors with Multi-Task Learning and Item Response Theory
Kosuke Doi, Katsuhito Sudoh, Satoshi Nakamura
This study examines the effect of grammatical features in automatic essay scoring (AES). We use two kinds of grammatical features as input to an AES model: (1) grammatical items th…
Evaluating the IWSLT2023 Speech Translation Tasks: Human Annotations, Automatic Metrics, and Segmentation
Matthias Sperber, Ondřej Bojar, Barry Haddow +8
Human evaluation is a critical component in machine translation system development and has received much attention in text translation research. However, little prior work exists o…
TransLLaMa: LLM-based Simultaneous Translation System
Roman Koshkin, Katsuhito Sudoh, Satoshi Nakamura
Decoder-only large language models (LLMs) have recently demonstrated impressive capabilities in text generation and reasoning. Nonetheless, they have limited applications in simult…
Average Token Delay: A Duration-aware Latency Metric for Simultaneous Translation
Yasumasa Kano, Katsuhito Sudoh, Satoshi Nakamura
Simultaneous translation is a task in which the translation begins before the end of an input speech segment. Its evaluation should be conducted based on latency in addition to qua…
Tagged End-to-End Simultaneous Speech Translation Training using Simultaneous Interpretation Data
Yuka Ko, Ryo Fukuda, Yuta Nishikawa +3
Simultaneous speech translation (SimulST) translates partial speech inputs incrementally. Although the monotonic correspondence between input and output is preferable for smaller l…
NAIST-SIC-Aligned: an Aligned English-Japanese Simultaneous Interpretation Corpus
Jinming Zhao, Yuka Ko, Kosuke Doi +3
It remains a question that how simultaneous interpretation (SI) data affects simultaneous machine translation (SiMT). Research has been limited due to the lack of a large-scale tra…
Modeling Multiple User Interests using Hierarchical Knowledge for Conversational Recommender System
Yuka Okuda, Katsuhito Sudoh, Seitaro Shinagawa +1
A conversational recommender system (CRS) is a practical application for item recommendation through natural language conversation. Such a system estimates user interests for appro…
Evaluating the Robustness of Discrete Prompts
Yoichi Ishibashi, Danushka Bollegala, Katsuhito Sudoh +1
Discrete prompts have been used for fine-tuning Pre-trained Language Models for diverse NLP tasks. In particular, automatic methods that generate discrete prompts from a small set…
Average Token Delay: A Latency Metric for Simultaneous Translation
Yasumasa Kano, Katsuhito Sudoh, Satoshi Nakamura
Simultaneous translation is a task in which translation begins before the speaker has finished speaking. In its evaluation, we have to consider the latency of the translation in ad…
E2E Refined Dataset
Keisuke Toyama, Katsuhito Sudoh, Satoshi Nakamura
Although the well-known MR-to-text E2E dataset has been used by many researchers, its MR-text pairs include many deletion/insertion/substitution errors. Since such errors affect th…
Subspace Representations for Soft Set Operations and Sentence Similarities
Yoichi Ishibashi, Sho Yokoi, Katsuhito Sudoh +1
In the field of natural language processing (NLP), continuous vector representations are crucial for capturing the semantic meanings of individual words. Yet, when it comes to the…
Speech Segmentation Optimization using Segmented Bilingual Speech Corpus for End-to-end Speech Translation
Ryo Fukuda, Katsuhito Sudoh, Satoshi Nakamura
Speech segmentation, which splits long speech into short segments, is essential for speech translation (ST). Popular VAD tools like WebRTC VAD have generally relied on pause-based…
Simultaneous Neural Machine Translation with Constituent Label Prediction
Yasumasa Kano, Katsuhito Sudoh, Satoshi Nakamura
Simultaneous translation is a task in which translation begins before the speaker has finished speaking, so it is important to decide when to start the translation process. However…
Using Perturbed Length-aware Positional Encoding for Non-autoregressive Neural Machine Translation
Yui Oka, Katsuhito Sudoh, Satoshi Nakamura
Non-autoregressive neural machine translation (NAT) usually employs sequence-level knowledge distillation using autoregressive neural machine translation (AT) as its teacher model.…
ARTA: Collection and Classification of Ambiguous Requests and Thoughtful Actions
Shohei Tanaka, Koichiro Yoshino, Katsuhito Sudoh +1
Human-assisting systems such as dialogue systems must take thoughtful, appropriate actions not only for clear and unambiguous user requests, but also for ambiguous user requests, e…