5 citations · 18 across the 27 of their papers we have counts for
12 papers · 1 filter
Revisiting the Shape Convention of Transformer Language Models
Feng-Ting Liao, Guan-Ting Yi, Tzu-Quan Lin +2
The architectural shape of dense Transformers has remained remarkably stable: narrow-wide-narrow feed-forward networks (FFNs) consume most non-embedding parameters. Motivated by th…
TASTE: Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling
Liang-Hsuan Tseng, Yi-Chang Chen, Kuan-Yi Lee +2
Recent efforts target spoken language models (SLMs) that not only listen but also speak for more natural human-LLM interaction. Joint speech-text modeling is a promising direction…
The Breeze 2 Herd of Models: Traditional Chinese LLMs Based on Llama with Vision-Aware and Function-Calling Capabilities
MediaTek Research, :, Chan-Jan Hsu +6
Llama-Breeze2 (hereinafter referred to as Breeze2) is a suite of advanced multi-modal language models, available in 3B and 8B parameter configurations, specifically designed to enh…
BreezyVoice: Adapting TTS for Taiwanese Mandarin with Enhanced Polyphone Disambiguation -- Challenges and Insights
Chan-Jan Hsu, Yi-Cheng Lin, Chia-Chun Lin +10
We present BreezyVoice, a Text-to-Speech (TTS) system specifically adapted for Taiwanese Mandarin, highlighting phonetic control abilities to address the unique challenges of polyp…
Enhancing Function-Calling Capabilities in LLMs: Strategies for Prompt Formats, Data Integration, and Multilingual Translation
Yi-Chang Chen, Po-Chun Hsu, Chan-Jan Hsu +1
Large language models (LLMs) have significantly advanced autonomous agents, particularly in zero-shot tool usage, also known as function calling. This research delves into enhancin…
FineWeb-zhtw: Scalable Curation of Traditional Chinese Text Data from the Web
Cheng-Wei Lin, Wan-Hsuan Hsieh, Kai-Xin Guan +6
The quality and size of a pretraining dataset significantly influence the performance of large language models (LLMs). While there have been numerous efforts in the curation of suc…