52 citations · 56 across the 8 of their papers we have counts for
8 papers
Revisiting the Shape Convention of Transformer Language Models
Feng-Ting Liao, Guan-Ting Yi, Tzu-Quan Lin +2
The architectural shape of dense Transformers has remained remarkably stable: narrow-wide-narrow feed-forward networks (FFNs) consume most non-embedding parameters. Motivated by th…
Rethinking the shape convention of an MLP
Meng-Hsi Chen, Yu-Ang Lee, Feng-Ting Liao +1
Multi-layer perceptrons (MLPs) conventionally follow a narrow-wide-narrow design where skip connections operate at the input/output dimensions while processing occurs in expanded h…
RISE: Reasoning Enhancement via Iterative Self-Exploration in Multi-hop Question Answering
Bolei He, Xinran He, Mengke Chen +3
Large Language Models (LLMs) excel in many areas but continue to face challenges with complex reasoning tasks, such as Multi-Hop Question Answering (MHQA). MHQA requires integratin…
Latent Flow Transformer
Yen-Chen Wu, Feng-Ting Liao, Meng-Hsi Chen +3
Transformers, the standard implementation for large language models (LLMs), typically consist of tens to hundreds of discrete layers. While more layers can lead to better performan…
The Breeze 2 Herd of Models: Traditional Chinese LLMs Based on Llama with Vision-Aware and Function-Calling Capabilities
MediaTek Research, :, Chan-Jan Hsu +6
Llama-Breeze2 (hereinafter referred to as Breeze2) is a suite of advanced multi-modal language models, available in 3B and 8B parameter configurations, specifically designed to enh…
The Shrinking Landscape of Linguistic Diversity in the Age of Large Language Models
Zhivar Sourati, Farzan Karimi-Malekabadi, Meltem Ozcan +7
Language is far more than a communication tool; it encodes a wealth of information about a person's identity, psychological state, and social context, providing valuable insights f…
Mastering the Craft of Data Synthesis for CodeLLMs
Meng Chen, Philip Arthur, Qianyu Feng +13
Large language models (LLMs) have shown impressive performance in \emph{code} understanding and generation, making coding tasks a key focus for researchers due to their practical a…
On the Effectiveness of Large Language Models in Domain-Specific Code Generation
Xiaodong Gu, Meng Chen, Yalan Lin +6
Large language models (LLMs) such as ChatGPT have shown remarkable capabilities in code generation. Despite significant achievements, they rely on enormous training data to acquire…