6 papers
Motif-2-12.7B-Reasoning: A Practitioner's Guide to RL Training Recipes
Junghwan Lim, Sungmin Lee, Dongseok Kim +23
We introduce Motif-2-12.7B-Reasoning, a 12.7B parameter language model designed to bridge the gap between open-weight systems and proprietary frontier models in complex reasoning a…
Motif 2 12.7B technical report
Junghwan Lim, Sungmin Lee, Dongseok Kim +22
We introduce Motif-2-12.7B, a new open-weight foundation model that pushes the efficiency frontier of large language models by combining architectural innovation with system-level…
Grouped Differential Attention
Junghwan Lim, Sungmin Lee, Dongseok Kim +7
The self-attention mechanism, while foundational to modern Transformer architectures, suffers from a critical inefficiency: it frequently allocates substantial attention to redunda…
Expanding Foundational Language Capabilities in Open-Source LLMs through a Korean Case Study
Junghwan Lim, Gangwon Jo, Sungmin Lee +16
We introduce Llama-3-Motif, a language model consisting of 102 billion parameters, specifically designed to enhance Korean capabilities while retaining strong performance in Englis…
Motif 2.6B Technical Report
Junghwan Lim, Sungmin Lee, Dongseok Kim +22
Recent advancements in Large Language Models (LLMs) have revolutionized artificial intelligence, yet developing an effective foundational LLM that balances high performance with co…
Large Language Model-based Role-Playing for Personalized Medical Jargon Extraction
Jung Hoon Lim, Sunjae Kwon, Zonghai Yao +2
Previous studies reveal that Electronic Health Records (EHR), which have been widely adopted in the U.S. to allow patients to access their personal medical information, do not have…