5 papers
Incorporating Domain Knowledge into Materials Tokenization
Yerim Oh, Jun-Hyung Park, Junho Kim +2
While language models are increasingly utilized in materials science, typical models rely on frequency-centric tokenization methods originally developed for natural language proces…
C2A: Client-Customized Adaptation for Parameter-Efficient Federated Learning
Yeachan Kim, Junho Kim, Wing-Lam Mok +2
Despite the versatility of pre-trained language models (PLMs) across domains, their large memory footprints pose significant challenges in federated learning (FL), where the traini…
CleaR: Towards Robust and Generalized Parameter-Efficient Fine-Tuning for Noisy Label Learning
Yeachan Kim, Junho Kim, SangKeun Lee
Parameter-efficient fine-tuning (PEFT) has enabled the efficient optimization of cumbersome language models in real-world settings. However, as datasets in such environments often…
MELT: Materials-aware Continued Pre-training for Language Model Adaptation to Materials Science
Junho Kim, Yeachan Kim, Jun-Hyung Park +3
We introduce a novel continued pre-training method, MELT (MatEriaLs-aware continued pre-Training), specifically designed to efficiently adapt the pre-trained language models (PLMs)…
Mentor-KD: Making Small Language Models Better Multi-step Reasoners
Hojae Lee, Junho Kim, SangKeun Lee
Large Language Models (LLMs) have displayed remarkable performances across various complex tasks by leveraging Chain-of-Thought (CoT) prompting. Recently, studies have proposed a K…