5 papers
MUON+: Towards More Effective Muon via One Additional Normalization Step for LLM Pre-training
Ruijie Zhang, Yequan Zhao, Ziyue Liu +4
Muon has recently emerged as a strong optimizer for large language model pre-training, orthogonalizing the momentum matrix via Newton--Schulz polar iterations. A natural intuition…
Investigating Hallucination in Conversations for Low Resource Languages
Amit Das, Md. Najib Hasan, Souvika Sarkar +7
Large Language Models (LLMs) have demonstrated remarkable proficiency in generating text that closely resemble human writing. However, they often generate factually incorrect state…
LLM for Complex Reasoning Task: An Exploratory Study in Fermi Problems
Zishuo Liu, Carlos Rabat Villarreal, Mostafa Rahgouy +4
Fermi Problems (FPs) are mathematical reasoning tasks that require human-like logic and numerical reasoning. Unlike other reasoning questions, FPs often involve real-world impracti…
OffensiveLang: A Community Based Implicit Offensive Language Dataset
Amit Das, Mostafa Rahgouy, Dongji Feng +10
The widespread presence of hateful languages on social media has resulted in adverse effects on societal well-being. As a result, addressing this issue with high priority has becom…
Investigating Annotator Bias in Large Language Models for Hate Speech Detection
Amit Das, Zheng Zhang, Najib Hasan +12
Data annotation, the practice of assigning descriptive labels to raw data, is pivotal in optimizing the performance of machine learning models. However, it is a resource-intensive…