52 citations · 54 across the 8 of their papers we have counts for
9 papers
Aligning Training and Rollout Policies for End-to-End FP4 RL on Mixture-of-Experts LLMs
Hei Yi Mak, Hoang Le, Mehran Taghian Jazi +8
In this work, we study end-to-end FP4 reinforcement learning (RL) post-training for Mixture-of-Experts (MoE) large language models. Recent work on FP4 RL has introduced effective r…
Activation Outliers Matter: Robust Recovery for Quantized Multimodal LLMs
Tanzila Rahman, Mehran Taghian Jazi, Yunke Peng +10
Low-bit quantization offers a promising avenue for reducing the computational and memory demands of Multimodal Large Language Models (MLLMs). Recent hardware support for low-precis…
HiFloat4 Format for End-To-End Reinforcement Learning Post-Training of Large Language Models
Hei Yi Mak, Shadan Golestan, Hoang Le +10
We present, to our knowledge, the first end-to-end FP4 RL post-training, in which both the rollout and training policies, including their forward and backward passes, operate at 4-…
TabEmb: Joint Semantic-Structure Embedding for Table Annotation
Ehsan Hoseinzade, Ke Wang, Anandharaju Durai Raju
Table annotation is crucial for making web and enterprise tables usable in downstream NLP applications. Unlike textual data where learning semantically rich token or sentence embed…
HiFloat4 Format for Language Model Pre-training on Ascend NPUs
Mehran Taghian, Yunke Peng, Xing Huang +22
Training large foundation models at low numerical precision is one of the most promising directions for reducing the compute and memory cost of modern AI. Recent 4-bit floating-poi…
On building machine learning pipelines for Android malware detection: a procedural survey of practices, challenges and opportunities
Masoud Mehrabi Koushki, Ibrahim AbuAlhaol, Anandharaju Durai Raju +3
As the smartphone market leader, Android has been a prominent target for malware attacks. The number of malicious applications (apps) identified for it has increased continually ov…
A Survey on Cross-Architectural IoT Malware Threat Hunting
Anandharaju Durai Raju, Ibrahim Abualhaol, Ronnie Salvador Giagone +2
In recent years, the increase in non-Windows malware threats had turned the focus of the cybersecurity community. Research works on hunting Windows PE-based malwares are maturing,…
Echelon: Two-Tier Malware Detection for Raw Executables to Reduce False Alarms
Anandharaju Durai Raju, Ke Wang
Existing malware detection approaches suffer from a simplistic trade-off between false positive rate (FPR) and true positive rate (TPR) due to a single tier classification approach…
FAT ALBERT: Finding Answers in Large Texts using Semantic Similarity Attention Layer based on BERT
Omar Mossad, Amgad Ahmed, Anandharaju Raju +2
Machine based text comprehension has always been a significant research field in natural language processing. Once a full understanding of the text context and semantics is achieve…