3 citations · 7 across the 13 of their papers we have counts for
14 papers
Train Smarter, Not Longer: Memorization-Guided Data Reuse for Efficient LLM Training
Jingwei Zuo, Cong Zeng, Ilyas Chahed +6
The training paradigm of large language models has shifted from traditional one-pass training to multi-epoch training, as reasonable reuse of limited high-quality data can improve…
Learnable Multipliers: Freeing the Scale of Language Model Matrix Layers
Maksim Velikanov, Ilyas Chahed, Jingwei Zuo +3
Applying weight decay (WD) to matrix layers is standard practice in large-language-model pretraining. Prior work suggests that stochastic gradient noise induces a Brownian-like exp…
Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performance
Jingwei Zuo, Maksim Velikanov, Ilyas Chahed +24
In this report, we introduce Falcon-H1, a new series of large language models (LLMs) featuring hybrid architecture designs optimized for both high performance and efficiency across…
NeurIPS 2025 E2LM Competition : Early Training Evaluation of Language Models
Mouadh Yagoubi, Yasser Dahou, Billel Mokeddem +12
Existing benchmarks have proven effective for assessing the performance of fully trained large language models. However, we find striking differences in the early training stages o…
Falcon Mamba: The First Competitive Attention-free 7B Language Model
Jingwei Zuo, Maksim Velikanov, Dhia Eddine Rhaiem +4
In this technical report, we present Falcon Mamba 7B, a new base large language model based on the novel Mamba architecture. Falcon Mamba 7B is trained on 5.8 trillion tokens with…
MAGNETO: Edge AI for Human Activity Recognition -- Privacy and Personalization
Jingwei Zuo, George Arvanitakis, Mthandazo Ndhlovu +1
Human activity recognition (HAR) is a well-established field, significantly advanced by modern machine learning (ML) techniques. While companies have successfully integrated HAR in…
Re-thinking Human Activity Recognition with Hierarchy-aware Label Relationship Modeling
Jingwei Zuo, Hakim Hacid
Human Activity Recognition (HAR) has been studied for decades, from data collection, learning models, to post-processing and result interpretations. However, the inherent hierarchy…
Practical Insights on Incremental Learning of New Human Physical Activity on the Edge
George Arvanitakis, Jingwei Zuo, Mthandazo Ndhlovu +1
Edge Machine Learning (Edge ML), which shifts computational intelligence from cloud-based systems to edge devices, is attracting significant interest due to its evident benefits in…
AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors
Weize Chen, Yusheng Su, Jingwei Zuo +13
Autonomous agents empowered by Large Language Models (LLMs) have undergone significant improvements, enabling them to generalize across a broad spectrum of tasks. However, in real-…
Opportunistic Air Quality Monitoring and Forecasting with Expandable Graph Neural Networks
Jingwei Zuo, Wenbin Li, Michele Baldo +1
Air Quality Monitoring and Forecasting has been a popular research topic in recent years. Recently, data-driven approaches for air quality forecasting have garnered significant att…
Unleashing Realistic Air Quality Forecasting: Introducing the Ready-to-Use PurpleAirSF Dataset
Jingwei Zuo, Wenbin Li, Michele Baldo +1
Air quality forecasting has garnered significant attention recently, with data-driven models taking center stage due to advancements in machine learning and deep learning models. H…
On Handling Catastrophic Forgetting for Incremental Learning of Human Physical Activity on the Edge
Jingwei Zuo, George Arvanitakis, Hakim Hacid
Human activity recognition (HAR) has been a classic research problem. In particular, with recent machine learning (ML) techniques, the recognition task has been largely investigate…
Graph Convolutional Networks for Traffic Forecasting with Missing Values
Jingwei Zuo, Karine Zeitouni, Yehia Taher +1
Traffic forecasting has attracted widespread attention recently. In reality, traffic data usually contains missing values due to sensor or communication errors. The Spatio-temporal…
SMATE: Semi-Supervised Spatio-Temporal Representation Learning on Multivariate Time Series
Jingwei Zuo, Karine Zeitouni, Yehia Taher
Learning from Multivariate Time Series (MTS) has attracted widespread attention in recent years. In particular, label shortage is a real challenge for the classification task on MT…