3 citations · 5 across the 23 of their papers we have counts for
7 papers · 1 filter
Train Smarter, Not Longer: Memorization-Guided Data Reuse for Efficient LLM Training
Jingwei Zuo, Cong Zeng, Ilyas Chahed +6
The training paradigm of large language models has shifted from traditional one-pass training to multi-epoch training, as reasonable reuse of limited high-quality data can improve…
Learnable Multipliers: Freeing the Scale of Language Model Matrix Layers
Maksim Velikanov, Ilyas Chahed, Jingwei Zuo +3
Applying weight decay (WD) to matrix layers is standard practice in large-language-model pretraining. Prior work suggests that stochastic gradient noise induces a Brownian-like exp…
Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory
Aymane El Firdoussi, Mohamed El Amine Seddik, Soufiane Hayou +3
Synthetic data has gained attention for training large language models, but poor-quality data can harm performance (see, e.g., Shumailov et al. (2023); Seddik et al. (2024)). A pot…
Alignment with Preference Optimization Is All You Need for LLM Safety
Reda Alami, Ali Khalifa Almansoori, Ahmed Alzubaidi +3
We demonstrate that preference optimization methods can effectively enhance LLM safety. Applying various alignment techniques to the Falcon 11B model using safety datasets, we achi…
MAGNETO: Edge AI for Human Activity Recognition -- Privacy and Personalization
Jingwei Zuo, George Arvanitakis, Mthandazo Ndhlovu +1
Human activity recognition (HAR) is a well-established field, significantly advanced by modern machine learning (ML) techniques. While companies have successfully integrated HAR in…
Opportunistic Air Quality Monitoring and Forecasting with Expandable Graph Neural Networks
Jingwei Zuo, Wenbin Li, Michele Baldo +1
Air Quality Monitoring and Forecasting has been a popular research topic in recent years. Recently, data-driven approaches for air quality forecasting have garnered significant att…
Unleashing Realistic Air Quality Forecasting: Introducing the Ready-to-Use PurpleAirSF Dataset
Jingwei Zuo, Wenbin Li, Michele Baldo +1
Air quality forecasting has garnered significant attention recently, with data-driven models taking center stage due to advancements in machine learning and deep learning models. H…