112 citations · 155 across the 22 of their papers we have counts for
15 papers · 1 filter
Collaboratively adding new knowledge to an LLM
Rhui Dih Lee, Laura Wynter
We address the question of how to successively add new knowledge to an LLM whilst retaining previously-added knowledge. We consider two settings, semi-cooperative and fully-coopera…
Enhancing Training Efficiency Using Packing with Flash Attention
Achintya Kundu, Rhui Dih Lee, Laura Wynter +2
Padding is often used in tuning LLM models by adding special tokens to shorter training examples to match the length of the longest sequence in each batch. While this ensures unifo…
Efficiently Distilling LLMs for Edge Applications
Achintya Kundu, Fabian Lim, Aaron Chew +3
Supernet training of LLMs is of great interest in industrial applications as it confers the ability to produce a palette of smaller models at constant cost, regardless of the numbe…
Transfer-Once-For-All: AI Model Optimization for Edge
Achintya Kundu, Laura Wynter, Rhui Dih Lee +1
Weight-sharing neural architecture search aims to optimize a configurable neural network model (supernet) for a variety of deployment scenarios across many devices with different r…
Neural-Progressive Hedging: Enforcing Constraints in Reinforcement Learning with Stochastic Programming
Supriyo Ghosh, Laura Wynter, Shiau Hong Lim +1
We propose a framework, called neural-progressive hedging (NP), that leverages stochastic programming during the online phase of executing a reinforcement learning (RL) policy. The…
Order Constraints in Optimal Transport
Fabian Lim, Laura Wynter, Shiau Hong Lim
Optimal transport is a framework for comparing measures whereby a cost is incurred for transporting one measure to another. Recent works have aimed to improve optimal transport pla…
Decentralized Deterministic Multi-Agent Reinforcement Learning
Antoine Grosnit, Desmond Cai, Laura Wynter
[Zhang, ICML 2018] provided the first decentralized actor-critic algorithm for multi-agent reinforcement learning (MARL) that offers convergence guarantees. In that work, policies…
Efficient Reinforcement Learning in Resource Allocation Problems Through Permutation Invariant Multi-task Learning
Desmond Cai, Shiau Hong Lim, Laura Wynter
One of the main challenges in real-world reinforcement learning is to learn successfully from limited training samples. We show that in certain settings, the available data can be…
Probabilistic Inference for Learning from Untrusted Sources
Duc Thien Nguyen, Shiau Hoong Lim, Laura Wynter +1
Federated learning brings potential benefits of faster learning, better solutions, and a greater propensity to transfer when heterogeneous data from different parties increases div…
Robustness and Personalization in Federated Learning: A Unified Approach via Regularization
Achintya Kundu, Pengqian Yu, Laura Wynter +1
We present a class of methods for robust, personalized federated learning, called Fed+, that unifies many federated learning algorithms. The principal advantage of this class of me…
IBM Federated Learning: an Enterprise Framework White Paper V0.1
Heiko Ludwig, Nathalie Baracaldo, Gegi Thomas +21
Federated Learning (FL) is an approach to conduct machine learning without centralizing training data in a single place, for reasons of privacy, confidentiality or data volume. How…
Variational Bayesian Inference for Crowdsourcing Predictions
Desmond Cai, Duc Thien Nguyen, Shiau Hong Lim +1
Crowdsourcing has emerged as an effective means for performing a number of machine learning tasks such as annotation and labelling of images and other data sets. In most early sett…
A Deep Ensemble Multi-Agent Reinforcement Learning Approach for Air Traffic Control
Supriyo Ghosh, Sean Laguna, Shiau Hong Lim +2
Air traffic control is an example of a highly challenging operational problem that is readily amenable to human expertise augmentation via decision support technologies. In this pa…
Robust commuter movement inference from connected mobile devices
Baoyang Song, Hasan Poonawala, Laura Wynter +1
The preponderance of connected devices provides unprecedented opportunities for fine-grained monitoring of the public infrastructure. However while classical models expect high qua…
Smooth Inter-layer Propagation of Stabilized Neural Networks for Classification
Jingfeng Zhang, Laura Wynter
Recent work has studied the reasons for the remarkable performance of deep neural networks in image classification. We examine batch normalization on the one hand and the dynamical…