31 citations · 93 across the 30 of their papers we have counts for
8 papers · 1 filter
Structuring MoE Expert Selection for Agentic Reinforcement Learning
Bolian Li, Ting-Yao Hu, Cheng-Yu Hsieh +3
Long-horizon LLM agents are frequently implemented using sparse mixture-of-experts (MoE) models, yet the co-design of agentic behavior and MoE structures remains underexplored. In…
COMPASS: Benchmarking Constrained Optimization in LLM Agents
Tian Qin, Felix Bai, Ting-Yao Hu +8
Human decision-making often involves constrained optimization. As LLM agents are deployed to assist with real-world tasks like travel planning, shopping, and scheduling, they must…
Proxy-FDA: Proxy-based Feature Distribution Alignment for Fine-tuning Vision Foundation Models without Forgetting
Chen Huang, Skyler Seto, Hadi Pouransari +6
Vision foundation models pre-trained on massive data encode rich representations of real-world concepts, which can be adapted to downstream tasks by fine-tuning. However, fine-tuni…
TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining
Jeffrey Li, Mohammadreza Armandpour, Iman Mirzadeh +8
Large Language Models (LLMs) trained on historical web data inevitably become outdated. We investigate evaluation strategies and update methods for LLMs as new data becomes availab…
Weight subcloning: direct initialization of transformers using larger pretrained ones
Mohammad Samragh, Mehrdad Farajtabar, Sachin Mehta +5
Training large transformer models from scratch for a target task requires lots of data and is computationally demanding. The usual practice of transfer learning overcomes this chal…
CLIP meets Model Zoo Experts: Pseudo-Supervision for Visual Enhancement
Mohammadreza Salehi, Mehrdad Farajtabar, Maxwell Horton +7
Contrastive language image pretraining (CLIP) is a standard method for training vision-language models. While CLIP is scalable, promptable, and robust to distribution shifts on ima…
Towards Federated Learning Under Resource Constraints via Layer-wise Training and Depth Dropout
Pengfei Guo, Warren Richard Morningstar, Raviteja Vemulapalli +3
Large machine learning models trained on diverse data have recently seen unprecedented success. Federated learning enables training on private data that may otherwise be inaccessib…
Federated Training of Dual Encoding Models on Small Non-IID Client Datasets
Raviteja Vemulapalli, Warren Richard Morningstar, Philip Andrew Mansfield +4
Dual encoding models that encode a pair of inputs are widely used for representation learning. Many approaches train dual encoding models by maximizing agreement between pairs of e…