12 papers
Matryoshka Agent: Unfolding Sub-Agents for Long-Horizon Machine Learning Engineering
Rushi Qiang, Changhao Li, Haotian Sun +3
Machine learning engineering (MLE) tasks require long-horizon decision making over iterative solution debugging and refinement, under expensive and feedback-driven environment inte…
Forward-Free Diffusion Language Models with BPTT-Free Looped Refinement
Haotian Sun, Rushi Qiang, Yuqian Zheng +1
Diffusion language models generate text through iterative denoising, offering a powerful alternative to autoregressive generation. However, discrete language spaces lack a natural…
Exploration-Driven Optimization for Test-Time Large Language Model Reasoning
Changhao Li, Yuchen Zhuang, Chenxiao Gao +4
Post-training techniques combined with inference-time scaling significantly enhance the reasoning and alignment capabilities of large language models (LLMs). However, a fundamental…
Spectral Representation-based Reinforcement Learning
Chenxiao Gao, Haotian Sun, Na Li +2
In real-world applications with large state and action spaces, reinforcement learning (RL) typically employs function approximations to represent core components like the policies,…
Two Birds with One Stone: Enhancing Uncertainty Quantification and Interpretability with Graph Functional Neural Process
Lingkai Kong, Haotian Sun, Yuchen Zhuang +3
Graph neural networks (GNNs) are powerful tools on graph data. However, their predictions are mis-calibrated and lack interpretability, limiting their adoption in critical applicat…
Towards Better Instruction Following Retrieval Models
Yuchen Zhuang, Aaron Trinh, Rushi Qiang +4
Modern information retrieval (IR) models, trained exclusively on standard <query, passage> pairs, struggle to effectively interpret and follow explicit user instructions. We introd…
AmorLIP: Efficient Language-Image Pretraining via Amortization
Haotian Sun, Yitong Li, Yuchen Zhuang +3
Contrastive Language-Image Pretraining (CLIP) has demonstrated strong zero-shot performance across diverse downstream text-image tasks. Existing CLIP methods typically optimize a c…
Matryoshka Pilot: Learning to Drive Black-Box LLMs with LLMs
Changhao Li, Yuchen Zhuang, Rushi Qiang +4
Despite the impressive generative abilities of black-box large language models (LLMs), their inherent opacity hinders further advancements in capabilities such as reasoning, planni…
EC-DIT: Scaling Diffusion Transformers with Adaptive Expert-Choice Routing
Haotian Sun, Tao Lei, Bowen Zhang +5
Diffusion transformers have been widely adopted for text-to-image synthesis. While scaling these models up to billions of parameters shows promise, the effectiveness of scaling bey…
Spectral Representation for Causal Estimation with Hidden Confounders
Haotian Sun, Antoine Moulin, Tongzheng Ren +2
We address the problem of causal effect estimation where hidden confounders are present, with a focus on two settings: instrumental variable regression with additional observed con…
HYDRA: Model Factorization Framework for Black-Box LLM Personalization
Yuchen Zhuang, Haotian Sun, Yue Yu +4
Personalization has emerged as a critical research area in modern intelligent systems, focusing on mining users' behavioral history and adapting to their preferences for delivering…
BBox-Adapter: Lightweight Adapting for Black-Box Large Language Models
Haotian Sun, Yuchen Zhuang, Wei Wei +2
Adapting state-of-the-art Large Language Models (LLMs) like GPT-4 and Gemini for specific tasks is challenging. Due to the opacity in their parameters, embeddings, and even output…