256 citations · 1.2k across the 40 of their papers we have counts for
41 papers
SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning
Peng Xia, Jianwen Chen, Hanyang Wang +10
Large Language Model (LLM) agents have shown stunning results in complex tasks, yet they often operate in isolation, failing to learn from past experiences. Existing memory-based m…
Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvement
Xiyao Wang, Jiuhai Chen, Zhaoyang Wang +8
Large vision-language models (LVLMs) have achieved impressive results in visual question-answering and reasoning tasks through vision instruction tuning on specific datasets. Howev…
Calibrated Self-Rewarding Vision Language Models
Yiyang Zhou, Zhiyuan Fan, Dongjie Cheng +7
Large Vision-Language Models (LVLMs) have made substantial progress by integrating pre-trained large language models (LLMs) and vision models through instruction tuning. Despite th…
Aligning Modalities in Vision Large Language Models via Preference Fine-tuning
Yiyang Zhou, Chenhang Cui, Rafael Rafailov +2
Instruction-following Vision Large Language Models (VLLMs) have achieved significant progress recently on a variety of tasks. These approaches merge strong pre-trained vision model…
TrustLLM: Trustworthiness in Large Language Models
Yue Huang, Lichao Sun, Haoran Wang +67
Large language models (LLMs), exemplified by ChatGPT, have gained considerable attention for their excellent natural language processing capabilities. Nonetheless, these LLMs prese…
How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs
Haoqin Tu, Chenhang Cui, Zijun Wang +6
This work focuses on the potential of Vision LLMs (VLLMs) in visual reasoning. Different from prior studies, we shift our focus from evaluating standard performance to introducing…
Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challenges
Chenhang Cui, Yiyang Zhou, Xinyu Yang +4
While GPT-4V(ision) impressively models both visual and textual information simultaneously, it's hallucination behavior has not been systematically assessed. To bridge this gap, we…
Analyzing and Mitigating Object Hallucination in Large Vision-Language Models
Yiyang Zhou, Chenhang Cui, Jaehong Yoon +5
Large vision-language models (LVLMs) have shown remarkable abilities in understanding visual information with human languages. However, LVLMs still suffer from object hallucination…
Conservative Prediction via Data-Driven Confidence Minimization
Caroline Choi, Fahim Tajwar, Yoonho Lee +3
In safety-critical applications of machine learning, it is often desirable for a model to be conservative, abstaining from making predictions on unknown inputs which are not well-r…
Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback
Katherine Tian, Eric Mitchell, Allan Zhou +5
A trustworthy real-world prediction system should produce well-calibrated confidence scores; that is, its confidence in an answer should be indicative of the likelihood that the an…
Last-Layer Fairness Fine-tuning is Simple and Effective for Neural Networks
Yuzhen Mao, Zhun Deng, Huaxiu Yao +3
As machine learning has been deployed ubiquitously across applications in modern data science, algorithmic fairness has become a great concern. Among them, imposing fairness constr…
Improving Domain Generalization with Domain Relations
Huaxiu Yao, Xinyu Yang, Xinyi Pan +3
Distribution shift presents a significant challenge in machine learning, where models often underperform during the test stage when faced with a different distribution than the one…
Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time
Huaxiu Yao, Caroline Choi, Bochuan Cao +3
Distribution shift occurs when the test distribution differs from the training distribution, and it can considerably degrade performance of machine learning models deployed in the…
C-Mixup: Improving Generalization in Regression
Huaxiu Yao, Yiping Wang, Linjun Zhang +2
Improving the generalization of deep networks is an important open challenge, particularly in domains without plentiful data. The mixup algorithm improves generalization by linearl…
Knowledge-Driven New Drug Recommendation
Zhenbang Wu, Huaxiu Yao, Zhe Su +5
Drug recommendation assists doctors in prescribing personalized medications to patients based on their health conditions. Existing drug recommendation solutions adopt the supervise…
Multi-Domain Long-Tailed Learning by Augmenting Disentangled Representations
Xinyu Yang, Huaxiu Yao, Allan Zhou +1
There is an inescapable long-tailed class-imbalance issue in many real-world classification problems. Current methods for addressing this problem only consider scenarios where all…
Surgical Fine-Tuning Improves Adaptation to Distribution Shifts
Yoonho Lee, Annie S. Chen, Fahim Tajwar +4
A common approach to transfer learning under distribution shift is to fine-tune the last few layers of a pre-trained model, preserving learned features while also adapting to the n…
Spatio-Temporal Graph Few-Shot Learning with Cross-City Knowledge Transfer
Bin Lu, Xiaoying Gan, Weinan Zhang +3
Spatio-temporal graph learning is a key method for urban computing tasks, such as traffic flow, taxi demand and air quality forecasting. Due to the high cost of data collection, so…
Improving Meta-learning for Low-resource Text Classification and Generation via Memory Imitation
Yingxiu Zhao, Zhiliang Tian, Huaxiu Yao +5
Building models of natural language processing (NLP) is challenging in low-resource scenarios where only limited data are available. Optimization-based meta-learning algorithms ach…
Diversify and Disambiguate: Learning From Underspecified Data
Yoonho Lee, Huaxiu Yao, Chelsea Finn
Many datasets are underspecified: there exist multiple equally viable solutions to a given task. Underspecification can be problematic for methods that learn a single hypothesis be…
Improving Out-of-Distribution Robustness via Selective Augmentation
Huaxiu Yao, Yu Wang, Sai Li +4
Machine learning algorithms typically assume that training and test examples are drawn from the same distribution. However, distribution shift is a common problem in real-world app…
Meta-learning with an Adaptive Task Scheduler
Huaxiu Yao, Yu Wang, Ying Wei +4
To benefit the learning of a new task, meta-learning has been proposed to transfer a well-generalized meta-model learned from various meta-training tasks. Existing meta-learning al…
Knowledge-Aware Meta-learning for Low-Resource Text Classification
Huaxiu Yao, Yingxin Wu, Maruan Al-Shedivat +1
Meta-learning has achieved great success in leveraging the historical learned knowledge to facilitate the learning process of the new task. However, merely learning the knowledge f…
Meta-Learning with Fewer Tasks through Task Interpolation
Huaxiu Yao, Linjun Zhang, Chelsea Finn
Meta-learning enables algorithms to quickly learn a newly encountered task with just a few labeled examples by transferring previously learned knowledge. However, the bottleneck of…