10 citations · 32 across the 8 of their papers we have counts for
8 papers
Retrieval Heads are Dynamic
Yuping Lin, Zitao Li, Yue Xing +6
Recent studies have identified "retrieval heads" in Large Language Models (LLMs) responsible for extracting information from input contexts. However, prior works largely rely on st…
Why Fine-grained Labels in Pretraining Benefit Generalization?
Guan Zhe Hong, Yin Cui, Ariel Fuxman +2
Recent studies show that pretraining a deep neural network with fine-grained labeled data, followed by fine-tuning on coarse-labeled data for downstream tasks, often yields better…
Module-wise Adaptive Distillation for Multimodality Foundation Models
Chen Liang, Jiahui Yu, Ming-Hsuan Yang +5
Pre-trained multimodal foundation models have demonstrated remarkable generalizability but pose challenges for deployment due to their large sizes. One effective approach to reduci…
DaTaSeg: Taming a Universal Multi-Dataset Multi-Task Segmentation Model
Xiuye Gu, Yin Cui, Jonathan Huang +8
Observing the close relationship among panoptic, semantic and instance segmentation tasks, we propose to train a universal multi-dataset multi-task segmentation model: DaTaSeg.We u…
Towards Understanding the Effect of Pretraining Label Granularity
Guan Zhe Hong, Yin Cui, Ariel Fuxman +2
In this paper, we study how the granularity of pretraining labels affects the generalization of deep neural networks in image classification tasks. We focus on the "fine-to-coarse"…
Multimodal Open-Vocabulary Video Classification via Pre-Trained Vision and Language Models
Rui Qian, Yeqing Li, Zheng Xu +3
Utilizing vision and language models (VLMs) pre-trained on large-scale image-text pairs is becoming a promising paradigm for open-vocabulary visual recognition. In this work, we ex…