2 citations · 2 across the 5 of their papers we have counts for
6 papers
Making Large Language Models Efficient Dense Retrievers
Yibin Lei, Shwai He, Ang Li +1
Recent work has shown that directly fine-tuning large language models (LLMs) for dense retrieval yields strong performance, but their substantial parameter counts make them computa…
Understanding and Harnessing Sparsity in Unified Multimodal Models
Shwai He, Chaorui Deng, Ang Li +1
Large multimodal models have achieved remarkable progress in both understanding and generation. Recent efforts pursue unified multimodal models that integrate heterogeneous compone…
Dense Video Understanding with Gated Residual Tokenization
Haichao Zhang, Wenhao Chai, Shwai He +2
High temporal resolution is essential for capturing fine-grained details in video understanding. However, current video large language models (VLLMs) and benchmarks mostly rely on…
DualSparse-MoE: Coordinating Tensor/Neuron-Level Sparsity with Expert Partition and Reconstruction
Weilin Cai, Le Qin, Shwai He +3
Mixture of Experts (MoE) has become a mainstream architecture for building Large Language Models (LLMs) by reducing per-token computation while enabling model scaling. It can be vi…
Hierarchical Interaction Summarization and Contrastive Prompting for Explainable Recommendations
Yibin Liu, Ang Li, Shijian Li
Explainable recommendations, which use the information of user and item with interaction to generate a explanation for why the user would interact with the item, are crucial for im…
Domino: Eliminating Communication in LLM Training via Generic Tensor Slicing and Overlapping
Guanhua Wang, Chengming Zhang, Zheyu Shen +2
Given the popularity of generative AI, Large Language Models (LLMs) often consume hundreds or thousands of GPUs for parallelizing and accelerating the training process. Communicati…