5 papers
Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling
Xiang Hu, Xinyu Wei, Hao Gu +10
Scaling modern large language models (LLMs) to long contexts is limited by the quadratic computation cost, and poor length extrapolation of dense attention. Chunk-wise sparse atten…
Retrieval Feedback Memory Enhancement Large Model Retrieval Generation Method
Leqian Li, Dianxi Shi, Jialu Zhou +4
Large Language Models (LLMs) have shown remarkable capabilities across diverse tasks, yet they face inherent limitations such as constrained parametric knowledge and high retrainin…
CEIDM: A Controlled Entity and Interaction Diffusion Model for Enhanced Text-to-Image Generation
Mingyue Yang, Dianxi Shi, Jialu Zhou +4
In Text-to-Image (T2I) generation, the complexity of entities and their intricate interactions pose a significant challenge for T2I method based on diffusion model: how to effectiv…
Dynamic Embedding of Hierarchical Visual Features for Efficient Vision-Language Fine-Tuning
Xinyu Wei, Guoli Yang, Jialu Zhou +4
Large Vision-Language Models (LVLMs) commonly follow a paradigm that projects visual features and then concatenates them with text tokens to form a unified sequence input for Large…
Separation and Collaboration: Two-Level Routing Grouped Mixture-of-Experts for Multi-Domain Continual Learning
Jialu Zhou, Dianxi Shi, Shaowu Yang +5
Multi-Domain Continual Learning (MDCL) acquires knowledge from sequential tasks with shifting class sets and distribution. Despite the Parameter-Efficient Fine-Tuning (PEFT) method…