2 citations · 2 across the 11 of their papers we have counts for
7 papers · 1 filter
ATLAS: Adapter-Based Multi-Modal Continual Learning with a Two-Stage Learning Strategy
Hong Li, Zhiquan Tan, Xingyu Li +1
While vision-and-language models significantly advance in many fields, the challenge of continual learning is unsolved. Parameter-efficient modules like adapters and prompts presen…
Exploring Information-Theoretic Metrics Associated with Neural Collapse in Supervised Training
Kun Song, Zhiquan Tan, Bochao Zou +3
In this paper, we introduce matrix entropy as an analytical tool for studying supervised learning, investigating the information content of data representations and classification…
Can I understand what I create? Self-Knowledge Evaluation of Large Language Models
Zhiquan Tan, Lai Wei, Jindong Wang +2
Large language models (LLMs) have achieved remarkable progress in linguistic tasks, necessitating robust evaluation frameworks to understand their capabilities and limitations. Ins…
Unveiling the Dynamics of Information Interplay in Supervised Learning
Kun Song, Zhiquan Tan, Bochao Zou +2
In this paper, we use matrix information theory as an analytical tool to analyze the dynamics of the information interplay between data representations and classification head vect…
Provable Contrastive Continual Learning
Yichen Wen, Zhiquan Tan, Kaipeng Zheng +2
Continual learning requires learning incremental tasks with dynamic data distributions. So far, it has been observed that employing a combination of contrastive loss and distillati…
The Information of Large Language Model Geometry
Zhiquan Tan, Chenghai Li, Weiran Huang
This paper investigates the information encoded in the embeddings of large language models (LLMs). We conduct simulations to analyze the representation entropy and discover a power…