2 citations · 2 across the 11 of their papers we have counts for
11 papers
Generalized Category Discovery via Reciprocal Learning and Class-Wise Distribution Regularization
Duo Liu, Zhiquan Tan, Linglan Zhao +3
Generalized Category Discovery (GCD) aims to identify unlabeled samples by leveraging the base knowledge from labeled ones, where the unlabeled set consists of both base and novel…
Information-Theoretic Perspectives on Optimizers
Zhiquan Tan, Weiran Huang
The interplay of optimizers and architectures in neural networks is complicated and hard to understand why some optimizers work better on some specific architectures. In this paper…
ATLAS: Adapter-Based Multi-Modal Continual Learning with a Two-Stage Learning Strategy
Hong Li, Zhiquan Tan, Xingyu Li +1
While vision-and-language models significantly advance in many fields, the challenge of continual learning is unsolved. Parameter-efficient modules like adapters and prompts presen…
Exploring Information-Theoretic Metrics Associated with Neural Collapse in Supervised Training
Kun Song, Zhiquan Tan, Bochao Zou +3
In this paper, we introduce matrix entropy as an analytical tool for studying supervised learning, investigating the information content of data representations and classification…
Can I understand what I create? Self-Knowledge Evaluation of Large Language Models
Zhiquan Tan, Lai Wei, Jindong Wang +2
Large language models (LLMs) have achieved remarkable progress in linguistic tasks, necessitating robust evaluation frameworks to understand their capabilities and limitations. Ins…
Unveiling the Dynamics of Information Interplay in Supervised Learning
Kun Song, Zhiquan Tan, Bochao Zou +2
In this paper, we use matrix information theory as an analytical tool to analyze the dynamics of the information interplay between data representations and classification head vect…
Provable Contrastive Continual Learning
Yichen Wen, Zhiquan Tan, Kaipeng Zheng +2
Continual learning requires learning incremental tasks with dynamic data distributions. So far, it has been observed that employing a combination of contrastive loss and distillati…
The Information of Large Language Model Geometry
Zhiquan Tan, Chenghai Li, Weiran Huang
This paper investigates the information encoded in the embeddings of large language models (LLMs). We conduct simulations to analyze the representation entropy and discover a power…
Diff-eRank: A Novel Rank-Based Metric for Evaluating Large Language Models
Lai Wei, Zhiquan Tan, Chenghai Li +2
Large Language Models (LLMs) have transformed natural language processing and extended their powerful capabilities to multi-modal domains. As LLMs continue to advance, it is crucia…
Understanding Grokking Through A Robustness Viewpoint
Zhiquan Tan, Weiran Huang
Recently, an interesting phenomenon called grokking has gained much attention, where generalization occurs long after the models have initially overfitted the training data. We try…
OTMatch: Improving Semi-Supervised Learning with Optimal Transport
Zhiquan Tan, Kaipeng Zheng, Weiran Huang
Semi-supervised learning has made remarkable strides by effectively utilizing a limited amount of labeled data while capitalizing on the abundant information present in unlabeled d…