257 citations · 437 across the 40 of their papers we have counts for
40 papers
Provable Training Data Identification for Large Language Models
Zhenlong Liu, Hao Zeng, Weiran Huang +1
Identifying training data of large-scale models is critical for copyright litigation, privacy auditing, and ensuring fair evaluation. However, existing works typically treat this t…
Information-Theoretic Perspectives on Optimizers
Zhiquan Tan, Weiran Huang
The interplay of optimizers and architectures in neural networks is complicated and hard to understand why some optimizers work better on some specific architectures. In this paper…
ATLAS: Adapter-Based Multi-Modal Continual Learning with a Two-Stage Learning Strategy
Hong Li, Zhiquan Tan, Xingyu Li +1
While vision-and-language models significantly advance in many fields, the challenge of continual learning is unsolved. Parameter-efficient modules like adapters and prompts presen…
Exploring Information-Theoretic Metrics Associated with Neural Collapse in Supervised Training
Kun Song, Zhiquan Tan, Bochao Zou +3
In this paper, we introduce matrix entropy as an analytical tool for studying supervised learning, investigating the information content of data representations and classification…
Diabetica: Adapting Large Language Model to Enhance Multiple Medical Tasks in Diabetes Care and Management
Lai Wei, Zhen Ying, Muyang He +9
Diabetes is a chronic disease with a significant global health burden, requiring multi-stakeholder collaboration for optimal management. Large language models (LLMs) have shown pro…
Can I understand what I create? Self-Knowledge Evaluation of Large Language Models
Zhiquan Tan, Lai Wei, Jindong Wang +2
Large language models (LLMs) have achieved remarkable progress in linguistic tasks, necessitating robust evaluation frameworks to understand their capabilities and limitations. Ins…
Unveiling the Dynamics of Information Interplay in Supervised Learning
Kun Song, Zhiquan Tan, Bochao Zou +2
In this paper, we use matrix information theory as an analytical tool to analyze the dynamics of the information interplay between data representations and classification head vect…
Provable Contrastive Continual Learning
Yichen Wen, Zhiquan Tan, Kaipeng Zheng +2
Continual learning requires learning incremental tasks with dynamic data distributions. So far, it has been observed that employing a combination of contrastive loss and distillati…
RhythmFormer: Extracting Patterned rPPG Signals based on Periodic Sparse Attention
Bochao Zou, Zizheng Guo, Jiansheng Chen +3
Remote photoplethysmography (rPPG) is a non-contact method for detecting physiological signals based on facial videos, holding high potential in various applications. Due to the pe…
FinLLM-B: When Large Language Models Meet Financial Breakout Trading
Kang Zhang, Osamu Yoshie, Lichao Sun +1
Trading range breakout is a key method in the technical analysis of financial trading, widely employed by traders in financial markets such as stocks, futures, and foreign exchange…
ChemLLM: A Chemical Large Language Model
Di Zhang, Wei Liu, Qian Tan +12
Large language models (LLMs) have made impressive progress in chemistry applications. However, the community lacks an LLM specifically designed for chemistry. The main challenges a…
The Information of Large Language Model Geometry
Zhiquan Tan, Chenghai Li, Weiran Huang
This paper investigates the information encoded in the embeddings of large language models (LLMs). We conduct simulations to analyze the representation entropy and discover a power…
Diff-eRank: A Novel Rank-Based Metric for Evaluating Large Language Models
Lai Wei, Zhiquan Tan, Chenghai Li +2
Large Language Models (LLMs) have transformed natural language processing and extended their powerful capabilities to multi-modal domains. As LLMs continue to advance, it is crucia…
TransMed: Large Language Models Enhance Vision Transformer for Biomedical Image Classification
Kaipeng Zheng, Weiran Huang, Lichao Sun
Few-shot learning has been studied to adapt models to tasks with very few samples. It holds profound significance, particularly in clinical tasks, due to the high annotation cost o…
FD-Align: Feature Discrimination Alignment for Fine-tuning Pre-Trained Models in Few-Shot Learning
Kun Song, Huimin Ma, Bochao Zou +2
Due to the limited availability of data, existing few-shot learning methods trained from scratch fail to achieve satisfactory performance. In contrast, large-scale pre-trained mode…
AutoEval-Video: An Automatic Benchmark for Assessing Large Vision Language Models in Open-Ended Video Question Answering
Xiuyuan Chen, Yuan Lin, Yuchen Zhang +1
We propose a novel and challenging benchmark, AutoEval-Video, to comprehensively evaluate large vision-language models in open-ended video question answering. The comprehensiveness…
Understanding Grokking Through A Robustness Viewpoint
Zhiquan Tan, Weiran Huang
Recently, an interesting phenomenon called grokking has gained much attention, where generalization occurs long after the models have initially overfitted the training data. We try…
OTMatch: Improving Semi-Supervised Learning with Optimal Transport
Zhiquan Tan, Kaipeng Zheng, Weiran Huang
Semi-supervised learning has made remarkable strides by effectively utilizing a limited amount of labeled data while capitalizing on the abundant information present in unlabeled d…
Information Flow in Self-Supervised Learning
Zhiquan Tan, Jingqin Yang, Weiran Huang +2
In this paper, we conduct a comprehensive analysis of two dual-branch (Siamese architecture) self-supervised learning approaches, namely Barlow Twins and spectral contrastive learn…
MM-LIMA: Less Is More for Alignment in Multi-Modal Datasets
Lai Wei, Xiaozhe Li, Zihao Jiang +2
Multimodal large language models are typically trained in two stages: first pre-training on image-text pairs, and then fine-tuning using supervised vision-language instruction data…
FILM: How can Few-Shot Image Classification Benefit from Pre-Trained Language Models?
Zihao Jiang, Yunkai Dang, Dong Pang +2
Few-shot learning aims to train models that can be generalized to novel classes with only a few samples. Recently, a line of works are proposed to enhance few-shot learning with ac…
DiffKendall: A Novel Approach for Few-Shot Learning with Differentiable Kendall's Rank Correlation
Kaipeng Zheng, Huishuai Zhang, Weiran Huang
Few-shot learning aims to adapt models trained on the base dataset to novel tasks where the categories were not seen by the model before. This often leads to a relatively uniform d…
Rethinking Weak Supervision in Helping Contrastive Learning
Jingyi Cui, Weiran Huang, Yifei Wang +1
Contrastive learning has shown outstanding performances in both supervised and unsupervised learning, and has recently been introduced to solve weakly supervised learning problems…
Matrix Information Theory for Self-Supervised Learning
Yifan Zhang, Zhiquan Tan, Jingqin Yang +2
The maximum entropy encoding framework provides a unified perspective for many non-contrastive learning methods like SimSiam, Barlow Twins, and MEC. Inspired by this framework, we…