4 papers
DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs
Dongyuan Li, Shiyin Tan, Ying Zhang +4
Dynamic graph modeling aims to uncover evolutionary patterns in real-world systems, enabling accurate social recommendation and early detection of cancer cells. Inspired by the suc…
Diversity of Transformer Layers: One Aspect of Parameter Scaling Laws
Hidetaka Kamigaito, Ying Zhang, Jingun Kwon +3
Transformers deliver outstanding performance across a wide range of tasks and are now a dominant backbone architecture for large language models (LLMs). Their task-solving performa…
Revisiting Dynamic Graph Clustering via Matrix Factorization
Dongyuan Li, Satoshi Kosugi, Ying Zhang +3
Dynamic graph clustering aims to detect and track time-varying clusters in dynamic graphs, revealing the evolutionary mechanisms of complex real-world dynamic systems. Matrix facto…
Reconsidering Degeneration of Token Embeddings with Definitions for Encoder-based Pre-trained Language Models
Ying Zhang, Dongyuan Li, Manabu Okumura
Learning token embeddings based on token co-occurrence statistics has proven effective for both pre-training and fine-tuning in natural language processing. However, recent studies…