57 citations · 147 across the 34 of their papers we have counts for
9 papers · 1 filter
Transformers meet Stochastic Block Models: Attention with Data-Adaptive Sparsity and Cost
Sungjun Cho, Seonwoo Min, Jinwoo Kim +3
To overcome the quadratic cost of self-attention, recent works have proposed various sparse attention modules, most of which fall under one of two groups: 1) sparse attention under…
Knowledge Unlearning for Mitigating Privacy Risks in Language Models
Joel Jang, Dongkeun Yoon, Sohee Yang +4
Pretrained Language Models (LMs) memorize a vast amount of knowledge during initial pretraining, including information that may violate the privacy of personal lives and identities…
Grouping-matrix based Graph Pooling with Adaptive Number of Clusters
Sung Moon Ko, Sungjun Cho, Dae-Woong Jeong +3
Graph pooling is a crucial operation for encoding hierarchical structures within graphs. Most existing graph pooling approaches formulate the problem as a node clustering task whic…
Symbolic Music Loop Generation with Neural Discrete Representations
Sangjun Han, Hyeongrae Ihm, Moontae Lee +1
Since most of music has repetitive structures from motifs to phrases, repeating musical ideas can be a basic operation for music composition. The basic block that we focus on is co…
Pure Transformers are Powerful Graph Learners
Jinwoo Kim, Tien Dat Nguyen, Seonwoo Min +4
We show that standard Transformers without graph-specific modifications can lead to promising results in graph learning both in theory and practice. Given a graph, we simply treat…
Towards Diverse Evaluation of Class Incremental Learning: A Representation Learning Perspective
Sungmin Cha, Jihwan Kwak, Dongsub Shim +4
Class incremental learning (CIL) algorithms aim to continually learn new object classes from incrementally arriving data while not forgetting past learned classes. The common evalu…