Publications (26)
The Devil is in the Dark Pixels: Toward Brightness Bias-Robust Denoising
Sungjun Cho, Zhuangzhuang Chen, Xiaomeng Li
In this paper, we reveal an important yet overlooked problem in image denoising: under signal-dependent camera noise models, dark regions suffer from inherently low Signal-to-Noise…
Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs
Sungmin Cha, Sungjun Cho, Dasol Hwang +1
Large Language Models (LLMs) have demonstrated strong reasoning and memorization capabilities via pretraining on massive textual corpora. However, this poses risk of privacy and co…
Pure Transformers are Powerful Graph Learners
Jinwoo Kim, Tien Dat Nguyen, Seonwoo Min +4
We show that standard Transformers without graph-specific modifications can lead to promising results in graph learning both in theory and practice. Given a graph, we simply treat…
MEG-GPT: A transformer-based foundation model for magnetoencephalography data
Rukuang Huang, Sungjun Cho, Chetan Gohil +2
Modelling the complex spatiotemporal patterns of large-scale brain dynamics is crucial for neuroscience, but traditional methods fail to capture the rich structure in modalities su…
Improving Multi-fidelity Optimization with a Recurring Learning Rate for Hyperparameter Tuning
HyunJae Lee, Gihyeon Lee, Junhwan Kim +3
Despite the evolution of Convolutional Neural Networks (CNNs), their performance is surprisingly dependent on the choice of hyperparameters. However, it remains challenging to effi…
TARDIS: Mitigating Temporal Misalignment via Representation Steering
Changho Shin, Xinya Yan, Suenggwan Jo +3
Language models often struggle with temporal misalignment, performance degradation caused by shifts in the temporal distribution of data. Continuously updating models to avoid degr…
Test-Time Scaling Makes Overtraining Compute-Optimal
Nicholas Roberts, Sungjun Cho, Zhiqi Gao +7
Modern LLMs scale at test-time, e.g. via repeated sampling, where inference cost grows with model size and the number of samples. This creates a trade-off that pretraining scaling…
Partial-Multivariate Model for Forecasting
Jaehoon Lee, Hankook Lee, Sungik Choi +2
When solving forecasting problems including multiple time-series features, existing approaches often fall into two extreme categories, depending on whether to utilize inter-feature…
Learning Equi-angular Representations for Online Continual Learning
Minhyuk Seo, Hyunseo Koh, Wonje Jeung +7
Online continual learning suffers from an underfitted solution due to insufficient training for prompt model update (e.g., single-epoch training). To address the challenge, we prop…
HCCL: Collective Communication for Meta Training and Inference Accelerators
Wesley Bland, Tiago Antunes, Lars Paul Huse +63
We present HCCL, a collective communication library co-designed with Meta's MTIA 300 accelerator, the first Meta chip to integrate backend networking directly on chip package. MTIA…
LLM-Integrated Bayesian State Space Models for Multimodal Time-Series Forecasting
Sungjun Cho, Changho Shin, Suenggwan Jo +3
Forecasting in the real world requires integrating structured time-series data with unstructured textual information, but existing methods are architecturally limited by fixed inpu…
Practical and Reproducible Symbolic Music Generation by Large Language Models with Structural Embeddings
Seungyeon Rhyu, Kichang Yang, Sungjun Cho +3
Music generation introduces challenging complexities to large language models. Symbolic structures of music often include vertical harmonization as well as horizontal counterpoint,…
Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers
Sungmin Cha, Sungjun Cho, Dasol Hwang +3
Since the recent advent of regulations for data protection (e.g., the General Data Protection Regulation), there has been increasing demand in deleting information learned from sen…
Curve Your Attention: Mixed-Curvature Transformers for Graph Representation Learning
Sungjun Cho, Seunghyuk Cho, Sungwoo Park +3
Real-world graphs naturally exhibit hierarchical or cyclical structures that are unfit for the typical Euclidean space. While there exist graph neural networks that leverage hyperb…
Diabetic Retinopathy Grading with CLIP-based Ranking-Aware Adaptation:A Comparative Study on Fundus Image
Sungjun Cho
Diabetic retinopathy (DR) is a leading cause of preventable blindness, and automated fundus image grading can play an important role in large-scale screening. In this work, we inve…
Block-SSD: A New Block-Based Blocking SSD Architecture
Ryan Wong, Arjun Tyagi, Sungjun Cho +2
Computer science and related fields (e.g., computer engineering, computer hardware engineering, electrical engineering, electrical and computer engineering, computer systems engine…
Transformers meet Stochastic Block Models: Attention with Data-Adaptive Sparsity and Cost
Sungjun Cho, Seonwoo Min, Jinwoo Kim +3
To overcome the quadratic cost of self-attention, recent works have proposed various sparse attention modules, most of which fall under one of two groups: 1) sparse attention under…
Reference-Specific Unlearning Metrics Can Hide the Truth: A Reality Check
Sungjun Cho, Dasol Hwang, Frederic Sala +3
Current unlearning metrics for generative models evaluate success based on reference responses or classifier outputs rather than assessing the core objective: whether the unlearned…
Equivariant Hypergraph Neural Networks
Jinwoo Kim, Saeyoon Oh, Sungjun Cho +1
Many problems in computer vision and machine learning can be cast as learning on hypergraphs that represent higher-order relations. Recent approaches for hypergraph learning extend…
Rebalancing Batch Normalization for Exemplar-based Class-Incremental Learning
Sungmin Cha, Sungjun Cho, Dasol Hwang +3
Batch Normalization (BN) and its variants has been extensively studied for neural nets in various computer vision tasks, but relatively little work has been dedicated to studying t…
On-the-Fly Rectification for Robust Large-Vocabulary Topic Inference
Moontae Lee, Sungjun Cho, Kun Dong +2
Across many data domains, co-occurrence statistics about the joint appearance of objects are powerfully informative. By transforming unsupervised learning problems into decompositi…
Two-View Accumulation as the Primary Training Lever for Hybrid-Capture Gaussian Splatting: A Variance-Decomposition View of When Gradient Surgery Helps
Sungjun Cho
Hybrid-capture novel view synthesis combines images at substantially different camera distances (e.g., aerial drone and ground-level views). Standard 3D Gaussian Splatting (3DGS),…
Bandwidth-Effective DRAM Cache for GPUs with Storage-Class Memory
Jeongmin Hong, Sungjun Cho, Geonwoo Park +3
We propose overcoming the memory capacity limitation of GPUs with high-capacity Storage-Class Memory (SCM) and DRAM cache. By significantly increasing the memory capacity with SCM,…
Grouping-matrix based Graph Pooling with Adaptive Number of Clusters
Sung Moon Ko, Sungjun Cho, Dae-Woong Jeong +3
Graph pooling is a crucial operation for encoding hierarchical structures within graphs. Most existing graph pooling approaches formulate the problem as a node clustering task whic…
AERO: Adaptive Erase Operation for Improving Lifetime and Performance of Modern NAND Flash-Based SSDs
Sungjun Cho, Beomjun Kim, Hyunuk Cho +4
This work investigates a new erase scheme in NAND flash memory to improve the lifetime and performance of modern solid-state drives (SSDs). In NAND flash memory, an erase operation…
3D Denoisers are Good 2D Teachers: Molecular Pretraining via Denoising and Cross-Modal Distillation
Sungjun Cho, Dae-Woong Jeong, Sung Moon Ko +5
Pretraining molecular representations from large unlabeled data is essential for molecular property prediction due to the high cost of obtaining ground-truth labels. While there ex…