4 papers · 1 filter
REViT-v2: Hierarchical Windowed Roto-reflection Equivariant ViT for Equivariant Feature Extraction
Sheir A. Zaheer, Jihwan Moon, Chan Y. Park
We propose a scalable roto-reflection-group-equivariant vision transformer based on windowed group-convolutional self-attention and a hierarchical feature architecture. We demonstr…
Greedy Local Learning for Language Model Pretraining: Gaps and Objective Design
Jihwan Moon, Sheir A. Zaheer, Jinmyoung Lee +2
Greedy block-wise local learning splits a network into gradient-isolated blocks trained by local auxiliary losses, deleting the backward pass between blocks: inter-stage communicat…
Where the Numbers Come From: Auditing Evaluation in Provenance-Based Intrusion Detection
Jihwan Moon, Gunhee Kim, Myeongjang Pyeon
Reproducing a provenance-based intrusion detector's score does not establish what that score says about its emitted alarms or the information its encoder uses. We audit nine releas…
Raon-Speech Technical Report
Beomsoo Kim, Changho Choi, Dohyun Kim +23
We present Raon-Speech, a top-performing 9B-parameter speech language model (SpeechLM) for English and Korean speech understanding, answering, and generation, and Raon-SpeechChat,…