5 papers
Greedy Local Learning for Language Model Pretraining: Gaps and Objective Design
Jihwan Moon, Sheir A. Zaheer, Jinmyoung Lee +2
Greedy block-wise local learning splits a network into gradient-isolated blocks trained by local auxiliary losses, deleting the backward pass between blocks: inter-stage communicat…
REViT: Roto-reflection Equivariant Convolutional Vision Transformer
Sheir A. Zaheer, Alexander C. Holston, Chan Y. Park
In this paper, we propose a discrete roto-reflection group equivariant vision transformer with convolutional attention. Roto-reflection equivariant networks preserve the rotational…
ErA: Error-Aware Deep Unrolling Network for Single Image Defocus Deblurring
Tu Vo, Chan Y. Park
We introduce ErA (Error-Aware Deep Unrolling Network), an end-to-end frame work for single-image defocus deblurring. ErA jointly learns a compact kerne basis and per-pixel weights,…
Drift-Augmented Scoring: Text-Derived Noise Robustness for Zero-Shot Audio-Language Classification
Tu Vo, Sheir Zaheer, Chan Y. Park
Contrastive audio-language models such as CLAP enable zero-shot audio classification: a sound is labelled by matching its embedding to text prompt embeddings, with no labelled audi…
Deep Joint Unrolling for Deblurring and Low-Light Image Enhancement (JUDE)
Tu Vo, Chan Y. Park
Low-light and blurring issues are prevalent when capturing photos at night, often due to the use of long exposure to address dim environments. Addressing these joint problems can b…