6 papers
Compositionality Emerges in a Narrow Depth-Connectivity Regime: Architecture Constraints and Solution Manifolds
Dat H. Do, Rushi Shah, Duc V. Le +1
Compositionality is believed to be the foundation for generalization, enabling models to reuse meaningful primitives in novel combinations. Yet, models trained with standard gradie…
CXR-LanIC: Language-Grounded Interpretable Classifier for Chest X-Ray Diagnosis
Yiming Tang, Wenjia Zhong, Rushi Shah +1
Deep learning models have achieved remarkable accuracy in chest X-ray diagnosis, yet their widespread clinical adoption remains limited by the black-box nature of their predictions…
Early Quantization Shrinks Codebook: A Simple Fix for Diversity-Preserving Tokenization
Wenhao Zhao, Qiran Zou, Rushi Shah +3
Vector quantization is a technique in machine learning that discretizes continuous representations into a set of discrete vectors. It is widely employed in tokenizing data represen…
Improving Discrete Optimisation Via Decoupled Straight-Through Estimator
Rushi Shah, Mingyuan Yan, Michael Curtis Mozer +1
The Straight-Through Estimator (STE) is the dominant method for training neural networks with discrete variables, enabling gradient-based optimisation by routing gradients through…
Representation Collapsing Problems in Vector Quantization
Wenhao Zhao, Qiran Zou, Rushi Shah +1
Vector quantization is a technique in machine learning that discretizes continuous representations into a set of discrete vectors. It is widely employed in tokenizing data represen…
Gaussian Mixture Vector Quantization with Aggregated Categorical Posterior
Mingyuan Yan, Jiawei Wu, Rushi Shah +1
The vector quantization is a widely used method to map continuous representation to discrete space and has important application in tokenization for generative mode, bottlenecking…