7 papers
Compositionality Emerges in a Narrow Depth-Connectivity Regime: Architecture Constraints and Solution Manifolds
Dat H. Do, Rushi Shah, Duc V. Le +1
Compositionality is believed to be the foundation for generalization, enabling models to reuse meaningful primitives in novel combinations. Yet, models trained with standard gradie…
Early Quantization Shrinks Codebook: A Simple Fix for Diversity-Preserving Tokenization
Wenhao Zhao, Qiran Zou, Rushi Shah +3
Vector quantization is a technique in machine learning that discretizes continuous representations into a set of discrete vectors. It is widely employed in tokenizing data represen…
CXR-LanIC: Language-Grounded Interpretable Classifier for Chest X-Ray Diagnosis
Yiming Tang, Wenjia Zhong, Rushi Shah +1
Deep learning models have achieved remarkable accuracy in chest X-ray diagnosis, yet their widespread clinical adoption remains limited by the black-box nature of their predictions…
Representation Collapsing Problems in Vector Quantization
Wenhao Zhao, Qiran Zou, Rushi Shah +1
Vector quantization is a technique in machine learning that discretizes continuous representations into a set of discrete vectors. It is widely employed in tokenizing data represen…
Gaussian Mixture Vector Quantization with Aggregated Categorical Posterior
Mingyuan Yan, Jiawei Wu, Rushi Shah +1
The vector quantization is a widely used method to map continuous representation to discrete space and has important application in tokenization for generative mode, bottlenecking…
Improving Discrete Optimisation Via Decoupled Straight-Through Estimator
Rushi Shah, Mingyuan Yan, Michael Curtis Mozer +1
The Straight-Through Estimator (STE) is the dominant method for training neural networks with discrete variables, enabling gradient-based optimisation by routing gradients through…
VQSynery: Robust Drug Synergy Prediction With Vector Quantization Mechanism
Jiawei Wu, Mingyuan Yan, Dianbo Liu
The pursuit of optimizing cancer therapies is significantly advanced by the accurate prediction of drug synergy. Traditional methods, such as clinical trials, are reliable yet encu…