Showing 2024Show all
3 papers · 1 filter
cs.LG2024
Representation Collapsing Problems in Vector Quantization
Wenhao Zhao, Qiran Zou, Rushi Shah +1
Vector quantization is a technique in machine learning that discretizes continuous representations into a set of discrete vectors. It is widely employed in tokenizing data represen…
cs.LG2024
Gaussian Mixture Vector Quantization with Aggregated Categorical Posterior
Mingyuan Yan, Jiawei Wu, Rushi Shah +1
The vector quantization is a widely used method to map continuous representation to discrete space and has important application in tokenization for generative mode, bottlenecking…
cs.LG2024
Improving Discrete Optimisation Via Decoupled Straight-Through Estimator
Rushi Shah, Mingyuan Yan, Michael Curtis Mozer +1
The Straight-Through Estimator (STE) is the dominant method for training neural networks with discrete variables, enabling gradient-based optimisation by routing gradients through…