1 citations · 1 across the 3 of their papers we have counts for
3 papers
SoftWater: Class-Aware Rate Allocation for Softmax Quantization
Joao V. Cavalcanti, Ashia C. Wilson
Post-training quantization pipelines routinely leave the softmax output layer in high precision. Yet in small LLMs with modern vocabularies, the head holds 15--30\% of all paramete…
Adaptive Acceleration Without Strong Convexity Priors Or Restarts
Joao V. Cavalcanti, Laurent Lessard, Ashia C. Wilson
A longstanding challenge in optimization is achieving optimal performance when the strong convexity parameter m is unknown. In this paper, we propose NAG-free, a simple extension o…
Adaptive Backtracking Line Search
Joao V. Cavalcanti, Laurent Lessard, Ashia C. Wilson
Backtracking line search is foundational in numerical optimization. The basic idea is to adjust the step-size of an algorithm by a constant factor until some chosen criterion (e.g.…