3 papers
cs.CL2023
Bit Cipher -- A Simple yet Powerful Word Representation System that Integrates Efficiently with Language Models
Haoran Zhao, Jake Ryland Williams
While Large Language Models (LLMs) become ever more dominant, classic pre-trained word embeddings sustain their relevance through computational efficiency and nuanced linguistic in…
cs.LG2023
Explicit Foundation Model Optimization with Self-Attentive Feed-Forward Neural Units
Jake Ryland Williams, Haoran Zhao
Iterative approximation methods using backpropagation enable the optimization of neural networks, but they remain computationally expensive, especially when used at scale. This pap…
cs.LG2023
Reducing the Need for Backpropagation and Discovering Better Optima With Explicit Optimizations of Neural Networks
Jake Ryland Williams, Haoran Zhao
Iterative differential approximation methods that rely upon backpropagation have enabled the optimization of neural networks; however, at present, they remain computationally expen…