Publications (5)
CAMformer: Associative Memory is All You Need
Tergel Molom-Ochir, Benjamin F. Morris, Mark Horton +8
Transformers face scalability challenges due to the quadratic cost of attention, which involves dense similarity computations between queries and keys. We propose CAMformer, a nove…
Gemma 2: Improving Open Language Models at a Practical Size
Gemma Team, Morgane Riviere, Shreya Pathak +195
In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2 billion to 27 billion parameters. In th…
High Cadence Millimagnitude Photometric Observations of V1112 Persei (Nova Per 2020)
Neil Thomas, Kyle Ziegler, Peter Liu
The private Lookout Observatory (LO) monitored the classic nova V1112 Persei on 37 nights spanning over 80 days, beginning shortly after its discovery by Seiji Ueda on 25 Nov 2020.…
Hamming Attention Distillation: Binarizing Keys and Queries for Efficient Long-Context Transformers
Mark Horton, Tergel Molom-Ochir, Peter Liu +8
Pre-trained transformer models with extended context windows are notoriously expensive to run at scale, often limiting real-world deployment due to their high computational and mem…
A Multi-attribute Controllable Generative Model for Histopathology Image Synthesis
Jiarong Ye, Yuan Xue, Peter Liu +3
Generative models have been applied in the medical imaging domain for various image recognition and synthesis tasks. However, a more controllable and interpretable image synthesis…