4 papers
RoleQ: Role-Aligned Scales for Converting Language Models to Ternary
Johannes Knittel, Alec Helbling, Hanspeter Pfister
Weight quantization below four bits shrinks memory and bandwidth at a perplexity that looks acceptable, but often at the cost of factual knowledge and task accuracy. Ternary weight…
Sparse Inter-Layer Dependencies of Transformer FFN Neurons
Johannes Knittel, Hanspeter Pfister
Feedforward network (FFN) blocks account for a large fraction of the parameters and computation in Transformer architectures, yet their internal structure remains difficult to inte…
GPT-2 Through the Lens of Vector Symbolic Architectures
Johannes Knittel, Tushaar Gangavarapu, Hendrik Strobelt +1
Understanding the general priniciples behind transformer models remains a complex endeavor. Experiments with probing and disentangling features using sparse autoencoders (SAE) sugg…
Multimodal Learning for Embryo Viability Prediction in Clinical IVF
Junsik Kim, Zhiyi Shi, Davin Jeong +8
In clinical In-Vitro Fertilization (IVF), identifying the most viable embryo for transfer is important to increasing the likelihood of a successful pregnancy. Traditionally, this p…