2 papers
cs.LG2024
Uncovering Layer-Dependent Activation Sparsity Patterns in ReLU Transformers
Cody Wild, Jesper Anderson
Previous work has demonstrated that MLPs within ReLU Transformers exhibit high levels of sparsity, with many of their activations equal to zero for any given token. We build on tha…
cs.AI2024
Capabilities of Gemini Models in Medicine
Khaled Saab, Tao Tu, Wei-Hung Weng +64
Excellence in a wide variety of medical applications poses considerable challenges for AI, requiring advanced reasoning, access to up-to-date medical knowledge and understanding of…