27 citations · 109 across the 23 of their papers we have counts for
8 papers · 1 filter
Sylber: Syllabic Embedding Representation of Speech from Raw Audio
Cheol Jun Cho, Nicholas Lee, Akshat Gupta +4
Syllables are compositional units of spoken language that efficiently structure human speech perception and production. However, current neural speech representations lack such str…
FineZip : Pushing the Limits of Large Language Models for Practical Lossless Text Compression
Fazal Mittu, Yihuan Bu, Akshat Gupta +4
While the language modeling objective has been shown to be deeply connected with compression, it is surprising that modern LLMs are not employed in practical text compression syste…
Geometric Interpretation of Layer Normalization and a Comparative Analysis with RMSNorm
Akshat Gupta, Atahan Ozdemir, Gopala Anumanchipalli
This paper presents a novel geometric interpretation of LayerNorm and explores how LayerNorm influences the norm and orientation of hidden vectors in the representation space. With…
Is Bigger Edit Batch Size Always Better? -- An Empirical Study on Model Editing with Llama-3
Junsang Yoon, Akshat Gupta, Gopala Anumanchipalli
This study presents a targeted model editing analysis focused on the latest large language model, Llama-3. We explore the efficacy of popular model editing techniques - ROME, MEMIT…
A Unified Framework for Model Editing
Akshat Gupta, Dev Sajnani, Gopala Anumanchipalli
ROME and MEMIT are largely believed to be two different model editing algorithms, with the major difference between them being the ability to perform batched edits. In this paper,…
Rebuilding ROME : Resolving Model Collapse during Sequential Model Editing
Akshat Gupta, Sidharth Baskaran, Gopala Anumanchipalli
Recent work using Rank-One Model Editing (ROME), a popular model editing method, has shown that there are certain facts that the algorithm is unable to edit without breaking the mo…
Identifying Multiple Personalities in Large Language Models with External Evaluation
Xiaoyang Song, Yuta Adachi, Jessie Feng +6
As Large Language Models (LLMs) are integrated with human daily applications rapidly, many societal and ethical concerns are raised regarding the behavior of LLMs. One of the ways…
Model Editing at Scale leads to Gradual and Catastrophic Forgetting
Akshat Gupta, Anurag Rao, Gopala Anumanchipalli
Editing knowledge in large language models is an attractive capability to have which allows us to correct incorrectly learnt facts during pre-training, as well as update the model…