5 citations · 17 across the 8 of their papers we have counts for
8 papers
Spectra 1.1: Scaling Laws and Efficient Inference for Ternary Language Models
Tejas Vaidhya, Ayush Kaushal, Vineet Jain +5
Large language models (LLMs) are increasingly used across research and industry applications, yet their inference efficiency remains a significant challenge. As the computational p…
Spectra: Surprising Effectiveness of Pretraining Ternary Language Models at Scale
Ayush Kaushal, Tejas Vaidhya, Arnab Kumar Mondal +3
Rapid advancements in GPU computational power has outpaced memory capacity and bandwidth growth, creating bottlenecks in Large Language Model (LLM) inference. Post-training quantiz…
LORD: Low Rank Decomposition Of Monolingual Code LLMs For One-Shot Compression
Ayush Kaushal, Tejas Vaidhya, Irina Rish
Low Rank Decomposition of matrix - splitting a large matrix into a product of two smaller matrix offers a means for compression that reduces the parameters of a model without spars…
Efficient Encoders for Streaming Sequence Tagging
Ayush Kaushal, Aditya Gupta, Shyam Upadhyay +1
A naive application of state-of-the-art bidirectional encoders for streaming sequence tagging would require encoding each token from scratch for each new token in an incremental st…
What do tokens know about their characters and how do they know it?
Ayush Kaushal, Kyle Mahowald
Pre-trained language models (PLMs) that use subword tokenization schemes can succeed at a variety of language tasks that require character-level information, despite lacking explic…
Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLP
Zhijing Jin, Julius von Kügelgen, Jingwei Ni +4
The principle of independent causal mechanisms (ICM) states that generative processes of real world data consist of independent modules which do not influence or inform each other.…
Domain specific BERT representation for Named Entity Recognition of lab protocol
Tejas Vaidhya, Ayush Kaushal
Supervised models trained to predict properties from representations have been achieving high accuracy on a variety of tasks. For instance, the BERT family seems to work exceptiona…
Leveraging Event Specific and Chunk Span features to Extract COVID Events from tweets
Ayush Kaushal, Tejas Vaidhya
Twitter has acted as an important source of information during disasters and pandemic, especially during the times of COVID-19. In this paper, we describe our system entry for WNUT…