24 citations · 70 across the 8 of their papers we have counts for
9 papers
Continual Pre-Training of Large Language Models: How to (re)warm your model?
Kshitij Gupta, Benjamin Thérien, Adam Ibrahim +5
Large language models (LLMs) are routinely pre-trained on billions of tokens, only to restart the process over again once new data becomes available. A much cheaper and more effici…
ARB: Advanced Reasoning Benchmark for Large Language Models
Tomohiro Sawada, Daniel Paleka, Alexander Havrilla +6
Large Language Models (LLMs) have demonstrated remarkable performance on various quantitative reasoning and knowledge benchmarks. However, many of these benchmarks are losing utili…
Broken Neural Scaling Laws
Ethan Caballero, Kshitij Gupta, Irina Rish +1
We present a smoothly broken power law functional form (that we refer to as a Broken Neural Scaling Law (BNSL)) that accurately models & extrapolates the scaling behaviors of deep…
cViL: Cross-Lingual Training of Vision-Language Models using Knowledge Distillation
Kshitij Gupta, Devansh Gautam, Radhika Mamidi
Vision-and-language tasks are gaining popularity in the research community, but the focus is still mainly on English. We propose a pipeline that utilizes English-only vision-langua…
Temporal Latent Bottleneck: Synthesis of Fast and Slow Processing Mechanisms in Sequence Learning
Aniket Didolkar, Kshitij Gupta, Anirudh Goyal +4
Recurrent neural networks have a strong inductive bias towards learning temporally compressed representations, as the entire history of a sequence is represented by a single vector…
ViTA: Visual-Linguistic Translation by Aligning Object Tags
Kshitij Gupta, Devansh Gautam, Radhika Mamidi
Multimodal Machine Translation (MMT) enriches the source text with visual information for translation. It has gained popularity in recent years, and several pipelines have been pro…
Volta at SemEval-2021 Task 9: Statement Verification and Evidence Finding with Tables using TAPAS and Transfer Learning
Devansh Gautam, Kshitij Gupta, Manish Shrivastava
Tables are widely used in various kinds of documents to present information concisely. Understanding tables is a challenging problem that requires an understanding of language and…
Volta at SemEval-2021 Task 6: Towards Detecting Persuasive Texts and Images using Textual and Multimodal Ensemble
Kshitij Gupta, Devansh Gautam, Radhika Mamidi
Memes are one of the most popular types of content used to spread information online. They can influence a large number of people through rhetorical and psychological techniques. T…
White or Blue, the Whale gets its Vengeance: A Social Media Analysis of the Blue Whale Challenge
Abhinav Khattar, Karan Dabas, Kshitij Gupta +2
The Blue Whale Challenge is a series of self-harm causing tasks that are propagated via online social media under the disguise of a "game." The list of tasks must be completed in a…