4 citations · 8 across the 5 of their papers we have counts for
6 papers · 1 filter
Evaluating Large Language Models on Computer Science University Exams in Data Structures
Edan Gabay, Yael Maoz, Jonathan Stahl +7
We present a comprehensive evaluation of Large Language Models (LLMs) on Computer Science (CS) Data Structure examination questions. Our work introduces a new benchmark dataset com…
Understanding Transformer Memorization Recall Through Idioms
Adi Haviv, Ido Cohen, Jacob Gidron +3
To produce accurate predictions, language models (LMs) must balance between generalization and memorization. Yet, little is known about the mechanism by which transformer LMs emplo…
Transformer Language Models without Positional Encodings Still Learn Positional Information
Adi Haviv, Ori Ram, Ofir Press +2
Causal transformer language models (LMs), such as GPT-3, typically require some form of positional encoding, such as positional embeddings. However, we show that LMs without any ex…
SCROLLS: Standardized CompaRison Over Long Language Sequences
Uri Shaham, Elad Segal, Maor Ivgi +8
NLP benchmarks have largely focused on short texts, such as sentences and paragraphs, even though long texts comprise a considerable amount of natural language in the wild. We intr…
Can Latent Alignments Improve Autoregressive Machine Translation?
Adi Haviv, Lior Vassertail, Omer Levy
Latent alignment objectives such as CTC and AXE significantly improve non-autoregressive machine translation models. Can they improve autoregressive models as well? We explore the…
BERTese: Learning to Speak to BERT
Adi Haviv, Jonathan Berant, Amir Globerson
Large pre-trained language models have been shown to encode large amounts of world and commonsense knowledge in their parameters, leading to substantial interest in methods for ext…