5 papers
Are you Synthesizing or Recalling? Evaluating LLMs on Algorithmic Code Retrieval
Nickil Maveli, Antonio Vergari, Shay B. Cohen
Large language models (LLMs) have demonstrated strong performance in code generation, where success depends on both recalling relevant algorithmic knowledge and reasoning about how…
Can LLMs Compress (and Decompress)? Evaluating Code Understanding and Execution via Invertibility
Nickil Maveli, Antonio Vergari, Shay B. Cohen
LLMs demonstrate strong performance on code benchmarks, yet consistent reasoning across forward and backward execution remains elusive. We present RoundTripCodeEval (RTCE), a bench…
What can Large Language Models Capture about Code Functional Equivalence?
Nickil Maveli, Antonio Vergari, Shay B. Cohen
Code-LLMs, LLMs pre-trained on large code corpora, have shown great progress in learning rich representations of the structure and syntax of code, successfully using it to generate…
Co-training an Unsupervised Constituency Parser with Weak Supervision
Nickil Maveli, Shay B. Cohen
We introduce a method for unsupervised parsing that relies on bootstrapping classifiers to identify if a node dominates a specific span in a sentence. There are two types of classi…
EdinburghNLP at WNUT-2020 Task 2: Leveraging Transformers with Generalized Augmentation for Identifying Informativeness in COVID-19 Tweets
Nickil Maveli
Twitter and, in general, social media has become an indispensable communication channel in times of emergency. The ubiquitousness of smartphone gadgets enables people to declare an…