101 citations · 145 across the 25 of their papers we have counts for
26 papers
A Benchmark for Deep Information Synthesis
Debjit Paul, Daniel Murphy, Milan Gritta +14
Large language model (LLM)-based agents are increasingly used to solve complex tasks involving tool use, such as web browsing, code execution, and data analysis. However, current e…
Conjecturing: An Overlooked Step in Formal Mathematical Reasoning
Jasivan Alex Sivakumar, Philipp Borchert, Ronald Cardenas +1
Autoformalisation, the task of expressing informal mathematical statements in formal language, is often viewed as a direct translation process. This, however, disregards a critical…
TopoAlign: A Framework for Aligning Code to Math via Topological Decomposition
Yupei Li, Philipp Borchert, Gerasimos Lampouras
Large Language Models (LLMs) excel at both informal and formal (e.g. Lean 4) mathematical reasoning but still struggle with autoformalisation, the task of transforming informal int…
DRIFT: Decompose, Retrieve, Illustrate, then Formalize Theorems
Meiru Zhang, Philipp Borchert, Milan Gritta +1
Automating the formalization of mathematical statements for theorem proving remains a major challenge for Large Language Models (LLMs). LLMs struggle to identify and utilize the pr…
DReSD: Dense Retrieval for Speculative Decoding
Milan Gritta, Huiyin Xue, Gerasimos Lampouras
Speculative decoding (SD) accelerates Large Language Model (LLM) generation by using an efficient draft model to propose the next few tokens, which are verified by the LLM in a sin…
SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Masks
Fenia Christopoulou, Ronald Cardenas, Gerasimos Lampouras +2
Direct alignment algorithms have proven an effective step for aligning language models to human-desired behaviors. Current variants of the Direct Preference Optimization objective…
Mixture of Attentions For Speculative Decoding
Matthieu Zimmer, Milan Gritta, Gerasimos Lampouras +2
The growth in the number of parameters of Large Language Models (LLMs) has led to a significant surge in computational requirements, making them challenging and costly to deploy. S…
Human-inspired Episodic Memory for Infinite Context LLMs
Zafeirios Fountas, Martin A Benfeghoul, Adnan Oomerjee +4
Large language models (LLMs) have shown remarkable capabilities, but still struggle with processing extensive contexts, limiting their ability to maintain coherence and accuracy ov…
Code-Optimise: Self-Generated Preference Data for Correctness and Efficiency
Leonidas Gee, Milan Gritta, Gerasimos Lampouras +1
Code Language Models have been trained to generate accurate solutions, typically with no regard for runtime. On the other hand, previous works that explored execution optimisation…
HumanRankEval: Automatic Evaluation of LMs as Conversational Assistants
Milan Gritta, Gerasimos Lampouras, Ignacio Iacobacci
Language models (LMs) as conversational assistants recently became popular tools that help people accomplish a variety of tasks. These typically result from adapting LMs pretrained…
Encoder-Decoder Framework for Interactive Free Verses with Generation with Controllable High-Quality Rhyming
Tommaso Pasini, Alejo López-Ávila, Husam Quteineh +5
Composing poetry or lyrics involves several creative factors, but a challenging aspect of generation is the adherence to a more or less strict metric and rhyming pattern. To addres…
MULAN: A Multi Layer Annotated Dataset for Controllable Text-to-Image Generation
Petru-Daniel Tudosiu, Yongxin Yang, Shifeng Zhang +5
Text-to-image generation has achieved astonishing results, yet precise spatial controllability and prompt fidelity remain highly challenging. This limitation is typically addressed…
Text-to-Code Generation with Modality-relative Pre-training
Fenia Christopoulou, Guchun Zhang, Gerasimos Lampouras
Large pre-trained language models have recently been expanded and applied to programming language tasks with great success, often through further pre-training of a strictly-natural…
Findings of the First Workshop on Simulating Conversational Intelligence in Chat
Yvette Graham, Mohammed Rameez Qureshi, Haider Khalid +3
The aim of the workshop was to bring together experts working on open-domain dialogue research. In this speedily advancing research area many challenges still exist, such as learni…
Automatic Unit Test Data Generation and Actor-Critic Reinforcement Learning for Code Synthesis
Philip John Gorinski, Matthieu Zimmer, Gerasimos Lampouras +2
The advent of large pre-trained language models in the domain of Code Synthesis has shown remarkable performance on various benchmarks, treating the problem of Code Generation in a…
Exploring Data Augmentation for Code Generation Tasks
Pinzhen Chen, Gerasimos Lampouras
Advances in natural language processing, such as transfer learning from pre-trained language models, have impacted how models are trained for programming language tasks too. Previo…
Topic-Aware Response Generation in Task-Oriented Dialogue with Unstructured Knowledge Access
Yue Feng, Gerasimos Lampouras, Ignacio Iacobacci
To alleviate the problem of structured databases' limited coverage, recent task-oriented dialogue systems incorporate external unstructured knowledge to guide the generation of sys…
Training Dynamics for Curriculum Learning: A Study on Monolingual and Cross-lingual NLU
Fenia Christopoulou, Gerasimos Lampouras, Ignacio Iacobacci
Curriculum Learning (CL) is a technique of training models via ranking examples in a typically increasing difficulty trend with the aim of accelerating convergence and improving ge…
PanGu-Coder: Program Synthesis with Function-Level Language Modeling
Fenia Christopoulou, Gerasimos Lampouras, Milan Gritta +19
We present PanGu-Coder, a pretrained decoder-only language model adopting the PanGu-Alpha architecture for text-to-code generation, i.e. the synthesis of programming language solut…
Generalising Multilingual Concept-to-Text NLG with Language Agnostic Delexicalisation
Giulio Zhou, Gerasimos Lampouras
Concept-to-text Natural Language Generation is the task of expressing an input meaning representation in natural language. Previous approaches in this task have been able to genera…
Conversation Graph: Data Augmentation, Training and Evaluation for Non-Deterministic Dialogue Management
Milan Gritta, Gerasimos Lampouras, Ignacio Iacobacci
Task-oriented dialogue systems typically rely on large amounts of high-quality training data or require complex handcrafted rules. However, existing datasets are often limited in s…
Show Us the Way: Learning to Manage Dialog from Demonstrations
Gabriel Gordon-Hall, Philip John Gorinski, Gerasimos Lampouras +1
We present our submission to the End-to-End Multi-Domain Dialog Challenge Track of the Eighth Dialog System Technology Challenge. Our proposed dialog system adopts a pipeline archi…
Informed Sampling for Diversity in Concept-to-Text NLG
Giulio Zhou, Gerasimos Lampouras
Deep-learning models for language generation tasks tend to produce repetitive output. Various methods have been proposed to encourage lexical diversity during decoding, but this of…
Generating Texts with Integer Linear Programming
Gerasimos Lampouras, Ion Androutsopoulos
Concept-to-text generation typically employs a pipeline architecture, which often leads to suboptimal texts. Content selection, for example, may greedily select the most important…