9 papers
Seamlessly Integrating Tree-Based Positional Embeddings into Transformer Models for Source Code Representation
Patryk Bartkowiak, Filip Graliński
Transformer-based models have demonstrated significant success in various source code representation tasks. Nonetheless, traditional positional embeddings employed by these models…
Tackling prediction tasks in relational databases with LLMs
Marek Wydmuch, Łukasz Borchmann, Filip Graliński
Though large language models (LLMs) have demonstrated exceptional performance across numerous problems, their application to predictive tasks in relational databases remains largel…
CCpdf: Building a High Quality Corpus for Visually Rich Documents from Web Crawl Data
Michał Turski, Tomasz Stanisławek, Karol Kaczmarek +2
In recent years, the field of document understanding has progressed a lot. A significant part of this progress has been possible thanks to the use of language models pretrained on…
From Dataset Recycling to Multi-Property Extraction and Beyond
Tomasz Dwojak, Michał Pietruszka, Łukasz Borchmann +2
This paper investigates various Transformer architectures on the WikiReading Information Extraction and Machine Reading Comprehension dataset. The proposed dual-source model outper…
Successive Halving Top-k Operator
Michał Pietruszka, Łukasz Borchmann, Filip Graliński
We propose a differentiable successive halving method of relaxing the top-k operator, rendering gradient-based optimization possible. The need to perform softmax iteratively on the…
On the Multi-Property Extraction and Beyond
Tomasz Dwojak, Michał Pietruszka, Łukasz Borchmann +2
In this paper, we investigate the Dual-source Transformer architecture on the WikiReading information extraction and machine reading comprehension dataset. The proposed model outpe…