Showing 2024Show all
2 papers · 1 filter
cs.DS2024
Theoretical Analysis of Byte-Pair Encoding
László Kozma, Johannes Voderholzer
Byte-Pair Encoding (BPE) is a widely used method for subword tokenization, with origins in grammar-based text compression. It is employed in a variety of language processing tasks…
cs.DS2024
Online sorting and online TSP: randomized, stochastic, and high-dimensional
Mikkel Abrahamsen, Ioana O. Bercea, Lorenzo Beretta +2
In the online sorting problem, items are revealed one by one and have to be placed (immediately and irrevocably) into empty cells of a size- array. The goal is to minimize t…