4 papers
High-Throughput LLM-Based Compression and What It Takes: A Systems Study of Inference Engines and Entropy Coding Optimizations
David Adams, Luisa Neubauer, Ping-Lin Kuo +5
Large language models are attractive text compressors because next-token probabilities can be converted into code lengths. In practice, however, the main bottleneck is often not mo…
Autoregressive Synthesis of Sparse and Semi-Structured Mixed-Type Data
Thomas Rückstieß, Robin Vujanic
Synthetic data generation is an important capability for privacy-preserving data sharing, system benchmarking and test data provisioning. For mixed-type data, existing synthesizers…
LEAF: Knowledge Distillation of Text Embedding Models with Teacher-Aligned Representations
Robin Vujanic, Thomas Rueckstiess
We present LEAF ("Lightweight Embedding Alignment Framework"), a knowledge distillation framework for text embedding models. A key distinguishing feature is that our distilled leaf…
ORIGAMI: A generative transformer architecture for predictions from semi-structured data
Thomas Rückstieß, Alana Huang, Robin Vujanic
Despite the popularity and widespread use of semi-structured data formats such as JSON, end-to-end supervised learning applied directly to such data remains underexplored. We prese…