7 papers · 1 filter
Breaking the Likelihood Trap: Variance-Calibrated Modulation for Large Language Model Decoding
Yuanhao Ding, Meimingwei Li, Esteban Garces Arias +3
In open-ended generation, LLMs frequently fall into the "likelihood trap", marked by repetitive degeneration and vocabulary dullness, creating a discrepancy between machine-generat…
Beyond Temperature: Hyperfitting as a Late-Stage Geometric Expansion
Meimingwei Li, Yuanhao Ding, Esteban Garces Arias +1
Recent work has identified a counterintuitive phenomenon termed "Hyperfitting", where fine-tuning Large Language Models (LLMs) to near-zero training loss on small datasets surprisi…
GUARD: Glocal Uncertainty-Aware Robust Decoding for Effective and Efficient Open-Ended Text Generation
Yuanhao Ding, Esteban Garces Arias, Meimingwei Li +6
Open-ended text generation faces a critical challenge: balancing coherence with diversity in LLM outputs. While contrastive search-based decoding strategies have emerged to address…
Unveiling Factors for Enhanced POS Tagging: A Study of Low-Resource Medieval Romance Languages
Matthias Schöffel, Esteban Garces Arias, Marinus Wiedner +4
Part-of-speech (POS) tagging remains a foundational component in natural language processing pipelines, particularly critical for historical text analysis at the intersection of co…
Towards Better Open-Ended Text Generation: A Multicriteria Evaluation Framework
Esteban Garces Arias, Hannah Blocher, Julian Rodemann +3
Open-ended text generation has become a prominent task in natural language processing due to the rise of powerful (large) language models. However, evaluating the quality of these…
Decoding Decoded: Understanding Hyperparameter Effects in Open-Ended Text Generation
Esteban Garces Arias, Meimingwei Li, Christian Heumann +1
Decoding strategies for generative large language models (LLMs) are a critical but often underexplored aspect of text generation tasks. Guided by specific hyperparameters, these st…