7 papers · 1 filter
Regret Pre-training: Bridging Prior and Posterior Views for Enhanced Knowledge Grounding
Mingkuan Zhao, Xiayu Sun, Wentao Hu +5
Causal language models factorize sequence probabilities using only preceding context, leaving future information unexploited during training despite its availability in the trainin…
MIRROR: A Multi-Agent Framework with Iterative Adaptive Revision and Hierarchical Retrieval for Optimization Modeling in Operations Research
Yifan Shi, Jiayi Wang, Minyi Wu +3
Operations Research (OR) relies on expert-driven modeling--a slow and fragile process ill-suited to novel scenarios. While large language models (LLMs) can automatically translate…
Generation-Augmented Generation: A Plug-and-Play Framework for Private Knowledge Injection in Large Language Models
Rongji Li, Jian Xu, Yi Chen +7
In domains such as materials science, biomedicine, and finance, high-stakes deployment of large language models (LLMs) requires injecting private, domain-specific knowledge that is…
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
Multilingual Language Model Pretraining using Machine-translated Data
Jiayi Wang, Yao Lu, Maurice Weber +5
High-resource languages such as English, enables the pretraining of high-quality large language models (LLMs). The same can not be said for most other languages as LLMs still under…
Warmup Generations: A Task-Agnostic Approach for Guiding Sequence-to-Sequence Learning with Unsupervised Initial State Generation
Senyu Li, Zipeng Sun, Jiayi Wang +4
Traditional supervised fine-tuning (SFT) strategies for sequence-to-sequence tasks often train models to directly generate the target output. Recent work has shown that guiding mod…