2 papers
cs.LG2025
Probing the Limits of Compressive Memory: A Study of Infini-Attention in Small-Scale Pretraining
Ruizhe Huang, Kexuan Zhang, Yihao Fang +1
This study investigates small-scale pretraining for Small Language Models (SLMs) to enable efficient use of limited data and compute, improve accessibility in low-resource settings…
cs.AI2024
HGOT: Hierarchical Graph of Thoughts for Retrieval-Augmented In-Context Learning in Factuality Evaluation
Yihao Fang, Stephen W. Thomas, Xiaodan Zhu
With the widespread adoption of large language models (LLMs) in numerous applications, the challenge of factuality and the propensity for hallucinations has emerged as a significan…