7 papers
OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration
Shaobo Wang, Xuan Ouyang, Tianyi Xu +9
As high-quality public text approaches exhaustion, a phenomenon known as the Data Wall, pre-training is shifting from more tokens to better tokens. However, existing methods either…
Lifelong Learning of Large Language Model based Agents: A Roadmap
Junhao Zheng, Chengming Shi, Xidi Cai +5
Lifelong learning, also known as continual or incremental learning, is a crucial component for advancing Artificial General Intelligence (AGI) by enabling systems to continuously a…
A Unified Shape-Aware Foundation Model for Time Series Classification
Zhen Liu, Yucheng Wang, Boyuan Li +4
Foundation models pre-trained on large-scale source datasets are reshaping the traditional training paradigm for time series classification. However, existing time series foundatio…
HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
Boyuan Li, Yicheng Luo, Zhen Liu +3
Irregular multivariate time series (IMTS) are characterized by irregular time intervals within variables and unaligned observations across variables, posing challenges in learning…
Neural-Symbolic Message Passing with Dynamic Pruning
Chongzhi Zhang, Junhao Zheng, Zhiping Peng +1
Complex Query Answering (CQA) over incomplete Knowledge Graphs (KGs) is a challenging task. Recently, a line of message-passing-based research has been proposed to solve CQA. Howev…
Spurious Forgetting in Continual Learning of Language Models
Junhao Zheng, Xidi Cai, Shengjie Qiu +1
Recent advancements in large language models (LLMs) reveal a perplexing phenomenon in continual learning: despite extensive training, models experience significant performance decl…