23 citations · 35 across the 19 of their papers we have counts for
19 papers
OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration
Shaobo Wang, Xuan Ouyang, Tianyi Xu +9
As high-quality public text approaches exhaustion, a phenomenon known as the Data Wall, pre-training is shifting from more tokens to better tokens. However, existing methods either…
A Unified Shape-Aware Foundation Model for Time Series Classification
Zhen Liu, Yucheng Wang, Boyuan Li +4
Foundation models pre-trained on large-scale source datasets are reshaping the traditional training paradigm for time series classification. However, existing time series foundatio…
HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecasting
Boyuan Li, Yicheng Luo, Zhen Liu +3
Irregular multivariate time series (IMTS) are characterized by irregular time intervals within variables and unaligned observations across variables, posing challenges in learning…
Neural-Symbolic Message Passing with Dynamic Pruning
Chongzhi Zhang, Junhao Zheng, Zhiping Peng +1
Complex Query Answering (CQA) over incomplete Knowledge Graphs (KGs) is a challenging task. Recently, a line of message-passing-based research has been proposed to solve CQA. Howev…
Spurious Forgetting in Continual Learning of Language Models
Junhao Zheng, Xidi Cai, Shengjie Qiu +1
Recent advancements in large language models (LLMs) reveal a perplexing phenomenon in continual learning: despite extensive training, models experience significant performance decl…
ListConRanker: A Contrastive Text Reranker with Listwise Encoding
Junlong Liu, Yue Ma, Ruihui Zhao +3
Reranker models aim to re-rank the passages based on the semantics similarity between the given query and passages, which have recently received more attention due to the wide appl…
Lifelong Learning of Large Language Model based Agents: A Roadmap
Junhao Zheng, Chengming Shi, Xidi Cai +5
Lifelong learning, also known as continual or incremental learning, is a crucial component for advancing Artificial General Intelligence (AGI) by enabling systems to continuously a…
Cross-domain Named Entity Recognition via Graph Matching
Junhao Zheng, Haibin Chen, Qianli Ma
Cross-domain NER is a practical yet challenging problem since the data scarcity in the real-world scenario. A common practice is first to learn a NER model in a rich-resource gener…
Pathformer: Recursive Path Query Encoding for Complex Logical Query Answering
Chongzhi Zhang, Zhiping Peng, Junhao Zheng +3
Complex Logical Query Answering (CLQA) over incomplete knowledge graphs is a challenging task. Recently, Query Embedding (QE) methods are proposed to solve CLQA by performing multi…
Towards Lifelong Learning of Large Language Models: A Survey
Junhao Zheng, Shengjie Qiu, Chengming Shi +1
As the applications of large language models (LLMs) expand across diverse fields, the ability of these models to adapt to ongoing changes in data, tasks, and user preferences becom…
Self-Adaptive Reconstruction with Contrastive Learning for Unsupervised Sentence Embeddings
Junlong Liu, Xichen Shang, Huawen Feng +2
Unsupervised sentence embeddings task aims to convert sentences to semantic vector representations. Most previous works directly use the sentence representations derived from pretr…
Balancing the Causal Effects in Class-Incremental Learning
Junhao Zheng, Ruiyan Wang, Chongzhi Zhang +2
Class-Incremental Learning (CIL) is a practical and challenging problem for achieving general artificial intelligence. Recently, Pre-Trained Models (PTMs) have led to breakthroughs…
Conditional Logical Message Passing Transformer for Complex Query Answering
Chongzhi Zhang, Zhiping Peng, Junhao Zheng +1
Complex Query Answering (CQA) over Knowledge Graphs (KGs) is a challenging task. Given that KGs are usually incomplete, neural models are proposed to solve CQA by performing multi-…
Incremental Sequence Labeling: A Tale of Two Shifts
Shengjie Qiu, Junhao Zheng, Zhen Liu +2
The incremental sequence labeling task involves continuously learning new classes over time while retaining knowledge of the previous ones. Our investigation identifies two signifi…
Can LLMs Learn New Concepts Incrementally without Forgetting?
Junhao Zheng, Shengjie Qiu, Qianli Ma
Large Language Models (LLMs) have achieved remarkable success across various tasks, yet their ability to learn incrementally without forgetting remains underexplored. Incremental l…
Beyond Anti-Forgetting: Multimodal Continual Instruction Tuning with Positive Forward Transfer
Junhao Zheng, Qianli Ma, Zhen Liu +2
Multimodal Continual Instruction Tuning (MCIT) enables Multimodal Large Language Models (MLLMs) to meet continuously emerging requirements without expensive retraining. MCIT faces…
Learn or Recall? Revisiting Incremental Learning with Pre-trained Language Models
Junhao Zheng, Shengjie Qiu, Qianli Ma
Incremental Learning (IL) has been a long-standing problem in both vision and Natural Language Processing (NLP) communities. In recent years, as Pre-trained Language Models (PLMs)…
Preserving Commonsense Knowledge from Pre-trained Language Models via Causal Inference
Junhao Zheng, Qianli Ma, Shengjie Qiu +6
Fine-tuning has been proven to be a simple and effective technique to transfer the learned knowledge of Pre-trained Language Models (PLMs) to downstream tasks. However, vanilla fin…
Distilling Causal Effect from Miscellaneous Other-Class for Continual Named Entity Recognition
Junhao Zheng, Zhanxian Liang, Haibin Chen +1
Continual Learning for Named Entity Recognition (CL-NER) aims to learn a growing number of entity types over time from a stream of data. However, simply learning Other-Class in the…