69 citations · 84 across the 16 of their papers we have counts for
16 papers
PolicyLong: Towards On-Policy Context Extension
Junlong Jia, Jiang Zhou, Ziyang Chen +5
Extending LLM context windows is hindered by scarce high-quality long-context data. Recent methods synthesize data with genuine long-range dependencies via information-theoretic ve…
LongBench Pro: A More Realistic and Comprehensive Bilingual Long-Context Evaluation Benchmark
Ziyang Chen, Xing Wu, Junlong Jia +4
The rapid expansion of context length in large language models (LLMs) has outpaced existing evaluation benchmarks. Current long-context benchmarks often trade off scalability and r…
EntropyLong: Effective Long-Context Training via Predictive Uncertainty
Junlong Jia, Ziyang Chen, Xing Wu +5
Training long-context language models to capture long-range dependencies requires specialized data construction. Current approaches, such as generic text concatenation or heuristic…
LiteLong: Resource-Efficient Long-Context Data Synthesis for LLMs
Junlong Jia, Xing Wu, Chaochen Gao +8
High-quality long-context data is essential for training large language models (LLMs) capable of processing extensive documents, yet existing synthesis approaches using relevance-b…
AMix-1: A Pathway to Test-Time Scalable Protein Foundation Model
Changze Lv, Jiang Zhou, Siyu Long +22
We introduce AMix-1, a powerful protein foundation model built on Bayesian Flow Networks and empowered by a systematic training methodology, encompassing pretraining scaling laws,…
LongMagpie: A Self-synthesis Method for Generating Large-scale Long-context Instructions
Chaochen Gao, Xing Wu, Zijia Lin +2
High-quality long-context instruction data is essential for aligning long-context large language models (LLMs). Despite the public release of models like Qwen and Llama, their long…
NExtLong: Toward Effective Long-Context Training without Long Documents
Chaochen Gao, Xing Wu, Zijia Lin +2
Large language models (LLMs) with extended context windows have made significant strides yet remain a challenge due to the scarcity of long documents. Existing methods tend to synt…
Quest: Query-centric Data Synthesis Approach for Long-context Scaling of Large Language Model
Chaochen Gao, Xing Wu, Qi Fu +1
Recent advancements in large language models (LLMs) have highlighted the importance of extending context lengths for handling complex tasks. While traditional methods for training…
Layout-guided Indoor Panorama Inpainting with Plane-aware Normalization
Chao-Chen Gao, Cheng-Hsiu Chen, Jheng-Wei Su +1
We present an end-to-end deep learning framework for indoor panoramic image inpainting. Although previous inpainting methods have shown impressive performance on natural perspectiv…
InfoCSE: Information-aggregated Contrastive Learning of Sentence Embeddings
Xing Wu, Chaochen Gao, Zijia Lin +3
Contrastive learning has been extensively studied in sentence embedding learning, which assumes that the embeddings of different views of the same sentence are closer. The constrai…
RaP: Redundancy-aware Video-language Pre-training for Text-Video Retrieval
Xing Wu, Chaochen Gao, Zijia Lin +3
Video language pre-training methods have mainly adopted sparse sampling techniques to alleviate the temporal redundancy of videos. Though effective, sparse sampling still suffers i…
Text Smoothing: Enhance Various Data Augmentation Methods on Text Classification Tasks
Xing Wu, Chaochen Gao, Meng Lin +3
Before entering the neural network, a token is generally converted to the corresponding one-hot representation, which is a discrete distribution of the vocabulary. Smoothed represe…
DistilCSE: Effective Knowledge Distillation For Contrastive Sentence Embeddings
Chaochen Gao, Xing Wu, Peng Wang +4
Large-scale contrastive learning models can learn very informative sentence embeddings, but are hard to serve online due to the huge model size. Therefore, they often play the role…
TransAug: Translate as Augmentation for Sentence Embeddings
Jue Wang
While contrastive learning greatly advances the representation of sentence embeddings, it is still limited by the size of the existing sentence datasets. In this paper, we present…
ESimCSE: Enhanced Sample Building Method for Contrastive Learning of Unsupervised Sentence Embedding
Xing Wu, Chaochen Gao, Liangjun Zang +3
Contrastive learning has been attracting much attention for learning unsupervised sentence embeddings. The current state-of-the-art unsupervised method is the unsupervised SimCSE (…
Smoothed Contrastive Learning for Unsupervised Sentence Embedding
Xing Wu, Chaochen Gao, Yipeng Su +3
Contrastive learning has been gradually applied to learn high-quality unsupervised sentence embedding. Among the previous un-supervised methods, the latest state-of-the-art method,…