47 citations · 165 across the 90 of their papers we have counts for
26 papers · 1 filter
The Rise and Down of Babel Tower: Investigating the Evolution Process of Multilingual Code Large Language Model
Jiawei Chen, Wentao Chen, Jing Su +6
Large language models (LLMs) have shown significant multilingual capabilities. However, the mechanisms underlying the development of these capabilities during pre-training are not…
Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering
Xinyan Guan, Yanjiang Liu, Xinyu Lu +8
The evolution of machine learning has increasingly prioritized the development of powerful models and more scalable supervision signals. However, the emergence of foundation models…
StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization
Zhuoqun Li, Xuanang Chen, Haiyang Yu +7
Retrieval-augmented generation (RAG) is a key means to effectively enhance large language models (LLMs) in many knowledge-based tasks. However, existing RAG methods struggle with k…
A Unified View of Delta Parameter Editing in Post-Trained Large-Scale Models
Qiaoyu Tang, Le Yu, Bowen Yu +5
Post-training has emerged as a crucial paradigm for adapting large-scale pre-trained models to various tasks, whose effects are fully reflected by delta parameters (i.e., the dispa…
Multi-Facet Counterfactual Learning for Content Quality Evaluation
Jiasheng Zheng, Hongyu Lin, Boxi Cao +4
Evaluating the quality of documents is essential for filtering valuable content from the current massive amount of information. Conventional approaches typically rely on a single s…
Seg2Act: Global Context-aware Action Generation for Document Logical Structuring
Zichao Li, Shaojie He, Meng Liao +6
Document logical structuring aims to extract the underlying hierarchical structure of documents, which is crucial for document intelligence. Traditional approaches often fall short…
Transferable Post-training via Inverse Value Learning
Xinyu Lu, Xueru Wen, Yaojie Lu +6
As post-training processes utilize increasingly large datasets and base models continue to grow in size, the computational demands and implementation challenges of existing algorit…
Self-Steering Optimization: Autonomous Preference Optimization for Large Language Models
Hao Xiang, Bowen Yu, Hongyu Lin +7
The key to effective alignment lies in high-quality preference data. Recent research has focused on automated alignment, which involves developing alignment systems with minimal hu…
Rethinking Reward Model Evaluation: Are We Barking up the Wrong Tree?
Xueru Wen, Jie Lou, Yaojie Lu +7
Reward Models (RMs) are crucial for aligning language models with human preferences. Currently, the evaluation of RMs depends on measuring accuracy against a validation set of manu…
READoc: A Unified Benchmark for Realistic Document Structured Extraction
Zichao Li, Aizier Abulaiti, Yaojie Lu +5
Document Structured Extraction (DSE) aims to extract structured content from raw documents. Despite the emergence of numerous DSE systems, their unified evaluation remains inadequa…
DOMAINEVAL: An Auto-Constructed Benchmark for Multi-Domain Code Generation
Qiming Zhu, Jialun Cao, Yaojie Lu +4
Code benchmarks such as HumanEval are widely adopted to evaluate the capabilities of Large Language Models (LLMs), providing insights into their strengths and weaknesses. However,…
REInstruct: Building Instruction Data from Unlabeled Corpus
Shu Chen, Xinyan Guan, Yaojie Lu +3
Manually annotating instruction data for large language models is difficult, costly, and hard to scale. Meanwhile, current automatic annotation methods typically rely on distilling…
Critic-CoT: Boosting the reasoning abilities of large language model via Chain-of-thoughts Critic
Xin Zheng, Jie Lou, Boxi Cao +7
Self-critic has become a crucial mechanism for enhancing the reasoning performance of LLMs. However, current approaches mainly involve basic prompts for intuitive instance-level fe…
CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution
Ruiyang Xu, Jialun Cao, Yaojie Lu +6
Code benchmarks such as HumanEval are widely adopted to evaluate Large Language Models' (LLMs) coding capabilities. However, there is an unignorable programming language bias in ex…
Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models
Jiasheng Zheng, Boxi Cao, Zhengzhao Ma +5
In recent years, researchers have proposed numerous benchmarks to evaluate the impressive coding capabilities of large language models (LLMs). However, current benchmarks primarily…
Open Grounded Planning: Challenges and Benchmark Construction
Shiguang Guo, Ziliang Deng, Hongyu Lin +3
The emergence of large language models (LLMs) has increasingly drawn attention to the use of LLMs for human-like planning. Existing work on LLM-based planning either focuses on lev…
On-Policy Self-Alignment with Fine-grained Knowledge Feedback for Hallucination Mitigation
Xueru Wen, Jie Lou, Xinyu Lu +8
Hallucination occurs when large language models exhibit behavior that deviates from the boundaries of their knowledge during response generation. To address this critical issue, pr…
Towards Scalable Automated Alignment of LLMs: A Survey
Boxi Cao, Keming Lu, Xinyu Lu +10
Alignment is the most critical step in building large language models (LLMs) that meet human needs. With the rapid development of LLMs gradually surpassing human capabilities, trad…
URL: Universal Referential Knowledge Linking via Task-instructed Representation Compression
Zhuoqun Li, Hongyu Lin, Tianshu Wang +7
Linking a claim to grounded references is a critical ability to fulfill human demands for authentic and reliable information. Current studies are limited to specific tasks like inf…
Few-shot Named Entity Recognition via Superposition Concept Discrimination
Jiawei Chen, Hongyu Lin, Xianpei Han +4
Few-shot NER aims to identify entities of target types with only limited number of illustrative instances. Unfortunately, few-shot NER is severely challenged by the intrinsic preci…
Meta-Cognitive Analysis: Evaluating Declarative and Procedural Knowledge in Datasets and Large Language Models
Zhuoqun Li, Hongyu Lin, Yaojie Lu +3
Declarative knowledge and procedural knowledge are two key parts in meta-cognitive theory, and these two hold significant importance in pre-training and inference of LLMs. However,…
Executing Natural Language-Described Algorithms with Large Language Models: An Investigation
Xin Zheng, Qiming Zhu, Hongyu Lin +3
Executing computer programs described in natural language has long been a pursuit of computer science. With the advent of enhanced natural language understanding capabilities exhib…
ShortGPT: Layers in Large Language Models are More Redundant Than You Expect
Xin Men, Mingyu Xu, Qingyu Zhang +5
As Large Language Models (LLMs) continue to advance in performance, their size has escalated significantly, with current LLMs containing billions or even trillions of parameters. H…
Self-Retrieval: End-to-End Information Retrieval with One Large Language Model
Qiaoyu Tang, Jiawei Chen, Zhuoqun Li +10
The rise of large language models (LLMs) has significantly transformed both the construction and application of information retrieval (IR) systems. However, current interactions be…