3 citations · 5 across the 25 of their papers we have counts for
9 papers · 1 filter
Instruction Embedding: Latent Representations of Instructions Towards Task Identification
Yiwei Li, Jiayi Shi, Shaoxiong Feng +6
Instruction data is crucial for improving the capability of Large Language Models (LLMs) to align with human-level performance. Recent research LIMA demonstrates that alignment is…
Focused Large Language Models are Stable Many-Shot Learners
Peiwen Yuan, Shaoxiong Feng, Yiwei Li +7
In-Context Learning (ICL) enables large language models (LLMs) to achieve rapid task adaptation by learning from demonstrations. With the increase in available context length of LL…
Poor-Supervised Evaluation for SuperLLM via Mutual Consistency
Peiwen Yuan, Shaoxiong Feng, Yiwei Li +5
The guidance from capability evaluations has greatly propelled the progress of both human society and Artificial Intelligence. However, as LLMs evolve, it becomes challenging to co…
Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoning
Xinglin Wang, Shaoxiong Feng, Yiwei Li +6
Self-consistency (SC), a widely used decoding strategy for chain-of-thought reasoning, shows significant gains across various multi-step reasoning tasks but comes with a high cost…
CogLM: Tracking Cognitive Development of Large Language Models
Xinglin Wang, Peiwen Yuan, Shaoxiong Feng +5
Piaget's Theory of Cognitive Development (PTC) posits that the development of cognitive levels forms the foundation for human learning across various abilities. As Large Language M…
Integrate the Essence and Eliminate the Dross: Fine-Grained Self-Consistency for Free-Form Language Generation
Xinglin Wang, Yiwei Li, Shaoxiong Feng +5
Self-consistency (SC), leveraging multiple samples from LLMs, shows significant gains on various reasoning tasks but struggles with free-form generation due to the difficulty of ag…
Dynamic Stochastic Decoding Strategy for Open-Domain Dialogue Generation
Yiwei Li, Fei Mi, Yitong Li +4
Stochastic sampling strategies such as top-k and top-p have been widely used in dialogue generation task. However, as an open-domain chatting system, there will be two different co…
Generative Dense Retrieval: Memory Can Be a Burden
Peiwen Yuan, Xinglin Wang, Shaoxiong Feng +5
Generative Retrieval (GR), autoregressively decoding relevant document identifiers given a query, has been shown to perform well under the setting of small-scale corpora. By memori…
Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning
Yiwei Li, Peiwen Yuan, Shaoxiong Feng +5
Self-consistency (SC) has been a widely used decoding strategy for chain-of-thought reasoning. Despite bringing significant performance improvements across a variety of multi-step…