3 citations · 11 across the 19 of their papers we have counts for
6 papers · 1 filter
Fourier Position Embedding: Enhancing Attention's Periodic Extension for Length Generalization
Ermo Hua, Che Jiang, Xingtai Lv +7
Extending the context length of Language Models (LMs) by improving Rotary Position Embedding (RoPE) has become a trend. While prior works mainly address RoPE's limitations within a…
On the token distance modeling ability of higher RoPE attention dimension
Xiangyu Hong, Che Jiang, Biqing Qi +4
Length extrapolation algorithms based on Rotary position embedding (RoPE) have shown promising results in extending the context length of language models. However, understanding ho…
JoyType: A Robust Design for Multilingual Visual Text Creation
Chao Li, Chen Jiang, Xiaolong Liu +2
Generating images with accurately represented text, especially in non-Latin languages, poses a significant challenge for diffusion models. Existing approaches, such as the integrat…
MSI-Agent: Incorporating Multi-Scale Insight into Embodied Agents for Superior Planning and Decision-Making
Dayuan Fu, Biqing Qi, Yihuai Gao +3
Long-term memory is significant for agents, in which insights play a crucial role. However, the emergence of irrelevant insight and the lack of general insight can greatly undermin…
A Survey On Text-to-3D Contents Generation In The Wild
Chenhan Jiang
3D content creation plays a vital role in various applications, such as gaming, robotics simulation, and virtual reality. However, the process is labor-intensive and time-consuming…
On Large Language Models' Hallucination with Regard to Known Facts
Che Jiang, Biqing Qi, Xiangyu Hong +6
Large language models are successful in answering factoid questions but are also prone to hallucination. We investigate the phenomenon of LLMs possessing correct answer knowledge y…