4 papers
AttentionRetriever: Attention Layers are Secretly Long Document Retrievers
David Jiahao Fu, Lam Thanh Do, Jiayu Li +1
Retrieval augmented generation (RAG) has been widely adopted to help Large Language Models (LLMs) to process tasks involving long documents. However, existing retrieval models are…
CASPER: Concept-integrated Sparse Representation for Scientific Retrieval
Lam Thanh Do, Linh Van Nguyen, Jiayu Li +2
Identifying relevant research concepts is crucial for effective scientific search. However, primary sparse retrieval methods often lack concept-aware representations. To address th…
SLMFix: Leveraging Small Language Models for Error Fixing with Reinforcement Learning
David Jiahao Fu, Aryan Gupta, Aaron Councilman +3
Recent advancements in large language models (LLMs) have shown very impressive capabilities in code generation across many programming languages. However, even state-of-the-art LLM…
Towards Formal Verification of LLM-Generated Code from Natural Language Prompts
Aaron Councilman, David Jiahao Fu, Aryan Gupta +4
In the past few years LLMs have emerged as a tool that can aid programmers by taking natural language descriptions and generating code based on it. However, the reliability of LLM…