5 papers
MemForest: An Efficient Agent Memory System with Hierarchical Temporal Indexing
Han Chen, Zining Zhang, Wenqi Pei +6
Memory is a fundamental component for long-context LLM agents, supporting persistent state across interactions through a continuous serve-and-update lifecycle. Despite substantial…
ROSE: An Intent-Centered Evaluation Metric for NL2SQL
Wenqi Pei, Shizheng Hou, Boyan Li +3
Execution Accuracy (EX), the widely used metric for evaluating the effectiveness of Natural Language to SQL (NL2SQL) solutions, is becoming increasingly unreliable. It is sensitive…
NL2SQLBench: A Modular Benchmarking Framework for LLM-Enabled NL2SQL Solutions
Shizheng Hou, Wenqi Pei, Nuo Chen +3
Natural Language to SQL (NL2SQL) technology empowers non-expert users to query relational databases without requiring SQL expertise. While large language models (LLMs) have greatly…
InterFeedback: Unveiling Interactive Intelligence of Large Multimodal Models via Human Feedback
Henry Hengyuan Zhao, Wenqi Pei, Yifei Tao +2
Existing benchmarks do not test Large Multimodal Models (LMMs) on their interactive intelligence with human users, which is vital for developing general-purpose AI assistants. We d…
Feather-SQL: A Lightweight NL2SQL Framework with Dual-Model Collaboration Paradigm for Small Language Models
Wenqi Pei, Hailing Xu, Hengyuan Zhao +5
Natural Language to SQL (NL2SQL) has seen significant advancements with large language models (LLMs). However, these models often depend on closed-source systems and high computati…