6 papers
WeGenBench: A Multidimensional Diagnostic Benchmark towards Text-to-Image Model Optimization
Qian Liang, Xiaomin Li, Ying Zhang +6
Recent text-to-image generation models have demonstrated remarkable capabilities in synthesizing highly realistic images from text inputs alone. Although existing benchmarks can ev…
BashCoder-R1: Towards Robust and Explainable Bash Code Generation with Robustness-Aware Group Relative Policy Optimization
Lei Yu, Peng Wang, Jia Xu +7
Bash scripts are critical for system administration, DevOps, and CI/CD, where code quality affects stability and security. However, LLM-generated scripts often lack reasoning and c…
LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMs
Keda Tao, Yuhua Zheng, Jia Xu +13
Recent advancements in omnimodal large language models (OmniLLMs) have significantly improved the comprehension of audio and video inputs. However, current evaluations primarily fo…
The Agent's First Day: Benchmarking Learning, Exploration, and Scheduling in the Workplace Scenarios
Daocheng Fu, Jianbiao Mei, Rong Wu +7
The rapid evolution of Multi-modal Large Language Models (MLLMs) has advanced workflow automation; however, existing research mainly targets performance upper bounds in static envi…
Changes by Butterflies: Farsighted Forecasting with Group Reservoir Transformer
Md Kowsher, Abdul Rafae Khan, Jia Xu
In Chaos, a minor divergence between two initial conditions exhibits exponential amplification over time, leading to far-away outcomes, known as the butterfly effect. Thus, the dis…
Investigating Training Strategies and Model Robustness of Low-Rank Adaptation for Language Modeling in Speech Recognition
Yu Yu, Chao-Han Huck Yang, Tuan Dinh +10
The use of low-rank adaptation (LoRA) with frozen pretrained language models (PLMs) has become increasing popular as a mainstream, resource-efficient modeling approach for memory-c…