CFM-Bench: A Unified Multi-Domain, Multi-Task Benchmark for Channel Foundation Models
arXiv:2607.14975
The paper introduces CFM-Bench, a unified benchmark that evaluates channel foundation models across multiple wireless domains, tasks, and data configurations, enabling fair comparison of model transferability and downstream performance.
Abstract
Channel foundation models (CFMs) are commonly evaluated in model-specific pipelines that differ in data, radio configurations, partitions, adaptation procedures, task definitions, and metrics, preventing reproducible comparison across CFMs and against task-specific networks. We release CFM-Bench, a unified multi-domain, multi-task benchmark comprising 157,900 official single-frame examples from six domains spanning 3GPP statistical simulation, two ray-tracing pipelines, terrestrial and aerial measurements, and synchronized vehicular multimodal simulation. Source-specific interfaces preserve complex channel state information (CSI) and the physical metadata available in each domain while allowing documented model-specific preprocessing. To reduce spatio-temporal leakage, official partitions isolate complete trajectories, measurement sessions, flights, vehicle links, simulation realizations, or buffered spatial regions. CFM-Bench excludes all benchmark splits from foundation-model pretraining, reserves the official training split for downstream fine-tuning, and reports the data used during model development. Six task groups across PHY, RAN, and ISAC applications cover CSI feedback, frequency and temporal channel extrapolation, propagation-state classification, current- and future-beam prediction, and single-frame and temporal localization. Representative experiments on CSI feedback, channel extrapolation, current-beam prediction, and wireless positioning provide reproducible reference results for pretrained channel-prediction models and task-specific neural networks. These results show that relative model performance can vary across data domains, highlighting the importance of using common data partitions, task definitions, and evaluation metrics. CFM-Bench provides a common substrate for evaluating the transferability of channel representations across models, domains, and tasks.
CFM-Bench dataset download: https://www.chaspark.com/\#/s/CFM-Bench