SMA-X: Versatile information sharing in and around telescopes, and beyond

Abstract

We developed the SMA eXchange (SMA-X) as a real-time data sharing solution, built atop a central Redis database. SMA-X is a storage convention, facilitated by a set of server-side Lua scripts (or Redis functions) which enable efficient low-latency and high-throughput real-time sharing of hierarchically structured data among the various systems and subsystems of the telescope. It enables fast, atomic retrievals of specific leaf elements, branches, and sub-trees, including associated metadata (types, dimensions, timestamps, and origins, and more). At the Submillimer Array (SMA) we rely on it since 2021 to share a diverse set of ~10,000 real-time variables, including arrays, across more than 100 computers, with information being published every 10 ms in some cases. SMA-X is open-source, and is freely available to everyone through a set of public GitHub repositories, including C/C++ and Python3 libraries to allow integration with observatory applications. A set of command-line tools provide access to the database from the POSIX shell and/or from any scripting language, and we also provide a configurable system daemon for archiving the observatory state at regular intervals into a time-series Structured Query Language (SQL) database to create a detailed historical record.

15 pages, 2 figures, an earlier version of this paper was presented at SPIE Conference 13101