A Multimodal GUI Architecture for Interfacing with LLM-Based Conversational Assistants

Hans G. W. van Dam

Advances in large language models (LLMs) and real-time speech recognition now make it possible to issue any graphical user interface (GUI) action through natural language and recei…