4 papers
ICaRus: Identical Cache Reuse for Efficient Multi Model Inference
Sunghyeon Woo, Jaeeun Kil, Hoseung Kim +9
Multi model inference has recently emerged as a prominent paradigm, particularly in the development of agentic AI systems. However, in such scenarios, each model must maintain its…
PrefillShare: A Shared Prefill Module for KV Reuse in Multi-LLM Disaggregated Serving
Sunghyeon Woo, Hoseung Kim, Sunghwan Shim +8
Multi-agent systems increasingly orchestrate multiple specialized language models to solve complex real-world problems, often invoking them over a shared context. This execution pa…
Chopping Trees: Semantic Similarity Based Dynamic Pruning for Tree-of-Thought Reasoning
Joongho Kim, Xirui Huang, Zarreen Reza +1
Tree-of-Thought (ToT) reasoning boosts the problem-solving abilities of Large Language Models (LLMs) but is computationally expensive due to semantic redundancy, where distinct bra…
Command A: An Enterprise-Ready Large Language Model
Team Cohere, :, Aakanksha +227
In this report we describe the development of Command A, a powerful large language model purpose-built to excel at real-world enterprise use cases. Command A is an agent-optimised…