Duo-LLM: A Framework for Studying Adaptive Computation in Large Language Models

Keivan Alizadeh, Iman Mirzadeh, Hooman Shahrokhi +6

Large Language Models (LLMs) typically generate outputs token by token using a fixed compute budget, leading to inefficient resource utilization. To address this shortcoming, recen…