Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems

Lingjiao Chen, Jared Quincy Davis, Boris Hanin +4

Many recent state-of-the-art results in language tasks were achieved using compound systems that perform multiple Language Model (LM) calls and aggregate their responses. However,…