What We Observe as LLM Behavior Can Be a Side-effect of Inference Backend

Shahed Masoudian, Passant Shafaei, Monorama Swain +1

Benchmark scores are reported as properties of a model, yet the inference framework used to produce them, such as HuggingFace, vLLM, or Ollama, are considered non-influential and t…