Robust Failure, Conservative Repair: Textual Knowledge Distillation from Cross-Model Failures

Andrew Ren, Haokun Liu, Chenhao Tan

Failure-based textual knowledge distillation aims to discover gaps in a model's knowledge by examining its task errors. The distilled knowledge can be useful for the reasoning of b…