Mixing between the Cross Entropy and the Expectation Loss Terms

Barak Battash, Lior Wolf, Tamir Hazan

The cross entropy loss is widely used due to its effectiveness and solid theoretical grounding. However, as training progresses, the loss tends to focus on hard to classify samples…