PACSET (Packed Serialized Trees): Reducing Inference Latency for Tree Ensemble Deployment

Meghana Madhyastha, Kunal Lillaney, James Browne +2

We present methods to serialize and deserialize tree ensembles that optimize inference latency when models are not already loaded into memory. This arises whenever models are large…