Dissecting and Re-architecting 3D NAND Flash PIM Arrays for Efficient Single-Batch Token Generation in LLMs

Yongjoo Jang, Sangwoo Hwang, Hojin Lee +4

The advancement of large language models has led to models with billions of parameters, significantly increasing memory and compute demands. Serving such models on conventional har…