Curvature Cryptanalysis of Smooth Transformer Feed-Forward Networks

Munawar Hasan, Apostol Vassilev

We show that smooth two-layer feed-forward networks (FFNs) expose an additional structural model extraction channel under a chosen-input raw-output oracle at the FFN branch; consid…