5 papers
The Strong Lottery Ticket Hypothesis for Multi-Head Attention Mechanisms
Hikari Otsuka, Daiki Chijiwa, Yasuyuki Okoshi +3
The strong lottery ticket hypothesis (SLTH) conjectures that high-performing subnetworks, called strong lottery tickets (SLTs), are hidden in randomly initialized neural networks.…
Binary Quadratic Quantization: Beyond First-Order Quantization for Real-Valued Matrix Compression
Kyo Kuroki, Yasuyuki Okoshi, Thiem Van Chu +2
This paper proposes a novel matrix quantization method, Binary Quadratic Quantization (BQQ). In contrast to conventional first-order quantization approaches, such as uniform quanti…
Annealed Mean Field Descent Is Highly Effective for Quadratic Unconstrained Binary Optimization
Kyo Kuroki, Thiem Van Chu, Masato Motomura +1
In recent years, formulating various combinatorial optimization problems as Quadratic Unconstrained Binary Optimization (QUBO) has gained significant attention as a promising appro…
Partially Frozen Random Networks Contain Compact Strong Lottery Tickets
Hikari Otsuka, Daiki Chijiwa, Ángel López García-Arias +6
Randomly initialized dense networks contain subnetworks that achieve high accuracy without weight learning--strong lottery tickets (SLTs). Recently, Gadhikar et al. (2023) demonstr…
Multicoated and Folded Graph Neural Networks with Strong Lottery Tickets
Jiale Yan, Hiroaki Ito, Ángel López García-Arias +5
The Strong Lottery Ticket Hypothesis (SLTH) demonstrates the existence of high-performing subnetworks within a randomly initialized model, discoverable through pruning a convolutio…