Chip Placement with Deep Reinforcement Learning

Abstract

In this work, we present a learning-based approach to chip placement, one of the most complex and time-consuming stages of the chip design process. Unlike prior methods, our approach has the ability to learn from past experience and improve over time. In particular, as we train over a greater number of chip blocks, our method becomes better at rapidly generating optimized placements for previously unseen chip blocks. To achieve these results, we pose placement as a Reinforcement Learning (RL) problem and train an agent to place the nodes of a chip netlist onto a chip canvas. To enable our RL policy to generalize to unseen blocks, we ground representation learning in the supervised task of predicting placement quality. By designing a neural architecture that can accurately predict reward across a wide variety of netlists and their placements, we are able to generate rich feature embeddings of the input netlists. We then use this architecture as the encoder of our policy and value networks to enable transfer learning. Our objective is to minimize PPA (power, performance, and area), and we show that, in under 6 hours, our method can generate placements that are superhuman or comparable on modern accelerator netlists, whereas existing baselines require human experts in the loop and take several weeks.

References in corpus (4)

• Link Prediction Based on Graph Neural Networks
• GAP: Generalizable Approximate Graph Partitioning Framework
• Placeto: Learning Generalizable Device Placement Algorithms for Distributed Machine Learning
• GDP: Generalized Device Placement for Dataflow Graphs

Cited by in corpus (28)

• Benchmarking Graph Neural Networks
• A Survey of Machine Learning for Computer Architecture and Systems
• Quantum circuit optimization with deep reinforcement learning
• PrefixRL: Optimization of Parallel Prefix Circuits using Deep Reinforcement Learning
• Learning Collaborative Policies to Solve NP-hard Routing Problems
• Learning the Travelling Salesperson Problem Requires Rethinking Generalization
• Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search
• CLDG: Contrastive Learning on Dynamic Graphs
• Machine Learning for Electronic Design Automation: A Survey
• Deep Learning in Protein Structural Modeling and Design
• Large-scale graph representation learning with very deep GNNs and self-supervision
• Revisiting Design Choices in Proximal Policy Optimization
• mGNN: Generalizing the Graph Neural Networks to the Multilayer Case
• Automated Optical Multi-layer Design via Deep Reinforcement Learning
• Placement in Integrated Circuits using Cyclic Reinforcement Learning and Simulated Annealing
• Proportional integral derivative controller assisted reinforcement learning for path following by autonomous underwater vehicles
• Learning to Pool in Graph Neural Networks for Extrapolation
• Optimizing Memory Placement using Evolutionary Graph Reinforcement Learning
• Guiding Global Placement With Reinforcement Learning
• A Graph Deep Learning Framework for High-Level Synthesis Design Space Exploration
• Data-Driven Offline Optimization For Architecting Hardware Accelerators
• NVCell: Standard Cell Layout in Advanced Technology Nodes with Reinforcement Learning
• Program-to-Circuit: Exploiting GNNs for Program Representation and Circuit Translation
• Robust Risk-Sensitive Reinforcement Learning Agents for Trading Markets
• DeepNVM++: Cross-Layer Modeling and Optimization Framework of Non-Volatile Memories for Deep Learning
• Active Offline Policy Selection
• Safety Aware Reinforcement Learning (SARL)
• Fast IR Drop Estimation with Machine Learning