Trust-Region Method with Deep Reinforcement Learning in Analog Design Space Exploration

Abstract

This paper introduces new perspectives on analog design space search. To minimize the time-to-market, this endeavor better cast as constraint satisfaction problem than global optimization defined in prior arts. We incorporate model-based agents, contrasted with model-free learning, to implement a trust-region strategy. As such, simple feed-forward networks can be trained with supervised learning, where the convergence is relatively trivial. Experiment results demonstrate orders of magnitude improvement on search iterations. Additionally, the unprecedented consideration of PVT conditions are accommodated. On circuits with TSMC 5/6nm process, our method achieve performance surpassing human designers. Furthermore, this framework is in production in industrial settings.

arXiv comment6 pages, 3 figures, 5 tables

References in corpus (9)

• Continuous control with deep reinforcement learning
• Neural Architecture Search with Reinforcement Learning
• Trust Region Policy Optimization
• End-to-End Training of Deep Visuomotor Policies
• Model-Based Reinforcement Learning for Atari
• GCN-RL Circuit Designer: Transferable Transistor Sizing with Graph Neural Networks and Reinforcement Learning
• AI Safety Gridworlds
• An Efficient Multi-fidelity Bayesian Optimization Approach for Analog Circuit Synthesis
• Learning to Design Circuits

Cited by in corpus (1)

• AnaFlow: Agentic LLM-based Workflow for Reasoning-Driven Explainable and Sample-Efficient Analog Circuit Sizing