Fixed-Dimensional and Permutation Invariant State Representation of Autonomous Driving

Abstract

In this paper, we propose a new state representation method, called encoding sum and concatenation (ESC), for the state representation of decision-making in autonomous driving. Unlike existing state representation methods, ESC is applicable to a variable number of surrounding vehicles and eliminates the need for manually pre-designed sorting rules, leading to higher representation ability and generality. The proposed ESC method introduces a representation neural network (NN) to encode each surrounding vehicle into an encoding vector, and then adds these vectors to obtain the representation vector of the set of surrounding vehicles. By concatenating the set representation with other variables, such as indicators of the ego vehicle and road, we realize the fixed-dimensional and permutation invariant state representation. This paper has further proved that the proposed ESC method can realize the injective representation if the output dimension of the representation NN is greater than the number of variables of all surrounding vehicles. This means that by taking the ESC representation as policy inputs, we can find the nearly optimal representation NN and policy NN by simultaneously optimizing them using gradient-based updating. Experiments demonstrate that compared with the fixed-permutation representation method, the proposed method improves the representation ability of the surrounding vehicles, and the corresponding approximation error is reduced by 62.2%.

References in corpus (13)

• Continuous control with deep reinforcement learning
• Gaussian Error Linear Units (GELUs)
• End to End Learning for Self-Driving Cars
• A Convergence Theory for Deep Learning via Over-Parameterization
• Distributional Soft Actor-Critic: Off-Policy Reinforcement Learning for Addressing Value Estimation Errors
• Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car
• Hierarchical Reinforcement Learning for Self-Driving Decision-Making without Reliance on Labeled Driving Data
• Centralized Cooperation for Connected and Automated Vehicles at Intersections by Proximal Policy Optimization
• Formulation of Deep Reinforcement Learning Architecture Toward Autonomous Driving for On-Ramp Merge
• Adaptive dynamic programming for nonaffine nonlinear optimal control problem with state constraints
• On Learning Sets of Symmetric Elements
• Universal approximations of permutation invariant/equivariant functions by deep neural networks
• Alternating Direction Method of Multipliers for Constrained Iterative LQR in Autonomous Driving

Cited by in corpus (1)

• Encoding Distributional Soft Actor-Critic for Autonomous Driving in Multi-lane Scenarios