This paper presents a comprehensive comparison of four representative reinforcement learning (RL) methods in autonomous vehicle (AV) decision-making, including:
- Fitted Value Iteration (FVI)
- Rollout Classification Policy Iteration (RCPI)
- Deep Q-Network (DQN)
- a discrete-action adaptation of Deep Deterministic Policy Gradient (D-DDPG)
The four methods are applied to the same AV decision-making problem at an unsignalized intersection and are compared in terms of optimality and safety, training stability, and training and decision cost.
RCPI is the closest to a reference solution and has the most stable training process. However, it is the least data-efficient.
D-DDPG reaches the second-smallest gap to a reference solution with the highest data efficiency, but has the least training stability.
DQN is the most stable model-free method but has the highest collision rate.
FVI achieves a lower collision rate than DQN, but is less data-efficient and less stable.
Based on these results, we provide suggestions for selecting a method according to whether a model is available, the budget for training, and whether stable training is required.