首页 > AI前沿 > Infrastructure-Based Multi-Agent Reinforcement Learning with Attention for Highway Ramp Merging

Infrastructure-Based Multi-Agent Reinforcement Learning with Attention for Highway Ramp Merging

arXiv机器学习 2026-11-15 23:44 24 阅读 查看原文
用户现在需要把一段新闻正文排版成语义化HTML片段,按照要求,每个自然段落用p,重要句子用strong?不对,原文里没有需要特别强调的?不对,首先看正文,首先整个正文是一个大段落?不对,原文的正文是连续的?不对,等下,用户给的正文是一段?不对,再看,哦,整个正文是关于匝道汇入的研究的内容,首先需要把每个自然段?不对,原文是不是一个自然段?不对,再仔细看,用户给的正文是一整段?不对,等下,我看看:从“On-ramp merging is a typical safety and throughput problem on highways.”到“...multiple traffic situations.”是一整个段落?不对,不对,有没有可以分小节的?哦,不对,看内容,首先第一部分是问题背景,然后是提出的方法,然后是评估,然后是结果?不对,不对,规则里说长文用h2/h3划分小节,但首先要保留所有内容,不能增删。等下,先梳理: 首先,原文的段落,哦,用户给的正文是一整段吗?不对,再看,哦,不对,用户给的正文是连续的一段?不对,等下,仔细看: 正文是: On-ramp merging is a typical safety and throughput problem on highways. Without proper coordination, human drivers and self-interested autonomous vehicles may make risky manoeuvres and cause accidents or congestion. With recent advances in connected autonomous vehicles (CAVs) and multi-agent reinforcement learning (MARL), vehicles can be trained to coordinate for safe and efficient merging. However, in dense traffic, forming an effective observation of the traffic state is challenging. For example, observing only the neighbouring vehicles may lead to locally optimal decisions, while observing the entire traffic can introduce much irrelevant information. In this paper, we propose a transformer-based MARL vehicle control model to address these problems. A roadside infrastructure controller receives the states of all CAVs via vehicle-to-infrastructure (V2I) communication and sends control commands back to them. An attention-based observation encoder is designed where each vehicle can filter the relevant vehicle information from the global traffic state. The proposed model is evaluated in SUMO against three baselines: a model that observes only the neighbouring vehicles, a model that encodes the global traffic information, and the rule-based SUMO model. The results show that our model achieves the highest traffic flow, safety and smoothness. Furthermore, in a four-hour simulation with dynamic traffic demand, our model successfully avoids traffic jams, while other baseline models fail at high traffic flow. We also show that our model remains effective when human-driven vehicles are present. We further show that our model, trained in ideal communication conditions, can still remain collision-free and maintain throughput under realistic V2I communication errors. Finally, we visualise the relevance weights that an ego vehicle assigns to other vehicles in multiple traffic situations. 哦,这是一整段?那需要怎么划分?不对,有没有可以分成小节的?比如,首先,问题背景部分,然后是方法部分,然后是实验评估部分,然后是结果部分?但要注意不能增删内容,所以小节标题是不是不能用原文没有的?哦不对,规则里说保留原文所有信息,不得增删改内容,所以h2/h3的内容必须是原文里的?不对,不对,等下,看原文里有没有可以作为小节标题的?哦,不对,原文没有小标题,那是不是可以不用h2/h3?不对,规则说“长文用

/

划分小节”,如果不需要的话也可以?不对,再看,哦,不对,用户给的正文是学术论文的摘要?那有没有需要划分的?等下,再仔细看规则,3.重要句子可用,那原文里的关键句子,比如提出的模型,结果之类的?不对,首先,每个自然段落用p包裹,原文是一个自然段?不对,不对,等下,原文的句子都是连续的,那整个是一个p?不对,不对,再想,哦,不对,有没有可能我理解错了?不,用户给的正文就是一整段,那排版的话,首先,有没有需要强调的?比如,“In this paper, we propose a transformer-based MARL vehicle control model to address these problems.”这个可以加strong?还有“The results show that our model achieves the highest traffic flow, safety and smoothness.”也可以?然后,有没有列表?哦,原文里有“three baselines: a model that observes only the neighbouring vehicles, a model that encodes the global traffic information, and the rule-based SUMO model.”这里可以改成列表?对,因为是列举三个基线,所以可以把冒号后面的改成ul li。哦,对,这样更语义化。那现在梳理: 首先,第一个p,包含前面的背景部分:

On-ramp merging is a typical safety and throughput problem on highways. Without proper coordination, human drivers and self-interested autonomous vehicles may make risky manoeuvres and cause accidents or congestion. With recent advances in connected autonomous vehicles (CAVs) and multi-agent reinforcement learning (MARL), vehicles can be trained to coordinate for safe and efficient merging. However, in dense traffic, forming an effective observation of the traffic state is challenging. For example, observing only the neighbouring vehicles may lead to locally optimal decisions, while observing the entire traffic can introduce much irrelevant information.

然后,方法部分,可以用h2?不对,原文没有小标题,那h2的话不能自己加啊,哦不对,规则说不得增删改内容,所以不能自己加小标题文字,那是不是可以不用h2/h3?不对,等下,再看,哦,不对,规则是“长文用

/

划分小节”,但如果原文没有分节,那是不是可以不用?或者,有没有可能把提出方法和实验部分分开?不对,不能加标题啊,哦,不对,哦,等下,我是不是错了,规则说“保留原文所有信息,不得增删改内容”,所以不能添加任何原文没有的文字,包括小标题。那h2/h3如果用的话,内容必须是原文里的?那原文里有没有可以作为标题的句子?比如“In this paper, we propose a transformer-based MARL vehicle control model to address these problems.”这个可以作为h2?然后下面的内容是方法的细节,然后实验部分?不对,再想想,哦,对,用户给的是正文,可能就是摘要,那可以这样: 首先,背景部分用p:

On-ramp merging is a typical safety and throughput problem on highways. Without proper coordination, human drivers and self-interested autonomous vehicles may make risky manoeuvres and cause accidents or congestion. With recent advances in connected autonomous vehicles (CAVs) and multi-agent reinforcement learning (MARL), vehicles can be trained to coordinate for safe and efficient merging. However, in dense traffic, forming an effective observation of the traffic state is challenging. For example, observing only the neighbouring vehicles may lead to locally optimal decisions, while observing the entire traffic can introduce much irrelevant information.

然后,用h2?不对,不能加标题,哦,不对,哦,等下,我是不是理解错了,h2/h3是用来划分小节,但是小节的标题可以用原文里的核心句子?比如:

Proposed Model

?不对,不行,原文里没有“Proposed Model”这个文字,不能加。哦,对哦,不能增删内容,所以不能自己造小标题。那那怎么办?那是不是就不需要用h2/h3?因为原文没有分节,所以直接用p,然后处理里面的列表和强调。 哦,对,原文里的“three baselines: a model that observes only the neighbouring vehicles, a model that encodes the global traffic information, and the rule-based SUMO model.”这里可以把三个基线改成ul li,这样更语义化。那现在组织: 首先,第一个p,包含前面的背景,然后提出模型的句子用strong?比如:

On-ramp merging is a typical safety and throughput problem on highways. Without proper coordination, human drivers and self-interested autonomous vehicles may make risky manoeuvres and cause accidents or congestion. With recent advances in connected autonomous vehicles (CAVs) and multi-agent reinforcement learning (MARL), vehicles can be trained to coordinate for safe and efficient merging. However, in dense traffic, forming an effective observation of the traffic state is challenging. For example, observing only the neighbouring vehicles may lead to locally optimal decisions, while observing the entire traffic can introduce much irrelevant information. In this paper, we propose a transformer-based MARL vehicle control model to address these problems.

然后,接下来是模型的细节,用p:

A roadside infrastructure controller receives the states of all CAVs via vehicle-to-infrastructure (V2I) communication and sends control commands back to them. An attention-based observation encoder is designed where each