首页 > AI前沿 > Geometric Time Score Estimators: Bridging Self-Predictive Representations and Generative Occupancy Models For Reinforcement Learning

Geometric Time Score Estimators: Bridging Self-Predictive Representations and Generative Occupancy Models For Reinforcement Learning

arXiv机器学习 2026-10-29 04:57 3 阅读 查看原文
这个问题是要把给定的纯文本新闻正文排版成语义化的HTML片段,需要遵循几个规则:只输出HTML片段,不用Markdown围栏和解释;每个自然段落用

包裹,长文用

/

划分小节;列表用
  • ,引用用
    ,代码用
    ,重要句子可用;保留原文所有信息,不能增删改;不用文档级标签。首先看正文内容,是一段关于强化学习中GTSE框架的学术内容,没有明显的小节划分,所以不需要

    或

    。接下来处理每个自然段落,这里原文是一个连续的段落,但里面有不同的句子,不过根据规则,每个自然段落用

    包裹,所以需要看原文的结构。原文第一句是“In reinforcement learning (RL), temporal reasoning is essential for estimating the density of future outcomes under a policy's occupancy measure.”,这部分可以作为一个

    的开头吗?不对,原文是一个整体段落,但可能需要拆分吗?不,原文是一个完整的段落,不过再仔细看,原文的内容是连续的,但根据规则,每个自然段落用

    包裹,所以可能需要把整个正文拆成几个自然段落?不对,原文的正文是一个连续的段落,但里面有不同的内容点,不过再看规则,每个自然段落用

    包裹,所以应该把原文的每个自然段落分开。原文的正文是:第一句讲RL中的时间推理重要,第二句讲self-predictive表征的问题,第三句讲生成式占据模型的优势和局限,第四句讲提出GTSE,第五句讲分析GTSE的作用,第六句讲实验结果,第七句是网站链接。现在看这些句子,是否属于不同的自然段落?原文的正文是一个连续的段落,但根据排版规则,可能需要把每个自然段落用

    包裹,不过原文的正文是一个整体,但再仔细看,原文的正文其实是几个不同的句子,但可能需要拆分吗?不对,原文的正文是一个连续的段落,但根据规则,每个自然段落用

    包裹,所以应该把原文的每个自然段落分开。不过再看原文的正文,其实是一段,但里面有不同的内容点,不过可能需要拆分吗?比如,第一句到第三句是背景,第四句到第五句是方法,第六句是实验结果,第七句是网站。但原文的正文是一个连续的段落,所以可能需要把整个正文作为多个

    ?不对,再看规则,每个自然段落用

    包裹,所以原文的正文是一个自然段落吗?不,原文的正文是一个连续的段落,但可能需要拆分成几个自然段落?比如,原文的第一句:“In reinforcement learning (RL), temporal reasoning is essential for estimating the density of future outcomes under a policy's occupancy measure.” 作为第一个

    ;第二句:“While self-predictive representations have been a dominant paradigm for occupancy density estimation, we find that they yield surprisingly poor estimates.” 作为第二个

    ;第三句:“On the other hand, generative occupancy models can produce remarkably accurate samples from the occupancy measure.” 作为第三个

    ;第四句:“Yet, they cannot easily recover occupancy densities, limiting their use for control despite their expressivity.” 作为第四个

    ;第五句:“In this paper, we propose Geometric Time Score Estimators (GTSE), a framework that bridges the two paradigms by learning occupancy density estimators along probability paths.” 作为第五个

    ;第六句:“We analyze how GTSE implicitly induces better occupancy estimators through improved representation learning and show how to directly extract self-predictive representations from this framework.” 作为第六个

    ;第七句:“Through experiments across several locomotion and manipulation tasks, we show that GTSE improves performance over state-of-the-art methods by $2\times$ to $6\times$ in goal-conditioned RL settings, and that the representations learned using our framework transfer effectively to downstream RL tasks.” 作为第七个

    ;第八句:“Website: https://aravindvenu7.github.io/website/gtse/” 作为第八个

    。但这样会不会太碎?不过根据规则,每个自然段落用

    包裹,所以应该把每个自然段落分开。不过再看原文的正文,其实是一段,但可能需要拆分成几个自然段落,因为每个句子表达不同的意思。不过再看规则,保留原文所有信息,不得增删改内容,所以不能拆分句子,只能把每个自然段落用

    包裹。原文的正文是一个连续的段落,所以应该把整个正文作为一个

    ?不对,再看规则,每个自然段落用

    包裹,所以原文的正文是一个自然段落,所以用

    包裹整个正文?但这样可能不符合,因为里面有不同的句子,但根据规则,每个自然段落用

    包裹,所以应该把原文的正文作为多个

    ?比如,原文的正文是: In reinforcement learning (RL), temporal reasoning is essential for estimating the density of future outcomes under a policy's occupancy measure. While self-predictive representations have been a dominant paradigm for occupancy density estimation, we find that they yield surprisingly poor estimates. On the other hand, generative occupancy models can produce remarkably accurate samples from the occupancy measure. Yet, they cannot easily recover occupancy densities, limiting their use for control despite their expressivity. In this paper, we propose Geometric Time Score Estimators (GTSE), a framework that bridges the two paradigms by learning occupancy density estimators along probability paths. We analyze how GTSE implicitly induces better occupancy estimators through improved representation learning and show how to directly extract self-predictive representations from this framework. Through experiments across several locomotion and manipulation tasks, we show that GTSE improves performance over state-of-the-art methods by $2\times$ to $6\times$ in goal-conditioned RL settings, and that the representations learned using our framework transfer effectively to downstream RL tasks. Website: https://aravindvenu7.github.io/website/gtse/ 现在,这个正文是一个连续的段落,但根据规则,每个自然段落用

    包裹,所以应该把整个正文作为一个

    ?但这样可能不符合,因为里面有不同的句子,但根据规则,每个自然段落用

    包裹,所以应该把每个自然段落分开。比如,第一句到第三句是一个自然段落,第四句到第五句是另一个,第六句是另一个,第七句是另一个,第八句是另一个。但这样会不会增删内容?不会,只是用

    包裹每个自然段落。不过再看原文的正文,其实是一段,但可能需要拆分成几个自然段落,因为每个句子表达不同的意思。不过根据规则,保留原文所有信息,不得增删改内容,所以不能拆分句子,只能把每个自然段落用

    包裹。所以正确的做法是把每个自然段落用

    包裹,原文的正文是一个自然段落,所以用

    包裹整个正文?不对,再看规则,每个自然段落用

    包裹,所以原文的正文是一个自然段落,所以用

    包裹整个正文。但再看例子,比如用户给的正文是一个连续的段落,但需要排版成HTML片段,所以应该把整个正文作为一个

    ?不过再看规则,长文用

    /

    划分小节,但这里没有小节,所以不需要。列表内容用
    • ,但这里没有列表,所以不需要。引用用
      ,但这里没有引用,所以不需要。代码用
      ,但这里没有代码,所以不需要。重要句子可用,所以需要找出重要句子,比如“we propose Geometric Time Score Estimators (GTSE)”可以用包裹,“GTSE improves performance over state-of-the-art methods by $2\times$ to $6\times$”可以用包裹。所以正确的HTML片段应该是:
      
      

      In reinforcement learning (RL), temporal reasoning is essential for estimating the density of future outcomes under a policy's occupancy measure. While self-predictive representations have been a dominant paradigm for occupancy density estimation, we find that they yield surprisingly poor estimates. On the other hand, generative occupancy models can produce remarkably accurate samples from the occupancy measure. Yet, they cannot easily recover occupancy densities, limiting their use for control despite their expressivity.

      In this paper, we propose Geometric Time Score Estimators (GTSE), a framework that bridges the two paradigms by learning occupancy density estimators along probability paths. We analyze how GTSE implicitly induces better occupancy estimators through improved representation learning and show how to directly extract self-predictive representations from this framework.

      Through experiments across several locomotion and manipulation tasks, we show that GTSE improves performance over state-of-the-art methods by $2\times$ to $6\times$ in goal-conditioned RL settings, and that the representations learned using our framework transfer effectively to downstream