首页 > AI前沿 > Intelligent Embedding Pruning for Large-Scale Recommendation Models

Intelligent Embedding Pruning for Large-Scale Recommendation Models

arXiv机器学习 2026-11-19 07:26 11 阅读 查看原文
用户希望将一段纯文本新闻正文排版成语义化的HTML片段。 需要遵循以下规则: 1. 只输出HTML片段本身,不要Markdown围栏、不要解释。 2. 每个自然段落用`

`包裹,长文用`

`/`

`划分小节。 3. 列表内容用`
  • `,引用用`
    `,代码用`
    `,重要句子可用``。
    4. 保留原文所有信息,不得增删改内容。
    5. 不要输出``/``/``等文档级标签。
    
    分析正文内容:
    这是一段关于推荐系统嵌入剪枝(FI-ITEP)的技术新闻/摘要。
    可以分为几个小节:
    - 背景与问题(Modern recommendation systems... heuristics.)
    - 方法(We use feature importance... less important features.)
    - 实验结果与系统收益(Across three large-scale... GPU-hours per model launch.)
    - 开源复现与指导(We reproduce these trends... real-world systems.)
    
    由于文本不长,可以划分为几个小节,用`

    `或`

    `,或者直接作为段落用`

    `,但规则提到“长文用`

    `/`

    `划分小节”。为了体现排版,可以加几个小标题,但不能增删改内容,加小标题算不算“增”内容? 规则4:“保留原文所有信息,不得增删改内容”。如果我自己加小标题,就增加了原文没有的内容。因此,最好不加自创的小标题,直接将原文的自然段落用`

    `包裹,或者把原文中的逻辑层次用``强调,或者如果原文本身没有小标题,就不强行加`

    `/`

    `,以免违反“不得增删改内容”的规则。 等等,原文是一整段。如果分成几个`

    `,算不算“改内容”?通常分段排版是允许的,只要文字内容不增删。原文是一整段,我可以把它拆成几个自然段落,或者就保持一个`

    `。 为了更好排版,可以把原文按句群拆分为几个`

    `,或者用``标记重要句子。 原文: "Modern recommendation systems face substantial memory pressure, largely from the sparse embedding layer that maps high-cardinality categorical features into dense vectors. In-training embedding pruning (ITEP) removes less important embedding rows during training, yielding end-to-end memory savings but sometimes degrading accuracy because per-table pruning ratios are set by size-based heuristics. We use feature importance (FI) to guide these ratios (FI-ITEP), assigning higher pruning to tables tied to less important features. Across three large-scale recommendation models, FI-ITEP achieves 20--40% total memory reduction while matching or improving accuracy relative to percentile-based ITEP. Because the feature-importance signal requires minimal resources to compute, these memory savings translate into concrete systems benefits: the same models train on fewer GPU trainers (e.g., 112 to 80) with higher per-GPU throughput, saving 1,000+ GPU-hours per model launch. We reproduce these trends on the open-source DLRM (Criteo 1TB) and quantify the end-to-end training-time and GPU-cost tradeoff---including the burden of computing FI---offering practical guidance for embedding pruning in real-world systems." 可以将其分为几个段落: 段落1:背景与现有方法的问题。 段落2:提出的方法。 段落3:实验结果。 段落4:系统收益。 段落5:开源复现与指导。 或者直接保留为一个段落,但用``强调关键句。 不过规则2说“每个自然段落用`

    `包裹”,原文是一个自然段,那我就用一个`

    `包裹,或者在合理的地方换行分成几个`

    `。为了安全,不增加新文本,直接将原文作为整体放入`

    `,或者按句子逻辑拆分成几个`

    `(不改变文字)。拆分段落不算增加内容,只是改变排版。 我们可以拆分为:

    Modern recommendation systems face substantial memory pressure, largely from the sparse embedding layer that maps high-cardinality categorical features into dense vectors. In-training embedding pruning (ITEP) removes less important embedding rows during training, yielding end-to-end memory savings but sometimes degrading accuracy because per-table pruning ratios are set by size-based heuristics.

    We use feature importance (FI) to guide these ratios (FI-ITEP), assigning higher pruning to tables tied to less important features. Across three large-scale recommendation models, FI-ITEP achieves 20--40% total memory reduction while matching or improving accuracy relative to percentile-based ITEP.

    Because the feature-importance signal requires minimal resources to compute, these memory savings translate into concrete systems benefits: the same models train on fewer GPU trainers (e.g., 112 to 80) with higher per-GPU throughput, saving 1,000+ GPU-hours per model launch. We reproduce these trends on the open-source DLRM (Criteo 1TB) and quantify the end-to-end training-time and GPU-cost tradeoff---including the burden of computing FI---offering practical guidance for embedding pruning in real-world systems.

    检查是否有需要列表、引用、代码的地方?原文没有,所以不需要强行使用。 检查是否增删改内容: 原文: Modern recommendation systems face substantial memory pressure, largely from the sparse embedding layer that maps high-cardinality categorical features into dense vectors. In-training embedding pruning (ITEP) removes less important embedding rows during training, yielding end-to-end memory savings but sometimes degrading accuracy because per-table pruning ratios are set by size-based heuristics. We use feature importance (FI) to guide these ratios (FI-ITEP), assigning higher pruning to tables tied to less important features. Across three large-scale recommendation models, FI-ITEP achieves 20--40% total memory reduction while matching or improving accuracy relative to percentile-based ITEP. Because the feature-importance signal requires minimal resources to compute, these memory savings translate into concrete systems benefits: the same models train on fewer GPU trainers (e.g., 112 to 80) with higher per-GPU throughput, saving 1,000+ GPU-hours per model launch. We reproduce these trends on the open-source DLRM (Criteo 1TB) and quantify the end-to-end training-time and GPU-cost tradeoff---including the burden of computing FI---offering practical guidance for embedding pruning in real-world systems. 我的拆分: P1: Modern recommendation systems face substantial memory pressure, largely from the sparse embedding layer that maps high-cardinality categorical features into dense vectors. In-training embedding pruning (ITEP) removes less important embedding rows during training, yielding end-to-end memory savings but sometimes degrading accuracy because per-table pruning ratios are set by size-based heuristics. P2: We use feature importance (FI) to guide these ratios (FI-ITEP), assigning higher pruning to tables tied to less important features. Across three large-scale recommendation models, FI-ITEP achieves 20--40% total memory reduction while matching or improving accuracy relative to percentile-based ITEP. P3: Because the feature-importance signal requires minimal resources to compute, these memory savings translate into concrete systems benefits: the same models train on fewer GPU trainers (e.g., 112 to 80) with higher per-GPU throughput, saving 1,000+ GPU-hours per model launch. We reproduce these trends on the open-source DLRM (Criteo 1TB) and quantify the end-to-end training-time and GPU-cost tradeoff---including the burden of computing FI---offering practical guidance for embedding pruning in real-world systems. 完全一致,没有增删改。 重要句子用``: "20--40% total memory reduction" "1,000+ GPU-hours per model launch" 不需要加`

    `/`

    `,因为原文