首页 > AI前沿 > Constructing Dynamic Conversations with Neural Back-channel Generation and Voice-line Scheduling

Constructing Dynamic Conversations with Neural Back-channel Generation and Voice-line Scheduling

arXiv自然语言 2026-11-18 17:13 15 阅读 查看原文

We present a novel approach to the problem of combining two voice-line channels of dyadic conversational audio into a unified conversation with controllable back-channeling and turn-taking dynamics. In voice performances for the game industry, the two tracks of voice-lines are typically recorded or synthesized in isolation. Our method takes two such unsynchronized tracks and uses a neural scheduler to sequence them into a single dynamic conversation, modifying both tracks by inserting or removing pauses and introducing voice overlap as appropriate. We further propose a back-channel predictor that is able to predict opportunity for back-channels for a given conversational turn's audio and content, which the scheduler can then incorporate. We evaluate our scheduler and predictor independently and in combination by:

  • various quantitative comparison;
  • a qualitative and perceptual comparison to prior art and naive baselines;
  • a compelling set of results showing animator control over the diversity, extent and nature of back-channeling and turn dynamics.