Large audio-language models (LALMs) exploit multimodal evidence, yet task-irrelevant audio can alter text-reasoning decisions when listening is unnecessary.
Aggregate Accuracy can hide this paired drift because audio-induced repairs and damages may cancel.
Paired drift analysis and targeted interventions identify architecture-specific, intervention-sensitive late audio pathways as actionable control points.
We introduce ICAP-Gate, which applies mechanism-guided, task-conditioned control to each model's pathway.
Across four LALMs, two reasoning benchmarks, and environmental-sound and natural-speech interference, ICAP-Gate has lower point estimates for Influence Rate and Answer Flip than ungated inference in all 16 full-split model--condition evaluations.
Fixed suppression degrades automatic speech recognition (ASR) across all four models, whereas ICAP-Gate matches ungated ASR performance by preserving the pathway for explicit audio-demand instructions.
ICAP-Gate has lower paired-drift point estimates than mitigation prompting in all four evaluated settings and provides competitive stabilization relative to eight-sample Self-Consistency while using one generation per query; in controlled ARC measurements, Self-Consistency incurs $7.0$--$9.2\times$ ungated latency.
These results establish selective modality influence control as a design principle for robust multimodal reasoning.