Existing multi-task active learning (MTL-AL) methods predominantly rely on linear scalarization of heterogeneous objectives such as uncertainty, diversity, and gradient conflict.
We argue that this entangled formulation is fundamentally limited, as linear combination fails to capture Pareto-optimal trade-offs under conflicting objectives.
We propose a shift from combination to deconstruction
We introduce a principled framework that rethinks MTL-AL along three orthogonal dimensions.
First
We identify the Conflict Paradox: although the optimization literature treats gradient conflict as interference to be suppressed, penalizing conflict when ranking candidates is measurably worse than not scoring it at all.
We therefore decouple the two concerns: the acquisition function does not score conflict, and the resulting interference is resolved by gradient surgery during training.
Second
We introduce Orthogonal Decomposition, separating acquisition into Informativeness and Exploration.
This two-stage process, informativeness filtering followed by coverage maximization, prevents "diversity dilution" by excluding uninformative outliers before enforcing diversity.
Third
We propose Cross-Task Mutual Information (CTMI), a covariance-based statistic of cross-task dependency computed from MC-dropout task entropies.
Experiments and Results
Extensive experiments on COCO, Cityscapes, and NYUv2, including comparisons against recent MTL-AL and data-selection baselines, demonstrate consistent improvements over state-of-the-art methods.
Highlighting that structural rethinking of acquisition yields larger gains than incremental heuristic design.
Code is available at https://github.com/Juho-Jung/multitask_active_learning_official.