Introduction
Many modern learning problems require comparing probability distributions whose individual observations are themselves probability measures, naturally giving rise to meta-measures in $\mathcal{P}(\mathcal{P}(Ω))$. However, optimal transport distances on this space can be prohibitively expensive to evaluate.
Proposed Method
We introduce the vertical sliced Wasserstein ($\mathrm{VSW}$) distance, a scalable metric on $\mathcal{P}(\mathcal{P}(Ω))$. Rather than slicing along the transport geometry of $\mathcal{P}(Ω)$, $\mathrm{VSW}$ projects each inner measure using continuous test functions and compares the resulting one-dimensional distributions with one-dimensional Wasserstein distance.
We prove that $\mathrm{VSW}$ is a valid metric on $\mathcal{P}(\mathcal{P}(Ω))$, metrizes weak convergence, and admits dimension-independent statistical convergence guarantees.
Empirical Results
Empirically, $\mathrm{VSW}$ provides high accuracy and efficiency compared with existing distances for meta-measures.
We further derive a practical particle gradient for the empirical $\mathrm{VSW}_2^2$ objective and demonstrate its use for gradient-based transport of meta-measures.
Conclusion
$\mathrm{VSW}$ provides a simple and scalable alternative for comparing and optimizing distributions of distributions.