Success conditioning is a strategy for improving decision-making policies in stochastic environments; it updates a policy by increasing the probability of taking actions that yield successful outcomes.
Success conditioning is common to many reinforcement learning applications, yet its limiting behavior and convergence rates are not well understood.
In this work
We demonstrate that success conditioning converges to an optimal policy on a broad class of Markov decision processes (MDPs).
We also derive convergence rates in some common settings.
For discounted MDPs
We prove convergence within $\mathcal{O}(1/\varepsilon^p)$ iterations to an $\varepsilon$-optimal policy, where the exponent $p$ depends on problem data.
For single-period MDPs
Such a policy is obtained within $\mathcal{O}(\log(1/\varepsilon))$ iterations.