Dividing each trajectory's score contribution by its realized length can change the direction of a reward-centered policy update.
In a finite trajectory model, we derive an exact ascent threshold for a three-outcome softmax policy and exhibit a binary-reward example in which the expected Euclidean update decreases task success.
Natural-gradient ascent holds for fully controllable outcome distributions with deterministic binary labels, but can fail under a one-parameter restriction.
A third-order U-statistic estimates the first-order return change without bias from independent current-policy trajectories, one per request.
In the example, a same-sample plug-in estimator instead has the opposite sign in expectation for batch sizes two through forty-nine.
Exact policy-grid enumeration confirms the local sign reversal.
In a branching process, length-normalized learning initially loses return, then recovers and exceeds the unnormalized comparator.
The analysis concerns the expected update field at the current policy.