Watch the "Deadly Triad" of RL in action. Off-policy sampling with function approximation causes standard semi-gradient TD(0) to exponentially diverge to ∞, while Gradient TD algorithms (GTD0 / TDC) remain stable and converge.
Adjust primary learning rate α, auxiliary step size β, and discount γ.
Updating weights w ← w + α · ρ · δ · φ(s) without gradient correction. Watch weights explode!
Auxiliary weight vector corrects off-policy gradient projection, maintaining bounded stability and driving weights to 0.
Comparing weight norm ||w||₂ trajectory over step transitions (TD(0) red vs GTD/TDC green).