Watch standard deep neural networks (Vanilla SGD) suffer from Loss of Plasticity across 50 sequential task permutations, while Selective Weight Reinitialization (SWR) and Continual Backprop (CBP) maintain ~95% accuracy indefinitely.
Adjust SWR reinit threshold k and CBP replacement rate ρ.
Suffers from severe Loss of Plasticity as dead units accumulate and features collapse.
Periodically reinitializes low-utility weights and resets momentum, preserving network health.
Continuously replaces mature low-utility neurons and re-routes bias projections.
Online evaluation accuracy before training on each 784-dim input pixel permutation task.