Bigger Models Remember the Rare Stuff Long Enough to Learn It
A scaling paper argues that larger models win partly because frequent tasks stop overwriting rare-task features. The proposed mechanism links parameter scale to reduced interference and better retention of infrequent capabilities over time.