Neural collapse with unconstrained features

Dustin G. Mixon, Hans Parshall, Jianzong Pi

Neural collapse is an emergent phenomenon in deep learning that was recently discovered by Papyan, Han and Donoho. We propose a simple "unconstrained features model" in which neural collapse also emerges empirically. By studying this model, we provide some explanation for the emergence of neural collapse in terms of the landscape of empirical risk.

Knowledge Graph

arrow_drop_up

Comments

Sign up or login to leave a comment