Dustin G. Mixon, Hans Parshall, Jianzong Pi
2020.11.23Sampling Theory Signal Processing and Data Analysis
tlooto Summary
This work proposes a simple unconstrained features model in which neural collapse also emerges empirically, and provides some explanation for the emergence of neural collapse in terms of the landscape of empirical risk.
Abstract
Neural collapse is an emergent phenomenon in deep learning that was recently discovered by Papyan, Han and Donoho. We propose a simple unconstrained features model in which neural collapse also emerges empirically. By studying this model, we provide some explanation for the emergence of neural collapse in terms of the landscape of empirical risk.
Citation format
MIXON, Dustin G.; PARSHALL, Hans; PI, Jianzong. Neural collapse with unconstrained features [preprint]. arXiv, 2020. arXiv:2011.11619.