Why is Pruning at Initialization Immune to Reinitializing and Shuffling?
TL;DR
Pruning at init methods work even under randomization treatments, perhaps because they maintain the weight distribution.
Venue
In Sparsity in Neural Networks Workshop 2021.
BibTeX
@article{singh2021pruning,
title={Why is Pruning at Initialization Immune to Reinitializing and Shuffling?},
author={Sahib Singh and Rosanne Liu},
year={2021},
eprint={2107.01808},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
title={Why is Pruning at Initialization Immune to Reinitializing and Shuffling?},
author={Sahib Singh and Rosanne Liu},
year={2021},
eprint={2107.01808},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
Date
July, 2021
Links