CoNNect: A Swiss-Army-Knife Regularizer for Pruning of Neural Networks-北京大学战略研究所

题目	CoNNect: A Swiss-Army-Knife Regularizer for Pruning of Neural Networks
作者	Franssen,Christian Jiang,Jinyang Peng,Yijie Heidergott,Bernd
作者单位	Department of Operations Analytics, VU Amsterdam, Amsterdam, Netherlands Guanghua School of Management, Peking University, Beijing, China
关键词：	CoNNect A SwissArmyKnife Regularizer for Pruning of Neural Networks
时间：	2025年2月2日
出版者：	arXiv
摘要	Pruning encompasses a range of techniques aimed at increasing the sparsity of neural networks (NNs). These techniques can generally be framed as minimizing a loss function subject to an L0norm constraint. This paper introduces CoNNect, a novel differentiable regularizer for sparse NN training that ensures connectivity between input and output layers. CoNNect integrates with established pruning strategies and supports both structured and unstructured pruning. We proof that CoNNect approximates L0-regularization, guaranteeing maximally connected network structures while avoiding issues like layer collapse. Numerical experiments demonstrate that CoNNect improves classical pruning strategies and enhances state-of-the-art one-shot pruners, such as DepGraph and LLM-pruner. Copyright ?? 2025, The Authors. All rights reserved.
URL	http://hdl.handle.net/20.500.11897/740920
ISSN	10.48550/arXiv.2502.00744
收录情况	EI

首页

作者单位	Department of Operations Analytics, VU Amsterdam, Amsterdam, Netherlands Guanghua School of Management, Peking University, Beijing, China	时间	2025年2月2日
出版者	arXiv	URL	http://hdl.handle.net/20.500.11897/740920
ISSN	10.48550/arXiv.2502.00744	DOI
收录情况	EI	分类