Sparsity is popular in statistics and machine learning because it can avoid overfitting, speed up computations, and facilitate interpretations. In deep learning, however, the full potential of sparsity still needs to be explored. This presentation first recaps sparsity in the framework of high-dimensional statistics and then introduces corresponding notions for modern deep-learning pipelines. Along the way, we discuss vital connections between mathematical statistics, optimization, and applications.
