Deep CNN sparse coding analysis: Towards average case

Deep convolutional sparse coding (D-CSC) is a framework reminiscent of deep convolutional neural nets (DCNN), but by omitting the learning of the dictionaries one can more transparently analyse the role of the activation function and its ability to recover activation paths through the layers. Papyan, Romano, and Elad conducted an analysis of such an architecture [1], showed the relationship with DCNNs, and proved conditions under which a D-CSC is guaranteed to recover activation paths. A technical innovation of their work highlights that one can view the efficacy of the ReLU nonlinear activation function of a DCNN through the new variant of the tensor’s sparsity, referred to as stripe-sparsity, and by which they can prove that the density of activations can be proportional to the ambient dimension of the data. We extend their uniform guarantees to a slightly modified model and prove that with high probability the desired activation is typically possible to recover for a greater density of activations per layer. Our extension follows from incorporating the prior work on one step thresholding by Schnass and Vandergheynst into the appropriately modified architecture of [1].