Reading Assignment: Advanced Architectures & Introspection

Optional: Manifolds

If you would like to follow up on the concept of manifolds, read the blog post on “Neural Networks, Manifolds, and Topology” by Christopher Olah.

Main Reading

Start with the high-level overview provided in this blog post by Adit Deshpande. Anything beyond the section on ResNets is optional. (GANs are covered in the “Learning Generative Models” course.)

Next, read Densely connected convolutional networks (2016), Huang et al. There is a lot to learn from this paper as the authors do a very good job pointing out similarities and differences with many related approaches.

For feature visualization, start with this overview blog post and a very recent addition.

Now it’s time to look at some specific techniques. Dig as deep as you like! These websites provide plenty of background information.

1. Deepvis (code and paper are linked, optional further reading on fooling neural nets)

2. LSTMVis (code and paper are linked)

3. Heatmapping - especially Methods for Interpreting and Understanding Deep Neural Networks (2017), Montavon et al.

Further Reading

• (optional) on ResNets: Deep residual learning for image recognition (2016), He et al.
• (optional) on Inception architectures: Rethinking the inception architecture for computer vision (2016), Szegedy et al.