Reading Assignment: Model Compression & Transfer Learning

Model Compression

• Hinton’s infamous paper on “Dark Knowledge”: Hinton et al. “Distilling the Knowledge in a Neural Network” (NIPS 2014 Deep Learning Workshop) paper, talk at NIPS 2014, longer talk at TTIC (optional), slides

• blog post “Compressing and regularizing deep neural networks” by Song Han and optional paper: Han et al. “Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding” (ICLR 2016)

Transfer Learning

• blog post “A Gentle Introduction to Transfer Learning for Deep Learning” by Jason Brownlee

• blog post “Transfer Learning - Machine Learning’s Next Frontier” by Sebastian Ruder

Further Reading (Optional)

Follow the links from the blog posts to the referenced articles for further reading!