Reading Assignment: Attention and Memory

Overview

For an overview on the topic, start with the blog post “Attention and Memory in Deep Learning and NLP” by Denny Britz.

Main Reading

Continue by reading the following articles that provide details on two very recently proposed approaches,

• Graves et al. “Hybrid computing using a neural network with dynamic external memory” in Nature 538.7626 (2016): pp. 471-476.

• Ashish et al. “Attention Is All You Need” in NIPS (2017) (The version on arxiv seems to be most recent.)

Further Reading (Optional)

Follow the links from the blog post to the referenced articles for further reading.