Summer Paper Reading List 2024

1 July, 2024 | 10 min read

This is my summer reading list for 2024. The list covers the foundational concepts and cutting-edge research in machine learning, neural networks, transformers, and distributed systems. It is based on my own research interests and recommendations from others.

Pick a few that grab you and really dig in. Implement the key ideas if you can. It's amazing how much clearer things become when you actually build them.

Any suggestions? Send me a DM on Twitter @chrisbbh.
If I find or get recommended a paper that's relevant and insightful, I'll add it to the list.

I'll update this post with my progress as I work through the papers. Each paper will be marked with a colored dot indicating its status: unread (🔵), read (🟢), or abandoned (🔴).

Foundational Computer Science Papers

Fundamental Concepts and Techniques in Machine Learning

Advanced Neural Network Architectures

Transformer Models and Innovations

Retrieval-Augmented Learning (RAG)

Distributed Systems and Large-Scale Machine Learning

Recent Innovations and Applications

Other interesting papers

Ilya 30u30 papers

This list is inspired by a story involving Ilya Sutskever and John Carmack. As Carmack recounts:

So I asked Ilya Sutskever, OpenAI's chief scientist, for a reading list. He gave me a list of like 40 research papers and said, 'If you really learn all of these, you'll know 90% of what matters today.' And I did. I plowed through all those things and it all started sorting out in my head.

It's important to note that the papers listed below are not confirmed to be the exact ones Sutskever recommended. This story comes from an interview with Carmack, but the specific papers remain unverified.

It's an interesting collection, if true. The kind of thing that makes you wonder what you'd put on your own list if you had to distill an entire field down to its essence. What would be your "90% of what matters" in your area of expertise?

Of course, even if this is the real list, it's a snapshot in time. In a field moving as rapidly as AI, today's 90% might be tomorrow's 50%. But there's value in understanding the foundations, the papers that shaped the current landscape.

Source of list: Ilya 30u30.

Liked this post? Join the newsletter.