Resources for learning about ML

There are so many useful machine learning resources out there and even more posts reviewing these resources . The goal of this page is not to list everything but only those that I have used/(partially) completed/read and that I can review. Maybe it will be hopeful for someone but I also want to keep track of what I have seen and liked.

Side Notes :

• I have a strong preference towards videos, interactive visualization, and intuitive mathematical explanation.
• Click on the resources to get some additional information!
• I will mostly have review saying that these are excellent resources, but this is because I didn't finish reading/watching the ones I liked less.

General Machine Learning

Pattern Recognition and Machine Learning

• Author : C. Bishop.
• Review : I just love it, excellent mix between maths and intuition
• Recommend as :
  • Introduction self-learning book if you have some mathematical background
  • If you have applied machine learning algorithms but now to get a better theoretical knowledge.
• Notes : Probably the most famous ML book.
• Level : Intermediate.
• Link
• Price : ~60$

Machine Learning a Probabilistic Perspective

• Author : K. Murphy.
• Review : Excellent reference book which covers a wide range of ML topics with a statistical perspective. Probably not a book to read at once or to use as an introduction to the field. Aimed to people with a decent mathematical background. Can sometimes feel a bit disorganized, which is understandable due to the large amount of covered topics.
• Recommend as :
  • Reference book to use a refresher for those who already understand of the concepts.
  • To get a probabilistic view of some concepts you already know.
  • Instructors that want a book as a supplement to their class.
• Notes : famous as a reference book.
• Level : Intermediate - Advanced.
• Link
• Price : ~90$

UBC - Undergraduate Machine Learning

• Author : N. de Freitas.
• Review : Very good introductory class, very intuitive but also tries to get you used to the necessary math in ML.
• Recommend as :
  • Stand-alone videos when you are interested in getting mathematical intuition of a introductory method.
• Notes : this CPSC 340 class at UBC was my first ML class. My professor wasM. Schmidt (also great but no videos), who replaced Nando after he left to Oxford. I have watched all the talks and online classes from Nando and really enjoy all of them.
• Level : Beginner.
• Link

UBC - (Graduate) Machine Learning

• Author : N. de Freitas.
• Review : Very good class, with clear mathematical explanations.
• Recommend as :
  • Stand-alone videos when you are interested in getting mathematical intuition of a intermediate method.
• Notes : this was my graduate ML class at UBC, although the topics covered were very different, so I watched all of those videos to.
• Level : Intermediate.
• Link

Machine Learning Summer School 2013

• Author : multiple.
• Review : Excellent videos of the 2013 Machine Learning Summer School held at the Max Planck Institute for Intelligent Systems in Tübingen. Some of the most famous ML professors in Europe come to give introductory lectures about their domain. The complexity of the video really depends on each professor but they all give excellent intuition and explanations about why some methods work.
• Recommend as :
  • Stand-alone videos when you have some knowledge of machine learning but would like to get better insights from researchers.
• Level : Beginner-Advanced.
• Link

Coursera - Machine Learning

• Author : A. Ng.
• Review : Perfectly mixes mathematical theory, intuition and practice through coding. Covers a very wide range of core machine learning concepts. Gives you the necessary basis to start learning about state of the art machine learning. Doesn't require any prerequisites, if you have some background you will often watch in 2x or skip parts. The only small thing I would have done differently is use python rather than octave.
• Recommend as :
  • Course if you don't have a heavy mathematical background but are serious about starting in machine learning.
  • Go-to resource to learn about machine learning.
  • Videos to watch if you didn't really understand something in class.
• Notes : by far the most famous resource to get into the field, one of the most watched coursera MOOC.
• Level : Beginner.
• Link
• Price : free

Youtube - Machine Learning

• Author : J. Miller.
• Review : Excellent machine learning videos, that cover more advanced topics than the other MOOCs I have cited. It is often the best intuitive explanations you can find on some topics.
• Recommend as :
  • Supplementary material you should use if you cannot get an intuitive feeling of a certain topic.
• Level : Beginner.
• Link
• Price : free

Udacity - Machine Learning

• Author : M. Littman, C. Isbell, P. Kolhe
• Review : Very simple to understand course where the 2 professors try to explain things to each other. Doesn't go too much into mathematical details but gives a very good overview and understanding of the major machine learning concepts. Also a great introductory MOOC. In addition to more classical introductions, it has some great videos on reinforcement learning major concepts.
• Recommend as :
  • Introductory MOOC to machine learning as a replacement to coursera's one if you want less math.
  • Videos to watch if you didn't really understand something in class.
  • Videos to watch if you want to have a quick overview of a certain domain.
• Notes : I would recommend watching a few videos of both MOOC's, stick with one, and watch some videos of the other when you are not satisfied with an explanation.
• Level : Beginner.
• Link
• Price : free

Udemy - Data Science and Machine Learning Bootcamp with Python

• Author : J. Portilla
• Review : Excellent resource for starting implementing as soon as possible, while having enough theory to be productive.
• Recommend as :
  • If you want to start applying machine learning to your problem in Python
• Notes : J. Portilla is my favorite instructor on Udemy and has many good practical courses.
• Level : Beginner.
• Link
• Price : 200$ but often on sale for 15$

Udemy - Data Science and Machine Learning Bootcamp with R

• Author : J. Portilla
• Review : Excellent resource for starting implementing as soon as possible, while having enough theory to be productive.
• Recommend as :
  • If you want to start applying machine learning to your problem in R
• Notes : If you don't have any preferences I would advice you to start in Python.
• Level : Beginner.
• Link
• Price : 200$ but often on sale for 15$

TWiML&Ai

• Author : S. Charrington.
• Review : I listen to it during all of my daily commutes or when I cook! I really enjoy it: it's an excellent way of getting an overview of what people are doing in the domain (research or industry) while being able to multi task as most of the episodes aren't too technical. What is particular is the wide variety of speakers and covered domains. You can have some of the best researchers in the world, as well business people who learned ML on the fly. I rarely learn about new subjects, but it often gives interesting additional perspectives. The only disadvantage is that some of the industrial episodes can be a bit superficial. This is less the case in the latest episodes also because Sam Charrington asks more technical questions than in the first episodes.
• Recommend as :
  • Podcast for multi tasking : daily commute / cleaning / cooking.
  • Podcast for new machine learning enthusiast that do not want to go to much in the details.
  • Podcast for people working in industry who want to have an idea of how machine learning could be applied to their domain
• Level : Beginner - Intermediate.
• Link

The O’Reilly Data Show

• Author : B. Lorica.
• Review : Before TWiML&AI, I was listening to this one. Very good podcast and speakers. It is mostly focused on scalable enterprise machine learning. I had to try an other podcast while I was waiting for the next episode, I never came back as the subjects covered weren't my favorite (lots of Apache tools).
• Recommend as :
  • Podcast for people who want to use scalable machine learning in their work.
• Level : Beginner - Intermediate.
• Link

Data Skeptic

• Author : K. Polich.
• Review : Very nice podcast to learn about general data science. I stopped listening to it as I had a good knowledge of the majority of the topics discussed. Nevertheless very good introductory podcast.
• Recommend as :
  • Podcast for people who are interested in learning more about data science.
• Level : Beginner.
• Link

Distill

• Author : The Distill Team.
• Review : Amazing posts (publication to be exact) with an in depth visual and mathematical explanation of a specific subject. Probably the best visual explanations you can find.
• Level : Advanced.
• Link

Machine Learning Mastery

• Author : J. Brownlee.
• Review : Very good and simple blog which covers an impressive number of machine learning subjects. Focuses on coding directly the concepts learned.
• Level : Beginner.
• Link

Other Great Blogs

• FastML practical ML
• The Spectator mostly statistical ML
• Hunch broad ML

Reinforcement Learning

UCL - Reinforcement Learning

• Author : D. Silver.
• Review : Excellent class which covers the important Rl concepts as well as the deep reinforcement learning. Gives excellent intuitive explanations of the formulas and concepts.
• Recommend as :
  • Go-to course if you know about ML/math and want to quickly understand the reinforcement learning setting and methods.
• Level : Intermediate.
• Link

OpenAI/BAIR - Deep Reinforcement Learning Bootcamp

• Author : DeepRlBootcamp team.
• Review : Very good classes that cover many aspects of deep reinforcement learning. The lecturers are some of the most famous deep RL researchers. You probably should have some understanding of RL before watching these if you want to take full advantage of the content.
• Recommend as :
  • Stand-alone videos when you are interested about a specific topic / method.
• Notes : I participated to the bootcamp, so you might even hear a question from me . it was an amazing opportunity and I would recommend to anyone who is even slightly in the field.
• Level : Advanced - "Expert" (maybe for lecture 5).
• Link

Bayesian Methods

Note: the PRML, the MLPP and UBC's Graduate Machine Learning class, which I reviewed in the first section, are a good introduction to Bayesian methods and a Bayesian perspective of ML.

Gaussian Processes for Machine Learning

• Author : C. Rasmussen, C. Williams
• Review : Very good concise book that perfectly balances the amount of math to lay a formal foundation in Gaussian Processes (GP) for ML, while still being easy to follow. A must read for anyone interested in GP. The provided code turned into a easy to use Matlab toolbox (GPML).
• Recommend as :
  • Introductory book to Gaussian Processes (chapters 1-5).
  • Preparation for GP research and advance use-cases (chapters 6-9).
  • Reference for GP.
• Notes : Fortunately, Prof. Rasmussen teaches a few of my classes at Cambridge U., he really has a gift to explain complicated topics in easy terms.
• Level : Intermediate - Advanced.
• Link
• Price : Free HTML version, ~40$ hardcover.

NIPS 2016 - Bayesian Deep Learning Workshop

• Author : multiple.
• Review : Good videos of the workshop. Being able to listen to many of the best researchers in this quickly growing field is very interesting. Unfortunate the videos are too short to really be able to cover the subject with great depth.
• Recommend as :
  • Stand alone videos for anyone interested in Bayesian Deep Learning (should have some knowledge of ML and DL).
• Level : Intermediate-Advanced.
• Link

Deep Learning

Deep Learning

• Author : I. Goodfellow, Y. Bengio, A. Courville.
• Review : Excellent book which covers Deep Learning in depth while remaining accessible for newcomers in the field. Simple explanations with a relatively low the amount of maths. Very well organized such that people know what parts they can skip. Sometimes missing numerical examples.
• Recommend as :
  • Introductory book to deep learning (start from part I).
  • Transition book from machine learning to deep learning (jump to part II).
  • Advance readers and reference book (jump to part III)
  • Exhaustive bibliography !
• Notes : first in depth Deep Learning book which has received many positive feedbacks from the community.
• Level : Intermediate - Advanced.
• Link
• Price : Free HTML version, ~50$ hardcover.

Oxford - Deep Learning

• Author : N. de Freitas
• Review : Excellent class as all of Nando's classes. Explanations are made simple and mathematically intuitive. Starts from basic topics, so you don't even need a good understanding of machine learning before.
• Recommend as :
  • Course if you don't have a good machine learning knowledge but directly want to learn deep learning theory.
  • Course if you enjoy having different intuitive explanation of a certain method.
• Notes : Only small complain I could make is concerning the sound quality, but the class is too good for complaining .
• Level : Intermediate.
• Link

Coursera - Deep Learning Specialization

• Author : A. Ng.
• Review : Excellent MOOC which covers every detail of basic deep learning. Andrw Ng has the gift of explaining concepts extremely simply and make you want to listen to him. There are no requirements for this course. The course is too simple for someone who was a good knowledge of deep learning, but I was still very happy of watching it as you get great insights on what works in practice and how Andrew selects his algorithms. Andrew also made videos where he interviews many of the most important researches in deep learning, which I absolutely love!
• Recommend as :
  • Course if you want to apply deep learning, but would like to get a good understanding of it to.
• Notes : Only complain I would have is how easy the assignments are, and the fact that most coding is done for us. Basically we have functions to fill-in with comments about what to do.
• Level : Beginner - Intermediate.
• Link

Udemy - Practical Deep Learning with PyTorch

• Author : Deep Learning Wizard.
• Review : Good MOOC as an introduction to PyTorch and deep learning if you want to get started quickly with no maths. The author does a good job explaining 3 major algorithms with just enough details for you to code in a high level library such as PyTorch. I watched the class to get a simple intro to PyTorch not for the basic theory, but if you have stricly no math background it might be a good way to start. There's a very high amount of repetition, but this might be helpful for beginners.
• Recommend as :
  • Course if you want to start playing around with pytorch while having the strict minimum understanding of the algorithms you are implementing.
• Level : Beginner.
• Link

Cognitive Class - Deep Learning with TensorFlow

• Author : Cognitive Class Team.
• Review : Decent MOOC to start coding in tensorflow with the minimum theoretical knowledge. The theory is explained a too simply for me, and without maths. I basically used it as a quick intro to tensorflow.
• Recommend as :
  • Course if you want to start playing around with tensorflow while having the strict minimum understanding of the algorithms you are implementing.
• Level : Beginner.
• Link

Colah's Blog

• Author : Christopher Olah.
• Review : My favorite ML blog. "Colah" doesn't only explain very clearly, but he has a gift for making beautiful and helpful visualizations. Very similar to Distill as he is on the editor's team.
• Level : Intermediate - Advanced.
• Link

Graphical Models

Natural Language Processing

Stanford - CS224n : Natural Language processing with deep learning

• Author : C. Manning, R. Socher
• Review : Excellent class for understanding state of the art NLP methods. As the name of the class suggests, it talks very little in "older" NLP methods, it focuses on deep learning and the use of the "newer" word embeddings (~2013). You should probably have some machine learning knowledge to take ful advantage of the class.
• Recommend as :
  • Course to effectively apply and understand state-of-the art NLP methods.
• Notes : Probably the most famous state-of-the art NLP class (in 2017). The NLP version of the other famous Stanford class : CS231n. I watched the older version but I think that the review still holds (even though 2 courses merged in this one).
• Level : Intermediate-Advanced.
• Link

Coursera - Natural Language Processing

• Author : M. Collins.
• Review : Excellent MOOC which gives you a in depth view of the major algorithms which were done in NLP before the "deep-learning era". I've rarely seen a professor keeping such a level of clarity during the whole course.
• Recommend as :
  • Course if you are starting in NLP but are serious about continuing.
  • Course if you have applied deep learning to NLP, but would like to really understand this very interesting field.
• Notes : Unfortunately not given on Coursera anymore. I understand that giving such a class must take a huge amount of time, but I'm a bit disappointed that they didn't put the old videos on youtube. Thankfully, you can still find someone who put the videos up.
• Level : Beginner-Intermediate.
• Link

Ruder.io

• Author : S. Ruder.
• Review : Top quality NLP and Deep Learning posts, that are very easy to read.
• Level : Advanced.
• Link

Other Great Blogs

• Explosion applied NLP

NLP Highlights

• Author : M. Gardner, W. Ammar.
• Review : Very good NLP podcast where the 2 research scientists discuss about a new paper in the field or interview other scientists about their recent work. Great way to keep up with the research in the field of NLP! The only downside with this podcast compared to the more general ones is that I find it hard to multi-task while listening to it.
• Recommend as :
  • PoPodcast to keep up with the recent papers and research in the field of NLP.
• Level : Advanced-Expert.
• Link

Time Series

Forecasting: principles and practice

• Author : R. Hyndman, G. Athanasopoulos .
• Review : Very good basic introduction about time series forecasting. Easy and quick to read with simple language. Focuses on practice with many examples in R. Little math involved.
• Recommend as :
  • Introductory self-studying book giving you a good overview of the most import time series notions.
  • "Documentation" if you plan on using the R package forecast, as R. Hyndman is one of the main author of both. It gives you enough theory behind the package to use it sensibly and productively.
• Notes : A lot of the Time Series resources are from R. Hyndman, so it's nice to use the same notation as him.
• Link
• Price : Free HTML version, ~40$ hardcover.

Introduction to Time Series Forecasting With Python

• Author : J. Brownlee.
• Review : Good basic introduction. Mostly focused on practical examples and Python code. No math involved. I personally find that there isn't enough theoretical explanations and math involved, but it was definitely not the goal of the book.
• Recommend as :
  • Self-study book if you want to start coding as soon as possible.
  • Self-study book if you only speak in python but not in math.
• Notes : J. Brownlee is the author of the famous machine learning mastery blog, this book is basically copy pasted from his posts.
• Link
• Price : Free HTML version, ~40$ hardcover.

Hyndsight

• Author : R. Hyndman.
• Review : Excellent blog posts which covers time series theory, tools, and food for thoughts. The only problem is that it can be hard to find what you're looking for in all these posts which are not very well organized.
• Level : Beginner - Intermediate .
• Link

Automatic Speech Recognition

Optimization and Numerical Analysis

Scientific Computing with MATLAB and Octave

• Author : A. Quarteroni, F. Saleri,P. Gervasio.
• Review : Good book which is very well structured to read all at once. Although it is mostly focused on application of numerical methods, it still keeps a good mathematical rigor. Although the initial chapters aren't very math-heavy, you should have good understanding of analysis to really enjoy mthe very interesting final chapters.
• Recommend as :
  • Book for people who have a descent understanding of analysis and want to start applying these concepts to more practical problems.
  • Complement to a numerical analysis course, to improve your understanding and start coding.
  • Self-paced book for young applied mathematicians/engineers who would like to learn MATLAB while understanding the power of numerical tools.
• Notes : Prof. Quarteroni was my first professor of a class linking math and computer science. His course was probably my favorite at EPFL and he always had great examples of how he applied some methods to a variety of different domains.
• Level : Intermediate.
• Link
• Price : ~50$

A First Course in Numerical Methods

• Author : U. Ascher, C. Greif.
• Review : Relatively good book for an introduction to numerical methods. I really enjoy the comments you will find in colored boxed. It is written pretty informally both in language and mathematical rigor, and is more focused on application with little proofs/derivations. Although it does a good job covering many different topics it doesn't go enough into details and thus can be hard to understand. Overall good book if you know what you are doing.
• Recommend as :
  • Book if you already had an introduction class on numerical methods long time ago and would like to refresh all the topics you had covered, as well a their applications.
  • Complement to a numerical analysis course.
• Level : Intermediate.
• Link
• Price : ~100$

CMU - Convex Optimization

• Author : Javier Peña, Ryan Tibshirani
• Review : Very good class covering optimization while keeping in mind applications to machine learning. Both professors do a very good job to give mathematical intuition of the important concepts and algorithms in covex optimization. The class is focused on computational and numerical methods, so you don't have much real proofs. You definitely not need to know about ML to watch it but if you might get more ideas about how to use these concepts in practice. You should have good linear algebra knowledge.
• Recommend as :
  • Course for anyone who knows about basic machine learning but not much about optimization which is essential in ML.
  • Introductory class to convex optimization.
• Level : Intermediate.
• Link

Computational Neuroscience

EPFL - Neuronal Dynamics

• Author : W. Gerstner.
• Review : Excellent class where to learn how to mathematically / physically model a single neuron. Prof Gerstner is extremely clear and has a very good mathematical rigor. He gives very good intuition of the mathematical ways of describing bio-chemical processes. You should have some good calculus background, and it could be helpful to have some basic knowledge of physics / chemistry.
• Recommend as :
  • Course if you have a good mathematical background and are interested in "understanding the brain".
  • Course if you come from machine learning and would like to know how ANN differ from current mathematical models of neurons.
• Notes : I currently only have watched $\frac{2}{7}$ of the course, I definitely plan on finishing it but didn't have time yet. So my review might change afterwards.
• Level : Intermediate - Advanced.
• Link
• Price : free

Other