Because deep learning started working so recently and is moving so quickly, it's a relatively shallow field (no pun intended) and can be picked up without too much pre-existing background.
That being said, different people will find different learning approaches to be preferable.
If you want to read one main resource... the Goodfellow, Bengio, Courville book (available for free from http://www.deeplearningbook.org/) is an extremely comprehensive survey of the field. It contains essentially all the concepts and intuition needed for deep learning engineering (except reinforcement learning).
If you'd like to take courses... Pieter Abbeel and Wojciech Zaremba suggest the following course sequence:
- Linear Algebra — Stephen Boyd’s EE263 (Stanford) - Neural Networks for Machine Learning — Geoff Hinton (Coursera) - Neural Nets — Andrej Karpathy’s CS231N (Stanford) - Advanced Robotics (the MDP / optimal control lectures) — Pieter Abbeel’s CS287 (Berkeley) - Deep RL — John Schulman’s CS294-112 (Berkeley)
(Pieter also recommends the Cover & Thomas information theory and Nocedal & Wright nonlinear optimization books).
If you'd like to get your hands dirty... Ilya Sutskever recommends implementing simple MNIST classifiers, small convnets, reimplementing char-rnn, and then playing with a big convnet. Personally, I started out by picking Kaggle competitions (especially the "Knowledge" ones) and using those as a source of problems. Implementing agents for OpenAI Gym (or algorithms for the set of research problems we’ll be releasing soon) could also be a good starting place.
I recommend a mix of all of the above approaches. Deep learning requires a mix of theoretical and empirical understanding. It can be hard to know where to start, and looking up a term on Wikipedia usually just yields another six terms to look up. The books and courses can both solve this problem by giving you a well-paced introduction to the subject.
On the other hand, everything usually feels abstract until you start implementing. It's mostly important to implement a variety of models and make them really work. As Ilya likes to say, you need to be prepared to suffer: expect hours of debugging models that refuse to learn, many passes restructuring your code, and building up your own conventions for changing various hyperparameters. But each time you suffer, know that you've built a little bit of skill that will be invaluable for the future.
