Getting started with Machine Learning

5 minute read

Published: November 23, 2020

If you are interested in Machine Learning you will find a lot of content online about it. I give below some content I find most appropriate to start to code and generate results.

The first step is to find a problem you are interested in solving and work hard to do it. Some good sources:

Scientific papers/thesis with data available (there are many)
Kaggle
UC Irvine Machine Learning Repository

Where to start learning?

Firstly, read the book

Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow:

start with Chapters 1-5. Take a look at and run the codes from its companion GitHub repo

https://github.com/ageron/handson-ml2

This book has a good introductory level, without going into subtleties, and shows you how to build code to produce predictive models effectively using python and scikit-learn.

“I don’t know Python”

It’s fine, take a look at

https://cs231n.github.io/python-numpy-tutorial/
https://developers.google.com/edu/python/
https://numpy.org/doc/stable/user/numpy-for-matlab-users.html - very important, to understand the array indexing logic and other stuff which commonly confuses MATLAB/Octave users
Get familiar with the following python libraries: numpy, pandas, and scipy (know what they are and check some examples online)
Just do it https://github.com/rougier/numpy-100/blob/master/100_Numpy_exercises.ipynb

I’m assuming you know to code in another language. Do use the jupyter notebooks. The google colab allows you to code without having the hassle of installing anything, as most of the packages are readily available in a virtual machine.

Going deeper and focusing

Advance your reading in the book and running the code as you get more confident. Once finished, go through Chapters 6-9 – always reading and running code.

You will notice that there are many methods available for machine learning. At an early stage, give focus to learn and apply the following:

Linear-in-the-parameters models (e.g. Elastic-Net, LASSO, ridge, logistic regression)
Support vector machines
Decision trees
Random forest and other ensemble methods (e.g. xgboost)
Artificial neural networks (you may learn this at a later stage, Chapters 10-11)

For a review on the performance of many different models and computational implementations, see the results in:

“I don’t like Geron’s book”

Ok. Below is the book by Raschka and Mirjalili, which is somehow equivalent in content to Geron’s. Take a quick look, maybe you prefer it:

Sebastian Raschka and Vahid Mirjalili, Python Machine Learning, 3rd Edition, 2019.

It also has a companion code repo on GitHub: https://github.com/rasbt/python-machine-learning-book-3rd-edition

Checklist of fundamental concepts to master at an early stage

Classification vs. regression problems
Supervised vs. unsupervised learning
Resampling for cross-validation and hold-out
What are hyperparameters and how to define them
Creating machine learning pipelines to build predictive models

How to report my results?

Glad you asked. The links below will point you to quick reads with best practices.

https://arxiv.org/abs/2108.02497

https://arxiv.org/pdf/1811.12808.pdf

YouTube channels and courses

Steve Brunton - University of Washington

This channel is very good. Brunton has written many important works on Fluid Mechanics applications of Machine Learning, and also an introductory book on the topic.

An Introduction to Statistical Learning with Applications in R
There are more, will add here!

Theory

As you go deeper and need to increase your understanding, you may then read about detailed theory.

This will help you propose new methods and use them more effectively, for knowing their inner workings.

Make sure to do this after you read the basic stuff and knows how to code.

Proceeding like this hopefully will motivate you and give directions which you will find necessary to read further:

Tom Mitchell, Machine Learning, McGraw Hill, 1997.

http://www.cs.cmu.edu/~tom/mlbook.html

Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, 4th Ed., 2020.

http://aima.cs.berkeley.edu/

Elaine Rich and Kevin Knight, Artificial Intelligence, 3rd Ed. McGraw-Hill Higher Education, 2010.

https://www.cs.utexas.edu/~ear/

Simon O. Haykin, Neural Networks and Learning Machines, 3rd Edition, Pearson, 2009.

https://www.pearson.com/us/higher-education/program/Haykin-Neural-Networks-and-Learning-Machines-3rd-Edition/PGM320370.html

Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani, An Introduction to Statistical Learning with Applications in R, 8th Ed., 2017.

http://faculty.marshall.usc.edu/gareth-james/ISL/

Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning, 2nd Ed., Springer, 2009.

https://web.stanford.edu/~hastie/ElemStatLearn/

Steven L. Brunton and J. Nathan Kutz. 2019. Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control (1st. ed.). Cambridge University Press, USA.

http://www.databookuw.com/

R or Python?

Both. Don’t waste time reading discussions on which is better. It is also possible to learn R for the same goals.

I’ve found in the last couple of years more momentum in Python related to Engineering applications, that’s the reason I advise you to start with Py. It is just a hint, nothing that I can measure or prove :)

Nonetheless, R has also a vibrant community and is worth learning as well. Use both to develop similar projects and you will be able to choose for yourself.

Once you have time to learn R, take a look at the book from Kuhn: