Getting started with Machine Learning

5 minute read

Published:

If you are interested in Machine Learning you will find a lot of content online about it. I give below some content I find most appropriate to start to code and generate results.

The first step is to find a problem you are interested in solving and work hard to do it. Some good sources:

  • Scientific papers/thesis with data available (there are many)
  • Kaggle
  • UC Irvine Machine Learning Repository

Where to start learning?

Firstly, read the book

Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow:

start with Chapters 1-5. Take a look at and run the codes from its companion GitHub repo

https://github.com/ageron/handson-ml2

This book has a good introductory level, without going into subtleties, and shows you how to build code to produce predictive models effectively using python and scikit-learn.

“I don’t know Python”

It’s fine, take a look at

I’m assuming you know to code in another language. Do use the jupyter notebooks. The google colab allows you to code without having the hassle of installing anything, as most of the packages are readily available in a virtual machine.

Going deeper and focusing

Advance your reading in the book and running the code as you get more confident. Once finished, go through Chapters 6-9 – always reading and running code.

You will notice that there are many methods available for machine learning. At an early stage, give focus to learn and apply the following:

  • Linear-in-the-parameters models (e.g. Elastic-Net, LASSO, ridge, logistic regression)
  • Support vector machines
  • Decision trees
  • Random forest and other ensemble methods (e.g. xgboost)
  • Artificial neural networks (you may learn this at a later stage, Chapters 10-11)

For a review on the performance of many different models and computational implementations, see the results in:

“I don’t like Geron’s book”

Ok. Below is the book by Raschka and Mirjalili, which is somehow equivalent in content to Geron’s. Take a quick look, maybe you prefer it:

It also has a companion code repo on GitHub: https://github.com/rasbt/python-machine-learning-book-3rd-edition

Further reading at initial stage

While you read Geron’s or Raschka’s book, also read the corresponding parts of the paper A high-bias, low-variance introduction to Machine Learning for physicists. It also has a GitHub repo you can get inspiration from: https://github.com/drckf/mlreview_notebooks.

Do also check A Few Useful Things to Know About Machine Learning, which contains information regarding the ‘‘black-art’’ necessary for succesful machine learning applications.

Checklist of fundamental concepts to master at an early stage

  • Classification vs. regression problems
  • Supervised vs. unsupervised learning
  • Resampling for cross-validation and hold-out
  • What are hyperparameters and how to define them
  • Creating machine learning pipelines to build predictive models

YouTube channels and courses

This channel is very good. Brunton has written many important works on Fluid Mechanics applications of Machine Learning, and also an introductory book on the topic.

Theory

As you go deeper and need to increase your understanding, you may then read about detailed theory.

This will help you propose new methods and use them more effectively, for knowing their inner workings.

Make sure to do this after you read the basic stuff and knows how to code.

Proceeding like this hopefully will motivate you and give directions which you will find necessary to read further:

  • Tom Mitchell, Machine Learning, McGraw Hill, 1997.

http://www.cs.cmu.edu/~tom/mlbook.html

  • Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, 4th Ed., 2020.

http://aima.cs.berkeley.edu/

  • Elaine Rich and Kevin Knight, Artificial Intelligence, 3rd Ed. McGraw-Hill Higher Education, 2010.

https://www.cs.utexas.edu/~ear/

  • Simon O. Haykin, Neural Networks and Learning Machines, 3rd Edition, Pearson, 2009.

https://www.pearson.com/us/higher-education/program/Haykin-Neural-Networks-and-Learning-Machines-3rd-Edition/PGM320370.html

  • Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani, An Introduction to Statistical Learning with Applications in R, 8th Ed., 2017.

http://faculty.marshall.usc.edu/gareth-james/ISL/

  • Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning, 2nd Ed., Springer, 2009.

https://web.stanford.edu/~hastie/ElemStatLearn/

  • Steven L. Brunton and J. Nathan Kutz. 2019. Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control (1st. ed.). Cambridge University Press, USA.

http://www.databookuw.com/

R or Python?

Both. Don’t waste time reading discussions on which is better. It is also possible to learn R for the same goals.

I’ve found in the last couple of years more momentum in Python related to Engineering applications, that’s the reason I advise you to start with Py. It is just a hint, nothing that I can measure or prove :)

Nonetheless, R has also a vibrant community and is worth learning as well. Use both to develop similar projects and you will be able to choose for yourself.

Once you have time to learn R, take a look at the book from Kuhn:

and the powerful caret package:

Introduction to Artificial Neural Networks

https://sites.google.com/matematica.ufrj.br/seminario-redesneurais/cronograma?authuser=0 [in Portuguese]

Deep Learning: An Introduction for Applied Mathematicians

Deep learning courses

Yann LeCun’s Deep Learning Course at CDS - NYU

Maziar Raissi - Applied Deep Learning

https://sebastianraschka.com/blog/2021/dl-course.html

Further reading

https://developers.google.com/machine-learning/crash-course

Principal Component Analysis: A Natural Approach to Data Exploration