[PYTHON] Everything for beginners to be able to do machine learning

Introduction

This article is an article to enable machine learning while doing what was presented as ** "Knowledge required for machine learning" **, which was given as an assignment for my aspiring internship. is. Only by doing all of this article will you be able to become a ** apprentice machine learning engineer **. On the contrary, you can easily get the basics of machine learning by doing all of this article.

It is not a perfect commentary as it includes personality.

Development environment

Before I say anything, let's start with my development environment. OS ・ Windows -Windows Subsystem for Linux (I also make it work on Ubuntu) IDE Pycharm

What do beginners need to do machine learning?

Below is a list of the issues presented by the intern aspirants. This is ** "necessary things" **. I will explain each one later, but first let's suppress the whole picture.

** ① Absolutely necessary ability as an engineer ** To be able to use it smoothly. ・ Basic knowledge of python3 ・ Command line ・ Git

** ② Setup ** Grasp the following: ・ Python system ・ Pip ・ Pipenv ・ Indispensable for efficiency on Ubuntu terminal (command line) ・ Tmux ・ Others ・ Markdown notation (for Qiita)

・ Debugging work -Debugging with Pycharm

** ③ Machine learning ** As a library, grasp the following things. ・ Scikit-learn ・ Numpy ・ Pandas ・ Matplotlib ・ Keras

** ④ Book ** Must ・ [Readable Code](https://www.amazon.co.jp/%E3%83%AA%E3%83%BC%E3%83%80%E3%83%96%E3%83%AB%E3 % 82% B3% E3% 83% BC% E3% 83% 89-% E2% 80% 95% E3% 82% 88% E3% 82% 8A% E8% 89% AF% E3% 81% 84% E3% 82% B3% E3% 83% BC% E3% 83% 89% E3% 82% 92% E6% 9B% B8% E3% 81% 8F% E3% 81% 9F% E3% 82% 81% E3% 81% AE% E3% 82% B7% E3% 83% B3% E3% 83% 97% E3% 83% AB% E3% 81% A7% E5% AE% 9F% E8% B7% B5% E7% 9A% 84% E3% 81% AA% E3% 83% 86% E3% 82% AF% E3% 83% 8B% E3% 83% 83% E3% 82% AF-Theory-practice-Boswell / dp / 4873115655) ・ [Essence of Machine Learning](https://www.amazon.co.jp/%E6%A9%9F%E6%A2%B0%E5%AD%A6%E7%BF%92%E3%81%AE % E3% 82% A8% E3% 83% 83% E3% 82% BB% E3% 83% B3% E3% 82% B9-% E5% AE% 9F% E8% A3% 85% E3% 81% 97% E3% 81% AA% E3% 81% 8C% E3% 82% 89% E5% AD% A6% E3% 81% B6Python-% E3% 82% A2% E3% 83% AB% E3% 82% B4% E3 % 83% AA% E3% 82% BA% E3% 83% A0-Machine-Learning / dp / 4797393963) ・ [Machine learning starting from work](https://www.amazon.co.jp/%E4%BB%95%E4%BA%8B%E3%81%A7%E3%81%AF%E3%81% 98% E3% 82% 81% E3% 82% 8B% E6% A9% 9F% E6% A2% B0% E5% AD% A6% E7% BF% 92-% E6% 9C% 89% E8% B3% 80 -% E5% BA% B7% E9% A1% 95 / dp / 4873118255) ** Other personally good books ** ・ [Easy Learning Mathematics for Understanding Machine Learning](https://www.amazon.co.jp/%E3%82%84%E3%81%95%E3%81%97%E3% 81% 8F% E5% AD% A6% E3% 81% B6-% E6% A9% 9F% E6% A2% B0% E5% AD% A6% E7% BF% 92% E3% 82% 92% E7% 90 % 86% E8% A7% A3% E3% 81% 99% E3% 82% 8B% E3% 81% 9F% E3% 82% 81% E3% 81% AE% E6% 95% B0% E5% AD% A6 % E3% 81% AE% E3% 81% 8D% E3% 81% BB% E3% 82% 93-% E3% 82% A2% E3% 83% A4% E3% 83% 8E-% E3% 83% 9F % E3% 82% AA% E3% 81% A8% E4% B8% 80% E7% B7% 92% E3% 81% AB% E5% AD% A6% E3% 81% B6-% E6% A9% 9F% E6% A2% B0% E5% AD% A6% E7% BF% 92% E3% 81% AE% E7% 90% 86% E8% AB% 96% E3% 81% A8% E6% 95% B0% E5% AD% A6% E3% 80% 81% E5% AE% 9F% E8% A3% 85% E3% 81% BE% E3% 81% A7 / dp / 4839963525) ・ [Learning by moving with Python! New deep learning textbook From basic machine learning to deep learning](https://www.amazon.co.jp/Python%E3%81%A7%E5%8B%95%E3 % 81% 8B% E3% 81% 97% E3% 81% A6% E5% AD% A6% E3% 81% B6-% E3% 81% 82% E3% 81% 9F% E3% 82% 89% E3% 81% 97% E3% 81% 84% E6% B7% B1% E5% B1% A4% E5% AD% A6% E7% BF% 92% E3% 81% AE% E6% 95% 99% E7% A7% 91% E6% 9B% B8-% E6% A9% 9F% E6% A2% B0% E5% AD% A6% E7% BF% 92% E3% 81% AE% E5% 9F% BA% E6% 9C% AC % E3% 81% 8B% E3% 82% 89% E6% B7% B1% E5% B1% A4% E5% AD% A6% E7% BF% 92% E3% 81% BE% E3% 81% A7-AI -TECHNOLOGY / dp / 4798158577)

① Absolutely necessary ability as an engineer

・ Basic knowledge of python3 ・ Command line ・ Git

It's like you can't really do anything without these abilities. First and foremost, let's do these three things. Progate was recommended by the aspiring destination. You can do it for free, and the fact that you don't have to develop the environment yourself is an attractive first step. I myself finished it all in one day, so I think I can proceed without difficulty. I made a quick reference table that I will use a lot later with reference to Progate, so please use it. ・ Git basic / quick reference tableCommand line basics / quick reference table

② Setup

** Python family ** ・ Pip ・ Pipenv

These are the tools to get the libraries you need in Python. As a practical matter, pip is only used to install pipenv. See below for pip. ・ Pip quick reference table See good articles written by others about pipenv. This article describes the reason why Pipenv is used in the presence of Anaconda and pyenv. -Pyenv, pyenv-virtualenv, venv, Anaconda, Pipenv. I use Pipenv. For the installation of Pipenv, I referred to the following article. -Python development summary using Pipenv

** Essential for efficiency on Ubuntu terminals (command line) ** ・ Tmux

tmux is a tool that streamlines development. It allows you to operate on the same screen without having to create multiple windows in the terminal. I referred to the following website. ・ If you are an infrastructure engineer, do you master tmux! ??

** Other ** ・ Markdown notation (for Qiita)

At first I thought this "Markdown notation" was a program notation or something, but it is simply a convenient notation for the work I am currently doing in writing Qiita. There are just so many ways to make letters bigger and paragraphs. If you refer to the following article, everything is organized. ・ Qiita Markdown Notation List / Cheat Sheet

** Debugging work ** -Debugging with Pycharm

This is also a matter of efficiency, but it should not be just efficiency. There are useful ways to debug Python. That is "using Pycharm". Aspiring applicants were required to understand and master the contents of the following articles. -[Python for non-programmers] Basics of debugging with PyCharm

③ Machine learning system

It's about trying to master the library, not the Python grammar.

The important thing in learning machine learning is ** "It's enough to know what you can do" **.

At first I spent a lot of time studying each library, but I don't think it's necessary. If you understand and know what you can do, you can implement it by looking at a book in one hand or something like ** Qiita's cheat sheet **. The implementation of programming is not an exam. You can cheat as much as you want. Therefore, I write articles such as ** quick reference table ** including the meaning of "for myself".

There is a lot of great summary articles that I don't need to explain, so please check them out for yourself. The official tutorial is everything, so I'll post it.

From my point of view, I regret that I could have done everything in about a week.

You should study in the following order. ①scikit-learn scikit-learn Official Tutorial ②Numpy Numpy Official Tutorial All about Numpy Complete Basics ③Pandas Pandas Official Tutorial ④Matplotlib Matplotlib Official Tutorial ⑤Keras Keras Official (Japanese)

I'll say it again! The important thing is ** "If you know what you can do, it's OK!" **! !!

④ Book

Here are the books I read to learn machine learning.

The essence of machine learning

[Essence of Machine Learning](https://www.amazon.co.jp/%E6%A9%9F%E6%A2%B0%E5%AD%A6%E7%BF%92%E3%81%AE% E3% 82% A8% E3% 83% 83% E3% 82% BB% E3% 83% B3% E3% 82% B9-% E5% AE% 9F% E8% A3% 85% E3% 81% 97% E3 % 81% AA% E3% 81% 8C% E3% 82% 89% E5% AD% A6% E3% 81% B6Python-% E3% 82% A2% E3% 83% AB% E3% 82% B4% E3% 83% AA% E3% 82% BA% E3% 83% A0-Machine-Learning / dp / 4797393963) This book covers the basics of Python, the math stories needed for machine learning, numerical calculations using Python, and machine learning algorithms. It seems like all of machine learning is organized like no other. Those who have studied mathematics at university should start with numerical calculations using Python.

Machine learning to start at work

[Machine learning starting at work](https://www.amazon.co.jp/%E4%BB%95%E4%BA%8B%E3%81%A7%E3%81%AF%E3%81%98 % E3% 82% 81% E3% 82% 8B% E6% A9% 9F% E6% A2% B0% E5% AD% A6% E7% BF% 92-% E6% 9C% 89% E8% B3% 80- % E5% BA% B7% E9% A1% 95 / dp / 4873118255) As the name suggests, it gives you an image of business. Even if you study only theory, you don't know what it can be used for. I think it will be a good guide in such a case. Also, since it is the second book to read, it is best to read it after suppressing the basics. However, it also has the nature of a summary book, so I think it is ideal for reviewing machine learning methods.

Easy learning Mathematics to understand machine learning

[Easy Learning Mathematics for Understanding Machine Learning](https://www.amazon.co.jp/%E3%82%84%E3%81%95%E3%81%97%E3%81] % 8F% E5% AD% A6% E3% 81% B6-% E6% A9% 9F% E6% A2% B0% E5% AD% A6% E7% BF% 92% E3% 82% 92% E7% 90% 86% E8% A7% A3% E3% 81% 99% E3% 82% 8B% E3% 81% 9F% E3% 82% 81% E3% 81% AE% E6% 95% B0% E5% AD% A6% E3% 81% AE% E3% 81% 8D% E3% 81% BB% E3% 82% 93-% E3% 82% A2% E3% 83% A4% E3% 83% 8E-% E3% 83% 9F% E3% 82% AA% E3% 81% A8% E4% B8% 80% E7% B7% 92% E3% 81% AB% E5% AD% A6% E3% 81% B6-% E6% A9% 9F% E6 % A2% B0% E5% AD% A6% E7% BF% 92% E3% 81% AE% E7% 90% 86% E8% AB% 96% E3% 81% A8% E6% 95% B0% E5% AD % A6% E3% 80% 81% E5% AE% 9F% E8% A3% 85% E3% 81% BE% E3% 81% A7 / dp / 4839963525) Thanks to this book, I was able to understand the knowledge of mathematics in basic machine learning. If you find the essence of machine learning difficult, it's okay to start now.

Learn by moving with Python! New deep learning textbook From the basics of machine learning to deep learning

[Learn by moving with Python! New deep learning textbook From basic machine learning to deep learning](https://www.amazon.co.jp/Python%E3%81%A7%E5%8B%95%E3% 81% 8B% E3% 81% 97% E3% 81% A6% E5% AD% A6% E3% 81% B6-% E3% 81% 82% E3% 81% 9F% E3% 82% 89% E3% 81 % 97% E3% 81% 84% E6% B7% B1% E5% B1% A4% E5% AD% A6% E7% BF% 92% E3% 81% AE% E6% 95% 99% E7% A7% 91 % E6% 9B% B8-% E6% A9% 9F% E6% A2% B0% E5% AD% A6% E7% BF% 92% E3% 81% AE% E5% 9F% BA% E6% 9C% AC% E3% 81% 8B% E3% 82% 89% E6% B7% B1% E5% B1% A4% E5% AD% A6% E7% BF% 92% E3% 81% BE% E3% 81% A7-AI- TECHNOLOGY / dp / 4798158577) This is the book that I mainly studied. This is a book I received when I went to the laboratory to consult with a professor about how to learn artificial intelligence. All of deep learning is described (aside from the theoretical story). We do everything from environment development to implementation. It is recommended for true beginners to do this book first and then the above books. Also, I am able to learn all about the libraries I mentioned above (such as Numpy).

at the end

I've put everything I've done into this article. From now on, I would like to get in touch with the cutting-edge technology of artificial intelligence while reading the paper. It seems to be long, but thank you for reading.

Recommended Posts

Everything for beginners to be able to do machine learning
[For beginners] Introduction to vectorization in machine learning
First Steps for Machine Learning (AI) Beginners
An introduction to OpenCV for machine learning
Vim Command Memorandum ~ Beginners will be able to do what they want! ~
An introduction to machine learning for bot developers
Recommended study order for machine learning / deep learning beginners
[For beginners] What to do after installing Anaconda
Introduction to machine learning
Preparing to start "Python machine learning programming" (for macOS)
Data set for machine learning
Japanese preprocessing for machine learning
Learning flow for Python beginners
An introduction to machine learning
Machine learning beginners tried RBM
~ Tips for beginners to Python ③ ~
Super introduction to machine learning
Made icrawler easier to use for machine learning data collection
For those who want to start machine learning with TensorFlow2
How to use machine learning for work? 03_Python coding procedure
Machine learning beginners try to reach out to Naive Bayes (2) --Implementation
Machine learning beginners try to reach out to Naive Bayes (1) --Theory
Introduction to machine learning Note writing
[Python machine learning] Recommendation of using Spyder for beginners (as of August 2020)
[For IT beginners] What to do when the rev command cannot be used with Git Bash
Python learning memo for machine learning by Chainer Chapter 8 Introduction to Numpy
Newton's method for machine learning (from one variable to multiple variables)
Before the introduction to machine learning. ~ Technology required for machine learning other than machine learning ~
Python learning memo for machine learning by Chainer Chapter 10 Introduction to Cupy
Machine learning summary by Python beginners
Introduction to Machine Learning Library SHOGUN
Machine learning meeting information for HRTech
[Recommended tagging for machine learning # 4] Machine learning script ...?
[Machine learning] I tried to do something like passing an image
How to collect machine learning data
How to study for the Deep Learning Association G test (for beginners) [2020 version]
That's why I'll be able to search for single-seat constituencies by location
Approach commentary for beginners to be in the top 1.5% (0.83732) of Kaggle Titanic_3
How to use machine learning for work? 02_Overview of AI development project
Search for technical blogs by machine learning focusing on "easiness to understand"
Python beginners publish web applications using machine learning [Part 2] Introduction to explosive Python !!
Approach commentary for beginners to be in the top 1.5% (0.83732) of Kaggle Titanic_1
Approach commentary for beginners to be in the top 1.5% (0.83732) of Kaggle Titanic_2
Machine learning beginners take Coursera's Deep learning course
Introduction to Machine Learning: How Models Work
Amplify images for machine learning with python
Record the steps to understand machine learning
I installed Python 3.5.1 to study machine learning
Why Python is chosen for machine learning
"Usable" one-hot Encoding method for machine learning
[Shakyo] Encounter with Python for machine learning
Introduction to ClearML-Easy to manage machine learning experiments-
[Python] Web application design for machine learning
Support Vector Machine (for beginners) -Code Edition-
Beginners read "Introduction to TensorFlow 2.0 for Experts"
How to enjoy Coursera / Machine Learning (Week 10)
[Deep learning] Nogizaka face detection ~ For beginners ~
Creating a development environment for machine learning
Introduction to Deep Learning (1) --Chainer is explained in an easy-to-understand manner for beginners-
Machine learning
I tried to implement various methods for machine learning (prediction model) using scikit-learn.