[PYTHON] [For non-programmers] How to walk Kaggle

This article is the third day of the Advent Calendar in Kaggle Advent Calendar 2019.

I hope it will be helpful for those who think "** Kaggle is interested, but it's apt and scary ... **"!

• The following information is as of December 2019

Until I met Kaggle

Below is a brief history of my career before I met Kaggle.

--Marketer ⇒ Consultant --From liberal arts (art history major) --Inexperienced in programming (just touched HTML) --There is only awe on the black screen ...

I was in such a state, but in the last year I studied Python & machine learning and managed to understand Kaggle. So, I will write to myself a year ago that it is good to know this.

Target audience

To whom?

--I want to improve my data analysis / machine learning skills --Kaggle Looks fun, but inexperienced in programming --Mainly marketers and planning staff

what?

--Kaggle is worth a try!

• The content is not for Kaggle Tsuyotsuyo.
• Mainly personal subjectivity (so-called poem)

What is "Kaggle"?

What is Kaggle

In a word, it is the Tenkaichi Budokai of data analysis using "machine learning".

Data to analyze

--Companies provide data and themes and hold competitions --Themes range from medical care, marketing, finance, etc.

Competition period

――For about 1 to 3 months --Relatively long

Ranking

--Create a machine learning model and rank it according to the prediction result of the model

Prize money

――There is also a competition where prize money will be given to the top players --As of December 2019, the DSB competition currently being held is No. 1 for $ 100,000 (about 10 million yen!)

Entry fee

• free!

Registered user

--Approximately 120,000 people (as of December 2019) * Reference source

About rank

Kaggle has five ranks depending on its strength. (Strictly speaking, you can get a rank not only by your strength in the competition, but also by your contribution to the Kaggle community such as Kernels, Discussion, and Datasets.)

--Grandmaster * The strongest ...!

• Master
• Expert
• Contributor --Novice * First from here

It's no exaggeration to say that the people who work on Kaggle (commonly known as Kaggler) are spending a lot of time every day trying to compete in order to raise this rank (probably).

Number of people by rank

The number of people as of December 2019 is as follows

Image of each rank

It is an outline and an image diagram of each rank. (It's just subjective)

Rank up conditions

Click here for rank-up conditions. (In Competitions) The Expert suddenly asks for a medal. .. .. If you are a beginner, you want to aim for Expert first.

Medal acquisition conditions

There are three types of medals: Gold, Silver, and Bronze.

Click here for medal acquisition conditions. (In Competitions) Strict conditions vary depending on the number of participants, but the following is a guide.

• Gold：Top10
• Silver：Top 5%
• Bronze：Top 10%（※）

• Suddenly Top 10%! You might think, but one theory is that just picking up information about the competition (Notebooks, Discussion, etc.) is not a dream to get Bronze ... (possible survivorship bias). )

For example, such a competition

Prediction of survivors of the Titanic sinking accident

• Titanic: Machine Learning from Disaster --A "Hello World" competition at Kaggle ――Since it is a practice competition, there is no particular end of the period --Predict whether or not the passenger will survive based on data such as the passenger's gender, age, port of board, and cabin rank.

Predicting the speed at which dogs and cats are picked up at pet stores

• PetFinder.my Adoption Prediction --Held from December 2018 to March 2019 --The first prize is $ 10,000 --Predict the speed at which a pet will be adopted based on data such as the profile, description, and photos of each pet. --Practical competition to predict based on multiple sources such as table data, images, and text

Benefits of doing Kaggle

I've listed three benefits of doing Kaggle.

① Understand your own data science skills (also prove)

Data science skills are difficult to measure without practice. However, earning medals and ranks at Kaggle can prove your approximate data science skills. Recently, the number of companies adopting Kaggle is gradually increasing, which is also evidence when stepping up.

② Learn the know-how of data scientists from around the world (free of charge!)

As I learned by actually participating in the competition, Kaggle has a wonderful culture of sharing know-how and information, not just competing with others. In particular, "Notebooks" explains the know-how and trends of data science in an easy-to-understand manner along with the code, and is recommended for beginners.

③ You can understand what machine learning is doing at the skin level

Nowadays, machine learning is being incorporated more and more into the apps and marketing tools around us. With Kaggle, you can master the modeling process of machine learning, so you can get a feel for what you can and cannot do with machine learning.

Kaggle should start here

There seems to be various disagreements, but there are three personal points below.

① If you have no machine learning / Python knowledge, choose one "Matsuo Lab" textbook!

Matsuo Laboratory of the University of Tokyo, which is famous for machine learning / AI.

You can download the text of "Data Scientist Training Course" conducted by Matsuo Laboratory for free. This is amazing, it's an incident! The text and Python code carefully explain the beginner's graduation level of machine learning, so if you have difficulty understanding the contents of Kaggle, I think you should complete this.

▼ Click here to download the "Matsuo Lab" text https://weblab.t.u-tokyo.ac.jp/gci_contents/

② Use "Notebooks" to copy the ones with a high number of votes and get the hang of it!

As I mentioned earlier, the "Notebooks" in each competition are a treasure trove.

If you choose a competition that seems easy to work on, it is recommended to copy the notebook with a high number of votes in "Notebooks". Notebooks with a high number of votes are carefully selected and easy to understand, so you can relive the flow from data acquisition to model result submission by copying sutras, and you can learn the tips of machine learning.

③ If you are alone, you will get stuck without information, so it is recommended to connect with someone!

Kaggle is a basic individual battle. However, the competition period is as long as 1 to 3 months, and motivation may drop or it may get bogged down.

At such times, if you have friends who are doing Kaggle, you can share useful information (private sharing is NG) and encourage each other, so you can enjoy Kaggle even more.

Personally, I recommend the following three.

--Go to Kaggle related seminars (connpass, etc.) --Follow Kaggle on Twitter --Let's join the Kaggle community loosely

at the end

Kaggle is a great platform for beginners to advanced machine learning / data analysts.

When you're doing Kaggle, you'll be able to compete with TOP data scientists around the world, share information, and get taught. However, when I think about it carefully, I realized that this is amazing, and while writing this article, I realized it again.

We will continue to do our best to help Kaggle remain a good platform! (First of all, I want a medal ...)