[PYTHON] Try "100 knocks on data science" ①

Practical learning environment for data science beginners "Data Science 100 Knock (Structured Data Processing)" released on GitHub for free https://digitalpr.jp/r/39499

I will try this. By the way, I don't know R at all, so I don't think I'll do it.

environment

・ Windows 10 ・ Docker for Desktop ・ Git

Environment

Start Docker

 wsl -e docker-desktop

Clone from Git

git clone https://github.com/The-Japan-DataScientist-Society/100knocks-preprocess.git

Build Docker

cd 100knocks-preprocess
docker-compose up -d --build

It will take a while ... Something is dropped ...

Precautions for environment construction

If you are using Docker Toolbox, the URL of the access destination will change. http://192.168.99.100:8888 The host when accessing the DB using the client tool also changes.

Confirmation of 100 knock environment

Access below http://localhost:8888

A screen like this should open

When you open work / preprocess_knock_SQL.ipynb

There is a SQL problem like this. The same is true for Python and R, and there seems to be an answer in the anser folder. DB is Postgre12.

It's not that difficult, but when asked to normalize it, it's okay ...? I google for a moment w

bonus

This environment seems to use *** Jupyter Notebook ***. I thought I'd look at it a lot, but I didn't know it so I looked it up.

Jupyter notebook is one of the Jupyter projects and is an OSS web application. The Jupyter project is a project for developing services and OSS to realize interactive processing execution across multiple languages. And you can manage the execution result of the program collectively.

It seems that this screen with the extension * .ipynb is made, and when I opened the inside, it was a json format file like the one below.

Entity_Relationship.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#DB logical design (ER diagram)\n",
    "-Some data that is FK may have data that does not exist in the master table.\n",
    "-Example) Non-member customer ID is not included in the customer table\n",
    "-Therefore, the FK information does not meet the external reference constraints in a typical database.\n",
    "-Please use it as reference information when combining data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![ER diagram](data/100knocks_ER.png\"sample\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}

reference

Jupyter notebook https://qiita.com/szk3/items/920fd3e905ed16469780