[PYTHON] Build a machine learning environment on mac (pyenv, deeplearning, opencv)

Memo for building a python environment on mac

1. Install X-code

From the app store.

2. Install homebrew

Official site: http://brew.sh/index_ja.html

$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Bonus wget

$ brew install wget

3. Install Git

$ brew install git
$ git config --global user.name msrks
$ git config --global user.email <email>
# for check
$ git config —-global —-list

4. Set Solarized theme on terminal

$ mkdir config
$ cd config
$ git clone https://github.com/tomislav/osx-terminal.app-colors-solarized

Set solarized-dark as the default from the terminal preferences menu

5. Install atom

From https://atom.io.

editor settings

-Do not indent when pasting
- font-size: 16
-Soft wrap
-Tab width: 4

Package introduction

- atom-runner : Ctrl+Code can be executed in R
- japanese-menu
- autocomplete-python
- python-indent
- file-icons
- minimap
- language-restructuredtext
- atom-beautify :Insert autopep8 with pip.

6. Install Python

Updated system python2 (2.7.13). At the same time, pip is also included.

$ brew install python
$ pip install numpy scipy matplotlib seaborn pandas

Put pyenv

$ brew install pyenv-virtualenv
$ cd
$ touch .bash_profile
$ open .bash_profile

export PYENV_ROOT=${HOME}/.pyenv
if [ -d "${PYENV_ROOT}" ]; then
    export PATH=${PYENV_ROOT}/bin:$PATH
    eval "$(pyenv init -)"
    eval "$(pyenv virtualenv-init -)"
fi

Also include python 3.6.0

$ pyenv install -l
$ pyenv install 3.6.0

7. Build a python2 environment for research

$ pyenv virtualenv system ML-2.7.13
$ pyenv global ML-2.7.13

data analysis

$ pip install jupyter
$ jupyter nbextension enable --py widgetsnbextension
$ pip install scikit-learn
$ pip install networkx
$ pip install statsmodels

web-development

$ pip install flask
$ pip install sphinx_rtd_theme mkdocs sphinx_rtd_theme

documentation

$ pip install sphinx mkdocs
$ pip install sphinx_rtd_theme sphinx_bootstrap_theme numpydoc

web-scraping

$ pip install beautifulsoup4 lxml

deeplearning (tensorflow, theano, chainer, keras, tflearn)

$ pip install tensorflow
$ pip install tflearn
$ pip install keras h5py
$ brew install graphviz
$ pip install pydot pydot-ng
$ pip install chainer

Image processing

$ pip install scikit-image Pillow

# opencv
$ brew install gcc
$ brew install cmake
$ brew install tesseract
$ brew tap homebrew/science
# opencv2
$ brew install opencv
$ ln -s /usr/local/Cellar/opencv/2.4.13.2/lib/python2.7/site-packages/cv2.so /Users/riki/.pyenv/versions/ML-2.7.13/lib/python2.7/site-packages/

Docker for mac

https://docs.docker.com/docker-for-mac/ Whale icon in status bar> Preference ...> Set "HTTP proxy settings:" in the Advanced tab

OpenAI-gym

$ pip install gym
$ brew install swig
$ pip install gym[all]

OpenAI-Universe

$ brew install golang libjpeg-turbo
$ pip install numpy incremental
$ git clone https://github.com/openai/universe.git
$ cd universe
$ pip install -e .

Universe-Starter-Agent

$ brew install tmux htop
$ cd <workspace>
$ git clone https://github.com/openai/universe-starter-agent

wxpython [option]

$ brew install wxpython wxmac
#$ ln -s #/usr/local/Cellar/wxpython/3.0.2.0/lib/python2.7/site-packages/wx-3.0-osx_cocoa/wx #/Users/riki/.pyenv/versions/ML-2.7.13/lib/python2.7/site-packages/

opencv3 [option]

$ pyenv install anaconda3-4.1.1
$ cd <workspace with opencv3>
$ pyenv local anaconda3-4.1.1
$ conda install -c https://conda.anaconda.org/menpo opencv3
$ conda update hdf5

8. Operation check

opencv

$ mkdir workspace
$ cd workspace
$ wget --no-check-certificate -O gorilla.jpg https://upload.wikimedia.org/wikipedia/commons/thumb/b/bc/Male_silverback_Gorilla.JPG/236px-Male_silverback_Gorilla.JPG
$ touch test-cv2.py
$ atom test-cv2.py

test-cv2.py


import cv2
img = cv2.imread("gorilla.jpg ")
img
cv2.imshow('image', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
$ python test-cv2.py

keras: training

It didn't work at first. (Error in the link below) http://qiita.com/Kodaira_/items/1a3b801c7a5a41c9ce49 Modify matplotlibrc by referring to the link

$ python -c "import matplotlib;print(matplotlib.matplotlib_fname())"
/Users/riki/.pyenv/versions/ML-2.7.13/lib/python2.7/site-packages/matplotlib/mpl-data/matplotlibrc
$ atom /Users/riki/.pyenv/versions/ML-2.7.13/lib/python2.7/site-packages/matplotlib/mpl-data/matplotlibrc

# line 38
- backend : macosx
+ backend : Tkagg
$ wget --no-check-certificate https://gist.githubusercontent.com/msrks/ea9ce906fe0c6a2947e7dec44291ec91/raw/ac8e24077e4127ae3dc392c3a4ca2235f3861aba/mnist_mlp.py
$ python mnist_mlp.py
$ open model_mlp.png
$ open log_acc.png
$ open log_loss.png

keras: prediction

$ wget --no-check-certificate -O gorilla.jpg https://upload.wikimedia.org/wikipedia/commons/thumb/b/bc/Male_silverback_Gorilla.JPG/236px-Male_silverback_Gorilla.JPG
$ wget --no-check-certificate https://gist.githubusercontent.com/msrks/6422288f60f4f56a5d8fcbc95acf02dc/raw/e32966364ab08389490b5cadc260ba7220e42550/test_vgg16.py
$ python test_vgg16.py
$ open model_vgg16.png

9. Addendum

Display the IP address in the menu bar

$ brew cask install ip-in-menu-bar

Make the tree command available

$ brew install tree
$ tree -L <depth>

It is convenient to turn off input completion and automatic spelling conversion when entering alphanumeric characters.

installation of mongoDB

https://docs.mongodb.com/manual/tutorial/install-mongodb-on-os-x/#install-mongodb-community-edition-with-homebrew

$ brew update
$ brew install mongodb

$ sudo mkdir /var/lib/mongodb
$ sudo touch /var/log/mongodb.log
$ sudo mongod --fork --dbpath /var/lib/mongodb --logpath /var/log/mongodb.log
$ brew services start mongodb
$ brew services list

Recommended Posts

Build a machine learning environment on mac (pyenv, deeplearning, opencv)
Build a machine learning Python environment on Mac OS
Build a machine learning environment
Build a Python environment on your Mac using pyenv
Build a machine learning environment natively on Windows 10 (x64)
Build a Python + OpenCV environment on Cloud9
Build a Python environment on Mac (Mountain Lion)
Build a Python development environment on your Mac
Build a Python machine learning environment with a container
Build a Django development environment using pyenv-virtualenv on Mac
Build a local development environment for Laravel6.X on Mac
Build a Python development environment on Mac OS X
Build a machine learning application development environment with Python
Build a Python development environment using pyenv on MacOS
Building a Python environment on Mac
Build an environment for machine learning using Python on MacOSX
Build a machine learning environment using PyCharm on Ubuntu environment (TensorFlow will also be introduced!)
Create a Python (pyenv / virtualenv) development environment on Mac (Homebrew)
Build a machine learning scikit-learn environment with VirtualBox and Ubuntu
Create a Python environment on Mac (2017/4)
Build a python data analysis environment on Mac (El Capitan)
Steps to quickly create a deep learning environment on Mac with TensorFlow and OpenCV
Machine learning environment settings based on Python 3 on Mac (coexistence with Python 2)
Build a python3 environment on CentOS7
Build a PyData environment for a machine learning study session (January 2017)
Build a Python environment on your Mac with Anaconda and PyCharm
Build Python3 and OpenCV environment on Ubuntu 18.04
Build a python environment on MacOS (Catallina)
Create a python environment on your Mac
I installed Kivy on a Mac environment
Build Python environment with Anaconda on Mac
Build a python virtual environment with pyenv
Creating a development environment for machine learning
Build AI / machine learning environment with Python
After buying a new Mac, use pyenv + poetry to build a Python environment.
[Definitive Edition] Building an environment for learning "machine learning" using Python on Mac
Try to build python and anaconda environment on Mac (by pyenv, conda)
Create a Python3 environment with pyenv on Mac and display a NetworkX graph
Build a "Deep learning from scratch" learning environment on Cloud9 (jupyter miniconda python3)
Build a LAMP environment on your local Docker
Build a WardPress environment on AWS with pulumi
Build python environment with pyenv on EC2 (ubuntu)
Simply build a Python 3 execution environment on Windows
Build a Django environment on Raspberry Pi (MySQL)
Build a python environment with ansible on centos6
[Memo] Build a virtual environment with Pyenv + anaconda
Build a simple Python virtual environment without pyenv
Build a virtual environment with pyenv and venv
Build a Kubernetes environment for development on Ubuntu
Remove old pyenv environment on Mac and update
Build a Python development environment on Raspberry Pi
Install pyenv on mac
Pyenv + virtualenv on Mac
Build a GVim-based Python development environment on Windows 10 (3) GVim8.0 & Python3.6
Build an OpenCV4 environment on Raspberry Pi using Poetry
Install Python3 on Mac and build environment [Definitive Edition]
Build an interactive environment for machine learning in Python
# 2 Build a Python environment on AWS EC2 instance (ubuntu18.04)
Build a GVim-based Python development environment on Windows 10 (1) Installation
How to build a Django (python) environment on docker
How to build a development environment for TensorFlow (1.0.0) (Mac)