[PYTHON] Install Caffe on OSX 10.10 and classify images by reference model

** Information as of April 22, 2015 ** ** caffe updates frequently so we can't guarantee it will work right now **

Image discrimination is performed using the reference model from the installation. Up to image identification using the reference model bvlc_reference_caffenet.caffemodel and classify.py.

What is caffe

Deep learning framework A very fast, active and popular framework See the official for details http://caffe.berkeleyvision.org/

For those who want to see a little performance Caffe Demos If you paste the URL of the image you want to classify into Image classification is done with the official reference model

Install on MacOSX 10.10.3 (yosemite)

I think the following articles will also be helpful Install Caffe on OS X 10.10 (Yosemite) (libstdc ++ Problem Solving)

About the installation environment

・ IMac 27-inch, Late 2013 -CPU 3.5 GHz Intel Core i7 16 GB 1600 MHz DDR3 ・ GPU NVIDIA GeForce GTX 775M 2048 MB ・ OS MaxOSX10.10.3 yosemite

Introduce dependent libraries

Python Use Anaconda-2.1.0 as it is officially recommended by Anaconda Anaconda is a distribution that introduces various packages to build a numerical calculation environment in python. Use this

Managed by pyenv Building a Python environment on Mac using pyenv Please introduce with reference to

In this article, create pyenv in home dictation without using brew. Changed python environment to Anaconda There are 2 series and 3 series, but 2 series is better

python

pyenv install anaconda-2.1.0
pyenv global anaconda-2.1.0
pyenv local anaconda-2.1.0

It should have already been installed in anaconda-2.1.0, but let's check the package for the time being.

python

pip install -r caffe/python/python/requirements.txt

Let's install the required packages at

CUDA7.0 Introduced 7.0 and above from NVIDIA Now it's officially released so no registration required Compile with libc ++ if it is 7.0 or higher NAVIDIA CUDA7 Downloads

Other libraries

Install with Homebrew

python

brew install --fresh -vd snappy leveldb gflags glog szip lmdb

boost boost-install python

There was information that it would not work unless it was ver1.55 in Chimata. It worked with boost1.57 and boost-python1.57, so it's as it is

python

brew install --build-from-source --fresh -vd boost boost-python

install protobuf

Be sure to remember the --with-python option If you don't attach it, you will get angry with PROTOC at build time.

python

brew install --build-from-source --with-python --fresh -vd protobuf

install openBLAS

python

brew tap homebrew/science
brew install homebrew/science/openblas

installation of hdf5

** Probably improved with the update, so if you can runtest without any problems, you don't need to change the Formula **

When doing a run test libhdf5_hl.8.dylib May result in an error saying that is not found (as of April 22, 2015)

Apparently in hdf5-1.8.14 libhdf5_hl.9.dylib I changed Homebrew's Formula so that it would be installed. Introduce hdf5-1.8.13

python

brew edit hdf5

change url and sha1

python

class Hdf5 < Formula
  homepage "http://www.hdfgroup.org/HDF5"
 --> url "http://www.hdfgroup.org/ftp/HDF5/releases/hdf5-1.8.13/src/hdf5-1.8.13.tar.bz2"
 --> sha1 "712955025f03db808f000d8f4976b8df0c0d37b5"

python

brew install --build-from-source --fresh -vd hdf5

For the time being, the installation of dependent libraries is complete.

Modify makefile and makefile.config

Install Caffe on OS X 10.10 (Yosemite) (libstdc ++ Problem Solving) Please refer here as well

Modify Makefile

Let's modify the Makefile

python

#stdlib=libstdc++To-stdlib=libc++change to
CXXFLAGS += -stdlib=libc++
LINKFLAGS += -stdlib=libc++
			
#BLAS_Also pass INCLUDE (there are two lines, so let's pass both)
BLAS_INCLUDE ?= /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/Headers/

Fix Makefile.com fig

python

#Change compilation to clang
# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
CUSTOM_CXX := /usr/bin/clang++

#Comment out because it is 7 or more
# For CUDA < 6.0, comment the *_50 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
		-gencode arch=compute_20,code=sm_21 \
		-gencode arch=compute_30,code=sm_30 \
		-gencode arch=compute_35,code=sm_35 \
#-gencode arch=compute_50,code=sm_50 \
#-gencode arch=compute_50,code=compute_50

#Around Anaconda
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda
#It's complicated, so ANACONDA_Do not use HOME, go to ↓
 PYTHON_INCLUDE := /Users/name/.pyenv/versions/anaconda-2.1.0/include /Users/name/.pyenv/versions/anaconda-2.1.0/include/python2.7 /Users/name/.pyenv/versions/anaconda-2.1.0/lib/python2.7/site-packages/numpy/core/include

# open for OpenBlas
BLAS := open

 BLAS_INCLUDE := /usr/local/Cellar/openblas/0.2.14/include
 BLAS_LIB := /usr/local/Cellar/openblas/0.2.14/lib

Put it through the python path
# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /Users/name/.pyenv/versions/anaconda-2.1.0/lib

pass the lib path
# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /Developer/NVIDIA/CUDA-7.0/lib

Build

In the first folder of caffe

python

make all -j8
make test -j8

-j8 is an option to parallelize compilation, the number is the number of parallel, so change it according to the environment, I think that it will take a long time to build as a single.

Do a test

python

make runtest

After various test results are output for a while

python

YOU HAVE 2 DISABLED TESTS

Is displayed, it's okay

Build to use Caffe with python

In the first folder of caffe

python

make pycaffe

Hit! Don't forget to run it via python this time!

Put the path in .bash_profile

python

PYTHONPATH=/Users/name/caffe/Through python

If you get the following error during build

python

python/caffe/_caffe.cpp:1:10: fatal error: 'Python.h' file not found
#include <Python.h>  // NOLINT(build/include_alpha)

Add the following path to .bash_profile

python

CPLUS_INCLUDE_PATH=/Users/name/.pyenv/versions/anaconda-2.1.0/include/python2.7

Import caffe with python

Protoc is required, so install it in python with pip

pip install protobuf

Launch python in interactive mode

python

>>> import caffe

If you pass with this, it's ok In this environment

python

caffe Fatal Python error: PyThreadState_Get: no current thread

And python was forcibly terminated, so I passed the following path with .bash_profile

python

export DYLD_FALLBACK_LIBRARY_PATH=/Users/name/.pyenv/versions/anaconda-2.1.0/lib:/usr/local/cuda/lib:/usr/local/lib:/usr/local/cuda/lib:/usr/local/lib

Image classification with reference model

Easy image classification with Caffe There is an article saying, but the version has changed and it is not easy. This time, we will go to the first part of this page.

Download reference model

python

caffe/models/bvlc_reference_caffenet/readme.md

According to it, it seems to exist at the following URL http://dl.caffe.berkeleyvision.org/bvlc_reference_caffenet.caffemodel Download from

python

caffe/models/bvlc_reference_caffenet/

Save it below

Download related files

caffe/data/ilsvrc12/get_ilsvrc_aux.sh Execute the script, Get the related data of the model.

Image of object recognition

As a sample Let's classify the images of this African elephant.

python

caffe/python/Elefante.jpg

Save it as.

Image classification using classify.py

python

caffe/python/

Hit the following command with to classify

python

python classify.py --raw_scale 255 ./Elefante.jpg  ./result.npy

It's ok if you can clear it here Depending on the version

python

  File "classify.py", line 137, in <module>

    main(sys.argv)

  File "classify.py", line 109, in main

    channel_swap=channel_swap)

  File "/Users/name/caffe/python/caffe/classifier.py", line 34, in __init__

    self.transformer.set_mean(in_, mean)

  File "/Users/name/caffe/python/caffe/io.py", line 255, in set_mean

    raise ValueError('Mean shape incompatible with input shape.')

ValueError: Mean shape incompatible with input shape.

Error may come back.

It seems that the input'caffe / imagenet / ilsvrc_2012_mean.npy' cannot be handled well. The cause is used by classify.py

python

caffe/python/caffe/io.py

Lines 253-254

python

if ms != self.inputs[in_][1:]:
	raise ValueError('Mean shape incompatible with input shape.')

It seems to be in. Apparently, the newly created part isn't working well, so I'll rewrite it in the old Caffe style.

python

if ms != self.inputs[in_][1:]:
	print(self.inputs[in_])
	in_shape = self.inputs[in_][1:]
	m_min, m_max = mean.min(), mean.max()
	normal_mean = (mean - m_min) / (m_max - m_min)
	mean = resize_image(normal_mean.transpose((1,2,0)),in_shape[1:]).transpose((2,0,1)) * (m_max - m_min) + m_min
	#raise ValueError('Mean shape incompatible with input shape.')

The same problem is raised at the following URL http://stackoverflow.com/questions/28692209/using-gpu-despite-setting-cpu-only-yielding-unexpected-keyword-argument

After the change, use classify.py again

Execution result

Loading file: ./Elefante.jpg
Classifying 1 inputs.
Done in 0.90 s.
Saving results into ./result.npy

Is ok. The determination result is saved in result.pny.

For the contents of result.npy Easy image classification with Caffe Published in

python

show_result.py

You can see the result numerically by using.

python

$ python show_result.sh ../data/ilsvrc12/synset_words.txt result.npy 
1 | n02504458 African elephant, Loxodonta africana | 81.2%
2 | n01871265 tusker | 14.9%
3 | n02504013 Indian elephant, Elephas maximus |  3.3%

81% chance of African elephant A large creature with fangs at 14% 3% are Indian elephants Was judged