This article is reprinted from Our blog.

Introduction

This time, I will run OpenPose on Mac using chainer. (It will be a part of the function to generate the detection heatmap instead of the whole)

1. Download Caffe's trained model
Load Caffe's model from Chainer
1. Apply images to Chainer's network

It has been reported that OpenPose, a skeleton estimation algorithm using a monocular camera announced at CVPR2017, will realize real-time processing on Ubuntu and Windows. However, since GPU is required or Mac does not support it, I thought that it could be run on Mac or CPU somehow, so I will load Caffe's model with Chainer and execute only the DNN part of the first half.

Execution environment (should work on Linux)

Mac book Pro (2.7 GHz Intel Core i5,16 GB 1867 MHz DDR3) Python2.7 (Chainer, Numpy)

work

1. Download Caffe's trained model

Download the executable binary for Windows at here. It contains the Caffe parameter file. (If you are interested in direct links, download and unzip the portable OpenPose demo 1.0. In the middle of https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/installation.md. Please drop from 1.)

Load Caffe's model from Chainer

This time, we will use a COCO model that estimates 18 skeletons. (MPI model can be done by the same procedure) The Caffe model is stored in the following directory.

OpenPose_demo_1.0.1/models/pose/coco/pose_iter_440000.caffemodel

Read this from chainer.

from chainer.functions import caffe
func = caffe.CaffeFunction('pose_iter_440000.caffemodel')

1. Apply images to Chainer's network

OpenPose_demo_1.0.1/models/pose/coco/pose_deploy_linevec.prototxt If you read, the network configuration is written, and if you look at the last output layer

layer {
   name: "concat_stage7"
   type: "Concat"
   bottom: "Mconv7_stage6_L2"
   bottom: "Mconv7_stage6_L1"
   # top: "concat_stage7"
   top: "net_output"
   concat_param {
     axis: 1
   }
 }

The first output node is defined with the name Mconv7_stage6_L2. (Mconv7_stage6_L1 is more like a bone)

So, by specifying this in the output layer of func, you can get the output map of (1,19, h, w).

x = chainer.Variable([Image numpy array])
y, = func(inputs={'data': x}, outputs=['Mconv7_stage6_L2'])
print y.data.shape #(1,19,h,w)

The numpy array of images is 0.0 to 1.0 in size (1, 4, height, width) and values instead of 0 to 255. Also note that the dtype must be np.float32. (I don't know if RGBA is supposed to be used instead of RGB, but I'm typing in the 4th with 0s filled in)

↓ This image I tried

    POSE_COCO_BODY_PARTS {
        {0,  "Nose"},
        {1,  "Neck"},
        {2,  "RShoulder"},
        {3,  "RElbow"},
        {4,  "RWrist"},
        {5,  "LShoulder"},
        {6,  "LElbow"},
        {7,  "LWrist"},
        {8,  "RHip"},
        {9,  "RKnee"},
        {10, "RAnkle"},
        {11, "LHip"},
        {12, "LKnee"},
        {13, "LAnkle"},
        {14, "REye"},
        {15, "LEye"},
        {16, "REar"},
        {17, "LEar"},
        {18, "Bkg"},
    }

The heatmap shows the probability of existence of each part of the body written in the structure above. It takes about 10 seconds per sheet, but it works on the CPU, so it seems to be more versatile.

[PYTHON] I tried running the DNN part of OpenPose with Chainer CPU

Introduction

Execution environment (should work on Linux)

work