I will post to Qiita for the first time on Labor Thanksgiving Day. Thank you.

wrap up

Created a ROS node that runs Tensorflow's trained MNIST.
I posted the script on GitHub.

background

There is a robot development environment called ROS. (ROS official website) With ROS, data from sensors such as cameras can be processed in an integrated manner, and robots can be moved and simulated.

** I want to incorporate Deep Learning here! !! ** **

What i did

If you do it first, it's a tutorial! So Tensorflow [Deep MNIST for Experts](https://www.tensorflow.org/versions/r0.10/tutorials/mnist/pros/index. html) trained model now runs on ROS.

Previously, there was a person who runs a trained Tensorflow model that recognizes objects on ROS, so that The model was visualized with reference to.

By writing the model explicitly, I think it became a script that can be easily applied to other tasks. (You can change the network configuration, and if you learn it first, you can move it.)

Processing procedure

So what I actually wrote the script to do is as follows.

Configure CNN to read trained files
Subscribe image information from the camera node
Compress image information to 28 * 28 and binarize black and white so that it fits in MNIST CNN.
Show the image to CNN and estimate the numbers
Publish the result

Description of ROS Nodes Publisher and Subscriber Description

Code commentary

I wrote in Another article

How to move

As in README.md on GitHub $ roscore
$ python tensorflow_in_ros_mnist.py image:=/cv_camera/image_raw
$ rosrun cv_camera cv_camera_node
$ rostopic echo /result

Tensorflow, ROS, cv_bridge, camera driver, OpenCV, numpy need to be prepared in advance. If you make a mistake in the execution order, an error will occur but it will work, so don't worry.

result

When I displayed the image on my computer with paint and pointed the camera at it, it became as shown in the figure below. The image on the right is the image that came in from the camera, and the data: on the left is the number that CNN expected and published. It is correct. Screenshot from 2016-11-23 15_41_51.png

The handwritten 9 was also recognized properly.

Screenshot from 2016-11-23 15_44_07.png

However, the recognition accuracy fluctuates considerably depending on the nature of the camera. You can devise the distance between the camera and the image, trim it with numpy before putting it in CNN, and play with the binarized value.

Afterword

** I'm glad that it worked properly for the time being! !! ** ** I was nervous about the first post on both GitHub and Qiita. Especially, I was quite confused to find out and understand the license of GitHub. As a result, I put the Aphache 2.0 license by referring to this site. Is this correct? The code was pretty messy, I didn't add any comments, and I thought I should do my best.

** Overwhelming thanks on Labor Thanksgiving Day! !! ** **

References not included in the text:

[PYTHON] Set up a node to do MNIST on ROS using Tensorflow