[PYTHON] Set up a node to do MNIST on ROS using Tensorflow

I will post to Qiita for the first time on Labor Thanksgiving Day. Thank you.

wrap up

background

There is a robot development environment called ROS. (ROS official website) With ROS, data from sensors such as cameras can be processed in an integrated manner, and robots can be moved and simulated.

** I want to incorporate Deep Learning here! !! ** **

What i did

If you do it first, it's a tutorial! So Tensorflow [Deep MNIST for Experts](https://www.tensorflow.org/versions/r0.10/tutorials/mnist/pros/index. html) trained model now runs on ROS.

Previously, there was a person who runs a trained Tensorflow model that recognizes objects on ROS, so that The model was visualized with reference to.

By writing the model explicitly, I think it became a script that can be easily applied to other tasks. (You can change the network configuration, and if you learn it first, you can move it.)

Processing procedure

So what I actually wrote the script to do is as follows.

  1. Configure CNN to read trained files
  2. Subscribe image information from the camera node
  3. Compress image information to 28 * 28 and binarize black and white so that it fits in MNIST CNN.
  4. Show the image to CNN and estimate the numbers
  5. Publish the result

Description of ROS Nodes Publisher and Subscriber Description

Code commentary

I wrote in Another article

How to move

As in README.md on GitHub $ roscore
$ python tensorflow_in_ros_mnist.py image:=/cv_camera/image_raw
$ rosrun cv_camera cv_camera_node
$ rostopic echo /result

Tensorflow, ROS, cv_bridge, camera driver, OpenCV, numpy need to be prepared in advance. If you make a mistake in the execution order, an error will occur but it will work, so don't worry.

result

When I displayed the image on my computer with paint and pointed the camera at it, it became as shown in the figure below. The image on the right is the image that came in from the camera, and the data: on the left is the number that CNN expected and published. It is correct. Screenshot from 2016-11-23 15_41_51.png

The handwritten 9 was also recognized properly.

Screenshot from 2016-11-23 15_44_07.png

However, the recognition accuracy fluctuates considerably depending on the nature of the camera. You can devise the distance between the camera and the image, trim it with numpy before putting it in CNN, and play with the binarized value.

Afterword

** I'm glad that it worked properly for the time being! !! ** ** I was nervous about the first post on both GitHub and Qiita. Especially, I was quite confused to find out and understand the license of GitHub. As a result, I put the Aphache 2.0 license by referring to this site. Is this correct? The code was pretty messy, I didn't add any comments, and I thought I should do my best.

** Overwhelming thanks on Labor Thanksgiving Day! !! ** **

References not included in the text:

Recommended Posts

Set up a node to do MNIST on ROS using Tensorflow
How to set up a Python environment using pyenv
Set up a file server on Ubuntu 20.04 using Samba
How to set up SVM using Optuna
Set up a mail server using Twisted
How to set up a jupyter notebook on ssh destination (AWS EC2)
A addictive story when using tensorflow on Android
How to set up Random forest using Optuna
How to set up Random forest using Optuna
How to set up a local development server
Set up a Python development environment on Marvericks
I tried to make a ○ ✕ game using TensorFlow
Set up a file server using samba on ZeroPi of Friendly Arm [OS installation]
Set up a simple local server on your Mac
Set up a free server on AWS in 30 minutes
[Part 1] Let's set up a Minecraft server on Linux
Set up a file server using samba on ZeroPi of Friendly Arm [Purchased Items]
Set up a file server using samba on ZeroPi of Friendly Arm [Personal import]
How to set up WSL2 on Windows 10 and create a study environment for Linux commands
I want to set up a GUI development environment with Python or Golang on Mac
SSH connection to a private server using a bastion server on EC2
Set up a Google Cloud service account key on heroku
I made a VGG16 model using TensorFlow (on the way)
[ROS] How to write Publisher and Subscriber on one node
Set up Python 3.4 on Ubuntu
Set up Polyglot on Windows
Creating a Tensorflow Sequential model with original images added to MNIST
Python: Introduction to Flask: Creating a number identification app using MNIST
How to build a Python environment using Virtualenv on Ubuntu 18.04 LTS
Set up a yum repository server on CentOS7 system and refer to it locally and from other servers.
Make a face recognizer using TensorFlow
Set up Python environment on CentOS
How to install TensorFlow on CentOS 7
Using a serial console on Ubuntu 20.04
Run TensorFlow2 on a VPS server
Creating a learning model using MNIST
A story about trying to run JavaScripthon on Windows and giving up.
Try to infer using a linear regression model on android [PyTorch Mobile]
Send mail with mailx to a dummy SMTP server set up with python.
A story about trying to connect to MySQL using Heroku and giving up
Try to set up a Vim test environment quite seriously (for Python)
Set up a local web server in 30 seconds using python 3's http.server
A story about a beginner trying hard to set up CentOS 8 (procedure memo)