[PYTHON] Take an image with Pepper and display it on your tablet

Choregraphe has a box called ** Take Picture ** that allows you to save the image that Pepper is looking at as a file in the file system inside Pepper. There are various ways to handle this file, such as attaching it to an email outbox or referencing it in the box that displays an image on the tablet, but when referencing the file in each box What is important is the idea of paths within Pepper.

Here, I will explain some ideas of paths by creating an example of taking an image and displaying it on a tablet.

Image save destination path in Take Picture box

In Choregraphe's Box Library panel, select the standard Box Library> Vision> Take Picture box and you should see a description similar to the following:

take-picture-desc.png

Since it says store it in his memory in ~ / recordings / cameras., the captured images are saved in(home directory) / recordings / cameras /of the file system in Pepper. The image file name has an item called File Name in the parameter, which can be specified here.

take-picture-param.png

Image import path in tablet Show Image box

You can use the tablet box library> tablet> Show Image box to display images on your tablet, but the Image Url description for this box is as follows:

show-image-desc.png

[Pepper Tutorial (5): Using a Tablet](http://qiita.com/Atelier-Akihabara/items/87def938b6553ee9c1e9#%E7%94%BB%E5%83%8F%E3%81%AE%E5%87 As explained in detail in% BA% E5% 8A% 9B), among the files that make up the application (file structure tree shown in the [Project Contents] panel), the files under the html directory are on the tablet. It can be displayed.

project-content-html.png

In other words, if you can save what you took in the Take Picture box to your application's html directory, you can view it in the Show Image box.

Implementation example

Now, let's create an application that takes an image with the ** Take Picture box and displays it on the tablet with the Show Image box **.

In addition, when displaying the dynamically created file on Pepper's tablet, it is necessary to devise to avoid the cache, so this will be explained separately.

Try to make

  1. Place the following boxes
  1. Connect the boxes as follows

    connections.png

  2. Change the location of the Take Picture box to the application's html directory. Double-click the Take Picture box and change the code as shown below. a. Add code to the beginning of the ʻonLoad method to create an object to access the ʻALFrameManager

    def onLoad(self):
        #Additional code:from here
        self.framemanager = ALProxy("ALFrameManager")
        #Additional code:So far
        self.bIsRunning = False
        ...
    

b. Add code to get the path of the html directory at the beginning of the ʻonInput_onStart` method

```py
def onInput_onStart(self):
    #Additional code:from here
    import os
    self.recordFolder = os.path.join(
                self.framemanager.getBehaviorPath(self.behaviorId), "../html")
    #Additional code:So far
    if( self.bIsRunning ):
        return
    ...
```
  1. Show Image Box Path Problem (http://qiita.com/Atelier-Akihabara/items/87def938b6553ee9c1e9#%E8%A3%9C%E8%B6%B3show-image%E3%83%9C%E3% 83% 83% E3% 82% AF% E3% 82% B9% E3% 81% AE% E5% AE% 9F% E8% A3% 85% E3% 82% 92% E3% 81% BF% E3% 81% A6% E3% 81% BF% E3% 82% 8B) For workaround, double-click the Show Image box and add the following code to the _getAppName method

    def _getAppName(self):
        import os
        if self.frameManager:
            behaviorPath = os.path.normpath(self.frameManager.getBehaviorPath(self.behaviorId))
            appsFolderFragment = os.path.join("PackageManager", "apps")
            if not (appsFolderFragment in behaviorPath):
                self.logger.error("appsFolderFragment is not in behaviorPath")
            fragment = behaviorPath.split(appsFolderFragment, 1)[1]
            #Additional code:from here
            fragment = fragment.split('/')[1]
            #Additional code:So far
            return fragment.lstrip("\\/")
        else:
            self.logger.warning("No ALFrameManager")
    
  2. Set the save destination file name for each box. The Take Picture box must have no extension and the Show Image box must have an extension (.jpg)

    take-picture-box-param.png

    show-image-box-param.png

  3. Create a html directory to save the image and import an appropriate dummy file. Click the Add button in the project content panel, then click ** Create Directory ... ** create-html-directory.png

b. Click ** Import File ... ** to open a file selection dialog. Select a suitable dummy file. add-image.png

c. Drag the dummy file and move it into the html directory move-image.png The image file is now ready for viewing on your tablet.

Now you have a Take Picture box to store the captured data in the html directory for tablet display, and a Show Image box to display this captured data.

Operation check

When you connect to Pepper and press the [Upload to Robot and Play] button, the contents taken by Pepper's camera will be displayed on the tablet.

Avoiding tablet cache

Even if you run this application multiple times, the contents of the tablet will not be updated.

This is because the tablet web browser has cached the image. One way to avoid this is to include the runtime time in the URL query. You can work around this issue by adding code similar to the following to the ʻonInput_onStart` method of the Show Image box.

    def onInput_onStart(self):
        #Additional code:from here
        import time
        #Additional code:So far
        # We create TabletService here in order to avoid
        # problems with connections and disconnections of the tablet during the life of the application
        tabletService = self._getTabletService()
        if tabletService:
            try:
                url = self.getParameter("ImageUrl")
                if url == '':
                    self.logger.error("URL of the image is empty")
                if not url.startswith('http'):
                    url = self._getAbsoluteUrl(url)
                #Additional code:from here
                url += "?" + str(time.time())
                #Additional code:So far
                tabletService.showImage(url)

Recommended Posts

Take an image with Pepper and display it on your tablet
Take a photo with Pepper & display it on your chest display -Choregraphe 2.5.5 version-
Put your own image data in Deep Learning and play with it
Let's make an image recognition model with your own data and play!
Crop Numpy.ndarray and save it as an image
Image display and video playback on Neovim Terminal
I made an image classification model and tried to move it on mobile
Notes on HDR and RAW image processing with Python
Install selenium on Mac and try it with python
Build an image classification model explosively with Azure Custom Vision and implement it with Flask
POST the image with json and receive it with flask
Automate Chrome with Python and Selenium on your Chromebook
POST the image selected on the website with multipart / form-data and save it to Amazon S3! !!
Get an image from a web page and resize it
Build a Python environment on your Mac with Anaconda and PyCharm
Linking Python and Arduino to display IME On / Off with LED
[Linux] Register an undefined mime type and associate it with an application
Install lp_solve on Mac OS X and call it with python.
Cut out an image with python
Image segmentation with scikit-image and scikit-learn
Save and retrieve files with Pepper
Connect to VPN with your smartphone and turn off / on the server
Deploy a Python app on Google App Engine and integrate it with GitHub
I want to display an image on Jupyter Notebook using OpenCV (mac)
Build your Django app on Docker and deploy it to AWS Fargate
Scrap the published csv with Github Action and publish it on Github Pages
A python program that resizes a video and turns it into an image
Start the webcam to take a still image and save it locally
Create a Python3 environment with pyenv on Mac and display a NetworkX graph
[Rails] How to calculate latitude and longitude with high accuracy using Geocoding API and display it on Google Map
GAE --With Python, rotate the image based on the rotation information of EXIF and upload it to Cloud Storage.