Hello. I'm sorry. Today, it is published as one of Cognitive Services in Microsoft Azure Emotion API I would like to play with .microsoft.com/en-us/services/cognitive-services/emotion/).

Agenda --What is Emotion API? --Let's make a trial app --Impressions used --Summary

What is Emotion API?

Overview

It is one of the analysis APIs provided by Microsoft. It is offered as one of the Cognitive Services on Microsoft Azure. When you send a photo to the Emotion API,

Detect a human face from a photo
Quantify what kind of emotion the detected human facial expression is

Will do the work and return it. If you just want to know the emotions of the person in the picture, you can use it without any knowledge of analysis.

Let's pick up the specifications of the Emotion API below.

――The number of faces that can be detected is up to 64 per image. ――It looks like a photo of the graduation ceremony is okay! ――Eight kinds of emotions can be acquired --anger --contempt (contempt) --disgust --fear --happiness --neutral --sadness --surprise --There are 4 types of extensions available. However, for GIF, only the first frame image is used for judgment. - JPEG - PNG - BMP - GIF --Image capacity is up to 4MB --Image size is from 36 x 36 to 4096 x 4096 --The returned JSON objects are sorted by face area

price

Currently only available in the US West region. Two types are available of restricted type (Basic plan) and unlimited plan (Standard). ). The Basic plan can detect only the specified face rectangle. If you want to detect other faces (a little profile etc.), you need to subscribe to the Standard plan (* 1). Both plans are available for up to 10 lances / sec. Each can be used at the following prices per 1000 transactions.

--Basic plan --¥ 10.20 per 1000 transactions --Standard plan --¥ 25.50 per 1000 transactions

Let's make a trial system

Trial system overview

The created functions are as follows.

Upload a photo
If the uploaded image is larger than the specified size of Emotion API, resize it to 4096 x 4096.
Run the Emotion API 4.3 Accumulate the results obtained in 3, display the emotion name with the largest numerical value & graph the accumulated emotion index

Actually, it will be such two screens. Select ... ↓ Display! ↓

By the way, this photo was taken at the time of the 1st anniversary of PyLadies Tokyo in October 2015. It's a party photo, so the happiness value is by far the highest.

Technology used

Emotion API The service that was the cornerstone of this creation. As mentioned above. API documentation and [API test console](https://dev.projectoxford.ai/docs/services/5639d931ca73072154c1ce89/operations/ It is encouraging that both 563b31ea778daf121cc3a5fa / console) are available. In the trial system, when facial expression data of multiple people is acquired, each emotion value is simply SUMed.

Tornado It is a WEB framework of Python and an asynchronous communication library (this time it is simply used as a WEB framework). Made by Facebook and lightweight. It works with both Python 2 and 3 (I created it with Python 3 this time). Since the template engine is included, when you want to pass a value from Python to the screen, you can pass it and display it by using the specified notation. If you want to write Python code on html, you can call it. Personally, I'm happy that the demo system is available along with the source code (https://github.com/tornadoweb/tornado). And this demo system is quite abundant. It is easy to refer to because it covers all of Websocket and OAuth. But I don't think it's very popular. I like it though.

Pillow In the fork project of the Python Imaging Library (PIL), it is literally Python image processing library.

pip install pillow

It's very convenient because you can use it just by yourself. And I personally like it because it's easy to understand for the library that plays with images. This time I used only the image resizing function, but it is a versatile library that can also invert colors, combine images, write text, and so on.

Chart.js Graph drawing JS. I like it because it has cute movements, it is simple to use, and there are so many types of graphs that can be drawn. You can switch the display on / off by clicking the label.

When all are displayed ↓

When happiness and neutral are hidden ↓

Impressions using the Emotion API

It's very simple to use, just follow the steps below (see here for more information).

--Register for use from the Azure portal --Set the key issued at the time of registration in the POST request header --Send the URL of the photo!

It's easy and easy to use.

After trying various things, I felt that it was easy to get a relatively high number for the happiness of a smile (* 2). When you're laughing, it's easy to understand that you're laughing, even if you look at it normally. Somehow I fell in love with it. On the contrary, for items such as fear and contempt that seem difficult to judge, it was difficult for the judgment result to be a remarkable result such as happiness. I think I often saw fear and contempt being about the same number (* 3).

When the emotion data of multiple people could be acquired, this time we simply added the numerical values for each emotion to make the judgment result, but for example, if you want to acquire "emotion judgment for the entire photo" like this time, this time As a beginner in analysis, I couldn't decide whether to just add up each emotion like this, or whether to weight or reanalyze from the obtained results. It may be interesting to weight it by the area of the face in the picture.

In addition to the still image version of the Emotion API used this time, there is also a video version of the Emotion API, so I would like to try this next time.

Summary

――Emotion API is simple and easy to use. ――How to use the numerical values calculated from the Emotion API (usage ideas) is the key to fun app development.

That was all.

1: As of October 25, 2016
2: It is just an individual impression. It was not quantified.
3: The photo I used may not have been very good.

Let's judge emotions using Emotion API in Python