Ask Sota to speak back the database values
Hello!
My name is @ saike1119 and I am an engineer internship at Ozvision Co., Ltd..
I am usually a student, and I am warm in a laboratory that is strong in AI (artificial intelligence) and natural language processing.
There, I am developing an application using Sota as my graduation research.
In addition to that, AI, robots, and smart speakers are popular these days, so I would like to describe the implementation of simple processing using Sota.
The process to be implemented this time is to have Sota search the database for the content spoken and return the related content.
The image is here.
- You can build the Sota environment on the official page below, so please finish it!
http://www.vstone.co.jp/sotamanual/index.php?Java%E3%81%A7%E3%83%97%E3%83%AD%E3%82%B0%E3%83%A9%E3%83%9F%E3%83%B3%E3%82%B0%E3%82%92%E3%81%97%E3%81%A6%E3%81%BF%E3%82%8B
Implement the Sota side
Sota can be developed with either VStone Magic or Java, but this time we will implement it in Java.
The Java files to be implemented in Sota are as follows.
SearchDbSota.java
//Change the package searchdbsota to any package name
package jp.vstone.searchdbsota;
import java.awt.Color;
import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLEncoder;
import jp.vstone.RobotLib.CPlayWave;
import jp.vstone.RobotLib.CRobotMem;
import jp.vstone.RobotLib.CRobotPose;
import jp.vstone.RobotLib.CRobotUtil;
import jp.vstone.RobotLib.CSotaMotion;
import jp.vstone.sotatalk.SpeechRecog;
import jp.vstone.sotatalk.TextToSpeechSota;
public class SearchDbSota {
static final String TAG = "SearchDbSota";
//Specify the executable file
static final String getText_url = "Specify the executable file"; //Example:http://192.168.11.10/SearchFruitsSotaSample/SearchFruits.php
//VSMD and communication socket / memory access class
private static CRobotMem mem = new CRobotMem();
private static CSotaMotion motion = new CSotaMotion(mem);
//Motion control class for Sota
private static SpeechRecog recog = new SpeechRecog(motion);
private static CRobotPose pose;
public static void main(String[] args) {
CRobotUtil.Log(TAG, "Start " + TAG);
//VSMD and communication socket / memory access class
CRobotMem mem = new CRobotMem();
//Motion control class for Sota
CSotaMotion motion = new CSotaMotion(mem);
if (mem.Connect()) {
//Initialize VSMD to Sota spec
motion.InitRobot_Sota();
CRobotUtil.Log(TAG, "Rev. " + mem.FirmwareRev.get());
//Turn on the torque on the servo motor at the current position
CRobotUtil.Log(TAG, "Servo On");
motion.ServoOn();
//Initialize all axes
pose = new CRobotPose();
pose.SetPose(new Byte[] { 1, 2, 3, 4, 5, 6, 7, 8 } // id
, new Short[] { 0, -900, 0, 900, 0, 0, 0, 0 } // target pos
);
//Turn on the LED (left eye: red, right eye: red, mouth: Max, power button: red)
pose.setLED_Sota(Color.ORANGE, Color.ORANGE, 255, Color.ORANGE);
motion.play(pose, 100);
CRobotUtil.wait(100);
}
while (true) {
pose();
CPlayWave.PlayWave_wait(TextToSpeechSota.getTTSFile("What fruit do you want to know about? When you quit, say end."));
CRobotUtil.Log(TAG, "Mic Recording...");
String fruitName = recog.getResponse(15000, 3);
CRobotUtil.Log(TAG, fruitName);
if (fruitName.contains("End")) {
CRobotUtil.Log(TAG, "finish...");
break;
}
String speech_text = searchFruit(fruitName);
CPlayWave.PlayWave_wait(TextToSpeechSota.getTTSFile(speech_text));
}
}
/**
*Process to access the web server and get the text by GET
*
* @param strGetUrl
* @return
*/
public static String getStringByCallGET(String strGetUrl) {
HttpURLConnection con = null;
StringBuffer result = new StringBuffer();
try {
URL url = new URL(strGetUrl);
con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("GET");
con.connect(); //Send request by GET to URL
//HTTP response code
final int status = con.getResponseCode();
if (status == HttpURLConnection.HTTP_OK) {
//Successful communication
//Get the text
final InputStream in = con.getInputStream();
String encoding = con.getContentEncoding();
if (null == encoding) {
encoding = "UTF-8";
}
final InputStreamReader inReader = new InputStreamReader(in, encoding);
final BufferedReader bufReader = new BufferedReader(inReader);
String line = null;
//Read text line by line
while ((line = bufReader.readLine()) != null) {
result.append(line);
}
bufReader.close();
inReader.close();
in.close();
} else {
// System.out.println(status);
}
} catch (Exception e1) {
e1.printStackTrace();
} finally {
if (con != null) {
//Disconnect
con.disconnect();
}
}
// System.out.println("result=" + result.toString());
return result.toString();
}
/**
*Search for values from DB
*
* @param fruit
* @return
*/
public static String searchFruit(String fruit) {
String param = "?";
String speechSota = "";
String error = "error";
if (fruit != null && fruit.length() > 0) {
param += "fruitsName=" + encodeWord(fruit);
}
speechSota = getStringByCallGET(getText_url + param);
if (speechSota != null && speechSota.length() > 0) {
return speechSota;
}
return error;
}
/**
*Encode the received String
*
* @param word
* @return
*/
public static String encodeWord(String word) {
if (word != null) {
try {
word = URLEncoder.encode(word, "utf-8");
} catch (Exception e) {
e.printStackTrace();
}
}
return word;
}
/**
*Azatoi pose
*/
public static void pose() {
pose.SetPose(new Byte[] { 1, 2, 3, 4, 5 }, new Short[] { 0, 180, -850, -180, 850 });
motion.play(pose, 1000);
CRobotUtil.wait(100);
}
}
Place the above Java file in the hierarchy you want to execute. Example: SearchFruitsSotaSample / bin / jp / vstone / searchdbsota / SearchDbSota.java
Prepare the server side
Please prepare a simple LAMP environment that can be accessed from Sota on the server side.
The PHP file to be implemented on the server side is as follows.
db_def.php
<?php
//below$Set any value other than dbname
$dbhost = '127.0.0.1';
$dbuser = 'root';
$dbpasswd = 'root';
$dbname = 'fruits_data';
?>
SearchFruit.php
<?php
include_once('db_def.php');
$speech_txt = "";
$fruitsName = "";
$mysqli = new mysqli($dbhost, $dbuser, $dbpasswd, $dbname);
if ($mysqli->connect_error) {
echo $mysqli->connect_error;
exit();
}
if (isset($_REQUEST['fruitsName'])) {
$fruitsName = $_REQUEST['fruitsName'];
$fruitsName = mysqli_escape_string($mysqli, $fruitsName);
$speech_txt = $fruitsName;
}
$query = "select * from fruits WHERE fruit = '{$fruitsName}';";
$result = $mysqli->query($query);
if ($result->num_rows >= 1) {
while ($row = $result->fetch_assoc()) {
$speech_txt .= 'Is' . $row["description"];
}
$result->free();
} else {
$speech_txt .= 'Was not found.';
}
$mysqli->close();
echo $speech_txt;
Place the above PHP file in an executable hierarchy within the server. Example: /var/www/html/SearchFruitsSotaSample/db_def.php Example: /var/www/html/SearchFruitsSotaSample/SearchFruit.php
Prepare DB
This time we will look for the value in the test DB table. In this case, it is mysql. Log in to mysql and execute the following sql statement.
- Create a fruits_data database
create database fruits_data;
- Use fruits_data database
use fruits_data;
- Create a table of fruits
create table fruits (id int not null primary key auto_increment, fruit varchar(20), description varchar(255));
- Add values to the fruits table
insert into fruits (fruit, description) values ('Apple', 'Deciduous tree of the Rosaceae family. In spring, white or light red flowers bloom. The fruits are spherical and sweet and sour. There are many varieties, the earliest ripening in late June and the late in early December. There are also colors such as red, yellow, and light green.'), ('Mandarin orange', 'Mandarin orange科の常緑低木または高木。六月ごろ白い花が咲き、冬に黄色で丸い果実を結ぶ。食用。品種が多い。'), ('Grape', 'Grape科の落葉植物。茎がつるになって他の物にからみついて伸び、夏、球状の実が房のような形につく。実は食用、またGrape酒の原料。');
Try moving Sota
Once you've done the above, let's actually move Sota!
Send the contents implemented on the Sota side in send.xml, and if successful, log in to Sota with ssh. (Sota speaks the host name of send.xml and ssh)
Then run the SearchDbSota.java file you just created with ./java_run.sh!
- At runtime, pay attention to the permissions of java_run.sh.
result
It's done! With this, Sota will search the DB for what you said and speak back.
in conclusion
How was that? Sota is an excellent humanoid robot that is used in various scenes and is actually introduced by many companies. Here, we only use voice synthesis, voice recognition, and pause, but there are endless possibilities by incorporating image recognition and linking with various APIs! Even so, there aren't many articles about Sota, so it's a simple process, but I hope this content will be helpful to those who have read it! Also, if you have any questions or concerns, please leave a comment. Noshi
- The above source code is available on the GitHub repository. https://github.com/saike1119/SearchFruitsSotaSample
Recommended Posts