[PYTHON] An example of a mechanism that returns a prediction by HTTP from the result of machine learning

Introduction

The other day, I created a program that predicts accounts using machine learning results, but there was an opinion that the response of the prediction results was abnormally slow and unusable, so I tried a little ingenuity.

Learning accounting data and predicting accounts from the contents of the description when entering journals --Qiita

Response mechanism

I decided to use HTTP in order to easily realize that the learning result is stored in the memory and the account is returned when the abstract is sent.

So, I created an HTTP server with Python, read the learning result when the HTTP server was started, and when I sent the summary with GET, I predicted and returned the account.

Build an HTTP server with Python

I referred to the following article.

Easily create an HTTP server with Python-Qiita

I'm using it as it is, but it has been corrected a little because the library called BaseHTTPServer has been changed and an error occurred in Japanese processing. By the way, it's Python3.

CallbackServer.py


#!/usr/local/bin/python
# coding: utf-8

import requests
import http.server
import socketserver
from http.server import BaseHTTPRequestHandler
from urllib.parse import urlparse, unquote

def start(port, callback):
    def handler(*args):
        CallbackServer(callback, *args)
    server = socketserver.TCPServer(('', int(port)), handler)
    server.serve_forever()

class CallbackServer(BaseHTTPRequestHandler):
    def __init__(self, callback, *args):
        self.callback = callback
        BaseHTTPRequestHandler.__init__(self, *args)

    def do_GET(self):
        parsed_path = urlparse(self.path)
        query = unquote(parsed_path.query)
        
        self.send_response(200)
        self.end_headers()
        
        result = self.callback(query)
        
        self.wfile.write(result.encode('utf-8'))
        
        return

Server startup script

It reads the learning result at startup and returns the result predicted by the GET callback.

server.py


#!/usr/local/bin/python
# coding: utf-8

import sys
import CallbackServer

import pandas as pd
import numpy as np

homedir = "/home/scripts/"
filename = "data/code.csv"

df = pd.read_csv(homedir + filename, header=None)
df.index = df.pop(0)

df_rs = df.pop(1)

from sklearn.externals import joblib

scaler = joblib.load(homedir + 'data/scaler.pkl')
clf = joblib.load(homedir + 'data/clf.pkl')
vect = joblib.load(homedir + 'data/vect.pkl')

from janome.tokenizer import Tokenizer

t = Tokenizer()

def callback_method(query):
    texts = [
        query,
    ]

    notes = []
    for note in texts:
        tokens = t.tokenize(note)
        words = ""
        for token in tokens:
            words += " " + token.surface
        notes.append(words)
    
    X = vect.transform(notes)
    
    result = clf.predict(X)
    ans = ""

    for i in range(len(texts)):
        ans = df_rs.ix[result[i]]
        
    return ans

if __name__ == '__main__':
    port = sys.argv[1]
    CallbackServer.start(port, callback_method)

Start with the following command.

python


$ chmod a+x server.py
$ ./server.py 8080 &

Operation test

Let's get the prediction result using Ruby.

test.rb


require 'net/http'
require 'uri'

puts Net::HTTP.get_print('localhost', URI.escape('/?Expressway usage fee'), 8080)

I will do it.

python


$ ruby test.rb
Travel expenses transportation

It was done (^-^)

Try incorporating it into a LINE bot.

スクリーンショット 2017-03-11 7.33.33.png

Good feeling (^-^)

Please refer to the following article for how to make a LINE bot.

Create an autoresponder BOT with LINE's Messaging API

By the way, you can make friends with this LINE bot with the following QR code.

2e099ea3-2a40-49b9-ca84-431c809cf153.png

What should I do next?

Recommended Posts

An example of a mechanism that returns a prediction by HTTP from the result of machine learning
Implementation of a model that predicts the exchange rate (dollar-yen rate) by machine learning
I made an API with Docker that returns the predicted value of the machine learning model
[Machine learning] Understand from mathematics that standardization results in an average of 0 and a standard deviation of 1.
I tried calling the prediction API of the machine learning model from WordPress
Judge the authenticity of posted articles by machine learning (Google Prediction API).
About the development contents of machine learning (Example)
Installation of TensorFlow, a machine learning library from Google
# Function that returns the character code of a string
An introduction to machine learning from a simple perceptron
Simple code that gives a score of 0.81339 in Kaggle's Titanic: Machine Learning from Disaster
Create a bot that only returns the result of morphological analysis with MeCab on Discord
A library that monitors the life and death of other machines by pinging from Python
Django returns the contents of the file as an HTTP response
The result of Java engineers learning machine learning in Python www
A concrete method of predicting horse racing by machine learning and simulating the recovery rate
Predict the presence or absence of infidelity by machine learning
The procedure from generating and saving a learning model by machine learning, making it an API server, and communicating with JSON from a browser
I made a Line bot that guesses the gender and age of a person from an image
Create an API that returns data from a model using turicreate
A memorandum of scraping & machine learning [development technique] by Python (Chapter 4)
A memorandum of scraping & machine learning [development technique] by Python (Chapter 5)
Evaluate the accuracy of the learning model by cross-validation from scikit learn
One-click data prediction for the field realized by fully automatic machine learning
Python learning memo for machine learning by Chainer until the end of Chapter 2
I tried to verify the result of A / B test by chi-square test
From a book that makes the programmer's way of thinking interesting (Python)
Learning notes from the beginning of Python 1
A simple mock server that simply embeds the HTTP request header in the body of the response and returns it.
(Updated from time to time) Summary of machine learning APIs that allow you to quickly build apps by Team AI
Learning notes from the beginning of Python 2
Judgment of igneous rock by machine learning ②
An article that just tries a little HTTP request with the curl command
Free version of DataRobot! ?? Introduction to "PyCaret", a library that automates machine learning
I tried to predict the presence or absence of snow by machine learning.
Horse Racing Prediction: If you think that the recovery rate has exceeded 100% in machine learning (LightGBM), it's a story
Machine learning memo of a fledgling engineer Part 1
A set of integers that satisfies ax + by = 1.
Classification of guitar images by machine learning Part 1
List of links that machine learning beginners are learning
Overview of machine learning techniques learned from scikit-learn
Analysis of shared space usage by machine learning
[Translation] scikit-learn 0.18 Tutorial Introduction of machine learning by scikit-learn
Machine learning memo of a fledgling engineer Part 2
Reasonable price estimation of Mercari by machine learning
Classification of guitar images by machine learning Part 2
Get a glimpse of machine learning in Python
Extract only complete from the result of Trinity
A story about data analysis by machine learning
An easy way to measure the processing speed of a disk recognized by Linux
A python script that gets the number of jobs for a specified condition from indeed.com
Predicting the goal time of a full marathon with machine learning-③: Visualizing data with Python-
[CodeIQ] I wrote the probability distribution of dice (from CodeIQ math course for machine learning [probability distribution])
[Ansible] Example of playbook that adds a character string to the first line of the file
What is a rational decision that maximizes the chances of encountering an "ideal home"?
[Go] Create a tool that returns the Pokemon race value received from the standard input
A memo that detects and returns an image acquired from a webcam with Django's OpenCV
An article that gives you a little understanding of the rigid sphere collision algorithm
I made a twitter app that identifies and saves the image of a specific character on the twitter timeline by pytorch transfer learning
The end of programming beginners (my pattern) who wrote the code motivated by the results that bring programming, not from a technical perspective