[PYTHON] (Note) A story about creating a question answering system using Spring Boot and machine learning (SVM)
Introduction
It is a story of making a question answering system using Spring Boot and machine learning (SVM). I made it by myself a long time ago (around mid-2018) and left it alone. It's a waste, so it's an article organized for myself (memorial). Please note that this is an article I made for myself, so it may be difficult to understand, information, and technology may be out of date: bow: Also, I hope it will be helpful for those who want to make some kind of web application by themselves. If you don't know about the question answering system, @ VegaSato's article is very detailed. I hope you can read this. $ \ tiny {* I used it as a reference. Thank you} $: bow_tone1: Create a Japanese question answering system using Bluemix's Watson API Simply put
It is a system in which the system searches for and answers the best answer when the user inputs a question.
Looking at the article related to the question answering system on Qiita, the core part uses a proper API such as IBM Watoson, but
Please note that I am a self-made SVM (Support Vector Machine) and the implementation is bad or not very accurate: no_good:
The real thing is like the GIF below.
App Link
When I asked how to be happy, I managed to get the answer ** "Do your best in any situation" **: clap:
Goal (purpose?) Of this theme
- Minimum: Outline the procedure when creating an application and the clogged part.
- Full: Incorporate new technology into the current app to make an article.
It's going to be long, so this time it's up to the environment. .. ..
Target audience:
- People who are interested in machine learning but want to see what it can be applied to
- People who have somehow made or have knowledge of web applications in some language
- People who can create an environment by referring to the net and books, and can read the code somehow
App overview
The screen and usage are very simple.
If you enter a question as shown in the capture below, multiple answers will be displayed at the bottom of the screen.
$ \ tiny {* Don't worry about the example question} $: sob:
Also, if you click the answer link, the screen will change to the link destination in another tab.
Processing flow
The flow from the user inputting the question text to displaying the answer on the screen on the system side is as follows. The same is true for humans, but first we classify what kind of question (what, when, how) to the questioned content. Perform machine learning in advance to classify, and prepare a trained model (question type). $ \ tiny {* I'd like to write an article about machine learning methods later when I have time. .. } $: sweat_smile: After classifying the questions, get the sentences from the internet (scraping). Classify the acquired sentences. Again, machine learning is done in advance to do the classification, Prepare a trained model (answer type). Then, the answer type and the acquired text are temporarily stored in the answer candidate list. $ \ tiny {* I feel that there is a waste in processing. .. } $: sob: Loop through the answer candidate list, and if the question type matches the answer type, store it in the answer list. Finally, the answer list is displayed on the screen.
Environment * I think that it will work even if it is not the following Ver, but please note that it is old: no_good_tone2: </ sup>
- Development environment
- OS : Ubuntu 18.04.4 LTS
- Language: Java1.8, HTML, CSS, JavaScript, Python 3.7
- Framework: Spring Boot 1.5, Bootstrap
- Source code control: Github
- Build tool: Gradle 3.4.1
- IDE：Spring Tool Suite(STS)
- Server environment
- Deploy: heroku,
- Ap server: Tomcat (included)
- Deployment method: Push to Github and it will be automatically deployed to Heroku
the next deployment
This time, I wrote an article about the environment. It's a shabby article, but I've used it for almost a day off: sob: Also, when you have time, you can brush up, sauce, etc. little by little. I would like to write an article: sob: