[PYTHON] Survey on the use of machine learning in real services

I understand how machine learning works, but how is it used in actual services? I wonder if the learning environment is buying a GPU on its own. Or is the cloud the mainstream?

I think that everyone has once had questions about how to use it in practice and how to develop it. I took the plunge and took a questionnaire when I was in charge of the course for those who were wondering and thinking about using it in practice.

The target was people who are using machine learning in practice, and we were able to get 32 answers (Thanks to those who answered!). Please note that there may be a sampling bias because the answers were solicited on Twitter / Facebook. Below, I would like to publish the results with my findings.

Q1. Please tell me the usage of machine learning.

image

This was a question to ask, "What purpose do you use machine learning in the first place?" It was a convincing number because there are many cases where it is used to improve recommendations and customer support, but it was surprising that the improvement in manufacturing operations has also increased considerably. I had the impression that there are many (true) solutions sold around here, but they haven't been put to practical use yet, so I was personally surprised that they would grow to this point.

Q2. Please tell me how to provide machine learning to customers.

image

This was a question to confirm that there are various forms of provision, which can be said to be machine learning. It can be assumed that the integration into the company's service is the first, and there are many uses for "using it to improve the company's service". Personally, I thought that there were many forms like machine learning consulting, but it didn't grow much.

Q3. Please tell us about the field of machine learning you are using.

image

This was a question to ask the field of technology being used. Images and natural language are still two giant towers, but what I wanted to confirm here was how much so-called "generational" technology such as GAN and Neural Conversational Model is used in actual services. There are not many examples of the results yet, and it turned out that the utilization of the recognition system is the main focus. What was surprising was that there were about 5 reinforcement learning cases. Bandit type of advertisement or robot control ... I don't know because I didn't take that much, but I would like to use it in practice.

Q4. Please tell us about the environment used for learning machine learning.

image

Where is everyone doing this in machine learning learning? It was a question to confirm that. The result is overwhelmingly our own environment! After all, if you use it seriously in the service, it seems that it is better to build the environment firmly. However, since each company is focusing on the GPU environment, the ratio may change in the future.

Q5. Please tell me where to get the data for machine learning.

image

This was a question to confirm the big problem in machine learning, "How to prepare data?". Not only in-house data, but also open data and scraping are doing quite well. Of course, we start from the place where there is no data at first, so knowing what kind of public data is available, and scraping technology (although I can not do it openly) is important for collecting data. ..

Q6. Please tell us about the painful events related to machine learning.

image

This was a question about how sympathetic I was about my personal concerns. After all it is a problem of "data" and "human resources". This seems to be a headache problem everywhere. It seems that there are many problems such as "the accuracy does not come out" that came out as the runner-up, and "it seems that you can do anything and it is difficult to explain". New machine learning frameworks and methods are appearing one after another in this field, but in that sense, it was surprising that the changes in frameworks and methods were too fast. In practice, there are probably a few fixed ones, and it is unlikely that you will follow the others (in the personal observation range, it seems that TensorFlow / Chainer is almost solidified in the DNN system (domestic). only)).

Also, since it is difficult to guarantee the results of machine learning, I wondered if there were some problems with the order amount and contract, but it did not grow much here. In the first place, there are many people who use it for their own services, so it may not be growing that much here either.

Q7. Other opinions

I didn't fill in a lot of free-form questions, but the problem of how to hire and train, and the inconsistency (noise) contained in the data entered by people were mentioned as headaches. Was there. In addition, there is a statement that there is a problem about the model that has produced results in natural language, "Does it work in Japanese?", So I think this is certainly a difficult point.

How was that. We hope that this result will help you to imagine "machine learning in the field" more concretely. We are doing this! If you have a voice like that, please leave a comment.

Recommended Posts

Survey on the use of machine learning in real services
About testing in the implementation of machine learning models
The result of Java engineers learning machine learning in Python www
How to use machine learning for work? 01_ Understand the purpose of machine learning
Full disclosure of methods used in machine learning
Use the latest version of PyCharm on Ubuntu
Read the output of subprocess.Popen in real time
About the development contents of machine learning (Example)
Summary of evaluation functions used in machine learning
Get a glimpse of machine learning in Python
Install the machine learning library TensorFlow on fedora23
Count the number of parameters in the deep learning model
About data preprocessing of systems that use machine learning
Impressions of taking the Udacity Machine Learning Engineer Nano-degree
Predict the gender of Twitter users with machine learning
Difference in results depending on the argument of multiprocess.Process
Let's use the open data of "Mamebus" in Python
Summary of the basic flow of machine learning with Python
Record of the first machine learning challenge with Keras
[Python] Use the Face API of Microsoft Cognitive Services
[Machine learning] "Abnormality detection and change detection" Let's draw the figure of Chapter 1 in Python.
A note on the default behavior of collate_fn in PyTorch
Try to evaluate the performance of machine learning / regression model
Machine learning in Delemas (practice)
Basics of Machine Learning (Notes)
I installed the automatic machine learning library auto-sklearn on centos7
Predict the presence or absence of infidelity by machine learning
Analyzing data on the number of corona patients in Japan
Count the number of characters in the text on the clipboard on mac
Try to evaluate the performance of machine learning / classification model
How to increase the number of machine learning dataset images
Used in machine learning EDA
[Machine learning] I tried to summarize the theory of Adaboost
Importance of machine learning datasets
Notes on how to use marshmallow in the schema library
Find the eigenvalues of a real symmetric matrix in Python
Verification of the spread of hoaxes in the "State of Emergency Declaration on April 1"
A story stuck with the installation of the machine learning library JAX
Wrap (part of) the AtCoder Library in Cython for use in Python
Use the vector learned by word2vec in the Embedding layer of LSTM
[Machine learning] Check the performance of the classifier with handwritten character data
Perform morphological analysis in the machine learning environment launched by GCE
Use PyCaret to predict the price of pre-owned apartments in Tokyo!
How to use Jupyter on the front end of supercomputer ITO
Find the rank of a matrix in the XOR world (rank of a matrix on F2)
People memorize learned knowledge in the brain, how to memorize learned knowledge in machine learning
Get the number of readers of a treatise on Mendeley in Python
Significance of machine learning and mini-batch learning
The story of participating in AtCoder
Automate routine tasks in machine learning
Machine learning ③ Summary of decision tree
Classification and regression in machine learning
The story of the "hole" in the file
Survey for practical use of BlockChain
Machine learning in Delemas (data acquisition)
Python: Preprocessing in Machine Learning: Overview
Preprocessing in machine learning 2 Data acquisition
The meaning of ".object" in Django
Random seed research in machine learning
Sakura Use Python on the Internet
Preprocessing in machine learning 4 Data conversion