[PYTHON] [Anomaly detection] Try using the latest method of deep distance learning

Previously, using ** Deep Distance Learning ** ["Image Anomaly Detection"](https://qiita.com/shinmura0/items/06d81c72601c7578c6d3#%E3%83%99%E3%83%B3%E3% 83% 81% E3% 83% 9E% E3% 83% BC% E3% 82% AF) was performed. After that, ** the latest method of deep distance learning ** "AdaCos" appeared.

In this article, I would like to apply AdaCos to "anomaly detection" for a simple benchmark. The entire code is here

From the conclusion

By applying AdaCos to anomaly detection, we found the following:

AdaCos Although AdaCos is said to be the latest method for deep-distance learning, as of January 2020, it has been more than half a year since the paper was published. It has passed. However, as far as I know, it is still in the frame of "pure deep distance learning". It seems to be SOTA.

AdaCos is a method based on ArcFace etc., and the parameters used in ArcFace etc. are used. It is a method to decide automatically. The accuracy of ArcFace changes dramatically depending on the parameter selection, so Parameter tuning was very severe. However, with the introduction of AdaCos, it automatically Being able to determine the parameters frees you from this tuning task.

In addition, it has been confirmed that the accuracy is also improved, which greatly improves work efficiency and accuracy. This is a highly useful method. For details, refer to the following articles.

Read AdaCos: Adaptively Scaling Cosine Logits for Effectively Learning Deep Face Representations

Constraint

There are two types of AdaCos, Fixed and Dynamic. The method used in this paper is "Fixed Ada Cos".

Also, AdaCos can only be applied when the number of classes is 3 or more. This becomes a bottleneck when applying Self-supervised learning, but the consideration there is I will do it in Next article. This is not a problem because the number of classes in this experiment is 9.

Experiment

[Previous experiment](https://qiita.com/shinmura0/items/06d81c72601c7578c6d3#%E3%83%99%E3%83%B3%E3%83%81%E3%83%9E%E3%83 Perform the experiment under the same conditions as% BC% E3% 82% AF).

The code of AdaCos itself is almost the same as that of ArcheFace. However, the method of giving parameters has been changed to the AdaCos style.

The entire code is here

conditions


Fashion-MNIST
The breakdown of the data is as follows. Normal is "sneakers", abnormal is "boots".

Quantity Number of classes Remarks
Reference data for learning 8000 8 Excluding sneakers and boots
Normal data for learning 1000 1 sneakers
Test data (normal) 1000 1 sneakers
Test data (abnormal) 1000 1 boots


cifar-10
The breakdown of the data is as follows. Normal is "deer", abnormal is "horse".

Quantity Number of classes Remarks
Reference data for learning 8000 8 Excluding deer and horse
Normal data for learning 1000 1 deer
Test data (normal) 1000 1 deer
Test data (abnormal) 1000 1 Horse

Fashion-MNIST results

image.png

"L2-Softmax Loss" and "ArcFace" show the results of the previous experiment. The result of "AdaCos" is the result of this experiment.

The median "AdaCos" is now about ** the same AUC as ArcFace. ** **

After all, I am grateful that parameter tuning is no longer necessary.

Results of CIFAR-10

image.png "L2-Softmax Loss" and "ArcFace" show the results of the previous experiment. The result of "AdaCos" is the result of this experiment.

Similar to Fashin-MNIST, the median "AdaCos" is about the same AUC as ArcFace.

Summary

Recommended Posts

[Anomaly detection] Try using the latest method of deep distance learning
[Anomaly detection] Detect image distortion by deep distance learning
I tried running an object detection tutorial using the latest deep learning algorithm
Examination of Forecasting Method Using Deep Learning and Wavelet Transform-Part 2-
Othello-From the tic-tac-toe of "Implementation Deep Learning" (3)
Try deep learning of genomics with Kipoi
Try cluster analysis using the K-means method
Visualize the effects of deep learning / regularization
Othello-From the tic-tac-toe of "Implementation Deep Learning" (2)
I tried using the trained model VGG16 of the deep learning library Keras
I tried the common story of using Deep Learning to predict the Nikkei 225
I tried the common story of predicting the Nikkei 225 using deep learning (backtest)
Examination of exchange rate forecasting method using deep learning and wavelet transform
The story of doing deep learning with TPU
Try using the collections module (ChainMap) of python3
Try using Jupyter Notebook of Azure Machine Learning
Calculation of the shortest path using the Monte Carlo method
Count the number of parameters in the deep learning model
Deep learning 1 Practice of deep learning
Techniques for understanding the basis of deep learning decisions
Othello ~ From the tic-tac-toe of "Implementation Deep Learning" (4) [End]
Collection and automation of erotic images using deep learning
The copy method of pandas.DataFrame is deep copy by default
[Deep Learning from scratch] Implementation of Momentum method and AdaGrad method
Reuse the behavior of the @property method by using a descriptor [16/100]
Try to evaluate the performance of machine learning / classification model
Deep running 2 Tuning of deep learning
Try using the Twitter API
Try Deep Learning with FPGA
Basic flow of anomaly detection
Try using the Twitter API
Try using the PeeringDB 2.0 API
Deep reinforcement learning 2 Implementation of reinforcement learning
cv2.Canny (): Makes the adjustment of edge detection by the Canny method nice
Feature extraction by TF method using the result of morphological analysis
[Python] LINE notification of the latest information using Twitter automatic search
Try using Elasticsearch as the foundation of a question answering system
"Deep Learning from scratch" self-study memo (No. 13) Try using Google Colaboratory
Try to predict the triplet of boat race by ranking learning
Summary of pages useful for studying the deep learning framework Chainer
Sabayon Linux Portage Profile from 17.0 to 17.1 / Try installing the latest version of Sabayon using the Daily builds installation media
Try to predict the value of the water level gauge by machine learning using the open data of Data City Sabae
Try deep learning with TensorFlow Part 2
Reinforcement learning 8 Try using Chainer UI
Try using the Python Cmd module
Anomaly detection introduction and method summary
I tried deep learning using Theano
Graph of the history of the number of layers of deep learning and the change in accuracy
Installation method using the pip command of the Python package (library) Mac environment
Sentiment analysis of corporate word-of-mouth data of career change meetings using deep learning
Until the Deep Learning environment (TensorFlow) using GPU is prepared for Ubuntu 14.04
Basic machine learning procedure: ③ Compare and examine the selection method of features
Half-day course to try 5 famous deep learning libraries in the shortest distance (TensorFlow, Chainer, Caffe, DeepDream, style conversion)