Purpose

There seems to be ** tf_explain **. It seems that ** GradCAM ** is included as a module there. Looking at the github that uses it, ** GradCAM ** could be executed with only 22 lines (with blank lines). I think it's wonderful I tried to move it.

22 lines of code

You can find this code on github below. https://github.com/sicara/tf-explain/blob/master/examples/core/grad_cam.py

The code below is all, so

python grad_cam_py.py

It just works.

`grad_cam.py`


import tensorflow as tf

from tf_explain.core.grad_cam import GradCAM

IMAGE_PATH = "./cat.jpg "

if __name__ == "__main__":
    model = tf.keras.applications.vgg16.VGG16(weights="imagenet", include_top=True)

    img = tf.keras.preprocessing.image.load_img(IMAGE_PATH, target_size=(224, 224))
    img = tf.keras.preprocessing.image.img_to_array(img)

    model.summary()
    data = ([img], None)

    tabby_cat_class_index = 281
    explainer = GradCAM()
    # Compute GradCAM on VGG16
    grid = explainer.explain(
        data, model, class_index=tabby_cat_class_index, layer_name="block5_conv3"
    )
    explainer.save(grid, ".", "grad_cam.png ")

Execution result

The image of the cat ("tabby cat" in it) is taken from wikipedia. https://en.wikipedia.org/wiki/Tabby_cat This kind of cat seems to be characterized by having the letter M around the eyes in the middle of the face ~~ ** the letter M on the "forehead" *. ( "There is a letter M so as to surround the eyes in the middle of the face" (wrong), but it is not invisible to M ...)

sample1

sample2

Comments on the results

The result of GradCAM seems to be right. (However, The purpose of this article is that tf_explain is easy to use, so No comment on the good or bad of GradCAM itself. Just in case, ** If you put too much feeling on the GradCAM display, you will feel that the fortune-teller's fortune-telling will be a hit, so I think you should look at it as cold as possible, which is a complete digression. **)

environment

It's Windows 10. There is no GPU. tensorflow 2.0.0 tf-explain 0.2.1

The above tf-explain is probably not included, so install it using pip or other normal method. ** I think that 2.1.x is the latest version of tensorflow now (March 11, 2020), but I got a terrible error. ** ** ** I don't know how to cure it, I have a complete idea! So, when I set it to 2.0.0, it worked. ** **

This section is a section to actually see the nature of CNN rather than tf_explain (a digression from this article?)

In the case of this example Is it the range of about 60x60 pixels that captures the features all at once? ⇒ In the last image, it feels like you are reacting individually to individual areas. Also, it may be more accurate to display the image and the point of interest even in the non-superimposed mode (is there?). (Because it is difficult to understand if it is affected by the brightness of the original image.) ⇒⇒ CNN will be vulnerable to "more images (up images)". ⇒⇒⇒ I'm impressed when a model that pursues the overall configuration and shows the feeling of grasping with GradCAM appears. .. .. (Whether it can be expressed by GradCAM is another story ... Apart from the fact that it can not be expressed by GradCAM, I think it would be great if such a model came out. Capsule system etc. If there is a model that pops in. ）

Related to the above section (hence, aside). CNN is "weak to leaning (weak to zooming up)".

We've also added diagrams to help you better understand the focus of GradCAM. I also added the probability.

[[('n02123045', 'tabby', 0.40362296),
  ('n02124075', 'Egyptian_cat', 0.34350035),
  ('n02123159', 'tiger_cat', 0.1646882),
  ('n02747177', 'ashcan', 0.022324266),
  ('n02127052', 'lynx', 0.009675921),
  ('n03223299', 'doormat', 0.008641529),
  ('n02123394', 'Persian_cat', 0.00528028),
  ('n02909870', 'bucket', 0.0034843169),
  ('n04040759', 'radiator', 0.0028082374),
  ('n03958227', 'plastic_bag', 0.002630277),
  ('n04265275', 'space_heater', 0.002209679),
  ('n04493381', 'tub', 0.0015652123),
  ('n04049303', 'rain_barrel', 0.001464855),
  ('n04553703', 'washbasin', 0.0014180988),
  ('n04589890', 'window_screen', 0.0012623073),
  ('n03887697', 'paper_towel', 0.0012330494),
  ('n04522168', 'vase', 0.0012083148),
  ('n02123597', 'Siamese_cat', 0.0010707852),
  ('n03950228', 'pitcher', 0.0010204213),
  ('n03255030', 'dumbbell', 0.00096622825)]]

[[('n03958227', 'plastic_bag', 0.23590706),
  ('n04209133', 'shower_cap', 0.117050014),
  ('n02124075', 'Egyptian_cat', 0.068308175),
  ('n01968897', 'chambered_nautilus', 0.052455623),
  ('n03825788', 'nipple', 0.042889122),
  ('n02123597', 'Siamese_cat', 0.040462725),
  ('n02120079', 'Arctic_fox', 0.02897999),
  ('n03868863', 'oxygen_mask', 0.018255476),
  ('n04370456', 'sweatshirt', 0.018049669),
  ('n02123045', 'tabby', 0.017420992),
  ('n04525038', 'velvet', 0.01728542),
  ('n02123394', 'Persian_cat', 0.0140852835),
  ('n03534580', 'hoopskirt', 0.012244948),
  ('n03724870', 'mask', 0.0106809465),
  ('n03045698', 'cloak', 0.007704126),
  ('n02120505', 'grey_fox', 0.0072637224),
  ('n02326432', 'hare', 0.006367313),
  ('n04127249', 'safety_pin', 0.006034479),
  ('n03887697', 'paper_towel', 0.0056772656),
  ('n04033995', 'quilt', 0.0056173983)]]

** tabby is about 10th. .. .. ** ** ⇒ It may be said that you should do it in various sizes, but I think it's better to be a little more conscious of the composition, CNN. ** I made a sample by misunderstanding the location of the letter M, which is a feature of tabby **, so the letter M, which is important, is outside the area of the image. (Actually, it's better to start over ... I think it's probably not good even if I try again, so I'll postpone it for a moment.)

I referred to the following to get the rate. https://medium.com/@gkadusumilli/image-recognition-using-pre-trained-xception-model-in-5-steps-96ac858f4206

Indicates the modified source.

import tensorflow as tf

import numpy as np ##
import pprint ##

from tf_explain.core.grad_cam import GradCAM

from tensorflow.keras.applications.vgg16 import decode_predictions ##

IMAGE_PATH = "./cat2_2nd_224.jpg "

if __name__ == "__main__":
    model = tf.keras.applications.vgg16.VGG16(weights="imagenet", include_top=True)

    img = tf.keras.preprocessing.image.load_img(IMAGE_PATH, target_size=(224, 224))
    img = tf.keras.preprocessing.image.img_to_array(img)
    
    predictions=model.predict(np.array([img])) ##
    pprint.pprint(decode_predictions(predictions,top=20))##

    model.summary()
    data = ([img], None)

    tabby_cat_class_index = 281
    explainer = GradCAM()
    # Compute GradCAM on VGG16
    grid = explainer.explain(
        data, model, class_index=tabby_cat_class_index, layer_name="block5_conv3"
    )
    explainer.save(grid, ".", "grad_cam_cat2_2nd_224.png ")
    
    grid = explainer.explain( ##
        data, model, class_index=tabby_cat_class_index, layer_name="block5_conv3", image_weight=0.01 ##
    ) ##
    explainer.save(grid, ".", "grad_cam_cat2_2nd_224ex.png ") ##

Summary

The environment is advancing every day (** tf_explain **). If you have any comments, please let us know.

[PYTHON] GradCAM with 22 lines of code. tf_explain may be easy to use, I recommend it!