[Python] I tried to judge the member image of the idol group using Keras
Article content
Image judgment processing using Keras was performed. Basically, I created it by diverting the article that I referred to, so I will write only the points that I devised as a memo.
reference
Building image recognition CNN with Python and Keras
environment
Python：3.7.4 Keras：2.3.4 tensorflow：1.13.1
Constitution
This time, I learned the image of an idol group called "// Necopla //".
Created a folder for each member under the img folder for member images, and placed images for each member.
What i did
Image acquisition
I got the image of each member from Twitter.
import json
import config
from urllib import request
from requests_oauthlib import OAuth1Session
from time import sleep
# API key setting (defined in another file config.py)
CK = config.CONSUMER_KEY
CS = config.CONSUMER_SECRET
AT = config.ACCESS_TOKEN
ATS = config.ACCESS_TOKEN_SECRET
# Authentication process
twitter = OAuth1Session(CK, CS, AT, ATS)
# Timeline acquisition endpoint
get_time_line_url = "https://api.twitter.com/1.1/statuses/user_timeline.json"
# Acquisition account
necopla_menber = ['@yukino__NECOPLA', '@yurinaNECOPLA', '@riku_NECOPLA', '@miiNECOPLA', '@kaori_NECOPLA', '@sakuraNECOPLA', '@miriNECOPLA', '@renaNECOPLA']
necopla_name = {'@yukino__NECOPLA': 'yukino',
'@yurinaNECOPLA': 'yurina',
'@riku_NECOPLA': 'riku',
'@miiNECOPLA': 'mii',
'@kaori_NECOPLA': 'kaori',
'@sakuraNECOPLA': 'sakura',
'@miriNECOPLA': 'miri',
'@renaNECOPLA': 'rena'}
# Parameter definition
params = {'q': '-filter:retweets',
'max_id': 0, # ID to start getting
'count': 200}
for menber in necopla_menber:
print(menber)
del params ['max_id'] # Clear ID to start acquisition
# Get the latest 200 tweets / Get tweets older than the ID set in params ['max_id'] from the second time onwards
index = 1
for j in range(100):
params['screen_name'] = menber
res = twitter.get(get_time_line_url, params=params)
print(params)
print(res.status_code)
if res.status_code == 200:
#API remaining count
limit = res.headers['x-rate-limit-remaining']
print("API remain: " + limit)
if limit == 1:
sleep(60*15)
n = 0
tweets = json.loads(res.text)
# Exit the loop if you can't get tweets from the account you're processing
print(len(tweets))
if len(tweets) == 0:
break
# Process by tweet
for tweet in tweets:
if 'extended_entities' in tweet:
for media in tweet['extended_entities']['media']:
url = media['media_url']
if url != '':
request.urlretrieve(url, './img/' + necopla_name[menber] + '/' + necopla_name[menber] + '_' + str(index).zfill(5) + url[-4:] )
index += 1
if len(tweets) >= 1:
print('max_id set')
params['max_id'] = tweets[-1]['id']-1
With this process, images are acquired from each member's Twitter, and the images are saved under the folder with each member's name.
Image sorting
The next thing I did was sort out the images. I got 1200 for many members and about 300 for few members. From that, I deleted announcements, food, two-shot images, etc., and manually narrowed down to only the images that are shown alone.
We further narrowed down the images of each member to 250, according to the member with the smallest number of images after narrowing down.
Image judgment
coding:utf-8
import keras
from keras.preprocessing.image import load_img, img_to_array
from keras.callbacks import LambdaCallback
from keras.utils import np_utils
from keras.models import Sequential
from keras.layers.convolutional import Conv2D, MaxPooling2D
from keras.layers.core import Dense, Dropout, Activation, Flatten
import matplotlib.pyplot as plt
import numpy as np
from sklearn.model_selection import train_test_split
from keras.models import model_from_json
from PIL import Image
import glob
necopla_name = ['yukino', 'yurina', 'riku', 'mii', 'kaori', 'sakura', 'miri', 'rena']
member_color = ['black', 'blue', 'red', 'green', 'magenta', 'aqua', 'purple', 'yellow']
image_size = 50
epochs = 100
hw = {"height":image_size, "width":image_size}
print('Load images...')
X = []
Y = []
for index, name in enumerate(necopla_name):
dir = "./image/" + name
files = glob.glob(dir + "/*.jpg ")
for i, file in enumerate(files):
image = Image.open(file)
image = image.convert("RGB")
image = image.resize((image_size, image_size))
data = np.asarray(image)
X.append(data)
Y.append(index)
X = np.array(X)
Y = np.array(Y)
X = X.astype('float32')
X = X / 255.0
# Convert the correct label format
Y = np_utils.to_categorical(Y, 8)
print('Create test data...')
# Training data and test data
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.20)
print('Build model...')
# Build CNN
model = Sequential()
model.add(Conv2D(32, (3, 3), padding='same',input_shape=X_train.shape[1:]))
model.add(Activation('relu'))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), padding='same'))
model.add(Activation('relu'))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(8))
model.add(Activation('softmax'))
# compile
model.compile(loss='categorical_crossentropy',optimizer='SGD',metrics=['accuracy'])
json_string = ''
passage_array = [[0] * epochs] * len(necopla_name)
passage_array = []
for i in range(len(necopla_name)):
passage_array.append([0] * epochs)
# Execute at the end of learning
def on_train_end(logs):
print('----- saving model...')
model.save_weights("necopla_image_model" + 'w.hdf5')
model.save("necopla_image_model.hdf5")
# Execute at the end of epoch
def on_epoch_end(epoch, logs):
print ('##' + str (epoch + 1) +'th time')
print('## yurina.jpg evaluate...')
img = load_img('yurina.jpg', target_size=(hw["height"], hw["width"]))
TEST = img_to_array(img) / 255
pred = model.predict(np.array([TEST]), batch_size=1, verbose=0)
print (">> Calculation result ↓ \ n" + str (pred))
print (">> This image is" "+ necopla_name [np.argmax (pred)] +" ".")
for i, p in enumerate(pred):
for j, pp in enumerate(p):
passage_array[j][epoch] = pp
print('## yukino.jpg evaluate...')
img = load_img('yukino.jpg', target_size=(hw["height"], hw["width"]))
TEST = img_to_array(img) / 255
pred = model.predict(np.array([TEST]), batch_size=1, verbose=0)
print (">> Calculation result ↓ \ n" + str (pred))
print (">> This image is" "+ necopla_name [np.argmax (pred)] +" ".")
print_callback = LambdaCallback(on_epoch_end=on_epoch_end,on_train_end=on_train_end)
# Training
history = model.fit(X_train, y_train, epochs=epochs, callbacks=[print_callback])
# Evaluation & evaluation result output
print(model.evaluate(X_test, y_test))
print('Output picture...')
x_datas = range(1, epochs+1)
y_datas = [ i / 10 for i in range(0, 10) ]
plt.xlim(1, epochs + 1)
plt.ylim(0, 1)
fig = plt.figure(figsize=(6, 4), dpi=72, facecolor="black", edgecolor="black", linewidth=2)
ax = plt.subplot(1, 1, 1)
for i, passage in enumerate(passage_array):
ax.plot(passage, color=member_color[i], label=necopla_name[i])
fig.savefig("passege.png ", edgecolor="black")
plt.close()
Basically, most of the codes that I referred to are diverted.
As a point I devised, I wanted to see what the learning process would be, so I added the following two points.
・ Judge the image at the end of epoch -Display the calculation result for each member in a graph
To judge the image, give the image of two members as input and confirm who is judged.
## First time
yurina.jpg evaluate...
>> Calculation result ↓
[[0.12650199 0.12684263 0.12742536 0.12854463 0.11904926 0.1264687
0.1201202 0.12504727]]
>> This image is "mii".
yukino.jpg evaluate...
>> Calculation result ↓
[[0.13068683 0.12408765 0.1275352 0.12792543 0.12050408 0.13144182
0.11644448 0.1213745 ]]
>> This image is "sakura".
Epoch 2/100
1600/1600 [==============================] - 12s 8ms/step - loss: 2.0785 - accuracy: 0.1456
## Second time
yurina.jpg evaluate...
>> Calculation result ↓
[[0.12513563 0.12735145 0.12925902 0.12473622 0.12179873 0.12717892
0.11717195 0.12736808]]
>> This image is "riku".
yukino.jpg evaluate...
>> Calculation result ↓
[[0.12863007 0.12423173 0.12936181 0.12356193 0.12369796 0.13277659
0.11367439 0.1240655 ]]
>> This image is "sakura".
(Omitted)
## 100th time
yurina.jpg evaluate...
>> Calculation result ↓
[[4.8989324e-10 8.2380754e-01 7.1863423e-07 3.2110822e-03 1.7282969e-01
1.9806185e-08 3.1989657e-05 1.1890962e-04]]
>> This image is "yurina".
yukino.jpg evaluate...
>> Calculation result ↓
[[6.1400205e-01 2.8108407e-03 4.0069546e-04 1.0979763e-03 3.1650570e-01
6.4887889e-02 8.5816224e-05 2.0912322e-04]]
>> This image is "yukino".
The first one returned the wrong result, but when it reached the 100th time, both of them came to make a correct judgment.
The graph below is a graph of the calculation results at the end of each epoch. The graph only graphs the judgment result of "yurina"'s image.
The blue of the graph is "yurina" and the pink is "kaori".
As a result I expected, I expected a graph in which the blue line converges toward 1.
Looking at this result, we can see that it is difficult to judge these two people with the current learning data.
Impressions
It's honestly interesting. For the time being, I tried to implement it while thinking about making it for studying, but I will try to learn more while reading books etc. for ideas to improve accuracy.
Recommended Posts