[PYTHON] Try drawing a social graph using Twitter API v2

Thing you want to do

1. I want to check if I can draw a graph ← here ――I want to make a social graph of relationships on Twitter ――I want to adjust the closeness of distance and the thickness of lines according to the strength of the relationship. ――I want to observe the process of cluster formation

What is Twitter API

API that can get data on Twitter

For more information,

-About Twitter API -Official document

For the API acquisition method, refer to Summary of procedures from Twitter API registration (account application method) to approval * Information may be out of date

Social graph

Social graph - Wikipedia The social graph is a graph that represents social relations between entities. In short, it is a model or representation of a social network, where the word graph has been taken from graph theory.

A graph showing the social relationship between objects (= nodes). Express the friendship with a line (= edge).

Main subject

Preparation

API acquisition is omitted

Library import

import json
import random
import pandas as pd
import matplotlib.pyplot as plt
from requests_oauthlib import OAuth1Session
import emoji
import networkx as nx

Store token in variable

CK = '***************************'
CS = '***************************'
AT = '***************************'
ATS ='***************************'

Function definition

def get_twitter_session():
    return OAuth1Session(CK, CS, AT, ATS)

Decide what you want to draw

user_list = ['himemoriluna','tokoyamitowa', 'amanekanatach','tsunomakiwatame','kiryucoco']

Create a simple dictionary for drawing

user_id_list = []
user_name_list = []
for user_screenname_t in user_list:
    ENDPOINT = 'https://api.twitter.com/1.1/users/show.json'
    SEARCH_WORD_SCREENNAME = user_screenname # twitter_ID

    twitter = get_twitter_session()
    params = {'screen_name': SEARCH_WORD_SCREENNAME}
    req = twitter.get(ENDPOINT, params=params)
    req_json = json.loads(req.text)
    user_id_list += [req_json['id']]
    user_name_list += [req_json['name']]


target_id_dict = {}
for i in range(0,5):
    target_id_dict[user_id_list[i]] = user_list[i]

# {1200396798281445376: 'himemoriluna',
#  1200357161747939328: 'tokoyamitowa',
#  1200396304360206337: 'amanekanatach',
#  1200397643479805957: 'tsunomakiwatame',
#  1200397238788247552: 'kiryucoco'}

target_name_dict = {}
for i in range(0,5):
    target_name_dict[user_list[i]] = user_name_list[i]

# {'himemoriluna': 'Himemori Luna',
#  'tokoyamitowa': 'Everlasting Towa',
#  'amanekanatach': 'Amane Kanata',
#  'tsunomakiwatame': 'For square winding',
#  'kiryucoco': 'Kiryu Coco'}

Data acquisition

ʻGet the user that the user specified by user_list` is following and keep the result in dictionary type.

follow_ids_dict = {}
for user_id in user_id_list:
    ENDPOINT = 'https://api.twitter.com/1.1/friends/ids.json'
    SEARCH_WORD_ID = user_id

    twitter = get_twitter_session()
    params = {'screen_name': SEARCH_WORD_SCREENNAME}
    req = twitter.get(ENDPOINT, params=params)

    #Get up to 5000 follow-up information
    follow_ids = json.loads(req.text)
    follow_ids_dict[user_id] = follow_ids['ids']

Create a dictionary to hold user names

follow_name_dict = {}
follow_screenname_dict = {}
for user_id in follow_ids_dict.keys():
    follow_name_list = []
    follow_screenname_list = []
    for user_id_nt in follow_ids_dict[user_id]:
        ENDPOINT = endpoint['user']
        SEARCH_WORD_ID = user_id_nt

        twitter = get_twitter_session()
        params = {'user_id': SEARCH_WORD_ID}
        req = twitter.get(ENDPOINT, params=params)

        req_json = json.loads(req.text)

        follow_name_list += [req_json['name']]
        follow_screenname_list += [req_json["screen_name"]]

    follow_name_dict[target_name_dict[user_id]] = follow_name_list
    follow_screenname_dict[target_name_dict[user_id]] = follow_screenname_list

DataFrame creation

This time, we visualize a graph in which the branches are oriented (directed branches) and the branches are weighted.

(It is troublesome if the emoji is displayed by the way, so delete it)

def remove_emoji(str_):
    return ''.join(c for c in str_ if c not in emoji.UNICODE_EMOJI)

#Select one of the following
# use_dict = follow_ids_dict #Visualize with internal ID
# use_dict = follow_screenname_dict #User ID(@xxxx)Visualize with
use_dict = follow_name_dict #Visualize by user name

Row_From = [] #start point
Row_To = [] #end point

for i in use_dict.keys():
    for j in follow_name_dict[i]:
        Row_From += [remove_emoji(target_name_dict[i])]
        Row_To += [remove_emoji(j)]

result = pd.DataFrame(
    data={'From':Row_From, 'To':Row_To},
    columns=['From', 'To'])

result.head()

Since the purpose of this time is a visualization experiment, the weight of the branch is decided appropriately with a random number.

def return_randint(num):
    return random.randrange(1, 10000)/1000000000

result['WC'] = 0
result['WC'] = result['WC'].apply(return_randint)
network = result

Visualization

The default font_family can only display alphanumeric characters. Specify because you want to display Japanese.

network_np = network.values
G = nx.DiGraph()
G.add_weighted_edges_from(network_np)

weighted_edges = []
for i in range(len(network)):
    weighted_edges += [list(network.iloc[i])]

G_practice = nx.DiGraph()

G_practice.add_weighted_edges_from(weighted_edges)
pos=nx.spring_layout(G)

fig = plt.figure(figsize=(40, 40), dpi=100,facecolor='w', linewidth=0, edgecolor='w')
nx.draw_networkx(G_practice,pos,font_size=16,font_family='IPAexGothic')
fig.show()

↑ I was able to draw something like this ('ω')

Future outlook

This time, the graph of social relationships was realized based on the follow-up information on Tiwtter. In the future, we will search for variables that can describe relationships more accurately, and based on that, we will adjust the weights of branches and cut out.

reference

• Social graph - Wikipedia
• Twitter Standard API Endpoints -[Easy network analysis with Python networkx](https://deepblue-ts.co.jp/python/python%E3%81%AEnetworkx%E3%81%A7%E7%B0%A1%E5%8D% 98% E3% 81% AB% E3% 83% 8D% E3% 83% 83% E3% 83% 88% E3% 83% AF% E3% 83% BC% E3% 82% AF% E5% 88% 86% E6% 9E% 90 /) -Resolving garbled Japanese characters in Python library matplotlib and NetworkX