Create a list in Python with all followers on twitter

Why did you decide to do it

The timeline of twitter HOME is as follows.

--Promotion (advertisement) comes out --Not necessarily in chronological order --Tweets from a few hours ago come to the top with a time lag

The second and third are basically the same thing, but they are arranged arbitrarily on the twitter side anyway. Personally, this is unpleasant, and it was stressful that the promotion was frequent and noisy.

"If it's a list, there will be no promotion and it will be in chronological order!" I was told and tried to create a list with all the followers There was no way to add users to the list at once on twitter, so I summarized this article.

Advance preparation

Since it is processed using twitter API, it is necessary to get Token. I will omit it in this article because it takes a lot of time, but I am familiar with the following article. https://www.itti.jp/web-direction/how-to-apply-for-twitter-api/

Please register the app on twitter and publish the following.

• CONSUMER_KEY
• CONSUMER_SECRET
• ACCESS_TOKEN
• ACCESS_TOKEN_SECRET

processing

The source code is stored here https://github.com/wonohe/twitter_create_follow_list

How to use

Install tweepy

I am using tweepy for API operation of twitter. Install below

bash

pip install -r requirements.txt

config modification

Change config.json.sample to config.json and register each key.

Processing execution

Execute the following at the root of the cloned folder

bash

python main.py

Commentary

The code looks like this:

main.py

import datetime
import tweepy
import time
import json

# config
config = json.load(open('config.json', 'r'))

SCREEN_NAME = config['SCREEN_NAME']
CONSUMER_KEY = config['CONSUMER_KEY']
CONSUMER_SECRET = config['CONSUMER_SECRET']
ACCESS_TOKEN = config['ACCESS_TOKEN']
ACCESS_TOKEN_SECRET = config['ACCESS_TOKEN_SECRET']

auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_TOKEN, ACCESS_TOKEN_SECRET)
api = tweepy.API(auth, wait_on_rate_limit=True, timeout=180, retry_delay=10, retry_count=5)

#create list
now = datetime.datetime.now()
list = api.create_list(name=f'all-{now.strftime("%Y%m%d%H%M%S")}',mode='public')
print(f'list.id:{list.id}')
print(f'list.name:{list.name}')

#add following users to list
cursor = -1
error_usr = []
while cursor != 0:
    itr = tweepy.Cursor(api.friends_ids, id=SCREEN_NAME, cursor=cursor).pages()
    try:
        for friends_ids in itr.next():
            try:
                user = api.get_user(friends_ids)
                print(user.screen_name)
                #Supports communication error users. Erase if successful
                error_usr.append(user.screen_name)
                api.add_list_member(screen_name=user.screen_name, list_id=list.id, owner_screen_name=SCREEN_NAME)
                error_usr.pop()
            except tweepy.error.TweepError as e:
                print(e.reason)
    except ConnectionError as e:
        print(e)
    cursor = itr.next_cursor

print('Corresponding to communication error')
#Process until there are no errors
while error_usr:
    for name in error_usr:
        try:
            print(name)
            api.add_list_member(screen_name=name, list_id=list.id, owner_screen_name=SCREEN_NAME)
            error_usr.remove(name)
        except tweepy.error.TweepError as e:
            print(e.reason)

Basically, I just use the tweepy API to create a list → register as a user, but There were some points to be aware of.

Settings when calling API with tweepy

There is a limit to the number of times you can call the twitter API, so set wait_on_rate_limit to True in tweepy. The default is False.

api = tweepy.API(auth, wait_on_rate_limit=True, timeout=180, retry_delay=10, retry_count=5)

In addition, since an error may occur even if the number of times is not limited, the timeout and the number of retries are also set longer.

Response to unknown communication errors

I have 144 followers, which I don't think is that many, but there were always some users who failed in the process. The number of users who get an error is different, about 3 to 5 users for 144 people. The error is as follows,

error

Failed to send request: HTTPSConnectionPool(host='api.twitter.com', port=443): Max retries exceeded with url: /1.1/users/show.json?id=******* (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x10b1db410>: Failed to establish a new connection: [Errno 60] Operation timed out'))

I'm not sure just because I couldn't make a connection. The next user who failed was successful, so it doesn't seem to be a limit. I didn't know the exact cause even if I googled, so I tried to retry only for the error. Here, put only the error amount in the array,

main.py

#Supports communication error users. Erase if successful
error_usr.append(user.screen_name)
api.add_list_member(screen_name=user.screen_name, list_id=list.id, owner_screen_name=SCREEN_NAME)
error_usr.pop()

Registration processing is performed until there are no errors in this part.

main.py

while error_usr:
    for name in error_usr:
        try:
            print(name)
            api.add_list_member(screen_name=name, list_id=list.id, owner_screen_name=SCREEN_NAME)
            error_usr.remove(name)
        except tweepy.error.TweepError as e:
            print(e.reason)

Summary

The process itself is not that complicated, but it was a little confusing that the twitter API sometimes caused mysterious behavior (sudden error). Even though I didn't change the code, I was suddenly told "Don't allow that operation", so don't be too crazy when testing.

Other

I referred to this article. Thank you very much. https://note.com/j1ntube/n/n5a417a68178b