[PYTHON] tweepy's api.search isn't perfect! ??

Introduction

I made LINE Bot to automatically get the latest images of actress Mayu Matsuoka, who is my livelihood.

If you like, please add it with a QR code. I am pleased.

See the following article or GitHub for detailed code. ** [I made a LINE Bot that sends recommended images every day on time] ** https://qiita.com/soma_sekimoto/items/4c01d0ab890024d6f87c

GitHub https://github.com/SomaSekimoto/MayuDelivery

The image acquisition source of this image transmission bot is twitter. (Because the information is fast, that is, the latest image comes out quickly.)

I implemented it using tweepy and was happy to this day. I thought it was a tweepy god.

Oh, by the way, as you may have noticed, I don't write much about this technically.

I used api.search with tweepy, but something is wrong. .. ..

I came up with it recently Enter the search word described in the code in the actual twitter search window and it is exactly the same

Search part code

q = f"#Mayu Matsuoka OR Mayu Matsuoka-'Similar to Mayu Matsuoka' filter:media exclude:retweets min_faves:10 since:{yesterday}"

tweets = tweepy.Cursor( api.search, q=q, tweet_mode='extended', include_entities=True).items(20)

When I read the document, it was written properly.

https://github.com/tweepy/tweepy/blob/master/docs/api.rst

Please note that Twitter's search service and, by extension, the Search API is not meant to be an exhaustive source of Tweets. Not all Tweets will be indexed or made available via the search interface.

In short

** "Not all tweets are retrieved by the Search method !!" **

I interpreted it as that.

Was it the specification of tweepy in the first place? .. .. ..

But I want to know which tweets I can get and which I can't. I want to know.

I was able to get the tweet I wanted to take with result_type = "mixed".

I was able to get the tweets I wanted to get.

Before correction

q = f"#Mayu Matsuoka OR Mayu Matsuoka-'Similar to Mayu Matsuoka' filter:media exclude:retweets min_faves:10 since:{yesterday}"

tweets = tweepy.Cursor( api.search, q=q, tweet_mode='extended', include_entities=True).items(20)

Revised

q = f"#Mayu Matsuoka OR Mayu Matsuoka-'Similar to Mayu Matsuoka' filter:media exclude:retweets min_faves:10 since:{yesterday}"

tweets = tweepy.Cursor( api.search, q=q, tweet_mode='extended', result_type="mixed", include_entities=True).items(20)

There are three types of values that can be set with result_type. "recent": Search for the latest tweets in chronological order "popular": Search for popular tweets (it's unclear what criteria are used to determine popularity) "mixed": A mixture of the above. It has become.

Further investigation revealed that the default was set to "recent".

See article below

https://note.com/katomaru0510/n/n8797618a68ce https://qiita.com/mima_ita/items/ba59a18440790b12d97e

in conclusion

It was completely my own way.

Of course, I knew the existence of result_type, but I misunderstood that the default would be to get all of them regardless of the type of tweet.

I solved it this time, but I haven't been able to understand tweepy's api.search fundamentally, so I will continue to deepen my understanding while using tweepy.