[PYTHON] Post arXiv trend words on Slack! (Easier article search !!)

Introduction

What do you do when you want to know the latest trends from your treatise? It would be ideal if I could read all the treatises from one end, but the number of treatises is published in large numbers every day, and which one should I follow ... ** So, I tried to analyze keywords from recent papers published on arXiv and create a function to post trend words to slack **!

Completion example (Post to Slack like this!)

We are posting the 15 best trend words in order from the top! (In CS.AI category)

Creation flow

1. Get treatise information with arXiv API
2. Get trend words using TF-IDF
3. Post to slack via webhooks using IFTTT This time, in the category "CS (Computer Science)", the top words acquired by tf-idf are defined as trend words, and we will create one that displays the trend words of CS.AI!

1. Get treatise information with arXiv API

As for the treatise to be acquired this time, arXiv has decided to acquire it! There are two main reasons.

--Early information as a treatise (high trend) --Since the API is in place, it is easy to get data

A lot of data can be obtained with arXivAPI, but this time we will analyze trend words from the text of Abstract!

2. Get trend words using TF-IDF

This time, we performed TF-IDF for the CS category. By the way, do you know how many CS subcategories there are? (I hadn't counted the author, but it was more than I imagined ...)

What, 40 categories! !! If you look at these categories from the beginning, it will be darker than day (laughs). Knowing this fact, I wanted to complete this analysis even more!

(reference) CS category list

3. Post slack via webhooks using IFTTT

Using IFTTT, it is triggered by sending a request in programming, and when a request is received, it is posted to Slack. (It's convenient because it can be made quite simply!)

Complete

In these 3 steps, I was able to post the arXiv trend word to Slack! The photo below is a trend word for a different day than the completed example above.

in conclusion

It would be useful to know what the theme of many recent treatises is just by looking at Slack. It's difficult to keep reading the treatise, and if you just check this post, the hurdles will go down! I think that the author can also use it as a regular information catch-up tool. The function created this time is actually implemented in Slack of the Tokai Artificial Intelligence Study Group (TAIR)! If you are an undergraduate or graduate student and are interested, I would like to invite you, so please contact me by following + DM from twitter!

• The Tokai Artificial Intelligence Study Group (TAIR) is a community for students who are interested in AI / machine learning, mainly in the Tokai area.