Get 1000 posts in Python order from all Slack channels and put them together in a txt file

Why did you decide to do it

I wanted data for morphological analysis. There was no choice.

procedure

First, get the Token of slack. I think this article will be helpful. https://qiita.com/ykhirao/items/0d6b9f4a0cc626884dbb

code

Since it is organized on github, there is no problem if you see here for the basics, pull it, install the necessary library and execute it. A cohesive output.txt will appear. https://github.com/hiwatee/get-txt-slack-python

import os
import requests
from os.path import join, dirname
from dotenv import load_dotenv


def main():
    # .Read environment variables from env
    dotenv_path = join(dirname(__file__), '.env')
    load_dotenv(dotenv_path)
    token = os.environ.get("TOKEN")
    host = os.environ.get("HOST")
    #Get / format channel list
    url = host + 'channels.list?token=' + token + '&exclude_archived=true'
    r = requests.get(url)
    data = r.json()
    channels = [{'id': channel['id'], 'name': channel['name']}
                for channel in data['channels']]
    for channel in channels:
        #Get the top 1000 from each channel
        url = host + 'channels.history?token=' + \
            token + '&channel=' + channel['id'] + '&count=1000'
        r = requests.get(url)
        data = r.json()
        #Write to file
        with open(dirname(__file__) + 'output.txt', 'a') as f:
            for message in data['messages']:
                print(message['text'], file=f)


if __name__ == '__main__':
    main()

More detailed explanation

Python-dotenv is required for actual use. The intention of using it is to reuse environment variables in .env to make it easier to manage permissions and to prevent my Token from leaking to git.

python-dotenv pip install python-dotenv It can be installed with.

Please rewrite the sample writing method as .env.txt.

By the way

The extracted txt file contains reactions, bot messages, and reactions and cannot be used for morphological analysis as it is. Please clean and use by yourself.