[Go language] Collect and save Vtuber images using Twitter API

Introduction

I want to save a picture of my favorite person ...

Currently, painters are posting various images on Twitter.

I often look at the picture of Vtuber, but I thought it was good, I want to save it somewhere.

I noticed that the pictures of Vtuber that I often see are often posted by hashtags on Twitter, and I wondered if I could do something about it, so I made a "collection tool" while investigating.

Get images in bulk using Twitter API

By the way, I would like to take up my favorite "Usada Pekora" here.

Required process

・ Acquisition and authentication of Twitter API ・ Set the information to be collected (tweets containing the character string "#pekoratto", 100 acquisitions, etc.) ・ Creating a file to store the saved image, etc. ・ Acquisition of image URL ・ Save as jpg file

Let's look at each one.

Acquisition and authentication of Twitter API

Please jump to the following site and get the Twitter API.

>>> Get Twitter API

I will omit the detailed method.

I think it would be good to define it at the beginning of the source.


const (
	consumerKey       = ""
	consumerSecret    = ""
	accessToken       = ""
	accessTokenSecret = ""
)

Set the information to collect

here,

How many tweets to get Does it include retweets? Do not include tweets without images

Set a query such as.


    v := url.Values{}
    v.Set("count", "100")
	query := "Pekora and OR#Pekora and exclude:retweets filter:images"
	searchResult, _ := api.GetSearch(query, v)

Here, we use "count", which means the number of tweets, and get 100 tweets. Next, assume that the searched character string is "Pekora" or "#Pekora". Only tweets with images attached, not including retweets.

Failure point

I didn't know that I could set it in this part, so I got the tweet content and judged whether the image content existed from it.

Creating a file to store the saved image

If you save it normally, it will be messed up, so create a folder to save the image. When creating a json file, it is better to create another folder.

When I mkdir in the following source code, I use the number 0777, which describes the file permissions. 0: Cannot read / write / execute 1: Only executeable 2: Write only 3: Can be written and executed 4: Read only 5: Readable and executable 6: Can be read and written 7: Read, execute, write So why are there three digits? Didn't you think? This is from top to bottom -File and directory owner (creator) privileges -Permissions of other users in the same group as the created user ・ Privileges of other users Click here for reference articles

//Set the current directory with an absolute path
currentDir, err := filepath.Abs(".")
//Chdir is a command as an image$Feeling cd
    err = os.Chdir(currentDir)
	if err != nil {
		panic(err)
	}
//Check the file
	_, err = os.Stat("pekorato")
	if err != nil {
//Now create a folder with Mkdir
		err = os.Mkdir("pekorato", 0777)//0777 is explained in detail above
		if err != nil {
			panic(err)
		}
	}
//Move directories and save images together here
	err = os.Chdir(currentDir + "/pekorato")
	if err != nil {
		panic(err)
	}

Get image URL

Getting the URL of an image is also easy. If you omit the details, you can collect it with one line of code below.

Conversely, you can get different information by rewriting after tweet. For example, you can use tweet.User.ProfileImageURL to get the URL of the image for that user's profile.


mediaRawURL := tweet.ExtendedEntities.Media[0].Media_url_https

Save as jpg file

I think it's more like a copy than a save. I'm doing http.Get at the beginning, but this requires a little knowledge of http. What will be returned if I send the get method? If you don't know it, you should check it out.


        response, err := http.Get(mediaRawURL)
			if err != nil {
			panic(err)
		}
		defer response.Body.Close()

		//Generate jpg file
		pekoPic, err := os.Create(fmt.Sprintf("pekora%d.jpg ", i))
		if err != nil {
			panic(err)
		}
		defer pekoPic.Close()

		//Copy to downloaded pekoPic
		io.Copy(pekoPic, response.Body)

Whole source code


package main

import (
	"fmt"
	"io"
	"net/http"
	"net/url"
	"os"
	"path/filepath"

	"github.com/ChimeraCoder/anaconda"
)

//Please enter here by yourself.
const (
	consumerKey       = ""
	consumerSecret    = ""
	accessToken       = ""
	accessTokenSecret = ""
)

func main() {
	//Twitter authentication
	anaconda.SetConsumerKey(consumerKey)
	anaconda.SetConsumerSecret(consumerSecret)

	//Personal authentication
	api := anaconda.NewTwitterApi(accessToken, accessTokenSecret)

	/*************
Search query settings
	**************/
	v := url.Values{}
	v.Set("count", "100")
	query := "Pekora and OR#Pekora and exclude:retweets filter:images"
	searchResult, _ := api.GetSearch(query, v)

	/**********************************************
Creating a folder to save images and moving directories
	**********************************************/

	currentDir, err := filepath.Abs(".")
	if err != nil {
		panic(err)
	}

	_, err = os.Stat("pekorato")
	if err != nil {
		err = os.Mkdir("pekorato", 0777)
		if err != nil {
			panic(err)
		}
	}

	err = os.Chdir(currentDir + "/pekorato")
	if err != nil {
		panic(err)
	}

	/********************************
Analyze tweets and generate images
	********************************/

	for i, tweet := range searchResult.Statuses {
		mediaRawURL := tweet.ExtendedEntities.Media[0].Media_url_https

		response, err := http.Get(mediaRawURL)
		if err != nil {
			panic(err)
		}
		defer response.Body.Close()

		pekoPic, err := os.Create(fmt.Sprintf("pekora%d.jpg ", i))
		if err != nil {
			panic(err)
		}
		defer pekoPic.Close()

		io.Copy(pekoPic, response.Body)
	}
}

Summary

This time, I used the Twitter API to get the image. Next, I thought I'd try to make something like getting the tweeted image of a specific person. Those who saw it were all right.

Recommended Posts

[Go language] Collect and save Vtuber images using Twitter API
I tried using Twitter api and Line api
Tweet regularly with the Go language Twitter API
Automatically save images and videos hit by Twitter search to iPhone using Pythonista3
Collect images using icrawler
Search and save images of Chino Kafu from Twitter
I tried to automatically collect erotic images from Twitter using GCP's Cloud Vision API
Collect large numbers of images using Bing's image search API
Try using the Twitter API
Try using the Twitter API
Collect tweets using tweepy in Python and save them in MongoDB
[Go language] Use OpenWeatherMap and Twitter API to regularly tweet weather information from Raspberry Pi
Collect product information and process data using Rakuten product search API [Python]
Send messages and images using LineNotify
I made go language for api and minimum configuration of react for front
Buy and sell cryptocurrencies using Zaif API
Tweet using the Twitter API in Python
Try using Dropbox API v2 with Go
Generate API spec using GO / Gin gin-swagger
[MacOS] Installation of Go (Go language) using goenv
Collect data using scrapy and populate mongoDB
[Golang] About Go language Producer and Consumer