I tried to automate everything including Google OAuth with two-step verification

I personally like the implementation around authentication, but I didn't implement Google OAuth, so I took this opportunity to implement it.

The OAuth function is provided by various vendors such as FaceBook, Yahoo, and Twitter, but for Google, parameters are dynamically generated by JavaScript each time, so the authentication part uses chromedriver / Selenium (scraping library) and the browser. We decided to implement base automatic authentication.

Also, when automatic login is troublesome, Google requires two-step verification in addition to the user name and password, so it is quite worth automating. .. When you try to log in automatically, Google will send an verification code for two-step verification to the reset email address by email, but this time I skipped this email to a receive-only email server using AWS SES and that After storing the email in S3, the authentication code written in the email body is automatically read and the value is automatically entered.

The SES settings are renewed now, but just in case, the reference URL is listed below. https://docs.aws.amazon.com/ja_jp/ses/latest/DeveloperGuide/receiving-email-setting-up.html

The sequence is as follows, but I tried to automate all the following flow (browser startup> authentication request> authentication> access token issuance).

スクリーンショット 2019-10-18 3.43.28.png

environment:  Java: 1.8  Selenium: 3.12.0  ChromeDriver: 77.0.3865.40  Chrome: 77.0.3865.90

Since Maven was used as the build tool this time, the contents of pom.xml below.

pom.xml


<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>googleOAuth</groupId>
  <artifactId>googleOAuth</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <name>googleOAuth</name>
  <dependencies>
  <dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-java</artifactId>
    <version>3.12.0</version>
</dependency>
<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk-s3</artifactId>
    <version>1.11.651</version>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-core</artifactId>
    <version>2.9.7</version>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.9.7</version>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-annotations</artifactId>
    <version>2.9.7</version>
</dependency>
<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk-sts</artifactId>
    <version>1.9.6</version>
</dependency>
</dependencies>
  <build>
  	<plugins>
  		<plugin>
  			<groupId>org.apache.maven.plugins</groupId>
  			<artifactId>maven-shade-plugin</artifactId>
  			<version>3.2.1</version>
  		</plugin>
  		      <plugin>
        <artifactId>maven-assembly-plugin</artifactId>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>single</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
            <archive>
              <manifest>
                <mainClass>googleOAuth.GoogleOAuth2</mainClass>
              </manifest>
            </archive>
          <descriptorRefs>
            <descriptorRef>jar-with-dependencies</descriptorRef>
          </descriptorRefs>
        </configuration>
      </plugin>
  	</plugins>
  </build>
</project>

GoogleOAuth.java


package google;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.net.HttpURLConnection;
import java.net.URL;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.util.Base64;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;

import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.DeleteObjectRequest;
import com.amazonaws.services.s3.model.GetObjectRequest;
import com.amazonaws.services.s3.model.ListObjectsRequest;
import com.amazonaws.services.s3.model.ObjectListing;
import com.amazonaws.services.s3.model.S3Object;
import com.amazonaws.services.s3.model.S3ObjectSummary;
import com.amazonaws.util.IOUtils;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;

public class GoogleOAuth {

	private final String clientId;
	private final String clientSecret;
	private final String redirectUri;
	private final String scope;
	private final String responseType;
	private final String tokenUrl;
	private final String driverpath;
	private String emailAddress;
	private String password;
	private String recEmailAddress;
	private String lastName;
	private String firstName;
	private String BucketName;
	private String LocalPath;

	public GoogleOAuth() {
		clientId = "*****"; //Client ID
		clientSecret = "*****"; //Client secret
		redirectUri = "*****"; //Redirect URI
		scope = "https://www.googleapis.com/auth/userinfo.profile";
		responseType = "code";
		tokenUrl = "https://accounts.google.com/o/oauth2/token";
		BucketName = "*****"; //S3 bucket name
		LocalPath = "*****"; //Email storage path
		driverpath = "*****"; //chromedriver path
		emailAddress = "*****"; //log-in e-mail address
		password = "*****"; //password
		recEmailAddress = "*****"; //SES incoming mail server address
		lastName = "*****"; //LastName
		firstName = "*****"; //FirstName
	}

	public static void main(String[] args) throws IOException {
		GoogleOAuth google = new GoogleOAuth();
		String aZcode = google.getAzCode();
		JsonNode access_token = google.getAccessToken(aZcode);
		System.out.println(access_token);

	}

	public String getAzCode() throws IOException {
		final String PATH = this.driverpath;
		System.setProperty("webdriver.chrome.driver", PATH);
        WebDriver driver = new ChromeDriver();
        final String URL = "https://accounts.google.com/o/oauth2/auth?client_id=" + this.clientId + "&redirect_uri=" +
        		this.redirectUri + "&scope=" + this.scope + "&response_type=" + this.responseType;

        driver.get(URL);


        try {
        	driver.findElement(By.xpath("//*[@id=\"identifierId\"]")).sendKeys(this.emailAddress);;
	        driver.findElement(By.xpath("//*[@id=\"yDmH0d\"]")).click();
	        Thread.sleep(5000);

	        driver.findElement(By.xpath("//*[@id=\"recoveryIdentifierId\"]")).sendKeys(this.recEmailAddress);;
	        driver.findElement(By.xpath("//*[@id=\"queryPhoneNext\"]/span/span")).click();
	        Thread.sleep(5000);

	        driver.findElement(By.xpath("//*[@id=\"lastName\"]")).sendKeys(this.lastName);;
	        driver.findElement(By.xpath("//*[@id=\"firstName\"]")).sendKeys(this.firstName);;
	        driver.findElement(By.xpath("//*[@id=\"collectNameNext\"]/span/span")).click();
	        Thread.sleep(5000);

        	driver.findElement(By.xpath("//*[@id=\"idvpreregisteredemailNext\"]/span/span")).click();
        	Thread.sleep(5000);

        	this.downloadFile();
        	Thread.sleep(3000);
        	String token = this.getToken();

	        driver.findElement(By.xpath("//*[@id=\"idvPinId\"]")).sendKeys(token);;
	        driver.findElement(By.xpath("//*[@id=\"idvpreregisteredemailNext\"]/span")).click();
	        Thread.sleep(3000);

	        driver.findElement(By.xpath("//*[@id=\"view_container\"]/div/div/div[2]/div/div/div/form/span/section/div/div/div/div/ul/li[3]/div/div/div[2]")).click();
	        Thread.sleep(3000);

	        driver.findElement(By.xpath("//*[@id=\"identifierId\"]")).sendKeys(this.emailAddress);;
	        driver.findElement(By.xpath("//*[@id=\"identifierNext\"]/span/span")).click();
	        Thread.sleep(3000);

	        driver.findElement(By.xpath("//*[@id=\"password\"]/div[1]/div/div[1]/input")).sendKeys(this.password);;
	        driver.findElement(By.xpath("//*[@id=\"passwordNext\"]/span/span")).click();
	        Thread.sleep(3000);
	        } catch (InterruptedException e) {
	        	System.out.println("Caught InterruptedException: " + e);
	        }

        	String currentUrl = driver.getCurrentUrl();
        	//System.out.println(currentUrl);
    		int biginIdx = currentUrl.indexOf("code=");
    		int endIdx = currentUrl.indexOf("&scope");
    		String aZcode = currentUrl.substring(biginIdx+5, endIdx);

    		driver.quit();

    		return aZcode;
		}


	public JsonNode getAccessToken(String code) throws IOException {
		URL url = new URL(this.tokenUrl);
		StringBuffer result = new StringBuffer();
		HttpURLConnection  conn = null;
		conn = (HttpURLConnection) url.openConnection();
		conn.setDoOutput(true);
		conn.setRequestMethod("POST");
		conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
		conn.setUseCaches(false);
		String params = new String("code=" + code + "&client_id=" + this.clientId + "&client_secret=" + this.clientSecret +
				"&redirect_uri=" + this.redirectUri + "&grant_type=authorization_code");
        OutputStreamWriter out = new OutputStreamWriter(conn.getOutputStream());
        out.write(params);
        out.close();
        conn.connect();

        final int status = conn.getResponseCode();
        if (status == 200) {
            final InputStream input = conn.getInputStream();
            String encoding = conn.getContentEncoding();
            if(null == encoding){
                encoding = "UTF-8";
            }
            final InputStreamReader inReader = new InputStreamReader(input, encoding);
            final BufferedReader bufReader = new BufferedReader(inReader);
            String line = null;
            while((line = bufReader.readLine()) != null) {
                result.append(line);
            }
            bufReader.close();
            inReader.close();
            input.close();
        } else {
            System.out.println(status);
        }

        conn.disconnect();

        ObjectMapper mapper = new ObjectMapper();
        JsonNode root = mapper.readTree(result.toString());

        return root;
	}

	public void downloadFile() throws IOException {
		AmazonS3 s3 = AmazonS3ClientBuilder.standard()
                .withRegion("us-east-1")
                .build();

		ObjectListing objectListing = s3.listObjects(new ListObjectsRequest()
		        .withBucketName(this.BucketName));

		for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) {
		    System.out.println(objectSummary.getKey());
		    String objectKey = objectSummary.getKey();
		    S3Object object = s3.getObject(new GetObjectRequest(this.BucketName, objectKey));
		    try {
				FileOutputStream fos = new FileOutputStream(new File(this.LocalPath));
				IOUtils.copy(object.getObjectContent(), fos);
				fos.close();
				s3.deleteObject(new DeleteObjectRequest(this.BucketName, objectKey));
			} catch (FileNotFoundException e) {
				e.printStackTrace();
			}
		}
	}

	public String getToken() throws IOException {
		File file = new File(this.LocalPath);
		FileReader fr = new FileReader(file);
		BufferedReader br = new BufferedReader(fr);
		String json_str;
		String data = "";
		int cnt = 0;

		while ((json_str = br.readLine()) != null) {
			cnt += 1;
			if ((cnt>= 73)&&(128 >= cnt)){
			data += json_str;
			}
		}

        Charset charset = StandardCharsets.UTF_8;
        byte[] bytes = Base64.getDecoder().decode(data.getBytes());
        String decode_str = new String(bytes, charset);
		int biginIdx = decode_str.indexOf("<strong style=\"text-align: center; font-size: 24px; font-weight: bold;\">");
		int endIdx = decode_str.indexOf("</strong>");
		String token = decode_str.substring(biginIdx+72, endIdx);

		br.close();
		return token;
	}
}

After executing the above code and completing authentication / authorization, you can finally get the access token as follows.

{"access_token":"ya29.ImCiBwhJcCZ-JIW9ZlGDe2ysCBkNzRpG9mgr_-ocM_A32Dh1bzJbHAHOT_iKi6GNAVovWDLgyKclHJaP1uqTODQ61LJomWAUzhWSnMsd4ddGKTeIUfOeSQocVbUzikxcWjU","expires_in":3596,"scope":"https://www.googleapis.com/auth/userinfo.profile","token_type":"Bearer","id_token":"eyJhbGciOiJSUzI1NiIsImtpZCI6IjNkYjNlZDZiOTU3NGVlM2ZjZDlmMTQ5ZTU5ZmYwZWVmNGY5MzIxNTMiLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJhY2NvdW50cy5nb29nbGUuY29tIiwiYXpwIjoiNjUwMDQxMDk4ODAzLTBtMG81Y2IxZ2ZraW43cDJwN2ZuaWNhYnUxaWdvdnVnLmFwcHMuZ29vZ2xldXNlcmNvbnRlbnQuY29tIiwiYXVkIjoiNjUwMDQxMDk4ODAzLTBtMG81Y2IxZ2ZraW43cDJwN2ZuaWNhYnUxaWdvdnVnLmFwcHMuZ29vZ2xldXNlcmNvbnRlbnQuY29tIiwic3ViIjoiMTE1OTcwNTg3NjcwMjI4MTk2Njc4IiwiYXRfaGFzaCI6IjhGZmJSa3BYQ1VoWUduX0VTN2tENnciLCJpYXQiOjE1NzEzMzE4ODYsImV4cCI6MTU3MTMzNTQ4Nn0.oXK4EhivivgJqhO6cjJ7ZyeTBPW9IrTOvM_GGeDcNJ6XXTOVtRd8nv3pRT1SHDSTTgkco6tAI8ZN-dWDD-LVhqkhGTjM-USzR2UbXhMsd4z6WRYGMPfzuLCkGRGZwp0Gqd7ctITlIWJr0N8xLPEGNSHx_IPZtabOaP4ME9YN0-XQg-x2tE32LdRI4ttvW7D1lSeYPKdzWf_12i9zoLnblc2MiGTupww91PlQSaLGoprFPVutI_57c1IraBsCmMGuW0Nke4vps7YoFhaaxDhB4XuJCpyIvpADHuOydq3RPY5WCV8YIzFXsFCa1f-Z7QyJNuZgcFf7WLWcjXQZkD4i8g"}

Recommended Posts

I tried to automate everything including Google OAuth with two-step verification
I tried to automate sushi making with python
I tried to automate the watering of the planter with Raspberry Pi
I tried to implement Autoencoder with TensorFlow
I tried to get started with Hy
I tried to implement CVAE with PyTorch
I tried to solve TSP with QAOA
I tried to predict next year with AI
I tried to detect Mario with pytorch + yolov3
I tried to implement reading Dataset with PyTorch
I tried to use lightGBM, xgboost with Boruta
I tried to learn logical operations with TF Learn
I tried to move GAN (mnist) with keras
I tried to save the data with discord
I tried to detect motion quickly with OpenCV
I tried to integrate with Keras in TFv1.1
I tried to get CloudWatch data with Python
I tried to output LLVM IR with Python
I tried to detect an object with M2Det!
I tried to predict Titanic survival with PyCaret
I tried "Receipt OCR" with Google Vision API
I tried to operate Linux with Discord Bot
I tried to study DP with Fibonacci sequence
I tried to start Jupyter with Amazon lightsail
I tried to judge Tsundere with Naive Bayes
I tried simple image processing with Google Colaboratory.
I tried to automate internal operations with Docker, Python and Twitter API + bonus
I tried to learn the sin function with chainer
I tried to move machine learning (ObjectDetection) with TouchDesigner
I tried to create a table only with Django
I tried to extract features with SIFT of OpenCV
I tried to move Faster R-CNN quickly with pytorch
I tried to implement and learn DCGAN with PyTorch
I tried to implement Minesweeper on terminal with python
I tried to get started with blender python script_Part 01
I tried to touch the CSV file with Python
I tried to draw a route map with Python
I tried to solve the soma cube with python
I tried to automate the article update of Livedoor blog with Python and selenium.
I tried to automatically read and save with VOICEROID2
I tried to get started with blender python script_Part 02
I tried to generate ObjectId (primary key) with pymongo
I tried to implement an artificial perceptron with python
I tried to build ML Pipeline with Cloud Composer
I tried to implement time series prediction with GBDT
I tried to debug.
I tried to uncover our darkness with Chatwork API
I tried to automatically generate a password with Python3
[Introduction to Pytorch] I tried categorizing Cifar10 with VGG16 ♬
I tried to paste
I tried to solve the problem with Python Vol.1
I tried to analyze J League data with Python
I tried to implement Grad-CAM with keras and tensorflow
I tried to make an OCR application with PySimpleGUI
I tried to implement SSD with PyTorch now (Dataset)
I tried to interpolate Mask R-CNN with Optical Flow
I tried to step through Bayesian optimization. (With examples)
I tried to access Google Spread Sheets using Python
I tried to find an alternating series with tensorflow
[Introduction to AWS] I tried playing with voice-text conversion ♪
I tried to solve AOJ's number theory with Python