Introduction

Previously, I created a system that uses the AdWords API to get keyword planner data. Recently, I heard rumors that the AdWords API will end (?), So I hurriedly decided to try the new Google Ads API.

Click here for the official website of the Google Ads API Google Ads API Beta

At the time of writing (as of August 3, 2020), it seems that Beta has not been taken yet.

Click here for the previous article Get keyword planner search volume / CPC / competitiveness with AdWords API

Until you get the keyword planner data

As with the last time, I will explain in the following flow.

Preparation for using the Google Ads API
Install Google Ads API Client Library for Python
OAuth2 authentication settings
Get search volume / CPC / competitiveness with API

1. Preparation for using the Google Ads API

The Google Ads API is free to use, but you need to apply for and approve an MCC account and a developer token. Since it was acquired in the previous article, it is omitted here. ・ Google Ads API Official (Quick Start) ・ [Click here for the previous article](https://qiita.com/zak_y/items/58e07ef042605b113f86#1-adwords-api%E3%82%92%E5%88%A9%E7%94%A8%E3%81% 99% E3% 82% 8B% E3% 81% 9F% E3% 82% 81% E3% 81% AE% E6% BA% 96% E5% 82% 99)

2. Install Google Ads API Client Library for Python

This section is not necessary if you have already installed it.

This time, I will try to get it using the API client library (Python) provided by Google. Please install it referring to here. Google Ads API Beta> Get Client Library Specifically, enter with the following command (pip)

`python`


$ pip install google-ads

The version of google-ads verified in this article is 6.0.0.
Python version is 3.7 or higher

3. OAuth2 authentication settings

Since it was set in the previous article, it is omitted here. ・ Google Ads API (quick start) ・ [Click here for the previous article](https://qiita.com/zak_y/items/58e07ef042605b113f86#3-oauth2-%E8%AA%8D%E8%A8%BC%E3%81%AE%E8%A8%AD % E5% AE% 9A)

Unlike the previous AdWords API, this time the configuration file is google-ads.yaml.
For google-ads.yaml, refer to here. 　 https://github.com/googleads/google-ads-python/blob/master/google-ads.yaml
You need to enable the Google Ads API. The Google Ads API is displayed when you narrow down your API library to ads in the Google API console.

4. Get search volume / CPC / competitiveness with API

I will briefly explain the processing flow. The official description is here. https://developers.google.com/google-ads/api/docs/keyword-planning/generate-historical-metrics You need to use KeywordPlanService.GenerateHistoricalMetrics to get the keyword planner data. In this API, as a flow

Creating keyword plans (KeywordPlan, KeywordPlanCampaigns, KeywordPlanAdGroups, KeywordPlanKeywords, and KeywordPlanNegativeKeywords.)
Get past index data (KeywordPlanService.GenerateHistoricalMetrics) Because it becomes First of all, I started the keyword plan creation department with reference to the formula,

I referred to the official sample here. I have omitted KeywordPlanNegativeKeywords because I think it is unnecessary this time. After that, make a part to acquire the past index data.
Since there is no sample to acquire the past index data (Historical Metrics), rewrite it to the past index data acquisition by referring to the sample to acquire the forecast data (Forecast Metrics).

Below is the completed sample code (Python3.7)

`get_historical_metrics.py`


#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import argparse
import sys
import uuid

from google.ads.google_ads.client import GoogleAdsClient
from google.ads.google_ads.errors import GoogleAdsException

ONE_MILLION = 1.0e6

def main(client, customer_id, target_keywords):
    #Creating a keyword plan
    try:
        resource_name = add_keyword_plan(client, customer_id, target_keywords)
    except GoogleAdsException as ex:
        print(f'Request with ID "{ex.request_id}" failed with status '
              f'"{ex.error.code().name}" and includes the following errors:')
        for error in ex.failure.errors:
            print(f'\tError with message "{error.message}".')
            if error.location:
                for field_path_element in error.location.field_path_elements:
                    print(f'\t\tOn field: {field_path_element.field_name}')
        sys.exit(1)

    #Get past index data
    keyword_plan_service = client.get_service('KeywordPlanService')
    keyword_competition_level_enum = client.get_type('KeywordPlanCompetitionLevelEnum', version='v4').KeywordPlanCompetitionLevel

    try:
        response = keyword_plan_service.generate_historical_metrics(resource_name)
    except GoogleAdsException as ex:
        print('Request with ID "{}" failed with status "%s" and includes the '
              'following errors:'.format(ex.request_id, ex.error.code().name))
        for error in ex.failure.errors:
            print('\tError with message "{}".'.format(error.message))
            if error.location:
                for field_path_element in error.location.field_path_elements:
                    print('\t\tOn field: {}'.format(field_path_element.field_name))
        sys.exit(1)

    results = []
    for i, historical in enumerate(response.metrics):
        metrics = historical.keyword_metrics
        results.append({
            'keyword':              historical.search_query.value,
            'avg_monthly_searches': metrics.avg_monthly_searches.value,
            'competition':          keyword_competition_level_enum.Name(metrics.competition),
            'competition_index':    metrics.competition_index.value,
            'low_top_of_page_bid':  metrics.low_top_of_page_bid_micros.value / ONE_MILLION,
            'high_top_of_page_bid': metrics.high_top_of_page_bid_micros.value / ONE_MILLION
        })

    print(results)


def add_keyword_plan(client, customer_id, target_keywords):
    keyword_plan = create_keyword_plan(client, customer_id)
    keyword_plan_campaign = create_keyword_plan_campaign(client, customer_id, keyword_plan)
    keyword_plan_ad_group = create_keyword_plan_ad_group(client, customer_id, keyword_plan_campaign)
    create_keyword_plan_ad_group_keywords(client, customer_id, keyword_plan_ad_group, target_keywords)

    return keyword_plan

def create_keyword_plan(client, customer_id):
    operation = client.get_type('KeywordPlanOperation', version='v4')
    keyword_plan = operation.create

    keyword_plan.name.value = (f'Keyword plan for traffic estimate {uuid.uuid4()}')

    forecast_interval = client.get_type('KeywordPlanForecastIntervalEnum', version='v4').NEXT_MONTH
    keyword_plan.forecast_period.date_interval = forecast_interval

    keyword_plan_service = client.get_service('KeywordPlanService', version='v4')
    response = keyword_plan_service.mutate_keyword_plans(customer_id, [operation])
    resource_name = response.results[0].resource_name

    print(f'Created keyword plan with resource name: {resource_name}')

    return resource_name


def create_keyword_plan_campaign(client, customer_id, keyword_plan):
    operation = client.get_type('KeywordPlanCampaignOperation', version='v4')
    keyword_plan_campaign = operation.create

    keyword_plan_campaign.name.value = f'Keyword plan campaign {uuid.uuid4()}'
    #Try setting the maximum cost per click with the default value (260 yen) when creating on the screen
    #This setting can be set for each ad group or individual keyword instead of campaign
    keyword_plan_campaign.cpc_bid_micros.value = 260000000
    keyword_plan_campaign.keyword_plan.value = keyword_plan

    keyword_plan_network = client.get_type('KeywordPlanNetworkEnum', version='v4')
    network = keyword_plan_network.GOOGLE_SEARCH
    keyword_plan_campaign.keyword_plan_network = network

    geo_target = client.get_type('KeywordPlanGeoTarget', version='v4')
    # 2392:Japan
    geo_target.geo_target_constant.value = 'geoTargetConstants/2392'
    keyword_plan_campaign.geo_targets.extend([geo_target])

    language = client.get_type('StringValue', version='v4')
    # 1005:Japanese
    language.value = 'languageConstants/1005'
    keyword_plan_campaign.language_constants.extend([language])

    keyword_plan_campaign_service = client.get_service('KeywordPlanCampaignService', version='v4')
    response = keyword_plan_campaign_service.mutate_keyword_plan_campaigns(customer_id, [operation])

    resource_name = response.results[0].resource_name

    print(f'Created keyword plan campaign with resource name: {resource_name}')

    return resource_name


def create_keyword_plan_ad_group(client, customer_id, keyword_plan_campaign):
    operation = client.get_type('KeywordPlanAdGroupOperation', version='v4')
    keyword_plan_ad_group = operation.create

    keyword_plan_ad_group.name.value = f'Keyword plan ad group {uuid.uuid4()}'
    #keyword_plan_ad_group.cpc_bid_micros.value = 2500000
    keyword_plan_ad_group.keyword_plan_campaign.value = keyword_plan_campaign

    keyword_plan_ad_group_service = client.get_service('KeywordPlanAdGroupService', version='v4')
    response = keyword_plan_ad_group_service.mutate_keyword_plan_ad_groups(customer_id, [operation])

    resource_name = response.results[0].resource_name

    print(f'Created keyword plan ad group with resource name: {resource_name}')

    return resource_name


def create_keyword_plan_ad_group_keywords(client, customer_id, plan_ad_group, target_keywords):
    match_types = client.get_type('KeywordMatchTypeEnum', version='v4')

    keywords = []
    for target_keyword in target_keywords:
        keyword_plan_ad_group_keyword = client.get_type('KeywordPlanAdGroupKeyword', version='v4')
        keyword_plan_ad_group_keyword.text.value = target_keyword
        #keyword_plan_ad_group_keyword.cpc_bid_micros.value = 2000000
        keyword_plan_ad_group_keyword.match_type = match_types.BROAD
        keyword_plan_ad_group_keyword.keyword_plan_ad_group.value = plan_ad_group
        keywords.append(keyword_plan_ad_group_keyword)

    operations = []
    for keyword in keywords:
        operation = client.get_type('KeywordPlanAdGroupKeywordOperation', version='v4')
        operation.create.CopyFrom(keyword)
        operations.append(operation)

    keyword_plan_ad_group_keyword_service = client.get_service('KeywordPlanAdGroupKeywordService', version='v4')

    response = (keyword_plan_ad_group_keyword_service.mutate_keyword_plan_ad_group_keywords(customer_id, operations))

    for result in response.results:
        print('Created keyword plan ad group keyword with resource name: '
              f'{result.resource_name}')


if __name__ == '__main__':
    # GoogleAdsClient will read the google-ads.yaml configuration file in the
    # home directory if none is specified.
    google_ads_client = GoogleAdsClient.load_from_storage()

    parser = argparse.ArgumentParser(description='Creates a keyword plan for specified customer.')
    # The following argument(s) should be provided to run the example.
    parser.add_argument('-c', '--customer_id', type=str, required=True, help='The Google Ads customer ID.')
    args = parser.parse_args()

    target_keywords = ['diet', 'Muscle training', 'Gluten free']

    main(google_ads_client, args.customer_id, target_keywords)

Here, we are acquiring keyword planner data with the keywords "diet," "muscle training," and "gluten-free."
Google-ads.yaml is assumed to be located in your home directory. To specify the location, specify the path in the argument of GoogleAdsClient.load_from_storage ().
The maximum cost per click is set to the default value (260 yen) when created on the screen.
In the API, low_top_of_page_bid (low-priced CPC) and high_top_of_page_bid (high-priced CPC) are divided by 1000000 because the unit is micro.

in conclusion

The API for Google Ads has been revamped, and it seems that it will be unified with the Google Ads API in the future. I tried to migrate the data acquisition of the keyword planner that was done with the AdWords API to the Google Ads API this time, but I was a little confused because not only the API but also the data items etc. have changed. In particular, CPC has been divided into low-priced and high-priced bands from this time (as well as the keyword planner screen), and what should be the consistency with the average CPC acquired by the AdWords API so far? Whether to simply calculate the low and high prices by averaging the two, or to use the CPC of forecast data (Forecast Metrics) instead of the historical indicator data (Historical Metrics) used this time. If I have time, I'll try to find out about that.

[PYTHON] Get Keyword Planner Search Volume / CPC / Competitiveness with Google Ads API