Press [Create] when the settings are complete. A bucket for backups has been created like this. Next, set up to put the backup in the bucket

2. Create the source code that receives the Datastore entity and backs it up.

Here, https://cloud.google.com/datastore/docs/schedule-export?hl=ja I will add it with Yaml and Python while referring to it. First, create ** [app.yaml] ** in Yaml. (If it already exists, create a folder and create it in it) ← I put it in the backup folder like this

The app.yaml here is prepared for Python settings.

app.yaml

runtime: python27
api_version: 1
threadsafe: true
service: cloud-datastore-admin

libraries:
- name: webapp2
  version: "latest"

handlers:
- url: /cloud-datastore-export
  script: cloud_datastore_admin.app
  login: admin

Next, let's prepare ** [cloud_datastore_admin.py] ** in Python.

cloud_datastore_admin.py

import datetime
import httplib
import json
import logging
import webapp2

from google.appengine.api import app_identity
from google.appengine.api import urlfetch

class Export(webapp2.RequestHandler):

  def get(self):
    access_token, _ = app_identity.get_access_token(
        'https://www.googleapis.com/auth/datastore')
    app_id = app_identity.get_application_id()
    timestamp = datetime.datetime.now().strftime('%Y%m%d-%H%M%S')

    output_url_prefix = self.request.get('output_url_prefix')
    assert output_url_prefix and output_url_prefix.startswith('gs://')
    if '/' not in output_url_prefix[5:]:
      # Only a bucket name has been provided - no prefix or trailing slash
      output_url_prefix += '/' + timestamp
    else:
      output_url_prefix += timestamp

    entity_filter = {
        'kinds': self.request.get_all('kind'),
        'namespace_ids': self.request.get_all('namespace_id')
    }
    request = {
        'project_id': app_id,
        'output_url_prefix': output_url_prefix,
        'entity_filter': entity_filter
    }
    headers = {
        'Content-Type': 'application/json',
        'Authorization': 'Bearer ' + access_token
    }
    url = 'https://datastore.googleapis.com/v1/projects/%s:export' % app_id
    try:
      result = urlfetch.fetch(
          url=url,
          payload=json.dumps(request),
          method=urlfetch.POST,
          deadline=60,
          headers=headers)
      if result.status_code == httplib.OK:
        logging.info(result.content)
      elif result.status_code >= 500:
        logging.error(result.content)
      else:
        logging.warning(result.content)
      self.response.status_int = result.status_code
    except urlfetch.Error:
      logging.exception('Failed to initiate export.')
      self.response.status_int = httplib.INTERNAL_SERVER_ERROR

app = webapp2.WSGIApplication(
    [
        ('/cloud-datastore-export', Export),
    ], debug=True)

If you can input like this, let's deploy

1. Use [config set] to configure gcloud for the right project

gcloud config set project [Project name]

2. Deploy [app.yaml] to the project * Do not mistake the directory where [app.yaml] is located! !!

gcloud app deploy

If you can deploy it successfully, you are ready for automatic backup. We will automate it next.

3. Set to run automatically in cron job

See also https://cloud.google.com/datastore/docs/schedule-export?hl=ja I will proceed to

First to set up a cron job that calls the GCP schedule-datastore-exports Create ** [cron.yaml] **.

cron.yaml

cron:
- description: "Explanatory text"
  url: /cloud-datastore-export?output_url_prefix=gs://[Bucket name]
  target: cloud-datastore-admin
  timezone: Asia/Tokyo
  schedule: every １２ hours

If this is okay, let's deploy

gcloud app deploy cron.yaml

For detailed settings ~ Selection ~

Use kind if you want to export only certain types of entities when backing up

url: /cloud-datastore-export?output_url_prefix=gs://[Bucket name]&kind=[Entity type name]

It is also possible to specify multiple specific ones

url: /cloud-datastore-export?output_url_prefix=gs://[Bucket name]&kind=[Entity type name]&kind=[Entity type name]

If you want to export if the namespace exists for a particular entity

url: /cloud-datastore-export?output_url_prefix=gs://[Bucket name]&kind=[Entity type name]&kind=[Entity type name]

You can also make detailed settings by entering

For detailed settings ~ Time setting ~

Let's specify the time to execute the backup in ** [schedule] ** Currently, it is set to perform backup once every 12 hours, but if it is once a day

schedule: every 24 hours

If you want to perform a backup every day at 00:00

schedule: every day 00:00

Can be set widely For more information [https://cloud.google.com/appengine/docs/flexible/java/scheduling-jobs-with-cron-yaml?hl=ja#defining_the_cron_job_schedule](https://cloud.google.com/appengine/ docs / flexible / java / scheduling-jobs-with-cron-yaml? hl = ja #defining_the_cron_job_schedule) for details. By the way, if you want to set the reference time for backup to Japan time

timezone: Asia/Tokyo

If so, it will be done in Japan time.

4, implementation of cron job

Let's test if it actually works

If the App Engine cron job is displayed like this, cron is working well

Now let's move it with [Run Now] Now!

Wow, Elatter (´ ﾟ д ﾟ ｀) In such a case, let's see the error in [View] of the log.

See the error in the log

When I displayed the log, it was 403.

When you open it, it says "The caller does not have permission" ... In other words, "I don't have the authority to import and export."

Let's go give authority

Give authority

Open IAM in IAN and admin Edit [Project ID]@appspot.gserviceaccount.com in it Press [Add another role] Select [Cloud Datastore Import / Export Administrator] in Datastore and save!

So let's go back to running again If you press [Run Now] again ... It's done! !!

Now it will take the data every 12 hours!

Finally

How was everyone, did it work? I'm wondering if this commentary went well (´ ﾟ д ﾟ ｀)

But if you back up the data in the Datastore like this You can erase it by human error or deal with any troubles. I don't think there is any loss in doing it

It's been a long time, but thank you! !!

Reference URL

https://cloud.google.com/storage/docs/locations?_ga=2.5981299.-176719181.1568084299

https://cloud.google.com/datastore/docs/schedule-export?hl=ja

https://cloud.google.com/appengine/docs/flexible/java/scheduling-jobs-with-cron-yaml?hl=ja#defining_the_cron_job_schedule