[GCP] How to publish Cloud Storage signed URLs (temporary URLs) in Python

Introduction

It summarizes how to publish Cloud Storage signed URLs (URLs that are valid for a certain period of time) in Python.

procedure

1. Creating a service account and key

Open IAM and Administration-> Service Accounts-> Service Accounts and select Create Service Account.

Then enter the service account name and select Create. image.png

Then from Select Role, select Storage-> Storage Object Viewer and select Continue. image.png

Then select Create Key, select JSON, and then select Create. Then the JSON file will be downloaded to your local PC, then select [Finish]. image.png

2. Creating a program

The Google Cloud documentation was only fragmented, so you'll have to combine the information yourself. First, check the python program on the following site.

V4 signing process with Cloud Storage tools (Open Language with English) https://cloud.google.com/storage/docs/access-control/signing-urls-with-helpers

storage_generate_signed_url_v4.py


from google.cloud import storage
import datetime


def generate_download_signed_url_v4(bucket_name, blob_name):
    """Generates a v4 signed URL for downloading a blob.

    Note that this method requires a service account key file. You can not use
    this if you are using Application Default Credentials from Google Compute
    Engine or from the Google Cloud SDK.
    """
    # bucket_name = 'your-bucket-name'
    # blob_name = 'your-object-name'

    storage_client = storage.Client()
    bucket = storage_client.bucket(bucket_name)
    blob = bucket.blob(blob_name)

    url = blob.generate_signed_url(
        version="v4",
        # This URL is valid for 15 minutes
        expiration=datetime.timedelta(minutes=15),
        # Allow GET requests using this URL.
        method="GET",
    )

    print("Generated GET signed URL:")
    print(url)
    print("You can use this URL with any user agent, for example:")
    print("curl '{}'".format(url))
    return url

When I try to run the above program (storage_generate_signed_url_v4.py), I get an error saying you need a private key. The program says "Note that this method requires a service account key file.", So you know that you need a service account key file. The service account key file is the JSON file created in the previous step, but there is no description on how to specify it.

So next, check the following python program.

Authentication using the service account key file https://cloud.google.com/bigquery/docs/authentication/service-account-file?hl=ja

from google.cloud import bigquery
from google.oauth2 import service_account

# TODO(developer): Set key_path to the path to the service account key
#                  file.
# key_path = "path/to/service_account.json"

credentials = service_account.Credentials.from_service_account_file(
    key_path,
    scopes=["https://www.googleapis.com/auth/cloud-platform"],
)

client = bigquery.Client(
    credentials=credentials,
    project=credentials.project_id,
)

The above program is an authentication method using a service account key file for BigQuery, but let's remake it for Cloud Storage. The changes are the following three points.

--Changed import target from bigquery to storage --Changed bigquery.Client to storage.Client --Uncomment key_path and specify the save location of the JSON file created earlier

load_service_account.py


from google.cloud import storage
from google.oauth2 import service_account

# TODO(developer): Set key_path to the path to the service account key
#                  file.
key_path = "path/to/service_account.json"

credentials = service_account.Credentials.from_service_account_file(
    key_path,
    scopes=["https://www.googleapis.com/auth/cloud-platform"],
)

client = storage.Client(
    credentials=credentials,
    project=credentials.project_id,
)

When I ran the above program (load_service_account.py) as a trial, no error occurred. Therefore, load_service_account.py and storage_generate_signed_url_v4.py are combined and some modifications are made. The corrections are as follows.

--Delete duplicate imports --Changed client = storage.Client () to storage_client = storage.Client () --Enter the bucket name and object name for which you want to get the signed URL in the last line and call the function --Change the minutes = 15 part if you want to change the expiration date of the signed URL

storage_generate_signed_url_v4_auth.py


import datetime
from google.cloud import storage
from google.oauth2 import service_account

# TODO(developer): Set key_path to the path to the service account key
#                  file.
key_path = "path/to/service_account.json"

credentials = service_account.Credentials.from_service_account_file(
    key_path,
    scopes=["https://www.googleapis.com/auth/cloud-platform"],
)

storage_client = storage.Client(
    credentials=credentials,
    project=credentials.project_id,
)

def generate_download_signed_url_v4(bucket_name, blob_name):
    """Generates a v4 signed URL for downloading a blob.

    Note that this method requires a service account key file. You can not use
    this if you are using Application Default Credentials from Google Compute
    Engine or from the Google Cloud SDK.
    """
    # bucket_name = 'your-bucket-name'
    # blob_name = 'your-object-name'

    storage_client = storage.Client()
    bucket = storage_client.bucket(bucket_name)
    blob = bucket.blob(blob_name)

    url = blob.generate_signed_url(
        version="v4",
        # This URL is valid for 15 minutes
        expiration=datetime.timedelta(minutes=15),
        # Allow GET requests using this URL.
        method="GET",
    )

    print("Generated GET signed URL:")
    print(url)
    print("You can use this URL with any user agent, for example:")
    print("curl '{}'".format(url))
    return url

def generate_download_signed_url_v4('test_bucket', 'test_blob')

3. Run

Run python.

python3 storage_generate_signed_url_v4_auth.py

You will get the following results. This is the signed URL.

Generated GET signed URL:
https://storage.googleapis.com/test_bucket/test_blob/Abbreviation
You can use this URL with any user agent, for example:
curl 'https://storage.googleapis.com/test_bucket/test_blob/Abbreviation

Reference URL

Signed URL https://cloud.google.com/storage/docs/access-control/signed-urls?hl=ja

V4 signing process with Cloud Storage tools (Language in English) https://cloud.google.com/storage/docs/access-control/signing-urls-with-helpers

Authentication using the service account key file https://cloud.google.com/bigquery/docs/authentication/service-account-file?hl=ja

Recommended Posts

[GCP] How to publish Cloud Storage signed URLs (temporary URLs) in Python
How to switch python versions in cloud9
How to develop in Python
How to upload files to Cloud Storage with Firebase's python SDK
How to specify Cache-Control for blob storage in Azure Storage in Python
[GCP] How to output Cloud Functions log to Cloud Logging (Stackdriver Logging) (Python)
[Python] How to do PCA in Python
How to collect images in Python
How to use SQLite in Python
How to use Mysql in python
How to wrap C in Python
How to use ChemSpider in Python
How to use PubChem in Python
How to handle Japanese in Python
How to install OpenCV on Cloud9 and run it in Python
How to configure CORS settings for Azure storage service in Python
How to update the python version of Cloud Shell on GCP
[Introduction to Python] How to use class in Python?
How to access environment variables in Python
How to dynamically define variables in Python
How to do R chartr () in Python
[Itertools.permutations] How to put permutations in Python
How to work with BigQuery in Python
How to get a stacktrace in python
How to display multiplication table in python
How to extract polygon area in Python
How to check opencv version in python
[GCP] Operate Google Cloud Storage with Python
How to adjust image contrast in Python
How to use __slots__ in Python class
How to dynamically zero pad in Python
How to use regular expressions in Python
How to display Hello world in python
How to use is and == in Python
How to write Ruby to_s in Python
Convert absolute URLs to relative URLs in Python
How to deal with old Python versions in Cloud9 made by others
How to use the C library in Python
How to receive command line arguments in Python
[REAPER] How to play with Reascript in Python
How to clear tuples in a list (Python)
How to generate permutations in Python and C ++
How to embed a variable in a python string
How to implement Discord Slash Command in Python
Summary of how to import files in Python 3
How to simplify restricted polynomial fit in python
How to use Python Image Library in python3 series
How to implement shared memory in Python (mmap.mmap)
How to create a JSON file in Python
Summary of how to use MNIST in Python
How to call Cloud API from GCP Cloud Functions
How to specify TLS version in python requests
How to notify a Discord channel in Python
How to get the files in the [Python] folder
How to use tkinter with python in pyenv
How to run Leap Motion in non-Apple Python
[Python] How to draw a histogram in Matplotlib
How to output "Ketsumaimo" as standard output in Python
How to handle datetime type in python sqlite3
How to make Python Interpreter changes in Pycharm
How to plot autocorrelation and partial autocorrelation in python