Dump BigQuery tables to GCS using Python

Dump the table to GCS in CSV GZIP compression format.

environment

code

from google.cloud.bigquery import Client
from google.cloud.bigquery.job import ExtractJobConfig, DestinationFormat, Compression, ExtractJob

client = Client(project="your_prject_id", location="US")
job_config: ExtractJobConfig = ExtractJobConfig(
    compression=Compression.GZIP,
    destination_format=DestinationFormat.CSV
)
extract_job: ExtractJob = client.extract_table(
    source="your_dataset.your_table",
    destination_uris="gs://your_bucket/hoge.csv.gz",
    job_config=job_config)

extract_job.result() #Wait for completion

Recommended Posts

Dump BigQuery tables to GCS using Python
Connect to BigQuery with Python
Post to Twitter using Python
Start to Selenium using python
Copy S3 files from Python to GCS using GSUtil
How to install python using anaconda
Python logging and dump to json
From Python to using MeCab (and CaboCha)
Introduction to Discrete Event Simulation Using Python # 1
How to work with BigQuery in Python
Log in to Slack using requests in Python
Try to operate Excel using Python (Xlwings)
Introduction to Discrete Event Simulation Using Python # 2
Manipulate BigQuery tables from a Python client
Updated to Python 2.7.9
#Monte Carlo method to find pi using Python
Procedure to use TeamGant's WEB API (using python)
Sort BigQuery tables according to data in Dataflow
I want to email from Gmail using Python.
Start using Python
EC2 (Python3)-> BigQuery
Convert STL to Voxel mesh using Python VTK
Scraping using Python
"Backport" to python 2
[GCF + Python] How to upload Excel to GCS and create a new table in BigQuery
[BigQuery] How to use BigQuery API for Python -Table creation-
How to set up a Python environment using pyenv
(Python) Try to develop a web application using Django
How to auto-submit Microsoft Forms using python (Mac version)
Push notifications from Python to Android using Google's API
Use gcloud-python to populate BigQuery without going through GCS
Error due to conflict between python when using gurobi
Easy way to scrape with python using Google Colab
MessagePack-Call Python (or Python to Ruby) methods from Ruby using RPC
How to make a Python package using VS Code
Write data to KINTONE using the Python requests module
How to exit when using Python in Terminal (Mac)
How to retrieve multiple arrays using slice in python.
[Introduction to Python] How to stop the loop using break?
Try to extract high frequency words using NLTK (python)
Output to "7-segment LED" using python on Raspberry Pi 3!
How to execute a command using subprocess in Python
[Introduction to Python] How to write repetitive statements using for statements
What I was addicted to when using Python tornado
Query from python to Amazon Athena (using named profile)
I tried to access Google Spread Sheets using Python
[Technical book] Introduction to data analysis using Python -1 Chapter Introduction-
Operate Redmine using Python Redmine
How to install Python
Fibonacci sequence using Python
Changes from Python 3.0 to Python 3.5
Changes from Python 2 to Python 3.0
Data analysis using Python 0
Rewrite Python2 code to Python3 (2to3)
How to install python
python decorator to retry
Introduction to Python language
Introduction to OpenCV (python)-(2)
Using Python #external packages
WiringPi-SPI communication using Python
Note to daemonize python