Connect to BigQuery with Python

Contents

Let's use BigQuery's client library in Python to get data on BigQuery.

Install the client library

pip install --upgrade google-cloud-bigquery

Create a service account

To enable the client library, you need to create a service account and set up authentication. If not set, a DefaultCredentialsError will occur.

"IAM and management" → "Service account" → "Create service account"

After creating a service account with, the JSON file containing the key will be downloaded with "Create Key".

Try connecting to BigQuery

I have created the following dataset in BigQuery.

スクリーンショット 2020-05-25 4.39.18.png

from google.cloud import bigquery
import os

#Describe the path of the downloaded json file
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = './*****.json'

client = bigquery.Client()
sql = "select *from dataset name.table name"

df = client.query(sql).to_dataframe()

project_id = 'Project ID'
df = client.query(sql, project=project_id).to_dataframe()
print(df.head())

Execution result

I was able to get the data firmly. スクリーンショット 2020-05-25 5.33.02.png

Reference information

https://cloud.google.com/bigquery/docs/reference/libraries?hl=ja

Recommended Posts

Connect to BigQuery with Python
Connect to Wikipedia with Python
Connect to MySQL with Python within Docker
How to work with BigQuery in Python
Connect to s3 with AWS Lambda Python
Connect python to mysql
Connect to pepper with PEPPER Mac's python interpreter
Connect to MySQL with Python on Raspberry Pi
Post to slack with Python 3
Connect to Postgresql with GO
Connect to sqlite from python
Switch python to 2.7 with alternatives
Write to csv with Python
Python: How to use async with
Link to get started with python
Connect to multiple databases with SQLAlchemy
[Python] Write to csv file with Python
Create folders from '01' to '12' with python
Nice to meet you with python
Try to operate Facebook with Python
Output to csv file with Python
Convert list to DataFrame with python
MP3 to WAV conversion with Python
To do tail recursion with Python2
How to get started with Python
Connect to utf8mb4 database from python
What to do with PYTHON release?
Connect to Bitcoin Testnet with Pycoin
Unable to install Python with pyenv
How to use FTP with Python
Connect to Elastic MQ with boto
How to calculate date with python
Easily post to twitter with Python 3
I want to debug with Python
Connect Raspberry Pi to Alibaba Cloud IoT Platform with Python
FizzBuzz with Python3
Updated to Python 2.7.9
Try to reproduce color film with Python
Scraping with Python
Try logging in to qiita with Python
Change Python 64bit environment to 32bit environment with Anaconda
English speech recognition with python [speech to text]
Convert memo at once with Python 2to3
HTML email with image to send with python
Statistics with python
Memo to ask for KPI with python
Python to remember only with hello, worlds
Scraping with Python
Output color characters to pretty with python
Python with Go
Introduction to Python Image Inflating Image inflating with ImageDataGenerator
Output Python log to console with GAE
Twilio with Python
Convert Excel data to JSON with python
Convert Hiragana to Romaji with Python (Beta)
Fractal to make and play with Python
Integrate with Python
I wanted to solve ABC160 with Python
Play with 2016-Python
[Introduction to Python] Let's use foreach with Python
AES256 with python