Caution

I'm new to GCP, python and jupyter. I was very happy to be able to move it while going around, so I posted it. I would appreciate it if you could let me know if you have any mistakes.

reference

-[GCP] AI Platform Notebooks has been officially released, so try using it. -[BigQuery] Did you know that you can use CREATE statements? Create table with SQL

Create BigQuery data

Open the GCP console

https://console.cloud.google.com/

Select "BigQuery" from the side menu

Screen Shot 2020-08-23 at 18.27.52.png

Click "Create Dataset"

Screen Shot 2020-08-23 at 18.32.48.png

I tried to create it with the dataset ID set to "test_data_set"

Screen Shot 2020-08-23 at 18.33.46.png

Easily create tables and data for testing

Paste the following query into the query editor and click Run

create table test_data_set.t (i INT64);
insert into test_data_set.t values (1),(2),(5),(7),(12);

Screen Shot 2020-08-23 at 18.45.00.png

Now that we have the data, let's take a look at it using Jupyter.

Screen Shot 2020-08-23 at 18.45.59.png

Instantiate a notebook

Select Side Menu> "AI Platform"> "Notebook"

Screen Shot 2020-08-23 at 17.36.45.png

I selected "New Instance"> "Python 2 and 3"

Screen Shot 2020-08-23 at 17.53.28.png

You can select the region etc. and the estimated amount is also displayed, but this time I will delete the instance as soon as I try it, so I pressed "Create" with the default settings

Screen Shot 2020-08-23 at 18.03.53.png

Click "Open JUPYTER LAB"

Screen Shot 2020-08-23 at 18.10.47.png

Jupyter Lab opens

Screen Shot 2020-08-23 at 19.14.42.png

When I first tried to get bigquery data

"UserWarning: Cannot create BigQuery Storage client, the dependency google-cloud-bigquery-storage is not installed."

I got an error, so install google-cloud-bigquery-storage first

Click "Console"

Screen Shot 2020-08-23 at 19.12.40.png

Install google-cloud-bigquery-storage

$ pip install google-cloud-bigquery-storage

Screen Shot 2020-08-23 at 19.09.06.png

Press Python 3 to create a new notebook

Screen Shot 2020-08-23 at 18.55.47.png

By default, JupyterLab has a file called /tutorials/bigquery/BigQuery basics.ipynb and there is an easy-to-understand explanation, so I tried to easily create a process of data acquisition from bigquery by referring to that area.

from google.cloud import bigquery

client = bigquery.Client(location="US")

query = """
    select *
    from test_data_set.t
    order by i
"""

query_job = client.query(
    query,
    # Location must match that of the dataset(s) referenced in the query.
    location="US",
)  # API request - starts the query

dv = query_job.to_dataframe()
print(df)
df.plot()

Press "▷" to execute

Screen Shot 2020-08-23 at 19.33.57.png

I was able to display the data of the bigquery I made first as a graph!

There will be a charge at the end, so stop with "Stop"

Screen Shot 2020-08-23 at 19.41.54.png

Thank you for watching until the end m (_ _) m

[PYTHON] I tried to visualize BigQuery data using Jupyter Lab on GCP

reference

Create BigQuery data

Instantiate a notebook