GCP's Cloud Shell Editor was clogged up with the Python development environment, so I managed to rebuild it in a vacant lot.

Overview

From around the end of 2019, I was working on developing Python on GCP's Cloud Shell and deploying it to Cloud Function. After developing for a while and leaving it for a few months, a new development project has recently occurred, so when I thought about deploying and testing it, I started to throw out various errors at the time of deployment. I tried to deal with it while looking at the error, but it didn't work, and when I finally reset the environment, it worked, so I will summarize it.

Updating and switching versions of Python and pip was the biggest stalemate.

procedure

  1. Make a backup
  2. Reset the environment according to the official procedure
  3. Update Python with pyenv
  4. Switch pip to pip3 and update
  5. Follow the official instructions to update gcloud
  6. Restore working files
  7. Update from requirements.txt
  8. After that, deploy and crush bugs as appropriate

1. Make a backup

After exporting the installed packages with pip freeze> requirements.txt, let's download all the files on Cloud Sell Editor. The file will be tarred down. You don't need to unpack it because you will raise it as tar later.

2. Reset the environment according to the official procedure

** Disable or reset Cloud Shell ** https://cloud.google.com/shell/docs/resetting-cloud-shell?hl=ja

According to sudo rm -rf $HOME It seems that you can go by restarting Shell after resetting.

3. Update Python with pyenv

This Qiita was helpful, thank you to the person who wrote it.

** How to update the python version of Cloud Shell on GCP ** https://qiita.com/greenteabiscuit/items/cbecdf4f84f0b73ff96e

I'm running on python 3.7.9.

4. Switch pip to pip3 and update

If you pip it as it is, it will be displayed as python v2, so be careful. If left unattended, an error was thrown when deploying Cloud Function, so it seems better to use pip3 here as well. Switch to the command reference on the following site https://www.it-swarm-ja.tech/ja/pip/pip3%E3%82%B3%E3%83%9E%E3%83%B3%E3%83%89%E3%82%92pip%E3%81%AB%E5%A4%89%E6%9B%B4%E3%81%99%E3%82%8B%E3%81%AB%E3%81%AF%E3%81%A9%E3%81%86%E3%81%99%E3%82%8C%E3%81%B0%E3%82%88%E3%81%84%E3%81%A7%E3%81%99%E3%81%8B%EF%BC%9F/831723093/

alias pip=pip3 I can go.

After that, use pip install -U pip to update to the latest version.

5. Follow the official instructions to update gcloud

I also had to update gcloud. Officially this is the procedure:

gcloud components update https://cloud.google.com/sdk/gcloud/reference/components/update

gcloud components update Is the command. However, on Shell, it cannot be updated with this,

sudo apt-get update && sudo apt-get --only-upgrade install google-cloud-sdk-app-engine-python-extras google-cloud-sdk-kpt google-cloud-sdk-minikube google-cloud-sdk-app-engine-java google-cloud-sdk-datalab google-cloud-sdk-kind google-cloud-sdk-app-engine-grpc google-cloud-sdk-firestore-emulator google-cloud-sdk-bigtable-emulator google-cloud-sdk-skaffold kubectl google-cloud-sdk-cbt google-cloud-sdk-pubsub-emulator google-cloud-sdk-anthos-auth google-cloud-sdk google-cloud-sdk-datastore-emulator google-cloud-sdk-app-engine-python google-cloud-sdk-spanner-emulator google-cloud-sdk-cloud-build-local google-cloud-sdk-app-engine-go

You will be asked to execute a very long command. When I ran it, it was updated properly.

6. Restore working files

Let's upload the tar that was downloaded in 1. You can go to D & D or right-click where there is nothing in the file list → Upload Files. After uploading, decompress it with the tar command. I referred to this Qiita for how to do it:

** How to compress and decompress [Linux] files ** https://qiita.com/supersaiakujin/items/c6b54e9add21d375161f

tar -xvf xxxx.tar Command above

7. Update from requirements.txt

Now let's reinstall the pip package. I referred to the following site.

** Bulk installation of packages using requirements.txt with Python and pip ** https://note.nkmk.me/python-pip-install-requirements/

pip install -r requirements.txt

Occasionally, I get an error like "This package cannot be installed without another package", but in that case, if you specify the package name and install it, it will be quite good. I'm just using the bigquery api, but I had to install the related packages below:

pip install google-cloud-bigquery

8. After that, deploy and crush bugs as appropriate

After that, let's crush the bugs ~~ infinitely by doing gcloud functions deploy ~~.

Postscript

Perhaps you don't have to worry about changing the version with the blue bar at the bottom of the Editor screen.

Recommended Posts

GCP's Cloud Shell Editor was clogged up with the Python development environment, so I managed to rebuild it in a vacant lot.
In IPython, when I tried to see the value, it was a generator, so I came up with it when I was frustrated.
I want to set up a GUI development environment with Python or Golang on Mac
I was addicted to creating a Python venv environment with VS Code
I set the environment variable with Docker and displayed it in Python
I tried to build a Mac Python development environment with pythonz + direnv
I set up TensowFlow and was addicted to it, so make a note
In IPython, when I tried to see the value, it was a generator, so I came up with it when I was frustrated.
When it is troublesome to set up an SMTP server locally when sending mail with Python.
GCP's Cloud Shell Editor was clogged up with the Python development environment, so I managed to rebuild it in a vacant lot.
I managed to do it because the custom of attaching a zip with a password to an email and saying "I will send you the password separately" is troublesome.
When I tried to change the root password with ansible, I couldn't access it.
When I tried to create a virtual environment with Python, it didn't work
I was able to mock AWS-Batch with python, moto, so I will leave it
I wanted to know the number of lines in multiple files, so I tried to get it with a command
I set up TensowFlow and was addicted to it, so make a note
Make it easy to install the ROS2 development environment with pip install on Python venv
What I did when I was angry to put it in with the enable-shared option
I was addicted to scraping with Selenium (+ Python) in 2020
I want to work with a robot in python.
I was able to repeat it in Python: lambda
Set up a Python development environment with Sublime Text 2
When I tried to install PIL and matplotlib in a virtualenv environment, I was addicted to it.
I made a class to get the analysis result by MeCab in ndarray with python
What I was addicted to when dealing with huge files in a Linux 32bit environment
Recursively get the Excel list in a specific folder with python and write it to Excel.
It was great to edit the Python file in the Raspberry Pi with Atom's remote function
I made a function to crop the image of python openCV, so please use it.
I also tried to imitate the function monad and State monad with a generator in Python
I wrote a doctest in "I tried to simulate the probability of a bingo game with Python"
There was a doppelganger, so I tried to distinguish it with artificial intelligence (laughs) (Part 1)
I wrote a script to create a Twitter Bot development environment quickly with AWS Lambda + Python 2.7
How to build a python2.7 series development environment with Vagrant
How to get into the python development environment with Vagrant
When I tried to scrape using requests in python, I was addicted to SSLError, so a workaround memo
I tried to find out the difference between A + = B and A = A + B in Python, so make a note
A story that didn't work when I tried to log in with the Python requests module
I want to use a wildcard that I want to shell with Python remove
What I was addicted to with json.dumps in Python base64 encoding
[You have to know it! ] I tried to set up a Python environment profitably by making full use of the privileges of university students.
LINEbot development, I want to check the operation in the local environment
I wanted to solve the ABC164 A ~ D problem with Python
Homework was a pain, so I do management accounting with Python
I made a segment tree with python, so I will introduce it
Even if I converted jpg to png, I managed to do it because the transparency was not tampered with
I ran GhostScript with python, split the PDF into pages, and converted it to a JPEG image.
Every time I draw a graph with python, I check it, so I will summarize only the simplest usage
I tried to open the latest data of the Excel file managed by date in the folder with Python
[Python] How to save the installed package and install it in a new environment at once Mac environment
The story I was addicted to when I specified nil as a function argument in Go
[Python] I was hooked for an hour trying to use list comprehensions
How to save the feature point information of an image in a file and use it for matching
In IPython, when I tried to see the value, it was a generator, so I came up with it when I was frustrated.
I thought it would be slow to use a for statement in NumPy, but that wasn't the case.
I set up TensowFlow and was addicted to it, so make a note
When I tried to install PIL and matplotlib in a virtualenv environment, I was addicted to it.
I made a function to crop the image of python openCV, so please use it.
When I tried to scrape using requests in python, I was addicted to SSLError, so a workaround memo
GCP's Cloud Shell Editor was clogged up with the Python development environment, so I managed to rebuild it in a vacant lot.
When I tried to run Python, it was skipped to the Microsoft Store
I tried to make a calculator with Tkinter so I will write it
Use Python from Java with Jython. I was also addicted to it.
Environment maintenance made with Docker (I want to post-process GrADS in Python
The file name was bad in Python and I was addicted to import
I made a program to check the size of a file in Python
A note I looked up to make a command line tool in Python
I was addicted to trying Cython with PyCharm, so make a note
[Words spelled to me when I was in the first grade ①] I'm not afraid to build a programming environment.
[Python & SQLite] I tried to analyze the expected value of a race with horses in the 1x win range ①
Prepare a development environment that is portable and easy to duplicate without polluting the environment with Python embeddable (Windows)
When I made a treemap (area graph) with python, it was subtle, so when I used flourish, it felt pretty good.
Introduction to AI creation with Python! Part 2 I tried to predict the house price in Boston with a neural network
I want to build a Python environment
The file edited with vim was readonly but I want to save it
Even if the development language is changed to python3 in Cloud9, version 2 is displayed in python --version