Even if you are developing in a GAE / py environment, you can access the datastore of other projects, etc. To use Google's RPC API I think there are cases where you want to use the google-cloud-python library.
At that time, there were some points to note regarding the path, so make a note of them.
google-cloud-python
This library is for hitting the GCP API from GCE or locally, Because you are not using C? , GAE / py also works.
It used to be called gcloud-python, but around September 2016, namespace was renamed to gcloud-> google.cloud. gcloud-python (I'm writing this article because of this rename)
You can see the basic usage by looking at read the docs, Find google-cloud or the module you want to use from pypi and install it with pip.
$ pip install google-cloud-datastore
After that, the module will be expanded under google.cloud, so you can use it as it is.
from google.cloud import datastore
client = datastore.Client()
product_key = client.key('Product', 123)
print(client.get(product_key))
As you may have noticed if you read this far, In GAE / py, import the appengine module from under google.appengine and use it. Therefore, if you install google-cloud, Modules under google in the local virtualenv environment, It will be overwritten by google-cloud from GAE SDK.
At this time, if you execute python on virtualenv, Whether it's a REPL or a file execution such as a unit test,
>>> import google.appengine
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named appengine
I get an error like this:
In the virtualenv lib / python2.7 / site-packages / _virtualenv_path_extensions.pth
I added the module addition logic while adding the library path of google_appengine.
Of course, it would be the same if you add it to your own .pth file.
_virtualenv_path_extensions.pth
# ...Below is the location of the python sdk/usr/local/google_appengine case
/usr/local/google_appengine
import sys, types, os;has_mfs = sys.version_info > (3, 5);p = os.path.join('/usr/local/google_appengine', *('google',));importlib = has_mfs and __import__('importlib.util');has_mfs and __import__('importlib.machinery');m = has_mfs and sys.modules.setdefault('google', importlib.util.module_from_spec(importlib.machinery.PathFinder.find_spec('google', [os.path.dirname(p)])));m = m or not has_mfs and sys.modules.setdefault('google', types.ModuleType('google'));mp = (m or []) and m.__dict__.setdefault('__path__',[]);(p not in mp) and mp.append(p)
# ...
Since it is difficult to read in one line, insert a line break and look at the contents. Since I brought the description of other libraries of google, there is also a description of 3.5 correspondence, but GAE / py is 2.7 series, so I think it can be scraped.
import sys, types, os
has_mfs = sys.version_info > (3, 5)
# google_Create google module path in app engine
p = os.path.join('/usr/local/google_appengine', *('google',))
#Extract Module object
# 3.5 cases
importlib = has_mfs and __import__('importlib.util')
has_mfs and __import__('importlib.machinery')
m = has_mfs and sys.modules.setdefault('google', importlib.util.module_from_spec(importlib.machinery.PathFinder.find_spec('google', [os.path.dirname(p)])))
# 2.7 cases
m = m or not has_mfs and sys.modules.setdefault('google', types.ModuleType('google'))
mp = (m or []) and m.__dict__.setdefault('__path__',[])
#Add path to module
(p not in mp) and mp.append(p)
It's a survey or a reference. ..
Adding multiple modules to the same namespace without python I don't think it's very welcome. So I thought it would be difficult to solve it.
However, if you take a closer look at the google-cloud library, Packages for multiple pypi are installed. And they are added under the google and google.cloud modules. I thought I could fix the path in the same way, so I searched for the .pth file and found the answer.
I don't really understand the style of .pth files,
It seems that the above measures cannot be added to GAE / py, When I deployed, both modules were properly added to the google module. As expected. (I want you to distribute app engine sdk with pypi like google-cloud library)
nose
Also, there was an error in python -m unittest discover etc.
When I run nosetests, the test completes without any problems.
This is because the nose library wraps the unittest and before running the test It seemed to be because I was modifying PYTHONPATH.
-Prevent tampering with nose sys.path
In CircleCI, test runner defaults to nose I was late to realize that the path conflict wasn't resolved because the test passed successfully. I think that there is no problem if you are doing nosetests locally, It is a problem when debugging with REPL etc., so I think it is better to solve it.
I didn't understand much about PYTHONPATH and .pth files, so I searched a lot. .. -What is PYTHONPATH --How to add default path in Python --Control the Python library path -[210: Control Python library loader](http://lightson.dip.jp/zope/ZWiki/210Python_e3_81_ae_e3_83_a9_e3_82_a4_e3_83_96_e3_83_a9_e3_83_aa_e3_83_ad_e3_83_bc_e3_e3_8_e3_e3_e3_e3_e3_e3_e3_e3_e3_e3_ -Search for Python module search path
Recommended Posts