A story linked with Google Cloud Storage with a little ingenuity

This article is the 9th day article of WebCrew Advent Calendar 2017. Yesterday was @ kadota's "How to implement WEB push notification".

background

I needed to link with Google Cloud Storage (GCS) in a Java application (Java 6) running on CentOS 5 running as an application server. I realized it in a slightly acrobatic way, so I wrote this article.

General cooperation method

A common way to integrate with Java applications is to use Google Cloud Client Library for Java to integrate with GCS. When I actually implemented a function that requires GCS linkage in another application, I tried to link with GCS using Google Cloud Client Library for Java.

However, the Google Cloud Client Library for Java requires a Java 7 or higher execution environment. This time, the Java version of the linked application was Java 6, so another method was required.

Adopted cooperation method

** Install CLOUD SDK on CentOS5 and execute CLOUD SDK command by external call of Java **

Installing the CLOUD SDK on Cent OS 5

The procedure for installing the CLOUD SDK for Cent OS 5 is roughly divided into the following.

  1. Install Python 2.7
  2. Install CLOUD SDK

I will explain each of them based on the points to be noted.

Install Python 2.7

The CLOUD SDK requires Python 2.7 to be installed. The default Python version for Cent OS 5 is 2.4. So I'll start by upgrading the Python version.

important point

** You should think that you can't update Python using yum on Cent OS 5 ** ** You should think that you can't update Python using yum on Cent OS 5 ** ** You should think that you can't update Python using yum on Cent OS 5 **

I said it three times because it's important.

If you search google teacher for "centos python update", the method using yum may be hit, but it is better to think that Python update using the yum repository is not possible on Cent OS 5.

The official yum repository for Cent OS 5 is EOL and Even if you replace the repository with a mirror site, you can use yum install such as centos-release-scl-rh (Software Collections for CentOS).

Let's aim for the update by steadily decompressing from the compressed file.

Actual procedure

  1. Download and decompress the compressed file

    wget --no-check-certificate  https://www.python.org/ftp/python/2.7.9/Python-2.7.9.tgz
    tar xvzf Python-2.7.9.tgz
    
  2. Python installation

    cd Python-2.7.9
    ./configure --prefix=/usr/local
    make
    make altinstall
    
  3. Python replacement

    mv /usr/bin/python /usr/bin/python_bk
    

Install CLOUD SDK

  1. Download and decompress the compressed file

    cd /usr/local
    
  1. Install CLOUD SDK

    ./google-cloud-sdk/install.sh
    
  1. Confirm CLOUD SDK installation

    /usr/local/google-cloud-sdk/bin/gcloud --help
    

External calls in Java applications

After installing the CLOUD SDK on your application server, run the SDK commands externally in your Java application.

This section describes the procedure for uploading a file from the application server to GCS.

  1. Grant approval by gcloud auth
  2. Upload from application server to GCS with gsutil

Actual Java source

  1. Grant approval by gcloud auth

Granting approval using a service account See Cloud SDK Tool Authorization for more information.

```java
ProcessBuilder pb = new ProcessBuilder({"/usr/local/google-cloud-sdk/bin/gcloud", "auth", "activate-service-account", "--key-file", "[Service account key file]"});
process =pb.start();
processRes = process.waitFor();
//Error handling is performed depending on the result of processRes
```
  1. Upload from application server to GCS with gsutil

    ProcessBuilder pb = new ProcessBuilder({"/usr/local/google-cloud-sdk/bin/gsutil","mv","[Upload file path]","[Upload destination GCS bucket URL]"});
    process =pb.start();
    processRes = process.waitFor();
    //Error handling is performed depending on the result of processRes
    

Conclusion

Starting with a slight panic, "Google Cloud Client Library for Java is not available, GCS integration !? What should I do !!", "CentOS 5, default Python is 2.4 !? What should I do !!", "yum can't be used!" There were many cases such as "What should I do !!" and "I don't know the success judgment of the external call of the SDK command !? What should I do !!" I was able to implement it firmly. Obviously, it's important not to stop thinking because you can't do it, but to try an approach like that. This case may not be very common, but I hope you get used to helping someone who is having trouble with a similar phenomenon.

Tomorrow is @ t-itou. Thank you.

Finally

WebCrew is always looking for people to work with. Please feel free to enter.

Recruitment of development engineers Recruitment of front-end engineers Recruitment of database engineers

Recommended Posts

A story linked with Google Cloud Storage with a little ingenuity
[GCP] Operate Google Cloud Storage with Python
Use of Google Cloud Storage (GCS) with "GAE / Py"
A little stuck with chainer
Copy data from Amazon S3 to Google Cloud Storage with Python (boto)
Streaming speech recognition with Google Cloud Speech API
A story connecting Slack and google spreadsheets
A story about machine learning with Kyasuket
Try using Python with Google Cloud Functions
Make a cascade classifier with google colaboratory
English word book program linked with Google Docs
Get Google Cloud Storage object list in Java
Automatic voice transcription with Google Cloud Speech API
A story stuck with handling Python binary data
A story about implementing a login screen with django
A little advanced job scheduling with AP Scheduler
A story packed with absolute values in numpy.ndarray
A story about an error when loading a TensorFlow model created with Google Colab locally