I tried deploying Kubernetes Pods / Helm Chart with Pulumi (Python)

A memo written by a Pulumi beginner when he made an evaluation while also studying Kubernetes. The official documentation has a relatively large number of TypeScript examples, so I'll write it down as a Python example.

Overview

As the title suggests, we tried and evaluated Kubernetes deployment with Pulumi. The following are the main impressions.

--Since the template for Kubernetes (kubernetes-python) is provided, the threshold of introduction is relatively low (just by pulumi up from the initial state of the project, a simple nginx deployment is created. it can). --If you can hit the kubectl command, there seems to be no additional settings. --Pulumi's Helm Chart deploy function not only simply kicks Helm, but also deploys and interprets resources such as Servide and Deployment independently. For this reason, Helm 2.x does not require a Tiller Server. --This time, Helm 3.x was used, so you cannot receive this benefit. On the contrary, there is a disadvantage that you cannot check the contents with the helm list command. --The conversion implementation of Pulumi.Output (parameter for the output of the generated resource) required some trial and error (somewhat difficult to understand, such as around the ʻapply` function). ――The implementation code looks like Python code for sequential processing, but it may be a little difficult to understand because it handles declarative concepts internally (Preview, Diff control, etc.).

Evaluation

Premise

The Kubernetes cluster (using AWS EKS) has already been built, and the initial settings for Plumi and Helm are assumed to be completed. The repository of Helm Chart (bitnami / jenkins) used for the test is also pre-registered (it seems that it can be obtained from the repository with the function of the FetchOps class, but it is excluded from the evaluation target this time).

$ pulumi version
v1.8.1
$ kubectl get nodes
NAME                                            STATUS   ROLES    AGE     VERSION
ip-10-0-0-145.ap-northeast-1.compute.internal   Ready    <none>   1h51m   v1.14.7-eks-1861c5
ip-10-0-2-96.ap-northeast-1.compute.internal    Ready    <none>   1h50m   v1.14.7-eks-1861c5
$ helm version
version.BuildInfo{Version:"v3.0.2", GitCommit:"19e47ee3283ae98139d98460de796c1be1e3975f", GitTreeState:"clean", GoVersion:"go1.13.5"}

$ helm repo list
NAME   	URL
bitnami	https://charts.bitnami.com/bitnami

Project initialization

Create a new Pulumi Project using the kubernetes-python template (questions asked are answered by default).

$ mkdir python-k8s && cd python-k8s
$ pulumi new kubernetes-python

This command will walk you through creating a new Pulumi project.
 
 Enter a value or leave blank to accept the (default), and press <ENTER>.
 Press ^C at any time to quit.
 
 project name: (python-k8s)
 project description: (A minimal Kubernetes Python Pulumi program)
 Created project 'python-k8s'
 
 Please enter your desired stack name.
 To create a stack in an organization, use the format <org-name>/<stack-name> (e.g. `acmecorp/dev`).
 stack name: (dev)
 Created stack 'dev'
 
 Your new project is ready to go! ✨
 
 To perform an initial deployment, run the following commands:
 
    1. virtualenv -p python3 venv
    2. source venv/bin/activate
    3. pip3 install -r requirements.txt
 
 Then, run 'pulumi up'

Set up Python's virtualenv environment (and install dependent libraries) as shown in the message.

$ pip3 install virtualenv
$ virtualenv -p python3 venv
$ source venv/bin/activate
$ pip3 install -r requirements.txt

Implementation

Changed the Python implementation code (__main__.py) to create the following two types of resources. In addition, it is defined to output the host name of each service (host name assigned to AWS Elastic Load Balancer) in Output.

import base64
import pulumi
from pulumi_kubernetes.apps.v1 import Deployment
from pulumi_kubernetes.core.v1 import Service
from pulumi_kubernetes.helm.v2 import Chart, ChartOpts

def deploy_nginx_service():
    app_name = "nginx"
    app_labels = { "app": app_name }

    nginx_deployment = Deployment(
        app_name,
        spec={
            "selector": { "match_labels": app_labels },
            "replicas": 1,
            "template": {
                "metadata": { "labels": app_labels },
                "spec": { "containers": [{ "name": app_name, "image": "nginx" }] }
            }
        })

    nginx_service = Service(
        app_name,
        metadata={
            "labels": nginx_deployment.spec["template"]["metadata"]["labels"],
        },
        spec={
            "type": "LoadBalancer",
            "ports": [{ "port": 80, "target_port": 80, "protocol": "TCP" }],
            "selector": app_labels,
        })

    return nginx_service

def deploy_jenkins_chart():
    return Chart("jenkins", ChartOpts(
        chart="jenkins",
        repo="bitnami",
        values={},
    ))

nginx_service = deploy_nginx_service()
jenkins_chart = deploy_jenkins_chart()

pulumi.export("nginx_hostname",
    nginx_service.status.apply(
        lambda x: x["load_balancer"]["ingress"][0]["hostname"]))

pulumi.export("jenkins_hostname",
    jenkins_chart.resources.apply(
        lambda x: x["v1/Service:jenkins"].status["load_balancer"]["ingress"][0]["hostname"]))

Run

pulumi up
Previewing update (dev):

     Type                                         Name            Plan
 +   pulumi:pulumi:Stack                          python-k8s-dev  create
 +   ├─ kubernetes:helm.sh:Chart                  jenkins         create
 +   │  ├─ kubernetes:core:PersistentVolumeClaim  jenkins         create
 +   │  ├─ kubernetes:core:Secret                 jenkins         create
 +   │  ├─ kubernetes:core:Service                jenkins         create
 +   │  └─ kubernetes:apps:Deployment             jenkins         create
 +   ├─ kubernetes:apps:Deployment                nginx           create
 +   └─ kubernetes:core:Service                   nginx           create

Resources:
    + 8 to create

Do you want to perform this update? yes
Updating (dev):

     Type                                         Name            Status
 +   pulumi:pulumi:Stack                          python-k8s-dev  created
 +   ├─ kubernetes:helm.sh:Chart                  jenkins         created
 +   │  ├─ kubernetes:core:Secret                 jenkins         created
 +   │  ├─ kubernetes:core:Service                jenkins         created
 +   │  ├─ kubernetes:core:PersistentVolumeClaim  jenkins         created
 +   │  └─ kubernetes:apps:Deployment             jenkins         created
 +   ├─ kubernetes:apps:Deployment                nginx           created
 +   └─ kubernetes:core:Service                   nginx           created

Outputs:
    jenkins_hostname: "xxx.ap-northeast-1.elb.amazonaws.com"
    nginx_hostname  : "yyy.ap-northeast-1.elb.amazonaws.com"

Resources:
    + 8 created

ʻThe host name output to Outputs is Service (Load Balancer) corresponding to jenkinsandnginx, respectively. If you access each with curl`, a response will be returned.

$ curl -s --head xxx.ap-northeast-1.elb.amazonaws.com | grep X-Jenkins:
X-Jenkins: 2.204.1
$ curl -s --head yyy.ap-northeast-1.elb.amazonaws.com | grep Server
Server: nginx/1.17.6

Check resources

You can also check the generated resources with the kubectl command. However, Helm Chart is not displayed (the list remains empty) because Pulumi develops it independently.

$ kubectl get deployments
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
jenkins          1/1     1            1           32m
nginx-nziiq5rs   1/1     1            1           32m
$ kubectl get services
NAME             TYPE           CLUSTER-IP      EXTERNAL-IP                            PORT(S)                      AGE
jenkins          LoadBalancer   172.20.125.15   xxx.ap-northeast-1.elb.amazonaws.com   80:32525/TCP,443:31321/TCP   33m
kubernetes       ClusterIP      172.20.0.1      <none>                                 443/TCP                      1h42m
nginx-6hbjq6d7   LoadBalancer   172.20.14.82    yyy.ap-northeast-1.elb.amazonaws.com   80:32325/TCP                 33m
$ helm list
NAME	NAMESPACE	REVISION	UPDATED	STATUS	CHART	APP VERSION

Reference information

Description of Plugin for Kubernetes.

API Reference。

In this tutorial, we’ll use the Helm API of @pulumi/kubernetes to deploy v2.1.3 of the Wordpress Helm Chart to a Kubernetes cluster. The Tiller server is not required to be installed. Pulumi will expand the Helm Chart and submit the expanded YAML to the cluster.

Explanation that Pulumi's Helm Provider deploys and deploys Chart independently. Due to this, it seems that it is not output in the result of helm list.

Description of output conversion of Output. It is necessary to use the ʻapply function when processing the Output (Pulumi.Outputtype) output by Pulumi (maybe it is doing special processing internally, or simply converting the object does not work well. For example, I encountered a phenomenon that processing such aspulumi up` is blocked and processing does not proceed).

The apply method accepts a callback which will be passed the value of the Output when it is available, and which returns the new value. The result of the call to apply is a new Output whose value is the value returned from the callback, and which includes the dependencies of the original Output. If the callback itself returns an Output, the dependencies of that output are unioned into the dependencies of the returned Output.

Recommended Posts

I tried deploying Kubernetes Pods / Helm Chart with Pulumi (Python)
I tried fp-growth with python
I tried scraping with Python
I tried gRPC with Python
I tried scraping with python
I tried web scraping with python.
I tried running prolog with python 3.8.2.
I tried SMTP communication with Python
I tried scraping Yahoo News with Python
I tried sending an email with python.
I tried non-photorealistic rendering with Python + opencv
I tried a functional language with Python
I tried recursion with Python ② (Fibonacci sequence)
#I tried something like Vlookup with Python # 2
I tried "smoothing" the image with Python + OpenCV
I tried hundreds of millions of SQLite with python
I tried "differentiating" the image with Python + OpenCV
I tried L-Chika with Raspberry Pi 4 (Python edition)
I tried Jacobian and partial differential with python
I tried to get CloudWatch data with Python
I tried using mecab with python2.7, ruby2.3, php7
I tried function synthesis and curry with python
I tried to output LLVM IR with Python
I tried "binarizing" the image with Python + OpenCV
I tried running faiss with python, Go, Rust
I tried to automate sushi making with python
I tried playing mahjong with Python (single mahjong edition)
I tried running Deep Floor Plan with Python 3.6.10.
I tried sending an email with SendGrid + Python
I tried Python> autopep8
I tried Python> decorator
I tried to implement Minesweeper on terminal with python
I tried to get started with blender python script_Part 01
I tried to touch the CSV file with Python
I tried to draw a route map with Python
[OpenCV / Python] I tried image analysis of cells with OpenCV
I tried to solve the soma cube with python
I tried to get started with blender python script_Part 02
I tried to implement an artificial perceptron with python
I tried to automatically generate a password with Python3
Mayungo's Python Learning Episode 1: I tried printing with print
I tried to solve the problem with Python Vol.1
I tried to analyze J League data with Python
I tried "morphology conversion" of images with Python + OpenCV
I tried hitting the API with echonest's python client
I tried to solve AOJ's number theory with Python
I tried to find the entropy of the image with python
I tried "gamma correction" of the image with Python + OpenCV
I tried to simulate how the infection spreads with Python
I tried using the Python library from Ruby with PyCall
I tried to make various "dummy data" with Python faker
I tried various methods to send Japanese mail with Python
I tried sending an email from Amazon SES with Python
I tried Learning-to-Rank with Elasticsearch!
I made blackjack with python!
I tried running Movidius NCS with python of Raspberry Pi3
I tried clustering with PyCaret
I tried follow management with Twitter API and Python (easy)
[Python] I tried to visualize tweets about Corona with WordCloud
Mayungo's Python Learning Episode 3: I tried to print numbers with print
I tried Python C extension