Write Pulumi in Go

I was thinking of doing it, but I didn't touch it, so let's take this opportunity.

What is Pulumi?

https://www.pulumi.com/ Pulumi is a tool that allows you to provision and deploy infrastructure using your favorite programming language. It's like the Terraform used in the neighborhood these days.

While Terraform uses its own language called HCL, Pulumi seems to be able to use javascript/typescript/python/C #/F #/VB/Go (a language that can be confirmed with pulumi new -h now). It seems that AWS, of course, GCP/Azure, AliCloud, K8s, and OpenStack are also supported. Containerization is progressing as a recent trend, so it is wonderful that it is compatible with EKS/GKE/AKS.

I usually develop with Rails (Ruby), and sometimes I use typescript. So, this time I would like to write in Go.

Preparation

Since we are developing on macos, both go and pulumi are installed with brew. macos is Big Sur (11.1).

$ brew install go
$ brew install pulumi

By the way, I heard rumors about pulumi about last year, and I wonder if the version at that time was 1.0. When I ran the pulumi command to write this, I was prompted to upgrade to 2.16.0. (So ​​upgrade) It seems that the release cycle is very fast. I was prompted to upgrade to 2.16.1 the day after the upgrade, and today (12/25) I was prompted to upgrade to 2.16.2. .. It's amazing. The version of Go is 1.15.6.

After installing, set the environment variables related to AWS.

$ export AWS_REGION=ap-northeast-1
$ export AWS_PROFILE=<YOUR_PROFILE>
$ export AWS_ACCESS_KEY_ID=<YOUR_ACCESS_KEY_ID>
$ export AWS_SECRET_ACCESS_KEY=<YOUR_SECRET_ACCESS_KEY>

You can specify the profile with $ pulumi config set aws: profile hogehoge, but the environment variable is set because an error that the key is not set occurs every time when pulumi up is done later.

Constitution

This time I just wanted to write in Go, so the configuration will be simple. AWS respectively

It is a flow to deploy.

Write code

Create the first template.

$ mkdir pulumi-go
$ cd pulumi-go
$ pulumi new aws-go

It will flow in a sloppy manner, but in general it is OK with Enter as it is.

For the stack, set dev/prod and select dev.

$ pulumi stack init dev
$ pulumi stack init prod
$ pulumi stack select dev

The final file structure looks like this.

.
└── pulumi-go
   ├── alb.go
   ├── ec2.go
   ├── go.mod
   ├── go.sum
   ├── main.go
   ├── network.go
   ├── Pulumi.dev.yaml
   ├── Pulumi.yaml
   ├── README.md
   ├── s3.go
   ├── security.go
   └── setup.go

Main Since main.go is executed, we will write the code there.

main.go


package main

import (
	"github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
		err := setup(ctx)
		if err != nil {
			return err
		}

		return nil
	})
}

setup.go


package main

import (
	"github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func setup(ctx *pulumi.Context) error {
	// VPC
	opt, err := createVpc(ctx)
	if err != nil {
		return err
	}

	// SecurityGroup
	sg, err := createSg(ctx, opt)
	if err != nil {
		return err
	}

	// S3
	err = createS3(ctx)
	if err != nil {
		return err
	}

	// EC2
	err = createEc2(ctx, opt, sg)
	if err != nil {
		return err
	}

	// ALB
	err = createAlb(ctx, opt, sg)
	if err != nil {
		return err
	}

	return nil
}

VPC Create a new one without using the default VPC. I will put it in the structure because I will use it later.

network.go


package main

import (
	"github.com/pulumi/pulumi-aws/sdk/v3/go/aws/ec2"
	"github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

type Opt struct {
	vpc     interface{}
	subnet  interface{}
	subnet2 interface{}
	err     interface{}
}

func createVpc(ctx *pulumi.Context) (*Opt, error) {
	vpc, err := ec2.NewVpc(ctx, "app-dev-vpc", &ec2.VpcArgs{
		CidrBlock: pulumi.String("10.0.0.0/16"),
		Tags: pulumi.StringMap{
			"Name": pulumi.String("app-dev-vpc1"),
		},
	})
	if err != nil {
		return nil, err
	}

	gateway, err := ec2.NewInternetGateway(ctx, "app-dev-gw", &ec2.InternetGatewayArgs{
		VpcId: vpc.ID(),
		Tags: pulumi.StringMap{
			"Name": pulumi.String("app-dev-gw1"),
		},
	})
	if err != nil {
		return nil, err
	}

	_, err = ec2.NewRouteTable(ctx, "app-dev-rt", &ec2.RouteTableArgs{
		VpcId: vpc.ID(),
		Tags: pulumi.StringMap{
			"Name": pulumi.String("app-dev-rt1"),
		},
		Routes: ec2.RouteTableRouteArray{
			&ec2.RouteTableRouteArgs{
				CidrBlock: pulumi.String("0.0.0.0/0"),
				GatewayId: gateway.ID(),
			},
		},
	})
	if err != nil {
		return nil, err
	}

	subnet, err := ec2.NewSubnet(ctx, "app-dev-subnet1", &ec2.SubnetArgs{
		VpcId:            vpc.ID(),
		CidrBlock:        pulumi.String("10.0.2.0/24"),
		AvailabilityZone: pulumi.String("ap-northeast-1b"),
		Tags: pulumi.StringMap{
			"Name": pulumi.String("app-dev-subnet1"),
		},
	})
	if err != nil {
		return nil, err
	}

	subnet2, err := ec2.NewSubnet(ctx, "app-dev-subnet2", &ec2.SubnetArgs{
		VpcId:            vpc.ID(),
		CidrBlock:        pulumi.String("10.0.3.0/24"),
		AvailabilityZone: pulumi.String("ap-northeast-1c"),
		Tags: pulumi.StringMap{
			"Name": pulumi.String("app-dev-subnet2"),
		},
	})
	if err != nil {
		return nil, err
	}

	opt := new(Opt)
	opt.vpc = vpc
	opt.subnet = subnet
	opt.subnet2 = subnet2
	opt.err = err

	return opt, nil
}

SecurityGroup Create for EC2 and ALB. This is also put in the structure.

security.go


package main

import (
	"github.com/pulumi/pulumi-aws/sdk/v3/go/aws/ec2"
	"github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

type Sg struct {
	SecurityGroupEc2 interface{}
	SecurityGroupAlb interface{}
}

func createSg(ctx *pulumi.Context, opt *Opt) (*Sg, error) {
	// EC2 SecurityGroup
	sg1, err := ec2.NewSecurityGroup(ctx, "app-dev-ec2-sg", &ec2.SecurityGroupArgs{
		VpcId: opt.vpc.(*ec2.Vpc).ID(),
		Ingress: ec2.SecurityGroupIngressArray{
			ec2.SecurityGroupIngressArgs{
				CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
				FromPort:   pulumi.Int(22),
				ToPort:     pulumi.Int(22),
				Protocol:   pulumi.String("TCP"),
			},
			ec2.SecurityGroupIngressArgs{
				CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
				FromPort:   pulumi.Int(80),
				ToPort:     pulumi.Int(80),
				Protocol:   pulumi.String("TCP"),
			},
		},
		Egress: ec2.SecurityGroupEgressArray{
			ec2.SecurityGroupEgressArgs{
				CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
				FromPort:   pulumi.Int(0),
				ToPort:     pulumi.Int(0),
				Protocol:   pulumi.String("-1"),
			},
		},
		Tags: pulumi.StringMap{
			"Name": pulumi.String("app-dev-sg1"),
		},
	})
	if err != nil {
		return nil, err
	}

	// ALB SecurityGroup
	sg2, err := ec2.NewSecurityGroup(ctx, "app-dev-lb-sg", &ec2.SecurityGroupArgs{
		VpcId: opt.vpc.(*ec2.Vpc).ID(),
		Ingress: ec2.SecurityGroupIngressArray{
			ec2.SecurityGroupIngressArgs{
				CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
				FromPort:   pulumi.Int(80),
				ToPort:     pulumi.Int(80),
				Protocol:   pulumi.String("TCP"),
			},
		},
		Egress: ec2.SecurityGroupEgressArray{
			ec2.SecurityGroupEgressArgs{
				CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
				FromPort:   pulumi.Int(0),
				ToPort:     pulumi.Int(0),
				Protocol:   pulumi.String("-1"),
			},
		},
		Tags: pulumi.StringMap{
			"Name": pulumi.String("app-dev-sg2"),
		},
	})
	if err != nil {
		return nil, err
	}

	sg := new(Sg)
	sg.SecurityGroupEc2 = sg1
	sg.SecurityGroupAlb = sg2

	return sg, nil
}

S3 We will build a detailed policy etc. without once.

s3.go


package main

import (
	"github.com/pulumi/pulumi-aws/sdk/v3/go/aws/s3"
	"github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func createS3(ctx *pulumi.Context) error {
	_, err := s3.NewBucket(ctx, "app-dev-bucket", nil)
	if err != nil {
		return err
	}

	return nil
}

EC2 Launch an instance from the official AMI and specify the subnet and security group.

ec2.go


package main

import (
	"github.com/pulumi/pulumi-aws/sdk/v3/go/aws/ec2"
	"github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func createEc2(ctx *pulumi.Context, opt *Opt, sg *Sg) error {
	// Instance
	_, err := ec2.NewInstance(ctx, "app-dev-web", &ec2.InstanceArgs{
		Ami:                      pulumi.String("ami-01748a72bed07727c"), // Amazon Linux 2 AMI
		InstanceType:             pulumi.String("t3.micro"),
		SubnetId:                 opt.subnet.(*ec2.Subnet).ID(),
		VpcSecurityGroupIds:      pulumi.StringArray{sg.SecurityGroupEc2.(*ec2.SecurityGroup).ID()},
		KeyName:                  pulumi.String("app-dev"),
		AssociatePublicIpAddress: pulumi.Bool(true),
		Tags: pulumi.StringMap{
			"Name": pulumi.String("app-dev-web1"),
		},
	})
	if err != nil {
		return err
	}

	return nil
}

I was a little addicted to how to specify SubnetId and VpcSecurityGroupIds. Basic Pulumi solves the dependency of each resource well, but without thinking about it, for example

...(abridgement
		SubnetId:                 pulumi.String(opt.SubnetID),
		VpcSecurityGroupIds:      pulumi.StringArray{sg.SecurityGroupID},

Even if I wrote something like this, it didn't solve the dependency well. (An error occurs because I try to create an instance first and then create a vpc) So, I was able to resolve the dependency by putting the created object in interface {} in the structure and specifying it as . (* Ec2.Vpc) when using it. To be honest, I don't know if this is the case. ..

ALB It is a setting to simply receive at number 80 and send it to number 80.

alb.go


package main

import (
	"github.com/pulumi/pulumi-aws/sdk/v3/go/aws/ec2"
	"github.com/pulumi/pulumi-aws/sdk/v3/go/aws/lb"
	"github.com/pulumi/pulumi/sdk/v2/go/pulumi"
)

func createAlb(ctx *pulumi.Context, opt *Opt, sg *Sg) error {
	// alb(subnet)
	alb, err := lb.NewLoadBalancer(ctx, "app-dev-alb", &lb.LoadBalancerArgs{
		SecurityGroups: pulumi.StringArray{sg.SecurityGroupAlb.(*ec2.SecurityGroup).ID()},
		Subnets: pulumi.StringArray{
			opt.subnet.(*ec2.Subnet).ID(),
			opt.subnet2.(*ec2.Subnet).ID(),
		},
	})
	if err != nil {
		return err
	}

	// tg
	tg, err := lb.NewTargetGroup(ctx, "app-dev-tg", &lb.TargetGroupArgs{
		Port:     pulumi.Int(80),
		Protocol: pulumi.String("HTTP"),
		VpcId:    opt.vpc.(*ec2.Vpc).ID(),
	})
	if err != nil {
		return err
	}

	// listener
	_, err = lb.NewListener(ctx, "app-dev-listener", &lb.ListenerArgs{
		LoadBalancerArn: alb.Arn,
		Port:            pulumi.Int(80),
		Protocol:        pulumi.String("HTTP"),
		DefaultActions: lb.ListenerDefaultActionArray{
			&lb.ListenerDefaultActionArgs{
				Type:           pulumi.String("forward"),
				TargetGroupArn: tg.Arn,
			},
		},
	})
	if err != nil {
		return err
	}

	return nil
}

Build/deploy

Build and check Go.

$ go get .

Let's build it on AWS by pulumi up. If yes/no/details that comes in the middle is selected, the deployment to AWS will actually occur.

$ pulumi up -d -r
Previewing update (dev)

(abridgement)
     Type                        Name              Plan       Info
 +   pulumi:pulumi:Stack         pulumi-go-dev     create     3 debugs
 +   ├─ aws:ec2:Vpc              app-dev-vpc       create
 +   ├─ aws:s3:Bucket            app-dev-bucket    create
 +   ├─ aws:ec2:Subnet           app-dev-subnet2   create
 +   ├─ aws:ec2:Subnet           app-dev-subnet1   create
 +   ├─ aws:ec2:SecurityGroup    app-dev-lb-sg     create
 +   ├─ aws:lb:TargetGroup       app-dev-tg        create
 +   ├─ aws:ec2:SecurityGroup    app-dev-ec2-sg    create
 +   ├─ aws:ec2:InternetGateway  app-dev-gw        create
 +   ├─ aws:lb:LoadBalancer      app-dev-alb       create
 +   ├─ aws:ec2:Instance         app-dev-web       create
 +   ├─ aws:ec2:RouteTable       app-dev-rt        create
 +   └─ aws:lb:Listener          app-dev-listener  create

Do you want to perform this update? yes
Updating (dev)

(abridgement)

Pulumi destroy when you no longer need it.

$ pulumi destroy
Previewing destroy (dev)

(abridgement)

     Type                        Name              Plan
 -   pulumi:pulumi:Stack         pulumi-go-dev     delete
 -   ├─ aws:lb:Listener          app-dev-listener  delete
 -   ├─ aws:lb:LoadBalancer      app-dev-alb       delete
 -   ├─ aws:ec2:Subnet           app-dev-subnet2   delete
 -   ├─ aws:ec2:Instance         app-dev-web       delete
 -   ├─ aws:ec2:Subnet           app-dev-subnet1   delete
 -   ├─ aws:ec2:RouteTable       app-dev-rt        delete
 -   ├─ aws:ec2:InternetGateway  app-dev-gw        delete
 -   ├─ aws:ec2:Vpc              app-dev-vpc       delete
 -   ├─ aws:ec2:SecurityGroup    app-dev-lb-sg     delete
 -   ├─ aws:lb:TargetGroup       app-dev-tg        delete
 -   ├─ aws:s3:Bucket            app-dev-bucket    delete
 -   └─ aws:ec2:SecurityGroup    app-dev-ec2-sg    delete

Do you want to perform this destroy? yes
Destroying (dev)

(abridgement)

You can now do a general flow.

Impressions

Pulumi has a faster update speed than Terraform, and it is a good impression that you can write using the programming language you usually use. There is also a Web UI, and it's nice that the activity so far remains as a log as well as the appearance. I'm glad that the documentation was well prepared. However, Go seems to be new, and Coming soon!

The impression that Terraform is simpler in the amount of code to be written. However, if you write in typescript or python, there is no difference.

Unfortunately, Ruby/Rust is not supported. If you liven up the community, eventually ...! Anyway, if you ask me about infrastructure from today, ** the era is Pulumi! Let's respond with **.

Infrastructure as Code!!

Recommended Posts

Write Pulumi in Go
Write tests in GO language + gin
Write DCGAN in Keras
Write decorator in class
Write Python in MySQL
Write Kikagaku-style algorithm theory in Go Primality test
Write Pandoc filters in Python
Write standard input in code
Write beta distribution in Python
Write python in Rstudio (reticulate)
Write Spigot in VS Code
Implement recursive closures in Go
Write data in HDF format
Hello World in GO language
Write Spider tests in Scrapy
To write a test in Go, first design the interface
Write a binary search in Python
Write a table-driven test in C
Try implementing Yubaba in Go language
Use optinal type-like in Go language
Write JSON Schema in Python DSL
How to write soberly in pandas
Write an HTTP / 2 server in Python
Write AWS Lambda function in Python
Write A * (A-star) algorithm in Python
[Maya] Write custom nodes in Open Maya 2.0
Post to slack in Go language
Write foreign key constraints in Django
Write selenium test code in python
Write a pie chart in Python
Write a vim plugin in Python
Write a depth-first search in Python
Write C unit tests in Python
Do something object-oriented in GO language
Write documentation in Sphinx with Python Livereload
Implement and understand union-find trees in Go
Write the test in a python docstring
Write a short property definition in Python
Write O_SYNC file in C and Python
Write a Caesar cipher program in Python
Write and execute SQL directly in Elixir
Read and write JSON files in Python
Write a simple greedy algorithm in Python
Write python modules in fortran using f2py
Write a simple Vim Plugin in Python 3
Various comments to write in the program
Web application development in Go language_Hands-on day 1
How to write this process in Perl?
How to write Ruby to_s in Python