theme

GCP has a Secret Manager. If you know Kubernetes, you might think, "Is it something like Secrets?" However, since k8s Secrets is Base64 encoded, it can not be uploaded to a public GitHub repository (for example, as a name for managing the original data), but Secret Manager is encrypted (if you do not know the key) Exactly Secret.

For other things like melon, see below. https://cloud.google.com/blog/ja/products/identity-security/introducing-google-clouds-secret-manager

As mentioned in the above reference article, not only Secret Manager but also GCP services can be prepared by using Cloud SDK. You can easily operate resources on GCP just by hitting. So, as in this topic, you don't have to write a wrapper programmatically at all. This time, I simply wrapped the function to operate Secret Manager as a subject to write an appropriate command line tool using google's subcommands library.

Assumed reader

――I know about GCP. --Golang can be written as it is.

Premise

--Go development environment has been built locally. --GCP contract completed. --Cloud SDK has been set up locally. --The key JSON file path (of the service account with all the required permissions) has been set in the local environment variable GOOGLE_APPLICATION_CREDENTIALS.

Development environment

OS - Linux(Ubuntu)

$ cat /etc/os-release 
NAME="Ubuntu"
VERSION="18.04.5 LTS (Bionic Beaver)"

#Backend

#Language --Golang

$ go version
go version go1.15.2 linux/amd64

IDE - Goland

GoLand 2020.2.3
Build #GO-202.7319.61, built on September 16, 2020

All sources this time

https://github.com/sky0621/gcp-toolbox/tree/v0.1.0/secret-manager

Individual source

I thought I'd write a commentary, but it's almost all the promises when using google's subcommands and the promises when using the Secret Manager SDK (that is, the information on each site). No explanation.

main.go

As a function, only the " create "command for creating a secret and the" list" command for displaying a list of created secrets are prepared.

package main

import (
	"context"
	"flag"
	"os"

	"github.com/google/subcommands"
)

func main() {
	os.Exit(int(execMain()))
}

func execMain() subcommands.ExitStatus {
	subcommands.Register(subcommands.HelpCommand(), "")
	subcommands.Register(newCreateCmd(), "create")
	subcommands.Register(newListCmd(), "list")
	flag.Parse()
	return subcommands.Execute(context.Background())
}

create.go

package main

import (
	"context"
	"flag"
	"fmt"
	"io/ioutil"
	"log"

	secretmanager "cloud.google.com/go/secretmanager/apiv1"
	secretmanagerpb "google.golang.org/genproto/googleapis/cloud/secretmanager/v1"

	"github.com/google/subcommands"
)

type createCmd struct {
	projectID, key, value, path string
}

func newCreateCmd() *createCmd {
	return &createCmd{}
}

func (*createCmd) Name() string {
	return "create"
}

func (*createCmd) Synopsis() string {
	return "create secret"
}

func (*createCmd) Usage() string {
	return `usage: create secret`
}

func (cmd *createCmd) SetFlags(f *flag.FlagSet) {
	f.StringVar(&cmd.projectID, "p", "", "project id")
	f.StringVar(&cmd.key, "k", "", "key")
	f.StringVar(&cmd.value, "v", "", "value")
	f.StringVar(&cmd.path, "f", "", "file path")
}

func (cmd *createCmd) Execute(ctx context.Context, f *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
	if cmd.projectID == "" || cmd.key == "" || (cmd.value == "" && cmd.path == "") {
		log.Println("need -p [gcp project id] -k [secret key] -v [secret value] or -f [secret file path]")
		return subcommands.ExitFailure
	}

	var client *secretmanager.Client
	{
		var err error
		client, err = secretmanager.NewClient(ctx)
		if err != nil {
			log.Fatalf("failed to setup client: %v", err)
		}
	}

	// Create the request to create the secret.
	createSecretReq := &secretmanagerpb.CreateSecretRequest{
		Parent:   fmt.Sprintf("projects/%s", cmd.projectID),
		SecretId: cmd.key,
		Secret: &secretmanagerpb.Secret{
			Replication: &secretmanagerpb.Replication{
				Replication: &secretmanagerpb.Replication_Automatic_{
					Automatic: &secretmanagerpb.Replication_Automatic{},
				},
			},
		},
	}

	var secret *secretmanagerpb.Secret
	{
		var err error
		secret, err = client.CreateSecret(ctx, createSecretReq)
		if err != nil {
			log.Fatalf("failed to create secret: %v", err)
		}
	}

	// Declare the payload to storage.
	var payload []byte
	if cmd.value != "" {
		payload = []byte(cmd.value)
	}
	if cmd.path != "" {
		ba, err := ioutil.ReadFile(cmd.path)
		if err != nil {
			log.Fatalf("failed to read secret file: %+v", err)
		}
		payload = ba
	}
	if payload == nil {
		log.Fatal("payload is nil")
	}

	// Build the request.
	addSecretVersionReq := &secretmanagerpb.AddSecretVersionRequest{
		Parent: secret.Name,
		Payload: &secretmanagerpb.SecretPayload{
			Data: payload,
		},
	}

	var accessRequest *secretmanagerpb.AccessSecretVersionRequest
	{
		// Call the API.
		version, err := client.AddSecretVersion(ctx, addSecretVersionReq)
		if err != nil {
			log.Fatalf("failed to add secret version: %v", err)
		}

		// Build the request.
		accessRequest = &secretmanagerpb.AccessSecretVersionRequest{
			Name: version.Name,
		}
	}

	// Call the API.
	result, err := client.AccessSecretVersion(ctx, accessRequest)
	if err != nil {
		log.Fatalf("failed to access secret version: %v", err)
	}

	// Print the secret payload.
	//
	// WARNING: Do not print the secret in a production environment - this
	// snippet is showing how to access the secret material.
	log.Printf("Plaintext: %s", result.Payload.Data)

	return subcommands.ExitSuccess
}

list.go

package main

import (
	"context"
	"flag"
	"fmt"
	"log"
	"strings"

	secretmanager "cloud.google.com/go/secretmanager/apiv1"
	"google.golang.org/api/iterator"
	secretmanagerpb "google.golang.org/genproto/googleapis/cloud/secretmanager/v1"

	"github.com/google/subcommands"
)

type listCmd struct {
	projectID string
}

func newListCmd() *listCmd {
	return &listCmd{}
}

func (*listCmd) Name() string {
	return "list"
}

func (*listCmd) Synopsis() string {
	return "list secrets"
}

func (*listCmd) Usage() string {
	return `usage: list secrets`
}

func (cmd *listCmd) SetFlags(f *flag.FlagSet) {
	f.StringVar(&cmd.projectID, "p", "", "project id")
}

func (cmd *listCmd) Execute(ctx context.Context, f *flag.FlagSet, _ ...interface{}) subcommands.ExitStatus {
	if cmd.projectID == "" {
		log.Println("need -p [gcp project id]")
		return subcommands.ExitFailure
	}

	client, err := secretmanager.NewClient(ctx)
	if err != nil {
		log.Printf("failed to create secretmanager client: %v", err)
		return subcommands.ExitFailure
	}

	// Build the request.
	req := &secretmanagerpb.ListSecretsRequest{
		Parent: fmt.Sprintf("projects/%s", cmd.projectID),
	}

	// Call the API.
	it := client.ListSecrets(ctx, req)
	for {
		resp, err := it.Next()
		if err == iterator.Done {
			break
		}

		if err != nil {
			log.Printf("failed to list secret versions: %v", err)
			return subcommands.ExitFailure
		}

		names := strings.Split(resp.Name, "/")
		reqName := fmt.Sprintf("projects/%s/secrets/%s/versions/%s", names[1], names[3], "latest")

		// Build the request.
		req := &secretmanagerpb.AccessSecretVersionRequest{
			Name: reqName,
		}

		// Call the API.
		result, err := client.AccessSecretVersion(ctx, req)
		if err != nil {
			log.Printf("failed to access secret version: %v", err)
			return subcommands.ExitFailure
		}

		log.Printf("Found secret %s ... got value: %s\n", resp.Name, string(result.Payload.Data))
	}
	return subcommands.ExitSuccess
}

Practice

Add Secret

$ go run ./*.go create -p XXXXXXXX -k rdb-host -v localhost
2020/10/11 23:03:33 Plaintext: localhost
$ go run ./*.go create -p XXXXXXXX -k rdb-port -v 12345
2020/10/11 23:04:05 Plaintext: 12345
$ go run ./*.go create -p XXXXXXXX -k rdb-user -v user1
2020/10/11 23:04:24 Plaintext: user1
$ go run ./*.go create -p XXXXXXXX -k rdb-pass -v pass1234
2020/10/11 23:04:47 Plaintext: pass1234

The part of XXXXXXXX is the ID of your GCP project.

Looking at the GCP console manager, it looks like this. screenshot-console.cloud.google.com-2020.10.11-23_05_07.png

View Secret list

$ go run ./*.go list -p fs-work-21
2020/10/11 23:09:34 Found secret projects/999999999999/secrets/rdb-host ... got value: localhost
2020/10/11 23:09:35 Found secret projects/999999999999/secrets/rdb-pass ... got value: pass1234
2020/10/11 23:09:35 Found secret projects/999999999999/secrets/rdb-port ... got value: 12345
2020/10/11 23:09:35 Found secret projects/999999999999/secrets/rdb-user ... got value: user1

Summary

What is the Secret information managed by Secret Manager used for, for example? I think there are various uses, but in my case, it's time to pass a DB password or something to the service posted on Cloud Run through environment variables. In the build shell, set env the DB password obtained from Secret Manager with the gcloud command. Specific method Yeah, I'll see you later.

Wrap reading and writing of GCP to Secret Manager with google subcommands