[PYTHON] Get JSON template of specific Resource in Resource Group (Azure CLI)

background

When building a system with Azure, if you prepare a JSON template for later, you can quickly build the same system (infrastructure part such as server, network). Depending on the size of the system, a single JSON template for the entire Resource Group can be quite large.

Therefore, we try to output a JSON template for each resource. This is quite annoying.

This time, the content is for Linux & Azure CLI users. Since it was made instantly, it is up to the readers to optimize it. (Throwing) I will improve it if I have time and spare time.

I was writing in Python, thinking that PowerShell would be easier.

Azure CLI command to run

Details will be described later, but first I will introduce what to use in the command. The --json option is required because the Azure CLI cannot reuse each value without JSON output.

  1. Get the list of resources in the resource group

    azure resource list --resource-group [RG Name] --tags [Key]=[Value] --json
    
  2. Get detailed information about Resource Provider

    azure provider show --namespace [Resource Provider] --json
    
  3. Get the details of Resource (purpose of this time)

    azure resource show --resource-id [Resource ID] --api-version [API Version] --json
    

Overall picture

Python


import os
import commands
import json

def getresourcejson():
    #Prepare a command to get only the resources of a specific tag
    resourceListCmd = 'azure resource list --resource-group [RG Name] --tags [Key]=[Value] --json'
    #Execute the command and get the output
    resourceList = commands.getoutput(resourceListCmd)
    #Objectize JSON output
    resourceListJson = json.loads(resourceList)
    #Analyze each resource one by one
    for resource in resourceListJson:
        #Get each resource type to get the API Version
        #This time,'[Resourece Provider]/[Resource Type]'Since it is, split with a slash and make each variable
        resourceTypeStr = str(resource['type']).split('/')
        resourceProv = resourceTypeStr[0]
        resourceType = resourceTypeStr[1]
        #Prepare a command to get the latest API Version from the obtained resource type
        resourceTypeListCmd = 'azure provider show --namespace ' + resourceProv + ' --json'
        #Execute the command and get the output
        resourceTypeList = commands.getoutput(resourceTypeListCmd)
        #Objectize JSON output
        resourceTypeListJson = json.loads(resourceTypeList)
        #I want the Resource Type under the Resource Provider, so I take it out.
        rTypes = resourceTypeListJson['resourceTypes']
        #Prepare a box of API Version
        apiVer = ''
        #Get API Version
        for rType in rTypes:
            #I'm not interested in Resource Types other than the one I want, so I filter
            if str(rType['resourceType']) == resourceType:
                #Get it because the beginning of apiVersions is the latest (like)
                #Some resources have the latest end, so it is recommended to check in advance.
                apiVer = str(rType['apiVersions'][0])
        #Define the output file name
        outFileName = str(resource['name']) + '.json'
        #Prepare a command to get the JSON of each resource
        resCmd = 'azure resource show --resource-id ' + str(resource['id']) + ' --api-version ' + apiVer + ' --json'
        #Execute a command
        res = commands.getoutput(resCmd)
        #Since the tab size is 2 for standard output, set it to 4.
        #Objectize JSON output
        resJson = json.loads(res)
        #Write a file with a tab size of 4
        with open(outFileName, 'w') as f:
            # sort_If you set keys to True, it will be difficult to see as an Azure JSON template, so set it to False.
            json.dump(resJson, f, sort_keys = False, indent = 4)
        #Display output completion
        print str('Complete output. File: ') + outFileName

if __name__ == '__main__':
    getresourcejson()

Commentary

1. Specify "tag"

One point is that the tag is specified by the Azure CLI command at the beginning.

Python


#Prepare a command to get only the resources of a specific tag
resourceListCmd = 'azure resource list --resource-group [RG Name] --tags [Key]=[Value] --json'

Most resources that would be useful to have a JSON template later can be tagged. If you don't tag it, you'll need to filter what you don't need in your JSON template.

There are other useful things to do with tags, so I highly recommend it.

Since you can specify more than one, you can handle things like wanting only the VMs in the DMZ, for example. Below is an example of tagging.

azure vm set --resource-group [RG Name] --name [VM Name] --tags "ResourceType=VM;Segment=DMZ"

2. Output of "azure resource list --json"

The information output by this command is limited. But it's enough to achieve this goal.

All you need is the following three.

  1. Resource ID

Required to get the JSON template for the target resource.

  1. Name

Used for the file name to save the JSON template of the target resource.

  1. Resource Type

Required to get the API Version required to get the JSON template of the target resource.

The output JSON is treated as an array because information such as the ID of each resource is formatted separated by commas. That's why I'm turning it in Foreach.

3. Get API Version

From here, I'm doing a lot of trial and error to get the API Version. Let me get it more easily, Microsoft. .. ..

3-1. Divide Resource Type into Provider and Type

To get the API Version, you have to pull the information from the Resource Provider. This is where the ʻazure provider show` command comes in.

Moreover, it is a habit that you can not pick up the list of API Version without adding the --json option. It's okay because it can't be reused in the code unless it's converted to JSON, but you want to check it first, right? You have to convert it to JSON to check it. Be careful.

ʻThe required parameter when executing the azure provider show command is --namespace`. Note that it is the Resource Provider that is passed here, not the Resource Type.

An error will occur with the Resource Type picked up in ʻazure resource list`. The resource provider is before the slash of this Resource Type, and the resource type is after the slash.

So let's split it. Since Resource Type will be used later, it is easier to identify if both are variable.

3-2. Narrow down the target Resource Type from the Resource Type list under the Resource Provider and obtain the API Version.

The JSON output of ʻazure provider show` is one object, but since there are multiple Resource Types in this, Foreach again.

The Resource Type split earlier is used here. Because this Resource Type is the "target Resource Type".

API Version is included in the target Resource Type, so extract it from here. However, note that there are multiple.

As far as I've checked, the first one is the latest, so I'm picking up the first API Version. Depending on the Resource Type, it may not be the beginning, so please correct it if necessary.

4. Get the final JSON template

Once you have the API Version, you can specify the Resource ID and execute ʻazure resource show`.

You can write to the file directly from the standard output of the --json option, but if you're curious about the indented tab size of 2, change the tab size with json.dump.

Sorting by Key is the de facto standard for general JSON, but in the case of Azure JSON template, it is easier to see the order specified by Azure side, so it is recommended not to sort.

Cooking JSON template

If you execute this Python script, a JSON template will be output as a file for each resource, so let's template the Resource Group according to the convention of Azure Resource Manager.

After creating a template, you can almost automate it by installing the middle on the VM with Ansible etc.

Azure PowerShell version will be available at a later date

As mentioned at the beginning, this time the method for the Azure CLI. I will make an Azure PowerShell version at a later date.

Postscript (2016/12/28 1st time) --Apology

I have noticed. JSON cannot be output with Azure PowerShell ...

I thought I should convert to-Json with a pipe, but when I actually tried it, the contents were quite different. Could you make a -Json option like Azure CLI?

So, at first I was trying to write the Azure PowerShell edition, but I will stop it because it will post incorrect information. I'm sorry for those who were expecting it.

Postscript (2016/12/28 2nd time) --Source optimization

It was a little regrettable to publish the dirty source as it was, so clean the source.

Python


import sys
import commands
import json

def ExecuteCmdlet(cmdlet):
    result = commands.getoutput(cmdlet)
    return json.loads(result)

def GetResourceList(rgName, tags):
    cmdlet = 'azure resource list --resource-group ' + rgName + ' --tags ' + tags + ' --json'
    return ExecuteCmdlet(cmdlet)

def GetResourceTypeList(provider):
    cmdlet = 'azure provider show --namespace ' + provider + ' --json'
    result = ExecuteCmdlet(cmdlet)
    return result['resourceTypes']

def GetResource(resourceId, apiVersion):
    cmdlet = 'azure resource show --resource-id ' + resourceId + ' --api-version ' + apiVersion + ' --json'
    return ExecuteCmdlet(cmdlet)

def GetApiVersion(typeList, target):
    result = ''
    for t in typeList:
        if str(t['resourceType']) == target:
            result = str(t['apiVersions'][0])
    return result

def GetResourceJson(resourceGroupName, tagName, tagValue):
    tags = tagName + '=' + tagValue
    resourceList = GetResourceList(resourceGroupName, tags)
    for resource in resourceList:
        resourceName = str(resource['name'])
        resourceId = str(resource['id'])
        resourceProvider = str(resource['type']).split('/')[0]
        targetResourceType = str(resource['type']).split('/')[1]
        resourceTypeList = GetResourceTypeList(resourceProvider)
        apiVersion = GetApiVersion(resourceTypeList, targetResourceType)
        outFileName = resourceName + '.json'
        resources = GetResource(resourceId, apiVersion)
        with open(outFileName, 'w') as f:
            json.dump(resources, f, sort_keys = False, indent = 4)
        print str('Complete output. File: ') + outFileName

if __name__ == '__main__':
    argvs = sys.argv
    if len(argvs) != 4:
        print 'Usage: python GetResourceJson.py [Resource Group Name] [Tag Name] [Tag Value]'
        exit()
    rgName = argvs[1]
    tagName = argvs[2]
    tagValue = argvs[3]
    GetResourceJson(rgName, tagName, tagValue)
    exit()

Recommended Posts

Get JSON template of specific Resource in Resource Group (Azure CLI)
Get the number of specific elements in a python list
Get all IP addresses of instances in the autoscaling group
Handling of JSON files in Python
Get the key for the second layer migration of JSON data in python
Get rid of DICOM images in Python