[PYTHON] Get and create nodes added and updated in the new version

Introduction

This article is the 10th day article of ** Houdini Advent Calender 2019 **.

** Houdini 18.0 ** was released at the end of last month!

Many people are already using it, but the node introduced in ** New Feature Movie ** Besides, it seems that minor updates have been made.

So, in this article, I will show you how to get and create nodes added and updated in Houdini 18.0.

Definition of added and updated nodes

Consider the definitions of additions and updates to find out. The following two are defined.

-** Things that only exist in Houdini 18.0 compared to all the nodes in Houdini 17.5 ** -** Node exists but new parameters are added **

Based on this condition, I would like to create all the added and updated nodes in Python like a network editor. By the way, the versions to be compared are ** 17.5.391 ** and ** 18.0.287 **.

Script execution result

First, see the result of running the script.

When you run the script, a window will appear where you can choose which version to compare, and when you choose, the node comparison process will run. When the processing is completed, ** NewNodes ** will be created for the node with the added node itself, and ** Subnet ** will be created for the node with the added parameter ** NewParmNodes **, and the categories such as ** Sop ** will be created. There is a node for each.

** Sop node added in 18.0 **

By the way, the added parameter contains an expression called ** constant () **, so if you select ** Parameters with Non-Default Values ** from the parameter filter, only the added parameter can be displayed. I will.

The script can be downloaded and used from the ** GitHub ** repository below.

• Houdini Create Update Node

List of added and updated nodes

At first, I thought about listing all the added and updated nodes, but there were too many, so I summarized the number of added and updated nodes by category.

Number of nodes that only exist in Houdini 18.0

A total of ** 152 ** nodes have been added and updated.

Not only the newly added ** LOP **, but also ** SOP ** alone has added and updated ** 62 ** nodes.

Now let's look at the number of nodes with parameters added.

Nodes with parameters added and number of parameters added

Here, parameters have been added to ** 187 ** nodes in total, and ** 1054 ** parameters have been added in terms of the number of parameters.

Looking at this, ** Lop ** was in as of ** Houdini 17.5 **. And it seems that not only ** LOP ** and ** Sop ** but also ** Top ** and ** Dop ** have been updated considerably.

I don't feel like chasing anymore.

CSV data

For those who want to see the list of node names and parameter names, we have uploaded the CSV file below.

Nodes that only exist in Houdini 18.0

-Download CSV

--The figure opened in Excel. Each node is divided by category, node name, and node label.

Node with added parameters

-Download CSV

--The figure opened in Excel. Each parameter is divided by category, node name, node label, parameter name, and parameter label.

How to get added and updated nodes

The following is an explanation of Python. It is difficult to write everything, so I will explain only the key points.

Get the version of Houdini you have installed

First, I want to specify the version I want to compare, so I get all the versions of Houdini that I have installed.

-** Get installation directory ** Houdini registers the installation directory in an environment variable called ** HFS ** at startup, so get it with the ** os ** module.

import os
hfs = os.getenv('HFS')
#C:/PROGRA~1/SIDEEF~1/HOUDIN~1.287

-** Get all versions ** To get all versions from here, go up one directory and get all the folder names in that folder.

root = os.path.dirname(hfs)
versions = os.listdir(root)
#['Houdini 16.5.268', 'Houdini 17.0.376', 'Houdini 17.5.258', 'Houdini 17.5.360','Houdini 17.5.391', 'Houdini 18.0.287']

-** Omit the version of Houdini that is running ** Now that I have all the versions, I don't need the version of Houdini that is running, so I'll omit it. Originally, it can be omitted from the path obtained by ** HFS **, but ** HFS ** is returned in abbreviated form like HOUDIN ~ 1.287, so I omitted it by the following method.

import hou
current_version = hou.applicationVersionString()
versions.remove('Houdini ' + current_version)

-** Display the acquired version on the UI ** Display the acquired version on ** UI ** so that the user can select it. ** UI ** is generally made with ** PySide , but it is troublesome, so [ hou.ui **](https://www.sidefx.com/ja/docs/houdini/hom/ I used the ** selectFromList ** function of the hou / ui.html) module.

sel_version = hou.ui.selectFromList(
    versions, exclusive=True, title='Select Compare Version',
    column_header='Versions', width=240, height=240
)
if not sel_version:
    return
version = versions[sel_version[0]]
version_path = '{}/{}'.format(root, version)
#C:/PROGRA~1/SIDEEF~1/Houdini 17.5.391

Get all node types

You need to get all the node types for comparison.

To get all node types, use ** hou.nodeTypeCategories () ** to key each category. A dictionary with the ** hou.NodeTypeCategory ** object in the name and value will be returned. And the ** hou.NodeTypeCategory ** object ** nodeTypes ** By executing the .sidefx.com/ja/docs/houdini/hom/hou/NodeTypeCategory.html#nodeTypes) function, the key is the node type name and the value is [** hou.NodeType **](https: // www.sidefx.com/en/docs/houdini/hom/hou/NodeType.html) A dictionary with objects will be returned.

For example, to get all the node type names of SOP, execute the following code.

import hou
categories = hou.nodeTypeCategories()
sop_category = categories['Sop']
sop_data = sop_category.nodeTypes()
sop_nodes = sop_data.keys()

Get all parameters

To get all the parameters ** hou.NodeType ** Object ** parmTemplates () ** By executing the function, basically all parameters ** hou.parmTemplate ** You can get the object. However, since the parameters contained in ** Multiparm Block ** are not included, items with parameters such as ** Group Promote ** contained in ** Multiparm Block ** cannot be acquired.

To get the parameters included in ** Multiparm Block **, you can get them by using the recursive function as shown below.

def get_all_parm_templates(all_parms, node_type):
    parms = node_type.parmTemplates()
    for parm in parms:
        if parm.type() == hou.parmTemplateType.Folder:
            get_all_parm_templates(all_parms, parm)
        elif parm.type() != hou.parmTemplateType.FolderSet:
            all_parms.append(parm)
    return all_parms

Get all node types and parameters

Based on the above, get all node types and parameters.

# -*- coding: utf-8 -*-
import hou

def get_all_parm_templates(all_parms, node_type):
    parms = node_type.parmTemplates()
    for parm in parms:
        if parm.type() == hou.parmTemplateType.Folder:
            get_all_parm_templates(all_parms, parm)
        elif parm.type() != hou.parmTemplateType.FolderSet:
            all_parms.append(parm)
    return all_parms

def main():
    node_data = {}
    categories = hou.nodeTypeCategories()
    for category_name, category in categories.items():
        category_data = []
        nodes = category.nodeTypes()
        for node_name, node_type in nodes.items():
            node_info = {}
            node_info['node_name'] = node_name
            node_info['node_label'] = node_type.description()
            all_parms = get_all_parm_templates([], node_type)
            node_info['parms'] = [parm.name() for parm in all_parms]
            category_data.append(node_info)
        node_data[category_name] = category_data

    return node_data

When the above code is executed, a dictionary with node names, node labels, and parameter names will be returned for each category as shown below (only the PolySplitSop part is displayed).

"Sop": [
    {
        "node_label": "PolySplit", 
        "parms": [
            "splitloc", 
            "pathtype", 
            "override", 
            "newt", 
            "updatenorms", 
            "close", 
            "tolerance"
        ], 
        "node_name": "polysplit"
    }, 

However, even if you execute this, you can only get the node information of the version that is currently running, but not the node information of the specified version.

Get the node information of the specified version

Use ** Hython ** to get the node information for the specified version. ** Hython ** is a ** Python shell ** located in ** $ HFS / bin **, which exists for each version. Since the ** hou ** module is automatically loaded at startup, you can execute Houdini's own processing without starting ** Houdini. ** **

To run the script on the specified version of ** Hython **, save the above code in a **. Py ** file and use ** subprocess ** to specify it as an argument to ** Hython **. I will.

import subprocess
from subprocess import PIPE

hython = 'Path to Hython'
script = 'The path of the script to execute'
p = subprocess.Popen([hython, script], shell=True, stdout=PIPE, stderr=PIPE)
#Get the return value from the script
stdout, stderr = p.communicate()
#Since the returned value is a string, convert it to a dictionary with eval
node_data = eval(stdout)

However, when I did this, nothing was returned to ** stdout **, and the following string was returned to ** stderr **.

'EnvControl: HOUDINI_USER_PREF_DIR missing __HVER__, ignored.\r\nTraceback (most rec
ent call last):\n  File "<string>", line 8, in <module>\n  File "C:/PROGRA~1/SIDEEF~
1/HOUDIN~1.287/houdini/python2.7libs\\hou.py", line 19, in <module>\n    import _hou
\nImportError: DLL load failed: \x8ew\x92\xe8\x82\xb3\x82\xea\x82\xbd\x83v\x83\x8d\x
83V\x81[\x83W\x83\x83\x82\xaa\x8c\xa9\x82\xc2\x82\xa9\x82\xe8\x82\xdc\x82\xb9\x82\xf
1\x81B\nTraceback (most recent call last):\r\n  File "D:\\create_update_node\\get_node_data.py", lin
e 2, in <module>\r\n    import hou\r\n  File "C:/PROGRA~1/SIDEEF~1/HOUDIN~1.287/houd
ini/python2.7libs\\hou.py", line 19, in <module>\r\n    import _hou\r\nImportError: 
DLL load failed: \x8ew\x92\xe8\x82\xb3\x82\xea\x82\xbd\x83v\x83\x8d\x83V\x81[\x83W\x
83\x83\x82\xaa\x8c\xa9\x82\xc2\x82\xa9\x82\xe8\x82\xdc\x82\xb9\x82\xf1\x81B\r\n'

Looking at the error, it seems that the ** hou ** module import has failed.

Cause of error

The reason I get the error is that ** Hython ** goes to ** $ HFS ** and ** $ HFS / houdini / python2.7libs ** when importing the ** hou ** module. In other words, if you execute ** Hython ** from ** 18.0 **, only ** HFS ** and ** PYTHONPATH ** for ** 18.0 ** will be recognized, so you will get an import error.

Solution

To solve this, you need to run the code below to change the settings for that version before running ** Hython ** (for safety, restore the original settings after running Hython). ..

import sys

#Overwrite with the path of the comparison version that obtained HFS
os.putenv('HFS') = version_path
#Since the DLL is also loaded when the hou module is imported, the PATH environment variable is also rewritten.
path = '{}/bin;{}'.format(version_path, os.getenv('PATH'))
os.putenv('PATH', path)

Whole code

The whole code looks like this (it's long so it's folded). Script execution calls the main function.

# -*- coding: utf-8 -*-
import hou
import os
import subprocess
from subprocess import PIPE

from .get_node_data import get_all_parm_templates

def get_compare_version(hfs):
    version_root = os.path.dirname(hfs)
    versions = os.listdir(version_root)
    current_version = 'Houdini ' + hou.applicationVersionString()
    if current_version in versions:
        versions.remove(current_version)
    #UI option dictionary
    kwargs = {
        'exclusive': True,
        'title': 'Select Compare Version',
        'column_header': 'Versions',
        'width': 240,
        'height': 240
    }
    #Show list view for selecting version
    sel_version = hou.ui.selectFromList(versions, **kwargs)
    if not sel_version:
        return
    version = versions[sel_version[0]]
    return version

def get_env_from_version(version, hfs, pref_dir):
    old_hfs = '{}/{}'.format(os.path.dirname(hfs), version)
    old_pref_dir = '{}/{}'.format(
        os.path.dirname(pref_dir),
        '.'.join(version.replace('Houdini ', 'houdini').split('.')[:2])
    )
    return old_hfs, old_pref_dir

def set_base_env(path, hfs, pref_dir):
    #Set environment variables and Python path
    os.putenv('PATH', path)
    os.putenv('HFS', hfs)
    os.putenv('HOUDINI_USER_PREF_DIR', pref_dir)

def get_old_node_data(old_hfs, old_pref_dir):
    script_root = os.path.dirname(__file__)
    script = os.path.normpath(script_root + "/get_node_data.py")
    hython = os.path.normpath(old_hfs + '/bin/hython.exe')
    #Pass the required environment variables and Python path before throwing to hython
    path = '{}/bin;{}'.format(old_hfs, os.getenv('PATH'))
    set_base_env(path, old_hfs, old_pref_dir)
    #Run the script with hython
    p = subprocess.Popen([hython, script], shell=True, stdout=PIPE, stderr=PIPE)
    #Get the return value from the script
    stdout, stderr = p.communicate()
    if stderr:
        hou.ui.displayMessage('Script Error', severity=hou.severityType.Error)
        return
    #Since the returned value is a string, convert it to a dictionary with eval
    old_node_data = eval(stdout)
    return old_node_data

def get_node_info(node_name, node_label):
    node_info = {}
    node_info['Node Name'] = node_name
    node_info['Node Label'] = node_label
    return node_info

def compare(old_node_data):
    new_node_data = {}
    new_parm_node_data = {}
    categories = hou.nodeTypeCategories()
    for category, type_category in categories.items():
        new_nodes = []
        new_parm_nodes = []
        nodes = type_category.nodeTypes()
        old_nodes = old_node_data.get(category)
        #What to do if the category itself does not exist
        if not old_nodes:
            for node_name, node_type in sorted(nodes.items()):
                node_label = node_type.description()
                node_info = get_node_info(node_name, node_label)
                new_nodes.append(node_info)
            if new_nodes:
                new_node_data[category] = new_nodes
            continue
        #If the category exists
        old_node_names = [node_info['node_name'] for node_info in old_nodes]
        for node_name, node_type in sorted(nodes.items()):
            node_label = node_type.description()
            node_info = get_node_info(node_name, node_label)
            if node_name in old_node_names:
                all_parms = get_all_parm_templates([], node_type)
                index = old_node_names.index(node_name)
                parm_sets = set(old_nodes[index]['parms'])
                new_parms = [parm.name() for parm in all_parms if not parm.name() in parm_sets]
                if new_parms:
                    node_info['parms'] = new_parms
                    new_parm_nodes.append(node_info)
            else:
                new_nodes.append(node_info)
        if new_nodes:
            new_node_data[category] = new_nodes
        if new_parm_nodes:
            new_parm_node_data[category] = new_parm_nodes
    return new_node_data, new_parm_node_data

def create_nodes(node_data, root_node):
    for category, nodes in node_data.items():
        #Create a parent node according to the category for creating a node
        if category == 'Object':
            parent_node = root_node.createNode('subnet', category)
        elif category == 'Driver':
            parent_node = root_node.createNode('ropnet', category)
        elif category == 'Sop':
            parent_node = root_node.createNode('geo', category)
        elif category == 'Vop':
            parent_node = root_node.createNode('matnet', category)
        elif not 'Net' in category:
            try:
                parent_node = root_node.createNode(
                    category.lower() + 'net', category, run_init_scripts=False)
            except:
                continue
        else:
            parent_node = root_node.createNode(category.lower(), category)
        #Creating a node
        for node_info in nodes:
            #Get the name of the node and create it
            node_name = node_info['Node Name']
            try:
                new_node = parent_node.createNode(node_name)
            except:
                continue
            #Get parameters
            parms = node_info.get('parms')
            if not parms:
                continue
            #Set expression in parameter
            for parm_name in parms:
                try:
                    if parm_name[-1] == '#':
                        parm_name = parm_name[:-1] + '1'
                    parm_tuple = new_node.parmTuple(parm_name)
                    if not parm_tuple:
                        continue
                    for parm in parm_tuple:
                        parm.setExpression('constant()')
                except:
                    pass
        #Node organization
        parent_node.layoutChildren()
    root_node.layoutChildren()

def create_new_nodes(new_node_data):
    root_node = hou.node('/obj').createNode('subnet', 'NewNodes')
    create_nodes(new_node_data, root_node)

def create_new_parm_nodes(new_parm_node_data):
    root_node = hou.node('/obj').createNode('subnet', 'NewParmNodes')
    create_nodes(new_parm_node_data, root_node)

def main():
    hfs = os.getenv('HFS')
    #Get the version to compare
    version = get_compare_version(hfs)
    if not version:
        return
    pref_dir = os.getenv('HOUDINI_USER_PREF_DIR')
    path = os.getenv('PATH')
    #Get the environment variables of the version to compare
    old_hfs, old_pref_dir = get_env_from_version(
        version, hfs, pref_dir)
    #Get node information of the version to be compared
    old_node_data = get_old_node_data(old_hfs, old_pref_dir)
    if not old_node_data:
        return
    #Restore the environment variables set for hython
    set_base_env(path, hfs, pref_dir)
    #Get node information compared to nodes in the current version
    new_node_data, new_parm_node_data = compare(old_node_data)
    #Create a node that exists only in the current version
    create_new_nodes(new_node_data)
    #Create a node with parameters added in the current version
    create_new_parm_nodes(new_parm_node_data)
    #Organize nodes
    hou.node('/obj').layoutChildren()

Summary

This is how to check the nodes added and updated in the new version. By using this script, you can check the updated node as soon as a new version comes out!

If you have any mistakes or unclear points in the article, I would appreciate it if you could write them down. Until the end Thank you for reading.