[PYTHON] How to communicate asynchronously only with Touch Designer

Introduction

TouchDesigner (TD) can execute python scripts, If processing such as HTTP communication takes time, it will freeze a little until the processing is completed. At this time, ** the timeline of the entire TD will also stop and the drawing will not be updated **, so To prevent this, we decided to communicate asynchronously.

If you just want to get the value asynchronously, you can use node etc. in addition to TD, but Due to the circumstances of the environment, I wanted to complete it with only TD, so I considered the following method.

--Use Web DAT --Use the threading module

I will explain each of these.

Sample code

This sample is uploaded below. https://github.com/genkitoyama/TD_with_asynchronous_communication

Use Web DAT

This is the easiest. With the parameter ʻAsynchronous Fetchturned on, Enter the URL and pulse theFetch` button, The TD timeline doesn't stop and communicates behind the scenes (maybe curl?). Reference: https://docs.derivative.ca/Web_DAT

(In the above example, API for finding the altitude of the Geographical Survey Institute is used.)

In the case of GET, it is OK if you enter it directly in the URL, In the case of POST, it is OK if you pulse the Submit button with the table of the data you want to send connected to the upper inlet ʻInput 0`. (By the way, you can put a custom HTTP header in the lower inlet)

Then, according to the format of the returned one, pull the required value using the XML DAT or json module, and you're done.

Reference: Parse JSON to DAT Table with TouchDesigner

Use the threading module

threading module Is a module that performs parallel processing with multiple threads. It is included as standard in python3.5 inside TD.

By putting the function you want to execute in another thread and the argument of that function in the argument of threading.Thread (), it will be executed in another thread.

Here, requests is used as the module for HTTP communication. requests is also included as standard in python inside TD. (There are several other modules that communicate with HTTP, but I feel that you can use whatever you like.)

import threading
import requests

class HttpRequest():
	def __init__(self):
		self.url = op('URL').text
		self.city_id = op('CITY_ID').text
	
    #Functions you want to execute asynchronously
	def request(self):
		response = requests.get(self.url, params={'city':self.city_id})
		
	def start_request(self):
		myThread = threading.Thread(target=self.request)
		myThread.start()

This is a great reference to Article by Dr. Matthew Ragan.

Disadvantages and countermeasures

The threading module is also very useful, but it has some drawbacks.

--Cannot detect when the process in another thread ended --Cannot refer to TD operator while processing another thread

Unable to detect when processing in another thread ended

I can send a stop command from the main thread, It is not possible to detect when the processing in another thread is completed. You can use the join () method to wait for the end of another thread, During that time, the main thread will stop, so the TD will eventually stop. ..

Therefore, it is not easy to do something like "execute this process when the value is returned by hitting the URL in another thread".

Cannot refer to TD operator while processing another thread

This is natural if you think about it, but since the process is run in another thread, Unable to access the operator that resides in the TD's main thread.

Therefore, it is not possible to "parse the result returned by hitting the URL in another thread to Table DAT as it is in the same function ".

(By the way, if you do that, you will get a dialog like this)

Countermeasures

So, it's quite a skill, but I decided to use Timer CHOP to monitor whether the value is returned (≒ whether the processing of another thread is completed) ** at regular intervals.

I would like you to see the sample for details, but roughly speaking, the flow is as follows.

1. Write the process to store the obtained value in the member variable in the function executed in another thread.
2. Check if the variable contains a value in ʻonDone ()of theTimer CHOP` callback.
3. If it is, the timer will end and proceed to another process.
4. If not, restart the timer

In the above example, livedoor's Weather Hacks is used. With this API, the result will be returned in an instant, so even if you do it this way, it will not be very delicious, but if it is the original API you used before, it will take 7 to 8 seconds to return due to circumstances. So I devised this method.

python tried to create RequestManager class and HttpRequest class as follows.

import http_request
import json

class RequestManager():
	def __init__(self):
		self.http = http_request.HttpRequest()
		self.can_start_request = True			#May I make a request
		self.max_confirmation_count = 10		#Maximum confirmation count
		self.current_confirmation_count = 0		#Current confirmation count
		self.timer = op('timer1')
	
	def init_timer(self):
		print('init http request timer')
		
	def start_timer(self):
		#If you can make a request, throw a request, if not, wait
		if self.can_start_request == True:
			print('start http request timer')
			self.http.start_request()
		else:
			print('restart timer')
			
	def done_timer(self):
		print('done http request timer')
		#Check if the value is returned
		output = self.http.get_data()
		#If there is no value, increase the confirmation count and try again
		if output is None:
			print('http response is none')
			self.can_start_request = False
			self.current_confirmation_count += 1
			#When the confirmation count reaches the maximum, it is judged that communication has failed.
			if self.current_confirmation_count >= self.max_confirmation_count:
				print('http out of request')
				self.timer.par.initialize.pulse()
				self.http = http_request.HttpRequest()
				self.current_confirmation_count = 0
			else:
				self.timer.par.start.pulse()
		else:
		#If there is a value, json is parsed and stored
			self.can_start_request = True
			out_json = json.loads(output)
			op('out_json')[0,0].val = out_json['forecasts'][0]
			
			self.current_confirmation_count = 0
			self.timer.par.initialize.pulse()

import threading
import requests

class HttpRequest():
	def __init__(self):
		self.url = op('URL').text
		self.city_id = op('CITY_ID').text
		self.out_data = None
		print('http request init')
	
	def request(self):
		get_data = {'city':self.city_id}
		response = requests.get(self.url, params=get_data)
		self.out_data = response.text
		print(self.out_data)
		
	def start_request(self):
		self.out_data = None
		myThread = threading.Thread(target=self.request)
		myThread.start()
		
	def get_data(self):
		return self.out_data

in conclusion

――We have summarized two types of asynchronous communication methods using only TouchDesigner. ――Because I'm a python beginner, I would be grateful if you could comment on the last part of the brute force, especially if you have any ideas for a better method ...!

bonus

――This time, I used TouchPlayer to run each on multiple terminals, but since I can't see the Textport with TouchPlayer, I used the method of spitting out the log to an external file and checking it if an error occurred. -(There is also an option to open TouchDesigner as ReadOnly.) ――At that time, loguru was very convenient as a module to spit out logs, so I recommend it.

Referenced links

• https://docs.derivative.ca/Web_DAT
• https://matthewragan.com/2018/05/14/touchdesigner-multi-threading/
• https://qiita.com/tchnkmr/items/b05f321fa315bbce4f77
• https://qiita.com/shoch0922/items/f273f785221773c51277