When I make a new program, I often experiment with various parameters and measure what kind of parameter and how long it takes to calculate.
In order to measure this, it is necessary to extract and collect time from the output of each result. You can do this manually if you have a few, but if you have a lot of parameter combinations, you can't do it by hand, so use a script. To do this, you have to write scripts for parameter generation, result parsing, and result collection. I feel that such processing is often used in common, so I found a maf in the place where I was wondering if I could make a library and tried using it.
Since there was no sample of calculation time measurement in the current maf sample, I made it.
For details, it is best to read here (http://research.preferred.jp/2013/12/maf/), but in simple terms, it is made by extending the build system written in python called waf. It is a tool for supporting experiments.
With this, you can easily enumerate the combinations of parameters and easily summarize the experimental results. Wonderful!
Like waf, maf needs to write Makefile in the make command called wscript. wscript etc. are in the following. https://github.com/shu65/maf_measure_computing_time
In this wscript, "sleep 0", "sleep 1", "sleep 2" are executed 3 times each, the time is measured using the time command, and the average calculation time is calculated for each parameter, which is a very simple sample. It has become.
The execution method after git clone is as follows.
python
cd maf_measure_computing_time
./waf configure
./waf experiment -j 1
Note that without the final "-j 1", multiple combinations of parameter experiments cannot be performed simultaneously and accurate calculation time cannot be measured.
Currently, I don't want to execute in parallel only for measuring time, but I can't specify it with wscript, so I'd like to improve this area and send a Pull Request in the future ...
Recommended Posts