[PYTHON] Web application performance measurement tool Funkload ...

Funkload

About the performance measurement tool of the web application (which also serves as the load on the server side), Apache Bench and [Siege](http: There are //www.joedog.org/siege-home/) and JMeter, I was wondering if there was anything else, and I was looking into Stack Overflow I found something called Funkload.

It seems to be made by Python. Then I tried to investigate it, so I tried to play with it.

Operating environment maintenance

As an operating environment, my own environment is as follows.

• Mac OS X 10.8
• Python 2.7.5

Funkload seems to support Python 2.x series, so select the latest version of 2.x series (as of June 2013). Refer to How to install on Mac OS X in the Funkload documentation. ... Apparently it's enough to have gnuplot. Let's put it in with homebrew.

> brew install gnuplot

Create an environment that can get dirty

Create an environment that can get dirty with virtualenv.

With Python 2.7.5 on the system (installed with homebrew etc.) If you haven't installed it with a Python processing system that uses easy_install, Download and install the distribute package, and in the following form install easy_install, pip

> curl -O http://python-distribute.org/distribute_setup.py
> python distribute_setup.py
> rehash
> easy_install pip
> rehash
> pip install virtualenv

Install Funkload

If virtualenv is included, in the following form Install Funkload using the pip command.

> mkdir ~/Sandbox/Funkload
> cd ~/Sandbox/Funkload
> virtualenv .venv
> source .venv/bin/activate
> pip install funkload

Test Funkload for now

Prepare a web application for testing for the time being

I can't help without a web app for testing, so Use the skelton creation function of express (node.js) to support the test application. If the node environment is not built, it is recommended to use nodebrew ...

If express is not included

Install it as npm_modules in a global location. Well npm install.

> npm install -g express

Test app creation

Easy with skelton creation function (Scaffold was not officially written)

> express webserver
> cd webserver && npm install
> node app

This is the end of starting the test application.

Creating a test case

While looking at the FunkLoad test case Funkload Tutorial I will write it. You can create a test case template with commands. It is good.

> fl-install-demo
> cd funkload-demo/simple

For the created test case, rewrite the main section of the Simple.conf file as follows.

[main]
title=Express skelton page test
description=express.js skelton application testing a root page.
# the server url to test
url=http://localhost:3000/

Launch Funkload to test access to the app

Let's execute the following command

> fl-run-test -dv test_Simple.py

The test case for the web application seems to be fine.

Actually tested with Funkload

Let's actually test it.

> fl-run-bench -c 1:10:20 test_Simple.py Simple.test_simple

The -c option is a thread count option for launch tests. In the above example, the test is performed with 1 thread, 10 threads, and 20 threads.

The actual test was completed successfully.

Generate test report

Convert the test report from an xml file to an html file with the following command.

> fl-build-report --html simple-bench.xml

A directory is created and under it The report is generated in the form of an html file. Let's see it in action ...

> cd test_simple-20130601T104942
> open index.html

Oh, something is being output ... page stats is the number of pages that were successfully acquired, I don't know E or G, but there was a proper explanation at the bottom of the output html file.

If the score is 0.94 or higher, it seems that a blue stick is standing because it is E. It's simple, the number of expected users is as small as 20, and there is usually no problem with the response speed. Will that happen?

I want to write a slightly complicated application and test it

So write a slightly more complex application and test it I will prepare a scenario.

The complex applications are prepared below. https://github.com/futoase/bottle-application-sample It simply has a form and displays it when it is filled.

I have already registered the test scenario / application prepared on github. The following repositories can be checked by doing git clone && git submodule update --init.

https://github.com/futoase/funkload-sample

Write a scenario

As a condition, the target Web application has the following functions.

--Receive user_name, comment by POST method in / comment and add to sqlite3 DB file -Display the content of the comment posted by / (index)

It's simple. Let's write a scenario to check the above conditions

-Access / (Get resources by GET) --POST the contents of user_name and comment to / comment and add them to the record

As a point that you can not go unless you are careful

--The file name, class name, and configuration file name used in the test must be the same. --The main section written in the configuration file (conf) cannot be omitted.

I can't omit the content written in the main section, Because it is hard-coded in the source code ...

Scenario file

I'm writing a scenario in CommentBoard.py. The test scenario is based on the Funkload documentation page.

CommentBoard.py

# -*- coding:utf-8 -*-

import unittest
import random
import json
from funkload.FunkLoadTestCase import FunkLoadTestCase

with open('conf/user_names.json') as f:
  USER_NAMES = json.loads(f.read())

with open('conf/comments.json') as f:
  COMMENTS = json.loads(f.read())

class CommentBoard(FunkLoadTestCase):
  def setUp(self):
    self.target_url = self.conf_get('main', 'url')
    self.try_count = self.conf_getInt('comment_board_setting', 'try_count')

  def test_index(self):
    target_url = self.target_url
    try_count = self.try_count

    for i in range(try_count):
      self.get(target_url, description='Get comment board index.') 

  def test_posts(self):
    target_url = self.target_url + 'comment'
    try_count = self.try_count

    user_name = random.choice(USER_NAMES)
    comment = random.choice(COMMENTS)

    self.post(target_url,
      params=[
        ('user_name', user_name),
        ('comment', comment)
      ],
      description='Comment as {user_name}'.format(user_name=user_name)
    )

if __name__ == '__main__':
  unittest.main()

--Test scenario where test_index is / (index) --Test_posts is a test scenario for / comment

Will be.

Configuration file for use in scenarios

The configuration file is written in CommentBoard.conf.

[main]
title=Comment board test
description=Comment board test
url=http://localhost:3000/

[index]
title=Comment board index tests
description=Comment board index

[comments]
title=Comment board posts tests
description=Comment board posts

[comment_board_setting]
try_count=30

[credential]
log_to = console file
log_path = comment-board.log
result_path = comment-board-test.xml
ok_codes =  200:301:302
sleep_time_min = 0
sleep_time_max = 0

[bench]
cycles = 50:75:100:125
duration = 10
startup_delay = 0.01
sleep_time = 0.01
cycle_time = 1
log_to =
log_path = comment-board-bench.log
result_path = comment-board-bench.xml
ok_codes =  200:301:302
sleep_time_min = 0
sleep_time_max = 0.5

[distribute]
log_path = log-distributed
funkload_location=http://pypi.python.org/packages/source/f/funkload/funkload-1.16.1.tar.gz

I haven't done anything special about this.

Test run

Before running the test, you have to launch the application to be tested, so Let's start up as follows.

> git clone git://github.com/futoase/funkload-sample.git
> git checkout -b v1.0.0 v1.0.0
> cd funkload-sample
> git submodule update --init
> virtualenv .venv
> source .venv/bin/activate
> pip install -r requiements.txt
> cd bottle-application-sample
> python migrate.py
> python app.py

After starting, create a new session with tmux as appropriate Run the Funkload test case.

cd funkload-sample
> fl-run-bench -c 10 CommentBoard.py CommentBoard.test_index
> fl-build-report --html comment-board-bench.xml
> cd test_index-20130604T175903
> open index.html
> cd ..
> fl-run-bench -c 10 CommentBoard.py CommentBoard.test_posts
> cd -test_posts-20130604T180037
> open index.html

Check the state of the test Web application that is running.

You can post properly in the form specified in the test. The result is also output.

Try using

It's more gorgeous than Apache Bench, but it might be nice to "see trends". You can make a self-judgment by looking at the transition of how to apply the load.

However, there is little information in Japanese, so There are no indicators such as the results measured by others, I wondered if it would be difficult to use it at work unless I was in a position to take responsibility. I think it can be used as long as it is judged as a benchmark, To make a "slow" decision, you have to write and build your own "slow" and "fast" implementations.

Apache Bench is simple, so the application server load on the reference system such as GET Since you can check it, Socci has more operational know-how, and if you search the net, it may be the result of performance. I wonder if it's good ...

The tool depends on the person.