How to dynamically write iterative test cases using test / unit (Test :: Unit)

Code from theory

Since Ruby can generate dynamic code even for Class and Method, it can be written as follows.

ARR = [true, false, true]
Class.new(Test::Unit::TestCase) do
  ARR.each_with_index do |e, idx|
    define_method "test_#{idx}" do
      assert e
    end
  do
end

This is synonymous with writing as follows.

ARR = [true, false, true]
class AnyTestCase < Test::Unit::TestCase
  def test_0
    assert ARR[0]
  end

  def test_1
    assert ARR[1]
  end

  def test_2
    assert ARR[2]
  end
end

Test code is DAMP from DRY

It's often said that test code should focus on ** DAMP (Descriptive and Meaningful Phrases) ** rather than DRY (Do n’t Repeat Yourself).

• What does “DAMP not DRY” mean when talking about unit tests?
• Expected value of test code is written solidly by discarding DRY-What is the important role of test code? ~

I basically agree with this matter. The test code should be a "specification", and in that sense, the same expression appears many times in a "specification written in natural language", which is unavoidable as a result of prioritizing clarity. However, it goes without saying that balance is important in everything.

Now, the problem here is "verification processing for a large amount of data". For example, there is something like this

• Verification of consumption tax calculation logic. The number of product masters is 100,000 (^^ ;;
• Moving a blog. The number of entries is 1000 (tsu q`)

If you implement it honestly

Let's target moving blogs. If you honestly verify that the blog you're moving to has more than one byte of content inside the <article> tag, you'll see:

require 'test/unit'
require 'httpclient'
require 'nokogiri'
  
class WebTestCase < Test::Unit::TestCase
  def setup
    @c = HTTPClient.new
  end

  def test_0
    doc = Nokogiri::HTML(@c.get("https://example.com/entry/one").body)
    assert_compare 1, "<", doc.css('article').text.length
  end

  def test_1
    doc = Nokogiri::HTML(@c.get("https://example.com/entry/two").body)
    assert_compare 1, "<", doc.css('article').text.length
  end

  def test_2
    doc = Nokogiri::HTML(@c.get("https://example.com/entry/three").body)
    assert_compare 1, "<", doc.css('article').text.length
  end
end

It is subtle to write this up to def test_1000.

Repeated implementation with iterator

Since the verification condition (assert_compare) does not change, I can think of a plan to make the URL list an iterator (array) and test it repeatedly, but this is not recommended for the reason described later.

require 'test/unit'
require 'httpclient'
require 'nokogiri'

URLS = %w(
  https://example.com/entry/one
  https://example.com/entry/two
  https://example.com/entry/three
)

class WebTestCase < Test::Unit::TestCase
  def setup
    @c = HTTPClient.new
  end

  def test_content_length
    URLS.each do |url|
      doc = Nokogiri::HTML(@c.get(url).body)
      assert_compare 1, "<", doc.css('article').text.length
    end
  end
end

% w notation is convenient. Well, it looks good at first glance. However, the problem actually occurs when the assert fails. Below is the result of the execution, but ** I don't know which URL failed **.

$ bundle exec ruby test_using_array.rb
Started
F
=========================================================================================================================================================================================
test_using_array.rb:25:in `test_content_length'
test_using_array.rb:25:in `each'
     24:   def test_content_length
     25:     URLS.each do |url|
     26:       doc = Nokogiri::HTML(@c.get(url).body)
  => 27:       assert_compare 1, "<", doc.css('article').text.length
     28:     end
     29:   end
     30: end
test_using_array.rb:27:in `block in test_content_length'
Failure: test_content_length(WebTestCase):
  <1> < <0> should be true
  <1> was expected to be less than
  <0>.
=========================================================================================================================================================================================

Finished in 0.0074571 seconds.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1 tests, 3 assertions, 1 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
0% passed
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

This is because the unit of the test is each method (def test_content_length in the above example).

It's better to let Ruby create it dynamically than to generate code

For each iterator (array), we want to create a method for each test and test it in it. After all, is there no choice but to implement it honestly by copying? Is there no choice but to generate code with Hidemaru's macro? When I came to this idea, I would like you to remember the dynamic code generation that is Ruby's black magic (metaprogramming).

As mentioned at the beginning, Ruby can generate code dynamically. Class and Method are no exception. You can use it to dynamically create test methods one by one for the contents of the iterator.

URLS = %w(
  https://example.com/entry/one
  https://example.com/entry/two
  https://example.com/entry/three
)

Class.new(Test::Unit::TestCase) do
  def setup
    @c = HTTPClient.new
  end

  URLS.each_with_index do |url, idx|
    define_method "test_#{idx}" do
      doc = Nokogiri::HTML(@c.get(url).body)
      assert_compare 1, "<", doc.css('article').text.length
    end
  end
end

This code is equivalent to the following, which is the same as the straightforward implementation.

URLS = %w(
  https://example.com/entry/one
  https://example.com/entry/two
  https://example.com/entry/three
)

class AnyTestCase < Test::Unit::TestCase
  def setup
    @c = HTTPClient.new
  end

  def test_0
    doc = Nokogiri::HTML(@c.get(URLS[0]).body)
    assert_compare 1, "<", doc.css('article').text.length
  end

  def test_1
    doc = Nokogiri::HTML(@c.get(URLS[1]).body)
    assert_compare 1, "<", doc.css('article').text.length
  end

  def test_2
    doc = Nokogiri::HTML(@c.get(URLS[2]).body)
    assert_compare 1, "<", doc.css('article').text.length
  end
end

In the following example, the location of the failure is specified as Failure: test_2 ().

$ bundle exec ruby test_using_black_magic.rb
Loaded suite test_using_black_magic
Started
..F
=========================================================================================================================================================================================
test_using_black_magic.rb:27:in `block (3 levels) in <main>'
Failure: test_2():
  <1> < <0> should be true
  <1> was expected to be less than
  <0>.
=========================================================================================================================================================================================

Finished in 0.007288 seconds.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
3 tests, 3 assertions, 1 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
66.6667% passed
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

#each_with_index is 0 origin (beginning). Failure in test_2 means failure when the value is ʻARR [2] `, so you should verify that value.

It is also possible to specify with --name

If the order of the values stored in the iterator is guaranteed, it will also support specification with --name. For example, if you want to test only on the value of ʻARR [1] `, specify as follows.

$ bundle exec ruby test_multiple.rb --name test_1

Implementation points

Dynamic class generation with Class.new and dynamic method generation with define_method are the keys. If you are interested in this, please search for "Ruby Black Magic".

Be careful not to overdo it

This is valid when the assertion condition is uniform. On the contrary, if the assertion to be applied differs depending on the input value (= conditional branching occurs), it is better to implement it honestly as DAMP.

• In the first place, the test case itself should be reviewed when a conditional branch occurs in the test case.

Afterword

I'm an amateur in this field, so I don't know if this suits me.

EoT