[PYTHON] Generate error correction code to restore data corruption with zfec library

Using the zfec library, I tried to generate an error correction code that can be restored even if some data is damaged.

Image to restore data corruption

The zfec library uses the RAID-5 algorithm to restore data corruption. The Raid-5 algorithm can be easily understood by reading here. It's a simple mechanism using XOR, so you can understand it immediately.

Features of zfec library

·early ・ There are APIs for python, c, Haskel ・ Can handle char arrays

https://pypi.python.org/pypi/zfec

Installation

pip install zfec

Generate error correction code from 2 blocks and verify

2DataBlock

# -*- coding: utf-8 -*-
from __future__ import absolute_import, unicode_literals
import zfec

#Error detection correction
data_block1 = str('aiueo')
data_block2 = str('kakik')

#Error detection code from data blocks 1 and 2:Generate 3
_, _, data_block3 = zfec.Encoder(2, 3).encode([data_block1, data_block2])
print 'data block 3 is... {}'.format(data_block3)
# >>> data block 3 is... uyI}g

#Data block 1 and error detection code:Restore 3 to 2
_, repaired_data_block2 = zfec.Decoder(2, 3).decode([data_block1, data_block3, ], [0, 2])
assert data_block2 == repaired_data_block2

#Data block 2 and error detection code:Restore 3 to 1
repaired_data_block1, _ = zfec.Decoder(2, 3).decode([data_block2, data_block3, ], [1, 2])
assert data_block1 == repaired_data_block1

Generate error correction code from 5 blocks and verify

5DataBlock

# -*- coding: utf-8 -*-
from __future__ import absolute_import, unicode_literals
import zfec

#Error detection correction
data_block1 = str('aiueo')
data_block2 = str('kakik')
data_block3 = str('ukeko')
data_block4 = str('sasis')
data_block5 = str('useso')

#Error detection code from data blocks 1 and 2:Generate 3
blocks = zfec.Encoder(5, 6).encode([data_block1,
                                    data_block2,
                                    data_block3,
                                    data_block4,
                                    data_block5])

ecc_block = blocks[-1]
print str('ecc block is... {}').format(ecc_block)
# >>> ecc block is... J?\j

#Data block 1-Restore 5 from 4 and ECC blocks
repaired_blocks = zfec.Decoder(5, 6).decode([data_block1,
                                             data_block2,
                                             data_block3,
                                             data_block4,
                                             ecc_block], [0, 1, 2, 3, 5])
assert data_block5 == repaired_blocks[4]

#Data block 1,2,4,Restore 3 from 5 and ECC blocks
repaired_blocks = zfec.Decoder(5, 6).decode([data_block1,
                                             data_block2,
                                             data_block4,
                                             data_block5,
                                             ecc_block], [0, 1, 3, 4, 5])
assert data_block3 == repaired_blocks[2]