What I was addicted to with json.dumps in Python base64 encoding

Introduction

You may want to return base64-encoded data when creating APIs, right? This time I was addicted to it, so I left it down. Since the thoughts are written as sloppy, if you write it briefly

data_bytes = bytes('Akanechan Kawaii Yatta', 'utf-8')
data_encode_bytes = base64.b64encode(data_bytes)
data_encode_str = data_encode_bytes.decode('utf-8')

These three lines are the result.

Prepared data

In Python, Dict type and Json have the same shape, so after processing with Dict type which is easy to process, it is finally converted to Json.

akane_dict = {
            'kind': 'voiceroid',
            'data': 'Akanechan Kawaii Yatta'
        }

It's packed in various ways. This time I want to encode 'data'` `` with base64 and do` `json.dumps.

Data encoding preparation

Make `'data'` of bytes type to encode.

data_bytes = b'Akanechan Kawaii Yatta'

akane_dict = {
            'kind': 'voiceroid',
            'data': data_bytes
        }

This is no good.

    data = b'Akanechan Kawaii Yatta'
          ^
SyntaxError: bytes can only contain ASCII literal characters.

'Akanechan Kawaii Yatta'Is not ascii, so it's natural (1 loss)

I'm typing characters in UTF-8 for VS Code input, so this seems to work.

data_bytes = bytes('Akanechan Kawaii Yatta', 'utf-8')

akane_dict = {
            'kind': 'voiceroid',
            'data': data_bytes
        }

data encoding

Since base64 is included as standard, you can import it as it is.

import base64

data_bytes = bytes('Akanechan Kawaii Yatta', 'utf-8')
data_encode_bytes = base64.b64encode(data_bytes)

akane_dict = {
    'kind': 'voiceroid',
    'data': data_encode_bytes
}

print(akane_dict) #For confirmation

Convert to Json

We will convert Dict type to Json.

import base64
import json

data_bytes = bytes('Akanechan Kawaii Yatta', 'utf-8')
data_encode_bytes = base64.b64encode(data_bytes)

akane_dict = {
    'kind': 'voiceroid',
    'data': data_encode_bytes
}

akane_json = json.dumps(akane_dict)

The following error occurs.

TypeError: Object of type bytes is not JSON serializable

This can't be converted to Json if it's a bytes type! You will get angry at once, so you need to convert it to str type. (2 losses)

Strategy 1

It's a cast that seems to change the str type! Then it will be like this.

import base64
import json

data_bytes = bytes('Akanechan Kawaii Yatta', 'utf-8')
data_encode_bytes = base64.b64encode(data_bytes)
data_encode_str = str(data_encode_bytes)

akane_dict = {
    'kind': 'voiceroid',
    'data': data_encode_str
}

print(akane_dict)
#Send
akane_json = json.dumps(akane_dict)

#Receive
res = json.loads(akane_json)
print(res)
res_data = base64.b64decode(res['data'])
print(res_data)

binascii.Error: Invalid base64-encoded string: number of data characters (53) cannot be 1 more than a multiple of 4 

I agree. If you cast to str type, it will be like this. Let's take a look at the contents of ``` res``.

{'kind': 'voiceroid', 'data': "b'44Ki44Kr44ON44OB44Oj44Oz44Kr44Ov44Kk44Kk44Ok44OD44K/'"}

It may be possible to send it, but it seems impossible to decode `` `data```.

Strategy 2

Article (1) was found, so refer to it and change the code as follows

import base64
import json

data_bytes = bytes('Akanechan Kawaii Yatta', 'utf-8')
data_encode_bytes = base64.b64encode(data_bytes)
data_encode_str = data_encode_bytes.decode('utf-8')

akane_dict = {
    'kind': 'voiceroid',
    'data': data_encode_str
}

print(akane_dict)
#Send
akane_json = json.dumps(akane_dict)

#Receive
res = json.loads(akane_json)
res_data = base64.b64decode(res['data']).decode('utf-8')
print(res_data)

The strange thing here is that the bytes type is decoded into a character string with utf8. I don't understand why I should do this due to lack of study, but if I do this, json.dumps will work well and the receiving side will not be sharpened with an error. On the receiving side, if it is handled as bytes type, it is not necessary to decode it with utf8. Then display

{'kind': 'voiceroid', 'data': '44Ki44Kr44ON44OB44Oj44Oz44Kr44Ov44Kk44Kk44Ok44OD44K/'}
Akanechan Kawaii Yatta

Yeah, looks good

at the end

It seems a little unorganized, but I wrote an article to find out where I was defeated.

reference

(1) Python: Handle binaries with JSON