--When making a request with the POST method using the requests module of python, URL encode it, but be careful when passing Japanese parameter values.
--Some sites decode in Shift-JIS format, in which case the requester must also encode and call in Shift-JIS format.
--If you pass it to requests.post in dict format, the default encoding will be performed, so by using the urllib module to encode the URL, make it str type and pass it directly to requests, the default encoding is also avoided and encoded with Shift-JIS. It is possible to pass parameters.
import requests
import urllib.parse
#value of post parameter (example)
item = 'Hoge'
post_data = {'KEY1':item}
# shift-Encode with jis and make it str type
urlencode_post_data = urllib.parse.urlencode(post_data, encoding='shift-jis')
headers = {'content-type': 'text/html; charset=Shift_JIS'}
response = requests.post(<Target URL>, urlencode_post_data, headers=headers)
――When dealing with Japanese, such a character code problem comes up. When I searched for a solution, I couldn't find a solution that was applicable to sites other than Japanese-speaking countries, so I solved it by trial and error on my own. I will post it if it helps even a little.
Recommended Posts