[Now] Python beginner tried to implement HTTP client using requests (header operation included)

I'm sorry to have used the resources with the material full of feelings now, Mr. Qiit.

It is necessary to hit while changing the Web API in the business system of the company, so I thought that I would try it with Python. I did some communication using the socket object before, so I thought it would be somehow.

So, while making an HTTP client based on socket, I tried various gugrecas and found that it seems different. Upon further investigation, I came to the conclusion that it would be better to use requests.

So, for the time being, I made something that I could just get.

Feature

About requests

Information about request

In this case, it is mostly described in the above document.

setup

requests need to be installed with pip. In my case, I checked with Cygwin (32bit) of Windows 10.

(Reference) Using pip with Cygwin

The information on this site was very helpful. Thank you, tanao.

Looking at the above site, it seems that there were various handling around pip in Cygwin, but as of July 18, 2016, it seems that the following can be done if the environment is Windows10 / Cygwin32bit, Python2.7.

HTTP client class code

For the time being, I tried to make it a class with a pitiful reason to remember how to write the class.

import requests

class SampleHTTPclient:

    def __init__(self, url):
        self.url = url
        self.protocol = ""
        self.error_message = ""
        self.request_headers = None
        self.response_headers = None
        self.status_code = None

    def check_protocol(self):
        # ex
        # if self.protocol = "http://" , self.uri = "hoge.org":
        #  self.url = "http://hoge.org"
        if self.url.find("://") == -1 and len(self.protocol) > 3:
            self.url = self.protocol + self.url

    def set_protocol(self, protocol):
        self.protocol = protocol
        self.check_protocol()

    def set_headers(self, headers):
        self.request_headers = headers

    def get(self, fp):
        try:
            if self.request_headers is None:
                response = requests.get(self.url)
            else:
                response = requests.get(self.url, headers=self.request_headers)
            fp.write(response.content)
        except Exception as ex:
            self.error_message = str(ex)
            return False
        else:
            self.status_code = response.status_code
            self.response_headers = response.headers
            return True

    def get_code(self):
        return self.status_code

    def get_headers(self):
        return self.response_headers

    def get_error_message(self):
        return self.error_message

The entity is two lines, but ... I've messed it up. ..

The usage is the following image.

  1. First, create an instance with URL
  2. If the URL data does not include http: //, you can specify the protocol to be used with set_protocol (* 1).
  3. If there is a header to add, enter the dict according to requests specification in set_headers Please. There is also a sample code below, so you may refer to that as well.
  4. Use open to keep the file open in write mode.
  5. Use it to call get. You can see success (True) and failure (False) with bool
  6. If the communication is successful, the body is in the specified file (* 2).
  7. If the communication is successful, you can get the HTTP status code with get_code.
  8. If communication is successful, use get_headers [response header in the form of dict specified in the request specification](http://docs.python-requests.org/en/master/user/quickstart/#response- You can get headers).
  9. If it fails, you can get the error message with get_error_message.

Code example to actually get

Here is an example of using SampleHTTPclient with the above description coded.

print "sample httpc program."

url = "<<The URL of the resource to get here>>"
headers = {"X-gdaigo_para1": "fujiko", "X-gdaigo-para2":"7890"}
filename = "test.html"
print url, " -> " , filename

httpc = SampleHTTPclient(url);
httpc.set_protocol("http://")
httpc.set_headers(headers)
fp = open(filename, "w")
success = httpc.get(fp)
fp.close()
if success == True:
    headers = httpc.get_headers()
    print "response code=", httpc.get_code()
    for key, value in headers.iteritems():
        print key, ":" , value
    print "success."
else:
    print "httpc:read error\n" , httpc.get_error_message()
    print "error."

print "complete."

The above example is an example of acquiring resources based on the URL and acquiring the data as test.html. The header is set (although it is meaningless), and the acquisition result is also displayed. GET seems to be okay like this.

So I was able to do it relatively easily by using requests.

license

I used it below. Thank you for providing the wonderful software.

that's all.

Recommended Posts

[Now] Python beginner tried to implement HTTP client using requests (header operation included)
I tried to implement PLSA in Python
I tried to implement permutation in Python
I tried to implement PLSA in Python 2
Log in to Slack using requests in Python
I tried to implement ADALINE in Python
I tried to implement PPO in Python
[Python] Deep Learning: I tried to implement deep learning (DBN, SDA) without using a library.
I tried to implement TOPIC MODEL in Python
I tried to implement selection sort in python
I tried to implement Minesweeper on terminal with python
I tried to implement Dragon Quest poker in Python
I tried to implement an artificial perceptron with python
Write data to KINTONE using the Python requests module
I tried to implement SSD with PyTorch now (Dataset)
I tried to access Google Spread Sheets using Python
HTTP server and HTTP client using Socket (+ web browser) --Python3
I tried to search videos using Youtube Data API (beginner)
I tried to implement a one-dimensional cellular automaton in Python
I tried to get Web information using "Requests" and "lxml"
I tried to implement the mail sending function in Python
I tried to make a stopwatch using tkinter in python
Tweet Now Playing to Twitter using the Spotify API. [Python]
A python beginner tried to intern at an IT company
I tried to implement blackjack of card game in Python
I tried to implement SSD with PyTorch now (model edition)
When I tried to scrape using requests in python, I was addicted to SSLError, so a workaround memo