[PYTHON] A story that Qiita really wanted to see under a proxy environment

background

** At the company I work for, I can't access Qiita because of a proxy. ** **

When I listen to the story at a study session, I hear that there is a company with an unexpectedly similar environment. But by all means ** "I want to see Qiita at the company !!" **

One day, when I consulted with a person who was attending an AWS study session, I got a good idea. Recently, I was studying AWS, so I implemented it.

Overall picture

The simple flow of the created service is to pull out the page information of Qiita you want to see and relocate it to a place that can be accessed from within the company. By the way, if you can't access AWS from within your company, you're out.

Service used

This time, we use three main services on AWS. Detailed explanation of each service is omitted.

Constitution

image.png

  1. GET request to API that accepts the URL of the Qiita page you want to see as a query parameter
  2. Pass the query parameter (Qiita URL) to Lambda
  3. Access the Qiita page from Lambda and get Markdown
  4. Markdown → Convert to HTML and save as static content in S3
  5. Response with the destination URL of the S3 object as the redirect destination
  6. Redirect to S3

The request is as follows when viewing the article at https://qiita.com/guromityan/items/5846fcefd87abcf76f7f. In target, specify the URL of the article you want to see.

https://<API Gateway>.ap-northeast-1.amazonaws.com/previewQiita?target=https://qiita.com/guromityan/items/5846fcefd87abcf76f7f

The response looks like this:

{
  "statusCode": 301,
  "headers": {
    "Location": "https://<S3 Bucket>.s3-ap-northeast-1.amazonaws.com/guromityan/5846fcefd87abcf76f7f"
  }
}

Because of " sutatusCode ": 301, the browser that receives it redirects to the URL specified in the Location header. The transition destination is a page that exposes the S3 object converted from Markdown to html as static content.

By the way, I made a Chrome extension and rewrote the URL to Qiita to automatically hit API Gataway. I will write that story someday.

manner

What I wanted to talk about this time is that I have such an idea, so I will omit the explanation of how to set up AWS services.

The source code of Lambda is published on [GitHub] guromityan / previewQiita, so if you want to do it, please use it.

** [Caution] ** In order to use it with Lambda, it is necessary to zip the dependent libraries together, so please drop the dependent libraries locally with docker-compose build etc.

Recommended Posts

A story that Qiita really wanted to see under a proxy environment
A story that stumbled when using pip in a proxy environment
A story that makes it easier to see Model debugging in the Django + SQLAlchemy environment
How to use pip3 under proxy environment Note
A story that I was addicted to at np.where
Docker under proxy environment
A story that took time to understand python's argsort (memorial)
Let's create a Docker environment that stores Qiita trend information!
[Python] A story that seemed to fall into a rounding trap
A story that I was addicted to calling Lambda from AWS Lambda.
I tried to create a server environment that runs on Windows 10
A story that struggled to handle the Python package of PocketSphinx
A story that was struggling to loop processing 3 million ID data
Pip install (Windows) under Proxy environment
When pip fails under proxy environment
A story addicted to Azure Pipelines
Quick reference table under Proxy environment
A story that visualizes the present of Qiita with Qiita API + Elasticsearch + Kibana
A memo that allows you to change Pineapple's Python environment with pyenv
Use the pip command under an HTTP proxy environment that requires authentication
A story that suffered from OS differences when trying to implement a dissertation