Miscellaneous notes that I tried using python for the matter

Introduction

Hi, this is UNIBA of MJ. This story is about working with python in a certain project. python is one of the famous languages that needs no special explanation. I usually work as a front engineer, but I started to touch python around April, thinking that I want to know the back end recently. At Universal, I mainly use languages such as javascript, which is node, and ruby, which is rails, at work. As a chuunibyo, I chose python, which no one had done (I regret it because I was in agony because no one could support me). At that time, at just the right time, there was a project with a backend (I'm really wondering if it's a backend because I didn't set up a server in the end, but ...), so I challenged it. It depends on what I saw.

By the way, I write it quite messy. I don't know

Matter

I will look back on what kind of project it was.

Regarding implementation

The lucky thing about this implementation was that it didn't include a server-side implementation. I was very fortunate that I didn't have to think about infrastructure development in addition to DB and so on. Or rather, I was pretty worried (no one in the company runs a server with python !!!).

Let's look back one by one.

Mechanism to automatically upload images and html files when saving photos (folder monitoring + template)

Watchdog to monitor folders, jinja2 to generate html I used / dev /). Both are fairly easy-to-use libraries. It's a well-made library that doesn't need any special explanation. This made the main script a lot easier. When it comes to using the watchdog, some of the contents of the folders you want to monitor (such as the number of files and the number of changes) need to be tested. There was no particular problem with this project.

Data upload destination is aws

Regarding the setting of aws, I was in charge of the part that I usually leave the setting to the back end person in the company. It's a good memory that I got a little trouble in the contract part when I got the domain with route53. s3 cannot be basic authenticated, but it can be restricted by ʻIP, so the test period was supported there. However, the verification around SNS share` was a little troublesome, so I want to make it easier next time.

I used boto for uploading on the python side. It is said that ʻamazonhas officially issued it, but honestly it was hard to read the reference and I could not set the communication timeout well, so I felt a bit painful. In addition, if the bucket name contains., you have to write something special. I noticed the fact just before the release and got a little cold sweat. If you associate it with a domain, you will have to add .`, so I want you to pay attention to this on the library side ...

Encrypt the upload destination path with base62

It's a precautionary measure to prevent mischievous downloads when publishing photos. At first, I implemented it myself, but in the end I used a library called basehash. The part that became a little problem was the cooperation with other apps. As a system, an application made by ʻopenFrameworksis running in addition topython, and a QR code was generated as a link to the web, but the value generated on the C ++ side is the pythonside Do not fit. The algorithm is the same, but it still doesn't fit. The cause is that the hash value is too large forC ++to handle normally. In short, it was a big int problem. It was said thatpythonwill calculate well as much as there is memory in the specification. I guess this is the reason whypythonis used in research fields such as universities. After all, the encrypted part hit python fromC ++ and returned the result. ʻIf __name__ =='__main__': is very useful. happy.

Other applications to care for

The area around here was sober and difficult. .. ..

For example, if communication is lost, a frame will be generated and a warning such as "Please retry" will be issued.

The warning screen must be displayed in the main thread (an amateur who was initially confused without knowing it), but the upload process called from watchdog runs in a sub thread due to the nature of the library, so between threads I had to communicate. To be honest, it was a world unknown to the uncle who usually plays with html. The Queue class solved this problem. We were able to use join and task_done to facilitate process stop and go coordination. No, it's convenient.

I used logging to output the log. To be honest, I want to fill in the log rotation part a little more.

Also, with ʻapplescript`, I made it start automatically at login, but that story is omitted this time.

Impressions

What I thought when I touched the language python

How is it? I hope you will come to work with python again.

Recommended Posts

Miscellaneous notes that I tried using python for the matter
vprof --I tried using the profiler for Python
I tried using the python module Kwant for quantum transport calculation
I tried using scrapy for the first time
[Python] I tried substituting the function name for the function name
I tried python programming for the first time.
I tried using the Datetime module by Python
[Python] I tried using OpenPose
I tried python on heroku for the first time
I tried using the Python library "pykakasi" that can convert kanji to romaji.
A story that was convenient when I tried using the python ip address module
I tried using the Python library from Ruby with PyCall
I tried using Thonny (Python / IDE)
[Python] I tried collecting data using the API of wikipedia
I tried using the checkio API
[Python] I tried using YOLO v3
[For beginners] I tried using the Tensorflow Object Detection API
I tried running the Python Package Repository (Warehouse) that supports PyPI
Movement that changes direction in the coordinate system I tried Python 3
Notes for using OpenCV on Windows10 Python 3.8.3.
I tried tensorflow for the first time
I tried using UnityCloudBuild API from Python
Python: I tried the traveling salesman problem
Notes for using python (pydev) in eclipse
I tried the Python Tornado Testing Framework
I tried using the BigQuery Storage API
I tried logistic regression analysis for the first time using Titanic data
I tried to refer to the fun rock-paper-scissors poi for beginners with Python
I tried using "Streamlit" which can do the Web only with Python
[Text classification] I tried using the Attention mechanism for Convolutional Neural Networks.
I tried to analyze the New Year's card by myself using python
I tried "smoothing" the image with Python + OpenCV
I checked the library for using the Gracenote API
I tried web scraping using python and selenium
I tried "differentiating" the image with Python + OpenCV
I tried object detection using Python and OpenCV
I tried simulating the "birthday paradox" in Python
I tried the least squares method in Python
I tried using PyCaret at the fastest speed
I tried using the Google Cloud Vision API
I tried using mecab with python2.7, ruby2.3, php7
[Python] I searched for the longest Pokemon Shiritori
I tried "binarizing" the image with Python + OpenCV
I tried reading a CSV file using Python
I tried Mind Meld for the first time
I tried using firebase for Django's cache server
I tried using the image filter of OpenCV
I tried using the functional programming library toolz
[Python] I tried to judge the member image of the idol group using Keras
I tried searching for files under the folder with Python by file name
I tried using parameterized
I tried using argparse
I tried using mimesis
I tried using anytree
I tried using aiomysql
I tried Python> autopep8
I tried using coturn
I tried using Pipenv
I tried using matplotlib
I tried using "Anvil".
I tried using Hubot