After learning scraping with python, I wanted to make a web application using the acquired data, so this is a memo that I investigated how to realize it.
A framework called Django is recommended.
--Reference article: Developing a web application with Python! Introducing popular frameworks
Learn more about Django.
See below [Introduction to Django] Install Django on your computer!
When I tried to run $ pip install django, it said that the version of pip itself was old and needed to be updated ...
--Reference article: Python Tips: I want to update pip itself
Did not launch python shell with $ python in Git Bash.
It seems to be a specification ...
It seems better not to handle python with Git Bash (execute quietly at the command prompt)
--Reference article: What to do if Python doesn't work on Git for Windows
--Reference article: [Introduction to Django] Django App Design Philosophy! Learn MTV models through models!
The command to create a project is as follows
django-admin startproject mySite
The command to create a template is as follows
python manage.py startapp myapp
By using the function called model, it seems that you can mess with the database without directly touching the DB (I wanted to make it with a three-tiered architecture, so I am grateful that the DB can be operated easily)
--Reference article: [Introduction to Django] Let's display a web page using template and HTML
First, add the following description of 'DIRS' in TEMPLATES in the settings.py file.
os.path.join(BASE_DIR, 'templates'),
This declares "put the template file in a directory called templates" (apparently).
After that, as a flow to create a template display process after creating a templates directory
--Definition of view function --Linking URL and view function (for each project and application)
Feeling like that.
After tying
python manage.py runserver
You can run the command and start the server on the local host to see if the template is displayed.
The django template has a lot of useful features, so I wanted to use it. See the official documentation (https://docs.djangoproject.com/ja/3.1/topics/templates/) for more information.
Reference article: python: Let's make a web application (scraping with django)
It will be described in various ways in the views.py file
view is responsible for the function of "acquiring the page/data to be displayed and generating a response based on the request from the browser" in django, which adopts the design concept of MTV (Model, Template, View).
--Reference article: Understanding MTV
And django has a feature called "general view" that sets the appropriate default values for your use case.
--Reference article: Introduction to class-based general-purpose views in Django and sample usage
See below for how to use different types of general-purpose views
--Reference article: -Introduction to class-based generic views in Django and sample usage -Collect Django's generic class views and mention the implementation
This time, I want to run scraping based on the data input by the web page, so I will use CreateView which can use form.
When specifying the success_url variable of CreateView (URL of the page that transitions when the creation is successful), use a function called reverse_lazy.
Functions such as reverse and reverse_lazy are used when "calling a url by name".
--Reference article: Easy-to-understand explanation of how to use reverse [with specific code]
Specifically, when reverse ('home') is set, the url specified as name ='home' in urlpatterns is called.
The difference between reverse and reverse_lazy is that "if you want to use the reverse function before reading URLConf (= urls.py), use reverse_lazy".
--Reference article: [Django] success_url and get_success_url and reverse and reverse_lazy properly used
When using it in a class, it seems to correspond to "before reading URLConf (= urls.py)", so use revese_lazy.
It seems that in order to understand lazy evaluation correctly, it is necessary to understand what is evaluated in python at what timing, but honestly I do not know well ...
--Reference article: Difference between reverse () and reverse_lazy () in Django
Reference article: [Python] Let's understand how to use append! Thorough explanation of roles and practical methods!
You can add a single element to an existing list by using the append function.
Reference article:
-Python Django Tutorial (3) -Official Document
A function that returns the rendering result using that template as HttpResponse when you pass request and template path. By passing a dictionary type value as the third argument, you can pass the value to template.
⇒ Now you can use the append function to pass the element (obtained by scraping) that is inserted into the dictionary type list to the template and fit it as an element in HTML
--Reference: Official Document
One Manager is added by default to the model class.
--Function: Provides an interface for DB operations
--Name: objects (Why isn't it a manager ...?)
The model manager has various methods.
--.objects.all (): Returns all elements in the DB
--. Objects.filter (): Return with specified conditions
Note: .objects is required to call the methods that the model manager has.
Reference: [For beginners] Understand when objects are required / not required for Django model operations
Recommended Posts