[PYTHON] Performance optimization in Django 3.xx

Overview

There are a lot of introductory articles for beginners, but there aren't many topics like this, so it's private Django project. I decided to write a combination of the knowledge I had when I was involved in the translation of Django and the knowledge of the part that was originally translated.

In addition, as a precedent example posted on Qiita

-High Performance Django --Model

There was, but since it was an article in 2016, I would like to write it according to the scope of the official document of Django 3.0.

General performance tuning

Many Django users are conscious of whether the logic works correctly when writing code. However, that alone is not enough to make apps efficiently with Django.

How to improve performance efficiently while keeping the output of logic is a very important point in developing an application.

In many cases, efficiency in one area leads to better performance in other areas. However, this is not always the case. Often, one area is improving performance at the expense of the other. For example, consuming more memory when trying to speed up a program is a prime example.

In a more vicious case, it can run out of memory in an attempt to speed up the program. It can be said that this case is harmful and has no advantage. Be aware of these trade-offs when improving performance.

Therefore, you need to know what the purpose of the performance improvement is. You also need to know if there is a good reason for the policy you are aiming for.

benchmark

It's not a good idea to imagine and look for good or bad performance in your code. Because Django has many tools, of which django-debug-toolbar is a very useful tool. Above all, the ability to view the SQL queries generated by the page and the execution time of each is very useful for performance tuning.

There are other tools that can analyze the performance of third-party sites. However, these services are not meant to diagnose code performance, they are limited to site-wide performance diagnostics.

As a typical service

And so on.

Tuning in Django

caching

Often, calculating the value is costly (that is, it consumes a lot of resources and is slow). Therefore, the value is stored in an readily accessible cache so that it can be retrieved the next time it is needed.

This is an important and powerful technique, and Django includes a comprehensive caching framework and some small caching features.

Caching Framework

The caching framework in Django greatly improves performance by storing dynamic content and avoiding recalculation on every request.

To be more flexible, Django offers different levels of cache granularity, with a wide range of support, from caching the output of a particular view to caching the entire site, for example.

The caching implementation is poorly written and should not be recognized as an alternative to bad code. Therefore, it is one of the last methods to consider when tuning performance.

cached_property

Use cached_property if you need to call a method of an instance of a class multiple times and it consumes a lot of resources to execute it.

Use cached_property to save the value returned by the property and return the value stored in the cache when the function is called again on the same instance.

Note that this decorator works only if the method takes only self as an argument and the method can be returned to a property.

delay

Whereas the cache stores values as described above, laziness is a technique that deliberately delays the execution of calculations until the values are actually needed.

By using this technology, you can refer to it before instantiating it, so it has many uses. For example, Deferred Translation (https://docs.djangoproject.com/en/3.0/topics/i18n/translation/#lazy-translations) runs until the translated string is needed in a rendered template, etc. It has the advantage that it can be used even before the language is known.

Also, delays are a way to save effort by trying to avoid work, and do nothing until you actually need the value, as mentioned above. In other words, the more resources you spend on related work, the greater the benefits of delay.

Python is a number of tools for lazy evaluation by using Generators and Generator Expression Components (https://docs.python.org/en/3/reference/expressions.html#generator-expressions). Is prepared. If you want to take advantage of delay patterns in your code, you can read more about Python delays for a wider range of uses.

Delay in Django

A good example of Django's delay is in the evaluation of QuerySet. QuerySet is delayed. Therefore, you can create a QuerySet and pass it in combination with otherQuerySets. You don't have to actually access the database to fetch the items described, you are passing a QuerySet object, not a collection of the items you will eventually need from the database.

On the other hand, certain operations force the evaluation of QuerySet. By deliberately avoiding the evaluation of QuerySet, you can avoid expensive and unnecessary trips to the database.

Django also provides a keep_lazy () decorator. This causes the function called with the deferred argument to behave deferred and be evaluated only when needed. Therefore, it will not be called for evaluation until the argument you want to delay is strictly requested.

Database

--Optimize database access -Database

We recommend that you take a look at. It is very important to be careful about queries and their costs when it comes to optimization, and read about the DBMS you are using for databases.

HTTP performance

Middleware

Django has middleware for improving site performance, such as:

ConditionalGetMiddleware provides GET response support for modern browsers based on ETag and Last-Modified headers To add.

GZip Middleware compresses the response of all modern browsers, saving bandwidth and transfer time. I will.

session

Cached sessions eliminate the need to read session data from slow storage sources such as databases, which can improve performance.

Static file

Static files are one of the things you should optimize, and Django has the following features:

ManifestStaticFileStorage caches for a long time by adding tags to static filenames I will be able to do it. If the file is changed, the tag is also changed, so the browser is automatically updated to reload the static file.

Minification

Some third-party Django tools and packages provide the ability to "minify" HTML, CSS, and JavaScript. They reduce the size of the document published by the site by removing unnecessary whitespace, line breaks and comments and shortening variable names.

Template performance

important point

--{% bloak%} is faster than {% include%}. --Generating a page from a large number of split template parts can impact performance.

Cache template loader

If enabling cached template loader (https://docs.djangoproject.com/en/3.0/ref/templates/api/#django.template.loaders.cached.Loader) can dramatically improve performance There are many. This loader saves you from having to compile each template each time you need to render the templates.

Use of different versions of available software

It's a good idea to check if different, higher performing versions of the software you are using are available.

This technique is already aimed at users who want to further improve the performance of well-optimized Django sites, and these aren't magical solutions to performance problems, they do the basics right. Keep in mind that it's unlikely that you'll benefit more than you can for a site that doesn't.

Django template language alternative

In most cases, Django's built-in template language is perfectly appropriate, but if you feel that the Django project bottleneck is in the template system and there seems to be no other way to fix it, then a third-party You may be able to improve it with a tool.

For example, using Jinja2 can improve performance, especially speed.

Alternate template systems differ in the extent to which Django's template language is shared.

Keep in mind that if you encounter performance issues with your template, the first thing you should do is understand exactly why, and using an alternative template system can prove faster. Yes, but the same issue may be available without addressing the issue. For example, you can perform expensive template processing and logic more efficiently in views.

Alternative software implementation

It's a good idea to check if the Python software you're using is provided by another implementation that can run the same code faster.

However, most performance issues with a well-crafted Django site aren't at the Python execution level, but with inefficient database queries, caches, and templates. For poor Python code, performance issues are rarely resolved by running the code faster.

Alternate implementations can cause compatibility, deployment, portability, or maintenance issues. Needless to say, before adopting a non-standard implementation, you need to make sure that your application offers sufficient performance gains to outweigh the potential risks.

With these caveats in mind, keep the following in mind:

PyPy is a Python implementation (the "standard" Python implementation is in C). PyPy can significantly improve performance, typically in large applications.

The main purpose of the PyPy project is Compatibility with existing Python APIs and libraries. Please note that Django is compatible, but you need to check compatibility with other libraries it depends on.

Summary

That's it. Now that I've summarized the basics, I'm thinking of writing a way to actually introduce each content and improve performance.

Recommended Posts

Performance optimization in Django 3.xx
Python in optimization
Models in Django
Forms in Django
Model changes in Django
High Performance Django --Model
PHP var_dump in Django templates
Handle constants in Django templates
Implement follow functionality in Django
Rename table columns in Django3
Output table structure in Django
(Note) Django in Vagrant environment
Solve optimization problems in Python
Show Django ManyToManyField in Template
reload in django shell with ipython
Solving "cubes in cubes" by combinatorial optimization
Set placeholders in input fields in Django
8 Frequently Used Commands in Python Django
Dynamically add form fields in Django
FX Systre Parameter Optimization in Python
Errors related to memcached in django
Implementation of login function in Django
Register your Django application in your project
Bayesian optimization package GPyOpt in Python
Write foreign key constraints in Django
How to reflect CSS in Django
Switch the language displayed in Django 1.9
Get parameter values ​​in Django templates
Debugging in Houdini Vol.3 Debugger / Optimization
The meaning of ".object" in Django
Deploy Django in 3 minutes using docker-compose
Pin factory_boy seed value in Django
GraphQL API with graphene_django in Django
Like button implementation in Django + Ajax
Combinatorial optimization techniques seen in puzzles
Get the query string (query string) in Django
Create a LINE Bot in Django