mixi Group Advent Calendar This is the second day.
"TicketCamp" operated by mixi Group is developed with Python and Django on the server side.
I'm currently using Django version 1.7.11, which is a note of what I did when updating to Django 1.8.
I'm sure there are some ticket camp-specific parts, but I hope it's helpful for anyone considering updating Django.
In the first place, what is the policy for upgrading the framework library of TicketCamp is that it depends on my intuition and it is difficult to clarify, but generally the following three rules are used. I can explain.
--Security updates applied as soon as possible --Follow Django's major updates as much as possible --Do not update unless there is another reason
The third "unless you have a specific reason" is ambiguous, and the reason is clear in cases like "I want to use the feature X, but in that case I need to update the library Y". However, there are cases where there is no deeper reason than "I tried to raise it by the way" such as "There was a case to modify the image upload API, so I will update Pillow as well".
Anyway, I would like to emphasize that we are not developing with a policy such as "prohibit version upgrade".
The second "follow major updates as much as possible" means that as a person who has been using Django before 1.0, it is better to always follow the latest version to maintain the health of the application, and when updating It's based on the rule of thumb that it's easy after all.
From that point of view, I wanted to update from Django 1.8 Beta Stage, but with some update failures There was a point that I hadn't touched for a long time. While doing so, I got up to Django 1.9 RC, so this is bad. I finally started working on the version upgrade seriously.
Then, when I checked the Django site when I was about to press the publish button in this article, [Django 1.9 was already published](https://www.djangoproject.com/weblog/2015/dec/01/] django-19-released /) ・ ・ ・
Ticketcamp uses the standard Celery in Python for the job queue, and processes that require external network I / O such as push notifications via AWS SNS , Ticket listing / request matching processing, which takes some time, is processed by Celery's worker process.
When I switched to Django 1.8, an error occurred with the celery == 3.1.7 I was using so far, so I updated to the latest version of celery == 3.1.9.
pip install --upgrade celery==3.1.9
Celery is very stable and has a lot of users, so I don't feel much risk when it comes to minor updates.
I'm using the django_nose test runner instead of the Django standard test runner for unit tests.
This was also fixed to a slightly older version, so an error occurred, so I updated to the latest version django_nose == 1.4.2.
pip install --upgrade django_nose==1.4.2
TEST_NAME to TESTStarting with Django 1.7, the test database setup method has changed from the TEST_ prefix to using an independent dict called TEST](https://docs.djangoproject.com/en/1.8/ref/settings) / # test), so fix it at this timing.
Before correction.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'USER': 'root',
'NAME': 'ticketcamp_dev',
'TEST_NAME': 'test_ticketcamp_dev',
},
}
Revised.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'USER': 'root',
'NAME': 'ticketcamp_dev',
'TEST': {
'NAME': 'test_ticketcamp_dev',
},
},
}
loader.get_template_from_string disappearedThere was only one place in the ticket camp that used the method django.template.loader.get_template_from_string.
At a minimum, the code looks like this:
from django.template import loader, Context
tmpl = loader.get_template_from_string(content)
output = tmpl.render(Context(ctxt))
Code like this
AttributeError: 'module' object has no attribute 'get_template_from_string'
I started to get the error.
Apparently django.template.loader.get_template_from_string has been removed in Django 1.8.
This kind of sudden method disappearance is rare in Django and never happens in public APIs. Normally, you'll get a DeprecatedWarning that will be deprecated in Django 1.x, so you'll notice it when you're developing it.
This means that this method was treated as a private method, not a public method.
Fortunately, in this case it was easy to solve by replacing loader.get_template_from_string with Template.
from django.template import Template, Context
tmpl = Template(content)
output = tmpl.render(Context(ctxt))
Foreign Key (unique = True) has come out.TicketCamp uses django.contrib.auth for user registration and user authentication, but it's a remnant of what we've been developing since the days of Django 1.4, Pattern using Profile objects to give users additional information still remains.
The code looks like this:
# -*- coding: utf-8 -*-
#This code has a trap, so don't copy it!
from django.conf import settings
from django.db import models
class Profile(models.Model):
user = models.ForeignKey(settings.AUTH_USER_MODEL, unique=True)
#abridgement
After updating to Django 1.8 and running Celery workers, I'm getting the following warning:
myapp.Profile.user: (fields.W342) Setting unique=True on a ForeignKey has the same effect as using a OneToOneField.
HINT: ForeignKey(unique=True) is usually better served by a OneToOneField.
I knew it for the first time when I got here, but in the first place, the above code is
# -*- coding: utf-8 -*-
from django.conf import settings
from django.db import models
class Profile(models.Model):
user = models.OneToOneField(settings.AUTH_USER_MODEL)
#abridgement
It seems that I should have used ʻOneToOneField ()instead ofForeignKey (unique = True)` like. Moreover, because I made a mistake in the first stage, there were some places where similar warnings were issued.
It seemed like I should change it to ʻOneToOneField ()` as in the hint in the warning, so I was able to fix everything at this opportunity.
Default, ʻauto_now, ʻauto_now_add of DateTimeField is now exclusiveThis is where the update is difficult.
The definition of the ticket camp model has the following pattern, and the model creation date and time and update date and time are automatically updated as created_at, ʻupdated_at`.
from django.db import models
class Ticket(models.Model):
price = models.IntegerField()
count = models.IntegerField()
created_at = models.DateTimeField(default=timezone.now, auto_now_add=True)
updated_at = models.DateTimeField(default=timezone.now, auto_now=True)
When I try to run code that contains such a model definition in Django 1.8, the Celery worker doesn't start because of the following error:
myapp.Ticket.created_at: (fields.E160) The options auto_now, auto_now_add, and default are mutually exclusive. Only one of these options may be present.
myapp.Ticket.updated_at: (fields.E160) The options auto_now, auto_now_add, and default are mutually exclusive. Only one of these options may be present.
The cause is that, as shown in the error message, ʻauto_now, ʻauto_now_add, default are mutually exclusive options and cannot be specified at the same time.
To solve this problem, I thought of the following solutions.
--Same as before, create your own DateTimeField so that the current time is entered by default and the creation date and update date and time are updated when the model is saved.
--The model definition is ʻupdated_at = models.DateTimeField (default = timezone.now), and the schema definition on MySQL is forced to be ʻupdated_at DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP.
――Allow the initial value of ʻupdated_at to be None`.
The first option is that you don't want to write the code yourself as much as possible, and that Django made these changes means that you can specify ʻauto_now, ʻauto_now_add, and default at the same time. I gave up because I guessed that it would be difficult to make such an implementation.
The second option was predicted to have less code changes and less impact on the application, but I've taken it so far as "do not define schemas that can't be represented in the Django model as much as possible". I put it on hold from the design principle.
We are currently testing the update with a third fix.
First of all, in the example of the above model, created_at can safely remove the option of ʻauto_now_add = True`.
When you create an instance of the model, the default option timezone.now is evaluated and the value of created_at becomes the current time, which is saved in the DB when you callsave (). Because it will be. (Thinking about that, I think that ʻauto_now_add = True` was unnecessary in the first place.)
The problem is ʻupdated_at`
updated_at = models.DateTimeField(default=timezone.now)
Or
updated_at = models.DateTimeField(auto_now=True)
You have to choose.
In the former case, ʻupdated_at is nonNone when the instance is created, but save () does not update the ʻupdated_at column of the database.
$ python manage.py shell
>>> from myapp.models import Ticket
>>> ticket = Ticket(price=1000, count=1)
>>> ticket.updated_at is not None
True
>>> ticket.save()
>>> ticket.price = 1500
>>> ticket.save() #I changed it, but the DB was updated_at is not updated
I've considered a way to update ʻupdated_at using [post_save` signals](https://docs.djangoproject.com/en/1.8/ref/signals/#post-save). , I thought it would cause a problem that was difficult to find the cause, so I decided to forgo it.
The latter, the disadvantage of selecting DateTimeField (auto_now = True), is that ʻupdated_at is None when the instance is created, and the current time is the instance for the first time when save () is called. It means that the behavior will be reflected in the updated_at` property.
$ python manage.py shell
>>> from myapp.models import Ticket
>>> ticket = Ticket(price=1000, count=1)
>>> ticket.updated_at is not None
False # updated_at is still None
>>> ticket.save()
>>> ticket.updated_at is not None
True # save()Become non-None for the first time after calling
In other words, there are cases where it will not work if there is code that references the instance's ʻupdated_at before save () `.
However, each method has advantages and disadvantages, and after all, the last method seemed to follow the usage expected by the framework creator, so I adopted the last method and adopted the following method. I decided to change it with feeling.
from django.db import models
class Ticket(models.Model):
price = models.IntegerField()
count = models.IntegerField()
created_at = models.DateTimeField(default=timezone.now)
updated_at = models.DateTimeField(auto_now=True)
The obvious error has disappeared, so when I started checking it, it looks like it's working fine. After that, I thought that the update would be completed if I referred to ʻupdated_at before save () `and moved the part where the error occurred, but there were still pitfalls ...
Actually, I wanted to reach the point where the update was completed by the time this Advent calendar was released, but there are still some unsolved and unexplained problems.
When you run a unit test,
$ python manage.py test myapp
#Abbreviation
django.db.utils.IntegrityError: (1215, 'Cannot add foreign key constraint')
I got the error.
I took a lot of time to investigate the cause, but it seems that the cause is that the schema is not reflected correctly when creating the database for unit test because migrations does not exist under each Django App directory.
TicketCamp was developed before Migrations was introduced in Django 1.7, and even when updating to Django 1.7, "Migrations" I didn't do manage.py make migrations because I didn't plan to use it, but it seems that I finally need to use Migrations.
Go back to Django's standard test runner, manually create a test database, and then
$ python manage.py test myapp --keepdb
When I ran the test with the --keepdb option, I was able to run the test.
However, as I had expected in advance, the definition of the model for ʻupdated_at changed, so I was trying to INSERT NULL into ʻupdated_at and could not load fixtures.
Regarding this issue
--Fix YAML for all fixtures.
--Make ʻupdated_at non-null in the pre_save` signal only during testing.
--Stop making fixtures in YAML and move to a library like Factory Boy.
We are considering a correction method such as.
With Django 1.8, there are a lot of warnings about functions that will be removed in 1.9, so I'm crushing them one by one.
Warning for django.core.cache.get_cache.
RemovedInDjango19Warning: 'get_cache' is deprecated in favor of 'caches'
Before correction.
from django.core.cache improt get_cache
get_cache('default').get('key')
Revised. This was replaced mechanically.
from django.core.cache improt caches
caches['default'].get('key')
Warning for django.utils.functional.memoize.
RemovedInDjango19Warning: memoize wrapper is deprecated and will be removed in Django 1.9. Use django.utils.lru_cache instead.
Warning for django.utils.datastructures.SortedDict. This is happening because the version of django-redis I'm using is quite old, rather than the code I wrote myself, so I'm upgrading. Seems to be necessary.
RemovedInDjango19Warning: SortedDict is deprecated and will be $
emoved in Django 1.9.
A warning that occurs when you pickle a Django model.
RuntimeWarning: Pickled model instance's Django version is not specified.
It feels like we're still in the middle of an update, but with Django's release policy, the 1.7 we're currently using will end maintenance when 2.0 comes out, so we definitely need to move to the next LTS version, 1.8. There is.
We're also investigating whether it's possible to move from Python 2.7 to Python 3.5, and I feel the need to keep up with the latest version of Django for the move to Python 3.5.
-Migrate Django applications running on Python 2.7 to Python 3.5
I hope to settle the matter by the end of the year and welcome the new year comfortably.
Next, akkuma will write about Android Action Mode.
Recommended Posts