[PYTHON] About "RuntimeWarning: Pickled model instance's Django version is not specified"

This article is the 6th day article of Python Part 2 Advent Calendar 2015.

If you upgrade Django from 1.7 to 1.8 or higher,

RuntimeWarning: Pickled model instance's Django version is not specified.
  obj = pickle.loads(value)

You may see a warning like this.

I was wondering if it would always occur when deserializing a model instance with Pickle, but sometimes this warning was not displayed and the conditions were unknown, so I investigated the cause.

Cause of warning

Simply put, it happens when you deserialize model data serialized in an application running Django 1.7 or earlier in an application running Django 1.8 or higher.

The code that is causing the warning is here.

You can easily reproduce it by creating an environment with 1.7, 1.8 / 1.9 installed with virtualenv etc. and switching the version with the same application.

Set the cache with the virtualenv of Django 1.7 as follows.

$ python manage.py shell
>>> from django.core.cache import caches
>>> from django.utils.six.moves import cPickle as pickle
>>> from myapp.models import MyModel
>>> obj = MyModel.objects.get(pk=1)
>>> caches['default'].set('my-model-instance', pickle.dumps(obj))

Deserialize the model instance from the cache with the Django 1.8 virtualenv like this:

$ python manage.py shell
>>> from django.core.cache import caches
>>> from django.utils.six.moves import cPickle as pickle
>>> from myapp.models import MyModel
>>> pickle.loads(caches['default'].get('my-model-instance'))
<string>:1: RuntimeWarning: Pickled model instance's Django version is not specified.

Workaround

The following can be considered as countermeasures.

ignore

I'm not sure, but as far as I read the comments in the Django source code, it doesn't seem to have any effect without the lazy-loaded property in defer.

Also, volatile caches like Memcache will be replaced by new caches over time, so the problem should go away.

Clear the cache when you upgrade Django

I do not recommend it.

$ python manage.py shell
>>> from django.core.cache import caches
>>> caches['default'].clear()

Or set CACHES in settings.py to [KEY_PREFIX](https://docs.djangoproject. Change com / en / 1.9 / ref / settings / # key-prefix) or increment VERSION Has almost the same effect as clearing the cache.

Change only the cache key of the pickled data

This is the method I'm trying to catch.

By putting the Django version number in the cache key, even if the version goes up in the future, a different cache will be used automatically for each version.

>>> from django.core.cache import caches
>>> from django.utils.six.moves import cPickle as pickle
>>> from django.utils.version import get_version
>>> from myapp.models import MyModel
>>> obj = MyModel.objects.get(pk=1)
>>> caches['default'].set('{}:my-model-instance'.format(get_version()), pickle.dumps(obj))

The problem is only when caching the pickled model instance, so if you fix the relevant part like the above, you should not be in trouble even if you upgrade to 1.9 or higher. ..

Recommended Posts

About "RuntimeWarning: Pickled model instance's Django version is not specified"
[Django error]'RenameAttributes' object is not iterable
About the matter that softmax is not needed at the end of Torchvision model.