Temporarily save a Python object and reuse it in another Python

This article is the 16th day article of Takumi Akashiro Alone Advent Calendar 2020.

Finally, the stock of this ad-care that had been for several days has run out, The deadline was so high that I couldn't post in advance in the morning!

The feeling of being driven is fun!

At the beginning

Have you ever wanted to temporarily save a Python object?

For example, debugging ... No, nowadays, with DCC tools, debugging via ptvsd with VS Code is a major [^ 1] ... For example, you want to use a variable with a complicated structure used in Maya in Python of another process ... This seems to be the last minute. Let's assume this today!

[^ 1]: I haven't faced Python very much this year, so I'm scared to write that even this method may be out of date.

What is serialization?

In object-oriented programming, writing an object as a file is called serialization.

According to Wikipedia, it is written as follows.

Serialization-Wikipedia The second meaning, serialization, is a term used in object-oriented programming to convert an object that exists in an environment into a byte string or XML format. More specifically, it means writing out a variable (field) that represents the state of the object and, in some cases, some identifier that represents the type (class) of the object in the form of a byte string or XML format that can be filed. .. This makes it possible to save the data represented by the object as a file or send it over the network. The byte string and XML format obtained in this way are restored to the original object by serialization restoration or deserialization.

Also, serializing objects and saving them in persistent storage such as files is called persistence. ``

... I knew the word itself, but for the first time I looked it up for today's article and knew it was an object-oriented term.

And the standard module that does serialize in Python is ... pickle! !! Let's use it for the time being!

Try using pickle

Let's save it first!

#! python3.8
import pickle
import tempfile
import pathlib
import re

data = {"nemui": "nero", "ohayou": "oyasumi", "nanka": "kanka", "hoge": pathlib.Path("d:/"), "reg": re.compile("\D{2}\d{16}")}

with open(pathlib.Path(tempfile.gettempdir()) / 'sample.pickle', 'wb') as f:
    pickle.dump(data, f)

It's coming, let's read this next.

#! python3.8
import pickle
import tempfile
import pathlib

with open(pathlib.Path(tempfile.gettempdir()) / 'sample.pickle', 'rb') as f:
    data = pickle.load(f)

for k, v in data.items():
    print(f"\t{k}:\t{v} ({type(v)})")

You can restore the variables properly, and the order is preserved according to the implementation of dict in Python3! (It is natural because the order is maintained for the convenience of the mounting side ...)

Read the pickle output in Python3 in Python2

Now let's bring this to Python2! Ah ... Python2 isn't on my current personal computer!

It can't be helped, do you move it in the old Houdini ...

I wonder if the position of temp is different, you ... Aside from the farce (?), Let's take a second look and run it in Python2 this time.

#! python2
import pickle
import tempfile

with open(tempfile.gettempdir() + '/sample.pickle', 'rb') as f:
    data = pickle.load(f)

for k, v in data.items():
    print("\t{0}:\t{1} ({2})".format(k, v, type(v)))

I get an error saying "Protocol Gachigaimasu"!

You did it! (?)

Read the official documentation

If you have a problem, read the official document!

pickle --- Serialization of Python Objects — Python 3.9.1 Documentation Currently, there are 6 protocols available for pickle. The higher the protocol used, the more you will need the latest version of Python needed to read the generated pickle.

• Protocol version 3 was added in Python 3.0.
  It explicitly supports byte objects and cannot be unpickled in Python 2.x.

• Protocol version 4 was added in Python 3.4.
  Added support for very large objects, pickling for more types of objects, and some data format optimizations. This is the default protocol starting with Python 3.8. See PEP 3154 for improvements made with Protocol 4.

• Protocol version 5 was added in Python 3.8.
  Added support for out-of-band data and faster in-band data. See PEP 574 for improvements to Protocol 5.

(Omitted)

pickle.dump(obj, file, protocol=None, *, fix_imports=True, buffer_callback=None)

So, if you replace it with pickle (data, f, protocol = 2) ...

Ah ... yes ... pathlib wasn't in Python 2 ... It can't be helped, so if you replace the path that was set to pathlib with string

You can read the variables of python3 safely with Python2 system! (Do you need this crap to stop using Python2 anymore?)

Tightening

I use it very rarely, but I recommend it because it is reasonably convenient.

Well, if it's a variable with a simple structure, or if you can sacrifice speed Personally, I prefer json, which is highly readable.

It's easy to deal with when you traverse.