[PYTHON] What I investigated in the process of expressing (schematicizing) containers in a nested frame with Jupyter and making it

How can I easily diagram a container in Python? I thought that was the beginning, and from there, I made a note of what I realized using Jupyter notebook and what I thought was indirectly useful in the process of realizing it.

Introduction

It's not limited to Python, but I wondered if I could easily prepare a visually easy-to-understand diagram or something when explaining data with multiple nested containers (especially for programming beginners). That was the reason I made this.

Even so, I'm not satisfied with just preparing some samples as images, so I wondered if the code could be schematized as it is.

At first, traverse the container to make an image? I thought, but it seems difficult to calculate the part that expresses nesting ...

But when I think about it, it was familiar to me when it came to expressing nesting. Yes, it's easy to write in HTML and display it in a web browser.

I was just starting to touch Jupyter, so it would be better if it could be displayed on Jupyter. Then, in order to do trial and error, the conversion process is made into a module ...

As a result of various thoughts like this, it became "display as a nested frame in HTML".

The links of the referenced sites are posted directly in the text.

Development environment

I am developing with Windows7 + Anaconda (Python3), Anything will do as long as you can use Python 3.5.1 and Jupyter notebook.

Realization method

Basically, all you have to do is traverse the container and create the HTML. There were still many things I didn't understand in Python, so I decided to make it through trial and error. I will make a note of some of the things I learned in the process.

Output HTML as it is with Jupyter

You can use IPython's display module to output HTML directly to Jupyter results.

Module: display — IPython 4.2.0 documentation http://ipython.readthedocs.io/en/stable/api/generated/IPython.display.html

--Example

from IPython.display import display, HTML

display(HTML('<p>Hello.</p>'))

Convert objects such as containers to HTML

Traverse the container and convert it to HTML. It transforms recursively, resulting in nesting.

Since it is difficult to understand the appearance just by outputting it in HTML, we have made it possible to distinguish several types by line type and color. So add a style (class) to the tag.

Create a module with the name container2html to convert these containers to HTML.

In order to apply the style definition, output the <style> tag as HTML in advance.

Temporarily use modules in the local environment

To use a module written temporarily in the local environment, add a directory to the module path. If you add the project directory to the module path, you can import it like a regular module.

For example, I'm writing a module in PyCharm on Windows right now There is a project directory C: \ Users \ argius \ PyCharm \ MyModule1 And suppose you want to create a file called container2html.py under it and make it a module.

In that case, you can use the module with the name c2h by doing the following.

import sys
sys.path.append(r'C:\Users\argius\PyCharm\MyModule1')

import container2html as c2h

Reload module

Once the module is loaded, the code updates will not take effect until it is reloaded. In this case, it's a trial and error process, so I'd like to reflect the update every time I modify the module ...

In that case, that is, to re-evaluate the modified module, use the reload function of the ʻimport lib` module.

import importlib
importlib.reload(c2h)

2016-07-21: I was using the old ʻimp module, so I modified it to ʻimport lib.

Examine the parent-child relationship of types

I wanted to investigate the parent-child relationship such as the basic type so that it could handle unknown types to some extent, but at first I didn't know how to do it.

The site below shows that you can look it up using the mro method. You can trace your ancestors with type (o) .mro () for object ʻo. You can also specify the type name directly, such as list.mro ()`.

Python Tips: I want to check the inheritance relationship of classes --Life with Python http://www.lifewithpython.com/2014/08/python-check-class-inheritance-relationship.html

For example, the Counter in the collections module is similar to a dictionary, but to see if it can be interpreted like a dictionary, I checked:

>>> from collections import Counter
>>> Counter.mro()
[<class 'collections.Counter'>, <class 'dict'>, <class 'object'>]

Implementation

Module container2html.py

"""Container to HTML"""


def convert(o, **kwargs):
    label = kwargs.get('label', None)
    labelhtml = '' if label is None else '<p class="label">%s</p>' % label
    return '<div class="c2h">%s %s</div>' % (labelhtml, HtmlConverter(kwargs).object_to_html(o))


def escape(o):
    return str(o).replace('&', '&amp;').replace('"', '&quot;').replace('<', '&lt;').replace('>', '&gt;')


class HtmlConverter:
    def __init__(self, kwargs):
        self.options = kwargs
        self.showtypeinfo = self.options.get('showtypeinfo', False)

    def object_to_html(self, o):
        def f():
            if isinstance(o, dict):
                return self.dict_to_html(o, 'dict')
            t = type(o)
            if t is tuple:
                return self.seq_to_html(o, 'tuple')
            if t is str:
                return '<span class="value">%s</span>' % repr(o)
            if t is list or hasattr(o, '__iter__'):
                return self.seq_to_html(o, 'list')
            return '<span class="value">%s</span>' % str(o)

        if self.showtypeinfo:
            return '<div class="typeinfo"><div>[%s]</div>%s</div>' % (escape(type(o).__name__), f())
        return f()

    o2html = object_to_html

    def seq_to_html(self, seq, style_name=None):
        a = list(seq)
        if len(a) == 0:
            return '<div style="nil">%s(empty)</div>' % ''
        items = ['<li>%s</li>' % self.o2html(x) for x in a]
        return '<ul class="%s">%s</ul>' % (style_name, ''.join(items))

    def dict_to_html(self, d, style_name=None):
        if len(d) == 0:
            return '<ul class="%s"><li>(empty)</li></ul>' % style_name
        items = ['<ul><li>%s</li><li>%s</li></ul>' % (self.o2html(k), self.o2html(v)) for k, v in d.items()]
        return '<ul class="%s"><li>%s</li></ul>' % (style_name, ''.join(items))

style

div.c2h {
  font-family: consolas, monospace;
  border: 1px solid black;
  margin: 0px 1px 3px 0px;
  padding: 3px 1em;
}
div.c2h .label {
  font-size: 100%;
  color: blue;
  text-decoration: underline;
  padding-bottom: 1ex;
}
div.c2h span.value {
  background-color: #fff;
  padding: 1px 4px;
}
div.c2h div.typeinfo {
  border: 1px dashed grey;
}
div.c2h ul {
  border: 4px solid black;
  display: table;
  margin: 1ex;
  padding-left: 0;
  list-style: none;
  text-align: center;
}
div.c2h ul li {
  display:table-cell;
  vertical-align: middle;
  text-align:center;
  padding: 1ex;
  border-style: solid;
  border-right-width: 4px;
  border-top-width: 0px;
  border-bottom-width: 0px;
}
div.c2h ul.list {
  border-color: #cc3311;
}
div.c2h ul.list li {
  border-color: #cc3311;
}
div.c2h ul.tuple {
  border-color: #33bb33;
  border-radius:10px;
}
div.c2h ul.tuple > li {
  border-color: #33bb33;
}
div.c2h ul.dict {
  border-color: #3333cc;
}
div.c2h ul.dict > li {
  border-color: #3333cc;
}
div.c2h ul.dict > li ul {
  border-color: #3333cc;
  border-style: inset;
  border-radius:10px;
}
div.c2h ul.dict > li ul li {
  border-color: #3333ff;
  border-width: 1px;
}

Execution result

Prepare in advance.

from IPython.display import display, HTML

import sys
sys.path.append(r'C:\Users\argius\PyCharm\MyModule1')

import container2html as c2h

display(HTML('''
<style>
(abridgement)
</style>
'''))

Now you can display the result of the c2h.convert function bydisplay (HTML (...)), It's a hassle to write this every time, so Define a function v as a shortcut that outputs the HTML-converted version of the code (expression) string with display.

def v(code, **kwargs):
    display(HTML(c2h.convert(eval(code), label=code, **kwargs)))

All you have to do is run it.

Execution code

If you add showtypeinfo = True as an option, the type name will also be displayed.

v('[1, 2, (3, 4, 5)]')
v('[1, 2, (3, 4, 5)]', showtypeinfo=True)
v('zip([1, 2, 3, 4], [1, 2, 3, 4], [5, 6, 7, 8])', showtypeinfo=True)
v('dict(one=1, two=2)')
v('range(5)')

from collections import Counter
v("Counter(['a', 'a', 'b', 'c'])", showtypeinfo=True)

import numpy as np
v('np.arange(9).reshape(3, 3)', showtypeinfo=True)

result

dict is displayed in a blue frame, tuple is displayed in a green frame, and other list and other ʻiterable`s are displayed in a red frame.

_qiita.png

in conclusion

I don't know if this will actually be used, but I think it was more meaningful to investigate in the process of making it. Actually, this time, I had a harder time with CSS than with Python.

Above all, it's fun to make something by trial and error like this!

Recommended Posts

What I investigated in the process of expressing (schematicizing) containers in a nested frame with Jupyter and making it
The result of making a map album of Italy honeymoon in Python and sharing it
The process of making Python code object-oriented and improving it
Process the contents of the file in order with a shell script
[Python / Jupyter] Translate the comment of the program copied to the clipboard and insert it in a new cell
I set the environment variable with Docker and displayed it in Python
I vectorized the chord of the song with word2vec and visualized it with t-SNE
The story of making a sound camera with Touch Designer and ReSpeaker
What I did when I couldn't find the feature point with the optical flow of opencv and when I lost it
I made a GAN with Keras, so I made a video of the learning process.
I made a mistake in fetching the hierarchy with MultiIndex of pandas
I wanted to know the number of lines in multiple files, so I tried to get it with a command
To output a value even in the middle of a cell with Jupyter Notebook
If you define a method in a Ruby class and define a method in it, it becomes a method of the original class.
What I did when I was angry to put it in with the enable-shared option
I investigated the calculation time of "X in list" (linear search / binary search) and "X in set"
Image processing with Python (I tried binarizing it into a mosaic art of 0 and 1)
Read the csv file with jupyter notebook and write the graph on top of it
I don't like to be frustrated with the release of Pokemon Go, so I made a script to detect the release and tweet it
I wrote python3.4 in .envrc with direnv and allowed it, but I got a syntax error
Make a note of what you want to do in the future with Raspberry Pi
Recursively get the Excel list in a specific folder with python and write it to Excel.
I also tried to imitate the function monad and State monad with a generator in Python
I wrote a doctest in "I tried to simulate the probability of a bingo game with Python"
How to insert a specific process at the start and end of spider with scrapy
[Python] Visualize Arashi's lyrics with WordCloud and try to understand what I wanted to convey to fans in the 20th year of formation.
I wrote the basic grammar of Python with Jupyter Lab
I wrote the basic operation of Seaborn in Jupyter Lab
What I'm glad I studied in 2015 and what I'm thinking of learning in 2016
Process the files in the folder in order with a shell script
I wrote the basic operation of Numpy in Jupyter Lab.
I wrote the basic operation of matplotlib with Jupyter Lab
The story of making a question box bot with discord.py
I tried hitting the Google API with Ruby and Python-Make the database a Spreadsheet and manage it with Google Drive
Process the gzip file UNLOADed with Redshift with Python of Lambda, gzip it again and upload it to S3
Find the white Christmas rate by prefecture with Python and map it to a map of Japan
[Python] The role of the asterisk in front of the variable. Divide the input value and assign it to a variable
When I cut the directory for UNIX Socket under / var / run with systemd, I got stuck in a pitfall and what to do
Minimum knowledge required when dealing with "angles" and "coordinates" in Ruby + In what direction is Mr. B from the perspective of Mr. A? Algorithm
I made a LINE bot that tells me the type and strength of Pokemon in the Galar region with Heroku + Flask + PostgreSQL (Heroku Postgres)
I compared the speed of Hash with Topaz, Ruby and Python
Scraping the schedule of Hinatazaka46 and reflecting it in Google Calendar
[Introduction to StyleGAN] I played with "The Life of a Man" ♬
I tried to process the image in "sketch style" with OpenCV
The story of making a standard driver for db with python.
I tried to make a periodical process with Selenium and Python
I wrote the basic operation of Pandas with Jupyter Lab (Part 1)
I want to create a pipfile and reflect it in docker
I tried to process the image in "pencil style" with OpenCV
I investigated the behavior of the difference between hard links and symbolic links
I wrote the basic operation of Pandas with Jupyter Lab (Part 2)
The story of making a module that skips mail with python
I made a chatbot with Tensor2Tensor and this time it worked
Note that I understand the algorithm of the machine learning naive Bayes classifier. And I wrote it in Python.
I made a POST script to create an issue on Github and register it in the Project
[Cliff in 2025] The Ministry of Economy, Trade and Industry's "DX Report 2" was published, so I read it.
I ran GhostScript with python, split the PDF into pages, and converted it to a JPEG image.
The story of making a tool to load an image with Python ⇒ save it as another name
Upload data to s3 of aws with a command and update it, and delete the used data (on the way)
[CleanArchitecture with Python] Apply CleanArchitecture step by step to a simple API and try to understand "what kind of change is strong" in the code base.