Expand a Python nested dictionary to do something like Pandas' MultiIndex

Thing you want to do

――I want to expand a nested dictionary by connecting it with an underscore. --Input: d1 = {'A': {'i': 0,'j': 1},'B': {'i': 2,'j': 3}} Data that contains a dictionary --Output: d2 = {'A_i': 0,'A_j': 1,'B_i': 2,'B_j': 3}, open data in one dimension

Function (Not Smart)

--In the end, nested dictionaries have a tree structure. ――As an image, the process is like breaking a tree structure and directly connecting the roots and ends. --A recursive function seems to be good when it comes to implementing it like a depth-first search.

So I wrote the following code.


#key change
def changekey(dict,k1,k2):
    dict[k2] = dict[k1]
    del dict[k1]

#Add modifiers to all keys with underscores
def addkeyheader(dict,key):
    ks = list(dict.keys())
    for k in ks:
        changekey(dict,k,key+'_'+k)

#After performing a depth-first search of the favorite function, the element names are combined and returned.
def dict_flatten(dict_,depth=2):
    newdict = {}
    for key in list(dict_.keys()):
        #If the element is also dict, call it recursively
        if isinstance(dict_[key], dict):
            if depth > 0:
                dic = dict_flatten(dict_[key],depth-1)
                addkeyheader(dic,key)
                newdict.update(dic)
            else:
                newdict[key] = dict_[key]
        #If the element is not a dict, leave it as it is
        else:
            newdict[key] = dict_[key]
    return newdict

Smarter implementation

In the comments, I told you about a smarter implementation.

def _unwrap(dct, prefix):
    for key, value in dct.items():
        new_key = f'{prefix}_{key}'
        if isinstance(value, dict):
            yield from _unwrap(value, new_key)
        else:
            yield new_key[1:], value       

def unwrap(dct):
    return dict(_unwrap(dct, ''))

d2 = unwrap(d1)

Expected usage

--The nested dict is arranged in two dimensions as shown below (Reference). --Unlike this, there are times when you just want to arrange the data.

When the Input is converted, it becomes as follows.

>> pd.DataFrame.from_dict(d1)
   A  B
i  0  2
j  1  3

The output is converted as follows.

>> pd.DataFrame(d2,index=[""])
  A_i  A_j  B_i  B_j
    0    1    2    3

I thought it was a good idea, but I didn't understand pandas very well.

Can be decomposed naturally using MultiIndex in Pandas

You can edit it with the MultiIndex function in practice without having to disassemble it. https://qiita.com/Morinikiz/items/40faa91e7a83807c0552

--It can be summarized by stacking () or unstack () df. --Stack can put the column side at a high level, unstack can put the row side at a high level ――However, why is the behavior opposite to Explanation here?

>>> df = pd.DataFrame(d1)
>>> df
   A  B
i  0  2
j  1  3
>>> df.unstack()
A  i    0
   j    1
B  i    2
   j    3
dtype: int64
>>> df.stack()
i  A    0
   B    2
j  A    1
   B    3
dtype: int64

To return to the Data Frame, use to_frame () at the end.

>>> df.unstack().to_frame("name")
     name
A i     0
  j     1
B i     2
  j     3

Recommended Posts

Expand a Python nested dictionary to do something like Pandas' MultiIndex
I want to do something like sort uniq in Python
Do something like a Python interpreter in Visual Studio Code
[Python] How to force a method of a subclass to do something specific
I wanted to do something like an Elixir pipe in Python
Do you make something like a rocket?
Do something like Redis transactions in Python
I want to improve efficiency with Python even in an experimental system (3) I want to do something like Excel with Pandas
[Python] Road to a snake charmer (6) Manipulate Pandas
[Python] How to expand variables in a character string
How to write a list / dictionary type of Python3
Forcibly draw something like a flowchart with Python, matplotlib
[Python] I want to make a nested list a tuple
[Python] A memo to write CSV vertically with Pandas
I made a python library to do rolling rank
I want to do something in Python when I finish
It's a hassle to write "coding: utf-8" in Python, so I'll do something with Shellscript
How to convert an array to a dictionary with Python [Application]
How to import a file anywhere you like in Python
[Python] How to output a pandas table to an excel file
Create a dictionary in Python
[Python] Convert list to Pandas [Pandas]
A road to intermediate Python
Add a dictionary to MeCab
[Road to Python Intermediate] Call a class instance like a function with __call__
What to do if you get a minus zero in Python
[Python] How to add rows and columns to a table (pandas DataFrame)
I want to do a full text search with elasticsearch + python
How to check the memory size of a dictionary in Python
Let's feel like a material researcher with python [Introduction to pymatgen]
[Machine learning] I tried to do something like passing an image
[Python] What to do if you get a ModuleNotFoundError when importing pandas using Jupyter Notebook in Anaconda
How to write a Python class
[Python] How to do PCA in Python
[Python] How to use Pandas Series
Something like JS setTimeout in python
Do you need a Python re.compile?
Create a nested dictionary using defaultdict
To do tail recursion with Python2
[Introduction to Python] Let's use pandas
What to do with PYTHON release?
Metaclass (wip) to generate a dictionary
I want to do ○○ with Pandas
[Introduction to Python] Let's use pandas
5 Ways to Create a Python Chatbot
[Introduction to Python] Let's use pandas
Something like tail -f in Python
What to do if there is a decimal in python json .dumps
[Introduction to Udemy Python3 + Application] 47. Process the dictionary with a for statement
Note: [Python3] Convert datetime to a string in any format you like
[Python / Tkinter] Search for Pandas DataFrame → Create a simple search form to display
[Python / Pandas] A bug occurs when trying to replace a DataFrame with `None` with` replace`
[Python] How to read a csv file (read_csv method of pandas module)
Q. Do you want to do something like generics that takes value from []?
What is the fastest way to create a reverse dictionary in python?
I want to do a monkey patch only partially safely in Python