Batch processing notes in IPython Notebook

Very useful as an IPython Notebook, interactive data analysis, programming environment. And if you use it heavy, you will want to write and execute code like batch processing. I will summarize how to execute the .ipynb file when the note is saved. First of all, two.

Generate a .py file with the ipython nbconvert command

Before I knew it, nbconvert was incorporated into iPython itself.

ipython nbconvert mynote.ipynb --to=python

For example, try running the next nightly batching note on `` `nbconvert```.

Note that the generated source does not include the startup script even if you specify the profile as follows. Also, the code that assumes pylab mode will not work.

# coding: utf-8

# In[19]:

from pymongo import MongoClient
from datetime import timedelta, datetime


# In[26]:

#Process local MongoDB
con = MongoClient('localhost')
db = con['test']


# In[27]:

#Date to be processed(Aggregate the day before after midnight)
process_date = datetime.now().date() - timedelta(days=1)
process_date


# In[28]:

xxx_count = 0
yyy_count = 0
with db.events.find({'time': {}}) as cur: # from-to specification omitted
    for doc in cur:
        pass
        #Some kind of aggregation


# In[ ]:

#Update of aggregation result
db.daily_state.update({'date': process_date}, {'$set': 'values': {
    'xxx_count': xxx_count,
    'yyy_count': yyy_count
}})

Use runipy to run as .ipynb file

paulgb/runipy https://github.com/paulgb/runipy

A command that parses .ipynb files and executes the Python code inside in ipython in sequence.

runipy mynote.ipynb

It is similar to nbconvert, but the result after execution can be output as an HTML file. It can be used for purposes such as charging cron and generating daily reports. You can also pass arbitrary parameters used in python scripts from the command line. However, since IPython profile cannot be specified, it is difficult to use if you make full use of the setting of ipython_notebook_config.py.

It is inconvenient that Profile cannot be specified, so the version that allows --profile-dir to be specified by forking has already been pulled.

hagino3000/runipy https://github.com/hagino3000/runipy

Development of runipy seems to be stagnant, so those who want to use it should commit with all their might.