Parse the Researchmap API in Python and automatically create a Word file for the achievement list

Dear researchers, Thank you for your hard work today. It was awkward to copy and paste the achievements entered in the researchmap, so I tried to parse API. Let's go as far as creating a Word file from Python.

environment

• MacOS Mojave 10.14.5
• Anaconda 2020.02
• Python 3.7.6
• Jupyter Notebook 6.0.3

Install python-docx

Please see here. But it's one line:

bash

pip install python-docx

JSON parsing

I'm a weak JSON person, so what are @ id and @ type? !! ?? !! I did it with tears. Please let me know if there is a more efficient method. (By the way, I don't really understand the scope of the word "parse". I think it refers to the point where you get JSON and retrieve the necessary information, but please tell me if you make a mistake. )

Get JSON

I referred to here.

python

import requests
import json

url = "https://api.researchmap.jp/kage"
response = requests.get(url)
jsonData = response.json()

You can get data like this. (When I used the print statement, it was written tightly without line breaks, so I didn't use it.)

Extract the required data from JSON

Identifying the location of performance data

I'm a weak JSON person, so scroll through the displayed JSON and take a closer look at the structure. It seems that performance data is stored in the part called @ graph.

Let's display some of the elements of @ graph.

python

for i in range(len(jsonData['@graph'])):
    print(i)
    print(jsonData['@graph'][i]['@type'])

Output result

0
research_interests
1
research_areas
2
research_experience
3
published_papers
4
books_etc
5
misc
6
presentations
7
awards
8
research_projects
9
education
10
teaching_experience
11
committee_memberships

Performance data is included in this. Let's extract the published_papers of element 3.

Display published_papers like a performance list

I was able to display it like that with the script below.

python

for i in range(len(jsonData['@graph'][3]['items'])):
    author_list = []
    authors = jsonData['@graph'][3]['items'][i]['authors']['en']
    title = jsonData['@graph'][3]['items'][i]['paper_title']['en']
    journal = jsonData['@graph'][3]['items'][i]['publication_name']['en']
    date =  jsonData['@graph'][3]['items'][i]['publication_date']
    vol = jsonData['@graph'][3]['items'][i]['volume']
    doi = jsonData['@graph'][3]['items'][i]['identifiers']['doi'][0]
    
    #Arrange the appearance of the author
    for j in range(len(authors)):
        author = ''.join(authors[j].values())
        author_list.append(author)
    
    print(', '.join(author_list))
    print(title)
    print(journal, vol, date)
    print('doi:', doi)
    print('')

You can copy and paste this into a Word file, but it's a big deal, so let's do our best.

Writing to a Word file

List with numbers in python-docx

I want to make a list of achievements with numbers. First, let's test the list writing. I referred to here.

python

from docx import Document

doc = Document()
docx_file = 'test.docx'

p0 = doc.add_paragraph('Item 1', style='List Number')
p1 = doc.add_paragraph('Item 2', style='List Number')

doc.save(docx_file)

A Word file has been created and the list has been written.

Format performance data for writing

A little change from the print statement above.

python

#List for storing all paper data
papers = []

for i in range(len(jsonData['@graph'][3]['items'])):
    author_list = []
    authors = jsonData['@graph'][3]['items'][i]['authors']['en']
    title = jsonData['@graph'][3]['items'][i]['paper_title']['en']
    journal = jsonData['@graph'][3]['items'][i]['publication_name']['en']
    date =  jsonData['@graph'][3]['items'][i]['publication_date']
    vol = jsonData['@graph'][3]['items'][i]['volume']
    doi = jsonData['@graph'][3]['items'][i]['identifiers']['doi'][0]
    
    #Arrange the appearance of the author
    for j in range(len(authors)):
        author = ''.join(authors[j].values())
        author_list.append(author)
    
    #Format the treatise and store it in the list
    paper = [', '.join(author_list), title, ', '.join([journal, vol, date]), ' '.join(['doi:', doi])]
    papers.append('\n'.join(paper))

Write the list papers to Word.

Writing data to a Word file

I referred to here for adding headings.

python

doc = Document()
docx_file = 'publication.docx'

#title
heading = doc.add_heading('Research achievements', level=1)
subheading = doc.add_heading('Original treatise', level=2)

#Treatise list
for paper in papers:
    p = doc.add_paragraph(paper, style='List Number')

doc.save(docx_file)

Data has been written to the Word file! The rest is just to arrange the fine appearance. While looking at here, let's make all the font colors black for the time being.

python

from docx.shared import RGBColor

doc = Document()
docx_file = 'publication.docx'

#title
heading = doc.add_heading('Research achievements', level=1)
heading.runs[0].font.color.rgb = RGBColor(0, 0, 0)
subheading = doc.add_heading('Original treatise', level=2)
subheading.runs[0].font.color.rgb = RGBColor(0, 0, 0)

#Treatise list
for paper in papers:
    p = doc.add_paragraph(paper, style='List Number')

doc.save(docx_file)

You can now create a Word file of your achievement list from researchmap without manual copy and paste!

reference

-About WebAPI -Read the old Word file (.doc) of the Gakushin DC application form from Python and operate it -Hit Web API in Python to parse JSON

• Bullet Lists in python-docx -Edit Microsoft Word document with python-docx (1) Basic editing operations -Rewrite docx file using python-docx