I / O related summary of python and fortran

Introduction

Regarding the input / output of python <-> fortran, it is difficult to find a cohesive description on the Internet, so I decided to summarize it as far as I know.

As an aside, np.savez and np.load are recommended for data input / output between pythons.

Read python output with fortran

How to use numpy

Output data according to the following procedure

  1. Prepare a numpy array
  2. Convert to fit fortran with reshape
  3. Convert to binary with ʻas type`
  4. Output with tofile

An example of python code that outputs a 3D double precision real number with little endian

import numpy as np

ix, jx, kx = 3, 4, 5
endian='<'
a = np.ones((ix,jx,kx))
a.reshape([ix*jx*kx],order='F').astype(endian+'d').tofile('test.dat')

To read this with fortran, do as follows. Suppose the calculator you are using is little endian. ʻAccess ='stream' is required for ʻopen in fortran

program test
  implicit none

  integer :: idf
  integer, parameter :: ix = 3, jx = 4, kx = 5
  real(KIND(0.d0)), dimension(ix,jx,kx) :: a

  open(newunit=idf, file='test.dat',form='unformatted',access='stream')
  read(idf) a
  close(idf)
  
  stop
end program test

If you want to exchange big endian data, use python code

endian='>'

You can change it to. This ʻendian variable controls how to convert to binary with ʻas type

In fortran code

  open(newunit=idf, file='test.dat',form='unformatted',access='stream',convert='big_endian')

And. If you want to output / input in single precision, rewrite the python code as follows

import numpy as np

ix, jx, kx = 3, 4, 5
endian='<'
a = np.ones((ix,jx,kx),dtype=np.float32)
a.reshape([ix*jx*kx],order='F').astype(endian+'f').tofile('test.dat')

Changed the parts of np.ones and ʻas type` to handle single precision.

The corresponding fortran code looks like this:

program test
  implicit none

  integer :: idf
  integer, parameter :: ix = 3, jx = 4, kx = 5
  real(KIND(0.e0)), dimension(ix,jx,kx) :: a

  open(newunit=idf, file='test.dat',form='unformatted',access='stream')
  read(idf) a
  close(idf)
  
  stop
end program test

Only the part of real (KIND (0.e0)) was changed.

If you want to output multiple arrays to one file, follow the steps below.

  1. Combine multiple arrays with np.array
  2. Use reshape to make a one-dimensional array.
  3. Swap the index positions with T.
import numpy as np

ix, jx, kx = 3, 4, 5
endian='<'
a = np.ones((ix,jx,kx),dtype=np.float32)
b = np.ones((ix,jx,kx),dtype=np.float32)*2
c = np.ones((ix,jx,kx),dtype=np.float32)*3
np.array([a,b,c]).T.reshape([3*ix*jx*kx],order='F').astype(endian+'f').tofile('test.dat')

The fortran code to read is

program test
  implicit none

  integer :: i,j,k
  integer :: idf
  integer, parameter :: ix = 3, jx = 4, kx = 5
  real(KIND(0.e0)), dimension(ix,jx,kx) :: a,b,c

  open(newunit=idf, file='test.dat',form='unformatted',access='stream')
  read(idf) a,b,c
  close(idf)
  
  stop
end program test

Should be

On the other hand, when outputting arrays of different sizes

  1. Use reshape to make a one-dimensional array
  2. Use np.concatenate to organize the arrays You just have to follow the procedure
The python code looks like this

import numpy as np

ix, jx, kx = 3, 4, 5
lx, mx = 2, 4
endian='<'
a = np.ones((ix,jx,kx))
b = np.ones((lx,mx))*2

a1 = a.reshape([ix*jx*kx],order='F')
b1 = b.reshape([lx*mx],order='F')
np.concatenate([a1,b1]).astype(endian+'d').tofile('test.dat')

When reading from fortran, it is the same as before, but it is as follows.

program test
  implicit none

  integer :: idf
  integer, parameter :: ix = 3, jx = 4, kx = 5
  integer, parameter :: lx = 2, mx = 4
  real(KIND(0.d0)), dimension(ix,jx,kx) :: a
  real(KIND(0.d0)), dimension(lx,mx) :: b

  open(newunit=idf, file='test.dat',form='unformatted',access='stream')
  read(idf) a,b
  close(idf)
  
  stop
end program test

How to use scipy.io.FortranFile

The above method is easier for python code, but fortran has to edit the data headers by yourself unless ʻaccess ='stream'is used. It isscipy.io.FortranFile` that adjusts this part well.

Python code looks like this

import numpy as np
from scipy.io import FortranFile

ix, jx, kx = 3, 4, 5
endian='<'
a = np.ones((ix,jx,kx))
f = FortranFile('test.dat','w')
f.write_record(a)
f.close()

The Fortran code for reading is

program test
  implicit none

  integer :: i,j,k
  integer :: idf
  integer, parameter :: ix = 3, jx = 4, kx = 5
  real(KIND(0.d0)), dimension(ix,jx,kx) :: a

  open(newunit=idf, file='test.dat',form='unformatted')
  read(idf) a
  close(idf)
  
  stop
end program test

It should be done.

Outputting multiple arrays is easier than using Numpy

a = np.ones((ix,jx,kx))
b = np.ones((ix,jx,kx))*2
f = FortranFile('test.dat','w')
f.write_record(a,b)
f.close()

And just add it to the argument of write_record. Fortran reading code

program test
  implicit none

  integer :: i,j,k
  integer :: idf
  integer, parameter :: ix = 3, jx = 4, kx = 5
  real(KIND(0.d0)), dimension(ix,jx,kx) :: a,b

  open(newunit=idf, file='test.dat',form='unformatted')
  read(idf) a,b
  close(idf)
  
  stop
end program test

All you have to do is arrange the arrays to be read side by side.

However, if you look at the scipy website

Since this is a non-standard file format, whose contents depend on the compiler and the endianness of the machine, caution is advised. Files from gfortran 4.8.0 and gfortran 4.1.2 on x86_64 are known to work.

Consider using Fortran direct-access files or files from the newer Stream I/O, which can be easily read by numpy.fromfile.

it is written like this. The fortran format seems to be highly compiler-dependent, and the method using numpy seems to be recommended.

Read fortran output with python

How to use numpy

In order to read fortran data with numpy, it is recommended to output with ʻaccess ='stream'` (it is not essential because you can adjust the header etc. yourself).

Define an array with fortran and output

program test
  implicit none

  integer :: i,j,k
  integer :: idf
  integer, parameter :: ix = 3, jx = 4, kx = 5
  real(KIND(0.d0)), dimension(ix,jx,kx) :: a

  a = 1.d0
  
  open(newunit=idf, file='test.dat',form='unformatted',access='stream')
  write(idf) a
  close(idf)
  
  stop
end program test

The simplest way to load it in python is to use np.fromfile.

import numpy as np

ix, jx, kx = 3, 4, 5
endian='<'
f = open('test.dat','rb')
a = np.fromfile(f,endian+'d',ix*jx*kx)
a = a.reshape((ix,jx,kx),order='F')
f.close()

Even if you use np.fromfile, you can use np.dtype for more versatile reading. For example, do as follows

import numpy as np

ix, jx, kx = 3, 4, 5
endian='<'
f = open('test.dat','rb')
dtype = np.dtype([('a',endian+str(ix*jx*kx)+'d')])
a = np.fromfile(f,dtype=dtype)['a']
a = a.reshape((ix,jx,kx),order='F')
f.close()

It is more convenient to use np.dtype when dealing with multiple arrays. First, fortran outputs multiple arrays to one file as shown below.

program test
  implicit none

  integer :: i,j,k
  integer :: idf
  integer, parameter :: ix = 3, jx = 4, kx = 5
  real(KIND(0.d0)), dimension(ix,jx,kx) :: a,b

  a = 1.d0
  b = 2.d0
  
  open(newunit=idf, file='test.dat',form='unformatted',access='stream')
  write(idf) a,b
  close(idf)
  
  stop
end program test

To load this in python:

import numpy as np

ix, jx, kx = 3, 4, 5
endian='<'
f = open('test.dat','rb')
dtype = np.dtype([('a',endian+str(ix*jx*kx)+'d'),('b',endian+str(ix*jx*kx)+'d')])
data = np.fromfile(f,dtype=dtype)
a = data['a'].reshape((ix,jx,kx),order='F')
b = data['b'].reshape((ix,jx,kx),order='F')
f.close()

How to use scipy.io.FortranFile

It is easy to use scipy.io.FortranFile to read fortran data that does not use ʻaccess ='stream'`.

The output of fortran is as follows

program test
  implicit none

  integer :: i,j,k
  integer :: idf
  integer, parameter :: ix = 3, jx = 4, kx = 5
  real(KIND(0.d0)), dimension(ix,jx,kx) :: a

  a = 1.d0
  
  open(newunit=idf, file='test.dat',form='unformatted')
  write(idf) a
  close(idf)
  
  stop
end program test

The python code to read this looks like this

import numpy as np
from scipy.io import FortranFile

ix, jx, kx = 3, 4, 5
endian='<'
a = np.ones((ix,jx,kx))
b = np.ones((ix,jx,kx))*2
f = FortranFile('test.dat','r')
a = f.read_record(endian+'('+str(ix)+','+str(jx)+','+str(kx)+')d')
f.close()

It can also be read using np.dtype as follows:

import numpy as np
from scipy.io import FortranFile

ix, jx, kx = 3, 4, 5
endian='<'
a = np.ones((ix,jx,kx))
b = np.ones((ix,jx,kx))*2
f = FortranFile('test.dat','r')
dtype = np.dtype([('a',endian+str(ix*jx*kx)+'d')])
a = f.read_record(dtype=dtype)
a = a['a'].reshape(ix,jx,kx,order='F')
f.close()

Recommended Posts

I / O related summary of python and fortran
Summary of Python indexes and slices
Correspondence summary of array operation of ruby and python
Summary of the differences between PHP and Python
Installation of Python3 and Flask [Environment construction summary]
Python iteration related summary
Summary of Python arguments
Python --Explanation and usage summary of the top 24 packages
[Python] Type Error: Summary of error causes and remedies for'None Type'
I checked out the versions of Blender and Python
Summary of date processing in Python (datetime and dateutil)
Summary of python file operations
I compared Java and Python!
Asynchronous I / O and non-blocking I / O
Source installation and installation of Python
[python] Summary of how to retrieve lists and dictionary elements
Learn "English grammar" instead of Python and AI related English words. .. ..
[Python] Summary of how to use split and join functions
I want to know the features of Python and pip
Environment construction of python and opencv
The story of Python and the story of NaN
Installation of SciPy and matplotlib (Python)
A brief summary of Python collections
H29.2.27 ~ 3.5 Summary of what I did
This and that of python properties
I played with PyQt5 and Python3
Coexistence of Python2 and 3 with CircleCI (1.0)
Reputation of Python books and reference books
[OpenCV; Python] Summary of findcontours function
I compared the speed of Hash with Topaz, Ruby and Python
[Introduction to Python] I compared the naming conventions of C # and Python.
[Python] I thoroughly explained the theory and implementation of logistic regression
I want to use both key and value of Python iterator
Summary of differences between Python and PHP (comparison table of main items)
Installation of Visual studio code and installation of python
[Python] Summary of how to use pandas
Python Summary
I compared python3 standard argparse and python-fire
Extraction of tweet.js (json.loads and eval) (Python)
Summary of various for statements in Python
[Python] Summary of array generation (initialization) time! !! !!
Connect a lot of Python or and and
Python summary
Learn Python and AI related English words. .. ..
[Python] Summary of conversion between character strings and numerical values (ascii code)
Summary of numpy functions I didn't know
[Python2.7] Summary of how to use unittest
I installed and used Numba with Python3.5
I have 0 years of programming experience and challenge data processing with python
I tried to verify and analyze the acceleration of Python by Cython
The Python project template I think of.
Summary of modules and classes in Python-TensorFlow2-
Summary of built-in methods in Python list
Summary of useful techniques for Python Scrapy
Summary of how to use Python list
Easy introduction of python3 series and OpenCV3
[Python] Various combinations of strings and values
[Python2.7] Summary of how to use subprocess
Axis option specification summary of Python "numpy.sum (...)"
I measured the speed of list comprehension, for and while with python2.7.
Idempotent automation of Python and PyPI setup