[PYTHON] About endian

What is endian?

Endian is the byte order in a binary file.

For example, suppose some data is stored in 2 bytes. 00000001 00000000 At this time, which byte indicates a value from 0 to 255 (that is, from the smallest digit to the 1,2,4,8 ... digit). The other represents a value between 256 and 65535 (ie, from the smallest to 256,512,1024 ...).

For big endian, the first byte represents a larger number. In the above example, the first byte is in the order of 256 to 32768, and the second byte is in the order of 1 to 128. This is the same number we usually see.

On the contrary, in the case of little endian, the first bite is smaller.

In addition, the last number in the byte is still the smallest (?) (← I'm not sure, so please comment if you understand)

In modern times, many systems have adopted Intel chips, so both Win and Mac have become little endian, but big endian data is rolling around in the streets.

If you touch the data of big endian with the intention of little endian, the result will be strange and it will hurt, so be careful. This page summarizes how to switch endianness (for my favorite language).

Fortran In Fortran, there is a method of originally determining the endianness at compile time and a method of switching when reading a file in a program.

Decide at compile time

This is super easy. If you are using gfortran, compile it as follows. However, be aware that everything in the program will be big endian.

gfortran -fconvert=big-endian -o test test.f90

Change for each device number with environment variables

export F_UFMTENDIAN="little;big 87"

This makes basic little endian and device number 87 big endian. Of course, you can flip big and small over. You can omit the rest from ;.

Python Easy to decide when reading a file. For example, suppose you want to read data to create an image. First, put the data of the opened file in the array.

imagearr=np.fromfile(rfile,'<h',NROWS*NCOLS*1)

An array that contains the original data that ʻimagearrcreates the image for. npis numpy. rfileis the directory of files. NROWS * NCOLS * 1` indicates the number of rows x the number of columns.

Here, <h indicates the type of variable to be read as endian. The first < indicates little endian. h indicates int16.

Other examples are shown below.

characters byte order size Alignment
@ native native native
= native standard none
< Little endian standard none
> Big endian standard none
! Network (= Big Endian) standard none
Format Type in C Python type standard size Remarks
x pad byte no value    
c char String of length 1 1  
b signed char Integer type (integer) 1 (3)
B unsigned char Integer type 1 (3)
? _Bool boolean type (bool) 1 (1)
h short Integer type 2 (3)
H unsigned short Integer type 2 (3)
i int Integer type 4 (3)
I unsigned int Integer type 4 (3)
l long Integer type 4 (3)
L unsigned long Integer type 4 (3)
q long long Integer type 8 (2), (3)
Q unsigned long long Integer type 8 (2), (3)
f float Floating point type 4 (4)
d double Floating point type 8 (4)
s char[] string    
p char[] string    
P void * Integer type   (5), (3)

Source: http://docs.python.jp/2/library/struct.html

Recommended Posts

About endian
About LangID
About CAGR
About virtiofs
About python-apt
About Permission
About sklearn.preprocessing.Imputer
About gunicorn
About requirements.txt
About locale
About permissions
About Opencv ②
About axis = 0, axis = 1
About Opencv ③
About import
About numpy
About pip
About Linux
About numpy.newaxis
About Linux
About import
About Opencv ①
About Linux
About Linux
About Linux ①
About cv2.imread
About _ and __
About wxPython
Notepad about TecoGAN
About python slices
Briefly about __name__
About python comprehension
About Docker Volume
About reference type
About Twitter scraping
Learn about programming
About Flask customization
About Python tqdm.
About python yield
Notes about with
About python, class
About Linear Models
About Go functions
About pandas describe
About Kivy root
About Firestore timeout
About python inheritance
About python, range ()
About Confusion Matrix
[Linux] About PATH
About python decorators
Linux (about groups)
Note about awk
About python reference
About Bitnami Autostart
About Python decorators
About Milkcocoa SDK
Notes about pytorch
[Python] About multi-process
About Numpy broadcast
About symbolic links