Test & Debug Tips: Create a file of the specified size in Python

Introduction

This is a trick that can be used for testing and debugging. Want to create a large file with random content for testing? .. Moreover, it is instant.

Create a huge file "instantly" on one command line

Create a 1024 byte file

Execution example (creating a 1024-byte file)


C:\>python -c "import sys; import numpy; sys.stdout.buffer.write(numpy.random.bytes(1024));" > 1024.bin

Create a 1MB file

Execution example (1MB/1,048,576 byte file creation)


C:\>python -c "import sys; import numpy; sys.stdout.buffer.write(numpy.random.bytes(1048576));" > 1048576.bin

Create a 1GB file

Execution example (1MB/1,073,741,824-byte file creation)


python -c "import sys; import numpy; sys.stdout.buffer.write(numpy.random.bytes(1073741824));" > 1073741824.bin

It took about 1 minute to create a 1GB file with Core i7-5500U-2.4GHz/SSD-DISK. (If you want a zero-padded file, please use fsutil. It will be finished in an instant)

See the result

Execution result


C:\work>dir
2021/01/11  10:00             1,024 1024.bin
2021/01/11  10:08         1,048,576 1048576.bin
2021/01/12  19:28     1,073,741,824 1073741824.bin

Summary

I posted a method to realize it instantly only on the command line. I hope it helps you when you are busy.

bonus

An error occurs with sys.stdout.write ()

Execution example that causes an error

>>> sys.stdout.write(numpy.random.bytes(1024))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: write() argument must be str, not bytes

I got an error when I tried to output. The part-time job is useless. I thought it was okay because numpy.random.bytes is returned as a string type like "b'\ xe5'". ..

What is the type of sys.stdout?

C:\>python
>>> import sys;
>>> print(type(sys.stdout));
<class '_io.TextIOWrapper'>

The type of sys.stdout is'_io.TextIOWrapper'.

What is class'_io.TextIOWrapper'?

────────────────────────── class '_io.TextIOWrapper' A buffered text stream providing higher-level access to a BufferedIOBase buffered binary stream. It inherits TextIOBase. (** Inherit TextIOBase **) ────────────────────────── '_io.TextIOWrapper' was a class for text processing. ** I have no choice but to look for a byte interface **. It is said to inherit'TextIOBase'.

What is class io.TextIOBase?

────────────────────────── class io.TextIOBase Base class for text streams. This class provides a character and line based interface to stream I/O. It inherits IOBase. There is no public constructor. ────────────────────────── This class had a member called buffer that accessed the binary IF.

What is io.TextIOBase.buffer (sys.stdout.buffer)?

────────────────────────── buffer The underlying binary buffer (BufferedIOBase instance) that TextIOBase handles. ────────────────────────── It seems that you can use buffer for binary output.

Recommended Posts

Test & Debug Tips: Create a file of the specified size in Python
I made a program to check the size of a file in Python
Create a Python image in Django without a dummy image file and test the image upload
Create a binary file in Python
[Note] Import of a file in the parent directory in Python
Write the test in a python docstring
[GPS] Create a kml file in Python
[Python] Read the specified line in the file
How to check the memory size of a variable in Python
How to check the memory size of a dictionary in Python
Get the caller of a function in Python
How to create a JSON file in Python
Output in the form of a python array
Create a MIDI file in Python using pretty_midi
Various ways to read the last line of a csv file in Python
Python scikit-learn A collection of predictive model tips often used in the field
Python scikit-learn A collection of predictive model tips often used in the field
Get the size (number of elements) of UnionFind in Python
A reminder about the implementation of recommendations in Python
Create a function in Python
Create a dictionary in Python
Download the file in Python
I created a script to check if English is entered in the specified position of the JSON file in Python.
Output the specified table of Oracle database in Python to Excel for each file
[Python] Open the csv file in the folder specified by pandas
Create a local scope in Python without polluting the namespace
Change the standard output destination to a file in Python
Get the number of specific elements in a python list
[Tips] Problems and solutions in the development of python + kivy
Google search for the last line of the file in Python
Find the eigenvalues of a real symmetric matrix in Python
Save the binary file in Python
Create a DI Container in Python
Get the value of a specific key up to the specified index in the dictionary list in Python
[Python] Programming to find the number of a in a character string that repeats a specified number of times.
The story of the "hole" in the file
Parse the Researchmap API in Python and automatically create a Word file for the achievement list
I tried to create a Python script to get the value of a cell in Microsoft Excel
Create a Kubernetes Operator in Python
Create an executable file (EXE) by PyInstaller in a hybrid environment (Nimporter) of Python + Nim
Create a random string in Python
Create a docx file with thumbnails of photos in the folder pasted with the Python python-docx library and the free software "Reduction only."
Create an instance of a predefined class from a string in Python
Process the contents of the file in order with a shell script
How to determine the existence of a selenium element in Python
Create a shell script to run the python file multiple times
Read the standard output of a subprocess line by line in Python
A function that measures the processing time of a method in python
Read a file in Python with a relative path from the program
How to create a large amount of test data in MySQL? ??
The idea of feeding the config file with a python file instead of yaml
Tips: [Python] Calculate the average value of the specified area with bedgraph
Dig the directory and create a list of directory paths + file names
Create a function to get the contents of the database in Go
Get the formula in an excel file as a string in Python
Get the number of readers of a treatise on Mendeley in Python
Trial of writing the configuration file in Python instead of .ini etc.
Create a compatibility judgment program with the random module of python.
Check the behavior of destructor in Python
Display a list of alphabets in Python 3
Check the existence of the file with python