[PYTHON] What to do when gdal_merge creates a huge file

gdal_merge creates a huge file ...

Gdal module that handles tif data etc. gdal_merge.py is a script in that module that merges tif data based on location information.

Official Documents

When I ran the script (below) in the document ...

python gdal_merge.py -init 255 -o out.tif in1.tif in2.tif

Even though I just combined two files of about 150MB, a huge file over 30GB was generated. (I looked back many times that the Finder might have mistaken G and M)

Solution

It seems that the tif file is basically compressed, and if you add a tag indicating the compression method to the script above, it will be compressed. It seems that a large file will be created by synthesizing images at distant points in order to compress parts that have no value.

python gdal_merge.py -init 255 -o out.tif in1.tif in2.tif -co COMPRESS=DEFLATE

This will generate a composite file of about 150MB. By the way, there seems to be a compression method with high file compatibility, although the compression rate will be slightly lower.

python gdal_merge.py -init 255 -o out.tif in1.tif in2.tif -co -co COMPRESS=LZW

Recommended Posts

What to do when gdal_merge creates a huge file
What to do when Ubuntu crashes
[AWS] What to do when the ping command causes a "timeout"
What to do when a video cannot be read by cv2.VideoCapture
[Memorandum] What to do when a warning appears after executing pip list
What to do if fprintd requires a password when registering your fingerprint
What to do when a warning message is displayed in pip list
What to do when PermissionError of tempfile.mkstemp occurs
What to do when a warning appears around Python integration in Neovim's CheckHealth
What to do when xlrd.biffh.XLRDError: Unsupported format, or corrupt file: Expected BOF record;
What to do when [Errno 2] No such file or directory appears in Python
What to do when you want to receive files from a Windows client remotely
[Go 1.13] What to do when unexpected directory layout: appears
[openpyxl] What to do when IllegalCharacterError appears in pandas.DataFrame.to_excel
What to do when raise ValueError, "unsupported hash type"
What to do if a UnicodeDecodeError occurs in pip
What to do when "cannot import name xxx" [Python]
What to do when you can't bind CaboCha to Python
What to do if you get a memory error when converting from PySparkDataFrame to PandasDataFrame
Upload a file to Dropbox
[AWS] What to do when you want to pip with Lambda
What to do when Japanese is not displayed on matplotlib
What to do if pip gives a DistributionError in Homebrew
What to do when PyCharm font is strange or garbled
What to do when Unalignable boolean Series provided as indexer
What to do if a 0xC0000005 error occurs in tf.train.start_queue_runners ()
What to do when an error occurs with import _ssl
What to do if package installation fails when deploying to heroku
What to do when "SSL: CERTIFICATE_VERIFY_FAILED _ssl.c: 1056" appears in Python
OSError: [Errno 40] What to do when Message too long appears
What to do when "Invalid HTTP_HOST header" appears in Django
ImportError: No module What to do when you are told
What to do when a Missing artifact occurs in a jar that is not defined in pom.xml
What I was addicted to when dealing with huge files in a Linux 32bit environment
[Beginner] What to do when "[Errno 2] File b'test.csv' does not exist: b'test.csv" is displayed when reading pandas csv
What to do if you cat or tail a binary file and the terminal is garbled
What to do if yum breaks
What to do with Magics install
What to do if you get a minus zero in Python
What to do if you grep a text file and it becomes Binary file (standard input) matches
Write standard output to a file
What to do if python says "fatal error:'stdio.h' file not found"
What to do when the jupyterlab extension settings are not reflected
What to do if you get a UnicodeDecodeError with pip install
What to do with PYTHON release?
What to do if Insecure Platform Warning appears when running Python
What to do when a part of the background image becomes transparent when the transparent image is combined with Pillow
What to do if you get a must override `get_config` error when trying to model.save in Keras
What to do when you get "I can't see the site !!!!"
What to do when UnicodeDecodeError occurs during read_csv in pandas (pd.read_table ())
[Django] I wanted to test when POSTing a large file [TDD]
What to do to get tensorflow-gpu to work
What to do when ModuleNotFoundError: No module named'XXX' occurs in Python
How to create a config file
What to do when the value type is ambiguous in Python?
A story about what to do when a bad interpreter: Not such file or directory appears in Anaconda3 and how to investigate the cause.
What to do if you get a "Wrong Python Platform" warning when using Python with the NetBeans IDE
[Mac OS] What to do when Python is not installed as a framework. Is displayed when import matplotlib is performed.
What to do if you get a Permission denied (public key) error when trying to pull on Github
[Python] What to do if you get a ModuleNotFoundError when importing pandas using Jupyter Notebook in Anaconda
What to do if there is a decimal in python json .dumps