[PYTHON] Image analysis performed with google Colab for those who start space research

Introduction

For those who are new to fits files. Since the image of the galaxy is familiar, it is a sample that works only with google Colab using this as an example. There is also How to search using python's astroquery and get fits images with skyview written for more advanced users.

Basic edition

Server and client basics

In space astronomical, a large amount of data is exchanged, so knowledge about servers and clients is required. Client server system is a basic concept in space astronomical, so let's know. Simply put, a server is an OS or software group that has the ability to provide information to many devices on the network, and the client is a useless machine that retrieves data from the server.

There are considerable differences depending on the OS. In space astronomical, I think that linux-based servers are almost 100%, so if you know the basic interaction and mechanism with the server in advance, it will be a little easier to acquire space astronomical data. think. What is a server OS? etc.

urllib example

If you want to use the server and client mechanism, it's very easy now, so let's use it comfortably first. For example, let's get the file of a famous page called heasoft from NASA.

python

import urllib.request
from bs4 import BeautifulSoup
url = 'https://heasarc.gsfc.nasa.gov/docs/software/heasoft/'
req = urllib.request.Request(url)
html = urllib.request.urlopen(req)
soup = BeautifulSoup(html, "html.parser")
topicsindex = soup.find('div', attrs={'class': 'navigation'})

Now, the information of all html is acquired in soup. The browser is displaying this. From this, only the part called navigation was acquired, which is the topicsindex.

python

print(topicsindex) # extract sentences begining with div navigation 
<div class="navigation">
<div id="nasaHeader">
<div>
....

Only some information is displayed. Note that this is the task of accessing the NASA server from the client and getting the data, which can be written in a few lines of program. I want you to recognize that acquiring data is not just about pressing various buttons with the mouse from the browser to acquire data, and in the case of space astronomical, acquiring data with such a command line. Often.

astroquery settings

python

pip install astroquery 

Install astroquery with. This is a tool for retrieving data with the same coding from many catalogs such as skyview and vizier.

How to use vizier

vizier is a catalog extracted for each paper. For example, searching for blackhole reveals a list of papers containing a large number of catalogs. You can quickly make simple plots on vizier. Since it is also linked to the paper, it is convenient to be able to quickly trace the original information. From astroquery,

python

# download galaxy data via Vizier
from astroquery.vizier import Vizier 
v = Vizier(catalog="J/ApJS/80/531/gxfluxes",columns=['Name',"logLx","Bmag","dist","type"],row_limit=-1)
data = v.query_constraints()
sname = data[0]["Name"] 
namelist = []
olist=["HRI","DSS"]

J/ApJS/80/531/gxfluxes is a catalog of galaxies, from here,'Name', "logLx", The data of "Bmag", "dist", "type" is extracted, and only Name is extracted by setting sname = data [0] ["Name"].

How to use skyview

skyview is a convenient catalog for acquiring images of celestial bodies. This time, let's get an image of the galaxy from here.

python

# get image from skyview
from astroquery.skyview import SkyView
name=namelist[0]
paths = SkyView.get_images(position=name, survey=olist)
save(paths,name,olist)

Here, specify position = "celestial body name", survey = ["DSS", ..], etc. Get an image from skyview and save it in a fits file with a function called save.

How to save to fits file

python

def save(p,name,obs): # save file into fits 
    for onep,oneo in zip(p,obs):
        onep.writeto(name+"_"+oneo+".fits",overwrite=True)

Name the category list written in obs and save it in the fits file.

Plot example in celestial coordinates using matplotlib

Import the modules needed to plot the image.

python

import matplotlib.pyplot as plt
from matplotlib.colors import LogNorm
from astropy.wcs import WCS
from astropy.io import fits
import glob 
import numpy as np

Next, read the fits file with astropy and extract the WCS information from the header. WCS stands for World Coordinate System (WCS) and is the information required for coordinate conversion of celestial images. Specifically, it is the information required for conversion from detector coordinates to celestial coordinates and galactic coordinates. .. Coordinate conversion can be done simply by specifying it in the keyword projection of matplotlib.

python

# set IC342 imaga of DSS
dssname = "IC0342_DSS.fits"
dsshdu = fits.open(dssname)[0]
dsswcs = WCS(dsshdu.header)
dssdata = dsshdu.data

Plot with matplotlib.

python

# plot image 
F = plt.figure(figsize=(8,8))
plt.subplot(111, projection=dsswcs)
plt.imshow(dssdata, origin='lower', norm=LogNorm())
plt.colorbar() 

If you see a picture like this, you are successful.

Image analysis example

Let's play with the image by performing basic operations such as rotating, transposing, and diffing the image. For example, you can easily display a difference image.

The fits file is just a file that combines basic information written in text and data written in binary, so let's handle it without being weak at first. As for image data, there are many modules in python such as openCV, so let's play around with them first. Images and galaxies look different if they have different wavelengths, and you can play with other images such as supernova remnants and clusters of galaxies. You can search for images using vizier, but you may not find what you want, so it may be faster to search on google. ..

Example of image analysis using morphology transformation

In specialized astronomical analysis, the response functions of the telescope (image spread, sensitivity, etc.) are used to investigate whether the "point" in the image is a point source or a spread structure. For example, the Chandra satellite has a tool called wavdetect that performs pattern matching for point source analysis.

Here, I will introduce a method to play astronomical images a little easier. Morphology conversion is a method of binarizing an image (0 or 1) and repeating and / or operations with an appropriate small area to transform the shape of the image. For example, if the area of the galaxy is connected by 100 pixels and the point source is only 1 pixel, the erosion process (= the operation of setting 0 if even 1 pixel is 0 in an appropriate small area) is performed. If you do, the noise will disappear. Next, if you perform the expansion process (== the operation of setting 1 if there is even 1 pixel in an appropriate small area), it will return to the original state, but the noise that once became 0 will not appear. The method of processing noise by repeating erosion and dilation in this way is called opening processing. It can be executed using cv2.morphologyEx (binarydssdata, cv2.MORPH_OPEN, kernel). Morphological conversion was helpful. The original is written in C language, and there is [Execution example of Morphology_2_8cpp-example](# https://docs.opencv.org/4.1.2/da/d34/samples_2cpp_2tutorial_code_2ImgProc_2Morphology_2_8cpp-example.html#a13).

Easy to use, images

python

# convert the data into binary image
binarydssdata = 1.0 * (dssdata > np.median(dssdata)) 

Like, it is binarized by the size of median. It seems that some functions do not work with int, so set float to 1.0.

Next, getStructuringElement (cv2.MORPH_ELLIPSE, (8,8)) creates an elliptical 1,0 matrix with an 8x8 image and converts it as cv2.morphologyEx as a kernel.

# 1.0 is used because morphologyEx may not work for int32
import cv2
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(8,8))
opening = cv2.morphologyEx(binarydssdata,cv2.MORPH_OPEN, kernel)

This is the only code. Here is an example of execution.

From left to right, "original image", "binarized image", and "after morphology conversion", and on the far right is "original image" x "after morphology conversion". In actual astronomical analysis, point source removal operates in the same way.

Execution example in google Colab

To run on google Colab, astroquery is not installed by default, so you need to install astroquery. Please refer to Example of execution on google Colab.

Also, if you do not save the fits file on google drive, you can not get the data out, so mount google drive if necessary and write the fits file there.

Related page

-Searching the space astronomical catalog using python's astroquery and simple plotting method using galaxies -How to merge images with WCS coordinates and plot with python -How to plot multiple fits images side by side using python