In this article, I will talk about ** "Shodan" **, how it works, and how to incorporate it into tools and scripts.

This blog is a translation from the English version. You can check the original from here. We use some machine translation. We would appreciate it if you could point out any translation errors. *

What Shodan can do

Shodan detects devices that are connected to the Internet at any given time, the location of those devices, and the current user. Such devices can be present in almost any type of system, including business networks, surveillance cameras, industrial control systems (ICS), and smart homes. Shodan is the first important step in penetration testing because trying to get the system banner directly helps identify vulnerable systems. He can also do more by supporting the Boolean operator and provides filters to improve search efficiency. Search engines offer 50 results for free and offer paid subscriptions for a wider range of results.

How it works

There are three ways to use Shodan.

--Web interface

CLI --With tools and scripts This time, we will look at how to use Shodan using all three methods.

For this tutorial, we will use Alibaba Cloud Elastic Compute Service (ECS). In order to secure your instance, you need to configure it accordingly.

--Create rules to allow traffic in and out of security groups --Create a pen test approval We won't go into server configuration here, but see the ECS documentation (https://www.alibabacloud.com/help/ja/product/25365.htm) for more information.

After doing the above two setups, you can install Apache on your ECS and access it with a web browser to test it.

Shodan web interface

Shodan, like most other search engines, can be accessed by accessing shodan.io in a web browser. I can.

I would like to know how many servers are using Apache around the world. Just enter the keyword "apache".

As you can see in the image, the results show that 25,544,783 servers are running on the Internet.

Shodan has many other features, but you need to register to use them. An error occurred when trying to do a deep analysis without an account, as shown below.

Now let's say you have successfully created an account and logged in. One of Shodan's features is the filter, but let's see what you can do with it.

Filters are special keywords that Shodan uses to filter search results based on service or device metadata. The format for entering the filter is

filtername:value

For example

product : "apache tomcat" --country: Filter specific countries --city: Filter specific cities --geo: Filtering by coordinates --hostname: Find a matching hostname --net: Limited to IP --os: Operating system based filter --port: Filter base for open ports

For example, suppose you want to do an accurate search by searching all Apache servers with Tomcat.

The keyword is "product:" apache tomcat "". The results are in the screenshot below.

As a result, we found that there are 1,445,150 servers running Apache Tomcat in the world.

You can further refine your research by looking for all Apache Tomcat servers in China.

Product:"apache tomcat" country:cn

What I did was ask Shodan to search for Apache servers in Tomcat, China.

The results show that there are 409,609 Apache servers with Tomcat in China.

Now suppose you are running a service on one of the ECSs and want to see what is happening. In my case, as mentioned above, I use Alibaba ECS as an example.

To do the task, we need to use a net filter and the value will be our IP_address.

Net: IP address

Suppose you have Apache installed, considering that your security group has not yet created a rule to allow traffic in and out. When I try to access it by entering my IP address in the navigator, I can't see anything because Alibaba Cloud restricts traffic by default. This is to ensure the security of the ECS server even if you have not created any rules for the security group.

As you can see in this image, the server is running on Alibaba and the IPS is Aliyun Computing. Since we are using two ports (80,443), we have two services running on the server, and the web technology used is jQuery. This will be part of the information that hackers collect about your server to see if it is vulnerable, such as an SSL version. ..

Now let's see how to use Shodan using the command line interface (CLI).

Shodan command line interface

Shodan's command line interface (CLI) is packaged with the official Python library for Shodan. To install a new tool, just run it.

$ easy_install shodan

After installing the tool, you need to initialize it with your API key.

$ shodan init YOUR_API_KEY

To https://account.shodan.io Go to it and get the API key for your account. It will be displayed in the upper right corner of the My Account button. Once the API_KEY is initialized, you can now start using Shodan commands.

You can check the account information.

Now, imagine you are on an instance of ECS and want to know what your public IP address is. With ifconfig, Alibaba Cloud protects the service by default, so you only know the private IP address. If you want to know your public IP address, just enter the command Shodan my ip.

$ shodan myip

The result shows my public IP 47.89.249.0.

If you want to have information about a host, we can now use Shodan, such as where such a host is, which port is open, which organization owns the IP, and so on. Let's say 42.120.226.13 is our server and we want to test it to see what's running.

$ shodan host 42.120.226.13

Our server is in China, last updated 2019-01-21, 2 ports are working (80, 443), the result gave us multiple versions, so check the SSL version need to do it.

The CLI has other features such as network analysis, maltego add-ons, and browser plugins, but you will need to upgrade your plan account to use them.

Script Shodan

Shodan provides a developer API for programmatically accessing the collected information. All websites and tools, including Shodan's main site, use this API. Everything you can do via a website can be achieved from your own code.

The API is divided into two parts. "REST API" and "Streaming API". The REST API provides a variety of utility methods for searching for Shodan, looking up hosts, getting summary information for queries, and facilitating development. The Streaming API provides a raw, real-time feed of the data that Shodan is currently collecting. There are several feeds you can subscribe to, but you can't search for data or interact with it in any other way.

Usage restrictions

There are three API methods that are restricted by the API plan.

1, ** Searching **: Shodan uses query credits to limit the number of searches that can be performed in a month. One query credit is used when you perform a search that includes a filter or when you pass the first page. For example, if you search for "apache", no query credits will be used. Also, if you search for "apache country: US", 1 query credit will be used. Similarly, if you search for search results on the second page with "apache", use 1 query credit. Finally, the search query on the second page of "apache country: US" also uses 1 query credit.

2, ** Scanning **: The On-Demand Scanning API uses scan credits to limit the number of hosts that can request Shodan to scan each month. Deduct 1 scan credit for all hosts you request for Shodan scans.

3, ** Network Alerts **: The number of IPs that can be monitored using alerts is limited based on your API subscription. Only paid customers can access this feature. Also, you cannot create more than 100 alerts for your account.

** Note **: Query and scan credits will be reset at the beginning of each month.

To install the Shodan library for Python, run the following command.

$ easy_install shodan

If you already have it installed and want to upgrade to the latest version.

$ easy_install -U shodan

The first thing you must do is initialize the Shodan API object.

import shodan
api = shodan.Shodan('YOUR API KEY')

Here, your API key is the API key for your account and can be obtained from: https://account.shodan.io

Now that you have all the APIs, you can search with a small script.

As you can see in this image, we are writing a small script that is trying to scan the internet for servers and queries. The code is written in python3.

When you run the script with a parameter called Apache, Shodan scans the internet and provides all the Apache server information.

Let's run the script and see if it works and what the result will be.

The result of the script looks like the one above. As you can see, you can see that the Apache server is running on the IP address.

Conclusion

As technology evolves, we need to be up-to-date to ensure that our security is not compromised. Especially in IoT applications where multiple devices are exposed to the Web, security is important not only to ensure the proper functioning of assets, but also to protect privacy. One of the most powerful tools available for that is Shodan. But like all good tools, Shodan can also be a double-edged sword. It can easily be exploited by hackers, but it also helps you better understand your network. It is not the tool itself that is dangerous, but rather the person who uses it.

Alibaba Cloud is the No. 1 (2019 Gartner) cloud infrastructure operator in the Asia Pacific region with two data centers in Japan and more than 60 availability zones in the world. Click here for more information on Alibaba Cloud. Alibaba Cloud Japan Official Page *

[PYTHON] Use and integration of "Shodan"