[PYTHON] 2019 version: Unauthorized access trend analysis (example of general-purpose server on the cloud)

Introduction

This article was posted as Day 23 of Cisco Systems Japan Advent Calendar 2018 </ B> </ font> by our fellow Ciscos.

2017 Edition </ B>: https://qiita.com/advent-calendar/2017/cisco 2018 Edition </ B>: https://qiita.com/advent-calendar/2018/cisco 2019 Edition </ B>: https://qiita.com/advent-calendar/2019/cisco

I think there are many network / infrastructure / server engineers who are building home labs. I had an actual machine at home in that way for several years, but about a year and a half ago I stopped having equipment at home and switched to multiple cloud services. It's been almost a year since I tried some software and management tools to fortify the server, and finally let go of it while repeatedly changing the settings, so it's been illegal for the last year I would like to summarize the access trends.

The theme covered in this entry is " 2019 Edition: Unauthorized Access Trend Analysis (Example of a General-Purpose Server on the Cloud) </ B>". In this,

- Annual Unauthorized Access Statistics </ font> </ B> - Environment assumptions and fail2ban tuning example </ font> </ B> - Unauthorized access connection source malicious score investigation tips (Umbrella Investigate API) </ font> </ B>

I will summarize the three main items of. As time passed from the posting of Advent Calender in 2018, it was a theme that I wanted to summarize as a story someday.

Annual Unauthorized Access Statistics </ font>

--Period

  • 2019/01/01 ~ 2019/11/31 --Item - Monthly ssh unauthorized access attempts (banned by ssh-sasl) </ B> - Monthly ssh root login attempts (root login fail) </ B> - Monthly postfix unauthorized access attempts (banned by postfix-sasl) </ B> - Monthly Ban Table entry average (ssh, postfix) </ B> - Number of ssh unauthorized access attempts by time (banned by ssh-sasl) </ B> - Unauthorized access source country code Top26 </ B> - Monthly unauthorized access source IP address Domain reverse lookup possible </ B> - ssh account Top20 </ B> - dovecot account Top14 </ B> - Unauthorized Access Source RiskScore (Overall) </ B> - Unauthorized access source RiskScore (Japan) </ B> - postfix RBL access violation statistics </ B> --Statistical source data & Tool -Action mail sent by fail2ban </ B> (mail sent by action at ban) -Statistical reports sent by logwatch </ B> (/ vat / log / maillog, / var / log / secure, / var / log / messages, / var / log / dovecot / dovecot.log) - Cisco Umbrella Investigate </ B> (for fraudulent access source risk score investigation)

Monthly ssh unauthorized access attempts (banned by ssh-sasl), monthly ssh root login attempts (root login fail)

1-1.png 1-2.png Monthly ssh unauthorized access attempts and monthly ssh root account attempts during the 12 months of the period.

Due to the fail2ban / iptables setting, the source IP address that failed 3 times within 1800 seconds cannot be reconnected for at least 36000 seconds (10 days). The number of unauthorized access attempts in this graph is a record of the number of first and second accesses in which a brute force attack is executed almost automatically. Normally, in a system that does not have such access control, it is considered that a huge amount of access with different digits is performed.

Before December 2018, we frequently turned off the power to improve the environment, and full-scale continuous operation started around November 2018. Since then, the number of ssh access attempts has increased, but since May 2019, the number of accesses has decreased at a stretch.

Monthly postfix Unauthorized access attempts (banned by postfix-sasl), Monthly Ban Table entry average (ssh, postfix)

2-1.png 2-2.png Monthly Postfix authentication unauthorized access attempts over a 12-month period and monthly Active Ban Table average.

In this target system, SMTP-AUTH is enabled as a part of 3rd Party relay countermeasures when sending mail by Postfix. The maillog counts the number of times this SMTP-AUTH attempt failed. As with ssh, the connection source IP address that failed twice within 5400 seconds is entered in iptables, so authentication failure at the time of initial connection is targeted. Although the total number is considerably smaller than that of ssh, it can be confirmed that the total number is gradually increasing from the time when access to ssh is decreasing.

Ban Table entries are the result of a fixed point observation of the total number of Active Banned Table entries for ssh and postfix reported by logwatch (3:00 am). You can see that it is proportional to the number of ssh accesses.

Number of ssh unauthorized access attempts by time (banned by ssh-sasl)

3.png This is the rate of occurrence of ssh unauthorized access events by time in Japan during the 12 months of the period. Unauthorized access to email services is not counted.

There is no bias in the number of accesses by time in Japan. It is considered to be executed almost mechanically from access sources all over the world.

Unauthorized access source country code Top26

4.png It is the top 26 countries (until Japan ranks in) of the connection sources of ssh unauthorized access events during the 12 months of the period. Unauthorized access to email services is not counted.

Overwhelmingly China </ font> </ B>, America </ font> </ B>, followed by You can see that there are many accesses from B> France </ font> </ B>.

Monthly unauthorized access source IP address Domain reverse lookup possible

5.png Monthly number of ssh unauthorized access attempts in 12 months within the period ( blue vertical line </ font> is the same as the previous' monthly ssh unauthorized access attempt count </ B>' graph) Is a comparison with the IP address that was able to reverse-lookup the host domain from the connection source IP address.

In all 65507 ssh unauthorized access attempts, 43% of the connection source IP addresses have a registered FQDN. There is no big difference in the monthly ratio. (39% ~ 43%)

ssh account Top20, dovecot account Top14

6.png This is a table with the highest usage rates of non-root accounts used to enforce unauthorized access to ssh and mail services during the 12 months of the period.

There are many accounts that can be imagined that ssh is created easily, and there is a bias in the number of times. In dovecot and postfix, many accesses can be confirmed with individual accounts that are less biased and are not shown in the graph and are used less frequently.

Unauthorized Access Source RiskScore (Overall), Unauthorized Access Source RiskScore (Japan)

7-1.png 7-2.png Distribution of Predictive Severity Assessment (RiskScore) </ B> by Cisco Umbrella Investigate </ B> from ssh unauthorized access sources over a 12-month period.

I tried to extract it from Umbrella Investigate as the reputation information of the connection source that performs fraudulent activity. The score is 0-100, 100 is the highest grade score. You can see that most are sources with relatively benign scores.

postfix RBL access violation statistics

8.png Postfix connection violation statistics by RBL rules for 12 months within the period.

- RBL: 1 </ B>: Reputation check by zen.spamhaus.org - RBL: 2 </ B>: Reputation check by bl.spamcop.net

  • No Service : Name or service not known
  • No Address : Address not listed for hostname

I have set zen.spamhaus.org, bl.spamcop.net as the Junk E-mail Source Reputation Blacklist RBL (Realtime Blackhole List), which checks this list when connecting to the MTA and rejects the sources that match the list. To do. The number of hits on zen.spamhaus.org spiked on June 2019 and October 2019, respectively.

Environment assumptions and fail2ban tuning example </ font>

This is a prerequisite for annual unauthorized access statistics. The count itself displayed in the graph of unauthorized access statistics is not the total number of access attempts, but the analysis result after being sufficiently filtered by the tool used. Let's sort out some of the conditions.

In addition to ssh, we are building a remote desktop environment using XRDP + GNOME and a mail service using postfix + dovecot on the system targeted for this analysis. We do not use other services such as Web servers. Open ports are 22 / tcp (ssh) </ B>, 25 / tcp (SMTP) </ B>, 993 / tcp (imaps) </ B>, 995 / tcp (pop3s) </ B>, 3389 / tcp (RDP) </ B>.

Simply protect Remote Desktop Services with static iptables that locks the source, and ssh and mail services use fail2ban </ B> to register and protect dynamic iptables entries according to specified rules. The result is sent to an email each time a banned table is entered by fail2ban, security events are analyzed every 24 intervals, and the result is sent by logwatch </ B>. Each service does not use advanced tools such as IPS / IDS to deal with vulnerabilities on the premise that it will be updated regularly. This time, the main purpose is to output the tendency of unauthorized access based on the information blocked by this traditional fail2ban </ B>.

fail2ban fail2ban is not a new tool at all. The log file specified by the setting that reports unauthorized access to the service such as / var / log / apache / error_log is scanned in real time, and the connection source IP is based on the malicious sign in the log file. Ban (automatic addition / update to iptables) of the address at the specified time protects the system from login password failure and Exploit usage. While fail2ban works well to reduce the rate of repeated login attempts, such as brutefoce attacks, it does not reduce access to weak authentication itself.

jail.local In the case of ssh, prevent consecutive'Failed passwords' from the same IP address from being output to / var / log / secure, and in the case of postfix, consecutive'LOGIN authentication failed' from the same IP address as much as possible. Continue adjusting to prevent output from / var / log / mail / maillog for a while, and finally findtime = 1800 / maxentry = 3, findtime = 5400 / maxentry = 2, respectively. I came to the conclusion that is the best. In other words, in the case of postfix, if there is a login failure from the same connection source three times within 90 minutes, it is set to Ban. In many cases, they tried to re-access from the same source within 17-18 minutes.

jail.local


[DEFAULT]
ignoreip = x.x.x.0/24
backend = polling
bantime = 360000
maxretry= 3
usedns = yes

[ssh-iptables]
enabled = true
filter = sshd
action = iptables[name=SSH, port=ssh, protocol=tcp]
         sendmail-whois[name=ssh-sasl, dest=root, [email protected]]
logpath = /var/log/secure
findtime = 1800

[sasl-iptables]
enabled = true
filter = postfix-sasl
action = iptables-multiport[name=postfix-sasl, port="smtp,smtps", protocol=tcp]
         sendmail-whois[name=postfix-sasl, dest=root, [email protected]]
logpath = /var/log/mail/maillog
maxretry= 2
findtime = 5400

Unauthorized access connection source malicious score investigation tips (Umbrella Investigate API) </ font>

Cisco Umbrella Investigate Umbrella </ B> uses Public Recursive DNS to score potential malicious domains and IP addresses that are automatically analyzed from more than 180 billion requests per day as predictive indicators. Security statistical analysis for these DNS and IP addresses is a function of Umbrella Investigate </ B>, and score information can be obtained from many perspectives. Scores generated for this particular domain, IP address can be used by advanced security operators and security researchers primarily to assist predictive analysis and find network activity information that is considered suspicious. Is possible.

Umbrella Investigate Risk Score The Umbrella Investigate Risk Score is a predictive potential score in the range 0-100 based on domain name lexical characteristics (construction strings), domain queries to the Umbrella infrastructure, and request pattern analysis. This value is not necessarily directly linked to the malignancy judgment due to past fraudulent activities under investigation, but is positioned as a predicted risk value. This information is also linked to past Botnet, phishing domains, malvertising, and ransomware relevance to the subject of the survey.

pyinvestigate (Umbrella Investigate API python module)

Umbrella Investigate was used as information on the risk score distribution of unauthorized access information statistics. Umbrella Investigate can be queried by REST API as well as Web UI. We used pyinvestigate </ B>, which is a python module for Umbrella Investigate, in order to collectively acquire RiskScore from REST API from more than 65,000 acquired unauthorized access source IP addresses and host FQDNs.

1. Install pyinvestigate

# pip3 install investigate
# pip3 install pytest
# pip3 install pytest-django
# pip3 install pytest-pythonpath

2. Investigate API key acquisition

--Login to https://dashboard.umbrella.com/ --Investigate> Navigate to Investigate API Access

  • CREATE NEW TOKEN Token.png Token2.png Refer to the created Token and copy

3. Run pyinvestigate

pyinvestigate can be run from the python interpreter. Specify the copied API Key and query the survey IP address / domain. A separate script is created for unauthorized access investigation and collectively acquired.

# python36

Python 3.6.8 (default, Aug 10 2019, 06:52:10)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-23)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> import investigate

>>> import datetime
>>> api_key = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
>>> inv = investigate.Investigate(api_key)
>>>
>>> inv.risk_score('xxx.com')
{'features': [{'feature': 'Geo Popularity Score', 'normalizedScore': 50, 'score': 0}, {'feature': 'Keyword Score', 'normalizedScore': 26, 'score': 0.26108091120959626}, {'feature': 'Lexical', 'normalizedScore': 89, 'score': 0.898}, {'feature': 'Popularity 1 Day', 'normalizedScore': None, 'score': None}, {'feature': 'Popularity 30 Day', 'normalizedScore': None, 'score': None}, {'feature': 'Popularity 7 Day', 'normalizedScore': None, 'score': None}, {'feature': 'Popularity 90 Day', 'normalizedScore': None, 'score': None}, {'feature': 'TLD Rank Score', 'normalizedScore': 0, 'score': 0.007835250367488503}, {'feature': 'Umbrella Block Status', 'normalizedScore': 0, 'score': False}], 'riskScore': 71}
>>>

{'features': [{'feature': 'Geo Popularity Score', 'normalizedScore': 3, 'score': -3.1797661699999997}, {'feature': 'Keyword Score', 'normalizedScore': 0, 'score': 0.005737760395203566}, {'feature': 'Lexical', 'normalizedScore': 50, 'score': 0.504}, {'feature': 'Popularity 1 Day', 'normalizedScore': 95, 'score': 95.59}, {'feature': 'Popularity 30 Day', 'normalizedScore': 96, 'score': 96.94}, {'feature': 'Popularity 7 Day', 'normalizedScore': 96, 'score': 96.26}, {'feature': 'Popularity 90 Day', 'normalizedScore': 96, 'score': 96.41}, {'feature': 'TLD Rank Score', 'normalizedScore': 0, 'score': 0.00562125007281412}, {'feature': 'Umbrella Block Status', 'normalizedScore': 0, 'score': False}], 'riskScore': 1}
{'features': [{'feature': 'Geo Popularity Score', 'normalizedScore': 84, 'score': 1.68891283}, {'feature': 'Keyword Score', 'normalizedScore': 50, 'score': 0.5030948281811238}, {'feature': 'Lexical', 'normalizedScore': 75, 'score': 0.751}, {'feature': 'Popularity 1 Day', 'normalizedScore': 11, 'score': 11.45}, {'feature': 'Popularity 30 Day', 'normalizedScore': 8, 'score': 8.72}, {'feature': 'Popularity 7 Day', 'normalizedScore': 9, 'score': 9.59}, {'feature': 'Popularity 90 Day', 'normalizedScore': 10, 'score': 10.8}, {'feature': 'TLD Rank Score', 'normalizedScore': 28, 'score': 0.28834641075042694}, {'feature': 'Umbrella Block Status', 'normalizedScore': 0, 'score': False}], 'riskScore': 29}
{'features': [{'feature': 'Geo Popularity Score', 'normalizedScore': 86, 'score': 1.8171738299999998}, {'feature': 'Keyword Score', 'normalizedScore': 50, 'score': 0.5030948281811238}, {'feature': 'Lexical', 'normalizedScore': 99, 'score': 0.991}, {'feature': 'Popularity 1 Day', 'normalizedScore': None, 'score': None}, {'feature': 'Popularity 30 Day', 'normalizedScore': None, 'score': None}, {'feature': 'Popularity 7 Day', 'normalizedScore': None, 'score': None}, {'feature': 'Popularity 90 Day', 'normalizedScore': None, 'score': None}, {'feature': 'TLD Rank Score', 'normalizedScore': 28, 'score': 0.28834641075042694}, {'feature': 'Umbrella Block Status', 'normalizedScore': 0, 'score': False}], 'riskScore': 94}

reference

[1] fail2ban https://www.fail2ban.org/wiki/index.php/Main_Page [2] Umbrella Investigate REST API - Security Information for a Domain https://docs.umbrella.com/investigate-api/docs/security-information-for-a-domain-1#section-risk-score-for-a-domain [3] pyinvestigate https://github.com/opendns/pyinvestigate

Recommended Posts