Threat Level: green Handler on Duty: Jan Kopriva

SANS ISC: SANS Internet Storm Center SANS Internet Storm Center


Sign Up for Free!   Forgot Password?
Log In or Sign Up for Free!

Latest Diaries

TriOp - tool for gathering (not just) security-related data from Shodan.io (tool drop)

Published: 2021-01-27
Last Updated: 2021-01-27 11:11:40 UTC
by Jan Kopriva (Version: 1)
0 comment(s)

If you’re a regular reader of our Diaries, you may remember that over the last year and a half, a not insignificant portion of my posts has been devoted to discussing some of the trends in internet-connected systems. We looked at changes in the number of internet-facing machines affected by BlueKeep[1], SMBGhost[2], Shitrix[3] and several other vulnerabilities [4] as well as at the changes in TLS 1.3 support over time[5] and several other areas [6,7].  Today, we’re going to take a look at the tool, that I’ve used to gather data, on which the Diaries were based, from Shodan.io.

In keeping with the Shodan theme, I’ve called the tool TriOp. It is a Python script, which enables you to quickly build a list (or multiple lists) of Shodan search terms and then repeatedly (e.g. once every day) get the count of public IPs, which satisfy each of them, from the Shodan API.

The basic use of the tool is quite straightforward – in addition to creating a Shodan account and getting its API key, one only needs to create a list of Shodan queries one wishes to monitor over time, input this list into TriOp (as a CSV file where the first row specifies the queries or through a command line as a comma-separated list) and the tool will do the rest. It can output the results of the queries to command line, but its more useful feature is the option to save the results in a CSV, which can later be used as an input for TriOp.

 

The basic search can be done using the -s/--search option in the following way:

triop.py -s "port:80,port:443"

In such a case, the output might look like this:

TriOp 1.0
Current IP count for query port:80 is 72010982
Current IP count for query port:443 is 59072465

Where things get interesting is the output to a file using the -O/--output_file option:

triop.py -s "port:80,port:443" -O http_ports.csv

The resultant CSV file would have the following structure:

Date,2021-01-26
port:80, 72036704
port:443, 59145503

You can probably see why using the same file as input on another day might be useful. If one were to use this file as an input today using the -S/--search_file option and add the -a/--append option, TriOp would add another row to the file with today’s date.

triop.py -S http_ports.csv -a

The updated CSV file would then look like this:

Date,2021-01-26,2021-01-27
port:80,72036704,72010982
port:443,59145503,59072465

If one wanted to monitor the situation on a day to day basis, one would only need to run the same command each day (preferably using some automatic scheduling mechanism).

Although gathering data about the number of public IPs with different ports open to the internet may be interesting, as it gives us some idea about how the global network changes over time[8], we are certainly not limited to just the “ports:” filter.

Since TriOp only gets a "count" for each of the queries and not the related list of IP addresses, which satisfy the queries, one may use any combination of Shodan search filters with it (even those, which are normally accessible only to enterprise or researches-level accounts), even with a free account.

This means that one may use TriOp to monitor the changes in different open ports in specific IP ranges (filter “net:”), ASNs (filter “asn:”) or countries (filter “country:”), but also to monitor changes in the number of IPs affected by specific vulnerabilities (filter “vuln:”), systems with specific JARM[9] fingerprints (filter “ssl.jarm:”), etc.

Since we’ve mentioned vulnerabilities, if this is area that interests you, you may also use TriOp as a high-level “passive” vulnerability scanner. Shodan itself detects machines affected by some vulnerabilities – currently it seems to be able to identify about 2246 of the approximately 190k CVEs published so far, according to the results of my tests[10] – and nothing is stopping us from getting the “count” for these. List of the CVEs “supported” by Shodan, which I’ve been able to identify, are included in TriOp and one may search for them simply using any query in combination with the --vuln_search_level option:

triop.py -s "country:US" --vuln_search_level 3

The previous command would result in a very long output giving us the number of public IPs in the US, on which systems vulnerable to specific CVEs might be found:

TriOp 1.0
Checking whether Shodan detects any vulnerabilities for search country:US.
Current IP count for query country:US has_vuln:true is 10398899
Current IP count for query country:US is 160792718
Current IP count for query country:US vuln:CVE-1999-0045 is 1
...

The tool has additional features as well (adding new queries to existing search files, exporting data related to similar searches from multiple search files, etc.) and you may find some of them demonstrated in the tutorial video bellow.

The one last feature I will mention here is the ability to load multiple input files based on a specified “mask”. I’ve originally created TriOp with the intention to monitor changes in the number of vulnerable systems, ports and services both globally as well as in different countries and I’ve created over a hundred different search files by now (one per each country I was interested in as well, several for different sets of vulnerabilities, etc.). In order for me to be able to “update” each of them on a daily basis, TriOp supports the --filename_load option, which enables one to specify a string, which is then used to select files which should be used as inputs. If one were to use the following mask for example, all CSV files in the current folder would be used as inputs and consequently updated.

triop.py --filename_load .csv -a

As you may see, although TriOp is a fairly simple tool, which only gathers “counts” for each of the submitted queries, its outputs can be quite useful. This goes especially for any (national) CSIRT, which wants to monitor public IPs of its constituency, but lacks a capability to scan it on a daily basis, or for any security researcher who wants to, for example, compare the number of devices affected by specific vulnerabilities in different countries.

In any case, if you’d like to try TriOp yourself, you may download it from my GitHub page.

[1] https://isc.sans.edu/diary/25506
[2] https://isc.sans.edu/diary/26732
[3] https://isc.sans.edu/diary/26900
[4] https://isc.sans.edu/diary/26798
[5] https://isc.sans.edu/diary/26936
[6] https://isc.sans.edu/diary/25854
[7] https://isc.sans.edu/diary/26374
[8] https://untrustednetwork.net/en/2021/01/01/open-ports-statistics-for-2020/
[9] https://engineering.salesforce.com/easily-identify-malicious-servers-on-the-internet-with-jarm-e095edac525a
[10] https://untrustednetwork.net/en/2020/11/18/most-common-vulnerabilities-based-on-shodan/

-----------
Jan Kopriva
@jk0pr
Alef Nula

Keywords: Shodan Tool TriOp
0 comment(s)

If you have more information or corrections regarding our diary, please share.

Recent Diaries

TA551 (Shathak) Word docs push Qakbot (Qbot)
Jan 26th 2021
19 hours ago by Brad (0 comments)

Fun with NMAP NSE Scripts and DOH (DNS over HTTPS)
Jan 25th 2021
1 day ago by Rob VandenBrink (0 comments)

Video: Doc & RTF Malicious Document
Jan 24th 2021
2 days ago by DidierStevens (0 comments)

CyberChef: Analyzing OOXML Files for URLs
Jan 23rd 2021
4 days ago by DidierStevens (0 comments)

Another File Extension to Block in your MTA: .jnlp
Jan 22nd 2021
5 days ago by Xme (0 comments)

Powershell Dropping a REvil Ransomware
Jan 21st 2021
6 days ago by Xme (0 comments)

Qakbot activity resumes after holiday break
Jan 20th 2021
1 week ago by Brad (0 comments)

View All Diaries →

Latest Discussions

PFSense
created Dec 23rd 2020
1 month ago by bas.auer@auerplace.nl (3 replies)

Port 23 & 2323 107.173.58.179
created Nov 15th 2020
2 months ago by Anonymous (0 replies)

Gmail hacked vis MS Outlook / request.zip virus/malware
created Oct 13th 2020
3 months ago by Anonymous (3 replies)

Why is the entire community so... I don't know the words...
created Sep 8th 2020
4 months ago by Everseeker (0 replies)

I can not find the Bluetooth channel!
created Aug 31st 2020
4 months ago by Martin (0 replies)

View All Forums →

Latest News

Top Diaries

An infection from Rig exploit kit
Jun 17th 2019
1 year ago by Brad (0 comments)

Old Worm But New Obfuscation Technique
Nov 13th 2020
2 months ago by Xme (0 comments)

Is IP 91.199.118.137 testing Access to aahwwx.52host.xyz?
Dec 5th 2020
1 month ago by Guy (0 comments)

AV Cleaned Maldoc
Nov 2nd 2020
2 months ago by DidierStevens (0 comments)

Traffic Analysis Quiz: Ugly-Wolf.net
Oct 16th 2020
3 months ago by Brad (0 comments)