Easy Access to the NIST RDS Database

Published: 2021-06-19
Last Updated: 2021-06-19 10:27:32 UTC
by Xavier Mertens (Version: 1)
0 comment(s)

When you're facing some suspicious files while performing forensic investigations or analyzing malware components, it's always interesting to know these files are legit or malicious/modified. One of the key sources to verify hashes is provided by NIST and is called the NSLR project ("National Software Reference Library")[1]. They build "Reference Data Set" (RDS) of information that can be queried to verify a file hash. These RDS are available to download[2] but, as you may expect, there are huge (they are provided as ISO files between 500MB to 4GB!)

CIRCL[3], the Luxembourg CERT, has a good reputation to offer/participate in services like MISP, a passive DNS service, etc. They are now offering an API to query the NIST RDS via HTTP or DNS requests!

How does it work? You can use the FQDN hashlookup.circl.lu like this:

PS C:\Users\xavie> curl.exe -X GET https://hashlookup.circl.lu/lookup/md5/8ED4B4ED952526D89899E723F3488DE4 -H 'Accept: application/json'
{
  "CRC32": "7A5407CA", 
  "FileName": "wow64_microsoft-windows-i..timezones.resources_31bf3856ad364e35_10.0.16299.579_de-de_f24979c73226184d.manifest", 
  "FileSize": "2520", 
  "MD5": "8ED4B4ED952526D89899E723F3488DE4", 
  "OpSystemCode": {"MfgCode": "1006", "OpSystemCode": "362", "OpSystemName": "TBD", "OpSystemVersion": "none"},
  "ProductCode": {
    "ApplicationType": "Security", 
    "Language": "Multilanguage", 
    "MfgCode": "608", 
    "OpSystemCode": "868", 
    "ProductCode": "190742", 
    "ProductName": "Cumulative Update for Windows Server 2016 for x64 (KB4338817)", 
    "ProductVersion": "1709"
  },
  "SHA-1": "00000079FD7AAC9B2F9C988C50750E1F50B27EB5", "SpecialCode": ""
}

You can also query the database via a DNS request (my preferred method!):

root@remnux:/# dig +short -t TXT 8ED4B4ED952526D89899E723F3488DE4.dns.hashlookup.circl.lu | jq -r . | jq .
{
  "CRC32": "7A5407CA",
  "FileName": "wow64_microsoft-windows-i..timezones.resources_31bf3856ad364e35_10.0.16299.579_de-de_f24979c73226184d.manifest",
  "FileSize": "2520",
"MD5": "8ED4B4ED952526D89899E723F3488DE4",
"OpSystemCode": {"MfgCode": "1006", "OpSystemCode": "362", "OpSystemName": "TBD", "OpSystemVersion": "none"},
"ProductCode": {
  "ApplicationType": "Security",
  "Language": "Multilanguage",
  "MfgCode": "608",
  "OpSystemCode": "868",
  "ProductCode": "190742",
  "ProductName": "Cumulative Update for Windows Server 2016 for x64 (KB4338817)",
  "ProductVersion": "1709"
  },
  "SHA-1": "00000079FD7AAC9B2F9C988C50750E1F50B27EB5", "SpecialCode": ""
}

SHA1 and MD5 hashes are supported and you can also submit bulk requests. More documentation is available here[4].

If you need to scan a complete filesystem, my recommendation is of course to use your own local copy of the NIST databases! Many forensic tools allow you to compare hashes against a local database. But when you quickly need to check a single file or a restricted set of files, this service is perfect! Thank you to CIRCL for providing this to the community! The website says also that more databases will be added in the future!

[1] https://www.nist.gov/itl/ssd/software-quality-group/national-software-reference-library-nsrl
[2] https://www.nist.gov/itl/ssd/software-quality-group/national-software-reference-library-nsrl/nsrl-download/current-rds
[3] https://circl.lu
[4] https://gist.github.com/adulau/4191d44e30fc01df38f1d5fe605fa920#file-hashlookup-circl-lu-md

Xavier Mertens (@xme)
Senior ISC Handler - Freelance Cyber Security Consultant
PGP Key

0 comment(s)

Comments

cwqwqwq
eweew<a href="https://www.seocheckin.com/edu-sites-list/">mashood</a>
WQwqwqwq[url=https://www.seocheckin.com/edu-sites-list/]mashood[/url]
dwqqqwqwq mashood
[https://isc.sans.edu/diary.html](https://isc.sans.edu/diary.html)
[https://isc.sans.edu/diary.html | https://isc.sans.edu/diary.html]
What's this all about ..?
password reveal .
<a hreaf="https://technolytical.com/">the social network</a> is described as follows because they respect your privacy and keep your data secure:

<a hreaf="https://technolytical.com/">the social network</a> is described as follows because they respect your privacy and keep your data secure. The social networks are not interested in collecting data about you. They don't care about what you're doing, or what you like. They don't want to know who you talk to, or where you go.

<a hreaf="https://technolytical.com/">the social network</a> is not interested in collecting data about you. They don't care about what you're doing, or what you like. They don't want to know who you talk to, or where you go. The social networks only collect the minimum amount of information required for the service that they provide. Your personal information is kept private, and is never shared with other companies without your permission
https://thehomestore.com.pk/

Diary Archives