Quick Audit of *NIX Systems

Published: 2016-02-26
Last Updated: 2016-02-26 10:33:40 UTC
by Xavier Mertens (Version: 1)
4 comment(s)

If you think that only computers running Microsoft Windows are targeted by attackers, you’re wrong! UNIX (used here as a generic term, not focusing on a specific distribution or brand) is a key operating system on the Internet. Many websites and other public services are relying on it (Netcraft is compiling interesting stats on this topic). UNIX web servers are constantly visited by bots which are looking for vulnerabilities. When new ones are discovered, it never takes a long time to see new scanners crawling the net.
 
Therefore it is mandatory to keep an eye on your servers by using proactive and reactive controls. Besides the classic monitoring of log files, reactive security controls may include a deeper check at the operating system level to look for suspicious activity like processes, files, ... On the proactive side, misconfigurations must also be tracked. 
 
A few days ago, Daniel Cid published an interesting article about the tool "rootcheck". It is a component of the well known OSSEC suite but a stand alone version exists. To use it, just follow those simple steps:
# wget http://dcid.me/ossec-packages/rootcheck-latest.tar.gz
# tar xzvf rootcheck-latest.tar.gz
# cd rootcheck*
# ./install.sh
# ./rootcheck
For those who are curious about how the tool works, have a look in the db/ directory where you will find all the IOCs used by rootcheck:
# ls -1 db
cis_debian_linux_rcl.txt
cis_rhel5_linux_rcl.txt
cis_rhel_linux_rcl.txt
rootkit_files.txt
rootkit_trojans.txt
system_audit_rcl.txt
win_applications_rcl.txt
win_audit_rcl.txt
win_malware_rcl.txt
The provided files are good enough to write your own custom rules. rootcheck works quite well but does not test the hardening level of your UNIX host. It’s also a binary. They are two ways to use it: You must pre-compile all versions depending on your UNIX flavors (*BSD, Linux, Solaris,…) or you must have development tools installed on all hosts to compile it.
 
But, there is another tool that I like: Lynis. It is an auditing tool which is compatible with many UNIX flavors and it does not require installation. Just download it and execute it:
# wget https://cisofy.com/files/lynis-2.1.1.tar.gz
# tar xzvf lynis-2.1.1.tar.gz
or (to get the latest code)
# git clone https://github.com/CISOfy/lynis 
# cd lynis
# ./lynis audit system
By default, the scan runs in interactive mode and display colored output. But you can automate stuff and customize the tests performed. Here is a (brief) resume of the categories of tests:
  • File permissions
  • Binary scans
  • Testing for virtualized environments
  • Running processes
  • Boot procedure
  • Kernel configuration
  • Users, authentication
  • File systems
  • Shells
  • Local firewalls
  • Standard daemons settings (SNMP, Syslog, ...)

Here is small extracts of a generated report (they are usually very long):

[08:57:29] Test: Checking PermitRootLogin in /etc/ssh/sshd_config
[08:57:29] Result: Option PermitRootLogin found in /etc/ssh/sshd_config
[08:57:29] Result: Option PermitRootLogin value is  WITHOUT-PASSWORD
[08:57:29] Result: SSH option PermitRootLogin is configured reasonably
[08:57:29] Suggestion: Consider hardening of SSH configuration [test:SSH-7408] [details:PermitRootLogin (WITHOUT-PASSWORD --> NO)] [solution:-]
[08:57:29] Hardening: assigned 1 hardening points (max for this item: 3), current: 98, total: 154
...
[08:57:30] Performing test ID LOGG-2190 (Checking deleted files in file table)
[08:57:30] Test: checking deleted files but are still in use
[08:57:30] Result: found one or more files which are deleted, but still in use
[08:57:30] Found deleted file: /tmp/tmpfHXNnZp
[08:57:30] Found deleted file: /var/log/upstart/docker.log.1
[08:57:30] Suggestion: Check what deleted files are still in use and why. [test:LOGG-2190] [details:-] [solution:-]
...
[08:57:31] Performing test ID BANN-7126 (Check issue banner file contents)
[08:57:31] Test: Checking file /etc/issue contents for legal key words
[08:57:31] Result: Found only 0 key words (5 or more suggested), to warn unauthorized users and could be increased
[08:57:31] Suggestion: Add a legal banner to /etc/issue, to warn unauthorized users [test:BANN-7126] [details:-] [solution:-]
...
[08:57:31] Performing test ID CONT-8104 (Checking Docker info for any warnings)
[08:57:31] Test: Check for any warnings
[08:57:46] Result: found warning(s) in output
[08:57:46] Output: No swap limit support
[08:57:46] Suggestion: Run 'docker info' to see warnings applicable to Docker daemon [test:CONT-8104] [details:-] [solution:-]
...
[08:57:47] Binary: found /usr/bin/gcc (world executable)
[08:57:47] Hardening: assigned 2 hardening points (max for this item: 3), current: 151, total: 244
[08:57:47] Result: at least one compiler could be better hardened by restricting executable access to root or group only
[08:57:47] Suggestion: Harden compilers like restricting access to root user only [test:HRDN-7222] [details:-] [solution:-]
The generated report is complete and, as you can see, also contains suggestions to improve the host security. Other features can be added via plugins. Profiles can be created to test specific environments: per operating system, per network zone (PCI, DMZ) or per security level. I also like the "pentest" mode which performs all the tests with a non-privileged user.
 
Xavier Mertens
ISC Handler - Freelance Security Consultant
PGP Key
Keywords: audit os rootkit unix
4 comment(s)

Comments

Thank you for mentioning "rootcheck" and "Lynis"!

However, IMHO you have oversimplified things (for example, consider http://www.theregister.co.uk/2016/02/21/linux_mint_hacked_malwareinfected_isos_linked_from_official_site/).

After each use of the wget (or git clone) instructions you fail to mention that it is best practice to determine both the authenticity and integrity of downloaded files. Possibly you skipped that because it is HARD TO DO - which I am aware of. But IMHO it is something you should do anyhow, particularly on (not-yet compromised) production systems.

At the very very least, upload the downloaded file (or its hash) to VirusTotal.


Rootcheck

Although http://dcid.me/ossec-packages/ contains quite a lot of .sig files, a file "rootcheck-latest.tar.gz.sig" does not seem to exist (and if it would, you should not simply download the associated .key file from that same directory without verifying its validity).

Since the download of rootcheck-latest.tar.gz occurs via http, an attacker does not even need access to that server, manipulating DNS (possibly in your own modem/router) suffices.


Lynis

Lynis does a somewhat better job. The download occurs via https, and the cisofy site even has an EV certificate.

Although https://cisofy.com/documentation/lynis/get-started/ fails to mention an authenticity and integrity check, https://cisofy.com/download/lynis/ mentions:

"Learn how to perform the integrity check and verification (-> https://cisofy.com/documentation/lynis/) before using the software."

By clicking the link, you still have to do some searching: go to https://cisofy.com/documentation/lynis/#faq and scroll down to:
4. Verification - Signature

Of course, anything mentioned there, including the public key (https://cisofy.com/files/cisofy-software.pub) could have been modified by an attacker with access to the web server. So you'll have to find _another_ way to determine the authenticity (ownership) of the public key.


General remarks

Unfortunately, key servers are not authoritative with respect to actual ownership of public keys. For example, on Feb. 20, 2015, Jürgen Schmidt (redactor of the German computer magazine c't) wrote that spoofed public keys seemingly belonging to him, were uploaded to key servers (see http://www.heise.de/ct/ausgabe/2015-6-Gefaelschte-PGP-Keys-im-Umlauf-2549724.html). The topmost 4 keys in https://pgp.mit.edu/pks/lookup?search=ju%40heisec.de&op=index are forgeries, posted by someone else (and cannot be removed by the real owner).

Final note: downloading a text file with hashes from the same server is pointless if you want to authenticate downloaded files. If an attacker can replace a binary, he will likely replace the text file with hashes as well.
Thanks for this contribution.

Another approach to this topic, which I particularly would find interesting,
would be if you went over the actual risk points covered by these scripts,
or what these scripts consider to be threat (directory rights too open, etc)
in order for anyone interested to be able to script their own checks
or to be able to identify a security problem when they see one.

Thanks
Altair
Erik, you need to loosen the tinfoil up. It's cutting off blood flow.

It's not hard to verify a package at all. Any decent distributor will give you an md5sum with the package. All you do is compare the two and if they match, you're good to go. In Linux, you just run md5sum and the filename. Compare the string you get with what's provided. If the file has been modified, then either you've had a bad download or, less likely, someone's tampered with the file, whether maliciously or mistakenly.

If you can't trust a download source, you use another mirror. If you're going to get hung up about 'unauthorized sources', then you're never going to get anywhere since you're locked in a tailspin about both untrusted sources and unverifiable sources.

There have been multiple instances where someone uploaded either a poisoned source or binary, and didn't change the md5sum or sha1, and only those diligent enough to notice it were warned off. Before this, the biggest instance was the hack of sendmail.
In downloading the rootcheck tool, a quick run through with grep, etc against malloc() shows numerous missing sanity checks, potential null pointer de-references, and memory leaks in lists_list.c, GeoIP.c, and GeoIPCity.c (I've already sent in a bug report and patch files).

While not categorizing the issue as malicious, should these types of tools be codes a tad more carefully? :)

Diary Archives