Finding hidden gems (easter eggs) in your logs (packet challenge!)
Last Updated: 2008-03-23 20:23:58 UTC
by Johannes Ullrich (Version: 1)
Anyway. He used some nice shell scripting to summarize his logs. I like this technique and frequently use it myself. You should all become familiar with the little gems hidden in your respective shells. Even Windows got something to offer in that department (search for past "Windows Shell Kung-Fu" posts by Ed for example).
In my opinion, the goal is to find "unusual" log entries. I am typically less interested in the thousands of SQL slammer infected hosts hitting me each day. And yes, they still exist :-(. What I am interested in are the "one off" hits.
Here a quick shell command I like to summarize my iptables firewall logs:
sed 's/.*SRC=//' < /var/log/firewall | sed 's/ .*DPT//' | sort | uniq -c | sort -n
Its not exactly pretty... but well, it works.
Browsing through the output, this one entry caught my attention:
1 184.108.40.206=27639 LEN=541
Its UDP (otherwise you would see more details about flags and such given the command sequence I used), it got payload, and it is sent to an odd port
Next lets look at the full line:
grep 27639 /var/log/firewall Mar 23 18:17:20 defianta kernel: filter: cable-dmz IN=eth2 OUT=eth0 SRC=220.127.116.11 DST=10.0.0.2 LEN=561 TOS=0x00 PREC=0x20 TTL=43 ID=0 DF PROTO=UDP SPT=53 DPT=27639 LEN=541
hm. from port 53. So maybe just a DNS response? But why did it get blocked? Now it is helpful to have full packet captures. I tend to capture each packet entering and leaving my network. Its not a huge amount of data (few gbytes per day), so I can keep about a month worth.
tcpdump -r snort.log.1206295201 -nn host 18.104.22.168 reading from file snort.log.1206295201, link-type EN10MB (Ethernet) 18:17:20.655144 IP 22.214.171.124.53 > 126.96.36.199.27639: 64286- 0/13/16 (533)
Interesting. The only thing I see is a DNS reply, with no answers, but plenty of authority records. Its also rather large. Unless edns0 is used, DNS replies should stick to 512 Bytes!
off I am into packet land to figure out what this was exactly about. More to come soon... it will involve an AAAA record (hehe... what a cliffhanger)
If you want to take a look, see http://isc.sans.org/odddns for the packet. (no... finding the packet was not staged. That's exactly how I got it.)
Johannes B. Ullrich, Ph.D. jullrich at sans.org