Top20 List Updated
The SANS Top 20 Internet Security Vulnerabilities list was updated today with information for the 2nd quarter of 2005. See the current information
TCP/1433 Remains Elevated
We continue to see an increase in probes on TCP port 1433. If you have any interesting packet captures, please submit them
ZoneAlarm 6.0 Released
The entire ZoneLabs ZoneAlarm product family has been updated. Version 6.0 of each of the products was released a couple of days ago. For more information, go .
The Penetrating Packets: Spam E-Mail
Background: What Happened?
Last week, I was awoken early in the morning by a ringing cell phone. A co-worker was calling to say that they hadn't received any email in about 6 hours, which for them was unusual. I got up, went downstairs, got online, connected to our relay email server, checked the queue and was immediately wide-awake when I saw that there were 150,000 plus messages sitting in our email queue waiting to be processed. That's outside the normal scope of our queue size by just a tad ;-)
So I started digging through the messages in the queue looking to see if I could figure out the problem. I knew our email server was not an open email relay (or at least it hadn't been yesterday). I saw that most of the email had "From:" and "To:" headers involving domains in China and Taiwan. None of the obvious spam email was to or from any of our domains or addresses.
Ok, so why was the email being accepted by our server? I noticed that the values in the "From:" and "To:" fields were using the Big5 character set and started wondering if perhaps this was a new trick of spammers to get around SMTP header filters on email servers. So I shutdown the email server and start working to clean out the queued spam email.
Several hours later, I have the queue reasonably cleaned up and restarted the email server to start processing email ... and see the queue immediately start to accumulate lots of fresh spam. All right, what's going on here?!?!
Research: Where's It Coming From?
So I started looking through the logs further and realized that all this spam email wasn't coming from the Internet ... it was coming from our internal email server used by employees to send/receive email. Checking the logs on that email server, I saw LOTS of SMTP connections from our SSH gateway. In order to access the end-point email server from outside the office, employees needed to connect through an SSH tunnel. The computers on our local network are considered to be trusted by our relay email server and so it will accept any email from these computers no matter what the destination address is.
Ok, so this implied that the spam was coming through an employees computer. Killing off the established SSH connections of the dozen or so folks currently connected terminated the ongoing flood of new spam messages entering our system. Now to try to figure out where the problem computer is. We contacted each of the employees who had been connected at the time and one by one started ruling them out as the source. Almost everyone had a broadband Internet connection and was behind a hardware firewall which blocked connection attempts from the Internet from reaching their computer. The few folks without hardware firewalls had personal software firewalls on their computers and we verified that the firewall was working correctly. One down, another eliminated, and so on until, hey wait a minute, we just eliminated the last employee who was online this morning!!! So if everyone had a hardware or software firewall that was correctly blocking connection requests from the Internet, how did someone connect to them? (Hint: It's a 3-letter word).
Back to the logs to try to find an IP address that will hopefully match up with one of the IP addresses an employee had earlier in the morning. Most of the SPAM messages had forged HELO values that were completely bogus but some had IP addresses and after lots and lots of searching , I got a match with someone who was connected throughout the night when the spam flood was ongoing.
So I contacted the suspected employee again and we went through the firewall settings. The problem turned out to have two parts.
Part 1 - Contributing Factor: When you setup SSH tunnels on your local machine to connect to a remote server, you usually associate the listening port with your local loopback adapter. Thus, only processes on your computer can connect to the tunnel endpoint and be connected to the whatever is on the far end. You can configure many SSH clients to bind the listening port to *ALL* interfaces on your computer (which will include any Ethernet or Wireless interfaces). In such cases, a remote user could then initiate a connection request to the tunnel port on your computer (if not blocked by a firewall) and go through the tunnel to whatever is on the remote end.
As it turned out, this employee's SSH client was so configured to bind tunnel end-points to all interfaces instead of to just the loopback adapter, thus creating the potential for a problem. But still, the hardware and software firewalls should have prevented such connections. What happened?
Part 2 - The Culprit: In the first "Die Hard" movie, Hans Gruber says "You asked for a miracle. I give you the F.B.I." Today, I say "You asked for a culprit. I give you the A.O.L."
If you have an existing Internet connection and then run the AOL software to have access to AOL content, you are in fact, setting up a VPN across the Internet between your computer and the AOL servers. If you run "ipconfig", you will see that you now have an additional virutal Ethernet adpater that has an IP address that is from AOL. When you access any AOL content, the packet comes from your computers virtual AOL IP address and is encapsulated in a packet that comes from your computer's real IP address. The responses from AOL come back in an encapsulated packet as well.
So how does this help someome bypass your firewall? Your computer has two IP addresses while you are connected to AOL. IP Address 1 is the real one and IP address 2 is the virtual one from AOL. An attacker who tries to connect to your real IP address from the Internet will be blocked by either your hardware or software firewall (if you haven't configured a hole for that type of traffic). But what happens if they scan AOL's network and happen to try to connect to the IP address that AOL has currently assigned to you? AOL will accept the traffic, determine who currently has that IP address and then encapsulates that packet inside of a new one that it addressed not to your virtual AOL IP address but to your real IP address. Because you established the connection to AOL, your firewall has tagged this as an established connections and so any data coming back is considered to be part of an approved, established connection as well and is allowed to pass through your firewall.
Make sure you know what software you have installed on your computer and how it is configured. Defense-in-depth doesn't help if you have provided an access path that bypasses all of your defenses.
Jul 25th 2005
1 decade ago