Health or Performance monitoring to detect security events.
Brent wrote in in response to ChrisM's diary about helping us help you.
"One of the things I stress to other admins is the importance of performance monitoring. Not only is it useful for
diagnosing performance bottlenecks, but it's useful from a security perspective too, provided someone is willing to
skim performance graphs on a regular basis to get a feel for what "normal" is.
For instance, we track the query stats on our DNS servers and back in March I saw an odd jump in query failures on
one of our external DNS servers.
.png)
A look at a 2nd graph
.png)
showed that these queries were for A records. When I see an anomaly like this (things that make me say "hmmm") I go investigate. In this case, it was a flood of queries for hostnames/domains our DNS servers weren't authoritative for (and, of course, they're set up to refuse recursive queries).
What was interesting was these queries initially came from a wide variety of IPs (many of which were in RBLs as
compromised systems) and soon thereafter, they were coming from our IP space, but mostly from blocks not currently
in use.
Checking performance stats has exposed all sorts of things - misbehaving software doing dozens of queries per second
for the same hostname, a compromised system looking up millions of MX records to try to send spam, someone running a
portscanner (and causing a big spike in rejected packets from our egress filters), etc. Ya never know what you'll find, if you just go look regularly. :-)"
I couldn't agree with Brent more. Health and performance monitoring tools can and should be used to detect security related events. "Peacetime learning" or monitoring while not under attack or unusual load is used in DDOS detection. Netflow which is commonly used to detect DDOS attacks today was originally designed for BILLING on "burstable" pipes:)
SNMP monitoring is frequently used to detect attacks against a system. If the memory or other resources suddenly goes
WAY UP you can bet something is wrong and in many cases that will be a security related event. So if your performance and health monitoring team isn't tied tightly to your security team you may want to introduce them.
Lastly the "triad" of security are frequently referred to by the TLA, CIA.
Confidentiality, Integrity, and Availability (2 "new" ones were added a while back Authenticity and Non-Repudiation).
Availability is either one third or one fifth of security practitioner's job, depending on which version of the "triad" your following.
SMS Phishing at the SANSFire 2011 Handler Dinner
After a great "State of the Internet" Panel at SANSFire 2011 with the Internet Storm Center Handlers we began to reflect on Phishing, Spear Phishing, FAKE-AV etc and how this threat is never going away.
In another episode of "Handlers have lives too" we get Phishing and run into strangeness as well. While sitting at our Handler Dinner a Handler Phone buzzed with a text message. Not unusual, but when examined a good gut chuckle rumbled out of the handler (By the way, that handler was me).
The message then got passed around to the rest of the handlers. It was then that Dr Johannes Ullrich, our boss, said "Take a screenshot and post it."
On a serious note, after taking a look at this screen shot, ask yourself, who would fall victim to this? Notice the optout,reply,stop?
.jpg)
One of our sister sites has great information on "Securing the Human OS" and this plays right into that shameless plug [1]. Technology is so pervasive and only going to get more complex.
[1] http://www.securingthehuman.org
Richard Porter
--- ISC Handler @ SANSFire 2011
UPDATE: Image has been moved and hosted on ISC server.
Helping us to help you
Readers and Handlers, Handlers and Readers, it’s a fantastic symbiotic relationship that keeps both parties informed, on their toes, looking at another side of the story which, in my humble option, makes us all better security professionals.
Without the support, information, questions, comments, heads-up, jokes, packet captures*, time and energy supplied by you, the readers, the Internet Storm Center (ISC) can’t be the resource it is today. If you attended the Handlers’ talk at SANSfire this year, this was the final comment from the assembled handlers. We need you to help us to help the collective you. Keep writing in with what you’re seeing, what you have to deal with and, heck, if you disagree with what we’ve said.
Being the new kid on the handlers’ team, seeing the information coming makes me want to be better at my day job. I’ve been reading the ISC for a good number of years, but never thought of writing in with what I was seeing on my systems and networks. My mistake. The more we share, discuss and debate, the more we learn. To steal a film quote "The only way to get smarter is by playing a smarter opponent."** Well, there are plenty of smarter, well-funded and co-ordinated opponents out there, so give yourself a helping hand and share what you’re seeing the ISC.
If you agree, we can pass that information on via the diary pages; it may help someone else make sense of what they are seeing and from a collaborative effort provide an answer for you.
So drop us links to stories and events that you think are important, add comments to the diaries, share with us what you are encountering and struggling with. We won’t always have the answer, but at least you’ll have someone else to offer their suggestions.
*We NEVER get bored of looking through packet captures, especially when trying to solve a puzzle posed or determine if something is happening
** Quote from the Guy Ritchie film “Revolver” which appears to have been made up for the film, not the mythical “Fundamentals of Chess -- 1883” unless they were Geezers back in the day. Unlikely, but Johannes Zukertort was a bit of a card [1].
[1] http://en.wikipedia.org/wiki/Johannes_Zukertor
Chris Mohan --- Internet Storm Center Handler on Duty
Comments