Developing Security

The Curious Case of the Careless Civic

Sometimes the big picture deceives us.

Last night as I was drifting off to sleep I was interrupted by the sound of the doorbell. In my haze I assumed it was some kids pulling a prank and tried to go back to sleep, but still the ringing persisted. Finally I was forced to investigate, so I went downstairs to the door where I found one scary looking dude on my doorstep.

"Your car rolled into the street", he said.

"My car is inside the garage", I replied (hoping that it was indeed true).

"Is you car a Civic?", he asked. To which I responded, "No", and just like that the Midnight Ringer was off.

Puzzled, I went back upstairs and looked out the window. Sure enough there was a Honda Civic in the middle of the road looking indeed like it had just rolled out of my driveway.

I dismissed it and went back to bed. A little while later I was awoken again by the doorbell. This time I went downstairs and opened the door, immediately stating, "The car is not mine". That took care of the second visitor quickly, but I realized that if my family and I were going to get any sleep I was going to need some help. So I taped a sign to the door, "THE CAR IS NOT OURS!!!"

A little later on I was awoken yet again, this time by flashing lights. The city police were investigating the oddly parked Civic. Like any rational person with access to the DMV's database, the officer started with a lookup of the plate number. The problem was, the car did not belong to someone in the neighborhood, so there was no easy win. Then the officer, like the others looked at the car's positioning and concluded that it had to have come from my driveway, so to my door he came. Seeing my posted message he then had little alternative but to begin the slow process of waking my surrounding neighbors in search of the culprit.

This rather annoying incident reminds me of the traditional approach to incident response. We take a look at the SI/EM alerts and start digging on the most likely source and target of the event. We look at the big picture, make a hypothesis and when that doesn't pan out, move on to the next theory. It's all part of the investigative process.

We are missing something here. Sometimes the big picture isn't the key metric that we need. Sometimes the devil is in one little detail.

Had the officer started with the positioning of the wheels, he would have seen that the angle on them would have left the car quite crookedly parked in my driveway, but was a perfect fit for the curb up the street. Instead he spent an hour working through the investigative process because he was mislead by the big picture (the position of the car).

When we have an incident in Information Security, looking at the big picture in metrics and correlated events can sometimes obscure the nugget of information that could save us hours or days of investigation. With the right information, you can jump to the correct answer without a full investigation. A quick scan of the raw logs can often highlight that one piece of the puzzle which explains how the rest of the event unfolded, in a much better way than a high level summary. The human eye is really adept at this type of scan, however volume is the enemy. Perhaps in the future Information Security Software will be able to assist us. The management systems need to get analysts and investigators beyond the dashboard and highlight the key parts of the trace events that can unlock the mystery.

The fact that even my sleep-deprived mind can take a random event like this and relate it to information security is a little disturbing. Now you'll have to excuse me while I take a nap...

Defense in the Deep End

Let's say I had a hypothetical large scale environment to secure with an unlimited budget. I could choose any form of countermeasures I wanted, provided that nothing I purchased remained unused. How would I choose to apply defense in depth?

When defining the required compensating controls for threat management, context is important. URL filtration isn't very exciting for a server that no one logs on to. Neither is database activity monitoring for the PC's in my call center, or a WAF for a sales person's laptop.

In addition to ensuring that the technology fits the role, it also has to provide a complimentary 'threat funnel' with little overlap. Each piece of the puzzle would have to provide some form of value against external or internal threats.

Here are the technologies I would deploy and where:

	Perimeter	Desktops	Laptops	Servers
Firewall
Network Anti-Malware (HTTP, SMTP, etc.)
File-Based Anti-Malware
Network IDS/IPS
System IDS/IPS
Anti-Spam				(On Mail Servers)
Anti-Phishing
URL Filtering & Web Threat Protection
Hard Drive Encryption
VPN
WAF				or (On Web Servers)
DLP
System/Application Log Forwarding (To SI/EM)
File Integrity Monitoring
DAM and Database Security				(On Database Servers)
Directory Integrity Monitoring				(On Directory Servers)

You'll likely see a few things missing (most notably NAC). Working within my no-shelfware condition I would likely have to pass on several technologies that are too hard to deploy, manage, or provide niche protection.

I would want to manage carefully the balance between best-in-breed and vendor sprawl. Ideally all of these functions could be accomplished by 2-3 vendors and in all cases reporting back to my SI/EM. I would compliment these point products with vulnerability scanning, risk management, patch management, event correlation and change control.

The problem is, no one has a fantasy budget of unlimited money and more importantly, time to assess, configure and monitor all of the controls they really need. Defense in depth is complicated, expensive and does require a very diverse feature mix. We are constrained, unable to create our dream suite of protection.

When it comes to enterprise threat management, I can't help but feel we are in over our heads.

 

Know Thine Enemy

How do you defend against an enemy of unseen size and composition? Every day security software and security professionals are asked to do just that. None of us really know who we are dealing with in this new age of financially motivated cyber crime.

In the latest episode of the Network Security Podcast (number 151) Martin and Rich discuss the fact that conferences like BlackHat are full of security professionals and ethical hackers. Today's true cyber criminals, if present at the conferences, lay low for good reason. This wasn't always the case. Hackers seeking notoriety in the past were happy to share their skills. The current fraudsters are in it only for the money not the fame, so we are left to learn from researchers and audit logs from known breaches ('known' being the painfully operative word). You can take hacking courses, but how close do these parallel the real skills and techniques of highly funded, organized criminals?

Compare this to the ethical side of security where there is a wealth of books, classes, certifications, magazines, blogs, podcasts and so on. If the organized cyber criminals are openly collaborating, there is little evidence. Most of the so-called 'underground' hacking sites are filled only with script kiddies like Mikey.

In the research I have seen, we have at best a rough idea of how attacks are staged and coordinated. I want to know who we are dealing with. How are they organized? How do they recruit? How are they paid? How do they pick their victims? How do they cover their tracks? What tools do they use and what do they build? We have only a partial picture in all of these areas. How do I defend my keep if I don't know the size of the opposing force and the weapons they wield?

There is an old adage in security that says "We have to protect everything where the bad guy only has to find one weakness". That is an annoyingly true axiom, made all the worse by the fact that we are being constantly stabbed in the back by unseen assailants.

I badly want to see someone infiltrate the groups behind the headline attacks and write a meaty expose. Law enforcement or the intelligence community won't cut it since they wouldn't be able to reveal the results. What we need are REAL investigative journalists who happen to be a top notch hackers and doesn't mind some jail time (or worse) if they get caught. Any takers?

Hypervisor Security APIs according to Shakespeare

There is a feud raging over the role of the Hypervisor in providing security services to Virtual Machines. One family, VMware (The Montagues) believes that external VM introspection opens up a world of new opportunities for security. On the other side, Citrix (The Capulets), believes the Hypervisor should just get out of the way and leave security to the existing in-guest and network paradigm. Before you ask... no, Crosby and Hoff are not the star crossed lovers.

I'm going to wade into this debate, despite my admittedly biased view (I'm working with a team on a VMsafe Virtual Appliance). To do so I'm going to borrow a little help from The Bard.

"Nothing can come of nothing."

As King Lear required penance for inaction, so to does Citrix risk its opportunity to foster innovation through inaction. Security APIs like VMsafe do offer an innovative way to secure and monitor Virtual Machines through introspection of CPU, memory, network and disk. With the exception of network, the other inspection points are not currently available unless you put software in the guest. The moment you do, you are vulnerable to evasions, being crippled by malware, and the challenges of inspection across a diverse set of platforms. If you allow inspection externally (via Security APIs) you immediately have a level platform to inspect all logical processing and disk/network usage, opening up a world of opportunities.

The first wave of products will likely replicate existing security controls (AV, Network/Behavioral IDS/IPS, DLP, Offline patching, etc). Shortly though, we are likely to see a new wave of security controls that currently don't have a handy acronym, which will be enabled only by this new method. There are community XEN projects to introduce introspection, however if they are not embraced by Citrix a large opportunity will be missed.

"I go, and it is done; the bell invites me."

Simon Crosby's main argument against providing APIs in the Hypervisor is that it fattens the Hypervisor and makes it less secure. I see no earthly reason why these APIs cannot be implemented in a compact nature with the most stringent security. An API is by nature thin, since it's the responsibility of the consumer to instrument it. Why not, like Macbeth just get it over with and do it securely.

"Though this be madness, yet there is method in't."

While there are non-API methods for network introspection available or coming (see Hoff's posts about vShield Zones and Citrix Network Changes) VMsafe-NET provides a low impact way of tapping the network traffic as close to the virtual NIC as possible. There is no need to alter virtual networking or re-route traffic through appliances (virtual or otherwise).

The simplicity for users justifies the complexity for vendors who must retool to utilize the APIs.

"To be, or not to be, that is the question"; when it comes to Hypervisor Security APIs the benefits to existence outweigh the consequences.

Security as a Service and the question of Privacy

During a panel session at the RSA conference, Eva Chen (CEO of Trend Micro) and Jay Chaudhry (CEO of Zscaler) had a lively debate about the role of Cloud computing in providing Security as a Service. Trend Micro recently released the Smart Protection Network, which uses an end-point agent that works closely with a Cloud service to provide web, email and file reputation services. Zscaler by contrast, proxies all web traffic through the service requiring no end-point agent. The debate really focused around how much of the security service should be located externally. There were a lot of points debated including latency, bandwidth, and reliability but one of the most contentious issues was that of data privacy.

Before I elaborate, it's important to mention why an organization would want to consider a Cloud offering for outsourcing aspects of security. One of the earliest Security as a Service offerings was Postini in early 2000 (acquired by Google in 2007). Moving Anti-Spam and Email Anti-Malware outside of the organization was a simple matter of changing the MX record. Organizations could rapidly acquire these services without the costs and complexities of a physical appliance or software solution, and the per user pricing ensured a fair cost for organizations of all sizes.

So clearly, cost (both OPEX and CAPEX) was an initial motivator but over time providers have utilized the central nature to provide significant benefits other than cost. The providers can be much more responsive with real time services replacing staged updates, and multi-tenant data correlation enhances everyone's service. The central infrastructure also proves more fault tolerant than individual appliances or software solutions for the most part, though when service provider failures do occur they have a much broader impact. In short, moving to the Cloud enhances security providing for much bigger and more intangible benefits.

Recently a new generation of Security as a Service Anti-Malware offerings have emerged. Trend Micro, Panda Security and McAfee have all recently released Cloud-based services. They all use a driver at the end points which instead of scanning against a set of signatures, creates a hash of the file. The hash is looked up in the Cloud and the service returns a result of good, bad or unknown. In the case of unknown files, the contents are then fully scanned (either locally or uploaded to the Cloud depending on the provider and circumstance). A local cache of hashes allows future full scans to determine safe files much faster than signature matches. The collective nature means that 99.9% of the files on your system have likely already been scanned and the hash categorized by someone. There is actually a lot of overlap here with another type of security control: File Integrity Monitoring, but I'll leave that for another day. These capabilities are augmented by other reputation and URL blocking services to create a strong shield against web threats. As with Cloud-based email proxies, because hashes (and sometime content) are being sent to the Cloud, there is a measure of trust required in the provider.

Zscaler, Purewire and others offload all of the work to the Cloud by providing the various security services (URL Filtering, Anti-Malware, DLP, etc.) as a web proxy/gateway. The advantage, of course is the lack of end-point agents to deploy and maintain as well as platform neutral protection for many devices. Though I suspect for many organizations the idea of all of their web traffic going through an external provider may be a barrier.

Other Cloud-based scanning services, like Qualys' Vulnerability Scanner or HP's Application Security do not deal with raw data from an organization, though the findings from there are still very sensitive.  

Clearly, the architecture of the solution is important. The more of the stack we move to the Cloud the less there is to deploy and manage locally, but the more trust we have to put in the provider and the more reliant we are on Internet connectivity and availability of the provider. The axiom "Trust but verify" is hard to apply in these environments because of the lack of visibility into the data end-to-end. The Cloud Security Alliance is working hard to address the complex issues of shifting layers of the stack to providers in all areas of Cloud computing.

As much as organizations may hesitate on Security as a Service, it is too important of an advancement to simply ignore. The enhanced security services and significant cost savings are going to inevitably pull us closer to the Cloud. It is key to evaluate the architecture and determine the acceptable risk level. Trusting a provider's service quality, up time or ROI is nothing new, but trusting a provider with our precious data does require a new level of something else usually associated with the sky... Faith. If we put the faith in the providers we can realize significant benefits in return. Other SaaS offerings like CRM giant SalesForce.com have done a good job of instilling the level of trust needed to be successful and I suspect the Security as a Service market will do the same.

I'm interested to see where this goes and how many of our current 'point solutions' end up in the Cloud (or partially in the Cloud). I'm also interested to see how this impacts how the Cloud itself is secured. It seems only a natural fit that Security as a Service will ultimately have a large role in other SaaS, PaaS and IaaS offerings both external and internal to the organization. 

M&A, RSA, CSA, and why I have been MIA

This has been one seriously busy couple of weeks. Between major changes at work, post-RSA wrap up and reviewing the initial material from the Cloud Security Alliance there has been a lot going on!

Trend Micro Acquires Third Brigade

The company I have been working for, since it started nearly five years ago has signed a definitive agreement to be acquired by Trend Micro. As you can imagine this is a pretty exciting time for us and the reaction in the press and community has been very positive. I'm personally looking forward to this new era and excited about what we will produce. We have been working with Trend over the last 18 months on the Intrusion Defense Firewall and I firmly believe this deal will result in even better products to come. (Press Release) (Media/Analyst Teleconference)

RSA Wrap Up

Digging through the post RSA aftermath takes some time, though there are some worthy nuggets of information. After reviewing all of the keynotes I missed I highly suggest checking out Brian Smith from TippingPoint as he introduces an interesting method of making multiple products work together more effectively. I also recommend watching the Qualys keynote, Philippe Courtot does a really good job of putting security as a service in perspective.

If you missed RSA there are some good summaries out there including Anton Chuvakin's four part series (I, II, III, IV) and Ben Tomhave's summary. Dan O’Neill does a good job of summarizing what was for me, the main event (and I agree, it was unfortunate that it lacked the sumo suits).

Cloud Security Alliance

I finally had the opportunity to read the first document from the Cloud Security Alliance, "Security Guidance for Critical Areas of Focus in Cloud Computing". If you are not familiar with the CSA, it is a grassroots effort to facilitate the mission to create and apply best practices to secure Cloud computing. This first document tackles the critical job of providing definitions for cloud architecture and coverage of 14 domains related to Cloud computing.

Anyone familiar with Rational Survivability will find the Cloud architecture domain familiar as Christofer Hoff has used his blog to vet most of the material present here. Moving beyond the architecture discussion is critical and I sincerely hope this will serve as the defacto standard, it certainly is the most comprehensive and vendor neutral way to define the Cloud I have seen (though I do see discrepancies in the definition of databases as IaaS or PaaS).

The other domains are very much initial material to start the discussion. Having expert authorship in each of the areas was an excellent way to start (Disclaimer: My CTO, Brian O'Higgins contributed domain 15 on virtualization), however an important second version of the document will be developed through community involvement later this year. I'm looking forward to the second version where more industry experts have an opportunity to contribute thereby balancing and enriching the material.

I strongly encourage anyone with a stake in Cloud computing to get involved. They will be looking for additional volunteers in a wide variety of capacities soon.