Here is my next monthly "Security Warrior" blog round-up of top 5 popular posts/topics this month.

Disclaimer: all this content was written before I joined Gartner on Aug 1, 2011 and is solely my personal view at the time of writing.

1. “The Last Blog Post!” is obviously BY FAR the most popular post in August. It announces my departure from consulting business in order to join Gartner as a Research Director with SRMS team.
2. “Top 10 Criteria for a SIEM?” is an EXAMPLE criteria list for choosing a SIEM.  Also see “On Choosing SIEM” which is about the least wrong way of choosing a SIEM tool – as well as why the right way is so unpopular.
3. “On SIEM Services” is a quick overview of services that you really should be getting with that SIEM purchase
4. “Log Management at $0 and 1hr/week?” is pretty much what it is. How to do log management under extreme budget AND time constraints?
5. A very old post (2009), “Log Management + SIEM = ?", is about architecting SIEM together with log management.

Also see my past annual “Top Posts” - 2007, 2008, 2009, 2010).

Enjoy!

About me: http://www.chuvakin.org

Executive summary: you need to procure services when you buy a SIEM tool, if you don’t – you’d be sorry later.

Even if you are amazingly intelligent and have extensive SIEM experience – see above.  Even if you saw a successful SIEM project that didn’t include vendor or 3rd party services with your very eyes – see above. Even if your SIEM vendor tells you “you don’t need services” – see above. See above! See above!! See above!!!

Let’s analyze this “SIEM services paradox.” A lot of organizations – way too many, in fact – balk at the need to procure related services before, during and after their SIEM purchase. The thinking often goes like this: we need a SIEM and this box <points at the appliance still in the box> is a SIEM. That’s all we need. What services? Why services? Huh?

In reality -  and this is what I sometimes call “secret to SIEM magic” – that box is not a SIEM. That box, when racked and connected to your network, is STILL not quite a SIEM. Only when you “operationalize” it (see picture), then you can say that you have Security Information and Event Management (SIEM) capability in your organization and that you do “real-time” security monitoring.

Now, be honest, do you know how to deploy a SIEM tool and then figure out the shortest path to its operational success? Probably not… thus services/consultants who will work WITH you to make it a reality by arriving at the best possible way of benefitting from SIEM in your environment. Which use case give you the best bang for the SIEM buck? Which one will show a “quick win” to your management? Which one is more likely to detect an attacker in your network?

When a SIEM vendor tries to sell you services, it is NOT vendor greed – but simply common sense. And if you say “no”, it is not “saving money” – but being stupid. SIEM success  out-of-the-box (while real, in some cases!) is a pale shadow of what a well-thought through deployment looks like! My [broken] analogy is: you buying a nice shiny Aston Martin and then only using it to commute to a train station 1 mile from home. Will it work? Yes. Is this a good investment and a good experience? Hell  no!

So, no, SIEM is NOT useless without services, but it is unlikely to reach its full potential. Pitfalls to SIEM success are many, and navigating them requires help.

And, no, outsiders alone cannot do it. You will need to help them  help you.

This also leads to the rise of managed or co-managed SIEM options (which are NOT MSSPs, BTW!) as more people realize that a) they need a SIEM and b) they cannot handle a SIEM. Future cloud SIEM will (when it emerges) try to tackle the same problem of being simpler to operate and thus simpler to operationalize.

Today, most SIEM vendors offer an extensive menu of services to go with a product, and there are also some smart third parties.  Many services around SIEM can be organized as follows.

Pre-sale services examples:

• Product selection help
• Vendor differentiation analysis and shortlist definition
• Regulation analysis and business cases review
• Product strengths/weaknesses analysis
• Product fit for type of project
• Product fit for vertical / business type
• RFP definition assistance
• Current tools vs requirements gap analysis

Services offered during SIEM acquisition and deployment:

• SIEM implementation
• SIEM project planning
• Proof-of-concept deployment management
• Product testing in production environment
• Data source integration and collection architecture
• Default contents tuning

Post-sale, operational services:

• SIEM analyst training
• Performance tuning and capacity planning
• SIEM project management
• Custom content creation
• Custom device integration
• SOC building

Vendors and consulting firms offer other types of services as well all the way up to “co-managed SIEM” where a 3^rd party firm manages your SIEM deployment for you. Will future SIEM work better out of the box? Yes, I think so. Will SIEM ever be as simple as a firewall? No, never: it is inherent complexity of security monitoring that cannot be squeezed out even by creative engineering…

Enjoy ... as this was my final blog post on SIEM.

Possibly related posts on SIEM:

• On Broken SIEM Deployments
• Top 10 Criteria for a SIEM?
• Algorithmic SIEM "Correlation" Is Back?
• How Do I Get The Best SIEM?
• Log Management->SIEM Graduation Criteria: Violate at Your Own Peril!
• How to Replace a SIEM?
• SIEM Resourcing or How Much the Friggin’ Thing Would REALLY Cost Me?
• How to Write an OK SIEM RFP?
• On Choosing SIEM
• "So, What Should I Want?" or How NOT to Pick a SIEM-III?
• The Myth of SIEM as "An Analyst-in-the-box" or How NOT to Pick a SIEM-II?
• I Want to Buy Correlation” or How NOT to Pick a SIEM?
• Log Management + SIEM = ?
• On SIEM Complexity
• SIEM Bloggables: SIEM Use Cases and Whitepaper with detailed SIEM use cases
• Log Management / SIEM Users: "Minimalist" vs "Analyst"
• All posts labeled SIEM

About me: http://www.chuvakin.org

Imagine you own a broken, dilapidated, failing SIEM crap deployment. What? Really… that, like, never happens, dude! SIEM is what makes unicorns shine and be happy all the time, right?

Well…mmm… no comment. In this post, I want to address one common  #FAIL scenario: a SIEM that is failing because it was deployed with a goal of real-time security monitoring, all the while the company was nowhere near ready (not mature enough) to have any monitoring process and operations (criteria for it).  On my log/SIEM maturity scale (presented here, also see this related post from Raffy), they are either in the ignorance phase or maybe log collection phase.

And herein lies the problem: if you deployed one of the legacy, born in the 1990s SIEMs that are not based on a solid log management platform, the tool will actually suck at the very fundamental level: log collection. The specific issue here is that most of these early tools were designed to only selectively collect what was deemed necessary for real-time security monitoring (vs all log data). In essence, you have a tool with monitoring features (that you don’t use) and with weak collection features (that you can use, but they are weak).

What to do? You have these options:

1. Leave it to rot; you can always keep it just to boast to your friends (and PCI QSAs) that “ye own one of ‘em olde SIEMs”
2. Blow it away and join the “SIEM doesn’t work” crowd – and maybe buy a simple log management tool later
3. Deploy a log management tool to “undergird” your crappy SIEM; you have a choice of buying from the same SIEM vendor (if they have it) or a different vendor
4. Built your own log management layer on syslog and open source tools

I have seen people take either of the above four. Personally, I have seen much more success with the option #3 (buy log management) and not infrequently with #4 (built log management) – BTW, this deck might help you choose. You want to move your SIEM setup from “get some logs – ignore all logs” model to “collect all/more logs – review some logs” which is typically much more aligned with your level of maturity. And then grow and solve more problems with your SIEM and demonstrate “quick wins.” While you are at it, review some architecture choices discussed here.

Enjoy …while it lasts.

Possibly related posts on SIEM:

• Top 10 Criteria for a SIEM?
• Algorithmic SIEM "Correlation" Is Back?
• How Do I Get The Best SIEM?
• Log Management->SIEM Graduation Criteria: Violate at Your Own Peril!
• How to Replace a SIEM?
• SIEM Resourcing or How Much the Friggin’ Thing Would REALLY Cost Me?
• How to Write an OK SIEM RFP?
• On Choosing SIEM
• "So, What Should I Want?" or How NOT to Pick a SIEM-III?
• The Myth of SIEM as "An Analyst-in-the-box" or How NOT to Pick a SIEM-II?
• I Want to Buy Correlation” or How NOT to Pick a SIEM?
• Log Management + SIEM = ?
• On SIEM Complexity
• SIEM Bloggables: SIEM Use Cases and Whitepaper with detailed SIEM use cases
• Log Management / SIEM Users: "Minimalist" vs "Analyst"
• All posts labeled SIEM

About me: http://www.chuvakin.org

OK, this WILL be taken the wrong way! I spent years whining about how use cases and your requirements should be THE MAIN thing driving your SIEM purchase. And suddenly Anton shows up with a simple ‘Top 10 list’, so…. blame it on that cognac.

This list is AN EXAMPLE. SAMPLE. ILLUSTRATION. It is here FOR FUN. If you use it to buy a SIEM for your organization, your girlfriend will sleep with your plumber.  All sorts of bad things can and likely will happen to you and/or your dog – and even your pet squirrel might go nuts. Please look up the word “EXAMPLE” in the dictionary before proceeding!

On top of this, this list was built with some underlying assumptions which I am not at liberty to disclose. Think large, maybe think SOC, think complex environment, etc. Obviously,  an environment with its own peculiarities … just like yours.

With that out of the way, Top 10 Criteria for Choosing a SIEM … EXAMPLE!

1. User interface and analyst experience in general: ease of performing common tasks, streamlined workflows, small number of clicks to critical functions and efficient and quick information lookups (including external information) when needed during the investigation

2. Correlation: correlation engine performance, ease of rule creation and modification, canned rule content, cross-device correlation based on normalized/categorized data; additional analytics methods including analysis of stored/historical log data; ability to test rules before production deployment

3. Log source coverage: full integration of most (better: all) needed log sources before operational deployment, detailed parsing and normalization of all fields needed for the analysts’ work; coverage of device, OS and application logs; wide use of real-time log collection methods, even at a cost of using agents

4. Dashboards and analyst views: availability of required analyst views, flexibility and customization, drilldown capability to see additional details, ease of modification and tuning, real-time operation (not periodic polling)

5. Reporting: report performance, visual clarity, ease of modification and default/canned report content, ability to create custom reports on all data in a flexible manner without knowing the SIEM product internal structures and other esoterica

6. Search and query: high (seconds) performance of searches and queries when investigating an incident, access to raw log data via an efficient search command, tied to the main interface

7. Escalation, shift and analyst collaboration support: a system to manage collaborative investigations of security issues, take notes, add details and review/approve the workflow; likely this requires an advanced case management / ticketing system to be built in

8. Ability to gradually expand storage on demand when the environment is growing; this applies to both parsed/normalized data storage as well as raw log storage

9. Complete log categorization and normalization for cross-device correlation that enables the analysts to “cross-train” and not “device-train” before using the SIEM well.

10. New log source integration technology and process: ability to either quickly integrate new log sources or have vendor do it promptly (days to few weeks) upon request

Got any comments?

If not, well, enjoy it … while it lasts.

Possibly related posts:

• Algorithmic SIEM "Correlation" Is Back?
• How Do I Get The Best SIEM?
• Log Management->SIEM Graduation Criteria: Violate at Your Own Peril!
• How to Replace a SIEM?
• SIEM Resourcing or How Much the Friggin’ Thing Would REALLY Cost Me?
• How to Write an OK SIEM RFP?
• On Choosing SIEM
• "So, What Should I Want?" or How NOT to Pick a SIEM-III?
• The Myth of SIEM as "An Analyst-in-the-box" or How NOT to Pick a SIEM-II?
• I Want to Buy Correlation” or How NOT to Pick a SIEM?
• Log Management + SIEM = ?
• On SIEM Complexity
• SIEM Bloggables: SIEM Use Cases and Whitepaper with detailed SIEM use cases
• Log Management / SIEM Users: "Minimalist" vs "Analyst"
• All posts labeled SIEM

About me: http://www.chuvakin.org

Just FYI, I am speaking at Gartner Catalyst 2011 event in San Diego tomorrow. The topic is “Five Best and Five Worst Practices for SIEM.”

“Implementing SIEM sounds straightforward, but reality sometimes begs to differ. In this session, Dr. Anton Chuvakin will share the five best and worst practices for implementing SIEM as part of security monitoring and intelligence. Understanding how to avoid pitfalls and create a successful SIEM implementation will help maximize security and compliance value, and avoid costly obstacles, inefficiencies, and risks.”

Time: Tuesday, 26 July 2011
02:45 PM to 03:20 PM

Location:   Hilton San Diego Bayfront
1 Park Boulevard
San Diego, CA 92101

If you are around, come see me here.

About me: http://www.chuvakin.org

As I was drinking cognac on the upper deck of a 747, flying TPE-SFO back from a client meeting, the following idea crossed my mind:  CAN one REALLY do a decent job with log management (including log review) if their budget is $0 AND their “time budget” is 1 hour/week? I got asked that when I was teaching my SANS SEC434 class a few months ago and the idea stuck in my head – and now cognac, courtesy of China Airlines, helped  stimulate it into a full blog post.

So, $0 budget points to using open-source,  free tools (duh!), but 1hr/week points in exactly the opposite direction: commercial or even outsourced model.

The only slightly plausible way it that I came up with is:

1. Spend your 1st hour building a syslog server; it can be done, especially if starting from a old Linux box that you found in the basement (at $0); don’t forget logrotate or equivalent
2. Spend a few next weeks (i.e. hours) configuring various Unix, Linux and network devices (essentially, all syslog log sources) to log to it
3. Consider deploying Snare on a few Windows boxes (if needed); it would likely be easier to do than doing remote pull – too much tuning might be needed
4. Next, drop a default OSSEC install on your log server and – gasp! – enable all alerts
5. Spend the next  few hours (in the next few weeks) turning off the ones that are too numerous, irrelevant or don’t trigger any action in your environment.
6. If you log volume fits within a free splunk license size (500MB/day), also spend an hour deploying splunk on your log server and have it index all gathered logs
7. Now you’d be spending your “one log hour each week” on reviewing alerts and (if installed) digging in splunk for additional details
8. Congrats! $0 and 1hr/week gave you semblance of log management and even monitoring….

What do you think? It just might work for organizations with severe time AND money constraints.

Enjoy the post … while it lasts.

BTW, on a completely unrelated note:  do you think EVERY organization above a certain size NEEDS a SIEM? Or WILL NEED a SIEM in the future?

About me: http://www.chuvakin.org

Somebody asked me that question “Do I need SIEM or do I need log management?” yesterday again, and I figured I’d repost this “bit of Anton’s wisdom” (ego alert!), so that people can just read this instead of repeatedly bugging me with this question.

Q: How do I figure out whether I need SIEM or log management?

A: You need log management – if you have computers, IT, data, etc. Period! This is not really a discussion item at all, since about 1986 or so. But do you also need a SIEM? You might think you need it, but you would only be able to benefit from it and satisfy that need if your organization fits the following "graduation criteria from log management to SIEM:”

1. Response capability: The organization must be ready to respond to alerts soon after they are produced. Incident response process/procedures are a must
2. Monitoring capability: The organization must have or start to a build security monitoring capability such as a Security Operations Center (SOC), or at least a team/person/resource dedicated to ongoing periodic monitoring.
3. Tuning and customization capability: The organization must accept responsibility for tuning and customizing the deployed SIEM tool; pure out-of-the-box SIEM deployments rarely succeed.

(originally written for this paper where the above are clarified in more detail)

Possibly related posts:

• All my posts about SIEM

About me: http://www.chuvakin.org

As all of you know, Gartner SIEM MQ 2011 is out – you can see it here (or here) without registration. The quadrant mostly matches my recent SIEM project experience.

My observations follow below:

• CA “SIEM” and “Log Manager” are finally wiped off the face of the Earth (=removed from SIEM MQ), NetIQ is dumped down to the Niche. As they should be.
• Honestly, Symantec SSIM in Leaders is a mystery to me; must be those invisible non-competitive deals or EU/APAC deals. I’ve not seen them on an enterprise SIEM shortlist in the US for a loooooooong time. The rest of the leaders match my expectations fully (and four of them have been at some point my consulting clients)
• Splunk is now officially a [sub-par] SIEM, even though it is really not. Is that good or bad? Well, they got their “honorable mention” for the last few years and now they are in the quadrant. BTW, this example shows that you can make A LOT of money by being free and not in any Magic Quadrant!
• Visionary sector of the MQ galaxy is extremely crowded – but with very different tools, ranging from Prism to Trustwave. Many organizations will choose a tool from this sector, but need to be careful – read the related posts below for some selection ideas and pitfalls.

BTW, congrats to all the vendors who got added this year: AlienVault, Tripwire, splunk and the regional SIEM guys.

As always, apart from insight, the MQ document has a good share of unintentional hilarity, for example:

• “This company declined to provide any information to Gartner for this research” (Darwin Awards anybody?)
• “Customer feedback on product function and support is mixed.” (Anton translation: product usually doesn’t work?)
• “Non-English-language versions of XYZ are not available.” (Anton’s comment: is everything else about the product perfectly perfect?)

Finally, if anybody is wondering, I think the concept of Magic Quadrant (whoever at Gartner came up with) is brilliant. However, many wrong  SIEM purchase decisions I’ve seen made usually stem from the decision maker’s own ignorance and not from whatever document or market visualization he has in his possession. Keep this in mind…

Rocky, your turn!

Possibly related posts:

• How Do I Get The Best SIEM?
• How to Replace a SIEM?
• SIEM Resourcing or How Much the Friggin’ Thing Would REALLY Cost Me?
• How to Write an OK SIEM RFP?
• On Choosing SIEM
• "So, What Should I Want?" or How NOT to Pick a SIEM-III?
• The Myth of SIEM as "An Analyst-in-the-box" or How NOT to Pick a SIEM-II?
• I Want to Buy Correlation” or How NOT to Pick a SIEM?
• Log Management + SIEM = ?
• On SIEM Complexity
• SIEM Bloggables: SIEM Use Cases and Whitepaper with detailed SIEM use cases
• Log Management / SIEM Users: "Minimalist" vs "Analyst"
• All posts labeled SIEM

About me: http://www.chuvakin.org

Note: this has been written for “Cisco MARS blog” as a guest post and is reposted here for posterity.

Ouch! That “Venus” SIEM  appliance that we got with routers has finally croaked. That piece of PHP brilliance that pre-pre-previous security engineer wrote has been buried under the thick pile of XML. That managed SIEM provider has annoyed us one last time.

What do the above situations have in common? The unfortunate time to replace your SIEM has come. What to expect, apart from copious amounts of pain? This post will shed some light on this conundrum, based on author’s experiences.

First, it goes without saying that it is better to choose the right SIEM the first time (e.g. see “On Choosing SIEM” and other posts mentioned below) than to migrate from a SIEM that has been collecting logs (and dust) for a few years. However, you might not have any say in the matter – you might have inherited it, your “evil boss” might have procured the previous SIEM without asking you or you might have built it yourself after a particularly bad hangover… Also, your organization might have simply outgrown the SIEM or your early generation SIEM vendor has not kept up with innovation in the space. In any case, you have a SIEM and you need a new one.

Let’s look at the good side of the situation:

• It is very likely that you learned some super-valuable lessons from your previous SIEM experience (other people have to hire consultants to get to those lessons) and now can avoid the common purchasing process pitfalls (some discussed here, BTW)
• You have much more confidence while discussing confusing SIEM features with vendors – speaking from your previous SIEM experience (this alone will make your new SIEM purchase process much less painful)
• You have some semblance of the logging policy across the systems that log into SIEM – that puts you ahead of those organizations who are just getting their first SIEM or log management tool
• It is possible that you built some operational procedures around SIEM (such as for PCI DSS log review or other purposes) and those would be handy for a new SIEM as well
• If you have to write an RFP (as I discuss here), the chances are that your new RFP would be MUCH better and more likely to result in a good vendor short list
• Treat this situation as positive, think “I now know more than 90% of people buying a SIEM, thus my new SIEM project will be a success”

A few things to avoid and pay attention to:

• Suppress that “I’d buy anything but this crap” mentality – think “what problems will a new SIEM solve or solve better?”
• Avoid taking shortcuts (such as not doing a PoC); you are more knowledgeable, but not prescient…

How might a migration process look like? This assumes that you have already selected a new product, tested it in the lab and are ready for production deployment.

• Prepare to run both products for some time – this might range from a few weeks to months
• Draft the new SIEM vendor to help you migrate the data; after all, they are getting the prize
• Potentially, be prepared to keep the old SIEM running (without paying for the support contract, of course) or at least keep the old data backups – this becomes important if complete data migration is impossible due to architecture differences between the new and old SIEMs.  Ideally, your log management tool will hold raw log backups and so keeping the old SIEM in operation won’t be needed.
• One of the biggest migration efforts will be migrating SIEM content: reports, rules, views, alerts, etc. As well all know, such content is not really portable across SIEMs and you should be prepared to simply recreate all the custom content AND all the default content that you used in the the old SIEM and that the new SIEM might lack.

By the way, I have seen more than a few organizations start from an open source SIEM or home-grown log management tool, learn all the lessons they can without paying any license fees – and then migrate to a commercial SIEM tool. Their projects are successful  more often than just pure “buy commercial SIEM on day 1” projects and this might be a model to follow (I once called this  “build then buy” approach)

Possibly related posts:

• SIEM Resourcing or How Much the Friggin’ Thing Would REALLY Cost Me?
• How to Write an OK SIEM RFP?
• Log Management Tool Selection Checklist Out!
• On Choosing SIEM
• "So, What Should I Want?" or How NOT to Pick a SIEM-III?
• The Myth of SIEM as "An Analyst-in-the-box" or How NOT to Pick a SIEM-II?
• I Want to Buy Correlation” or How NOT to Pick a SIEM?
• Logging, Log Management and Log Review Maturity
• Log Management + SIEM = ?
• On SIEM Complexity
• SIEM Bloggables: SIEM Use Cases
• Whitepaper with detailed SIEM use cases (using a particular SIEM as an example)
• Log Management / SIEM Users: "Minimalist" vs "Analyst"
• All posts labeled SIEM

About me: http://www.chuvakin.org

My account of RSA 2011 cannot be complete without-  yes! - SecurityBSides San Francisco. I was holding this post hoping to include links to videos, but – despite the power of Google – I was not able to figure out where AND whether the video are posted.  So, you have to enjoy my new fun SIEM presentation (below) without my voice and an image of me pointing at the sky

Enjoy!

Possibly related notes:

• RSA 2011 Conference Notes
• RSA 2011 PCI Council Interview
• All posts tagged RSA

About me: http://www.chuvakin.org

One of the ugliest, painfulest, saddest issues with SIEM is resourcing. Yes, that SIEM appliance might set us back $75,000 in hard earned security budget dollars, but how much more will we have to spend in the next 3 years deploying, integrating, using, tuning, cursing, expanding the thing? How much manpower will the new operational procedures (example) cost us? And if we actually build a SOC or “a virtual SOC”, how much will we have to spend on an ongoing basis to get the value and benefits? In fact, how much will the coffee cost if we have to work 20 hours in a row recovering that crashed SIEM database partition?

These and other questions are super-important for every SIEM and log management project. And the time has come for me to reveal my secret knowledge of SIEM resourcing. OK, that’s a joke – it is not a secret, just a bunch of things that are often unpleasant for many SIEM buyers, users and sellers to hear.

So:

NEWSFLASH! SIEM costs money. “Free” SIEM (example) costs money too, BTW. Let’s try to delve into what those costs are. I will be not-quite-scientific in regards to real “hard money” costs (e.g. software license purchasing) and “soft costs” costs (e.g. staff time costs), but I will try to clearly mark each kind of SIEM cost below.

First, assumptions and limitations:

• This is NOT “SOC staffing” , but simply “running a SIEM” staffing. SOC implies more processes and more tasks and a broader mission.
• Assumes in-sourced, traditional “buy and run” SIEM; outsourced, co-managed/co-sourced cost model would look different.

Here is the rough cost model for some of the most common SIEM cost categories:

1. Initial “hard” costs 
   1. SIEM license costs: base price +per user, per node, per EPS, per CPU (and per CPU core), etc costs – however your favorite vendor charges
   2. For software SIEM, also hardware, OS, database costs for as many servers as you need
   3. If any, mandatory 3rd party software license costs (occasionally, agents, reporting tools, etc)
   4. If chosen, vendor or consultant deployment services costs. If not chosen, staff time for deployment will pop out in soft costs below!
   5. Vendor training or 3rd party training on logs, log management,  SIEM, etc
   6. Additional external storage (in most cases)
2. SIEM ongoing, operating “hard” costs
   1. Various SIEM vendor services: support (typically mandatory), ongoing professional services costs
   2. Personnel to operate a SIEM: from part of FTE (very small scale, few use cases for a SIEM) to 1 FTE (small appliance deployments) to many FTEs of various roles (+much more for SOC staffing if live monitoring is implemented). 0 FTE for SIEM = SIEM project FAIL with 100.00% probability.
3. Periodic or occasional “hard” costs
   1. Various SIEM vendor services: professional services, custom development work for device integration (some of these may go into soft costs if done internally – for advanced organization or those experienced with SIEM already)
   2. Periodic staff training on SIEM operation and tuning
   3. Specialty staffing: DBA, sysadmins, node sysadmins, in-house developers for custom connectors, Crystal Reports administrator (yuck!), etc – some of these might go into “soft” costs if “poaching” existing personnel time
   4. Deployment expansion costs: same as initial costs, but for additional systems, software, hardware, etc as you grow; these sneak up really fast if SIEM is licensed using many dimensions such as user+CPU+node+server+something else.
   5. External storage expansion costs – yes, you will buy more storage if your volume grows, and log retention time stays the same (e.g. 1 year for PCI DSS)

On the other hand, here are some of the “soft” costs, such as time expenditures by existing resources:

1. Initial “soft” costs 
   1. Deployment time for the SIEM project – allocate more time if deploying purely using internal personnel, not vendor or consultant
   2. Log source configuration and integration – this will likely take way more than simply installing the tool. This is what makes SIEM deployment projects go for months in complex, distributed organizations with many silos.
   3. Initial tuning, content creation  and adapting the tool to your environment  (however lightweight it may be)
   4. Training and other staff time costs to jumpstart the operation (Congratulations! You bought ta SIEM. Now you need to operate it…)
2. SIEM ongoing, operating “soft” costs
   1. Report review and other ongoing monitoring tasks – from 24/7 to daily to weekly
   2. Alert response and escalation; SIEM implies correlation and automated alerting
   3. Other “using SIEM” tasks such as reviewing the dashboards
   4. Uptime maintenance tasks i.e. caring for your SIEM as well as storage – backups, updates, minor troubleshooting, etc
3. Periodic or occasional “soft” costs
   1. SIEM rule tuning, reports creation, dashboard customization, new log source integration, other ongoing SIEM tasks
   2. Periodic training and related staff time costs
   3. Expansion: same as initial soft costs

As was suggested by a discussion on SIEMusers.org (shhh…the site is not ready for launch yet), it is useful to separate soft costs  that are mandatory FOR SIEM operation from those which commonly arise FROM SIEM operation. The most obvious example is incident response due to increased awareness of network and system activities, delivered by your SIEM.

1. Added cost of incident response: more incidents are likely to be detected
2. Resulting incident remediation costs and even cost of new technologies deployed for preventing the discovered issues
3. Other department personnel time for dealing with issues revealed by SIEM – the soft costs do leak out of the security department to IT and even beyond (legal, HR, etc).

Anything big I missed that you experienced? BTW, in my experience, I have seen the total cost of a SIEM project (hard + soft) range from 10% of SIEM license costs (for shelfware SIEM “deployments”) to a mind-boggling 20x of license cost.

P.S. Finally, if you want to really annoy Anton, mention “SIEM ROI.” If you do that, I will send you to Gal Shpantser for a mandatory “why he avoids SIEM!” class

Possibly related posts:

• Log Management Tool Selection Checklist Out!
• "So, What Should I Want?" or How NOT to Pick a SIEM-III?
• On Choosing SIEM
• I Want to Buy Correlation” or How NOT to Pick a SIEM?
• The Myth of SIEM as "An Analyst-in-the-box" or How NOT to Pick a SIEM-II?
• Logging, Log Management and Log Review Maturity
• Log Management + SIEM = ?
• On SIEM Complexity
• SIEM Bloggables: SIEM Use Cases
• Whitepaper with detailed SIEM use cases (using a particular SIEM as an example)
• Log Management / SIEM Users: "Minimalist" vs "Analyst"
• All posts labeled SIEM

About me: http://www.chuvakin.org

1. We plan for this to happen periodically, such as maybe every three weeks - recorded on Wednesday, posted on Thursday. However, due to our work schedules, irregularities occur all the time. If you have not seen or heard a new LogChat podcast for a few weeks, be aware that we are not dead; just busy taking over the world.
2. No, we are still not ready with transcribing and, yes, we still want it.  I did try Amazon Mechanical Turk, but it didn't turn to be as inexpensive as people claimed. If you have ideas for a good AND cheap transcribing service, we are all ears.
3. Please suggest topics to cover as well - even though we are not likely to run out of ideas for a few years.
4. Any other feedback is HUGELY useful. Is it too long? Too loud? Too rant-y? Too technical? Not enough jokes? Too few mentions of the "cloud"? Feedback please!

• LogChat Podcast 1: Anton Chuvakin and Andrew Hay Talk Logs
• LogChat Podcast 2: Anton Chuvakin and Andrew Hay Talk Logs
• LogChat Podcast 3: Anton Chuvakin and Raffy Marty Talk Logs
• LogChat Podcast 4: Anton Chuvakin and Andrew Hay Talk Logs

About me: http://www.chuvakin.org

Once upon a time, I was retained to create a comprehensive PCI DSS-focused log review policies, procedures and practices for a large company.  They are focused on PCI DSS compliance, but based on generally useful log review practices that can be utilized by everybody and with any regulation or without any compliance flavor at all. The practices can be implemented with commercial log management or SIEM tools, open source log analysis tools or manually. As you well know, tools alone don’t make anybody compliant!

This is the FINAL, 18th post in the long, long series  (part 1, part 2, part 3 – all parts). A few tips on how you can use it in your organization can be found in Part 1. You can also retain me to customize or adapt it to your needs.

And so we end our Complete PCI DSS Log Review Procedures. Please start reading from Part 1 – at this stage we are deep in the details and these sections might seem out of context without reading earlier parts:

References

The following references are useful for PCI DSS log review program and log management in general:

SANS CAG/CSC

“Twenty Critical Security Controls for Effective Cyber Defense: Consensus Audit Guidelines”

http://www.sans.org/critical-security-controls/

Specifically, the relevant control on audit logs is shown below:

“Critical Control 6: Maintenance, Monitoring, and Analysis of Audit Logs”

NIST 800-92 Logging Guide

“Guide to Computer Security Log Management: Recommendations of the National Institute of Standards and Technology by Karen Kent and Murugiah Souppaya”

http://csrc.nist.gov/publications/nistpubs/800-92/SP800-92.pdf

NIST 800-66 HIPAA Guide

“An Introductory Resource Guide for Implementing the Health Insurance Portability and Accountability Act (HIPAA) Security Rule ”

http://csrc.nist.gov/publications/nistpubs/800-66-Rev1/SP-800-66-Revision1.pdf

Appendix A Recommended Logbook Format

Logbook entry:

1. Date/time/time zone this logbook entry was started

2. Name and role of the person starting the logbook entry

3. Reason it is started: log exception (copied from log aggregation tool or from the original log file), make sure that the entire log is copied, especially its time stamp (which is likely to be different from the time of this record) and system from which it came from (what/when/where, etc)

4. Detailed on why the log is not routine and why this analysis is undertaken

5. Information about the system that produced the exception log record or the one this log exception is about

a. Hostname

b. OS

c. Application name

d. IP address(s)

e. Location

f. Ownership (if known)

g. System criticality (if defined and applicable)

h. Under patch management, change management, FIM, etc

6. Information about the user whose activity produced the log (if applicable)

7. Investigation procedure followed, tools used, screenshots, etc

8. Investigative actions taken

9. People contacted in the course of the log analysis

10. Impact determine during the course of the analysis

11. Recommendations for actions, mitigations (if needed)

Follow PCI_Log_Review to see all posts.

Possibly related posts:

• Incident Log Review Checklist
• All posts tagged PCI_Log_Review

About me: http://www.chuvakin.org

FYI, this piece has been specially created for LogManagementCentral (original post), an awesome resource for all logging things. It is reposted here for posterity.

While many people have seen 10 things that your chef, real-estate agent, wedding planner or pilot won’t tell you, the world has not yet seen Top 10 things your log management vendor won't tell you. Finally, this gap is now closed.

1. “We talk analytics, but really, most of our customers use us for collection only.” While some products within SIEM and log management offer advanced analytics features, many of their customers are not truly ready for them. They need to start dealing with the basics – logging, log collection, log review before delving into advanced areas. Buying a product based on features you won’t use is a mistake. For example, see “Log Management Before SIEM”

2. “Our tool won’t make you PCI compliant. You’d have to do A LOT of things yourself – every day – to get and maintain compliance.” Sadly, many security solutions – and SIEM / log management are no exception – are sometimes sold as “compliance in a box.” You need to be aware that to stay PCI compliance you need to do more than purchase tools. For example, see “How to Stay PCI Compliant”

3. “No, you cannot buy an entire SOC in this small box.” Just as with compliance, you cannot buy an entire Security Operations Center in a box, big or small. However, some will try to sell you their SIEM as “SOC-in-a-box.” Running an effective SOC includes multiple processes and procedures which are just as necessary as a market-leading SIEM tool

4. “We are cloud-ready, because … mmmmm… well, we are ready for it!” Many vendors will tell you that their tools are cloud-ready – without really thinking what they mean. Effectively monitoring traditional and multi-tenant cloud environments distributed across regions and countries requires more than updated marketing materials. You need to carefully test the tool in your own hybrid environment before concluding that it is “cloud ready”

5. “Our SIEM is really just a renamed log management tool. But that’s all you probably need.” The confusion around SIEM and log management functionality rages on – it also allows some tools to be sold as SIEM without having any critical SIEM functionality such as correlation and real-time dashboards. Even though it might be all many customers need, it does not make such tool a SIEM tool. For additional reading, see this whitepaper.

6. “We can do everything with logs, but it might require some SMALL customizations. Our PS team is standing by!” More than a few SIEM vendors will promise support for every possible log – including logs they have never seen. However, fully integrating a new log source for reporting, correlation and visualization will always takes work and cannot be taken for granted.

7. “If you make a mistake with capacity planning, we’d be happy to sell you more log management than you really need.” Many organizations are having trouble estimating how much log data will be coming into their SIEM or log management tools. Both under as to making and overestimating are common. It is recommended that you spend about a week measuring log volumes across the systems that will be reporting to a SIEM.

8. “We think our tool is scalable, but we don’t really have production customers of your size. Our engineers believe that it might work.” Scalability claims are cheap and would often be made by SIEM and log management vendors. However, the only real proof that the tool will scale to your requirements is testing the tool in your environment. Thus, you should insist on performance testing during the pilot if there are any doubts.

9. “Out tool offers predictive security intelligence. No, we don’t know what it means either – and we can’t really predict it.” SIEM is one of the most over-hyped and over-marketed security technologies out there. The only way to get the tool that satisfies your requirements is too carefully spelled out those requirements and then test the tool yourself.

10. “We estimate our performance using really small log messages sizes.” Yes, our tools can do a million message an instant – but these are our special messages that we create in the lab. Nowadays, application logs and proliferation of XML-based logging has pushed the message sizes up to 1 kb or more from a traditional 200 byte logs from firewalls. Thus, you need to be wary of performance estimates based on such artificially short logs.

So what is your vendor NOT tellin’ ya?

About me: http://www.chuvakin.org

Once upon a time, I was retained to create a comprehensive PCI DSS-focused log review policies, procedures and practices for a large company.  They are focused on PCI DSS compliance, but based on generally useful log review practices that can be utilized by everybody and with any regulation or without any compliance flavor at all. The practices can be implemented with commercial log management or SIEM tools, open source log analysis tools or manually. As you undoubtfully know, tools alone don’t make anybody compliant!

This is the 17th, one before last,  post in the long series of 18 posts (part 1, part 2, part 3 – all parts) – this is a very important part as it contains the summary of key periodic operational procedures. Please consider reading from Part 1 – at this stage we are deep in the details and these sections might seem out of context without reading earlier parts. A few tips on how you can use it in your organization can be found in Part 1. You can also retain me to customize or adapt it to your needs.

And so we continue with our Complete PCI DSS Log Review Procedures:

Periodic Operational Task Summary

The following chapter contains a summary of operational tasks related to logging and log review. Some of the tasks are described in detail in the document above; others are auxiliary tasks needed for successful implementation of PCI DSS log review program.

Daily Tasks

The table below contains daily tasks, responsible role that performs them as well as what record or evidence is created of their execution:

Task	Responsible Role	Evidence
Review all the types of logs produced over the last day as described in the daily log review procedures	Security administrator, security analyst, (if authorized) application administrator	Record of reports being run on a log management tool
(As needed) investigate the anomalous log entries as described in the investigative procedures	Security administrator, security analyst, (if authorized) application administrator	Recorded logbook entries for investigated events
(As needed) take actions as needed to mitigate, remediate or reconcile the results of the investigations	Security administrator, security analyst, (if authorized) application administrator, other parties	Recorded logbook entries for investigated events and taken actions
Verify that logging is taking place across all in-scope applications	Application administrator	Create a spreadsheet to record such activities for future assessment
(As needed) enabled logging if disabled or stopped	Application administrator	Create a spreadsheet to record such activities for future assessment

Weekly Tasks

The table below contains weekly tasks, responsible role that performs them well as what record or evidence is created of their execution:

Task	Responsible Party	Evidence
(If approved by a QSA) Review all the types of logs produced on less critical application over the last day as described in the daily log review procedures	Security administrator, security analyst, (if authorized) application administrator	· Record of reports being run on a log management tool. · Record of QSA approval for less frequent log reviews and reasons for such approval
(As needed) investigate the anomalous log entries as described in the investigative procedures	Security administrator, security analyst, (if authorized) application administrator	Recorded logbook entries for investigated events
(As needed) take actions as needed to mitigate, remediate or reconcile the results of the investigations	Security administrator, security analyst, (if authorized) application administrator, other parties	Recorded logbook entries for investigated events and taken actions

Monthly Tasks

The table below contains daily tasks, responsible role that performs them as well as what record or evidence is created of their execution:

Task	Responsible Party	Evidence
Prepare a report on investigated log entries	Security analyst, security manager	Prepared report (to be filed)
Report on observed log message types	Security analyst, security manager	Prepared report (to be filed)
Report on observed NEW log message types	Security analyst, security manager	Prepared report (to be filed)
(If approved by a QSA) Review all the types of logs produced on non-critical applications over the last day as described in the daily log review procedures	Security administrator, security analyst, (if authorized) application administrator	· Record of reports being run on a log management tool. · Record of QSA approval for less frequent log reviews and reasons for such approval
(As needed) investigate the anomalous log entries as described in the investigative procedures	Security administrator, security analyst, (if authorized) application administrator	Recorded logbook entries for investigated events
(As needed) take actions as needed to mitigate, remediate or reconcile the results of the investigations	Security administrator, security analyst, (if authorized) application administrator, other parties	Recorded logbook entries for investigated events and taken actions

Quarterly Tasks

The table below contains daily tasks, who performs them as well as what record or evidence is created of their execution:

Task	Responsible Party	Evidence
Verify that all the system in scope for PCI are logging and that logs are being reviewed	Security analyst, security manager	Recorded logbook entries for review and exception follow-up
Review daily log review procedures	Security analyst, security manager	Updates to logging procedures; change log
Review log investigation procedures	Security analyst, security manager	Updates to logging procedures; change log
Review collected compliance evidence	Security analyst, security manager	Compliance evidence; evidence review log
Review compliance evidence collection procedures	Security analyst, security manager	Updates to procedures; change log

Annual Tasks

The table below contains daily tasks, who performs them as well as what record or evidence is created of their execution:

Task	Responsible Party	Evidence
Review logging and log review policy	CSO	Policy changes; change log; policy review meeting minutes
Review compliance evidence before the QSA assessment	PCI DSS compliance project owner	Meeting minutes or other record
Live tests with anomalies	As needed	Logs or other records of such tests

To be continued.

Follow PCI_Log_Review to see all posts.

Possibly related posts:

• Incident Log Review Checklist
• All posts tagged PCI_Log_Review

About me: http://www.chuvakin.org

Management Reporting

• Incident Log Review Checklist
• All posts tagged PCI_Log_Review

About me: http://www.chuvakin.org

As a favor to yet another friend, I am posting yet another SIEM-related job. IMHO, it is an ideal position for a good architect looking to jump ship from a failing or “non-performing” SIEM vendor:

The RSA Security’s fast-growing  Security Management group is looking for the best technical minds to develop the next generation of Security Information and Event Management (SIEM) software.  We are building a great organization with talented employees with the highest ethical and professional standards who deliver a portfolio of products to enable our customers to protect their information assets.

Ideal candidate will have broad knowledge of IT security with proven ability to architect and build complex enterprise systems.  You must enjoy working in a rapidly-changing, high-pressure environment spanning multiple geo locations. As a lead architect, you will exert significant influence over the technical strategy and the architectural definition of the next generation of RSA’s Security Management products.  Practical experience in one of the following areas is required: large-scale database systems, real-time design, network monitoring and analysis.

This position is full-time, based in Bedford, MA. If you are interested in joining the Security Management group in RSA, please send your inquiry or resume to Lauren Day at  lauren.day@emc.com or 978-686-2234.

… and somebody now owes me beer at RSA 

Possibly related posts:

• SIEM-related Job: Principal SIEM Consultant 
• SIEM-related Product Management Job: Atlanta, GA
• All jobs posts on my blog

About me: http://www.chuvakin.org

Once upon a time, I was retained to create a comprehensive PCI DSS-focused log review policies, procedures and practices for a large company.  They are focused on PCI DSS compliance, but based on generally useful log review practices that can be utilized by everybody and with any regulation or without any compliance flavor at all. The practices can be implemented with commercial log management or SIEM tools, open source log analysis tools or manually. As you undoubtfully know, tools alone don’t make anybody compliant!

This is the 15th post in the long, long series  (part 1, part 2, part 3 – all parts). A few tips on how you can use it in your organization can be found in Part 1. You can also retain me to customize or adapt it to your needs.

And so we continue with our Complete PCI DSS Log Review Procedures (please consider reading from Part 1 – at this stage we are deep in the details and these sections might seem out of context without reading earlier parts):

PCI Compliance Evidence Package

Finally, it is useful to create a “PCI Compliance Evidence Package” based on the established and implemented procedures to show it to the QSA. It will help establish your compliance with three key of PCI DSS logging requirements:

· Presence and adequacy of logging

· Log review

· Exception handling

While it is possible to prepare the evidence package before the assessment, it is much easier to maintain it on the ongoing basis. For example, keep printed or electronic copies of the following:

1. Logging policy that covers all of the PCI DSS in-scope systems

2. Logging and log review procedures (this document)

3. List of log sources – all systems and their components (applications) from the in-scope environment

4. Sampling of configuration files that indicate that logging is configured according to the policy (e.g. /etc/syslog.conf for Unix, screenshots of audit policy for Windows, etc)

5. Sampling of logs from in-scope systems that indicate that logs are being generated according to the policy and satisfy PCI DSS logging requirements

6. Exported or printed report from a log management tools that shows that log reviews are taking place

7. Up-to-date logbook defined above

This will allow always establishing compliant status and proving ongoing compliance.

To be continued.

Follow PCI_Log_Review to see all posts.

Possibly related posts:

• Incident Log Review Checklist
• All posts tagged PCI_Log_Review

About me: http://www.chuvakin.org

Once upon a time, I was retained to create a comprehensive PCI DSS-focused log review policies, procedures and practices for a large company.  They are focused on PCI DSS compliance, but based on generally useful log review practices that can be utilized by everybody and with any regulation or without any compliance flavor at all. The practices can be implemented with commercial log management or SIEM tools, open source log analysis tools or manually. As you undoubtfully know, tools alone cannot and do not make anybody compliant!

This is the 14th post in the long, long series  (part 1, part 2, part 3 – all parts). A few tips on how you can use it in your organization can be found in Part 1. You can also retain me to customize or adapt it to your needs.

And so we continue with our Complete PCI DSS Log Review Procedures (please consider reading from Part 1 – at this stage we are deep in the details and these sections might seem out of context without reading earlier parts):

Example Logbook Entry

Here is an example following the above pattern:

1. Date/time/time zone this logbook entry was started: November 23, 2009, 4:15PM PST

2. Name and role of the person starting the logbook entry: Anton Chuvakin, principal consultant.

3. Reason the logbook entry is started: log exception (copied from log aggregation tool or from the original log file), make sure that the entire log is copied, especially its time stamp (which is likely to be different from the time of this record) and system from which it came from (what/when/where, etc):

Time/date of log: 10/21/2009 10:01:23 PM PST

System: OLGA.example.com

4. Detailed on why the log is not routine and why this analysis is undertaken: this event ID (Windows event ID 11) from this application event source (Source crypt32) was never seen before on any of the systems where logs are reviewed across our organization.

5. Information about the system that produced the exception log record or the one this log exception is about

a. Hostname: OLGA.example.com

b. OS: Windows XP SP 3

c. Application name: N/A

d. IP address(s): 10.1.1.1

e. Location: Home office

f. Ownership (if known): Olga Chuvakin, President and CEO

g. System criticality (if defined and applicable): critical, main laptop of the executive

h. Under patch management, change management, FIM, etc: yes

6. Information about the user whose activity produced the log: N/A, no user activity involved

7. Investigation procedure followed, tools used, screenshots, etc: procedure for “Initial Investigation” described above

8. Investigative actions taken: following the procedure for “Initial Investigation” described above, it was determined that this log entry is followed by a successful completion of the action logged. Specifically, on the same day, 1 second later the following log entry appeared:

This entry indicates the successful completion of the action referenced in our exception log entry and thus no adverse impact from the error/failure is present.

9. People contacted in the course of the log analysis: none

10. Impact determine during the course of the analysis: impact was determined to be low to non-existent; no functionality was adversely affected, no system was at risk.

11. Recommendations for actions, mitigations (if needed): no mitigation needed, added this log entry to baseline to be ignored in the future as long as the subsequent log entry exists.

The logbook of that sort is used as compliance evidence since it establishes log exceptions follow-up, required in item 10.6.a of PCI DSS validation procedure, which states “Obtain and examine security policies and procedures to verify that they include procedures to review security logs at least daily and that follow-up to exceptions is required.”

The logbook (whether in electronic or paper form) can be presented to a QSA or other auditor, if requested. I recommend retaining the log book for 3 years or at least 2x of the log retention period (1 year for PCI DSS)

To be continued.

Follow PCI_Log_Review to see all posts.

Possibly related posts:

• Incident Log Review Checklist
• All posts tagged PCI_Log_Review

About me: http://www.chuvakin.org

Once upon a time, I was retained to create a comprehensive PCI DSS-focused log review policies, procedures and practices for a large company.  They are focused on PCI DSS compliance, but based on generally useful log review practices that can be utilized by everybody and with any regulation or without any compliance flavor at all. The practices can be implemented with commercial log management or SIEM tools, open source log analysis tools or manually. As you undoubtfully know, tools alone don’t make anybody compliant!

This is the 13th post in the long, long series  (part 1, part 2, part 3 – all parts). A few tips on how you can use it in your organization can be found in Part 1. You can also retain me to customize or adapt it to your needs.

And so we continue with our Complete PCI DSS Log Review Procedures (please consider reading from Part 1 – at this stage we are deep in the details and these sections might seem out of context without reading earlier parts):

Logbook – Evidence of Exception of Investigations

How to create a logbook that proves that you are reviewing logs and following up with exception analysis, as prescribed by PCI DSS Requirement 10? The logbook is used to document everything related to analyzing and investigating the exceptions flagged during daily review. While the same logbook approach is used in the incident handling process (such as SANS Incident Response Workflow), in this document it is utilized as compliance evidence.

The logbook should record all systems involved, all people interviewed, all actions taken as well as their justifications, what outcome resulted, what tools and commands were used (with their results), etc.

Here is one recommendation for a logbook entry:

Recommended Logbook Format

Logbook entry:

1. Date/time/time zone this logbook entry was started

2. Name and role of the person starting the logbook entry

3. Reason it is started: log exception (copied from log aggregation tool or from the original log file), make sure that the entire log is copied, especially its time stamp (which is likely to be different from the time of this record) and system from which it came from (what/when/where, etc)

4. Detailed on why the log is not routine and why this analysis is undertaken

5. Information about the system that produced the exception log record or the one this log exception is about

a. Hostname

b. OS

c. Application name

d. IP address(s)

e. Location

f. Ownership (if known)

g. System criticality (if defined and applicable)

h. Under patch management, change management, FIM, etc.

6. Information about the user whose activity produced the log (if applicable)

7. Investigation procedure followed, tools used, screenshots, etc

8. Investigative actions taken

9. People contacted in the course of the log analysis

10. Impact determine during the course of the analysis

11. Recommendations for actions, mitigations (if needed)

To be continued.

Follow PCI_Log_Review to see all posts.

Possibly related posts:

• Incident Log Review Checklist
• All posts tagged PCI_Log_Review

About me: http://www.chuvakin.org

Validation of Log Review

Proof of Logging

Proof of Log Review

Proof of Exception Handling

• Incident Log Review Checklist
• All posts tagged PCI_Log_Review

About me: http://www.chuvakin.org

Once upon a time, I was retained to create a comprehensive PCI DSS-focused log review policies, procedures and practices for a large company.  It is focused on PCI DSS, but based on generally useful log review practices that can be utilized by everybody and with any regulation or without any compliance flavor at all.

This is the 11th post in the long, long series (part 1, part 2, part 3 – all parts). A few tips on how you can use it in your organization can be found in Part 1. You can also retain me to customize or adapt it to your needs.

And so we continue with our Complete PCI DSS Log Review Procedures (please read in order- at this point we are pretty deep in the details and this piece might look out of context):

External Information Sources Investigation

Here is the procedure to follow in this case:

This procedure can be expanded to cover other sources of information available at the organization.

The main idea of this procedure it to identify and then query information sources (such as IdM, change management, integrity checking, network flow analysis, etc), based on the type of the exception log entry and then to identify its impact and the required actions (if any)

The procedure works to roughly identify the type of a log entry and then to query the relevant information sources. In some cases, then the log entry is deemed to be an indication of a serious issue, incident response process is triggered.

However, it sometimes happens that neither the preliminary analysis nor the query of external systems yields the results and the “exception” log entry is exceptional. In this case, the collaborative workflow is triggered. See the next section for details

Escalation to Others Procedure – Collaborative Workflow

The investigation and escalation process is shown below:

This process allows tapping into the knowledge of other people at the organization who might know what this “anomaly” is about.

The main idea of this procedure it to identify and then interview the correct people who might have knowledge about the events taking place on the application then to identify its impact and the required actions (if any).

The very last resource is to query the application vendor; such info request is typically time consuming or even expensive (depends on the support contract available) so it should be used sparingly.

To be continued.

Follow PCI_Log_Review to see all posts.

Possibly related posts:

• Incident Log Review Checklist
• All posts tagged PCI_Log_Review

About me: http://www.chuvakin.org

Once upon a time, I was retained to create a comprehensive PCI DSS-focused log review policies, procedures and practices for a large company.  It is focused on PCI DSS, but based on generally useful log review practices that can be utilized by everybody and with any regulation or without any compliance flavor at all.

This is the tenth post in the long, long series (part 1, part 2, part 3 – all parts). A few tips on how you can use it in your organization can be found in Part 1. You can also retain me to customize or adapt it to your needs.

And so we continue with our Complete PCI DSS Log Review Procedures:

Exception Investigation and Analysis

A message not fitting the profile of a normal is flagged “an exception.” It is important to note that an exception is not the same as a security incident, but it might be an early indication that one is taking place.

At this stage we have an individual log message that is outside of routine/normal operation. How do we figure out whether it is significant, determine impact on security and PCI compliance status?

Initial Investigation

The following high-level investigative process (“Initial Investigation”) is used on each “exception” entry (more details are added further in the document):

Specifically, the above process makes use of a log investigative checklist, which is explained below in more details.

1. Look at log entries at the same time: this technique involves looking at an increasing range of time periods around the log message that is being investigated. Most log management products can allow you to review logs or to search for all logs within a specific time frame. For example:

a. First, look at other log messages triggered 1 minute before and 1 minute after the “suspicious” log

b. Second, look at other log messages triggered 10 minute before and 10 minute after the “suspicious” log

c. Third, look at other log messages triggered 1 hour before and 1 hour after the “suspicious” log

2. Look at other entries from same user: this technique includes looking for other log entries produced by the activities of the same user. It often happens that a particular logged event of a user activity can only be interpreted in the context of other activities of the same user. Most log management products can allow you to “drill down into” or search for a specific user within a specific time frame.

3. Look at the same type of entry on other systems: this method covers looking for other log messages of the same type, but on different systems in order to determine its impact. Learning when the same message was products on other system may hold clues to understanding the impact of this log message.

4. Look at entries from same source (if applicable): this method involves reviewing all other log messages from the network source address (where relevant).

5. Look at entries from same app module (if applicable): this method involves reviewing all other log messages from the same application module or components. While other messages in the same time frame (see item 1. above) may be significant, reviewing all recent logs from the same components typically helps to reveal what is going on.

In some cases, the above checklist will not render the result. Namely, the exception log entry will remain of unknown impact to security and PCI DSS compliance. In this case, we need to acquire information from other systems, such as File Integrity Monitoring, Vulnerability Management, Anti-malware, Patch Management, Identity Management, Network Management and others.

To be continued.

Follow PCI_Log_Review to see all posts.

Possibly related posts:

• Incident Log Review Checklist
• All posts tagged PCI_Log_Review

About me: http://www.chuvakin.org