1. We need a new name! We are not entirely happy with "LogChat" and, sadly, "LogTalk" is taken. Please suggest a name - if we pick yours, you get a free signed  copy of my "PCI Compliance" book.
2. We will post the transcript, not just the MP3 file - in a few days. If you have ideas for a good/inexpensive transcribing service, we are all ears. I will try Amazon Mechanical Turk first, but it might not be good enough for a technical podcast.
3. Please also suggest topics to cover as well - even though we are not likely to run out of ideas for a few years. Our first topic today is new log source integration - if it sounds boring...well...listen first/judge second :-)
4. We plan for this to be a monthly podcast. So, the next one will happen sometime early October.
5. Any other feedback is HUGELY useful. Is it too long? Too loud? Not enough jokes? Too few mentions of the "cloud"? Feedback please! Who knows...maybe there are more PCI books left in my secret stash and you too will earn that glorious prize for the most useful piece of feedback  :-)

About me: http://www.chuvakin.org

As promised, here is another detailed SIEM whitepaper called “A Pragmatic Approach to SIEM: Buy for Compliance, Use for Security” that I wrote for a great team at Tripwire earlier this year.

“While recent economic troubles might have something to do with it, many organizations today seek to only do a bare minimum of security. To be more precise, they try to do what they think is the bare necessary minimum. Their perception that security “due diligence” can be reduced all the way down to the level prescribed by regulations, such as PCI DSS, is more common than ever today. All too common result of this thinking is security breaches and other damaging events.

This trend has affected many security safeguards, and SIEM and log management are hard hit by this as well. It is very common to deploy these technologies in order to satisfy the compliance check box. In this paper we will analyze this trend and provide useful guidance for getting value out of SIEM and log management tools while focusing on protecting systems and data – and not simply on checking the box.”

Get the paper here.

Possible related posts:

• “New SIEM Whitepaper on Use Cases In-Depth OUT!”
• “Two New Logging Resources Published”

About me: http://www.chuvakin.org

The future of logging is finally here! Common Event Expression (CEE) team releases CEE Architecture Overview [PDF] for public comments. HUGE thanks to MITRE side of team for finally clearing all the hurdles and releasing “our baby.”

The Common Event Expression (CEE™) Architecture Overview document defines the structure and components that comprise the CEE event log standard. This architecture was developed by MITRE, in collaboration with industry and government, and builds upon the Common Event Expression Whitepaper. This document defines the CEE Architecture for an open, practical, and industry-accepted event log standard. It provides a high-level overview of CEE along with details on the overall architecture and introduces each of the CEE components including the data dictionary, event taxonomies, syntax encodings, and profiles. The CEE Architecture is the first in a collection of documents and specifications, whose combination provides the necessary pieces to create the complete CEE event log standard. 

We encourage community members to offer feedback on this document on the CEE
Email Discussion list. You may also contact us directly at cee@mitre.org.

Again, the document is at: http://cee.mitre.org/docs/CEE_Architecture_Overview-v0.5.pdf

The day we were working towards for nearly five years (!) has finally come and more of CEE is revealed to the world! Of course, detailed specifications are still in development and we will release them when they are ready for public review.

Possibly related posts:

• All posts tagged CEE
• My first post on CEE in April 2007

About me: http://www.chuvakin.org

Reposted (from here) for those who don’t monitor public Common Event Expression (CEE) log standard effort.

“First of all, let me answer the question that is in all of your heads: No, CEE is not dead... After a slow start, the work pace has picked up over the past couple of months. I apologize for the extremely long delay in posting any details regarding the state of CEE.

The CEE Board [that’s us – A.C. :-)] has been working on bringing the CEE Dictionary, Taxonomy, and Syntax requirements together. We have drafted specification documents detailing the CEE Architecture as well as the initial Dictionary and Taxonomy specifications. These documents are the first of the CEE v0.5 (previous versions were internal) document series and have already been shared with some of you for your feedback. We are waiting for the final authorization before they are posted to the CEE website for your review.

This authorization is expected to be granted next week. A follow-up e-mail will be sent to this discussion list as well as the CEE Announce list notifying you that the documents were posted along with the URL where said documents may be obtained. It is important to note that these are only the first iterations of many. We need your help in improving CEE.
 

In addition to helping critique and improve the CEE documents, we are also requesting your help in building and improving the CEE Use Cases. Within the next week or two, you will also receive the beginnings of a list of CEE Use Cases for review.”

Sign up for the public CEE list to see more. Meanwhile, the effort for log standardization makes another step!

About me: http://www.chuvakin.org

Comprehensive firewall monitoring
(security + network)
Since the early days of SIEM technology, firewall log data
has been considered as one of the most useful and
commonly collected information sources.
Apart from allowing and denying connections to and from
the network, firewalls allow recording or logging of every
single connection denied or allowed by the firewall. An
example would be connections from the outside world to
the DMZ Web server, or connections by users inside the
company to their favorite social media Web site.
Analysis of such logs is extremely useful for security,
compliance and even operational purposes such as
network management, bandwidth management, etc.
For example, on the compliance side, PCI DSS, HIPAA,
NERC/FERC all have firewall logging implications. Firewall
logs are also extremely useful for incident response and
forensics since they can help identify the connectivity
pattern and serve as “poor man netflow.” On top of this,
firewall logs can be used to assess the health of the
firewall itself and to optimize the rule set performance.

Collection: comprehensive firewall log collection is
mandatory for this use case, and it is important to
remember that firewalls can record both failed and
successful connections through the firewall – both
types are essential for SIEM.
…

• Two New Logging Resources Published
• How Do I Get The Best SIEM? 
• On Choosing SIEM
• One More Time on SIEM vs Log Management

About me: http://www.chuvakin.org

Thanks for overwhelming community response (here, here, here, and separate blog posts here and here and I might have missed a few places too). The list has grown and is on the verge of becoming unwieldy and not “top” and “essential” so I am about to close the comment period, write up the doc and send it to SANS to update the legacy SANS Top 5 Log Reports [PDF].

Any last second thoughts before I document this baby? Any smokin’ hot log reports to add?!  Also, anything I should take OFF the list for not being “top” and “essential”?

1. Authentication and Authorization Reports

a. All login failures and successes by user, system, business unit – must have login success logs, not just failure!

b. Login attempts (successes, failures) to disabled/service/non-existing/default/suspended accounts

c. All logins after office hours / “off” hours

d. Users failing to authentication by count of unique systems they tried

e. VPN authentication and other remote access logins (success, failure)

f. Privileged account access: logins, su use, Run As use, etc (success, failure)

g. Multiple login failures followed by success by same account – needs to have correlation for that

2. Change Reports

a. Additions/changes/deletions to users, groups – even a trend on user additions across systems would be useful

b. Additions of accounts to administrator / privileged groups

c. Password changes and resets – by users and by admins to users

d. Additions/changes/deletions to network services

e. Changes to system files – binaries, configurations – likely needs a list to run

g. Changes in file access permissions

h. Application installs and updates (success, failure) by system, application, user

3. Network Activity Reports

a. Log volume trend over days – watch for both drops and increases in logging levels on systems

b. All outbound connections from internal and DMZ systems by system, connection count, user, bandwidth, count of unique destinations, hour of access (focus on “off hours”)

c. Top largest file transfers (inbound, outbound) OR Top largest sessions by bytes transferred

d. Web file uploads to external sites - based on proxy logs

e. All file downloads by content type (exe, dll, scr, upx, etc) and protocol (HTTP, IM, etc)

f. Internal systems using many different protocols/ports

g. Top internal systems as sources of multiple types of NIDS, NIPS or WAF Alerts

h. VPN network activity by user name, total session bytes, count of sessions, usage of internal resources

i. P2P use by internal systems

j. Wireless network activity

i. Rogue AP detection

ii. Wireless network access by user

iii. WIDS/WIPS alert activity

4. Resource Access Reports

a. General

i. Access to resources on critical systems after office hours / “off” hours

b. Web

i. Top internal users blocked by proxy from accessing prohibited sites (malware sources, pornography, etc)

c. File

i. File, network share or resource access (success, failure) - for specific audited resources

d. Database

i. Top database users - excluding known application access

ii. Summary of query types - excluding known application queries

iii. All privileged user access

iv. All users executing INSERT, DELETE commands - excluding known application queries

v. All users executing CREATE, GRANT, schema changes, etc

vi. Database backups

e. Email

i. Top internal email addresses by count of messages, byte volume

ii. Top internal email addresses sending attachments to public/hosted addresses

iii. All emailed attachment content types, sizes

iv. All internal systems sending mail – excluding known mail servers

5. Malware Activity Reports

a. All systems with AV events by user, system name, time trend

b. Detect-only events from anti-virus tools (leave-alones)

c. All anti-virus protection failures (crashes, unloads, update failures, etc)

d. Internal connections to known malware IP addresses – a public blacklist needed

6. Failures and Critical Errors

a. Critical errors by system, application, business unit

b. System and application crashes, shutdowns, restarts

c. Backup failures

d. Capacity / limit exhaustion - memory, disk, CPU, etc

7. Analytic Reports – Mostly Using “Never Before Seen” (NBS) aka “NEW Type/Object” Analysis = also add “rarely seen” / OSO / “bottom X by …”

a. NEW (NBS) Log message types / event types

b. NEW (NBS) Users authenticating successfully

c. NEW (NBS) Sources that connected to systems using privileged accounts

d. NEW (NBS) Internal system connecting to external systems

e. NEW (NBS) External IPs connecting to NEW Entry Points – not sure how to collect this

f. NEW (NBS) Ports accessed on internal systems

g. NEW (NBS) HTTP request types

h. NEW (NBS) Downloaded/uploaded content types

i. NEW (NBS) Query types on databases

More last-second comments? If not, I will be adding documentation for all report examples and submitting it to SANS for distribution.

Also, if you commented, please let me know if you do NOT want your name in the credits. Default:  you will be mentioned as valuable contributor as long as your contribution was, you know, valuable :-)

Possibly related posts:

• Simple Log Review Checklist
• SANS Top 5 Essential Log Reports Update!

About me: http://www.chuvakin.org

As far as awesomeness is concerned  [and I am a big student of it :-)], this is full of it. BrightTalk Log Management Summit promises to be as awesome as logging events go... Here is an agenda:

WHEN: Thursday, August 19, 2010, attend live online throughout the day or afterward on-demand

HOW: Register Now: http://www.brighttalk.com/r/vbf

TOPICS AND PRESENTERS:

• “Log Standards & Future Trends” by Dr. Anton Chuvakin, Principal, Security Warrior Consulting
• “Leveraging Logs, Information and Events” by Derek Brink, VP & Research Fellow for IT Security, Aberdeen Group
• “Log Visualization in the Cloud” by Raffael Marty, Chief Logger, SecViz.org <– how come they don’t mention Loggly here?
• “The Integration Lifecycle: Loving Long Logging Lifecycles” by Andrew Hay, CISSP, Senior Analyst, Enterprise Security Practice, The 451 Group <- high chance for an awesomeness boost from Andrew!
• “Best Practice and Approaches for Log Management” by Ritesh Singhai, Senior Security Engineer, SecureWorks
• “Delivering Value from SIEM” by Chris Burtenshaw, Information & Technology Risk Manager, Deloitte

Enjoy! And “see” you there on August 19th.

Possibly related posts:

• Brief Log Management Class
• SANS Log Management Class in California?
• FINALLY! SANS SEC434 "The" Log Management Class (2-day version!) in Northern California on Sep 9-10, 2010

About me: http://www.chuvakin.org

It will happen! My SANS SEC434 Log Management Class will be taught in in Northern California on Sep 9-10, 2010 in its never-before-seen extended 2-day version (with loads of cool hands-on log mangling exercises). The announcement follows below:

Log Management In-Depth: Compliance, Security, Forensics, and Troubleshooting
Thursday, September 9, 2010 - Friday, September 10, 2010

“This first-ever dedicated log management class for IT and security managers will cover system, network, and security logs and their management at an organization. We will start with the basics, like making sure that logs exist, and then go on to touch upon everything from managing log storage, to analysis techniques, to log forensics and regulatory issues related to logging.

In the beginning, we will cover various log types and provide configuration guidance, describe a phased approach to implementing a company-wide log management program, and go into specific tasks that IT and security managers need to be focusing on a daily, weekly, and monthly basis in regards to log monitoring.

A unique and comprehensive section that covers the hot topic of using logs for regulatory compliance, such as PCI DSS, will also be presented. Everybody knows that logs are essential for resolving compliance challenges; this class will teach you what you need to concentrate on and how to make your log management compliance-friendly.

The class will also touch upon various uses of logs for incident response, forensics, and operational monitoring. Common logging mistakes, learned from many years of working with logs, will also be explained.”

Class Location:

UC Davis
Room 1065, Kemper Hall, UC Davis
1 Shields Ave
Davis, CA
Web site: www.ucdavis.edu

The price is actually VERY reasonable.

Sign up … NOW! I mean it!! :-)

Possibly related posts:

• SANS Log Management Class in California?

About me: http://www.chuvakin.org

NEW PROPOSED Top 7 Essential Log Reports

• Simple Log Review Checklist

About me: http://www.chuvakin.org

This post is not just an announcement; it contains a BIG question to my readers, mostly in California and around.

As you know, I have authored a SANS Log Management Class (SEC434) which is almost out of beta and near production stage, after a few years of tuning and trial runs. We are thinking of teaching it in California during the second week of August 2010. Via this blog post, I wanted to get some quick feedback from my readers about how many might want to sign up for it. So, please just leave a comment here if you’d like to attend!

Also, I wanted to check whether anybody’s employer (a log management or SIEM vendor perhaps…) would be willing to provide a venue to teach a class. We just need a room with a projector, nothing fancy. In exchange for that, SANS will give you some free attendance seats for the class. So, drop me an email, DM or something, if you’d like to take this opportunity.

The updated information on the class follows below:

“This first-ever dedicated log management class teaches system, network, and security logs, their analysis and management and covers the complete lifecycle of dealing with logs: the whys, hows and whats.

You will learn how to enable logging and then how to deal with the resulting data deluge by managing data retention, analyzing data using search, filtering and correlation as well as how to apply what you learned to key business and security problems. The class also teaches applications of logging to forensics, incident response and regulatory compliance.

In the beginning, you will learn what to do with various log types and provide brief configuration guidance for common information systems. Next, you will learn a phased approach to implementing a company-wide log management program, and go into specific log-related tasks that needs to be done on a daily, weekly, and monthly basis in regards to log review and monitoring.

Everyone is looking for a path through the PCI DSS and other regulatory compliance maze and that is what you will learn in the next section of the course. Logs are essential for resolving compliance challenges; this class will teach you what you need to concentrate on and how to make your log management compliance-friendly. And people who are already using log management for compliance will learn how to expand the benefits of you log management tools beyond compliance.

You will learn to leverage logs for critical tasks related to incident response, forensics, and operational monitoring. Logs provide one of the key information sources while responding to an incident and this class will teach you how to utilize various log types in the frenzy of an incident investigation.

Finally, the class author, Dr. Anton Chuvakin, probably has more experience in the application of logs to IT and IT security than anyone else in the industry. This means he and the other instructors chosen to teach this course have made a lot of mistakes along the way. You can save yourself a lot of pain and your organization a lot of money by learning about the common mistakes people make working with logs.”

P.S. Response to comments might be delayed, I am away from my computers.

Possibly related posts:

About me: http://www.chuvakin.org

This year, Workshop on the Analysis of System Logs (WASL) is reborn as SLAML. Please consider submitting a short paper (no need to do a full academic write-up!). The deadline is July 11.

“Join us in Vancouver, BC, Canada, October 2–3, 2010, for the Workshop on Managing Systems via Log Analysis and Machine Learning Techniques. Modern large-scale systems are challenging to manage. Fortunately, as these systems generate massive amounts of performance and diagnostic data, there is an opportunity to make system administration and development simpler via automated techniques to extract actionable information from the data. SLAML '10 workshop addresses this problem in two thrusts: (i) the analysis of raw system data logs and (ii) the application of machine learning to systems problems. The large overlap in these topics should promote a rich interchange of ideas between the areas.

SLAML '10 combines the Workshop on the Analysis of System Logs (WASL) and the Workshop on Tackling Computer Systems Problems with Machine Learning Techniques (SysML)."

The part related to logs is:

“Log Analysis: It is well known that raw system logs are an abundant source of information for the analysis and diagnosis of system problems and prediction of future system events. However, a lack of organization and semantic consistency between system data from various software and hardware vendors means that most of this information content is wasted. Current approaches to extracting information from the raw system data capture only a fraction of the information available and do not scale to the large systems common in business and supercomputing environments. It is thus a significant research challenge to determine how to better process and combine information from these data sources.”

The topics sought are:

“Topics include but are not limited to:

• Reports on publicly available sources of sample system logs
• Prediction of malfunction or misuse based on system data
• Statistical analysis of system logs
• Applications of Natural-Language Processing (NLP) to system data
• Techniques for system log analysis, comparison, standardization, compression, anonymization, and visualization
• Applications of log analysis to system administration problems
• Use of machine learning techniques to address reliability, performance, power management, security, fault diagnosis, scheduling, or manageability issues
• Challenges of scale in applying machine learning to large systems
• Integration of machine learning into real-world systems and processes
• Evaluating the quality of learned models, including assessing the confidence/reliability of models and comparisons between different methods”

Please submit to advance the state of log analysis research! Past workshop information is here (2008, 2009).

P.S. This is posted by a scheduler; response to comments may be delayed since I might be away from computers.

Possibly related posts:

• Free Log Data For Research – Update

• Workshop on the Analysis of System Logs (WASL) 2009 CFP

• Workshop on the Analysis of System Logs (WASL) 2010 CFP Out!

About me: http://www.chuvakin.org

Given that I spent this entire week getting back into a SIEM-building game [don’t ask :-)], a few thoughts on the state of Security Information and Event Management have dawned on me.

Some security technologies – like network firewalls - are getting pretty darn close to being commoditized and differences between products are ever-so-close to being wiped out.

SIEM, let me tell you, is nowhere near this.  Maybe this also has something to do with the fact that Gartner SIEM MQ 2010 (see this fun commentary from Rocky and his view on SIEM history) contain so many players for so many years. To follow up on this, here is a fun quote from Gartner MQ on SIEM: “There are signs of general convergence on a core set of [SIEM] capabilities.”

Do you know WHEN the above was written? March 2003!

2003! In other words, full 7 (!) years after first SIEM products were built. And also - full 7 (!) years  ago. Look to the right to see how SIEM realm looked back then [yes, Brian, I just reread all SIEM MQs from 2003 to 2010 – just for fun :-)]

Today, in 2010, there is still NO “best SIEM for everybody” and there is NO feature parity even across key capabilities.

Yes, there is a SIEM tool that seems better for large enterprises with unlimited budget. But overall “best SIEM"? Nope.

In fact, I bet that …

If you pick five top SIEM requirements AND 5 “top” SIEM vendors, then at least one of the tools will REALLY SUCK on at least one of the key requirements.

The reality is that after so many years, all – well, most -  SIEM tools actually “run” - but do they always “work?” Let me explain the difference between a software that RUNS from the one that WORKS. “Runs” means that code compiles and, when executed, does not throw an exception. On the other hand, “works” means that it delivers value to its buyer. For example, rule-based correlation runs (well, unless it runs out of memory…oops!), but doesn’t work in many environments (see recent Securosis piece on that). Real-time dashboards run, but aren’t even utilized in many environments. Visualization tools run, but often users cannot get them to work. Risk scoring / statistical correlation runs, but often doesn’t deliver useful results.

And you known, believe it or not, SIEM vendors are NOT the ones to blame for it. Many are honest in saying that “Yes, to succeed,  a SIEM project takes work BY it’s buyer/user.” So, your SIEM likely will WORK, if you WORK on it.

Now, let’s turn this into something practical and useful? What’s a poor SIEM buyer – whether enterprise or mid-market - to do? How to pick the right SIEM?

The only choice I see is the one that won’t surprise my readers: focus on requirements, define your SIEM use cases – and then test the products. Buy the one that WORKS FOR YOU! Some ideas on the selection process can be found here.

Enjoy!

Possibly related posts:

About me: http://www.chuvakin.org

Since people keep asking me again and again, here is another post on the subject.

40,000 ft view:

SIEM = SECURITY information and event management; the emphasis is on SECURITY. Security information is not just logs.

while

LM = LOG management; the emphasis is on LOGS. Logs aren’t just for security.

10,000 ft view:

(with slight risk of oversimplification)

1000 ft view:

Read this paper – then ask me questions if it is not clear.

Finally, people, please STOP obsessing on “SIM vs SEM.” The 1990s are officially over [darn, even 2000s are over!] SIEM is what exists today – that and log management.

Possibly related posts:

• On Choosing SIEM
• The Myth of SIEM as "An Analyst-in-the-box" or How NOT to Pick a SIEM-II?
• Log Management / SIEM Users: "Minimalist" vs "Analyst"
• "I Want to Buy Correlation" or How NOT to Pick a SIEM?

About me: http://www.chuvakin.org

As you remember, I’ve done this webcast/IPC with IANS called “Navigating the Data Stream without Boiling the Ocean: Case Studies in Effective Log Management.” My role as IANS faculty was to moderate the discussion.

My intro slides can be found here. A recording can be found somewhere here – grab it since we had a great panel discussion with a bunch of useful and juicy bits about log management in the real world. Below I am answering some of the fun questions we got at the show for a broader audience of this blog – and sorry for a delay with that.

Q: What, if anything, has anyone done to overcome privacy restrictions in some countries like Germany, France, Italy regarding log collection activity of users?

A: Sorry, but I have to give you a cynical answer. From what I am hearing, those countries are making a choice in favor of - what they think of as – “privacy” over security monitoring and activity auditing. As a result, many of the logging and log review tasks legality is becoming questionable or the burden of performing such tasks grows exponentially. The only advice I can give is to follow the law - even if you screw yourself and your organization in the process. Under democracy, you're supposed to act towards changing the law and not simply ignoring it.

Q: Can you describe your process for determining what to periodically review from your logs?  Did a committee comprised of sysadmin and information security team identify what to review?

A: Ideally, such process should and include all stakeholders, namely, people who can benefit from the information in log files. This would certainly include system administrators and a security team. However, it is not uncommon that the security team will do it on its own if other parties show no interest in participating. Regarding the process itself, the key approach to doing is “use cases.“ What do regulations say about logging and log review? What business units ask for, if anything? What level of details you'd prefer to have during incident response? What are the things I trying to accomplish? Look for future blog posts about this subject.

Q: Would you use log management without a SIEM?

A: Absolutely. I would not use a SIEM without log management though; I would also try not to use a SIEM without a good log management tool. For more info on this subject read this, this, this.

Q: Does using a complete SIEM solution reduce the number of staff required?

A: Hard to say what is meant by ”complete” here, but the answer is either “no” or “it depends.”

Overall, I do not like this type of positioning at all: if you are trying to purchase a SIEM solution in order to fire your security analysts, you'll fail miserably. On the other hand, if you'd like to reduce the number of people whose jobs consist of only reading logs every day, then SIEM can help reduce that staffing need so that you can allocate people to more productive security monitoring tasks. Still, the main value of a SIEM tool lies in the skilled personnel that operates it! For example, see this one.

Q: What is your definition of structured and un-structured data [mentioned in the discussion]?

A: Structured data is more like a database table, it has named fields such as “username”, “source IP”, etc. Name=value pairs is another example of log data with structure. On the other hand, plain English text is not structured [at least, not for our purposes of log analysis] and needs to be either structured (“parsed”, tokenized, etc) or directly analyzed using text mining tools.

Q: How visualization tools technically help in log review?

A: See http://secviz.org for more information on the subject than you ever wanted to learn :-) While you're in the subject, get a great book about it.

Q:  What level from the log management maturity curve [A.C. - reference to this graph] does HIPAA compliance require?

A: Based on the fact that HIPAA prescribes logging (164.312(b) Audit Controls) and some monitoring for specific events, such as logins (164.308(a)(5)(ii)(C)    Log-in Monitoring), I’d venture a guess that HIPAA compliance will require an organization to have a fairly mature log management and security monitoring operation. And is this reality? No, many healthcare organizations are nowhere near that stage with their logging.

Also, see awesome coverage of this webcast from Rocky DeStefano is here at his VisibleRisk blog.

Enjoy!

Possibly related posts:

• Fun Logging Webcasts: 4/1/2010 and 5/12/2010
• Open Group Log Webcast Slidea and Q&A: "Enterprise Logging and Log Management: Hot Topics"
• Thursday 3/25 IANS Webcast + Panel on Log Management: "Awesome++"
• Log Management / SIEM Users: "Minimalist" vs "Analyst"
• The Myth of SIEM as "An Analyst-in-the-box" or How NOT to Pick a SIEM-II?
• Log Management + SIEM = ?
• SIEM Bloggables talks about SIEM use cases.

About me: http://www.chuvakin.org

Two new resources about logging, log management and SIEM , written by myself, have just been published.

“The Complete Guide to Log and Event Management” (a lofty title, I know) was written for Novell Sentinel team (this doc is behind the sign-up form, but it is totally worth it :-)) BTW, if you mistakenly wrote off Sentinel from the SIEM battlefield, you are sadly mistaken!

Quote:

“What happens after an organization deploys a log management tool and starts using it effectively for security and compliance as well as operational purposes? The natural and logical progression is for organizations to graduate to near-real-time event management by deploying a SIEM tool.

This paper is the first document that formulates “graduation criteria” for such development. Organizations that graduate too soon will waste time and effort, and won't any increased efficiency in their security operation. However, waiting too long also means that the organization will never develop the necessary capabilities to secure themselves.

[…]

One of the paramount conclusions from this work it to remember that everybody has logs and that means that everybody ultimately needs log management. In its broadest form, log management simply means “dealing with logs.” And if you have logs you have to deal with them – if only because many recent regulatory mandates prescribe that.

It's also important to remember that logs are used for a very large number of situations: from traditional (incident response) to highly esoteric. Most of such uses of logs happen much later, after the event happens and is recorded in logs. It is much easier to be prepared to respond then to monitor.

Your organization might need to go “back to logging school” before it is ready to “graduate to SIEM.” Such graduation requires an ability to respond to alerts and customize and tune products.”

“PCI Logging HOWTO” (part 1)  was written for Prism Microsystems, the home of 100 Uses for Log Management.

Quote:

“To satisfy those requirements, it is recommended that an organization create “PCI System Log Review Procedures” and related workflows that cover:

• Log review practices, patterns and tasks
• Exception investigation and analysis
• Validation of these procedures and management reporting.

[…]

First, do determine the scope for PCI compliance, including systems, databases that store the data, application that process the data, network equipment that transmits unencrypted card data as well as any system which is not separated from the above by the firewall. In the case of so-called “flat network”, the entire IT environment is in scope and thus must be made compliant by implementing DSS controls.

Second, identify system components that touch the data: apart from the operating systems (Windows or Linux, for example) databases and web servers need to considered as well as payment application, middleware, Point-of-Sale (PoS) devices, etc.

Next, a logging policy must be created. PCI-derived logging policy should at least cover logged event types for each application and system deployed as well as details for all systems in scope for PCI DSS.  For custom applications, this logging policy needs to be communicated to internal or outsourced developers.”

Enjoy! And make sure you remember that I am available for consulting projects to deliver value to your organization as well.

Possibly related posts:

• “Log Management + SIEM = ?”
• "I Want to Buy Correlation" or How NOT to Pick a SIEM?”
• “The Myth of SIEM as "An Analyst-in-the-box" or How NOT to Pick a SIEM-II?”

About me: http://www.chuvakin.org

The famous SANS Log Management Survey 2010 is out; grab the document here [PDF], some highlights follow below:

• One of the highlights this year is “there is room for improvement in the ability of log management systems to deliver value from logs being collected, specifically in the areas of searching (where 36% of respondents reported problems), and analysis (where 34% had problems)”
• It seems like collection beast has been beaten back by most organizations ”in 2005, respondents reported their biggest problem was simply collecting log data, in
  2010, collection was cited by only 10% as the biggest problem.” Looks like we continue to nicely evolve along the log maturity curve.
• ”The top reason for collecting logs was to “Detect/prevent unauthorized access and insider abuse,” with 63% of respondents rating this as most critical.“ Compliance, BTW, got the 2nd place with 41%. BTW, I think this is a fluke since the survey unexplicably had two of the identical options “meet regulatory requirements” and “ensure regulatory compliance."  These two together beat the current response leader!
• As expected, “organizations aren’t installing log management systems for their troubleshooting value, yet an increasing
  number are finding troubleshooting assistance to be a useful added bonus.” The magic of “Compliance+” model where you buy from compliance (say PCI DSS) and then use for security, troubleshooting, etc is alive and well!
• Another long expected finding is that  “we’re also seeing a large increase in the amount of log data being collected from a variety of other types of sources.” (there is a curve for that too). Log management is not only about firewalls, routers, switches, IDS/IPS, servers anymore (collection rates for these run in 90-95% range)
• And here is an OMFG: “Last year [2009], 41% of respondents collected log data from homegrown applications. This year, the definition was expanded to include both homegrown and commercial applications, and 73% of
  respondents say they’re collecting logs from these sources.” Mind-blowing indeed! I’d never guess that 73% of all organization collect application logs today. I am sure folks at Brannan St are preparing for an IPO, given this news.
• Similarly shocking is a finding that ”Logs from desktops are also being collected—49% of respondents report collecting data from desktops.” And so is “48% of respondents indicated they collect log data from “Physical devices (badge access systems, plant control systems).” This is great! This totally makes me think we live in the 21st century :-)
• Even though “The amount of log data being collected is growing at the rate of 15-20% per year,”  log retention is up, as expected, “This year, the largest group retained logs for one to two years.” However, survey says that even some PCI DSS impacted organizations store logs for less than 90 days. Sorry, guys, you are just not compliant :-) And don’t blame the council!
• At the end, the survey teams gives some useful tips such as “Know Your Data” and “Know Your Logs,” and, specifically, “Get to learn what normal is for your organization”

So, enjoy the survey! BTW, the webcast (part 2) focused on the use of the data – which is more fun (here is a link to part 1 webcast which focused on collection). The rumor is, however, that the recording for part 2 might not be available… :-)

About me: http://www.chuvakin.org

As you remember, I’ve done this webcast with Open Group called “Enterprise Logging and Log Management: Hot Topics.”

Here are the slides from the webcast. Full recording with voice can be found here. Below I am answering some of the fun questions we got at the show for a broader audience of this blog.

Q: As the log management curve matures [reference to this graph], how do you ensure that the log data is secure?

A: Check out my “Top 11 Reasons to Secure and Protect Your Logs.” In reality, access control, occasionally hashing (yes, sometimes even with MD5) and sometimes encryption of archived logs is the “state of the art” for log protection. Think about it! People don’t encrypt and poorly protect SSNs, payment card numbers and their own key intellectual property… do you think they will protect logs well? Thus this is in mane cases an academic question…

Q: What do you mean by “use cases” here, is it the same concept as in software engineering or it has diff context here?

A: Yes, same use case definition – pardon for a bit of PM-speak. Example brief SIEM use cases are here.

Q: Are there any templates or best practices to decide as what to log in order to cover wide range of domains/purposes e.g. hacking, policy,

A: This is a million dollar question, really. What exactly needs to be logged for PCI has been discussed here and here and I was involved in some consulting projects to define that for a particular company (recent project example). In the near future, an attempt will be made to answer this question more consistently… sorry, can’t say more yet, but watch this blog for updates.

Q: How have you dealt with the trade off of logging requirements & mandates vs scale & performance needs in the area of application architecture? 

A: Poorly? :-) In most cases the mandate/security requirement HAS TO WIN and the only way for the developer to present this situation as “a tradeoff” is to avoid the security guy like a plague until the application is fielded – and the present this fake “dilemma.” In reality, if your application crashes or slows to the crawl when you enable logging of, say, all transactions, it needs to go back to the drawing board. Think of an example: can you field a payment app that can operate without logging all transactions? There is no tradeoff here.

Q: Would you please suggest a log management application?

A: Free tools are listed here and some commercial ones are here; you can pay me to select the right tool for your requirements  since log management is broad enough and complex enough to make “one best log management tool” a pipe dream at best. Are you collecting Cisco ASA log data or Oracle Finacials audit table? For PCI DSS or against web application attacks? Or maybe for web server debugging? These are other cases will have different “best app choices.” You can try reading this to learn maybe you need to write your very own log analysis application.

Q: What is your opinion of OVAL/CVE and SCAP as standards for log management?

A: CEE by MITRE is an active effort to create such set of log standards; NIST plan to later adopt them as “EMAP” (SCAP’s logging brother). As we work on the standard, I occasionally blog about it here. Right now the team is actively engaged at weekly workgroup calls and email discussions, mostly focusing on finalizing the taxonomy draft (the famous “O-A-S”), “logging profiles” and other fun things.

Enjoy!

Possibly related posts:

• “Thursday 3/25 IANS Webcast + Panel on Log Management: “Awesome++””
• My other recent presentations and webcasts
• All of my speaking
• My decks at Slideshare.

About me: http://www.chuvakin.org

As we all know, blogs are a bit "stateless" and a lot of useful security reading material gets lost since people often only pay attention to what they see today. These monthly round-ups is my way to remind people of useful content from the past month! If you are “too busy to read the blogs,” at least read these.

So, here is my next monthly "Security Warrior" blog round-up of top 5 popular posts/topics.

1. By a HUGE margin (20x?), the #1 post this month is “Simple Log Review Checklist Released!” Grab our log review checklist here, if you have not done so already.
2. My RSA 2010 interview with Bob Russo and Troy Leach from PCI SSC holds the #2 spot: “RSA 2010 EXCLUSIVE PCI Security Standards Council Interview.” BTW, my other RSA coverage shows up on the top list as well: “RSA 2010 – Day 1 Metricon” and other RSA 2010 posts.
3. “The Myth of SIEM as “An Analyst-in-the-box” or How NOT to Pick a SIEM-II?“ and its predecessor ““I Want to Buy Correlation” or How NOT to Pick a SIEM?” hold the next position this month. They present some sadly popular misconceptions about acquiring and implementing SIEM and log management tools.
4. “Log Management / SIEM Users: “Minimalist” vs “Analyst”” being next on the hotlist made me think that maybe my blog is back to being the best blog on SIEM and log management :-) Also, my post on progressing from logging to log management to log monitoring discussion in “Logging, Log Management and Log Review Maturity”  is still popular as well. It presents a maturity scale for organization selecting log management or SIEM.
5. Open source SIEM – and now also open source log management - theme continues to drive a lot of traffic (starting from “Short Observation on Open Source SIEM”) – it looks like folks are still desperately googling for it. “Why No Open Source SIEM, EVER?” post takes the spot in Top5 this month again – and so does “Open Source CLOUD SIEM, Anybody?” The older inspiration for these posts is “On Open Source in SIEM and Log Management.”

This month I am continuing a new tradition: I am going to thank my top 5 referrers this month (those that are actual humans, that is). So, thanks a lot to the following people whose blogs sent the most visitors to my blog:

1. Walt Conway
2. Sandro Süffert
3. Dancho Danchev
4. Lenny Zeltser
5. Stefano Zanero

Thank you for all the link-love!

See you in April; also see my annual “Top Posts” - 2007, 2008,  2009!

Possibly related posts / past monthly popular blog round-ups:

• Monthly Blog Round-Up – February 2010
• Monthly Blog Round-Up – January 2010
• Monthly Blog Round-Up – December 2009
• Monthly Blog Round-Up – November 2009
• Monthly Blog Round-Up – October 2009
• Monthly Blog Round-Up – September 2009
• Monthly Blog Round-Up – August 2009
• Monthly Blog Round-Up – July 2009
• Monthly Blog Round-Up – June 2009
• Monthly Blog Round-Up – May 2009
• Monthly Blog Round-Up – April 2009
• Monthly Blog Round-Up – March 2009
• Monthly Blog Round-Up – February 2009
• Monthly Blog Round-Up - January 2009
• Monthly Blog Round-Up - December 2008
• Monthly Blog Round-Up - November 2008
• Monthly Blog Round-Up - October 2008
• Monthly Blog Round-Up - September 2008
• Monthly Blog Round-Up - August 2008
• Monthly Blog Round-Up - July 2008
• Monthly Blog Round-Up - June 2008
• Monthly Blog Round-Up - May 2008
• Monthly Blog Round-Up - April 2008
• Monthly Blog Round-Up - March 2008
• Monthly Blog Round-Up - February 2008
• Monthly Blog Round-Up - January 2008
• Monthly Blog Round-Up - December 2007
• Monthly Blog Round-Up - November 2007
• Monthly Blog Round-Up - October 2007
• Monthly Blog Round-Up - September 2007
• Monthly Blog Round-Up - August 2007

Obligatory “added everywhere” posts :-)

• I am available for consulting projects related to logging, log management, SIEM, PCI DSS etc. Please see the services list at my consulting site.

About me: http://www.chuvakin.org

In the next few days, I will be doing two fun logging webcasts with The Open Group. Here is the info, quoted from their site:

Title: Enterprise Logging and Log Management: Hot Topics
Date & Time: Thursday, April 1, 2010, 11:00am Eastern Time

Capturing log information is critical to IT organizations for many reasons, including for security incident detection and response, and for compliance with numerous regulations and standards. Join one of the foremost experts on log management, Dr. Anton Chuvakin, as we discuss enterprise logging challenges and issues.

Moderator: Jim Hietala, VP Security, The Open Group
Panelist: Dr. Anton Chuvakin, Security Warrior Consulting

To register and attend: https://opengroupevents.webex.com/opengroupevents/onstage/g.php?t=a&d=664303043

Title: Logging Use Cases and Standards Update
Date & Time: Wednesday, May 12, 2010 11:00 am Eastern Time

Following on from our April 1 Log Management Challenges webcast, this second webcast will explore some log management use cases, including around accountability for data access. In addition, an update on progress in standards work from The Open Group (XDAS) and MITRE (CEE) will be presented.

Moderator: Jim Hietala, VP, Security, The Open Group
Panelists:

• Ian Dobson, Director, Security & Jericho Forums, The Open Group
• Dr. Anton Chuvakin, Security Warrior Consulting
• Joel Winteregg, Netguardians
  

To register and attend:
https://opengroupevents.webex.com/mw0306l/mywebex/default.do?siteurl=opengroupevents&rnd=0.20892260101881588

Possibly related posts:

• All posts tagged presentation

About me: http://www.chuvakin.org

“What makes a log management program effective? Log management activities must be prioritized in order to operate your security team effectively. We will review and analyze best practices for implementing  log management programs as well as address SIEMs’ influence on the goal of optimization. This virtual discussion is ideal for risk, compliance, and security managers, as well as anyone looking for new approaches to gain intelligence from their log data.”

• All posts about SIEM
• All posts about log management

About me: http://www.chuvakin.org

Just a random piece of some research project I did at some random point :-) In discussions at RSA 2010 conference, somebody mentioned that SIEM, log management and other monitoring/detection security product users are split into two major categories: one actually uses the product while the other “buys it for compliance” and then eventually uses … as a doorstop, for example.

And I actually had  an old presentation about this that was offered as strategic guidance to my consulting client (a vendor).

Here is that picture and text: two types of SIEM/log managements users that your solution has to make happy:

“Minimalist” SIEM/LM User

•Still evolves from “logs are dirt” to raw collection of log data

•Pure compliance focus – “deliver me from evil… eh… auditors” (or assessors, in case of PCI DSS)

•Collecting logs is the primary “activity”; not even thinking about log review yet

•Checkbox mentality is rampant among that type of user (sometimes, “correlation” is one of the checkboxes, sadly)

•Less mature; needs more hand-holding when deploying the product (might not want any help though…)

“Analyst” SIEM/LM User

•Evolved to “so we have them collected – now what?”; stuck now and not sure how to use “all that data”

•“Compliance+” or even pure security/operational focus; for example, SOC operation

•Using logs – review, analysis, at the very least investigations

•Explore and use logs mentality, focuses on getting the value of the data and solving problems

•More mature; needs more “cool tools”

So, before you plan/design/build your solution, think what is the primary user type… but keep in kind that to be truly successful you might need to entice both.

Enjoy!

Possibly related posts:

About me: http://www.chuvakin.org

Today, many people are looking for very simple solutions to big and complex problems – and the area of logging and log management is no exception. Following that theme, we have created a "Critical Log Review Checklist for Security Incidents" which is released to the world today.

In addition to HTML, PDF or DOC versions are available as well (alternative hosting location is here). Feel free to modify the checklist for your own purposes or for internal distribution in your organization - but please keep the attribution to the authors.

The log cheat sheet presents a checklist for reviewing critical system, network and security logs when responding to a security incident. It can also be used for routine periodic log review. It was authored by Dr. Anton Chuvakin and Lenny Zeltser (BTW, Lenny has other useful security cheat sheets on malware analysis, security architecture, DDoS, etc  here)

Here is the embedded version from DocStoc:

Enjoy!

About me: http://www.chuvakin.org

In response to one of my previous SIEM posts (“I Want to Buy Correlation” or How NOT to Pick a SIEM?”), one of my readers grabbed onto my analogy (“correlation engine as engine, SIEM with content as car”) and said:

“They want a Security Analyst in a Box, because once you deliver the engine in a car, they realize they don't want to drive either”

This is, sadly, very true, despite the deep and obvious ridiculousness of such sentiment.

So, WTF? Did anybody sell you a tiny teeny security analyst stuck up in one of those 1U SIEM appliances you can buy at Walmart nowadays – or at least from your friendly neighborhood VAR? Where did this come from and what we can do about it?

Well, one thing we can do is simply say: if a Security Information and Event Management vendor came to you and said “this little box will manage your security” and you believed it, you need to have your head examined. But just saying this wouldn’t be funny enough for this blog! Noooooo…

So, instead, I came up with 7 reasons why SIEM is NOT “an analyst in the box”:

1. SIEM requires the buttons to be pressed, and analyst presses those buttons. See? There is a difference.
2. Imagine the best SIEM out there with the best correlation engine, the best rules, the best interface, etc [it is also the most expensive…] Now imagine the dumbest person ever to be tasked with event analysis. Well, they are about equally intelligent [*].
3. Despite what you might think, log analysis (and, definitely, security monitoring in general) is a pretty subjective “art.” Boxes don’t do art well, humans do.
4. Modern SIEM products have some “value out of the box”: report, stock rules, knowledge bases, etc. And, indeed, a vendor system engineer can even schedule the reports and set the rules for you – before leaving. However, do you have anybody who actually understands the information coming out of the product? Well, that’s the analyst, and he ain’t in the box. You have to hire him or her! (or have a consultant help you :-))
5. In addition to art, log management often involves politics: can we get those logs? Can we get the context needed for analysis (e.g who owns that system … eh…in addition to RBN that 0wns it, that is :-)). Just like art, boxes don’t do politics well, humans do.
6. Think of a good SIEM as a robot defender [assuming that you turned on automated response, oh the Adventurous One]. Do you see the military switching from human soldiers completely to robots? Exactly! SIEM + analyst = security defense. SIEM alone = gun rusting in the trench.
7. Analysts needs to be fed, SIEM can survive on just the diet of logs.

Thus, if you expect a security information and event management system to “be an analyst in the box”, stop expecting it. If you don’t want or can’t run a SIEM, don’t buy it (look here to see whether you are ready) or don’t download it. In other words, SIEM requires ongoing commitment to keep delivering value: no commitment – no value, it is that SIEMple.

BTW, I am thinking of writing a whole mammoth paper on picking the right SIEM. My dear vendor friends reading this blog, wonna sponsor it?

[*] I have seen some data mining algorithm mimic – and actually rival! – the performance of a junior security analyst. Sadly, they were build for a home-grown SIEM, now defunct… Oh, the lore of civilizations long gone :-)

Possible related posts:

• “I Want to Buy Correlation” or How NOT to Pick a SIEM?”
• “Logging, Log Management and Log Review Maturity” (helps to know whether you are ready for that SIEM)
• “Log Management + SIEM = ?“
• “SIEM Bloggables” (has SIEM use cases)
• “On SIEM Complexity“
• All blog posts on SIEM

About me: http://www.chuvakin.org