Event Analyst ® 7 Can Slice and Dice Your Security Event Logs ... Any Way Your Auditors Want Them Served

After more sweat and tears, the Dorian Software Development Team is happy to announce the release of Event Analyst Version 7. Version 7 comes with a huge overhaul to the custom reporting engine inside Event Analyst, allowing our users to parse the smallest details out of Windows Security Log Events, grouping and sorting them to their (and their auditors!) hearts' content. We call this Event Analyst's PrecisionParser ™ capability, and it's a subcomponent of our greater LogRefiner ™ technology.

Some log management software in the marketplace attempts to tokenize and normalize security log data at the time of collection/import, necessitating 1.) a database platform for analysis, 2.) numerous table schemas to store the different types of tokens for different categories of events (e.g. taxonomies), 3.) revisions of said schemas as event tokens expand over time (often as a result of new operating systems and service packs).

The whole process above is pretty labor intensive, and if you're a forensic auditor or the administrator of a small network, setting up a database for this purpose can be a costly endeavor. You may just want to open an EVT/EVTX file and rip it down every which way to produce some nifty reports. Or import a handful of said files into Access ®, and then rip them down together.

We have opted for a different approach. Our PrecisionParser does the parsing of key Windows Security Log Description subfield data at the time the data is analyzed and reported against. It can work against a bunch of different formats, such as security log data still inside EVT/EVTX files, to comma-delimited text files and database tables produced by Event Archiver, our log collection and centralization software package.

Yes, you heard that EVTX part right. While some vendors still have their heads in the sand regarding EVTX compatibility for Windows Vista ™ and Windows Server ® 2008, Event Analyst can already parse the EVTX logs just as easily as the EVT versions, even if security log data from both operating systems resides together in one database table. This is a good thing, because the number of security events (as well as the tokens in their Description fields) have only expanded within Vista and 2008.

Here are some of the details on PrecisionParser inside Event Analyst:

As any veteran of security event log analysis can tell you, the subvalue name/data pairs in the Description field of Windows Security events are the golden nuggets that must be mined to generate meaningful reports. Existing users of Event Analyst have already enjoyed the capabilities of Event Analyst's prebuilt reports to extract, group, and sort this level of detail in a variety of categories, like logon activity and group management.

Now, Dorian Software has incorporated its exclusive PrecisionParser capability - a component of Dorian Software's exclusive LogRefiner technology - into Event Analyst's custom reporting engine. What does this mean to you? Plenty! Virtually any type of security event can now have its key subfields parsed out, grouped, and sorted inside Event Analyst's custom reporting engine. Want to group your 529 logon failures by Source IP Address and Authentication Package? No problem. Need to sort file access events by Handle ID? We've got that covered as well.

The benefits of Dorian's PrecisionParser capability are tremendous, and include:

True Log Format Independence - Parsable security log data formats include native EVT and EVTX files, comma-delimited text files produced by Event Archiver and Event Analyst, and Microsoft Access, SQL, or Oracle database tables produced by Event Archiver and Event Analyst. Dorian's multiple log format support stands in stark contrast to other vendor packages, which depend on multiple database table schemas in attempt to normalize log data at time of collection, rather than normalizing data at time of analysis.

True Operating System and Service Pack Level Independence - PrecisionParser can handle virtually all security log data collected from different Microsoft operating systems - from Windows NT 4.0 to Windows Server 2008. This is important as Microsoft frequently expands reported data in security log events over time, often after service packs are applied. If a custom-defined subfield is not present in a legacy operating system event, the custom reporting engine degrades gracefully, simply indicating that the field was not found.

Correlation Across Related, Yet Different Security Events - Correlation is possible among different security events that share common subfields in their descriptions. For example, many security events log handle identifiers, logon identifiers, and IP addresses. Custom reports paired with advanced filters can now be designed to show a variety of event activity that is in fact related via these fields.

Support For Multiple Occurrences of the Same Subfield - While less common in legacy security events, Windows Vista and Windows Server 2008 now often include the same subfield name twice in the Description field. For instance, Event ID 4724 describes the resetting of user passwords by an administrator. Yet the order of the occurrence of the user in the Description determines whose password was reset, and who actually reset the password. When defining custom fields for reports, Event Analyst allows you to make this subtle distinction by indicating if you would like to parse out the second, third, or nth occurence of that field.

Multiple Report Formats Remain Available For Presentation and Data Mining - As in previous versions, custom reports in Event Analyst will continue to be generated in both HTML and CSV formats. The printer-friendly HTML version of the report is excellent for presentation and review by management, whereas the CSV version of the report allows you to import raw, parsed subfield data from the description field into other software packages, such as Microsoft Excel ®. Frequent users of Microsoft Excel will be amazed at the level of analysis possible when reviewing CSV files with Excel's AutoFilter feature.

New EVTX Log Format Whitepaper Released

Earlier this week, we released a new whitepaper that outlines many of the key changes in the new EVTX event log format found in both Microsoft Vista and Windows Server 2008. We've included some choice pull quotes below, but you can download the full version by registering here:

http://www.doriansoft.com/evtx

Beginning with Microsoft® Windows Vista™ and Windows Server® 2008, Microsoft has completely redesigned its event log format. This new EVTX file format stores event log records as a stream of binary XML records. Accessing data in the new EVTX files requires the use of a new application programming interface that is not available in older Windows operating systems. In addition, the number of, structure of, and data within the fields in the EVTX log records has changed significantly.

Because the new Windows Event Log API functions are only available inside Windows Vista and later operating systems, legacy Windows operating systems like XP and 2003 cannot read previously saved EVTX files at all - there is simply no forward compatibility for consuming saved EVTX files. And while the legacy Event Log API can be used to read some of the events from an "active" EVTX file (that is to say an EVTX file currently being maintained by the EventLog service on a Vista machine), it cannot properly read and parse some events recorded by the new API.

In summary, both forward compatibility to EVTX files from legacy Windows operating systems and backward compatibility to EVT files are severely hampered, if available at all. As a result, organizations that rely on their own scripts and automation techniques may be tempted to develop two different systems for log management - one supporting legacy EVT files on legacy operating systems, and another supporting EVTX files on Windows Vista and Windows Server 2008. Such a strategy has the potential to decentralize log collection and reporting, as well as substantially increase costs over time.

Again, to read the full version, please register here:

http://www.doriansoft.com/evtx

That Infernal Road, Paved With Good Intentions...

Eric, the head auditing guru at Microsoft, posted today on his blog that he is receiving an ever-increasing number of complaints on the lack of documentation regarding the new Event IDs in the Windows Vista™ security log. Specifically, he says that our earlier post "complains" about how sometimes the "add 4096" rule works in Vista's security log, but not in all cases.

With that background, let me take some time here to clarify our original comments and attempt to speak to the source of the frustration Eric is hearing from log management vendors, log scripting enthusiasts, and security admins.

First off, our earlier post on the 4096 offset trick in Vista was not a complaint in so much as it was an attempt to draw attention to a very significant change in the Windows Vista security log. Keep in mind, while Microsoft has made subtle changes to security events ever since Windows NT, the changes in auditing from Windows® NT to Windows 2000 to Windows XP to Windows 2003 are nowhere near as complex as the changes from Windows 2003 to Windows Vista and the forthcoming Windows Server 2008™.

Expanding on this, the complete renumbering of security events in Vista is just the tip of the iceberg. Compounding this trauma of sorts is:

A.) A completely new logging file format, the EVTX file
B.) A completely new API that is used to manage these EVTX files
C.) New, different auditing categories (Tasks) in the Vista security log
D.) Shifting of user account information out of the User field altogether in security events
E.) Other changes to the "traditional" log fields that were present in the legacy EVT files (e.g. the Level/Type field)
F.) Other issues related to forward and reverse compatibility as it relates to log management on pre-Vista and Vista.
... etc

That being said, we know that Eric is not responsible for all of these changes. He did not create the new EVTX log format or the API used to access it, for instance.

Collectively, though, all of these challenges together are most likely frustrating third-party log management vendors, as well as the admins who have developed scripts to automate security event management. Unfortunately, it would appear that Eric is getting the brunt of that frustration. Perhaps he should post contact information for the team at Microsoft that developed the Crimson logging format and accessory APIs so that constructive criticism and questions can be more properly distributed.

At Dorian, our approach is to adapt and innovate around the changes to Microsoft Vista's new logging format and auditing system, and we are proud of our efforts to date. Still, we hear every day the issues that small and medium sized businesses face regarding log management, often directly due to compliance regulations. Not every organization has the budget or resources needed to procure a commercial log management package, and for those facing a complete rearchitecture of their log automation scripts in Windows Vista and Windows Server 2008, those limited resources just got stretched even tighter.

Storage Requirements for the Windows Vista™ Security Log

Recently, we've created a few blog postings that talk about some of the new events present in the Microsoft Windows Vista™ security log. From a security standpoint, Vista's increased number of auditable events is excellent, as administrators and compliance officers can get a much deeper picture of the actions taking place on a computer prior to and during a security incident.

However, if you are required to retain those security events, either by law (e.g. HIPAA, SOX, GLB, PCI, etc) or by policy, you need to start budgeting for more storage before you start your Vista and Windows Server 2008™ migrations.

Here are a few examples of how Vista security logs tend to grow much more quickly than their predecessors:

1.) Looking at some of our internal Vista security logs, there are tons of events relating to the blocking or accepting of network data via the Windows Filtering Platform. Some organizations may find this data valuable, especially if the machine is exposed to the public, however others may not.

2.) Some events log extra information at the end of the Description field that serves no other purpose than to further explain the parameters in the Description field. For instance, every 4608 event (Windows is starting up) also tells you that:

"This event is logged when the LSASS.exe starts and the auditing subsystem is initialized."

Similarly, every 4634 event (An account was logged off) feels the need to mention that:

"This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer."

These are just two brief examples, but note well: your Vista logs will use up more space than your XP and Windows 2000 workstation logs. If you are reassuring yourself now by thinking that you only need to retain server logs, bear in mind that Windows Server 2008 will share Vista's new events and logging tendencies!

Fortunately, the current release (and several prior releases) of our Event Archiver™ software offers you techniques to help you manage your storage of log data. Event Archiver allows you to automatically prune your database tables by date, selectively import only key events or exclude non-key events into database tables with global import filters, and keep your data in multiple compressed formats for storage efficiency. As the number of auditable events increase and expand in size, these features become increasingly important.

Vista-Compatible Release of Event Archiver is Near

With all the posting we've been doing about Vista events recently, you may have assumed we've been doing a lot of work to get our software titles Vista-ready. You'd be correct.

Stay tuned to the blog, as next week we're going to reveal Version 7.0 of Event Archiver, with tons of really cool new Vista-specific features. We're introducing some pretty radical technology, and we think you'll be quite impressed! :)

Auditing Changes To Your Auditing (Event ID 4907)

Here's another nice new security event that has been added to Vista - Event ID 4907. It has been tasked with recording every time an administrator or program changes the SACL on an object, typically a file or folder. What's a SACL, you ask? Let's have a brief primer on that subject.

Every securable object (e.g. file, folder, registry key, etc) in Windows has a Security Descriptor assigned to it. The security descriptor, among other things, specifies:

1.) the user owner of the object
2.) the group of the object (used by Unix apps that run under POSIX)
3.) the DACL (Discretionary Access Control List), and
4.) the SACL (System Access Control List)

When you use Windows Explorer or Group Policy to change who can access a file or folder, you are changing the DACL. Similarly, when you click the "Advanced" button in Windows Explorer on a file or folders property page, and visit the Auditing tab, you are changing the SACL.

The SACL is what the operating system uses to determine which users, groups, and identities cause auditing events to be generated in the Security log when said users perform various actions on files, folders, registry keys, etc.

So to summarize: When you change the users/groups that *can access* a file/folder, you are changing the DACL. When you change the users/groups who generate auditing events *when they access* a file/folder, you are changing the SACL.

Anyway, back to Event ID 4907. In Vista, this event gets logged any time an administrator changes how a file/folder is audited. Here's a sample of the event description:

Auditing settings on object were changed.

Subject:
Security ID: DOMAIN\Admin
Account Name: Admin
Account Domain: DOMAIN
Logon ID: 0x1f472

Object:
Object Server: Security
Object Type: File
Object Name: C:\Folder
Handle ID: 0x28c

Process Information:
Process ID: 0x690
Process Name: C:\Windows\explorer.exe

Auditing Settings:
Original Security Descriptor:
New Security Descriptor: S:ARAI(AU;OICISAFA;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;WD)


Reviewing the description of this event, we see that we can determine:

1.) Who changed the SACL (DOMAIN\Admin)
2.) What program they used to change the SACL (explorer.exe)
3.) The name and type of the object changed (c:\folder, file)
4.) A SDDL representation of the old Security Descriptor and new Security Descriptor.

So now, in Vista, you can track anyone who changes how a critical file/folder is audited, including how it was audited BEFORE the change, and how it will be audited AFTER the change. Again, this is great from an accountability standpoint in organizations governed by compliance regulations.

Oh, and if you're curious about how to translate the SDDL string into something meaningful, please read this article.