That Infernal Road, Paved With Good Intentions...

Eric, the head auditing guru at Microsoft, posted today on his blog that he is receiving an ever-increasing number of complaints on the lack of documentation regarding the new Event IDs in the Windows Vista™ security log. Specifically, he says that our earlier post "complains" about how sometimes the "add 4096" rule works in Vista's security log, but not in all cases.

With that background, let me take some time here to clarify our original comments and attempt to speak to the source of the frustration Eric is hearing from log management vendors, log scripting enthusiasts, and security admins.

First off, our earlier post on the 4096 offset trick in Vista was not a complaint in so much as it was an attempt to draw attention to a very significant change in the Windows Vista security log. Keep in mind, while Microsoft has made subtle changes to security events ever since Windows NT, the changes in auditing from Windows® NT to Windows 2000 to Windows XP to Windows 2003 are nowhere near as complex as the changes from Windows 2003 to Windows Vista and the forthcoming Windows Server 2008™.

Expanding on this, the complete renumbering of security events in Vista is just the tip of the iceberg. Compounding this trauma of sorts is:

A.) A completely new logging file format, the EVTX file
B.) A completely new API that is used to manage these EVTX files
C.) New, different auditing categories (Tasks) in the Vista security log
D.) Shifting of user account information out of the User field altogether in security events
E.) Other changes to the "traditional" log fields that were present in the legacy EVT files (e.g. the Level/Type field)
F.) Other issues related to forward and reverse compatibility as it relates to log management on pre-Vista and Vista.
... etc

That being said, we know that Eric is not responsible for all of these changes. He did not create the new EVTX log format or the API used to access it, for instance.

Collectively, though, all of these challenges together are most likely frustrating third-party log management vendors, as well as the admins who have developed scripts to automate security event management. Unfortunately, it would appear that Eric is getting the brunt of that frustration. Perhaps he should post contact information for the team at Microsoft that developed the Crimson logging format and accessory APIs so that constructive criticism and questions can be more properly distributed.

At Dorian, our approach is to adapt and innovate around the changes to Microsoft Vista's new logging format and auditing system, and we are proud of our efforts to date. Still, we hear every day the issues that small and medium sized businesses face regarding log management, often directly due to compliance regulations. Not every organization has the budget or resources needed to procure a commercial log management package, and for those facing a complete rearchitecture of their log automation scripts in Windows Vista and Windows Server 2008, those limited resources just got stretched even tighter.

Highlights From the Event Archiver 7 Press Release

Initial feedback on Version 7 of Event Archiver® from customers has been very good. Version 7, in case you didn't read the earlier blog posting, has direct support for Microsoft Windows Vista™ EVTX logs throughout the program. We also added a bunch of cool new features to help overcome some shortcomings in Vista eventing which we are calling LogRefiner™ technology.

This week, we sent out a press release regarding our launch of Event Archiver 7. Here are some highlights, with some of the most interesting sections highlighted in bold:

Dorian Software Creations, Inc. www.doriansoftware.com today announced the release of Event Archiver 7 (www.eventarchiver.com), the latest version of its automated log file collection and consolidation tool.

Having announced earlier in the year a U.S. patent for its Total Event Log Management Solution ™, the globally recognized leader in log management is again charting new territory within the SEM and SIEM markets. This time, Dorian is striking early at the looming onslaught of EVTX files – logs generated by the new Windows Vista and upcoming Windows Server ® 2008 operating systems – that compliance and security specialists face.

Dorian’s development team has been warning for some time in its blog at http://eventlogs.blogspot.com/ that the change in log formats from the existing EVT format to the new EVTX is rife with pitfalls - for admins and particularly, compliance and security specialists seeking consistency and reliability for log audits. The warnings have not articulated a preference between the log types but have instead stressed the importance of understanding the pitfalls before moving forward with Windows Vista and Windows Server 2008 migrations.

Many network administrators and those attempting to audit existing log data have just gotten the hang of the EVT format. Now, within the Windows ®platform alone, these security professionals face the specter of disparate formats and all the problems those differences bring: new event IDs; different formatting of data; and last but not least, changes in the way logs are handled for collection, monitoring, and reporting. Microsoft's shift to the EVTX format in Windows Vista and Windows Server 2008 is truly the elephant in the room for those tasked with ensuring compliance and log retention.

The differences in the log formats and the methodologies behind them are far greater than many in the industry are willing to admit. We are responding to these changes not by forcing upgrades to our software or encouraging adoption of the new format, but by focusing instead on the management of these log types side-by-side. After all, the adoption of the new log format within the private and public sectors is just beginning, and many requirements force organizations to store years-worth of log data. That means, in many cases, auditors and forensic investigators will be looking at the “old” EVT logs for another 5-10 years at least.

...

As a result, Dorian Software Creations, Inc. is introducing its exclusive LogRefiner ™ technology. The focus of this new technology is the careful management of both log formats side-by-side, streamlining the management of both formats via consistent logic and methodology. Therefore, early adopters of Windows Vista and Windows Server 2008 - the operating systems that generate the new EVTX format - can take advantage of log management capability in Event Archiver today. This again sets Dorian Software apart from other log management vendors - almost all of which have been notably mute or at least guarded in their response to the major changes facing SEM and SIEM efforts.

...

Because the management of both log file formats will be necessary for yearsto come, Dorian Software stresses that any releases including the LogRefiner technology will not abandon those who continue to work with the EVT format.

...

Windows Vista EVTX File Support

Event Archiver has the capability to collect and convert EVTX log files. This is the new logging format first introduced in Windows Vista and planned for use in Microsoft Windows Server 2008. Simply install Event Archiver to a Windows
Vista workstation to start collecting EVTX files from other Vista workstations.

LogRefiner ™ Technology Makes Downlevel EVT File Processing in Windows Vista Possible

Dorian's exclusive LogRefiner technology can archive and convert EVT files from downlevel systems directly alongside the EVTX files from Windows Vista and newer operating systems - the converting and reading of EVT files being the very thing that the Microsoft Event Viewer on Windows Vista has difficulty doing correctly. With Event Archiver's special new technology, no information goes missing when converting downlevel EVT files into new formats – all event log fields are processed properly the first time.

Streamlines Fields Between EVT and EVTX Logs With LogRefiner Technology

Did you know that Windows Vista’s EVTX logs have even more fields? Event Archiver 7 can be instructed to automatically consolidate these fields - the Keyword and Opcode fields specifically - into the Task (Category) field so that you can have a uniform data structure for EVT and EVTX exported log files.

LogRefiner Technology Maintains Field Consistency Across
Logs

In the Windows Vista Security Log, no information about the user performing the action or affected by the action is recorded in the User field when an event is logged. Instead, all user information is placed in the Description of the event. Event Archiver 7, however, has the ability to place the most relevant user information back into the User field as it converts EVTX files into new formats. By helping maintain the consistency of log data and its formatting, this feature greatly aids the administrator or compliance officer in charge of reviewing the consolidated data.

Defines Success Audits Versus Failure Audits Using LogRefiner
Technology

Another major change in the Windows Vista security log is that all events are recorded as “Informational.” To discern whether or not the event represents a failed or successful action, the administrator must refer to the Keyword of the event.

But, Event Archiver 7 - when converting security EVTX Files - has the ability to properly record whether or not the event was a Success Audit or Failure Audit, greatly aiding the reviewer of log data generated from both EVT and EVTX log files.

To sum up, our LogRefiner™ technology in Event Archiver 7 means that:

1.) You can migrate to Windows Vista and Windows Server 2008 when you are good and ready, knowing that,
2.) Our software will process the downlevel EVT files for you right alongside the newer EVTX files, and
3.) Event Archiver has advanced technology that standardizes the collected data for reporting and other compliance purposes.

From Windows NT to Windows Server 2008, Event Archiver 7 has you covered. If you'd like to take it for a test drive, you can download your free 30-day evaluation copy at http://www.doriansoft.com/download. Happy archiving!

Storage Requirements for the Windows Vista™ Security Log

Recently, we've created a few blog postings that talk about some of the new events present in the Microsoft Windows Vista™ security log. From a security standpoint, Vista's increased number of auditable events is excellent, as administrators and compliance officers can get a much deeper picture of the actions taking place on a computer prior to and during a security incident.

However, if you are required to retain those security events, either by law (e.g. HIPAA, SOX, GLB, PCI, etc) or by policy, you need to start budgeting for more storage before you start your Vista and Windows Server 2008™ migrations.

Here are a few examples of how Vista security logs tend to grow much more quickly than their predecessors:

1.) Looking at some of our internal Vista security logs, there are tons of events relating to the blocking or accepting of network data via the Windows Filtering Platform. Some organizations may find this data valuable, especially if the machine is exposed to the public, however others may not.

2.) Some events log extra information at the end of the Description field that serves no other purpose than to further explain the parameters in the Description field. For instance, every 4608 event (Windows is starting up) also tells you that:

"This event is logged when the LSASS.exe starts and the auditing subsystem is initialized."

Similarly, every 4634 event (An account was logged off) feels the need to mention that:

"This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer."

These are just two brief examples, but note well: your Vista logs will use up more space than your XP and Windows 2000 workstation logs. If you are reassuring yourself now by thinking that you only need to retain server logs, bear in mind that Windows Server 2008 will share Vista's new events and logging tendencies!

Fortunately, the current release (and several prior releases) of our Event Archiver™ software offers you techniques to help you manage your storage of log data. Event Archiver allows you to automatically prune your database tables by date, selectively import only key events or exclude non-key events into database tables with global import filters, and keep your data in multiple compressed formats for storage efficiency. As the number of auditable events increase and expand in size, these features become increasingly important.