Tip of the hat to Eric at Microsoft

Many kudos to Eric Fitzgerald, the head of Microsoft's Windows Auditing Team, for his very candid explanation on the transposing of security event ids in Vista. He writes:

Doriansoft noticed that there's a relationship between our pre-Vista security event IDs and our Vista-era security event IDs.

For most security events:VistaEventId = PreVistaEventId + 4096

Why is this?

We needed to differentiate the Vista events from the pre-Vista events, because we were significantly changing the event content and didn't want to break automation. However we wanted to preserve the knowledge that security professionals already had in their heads about security events, so we wanted to make sure that there was a relationship between old and new event IDs.

We decided to offset the old IDs by some constant to get the new IDs. I wanted to offset them by a decimal number (say 6000, so 528 would become 6528, etc.). However event IDs are declared in hex in the source code and are all 3 digits long (528 = 0x210), and Raghu, my developer, wanted to conserve effort, and he won that battle so we added 0x1000 (4096) to the existing event IDs.

For what it's worth, I think Eric's initial approach would have been best, as I think most non-developers can deal with Base 10 offsets in their head much more easily than Base 16. Still, his candor in addressing the issue is refreshing, and is much appreciated.

Crash ... Into Me

Many of our high-security clients must enable the CrashOnAuditFail setting on their servers, as per government policy. In case you're not familiar with this setting, open up your Registry Editor, and visit the following key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa

One of the values under this registry key is "CrashOnAuditFail" ... By default, this value is set to 0. If you set it to 1 (and I don't recommend you do, unless you have a test machine you're tinkering with), your system will Blue Screen as soon as the security log fills up, provided you have also prevented your workstation or server from overwriting events automatically in the security log. At that point, only an Administrator can log back on to the machine after a restart to clear the security log and reset the CrashOnAuditFail flag.

The purpose of this special setting is to prevent a computer from being used (e.g. logged into) by anyone other than administrators unless auditable events can be recorded in the security log. Thus, it is a very important setting in high-security networks.

Interestingly enough, Vista adds a new event related to this special registry value. In Vista (and presumably Longhorn server as well), Event ID 4621 gets logged when an administrator successfully recovers the system from a crash related to the Security log filling up. From a documentation and accountability standpoint, this is a nice new event that Vista brings to the table.

References Available On Request...

Today we're going to tackle one of the biggest misconceptions among admins about the Windows Event Log. In particular, the notion that it is a.) a self-contained file and/or b.) a text file. It is in fact neither one.

If you drop a saved EVT file into a text editor like Notepad, you will see some text strings, but you're also going to see a lot a gibberish. Together, the readable text and the gibberish make up a bunch of linked data structures. Microsoft calls this structure an EVENTLOGRECORD, and a saved EVT file has a ton of them, one for each log entry in the file. An EVENTLOGRECORD contains a whole bunch of information, including the date/time the log entry was written to the file, the security identifier (SID) of the user performing or logging the action, the category of the event, the source, event identifier, etc.

Earlier, it was mentioned that event log files are not self-contained. For example, the log file doesn't contain the name of the user performing the action - instead, it contains the SID number (security identifier). In other words, a log entry generated by NT AUTHORITY\SYSTEM wouldn't have NT AUTHORITY\SYSTEM as a text string in the EVT file - it would contain the SID "S-1-5-18" represented in binary. In some respects this seems tedious, but since usernames get renamed often in a Windows domain, it makes sense. If you decide to review an EVT file again in the Microsoft Event Viewer many months after it was initially saved, you want to see the usernames the SIDs resolve to at that instant, not what they resolved to months ago. So, any application that consumes or reads event log files needs to be able to resolve that information, typically via the domain controller or the member server/workstation where the log file originated from.

Another wrinkle in the Windows Event Log format is how the Description field - which often contains very valuable information - gets put together. EVT files don't contain all the data that you see in the Description field of events you view in the Microsoft Event Viewer. Instead, they contain the key nuggets of data that get combined together with message frameworks from resource files to form a complete description.

For instance, your antivirus software may elect to write Event ID 20 to the Application Event Log. Event ID 20 corresponds to the following message template in the Antivirus software's message files: Virus %1 was detected in file %2

However, the Antivirus software will only send the two parameters needed by the message framework when it logs the event. In this example, MyDoom for the virus name and c:\dropper.exe for the file.

In other words, your EVT file will only contain MyDoom and c:\dropper.exe ... and it's up to the program displaying the event to you to put together the complete description, which in this case is: Virus MyDoom was detected in file c:\dropper.exe.

Given that event log files contain so many references to information that in many cases can only be obtained from the machine where the files were generated, here are some tips that will serve you well:

1.) If you routinely save your event log files, consider saving them in both the EVT/EVTX format and another self-contained format like comma-delimited text or a database table. The EVT file will contain valuable information like the original SIDs, and the text file/database table will contain a snapshot of how that data was translated at a certain instant in time.

2.) If you uninstall software, you may find entries in your event logs where the Description field cannot be correctly formatted. This is because message files often get uninstalled alongside the software. If you reinstall the software, even temporarily, you should be able to see the full event descriptions again.

3.) Transplanting Windows Event Log files to a network outside the one they originated from is a losing proposition. When the Microsoft Event Viewer or other log viewing application attempts to resolve the many references in the file, it will more often than not fail and produce incomplete data. As a result, you will only see limited amounts of information, such as Well-Known SIDs and events from software/hardware that match the software/hardware installed on the originating machine.

4096 Security Events Lane

What's a big difference in the Vista security log? Here's your clue ...

2^12
1000 in Base 16/Hex
1000000000000 in Base 2/Binary
4096 in Base 10

If you scan through your security log in Vista, you're going to see some very unfamiliar Event IDs.... 4616 (System Time Changed), 4624 (Successful Logon), etc.

Let's do some quick math:

4616 - 4096 = Our old friend Event ID 520
4624 - 4096 = Our old friend Event ID 528

For fun (I'm sure they had a more legitimate reason, right?), Microsoft decided to add 4096 to quite a few of the old well-known Security Event IDs in Vista. Now bear in mind this "subtract 4096" trick doesn't work for every event, and also understand that some of your favorite Event IDs have gone missing.

Missing Event IDs? Sure.

Like 540 (Successful Network Logon) ... he's been forced to reside with his first cousin 528 (Successful Logon) at 4624 No Caps Lock Drive.

Don't feel bad for 540 though. Just ask those naughty logon failure IDs of yesteryear, like 530 (Account Logon Time Restriction Violation) and 535 (The account password has expired). They - and several of their siblings - now have to live at 4625 Fat Fingers Boulevard.

For all those folks out there using scripts for security log management ... you have some updating to do.

Seeing Crimson...

Sorry, couldn't resist that little play on words for this inaugural blog post. We're wrapping up development right now on new libraries to support the management of logs on Microsoft Vista. In case you haven't heard, Microsoft has introduced a new logging format in Vista, EVTX files. Back in Vista's beta days, this logging format was code-named Crimson.

So far, the development experience has been quite an eye opener. While the ReportEvent function, which is the cornerstone function for writing to the event log in the legacy EventLog API, works great in Vista, other Legacy API techniques do not. For instance, if you try calling the OpenBackupEventLog function on Vista to open a saved legacy EVT file, the function will fail. Interestingly enough, Microsoft has still not updated its documentation at MSDN to reflect this problem as of this writing: http://msdn2.microsoft.com/en-us/library/aa363671.aspx

Going in the other direction, legacy Windows clients (e.g. NT/XP/2000/2003) can open a handle to a "live" Crimson/EVTX log on Vista remotely, but the traditional techniques used to parse through and render the data on such a log will most likely fail due to a variety of other reasons that relate to the hardening of Vista's networking and new message provider data stores. On top of that, legacy Windows clients simply have no mechanism for reading saved Vista EVTX log files whatsoever.

While the Crimson/EVTX format does confer advantages over its predecessor, such as XPath queries, we're still not sure why Microsoft elected to cripple the OpenBackupEventLog function on Vista while supporting other legacy EventLog API calls. Certainly, from a programmatic standpoint, it would appear that the function does little more then read linked data structures out of a saved binary file. One perhaps controversial theory is that Microsoft wanted to make transitioning to Vista much easier for publishers of Windows events (e.g. software developers whose programs write to the log), than for the consumers of the those events (e.g. the utility software vendors whose programs manage and analyze log files). Considering Microsoft's efforts to increase market share in the server management market, that could be the case.

Regardless, we've found cool new ways to work around these potential trouble spots and look forward to introducing Vista/Longhorn compatible versions of our software very shortly.